Pre-Registration on OSF: AI-Assisted Drafting That Doesn't Box You In
When I pre-registered ARM4 on OSF, AI helped draft most sections in a fraction of the usual time. But I pushed back on three analysis-plan passages — the AI had specified tests that depended on distributional properties I couldn't verify before data collection. OSF pre-registration AI is a genuine productivity gain only if you know which sections to trust and which to treat as drafts, not commitments.
Why Pre-Registration Fails in Two Opposite Ways
Over-specification is the failure mode AI makes easy. You commit to an exact analysis pipeline before knowing the distribution of your data, the rate of missing values, or whether baseline randomization held. Six months later, when reality diverges, you're explaining a "deviation from pre-registration" — which reviewers treat as a yellow flag regardless of the reason.
Under-specification is the mirror problem. A plan so vague it provides no real constraint looks like post-hoc rationalization to peer reviewers, even when it isn't.
AI defaults to over-specification. It tries to be thorough: the exact test, the exact alpha, the exact covariate list. That reads like rigor. It functions like debt.
The OSF Template Fields Where AI Is Safe
For most OSF template fields, AI drafting is fast and low-risk:
- Background and rationale: AI builds a coherent paragraph from your PICO question. Edit for accuracy; done.
- Hypotheses: Give AI the direction and it converts research questions into directional hypotheses cleanly.
- Sample size justification: AI explains the power calculation rationale well — though the actual calculation still needs a dedicated tool, not a language model.
- Inclusion/exclusion criteria: Mostly a transcription task. AI handles it without introducing risk.
The Two Fields That Need Human Control
Analysis plan. AI will write a complete pipeline with named tests, alpha thresholds, and software function calls. The problem is that this assumes you already know the shape of your data. If you plan a paired t-test and your difference scores are non-normal, you need a fallback — but your pre-registration didn't mention one. That's a post-hoc deviation waiting to happen.
Decision rules and stopping criteria. AI defaults to textbook thresholds (stop at interim if p<0.001). Real trials often need softer rules tied to context — and locking in the textbook version creates problems the moment your trial veers from the idealized design.
How to Draft the Analysis Plan Without Over-Specifying
The working principle: describe intent, not implementation. OSF templates explicitly allow "exploratory" or "conditional" labels — use them deliberately. The goal is to distinguish what you will do from what you might do, not to pretend everything is locked down.
Use this prompt:
"Here is my research question and primary outcome: [paste]. Draft an analysis plan for OSF pre-registration. For each analysis decision, classify it as: (A) pre-specified primary — highly committed, (B) pre-specified exploratory — conditional on data characteristics, or (C) deferred — to be determined after data inspection. Avoid specifying software package versions or exact function calls."
The AI will categorize each decision by commitment level rather than treating everything as primary. In ARM4, three passages fell into category B or C after this review — tests that depended on distributional assumptions I couldn't verify pre-collection. Each became a conditional specification:
"If normality is confirmed by Shapiro-Wilk (p>0.05), a paired t-test will be used; otherwise, the Wilcoxon signed-rank test."
One sentence. Defensible. Honest about uncertainty without abandoning the plan.
The Over-Specification Audit Before Filing
Before you submit the pre-registration, run a second AI pass on the analysis plan specifically:
"Read this pre-registration analysis plan. Flag any specification that commits to a specific test family before data collection, where a plausible distributional violation would require a different approach. For each flagged item, suggest a conditional specification."
This systematically surfaces over-specifications the first pass created. In my experience, there are always 2–4 items that need converting from fixed commitments to conditionals. Finding them before filing is much cheaper than explaining them to a reviewer.
Once the paper is drafted, combine the pre-registration review with the broader manuscript audit described in Self-Peer-Review with AI: A 5-Step Manuscript Audit Workflow. Pre-registration and manuscript auditing are separate loops, but they check overlapping claims — a claim in your Discussion that doesn't trace back to a pre-specified analysis is a yellow flag worth catching before submission, not after.
For the statistical specifics that feed your conditional specifications, the statistical assumption check workflow gives you the standard requirement set per test family. Knowing which assumptions you'll need to test informs what conditionals to write into the pre-reg in the first place.
The Research Mentor on aiforacademic.world works through hypothesis generation and PICO outlining — the structural decisions upstream of the analysis plan. Getting those right makes the pre-registration faster and tighter. Start there before you open the OSF template; the analysis plan will follow more naturally once the research question is sharp.