Citation Verification with CiteCheck: A 5-Minute Setup
A polished reference can still be wrong. The DOI may not resolve, the title may belong to another article, or the year may have drifted during a round of edits. Those are metadata problems: tedious to inspect one by one, but well suited to an automated first pass.
That is the narrow job of CiteCheck. It extracts references from a manuscript and compares them with records returned by Crossref, PubMed, Semantic Scholar, and OpenAlex. It does not decide whether the cited paper supports your sentence. Keeping those two questions separate is essential.
Install and run
Install the DOCX extra in a Python 3.9 or newer environment:
pip install 'citecheck[docx]'
Then point it at the manuscript and save a machine-readable report:
citecheck manuscript.docx --format json --output citecheck-report.json
The input can be a supported manuscript file rather than a specially prepared list. The JSON output is useful when you want to sort results or preserve the check with a submission archive.
What the output contains
Each extracted reference receives one of three states: verified, not_found, or error. A verified result includes the matched title, authors, year, journal, DOI, source database, and a title-similarity value. That detail matters: it lets the reviewer inspect what the software matched rather than accepting a green icon without context.
not_found means that no match passed the configured threshold across the selected sources. error means the check itself failed, for example because a service was unavailable. Do not combine those states. A database error is not evidence that a citation is fabricated.
The default fuzzy-title threshold can also create borderline cases. A low threshold may accept a related paper with a similar title; a high threshold may reject a legitimate record with abbreviated or translated metadata. Keep the raw reference and proposed match beside each other during review.
Read the report as triage
A positive match means that the bibliographic details were similar enough to a record returned by one or more scholarly databases. A non-verified result means “inspect this reference,” not “delete it.” Recent articles, books, conference abstracts, non-English titles, and records with incomplete metadata can all produce uncertain matches.
For each warning:
- Open the DOI or the journal record.
- Compare title, first author, journal, year, volume, and pages or article number.
- Correct the reference in the reference manager, not only in the exported manuscript.
- Re-export the bibliography and run CiteCheck again.
This loop prevents the same error from returning in the next draft.
For a multi-author paper, assign warnings by manuscript section and keep the resolved JSON report with the submitted version. Re-run the check after bibliography edits, because changing citation style or merging reference libraries can introduce new mismatches late in revision.
What CiteCheck does not establish
Metadata verification answers “Does this paper appear to exist, and is this reference pointing to it?” Claim verification asks “Does the paper support the sentence beside the citation?” The second question requires reading the relevant methods and results, not merely resolving a DOI or comparing an abstract.
CiteCheck also does not assess study quality, risk of bias, statistical correctness, or reporting-guideline compliance. Treat it as one gate in a wider pre-submission review. The distinction is explained further in Reference–Claim Alignment Checks, while Citation Hallucination in AI-Assisted Writing covers the underlying failure modes.
Five minutes is enough to install the tool and start a first run. Reviewing every warning will take longer, and that is appropriate: automation should narrow the manual work, not impersonate scholarly judgment.
CiteCheck is open-source under the MIT license; the implementation is available in the project repository. For plain-text or Markdown input, the base install is pip install citecheck. Preserve the report, and let a human author make the final call on every unresolved reference.