Skip to content
Accelerated ScienceJournals
Submit researchSubmit

Review

Claim verification

Every published paper lists the checks that ran on it and names the tool that ran each one.

What verification covers

Verification is mechanical and does not assess the quality of a manuscript. It establishes what can be determined by lookup and calculation. The reference and algebra checks are deterministic and return the same result for the same manuscript. Extraction of figures and equations from a document is not deterministic, as the document must first be read, and that step may misread. A check that cannot be completed with confidence is recorded as not run rather than as passed. Each published article lists the checks performed and the tool that performed each. A check that is not listed was not run.

The checks

Each check returns a located and falsifiable result, identifying the figure, its location, and the discrepancy. Checks are added as they enter the pipeline.

  • Reference resolution: cited references are looked up in Crossref; any that do not resolve are flagged, with the reference named.
  • Statistical consistency: reported numbers, tables and magnitudes are checked for internal agreement with the stated formulas and values.
  • Symbolic mathematics: the algebra in a paper's derivations is checked symbolically for sign, algebra and limit errors. It checks the identities a paper states rather than every step of a proof.

What verification is not

A resolved reference is a reference that exists, not a reference used correctly, and Crossref matches by title, so a close match is possible. Verification does not detect plagiarism. It does not certify data or code it cannot see. Judgement of the argument, the methods and the contribution is the work of the model-assisted reading and the subject reviewer. It is labelled where it is machine-generated.

Reference resolution depends on the record it queries. Work deposited only as a preprint, including much of the current literature in computer science and machine learning, may carry no Crossref entry, and an unresolved reference of that kind is reported as unresolved rather than treated as an error by the author.