---
title: "Introducing Rigor Scan: A Free Reporting Check for Research Papers, Built on Jev"
description: "Rigor Scan checks how completely a paper reports its design, statistics, materials, data and claims, using Jev to tie every result to the sentence behind it."
updatedAt: "2026-09-23"
author: "Timothy Kassis, PhD"
authorTwitter: "timothykassis"
authorLinkedIn: "https://www.linkedin.com/in/timothykassis/"
tags: ["Product", "AI", "Research", "Scientific Integrity", "Reproducibility"]
canonical: "https://www.k-dense.ai/blog/introducing-rigor-scan"
---
Every methods section makes a promise: here is enough detail that someone else could check this work, or repeat it. Reporting guidelines exist to make that promise concrete. [ARRIVE 2.0](https://doi.org/10.1371/journal.pbio.3000410) lists 21 items for animal research, CONSORT does the same for trials and STROBE for observational studies, and most journals have a checklist of their own. The problem has never been a lack of rules. It is that checking a paper against them is slow and tedious, so it happens late, by a reviewer short on time, or not at all.

The gaps that result are well documented. A [survey of 271 animal studies](https://doi.org/10.1371/journal.pone.0007824) found that only 59% stated the hypothesis or objective and the number and characteristics of the animals used, and only 12% reported randomly allocating animals to groups. When [statcheck](https://doi.org/10.3758/s13428-015-0664-2) was run over more than 250,000 p-values in eight psychology journals, half of the papers that used significance testing contained at least one p-value inconsistent with its own test statistic, and one in eight contained a gross inconsistency that may have changed the statistical conclusion. Retracted papers keep being cited too: in a [study of post-retraction citations in biomedicine](https://doi.org/10.1162/qss_a_00155), only 5.4% of the passages citing a retracted paper acknowledged the retraction.

None of these problems is hard to catch on its own. What makes them persist is that catching all of them, for every paper, means working through sixty-odd items, cross-referencing the methods, and looking up every reference by hand. So today we are releasing **[Rigor Scan](/tools/rigor-scan)**, a free tool on the K-Dense website that does that first pass for you. Drop in a PDF and within seconds you get a reporting check across the paper's design, statistics, materials, data and claims, with every result pointing to the sentence it rests on. The checks are answered by **Jev**, a new structured decision model from TypeSafe, and this post explains how that works and why we built it that way.

## What Rigor Scan checks

Rigor Scan asks 74 questions of a paper, grouped into six domains: study design, statistics, materials, data and code, ethics and disclosure, and claims. The questions are adapted from ARRIVE 2.0 and common journal reporting checklists, and they are the kind a careful reviewer asks. Does the paper explain how the sample size was chosen? Does it say what its error bars represent? Does it give catalogue numbers or RRIDs for its antibodies? Does it say whether its cell lines were authenticated? Is its data in a public repository, or only "available on request"?

Not every question applies to every paper, and a paper should not lose points for leaving out animal housing when it has no animals. So before anything is scored, the scan works out which of nine study types the paper contains: new experimental data, assigned treatments, animal experiments, human participants, clinical trials, cell lines, antibodies, prediction models and omics data. Checks tied to a study type the paper doesn't contain are skipped and left out of every score. Four of the checks run the other way round. These are warning signs (results reported only as significant or not, data only on request, correlation described as causation, and conclusions generalized beyond what was studied), and finding one lowers the score.

![Rigor Scan results for the example paper, showing the detected study types, a summary of 47 checks in good shape, 2 unclear, 12 missing or flagged and 13 that don't apply, the PDF with the supporting sentence highlighted, the rigor fingerprint with a completeness index of 79, and six domain scores](./results-view.png)

The result is a completeness index from 0 to 100, a score for each domain, and a **rigor fingerprint**: one spoke per check that applies, grouped by domain, with spoke length showing how strongly the paper reports that item. Select any spoke or check and the PDF scrolls to the passage it rests on, with the sentence highlighted. Next to the score, Rigor Scan gives two graded reads that don't count towards it: how much procedural detail the methods give (from minimal to protocol-level) and how strongly the abstract states its main claim (from tentative to sweeping).

It is worth being precise about what the index measures. It measures how completely a paper reports what it did, and it does not judge whether the findings are right. A paper can report every item and still be wrong, and a paper with gaps can be entirely correct. What a low score tells you is that a reader has less to go on when deciding.

## How a scan works

A scan runs in four stages: the paper is read on your device, software checks run over its numbers and identifiers, Jev answers each reporting check, and the answers are scored against published rules. The first two stages involve no AI model at all and the last is plain arithmetic, so Jev's role is confined to the one step that genuinely needs reading comprehension.

### 1. The PDF is read in your browser

Your browser opens the PDF and reads its text layer directly, page by page. From the position of every line it rebuilds the reading order, separates side columns and running headers from the body, joins lines into paragraphs, labels sections such as the abstract, methods and results, and pulls out the reference list. The file itself never leaves your computer. What gets sent on for checking is the extracted paragraph text, without the references, and for the database lookups only the identifiers and citations those need.

### 2. Software checks the numbers and identifiers

Some things in a paper don't need a model to check. They need arithmetic or a database lookup, and Rigor Scan does those in plain code:

- **Reported statistics.** Every t, F, χ², r and z result that comes with a p-value is recomputed, following the [statcheck](https://doi.org/10.3758/s13428-015-0664-2) method, with the rounding of the reported statistic allowed for. A mismatch is flagged, and marked serious when the recomputed p falls on the other side of 0.05.
- **Ratios and intervals.** For hazard, odds and risk ratios with a 95% interval, the estimate has to sit inside the interval and near its centre on the log scale, and any p-value has to agree with the one the interval implies.
- **Percentages and means.** Every "x of n (y%)" is recomputed. Means of whole-number data such as counts or Likert scores are checked with the [GRIM test](https://doi.org/10.1177/1948550616673876), which asks whether that mean is even possible with the reported sample size.
- **Retractions.** Each reference's DOI is looked up in Crossref, which has included the Retraction Watch database since [acquiring it and making it open](https://www.crossref.org/blog/news-crossref-and-retraction-watch/) in 2023, and each PubMed ID is looked up in PubMed. References without an identifier are matched by their citation, and a match only counts when the title agrees. Retractions and expressions of concern are flagged, corrections are listed, and the paper's own DOI is checked as well.
- **Cell lines.** Cell line names are compared against the roughly 1,400 lines [Cellosaurus](https://www.cellosaurus.org/) lists as contaminated, misidentified or misclassified, and named lines are looked up for their accession and STR profile.
- **Data, code and registries.** Repository DOIs have to resolve and, where the repository reports it, contain files. GitHub repositories have to exist, GEO accessions have to be public, trial registration numbers have to exist in their registry, and RRIDs have to resolve.

These checks are reported in their own panel and never change the completeness index. They answer a different kind of question (is this number internally consistent, does this link actually open) and they are either right or wrong in a way the reporting checks are not.

### 3. Jev answers each reporting check

The 74 reporting checks need reading comprehension, and this is where Jev comes in. [Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev) is the first of what TypeSafe calls System One models, a class of models built to make fast, structured decisions that software can use directly. TypeSafe describes it as "unstructured state in, typed probabilistic decisions out." You send Jev some state (here, the text of the paper) along with a set of typed questions, and it returns a typed answer with probabilities for each one. It does not write prose. In the words of its [documentation](https://openrouter.ai/docs/guides/community/jev), "Jev does not produce reasoning traces, explanations, or free-form text."

Jev answers three kinds of question, and Rigor Scan uses all of them:

- **Does this condition hold?** Jev returns the probability that the answer is yes (TypeSafe calls this question type a *noul*). Every reporting check is asked this way: "Does the paper describe the method used to generate the random allocation sequence?" comes back as a number between 0 and 1. The nine study-type questions are asked the same way.
- **Which one of these options?** Jev returns the option it picks, with a probability for each option. Rigor Scan uses this to find the evidence. Alongside each check it asks which paragraph of the paper best answers the question, and the only options it offers are the paper's own paragraph IDs. Once every domain has answered, a second pass asks which sentence inside that paragraph is the one, again offering only the sentences that are actually there.
- **Where does this fall on an ordered scale?** Jev returns a position on the scale with a probability for each level. The two graded reads, methods detail and claim strength, are asked this way.

A domain's questions are asked together over the same text, and the six domains run in parallel. Long papers are split into windows that fit Jev's 32,000-token context, each window is asked the same questions, and for each check the window with the strongest evidence wins.

### 4. Scoring against published rules

Each check's probability becomes evidence (flipped for warning signs), and the evidence decides its status:

```
evidence = p          for a reporting check
         = 1 − p      for a warning-sign check
status   = reported if evidence ≥ 0.70, missing if < 0.35, else unclear
index    = 100 × mean(evidence over the checks that apply)
```

A domain score is the average evidence across that domain's checks that apply, and the completeness index averages across all of them, so larger domains carry more weight. Everything that decides a result is published on the tool page under [How scoring works](/tools/rigor-scan#method): every question exactly as it is put to Jev, every threshold and every software rule, and you can copy the whole checklist as JSON. The checklist is versioned (this is v0.3), and scores are only comparable within a version.

## Why we built it on a decision model

The obvious way to build a tool like this is to put a chat model in front of the text and ask it for an assessment. That produces a readable report, but one that is hard to hold to account. The output is prose that has to be parsed, its judgments can drift from the question that was actually asked, and when it quotes the paper you have to go and check that the quotation exists. For a tool whose whole purpose is to help people verify research, that felt backwards. We wanted every result to be something you could check in a click, and Jev's design lines up with that requirement in four ways.

**A checklist is already a list of typed questions.** Reporting guidelines are made of yes-or-no items, and that is exactly the shape Jev answers. Rigor Scan doesn't need to ask for a review and then extract verdicts from it. It asks the checklist's questions verbatim, and what comes back is one probability per question. That is also why we can publish the questions exactly as they are asked, since there is no free-form review in between for them to disappear into.

**Probabilities make the thresholds meaningful.** Because every answer is a probability, the rule for "reported" can be written down (0.70 or above) and applied the same way to every paper, and a genuinely ambiguous passage can land in "unclear" instead of being forced into a yes or a no. TypeSafe trains Jev with a method it calls Reinforcement Learning for Calibrated Decisions, with the aim that higher confidence means higher accuracy. Calibration is a property that holds on average across many answers, though, and any single answer can still be wrong. That is the reason for the next point.

**Evidence can only point at text that exists.** Because the evidence question is a choice among the paper's own paragraphs, and then among that paragraph's own sentences, Jev cannot invent a quotation. When it comes to evidence, the worst it can do is pick the wrong sentence, and when it does, you can see that immediately, because the highlighted sentence is right there on the page. The model picks and the page shows. We think that division of labour is the right one for any AI tool that makes claims about scientific papers.

**It's fast enough to feel like a tool.** TypeSafe quotes end-to-end response times of 70 to 500 milliseconds for Jev. That is what lets a full scan, with a probability and a supporting sentence for every check, finish in seconds instead of the minutes a long generated review would take.

Building on a model that writes no text has one more consequence: nothing in a Rigor Scan result is generated prose. The labels, the questions, the statuses and even the brief that hands your gaps to K-Dense Web (more on that below) all come from fixed templates. What Jev contributes is numbers and picks, and every one of them is on the page for you to inspect.

## A worked example

The tool loads with an example so you can explore it before uploading anything: [Dubowitz et al. (2023)](https://doi.org/10.1371/journal.pone.0293905), an open-access PLOS ONE study in mouse models of breast cancer that compared sevoflurane and propofol anaesthesia during tumour resection and found no difference in cancer recurrence. We chose it because it is openly licensed and because it is a good paper, not a weak one, which makes the gaps it does have more instructive.

Rigor Scan detects new experimental data, assigned treatments, animal experiments, cell lines and antibodies, and rules out human participants, clinical trials, prediction models and omics data. That leaves 61 of the 74 checks, and the paper scores 79. Its strongest domains are materials (89), ethics and disclosure (89) and claims (87). It reports the species, strain, sex, age, source and housing of its mice, and it states that its cell lines were tested for mycoplasma and authenticated. Select that check and the scan highlights the sentence it rests on: "Cell line identities were confirmed by short tandem repeat profiling (Cellbank, Australia)."

The gaps sit mostly in study design (67) and data and code (62). The scan finds no rationale for the sample size, and it finds the antibodies listed with their suppliers but without the catalogue numbers, clones or RRIDs needed to identify them. Nor does the paper name a reporting guideline. Some of the results turn on distinctions that a quick read would blur. The paper states that mice were randomly assigned, so randomization counts as reported, but it doesn't describe how the allocation sequence was generated, which is a separate item. Similarly, it reports that "statistical analyses were undertaken by an investigator (RB) blinded to treatment conditions," so blinded assessment counts as reported, while blinding at the point of allocation does not. ARRIVE 2.0 treats these as separate items, and so does Rigor Scan.

![Software check results for the example paper: three hazard ratios with consistent intervals, 48 references with no retractions found and two with published corrections, three cell lines with no problem lines, and a Zenodo data record that opens and includes analysis code](./software-checks.png)

The software checks come back clean. The three hazard ratios the paper reports are consistent with their confidence intervals. Of its 48 references, the 45 that could be matched include no retractions, and the two with published corrections are listed. None of its three cell lines appears on the problem list, and a trial registration it cites resolves in ClinicalTrials.gov. The most interesting result is where the two kinds of check meet. The reporting checks find no statement that the analysis code is available, but when the software check opened the paper's Zenodo data record, it found R scripts inside. The code is shared and the paper just doesn't say so, which makes it a one-sentence fix, and it is the kind of thing that only surfaces when something actually opens the link.

The example also shows why every result carries its evidence. The one warning sign raised, for overgeneralization, only just clears the threshold, and the sentence it points to is the authors' own caveat that "the findings are limited by the extent to which mouse models replicate the clinical scenario experienced by breast cancer patients." Most readers would look at that and discount the flag, and the point of the design is that they can do so in one click, without having to take the tool's word for it.

## From gaps to fixes with K-Dense Web

A list of gaps is only useful if you can close them, so every missing or unclear item in Rigor Scan comes with a concrete next step. A missing sample size rationale becomes "Run a power analysis." Missing antibody identifiers become "Build an antibody table" with supplier, catalogue number, clone and RRID. Code that isn't shared becomes "Package the analysis code," with a README, an environment file and a draft availability statement. A missing reporting guideline becomes "Fill in the ARRIVE 2.0 checklist" with the page for each item.

![The Close the gaps with K-Dense Web panel for the example paper, listing 15 gaps such as sample size justified, randomization method and antibody identifiers, each with a suggested task, and a button to copy the brief and open K-Dense Web](./handoff.png)

Pick the items you want, copy the brief, and open [K-Dense Web](/products/k-dense-web), our AI co-scientist, which can run the power analysis, compute effect sizes with confidence intervals, package code and draft the missing methods text from your own files. The brief lists the gaps and what to do about each one, and it is assembled from fixed templates, so it contains no text from your paper. If you want K-Dense Web to work from the full manuscript, you attach it there yourself.

## Private by design

Unpublished manuscripts are sensitive, so Rigor Scan is built so that your paper stays with you. The PDF is read in your browser and never uploaded. Only the extracted text is sent for checking, only identifiers and reference citations are sent for the database lookups, and K-Dense doesn't store any of it. Results live in your browser tab, and when you close it, they're gone.

You can still share a result. A share link carries scores only: one number per check, plus the paper's title and DOI if you choose to include them. It holds no text from the paper, and the share image is drawn from the same link, so what a recipient sees is exactly what the link contains.

## What Rigor Scan can't do

Rigor Scan reads text, so it can't read figures or tables embedded as images, and supplementary files are only included when they are part of the PDF. Scanned PDFs without a text layer can't be read at all, and English papers work best. Its segmentation is heuristic, and unusual layouts can split paragraphs or sections imperfectly. Above all, it checks what a paper says, not whether it is true, and the model can misread a passage. That is why every result links to its evidence, and why the software checks sit apart from the score.

## Who it's for

We built Rigor Scan for anyone who has to decide how much to trust a paper, or wants others to trust theirs. For authors, it is a pre-submission check that catches the missing power analysis or antibody table while it is still cheap to fix. For reviewers and editors, it is a fast first pass that points straight to the relevant sentences, so reviewing time goes to the science instead of the checklist. For journal clubs and trainees, it is a concrete picture of what complete reporting looks like, item by item.

It also fits a broader argument we have been making. As AI makes it cheaper to produce papers, analyses and manuscripts, the bottleneck shifts to [verification](/blog/ai-co-scientist-verification-bottleneck), and the tools that help most are the ones whose output you can check. Rigor Scan is our attempt to build one: every question published, every threshold written down, and every result tied to the sentence it rests on.

## Try it

[Rigor Scan](/tools/rigor-scan) is free and needs no account. The example paper loads by default, so you can click through the fingerprint and the evidence before you scan anything of your own. When you're ready, drop in a PDF. If you have a check you think we should add, or a result you think we got wrong, we would like to hear about it at [contact@k-dense.ai](mailto:contact@k-dense.ai).