The review that finishes this year.
Screening four thousand abstracts twice is why systematic reviews take eighteen months and why so many are abandoned at the halfway point. We do the mechanical half at machine speed and keep the judgement where it belongs — two human reviewers, an agreement statistic, and a PRISMA flow generated from what actually happened.
Systematic reviews get abandoned somewhere around abstract eleven hundred, on a Sunday, while the question is still worth answering.
Title and abstract screening is a huge job that cannot be skipped, and almost all of it is mechanical. Two people read the same four thousand abstracts and agree about 95% of the time. The 5% is where the expertise is; the rest is stamina.
So we put a model in front of the queue to rank and pre-classify, keep both human reviewers, and spend their attention on the records where it changes something: the uncertain middle and the disagreements.
The result is a review with the same method and a defensible audit trail, finished within a quarter.
The 5% that needs a human.
Records arrive ranked by the model. Both reviewers still decide every one, but they see the likely includes first, and the queue tells them where they disagree. The records are illustrative, but this is the workflow we run.
The model orders the queue. It never excludes a record on its own — a record leaves the queue only when two humans have agreed, or a third has adjudicated.
Eight stages, and you can buy one.
Written before you look
The question in PICO form, eligibility criteria that can actually be applied by two people consistently, and a PROSPERO registration that will not need amending in month four.
- PRISMA-P compliant protocol
- PROSPERO or OSF registration
- Criteria pilot-tested on 50 records before launch
The part reviewers attack first
Database-specific strategies with controlled vocabulary and free text, peer reviewed against the PRESS checklist, and recorded so somebody can re-run them in three years.
- MEDLINE, Embase, CENTRAL, CINAHL, PsycINFO, Web of Science, Scopus
- Grey literature, trial registries and citation chasing
- Every strategy, date and hit count reported verbatim
Boring, and worth doing properly
Records from seven databases overlap messily — different capitalisation, missing DOIs, conference abstracts of the same study. Aggressive deduplication loses papers; timid deduplication wastes weeks.
- Match on DOI, then on fuzzy title, author and year
- Uncertain pairs shown to a human rather than merged silently
- Duplicate count reported for the PRISMA diagram
Ranked queue, human decisions
A model trained on your own inclusion decisions ranks the queue so likely includes surface early. Both reviewers still screen, and the model changes only the order.
- Agreement statistics computed continuously
- Conflicts routed to a third reviewer with the reasons attached
- Recall estimated on a human-screened sample, and reported
Into a versioned form
A piloted extraction form, dual extraction with reconciliation, and pre-filled candidate values that a human confirms against the paper — with the page and sentence they came from.
- Extraction form built in REDCap or COHORT, so it is versioned
- Every value traceable to its location in the source
- Author contact tracking for missing outcomes
The judgement, with its reasons
RoB 2, ROBINS-I, QUADAS-2, PROBAST or the Newcastle-Ottawa Scale as the design demands, with the supporting quotation recorded for every domain judgement.
- Two assessors, disagreements adjudicated and logged
- Traffic-light and summary plots as generated output
- Domain judgements defensible line by line
Meta-analysis, or an honest refusal
Random and fixed effects, network meta-analysis where the comparisons connect, meta-regression, and a clear statement when the studies are too heterogeneous to pool.
- Forest, funnel and league tables to journal spec
- Heterogeneity investigated rather than reported and ignored
- GRADE certainty assessment per outcome
PRISMA from the record
The flow diagram, the characteristics table and the checklist generated from the screening database — and a re-run on a schedule if the field is moving fast enough to need one.
- PRISMA 2020 flow and checklist as output
- Searches re-run quarterly for living reviews
- The whole screening record archived and citable
A missed study is not a rounding error.
The worst failure in automated screening is a relevant trial that never enters the synthesis and therefore never appears in the conclusion. Nobody notices the omission, and that is why it is serious.
So the model is confined to ordering the queue and pre-classifying the obvious. Exclusion always takes two humans. And because Cochrane and comparable bodies are still consolidating guidance here, we report the automation in enough detail that a methods editor can judge it themselves.
- It does Rank the queue Likely includes first, so the reviewers' first hour is their most useful. Order only — every record is still screened.
- It does Pre-classify the obvious An animal study in a review of human trials is not a judgement call. Flagged, and still confirmed by a person.
- It does not Exclude anything alone A record leaves the queue when two humans agree or a third adjudicates. There is no configuration in which the model decides.
- It does not Stop the screening early Stopping rules based on predicted recall exist in the literature. We do not use them without your explicit, documented agreement.
Every box is a query.
The flow diagram is generated from the screening database, so it cannot disagree with the review. When a reviewer asks why 287 became 34, the exclusion reasons are already tabulated because each exclusion had to carry one at the time.
The same applies to the characteristics table, the risk-of-bias plots and the checklist. Regenerating them after a late-arriving study takes one command.
- Live during screening — the current flow, at any moment
- Exclusion reasons tabulated because they were mandatory fields
- Re-runnable when the search is updated for a living review
A quarter, with the method intact.
Protocol
Question, criteria and outcomes fixed, piloted on fifty records, and registered before anybody screens anything.
- PICO and eligibility
- Criteria pilot
- PROSPERO submitted
Search
Strategies built and peer reviewed, run across the databases, deduplicated with the uncertain pairs checked by hand.
- PRESS-reviewed strategy
- Grey literature and registries
- Counts locked for PRISMA
Screen
Dual screening on a ranked queue, with agreement watched from the first hundred records and conflicts adjudicated as they arise.
- Two reviewers, always
- Kappa monitored live
- Recall estimated on a sample
Extract & appraise
Dual extraction into a versioned form, risk of bias by two assessors, and authors contacted for the outcomes nobody reported properly.
- Values traceable to the page
- RoB with quotations
- Author correspondence tracked
Synthesise & write
Meta-analysis where pooling is defensible, GRADE per outcome, and every figure, table and checklist generated rather than assembled.
- Forest and funnel plots
- GRADE summary of findings
- Reproducible supplement
What methods editors ask.
Is AI-assisted screening acceptable to Cochrane and to journals?
Automation to prioritise screening is well established and used in Cochrane workflows. Automation that replaces one of the two human screeners, or that stops screening early on a predicted-recall rule, is an active methodological argument and not settled.
We work on the settled side of that line by default: the model orders the queue, both humans screen everything, and the automation is described precisely in the methods so an editor can form their own view. If you want to use a stopping rule, we will implement it — after writing down together what it risks.
Can you take over a review that has stalled?
Frequently. The usual state is a registered protocol, a search from eighteen months ago, a spreadsheet of partially screened records, and a co-author who has left. That is recoverable, but the search almost certainly has to be re-run and the new records screened, which we would rather tell you at the start.
We reconstruct what was decided from whatever record exists, re-run the search with the original strategy, and screen only the delta with the original criteria. Where the earlier screening cannot be reconstructed, we will say that the honest options are re-screening or a documented limitation.
Who are the two reviewers — us or you?
Either, and it changes the cost substantially. Many groups screen themselves and use us for the protocol, the search, the platform, the statistics and the reporting — which is the cheapest version and keeps the clinical judgement with the people who have it.
Where you need screeners, we provide them, with a clinical or methodological background matched to the topic. The clinical eligibility judgement is best made by somebody who knows the field, and if that is not us for your topic, we will say so and screen alongside one of your team instead.
Do you do scoping reviews and rapid reviews too?
Yes, and choosing between them properly is worth an hour at the start. A scoping review maps what exists and follows PRISMA-ScR; a rapid review deliberately trades some methodological rigour for speed and has to declare which trades it made; an umbrella review synthesises existing reviews.
Picking the wrong one is a common and expensive mistake — usually a full systematic review attempted when a scoping review would have answered the actual question in six weeks.
What if the studies are too different to pool?
Then we say so and do a structured narrative synthesis instead, with the reasoning stated. A meta-analysis of studies that should not be combined produces a confident number about nothing, and it is the criticism such papers most often receive.
We investigate heterogeneity with subgroup analysis and meta-regression where there are enough studies to support them, and say so plainly when there are not. A paper that says “We chose not to pool, and here is why” is stronger than one with a forest plot and an I² of 92%.
Can the review be kept up to date?
Yes — living reviews are one of the better arguments for doing the screening on a platform rather than in a spreadsheet. The searches re-run on a schedule, new records enter the same ranked queue, and the PRISMA numbers update because they are queries.
What needs a decision up front is the trigger for republication: every new study, a change in the pooled estimate, or a fixed calendar. We recommend the second, and it has to be pre-specified to be credible.
A review is a study with different data.
Send the question. We will send back the hit count.
Give us the review question and we will build a draft search strategy and run it — so you know within a week whether you are facing four hundred abstracts or fourteen thousand. That number decides the design of everything else, and most reviews start without it.