A systematic review is a literature review that someone else can repeat. It states the question, the sources, the search strings, the screening rules and the counts, so a reader can see what was found and what was left out. That is the reason decision makers ask for them. It is also why they are slow, and why AI tools are tempting.
Why systematic reviews are hard
67 weeks
average time to complete a systematic review
0.98 and 0.27
recall and precision reported for one LLM screening study
39.5%
pooled sensitivity of one AI search tool in an independent test
Pick the right reporting standard
PRISMA 2020 is the best-known reporting checklist, and it grew out of medical reviews. A 2018 paper on reporting in environmental evidence looked at the earlier QUOROM and PRISMA (2009) standards, noted that they were developed for medicine, and highlighted 12 key problems with applying PRISMA to conservation and environmental management.
The environmental answer is ROSES, short for RepOrting standards for Systematic Evidence Syntheses. The Collaboration for Environmental Evidence announced it on 1 March 2018, and the article introducing it was by Haddaway, Macura, Whaley and Pullin. It offers forms for review and map protocols and reports, plus a flow-diagram template. It was designed for environmental topics but is described as broadly applicable across disciplines. EQUATOR lists an update under development, registered on 28 October 2026, so versions matter.
Two further points on process. A protocol for a Collaboration for Environmental Evidence review must be registered and peer reviewed through its editorial process, so a draft from software is not a substitute. And evaluations of published reviews show why standards matter: one evaluation of more than 1,000 evidence syntheses published between 2018 and 2020 found that most had problems with transparency and replicability.
Frame the question
| Framework | Elements | Use it for |
|---|---|---|
| PICO | Population, Intervention, Comparator, Outcome | Effect of an intervention |
| PECO | Exposure replaces Intervention | Effect of an exposure; recognised by the Collaboration for Environmental Evidence |
| SPIDER | Sample, Phenomenon of Interest, Design, Evaluation, Research type | Qualitative and mixed-methods questions |
Whichever you choose, say why. The PRISMA 2020 expanded checklist asks reviewers to describe any search structure that is not PICO-based.
What must be logged
PRISMA 2020 is specific about the search record:
- Item 6 requires every database, register, website, organisation and reference list searched or consulted, and the date each was last searched. - Item 7 requires the full search strategy for each source, including filters and limits. - Item 8 requires the selection process: how many reviewers screened each record, whether they worked independently, and details of any automation tools. - Item 16a requires results from records identified to studies included, ideally in a flow diagram.
The checklist is published under CC BY 4.0, so the wording can be reused with attribution. If an LLM screened any records, item 8 says it must be reported as an automation tool.
For sources, OpenAlex offers free basic use without a key, and a free key raises the daily budget tenfold, according to its documentation. Crossref can verify DOIs. Whatever you use, log the date.
What AI screening has been measured to do
Screening thousands of titles and abstracts is the step where AI help is most tempting. The evidence is mixed:
| Study or result | Finding |
|---|---|
| Review of prior LLM screening work | Up to, and sometimes beyond, master's-student level, but not PhD level |
| One reported result | Recall 0.98 with precision 0.27 |
| 34,528 studies across 24 reviews and 9 LLMs | Small average differences among top models, much larger differences between screening tasks |
| Ten reviews, four prompting methods | A dynamic few-shot method gave the highest F1 |
| A 2026 workflow study on 1,131 records | LLMs suit validated, auditable, human-supervised workflows better than autonomous exclusion |
High recall at low precision is useful: the model flags nearly everything relevant, and people discard the false alarms. Performance depends on the topic, so validating on your own review beats trusting a headline figure. The 2026 workflow study adds that no single metric describes a screening workflow. Agreement, recovery of eligible records and workload saved are different things, and each should be reported.
Active-learning tools raise a related caution. ASReview can reduce screening burden by up to 95% while keeping recall high, but choosing where to stop is subjective. Commentary on a 2026 study (Kempny and colleagues) calls common stopping habits, such as stopping after 100 consecutive irrelevant records or after 5% of the dataset, highly questionable. The study itself found the extra screening needed ranged from 2% to 61% depending on the dataset.
AI search tools are not a search strategy
An independent study tested whether the AI research tool Elicit could find the studies included in four published reviews: one pharmacology, one surgery and two public health. Its pooled sensitivity was 39.5% (range 25.5% to 69.2%), against 94.5% for traditional searching. Precision was higher, 41.8% against 7.55%. Sensitivity was lowest in public-health reviews and highest in drug-intervention reviews. The authors concluded that the tool gives quick, high-precision results and finds some unique studies, but should be a supplement, not a standalone method.
Treat vendor claims on speed and coverage as claims. The study notes that most evaluations of such tools come from their developers and read like marketing. For the tool Consensus, a research-library guide states that it cannot be used for systematic reviews or meta-analyses because its retrieval is not reproducible or auditable in the PRISMA sense.
A workable routine
1. Choose ROSES or PRISMA and the question framework, and record why. 2. Search several sources, log each with its date, and keep the full strings. 3. Let an LLM pre-screen if you wish, but log it as an automation tool, keep humans in the loop, and validate recall on a sample you screen yourself. 4. Do not let the model exclude records alone. 5. Check every cited reference against its DOI before it enters the report.
For related cautions on AI claims, see what peer-reviewed benchmarks show about AI biomass estimates and why AI reading of regulations needs checks.
Frequently Asked Questions
It is widely used, but a 2018 paper on the 2009 version highlighted 12 key problems in applying PRISMA to conservation and environmental management. ROSES was designed for environmental evidence syntheses.
Turn climate goals into a verified portfolio
TREEO connects the full carbon cycle — eligibility, simulation, real-time monitoring and registry-ready reporting — combining expert consulting with dMRV technology so the evidence exists before anyone asks for it.



