Public debate moves quickly and produces more material than anyone can easily follow. AI-generated and AI-assisted content adds to that volume. We tested whether LLM snapshots could help us keep up. Choose a country and period to see which issues, voices and developments an LLM identified in a defined sample of documents collected by TRACE.
Unlike a one-off response from a general chatbot, each snapshot was produced from a recorded sample using a fixed instruction and named model. The output was stored, and its quoted phrases and multi-word names were checked against the material supplied to the model. Single-word names such as Ofsted, and acronyms, are not covered by that check.
For each country and period, a language model summarised a rule-based sample of up to 400 documents. The rule: take the twenty largest topics in that country and period, and from each take the twenty documents sitting closest to the centre of that topic. So the model sees each topic’s most typical material, not a random draw, and the snapshot will read as more coherent than the collection actually is. The sampling rule, instruction and model were fixed and recorded. A person did not select documents individually or prompt the model through a chat window.
A script then checked quoted phrases and multi-word capitalised names against the documents supplied to the model and removed those it could not verify. This can catch invented names and altered or unsupported quotations. It cannot determine whether the snapshot selected the right issues, represented them fairly or omitted important evidence.
Generated text may contain unsupported material, but factual invention is not the only risk. A snapshot can quote accurately and still give disproportionate attention to one issue, overlook another or impose a misleading interpretation on the material.
The snapshot is mechanically checked in specific ways, not independently validated as a complete, balanced or correct account.
Pick a country and time period, then select View snapshot. Each snapshot was generated in advance and stored, so what you see is a replay of the recorded output rather than a new model response.
An LLM can summarise information quickly, but it does this by leaving things out, including details that someone who knows the system might see as crucial.
What the model highlighted in a fixed sample from one country and one period.
Treat it as a quick route into the documents, not as a finding. We checked quotations and multi-word names against the sources. The original output is stored, so this display replays it rather than generating a new answer.
Whether the sample represents the wider debate or whether the model gave each issue fair weight. Checking quotations catches invented text, not distorted emphasis.
No one has independently read the full sample and assessed the snapshot against it. Use it to identify questions worth investigating, never as evidence for a decision.