Four models read your paper in four different roles, then cross-examine each other

Set the lineup

Four models, four different jobs. Which model takes which seat is our choice, not a finding: the two heavy analytical seats go to the ones we have found steadiest on long technical documents, and the devil’s advocate seat to the one that diverges most, because that seat exists to break a consensus rather than to be right.

A vague label such as “tough referee” gets you four copies of one review. What produces different failure modes is a concrete prior, so each role below says what it suspects before it has read anything. Edit them freely. You can run other models in these seats too, though the page keeps these four names, so match them in your own bundle headings.

Reset the roles to the protocol’s own lineup

Round 1: four independent readings

Upload: the document, to each of four fresh conversations.

Give all four the same file, changing nothing but the role and the label. No model sees another’s answer in this round, which is the point: four readings that were never negotiated.

Use a PDF if you can: page numbers survive across interfaces, and every criticism has to carry a quote and a page. If a model truncates the file or answers as though it read only part of it, fix that before you continue.

Round 2: cross-examination

Upload: the document and your Round 1 bundle, to the same four conversations.

First build the bundle: one file, the four Round 1 answers in full, each under a heading naming the model and its seat. Keep them verbatim. Tidy them and you become the hidden bottleneck, editing in the direction you already believe. If a model failed, keep its heading and say so underneath, because a silent gap reads as agreement.

The same shape does for the Round 2 bundle you will assemble before the arbiter. Only the contents change.

This is the round that does the work: most of what Round 1 produced is generic, and this is where it dies. Every criticism carries an ID such as Claude-C3, so a model can name exactly which point it is attacking. If one has drifted back into a generic reviewer by now, restate its role in the chat.

Round 3, only if the gate is met

Run this only when, for one and the same criticism, one model said keep and another said reject, both cited evidence from the document, and it is high severity. Two models saying “identification is weak” and “parallel trends is not defended” are agreeing in different words. Two models reading one table and disagreeing about whether it shows what the text claims are not.

Run it once, on one model. Whatever it leaves unsettled goes to the arbiter as an open disagreement rather than to a fourth round.

The button stays off until you name the criticism: the prompt says nothing without it.

The arbiter

Upload: the document and both bundles, to a fresh ChatGPT conversation.

Collect the four Round 2 answers into a second bundle, same shape as the first. Then open a fresh conversation, fresh even though ChatGPT was one of the four: a model that argued a position in Round 2 will defend its own phrasing when asked to judge. The prompt makes it merge duplicates before ruling on the evidence, and forbids counting how many models agreed, because four models trained on overlapping data agreeing is weak evidence.

Read the rejected points, the open disagreements and the minority report before the action list. They exist only because four readings disagreed, which is what you paid for.

Check it yourself

The memo is evidence, not a verdict. Do this before you change a line.

1. Search your own file for every quotation behind a surviving criticism. Drop any you cannot find. Models fabricate quotations, and a fabricated one reads exactly like a real one.
2. Open every external source the memo names. Those points are unverified by construction. A confidently cited paper that does not exist, or exists and says something else, is the usual way a literature criticism goes wrong.
3. Redo any arithmetic the memo leans on. Do not accept a recomputed number from any of the five conversations.
4. Decide, and write down what you decided. You keep authorship and responsibility. Where the memo and a reader you trust disagree on a high-severity finding, settle it on the evidence rather than on who sounded surer. The note of what you accepted is what stops a second run relitigating the first.