idea-forge is a Claude Code skill that hardens an idea by making rival versions of it fight. To test it, we pointed it at the one idea it knew best — its own design — and let 8 Opus agents argue it down to a single, reinforced winner. This is the measured before → after.
An agent fleshed out the concept, wrote v1 of the skill, then the skill was run on itself. Every box below was executed by Claude agents — the only human step was granting GitHub access to publish this page.
An agent self-ran a Socratic interview across 8 dimensions + an adversarial stress-test, producing an idea brief.
A single-elimination debate bracket: 8 variants, 7 debates, a judge per pair, carry-forward reinforcement.
v1's own tournament was run with the seed idea set to “improve the idea-forge skill.”
A king-of-the-hill ladder with an audited merge and a provable floor guarantee. This page.
The tournament kept idea-forge's adversarial, carry-forward soul but rebuilt the machine around it. The winning direction came from contender #3 — “minimize feasibility / cost risk.”
“v1 calibrated the selector carefully but the carry-forward edit was asserted, never audited, and the one thing it shipped was never measured… v2 keeps the adversarial, divergent, carry-forward soul but audits the synthesizer as hard as the selector — each graft is re-validated against a composable ledger and must match-or-beat the previous champion or roll back, and a mandatory audit rung ships the champion only if it provably clears a calibrated margin over the reserved original. The result is cheaper per round and carries a real guarantee v1 lacked: a champion provably no worse than the best original.”
| Dimension | v1 — before | v2 — after |
|---|---|---|
| Selection structure | Single-elimination bracket, 7 debates | King-of-the-hill ladder, running merge re-architected |
| Seeding | Spawn order; first mover gets incumbency | Cheap pre-screen ranks the 8; the a-priori-best seeds champion-zero new |
| Reinforcement | Winner absorbs one objection (edit asserted) | Audited graft + composable graft ledger audited |
| Synthesizer safety | None — merge never re-checked | Predecessor audit + rollback on regression new |
| Floor guarantee | None — can ship worse than a parent | Audit rung vs frozen reserve: ship only if provably better new |
| Judge calibration | Quoted lines + tie-break ladder | Noise-floor band + position-swapped recheck stronger |
| Cost per run | ~24–30 Opus calls | ~18–24 (lite ~9–12) cheaper |
| Honesty about limits | Claimed carry-forward “is the product” | Claims only “no worse than best original”; names residual path-dependence honest |
Each contender attacked v1 from a different angle. They debated in pairs; a judge scored each on five criteria and quoted the deciding line; the winner absorbed the loser's strongest objection before advancing. Contender #3 — the one obsessed with cutting cost and feasibility risk — won the final.
“The apparatus certifies every pairwise verdict to the hilt but never once evaluates the thing it actually ships — the accumulated champion. You can win all rungs by clearing margins and still ship a chimera worse than the best single original, because the load-bearing claim is the one quantity the system never measures.”
The champion was then amended to answer exactly this: the predecessor audit, the graft ledger, and the final reserve audit rung all exist because this objection landed. That is reinforcement under selection — working on the skill itself.
This is the actual changelog the synthesis step emitted — not a summary written after the fact.
Replaced the single-elimination 8→4→2→1 bracket with a ladder whose advancing artifact is a running merge that compounds every compatible fix; the champion only ever accumulates and is never wholesale-replaced.
driven by → gauntlet's path-dependence + unaudited-synthesizer objectionOne cheap Opus pass ranks the 8 originals, names the strongest as champion-zero, and freezes a pristine copy in reserve — so incumbency accrues to the a-priori-best, not to spawn order.
driven by → open risk: incumbency was handed to a random first moverEach round is a single gated call returning steelman → scores (with verbatim rubric quotes) → winner (margin + self-consistency retry) → salvage → merge → predecessor audit — cutting cost from ~6 calls/debate to 1–3/rung.
driven by → the winning contender's cost mandateEvery new graft must assert non-contradiction against each live ledger entry; a flagged conflict forces a resolve-edit that keeps the stronger graft and updates the ledger, instead of silently accumulating contradictions.
driven by → riskiest-assumption: compatibility may be non-monotonicAny non-no-op merge is re-judged against the immediately previous champion; a regression is independently confirmed and rolled back — costing the graft but not the recorded verdict. Selection and synthesis are decoupled.
driven by → core gauntlet objection: the synthesizer was never auditedThe accumulated champion ships only if it clears the noise floor over the pristine original — otherwise the original ships. “Better than the best original” becomes a measured gate and a hard floor.
driven by → “the system never measures the thing it ships”Margin ≥2 auto-certifies; a verdict inside the ~1-point band triggers one fresh position-swapped judgment with the prior verdict hidden — agree certifies, disagree means incumbency holds. Applied to ladder rungs and the audit rung.
driven by → judge-reliability risk on near-tiesStrengthened axis 3 and the feasibility criterion to “feasibility & cost,” with explicit verbatim rubric lines the judge must quote for every score.
driven by → the winning contender axisUpdated framing, red flags, cost numbers, lite-mode axes, worked example, and checklists to claim only “no worse than the best original, hardened by every surviving graft” — never a global optimum — and to name the residual path-dependence.
driven by → residual path-dependence open riskv1 could ship something worse than its own inputs. v2 cannot — the reserve audit rung is a provable floor. That is a categorical, not cosmetic, gain.
~24–30 → ~18–24 Opus calls per run, with a ~9–12 lite mode — even as three new audit mechanisms were added.
All 9 changes map to a specific objection raised and survived in the bracket — the mechanism did what it claims to do, on itself.
All 8 contenders and every judge are the same model family. The debate's diversity is real but capped — v2 names this and flags mixing model families as future work.
A running merge is path-dependent. The reserve caps the downside but a different seeding could compound a different set. v2 claims a floor, not an optimum — deliberately.
“Better” here is an LLM-judge verdict, not an external eval. Compelling and self-consistent — but the next honest step is an A/B against a flat ensemble.
Nothing here is paraphrased marketing — these are the actual artifacts the agents produced.