Raid simulator accuracy

Known gaps

• Mewtwo's #1 is still not Pokebattler's — and neither is Pokebattler's. We lead with Tyranitar (637 s) and Hydreigon second (639 s). Read live on 2026-09-07 that page led with Hydreigon (649.3 s); our own scrape of the same page on 2026-10-03 leads with Gholdengo (651.0 s), with Hydreigon second — the site's own answer for this boss moved inside a day without our code changing, because its first five sit inside 1.5% of one another. That is the honest shape of this gap. It is worth being precise about the pair anyway: Tyranitar and Hydreigon both run Bite / Brutal Swing, so their damage ratio is fixed by Attack alone (Hydreigon 271 to Tyranitar 266, 1.9% in Hydreigon's favour) and Pokebattler's own times differ by almost exactly that (663.4 / 649.3 = 2.2%), i.e. on this pair its answer is pure damage output. Ours is not: Tyranitar is the bulkier of the two into most of Mewtwo's movepool, and in our simulator that buys back the 1.9% and a second more, so it leads on time and on damage per life. Closing it needs the surviving few percent of per-attacker error, not a different column — so until then /raids names every attacker within 2.1% of the leader as a joint top pick instead of printing a #1 it cannot defend. On Mewtwo that band is Tyranitar and Hydreigon; on Rayquaza it is Mamoswine (the live page's own #1) and Baxcalibur.
• What 2.1% is. Two error bars, measured separately, one of them much larger than the other. Re-rolling the Monte Carlo seed moves an attacker's time to win by about 0.1% (five seeds on Mewtwo span 636-638 s; the page prints the leader's own figure). The same attacker's time differs from Pokebattler's published number by a median of 1.6% (p90 4.0%, worst 14.5%). So this simulator is precise to a tenth of a percent and accurate to about two and a half, and any two attackers inside that are a tie it cannot break. The band is the median, not the p90: a p90 band would swallow most of a top ten and stop being information.
• Pokebattler's Mega Level slider scales a Mega's “+” signature move only (Future Sight+, Drill Peck+, Outrage+). We simulate a Base-level Mega, so the reference is pinned to Mega Level 1; at the site's default Max level those three rank higher there than here.
• The weakest row is Mewtwo, page default, at 70% — the one cell under the 70% bar this round. It is a boundary shuffle in a tightly clustered field: every attacker we place that Pokebattler does not, and every one it places that we do not, sits within 8% of its own time for that species, on a list whose top ten spans 829 s to 1010 s. Recorded rather than tuned away.
• The boss no longer charges off the damage it takes — and that was this simulator's biggest error. Until this round a raid boss gained 0.5 energy per HP of damage it took, the rate a trainer's Pokemon gains. A Tier 5 boss loses 15,000 HP in one fight: that is 7,500 energy against a 100 cap, so it sat at full energy from the opening seconds and fired its most expensive charged move on every decision it was allowed. Real raid bosses land a charged attack every 20-30 s. Graded against Pokebattler's own published effectiveDeaths over the 179 attacker rows of these six bosses, our faint counts were 2.56x too many; a raid boss that charges only from its own fast moves brings that to 1.47x, and our damage dealt per attacker life — the bulk half of the Overall column — from 2.93x off its published Power to 1.71x. The one knob this leaves free, how long a wipe-and-relobby costs, is swept against the same 179 rows by npx tsx gauntlet/validate-raid.ts --calibrate and set to 25 s, where the median time-to-win bias is +0.2%.
• Faint counts still are not quite the same quantity on the two sides and are reported as a ratio rather than a bias. We simulate one trainer soloing the boss and relobbying a fresh party of six after every wipe; Pokebattler simulates a group and publishes a group-normalised figure with no relobby in it at all (its published Time to Win adds “2 seconds ... for each time the Pokemon fainted” and nothing else). The residual ratio now runs 0.9x to 2.2x rather than the 3x it was.
• Why Mewtwo's faint ranking disagrees. Mewtwo's own movepool contradicts itself about who is bulky, and it is the only reference boss that does. Rank the same fifteen attackers by faints under Psycho Cut/Focus Blast, rank them again under Confusion/Psychic, and the two orders correlate -0.75 — they nearly invert. Both of Mewtwo's fast moves are Psychic, which the Dark half of its counter list triple-resists at 0.39x while the Ghost half takes it neutrally; Focus Blast is the exact mirror (1.6x into Dark, 0.39x into Ghost). So “who survives Mewtwo” has two opposite answers decided by a coin flip, and the mean over an even roll of its ten movesets is a fragile ordering for either simulator. No other reference boss is even negative (Machamp -0.27, the rest +0.2 to +1.0), and no other boss has a faint correlation under 0.71. Time to win is untouched at 0.95, because it is driven by the attacker's own damage output, which does not invert. Pokebattler agrees with us on the same boss's time-to-win ordering and on all ten of its top ten.
• Two candidate fixes for that were tested this round and rejected on the evidence, so neither is in the engine. (1) Pokebattler's reference URL carries dodgeStrategy=DODGE_REACTION_TIME, which would explain its attackers lasting 2-4x longer than ours — but simulating dodging pushes the time-to-win bias from +0.4% to +6.3% on Mewtwo and +20.8% on Rayquaza and drops the faint correlation further, to 0.25. Its numbers behave as if nothing dodges, so nothing dodges here. (2) Counting only the faints inside one raid timer, the window Pokebattler's group fight occupies, is the more like-for-like quantity in principle; measured, it correlates worse than the whole-fight count on five of six bosses (Mewtwo 0.23 against 0.42). The engine records it as deathsWithinTimer but does not rank on it.