Benchmark appendix · Ground truth · June 2026
FindAll benchmark: the full answer set, and who found what
This is the complete, auditable ground truth behind the Subindex vs Parallel.ai FindAll set-finding benchmark. It lists every one of the 268 approved answer-set entities across six queries and marks, per entity, which systems actually returned it. Every recall figure on the summary page traces directly to a ✓ below — nothing is credited that was not retrieved.
Summary — coverage by system
One shipped run of Subindex (config R15) per query; each FindAll tier run through Parallel.ai's public API. "Found" counts distinct approved entities returned. Recall shown two ways — see definitions below.
| System | Found | Recall (micro) | Recall (macro) | Cost | Wall time |
|---|---|---|---|---|---|
| Subindex | 132/268 | 0.493 | 0.525 | $1.74 | 13 min |
| FindAll preview | 7/268 | 0.026 | 0.057 | $0.60 | 18 min |
| FindAll base | 30/268 | 0.112 | 0.146 | $6.51 | 64 min |
| FindAll core | 42/268 | 0.157 | 0.167 | $29.25 | 145 min |
Micro-recall (headline)
Pooled across all six queries: total entities found ÷ 268. Larger queries (e.g. 96-entity F500) weigh more. This is the number the per-entity ✓ marks sum to, and the one used on the summary page.
Macro-recall
Each query scored on its own, then the six recall values averaged with equal weight. A 15-entity query counts as much as a 96-entity one. Shown for completeness; it runs slightly higher here.
Per-query ground truth & coverage
Entities are the human-approved ground truth for each query, listed alphabetically. Matching uses a canonical-key normalization (lower-case, punctuation- and legal-suffix-stripped) applied identically to every system's output.
✓ found by Subindex
✓ found by that FindAll tier
· not returned
Fortune 500 companies that hired a new CEO in 2025
20/96 · 0.210/96 · 0.001/96 · 0.016/96 · 0.06
FindAll core's run on this query ran past our 30-minute poll window, but it completed on Parallel's side (candidates_exhausted, 19 matched). We fetched the finished run from Parallel's API and scored its actual delivered candidates — 6 of the 19 are approved ground truth (marked below). Nothing here is estimated or self-reported; its real billed cost ($4.85) is included in the core total.
| Ground-truth entity | Subindex | FindAll preview | FindAll base | FindAll core |
|---|---|---|---|---|
| Andrew Sullivan | ✓ | · | · | · |
| Andrew Sullivan (Prudential Financial) | · | · | · | · |
| Antonio Filosa, Stellantis | · | · | · | · |
| Arthur Starrs, Harley-Davidson | · | · | · | · |
| Bill Meury, Incyte | · | · | · | · |
| Brent Shafer | · | · | · | · |
| Brent Shafer (Baxter International) | · | · | · | · |
| Brian D. Pieninck | · | · | · | · |
| Chris Rogers (Instacart) | · | · | · | · |
| Chris Turner (Yum! Brands) | ✓ | · | · | · |
| Christopher Nelson, Stanley Black & Decker | ✓ | · | · | · |
| Creed (Caterpillar) | · | · | · | · |
| Damien McDonald, Enovis Corporation | · | · | · | · |
| Dan Schulman (Verizon) | ✓ | · | · | · |
| Daniel Heaf | ✓ | · | · | ✓ |
| Daniel Heaf (Bath and Body Works) | · | · | · | · |
| Daniel Heaf, Bath & Body Works | · | · | · | · |
| David Joyner | · | · | · | · |
| David Joyner (CVS Health) | · | · | · | · |
| David McCreight | · | · | · | · |
| Deanna Strable | · | · | · | · |
| Deanna Strable, Principal Financial Group | ✓ | · | · | · |
| Derek Chang | ✓ | · | · | · |
| Derek Chang, Liberty Media | · | · | · | · |
| Eduardo F. Menezes (Air Products) | · | · | · | · |
| Greg Abel (Berkshire Hathaway) | · | · | · | · |
| Greg Foran | · | · | · | · |
| Greg Foran, Kroger | · | · | · | · |
| Gunjan Kedia | · | · | · | ✓ |
| Gunjan Kedia, U.S. Bancorp | ✓ | · | · | · |
| Harry Sideris | · | · | · | · |
| Harry Sideris, Duke Energy Corporation | · | · | · | · |
| Henrique Braun (Coca-Cola) | ✓ | · | · | · |
| Jeffrey M. Ettinger | · | · | · | · |
| Jeffrey M. Ettinger (Hormel Foods) | · | · | · | · |
| Joakim Weidemanis (Johnson Controls) | · | · | · | · |
| John Furner (Walmart) | · | · | · | · |
| John Malone | · | · | · | · |
| John Malone (Liberty Media) | · | · | · | · |
| Joseph E. Creed, Caterpillar | · | · | · | · |
| Joseph Morrissey (Organon) | · | · | · | · |
| Juan C. Andrade (USAA) | ✓ | · | · | · |
| Juan C. Andrade, USAA | · | · | · | · |
| Karl K. Mistry | · | · | · | · |
| Karl K. Mistry (Toll Brothers) | ✓ | · | · | · |
| Kecia Steelman | · | · | · | ✓ |
| Kecia Steelman, Ulta Beauty | ✓ | · | · | · |
| Keith Barr | · | · | · | · |
| Ken Cook (Wendy's) | · | · | · | · |
| Kevin Hammons | · | · | · | · |
| Kirk L. Perry | · | · | · | · |
| Kirk L. Perry (Kenvue) | · | · | · | · |
| Kirk Perry | · | · | · | · |
| Kirk Perry (Kenvue) | · | · | · | · |
| Kirk Tanner (Hershey) | · | · | · | · |
| Lip-Bu Tan | · | · | · | · |
| Lip-Bu Tan, Intel Corporation | · | · | · | · |
| Markus Strobel (Coty) | ✓ | · | · | · |
| Matthew E. Avril (Marriott Vacations Worldwide) | · | · | · | · |
| Meg O’Neill (BP) | · | · | · | · |
| Michael Bender | · | · | · | ✓ |
| Michael Bender (Kohl's) | ✓ | · | · | · |
| Michael Fiddelke, Target | · | · | · | · |
| Michael P. Lyons | · | · | · | · |
| Michael P. Lyons, Fiserv | · | · | · | · |
| Michael T. Hutchins | · | · | · | · |
| Mick Beekhuizen | ✓ | · | · | · |
| Mick Beekhuizen (Campbell’s) | · | · | · | · |
| Nicholas Fink | · | · | · | · |
| Nicholas Fink, Constellation Brands | · | · | · | · |
| Patrick Kelleher | · | · | · | · |
| Paul Shoukry | ✓ | · | · | · |
| Paul Shoukry, Raymond James Financial | · | · | · | · |
| Richard Jackson | · | · | · | · |
| Robert J. Gaudette | · | · | · | · |
| Robert J. Gaudette (NRG Energy) | ✓ | · | · | · |
| Ron Sargent | · | · | · | ✓ |
| Ron Sargent (Kroger) | ✓ | · | · | · |
| Ronald J. Lewis | · | · | · | · |
| Ronald J. Lewis (Ball) | · | · | · | · |
| Ronnie Pruitt | · | · | · | · |
| Ronnie Pruitt, Vulcan Materials | · | · | · | · |
| Sanoke Viswanathan | · | · | · | · |
| Scott A. Richardson | ✓ | · | · | · |
| Scott A. Richardson, Celanese | · | · | · | · |
| Scott McPherson | · | · | · | · |
| Scott McPherson, Performance Food Group | · | · | · | · |
| Shailesh Jejurikar | · | · | ✓ | · |
| Shailesh Jejurikar (P&G) | · | · | · | · |
| Shailesh Jejurikar (Procter & Gamble) | ✓ | · | · | · |
| Srini Gopalan (T-Mobile) | · | · | · | · |
| Stephen Hemsley | · | · | · | ✓ |
| Steve Cahillane | · | · | · | · |
| Steve Cahillane (Kraft Heinz) | · | · | · | · |
| Steve Cahillane (The Kraft Heinz Company) | · | · | · | · |
| Tom Conrad | · | · | · | · |
Michelin 1-star restaurants in Manhattan (2025 guide)
62/76 · 0.820/76 · 0.0013/76 · 0.1720/76 · 0.26
| Ground-truth entity | Subindex | FindAll preview | FindAll base | FindAll core |
|---|---|---|---|---|
| 63 Clinton | ✓ | · | ✓ | · |
| Acru | ✓ | · | · | · |
| Aquavit | · | · | · | ✓ |
| Atera | · | · | · | · |
| Atomix | ✓ | · | · | · |
| Bar Miller | ✓ | · | · | ✓ |
| Bom | ✓ | · | ✓ | ✓ |
| Bridges | ✓ | · | ✓ | ✓ |
| Café Boulud | ✓ | · | · | · |
| Casa Mono | ✓ | · | · | · |
| Caviar Russe | · | · | · | · |
| Corima | ✓ | · | · | · |
| Cote | ✓ | · | ✓ | · |
| Crown Shy | ✓ | · | ✓ | · |
| Daniel | ✓ | · | ✓ | ✓ |
| Dirt Candy | ✓ | · | · | ✓ |
| Don Angie | ✓ | · | · | · |
| Essential by Christophe | ✓ | · | · | ✓ |
| Estela | ✓ | · | ✓ | · |
| Family Dog | · | · | · | · |
| Family Meal at Blue Hill | ✓ | · | · | · |
| Francie | · | · | · | · |
| Frevo | ✓ | · | · | ✓ |
| Gramercy Tavern | ✓ | · | ✓ | ✓ |
| Hirohisa | ✓ | · | ✓ | · |
| Huso | ✓ | · | · | ✓ |
| Icca | ✓ | · | ✓ | · |
| Jeju Noodle Bar | ✓ | · | · | · |
| Jua | ✓ | · | · | · |
| Jōji | ✓ | · | · | · |
| Kochi | ✓ | · | · | · |
| Kono | · | · | · | · |
| Kosaka | ✓ | · | · | ✓ |
| L'Abeille | ✓ | · | · | · |
| La Bastide by Andrea Calstier | · | · | · | · |
| Le Coucou | ✓ | · | · | · |
| Le Jardinier | ✓ | · | · | · |
| Le Pavillon | ✓ | · | · | · |
| Mari | ✓ | · | · | · |
| Maru | ✓ | · | · | · |
| Meju | ✓ | · | · | · |
| Muku | ✓ | · | · | · |
| Noda | ✓ | · | · | ✓ |
| Noz 17 | ✓ | · | · | · |
| Nōksu | ✓ | · | ✓ | ✓ |
| Odo | · | · | · | · |
| Odre | · | · | · | · |
| Oiji Mi | ✓ | · | · | ✓ |
| One White Street | ✓ | · | · | · |
| Onedera | · | · | · | · |
| Oxalis | ✓ | · | · | · |
| Restaurant Yuu | ✓ | · | · | · |
| Rezdôra | ✓ | · | ✓ | ✓ |
| Semma | ✓ | · | · | · |
| Shion 69 Leonard Street | ✓ | · | · | ✓ |
| Shmoné | ✓ | · | · | · |
| Shota Omakase | ✓ | · | · | · |
| Sushi Ichimura | ✓ | · | · | · |
| Sushi Nakazawa | · | · | · | ✓ |
| Sushi Nakazawa New York | ✓ | · | · | ✓ |
| Sushi Noz | · | · | · | · |
| Sushi Sho | ✓ | · | · | · |
| Takeda | · | · | · | · |
| Tempura Matsui | ✓ | · | · | · |
| The Modern | ✓ | · | · | · |
| The Musket Room | ✓ | · | · | · |
| The Noortwyck | · | · | · | · |
| Torien | ✓ | · | · | · |
| Torrisi | ✓ | · | · | ✓ |
| Tsubame | ✓ | · | · | · |
| Tsukimi | ✓ | · | · | · |
| Tuome | ✓ | · | ✓ | ✓ |
| Vestry | ✓ | · | · | · |
| Yamada | ✓ | · | · | · |
| YingTao | ✓ | · | · | · |
| Yoshino | ✓ | · | · | · |
Notable OpenAI safety / alignment staff who departed, 2024–2026
23/32 · 0.723/32 · 0.098/32 · 0.258/32 · 0.25
| Ground-truth entity | Subindex | FindAll preview | FindAll base | FindAll core |
|---|---|---|---|---|
| Aleksander Madry | · | · | · | · |
| Andrea Vallone | ✓ | · | · | · |
| Barret Zoph | ✓ | · | · | · |
| Bob McGrew | ✓ | · | · | · |
| Carroll Wainwright | · | · | · | · |
| Collin Burns | ✓ | · | · | · |
| Cullen O'Keefe | · | · | · | ✓ |
| Daniel Kokotajlo | ✓ | ✓ | ✓ | ✓ |
| Gretchen Krueger | ✓ | · | ✓ | ✓ |
| Ilya Sutskever | ✓ | ✓ | ✓ | ✓ |
| Jan Hendrik Kirchner | ✓ | · | · | · |
| Jan Leike | ✓ | ✓ | ✓ | ✓ |
| Jeffrey Wu | ✓ | · | · | · |
| John Schulman | ✓ | · | · | · |
| Jonathan Uesato | ✓ | · | · | · |
| Leopold Aschenbrenner | ✓ | · | ✓ | ✓ |
| Lilian Weng | ✓ | · | ✓ | · |
| Max Schwarzer | · | · | · | · |
| Miles Brundage | ✓ | · | ✓ | ✓ |
| Pavel Izmailov | ✓ | · | · | · |
| Richard Ngo | · | · | · | · |
| Rosie Campbell | ✓ | · | · | · |
| Ryan Beiermeister | · | · | · | · |
| Ryan Lowe | · | · | · | · |
| Stephen Adler | ✓ | · | · | · |
| Steven Bills | ✓ | · | · | · |
| Suchir Balaji | · | · | · | · |
| Todor Markov | ✓ | · | · | ✓ |
| William Saunders | ✓ | · | ✓ | · |
| Yo Shavit | · | · | · | · |
| Yuri Burda | ✓ | · | · | · |
| Zoë Hitzig | ✓ | · | · | · |
EU-based seed-stage AI-safety startups
6/28 · 0.210/28 · 0.001/28 · 0.041/28 · 0.04
| Ground-truth entity | Subindex | FindAll preview | FindAll base | FindAll core |
|---|---|---|---|---|
| Adaptive ML | ✓ | · | · | · |
| AI Score | · | · | · | · |
| Alinia | · | · | · | · |
| Alinia AI | ✓ | · | ✓ | ✓ |
| Calvin Risk | · | · | · | · |
| DAIKI GmbH | · | · | · | · |
| Deep Safety GmbH | · | · | · | · |
| Formel AI | ✓ | · | · | · |
| Giskard | · | · | · | · |
| H | · | · | · | · |
| Harmattan AI | · | · | · | · |
| Kyutai | · | · | · | · |
| LangWatch | · | · | · | · |
| Lexsi Labs | · | · | · | · |
| Lighthouse AI | ✓ | · | · | · |
| Maisa | · | · | · | · |
| Noru | · | · | · | · |
| Phospho | ✓ | · | · | · |
| Principled Intelligence | · | · | · | · |
| PRISM Eval | · | · | · | · |
| Redpine | · | · | · | · |
| Root Signals | · | · | · | · |
| SaferAI | · | · | · | · |
| Saidot | ✓ | · | · | · |
| TetraxAI | · | · | · | · |
| Timaeus | · | · | · | · |
| trail | · | · | · | · |
| Transcendence | · | · | · | · |
Upscale restaurants in Lawrence, NY
11/21 · 0.521/21 · 0.053/21 · 0.144/21 · 0.19
| Ground-truth entity | Subindex | FindAll preview | FindAll base | FindAll core |
|---|---|---|---|---|
| Akimori | · | · | · | · |
| Anju | ✓ | · | · | · |
| Bellagio Cafe And Restaurant | · | · | · | · |
| Bellagio Cafe NY | · | · | · | · |
| BH Steakhouse | · | · | · | · |
| Cho-Sen Island | ✓ | · | · | · |
| Cork & Slice | ✓ | · | · | · |
| Doma Land + Sea | ✓ | · | ✓ | ✓ |
| Greenview at Lawrence Yacht & Country Club | ✓ | · | ✓ | ✓ |
| Hibana Kosher Hibachi | ✓ | · | · | · |
| MG Craft Kitchen | · | · | · | · |
| MUR restaurant and Bar | ✓ | · | · | · |
| NOVO | ✓ | · | · | · |
| Novo Restaurant | · | · | · | ✓ |
| Novo Steakhouse | · | · | · | · |
| Ondas by Fuego | ✓ | · | · | · |
| Prime 1024 | ✓ | · | · | · |
| Prime Bistro | ✓ | ✓ | ✓ | ✓ |
| Traditions | · | · | · | · |
| Traditions Restaurant | · | · | · | · |
| Traditions: Lawrence | · | · | · | · |
Top-10 performing S&P 500 stocks, Q1 2026
10/15 · 0.673/15 · 0.204/15 · 0.273/15 · 0.20
| Ground-truth entity | Subindex | FindAll preview | FindAll base | FindAll core |
|---|---|---|---|---|
| APA Corporation | · | · | · | · |
| Ciena Corp. (CIEN) | ✓ | · | · | ✓ |
| Comfort Systems USA Inc. (FIX) | ✓ | · | · | · |
| Corning Inc. (GLW) | ✓ | · | · | ✓ |
| Dow Inc. | · | · | · | · |
| Generac Holdings Inc. (GNRC) | ✓ | · | ✓ | · |
| Lumentum Holdings | · | · | · | · |
| Lumentum Holdings Inc. | · | ✓ | · | · |
| LyondellBasell | · | · | · | · |
| Moderna Inc. (MRNA) | ✓ | · | ✓ | · |
| Sandisk Corp. (SNDK) | ✓ | ✓ | ✓ | · |
| Seagate Technology Holdings (STX) | ✓ | · | ✓ | · |
| Teradyne Inc. (TER) | ✓ | · | · | · |
| Texas Pacific Land Corp. (TPL) | ✓ | ✓ | · | ✓ |
| Western Digital (WDC) | ✓ | · | · | · |
Method & caveats
- Recall metric
- Micro-recall = approved entities returned ÷ total approved ground truth (268), pooled over all six queries. Headline figure.
- Runs counted
- Subindex: one run of the shipped R15 configuration per query (batch 20260608T191556Z), candidate set scored directly. FindAll: best available run per generator tier per query via Parallel.ai's public API; match_status: matched candidates counted.
- Ground truth
- Built cumulatively from candidates surfaced across every system's runs, then human-reviewed. Frozen before these runs (all approvals dated on or before 2026-05-27), so denominators are fixed and identical for every system.
- FindAll core · F500 recovery
- FindAll core's F500-CEOs run outlasted our 30-minute poll cap, so it was first recorded as a zero. The run in fact finished server-side (candidates_exhausted); we fetched the completed run from Parallel's API (findall_f1845…) and scored its actual output: 19 matched candidates, 6 of them approved GT. That real result (and its real $4.85 cost) is what appears here — not FindAll's self-reported count, and not an estimate.
- Pro tier
- Not run. Projected on the summary page from Parallel's published pricing ($10 fixed + $1/match) and the recall-vs-tier trend; excluded from this per-entity appendix because there is nothing measured to list.
- Fairness
- Identical queries, ground truth, canonicalizer, and scorer for all systems. Cost is each system's own billed total; wall time is end-to-end.
- Measured
- 2026-06-08.