How I measure the work

Scoped Spider dev-split run of DBWhisper’s generation model (qwen/qwen3-32b at temp 0.1) with each database’s schema in context: 139 questions across 18 databases. Generated SQL is executed against the real Spider SQLite databases and compared by result set — ordered when the gold query has ORDER BY, multiset otherwise (standard execution match). 101/139 correct; malformed generations count as incorrect. 9 of the 148 sampled questions could not be scored after repeated provider throttling and are excluded, not counted either way. Measures the NL→SQL generation core; the deployed agent adds schema retrieval and a read-only validator on top.

22 natural-language golden queries + 4 unsafe/out-of-scope prompts over a read-only Postgres store, run end-to-end through DBWhisper’s live pipeline (schema retrieval → generation → read-only validator → execute). 82% exact result-set match (18/22); 95% (21/22) when crediting correct answers that returned an extra column. All 4 destructive or out-of-scope prompts were refused fail-closed (4/4).

5-fold stratified cross-validation on 15,115 NBA games — replacing an earlier post-hoc “best of 300 random splits” 68%. The base home-win rate is 57.5%, so this is a real ~8-point edge (ROC-AUC 0.685). Published picks are then graded against the real final score, and the track record is flagged “insufficient” below 20 settled picks.

The engine decides on closed bar i and fills at bar i+1’s open. A canary multiplies every future bar by 3× and asserts the past equity curve is byte-identical; any look-ahead leak changes the curve and fails the build. Correctness is proven structurally, not scored on returns — the platform deliberately publishes no flattering performance number.