- model accuracyHow this was measured — 5-fold stratified cross-validation on 15,115 NBA games; base home-win rate is 57.5%, so a real ~8-point edge (ROC-AUC 0.685).
- 65.2% ± 0.8%
- track-recordHow this was measured — Every published pick is settled against the real final score; the record is flagged 'insufficient' below 20 settled picks.
- graded vs finals
Context
Sports-betting "tip" sites have a credibility problem, and it's almost always the same three failures: unvalidated models (accuracy quoted from one lucky split), naive edge math (comparing a model probability to raw book odds, ignoring the vig), and no accountability (picks published, then quietly forgotten). CrownWager was built to be defensible on all three — a cross-validated model, explicit Decimal money math, and a track record graded against reality.
Constraints
It's a ground-up rebuild of a legacy monolith I audited and judged unshippable — no rate limiting, secrets in the repo, no auth hardening — so security was a first-class requirement rather than a follow-up. I'm not publishing the score I gave it: it was my own rubric, applied once, to a codebase nobody else can inspect. The rebuild's hardening is checkable in CI, which is the part that can actually be verified. And it is a responsible-gambling product, which constrains the assistant on purpose.
System architecture
Three services separate model loading and inference from the web backend: Django/DRF, a separate FastAPI ML service in its own container, and a Next.js frontend. Django never imports XGBoost — all inference happens over HTTP, and a model registry serves only a model whose checksum is validated, otherwise a clearly-labeled baseline.
System map
Tap to inspect
- 01
The frontend; talks to Django over REST + JWT.
- 02
Implied prob, edge, expected value, and half-Kelly staking — all in Decimal, no float drift.
- Instead of
- Float arithmetic (the legacy approach)
- Why
- Odds/EV/Kelly/arbitrage math accumulates float error — 1/1.91 + 1/1.95 doesn't land on exactly 1.00, so a narrow arbitrage can be missed. Every money calculation uses Decimal(str(value)) and quantizes to the cent. Staking is half-Kelly (0.5) — professional practice, ~50% less variance than full Kelly.
- Tradeoff
- Decimal is 2–3× slower than float — imperceptible on a single prediction, and worth it for money math.
The backend: odds, EV/Kelly, best bets, graded record, and CrownBot — heavy ML never runs in-process.
Every pick grades against the real final score; per edge tier; flagged insufficient below 20 settled picks.
- Instead of
- Backfilling or interpolating outcomes that can't be mapped
- Why
- A pick graded wrong inflates or deflates the model's record. Unmappable picks are voided, not guessed. Results settle against real final scores, are broken out per edge tier (0–2%, 2–5%, 5–10%, 10%+) to expose lucky buckets, and the record is flagged 'insufficient' until 20 picks settle.
- Tradeoff
- Voiding shrinks the sample, so the record builds slowly — but it's honest.
Claude, grounded strictly in the current best bets; forbidden from pushing wagers or promising profit.
- 03
Shallow model (max_depth 3) predicting NBA moneyline; accuracy from 5-fold stratified CV.
- Instead of
- A neural net, and the legacy 'best of 300 random splits' 68% claim
- Why
- The inherited claim was cherry-picked after the fact, never validated. XGBoost was chosen for reproducibility (one fixed split + 5-fold stratified CV → 65.2% ± 0.8% on 15,115 games vs a 57.5% base rate), interpretability (depth capped at 3, real feature importances), and fast JSON serving. Honesty over a marginal, unverifiable accuracy gain.
- Tradeoff
- 65% isn't state-of-the-art — but it's a number that holds up.
Inference over HTTP /predict; Django never imports XGBoost.
- Instead of
- Importing the ~500 MB ML stack into the Django web process
- Why
- Embedding XGBoost/TensorFlow in the web process makes every request thread carry the weight, and a slow model starves web workers. The model runs as a separate FastAPI service; Django calls it over HTTP (15s timeout) behind a circuit breaker — after 3 consecutive failures it fails fast for 30s and serves a labeled baseline instead of hanging.
- Tradeoff
- Two services to deploy and a network hop — bought decoupling, independent scaling, and zero downtime on model updates.
Validates checksums; serves only a model flagged validated:true, else a labeled baseline.
- Instead of
- Serving whatever model binary is on disk
- Why
- The model is promoted only if it clears a threshold on held-out data and its SHA-256 checksum matches the manifest. If no validated model is present, the service returns a transparent, documented baseline (home-court + rating + rest logits) — never vaporware.
- Tradeoff
- A gate between training and serving — deliberate friction that keeps unproven models out of production.
- 04
Odds, bets, and graded results; Celery + Redis for async jobs and cache.
Hover a node to preview · click one to see the decision behind it
Use the arrow keys, Home, or End to move between nodes. Press Enter or Space to pin a node's decision, and Escape to release it.
- Next.js 15: The frontend; talks to Django over REST + JWT.
- Django + DRF: The backend: odds, EV/Kelly, best bets, graded record, and CrownBot — heavy ML never runs in-process.
- EV / Kelly: Implied prob, edge, expected value, and half-Kelly staking — all in Decimal, no float drift. Why: Odds/EV/Kelly/arbitrage math accumulates float error — 1/1.91 + 1/1.95 doesn't land on exactly 1.00, so a narrow arbitrage can be missed. Every money calculation uses Decimal(str(value)) and quantizes to the cent. Staking is half-Kelly (0.5) — professional practice, ~50% less variance than full Kelly. Instead of: Float arithmetic (the legacy approach). Tradeoff: Decimal is 2–3× slower than float — imperceptible on a single prediction, and worth it for money math.
- Graded record: Every pick grades against the real final score; per edge tier; flagged insufficient below 20 settled picks. Why: A pick graded wrong inflates or deflates the model's record. Unmappable picks are voided, not guessed. Results settle against real final scores, are broken out per edge tier (0–2%, 2–5%, 5–10%, 10%+) to expose lucky buckets, and the record is flagged 'insufficient' until 20 picks settle. Instead of: Backfilling or interpolating outcomes that can't be mapped. Tradeoff: Voiding shrinks the sample, so the record builds slowly — but it's honest.
- CrownBot: Claude, grounded strictly in the current best bets; forbidden from pushing wagers or promising profit.
- FastAPI ML: Inference over HTTP /predict; Django never imports XGBoost. Why: Embedding XGBoost/TensorFlow in the web process makes every request thread carry the weight, and a slow model starves web workers. The model runs as a separate FastAPI service; Django calls it over HTTP (15s timeout) behind a circuit breaker — after 3 consecutive failures it fails fast for 30s and serves a labeled baseline instead of hanging. Instead of: Importing the ~500 MB ML stack into the Django web process. Tradeoff: Two services to deploy and a network hop — bought decoupling, independent scaling, and zero downtime on model updates.
- Model registry: Validates checksums; serves only a model flagged validated:true, else a labeled baseline. Why: The model is promoted only if it clears a threshold on held-out data and its SHA-256 checksum matches the manifest. If no validated model is present, the service returns a transparent, documented baseline (home-court + rating + rest logits) — never vaporware. Instead of: Serving whatever model binary is on disk. Tradeoff: A gate between training and serving — deliberate friction that keeps unproven models out of production.
- XGBoost: Shallow model (max_depth 3) predicting NBA moneyline; accuracy from 5-fold stratified CV. Why: The inherited claim was cherry-picked after the fact, never validated. XGBoost was chosen for reproducibility (one fixed split + 5-fold stratified CV → 65.2% ± 0.8% on 15,115 games vs a 57.5% base rate), interpretability (depth capped at 3, real feature importances), and fast JSON serving. Honesty over a marginal, unverifiable accuracy gain. Instead of: A neural net, and the legacy 'best of 300 random splits' 68% claim. Tradeoff: 65% isn't state-of-the-art — but it's a number that holds up.
- PostgreSQL: Odds, bets, and graded results; Celery + Redis for async jobs and cache.
Data flow
A request hits Django, which calls the ML service's /predict for a home-win probability, turns it into fair-odds edge and expected value, sizes a stake with half-Kelly, and persists the pick. When the game is final, the pick is graded against the real score.
Key decisions
Cross-validated accuracy over a lucky split
Instead of: report a single train/test split, quote accuracy without measuring it
The XGBoost model is trained deterministically and reported from 5-fold stratified cross-validation, and kept intentionally shallow (max_depth 3) to resist overfitting — a real edge above the base home-win rate.
Trade-off accepted — A shallow model leaves some accuracy on the table versus an aggressively-tuned one — in exchange for a number you can trust.
Explicit, Decimal-precise edge and staking math
Instead of: float arithmetic, full Kelly staking
Implied prob is 1/odds, edge is model − implied, EV is explicit, and stakes use half-Kelly — all in Decimal so real-money arithmetic doesn't drift.
Trade-off accepted — Half-Kelly grows a bankroll slower than full Kelly, deliberately, to survive variance.
A verifiable, graded track record — not a wall of pending picks
Instead of: show pending picks and hope, backfill an impressive history
Every pick settles only once its game is FINAL with both scores; results aggregate per edge tier and are flagged 'insufficient' below 20 settled picks; unmappable picks are voided.
Trade-off accepted — The record starts empty and grows slowly — the price of it being real.
Graceful degradation everywhere
Instead of: let a slow dependency hang the request
A circuit breaker on the ML client, plus fallbacks for the odds API and the assistant, provide a labeled demo mode for handled dependency failures.
Trade-off accepted — More fallback paths to maintain and test, and reduced functionality while dependencies are unavailable.
A grounded assistant over a persuasive one
Instead of: a salesy chatbot that pushes wagers
CrownBot is grounded strictly in the current best bets; its prompt forbids telling anyone to place a wager, forbids promising profit, and mandates responsible-gambling messaging.
Trade-off accepted — A less 'salesy' assistant — which is the entire point for this domain.
What failed
A slow or dead ML service used to block web requests while their calls timed out. The fix was a circuit breaker: after three consecutive failures it opens and fast-fails for thirty seconds, falling back to clearly-labeled demo predictions instead of hanging.
Cause — a slow or dead ML service blocked web requests on timeouts
Fix — circuit breaker opens after 3 failures, fast-fails 30s, and drops to labeled demo predictions
Performance & cost
| Metric | Value | Method |
|---|---|---|
| accuracy (CV) | 65.2% ± 0.8% | 5-fold stratified cross-validation on 15,115 NBA games (2012–24), one fixed random split — no cherry-picking. The base home-win rate in the data is 57.5%, so this is a real ~8-point edge. |
| held-out test | 64.3% | 80/20 hold-out on 3,023 unseen games; ROC-AUC 0.685, log-loss 0.628 — the calibration signal that keeps Kelly staking from over-sizing. |
| model shape | 106 features · depth 3 | 750 shallow trees (max_depth 3, learning rate 0.01) over 106 pre-game features; kept deliberately shallow to resist overfitting and stay interpretable. |
| track record | graded vs finals | Picks settle only on FINAL scores; aggregated per edge tier; unmappable picks voided; flagged insufficient below 20 settled picks. |
| money math | Decimal | Edge, EV, and half-Kelly (0.5) staking in Decimal; arbitrage flagged when Σ(1/odds) < 1, legs sized to equalize payout. |
| legacy → rebuild | hardened in CI | Rebuilt from a legacy monolith with proxy-aware per-IP rate limiting, admin 2FA (TOTP), constant-time token comparison, CSP, and blocking bandit + gitleaks + dependency audits in CI. The pre-rebuild audit was my own rubric on a private codebase, so no score is published for it. |
Operations
Celery + Redis run the async jobs; the ML service runs in its own container behind the circuit breaker. Security is hardened by default: proxy-aware per-IP rate limiting, constant-time token comparison, optional TOTP 2FA, DSN-gated Sentry, and a tested backup/restore runbook.
What I'd do differently
The model is deliberately simple; with more graded history I'd add calibration reporting (are the 60% picks actually winning 60%?) and a closing-line-value check, which is a stronger signal of edge than raw win rate.
Evidence
The app is live and the backend, ML service, and grading logic are in the repository above.
Changelog
A verifiable model track-record and prediction confidence grades; cut a cherry-picked split in a reality pass.
Best-bets credibility layer — fair odds, confidence grades, and filters.
Security hardening: admin 2FA, cookie-JWT auth, a circuit breaker on the AI client, and blocking CI gates.
Card-free free-tier deploy (Hugging Face Space + Vercel), bankroll ROI/P&L tracking, and launch hardening.
First build — a validated XGBoost NBA moneyline model served behind the analytics platform.