Method: Postgres, MySQL, and SQL Server via SQLAlchemy 2.0 + ODBC, with per-dialect generation directives.
Context
Natural-language-to-SQL is easy to demo and hard to trust. Calling an LLM to write SQL takes an afternoon; the reason most such tools never reach production is that a model will happily emit DROP TABLE, a multi-statement payload, or a query against a table that doesn't exist. DBWhisper turns a plain-English question into safe, read-only SQL across Postgres, MySQL, and SQL Server, runs it, and explains the answer — with the guarantee that it can read your data but never change it.
Constraints
Three constraints shaped the design. Safety must be structural— the read-only promise can't depend on the model behaving. Schemas are large — real databases have dozens to hundreds of tables, too many to paste into a prompt, so the agent has to retrieve the right ones. And it runs on a $0 stack (Vercel + a Hugging Face Docker Space + Neon Postgres), which rules out waste: small prompts, bounded loops, lean images.
System architecture
A split deploy in three tiers: a Next.js UI on Vercel, a FastAPI + LangGraph agent on a Docker Space, and the target database queried through a read-only user. Retrieval and conversation memory sit in Neon Postgres with pgvector.
System map
Tap to inspect
- 01
The browser client; sends a plain-English question over REST /query.
- 02
The agent loop — retrieves the right schema slice, generates SQL, validates, executes, and summarizes.
- Instead of
- Flask/Django request loops, or raw LLM calls with no structure
- Why
- LangGraph is a state machine for a multi-step job — retrieve schema → generate → validate → execute — with checkpoints persisted to Postgres for conversation memory. FastAPI gives async streaming, typed I/O, and clean middleware for the rate-limit and auth gates.
- Tradeoff
- Heavier runtime than raw httpx + Pydantic — paid back in typed tool contracts and state that survives a restart.
First provider with credentials wins; Gemini's free tier is the final fallback so it runs with zero paid keys.
- Instead of
- A single hardcoded provider
- Why
- OpenAI → OpenRouter → DeepSeek → Groq → Anthropic → Gemini, ordered by cost. Gemini's free tier is always appended as the last resort, so a quota wall on any one provider is invisible to the user. Every provider failure is logged, not just rate-limits.
- Tradeoff
- Six providers' credentials to manage, and slightly different response shapes to normalize.
Deterministic gate: SELECT-only, single statement, no DDL/DML, only enrolled tables — rejects when it can't prove safety.
- Instead of
- Trusting the model to write safe SQL; relying on the DB role alone
- Why
- A deterministic validator runs before every execution — whether the SQL came from the LLM or a user edit. sqlparse + regex reject DML/DDL/EXEC, multiple statements, SELECT INTO, and system-schema access, and check every table against the enrolled schema index. If the index is missing, it refuses.
- Tradeoff
- A heuristic parser isn't a formal proof — so it's one layer of defense-in-depth on top of a read-only DB user, timeouts, and row caps.
- 03
Semantic search over table summaries to find the candidate tables for the question — records which tables it touched.
- Instead of
- Stuffing the whole schema into every prompt
- Why
- Retrieval runs over documented table summaries, not raw DDL, so it stays accurate on large databases and keeps the prompt small. Top-4 tables per question; the agent is capped at 8 calls per tool per run so it can't loop.
- Tradeoff
- A one-time enrollment step per database to document + embed the schema.
Human-approved question→SQL pairs; the agent prefers adapting a close verified example over writing SQL from scratch.
- Instead of
- Generating every query from scratch, forever
- Why
- Approved (question → SQL) pairs are retrieved at generation time so the agent adapts a proven query instead of hallucinating one. Every approval makes the next answer for that database better.
- Tradeoff
- Needs a human to approve pairs — a deliberate quality flywheel, not free.
- 04
Table-documentation embeddings live in the same Postgres as everything else — retrieval searches the docs, not raw schema.
- Instead of
- A separate vector DB (Pinecone, Weaviate, Milvus)
- Why
- One managed Postgres holds embeddings, LangGraph checkpoints, metadata, and sessions — no second service to run, no ETL between stores. Embeddings are searched by cosine similarity with JSONB metadata filters on the same connection.
- Tradeoff
- pgvector's ivfflat index caps at 2000 dims, so the 3072-d Gemini embedding falls back to a sequential scan — fine at this scale, and worth it to avoid operating a second database.
LangGraph conversation memory, persisted per turn, so follow-ups have context.
- Instead of
- A stateless agent, or in-memory sessions lost on restart
- Why
- Conversation state (last few turns, tables touched, insights) is persisted per user + session, so a follow-up like "now break that down by month" has context — and survives a redeploy.
- Tradeoff
- A DB round-trip per turn to read + write the summary.
Postgres, MySQL, or SQL Server — queried through a read-only user, with a timeout and a row cap.
- Instead of
- One hardcoded dialect
- Why
- The same agent targets Postgres, MySQL, or SQL Server; a per-dialect directive in the system prompt keeps generated SQL dialect-correct. Execution runs through a read-only user with a 30s timeout and a 1000-row cap.
- Tradeoff
- Per-dialect prompt tuning — the alternative (dialect-blind SQL) shipped a real bug: T-SQL `TOP` emitted against Postgres.
Hover a node to preview · click one to see the decision behind it
Use the arrow keys, Home, or End to move between nodes. Press Enter or Space to pin a node's decision, and Escape to release it.
- Next.js UI: The browser client; sends a plain-English question over REST /query.
- FastAPI + LangGraph: The agent loop — retrieves the right schema slice, generates SQL, validates, executes, and summarizes. Why: LangGraph is a state machine for a multi-step job — retrieve schema → generate → validate → execute — with checkpoints persisted to Postgres for conversation memory. FastAPI gives async streaming, typed I/O, and clean middleware for the rate-limit and auth gates. Instead of: Flask/Django request loops, or raw LLM calls with no structure. Tradeoff: Heavier runtime than raw httpx + Pydantic — paid back in typed tool contracts and state that survives a restart.
- search_tables: Semantic search over table summaries to find the candidate tables for the question — records which tables it touched. Why: Retrieval runs over documented table summaries, not raw DDL, so it stays accurate on large databases and keeps the prompt small. Top-4 tables per question; the agent is capped at 8 calls per tool per run so it can't loop. Instead of: Stuffing the whole schema into every prompt. Tradeoff: A one-time enrollment step per database to document + embed the schema.
- verified queries: Human-approved question→SQL pairs; the agent prefers adapting a close verified example over writing SQL from scratch. Why: Approved (question → SQL) pairs are retrieved at generation time so the agent adapts a proven query instead of hallucinating one. Every approval makes the next answer for that database better. Instead of: Generating every query from scratch, forever. Tradeoff: Needs a human to approve pairs — a deliberate quality flywheel, not free.
- pgvector: Table-documentation embeddings live in the same Postgres as everything else — retrieval searches the docs, not raw schema. Why: One managed Postgres holds embeddings, LangGraph checkpoints, metadata, and sessions — no second service to run, no ETL between stores. Embeddings are searched by cosine similarity with JSONB metadata filters on the same connection. Instead of: A separate vector DB (Pinecone, Weaviate, Milvus). Tradeoff: pgvector's ivfflat index caps at 2000 dims, so the 3072-d Gemini embedding falls back to a sequential scan — fine at this scale, and worth it to avoid operating a second database.
- multi-LLM: First provider with credentials wins; Gemini's free tier is the final fallback so it runs with zero paid keys. Why: OpenAI → OpenRouter → DeepSeek → Groq → Anthropic → Gemini, ordered by cost. Gemini's free tier is always appended as the last resort, so a quota wall on any one provider is invisible to the user. Every provider failure is logged, not just rate-limits. Instead of: A single hardcoded provider. Tradeoff: Six providers' credentials to manage, and slightly different response shapes to normalize.
- validate_sql: Deterministic gate: SELECT-only, single statement, no DDL/DML, only enrolled tables — rejects when it can't prove safety. Why: A deterministic validator runs before every execution — whether the SQL came from the LLM or a user edit. sqlparse + regex reject DML/DDL/EXEC, multiple statements, SELECT INTO, and system-schema access, and check every table against the enrolled schema index. If the index is missing, it refuses. Instead of: Trusting the model to write safe SQL; relying on the DB role alone. Tradeoff: A heuristic parser isn't a formal proof — so it's one layer of defense-in-depth on top of a read-only DB user, timeouts, and row caps.
- Neon Postgres: LangGraph conversation memory, persisted per turn, so follow-ups have context. Why: Conversation state (last few turns, tables touched, insights) is persisted per user + session, so a follow-up like "now break that down by month" has context — and survives a redeploy. Instead of: A stateless agent, or in-memory sessions lost on restart. Tradeoff: A DB round-trip per turn to read + write the summary.
- Target DB: Postgres, MySQL, or SQL Server — queried through a read-only user, with a timeout and a row cap. Why: The same agent targets Postgres, MySQL, or SQL Server; a per-dialect directive in the system prompt keeps generated SQL dialect-correct. Execution runs through a read-only user with a 30s timeout and a 1000-row cap. Instead of: One hardcoded dialect. Tradeoff: Per-dialect prompt tuning — the alternative (dialect-blind SQL) shipped a real bug: T-SQL `TOP` emitted against Postgres.
Enrollment (once per database). A target database is introspected, each table is documented into structured sections (summary, columns, relationships, stats), and those chunks are embedded into pgvector. Retrieval later searches the documentation embeddings, not the raw schema — which is what makes it work on large databases at all.
Data flow
One question, end to end: the UI POSTs to /query; the agent runs search_tables to find candidate tables by semantic similarity, pulls only the needed section of each (columns / relationships / stats) so the prompt stays small, checks verified queries for a close human-approved example to adapt, then generates SQL. Generation passes through validate_sql before anything touches the database; the query executes under a timeout and row cap; the result is summarized back into natural language. A per-turn LangGraph checkpoint gives follow-ups context.
Key decisions
The decisions that make it trustworthy — each with the alternative it beat and the cost it accepted.
A deterministic, fail-closed SQL validator — not LLM trust
Instead of: trust the model's SQL, an LLM-as-judge safety check
It uses sqlparse (not fragile regex) to enforce SELECT/WITH-only, single-statement, no DDL/DML/EXEC, no information_schema/sys/pg_catalog — and, given a database, rejects any query touching a table outside the enrolled schema. If no schema index exists, it fails closed rather than open.
Trade-off accepted — Rejects some exotic-but-valid queries and adds an enrollment step — a false 'safe' is far costlier than a false 'unsafe'.
Defense in depth, not a single gate
Instead of: rely on the validator alone
Below the validator: read-only DB users (a probe warns if a connection looks writable), a query timeout + row cap, and log sanitization that masks passwords, API keys, and SQL literals. Any one layer failing doesn't expose the database.
Trade-off accepted — More moving parts to maintain and test.
Multi-provider LLM fallback over single-vendor lock-in
Instead of: one provider (e.g. OpenAI only)
A priority chain (OpenAI → OpenRouter → DeepSeek → Groq → Anthropic → Gemini) picks the first provider with credentials and always keeps Gemini's free tier as a final fallback, so no single outage or quota exhaustion takes the product down — and it stays free to run.
Trade-off accepted — Six provider adapters to keep working behind one interface.
Retrieval over context-stuffing
Instead of: dump the whole schema into the prompt
Schemas are documented, embedded, and retrieved per question, keeping the prompt small and the hallucination surface (and token cost) low.
Trade-off accepted — An enrollment/embedding step per database.
What failed
Early on, the model would emit SELECT TOP n against Postgres — valid T-SQL, invalid everywhere else — so cross-dialect queries failed. The fix was a deterministic dialect directiveinjected before generation, stating the target dialect's rule explicitly (LIMIT for Postgres/MySQL, TOP for SQL Server) rather than hoping the model inferred it.
Cause — the model defaulted to SELECT TOP n regardless of the target database
Fix — inject a per-dialect directive before generation, and cap tool loops so a confused agent can't spin
Performance & cost
Safety is a property of the architecture, not a single number — but the pipeline is still measured end to end. The structural guarantees first, then the results.
| Metric | Value | Method |
|---|---|---|
| write access to your data | none | Fail-closed validator + read-only DB user; a query is rejected if it references a table outside the enrolled schema. |
| LLM providers | 6 | One fallback interface; Gemini free tier is the final fallback so it runs with zero paid keys. |
| SQL dialects | 3 | Postgres, MySQL, SQL Server via SQLAlchemy 2.0 + ODBC. |
| per-run guardrails | k=4 · ≤8 calls · 30s | Retrieval returns the top 4 tables; each tool is capped at 8 calls per run; execution runs under a 30s timeout, a 1000-row cap, and a 5000-char SQL limit. |
| infra cost | $0 | Vercel + Hugging Face Docker Space + Neon Postgres, all free-tier. |
Results (measured)
The pipeline is scored end to end — two real runs, each linked to its method:
- 73% (101/139, dev)Spider · Execution accuracy
- Scoped Spider dev-split run of DBWhisper’s generation model (qwen/qwen3-32b at temp 0.1) with each database’s schema in context: 139 questions across 18 databases. Generated SQL is executed against the real Spider SQLite databases and compared by result set — ordered when the gold query has ORDER BY, multiset otherwise (standard execution match). 101/139 correct; malformed generations count as incorrect. 9 of the 148 sampled questions could not be scored after repeated provider throttling and are excluded, not counted either way. Measures the NL→SQL generation core; the deployed agent adds schema retrieval and a read-only validator on top.
- 82% exact · 100% fail-closedCustom golden-query set · Execution accuracy · fail-closed refusals
- 22 natural-language golden queries + 4 unsafe/out-of-scope prompts over a read-only Postgres store, run end-to-end through DBWhisper’s live pipeline (schema retrieval → generation → read-only validator → execute). 82% exact result-set match (18/22); 95% (21/22) when crediting correct answers that returned an extra column. All 4 destructive or out-of-scope prompts were refused fail-closed (4/4).
Operations
It runs as a split deploy with the agent in a Docker Space. Per-run guardrails keep it bounded: each tool result is cached per run, and a tool called more than eight times returns an abort hint that tells the model to stop looping and produce output. Logs are sanitized before write, and the validator's rules are covered by unit tests that gate CI (pytest + ruff, alongside a gitleaks secret scan).
def validate_sql(sql: str, db_flag: str | None) -> None:
stmt = _single_statement(sql) # exactly one statement
_require_read_only(stmt) # starts with SELECT/WITH; no DDL/DML/EXEC
_reject_system_tables(stmt) # no information_schema / sys. / pg_catalog.
if db_flag:
schema = _load_schema_index(db_flag) # fail closed if missing
_require_enrolled_tables(stmt, schema) # only tables we can vouch forWhat I'd do differently
Two things. I'd add a small golden-query eval per enrolled database so retrieval and generation quality are measured, not assumed — right now the verified-query flywheel is the feedback loop, but it isn't scored. And I'd move the six provider adapters behind a single typed capability interface earlier; retrofitting consistent retry/timeout behavior across them was more work than building it in from the start.
Evidence
The app is live and the source is public — the validator, the agent tools, and the multi-provider chain are all in the repository linked at the top of this page.
Changelog
Schema browser, a verified NL→SQL training flywheel, and the full marketing site.
Flagship web console — auto-generated charts, CSV export, sortable results, staged progress.
Production hardening: Argon2id auth, per-query tenancy, API-key + per-IP rate limits, a SQL allowlist, and optional Sentry.
Productionized from a prototype — Next.js frontend, dialect-agnostic schema extraction, and a live demo on Neon.