Giving an LLM raw SQL access to your database is a recipe for

大鹏的日常 Novice 1h ago 572 views 5 likes 2 min read

I've been building a personal health tracker to query my own data—mostly so I can check trends from my phone at 6am instead of being tethered to a desk with Claude Code. The setup uses a TimescaleDB backend, and while it seems straightforward, I realized that letting a model write its own queries is a disaster waiting to happen.

Most "chat with your data" tutorials suggest the obvious route: give the model the schema, a read-only connection, and let it rip. I tried that, and it failed miserably because my dataset has six specific "traps" that are invisible to a schema but critical for accuracy.

Why SQL-generation fails in real-world data

The problem isn't the LLM's ability to write syntax; it's the lack of domain context. A model sees a column named calories and assumes it's a fact. In reality, the data is messy. Here are the traps that broke my initial implementation:

  • Partial Day Bias: Today's numbers are still climbing. If the model compares today's current total to yesterday's final total, it "invents" a fast that didn't happen.
  • The Gap vs. Zero Fallacy: An unlogged day is missing data, not a day of zero intake. SQL treats NULLs or missing rows in a way that leads the LLM to assume I stopped eating.
  • Duplicate Entries: My workout app shadow-copies sessions. A naive SUM of training volume literally doubles my actual work.
  • Formulaic Overestimation: My energy_balance column is a calculated estimate. It overstates my deficit by nearly 3x because basal energy formulas and watch-measured burn are notoriously generous.
  • Noise vs. Signal: A single weigh-in is just water weight. A model querying a single row sees a "gain" or "loss" that is biologically meaningless.
  • Semantic Misinterpretation: In my logs, reps: 0 means I attempted a set and failed. An LLM treats it as a missing value or a zero-effort set.

The better AI workflow: Tool-based abstraction

Instead of giving the model a blank SQL canvas, I shifted to a tool-based architecture. I built thirteen read-only tools that act as an API layer between Claude and the database.

If the model wants to know about my VO2max or squat progress, it doesn't write a SELECT statement; it calls a specific function. This allows me to bake the "trap avoidance" directly into the code. The logic for handling partial days or filtering duplicate workouts happens in the backend, not in the prompt.

For the stuff that needs to be 100% deterministic (like protein adherence or weight trends), I didn't use an LLM at all. I wrote pure functions in lib/signals/ and unit-tested them against fixtures.

If you're designing an LLM agent for data analysis, stop focusing on the prompt and start focusing on the abstraction layer. The schema is never enough context for the model to understand the nuance of how the data was actually collected.

All Replies (3)

N
NeuralSmith Novice 1h ago
Most DB demos ignore the messiness of real-world data. SQL read-only access doesn't stop a model from hallucinating based on weird sentinel values or broken schemas. Shifting that logic into named domain operations is a smart move—it's way more reliable than praying the prompt handles every single edge case.
0 Reply
Z
ZenMaster Expert 1h ago
SQL lets the model spit out a number, but it doesn't give it the context to know if that number actually means anything. I really appreciate the distinction between missing data and zero here. In finance, that same slip-up can easily be mistaken for a fake recovery or a false drawdown.
0 Reply
G
GhostGeek Expert 1h ago
Read-only users and row-level security usually save me from the worst hallucinations.
0 Reply

Write a Reply

Markdown supported