Large enterprises often sit on enormous volumes of data — in Rocket Mortgage’s case, tens of petabytes. That data holds valuable insight, but the long-standing challenge has been making it accessible to the executives, analysts, and decision-makers who understand the business yet do not necessarily write SQL. Rocket Analytics is the company’s response: a text-to-SQL application, built on agentic retrieval-augmented generation (RAG), that lets people ask questions in plain language while the system handles the technical work.
What Rocket Analytics does
In practice, a user asks a natural-language question — for example, requesting a count of loans over the last six months. Behind the scenes the system converts the question into a SQL query, runs it against the relevant database, and returns the results in a clean, readable format. The greater value emerges with follow-up questions: users can build on earlier results or assemble dashboards that surface trends. In one demonstration, a user moved from raw loan figures to a dashboard showing total loans closed, average daily loans, peak dates, and trend analysis within seconds — a meaningful capability in an industry where decision speed matters.
The architecture behind the system
Rocket Analytics combines several components into a retrieval-augmented pipeline that turns a question into a verified answer. Each stage narrows and sharpens the context the language model works with, which improves both accuracy and cost. A related enterprise example is described in this case study on building an AI-ready data ecosystem.
Query transformation
The process begins when a user enters a question. Language models are powerful but do not inherently understand a company’s specific business context or data structures, so the raw question is first transformed into a clearer, context-aware version that maps better onto the underlying schema.
Building and managing the knowledge base
Before any question can be answered, the system needs a knowledge base. Database metadata is converted into embeddings using Amazon Titan and stored in a FAISS (Facebook AI Similarity Search) vector store, along with descriptive information about each table and column.
Intelligent retrieval
For a given question, semantic search selects the most relevant tables, while keyword triggers ensure that important tables are included even when semantic search alone might miss them. A knowledge base of, say, 15 tables can be narrowed to the four or five most relevant, which keeps the model from being overloaded and reduces errors.
Re-ranking for accuracy
Once candidate tables are retrieved, a Cohere rerank model refines the selection by scoring how likely each is to be relevant to the question. This step is important for limiting hallucinations, since it ensures the model generates SQL based only on genuinely relevant metadata.
Prompt engineering
The prompt is layered: standard guidelines (answer within the provided context; use exact table and column names), adaptive guidelines (a few worked examples for trickier question types), and domain-specific references that help the model interpret mortgage-industry terminology. A typical prompt presents the relevant tables and their metadata and asks the model to answer the question while following those guidelines, which improves accuracy on complex queries.
Execution and post-processing
After the model produces a SQL query, the system executes it against the appropriate database and post-processes the results into a user-friendly form. Users can view the output directly or ask follow-up questions to refine or extend it.
An agentic framework for cross-domain questions
A single agent can handle queries within one domain, but organizations frequently need cross-domain insight. The agentic framework adds a lead orchestrator agent coordinating sub-agents that specialize in domains such as sales, marketing, and operations, each with its own knowledge base and tailored prompts. The orchestrator routes a question to the right specialist and combines results when an answer spans several domains.
Performance optimization: speed and cost
Two caching strategies make a notable difference at scale.
Semantic caching
Reusing retrieved tables and prompts for semantically similar queries avoids redundant processing. Distinguishing genuinely similar requests from subtly different ones — “last six months” versus “last ten days” — requires careful tuning.
Prompt caching
Caching the static components of a prompt reduces repeated computation and improves both latency and cost efficiency. Together, these optimizations help the system scale across teams without slowing queries down.
Measuring success
The system is evaluated along several metrics: relevance, judged by whether retrieved tables are actually pertinent (assessed with an LLM-as-a-judge approach); hallucination detection, comparing output against a curated “golden” dataset; and screening to ensure responses remain professional. Running multiple iterations per query gives a more reliable performance picture and helps catch edge cases.
Real-world impact
Rocket Analytics supports teams across the organization. Sales and marketing use it to analyze campaign effectiveness, identify seasonal trends, and compare strategies across segments; operations teams monitor performance metrics and refine processes; finance teams generate reports and analyze trends on demand. Work that previously took analysts days can now be available in seconds.
The human element: trust but verify
Speed does not remove the need for oversight. The system uses human-in-the-loop checkpoints for high-risk queries and regular audits to catch irregularities or emerging errors, with the aim of preserving accuracy and trust alongside speed.
Looking ahead
Planned directions include deeper multi-agent coordination for cross-domain insight, continuous performance improvement, and expanded domain coverage as more of the business is brought into the system.
Key takeaways
The project reinforced a few lessons: AI works best as a way to augment rather than replace human judgment; speed, accuracy, and accessibility matter more than raw sophistication; and thoughtful, user-centered design is decisive. By removing technical barriers, a well-built text-to-SQL system lets more people explore and act on data — the goal being the most useful system, not the most complex one.
