Finding the right customer story at the right time is surprisingly harder than expected. To improve employee productivity, we created Refi – an app that enables users to search and analyze over 2,400 Databricks customer references, providing personalized responses, cross-story analysis, quotes, and more. In its first two months, more than 1,800 people in Databricks sales and marketing have asked more than 7,500 questions on ReFi. This translates into more relevant and consistent storytelling, faster campaign execution, and the confidence that customer testimonials are used at scale. By making these stories searchable and digestible, we solved the problem of tribal knowledge related to customer contexts and highlighted the valuable work of the many people who have collected them over the years.
In this article we will discuss the inspiration for Refi, the entire Databricks solution, its impact on our organization, and how we plan to take it even further internally.
Challenge of democratization of tribal knowledge
“Who else has done this?” This is a question every salesperson hears. A prospect is attracted to your pitch, but before they move forward, they want proof – a customer like them who has already walked this path. This should be easy to answer.
For our marketing team, customer stories are a core input to almost every activity – campaigns, product launches, advertising, PR, analyst briefings and executive communications. When those stories aren’t easy to find or evaluate, real problems arise: high-value references get overused, new use cases or industries are missed, and marketing effectiveness is limited by tribal knowledge.
Databricks has thousands of YouTube talks, case studies, internal slides, LinkedIn articles, Medium posts on Databricks.com. Somewhere there is exactly the right context – a financial services company in Canada that is doing fraud detection in real-time, a retailer that has replaced an old data warehouse, a manufacturer that is expanding GenAI. But looking for it? This is where things fall apart. Stories exist on a dozen platforms with no integrated search, and when you find something, you can’t immediately tell if it’s strong – does it have reliable business results, or just vague claims?
So people do what people do: They message the marketing team on Slack, dig through folders they only half-remember, or keep asking around until someone turns up something useful. Sometimes they find gold. Often, they settle for “good enough” or give up altogether – never knowing if the right story was out there.
Clearly, we needed a better way for sales and marketing to discover the most relevant customer stories.
Refi: A Full-Stack Solution on Databricks
To solve this problem, we consolidate all the stories into a single table, categorize them, then use a RAG-based agent to power search – all exposed through a Vibe-coded Databricks app. The architecture spans the full Databricks platform: Lakeflow Jobs orchestrates our ETL pipelines, Unity Catalog controls our data, Vector Search powers retrieval, Model Serving hosts our agent, Lakebase handles reads and writes in real-time, and Databricks Apps delivers the frontend. Let’s know the details.
Data Sources and ETL
Our pipeline is defined in a series of Databricks notebooks lakeflow jobs. The pipeline starts by collecting the text of stories from all of our data sources: we use standard Python webscraping libraries to collect YouTube transcripts, LinkedIn/Medium articles, and all public customer stories on databricks.com. Using Google Apps Script, we consolidate text from hundreds of internal Google Slides and Docs into a single Google Sheet. All these sources are processed with basic metadata and saved in a ‘Bronze’ Delta Lake table in the Unity Catalog (UC).
Now all our stories are in one place, but we still have no idea about their quality. To address this, we grade text by applying a rigorous 31-point scoring system (developed by our Values team) to each story. AI functions. We drive Gemini 2.5 to assess the quality of the overall story by identifying the business challenge, the solution, the credibility of the outcome, and how Databricks is uniquely positioned to deliver value. Rating stories like this also allows us to filter out the lowest quality stories from Refi. Prompt also extracts key metadata like country and industry, products used, competition and citations and tags stories based on whether they are publicly shareable or internal-only. This rich dataset is saved in ‘Silver’ table in UC.
The final steps of ETL involve filtering out low-scoring stories and creating a new ‘Summary’ column that ties together the essential story components. The idea is simple: we sync this ‘gold’ table databricks vector search Index, with summary columns that contain all the essential information needed for LLM to match customer stories to queries.
agent ai
Using the DSPY In the framework, we define a tool-calling agent that can look for the most relevant customer context with hybrid keyword and semantic search. We love DSPy! Agents built with it are easy to iteratively test in Databricks notebooks without redeploying to the model serving endpoint each time, resulting in faster development cycles. The syntax is highly intuitive compared to other popular frameworks, and it includes excellent quick customization Component. If you haven’t yet, definitely check out DSPy.
We structure our Customer Stories agent to facilitate lightning-fast pure keyword searches and a long-form LLM response with logic based on user input: If you ask a question, you’ll get a carefully thought-out answer with sources, but if you enter just a few keywords, Refi will return the top results in less than two seconds. We also use Databricks Re-Ranker for vector search to improve the results from RAG.
To ensure balanced and professional feedback, we use the following system prompts:
The agent is logged into MLflow and deployed to Databricks Model Serving using our agent framework. Since most of the processing is done on the model provider side, we can get away with deployment on a small CPU instance, saving on infrastructure costs compared to GPUs.
databricks app
Now that we have the data cleaned and indexed and the agent is working well, it’s time to tie it all together and create an app to make it accessible to non-technical users. We chose a React frontend with a FastAPI Python backend. React is beautiful and fast in the browser and supports streaming output from our model serving endpoints. FastAPI lets us take advantage of all the benefits of the Databricks Python SDK in our app, namely:
- integrated authentication – No code changes when authenticating locally during development vs. deploying to Databricks apps. Apps have the same environment variables as local authentication, so the code works seamlessly.
- Detailed API coverage – We can call model serving, execute SQL queries, or do whatever we need from the Databricks workspace, all through a single SDK.
Refi is primarily a chat app, so we use Lakebase to retain all conversation history, logs, and user identities for fast reads and writes, quality assurance, and thoughtful follow-up as users return or start new conversations.
Ongoing monitoring and metrics
Logs from Lakebase are processed in a separate Lakeflow job to surface key metrics like daily active users and average response time in the AI/BI dashboard. This dashboard also shows us recent inputs and reactions, and we go a step further to apply another AI function to summarize inputs and reactions in recent topics and gap analysis. We want to understand which customer stories are popular and where there may be shortcomings, and the logs collected from Refi help us do that. For example, we found that users were especially eager to find stories on Agent Bricks and Lakebase, the two newest Databricks products.
At the bottom of the dashboard, we include a static analysis of story quality across different industries and content types.
A note on development setup
Where does most project development take place? cursorAnd as mentioned earlier, integrated authentication between the Databricks CLI and SDK keeps things simple. We Sign in once via CLIAnd all our local builds of Refi that use the SDK are certified. When we want to test in Databricks apps, we use the CLI to sync the latest code to our workspace and then deploy the app. Databricks Apps checks it out Environment Variables We have authentication set up locally, so our calls to Model Serving and SQL Warehouse that depend on the SDK just work! Our iterative development becomes:
- Sign in to Workspace via CLI
- author code in cursor
- test locally
- Sync code to workspace and deploy app
- Test in Databricks Apps
Finally, to ensure proper CI/CD and portability, we use Databricks Asset Bundle To bundle all the code and resources used by Refi into one package. This bundle is then deployed to our target production workspace via GitHub Actions.
what we learned
Several teams at Databricks had already solved parts of this problem independently, naturally gravitating toward the most exciting work – the AI layer. However, data engineering is still at the core and getting ETL right, scoring stories for quality and structuring data for effective retrieval proved to be just as important as the agent.
Cooperation was equally important. Customer stories touch almost every corner of the organization: sales, marketing, field engineering, and PR all play a role. Building strong partnerships with these groups shaped both the product and the data that powers it.
what will happen next
While the application frontend provides immediate value, the real power will emerge from connecting ReFi with other solutions in Databricks. We plan to provide that connectivity through an API and MCP server, enabling teams to access customer intelligence directly within their existing workflows and tools.
With Databricks and Lakebase, we can also understand how thousands of users interact with ReFi over time. These insights will allow us to continually refine the tool and thoughtfully shape the stories we add to this growing ecosystem.
For Databricks teams struggling with customer context discovery today, Refi offers a solid example of what is possible when these capabilities are brought together. To get started building your own agentic app on Databricks, learn more about Databricks AppsOur RAG GuideLakebase, and agent bricks.
