This Startup’s New Mechanical Explainability Tool Lets You Debug LLM

by ai-intensify
0 comments
This Startup's New Mechanical Explainability Tool Lets You Debug LLM

The company says its mission is to make building AI models less like alchemy and more like science. Sure, LLMs like ChatGPT and Gemini can do amazing things. But no one knows how or why they work, and that can make it difficult to fix their flaws or stop unwanted behavior.

“We saw a growing gap between how well the models were understood and how widely they were being deployed,” explains Eric Ho, Goodfire’s CEO. MIT Technology Review In an exclusive conversation ahead of the release of In Silico. “I think the dominant sentiment in every major frontier laboratory today is that you just need more scale, more compute, more data, and then you get AGI (artificial general intelligence) and nothing else matters. And we’re saying no, there’s a better way.”

Goodfire is one of a handful of companies, including industry leaders Anthropic, OpenAI, and Google DeepMind, that are pioneering a technology called mechanistic interpretability, which aims to understand what happens inside an AI model when it performs a task by mapping its neurons and the pathways between them. (MIT Technology Review chose machine learning as one of its 10 defining technologies of 2026.)

Goodfire wants to use this approach not only to audit models – that is, studying people who have already been trained – but to help design them in the first place.

“We want to remove trial and error and turn training models into precision engineering,” Ho says. “And that means exposing the knobs and dials so you can actually use them during the training process.”

Goodfire has already used its technologies and tools to change LLM behavior—for example, reducing the number of hallucinations they cause. With In Silico, the company is now packaging many of those in-house technologies and shipping them as one product.

This tool uses agents to automate most complex tasks. “Agents have become so robust now that they can do a lot of the interpretive work that we were doing using humans,” Ho says. “It was kind of a gap that needed to be bridged before it could actually be a viable platform that customers could use themselves.”

Leonard Bereska, a researcher at the University of Amsterdam who has worked on the mechanistic explanation, thinks that in silico looks like a useful tool. But he pushes back against Goodfire’s lofty aspirations. “In fact, they are adding precision to alchemy,” he says. “Calling it engineering makes it sound more theoretical than it is.”

Related Articles

Leave a Comment