Why Flip AI built a custom large language model to run its observability platform

Observability is a bit of a buzz word in IT circles these days, but it basically involves monitoring a company’s systems, looking for issues or trying to find the root cause of problems after they happen — and they do happen all the time, sometimes slowing down a site or application, and in the worst case taking it offline.

There are many startups and established companies trying to solve this problem, but Flip AI is bringing a new twist to the category. The early-stage startup has built its own large language model, specifically designed to attack the monitoring problem.

Today, the company announced their product was generally available, as well as a previously unannounced $6.5 million seed investment.

CEO and co-founder Corey Harrison says that today, in spite of the number of tools out there, companies still often are using highly manual processes to track data between systems. He and his co-founders, CTO Sunil Mallya and CPO Deap Ubhi, saw an opportunity to put intelligence and automation to work to speed up time to resolution.

“So large enterprises are using [multiple] tools, yet still have difficulty when it comes time to actually troubleshoot incidents,” Harrison told TechCrunch. He said that this problem is often more acute in larger organizations where they have more tools and data often resides in different systems, making it especially challenging to track down the cause of the problem without a lot of manual querying.

By building a large language model trained on DevOps data, they believe they can speed up the troubleshooting process and the time to recovery. “We have our own large language model — we’re not using OpenAI or anything like that — which we trained on over 100 billion tokens of DevOps-specific data like logs, metrics, trace data, configuration files, etc. It can then rationalize the same way humans were meant to query between systems,” Harrison said.

The result is a tool that analyzes the data across systems and generates a root cause analysis in less than a minute, and typically just a few seconds, according to Harrison, and he says they leave the data in place, requiring just read access to complete the analysis.

Harrison acknowledges that no model can be right all of the time, but he says they provide the path of how the model got the answer, so a human developer can check the work. “So even if in the end the root cause analysis is not 100% correct, we’ve already localized the error, we’ve run the queries and pulled sample data. So we’ve still done 90% of the work for you,” he said.

It’s a big and bold idea to train your own LLM, but Mallya and Ubhi both previously worked at Amazon where Mallya was in charge of Amazon Comprehend, the company’s NLP service, and Ubhi was director of product management. Harrison also has a deep technical background, including most recently working as SVP of operations and chief of staff to the NFL Commissioner.

The company currently has 20 employees split between San Francisco and Bangalore, India. As it grows, it’s trying to balance customer demand, which he says is quite good, with moving in a methodical way. Harrison, who is Black, certainly recognizes the lack of diversity in the tech jobs market, something he says he thinks about a lot. “Just given my background, and who helped me get here, and a diverse set of people helped me get here, I want to make sure that Flip AI has the same, if not greater diversity level,” he said.

The $6.5 million seed investment was led by Factory with Morgan Stanley Next Level Fund and GTM Capital participating.