Fireworks.ai open source API puts generative AI in reach of any developer

Just about everyone is trying to get a piece of the generative AI action these days. While the majority of the focus remains on the model vendors like OpenAI, Anthropic and Cohere, or the bigger companies like Microsoft, Meta, Google and Amazon, there are in fact, a lot of startups trying to attack the generative AI problem in a variety of ways.

Fireworks.ai is one such startup. While lacking the brand name recognition of some of these other players, it boasts the largest open source model API with over 12,000 users, per the company. That kind of open source traction tends to attract investor attention, and the company has raised $25 million so far.

Fireworks co-founder and CEO Lin Qiao points out that her company isn’t training foundation models from scratch, but rather helping fine tune other models to the particular needs of a business. “It can be either off the shelf, open source models or the models we tune or the models our customer can tune by themselves. All three varieties can be served through our inference engine API,” Qiao told TechCrunch.

Being an API, developers can plug it into their application, bring their model of choice trained on their data, and add generative AI capabilities like asking questions very quickly. Qiao says it’s fast, efficient and produces high-quality results.

Another advantage of Firework’s approach is that it allows companies to experiment with multiple models, something that’s important in a fast-changing market. “Our philosophy here is we want to empower users to iterate and experiment with multiple models and have effective tools to infuse their data into multiple models and test with a product,” she said.

Perhaps even more importantly, they keep costs down by limiting the model size to between 7 billion and 13 billion tokens, compared with over 1 trillion tokens in ChatGPT4. While that limits the universe of words the large language model can understand, it enables developers to focus on much smaller, focused data sets designed to work with more limited business use cases.

Qiao is uniquely qualified to build such a system having previously worked at Meta, leading the AI platform development team with a goal of building a fast, scalable development engine to power AI across all of Meta’s products and services. She was able to take this knowledge from working at Meta and create an API-based tool that puts that kind of power in reach of any company without requiring the level of engineering resources of a company the size of Meta.

The company raised $25 million in 2022 led by Benchmark, with participation from Sequoia Capital and angel investors including Databricks and Snowflake. The two latter are particularly interesting strategic investors, given they both are data storage tools, and Fireworks will enable users to put that data to work.