PlanetScale forks MySQL to add vector support
It seems like everyone in the database business is thinking about how to capitalize on the growth of AI by adding support for vectors to their products. That, after all, remains to be the easiest way to feed up-to-date data to foundation models. Almost 28 years after its launch, MySQL may not have much hype surrounding it anymore, but it remains among the most often-used databases and it powers the vast majority of the web. Today, PlanetScale, the well-funded MySQL database company founded by the co-creators of the Vitess open source project, announced that it is forking MySQL in order to add vector support — in part because the team believes that MySQL-owner Oracle isn’t moving fast enough to do the same.
“MySQL is still the world’s most popular database; still runs 70% of the internet. Oracle has been slow to respond to this changing and evolving world of AI and generally, modern MySQL is being carried forward by PlanetScale,” PlanetScale CEO and President Sam Lambert told me when I asked him why the company decided to go this route. “What we’ve seen through industry trends is that AI is not just leading to a ballooning of vector data but relational data itself. People have proprietary data, they have the inputs that are used for these models, and then you create this kind of circular feedback loop: you’re seeing what people are searching for, you’re inputting that into your relational database, then you’re training on it, you’re storing vectors and you’re doing similarity search. We couldn’t really let ourselves get bound to Oracle’s lack of pace on this.”
Lambert noted that it only took his team about six weeks to complete the work on adding vector support. The team is now testing to ensure that its solution can scale before it will roll out to all PlanetScale users, but the company is also looking to see if it can contribute this code upstream into the MySQL open source edition. “We’ll make an attempt to upstream this to be good citizens,” Lambert, who came to PlanetScale in 2021 after being the VP of Engineering at GitHub, said. “We believe in open source. My background at GitHub gave me a long, long time of experience in the open source community, we love open source.”
Internally, PlanetScale has been using a MySQL fork for years, so the company is well-versed in maintaining this fork, but it now has to do so in public. And while the team was able to quickly add vector support to MySQL, Lambert argues that this was only possible because the team is highly versed in the internals of the database. “It’s no small feat, right? It’s a 25-year-old C++ code base, right? Postgres has two versions of this now because they’ve got an easy plugin ecosystem. This is deep work we’ve had to go and do to actually produce this — and it pays off.”
Image Credits: Planetscale
To a large degree, PlanetScale’s value proposition has always been that it can scale well beyond any of its competitors. Vitess, after all, came out of YouTube. The company now promises that it can bring this scalability to vector databases and search. Lambert argues that it makes sense to keep the vector features close to the database. In his view, storing vectors isn’t a very complicated way to store data, so it’s easy enough to add it to the existing database and keep the two systems close. This also now means that, ideally, WordPress and other tools that run on top of MySQL may soon get easy access to these features as well, which would allow the developers in those ecosystems to more easily add AI-based features that make use of custom data as well.
“We feel it’s a core part of our mission,” Lambert said. “If we were carrying forward modern MySQL, we have to take up the flag and own this and move forward. This is something we’re very excited to do to keep making MySQL better while this new modern audience falls in love with it.”
“It’s great MySQL is so reliable. No one questions it. No one has ever said that it’s not reliable, that it’s not scalable. That doesn’t give Oracle the excuse to stop shipping and responding to these trends and that’s why we’re here, that’s why we’re picking this up.”
