Spotify introduces new product to help software development teams with A/B testing

Spotify is introducing a new commercial product for software development teams called “Confidence,” which is based on its homegrown experimentation platform. Confidence is designed to make it easy for teams to set up, run, coordinate and analyze their own user tests so they can quickly optimize their ideas. Confidence is currently only available as a private beta.

“Spotify’s data scientists and engineers have been developing and honing our product testing methods for years,” the company wrote in a blog post. “Whether it’s automatically coordinating simultaneous A/B tests or orchestrating the rollout of an AI recommendation system across mobile, desktop, and web, the platform we built scales experimentation best practices and capabilities to all our teams. Soon this experimentation platform will be available to any company that wants to build, test, and iterate ideas the way we do at Spotify: quickly, reliably, and with confidence.”

Spotify says Confidence is designed to help software development teams whether they have outgrown their current testing platform or are looking for a quick and easy way to get started with A/B testing.

The music streaming service says its experimentation efforts began in the early 2010s when a few data scientists and engineers started conducting small A/B tests internally. Although these tests were manual and error-prone, Spotify understood the importance of experimenting and wanted to get better at it. The company then decided to build its own basic A/B testing platform, which it called ABBA.

The platform did feature flagging and analysis for a set of standardized metrics. Spotify says this unlocked a wave of experimentation across the company, and that it grew from running fewer than 20 priority experiments per year to running hundreds of experiments per year across multiple squads.

Confidence will be available to customers in three ways. First, it can be run as a managed service, which means you can access the experimentation platform as a standalone web service managed by Spotify. Second, you can run it as a Backstage plugin. Last, you can integrate the Confidence platform into your own infrastructure via APIs.

Software development teams can now sign up for the waitlist to be eligible for an invite to Confidence. Spotify did not say when it plans to make Confidence more widely available.

It’s worth noting that Confidence is Spotify’s second commercial product for monetizing developers. Last December, Spotify unveiled its plan to monetize developers via its open source Backstage project. Spotify Backstage is platform designed to bring order to companies’ infrastructure by enabling them to build customized “developer portals,” combining all their tooling, apps, data, services, APIs and documents in a single interface.

Today’s announcement indicates that although Spotify is mainly known for its consumer-facing music streaming service, it’s looking to branch out beyond that notion.