System of intelligence — generative AI at the app layer

Generative AI is a paradigm shift in technology, and it will spur a massive shift in enterprise spend over the next decade and beyond. Transformations of this magnitude can feel rapid on the surface, especially when they make a huge splash like generative AI has in recent months, but it’s a steep and steady climb to permeate the layers of the enterprise technology stack.

The infrastructure layer captures the initial spend as companies assemble the building blocks for power and performance — the capital pouring into Nvidia and GPU aggregators today indicate this is well underway. As adoption (and dollars) move up the stack, development focus will shift toward the new experiences and products that will reshape each subsequent layer.

We’re just getting a glimpse of how this transformation will unfold at the application layer, and early signals suggest the disruption will be profound.

Long before generative AI, enterprise applications began to deliver more consumer-like experiences by improving UIs and introducing interactive elements that would engage everyday users and accelerate workflow. This spurred a shift from “system of record” applications like Salesforce and Workday to “system of engagement” applications like Slack and Notion.

Collaboration was a defining characteristic of this new breed of enterprise tools, with features like multiplayer mode, annotation functionality, version history, and metadata. These apps also leveraged consumer-native viral components to drive adoption and enable seamless sharing of content within and between organizations. The core record retained its intrinsic value within these systems of engagement, and served as a bedrock for the growing volume of information created at the engagement layer.

As generative AI shapes the next generation of application products, we can expect even more sweeping evolution. The first players look a lot like ChatGPT integrators, building lightweight tools directly on top of generative models that deliver immediate but fleeting value. We have already seen a variety of generative AI products emerge that have explosive initial growth, but also extremely high churn due to limited workflow or lack of additional functionality. These applications typically produce a generative output that is a single-use type of content or media (i.e., not embedded into a user’s everyday workflow), and their value relies on off-the-shelf generative models that are widely available to others in the market.

The second wave of generative AI applications, which is just beginning to take shape, will leverage generative models to integrate the structured data that lies within the system-of-record applications and the unstructured data that lies within the system-of-engagement applications.

Developers of these products will have more potential to create enduring companies than first-wave entrants, but only if they can find a way to “own” the layer above the system-of-engagement and system-of-record applications — no mean feat when incumbents like Salesforce are already scrambling to implement generative AI to create a protective moat around their underlying layers.

This leads to the third wave, where entrants create their own, defensible “system of intelligence” layer. Startups will first introduce novel product offerings that deliver value by harnessing existing system-of-record and system-of-engagement capabilities. Once a strong use case is established, they will then build out workflows that can ultimately stand alone as a true enterprise application.

This does not necessarily mean replacing the existing interactive or database layers; instead, they will create new structured and unstructured data where generative models utilize these new datasets to enhance the product experience — essentially creating a new class of “super datasets.”

A core focus for these products should be integrations with the ability to ingest, clean, and label the data. For example, to build a new customer support experience, it’s not enough to simply ingest the knowledge base of existing customer support tickets. A truly compelling product should also incorporate bug tracking, product documentation, internal team communications, and much more. It will know how to pull out the relevant information, tag it, and weigh it in order to create novel insights. It will have a feedback loop that allows it to get better with training and usage, not only within an organization but also across multiple organizations.

When a product accomplishes all of this, switching to a competitor becomes very difficult — the weighted, cleaned data is highly valuable and it would take too long to achieve the same quality with a new product.

At this point, the intelligence lies not only in the product or model, but also in the associated hierarchy, labels, and weights. Insights will take minutes instead of days to deliver, with a focus on actions and decisions rather than just synthesis of information. These will be the true system-of-intelligence products that leverage generative AI, marked by these defining traits:

• Have deep integration with company workflows and ability to capture newly created structured and unstructured data.
• Be sophisticated around the characterization and digestion of data through hierarchy, labels, and weights.
• Create data feedback loops within and between customers to enhance the product experience.

One key question I love to ask customers is, “Where does a new product stack rank with the other tools you use?” Normally the system-of-record product is the most important, followed by the system-of-engagement product, with additional tooling at the bottom of the list.

The least important product will be the first to get cut when the budget is tight, so emerging system-of-intelligence products must provide enduring value in order to survive. They’ll also face steep competition from incumbents who will build generative AI–enabled intelligence capabilities into their products.  It will be up to the new wave of system-of-intelligence to couple their offerings with high-value workflows, collaboration, and the introduction of super datasets to endure.

Transformation of the AI space has accelerated over the last 12 months, and the industry is learning fast. Open source models are proliferating and closed proprietary models are also evolving at an atypically rapid pace. Now it’s up to founders to build enduring system-of-intelligence products atop this rapidly shifting landscape — and when it’s done right, the impact on enterprises will be extraordinary.