System of intelligence — generative AI at the app layer

Content originally published by Tech Crunch as a guest contribution on 12/9/23.

Generative AI is a paradigm shift in technology, and it will spur a massive shift in enterprise spend over the next decade and beyond. Transformations of this magnitude can feel rapid on the surface, especially when they make a huge splash like Generative AI has in recent months, but it’s a steep and steady climb to permeate the layers of the enterprise technology stack. The infrastructure layer captures the initial spend as companies assemble the building blocks for power and performance–the capital pouring into Nvidia and GPU aggregators today indicate this is well underway. As adoption (and dollars) move up the stack, development focus will shift toward the new experiences and products that will reshape each subsequent layer. We’re just getting a glimpse of how this transformation will unfold at the application layer, and early signals suggest the disruption will be profound.

Long before generative AI, enterprise applications began to deliver more consumer-like experiences by improving UIs and introducing interactive elements that would engage everyday users and accelerate workflow. This spurred a shift from ‘system of record’ applications like Salesforce and Workday to ‘system of engagement’ applications1 like Slack and Notion. Collaboration was a defining characteristic of this new breed of enterprise tools, with features like multiplayer mode, annotation functionality, version history and metadata. These apps also leveraged consumer-native viral components to drive adoption and enable seamless sharing of content within and between organizations. The core record retained its intrinsic value within these systems of engagement, but also served as a bedrock for the growing volume of information created at the engagement layer.

As generative AI shapes the next generation of application products, we can expect even more sweeping evolution. The first players look a lot like Chat-GPT integrators, building lightweight tools directly on top of generative models that deliver immediate but fleeting value. We have already seen a variety of generative AI products emerge that have explosive initial growth, but also extremely high churn due to limited workflow or lack of additional functionality. These applications typically produce a generative output that is a single-use type of content or media (i.e., not embedded into a user's everyday workflow), and their value relies on off the shelf generative models which are widely available to others in the market.

The second wave of generative AI applications, which is just beginning to take shape, will leverage generative models to integrate the structured data that lies within the ‘system of record’ applications and the unstructured data that lies within the ‘system of engagement’ applications. Developers of these products will have more potential to create enduring companies than first-wave entrants, but only if they can find a way to ‘own’ the layer above the ‘system of engagement’ and ‘system of record’ applications–no mean feat when incumbents like Salesforce are already scrambling to implement generative AI to create a protective moat around their underlying layers.

This leads to the third wave, where entrants create their own, defensible ‘system of intelligence’ layer. Startups will first introduce novel product offerings that deliver value by harnessing existing ‘system of record’ and ‘system of engagement’ capabilities. Once a strong use case is established, they will then build out workflows such that they can ultimately stand alone as a true enterprise application. This does not necessarily mean replacing the existing interactive or database layers; instead, they will create new structured and unstructured data where generative models utilize these new data sets to enhance the product experience–essentially creating a new class of ‘super data sets.’

A core focus for these products should be integrations with the ability to ingest, clean and label the data. For example, to build a new customer support experience, it’s not enough to simply ingest the knowledge base of existing customer support tickets. A truly compelling product should also incorporate bug tracking, product documentation, internal team communications and much more. It will know how to pull out the relevant information, tag it and weigh it in order to create novel insights. It will have a feedback loop that allows it to get better with training and usage, not only within an organization but across multiple organizations. When a product accomplishes all of this, switching to a competitor becomes very difficult–the weighted, cleaned data is highly valuable and it would take too long to achieve the same quality with a new product.

At this point the intelligence lies not only in the product or model, but also in the associated hierarchy, labels and weights. Insights will take minutes instead of days to deliver, with a focus on actions and decisions rather than just synthesis of information. These will be the true ‘system of intelligence’ products that leverage generative AI, marked by these defining traits:

1. Deep integration with company workflows and ability to capture newly created structured and unstructured data

2. Be sophisticated around the characterization and digestion of data through hierarchy, labels and weights

3. Create data feedback loops within and between customers to enhance the product experience

One key question I love to ask customers is where a new product stack ranks with the other tools they use. Normally the ‘system of record’ product is the most important, followed by the ‘system of engagement’ product, with additional tooling at the bottom of the list. The least important product will be the first to get cut when the budget is tight, so emerging ‘systems of intelligence’ products must provide enduring value in order to survive. They’ll also face steep competition from incumbents who will build generative AI-enabled intelligence capabilities into their products. It will be up to the new wave of ‘systems of intelligence’ to couple their offerings with high-value workflows, collaboration and the introduction of ‘super data sets’ to endure.

Transformation of the AI space has accelerated over the last 12 months, and the industry is learning fast. Open-source models are proliferating and closed proprietary models are also evolving at an atypically rapid pace. Now it’s up to founders to build enduring ‘system of intelligence’ products atop this rapidly shifting landscape–and when it’s done right, the impact on enterprises will be extraordinary.