The Reality of Implementing AI in the Enterprise

Introduction

Through the ages I suspect the term “AI” will always be used to refer to the most advanced form of automation available; in the 1980s AI research was all about rules and logic — barely a thought about data. Two years ago, the term got a refresh when ChatGPT was revealed.

Now “AI” refers to a machine’s ability to complete nuanced details of immense realities, like human language, based on small prompts by humans.

These superpowers are the result of efficient neural network architectures that use gigantic datasets — e.g. the internet — to train these foundation models. The resulting Generative AI (GenAI) applications will be pressed into our lives at different paces depending on the domain.

Sense Street has built an enterprise-grade platform that applies GenAI in various ways to dialogue data in capital markets.

This simple picture conceals a tremendous amount of complexity in both resources and coordination. This post will abstract from our experience and unpack that complexity to discuss some points that are important for enterprise GenAI in general — specifically the importance of:

• sky high expectations
• continuous data curation
• having the right team composition
• carefully constructing a product roadmap to guarantee successful adoption

Expectations

Achieving perfection in enterprise applications has always been more critical than it is for consumer apps. This is because enterprise apps operate within an ecosystem of other applications and use cases, where propagating errors can be devastating. GenAI’s promise of high performance should allow for more applications using statistical models.

The goal should not be an easier workload for your data science team, it should be outstanding performance

For those applications where data scientists previously could get you to 80% accuracy, expect your GenAI-powered app to achieve 95% or better. In other words, a genuine revolution is underway, and you should expect substantial improvements.

Consider your traditional ML stack: achieving 80% accuracy likely required considerable effort. With shiny new LLMs, reaching 80% can probably be achieved with less effort from your data scientists. But the goal should not be an easier workload — it should be outstanding performance. If an LLM strategy is getting you to the same performance as before (or slightly better), don’t waste your time making the change.

It is important not to be locked into a single model or platform to retain the benefit of fast marginal product development

Successfully deploying enterprise GenAI is challenging and costly, but this effort eventually amortises over an increasing number of use cases built on the same infrastructure. Spinning up new applications across a wide domain should be simplified to training a model. While building a dataset is not trivial, it is less costly and time-consuming than engineering deterministic software workflows. At Sense Street the platform we built for capital markets is designed to quickly spin up new applications anywhere natural languages are used, irrespective of asset class or chat platform.

Scaling like this requires an agile GenAI function. It is important not to be locked into a single model or platform to retain the benefit of fast marginal product development.

Data

We all know data is the alpha and the omega of modern machine learning (this wasn’t always the case — remember those expert systems from the 1980s). But the quantity and quality of data needed for machine learning varies widely.

Given an optimal annotation scheme and enough curated data with minimal errors, such a system can outperform humans

Machine learning applications can be categorised based on user expectations and the nature of the data used for training. In the lower-left quadrant are applications with forgiving users, where minor mistakes (like omitting salt in a recipe) aren’t critical. These applications often use widely available data.

In contrast, enterprise applications often fall in the top-right quadrant, where perfection is crucial. These applications aim to replicate human performance and require proprietary data generated by specific enterprise activities. This data is unique and not commonly found on the open web, meaning large foundational language models have limited exposure to it. To achieve high performance, models need to be trained on this bespoke data, often requiring detailed annotation to be effective.

Consider an experiment conducted at Sense Street. Synthetic data was used to simulate complex trade negotiations in repo markets. This data was submitted to GPT-4, guided by a few example constraints. GPT-4 struggled with the task, often refusing to provide input on certain fields and generating incorrect values for others.

By comparison, a much smaller language model — around 200 times smaller than GPT-4 — was trained on high-quality annotated data. This model, with well-partitioned fields in a complex schema, performed nearly at human levels. Given an optimal annotation scheme and enough curated data with minimal errors, such a system can outperform humans.

In production, data curation never ends

Assuming a well-curated dataset, how much data do you need? This depends on the complexity of the task, but in general you probably need in the region of a few thousand to tens of thousands of annotations for highly bespoke enterprise applications. But when you try to put such a system into production you’ll almost certainly need a lot more data.

There are two reasons for this. First, the sunk cost of implementing a system for an enterprise is large. That cost only makes sense if the system can be used for more than one use case — so you must demand multiple applications from your GenAI platform. And if all those applications are to perform perfectly, you’re going to have to manage multiple medium-sized task-specific datasets. Tread carefully: this situation can quickly become unscalable. At Sense Street we are constantly massaging a complex hierarchy of annotation schemes, balancing model performance and annotator efficiency.

The second reason your dataset will grow is to account for shifts in distribution. Consider a model trained to detect the direction of interest in dialogue, trained on data from a period of declining credit spreads — a market with a high ratio of buyers to sellers. That model failed when shown data from a period when the ratio became more balanced as spreads widened during Covid. As with all statistical systems, GenAI retains the bias of its training data. The only way to protect model performance from future unknowable changes in distribution is to continuously curate and retrain with fresh data. In production, data curation never ends.

Resources

Discussions about human-in-the-loop systems often focus on user feedback improving production machines. In some cases user input directly and quickly affects the service, such as in automated bidding systems. However, GenAI apps intended for the enterprise tend to replicate human-intensive tasks. Effective data curation for multiple such applications requires a team of engineers, data scientists, domain experts and users. Properly coordinated, GenAI applications should drive performance to human levels and beyond.

At sensitive enterprises production pipelines can take years to initiate

The production pipeline transforms new data which is served to users via applications. Accept that applications in the enterprise are tested intensely. At a minimum, plugging your inference pipeline into the application layer will involve engineers, business analysts and information security sign-off. While this might be a one-off effort, at sensitive enterprises production pipelines can take years to initiate.

Domain-specific data analysts, skilled in annotation design and machine learning, are the heart of the pipeline

Applications collect false positives through user feedback, while false negatives — which aren’t shown to the user — are captured by sampling output from the production pipeline. Over time this process becomes systematic as analysts better understand the data. Continuous analysis ensures edge cases and distribution shifts are accounted for. This requires data analysts who understand both the domain and the annotation scheme. Users can’t replace these analysts because they don’t see the parts of the domain excluded by the application. In Sense Street’s pipeline, traders can’t review conversations rejected by the production pipeline for errors, as that would defeat the pipeline’s purpose.

As ever, changes in production must be Pareto optimal

When a dataset changes, retraining the GenAI modules is necessary. Additionally, significant engineering work is needed for both pre-processing and post-processing software patches. Adjustments to the model architecture and hyper-parameters, particularly sampling strategies, require the expertise of machine learning scientists. All these changes are then subjected to traditional integration tests since, as ever, production changes must be Pareto optimal.

Roadmap

GenAI will enter the enterprise in various ways. At Sense Street we have stumbled on a promising roadmap for this transformation, conceptualised as three generations of generative applications.

First generation: unambiguous evaluation

The hallmark of first-generation applications is the necessity for unambiguous evaluation. These applications generate outputs that can be clearly and objectively assessed against test data. For instance, at Sense Street we use generative methods to extract complex trade negotiations — known as Voice RFQs — between human traders. Unlike traditional information extraction methods that pick elements from existing data, our models generate new outputs. This allows for straightforward evaluation against unambiguous test data.

Second generation: augmenting computational processes

Second-generation applications enhance other computational processes by providing previously inaccessible data. They do not necessarily involve additional machine learning, but leverage the output of first-generation software to enrich queries against other data sources. For example, a trading desk can now analyse how often it fails to secure lucrative trades negotiated in bilateral chats. Similarly, transaction cost analysis can programmatically integrate executable quotes from conversational data to augment external price sources.

Third generation: novel content creation

Third-generation applications, still emerging, involve language models prompted to create novel content. Evaluating this content will be challenging, as subjective factors come into play — the value of one trade idea over another. However, the content will be highly relevant, since the underlying models are built from first-generation applications and the prompts are informed by datasets aggregated in the second generation. These applications hold significant potential but have yet to be fully realised in the enterprise.

In summary, Sense Street’s roadmap for generative applications outlines a clear progression: starting with unambiguous evaluation, moving to augmenting existing computational processes, and ultimately leading to the creation of novel, contextually relevant content.