Making Sense of Chat: Generative AI in Capital Markets
What it takes to extract counterparty intentions from the messiness of chat — extraction, grounding, and why generic LLMs don’t work out of the box.
What do we use LLMs for at Sense Street?
At Sense Street, we build AI that extracts counterparty intentions in all the messiness of chat. This structured data powers analytics, automation and observability applications at investment banks. But what does this endeavour really involve, and how does generative AI fit into the picture? Going from chat to structured data involves two main steps: extraction and grounding.
The technical approach: extraction and grounding
Extraction
For capital markets conversations, this step aims to identify all market-relevant opinions in a conversation, converting them into a list of RFQs (requests for quotes) and IOIs (indications of interest).
Determining counterparty intentions in trader chats turns out to be hard. Speaker intentions are a mysterious thing. Counterparties lie and mislead so as not to reveal their book. Speakers use domain slang — slang that only exists in chat data. If you’re hoping to learn these phrases from financial news or earnings reports, try again: those are well-formed texts without the eccentricities we observe in clipped back-and-forths. And domain dynamics transform the meaning of a phrase depending on the role of the speaker.
Why generic models fall short
Generative models derive their power from learning the underlying distribution of the data they’ve seen at pre-training. LLMs don’t work out of the box on these problems in large part because the data in these chats is unlike anything the models have seen during pre-training.
This is where fine-tuning comes in. Fine-tuning allows us to leverage task-specific supervised signal on domain data to adapt a large generic model to a particular problem space. However, getting this process to work is not trivial.
Technical literature, and the domain itself
Some of our approaches leverage technical literature: we’ve been implementing state-of-the-art methods such as LoRA adapters to avoid overfitting and reduce memory consumption, making training tractable — and new methods keep coming out all the time.
Other times we leverage the domain to approach technical challenges. Open-source models have limited context windows, which means that given a long conversation they will arbitrarily cut off context once the window is saturated. We use domain knowledge to make sure models are considering and grouping relevant context, so that inevitable chunking doesn’t happen arbitrarily. In all of this, we continue using supervised learning.
More ways to use supervised signal
None of this, by itself, is particularly surprising. Nor is it surprising that annotations continue to have a place in adjusting generic models to particular tasks on domain-specific datasets. What has been surprising for us is that the possibilities for getting value from supervised signal have multiplied in the age of generative AI.
For example, we train a model to solve our ultimate task — but we also train it to solve ancillary tasks. We train the model to make decisions, and we also train it to produce explanations for those decisions, and to align those explanations with our own. In doing this, we’re able to use the underlying model’s understanding of natural language to create more fine-grained distinctions between classes. We have found this to be incredibly powerful.
Continuous learning
We have developed methods to sample the highest-value data arriving in production, to annotate, evaluate and readjust model weights over time.
About Sense Street
We are developing natural language understanding systems for capital markets. Our premise is simple: markets are conversations, and we aim to help participants have better and more efficient conversations. Through partnerships with global financial institutions we have access to datasets that have not been made available in the past. This allows us to create language models uniquely suited to capital markets while advancing the state of the art. Our platform ensures robust and safe delivery of LLMs into the financial enterprise, enhancing analytics, workflow automation and observability.