One of the most common tasks in GTM is evaluating the risk a given deal carries. Instead of reviewing each deal by hand, we now have AI to throw at the problem…but does it actually make the result better? Deals are complex. Different members of the GTM team see them from different perspectives, and much of what matters never makes it into the notes. Unfortunately, too many teams are relying on AI to assess deal risk without taking any of this into account.

Instead of asking “which model should we use?”, perhaps we should be asking how much the model matters at all.

A model reasoning over a thin record isn’t really evaluating the deal; it’s filling the gaps with its own prior knowledge. Different models fill those gaps differently, and the variance that results looks like a model quality problem when it’s really a context problem.

But I don’t like relying on assumptions, so this article walks through a benchmark I ran to test that idea directly, what the data shows, and what it means for anyone building deal intelligence on top of their GTM data.

The Hypothesis

The Question: Can AI be used to determine risk within an active deal?

Too much emphasis is placed on model choice, and not enough on the data that forms the model’s context. When the data is thin, the model matters, because it fills the gaps with its own knowledge. As context grows, model choice matters less, and the amount and shape of the context matter significantly more.

If that’s true, it changes where GTM teams should spend their effort. The instinct is to fix inconsistent AI output by moving up to a more advanced model, with a larger context window and deeper reasoning. This benchmark tests the opposite: that the better investment sits upstream, in the data the model receives and how it is joined together.

Creating the Benchmark

The Setup

To get started, we need a few key ingredients:

GTM Data. Since we want something that works across different data sets while protecting data privacy, the benchmark uses synthetic data, modeled on real CRM fields and inputs.

The Harness. Swappable models, memory, context building, and evals, with multi-run support and structured outputs for evaluation.

A GTM Schema. This is the part too many people skip. If you throw a pile of fields at an LLM, you’re letting the LLM decide what matters, what to discard, and where to substitute its own knowledge. Take the time to define a clean, logical schema that maps to an ontology your organization already agrees on.

Real-World Note: Because I am using synthetic data, I purposely had to introduce issues into the data to mirror the real world. For instance, everyone hates updating CRM notes so some of the deals don’t have any data or the note might be incomplete. The cap for the number of deals was set at 100 for this data set.

Considering Different Perspectives

The variable under test is the “lens” used to evaluate the risk itself. An AE and an SE are reading the same deal from fundamentally different vantage points, one is tracking commercial momentum, the other is tracking technical fit, and each carries signal the other can’t see. A single lens gives you one angle on risk. The combined view gives you the deal.

The AE view contains the Account Executive’s deal stage and their last note about the deal.

The SE view contains the Solution Engineer’s technical notes.

The combined view joins both sets of notes on the same deal.

Executing the Run

Each lens was scored by three models: Haiku 4.5, Sonnet 5.5, and Opus 5.5. I used the latest model in each Anthropic family, so every tier in the comparison was the best version available. Every configuration ran five times on identical input. That’s 9 configurations, 45 runs, and 4,500 scored assessments.

Labels and probabilities in the analysis below use each model’s majority vote and mean probability across its five runs, which separates a model’s actual reading of a deal from run-to-run noise.

Real-World Note: This benchmark measures agreement (between models and between runs) and evidence quality, not predictive accuracy. Agreement isn’t correctness. But when two models split on a binary call about the same deal, at least one of them is wrong, and that’s a meaningful signal in its own right.

Thin Input Exposes the Model

As I mentioned earlier, some of the deal data is purposely incomplete or missing. Looking at the data after the runs were complete, about 50% of the deals had no note from the SE. When the SE note was the only input for deal risk, the model had to decide what a blank note means, and each model decided differently.

The benchmark shows that on thin input, the model makes that call for you without anyone noticing:

• Haiku treated a blank SE note as a warning sign and flagged 86% of those deals.
• Sonnet treated it as “no news is good news” and flagged 10%.
• Opus landed in the middle at 50%.

That’s a 76-point spread across three models reading identical input.

Add the AE notes to those same 50 deals (the combined view) and the models land at 54%, 52%, and 44%. A 10-point spread. Nothing about the models changed between those two rows. Only the input did.

This is the most important finding in the benchmark. When the input is thin or missing, the model quietly makes a judgment call that your team should be making.

So picking a model looks like a technical or budget choice, but on these deals it also decides your rule about missing data.

Matching Totals, Different Deals

The second trap is subtler.

On the AE view, Haiku flags 46 deals, Sonnet flags 46, and Opus flags 45. At the dashboard level, that reads as near-perfect agreement. At the deal level, the three models give the same label on only 52 of the 100 deals. They disagree in balanced ways, and the totals hide it.

The SE view shows the opposite pattern: the totals diverge widely (78, 40, and 58) and deal-level agreement is the lowest of the three lenses at 41. The combined view has the highest deal-level agreement at 62, even though its totals aren’t identical.

Aggregate at-risk counts are a poor basis for comparing models.

Pipeline reviews happen one deal at a time, and that’s where disagreement surfaces.

Context Closes the Gap

Here’s the simplest way to look at it. Forget the lens labels entirely for a moment. Sort every deal by how much text the models had to read, then check how often all three gave the same answer.

With no text at all, the three models agreed on just 21% of deals. Give them 200+ characters and that climbs to 72%. It goes up at every step in between.

The risk scores tell the same story. On a deal with no notes, the most and least worried models were 0.26 apart. Above 350 characters, they’re 0.12 apart. The models aren’t only flipping fewer labels. They’re starting to read the deal the same way.

Even the Odd Couple Starts to Agree

Labels only tell part of the story, so I also compared the underlying risk scores for each pair of models, lens by lens, from thinnest input to richest.

Every pair got closer as the input got richer, with no exceptions. Haiku and Sonnet moved the most. On the SE view their scores were barely related (0.22), but on the combined view they got to 0.64. Sonnet and Opus were already close and ended up nearly in sync at 0.91.

Swapping the Model Matters Less

Here’s the whole argument in one number. Keep the deals and the input exactly the same, swap one model for another, and count how many labels change.

On the SE view, swapping models changes 39% of the labels. On the combined view, it changes 25%. The more context you give the models, the less the choice of model decides the answer.

Now flip the experiment. Keep the model the same and change only what it reads, moving from a single lens to the combined view. That changes 18% to 34% of the labels, depending on the model and where you start.

So on rich input, the model you pick moves fewer labels than it does on thin input. The price gap between them doesn’t shrink at all. Once the context is good, paying for the most expensive model buys you less than it looks like on paper.

Amount Versus Shape

A fair objection is that the combined view simply contains more characters. Is the improvement about joining the data, or just about having more of it?

To separate the two, I compared deals of matched length (the amount). Above 200 characters, single-lens deals and combined deals reach roughly the same three-way agreement, about 72%. Most of the effect comes from the amount of usable signal per deal. Joining the AE and SE views is simply the most direct way to produce it.

Shape still shows up in a few places:

Evidence quality. Single-lens signals cite a non-supporting quote 1.9% to 4.0% of the time. On the combined view, that falls to 1.0% for Sonnet and 0.8% for Opus.

Conflict detection. When both perspectives are in the prompt, the scorer can flag deals where the AE and SE accounts disagree. A single lens can’t surface that disagreement at all.

Blank inputs. On the 11 deals with nothing in either source, the combined view still reached three-way agreement 36% of the time, versus 18% for blank single-lens prompts. A prompt that knows which source is missing handles the gap more reliably than one left to interpret silence on its own.

The deeper point is that shape determines what the model can reason about, not just how confidently it scores. Amount gets you better agreement on the deals you already have; shape gets you coverage on the ones that would otherwise slip through; conflicts, absences, and signals the prompt is structured to catch.

Where Model Choice Still Matters

Context narrows the gap between models significantly, but it doesn’t eliminate it.

The smallest model remains the outlier. Take the 80 deals where Sonnet and Opus agree on the combined view. Haiku matches them on 62. That’s an improvement, but it doesn’t reach the consensus its larger counterparts share.

Agreement between models isn’t stability within one. Opus was the least consistent model from run to run on the combined view: 89% pairwise label agreement across its five runs, versus 95% for Sonnet and 96% for Haiku. On single lenses all three land in the 93 – 99% range, so the gap isn’t about raw capability; it’s about how each model responds to the added complexity of a richer prompt.

Haiku costs roughly $0.30 – $0.35 per 100 deals. Opus runs $1.96 – $2.21. Once context is in good shape, that difference in price becomes a critical decision point.

The Payoff: Token Economics

The full benchmark, 45 runs across three models, cost roughly $53 in tokens. Opus costs six to seven times more per run than Haiku, and roughly twice as much as Sonnet.

On thin input, that premium buys a different answer; though not necessarily a more correct one, since the three models disagree on what the missing data means. On rich input, most of what the premium buys is a few points of agreement. That’s a very different value proposition, and it’s one most teams never measure because they evaluate model choice before they evaluate their data.

The more durable investment sits upstream. Improving the completeness of SE notes, joining AE and SE perspectives into a single record, and defining how missing data should be interpreted all reduce variance for every model at once. That work compounds; a model upgrade doesn’t.

Acknowledging Limitations

This benchmark measures agreement and evidence quality, not predictive accuracy. The real test arrives when these deals close and the risk calls can be scored against outcomes. Convergence between models is evidence of a shared reading of the deal, not proof that the reading is right.

Context length and lens are partially confounded. The richest bucket is made up entirely of combined-view deals. The matched-length comparison helps separate the two, but it doesn’t fully isolate them.

Making This Real

The model choice isn’t irrelevant, but it does have limitations. The purpose of this benchmark is to underscore the importance of sequencing. For teams building deal intelligence on top of a CRM, the order of operations matters:

• Measure the context before the model. Count characters per record, per field. If half of the records are blank in the field being scored, no model will compensate for it.
• Join the perspectives the team already writes. AE and SE notes describe the same deal from two angles. They belong in the same prompt.
• Make missing data a written rule. Decide what a blank note means and state it explicitly, so the answer doesn’t depend on which model happens to be running.
• Build an eval layer before you pick a model. Agreement across runs, evidence quality, and quote-verified signals tell you whether the output is trustworthy, don’t wait for deal outcomes to find out.
• Select the least expensive model that converges. Not the largest model the budget allows.

When the input is thin, you aren’t choosing a model. You’re choosing a bias, and paying a premium for it. Deal intelligence isn’t really an AI problem, it’s a data hygiene problem. The model is not the unreliable part; the notes are. Fix those first, and the model choice gets a lot easier after that.

← Back to Writing