Skip to content
Julius Research

· By Julius Research

DeepFinance: Can AI diligence a $100MM deal?

Contents · 8 sections

DeepFinance is an ultra long-horizon benchmark for measuring models on real-world Private Equity and Investment Banking workflows that would take a human professional between 20 and 30 hours to complete. DeepFinance was composed and reviewed by Private Equity (PE) and Investment Banking (IB) professionals from firms like J.P. Morgan, UBS, and Evercore, and institutions like Harvard Business School, Wharton, and Columbia Business School.

  • Task count: 21
  • Average human time: 27.1 hours per task
  • Average rubric criteria: 84 criteria per task
  • Average environment size: 19 files per data room
  • Average prompt length: 240 words, ~29% shorter than GDPval1 prompts

DeepFinance Leaderboard

  1. Claude Opus 5.5 (max): score 54.8%, 95% confidence interval 46.2% to 64.4%.
  2. Claude Sonnet 5.5 (max): score 54.6%, 95% confidence interval 46.0% to 63.9%.
  3. GPT-6 Astra (xhigh): score 54.2%, 95% confidence interval 46.4% to 62.6%.
  4. GPT-6.1 Sol (xhigh): score 52.9%, 95% confidence interval 45.4% to 61.0%.
  5. Claude Fable 5.1 (max): score 45.6%, 95% confidence interval 40.7% to 51.3%.
  6. GPT-6.1 Terra (xhigh): score 43.5%, 95% confidence interval 37.1% to 50.6%.
  7. Gemini 4 Argon (high): score 41.3%, 95% confidence interval 34.7% to 48.8%.
  8. Muse Spark 1.3 (max): score 37.0%, 95% confidence interval 32.6% to 41.9%.
  9. Kimi K3.1 Pro (max): score 34.6%, 95% confidence interval 30.4% to 39.2%.
  10. Grok 4.7 (xhigh): score 34.1%, 95% confidence interval 29.9% to 38.7%.
  11. Qwen 3.8 Max (xhigh): score 30.3%, 95% confidence interval 26.3% to 34.6%.
  12. Inkling 2 Small (xhigh): score 29.8%, 95% confidence interval 26.1% to 33.5%.
  13. GLM 5.3 (max): score 27.7%, 95% confidence interval 23.9% to 31.5%.
  14. DeepSeek V4.1 Flash (max): score 27.3%, 95% confidence interval 23.6% to 31.3%.
  15. GPT-6 Luna (xhigh): score 24.6%, 95% confidence interval 20.1% to 29.2%.
  16. Mistral Large 4 (high): score 21.5%, 95% confidence interval 17.2% to 25.9%.
  17. DeepSeek V4 Pro (max): score 20.5%, 95% confidence interval 16.5% to 24.7%.
  18. Nemotron 3 Ultra (high): score 14.5%, 95% confidence interval 11.3% to 17.8%.
  19. Inkling (max): score 13.1%, 95% confidence interval 9.3% to 17.2%.
  20. Inkling Small (max): score 10.9%, 95% confidence interval 7.3% to 14.8%.
Error bars show 95% confidence intervals.

Philosophy

There is a fundamental gap in how AI models are evaluated on real-world high-finance work. We believe this gap stems from a deeply flawed understanding of how analysts in high-finance perform real-world tasks.

Existing evals measure the models in two ways: their ability to retrieve answers from a set of financial documents and their ability to produce numerically correct final outputs. However, real-world Private Equity and Investment Banking workflows are meaningfully more complicated.

In software benchmarks (like SWE-bench2 and DeepSWE3), models are graded on their output's ability to pass tests. If tests pass and the fix works, the path (or trajectory) the agent took matters less. Finance has no similar test suite. A high-finance valuation is like a pull request that has to pass a line-by-line code review, where every number cites its source and every assumption needs to hold up in front of the MD, the IC, and a client.

In PE and IB, while two analysts could produce identical numbers for the same task, one output could rest on completely incorrect assumptions. Change a single number upstream, and the entire workflow no longer makes sense. The trajectory of an ultra long-horizon high-finance task is full of assumptions and judgment calls. Without being able to audit the steps and judgment of the analyst, a professional in high-finance cannot blindly trust the work that is presented to them.

Failure modes of current evals

Existing high-finance benchmarks have two main limitations in how they currently evaluate models in high-finance tasks:

  1. Overemphasizing question-answer ability from financial documents
  2. Grading that is limited to numerical correctness of the outputs

Existing benchmarks often fail to consider the real-world working environment that high-finance professionals face daily. A set of financial documents and revisions in varied formats, long email chains, sparse meeting notes, and other data room objects almost invariably contains conflicting information that must be reconciled. The model's ability to judge and choose from overlapping sources is rarely measured.

  • FinanceBench4: Questions are simple and too easy; this is not realistic or representative of the deal work analysts are hired for.
  • BizBench5: Checks one final number against an answer key; DeepFinance grades end-to-end workflows against up to 140 criteria.
  • FinanceQA6: Spoon-feeds the model relevant 10-K sections with each question, leaving no room to evaluate model trajectory.
  • BigFinanceBench7: Uses solid trajectory reasoning but limits the scope of questions and stops well short of end-to-end workflows.

Benchmarks like OfficeQA8 by Databricks are great at properly tracking a company taxonomy, but rarely step beyond grading pure numerical correctness. For high-finance ultra long-horizon tasks, trajectory is crucial to understand and audit the underlying method and assumptions that were put into the work.

ELO-based preference ranking used by benchmarks like GDPval1 misses the nuance in the complex long-horizon workflows by only comparing binary final outputs, and doesn't grade on the atomic trajectory.

There are benchmarks, like Halluminate's Westworld Finance Diligence Bench9, that have attempted to measure semi-long-horizon work. These benchmarks only study the agent's trajectory and leave it out of the score, since every point on their leaderboard is from checks on the final output. While this might be an effective way to check final answers, this fundamentally is not how finance work is judged in the real-world. Correct numbers without a source or auditability will be scrutinized by an investment committee. DeepFinance grades whether every number is sourced and every judgment call is auditable and traceable.

These highlighted approaches do not represent real-world uses. DeepFinance was built to demonstrate that the path of the task itself carries the assumptions and therefore grading must account for the trajectory when evaluating model performance. Auditability is paramount to internal and external stakeholders like lenders, clients, and brokers. Understanding how a model (or an analyst) came to an assumption is the key to understanding what could go right, wrong, or sideways in a multi-million dollar deal.

Average Number of Criteria Per Task

Use the up and down arrow keys to inspect benchmarks, then Tab to read the details.

DeepFinance · 84 criteria on average
Mean rubric criteria per task

Methodology

Task construction

DeepFinance covers four primary pillars of high-finance work:

  • Valuation: estimate what a business is worth and justify the assumptions.
  • Due diligence: investigate the records, identify important risks, and explain their effect on the deal.
  • Underwriting: assess the deal's risks and economics, then determine acceptable financing or investment terms.
  • Financial modeling: build a model whose assumptions, calculations, and outputs remain consistent.

Each ultra long-horizon task pairs a request with a data room: the collection of records needed to complete the assignment. Tasks use real data that has been scrubbed and anonymized. The request states the objective, constraints, and expected deliverables; from there the agent must work out how to complete the analysis. Every DeepFinance problem statement has on average 240 words; we find this to be realistic in crafting a real-world end-to-end workflow. This compares to other evals that have prompts that are very short, but only handle simple arithmetic or quantitative tasks.

Each DeepFinance task comes with a data room of on average 19 files, comprising the same material analysts work with from day-to-day. File types include spreadsheets (.xlsx), Word documents (.docx), emails (.eml), PDFs, and raw CSV extracts. Inside are emails, chats, and deal-team notes; models and projections; CIMs, SPAs, term sheets, commitment letters, and transaction data; diligence reports; IC minutes; and market data and comps.

Each ultra long-horizon task is representative of end-to-end work done at top PE and IB firms like Evercore, J.P. Morgan, and UBS. Every task uses real data that has been anonymized and scrubbed. The work simulates analyst work on real deals, not just simple Q&A on sets of documents or finding one off answers. There is ambiguity, and there are conflicting sources.

Scoring

Graders in DeepFinance evaluate the finished deliverable together with evidence of the trajectory taken. Sources, tool results, calculations, and revisions can show whether the agent completed the required work, and agents are graded on labeled rubrics with label categories:

  • Output: Are the calculations, conclusions, and requested deliverables correct?
  • Workflow: Did the agent find the relevant evidence and use it in the analysis?
  • Judgment: Does the evidence support the agent's assumptions and decisions?

Each rubric criterion was written and verified by experts from the world's top PE and IB firms.

By tracking the steps taken with respect to these three categories, experts and reviewers are able to audit both workflow and judgment, irrespective of final output. When it comes to ultra long-horizon tasks that can take 20-30+ hours for an analyst to complete, trajectory is invaluable.

DeepFinance tasks test whether an agent can carry decisions through a long assignment and abide by "best practices" along the way. A source chosen early may affect a forecast, the amount a company can borrow, its valuation, and the final recommendation. When an input changes, every calculation that depends on it needs to change too. But that doesn't mean that an agent should only be penalized for upstream decisions that go wrong; we should also evaluate the quality of steps and judgments made along the way, irrespective of output. This is how we capture trajectory.

DeepFinance graders give partial credit when the models take the right steps on the trajectory.

Rubric score is the percentage of rubric criteria satisfied, averaged over attempts and then tasks. Attempts that errored were re-run, so every configuration is scored on 10 successful attempts per task.

Quality assurance

Every task in DeepFinance was reviewed by multiple layers of agentic verifiers and human expert peers with backgrounds from top firms like Evercore and institutions like Harvard Business School and Wharton. Each task on average has 84 rubric criteria.

Expert review and verifiers established four parameters:

  1. The task is clear and the source files contain enough information to complete it.
  2. The reference solution is supported by the evidence.
  3. The rubric checks the requested work and accepts valid solutions.
  4. A failed criterion reflects the agent's work, rather than a faulty reference, grader, or environment issue.

Reviewers traced disputed grades and expert commentary back to the source files and agent traces. This distinguishes model mistakes from evaluation mistakes, ensuring that errors in the evaluation process do not influence our conclusions about model capability.

Harness

Unlike many benchmarks, DeepFinance evaluates models in the harness they are most likely to be used in. This means that our environment supports a multi-harness setup where:

  • Claude models run in claude-code v.2.1.318
  • GPT models run in codex v.0.166.0
  • Gemini, Muse, DeepSeek, GLM, Nemotron, and Inkling models run in opencode-ai v.1.19.3
  • Kimi models run in kimi-code v.2.4.1
  • Qwen models run in qwen-code v.0.26.6
  • Grok models run in x.ai v.1.0.70
  • Mistral models run in mistral-vibe v.2.24.3
  • Node.js runs on v.22.23.4

Each harness has nearly every native tool enabled, including web search. The one exception is the ask-user tool, which we turn off so every run can finish on its own. Many current evals cut off the agent's access to the internet so agents cannot look up the answer keys. DeepFinance doesn't need to. Every run gets its own sandbox, and the answer key and rubric sit in separate, isolated grading environments.

The goal of DeepFinance is to test agents in environments that are representative of those that PE and IB professionals use in their day-to-day work. This approach provides a more realistic evaluation of model performance rather than using a single universal harness that imposes arbitrary or artificial constraints on the models.

Results

Quantitative Analysis

  1. The gap is widening on the frontier

    Anthropic and OpenAI models take the six highest scores. The top four configurations, Claude Opus 5.5, Claude Sonnet 5.5, GPT-6 Astra, and GPT-6.1 Sol (54.8, 54.6, 54.2, 52.9), are not statistically distinguishable (paired bootstrap, 95%). The best model from another provider, Gemini 4 Argon, scores 41.3, and the bottom group (Nemotron 3 Ultra, Inkling, and Inkling Small, at 10.9 to 14.5) sits roughly 40 points below the top.

  2. GPT-6.1 class models dominate the Pareto frontier

    GPT-6.1 Sol with Extra high reasoning averaged $0.81 per completed task with a 52.9% score. The cheapest model scoring above 30% was GPT-6.1 Terra at $0.38 per completed task, with a 43.5% score. The GPT-6 family accounted for 2 of the 4 cheapest models for finance across the 20 tested.

Score vs. cost

Mean rubric score against mean API cost per task. The dashed line marks the Pareto frontier: configurations not dominated on both score and cost.

Score vs. tokens

Mean rubric score against mean tokens per task. The dashed line marks the Pareto frontier: configurations not dominated on both score and tokens.

Results

Source: DeepFinance (Julius Research)Updated juliusresearch.ai

Results: mean rubric score (%) by model configuration with its 95% confidence interval, best first
RankModelLabMean rubric score with 95% confidence interval
1Claude Opus 5.5 (max)Anthropic54.8% mean rubric score, 95% CI 46.2 to 64.4
2Claude Sonnet 5.5 (max)Anthropic54.6% mean rubric score, 95% CI 46.0 to 63.9
3GPT-6 Astra (xhigh)OpenAI54.2% mean rubric score, 95% CI 46.4 to 62.6
4GPT-6.1 Sol (xhigh)OpenAI52.9% mean rubric score, 95% CI 45.4 to 61.0
5Claude Fable 5.1 (max)Anthropic45.6% mean rubric score, 95% CI 40.7 to 51.3
6GPT-6.1 Terra (xhigh)OpenAI43.5% mean rubric score, 95% CI 37.1 to 50.6
7Gemini 4 Argon (high)Google41.3% mean rubric score, 95% CI 34.7 to 48.8
8Muse Spark 1.3 (max)Meta37.0% mean rubric score, 95% CI 32.6 to 41.9
9Kimi K3.1 Pro (max)Moonshot AI34.6% mean rubric score, 95% CI 30.4 to 39.2
10Grok 4.7 (xhigh)SpaceXAI34.1% mean rubric score, 95% CI 29.9 to 38.7
11Qwen 3.8 Max (xhigh)Alibaba30.3% mean rubric score, 95% CI 26.3 to 34.6
12Inkling 2 Small (xhigh)Thinking Machines29.8% mean rubric score, 95% CI 26.1 to 33.5
13GLM 5.3 (max)Z.ai27.7% mean rubric score, 95% CI 23.9 to 31.5
14DeepSeek V4.1 Flash (max)DeepSeek27.3% mean rubric score, 95% CI 23.6 to 31.3
15GPT-6 Luna (xhigh)OpenAI24.6% mean rubric score, 95% CI 20.1 to 29.2
16Mistral Large 4 (high)Mistral21.5% mean rubric score, 95% CI 17.2 to 25.9
17DeepSeek V4 Pro (max)DeepSeek20.5% mean rubric score, 95% CI 16.5 to 24.7
18Nemotron 3 Ultra (high)NVIDIA14.5% mean rubric score, 95% CI 11.3 to 17.8
19Inkling (max)Thinking Machines13.1% mean rubric score, 95% CI 9.3 to 17.2
20Inkling Small (max)Thinking Machines10.9% mean rubric score, 95% CI 7.3 to 14.8
Mean rubric score by model configuration on DeepFinance (21 tasks, 10 attempts per task). Rubric score is the percentage of rubric criteria satisfied. Error bars denote 95% bootstrap confidence intervals (100,000 resamples over tasks and attempts).

Qualitative Analysis

  1. Fable for Finance: not worth the cost

    While Fable 5.1 has been regarded for its intelligence, this didn't apply to high-finance as it scored below GPT-6 Astra, Claude Opus 5.5, Claude Sonnet 5.5, and GPT-6.1 Sol. Furthermore, Fable 5.1 was the most expensive model that we tested while scoring under 50% on end-to-end tasks.

  2. Even frontier models are weak at turning evidence into proper judgement

    In most DeepFinance tasks, agents are able to find the relevant information without recognizing its implications. They also have difficulty challenging a supplied assumption or evaluating better opportunities unless strictly told to, even in situations where a human analyst would instinctively do so. This means that when a CFO says one thing and an analyst says another, there is little reconciliation effort between balancing the importance of both perspectives.

  3. Upstream mistakes cause cascading downstream failures

    Incorrect earnings, source selection or completion timing can affect debt capacity, fees, cash flows, valuations and investor returns. Atomic output checks reveal those consequences, but the number of failed checks should not be interpreted as the number of independent reasoning failures.

  4. Models often execute calculations correctly after selecting the wrong inputs

    In a lending task, agents correctly calculated debt capacity using an incorrect earnings subtotal. In a pharmaceutical royalty task, they correctly applied payment caps and milestone rules to a forecast that omitted an entire sales channel. The agents knew how to perform the calculations but failed to verify the numbers feeding them.

  5. Agents find decision-relevant evidence without acting on it

    In one acquisition task, moving the closing date changed shareholder payouts and lowered the required offer by $1.25 million. All attempts by frontier models missed this saving, even though several had identified the buyer's right to choose the date.

The path toward AGI in dealmaking

For AGI to be useful in dealmaking, its work needs to hold up when someone checks it. Across high-finance assignments that take professionals 20-30 hours, agents must choose sound assumptions, use them consistently, and explain the decisions that matter.

DeepFinance evaluates both the result and the work behind it. Auditability gives professionals a way to check the analysis and gives researchers a way to find the mistakes that need fixing. That brings us closer to measuring whether agents can take an assignment from a data room to a decision people can rely on.

References

  1. Patwardhan, T., et al. (2025). GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. arXiv:2510.04374. ↩ ↩2

  2. Jimenez, C. E., et al. (2023). SWE-bench: Can Language Models Resolve Real-World GitHub Issues? arXiv:2310.06770. ↩

  3. Huang, W., Lee, C., Tng, L., & Ge, S. (2026, May 26). DeepSWE: Measuring frontier coding agents on original, long-horizon engineering tasks. Datacurve. ↩

  4. Islam, P., et al. (2023). FinanceBench: A New Benchmark for Financial Question Answering. arXiv:2311.11944. ↩

  5. Koncel-Kedziorski, R., et al. (2023). BizBench: A Quantitative Reasoning Benchmark for Business and Finance. arXiv:2311.06602. ↩

  6. Mateega, S., et al. (2025). FinanceQA: A Benchmark for Evaluating Financial Analysis Capabilities of Large Language Models. arXiv:2501.18062. ↩

  7. Wang, A., et al. (2026). BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agents. arXiv:2606.03829. ↩

  8. Databricks AI Research. (2025, December 9). Introducing OfficeQA: A benchmark for end-to-end grounded reasoning. Databricks. ↩

  9. Halluminate. (2026, August 11). Westworld Finance Diligence Bench. Halluminate. ↩

Cite this work

Julius Research. (2026, November 5). DeepFinance: Can AI diligence a $100MM deal? Julius Research. https://juliusresearch.ai/blog/deep-finance

BibTeX
@misc{juliusresearch2026deepfinance,
  title        = {{DeepFinance}: Can {AI} Diligence a \$100{MM} Deal?},
  author       = {{Julius Research}},
  year         = {2026},
  month        = nov,
  howpublished = {\url{https://juliusresearch.ai/blog/deep-finance}},
  note         = {Blog post}
}

Contributors

Drew Hahn, Tyler Shibata, Jackson Grove, Rahul Sonwalkar, and the Julius Research team

All contributors contributed equally.