Researchers at Salesforce AI Research posted EGI-Bench to arXiv on Oct. 19, a suite that runs eight existing benchmarks for enterprise AI agents through one command-line tool and combines them into a single Enterprise General Intelligence (EGI) Score.
The paper’s lead authors are Zhiwei Liu and Zhujun Lan, with 13 co-authors including Silvio Savarese and Shelby Heinecke. The code is on GitHub under Salesforce AI Research.
The authors start from a practical complaint. Benchmarks for enterprise agents “live today in separate repositories with divergent runners, dependencies, and scoring conventions,” they write. That makes models hard to compare and “obscures what a single aggregate number actually represents.”
How the score works
The suite pulls in CRMArena, for customer-relationship software, along with HERB, PersonaBench, UserBench, MCP-Universe, MCPEval, Benchforce and LoCoBench. Together they cover tasks like searching company documents, answering questions about a user’s private data, calling business APIs and working in large codebases with up to one million tokens of context.
Instead of averaging each benchmark’s overall score, EGI-Bench routes individual sub-scores into six dimensions, according to the full paper. The authors argue that plain averaging dilutes the signal, since a benchmark that already averages nine sub-scores would count only once.
The six dimensions carry fixed weights. Task outcome accuracy, whether the agent got the right answer, counts for 28%.
Tool use reliability counts for 22% and planning for 16%. Retrieval and grounding in evidence, and safety and abstention, each count for 14%. Engineering quality counts for 6%.
The suite has two modes. Quick mode aims to finish in three hours or less, and deep mode in about 12 hours, the paper says. The published results use quick mode.
What they found
The team tested 14 models from the GPT, Claude and Gemini families. Overall scores ranged from 39.6 for GPT-4o mini to 65.6 for gpt-5.5.
The top was close. Claude Sonnet 4.5 scored 65.25, a fraction behind gpt-5.5, according to the paper’s results table. Gemini 3.1 Pro Preview scored 63.51 and Claude Opus 4.5 scored 63.31.
No model led everywhere. Gpt-5.5 led on task outcomes, planning and retrieval, while Claude Sonnet 4.5 led on tool use and on safety, where its score of 80.28 was the only one above 76. Claude Haiku 4.5, sixth overall, had the best engineering quality score of any model.
Planning was the weakest dimension for every model, with scores between 16 and 31, the authors report. They link that to low rewards on UserBench, a multi-turn dialogue test, and say it shows “multi-turn planning remains an open frontier.”
The cheaper models, including GPT-4o mini, GPT-5 nano and Gemini Flash versions, trailed the top tier by roughly 6 to 26 points, the paper says. GPT-5 mini and Gemini 2.5 Flash were nearly tied, at 51.91 and 51.36.
The weights come from an internal proposal and are applied the same way across the suite, the authors note. Teams that value some abilities over others would get different rankings with different weights.
Source: arXiv, Zhiwei Liu et al., “EGI-BENCH: An Enterprise General Intelligence Benchmark Suite”
Additional sources: arXiv (HTML), Zhiwei Liu et al., “EGI-Bench: An Enterprise General Intelligence Benchmark Suite (full text)”; GitHub, Salesforce AI Research, “egi-bench”
