Visa推出首个智能体购物基准测试：模型近半数测试未过关
Visa 11月5日在旧金山发布 Visa Agentic Commerce Benchmark——首个把主流超级智能（SI）模型放进真实购物场景里考试的行业基准：22个来自12家实验室的模型挑战15项购物任务，整体通过率只有约57.2%，即近半数测试未过关。
这项测试的难点在于：消费者往往只知道自己想买一台电视，却说不清全部规格。模型必须通过对话摸清消费者的需求再作购买决定；如果没有合适的商品，应当选择不买；为防蒙对，模型必须挖掘到足够信息来支撑自己的决定。成绩最好的模型通过率82.2%，最差的只有28.9%。
Visa 公布的细分结果显示：挖掘隐藏购买因素最多的五个模型平均通过率70.2%，挖掘最少的五个不到一半（44.4%）；单次运行成本最高的五个模型通过率67.1%，成本最低的五个为50.2%；一致性是普遍短板——每个模型对每个场景尝试三次，三次全过的比例只有34.2%；失败的运行中约54%仍以模型作出一笔购买告终，即要么买了不符合需求的商品，要么没问全需求就下了单。
Visa 全球增长主管 Rubail Birwadker 说：「智能体有潜力驱动下一个商业时代，但信任是关键。Visa Agentic Commerce Benchmark 是我们提升全行业标准的又一项工作。」Visa 9月发布的信任指数显示，72%的美国消费者用过智能助手，但只有23%信任它代为处理支付交易。
Visa 表示，计划给基准增加更多购物场景和品类，并把测试扩展到结账、支付和下单环节；公众也可以在 Visa Payments Frontier 页面亲自参加这项测试、观看模型尝试的回放。
（来源：Visa）
Visa Launches First-of-Its-Kind Agentic Commerce Benchmark; AI Models Fail Almost Half of Test Runs
Visa on November 5 in San Francisco launched the Visa Agentic Commerce Benchmark, a first-of-its-kind industry benchmark that puts leading AI models through realistic buying challenges — and lets people take the test, too. Across the first round, 22 AI models from 12 labs took on 15 shopping challenges, achieving an overall pass rate of about 57.2 percent — meaning almost half of the test runs failed.
Shopping is difficult for AI because consumers often do not clearly know what they want at the outset — they may know they want a TV, but not all the required specifications. Under the benchmark, models must identify a consumer's needs through conversation and make an informed purchase decision; if nothing fits the customer's requirements, the AI should opt not to buy, and it must uncover enough information to justify its decision.
The best-performing models passed 82.2 percent of attempts, while the worst passed 28.9 percent. The five models that uncovered the most hidden buying factors averaged a 70.2 percent pass rate, against less than half (44.4 percent) for the five that uncovered the least. The five models with the highest average cost per run passed 67.1 percent, compared with 50.2 percent for the five least expensive. Consistency is an issue: every model got three attempts at each scenario, and all three attempts passed in just 34.2 percent of cases. About 54 percent of failed runs still ended with the AI making a purchase that did not meet the customer's identified requirements, or without uncovering all of them.
"AI agents have the potential to power the next era of commerce, but trust is key to making it work," said Rubail Birwadker, Visa's Global Head of Growth. "Visa Agentic Commerce Benchmark is yet another way Visa is working to boost standards across the industry." Visa's Trust Index, released in September 2026, found that while 72 percent of U.S. consumers had used an AI assistant, only 23 percent trusted AI to handle payment transactions on their behalf.
Visa plans to add other shopping scenarios and product categories to the benchmark and extend testing to checkout, payment and order placement. The benchmark has been shared with leading model providers for feedback, and the public can attempt it on the Visa Payments Frontier page and watch replays of AI attempts.
(Source: Visa)
来源：Visa
发布：GSM·美国（发布链接）
图片来源：Visa 总部所在的旧金山 Mission Rock 办公楼群，Sarah Stierch 摄，Wikimedia Commons，CC0
GSM Global · 本篇英文版 English edition of this story
