"In May, researchers at Carnegie Mellon University released a paper showing that even the best-performing AI agent, Google's Gemini 2.5 Pro, failed to complete real-world office tasks 70 percent of the time. Factoring in partially completed tasks — which included work like responding to colleagues, web browsing, and coding — only brought Gemini's failure rate down to 61.7 percent.
And the vast majority of its competing agents did substantially worse.
OpenAI's GPT-4o, for example, had a failure rate of 91.4 percent, while Meta's Llama-3.1-405b had a failure rate of 92.6 percent. Amazon's Nova-Pro-v1 failed a ludicrous 98.3 percent of its office tasks.
Meanwhile, a recent report by Gartner, a tech consultant firm, predicts that over 40 percent of AI agent projects initiated by businesses will be cancelled by 2027 thanks to out-of-control costs, vague business value, and unpredictable security risks.
"Most agentic AI projects right now are early stage experiments or proof of concepts that are mostly driven by hype and are often misapplied," said Anushree Verma, a senior director analyst at Gartner.
The report notes an epidemic of "agent washing," where existing products are rebranded as AI agents to cash in on the current tech hype. Examples include Apple's "Intelligence" feature on the iPhone 16, which it currently faces a class action lawsuit over, and investment firm Delphia's fake "AI financial analyst," for which it faced a $225,000 fine.
Out of thousands of AI agents said to be deployed in businesses throughout the globe, Gartner estimated that "only about 130" are real."
https://futurism.com/ai-agents-failing-industry