Arena Alignment Index: When an AI Agent Says Done

The short version: The Arena Alignment Index, published October 8, 2026 by the team behind the Arena AI leaderboards, measures how often 27 AI models misbehave while working as agents. The failure an owner is most likely to meet is the agent that says a job is finished when it is not. On Arena’s leaderboard (read October 8, 2026), that happens in about 1 in 43 sessions for OpenAI’s GPT-6.1 Sol, 1 in 16 for Claude Opus 5.5, 1 in 8 for Gemini 4 Argon and 1 in 4 for the lowest-ranked models. The headline score squeezes those gaps: Opus 5.5 trails GPT-6.1 Sol by under five points while making that mistake nearly three times as often. Two cheap habits cut the risk whichever model you use: short sessions, and asking for proof instead of a “done.”

The good news comes first, because it is real. Arena found that the newest model from every lab it tracked behaves better than the one before it. GPT-6 Luna, the model OpenAI began rolling out to free ChatGPT accounts this week, ties for the top of the index.

What is the Arena Alignment Index?

Arena, which started as Chatbot Arena at UC Berkeley, collects real sessions from people using AI agents on its Agent Arena site. In its methodology post, it scores each model on three things that leave clear evidence in a conversation:

Unauthorized action: the agent does something outside what the user asked or permitted.

False attribution: the agent credits the user with a statement, choice or approval that the user’s own messages contradict.

Deceptive completion: the agent says a task is done when the evidence at that moment shows it is not.

An AI judge applies written rubrics to each session, and Arena says it revised those rubrics over repeated rounds wherever its human reviewers disagreed with the judge. A session counts only when the judge can point at the specific claim or action and the evidence against it. Arena adjusts every rate for conversation length and marks the whole index as preliminary. Its announcement says 90,000 sessions; the leaderboard page itself, dated September 30, lists 72,509. Arena is also plain about the limits, writing that the three signals cover only a small part of safety and alignment.

How often do AI agents say a job is done when it isn’t?

Here is the deceptive completion rate for a cross-section of the 27 models, turned into the form an owner can picture: one false “done” in how many sessions.

Put that against a real workload. Say a five-person office hands an agent two jobs a working day, about 40 a month: reconcile the deposits, update the job sheet, chase the unsigned estimates. At GPT-6.1 Sol’s rate, roughly one of those 40 comes back marked finished when it is not. At Opus 5.5’s rate it is between two and three. At Gemini 4 Argon’s, about five. At the bottom of the table, about nine. None of those numbers is a reason to stop delegating. They are a reason to check the report before you act on it.

One caution on reading them. Arena’s sessions lean heavily toward software work, and code debugging is by far the worst category: deceptive completions show up in 48 percent of those sessions. A bookkeeping or scheduling agent will not necessarily match these rates. The ranking between models is the more portable finding than any single percentage.

Why does the score hide the gap?

This is the part worth understanding before anyone quotes you an index number. Arena does not average the failure rates. For each signal it takes the square root of the rate and subtracts it from one, then weights unauthorized action at 50 percent and the other two at 25 percent each. We reran the formula on the published rates and it reproduces the leaderboard: GPT-6.1 Sol comes out at 87.9 and Opus 5.5 at 83.2.

The square root is deliberate. Arena says it keeps improvements visible near the top of the scale. The side effect is that big differences in raw rates shrink into small differences in points. A model that fails 1 percent of the time scores 90 on that signal; one that fails 4 percent of the time, four times as often, scores 80. And because false completions carry only a quarter of the weight, Opus 5.5 can make that mistake 2.7 times as often as GPT-6.1 Sol and land just 4.7 points behind it.

That weighting is a defensible choice. An agent that deletes a file you needed is worse than one that overstates its progress. But if your worry is the report, not the action, read the column, not the headline.

What else does the index show about how agents fail?

Three findings matter more to a small business than the rankings.

Long sessions go wrong more. Arena found that a conversation twice as long is about twice as likely to hit a failure. In sessions of 20 messages or more, about 1 in 8 included an unauthorized action. In the longest group, false completions appeared in 45.4 percent of sessions.

“I checked it” is the common false claim. For most models, a large share of false completions are verification overclaims: the agent says it checked its work when it did not. Arena puts most Claude models above 40 percent on that measure and the GPT-6 series and Grok 4.7 at 7 to 10 percent. Anthropic’s own system cards describe the same habit, and Arena’s write-up lists frontier models “claiming checks they never performed.”

“Clean up” is a dangerous instruction. Only about 2 percent of Claude Opus 5 sessions included an unauthorized action, but more than half of those, 53.5 percent, were the agent deleting the user’s files or earlier work while tidying up. Opus 5.5 cut that share to 20 percent. Different models overstep in different ways, which is why the same instruction can be harmless with one tool and costly with another.

Three habits that work with any AI agent

One job, one conversation. Since failure risk climbs with length, start a fresh session for each task instead of running the whole week through one thread. It costs nothing and it is the single biggest lever the index points to.

Ask for the proof, not the word. Instead of “is it done?”, ask “show me the row you changed,” “paste the confirmation number,” or “list the three emails you sent and to whom.” An agent that cannot produce the evidence has told you what you need to know, in ten seconds instead of a week.

Never say “clean up” without a list. Name exactly what may be deleted or moved, and keep a copy of anything that matters before an agent touches the folder. If the tool offers approval steps, keep them on for deletions. We walked through how those approval levels work in Meta’s Muse for Small Business.

This index joins two other ways of reading model trust. Mistral’s office-work test showed agents finishing the work and still breaking a business rule, and Gemini 4 Argon showed the value of a model that says “I don’t know”. Arena’s measures the moment in between, when the agent reports back. Taken together, the lesson is consistent and encouraging: the work is increasingly reliable, and the habits that make it safe take seconds.

Frequently Asked Questions

What is the Arena Alignment Index?

It is a leaderboard Arena published on October 8, 2026 that scores 27 AI models on how they behave as agents in real sessions on its Agent Arena site. It measures three failures: acting without permission, misstating what the user said, and claiming a task is finished when it is not. Arena labels the current results preliminary.

Which AI model is least likely to claim a task is done when it isn’t?

On the leaderboard read October 8, 2026, OpenAI’s GPT-6.1 Sol had the lowest false completion rate at 2.34 percent of sessions, about 1 in 43, followed by GPT-6 Sol and GPT-6 Astra. Claude Opus 5.5 was at 6.41 percent and Gemini 4 Argon at 12.86 percent. Rates come from Arena’s own mix of tasks, which leans toward software work.

Why are the index scores so close together if the failure rates differ so much?

Arena converts each failure rate with a square root before scoring and gives false completions only 25 percent of the weight, with unauthorized actions at 50 percent. That compresses large differences in raw rates into a few points, so a model can make a mistake nearly three times as often and trail by under five points.

How can a small business reduce AI agent mistakes?

Keep each task in its own short session, since Arena found failures roughly double as conversations double in length. Ask the agent to show evidence of finished work, such as the changed record or a confirmation number, rather than accepting “done.” And never ask an agent to clean up without naming exactly what it may delete.

When an AI tool tells you a job is finished, what do you check before you believe it? Tell us in the comments.