AI Tutor vs Human Tutor: 2,383 Students Call It a Draw

The short version: In the largest head-to-head test of an AI tutor vs a human tutor published so far, 2,383 adults were randomly assigned to one hour with an AI tutor, one hour with an expert human tutor on a video call, or an hour of unrelated videos, then retested on GRE math and verbal questions. The researchers found the AI and human tutors statistically equivalent. The cheapest AI tutor in the study, Google’s open-weight Gemma 4 31B, matched the humans for 6.7 cents of computing per session. The human reference rate was $75 an hour.

The useful part for a small business is not the price. It is one detail of how the study was set up: every tutor, human or AI, started the hour with the student’s own mistakes. That is a routine any owner can copy this week to bring a new hire up to speed, and the rest of this piece walks through it.

What did the StudentBench study actually find?

The paper, StudentBench, comes from Handshake AI Research and was revised on September 30. Participants were students recruited through the Handshake careers platform and paid $50 for a session. Each took a timed GRE-style pretest, got an hour of help, and then took a different but matched test. Human tutors were former ETS or Kaplan question writers or people with at least five years of GRE tutoring behind them.

Thirteen AI tutor setups were tested, built on models from Google, Anthropic and OpenAI. Pooled together, the AI tutors produced gains within a hair of the human tutors: on average 0.58 percentage points lower, a gap well inside the margin the researchers set for calling two methods equivalent. Against the group that watched unrelated videos, AI tutoring added 6.86 points on the math section and 5.47 on verbal.

The split by subject matters. On math, the best AI tutors beat the humans in three of four areas. On reading and vocabulary, the humans stayed above the average AI tutor in all three areas, with only the best AI tutors running close. If your training is closer to numbers and procedures than to judgment and nuance, the result fits your case better.

Why does the cheap model keep up?

Two findings explain it, and neither is about raw intelligence.

First, the researchers deliberately used minimal prompting so they could measure the models rather than their own engineering. There was no elaborate tutoring software. Each AI tutor got one prompt to plan a lesson and write practice problems, and one to run the conversation.

Second, speed mattered. On the math sessions, faster AI replies went with more messages from the student, more messages went with more correct practice, and more correct practice went with bigger gains. The paper reports each link as statistically significant. A quick, inexpensive model keeps a learner working problems; a slow, expensive one gives them time to drift. The paper notes that Gemini 3.5 Flash produced gains similar to GPT-5.5 Pro at one twentieth of the cost.

That is genuinely good news for an owner who has been assuming useful AI training needs the top paid tier. The models that did this work are the fast, inexpensive class, and that class now sits on free plans: Anthropic, for one, made Claude Sonnet 5.5 the default on its free plan last week.

What does an hour of AI tutoring cost a small business?

Here is the arithmetic the paper did not do, because it was measuring tutors, not payrolls. Say your new hire earns $22 an hour; put in your own number. An hour of expert human tutoring costs the tutor’s $75 plus that $22 of paid time, so $97, and the tutor is 77 percent of it. An hour with the Gemma tutor costs $22.07, and the tutor is under one percent of it.

So the expensive thing in the room is no longer the teacher. It is the hour of your employee’s attention. That changes the question from “can we afford a tutor?” to “are we spending this hour on the right things?” The study’s design answers that second question.

How do you run an AI tutoring hour for a new hire?

Copy the structure the researchers used, scaled down to a shop.

1. Test first, and keep it short. Write ten questions on one thing a new hire must get right: markup math on an estimate, the clearances you install to, the ten parts you stock and when each is used. An assistant can draft the questions; you check every one, because a wrong answer key teaches the wrong thing.

2. Hand the tutor the misses, not the whole manual. In the study, both tutors were given the student’s pretest mistakes to plan the lesson. Paste in the questions your hire got wrong, with their answers, and ask the assistant to plan a short lesson on those mistakes, then tutor one problem at a time and make the learner do the working.

3. Retest with questions the tutor never saw. The study’s tutors had no access to the post-test, and the two test forms were swapped between students. Keep five fresh questions back. If the score moves on questions the tutor could not have coached to, the learning is real.

4. Stay the teacher for the judgment calls. The AI drills the basics; you, or your most experienced person, teach the parts that live in your head: when a job is worth walking away from, how to talk a nervous customer through a price. That is where the human tutors in the study kept their edge, and it is where your time is now best spent. If you have never written those processes down, recording yourself doing the work is a fast way to start.

What did the study not test?

Three things, and the researchers say so themselves.

Retention. In the authors’ words, “we measured immediate learning gains without evaluating whether they persist over months.” An hour that lifts a same-day score may fade by the next job.

Practice on your own. The comparison group watched unrelated videos. Nobody in the study simply worked GRE problems alone for an hour, so we cannot tell how much of the gain came from tutoring and how much from practice. Since the paper ties the AI’s results to more correct practice, the honest reading is that the tutor’s job is mostly to keep the learner practicing on the right things.

Your trade. The test was the GRE, taken by paid student volunteers reading and writing in English, and Handshake AI funded the work. Nothing here was measured on an apprentice learning a refrigerant charge or a front-desk hire learning your booking rules. The method transfers easily; the result has to be earned again in your shop, which is what step three is for.

If you are taking your first steps with AI, a single training hour with a before-and-after score is about as clean a pilot as exists, and it fits the one-task-first approach in our 30-day starting plan. The tutor costs almost nothing. The hour you invest is what deserves the planning.

Frequently Asked Questions

Is an AI tutor as good as a human tutor?

For short-term learning on GRE-style questions, the StudentBench study found the two statistically equivalent across 2,383 participants, with the best AI tutors ahead on math and human tutors ahead of the average AI tutor on verbal reasoning. The study did not measure whether the gains lasted beyond the same session.

How much did the AI tutoring cost in the study?

The cheapest AI tutor that matched human results, Google’s open-weight Gemma 4 31B, cost an average of $0.067 in computing per session, or $0.0052 per percentage point gained, against $4.81 per point for human tutoring priced at a $75 hourly reference rate. That is the 918 times figure in the paper.

Can a small business use AI tutoring to train employees?

Yes, with a simple structure taken from the study: give a short quiz, hand the assistant only the questions the employee missed and ask it to tutor one problem at a time, then retest with questions the assistant never saw. The owner checks the answer key and keeps teaching the judgment calls.

Do you need a paid AI plan for tutoring?

Not necessarily. The study’s results came from fast, inexpensive models with minimal prompting, and faster replies were linked to more practice and larger gains on math. Models in that class are available on free plans from the major assistants, though usage limits on free tiers vary by provider.

What is the one thing every new hire in your business gets wrong in their first month? Tell us in the comments; it might be the best place to try a mistakes-first hour.