From speech to band score: grading IELTS speaking with an LLM

IELTS speaking is a subjective exam: a human examiner listens and assigns a band on four official criteria. EasyIELTS (1,400+ users) does the same job with an LLM — and the interesting part is everything you have to pin down to make a fuzzy human judgement reproducible.

The pipeline

A practice session is a real-time conversation. The user speaks; speech-to-text (DashScope) turns the audio into a transcript; an AI examiner asks the next question — Part 1 short Q&A, Part 2 a ~2-minute cue-card monologue, Part 3 deeper discussion. The conversation has to feel live, so scoring runs in the background: the examiner moves straight on while each turn is graded asynchronously and the results catch up.

The rubric as a contract

The judgement itself is a single strict-JSON contract. The scoring prompt is aligned to the official IELTS criteria and forces a machine-readable shape:

{
  "fluency_coherence": 0-9, "lexical_resource": 0-9,
  "grammatical_range": 0-9, "pronunciation": 0-9,
  "band": 0-9,
  "feedback": { "comment", "strengths": [], "improvements": [], "vocabulary_suggestions": [] }
}

Each dimension is scored 0–9 in 0.5 increments, the way a real examiner would, and the band is their average rounded to the nearest half. Crucially, the rubric is part-aware — the prompt encodes what each part actually demands:

Part 2 cue-card: the student should speak for ~2 minutes (180–260 words) and cover the card’s points; clearly under a minute, or missing points → fluency below 6.

That part-specific framing is the difference between “an LLM that rates English” and “a grader aligned to the real exam.”

Be honest about what the model can’t hear

Here’s the detail I’m most pleased with. The model scores from a transcript — it never hears the audio. So pronunciation can only be inferred indirectly. Rather than pretend otherwise, the prompt says so, out loud, and is told to pass that caveat through to the learner:

pronunciation is inferred from lexical complexity, sentence completeness and fluency; note in the feedback that this is based on the transcript and that real pronunciation needs human re-check.

A scoring system that overstates its confidence is worse than useless to a learner. Encoding the limitation into the output is what keeps the product trustworthy.

Guardrails that keep scores sane

An LLM’s output is a contract you have to enforce. Bands are clamped to the valid range and snapped to half-points:

function clampBand(value: unknown): number {
  const num = Number(value);
  if (!Number.isFinite(num)) return 0;
  return Math.round(Math.max(0, Math.min(9, num)) * 2) / 2; // 0–9, 0.5 steps
}

And inputs that can’t be graded are rejected — a two-word answer isn’t a band 3, it’s not gradeable:

if (wordCount < 5) {
  return {
    score: null,
    feedback: { comment: "This turn is too short — say a few more words so the AI can score it.", ... },
    skippedReason: "insufficient_audio",
  };
}

Cheap and resilient

Scoring fires on every turn for every user, so cost and reliability both matter. The model client uses DeepSeek as the primary (strong and inexpensive for structured output) with OpenRouter as a fallback when the primary errors:

const deepseek = process.env.DEEPSEEK_API_KEY
  ? new OpenAI({ baseURL: "https://api.deepseek.com", apiKey: process.env.DEEPSEEK_API_KEY })
  : null;
const openrouter = process.env.OPENROUTER_API_KEY
  ? new OpenAI({ baseURL: "https://openrouter.ai/api/v1", apiKey: process.env.OPENROUTER_API_KEY })
  : null;

The same pattern grades writing against its four criteria (Task Response, Coherence & Cohesion, Lexical Resource, Grammar) and backs a chat tutor.

Takeaways

• Turn the rubric into a schema. A subjective judgement becomes reproducible the moment you force it into a typed, per-criterion JSON contract.
• Encode the domain, not just the task. “Score this English” and “score this as IELTS Part 2” are different prompts; the exam’s expectations belong inside the prompt.
• Model your blind spots. When the LLM genuinely can’t judge something — pronunciation from text — say so in the output. Honesty is a feature, not a disclaimer.
• Score off the critical path. Background, asynchronous scoring keeps the conversation fast while grading catches up.

EasyIELTS is live, with 1,400+ learners.