How Handy 990 estimates your TOEIC score

The problem

Every time you answer a practice question, you tell the app one bit of information: right or wrong. From a long stream of these ones and zeros, spread across the seven parts of the TOEIC, we want to estimate a single number — your likely score out of 990 — and we want that number to be trustworthy, stable, and honest about its own uncertainty.

That is harder than it sounds, for four reasons:

1. Answers are noisy. Even at a fixed skill level you'll get streaks of right and wrong purely by chance. A good estimate shouldn't lurch every time you slip.
2. Your ability changes. You're studying. Evidence from three weeks ago is worth less than evidence from today. An estimate that weights everything equally can't track your improvement.
3. Cold start. After three questions we still have to show something, without pretending three questions tell us as much as three hundred.
4. We should know what we don't know. "Reading 320" and "Reading 320, but we're really not sure" are very different statements, and the app should be able to tell them apart.

This article explains the method we use — a decayed Beta–Bernoulli posterior — how it addresses all four problems at once, and why we chose it over the more obvious alternatives.

---

First attempts, and why they fall short

The plain running average

The simplest idea: for each part, track (questions correct) / (questions answered).

This is unbiased and dead simple, but it has no memory decay. After 500 questions, one answer barely moves it — which is fine if your skill never changes, and useless if it does. A learner who was 50% three weeks ago and is 75% today would see a running average stuck somewhere in the middle for a very long time. It also treats a question you saw yesterday exactly like a full mock exam. We need something that forgets the distant past at a controlled rate.

The fixed exponential moving average (what we used before)

The classic fix is an exponential moving average (EMA). Keep one number, the estimate m, and after each answer nudge it toward the latest result:

Each new answer counts for 10%, and the whole past counts for 90%. This does forget the past — its effective memory is roughly the last 1/α ≈ 10–20 answers — and it's what Handy 990 used originally. But it has three real defects:

• The learning rate never changes. α = 0.1 on your 3rd answer and on your 3,000th. Early on that's too slow (we're throwing away 90% of the little data we have); later it's the source of permanent noise.
• It never converges — it jitters forever. Because every answer keeps moving it by a fixed fraction, the estimate keeps twitching even when your true ability is perfectly flat. For a part where your true accuracy is 50%, a fixed-α EMA has a long-run standard deviation of about

Push that through the parts of a section and it's tens of points of wobble on the 990 scale that reflect luck, not learning. - It carries no sense of confidence. m is a bare number. It cannot tell you whether it rests on 3 answers or 3,000, so it can't show an honest range and can't avoid over-reacting when data is thin.

We want the good part of the EMA (bounded memory) without these three costs. The way to get there is to stop tracking a single number and start tracking a distribution.

---

The Bayesian idea: track a belief, not a point

Think of your accuracy on a given part as an unknown probability p — the chance you get a random question of that type right. Each answer is a Bernoulli trial: a coin flip that lands "correct" with probability p. We don't know p; we want to learn it from the flips.

The Bayesian move is: instead of committing to one guess for p, hold a whole probability distribution over what p might be — our belief — and update that belief as evidence arrives. When you've seen little data the belief is broad (we're unsure); as data accumulates it concentrates around the true value (we're confident). That breadth is the confidence information the EMA was missing.

The Beta distribution

For a probability p that lives between 0 and 1, the natural shape for this belief is the Beta distribution, written Beta(a, b). You can picture its two parameters as pseudo-counts:

• a behaves like a running count of correct answers,
• b like a running count of wrong answers.

The Beta distribution has two properties that make it perfect here:

1. Its average is exactly what you'd expect:

Just "corrects over total," but with pseudo-counts instead of raw tallies.

2. Updating it is trivial. The Beta is the conjugate prior for Bernoulli trials — a piece of mathematical good luck that means the update rule stays inside the same family. Observe a correct answer, and your new belief is simply Beta(a+1, b); a wrong answer gives Beta(a, b+1). No integrals, no approximation: you just add 1 to a counter. This is why the whole method costs a handful of arithmetic per answer and runs entirely on your device.

Where we start (the prior)

Before you've answered anything we begin at Beta(3, 3). Its mean is 3/6 = 50% — a neutral starting guess — and its total, a + b = 6, is deliberately small, encoding "we think 50%, but weakly, so real data will move us quickly." Answer a few questions and the pseudo-counts you contribute swamp the prior; the 50% assumption fades on its own.

Confidence comes for free

Because we carry a whole distribution, we get its variance without any extra bookkeeping. For a Beta distribution,

Look at what this does: the numerator m(1−m) is the usual coin-flip variance, and the denominator grows with N, the total evidence. More answers ⇒ smaller variance ⇒ a tighter estimate. The standard deviation of this (its square root) is what the app turns into the "±" range you see next to your score. It shrinks as you practice, which is both honest and, we hope, motivating.

---

The "decayed" part: forgetting at a controlled rate

Pure Bayesian updating has the running-average problem all over again: it never forgets. After 500 answers, N = 506 and a new answer barely registers — no good for a learner who is improving.

The fix is one line. Before folding in each new answer, gently shrink the existing pseudo-counts by a decay factor ρ slightly below 1. On each answer (x = 1 if correct, 0 if wrong):

Shrinking a and b a little each step stops the total evidence from growing without bound. It settles at a fixed ceiling:

So no matter how many thousands of questions you answer, each part behaves as if it remembers about the last 40, smoothly weighted toward the most recent. That's the "bounded memory" we liked about the EMA — but now with two bonuses.

Bonus 1: the learning rate adapts itself

Here's the elegant part. Work out how much the mean moves on a single answer. Writing N = a + b for the evidence before the answer and N′ = ρN + 1 for the evidence just after, a couple of lines of algebra collapse to:

This is exactly an EMA — but its step size is 1/N′, which is not fixed. When N is small (you've just started), N′ is small too — around 7 on your very first answer, so the step is about 1/7 ≈ 0.15 and the estimate moves fast, using your scarce data eagerly. As N climbs toward its ceiling, N′ settles at 40, the step settles to 1/40 = 0.025, and the estimate becomes calm and stable. In other words:

The decayed Beta posterior is an exponential moving average — but one whose learning rate starts high and anneals to a low value on its own, and that carries an evidence counter so it always knows how confident to be.

A fixed EMA is the special case where you throw that adaptivity away.

Bonus 2: half the noise, and it converges

Because the settled learning rate (≈ 0.025) is far smaller than the old fixed α = 0.1, the long-run jitter drops by roughly half in standard deviation — about a quarter of the variance. The estimate stops twitching on luck, so the trend chart and the "points gained this session" figures reflect real change instead of noise. And yet, thanks to the decay, it never freezes: genuine improvement still comes through, just without the false precision.

---

Two more kinds of "not all evidence is equal"

The same machinery absorbs a few refinements that a bare EMA simply can't express, because it has nowhere to put them.

Aging over calendar time

Bounded memory counts answers, but a two-month layoff should also cost you confidence even though you answered nothing in between. So once a part has sat idle past a one-week grace period, we shrink its pseudo-counts a little for each idle day (a factor of 0.99 per day). Crucially, we shrink a and b by the same factor — which leaves the mean a/(a+b) unchanged but lowers the total N. The effect: your displayed score doesn't move, but its ± range widens, and your first answers back carry more weight. "We haven't forgotten where you were, but we're less sure it still holds" — expressed in one multiplication.

Weighting individual answers

Not every answer is equally informative, so we let an answer fold in with a weight w (the update becomes a ← ρa + w·x, b ← ρb + w·(1−x)):

• A question you've seen before counts at half weight. Getting a repeat right leans partly on memory rather than ability, so it shouldn't move the estimate as much as a fresh item. This also cancels a subtle bias: once you've exhausted a category, practice starts re-serving the questions you previously missed, which would otherwise drag your estimate down unfairly.
• The source matters. An answer under real exam conditions (a full timed mock) is worth more than a casual drill, so it folds in at a higher weight (up to double). With the memory ceiling of ~40, a completed mock then genuinely dominates a part's recent evidence instead of washing out by the next day.
• A reported real score (if you enter one) seeds the belief as a strong prior — roughly 25 answers' worth of confidence at the implied accuracy — so a single later practice question can't erase it, but a few dozen eventually will.

None of these needed new architecture. They're all just the weight w and the two counters. That flexibility is a direct consequence of choosing a model with an explicit notion of "how much evidence," rather than a single opaque number.

---

From per-part beliefs to a 990 score

Each of the ten question categories carries its own decayed Beta posterior. To turn them into a section score, we weight each category's mean by how many questions of that type appear on a real exam (the counts sum to 100 per section), which gives an estimated number-correct out of 100. That number is mapped through the official TOEIC number-correct → scaled-score table to produce a 5–495 section score, and the two sections add up to your /990 estimate.

The uncertainty travels with it. Because the categories are independent, their variances add in quadrature (weighted by the square of each exam count), and the combined standard error becomes the ± band on your section and total scores.

A reading-specific reality check

TOEIC Reading is speeded: 100 questions in 75 minutes — an average of 45 seconds each, with no way to buy more time. Accuracy measured in untimed practice therefore flatters a slow reader, because on the real clock they never reach the last questions, and an unreached question scores like a random guess — 25%, one chance in four — not like your practiced accuracy.

So the app runs a simple simulation. Using the per-question pace it has measured for each reading part, it "spends" your 75-minute budget in exam order — Part 5, then 6, then 7 — until the time runs out, and scores every question you wouldn't reach at 0.25 instead of your real accuracy.

A concrete example. Suppose you read at 30 s/question on Part 5, 45 s on Part 6, and 80 s on Part 7. Parts 5 and 6 (30 + 16 questions) use up 30×30 + 16×45 = 1,620 seconds, leaving 2,880 for Part 7. At 80 s each you finish only 2,880 / 80 ≈ 36 of Part 7's 54 questions — the other 18 go unanswered. If your Part 7 accuracy is 70%, those 18 questions each fall from 0.70 to 0.25, costing about 0.45 × 18 ≈ 8 raw points — very roughly 30–40 points off your scaled reading score. Read faster and more of those questions get reached at your real accuracy, so the penalty shrinks; at a comfortable pace it vanishes entirely.

When your speed would cost points like this, the app shows it as an explicit "at exam pace" figure — a reminder that reading is about speed, not just correctness. (Listening is machine-paced, so it needs no such adjustment.)

---

Why this beats the alternatives

We considered the usual candidates. Here's the honest comparison.

Approach	What it gives	Why we didn't stop there
Plain running average	Unbiased, trivial	Never forgets; can't track improvement; treats all answers alike
Fixed-α EMA	Bounded memory, cheap	Constant learning rate; permanent jitter; no uncertainty; nowhere to add weights
Decayed Beta posterior (ours)	Adaptive rate, bounded memory, built-in confidence, natural weighting	—
Item-Response Theory (IRT)	The "gold standard" ability model	Needs each question's difficulty calibrated from thousands of test-takers — data we don't have on-device
Elo / Glicko ratings	Elegant, adaptive	Also needs per-item "opponent" difficulties we can't trust; its uncertainty bookkeeping duplicates what our decay already gives
Kalman filter	Tracks a changing quantity with uncertainty	For yes/no data this reduces to essentially our model, but with a clumsier likelihood; the Beta form is the natural fit

The decisive point is this. The two "more sophisticated" models — IRT and Elo — put all their power into per-question difficulty parameters, and estimating those reliably requires a large pool of test-takers answering the same items. Handy 990 runs privately on your device, with no such fleet-wide statistics. Feed those models the uncalibrated difficulty labels we do have and they'd be sophisticated machinery resting on shaky inputs.

Meanwhile, every problem a user actually feels — the jitter, the forgetting, the missing confidence range, the equal treatment of a mock and a warm-up — lives in the update rule, not in item difficulties. The decayed Beta posterior is the smallest, most principled model that fixes all of those at once:

• It is genuinely Bayesian, so the estimate and its uncertainty come from the same clean rule rather than being bolted on.
• It is an adaptive-gain EMA, so it starts fast, settles calm, and never freezes.
• It is conjugate, so each update is a few additions and multiplications — fast, exact, and offline.
• It has an explicit evidence count, so weighting by recency, repetition, and source all drop out naturally.

It is, in short, the right amount of statistics for the data we honestly have.

---

Honest caveats

• It's a study aid, not an official score. Real TOEIC uses IRT scaling on secured items; we can only ever approximate it from your practice.
• The constants are reasoned, not fitted. The memory ceiling (~40), the prior strength, the decay rates, and the answer weights are sensible defaults, not values tuned against a large calibration set. When you enter a real reported score, we log the gap between our estimate and that ground truth (privately) precisely so these numbers can be refined over time.
• Thin data is thin data. Early on, most parts still sit at the 50% prior, so the early estimate is rough — which is exactly why the app shows a wide ± range until you give it more to work with.

We haven't captured everything — and we know it. The factors above are the ones we could model from what your practice tells us: recency, repetition, exam-condition weighting, reading speed. But a real exam also turns on things we simply cannot see from here — test-day nerves and fatigue, the particular form you're handed, unfamiliar wording, how our question bank's difficulty lines up with the real one, and plain luck on your guesses that day, among others. We've weighed many factors, but certainly not all of them, so the estimate will never be exact and isn't meant to be. If you sit a real TOEIC and the result surprises you — above or below what we showed — we would genuinely like to hear about it through the app's feedback option. Telling us what you think moved your score helps us spot the factors we've missed and sharpen the estimate for everyone.

The honest version of the promise is: answer more questions, across more parts, under more realistic conditions, and the estimate will both move toward the truth and tell you it has grown more sure. That is the behavior the decayed Bayesian estimator is built to deliver.

---

The sequel is here: when your real score report arrives, how does this estimator put it to use — and what can the ABILITIES MEASURED panel most people skip actually do? Read Your TOEIC Score Report Is More Than Two Numbers.