"70% chance the stock rises this week." Was that a good forecast? If the stock rose, it looks right. If it fell, it looks wrong. But a forecast of 70% isn't a promise; it says the stock should fall about three times out of ten. To judge forecasts like that fairly you need a score that grades the chance itself. The most common one is the Brier score.
The formula, in one line
For each forecast, take the chance given (as a number from 0 to 1), subtract what happened (1 if it happened, 0 if not), and square it. Then average over all forecasts.
Brier score = average of (chance − outcome)²
- 0 is perfect: you said 100% for everything that happened and 0% for everything that didn't.
- 0.25 is what you get by saying 50% every time.
- Lower is better. The worst possible score is 1.
Worked examples
A forecaster says 70% up:
- The stock rises: (0.7 − 1)² = 0.09. Good.
- The stock falls: (0.7 − 0)² = 0.49. Bad.
Another says 55% up:
- It rises: (0.55 − 1)² = 0.20.
- It falls: (0.55 − 0)² = 0.30.
The 55% forecaster is punished less for being wrong, and rewarded less for being right. That's the point: the Brier score rewards being confident when you're right and punishes being confident when you're wrong.
Why "hit rate" misleads
Hit rate (the share of calls that went the right way) ignores how confident each call was. A forecaster who says 51% on everything and one who says 90% on everything can have the same hit rate, while one of them is badly overconfident. Hit rate also flatters anyone in a rising market: say "up" every week in a year the market climbs and you'll look skilled.
Compare against a baseline
A score only means something next to what you could have done without the forecaster:
- Always 50% scores 0.25. Any forecaster worth listening to has to beat it.
- The usual odds. Stocks rise a bit more often than they fall: over the past decades, the market has risen in roughly 54% of weeks and about 64% of years. Saying "54% up" for every stock every week is a tougher baseline than 50%, and surprisingly hard to beat.
- The stock's own history. How often this particular stock rose over that period in the past.
If a forecaster can't beat the usual odds, it's adding nothing you couldn't get from a single average.
Two more things to check
- Calibration. Of all the times the forecaster said about 60%, did about 60% actually happen? A well-calibrated forecaster's 60% means 60%.
- Sample size and time. Forecasts made in the same week rise and fall together, so a thousand calls in one week is closer to one piece of evidence than a thousand. Wait for many separate weeks before judging.
How Ticker Council uses it
Every Ticker Council call gives a chance of the price rising over the next week, 3 months and a year. Each call is saved unedited, then scored with the Brier score against what happened, and compared with the baselines above, counting each stock once a day and each market week as one piece of evidence. The details are in How Ticker Council scores its calls.
Not financial advice: this guide explains how forecasts are graded, not what to buy or sell.
Questions
What is a good Brier score?
Lower is better, from 0 (perfect) to 1 (always confidently wrong). Saying 50% every time scores 0.25, so a useful forecaster has to score below that, and ideally below a baseline that uses the usual odds for that kind of event.
What is the difference between Brier score and accuracy?
Accuracy (hit rate) only counts whether each call went the right way. The Brier score also grades how confident each call was, so overconfident wrong calls cost more than cautious ones.
How is the Brier score calculated?
For each forecast, subtract the outcome (1 if it happened, 0 if not) from the probability given, square the result, and average over all forecasts.