Evaluation
Log loss: scoring a forecast with a logarithm
Log loss is the average negative natural logarithm of the probability a forecast gave to the outcome that happened. Lower is better: a correct 100% forecast scores 0, a constant 50% on a two-way question scores 0.693 and an even 1X2 forecast 1.099. Unlike the Brier score, log loss grows without limit when a confident forecast is wrong.
Contents
Key terms
- Log loss
- Logarithmic loss, also called cross-entropy or the logarithmic score. It measures how surprised a forecast was by the outcome.
- Probability of the outcome
- The probability the forecast gave to what actually happened. If the forecast was p for “Yes” and the answer was “No”, it is 1 − p.
- Natural logarithm
- ln(1) = 0, and ln(p) gets more negative as p approaches 0. So −ln(p) grows the further the forecast was from the outcome.
Formula
LL = −(1 / N) × Σ ln(p of the observed outcome)Example with four forecasts
The same four forecasts as in the Brier score guide:
| Forecast | Outcome | p of outcome | −ln(p) |
|---|---|---|---|
| 0.70 | 1 | 0.70 | 0.3567 |
| 0.20 | 0 | 0.80 | 0.2231 |
| 0.55 | 0 | 0.45 | 0.7985 |
| 0.90 | 1 | 0.90 | 0.1054 |
| Total | 1.4837 |
1.4837 / 4 = 0.3709 → 0.371
A constant 50% forecast on the same four questions scores ln 2 = 0.693.
Log loss for 1X2
Forecast 1 / X / 2 = 50% / 30% / 20%.
| Match result | Calculation | Log loss |
|---|---|---|
| 1 | −ln 0.5 | 0.693 |
| X | −ln 0.3 | 1.204 |
| 2 | −ln 0.2 | 1.609 |
An even forecast of 33.3% for each outcome scores ln 3 = 1.099 whatever happens. A model that averages worse than that knows less than a random guess.
Log loss or Brier score?
| p of the observed outcome | Log loss | Brier |
|---|---|---|
| 0.90 | 0.1054 | 0.0100 |
| 0.70 | 0.3567 | 0.0900 |
| 0.50 | 0.6931 | 0.2500 |
| 0.30 | 1.2040 | 0.4900 |
| 0.10 | 2.3026 | 0.8100 |
| 0.01 | 4.6052 | 0.9801 |
The Brier score has an upper limit, log loss does not: a miss at 99% confidence costs 4.6, at 99.9% already 6.9. One such error can spoil the average of hundreds of good forecasts, so probabilities are often floored before scoring, for example at 1%.
Both scores reward an honest forecast: on average the best score goes to whoever states the probability they actually believe. Such a score is called a proper scoring rule.
The AI model performance page computes log loss with probabilities floored at 1%. As of 31 October 2026, matches from 1 September to 31 October 2026:
| Market | Margin-free odds | AI model | Base rate |
|---|---|---|---|
| 1X2 | 0.990 | 1.047 | 1.068 |
| Over/under 2.5 | 0.668 | 0.690 | 0.689 |
On 1X2 the AI model beats the base rate and the even 1.099, but margin-free odds do better still. The figures change daily on the AI model performance page.
Limitations
- Log loss depends on the number of outcomes: 1X2 and two-way questions are not comparable, so only forecasts for the same market are compared.
- Without a floor, one 0% forecast for an outcome that happens gives an infinite log loss.
- On a small sample the gap between two forecasts may be chance.
- A lower log loss does not mean profit at a bookmaker. This is statistics, not betting advice.
Related guides
Sources
- Good, I. J. (1952). Rational decisions. Journal of the Royal Statistical Society, Series B, 14(1), 107–114.