World Cup Forecasts

Assessing after first round of games...

I posted my two sets of forecasts last week - one based on Elo ratings and the other on Transfermarkt (TM) values. After one round of games they are both performing well, more or less matching the bookmaker odds (which I took from the Oddsportal website).

Out of 24 games played, the bookmakers gave the highest probability to the actual result in 13 out of 24 cases (54%). That’s similar to the success rate I find for bookmaker odds when predicting English Premier League games.

The TM model also got 13 out of 24 right, while Elo got only 12 right. I also calculated something similar to a Brier score. There are 3 possible results for each group stage result (team 1 wins, draw, team 2 wins). Assign a value of 1 to the actual outcome and 0 to each of the other two possible outcomes, then sum the squared difference between each probability and the 3 outcome values, divide by 9 and stake the square root.

This is essentially the average difference between the prediction and the result. If the forecast gave 100% probability to the actual oucome, the difference would be zero, if the model attributed a 100% probability to a different outcome the difference would be 1 - so lower numbers mean better forecasts.

For the bookmakers, the average difference is 0.2413 - the average odds were 24.13% away from the result. This is something of a benchmark. If you think you can systematically outperform the bookmakers by a significant margin then I take my hat off to you (or I would if I had one).

To the TM model the average difference is 24.47% - one third of one percent off the bookmakers. This seems pretty creditable. Bear in mind that the outcome of each game is forecast using only two numbers - the TM value for each team - that’s pretty basic.

The Elo model is doing a little worse - so far the average difference is 26.2%, that’s about one and three quarters percent off the bookmakers. Worse, but not so as it would lose you much money (it’s always the bookmaker margin and taxes that make gambling unprofitable for the intelligent gambler).

Just to note, I entered my forecasts in the competition being run by Futbolmetrix . I doubt my models will win, but I’m interested to see how they do against more sophisticated models - my guess is that they will be close - sophistication doesn’t add much. But we’ll see.

Let’s see how things go with the second round of games.