Now, it's time to START to assess what went down last weekend. This will certainly not be the last post on the Games, and in fact, I still have several things I'd like to look into for the 2012 season as a whole. But for starters, let's see how our predictions panned out.
Notes: Just like I did in my 2011 analysis, I assigned points to each athlete who was cut for those events that they missed. The method of estimating those points is explained a couple of posts back.
Men: I honestly expected to do a little better here than I did. The model I used had an R-squared of 66% on the training data set (the 2011 Games), and while I obviously would not expect to do that well again, I expected to do a pretty decent job picking this year's Games. In the end, the model had an R-squared of 49% (that was calculated based on the actual points for each athlete, not just the rank). We did pick the winner correctly - Rich Froning won convincingly, as was expected. Matt Chan, on the other hand, had a performance that I simply did not see coming. His regional performance was solid (7th), but he was only so-so in the Open (21st), and at 33 years old, I didn't know whether he could handle the volume required these days in the Games. The model had him picked 23rd, and he ended up outperforming the model (in terms of points) more than any other athlete. As far as placement, Scott Panchik had the biggest move, finishing fourth despite being projected 27th by the model.
Despite the performances of newcomers Panchik and Marcus Hendren (7th), Games experience still proved to be a factor. Here is a comparison showing the average ranks of previous competitors vs. newcomers (same comparison we did for 2011 a couple of posts back):
Except at the top end, prior Games competitors outperformed newcomers with similar regional rankings. Something we may want to consider is modeling the top athletes slightly differently than the ones finishing more modestly at regionals.
What was really NOT much of a predictor this year was Open results. After taking into account the regional results, the Open results basically told us nothing more (in fact it had a slightly negative coefficient in the regression). The results did show that age was still a factor (younger athletes are expected to do better), but not to the extent we expected. Each year over 26 was worth somewhere around 7 points, after accounting for prior experience, Regional and Open results - the model had assumed more like 20 (after scaling up to the 1,350 available points this year from 1,000 last year).
Overall, it looks like we would have been better off simply using the Regional rank. The R-squared there would have been was about 55% using my adjusted regional rankings and 53% using the raw regional rankings. With another year of data under our belt, hopefully we can do better next year.
Women: The women's model was much simpler than the men's, and it actually slightly outperformed the men's as well. The R-squared (using points) was 50%, and of the top 10 women, I had 6 predicted to be in the top 10. If we had simply used the adjusted regional ranks and converted them to Games-style points, then used that to predict the Games points, the R-squared would have been only 46%. So taking the Open results into account certainly helped out.
At the top, as expected, it was a dual between Julie Foucher and Annie Thorisdottir. While we expected some other top names, like Kristan Clever (predicted 3rd, finished 4th) and Camille Leblanc-Bazinet (predicted 4th, finished 6th), Talayna Fortunato was a surprise. She was 11th in the adjusted regional rankings and 8th in the Open, but a 3rd place finish was unexpected (predicted 10th). Overall, Jenny Davis outperformed the model more than anyone else, finishing 8th despite being picked 26th.
For the women, we did see a bit more correlation between prior Games experience and improved results this year (last year we saw basically none). Here is the same chart as above, except for women:
What stuck out, however, is that the Open results were a pretty darn good predictor of success for the women. After accounting for Regional results, the regression actually showed that the Open was a slightly stronger predictor than the Regionals. If you had used the Open results alone to predict the Games, the R-squared would have been 47%, slightly higher than using the Regional results alone. And again, like 2011, age did not appear to be a factor at all once you take into account regional peformance (see 40+ year-olds Becky Conzelman and Cheryl Brost finishing 14th and 15th, respectively). Obviously, in general, it helps to be younger, but if a woman has already qualified for the Games, there is no reason to believe their age will negatively impact them any more in the Games than it did at Regionals or in the Open.
I'll continue to dig into this in the coming weeks. Hopefully everyone enjoyed the Games. Only 7 months until the 2013 Open begins!
No comments:
Post a Comment