Today I was planning on posting my monthly meta analysis, just as I have planned to do all week. Things just keep coming up, though, and this is the latest. As it turns out, this post is also the latest in a series of posts about the concept of Win Probability and its treatment in the mainstream media. I’m getting tired of re-linking everything each time I post about this, so I’ve tagged all of these posts with the sub-categories “Win Probability” and “WPA & The Media.” You can visit those archives here if you so desire.
So, you may recall that ESPN introduced a win probability graphic during the College World Series, and I decided to track a game. Today, Dave Studeman (also credited as Dave Studenmund and Studes elsewhere, and the creator of the handy WPA tracking spreadsheet that I occasionally use), wrote an article for The Hardball Times after he noticed ESPN using their win probability stats during last night’s MLB broadcast. This is the first MLB use of which I’m aware, but I don’t watch that much baseball on ESPN, and I didn’t watch last night.
As it happens, the Worldwide Leader was doing the Braves-Mets game last night when they brought out their stat. As Studeman describes, Jeff Francoeur was up with the bases loaded and two out in the eighth, with the Braves down 3-1. This situation (visiting team -2 with that inning/base/out situation), as Studeman also mentioned, has only happened 58 times in the last 7 years, which comparatively is not a large sample size. The ESPN graphic showed that the Mets had a win probability of 88%, which as it turns out was about 7% too high. Check the play log here if you like.
Studeman was right in thinking that 88% seemed too high for the situation, and the difference in their numbers appears to be the result of two prominent methods of calculating win probability: one entirely based on play-by-play data, and another influenced by mathematical probability. The two usually yield very similar results, especially in common inning/base/out situations, but in the case of rarer situations, it’s best to use the mathematically-derived tables (mostly the result of Tom Tango‘s work). The play-by-play data for these situations does not yield a statistically significant probability, so it makes sense to defer to the theoretical values there. It appears that ESPN is not using this method, which is unfortunate given their tremendous influence and the relative unknown status of WPA as a mainstream number.
To put it more simply, it appears that ESPN is using some set of play-by-play data that, as of yet, remains undisclosed and unproven. Like Football Outsiders and the DVOA/DPAR method I mentioned yesterday, ESPN has not shown us specifically how they arrive at their numbers, giving us little reason to trust them. However, unlike FO, there are already better alternatives available for baseball win probability.
If you really want to know the correct win probability during a game, the best thing to do is download the spreadsheet and track it yourself. Now that ESPN has made it clear that they’re not even using the best method available (and instead just looking for something gimmicky to use in a late-season broadcast), there’s really no reason to give their numbers any credence.
Lastly, a blast from the past:
Tim Keown wrote a Page 2 article back in April 2005 suggesting the use of win probability in baseball broadcasts. It seems to be somewhat tongue-in-cheek, and it doesn’t go into anything specific, but it’s a suggestion nonetheless. I doubt this was the origin of their idea, but it did come up as one of the top Google results for ESPN and win probability.
jfwiii.net 