Don’t Overreact To Small Samples
Last Friday, Jeffrey Paternostro of Baseball Prospectus released an article discussing why they aren’t releasing updated prospect rankings for the month of May. It’s worth a read and touches on something that’s been bugging me lately, most notably in the fantasy and prospect community.
We’ve decided that it is in our best interest to oversaturate the reader with content, even if that content isn’t meaningful. For instance, I find monthly prospect rankings and pieces earnestly dissecting player performance, especially statistical performance early in the year, to be reading too much into small sample sizes. There is too much fluctuation and noise within a month’s worth of stats to draw meaningful conclusions for just about anything. Even something that a lot of people view as static, like fastball velocity, can fluctuate from month to month.
I’m not a member of the prospect community so I can’t speak for them, but I can speak toward this issue in our fantasy community. Chris Towers, senior editor and fantasy writer at CBS Sports, tweeted this out on April 16th:
As Russell Carleton notes, exit velocity becomes reliable very quickly, around 40 batted balls. What’s more important is that Russell makes the clear distinction that “[t]he idea that a number has become ‘reliable’ is not the same thing as saying that the player is now that number and that going forward this is what we should expect out of him.”
The same thing goes for statistics that “stabilize” (in actuality, a stat never completely stabilizes, it just continues to stabilize). These tests aren’t meant to predict future performance, but to evaluate past performance. If we’re going to continually base our analysis off of 60 PA samples, we’re not telling you anything that you don’t already know: that Player X is struggling, or that Player Y is hot. Russell Carleton broke it down better than I ever could in April of last year.
Players adjust. Players change. Players adjust to other players after they adjust. Just because a stat is stabilized doesn’t mean that it will hold that way for the remainder of the season and to either 1) posit that it will (as an analyst), or 2) believe that it will (as a fantasy player) is, in my estimation, a mistake. It’s natural to want answers NOW, but if you think you’ve found something meaningful in April, it’s not time to signal the lanterns over what very well could be a hot streak, a cold streak, or random variation. Make a note of it, watch for changes the player may have taken to develop or enhance this skill, look for further evidence from player and coach quotes.
Sometimes, it’s possible to infer a change in skill. Did a middle infielder who never actualized his raw power suddenly start hitting 440-foot moonshots to his pull field because he radically changed his swing? Every instance linked above had evidence provided by player quotes, coaches quotes, and could be identified on video. While you can’t immediately proclaim that he’s a 30-homerun-caliber hitter in June, he gave us reason to believe that a power surge could be real.
Sadly, not every skill is as easily identifiable as in-game power. None of us can tell you with confidence that Jed Lowrie is now a .300 hitter and us telling you he is would be disingenuous. This is no different, in my mind, then dropping a prospect’s position on a list solely because they had a bad month.
If you’re curious about a player, it helps to watch him play. You may not be able to answer a question such as “is Jed Lowrie capable of keeping a .300 average for the entire season,” but you might be able to confirm that his recent hot streak is valid. Seek out knowledgeable fans, analysts, or writers who teach, scout, or study hitting mechanics to get them to explain the nuances of a player’s swing.
It’s okay to wait and see and it’s okay for things to be vague. It’s necessary to wait to confirm new findings and the reader is better off for it.