With the 2018 Consensus Big Board live, we can start drilling down on what having all these big boards available to us means for draft analysis.
We’ve explained in the past one of the core pieces of our analysis — the difference between Forecasters and Evaluators — and it’s worth rehashing once more.
There are two general approaches to draft coverage: 1) Who is going to pick who, and 2) Who is good. We’ve separated the draft boards into two categories: Forecasters (who do a job more closely resembling question #1) and Evaluators (who are closer to answering question #2 than question #1).
Generally speaking, the forecasters have been or are currently employed by media organizations that thrive on access, and that gives them access as well. Beyond that, people like Dane Brugler (formerly of CBS.com and now working with DraftScout.com) publish draft guides that are driven in big ways by the access they have.
Sometimes that access influences the actual talent evaluation, but often it will influence the final grade by speaking to the gravity of character concerns, injury concerns or some other errata.
Last year, unusual clusters of similar rankings at odd points in the charts confirmed (to me) the clear separation between those two groups of draft boards. This year, there are far fewer clusters in that data (it’s a more polarizing draft), but they do show up.
Every year, we’ve found that the variance in draft rankings among the forecaster boards has been significantly less than the variance in rankings from the evaluator boards, and that they tend to cluster their rankings in similar ways.
We can look at the variance in rankings in the top 50 players between the two boards to see that.
That’s very strong evidence that the forecaster boards work off of similar data and their access to NFL evaluators lets them hone in on what the NFL will do.
That information will include where NFL teams value players, injury data, off-the-field incidents and more.
When the two boards have disagreed, the Forecaster Board has consistently proven to be closer to the prospect’s eventual NFL draft position. Last year, the Forecaster Board projected 12 of the 20 players more accurately than the Evaluator Board. This pattern has held true through four years of analysis and it’s a reason why the Forecaster Board has not only beaten the Consensus Board and Evaluator Board in predicting the picks in the NFL Draft, but every other single board we could find over the last four years.
Let’s take a look at which players this year drew the largest disagreement between the two boards and see which players might surprise analysts with a fall in the draft and which players might end up being Day 3 steals.
The fact that Forecasters ranked Josh Allen highly while Evaluators didn’t care as much for him shouldn’t be surprising. It’s also not surprising to see Michael Dickson on the list; many evaluators don’t rank punters and he therefore didn’t show up in a lot of boards.
Evaluators, like in most years, seem to be higher on small school prospects. Players like Colby Gossett, Alex Kappa, Chase Litton, Joe Ostman, Genard Avery and Brett Toth caught the eye of those on the Evaluator Board, with only KC McDermott, Lowell Lotulelei, Harold Landry and Azeem Victor standing out among those preferred by Evaluators.
Those might be the players that we see “fall” in the draft relative to their general position on the Consensus Board.
On the other hand, two small-school prospects were preferred by the Forecasters: Marcus Davenport and Darius Leonard. It should be interesting to see how those players end up performing in the NFL given how dramatic those differences are.
In addition to that, players who have had their true position debated among analysts have seen more love from Forecasters than Evaluators — Leighton Vander Esch, Rasheem Green and Harrison Phillips were favorites for Forecasters.