I observed that recently my fellow writers have been looking at EDHREC as a source of information (for example Sig's article here and David's here). These are great articles that I highly recommend reading. However, I am missing something from both articles – and I've decided to investigate.

Both of these articles as well as others citing EDHREC as a source for their data tend to start off with the assumption that EDHREC is actually a representative source for Commander deck builders in general. The general approach to analysis presented in these articles are still great examples of what to do when presented with a source of information. However, building on bad ground could lead to unreliable results even when applying the best building practices. I got in touch with Donald Minor, who was kind enough to answer a number of questions I had regarding the data gathering and filtering approach taken at EDHREC.

It quickly became clear to me that Don is also taking a critical look at his processes, and among other things, we discussed ways his data might be enhanced in the future to make it more reliable. Of course, his focus is not on MTG finance, but some improvements that would benefit our community would also be useful for the deck-building player who uses the site, so I suspect that these improvements might make it in anyway. This article applies to EDHREC as it currently stands, and a revisit may be needed in the future if or when changes have been made.

What Can We Find on EDHREC?

EDHREC was created as a tool to support building Commander decks. Decks are loaded into its database and players can search in various ways to find useful analytical information regarding these decks. You can find information like the most commonly played cards per Commander or color, which Commmanders are most popular, and you can even have it compile an average decklist for a given Commander.

There are currently more than 150,000 decks in the database. It's hard to make such a large volume of data easily accessible, and I think they did a great job at it. They definitely created the best resource available for this purpose, and I am glad to see they are not leaving it at that, but are thinking about improvements.

EDHREC uses TappedOut as its data source. In the past, DeckStats and MTG Salvation were also used, but EDHREC has since moved away from that (MTG Salvation was only in use in the beginning, DeckStats up to about a year ago).

To be imported, a deck needs to meet certain conditions:

• Be listed in TappedOut's Commander / EDH section – obviously.
• Be about 100 cards large – filters are in place that will keep out anything much larger or smaller.
• Contain no illegal cards – this is mostly about color identity, there is no filter on banned cards as there are different banned lists in use.
• Not be a duplicate – if the URL was already submitted to the database, the older entry will be deleted. Don sometimes manually runs a script that checks for duplicates after a new Commander set comes out, but he says it finds surprisingly few of them.

Decks remain in the database indefinitely, with the oldest being from 2015.

Analysis

While the previous section is fairly small, there is quite a bit we can surmise from it. It's important to realize that any concerns will keep influencing the data as long as a deck remains in the database.

Duplicates

• If a deck creator resubmits a deck rather than updating the original entry, it will be nearly duplicated (depending on how extensive the changes are). I know that I personally like to keep previous versions of decks available to be able to look back and reconsider updates I've made in the past, and I believe others do too.
• People are likely to create new decks inspired by what they find on EDHREC, then list those decks on TappedOut. This leads to a self-fulfilling prophecy where the most-played cards are likely found in most newly submitted decks. EDHREC does try to counter this to an extent by calculating how often a card is found in decks created since its release, leading to more balanced suggestions. This does introduce a concern that new cards often don't need to be found in all that many decks before being suggested, while older cards that work well with those newer cards won't be listed even if they are found in exactly the same decks.
• Some people like to keep both a current list (that they own) and an ideal list (what they like to work towards). They might also experiment with variants. To what extent these are duplicates obviously depends on the progress the player has made towards the ideal list and how far their experiments deviate.

Without direct database access, it's impossible to get a good feel for how many of those decks are complete or partial duplicates. I, however, do believe that all three of these scenarios are likely. I also feel that for a new legendary creature, there is likely a lot of experimentation when it's just been released, causing lots of lists to be submitted and frequently updated. I suspect that this might cause more duplication in the beginning and that the amount of duplication goes down a bit later on, but it's impossible to be sure of this. Duplication is bad for our purposes, because it artificially increases the number played for the cards affected by it.

Player Behavior

It's impossible to get any kind of reliable numbers on how representative the people who post on TappedOut are for Commander players as a whole. Even if you could get such numbers, the Commander players you know might not be like the group as a whole. Because of this, what follows is based on my personal interpretation and experience from real-life gameplay and online discussion. I feel confident about it myself, but I strongly encourage you to think about to what extent it applies to your local situation.

• By virtue of being players who post online, we can assume the creators of these decks fall on the more fanatical side of the spectrum. I know very few real-life casual players who ever discuss decks online. Those who do, and I count myself among them, are always the more fanatical and usually the better players in their respective groups. In most cases, these people are also the most competitively oriented players relative to their groups (not to be confused with actual competitive EDH players who tend to focus much more on winning). This would bias the numbers towards stronger decks, and it also means that the vast majority of decks will not be represented in EDHREC (I would guesstimate that at most, five percent of all Commander players post their decks online – and I wouldn't be surprised if the actual number was below one percent. Now consider that only a portion of that small percentage submits their decks to TappedOut).
• Casual and competitive players alike will be among the submitting players. A player who especially craves to win seems more likely to be posting their deck online for discussion and improvement. This would lead to a bias towards stronger and more competitively focused decks.
• Players are not limited by their collections for virtual decks or may play with proxies in their groups (which rules them out as potential customers in any case). This leads to interesting results, like 788 copies of Mishra's Workshop, 1190 copies of Grim Tutor and 1532 copies of Imperial Seal in listed decks, which seem to be extraordinary large numbers considering these cards' rarities and prices. In general, EDHREC seems likely to find decks that are biased towards bigger budgets.
• Some players may look at EDHREC and deliberately decide to go another way with a particular Commander. I don't think that these people are as likely to post their lists on TappedOut, because these people seem like they prefer to figure things out on their own and keep their tech to themselves.

My gut feeling tells me that the statistics found on EDHREC are biased to the point where they don't really represent the people I am likely to play with in real life. However, in some areas, EDHREC usage may very well be more common than in others, and the more it is in use, the more representative it becomes for the people you might be dealing with. You really need to do research on this and act accordingly.

TappedOut

When I analyze EDHREC, I should really also consider the reliability of TappedOut as its source. Based on a quick inspection, I'd guess that my analysis presented in the previous section indeed applies.  I do not intend to estimate the exact extent to which the biases I've identified apply, and as such, I think it's safe to stop at just analyzing EDHREC and assume my conclusions for it also apply to TappedOut, as they are each drawing from the same data. For older data, this does become a bit murky as other sources were also in use then, but I reckon new Commanders are what mostly triggers movement in any case.

Comparing to My Own Decks

The only Commander decks I know well enough to compare against what EDHREC suggests are my own. I have eight self-built decks, but only one of those Commanders commands a large number of decks, there's even one that only commands ten! (Jerrard of the Closed Fist). I know this deck very well and have thoroughly discussed it online, so I think it would be decent for some analysis of the results EDHREC presents.

Mayael the Anima (952 decks)

There is actually a 100-plus page Mayael discussion on the official mtgcommander.net forums that's been started by me and that has basically become the place where Mayael players on that forum gather to discuss.

I see Mayael as a card advantage engine that basically allows me to draw the best of the top five cards of my deck and play it for free. Seen as such, I've decided to focus my build on mass removal, reasoning that my big guys force my opponents to overextend while essentially not costing me any cards from hand when they are called forward by Mayael. Basically, I run my opponents out of resources and then steamroll them with any combination of fatties.

Other people take other directions, aiming to suddenly strike through cards that give haste, building a big board position and overwhelming opponents, or perhaps some mixture of my controllish style and building that big board position.

I see some overlap between the cards I play and the cards EDHREC suggests, naturally: there are only so many fatties to choose from, but it's telling that I only play three of the top cards (all of which are in the bottom half). The deck suggested seems to be one that is more singularly focused on dropping a bunch of fatties and hoping for the best. Everything is very acceleration and combat focused, with quite a few staples sprinkled in.

Basically, a list built based off of EDHREC's suggestions would do exactly what you expect if you haven't put a lot of thought into your deck and it has lots of interchangeable parts (as Mayael will work with any fatty in the right colors). But it doesn't look exciting, and it feels like someone needs to put more effort into it. Despite nearly a thousand Mayael decks listed, I see many cards suggested that we decided against in the discussion topic for various reasons. Something is definitely making these suggestions unreliable, and it leaves me skeptical about the usability of suggestions for other decks.

I've looked at other generals I am decently familiar with and saw similar results. The deck suggested seems to be full of staples that would be pretty unexciting to play or face. I also tend to see many ways in which decks could be improved, though obviously that's subjective. I observe a far heavier focus on acceleration than I am used to seeing in real life and more "good stuff" or staple cards in general.

On the whole, it looks like this tool would be useful for players who don't know many cards to use as a starting point for building their decks. Financially speaking, I suppose mostly this tells us that there are staples in EDH that you will see regularly, and that there may be some merit in looking at the top 100 lists of cards to figure out what they are.

Conclusions

What worries me most when considering EDHREC's usefulness for MTG finance is that due to its success, we get into a vicious circle where later decks likely confirm the best cards for a Commander while true innovation may very well get lost in the noise. We are ideally ahead of the curve so that we can identify good specs before they spike, but only when innovation starts showing up in the average deck for a Commander does it show on this particular radar.

In a sense, this vicious circle is also good for us: we know the next generation of decks is likely to contain the same top cards, and that demand therefore will keep increasing. However, we are seeing inbreeding in a group that is very small relative to all Commander players, leading to a potential big mismatch with what our local player is actually playing.

It basically comes down to whether the players you are dealing with align well with EDHREC. I absolutely consider this a possibility, but with the many ways the data is likely biased, this doesn't feel like something that will often happen. As such, I am very skeptical about the usefulness of looking to EDHREC as a replacement for having an actual understanding of your local players and Commander as a format. I will definitely continue to rely on my intuition rather than its numbers, but I may keep an eye on the top cards for interesting movement – as I do see merit there.