Nobody enjoys performance reviews. Even if they're positive. There's just something existentially distressing that comes with being assessed, aside from any professional implications stemming from said review. And they're also frequently boring for everyone involved. (From personal experience.) However, they can be useful for fostering improvement. Especially when they're self-administered in a public setting, so there's no weaselling out of self-criticism.

It's been long enough since the release of Modern Horizons 2 for the set to be reasonably explored and integrated into the metagame. There's always cards that get overlooked for years or need help to make the grade, but it looks like things are settling down. Relatively speaking, anyway. Therefore, this is an opportunity to revisit my periodic reflections on how spoiler predictions have played out. However, this time rather than trying to explain or excuse misses, I'm looking for the lessons from MH2 spoiler season. What did we miss, where did we miss, and why did we miss it? I'll also point out what we got right because criticism is easier to take when praise is mixed in.

The Top 5

The hardest part of evaluations is figuring out where to start. It's also quite hard to decided how to evaluate something, and criteria setting tends to eat up a lot of time. So I've decided to start as objectively as possible. And that will be working comparatively from a list. Jordan culminated the spoiler season with a Top 5 list, so it's fairly easy to compare that list to the most played cards list on MTGGoldfish and see how he did (According to that list as of Monday, 8/16).

Well, that's quite some variation, but Jordan did get 2/5 which isn't bad considering that these were evaluated without being tested in any tournament settings. That the two that Jordan got right are #1 and #3 is significant, even if they are in reverse order. That said, two out of five is not a passing grade, so shame on Jordan, right? Wrong. I'll admit, Sudden Edict is a huge whiff and has seen play in exactly one deck since release; everyone was high on Urza's Saga and yet it just hasn't worked out. It doesn't even make the MTGGoldfish's Top 50 list. Why? I'd say that the complete lack of splashability Jordan identified is the key. Saga is quite powerful, but accessing that power requires a commitment to artifacts that few decks can muster.

While Harvest hasn't really panned out either, that doesn't make Jordan wrong there either. Abundant Harvest has seen plenty of play since release. The catch is that it's been in a deck that has severely fallen off. Couple that with UR taking all the space for cantrips in the metagame, and there's no place for Harvest. That doesn't mean that Jordan's rating is wrong about what the card is capable of or its power in a vacuum. Things didn't work out the way that we expected.

It's Elementary

Another easy measurement should be the Incarnations list. There were five elemental incarnations and they had obvious power differences, which lent themselves well to ranking. And we weren't the only ones. StarCityGames recently did their Exit Interview for MH2 and ranked the incarnations based on how they've actually played. Averaging their scores yields a consensus place for each. Should be simple to just compare Jordan's list (which I agreed with at the time) to the SCG consensus and see how we did, right? Well, that's one way. However, it's also fair to ask how SCG's commentators did based on their initial impressions. But again, there's the MTGGoldfish ranking based on actual play frequency. Which is the most accurate? How about I sidestep that question and compare our list to all the options?

Jordan and SCG are in lockstep over Solitude as the most powerful. Swords to Plowshares is incredibly strong and having it as a pitch spell is invaluable. Jordan and SCG also agree that Endurance was third strongest. However, the only other point of agreement is with MTGGoldfish that Grief is the worst incarnation. Even the SCG guys were down on Grief compared to their initial impressions, and many admitted to rating it more on the basis on objective power than actual performance. It turns out that perspective and methodology really affect the evaluative process.

What it Means

The point here is that trying to measure spoiler season success in an objective way is quite hard if not impossible. Everything depends more on how the question is asked and what criteria that question is evaluated with. It's especially unfair since all jokes aside, we're not clairvoyant. There's no way to know how things will actually turn out without practical experience. So there's no objective criteria for success in these things.

However, that doesn't mean that we can't still learn and evaluate our performance. It just won't be in a nice list. This is about looking at what we actually wrote about the cards and whether we did a good job evaluating their potential. Again, did we totally miss on any played cards? What did we overestimate, and why? And what were we on the money about?

What Went Wrong

As I see things, it doesn't much matter if we got a card exactly right. Did we evaluate it correctly is what matters. We can't know how anything will play out until the set's released. The final value of a card is not the card's inherent power but contextual power once it finds a home. And we can only guess at that. Who saw Shardless Agent making Crashing Footfalls into a Tier 1 card? However, as long as we said the right things about the card I'm going to count it as a pass. Reading the future is hard, okay? Assess us on the things we could control. Thus, after going through everything that we both said about MH2, here's my evaluation of our performance.

Did We Miss Something Playable?

The biggest thing is was anything overlooked? That's easy to determine and also the biggest mistake we could make. And on that front I have good news. According to MTGGoldfish, the only commonly played MH2 card that we didn't talk about at all is Foundation Breaker. It turns out that Living End really likes having an evoke Naturalize to fight through hate cards. And I think we missed it thanks to fatigue. Every time there've been cycling creatures or sacrifice effect creatures we've said something to the effect of "it may find a home in Living End." And we're often right. However, having done this for so many years, it just slipped our minds. We'd said it so often that it started losing meaning to us and we overlooked a key card in a popular deck.

Missing a role player in what was at the time a newly resurgent deck is not that big a deal. We did at least identify all the potential playable cards from MH2 and were in the ballpark about how and why they'd see play. Overall, a good performance.

What Was Underestimated and Why?

We underestimated Dragon's Rage Channeler and to a lesser extent Unholy Heat and Murktide Regent. The problem on our end was experience. Jordan had tried years ago to make Delirium Zoo a thing. He found that Gnarlwood Dryad was great with delirium and terrible without, and had to contort his deck to make Dryad work. DRC looks a lot like Dryad stat-wise, so Jordan's experience said that DRC would only be great sometimes, which cooled our expectations. What we missed was how powerful repeatable surveil would prove in an UR shell and that it would synergize so well with Heat and Regent that UR Thresh would supplant UR Prowess. Jordan was right about what they would do and how they'd play, but they're all far more playable than expected. This is a case of experience leaving us gun-shy.

Meanwhile, I was too cool on Prismatic Ending in retrospect. Jordan was totally right about how splashable it's proven to be. I thought that it would see a lot of play (and it has) but only in control decks. Instead, the opportunity cost is so low and the flexibility so high that it sees play in basically every multicolor deck with white in it.

What Was Overestimated and Why?

I was too hot on Rishadan Dockhand in my initial review. Dockhand really hasn't played out well in Modern. A lot of that comes down to how the format has shaken out, but I also gushed too early. Tide Shaper was spoiled after I wrote that article and did what Merfolk needed much better. There was no way I could have seen that coming and nothing that made me expect that Dockhand would be superseded. And I did walk my assessment back a few weeks later.

Abundant Harvest has seen less play than expected, but I've covered it and Sudden Edict already. However, we did think that all the anti-Tron cards would see more play. That hasn't happened, though whether that's the fault of the cards being worse than expected or metagame considerations isn't clear. Tron fell off massively along with MH2 and it may not be related. Thus this may be a case of waiting for the right meta rather than an overestimated power level. On a similar note, Domain Zoo hasn't lasted. It did well initially, but has fallen away as Living End and UR Thresh have risen. The cards did what we expected in the deck, but the deck itself didn't work out. Or maybe just not yet.

The Big Picture

Overall, I'm happy with how our predictions shook out. We didn't get everything exactly right, but most of that is how the metagame has evolved, which is out of our control. When we made mistakes it was a combination of jumping the gun and relying too much on history to make judgement calls. Experience informs how a card will play, but it isn't deterministic. Let the cards speak for themselves rather than be spoken for by pervious cards.

What Went Right

I feel as though our greatest strength this spoiler season was card evaluation. We were very good at assessing where cards fit into Modern. We didn't always get their contextual power right nor do we know if the deck will succeed, but we succeeded at evaluating a card's role.

For example, I said that Counterspell largely replace the cheap counters but leave the expensive ones alone, and that's what's happened. I've also been proven right about the problems I identified with Grief, despite players desperately trying anyway. Jordan's evaluation of Urza's Saga and Prismatic Ending was right on the money (despite their places on his list). He also called out Fire // Ice and Suspend as playable cards when I didn't think either would see any play. We're good at understanding the cards. The metagame's the issue.

Lessons for Next Time

My lessons from MH2 spoilers are as follows:

1. Focus on the cards themselves. The metagame is certain to shift and decks will fall and rise. We can't predict that, so focus on our strength and evaluate the cards.
2. Focus on the cards in the current context. Circumstances and metagames change. Experience isn't always predictive, so don't let past experience dictate everything.
3. Remember that everything is relative. We don't know what we don't know. Don't sweat getting it exactly right. Instead, if you're retrospectively wrong, be wrong for the right reasons.

Sooner Than Later

And this is timely because the teasers for the Innistrad split set are already starting. I get wanting to do multiple themes and having parallel stories, but splitting the fall set in two is a bit extreme. I guess we'll just have to see how this plays out. While wondering why they couldn't just do this as an old-school block.