At last, the time has come to reveal the data about my Jace testing. As a reminder, these data come from 500 matches, well over 1000 individual games played over five months. I publish my methods so that you can replicate my experiment if you disagree with the results, but I have to warn you that this is a substantial time commitment. There is a reason nobody else is undertaking this experiment.

To recap the qualitative results from last week, Jace, the Mind Sculptor is still extremely strong. You don't need to use the ultimate because, left alone, Jace's other modes win the game. He is an incredible value engine and is more powerful, if less versatile, than Nahiri, the Harbinger. I cannot fathom why you would play Nahiri over JtMS, especially when Emrakul is a liability . All that's left is to look over the numbers and see what they say.

The Total Data

I will start with the overall results before the individual matchups. This is partially because the statistical analysis of the total set is far more accurate than the individual results, and mostly so I only have to explain the what and how of the analysis once. Everything I did with the big set applies to the small sets, though the smaller n's of the individual results make them less accurate.

Onto the numbers!

Total Match Wins:

• Jeskai - 123/250: 49.2%
• Jacekai - 149/250: 59.6%

The baseline Jeskai Nahiri deck won just under half of the matches while the test deck with Jace won nearly 60%. While I did get better with the decks as testing went on (and on, and on), I mitigated this by alternating my decks, never playing two Jeskai or Jacekai matches in a row.

The Analysis

When I committed to doing actual statistical analysis, I looked for how to test a binary data set. There were a number of tests suggested to me and I did all of them. I wasn't sure which one was best, because I'm not a statistician—I'm a researcher who uses stats. Since the results were relatively consistent (the numbers weren't all the same, but the conclusions they reached were) I will only publish the results from the z-test. It's one of the common tests so I expect more people to understand the results. If you want to do these tests yourself, I placed all 250 matches for each deck into columns, recorded match wins as ones, losses as zeros, and then performed the tests with a null hypothesis of H = 0.

As a result of this test, I reject my null hypothesis and accept the alternative that the means are different. This means that the test results were statistically different from the control results.

The key thing to look at is the P-value. If it is less than .05 your data is significant at the 95% confidence interval. I consider this a one-tail test as I'm only interested in one deck being better than the other, but I know arguments can be made for it being two. Seeing that both .0095 and .019 are less than .05, this is not an issue.

The Meaning

The addition of JtMS to Jeskai Control had a positive and statistically significant impact on its win percentage. On that basis, Jacekai would have been firmly Tier 1 in the old metagame. What does this mean in terms of his safety for Modern? Hard to say. All this result proves is that JtMS is very powerful and lends that power well to my test platform. To determine whether remaining banned is warranted, we need to look at the individual test results, in the order I tested them.

Jund

I need to stress at this point that all the individual results are less reliable, statistically, than the overall result. The smaller the n of your sample, the higher the threshold of significance becomes. With only 50 matches per deck (100 total) to compare, you expect more results not to be significant. With those excuses out of the way, here's my experimental data:

Vs. Jund: Total Match Wins

• Jeskai - 24/50: 48%
• Jacekai - 33/50: 66%

Yes, your eyes do not deceive you. That is an 18% jump from Nahiri to Jace. I should also mention that this was the hardest match for me to test. Jeskai vs. Jund is a long, hard-fought war of attrition. Sometimes Jund won by completely stripping Jeskai's hand with Inquisition of Kozilek and following up with Liliana of the Veil and Tarmogoyf, but mostly games were decided by attrition. Jeskai seemed to have a slight advantage thanks to having more card advantage, but it was the experience of the pilots that really determined individual matches.

Nahiri fared worse overall because she could only provide incremental advantage. Exiling a creature was nice and helped some, but it just wasn't as good as fixing your hand with Jace. Ancestral Vision is good against Jund for a reason, and with fetchlands Jace could replicate that repeatedly.

Sideboarding

Jeskai's Sideboarding:

On the Play:

-2 Spell Snare -2 Remand

+ 1 Celestial Purge +2 Spreading Seas +1 Timely Reinforcements

On the Draw:

-2 Remand -2 Mana Leak

+2 Spreading Seas +1 Celestial Purge +1 Timely Reinforcements

Tempo cards are really bad in attrition matchups, so Remand had to go. On the play I was more concerned about Liliana, so I kept in Mana Leak. On the draw Dark Confidant was my main concern, so I kept in more answers to Bob.

We spent a long time trying to determine if I was supposed to board in Wear // Tear. We decided against it since it was a dead card too often, but Cage shutting off Emrakul was the deciding factor in several games.

Jacekai's Sideboarding:

-2 Snapcaster Mage -3 Mana Leak -2 Anger of the Gods

+4 Spreading Seas +1 Supreme Verdict +2 Rest in Peace

Rest in Peace is a beating against Jund and barely affects Jeskai decks. Anger is worse than Verdict here and was only kept in the other deck because it didn't have Verdict. Anger over Verdict only mattered twice. Leak was a dead card too often and was cut. I could see cutting all my Mages and playing all the RiPs, but you need some creatures to draw fire from Colonnade and JtMS.

Jund's Sideboarding:

-1 Lightning Bolt -2 Abrupt Decay -1 Slaughter Pact -1 Maelstrom Pulse

+2 Painful Truths +2 Grafdigger's Cage +1 Pithing Needle

Jund's plan was to cut down on more expensive or situational removal in favor of card draw and hate. The Cages were much worse against Jacekai, but my opponent felt that he would have brought the card in anyway. It was amusing seeing it and RiP in play the one time it came up.

Analysis

The results of the z-test confirm what your instincts might tell you about the above results.

Once again P < .05 so we reject the null hypothesis. This result is from actual variation in the data, and the result is statistically significant.

Meaning

Jacekai did statistically significantly better than normal Jeskai Nahiri. This is to be expected, since Jace actively draws cards and card advantage is critical to the matchup. One interesting aspect was that when JtMS and Liliana faced off, you often hid your best card on top after you Brainstormed unless you were going to play it right away.

Based on these results and his observed utility in Legacy, I expect that Jace would have a similar impact against other fair midrange decks in Modern. This would likely make him the focal point of those matchups. Value engines are always very potent in attrition matchups, and attaching one to a hard-to-kill permanent is dangerous, as Birthing Pod will attest. Ancestral Vision is used for this now, and Jace is a much better topdeck. I would imagine this would lead to the fairer metagames warping around him. Again, this is not surprising. This is very similar to what happened back in Standard.

Ad Nauseam

The most powerful and consistent combo deck in Modern, Ad Naus is advantaged against Jeskai. Game one is heavily in its favor since most of Jeskai's cards are dead and the clock is slow. While after sideboarding the matchup improves, I still wouldn't want to face it repeatedly.

Jeskai is the beatdown deck and it isn't very good at the job. Pact of Negation is serious game when we're counterspell-light. Playing more helped some, but many of my wins came by Ad Naus failing to combo rather than me stopping him.

Vs. Ad Naus: Total Match Wins

• Jeskai - 25/50: 50%
• Jacekai - 27/50: 54%

A very slight increase, which I attribute to the extra Vendilion Clique game one more than anything else. Snagging Ad Nauseam in response to Lotus Bloom was the best thing Jeskai could do, and having more chances over the course of a match made the difference. I rarely tapped out unless I was desperate so I didn't record a difference between the planeswalkers, except for when I attacked for lethal and then exiled Phyrexian Unlife.

Sideboarding

Jeskai's Sideboarding

-4 Path to Exile -1 Timely Reinforcements -2 Anger of the Gods -1 Lightning Helix

+2 Dispel +2 Negate +2 Geist of Saint Traft +1 Vendilion Clique +1 Wear // Tear

Jacekai's Sideboarding

-4 Path to Exile -2 Blessed Alliance -2 Anger of the Gods -1 Supreme Verdict

+2 Dispel +2 Negate +1 Stony Silence +1 Wear // Tear +3 Spreading Seas

The plan for both decks was to take out dead cards for not dead cards. Seas is not actually good here, but it's better than the sweepers.

Ad Nauseam's Sideboarding

-3 Spoils of the Vault -1 Phyrexian Unlife

+1 Pact of Negation +2 Thoughtseize +1 Boseiju, Who Shelter's All

Spoils is mostly there as a desperation card. Since Jeskai gave my opponent more time to go off, he chose to play a longer game and reduce the chance of killing himself.

Analysis

The test just confirms what you'd expect looking at the raw data.

This data is not significant. The variation in the means can be explained by normal variance and not by an actual difference in the data.

Meaning

Jace had no noticeable effect on this matchup. Honestly I wasn't expecting him to—Valakut, the Molten Pinnacle psuedo-combo decks were the only decks that could hang with Cawblade back in the day. In Legacy Jace is just a finisher against combo, interchangeable with any creature. I doubt that Jace will impact matchups against combo in Modern.

Bant Eldrazi

I expected this matchup to be hard, and it was. Eldrazi prey on slower decks, and it was especially hard for Jeskai decks when they had Cavern of Souls. This didn't happen all that much, but it made going long with Cryptic Command a dicey plan. The best was to trade as much as possible and hope they started flooding, which happens more than you think.

My opinion on Ancient Stirrings came from these tests. Seeing how poorly Serum Visions compared really made me question why Stirrings is legal.

Vs. Bant Eldrazi: Total Match Wins

• Jeskai - 20/50: 40%
• Jacekai - 26/50: 52%

That is quite the spike, but I'm not certain that I can attribute it to Jace. Supreme Verdict and Blessed Alliance are shockingly good against the deck, and I think their inclusion had a larger impact—though Jace giving you more chances to find them certainly helped. Exiling tapped Eldrazi was hardly irrelevant, but I'd rather be wiping their board or digging for answers at four mana.

Sideboarding

Jeskai Sideboarding

-2 Spell Snare -1 Vendilion Clique -1 Timely Reinforcements

+1 Crumble to Dust +2 Spreading Seas +1 Wear // Tear

I didn't have much to bring in, so I went for answers to Cavern of Souls and Chalice of the Void. Spell Snare is amazingly dead and Clique is uninspiring.

Jacekai Sideboarding

-2 Spell Snare

+1 Supreme Verdict +1 Wear // Tear

Since I had more good cards maindeck, I sideboarded fewer cards than in the control test, but I think it was better overall. Clique could have come out for Seas but it was a wash impact-wise during practice.

Bant Eldrazi Sideboarding

-4 Path to Exile -2 Engineered Explosives

+4 Chalice of the Void +2 Grafdigger's Cage

Eldrazi only cared about Jeskai's creatures when the game was lost, so answers were cut for disruption. Jacekai's sweepers made Chalice much less of a problem. Cage prevented Jeskai from winning on several occasions.

Analysis

This particular test highlights the problem of working with smaller n's in statistics.

P > .05 so the null hypothesis is accepted and the data is not significant. This is nearly significant at the 90% level, but that is not good enough. I must conclude that this result is within normal variation.

Meaning

I believe that if I had a larger n this would have been a significant result. Jace finding you answers more efficiently was important in a number of games and I think more testing might prove this. If I were going to do a revisit, I would start here.

Aggressive decks are aggressive decks as far as the data is concerned and Jace had no detectable impact here.

Affinity

The Affinity matchup surprised me. Most of my losses came from really broken starts or Plating-equipped Etched Champion. All of the medium Affinity games went Jeskai's way. Power Card decks do not like facing waves of removal.

Vs. Affinity: Total Match Wins

• Jeskai - 29/50: 58%
• Jacekai - 35/50: 70%

This actually should have been more lopsided in Jacekai's favor, but something I didn't expect kept it close: Nahiri answers Blood Moon. Affinity got a turn-two Moon against Jacekai several times—and turn-one'd it once—and crushed me. Had Jeskai been turn-oned it would have lost too, but it never happened. Instead, as long as I fetched for a Plains right away, I could hold on until I hit Nahiri and exile the Moon. Once I got to play Magic again, winning was easy. This kept the spread at least five matches closer than it would have otherwise been (three Jacekai wins denied, two Jeskai wins gained).

Sideboarding

Jeskai Sideboarding

-2 Remand -1 Mana Leak

+1 Stony Silence +1 Timely Reinforcements +1 Wear // Tear

Jacekai Sideboarding

-3 Mana Leak -1 Serum Visions

+1 Stony Silence +1 Wear // Tear +1 Anger of the Gods +1 Supreme Verdict

Two-mana counters are fairly poor against Affinity, so out they go for hate and more removal. I was unsure what to cut for Verdict in Jacekai so I cut a card I didn't like in my opening hand.

Affinity Sideboarding

-2 Master of Etherium -4 Galvanic Blast -1 Vault Skirge

+2 Thoughtseize +2 Ghirapur Aether Grid +1 Etched Champion +2 Blood Moon

Sideboarding with Affinity is hard, but my opponent felt that the extra reach from Blast was worse than the red enchantments. Master was big but not great against waves of removal. As mentioned above, when Moon landed it was devastating.

Analysis

Much like Eldrazi, I think the small n really hurts my analysis here.

Again, P > .05 means that this data is not statistically significant. It is more significant than Eldrazi by a small margin, but not enough. Therefore I cannot conclude that Jace had a significant impact on the Affinity matchup.

Meaning

I'm not exactly surprised by the result. The fast aggro matches played out as I expected, with Jace only being a factor once the game was won. That said, this was the one matchup where playing specifically Nahiri actually made an impact. Had she been any other planeswalker, the difference between the sets would have been larger and yielded more significant data.

Infect

Remember, this was pre-banning Infect. This was a weird matchup because when Infect had Gitaxian Probe it was heavily in its favor, but if not then Jeskai's relentless removal won the day. It was important to remember not to fight during the combat step except when I had Blessed Alliance. I did lose a few matches to turn-two kills.

Vs. Infect: Total Match Wins

• Jeskai - 25/50: 50%
• Jacekai - 28/50: 56%

Not much change, though again having non-targeting removal was very strong. This was the only time when I always aggressively +2'd Jace. There are so few cards that matter in Infect that being able to deny Infect's draw step was more important than me drawing cards. Yes, Nahiri would have won me the game outright. But the Cage kept me from winning in two matches, so I think we'll call it a wash.

Sideboarding

Jeskai Sideboarding

-2 Spell Snare -2 Mana Leak -1 Cryptic Command

+1 Crumble to Dust +2 Dispel +1 Wear // Tear +1 Negate

Jacekai Sideboarding

-2 Spell Snare -3 Mana Leak -2 Cryptic Command

+2 Dispel +2 Negate +1 Supreme Verdict +2 Spreading Seas

I took out the clunky or expensive counters for the more efficient ones and extra removal. Removing Inkmoth Nexus from circulation was very good, which is why I had Crumble in Jeskai.

Infect Sideboarding

-2 Twisted Image -1 Dismember -1 Distortion Strike

+2 Grafdigger's Cage +2 Dispel

Infect removed what the pilot's experience said were useless cards for some disruption and protection. He decided against Spellskite because it was a one-of, but given the choice would have played two maindeck. That's not what the mainstream decks were doing at the time, but maybe his plan was correct.

Analysis

If you read the Ad Nauseam section, this should come as no surprise.

This is not significant data and only shows normal variation within a sample. JtMS does not impact the win percentage against Infect.

Meaning

JtMS would not impact the fast aggro matchup relative to Nahiri. As I said, this doesn't surprise me at all—against really fast gotcha! aggro a win condition is just a win condition. Jace put the game out of reach the same way a Baneslayer Angel or Gurmag Angler would.

Conclusion

I feel the need to hedge my conclusion. On the one hand, the totality of the data showed that JtMS has a significant impact on Jeskai Control and would dramatically increase its win percentage. However when you get into the individual results that impact gets more indeterminate. This makes it harder to be definitive.

What I can safely conclude is that JtMS would have an uneven impact on Modern. Fair decks would be substantially impacted, with lessening impact the more unfair a deck becomes. This suggests that in a very unfair environment Jace would have no measurable impact. However, unfair environments do not last long before Wizards steps in. It is important to consider whether this uneven impact would be good for Modern long-term rather than just in a given metagame. Again, this testing was conducted in a pre-ban metagame. The meta has slowed noticeably and Infect isn't a deck anymore. Given the data and the current context of the format, I see only one conclusion.

Jace, the Mind Sculptor is not an implausible card to unban, but it is risky. Fair decks would feel the brunt of the impact, with JtMS' power likely to function like a black hole and warp those matchups around himself. It is possible that Modern can adapt and benefit, but the potential for oppression and a total metagame warp also exists. An unban would need to be seen as a calculated risk and an experiment rather than a permanent addition. If it works out, that's great. If not, and he must be rebanned, it should come as no surprise. I don't think that's worthwhile, but I would also really enjoy wielding the power of Jace again.

And that's enough writing about banned cards for a while. Next week we look at new cards from the Amonkhet spoiler!