29 thoughts on “Testing Stoneforge Mystic in Modern: Part Two”

1. Pretty expansive data set there. I think your conclusion checks out well, and I’m glad you went through the higher-level analysis to note that decks such as Death’s Shadow and Infect would find themselves boosted by proxy (especially given that Burn’s decline would put a big dent in their major metagame predator). Do you have any further projects of this type planned, or was this one exhausting enough that you’ll put this sort of thing on the back burner for the time being?

   Log in to Reply

   1. It was pretty exhausting, and I used up a lot of good will and favors to get all the testing done. Don’t expect anything like this from me again for some time, I doubt I’ll find enough experts willing to work on such a project for quite a while.

      I’m also not sure what I would test, most of what’s left on the list is there for a good reason or doesn’t need testing like Splinter Twin (No, Bloodbraid Elf is not a consideration, look at how good Jund is without it, don’t boost Jund). Suggestions?

      Log in to Reply

      1. Hey David, really awesome experiment, truly.

         While I can’t say I’m happy with the results (I’d really like to see more White in Modern) it is good to know from a strategic and knowledge-seeking standpoint. It would have been good to see the Dredge match up, but I can understand that it wasn’t around at the time of testing. I would extrapolate it would play out similarly to Affinity.

         What I find interesting is that it does do a great job of fighting off fair aggro decks. As such, if a new weapon were introduced to fight off the “fair” decks, SFM might be a more reasonable unban. I’m not worried about homogenization so much. Most red decks run Bolt, most green decks run Goyf, etc. In fact, I think the reason White is largely considered the weakest color in the format is because it doesn’t have a defining card like the other colors do.

         In terms of suggestions, if you ever wanted to do something like this again, I would love to see a Preordain test. It’s the card I consider most reasonably unbanned in the format.

         Overall, really thorough and impressive job. You should be proud!

         Log in to Reply

2. Fascinating conclusion. Friends and I have discussed the safety of unbanning SFM, we considered unbanning multiple things such as JTMS and BBE before SFM; as those cards are much slower than SFM and require more setup and card selection in deck building. Twin being unbanned would help reign in the speed of the format, which needs to come first, before SFM can even be talked about it. However, I do think that including Jund with x2 Kolaghan’s Commands would affect testing greatly. All in all – great study, can’t wait for more.

   Log in to Reply

   1. The problem with Jund is that at the time Kommand was seeing very little play, and I barely see it at all now, so it would have had no impact in the actual testing. I also couldn’t find a willing Jund pilot.

      Should SfM be unbanned that would change, but I don’t think that the existence of answers is reason to unban a card if its impact is high enough.

      Log in to Reply

3. Awesome job. You put a serious amount of work into this so thanks for that.
   Im not gonna lie reading through this article the conclusion was quite the twist. I did not expect you arrived at that from the data presented.
   I can see where you are coming from, but it seems that the control deck had a fine time answering Stoneforge and even though really unfair decks had a slighty better match up saying that meta will turn to really unfair decks might be an exaggeration. I would have liked to see how a kolaghans command decks grixis/jund would fair against Stoneforge because that will determine if the statement is true that you have to play Stoneforge to be a competative fair deck. Since those decks also have fine unfair match up

   Log in to Reply

4. Your hypothesis seems wrong to me. Something should be on the banlist if it violates the bannable criteria (turn four rule etc). Whether or not mystic slows down aggro is completely irrelevant. We dont ban cards for not slowing down aggro, so why would that be a rationale for unbanning?

   If anything your results make it look mostly harmless and not format warping. It deals a blow to merfolk and burn and doesnt do much vs combo decks and infect.

   You should have started with “why is this card banned?” because every white midrange deck would play it? Whoop dee doo every black midrange deck plays iok, every red midrange deck plays bolt, every green one plays goyf, etc etc.

   Whats changed since modern started? We have far better and maindeckable answers to artifacts. Decay k command revelry d sphere stony silence etc etc. Decks also do far more disgusting things on turn 3 now than play a 4/4 lifelink vigilance.

   Log in to Reply

   1. But that 4/4 doesnt die…easily. It can come in eot to dodge removal, it can provide late game power or buff a creature for +4/4

      Log in to Reply

      1. Killing the germ makes them blow 8 mana to replay the batterskull. With an sfm on board its just four which is significant, but if you cant kill a 1/2 creature when it matters you probably arent winning a lot of matches anyways. Letting your opponent untap with a stoneforge on t3 is like letting them untap with a bob or young pyro – not something you ever want to happen.

         Log in to Reply

5. I disagree with this conclusion. Not enough decks were tested, and only two fair non-linear decks were tested. Missing decks include bant eldrazi, jund, junk variants, kikicord, grixis variants and edlritch evolution decks. These decks all interact with creatures, and win through the combat step. Without seeing how stoneforge affects these creature-centric midrange matchups, I can’t agree with the conclusion. I like the testing, I just would appreciate it if more deck types were included in the testing, to reach a more thorough conclusion.

   Log in to Reply

   1. I mean, the thoroughness of testing means that getting more decks is a lot of pain. Additionally, it required good pilots for the decks it was matched against. I wonder if you could make a magic computer program that played as close to optimally as possible. The idea of being able to test hundreds of matches a minute seems interesting but somewhat scary (It would be very easy to find optimal builds.)
      As a Mardu player facing a lack of 2 drop my attention has shifted somewhat to a Bedlam Reveler focused build rather than a confidant one. Or maybe both if Im feeling a little lucky.

      Log in to Reply

   2. Let’s try to be reasonable here. David played 600 matches to create this data set, over several months. The labor involved in adding 6 additional archetypes is hardly trivial. Of course he hasn’t made the definitive statement on Stoneforge Mystic in Modern, but these data contribute significantly to the debate.

      Log in to Reply

   3. I mentioned this on another message board, but I’ll repeat it here. On the one hand, you’re not wrong that N could be larger and more decks could have been included. This is often the case with analysis like David’s. On the other hand, however, this is substantially more testing than Wizards will likely do for any unban. I don’t know whether they will consider results like these, but even if they don’t, the results could still be predictive of how Wizards will treat the issue with more limited testing. That is to say, even if N is smaller than a statistician or scientist would like, it’s still big enough to make unban suggestions from.

      Log in to Reply

      1. Yeah more testing isnt really reasonable but the decks could have been chosen a bit better. If anyone expected batterskull to swing the ad nauseum or infect match im not sure what to tell them. Did not likely need a hundred games of those to confirm that lifelink doesnt matter vs poison and is almost always inconsequential vs ad naus burn spell finisher. Burn itself should be hit the hardest being by far the most vulnerable to a 44 lifelink vigilance on t3. Its nice to get a number tho (26 percent improvement for the sfm deck)

         The meta conclusion also bugs me – is it a failure if a bunch of aggro decks get displaced by sfm midrange decks? Isnt it more diverse having a bunch of interactive sfm decks fighting infect and deaths shadow than having them fight other linear aggro like burn and merfolk? Id also note sheridan found sfm low impact despite 12 percent improvement vs affinity – if so, we probably shouldnt exaggerate the 8 percent uptick vs merfolk.

         Log in to Reply

   4. It is ALWAYS the case that more testing leads to stronger conclusions, just like it is always the case that testing is constrained by time, funds, and personnel availability. I think it was like the first paragraph where Ermenwein invited readers to continue this testing themselves… so go nuts.

      Log in to Reply

   5. Dude the guy played 600 matches…

      Log in to Reply

6. This right here is some outstanding Modern content writing. Really excellent work, thank you very much for your extensive testing and thoughtful analysis.

   Log in to Reply

7. Disappointed this didnt test against a single k command deck. One of the huge arguments against sfm is that it would kill all non-sfm fair decks, and that k command isnt enough to keep the matchup similar. Showing that mystic did indeed help against aggro, whether or not they are ‘fair” or “unfair” aggro, shouts to me that she is a good unban. A huge advantage in the jund/grixis matchup would be enough for me to be against the unban, which considering the jeskai matchup nearly unaffected (keeping in mind this deck can run sfm themselves and didnt for this test), i find it hard to believe kommand wouldnt keep these matchups even.

   Log in to Reply

8. well that must have taken quite a bit of work, well done,your testing makes perfect sense and confirms what many suspected: unbaning powerful fair cards will probably lead to an increasingly linear meta in order to avoid having to deal with them, this effect would be magnified with something like JTMS which essentially adds burn,zooicide etc to the list of decks it’s not very effective against

   essentially SFM wants you to go over her with something like Tron or a game winning combo, while JTMS wants you to go either under (burn,infect etc) or over him (tron) but both give a clear message: to beat us play linear

   Log in to Reply

9. Again thanks alot for the great amount work that was put into this. I have a few questions and I can not help but feel the conclusion was maybe a little biased or an interpretation of the numbers which atleast is not what I see from the data presented here.

   So basically to recap. We confirm that stoneforge helps fair midrange and control decks against fast aggro decks which was the hypothesis. The data does not show that control without stoneforge would be crushed by the card and this doesn’t even take into account grixis/jund which have natural answers to it but the fact that the control deck can win even without special answers. Furthermore we confirm that stoneforge does not do too much against linear combo decks which we already had a hypothesis about that it wouldn’t, since batterskull is only a beatstick here. So to conclude that the meta will move towards an more unfair meta is at best a guess and is no way in correlation with numbers presented here. Since this doesn’t take into account the other deck that could emerge from the meta slowing down so it impossible to predict the meta based on the data here.

   All I can see is:
   Yes she helps midrange/fair decks against fast aggro decks and that would help slow the format down. And yes you can be competitive without her in your deck. I honestly fail to see how you can argue that isn’t good for the format and she should stay on the banlist.

   Log in to Reply

   1. The hypothesis was for all aggro decks, not just the fair ones. The aggro decks that weren’t fair were not measurably impacted by Stoneforge Mystic. Therefore we cannot fully confirm the hypothesis, and on that basis cannot definitively say that Stoneforge would be safe to unban.

      Part of the hypothesis also claimed that there would be more room for fair midrange decks in Modern and I cannot confirm that due to the negative impact that the Stoneforge-less Jeskai deck saw. It’s not definitive evidence, but if Jeskai is harmed by not playing Stoneforge it stands to reason that other midrange decks without it would also be harmed.

      Furthermore, when we consider the fact that the faster unfair aggro decks were unaffected as was the combo deck, we can extrapolate that the correct decision in a Stoneforge filled metagame would be to play a fast unfair deck.

      Putting this together, what I expect to see is a decrease in fair agro decks and a rise in Stoneforge decks. This will lead to a rise in unfair decks who are unaffected, and this trend towards unfair decks will negatively impact the format. How long or sustained that would be is impossible to say, but if we see the format moving towards the unfair end of things I think that’s a negative impact.

      Log in to Reply

      1. Thanks a lot for the reply.
         I think your reasoning is a little flawed since we could almost already tell that a slow 4/4 wouldnt do much against combo decks and “aggro combo” decks. When you put it like this it gets almost impossible to evalutate the card since this can be said of a lot of cards. But it did help in the matchs we hoped it would help.
         The jeskai deck were only slighty affected with 48 to 54 preboard vs 50 to 52 %. This suggest that the deck doesnt get eaten alive by the card and you can be competative without the card .
         So the real question that should be asked is if the card allows other midrange/control to flurrish. Because as far as aggro and combo it did IMO what it was supposed to.

         I would argue that the meta would shift to more fair meta in the long run, since grixis control would probably be the biggest winner with sfm unban. Since it faces less of its worst match ups in burn and to a less extent zoo. And can play with vision (80% win rate when suspended on 1 against mystic), play good cards to counter the sfm decks and still have a lot of tools against the really unfair decks in counters and good removal. I could see even jund coming out strong from this aswell with good removal, discard and goyf which laughs at a small 4/4. Hell even bant eldrazi comes of strong from this since its decent against some really unfair decks but can struggle against some fast decks. But mystic is quite lackluster against them on paper, with displacer for the token, thought knot to take the equipment, big creatures and I could go on.
         Those are the interrresting decks since they will determine how oppressive sfm is against midrange decks. If the card doesnt allow for them to exist and you have to play sfm to be competative then I agree it should stay on the banlist. But from my hypotheses this is not the case and the card should be unbanned.

         As I said earlier I dont mean to take anything away from your work since you did an awesome job but this is simple to start a discussion.

         Log in to Reply

10. I dislike one part of the methods.
    The mystic deks used to test against enemy decks should have been very high initially so you could find a mysticdeck that had the best chances against the testdecks to start with,
    then, when you found something solid, you could start the heavy testing.

    I do however like this new trend of “mega-meta testing” which I may have helped form with my own rogue deck products (I test each of my decks for years) that I post on forums.
    (While known as wickeddarkman)

    Log in to Reply

11. Hello David! Thank you for this great article and the effort you and the other testers have put into this to get these amounts of data! I am no native speaker, but i hope you put the effort in reading this wall of text and can estimate what i am trying to say.

    Looking over the data, i do come to a different conclusion. So if you don’t mind, i would like to add a bit of context:
    First of all, you have tested Stoneforge Mystic in Abzan as Sheridan Lardner did, reasoning its near Tier1 presence as a logical proven best-deck contender that can adapt Mystic without splashing a Color.

    However, to measure impact of the Card i do think we need to incorporate the Metagame.
    Abzan attacks unfair Decks on the same angle as Jund, which we know is not very useful at the time, as the Meta has become more linear although these Decks exist and are prevalent.

    With Blue Control slowly vanishing, answers that can regulate unfair decks on a different angle (counterspells) are dropping in numbers. As unfair Modern Decks are resilient enough to only be disrupted temporarily, decks like Esper and Jeskai have proven excellent at extending their timewindow to win, but fail to exploit the window they openend. Stoneforge Mystic could address these issues for blue based Decks.

    So testing with Abzan might not be the optimal choice, as Abzan absically gains a powerful addition to the already powerful Tarmogoyf, but this adds nothing to the deck that it hadn’t had before. Jeskai and Esper simply lack that option.

    Additionally it reads like you based your Conclusion on the pre-board Match data. But when i do look at the Jeskai Matchup including post-board data i don’t see a significant Impact:

    Stock Abzan:
    Game One Win %: 48% (24/50)
    Total Matches Won: 50% (25/50)

    Mystic Abzan:
    Game One Win %: 54% (27/50)
    Total Matches Won: 52% (26/50)

    While the preboard-Matchup seems to have improved significantly, the post-board matchup must have been significantly worse on the same rate. In addition since the average round for Midrange Matchups is higher, it is easier to fix with the Sideboard since you can get less sideboard cards more often as the games are longer.

    As this is the only Midrange/Control Matchup data available, so i can only partially agree to the statement
    “It negatively impacts the viability of fair aggro decks and non-Stoneforge midrange decks, while having a negligible impact on the less fair decks.”
    because the data does not support the claim that it would reduce the viability of non-Stoneforge Midrange decks. The Impact on less fair decks for the meta cannot be concluded, since adding stoneforge mystic to Abzan neither adds a new angle of attack or more pressure than before to these decks.

    ” The likely outcome would be a shift to more unfair decks and the speed of the format increasing to try to ignore and invalidate Batterskull and Swords.”
    The format already is linear, and linear decks only have two choices when a new threat is introduced into the format: Ignore it and go under it when the effect on the Win% is minor, or try to invalidate it when it cannot be ignored (e.g. Destructive Revelry versus Leyline of Sanctity in Burn). If Stoneforge would be introduced into the format, the meta also could temporarily shift away from linear strategies in order to explore the strength of stonforge mystic and then return to its previous state when the impact is too low, at worst switching the Midrange-To-Go Deck from Jund/Abzan to Jeskai/Esper/Bant.

    The same argument regarding Midrange decks could be made about Tarmogoyf:
    “It negatively impacts the viability of fair aggro decks and non-Tarmogoyf midrange decks”

    Again, I really appreciate all the work you’ve put into this, this is an excellent read and really a great contribution to further discussion!

    Log in to Reply

12. Another limitation in your testing is you tested stock decks *from a metagame where SFM doesn’t exist* against an SFM deck. As with any powerful card or strategy, the existence of SFM may cause small changes in deck construction which limit the effectiveness of the new threat (in this case, SFM). For example, Kolaghan’s command decks like Jund and Grixis have been around for a while, but the popularity of Kolaghan’s Command itself has been declining. Perhaps in an SFM-legal world, these strategies achieve success by increasing their counts of Kolaghan’s Command. These kind of adjustments aren’t a bad thing and can actually be healthy for a format. These shifts are near impossible to identify during this type of testing. Bear in mind, I have no data to say this would be what happens were SFM made legal, I’m merely using the example to demonstrate my point.

    Lastly, there is nothing intrinsically wrong with having a common win-condition in a color, or strategy. Saying that SFM would become the reason to play white or that decks would splash white for SFM is akin to saying decks play green for Tarmogoyf, red for Lightning Bolt, black for Inquisition of Kozilek, or blue for Snapcaster Mage. Having to pass the SFM-test isn’t significantly different than having to pass the Bolt-test. In fact, these common themes help aid sideboarding decisions (“this is how I deal with SFM strategies” is the same as “this is how I deal with big mana strategies”). Unifying the threat base for a subsection of decks can actually reduce the strain that Modern players have on their sideboard, because specific sideboard slots are more useful in a wider array of match-ups.

    This point obviously assumes that SFM doesn’t become the only strategy in Modern, and I believe your testing showed ample evidence that this would not be the case.

    Log in to Reply

    1. I’m actually impressed about the whole work you’ve continued to reach such results. However, and since i kind of have some academic background in data analysis and investigation, some few aspects bother me…

       – You missed some key decks while testing

       Band Eldrazi is the main one, but Tron and Scapeshift (the BTL version), natural prey of midrange decks, should have been taken into account as well, if not tested, at least as a way to notice they might change your results

       – Other decks would pack Mistyc as well

       I see no reason for Jeskai decks to divide into two different versions: Nahiri and “Jeskai Blade” decks. That by itself, can shape the whole meta you’re trying to predict.

       – The lists aren’t fine-tuned to such a meta.

       Just like when you had Splinter Twin, some cards would see more play than others. Abrupt Decay numbers would rise, for instance, thus affrecting the results as well. Just a single example, for instance

       In short, it was a very useful work, but i think it’s very audacious to conclude that unbanning Mystic would result in a massive metagame shift.

       Valakut, Bitterblossom, Sword of the Meek directly contradict these conclusions, and they were all supposed to be broken and opressive decks.

       Don’t get me wrong, it was excelent work, what you did here. But i think your results shouldn’t give you such conclusions…

       Log in to Reply

       1. While I appreciate the criticism on this point, as has been pointed out before both here and on reddit, what I have is the result of 600+ matches played over nearly three months. I know this isn’t as through as can be, that’s why I note my limitations and said that my conclusions were based on my results, not the definitive answer. I saw fair decks punished and unfair decks unaffected. Trying to do any more would add months on to this project, and at some point you just have to publish.

          On some specific points:
          Tron matchups would be unaffected because Tron comes online the same turn that Stoneforge could be activated and mostly doesn’t care about 4/4’s on three (see also Loxodon Smiter). Bant Eldrazi had not been proven as a metagame fixture when I began the project and I only wanted proven decks. I also only tested against experts on their deck and there was never a Scapeshift pilot willing to participate.

          The next two I’ll address together: That adds too many variables. To make this as applicable as possible to the actual metagame I needed proven decklists. Trying to add Mystic to lists or adapt existing lists to a Mystic filled metagame means that I am adding in the variable of my own deckbuilding skill, and that would necessarily make my results weaker. Especially if I make a mistake and don’t detect it early enough.

          Log in to Reply

          1. Yeah, i understood why you didn’t include Bant Eldrazi in your analysis, hence the reason i wasn’t criticizing it as well. The Scapeshift deck is also well explained, because a subpar pilot would ruin all your data. I can agree with that.

             Againt, i actually lean towards the theory that Mystic wouldn’t warp Modern, but again, i do not have the data to support it. I just htought, as a fellow investigator that it was… bold to reach such conclusion. But i understand that this piece of research woul have to have a conclusion either way.

             And again, wonderful piece of reserarch and arcticle(s)

             Log in to Reply

13. Eldrazi displacer would enable someone to really abuse stoneforger mystic.
    Add it in a UW control shell and soon you will have plenty of rare swords with protection from two colors as well as batterskulls. If that wont work, something similare will.

    Log in to Reply