BlizzCon 2016 Heroes of the Storm: State of the Game Panel Transcript
Alan: Cool, thank you Dustin. As Dustin mentioned, why don’t we dive a little bit deeper into some of the things that we actually finished during 2016?
One of the nice things is we collect all the stats, we can look at all these stats, and then kind of make a determination of how did these changes actually work out in practice, how are players using them? We get a lot of feedback from you guys; but It is also great to look at all these stats to help us make decisions.
First off, let’s take a look at what happens after we actually made the major changes to our ranked experience and also adding an unranked draft. This graph is going to show you– this is player time in Heroes before we released that update. This is where players are spending their time in Heroes of the Storm.
Here you can see about 60% of game time was spent in Quick Match. After that, we had about 22% in versus AI, and then finally followed up by about 19% in Ranked which while it is the least played prior to the update. It is still pretty good for a highly competitive mode.
Now let’s take a look at what happened after we actually introduced that update where we improved the ranked experience, and we added unranked draft.
First off, we can see that quick match dropped a good amount down to 46%, ranked jumped up quite a bit to 30%, versus AI dropped down to about 16%, and finally unranked draft made a nice showing at 8%; and while that might seem like a lower amount we are actually really excited with that engagement from the player community.
We weren’t entirely sure how people would use unranked draft. Was it something that everyone was going to use? or was it something that maybe no one would end up using?
So to come out with this new game mode and actually have about 10% of players use it, was awesome; and it is a cool bridge between quick match and a highly competitive ranked play.
So like I said these were just a snapshot from right after we released that update, as time has gone on things have changed, but we are actually really excited that more people are playing with some of these newer game modes.
Speaking of ranked play we just had our first season roll. We had season 1 ended, and prior to that we had pre-season. Pre-season went on for quite some time, so we are glad we are able to actually finally get out to the updated ranked experience; and so again we can turn to some numbers internally and look at how things looked versus pre-season, versus the end of season one.
First, I want to show you: this is a graph that is showing the distribution of players at the end of pre-season; and so if you weren’t familiar with the system back then, we had a system where ranks were from 50 to 1, and you kind of climbed up that ranking system that way. Already by looking at this graph you can see some weird things, and this is part of some of the reasons why you wanted to change the ranking system.
First off, you can see that between 50 and 40 there is basically nobody there, and it is really empty, and that is because of some original design ideas we had where we wanted to make some of the earlier levels more progression based.
We wanted to actually make it so that you felt like you were moving up even if it wasn’t necessarily directly reflecting your skill. As a result, you didn’t actually lose any points in that range, you only gained points.
So at the end of pre-season, of course, everyone is out of that range, and you see that large mass of players between 40 and 30. The other thing that you will see that is kind of odd is that large peak at rank 1; and this also speaks to an issue we had with the old system where we did not have a high-end aspect to the ranking system.
Rank 1 was just this open-ended system whereby anyone above a certain skill rating ended up in rank 1. In addition, we did not have season roles. We did not have the opportunity really redistribute the players along that ranking curve there, and put everyone in the right places; so these are just a couple examples of things that we weren’t happy with in this old system, and just showing you that distribution. Now let’s take a look at what happened at the end of season 1.
This is with the new ranking system now. So right off the bat, you can see that things are looking a little better here.
We do have a representation at the bottom end. One of our goals was to make the ranking be an accurate reflection of your skill, and so we want players at all of these areas within the range.
So we do have a representation at the bottom at about 11%. Then you can see the largest mass is in that silver to gold range. Silver being a little more represented than gold, but majority of players are in that range; and as we move up there, you can see it becomes more exclusive. Less than platinum, less than diamonds.
Finally, you get up to masters where there is only 1% of players in that region who are in masters; and finally, grandmasters where when we shipped our ranked update, we actually had 500 players in there.
Since then we have actually changed it down to 200 players to make it even more exclusive, and to really highlight the best players on your region. This is looking way better, but there are actually a couple things here that we’d still like to explore that we are thinking about changing for the future season.
Number 1: While we do want the majority of players to be in silver and gold (we think that’s a nice place for the masses to be), we think that right now silver is a little bit over-represented. When we first made this distribution, we thought that that would be the right place to put players, since then we think we’d like you to be more balanced between silver and gold, and we’d actually like to bring more of those players into gold. So that is one change we are going to be doing, is making some adjustments in that range.
Another change that we’d like to look at is, we actually think that diamond is a little too over-represented right now. We’d like that to be a little more exclusive, and to really represent that increase in skill as you move through those ranks.
So we are going to take a look at actually redistributing some of that as well into the rest of the range, and bring in diamonds to be a little more exclusive; but either way, this was a much better view than our pre-season; and as we go through seasons, the nice thing is we can make these adjustments, we can look at the data, and we can keep making things better.
RANKED PARTY SIZES
Continuing along with ranked stuff, I want to talk about party sizes a little bit. Right now we have got our two leagues in our ranked system: one of them Hero League (solo and duo players only) and Team League (full parties of 5).
We have heard a lot of feedback from you guys, we have been this way for a little while now, and we actually play “rank a ton” ourselves so we have some experiences, and some thought on this ourselves.
There are a couple of potential issues here. You guys have brought it up, we have thought about it ourselves, and I just want to outline those for you.
So right off the bat, there is a perception amongst a lot of the players that solo players are at a disadvantage when it comes to Hero League. Basically, the idea is that if you are not in a duo, there is no chance for you to get to the top of the Hero League ladder; and the reason why I put perception there is because it actually doesn’t entirely play out in reality in the sense that when we look at the top of Hero League, we do have a good representation of solo players, it is not just duo players; and In addition, when we actually look at the stats we see that duo players, the advantage that they have is a little less than 1% over solo players.
So it is not like this hard guarantee that duo players are going to be the only ones at the top of the ladder, but the reality is there is that perception, and it makes the ranking a little weaker as a result, since there is that perception.
Another issue here is 3’s and 4’s can’t participate in ranked. Just by nature of our party size restrictions, there is just no place for them. As a result you need to pick up or lose players, or you go play unranked draft or quick match.
And finally, simply by nature of how Team League works, the fact that you need a full party of 5, you need to kind of find those four other friends that you are going to log on with at the same time, your schedules need to lineup, your kind of playstyles kinda need to lineup. All that stuff. There are less times that you are going to get a game going in Team League. Queue times end up being longer, and as a result our matchmaking is actually not as good in Team League as well. So that is not ideal. We want to improve that.
So what are we going to do about this? We have actually heard a lot about your feedback, we talked internally a bunch as well, and so there is something we’d like to do for a future season. We are going to test something out, and see how this goes.
So what we are going to do is: we are actually going to make Hero League solo players only. Only solo players can enter in the Hero League, and we are going to move 2’s, 3’s, and 5’s into Team League.
What are the benefits here and why? Why would this be something that is actually better? First of all, Hero League truly becomes a representation of your personal skill. This is just you entering into the queue by yourself, we are going to match you up with people of even skill, and then we are going to see how you perform.
So now there is no more this question of: “Well, did he duo or did he solo?” It truly is a representation of your solo skill in this game. The other benefit is that Team League is going to reflect your coordinated team skill. Now you get all of the party sizes with the exception of 4’s (and I will talk about it in a second) into Team League.
These are the players who maybe are on voice chat, maybe they have coordinated with their team as far as what hero composition they want to have, talked about strategies… Team League is now really the place to do that.
Now unfortunately, we couldn’t get 4’s in here simply because all of our solos are in Hero League, but we think that the ability to maybe pick up another player to get in Team League is an easier option now. Alternatively, you can still go and play unranked draft or quick match.
And then finally, we are going to have (I just talked about it)– we can get parties of 3’s in there now. So that is an awesome addition; and finally we are going to have higher Team League engagement.
As a result of being able to get more players into the game for Team League, that mode is going to have quicker matches, better matchmaking; and as a result, the ranking will actually have more significance as well.
It is no longer going to be thought of as like: “Okay well, that is just Team League. You don’t get a good match.” Now it is actually a good group of people playing in this mode as well.
So like I said, we are going to try this out in a future season. We don’t have the specific season just yet, but we are going to explore it. If it works: great. We are going to make it permanent. If for some reason it doesn’t work, we are going to make improvements, or we are going to decide to go back. Either way.
Let’s talk a little bit about matchmaking. As Dustin mentioned, a little over a year ago, our matchmaking was in a very bad place. We were not doing well.
We track a lot of metrics on matchmaking, and internally, so that we can kind of evaluate our own matchmaker. This is in addition to the feedback you guys give us, and we play this game a ton ourselves, and we knew it was not good; and back then, even the metrics we keep track of were saying the matchmaker was not in a good situation.
Earlier this year, we completely rewrote the matchmaker, we made significant improvements, and the matchmaker in all of our metrics actually increased significantly. We made much better matches. In addition, we also have been rolling out fixes and updates all throughout the year, and this is the great thing about the matchmakers is since it is completely on the server side, we can roll out fixes independently of patches, and then we can look at the data, and see how that goes, and keep on making more fixes.
I just wanted to highlight a couple of the changes we have made. We have actually made a lot more than this, but just to touch on some of these things.
The first one is about hero level, and this is for quick match specifically. Prior to this change, we looked at your skill as an individual reflection of how good you are at Heroes of the Storm, but we didn’t necessarily actually consider how experienced you were with a specific hero.
So let me walk you through this graph a little bit. There are two things being shown here. This is before we actually included hero level, and this is after we started including hero level.
In the middle there, you have got that dash line and that is kind of the ideal win rate. That is where we have our even games. This graph is a little bit zoomed in actually, so it doesn’t mean the bottom is 0% percent and the top is 100%; but we wanted to highlight it a little better for you.
In the bottom on the X-axis, you have got hero levels going across to 20. So first let’s look at the blue line there. This is without considering hero level. This is before we took this into consideration.
You can see that at level 1 for hero level, your win rate is quite a bit lower than that ideal rate; and this is purely because even though you are really good at maybe Heroes of the Storm, you are playing with a brand-new hero. You don’t know how to use him yet, you don’t know what all the talents are, and when you should pick what, when to counteract, what the opposing team is doing; and as a result your win rate was lower.
As you kind of move up to the hero levels there and you get to around 7 or 8, you see that you start crossing over that line. That is like that sweet spot where players start learning how to use their heroes, how to counter opposing teams, compositions, and whatnot; and then, you see we actually jump quite a bit over that line.
As you get more experience with your hero and you go into the teams, now your win rate is a good amount higher. Either way, this is not an ideal kind of view. The purple line is what happens after we start considering hero level in our quick match matchmaking.
So already you can see from 1 to about 8 (or so), we are bouncing around that win line there. It is in a much better place. As we cross over again, once we get to 8 or 9, you see again players start understanding how to use that hero and we start crossing a little bit over.
While it is a little bit over that ideal win line, it is much closer than it was before. So this has actually been a great improvement to quick match matchmaking, and it is one of those things that we are going to look at. We are going to keep on tweaking those numbers and we are going to keep on enhancing as time goes on.
One of the other changes we made (and this is more for Hero League) is we also added an additional restriction where we will not match you up in a game where a player is two divisions lower than you.
So what that means is: if I am gold 1, we are going to match you up potentially with someone in gold 2, or gold 3; but we are not going to drop you down lower to silver, or bronze, or something like that.
This was a huge improvement as well for our Hero League matchmaking, and one of the reasons why this was necessary is occasionally we get into situations where the matchmaker says: “Look, I actually can’t find a game right now,” and we essentially say: “They have waited long enough. Let’s just start this game,” and we started a lot of games that weren’t ideal.
We put this restriction in so that we said: “Look, it doesn’t matter how long they wait, we don’t want to actually start a game that is significantly poor quality.”
So this has also made a lot of improvements. Again, these are just two changes we have made, we have made many more; but I just wanted to highlight a couple for you.
One of the things is all of these changes have actually had an impact to our queue times. Back when we started on this initiative to improve our matchmaker we said: “Look, we are willing to trade queue times up to a limit for match quality.”
We were laser focused on improving match quality. As we kept on making these improvements, we basically deliberately traded that aspect of queue time for match quality; and so in some cases just to give you some numbers, in quick match our times may have gone up by about 50%, and in some cases for Hero League it may have gone up by 200%.
Now we realize this is at the same time if you think back to my other graph where we actually have an increased number of players playing in Hero League; and so this is basically a great view on what happened here where we decided to really focus on match quality — which we still feel is the priority.
However, we don’t think it is the only thing. We want to get you in the games quickly, and by the way, let me just touch on, while some of these numbers looked very large in terms of going up 50% or whatnot, previously maybe we were finding you a game in 60 seconds or so, now in quick match we might be finding you a game in 90 seconds.
So regardless, even that increase, it is more than we want. We want you to get in the games quickly, and so this is something we are also looking at right now. We are going to be working on bringing those queue times back into check, and while still not actively hurting match quality, it is something that we still value and we still want to prioritize.
Next up we have Travis to talk about some cool new stuff coming up.
Next: Coming Soon™
|BLIZZCON 2016 HEROES OF THE STORM: STATE OF THE GAME PANEL TRANSCRIPT|
|Last Year Recap||Game Modes||Coming Soon||Panel Q&A|