0
For those of you too good for Reddit:
Quote
The change had the desired and anticipated effect. We had run several simulation runs using production telemetry against a local test environment both with and without these changes, and had gathered metrics on the expected outcome. Production group queue actually ended up exceeding the simulations by a bit. Solo queue not as much, surprisingly.
Your suggestion is noted. Russ simply felt like this particular issue wasn't worth increasing customer dissatisfaction over. He's asked me to revert the change for now. We can revisit this issue once emotional levels have calmed down, and everyone's had a chance to fully consider the change.
Your suggestion is noted. Russ simply felt like this particular issue wasn't worth increasing customer dissatisfaction over. He's asked me to revert the change for now. We can revisit this issue once emotional levels have calmed down, and everyone's had a chance to fully consider the change.
Quote
Player opinion is the majority of the decision in this particular case. We want to improve skill matching, because the matchmaker is currently producing games that it knows are imbalanced. We also have lots of nifty new game-mode stuff that we're working on for CW, and it would be very nice to bring at least some of that back into public queue at some point soon in the form of additional game modes. Hard-constraints are simply not a scalable solution, as every new game mode we add splits the queues further and further. Voting scales without harming match quality as a result.
Regardless, the Elo averages currently are not too bad in solo queue at least; but those outliers are really brutal. We have a fairly decent skill metric that is accurately tracking player ability as well as predicted match outcome, and weknow that those outlier matches are of unacceptably poor quality. The voting change was one relatively low impact way to reduce the number of outlier matches the matchmaker was producing.
Regardless, the Elo averages currently are not too bad in solo queue at least; but those outliers are really brutal. We have a fairly decent skill metric that is accurately tracking player ability as well as predicted match outcome, and weknow that those outlier matches are of unacceptably poor quality. The voting change was one relatively low impact way to reduce the number of outlier matches the matchmaker was producing.
Quote
The game mode restriction code is currently shared, due to the roots of solo and group queues initially sharing small groups (small groups initially could end up in either queue) at new matchmaker feature launch.
It would be possible to separate group and solo queues such that only group queue used a vote system, and solo queue used hard constraints; but this change is a bit more involved, and design would want time to clean up the UI so that these behavioral differences are clear to the user.
It's possible that we'll revisit adding this change back into the group queue only at a later date; but that decision will be driven by Russ and Paul, and will certainly involve additional community feedback.
It would be possible to separate group and solo queues such that only group queue used a vote system, and solo queue used hard constraints; but this change is a bit more involved, and design would want time to clean up the UI so that these behavioral differences are clear to the user.
It's possible that we'll revisit adding this change back into the group queue only at a later date; but that decision will be driven by Russ and Paul, and will certainly involve additional community feedback.
Quote
We're already discussing some potential compromise solutions, just in case the majority of the playerbase is never willing to lose all guarantees over game mode preference. I'm sure some clever ideas will also be brought up by the playerbase themselves, that might be valid compromise solutions that everyone can get behind.
As to the speed of reverting this change; I agree it is unfortunate, but this is really down to a customer satisfaction issue at this point. Russ really wanted a very high majority of support before executing this change. He thought support was there after the results of the first poll; but after throwing up the second poll, it is now very clear that this is a highly contentious issue.
As to the speed of reverting this change; I agree it is unfortunate, but this is really down to a customer satisfaction issue at this point. Russ really wanted a very high majority of support before executing this change. He thought support was there after the results of the first poll; but after throwing up the second poll, it is now very clear that this is a highly contentious issue.
Quote
This is true that the magnitude improvement was greater for group queue than solo, even in simulation.
However, the fundamental nature of this change allows the matchmaker to consider many more potential match combinations than before; solo and group. This resulted in approximately a 5 point average Elo delta improvement for solo queue in simulation, which never really materialized on production. That's why I say it was surprising.
However, the fundamental nature of this change allows the matchmaker to consider many more potential match combinations than before; solo and group. This resulted in approximately a 5 point average Elo delta improvement for solo queue in simulation, which never really materialized on production. That's why I say it was surprising.
When asked about the possibility of "hard veto" instead of "hard select":
Quote
Yes, that is a very valid suggestion, and is one of the compromise solutions we are discussing. This solution would also allow us to add additional public queue game modes without further harming match quality beyond where it's already at.
Edited by Rebas Kradd, 08 October 2014 - 02:41 PM.
