1 reply to this topic

[Omid Kiarostami]

It's been two months since our last major update to matchmaking, where we introduced our first pass at Elo and saw a tremendous improvement in the quality of matches being kicked off. With the next patch we're bringing in a new round of changes, and we've got two big ones lined up: Weight Class Matching and Elo Rating Adjustment.


WEIGHT CLASS MATCHING

We're adding the ability for us, the developers, to adjust our weight matching tolerances on the fly. This is giving us a controlled way to tighten up the weight imbalance between teams that players are experiencing.

Fun fact: About 74% of games kicked off are within what we consider tolerable weight matching limits (the difference between a heavy vs. assault Mech on the opposing team). Of the remaining 26% of matches, about 7% are what we would call "horrendously bad" (i.e., they carry the weight difference of one or two full assault Mechs between teams).

When we tighten up the Weight Matching, we're going to drop our weight tolerance to 0 and see what happens. Teams will be almost equally built in terms of weight class, though they will draw from a wider skill range of players to compensate. We'll be keeping an eye on the system while doing this and will continue to tune it until we're happy with the match quality and wait times.


ELO RATING ADJUSTMENT

We discovered a bug that was causing player Elo values to become inflated. Players that would win a game against a weaker opponent were being rewarded as though they were the underdog in that match. This caused Elo values to increase by a lot when they should have increased by only a little, skewing the player Elo distribution so that it became top-heavy.

What we've done (beyond correcting the bug) is a complete re-simulation of Elo ratings for every player in the game. Using match history we've been logging since February 5th, we determined what every players Elo rating would have been if we were using the corrected algorithm. The difference we noticed is huge, and the new player Elo distributions look healthy. We're going to push these ratings to the servers when we patch on April 16th.

So, how will this affect things? Well, without going into the details of how Elo works, this significantly improves our ability to match similarly skilled players. The old algorithm had an unintended (although interesting) bias towards people who had played more games, whereas the new rating measures only how you're playing. The bottom line is that we can kick off more close matches.


ROLLOUT PLAN

So when will we get to see this stuff? You won't have to wait long; this goes live with the April 16th patch. Other fixes, such as better detection of disconnected players while matching are also going in at this time as well. However, we'll be staging the release of the Weight Matching tweaks to be sure the system is working correctly (and to give us time to collect a bit of data on the new operating conditions).

Here's the plan:
On Tuesday, April 16th, we'll patch as normal and will put in the adjusted Elo values.
On Thursday, April 18th (tentative date), we'll drop the weight matching tolerance to zero. This won't require a patch or an interruption of service.

We're excited to bring the next round of changes live and look forward to matching up with you. See you on the Battlefield!

Any feedback on these features can go here.

[Omid Kiarostami]

We’ve done our tuning pass on the Matchmaker and have some good results to report. It’s hard to talk about this without going into details, so I’m going to get a little technical and share some of the data we’re looking at – just scroll down to the summary if you want the cliff notes.


ELO ADJUSTMENT

The Elo adjustment went in beautifully. To show you what we’re looking at, this is our distribution of players across Elo values:


For the curious, the darker portions of the graph indicate players who have played more than 50 matches. Visually, it indicates how the curve distributes itself over time.

The graph below is what our distribution was like before:


You can see how this curve favored people who had played more matches, distributing players across a much broader range. When we moved from the red curve to the blue one, we started measuring extremely positive results on our servers.

In my last post, I shared a statistic: That 74% of our matches were within “tolerable” weight balance. When we started matching with the blue curve, that number spiked to 88%. Beyond that, less than 2% of our matches now fall into the “horrendously bad” category we described earlier – that’s down from the over-7-percent number it was before. The difference in Elo for most matches produced is also is less than half of what it used to be.

We tried tuning the weight class tolerance down to further improve this, but got mixed results. The trade-off was between an improvement to average match quality (which has already changed with the Elo adjustment) and causing more of our outlying groups to fail to find matches entirely (specifically the 8-man queues). We’re not satisfied with the state of this, and have left the tolerance wide enough for 8-mans to continue to match while we work out a more complete solution. Note that our considerations include using tonnage as a direct factor to matchmaking, among other things.


IN SUMMARY

The result of all this is that we were able to cut down the number of unbalanced matches, but not eliminate them entirely. To summarize:

• There’s better skill matching of teams in game.
  
• Uneven weight balance matches occur less frequently.
  
• Wait times are lower (thanks to a denser curve).

This is something that we are monitoring heavily and will continue to address over time. As always, we appreciate the discussion and comments. You can more talk about this follow-up here.