3 replies to this topic

[Pineapple Salad]

For a while now, it’s been generally accepted that the matchmaker’s ability to produce high quality matches hasn’t been very good. Instead of (only) complaining about how much the matchmaker sucks, I’ve actually been thinking about how to fix it. The solution I came up with is two-fold. The first involves making the current PSR system to work better, in order to give the matchmaker better material to work with. The second one involves creating another metric to measure actual player performance in games and to allow the matchmaker to better balance the teams. I actually started writing this thing back in the fall, but with MWOWC burnout etc, I had pretty much lost all interest in the game and couldn’t be bothered to finish it. So I had deleted the original and what I’ve written here is mostly from memory with my current thoughts added in. All the numbers here are for example’s sake only, and I haven’t put any significant thought into what they should actually be.

Let’s talk about PSR first. Elo system had it’s own issues, and PSR was in my opinion a step in right direction, but was missing a few key components that would make it viable in the long run. When PSR was first introduced, many players noted that there was a definitive upwards trend in the PSR system, and any average player would eventually end in Tier 1, if they just played enough games. So in the end, Pilot Skill Rating is not doing a very good job at estimating player skill. What it is doing a good job at is estimating player experience however, and it could be assumed that the more experienced gamers tend also to be more skillful. But what happens if a reasonably skilled player takes a half-year, or even longer break? Surely they would be able to quickly return to their previous level, but in the meanwhile the game might have changed, they probably have forgotten a thing or two, and it just takes time to readjust. And while they’re at it, they are not performing at a level that is reflected by their PSR rank.

To make the PSR more meaningful, and better tool for the matchmaker I have a few suggestions.

• First of all, PSR shift should be much quicker than it is now. Maybe it shouldn’t take more than 100-200 “steps” (where small increase is 1 step, medium increase is 2 steps, large increase is 3 steps and vice versa) to move through an entire tier.
  
  
• Instead of using some arbitrary number to decide tier limits, PSR should split playerbase to 5 equally sized tiers, based on the number of active players. The tier limits should be recalculated once a month.
  
  
• Player activity could be determined by number of matches. For example, 15 matches a month, under that and PSR bar will degrade by 5% for each game under 15. So for example, a player who played 10 matches a month will lose 15-10*5%=25% of their PSR tier. One month break would take a 50% tier 1 to 75% tier 2.
  
  
• Players under the activity limit will not count in the tier size calculation, because their overall impact to the matchmaker is low
  
  
• PSR will determine the pool of players which matchmaker uses to create matches. Only adjacent tiers should be allowed, but NOT in both directions. IE a tier 2 can have only players from tier 2 and 3, not from 2, 3 and 1.
  
  
• A matchmaker is running for each tier. On set time periods (different timer for each tier) they will scoop all available players (up to 24) from the selected tier. If the match is not full, it check if either adjacent tier has enough players in queue to fill the match. If yes, it will fill the match with players from adjacent tier, if not, it will release the players and wait for a set amount of time before checking again.
  
  After the matchmaker has found 24 players for the match, we come to the second part of the matchmaker: balancing the match and determining which player goes to which side.

Here we are going to need the second metric. Let’s call it Pilot Efficiency Rating, or PER. Right now we have a pretty nice way to measure pilot efficiency with average match scores. This is not perfect, and maybe some modifiers should be applied to it. The formula could be something like Average Match Score * (1+(KDR/10)) * (1 + (WLR/20)) We should also take mechs into account. PGI should already have pretty good data available on how different mechs perform on general. Let’s say that light mechs on average perform at 0.9 efficiency compared to the average of all weight classes combined. And let’s say that jenner IIC-A performs 25% better than light mechs overall average. So let’s add *(0.9*1.25) to our formula. PER should be recalculated once per month, based on last month’s results if player does not have enough matches in given weight class (under the activity limit) then the previous month's matches should be added in until the minimum sample size of the activity limit is reached. If no matches in given weight class exist, PER should fall back to global average.


So using my stats with heavy mechs from season 6: 325 *(1+1.63/10)*(1+3.0/20)=435 and let’s say I’m dropping with a Night Gyr, heavy mechs perform at 1.05 and night gyr at 30% over the average heavy, so 435*(1.05*1.3)=593.
Proton’s PER would be 1461.

The example modifiers are probably a lot larger than real world numbers would be, but to make a point, this enables even a high skill players to occasionally have fun and bring that vindicator or commando without totally shafting their team, because PER will take the mech into account and adjust the teams accordingly. On the other hand it emphasizes that a highly skilled player in a high tier mech is going to be even more dangerous.

Now we have a match with 24 players, each with their own PER. Now to determine who goes to which team, we have to first sort the players by their PER. Then add players to the teams one by one, in a way that the next player will always go to the team with less total PER.

So using random.org I generated a list of 24 random numbers in the range of 300 and 1000, where I’d imagine most tier 1 players would land. Sorted from highest to lowest the numbers look like this:
Player 1:970
Player 2:956
Player 3:924
Player 4:868
Player 5:844
Player 6:828
Player 7:802
Player 8:744
Player 9:700
Player 10:658
Player 11:635
Player 12:631
Player 13:624
Player 14:609
Player 15:595
Player 16:516
Player 17:473
Player 18:473
Player 19:409
Player 20:384
Player 21:373
Player 22:355
Player 23:341
Player 24:318


Now we start placing players to the teams:

Team 1:7513 Team 2:7517
Player 1:970 Player 2:956
Player 4:868 Player 3:924
Player 5:844 Player 6:828
Player 7:802 Player 8:744
Player 10:658 Player 9:700
Player 11:635 Player 12:631
Player 13:624 Player 14:609
Player 16:516 Player 15:595
Player 17:473 Player 18:473
Player 19:409 Player 20:384
Player 21:373 Player 22:355
Player 23:341 Player 24:318


And as you can see, even if the difference between individual players is huge, the difference between the teams is miniscule, because the system automatically balances itself. By the way, formatting this forum post was a PITA, maybe consider enabling some safe HTML markup, like tables.

So, what do you think? I'm probably going to do a simulator soon to test average search times and team PER balance with this system.

[Little Details]

I'm willing to pay money for them to completely rewrite the MM - instead of a Feb mech pack, have a 'donate to hire two consultants to review and implement the best idea (internal or player suggestion)". Your ideas seem reasonable to me - it fixes the three parts the problem - PSR, team skill balance, and mech taken.

Edited by LT Satisfactory, 30 January 2017 - 01:07 PM.

[Ultimax]

1) One of the main issues is playerbase size. As such reducing teams to 8v8 would instantly help as the MM wouldn't need 24 players for a match and instead only requires 16 - this would allow matching to be narrower.

2) Trev from Emp already put forward a very well thought out solution to the PSR here (he forsesaw this exactly problem one year ago): https://www.reddit.c...for_psr_system/



As an aside, 8v8 would also instantly lengthen TTK without having to nerf any weapons or mechs, or even buff any mechs.

It would also create a nice system where we'd have:

• 4v4 Scouting Matches
  
• 8v8 Quick Play Matches
  
• 12v12 FW Matches
  
• 12v12 in private lobbies

[Tarogato]

looool, you can imagine my surprise when I saw this thread on reddit just after I finished writing up and posting my own thread.


+1.

PSR needs exactly the improvements you mentioned and matchmaker needs to be willing to shuffle players between sides after it finds the 24 it needs for a match. Heck, I've seen group queue matches where you have something like an [EmP] 4-man and and [-D5-] 4-man on the same side, and the the opposing sides has at least one absolutely potato 4-man group and the rest of the players in the match are average. Why not swap the [-D5-] group for the potato group, so at least there's one high-level competitive 4-man on each side? Because matchmaker doesn't know better.

Edited by Tarogato, 30 January 2017 - 02:08 PM.