11 replies to this topic

[Chrithu]

Ok so now that the Game Mode Voting System as a solution to better matchmaking is off the table I think it is time to discuss if it wouldn't be best to leave things where they are for the time being and to look at ways to completely rework matchmaking after CW is released and with the needs for CW in mind.

The Status Quo

The current system with all constraints it needs to factor in seems to be in a state where further improvement is only possible if the constraints are traded off for each other.

Also as some people pointed out during the last two days, maybe the root of the problems is that the currently used Elo system isn't the best fitting solution to begin with:

For instance Russ himself pointed out that he never was awfully happy with Elo rank being determined by win and loss exclusively as it doesn't properly reflect how well a player did in a match or did not.

Then there were numerous posts in which people pointed out that Elo is a pretty superficial measure of a players skill with a certain weight class (or was it chassis? I don't remember anymore if there is a Elo per weight class or per chassis), but it doesn't reflect loadouts in any way and I think most of us would agree that even if two players are equally Elo ranked their specific loadout is very decisive for a match. In fact the closer the skill gap between two players the more important the actual mech loadout becomes in my view. So especially when two players have the same Elo rank the loadout begins to play a major role. In order to reflect this we currently only have weight matching, which is an even more superficial measure than Elo based on win/loss.

My Proposal

What I would propose is a complete rework of the system after CW is out of the door in the following fashion:

Matchmaking is done based on battle value and - in queues where it applies - game mode selection. This would remove the weight class constraints as a seperate constraint set.

Battle Value would be a combined measurement of the theoretical strength of a Mech (which includes it's weight, explanation to follow) as well as the player's skill with that chassis (or if possible I'd even maintain a skill measurement per variant).

I think that for good matchmaking both the player skill AND the mech loadout must be considered to get truely even matches.

How it Works (Player Skill Measurement)

Elo based on team win/loss isn't a really good measurement for individual player skill. But since the game is team based and not 1v1 it isn't possible to truely measure individual relative player skill. What we can do is measure absolute individual player skills by using the match score.

What I propose is to use the average Match score of the last X matches in a chassis (variant). What X should be is hard to say, it shouldn't be a complete history of the player's past matches but only recent match history.

In order for this to properly reflect the whole skillset of a player some modifications to match score are needed though:

It needs to properly balance the succesful use of NARC/TAG with the use of weapons in the respective slots. My suggestion is to attribute damage done by LRMs to both the LRM shooting mech and the supporting mech.

Not sure about this one: Maybe the Match Score should even include bonus points for the mech condition at the end of the match to reflect the player's skill at avoiding damage, but standing way back behind cover hiding away from the enemy shouldn't boost your skill rating. Not sure how to handle that yet.

How it works (Mech strength)

The loadout strength is measured in Battle Points:

Each Piece of Equipment (ECM, BAP, AMS,etc.. even Heatsinks and Engine), each weapon, module and upgrade and each ton of ammo has a certain Battle Point value reflecting it's impact on a mech's strength.

Armor is also translated into Battle Points.

The Battle Points of everything a mech has in it's loadout are added up.

This should reflect even a mech's weight since even ammo and armor have Battle Point values appointed to them.

How it works (Player Skill + Mech Strength = Battle Value)

The Mech's Battle Points + The player's skill score in that chassis (or variant if possible to track Player Skill Score per variant) results in the player's Battle Value.

The Matchmaker now tries to form teams of equal battle value, by adding as equal as possible to both teams from the gamemode specific queues where needed.

That is the basic idea. It sure needs a lot of maths and balancing to get it working but if it can be done I am sure it would lead to a lot better matches since it takes loadouts into consideration.


Reason for edit: My originally proposed idea of a changed Elo based on kills/assist cannot work, thanks to Ice Serpent for pointing that out.

Edited by Jason Parker, 08 October 2014 - 03:28 PM.

[IceSerpent]

Jason Parker, on 08 October 2014 - 01:11 PM, said:

How it Works (New Elo)

The new Elo would be determined by kills/assists rather than win/loss:

If player B get's killed he gets his Elo adjusted based on the Elo values of the player that killed him and the players that scored assists (including spotting/Narcing/Taging) weighted according to their respective damage percentage with a weighting bonus for the player that got the kill shot into the target.

Can't do that with Elo, as Elo rating becomes meaningless when it's based on random factors. I.e. me hitting a target (and getting an assist) isn't based on relative skill levels - it merely means that I had LOS and pulled the trigger.
You certainly can create a new "Jason Parker's rating" for that, but then you would need to come up with appropriate math behind it that would make that rating mean something.

[endevite]

I would like to simply see BV or a tweaked version of BV be used, elo is pretty useless in concluding a poor elo player doing well with a cheese build is actually going to tilt the table, but if it were by BV, that cheese build would hold a pretty high mark and be placed properly. (you will never be able to balance by piloting skill as you will always have pilots who can do well with terrible builds, and players who can only do well with cheese builds)

Many poor elo players (some of it caused by long term playing of harder to play builds, and recent switching to the newer high dmg cheese clan builds) can push high marks but are placed as a low tier player due to their history and not their mechs abilities.

I think it really should entirely be based on your mechs ability than your piloting history and represented by BV or a BV styled system.

edit: I would like to add, if any form of pilot skill history were to be used in the mm calc combined with bv, it should be short term history not entire history, say last 10 matches or so.

Edited by endevite, 08 October 2014 - 01:44 PM.

[Chrithu]

IceSerpent, on 08 October 2014 - 01:29 PM, said:

Can't do that with Elo, as Elo rating becomes meaningless when it's based on random factors. I.e. me hitting a target (and getting an assist) isn't based on relative skill levels - it merely means that I had LOS and pulled the trigger.
You certainly can create a new "Jason Parker's rating" for that, but then you would need to come up with appropriate math behind it that would make that rating mean something.

Hmm I'd say that the ability of a player to avoid damage through cover or to at least spread it through torso twisting is part of his skillset and the ability to aim at damaged mech sections for faster kills while the target is torso twisting and using cover is part of a player's skill set as well. Thus one player killing another is indeed a measurement of their relative skillsets. And I would argue it is a much better measurement than the team based win/loss that is used now. Especially when talking about individual skill.

[DAYLEET]

Im in as long as it help 3/3/3/3 work and not just make team of similar bv with nothing but assault and heavy.

[Chrithu]

endevite, on 08 October 2014 - 01:39 PM, said:

I would like to simply see BV or a tweaked version of BV be used, elo is pretty useless in concluding a poor elo player doing well with a cheese build is actually going to tilt the table, but if it were by BV, that cheese build would hold a pretty high mark and be placed properly. (you will never be able to balance by piloting skill as you will always have pilots who can do well with terrible builds, and players who can only do well with cheese builds)

Many poor elo players (some of it caused by long term playing of harder to play builds, and recent switching to the newer high dmg cheese clan builds) can push high marks but are placed as a low tier player due to their history and not their mechs abilities.

I think it really should entirely be based on your mechs ability than your piloting history and represented by BV or a BV styled system.

edit: I would like to add, if any form of pilot skill history were to be used in the mm calc combined with bv, it should be short term history not entire history, say last 10 matches or so.

I understand your point of view. My thoughts behind keeping Elo or something similar in place to measure player skill was that a really bad or a brand new player in a killer mech will actually hamper his team. Especially when piloting Assaults you gotta know exactly what you're doing or all it's power just goes to waste and in the end that mech's battle value will have meant nothing.

I strongly believe that you have to consider both. Trading one for the other will not really result in much improvement.

[Chrithu]

DAYLEET, on 08 October 2014 - 01:47 PM, said:

Im in as long as it help 3/3/3/3 work and not just make team of similar bv with nothing but assault and heavy.

Hmm a hard 3/3/3/3 proved to be unenforceable with pure Elo. It will remain unenforceable with BV+ Elo or pure BV as well.

What could be feasible is enforcing 4 BV brackets filled by 3 mechs each, which would allow for biggest light mechs matched with smallest medium, biggest mediums matched with smallest heavy and biggest heavy matched with smallest assault.

[IceSerpent]

Jason Parker, on 08 October 2014 - 01:45 PM, said:

Hmm I'd say that the ability of a player to avoid damage through cover or to at least spread it through torso twisting is part of his skillset and the ability to aim at damaged mech sections for faster kills while the target is torso twisting and using cover is part of a player's skill set as well. Thus one player killing another is indeed a measurement of their relative skillsets.

It doesn't work this way in MWO. You don't need to aim at damaged section in order to get an assist. You also don't need to do any damage avoidance in order to get a kill.
Consider this scenario: 2 players with 2800 Elo are fighting each other, 3rd player with 1500 Elo shows up and fires a single medium laser (hits due to hit scan nature of the weapon) and gets an assist. Should this warrant the 3rd player's rating increase when they simply fired on a target of opportunity? Should this not warrant rating increase if that target was a 1000 Elo player? Chances of landing that hit are the same in both cases.
Same goes for kills - same scenario as above, 3rd player happens to show up when target is almost dead and lands the last shot. It doesn't tell us anything about that player's skill relative to the target.

[Chrithu]

IceSerpent, on 08 October 2014 - 02:55 PM, said:

It doesn't work this way in MWO. You don't need to aim at damaged section in order to get an assist. You also don't need to do any damage avoidance in order to get a kill.
Consider this scenario: 2 players with 2800 Elo are fighting each other, 3rd player with 1500 Elo shows up and fires a single medium laser (hits due to hit scan nature of the weapon) and gets an assist. Should this warrant the 3rd player's rating increase when they simply fired on a target of opportunity? Should this not warrant rating increase if that target was a 1000 Elo player? Chances of landing that hit are the same in both cases.
Same goes for kills - same scenario as above, 3rd player happens to show up when target is almost dead and lands the last shot. It doesn't tell us anything about that player's skill relative to the target.

Ok got it. Doesn't work indeed.
Maybe Elo and the whole idea of measureing relative player skill needs to be scrapped completely. I just think that the current Elo isn't a good measurement of a player's skill, yet I believe that matchmaking cannot produce good matches if it leaves either loadout or player skill out of the equation. They both must be present.

Maybe an absolute measurement of player skill can be used: As simple as using the average match score of the last X matches. But it would need modification of match scoring to better balance NARC/TAG on light mechs with using those hardpoints for weapons: something like attributing the damage done by LRM to both the LRM shooter and the supporting mech.

Edited by Jason Parker, 08 October 2014 - 03:19 PM.

[IceSerpent]

Jason Parker, on 08 October 2014 - 03:11 PM, said:

Maybe Elo and the whole idea of measureing relative player skill needs to be scrapped completely. I just think that the current Elo isn't a good measurement of a player's skill,

Problem with the current Elo implementation is that Elo simply doesn't work (rating becomes meaningless) when match outcome is based on random factors instead of just player skill. This is why averaging Elo within a team is generally a big no-no, as 12 average players usually have a considerable advantage over 6 elite ones + 6 baddies.

Quote

yet I believe that matchmaking cannot produce good matches if it leaves either loadout or player skill out of the equation. They both must be present.

I wholeheartedly agree with this. I've been advocating using some sort of BV scheme literally for years.

Quote

Maybe an absolute measurement of player skill can be used: As simple as using the average match score of the last X matches without the bonus for a won match. But it would need modification of match scoring to better balance NARC/TAG on light mechs with using those hardpoints for weapons: something like attributing the damage done by LRM to both the LRM shooter and the supporting mech.

It's not that simple to create a functional rating system from scratch. See WN8 rating in WoT for example - people have been working on it for a very long time, yet it's still not quite accurate.

[wuzy]

Damage number is usually a good indicator of skill, although it can be skewed by SRM/LRM. So maybe account match score into it as well.

The main reason why I avoid solo whenever possible is due to how MM averages ELO; throwing elites with bad players like many have said. The burden of having to carry the lance while grinding a new mech frustrates me. In a large group where you know everyone will be up to their task, if we lose then at least we know there's nobody to blame but ourselves.

[Chrithu]

wuzy, on 08 October 2014 - 03:35 PM, said:

Damage number is usually a good indicator of skill, although it can be skewed by SRM/LRM. So maybe account match score into it as well.

Actually a good point, though I wouldn't attribute it to SRM/LRM alone. In order to adequately reflect the player's skill the score from damage must be smoothened by the mech's damage potential. A high sustained DPS build with a lot of ammo/Heatsinks easily get's 1000+ dmg whereas a low sustained DPS high Alpha pinpoint build can easily rack up the same number of kills/assists with a lot less damage.

I wouldn't say that the guy in the high sustained DPS high ammo build has better skill just because he did more damage. Sure damage plays a role but what matters are the actual kills and assists. And I think when using an average score over a certain number of matches it also smoothes out random factors like getting in a flyby kill shot as a total newb or carrying the whoile team to victory as an elite pilot. You don't get that every match. For constant numbers of kills/assist over a stretch of games you definetly need skill and it has nothing to do with randomness.

wuzy, on 08 October 2014 - 03:35 PM, said:

The main reason why I avoid solo whenever possible is due to how MM averages ELO; throwing elites with bad players like many have said. The burden of having to carry the lance while grinding a new mech frustrates me. In a large group where you know everyone will be up to their task, if we lose then at least we know there's nobody to blame but ourselves.

Yep that's exactly the point good matching must rely on a good measurement for individual player skill but also on the mech loadout.