22 replies to this topic

[1985]

I do data science (i.e. use statistics + programming to make predictions) for a living, so the Mechwarriors Online ELO system excites me very much.

The current system is not bad, but there is a glaringly easy improvement: you can vastly improve the current system by giving each player a "per-mech" ELO score. Let me explain the TWO benefits of this:

• If you assign a different ELO score per mech, then you will get better matches because, obviously, the performance of a given player depends very heavily on the specific mech he or she chooses. Much of the variation of performance of a given player will be explained away by this. This means that you will be able to match better balanced teams.
  
• I play a variety of mechs, some of which are stronger (e.g. RVN-3L) than others (e.g. RVN-4X, RVN-2X). Currently, whether I will win or lose can be predicted solely on the particular mech I chose. This is a very strong sign that the ELO system is working "on spec" -- good job! The game achieves the following problematic but "technically correct" empirical behavior: I win when I play a strong mech, and lose when I play a weak mech. [Please pause and briefly ponder the previous sentence if "technically correct" was felt cryptic on your first pass.] Implementing per-player-per-mech ELO scores will directly fix this problem.

The current behavior of always winning in your good mechs and losing in your bad mechs detracts from mechwarriors online in two ways:

• This foresight of outcome destroys the suspense and thereby takes away from the mechwarriors online experience. This foresight is an unintentional side-effect of the current ELO implementation.
  
• Unlocking mech-efficiencies on a new mech is time consuming. This is directly exacerbated by the fact that I will now be consistently set up to lose when trying to unlock these efficiencies. Some players may resort to underhanded tactics like smurfing and losing some number of games intentionally. Thus, this detraction is further exacerbated in that it will encourage pathological "gaming-the-system" type behavior which can ruin the game for 15 other players over and over again. (In spirit and sometimes in practice, this behavior is just like the old suicide farming tactic.)

I would be happy to discuss these issues in detail, especially if the devs are interested. In summary, the three benefits of a per-mech-per-player ELO score are:

• Maching better balanced teams for closer games
  
• Restoring a sense of suspense (e.g. Choosing between RVN-3L vs. RVN-2X doesn't dictate outcome)
  
• Removing an incentive to game the system

Edited formatting/spelling/grammar etc.; added a brief summary.

Edited by 1985, 26 February 2013 - 12:44 PM.

[Argent Usher]

Uhmm well the current isn't only bad but it is "as usual" (sry PGI) not really good and again only half-baked.

Certainly is a KISS system to integrate easily and i would have no problems with it but honestly the current "only damage" is wrong. As you wrote we have different mech classes (and role) and mech variants to do their work well or not on the battlefield (your example a stronger (e.g. RVN-3L) than others (e.g. RVN-4X, RVN-2X).

I still think the lack of a battle value system in addition to an ELO score is another problem.

We don't need a complicated system here, imo you would need to take only the cbill value of the mech and it would be more balanced as yet.
I also think the current xp points system is more suitable for an ELO system as the current "dealt damage" ELO system.

It should be discussed in any case because i don't want to see one day only three mechs on the battlefield.



Cheers A.U.

[Madkp]

Really good post. I hope PGI looks carefully at what you have said and decide to implement it.

Do you have any insights regading the (perceived) difficulties in group play with the new system, where users are reporting that grouping and dropping with a less skilled pilot results in their whole team being compromised? I feel I have experienced this myself, and I find it frustrating the the current system seem to not be handling this sort of situation in a way that results in a level playing field.

Edited by Madkp, 25 February 2013 - 01:40 AM.

[Inertiaman]

Wouldn't this break any proposed dropship/respawn modes though?

[Madkp]

Inertiaman, on 25 February 2013 - 01:47 AM, said:

Wouldn't this break any proposed dropship/respawn modes though?

I suspect it would make them more difficult to manage/implement. A person would be matched with a team after the map and mech are chosen for instance, which might result in some delays. I doubt it would break or destroy those functions, however.

EDIT: Respawn would indeed be trickier to manage, and may have to rely on an across mech average for groupings, which could result in some less than desirable match ups.

Edited by Madkp, 25 February 2013 - 01:51 AM.

[1985]

Madkp, on 25 February 2013 - 01:40 AM, said:

grouping and dropping with a less skilled pilot results in their whole team being compromised?

The current system addresses the problem in a limited extent, but could be improved upon. In an ideal matching system, the match-making will make it such that the vast majority of matches will end very close, thereby accounting for for inexperienced pilots with a combination of weaker opponents and stronger teammates. Implementing per-mech-per-player ELO will directly contribute to the match equality, in this very specific and important sense.

Edited by 1985, 25 February 2013 - 01:52 AM.

[Madkp]

So the hunch is that these perceived stompings resulting from grouping with less experienced/skilled pilots will become hammered out once their elo's have been assessed more accurately (with more data, etc), regardless of whether or not further improvements are made (though I do hope improvements are made)?

[1985]

Madkp, on 25 February 2013 - 01:56 AM, said:

So the hunch is that these perceived stompings resulting from grouping with less experienced/skilled pilots will become hammered out once their elo's have been assessed more accurately (with more data, etc), regardless of whether or not further improvements are made

This is an intelligent, though not-fully-correct hunch. More data will certainly go a very long way to improving things.

The core of the issue with the current ELO system is that the system uses just a single average number for each person, regardless of which mech that person chooses. Their ELO score does not allow the system to represent the specifics of which mech was chosen. As a direct consequence, the system cannot account for the particular mech chosen by the player in a given match. I am usually a big fan of the KISS principle, except that the omission of the mechs chosen in their match-prediction-model is just such a massive omission.

Edited by 1985, 25 February 2013 - 02:09 AM.

[Inertiaman]

They don't include it - but I don't see that they should, and if they do wouldn't they they need to take loadout into account? Creating an entirely new metabase for mechs to underpin Elo with seems as far from KISS as it's possible to get.

Further - good examples of existing Elo games (starcraft/hon/lol) dont' include race/hero/summoner type and seem to get by. The assumption has to be that a good player will generally have more impact regardless of mech.

[1985]

Inertiaman, on 25 February 2013 - 02:12 AM, said:

They don't include it - but I don't see that they should, and if they do wouldn't they they need to take loadout into account? Creating an entirely new metabase for mechs to underpin Elo with seems as far from KISS as it's possible to get.

Regarding this slippery slope argument, I certainly have some modeling/implementation ideas for a feasible loadout-based-system. That being said, given the resource constraints, PGI doesn't have to go the whole way; the proposed per-mech-per-person ELO score is actually pretty simple to do in code and will get us most of the way there.

Inertiaman, on 25 February 2013 - 02:12 AM, said:

Further - good examples of existing Elo games (starcraft/hon/lol) dont' include race/hero/summoner type and seem to get by. The assumption has to be that a good player will generally have more impact regardless of mech.

Actually Starcraft does not get by. I was much more highly ranked (diamond/master) as protoss than zerg or terran (roughly platinum level, based on playing with friends). One reason that I didn't play Zerg/Terran is because I couldn't queue for a real match. If all I played was P/P, P/T, and P/Z, that was somewhat acceptable because that's 1/3 the game. However, I wouldn't want to be constrained to just three mechs in MWO.

Also, LOL, Dota, and Starcraft are balanced by design. MechWarriors online, in contrast, is designed to follow TT. Balance in MWO is still important, but it frequently is compromised in deference to other objectives. Another way to look at it is that the heros in Dota/LOL have been nerfed/buffed based on their performance in professional matches, so as to be balanced. This is not the case in MechWarriors online.

Edited by 1985, 25 February 2013 - 02:42 AM.

[Inertiaman]

Fair points on the balance trajectory.

[Madkp]

Bumping for more views/interest from others

[1985]

shameless self-bump...

[flackee]

1985, on 25 February 2013 - 02:02 AM, said:

The core of the issue with the current ELO system is that the system uses just a single average number for each person, regardless of which mech that person chooses.

Not quite, this system uses a rating for each weight class for each person. Granted that isn't the level of per-mech-per-person you are describing, but an interesting compromise.

[Deathlike]

It's not that "simple", because assigning some sort of ELO modifier to each variant is actually subjective for the most part.

For instance, I have seen really good Awesome and Stalkers, despite seeing plenty of Atlas DCs in force... so it's hard to exactly quantify the value of a mech because of the pilot.

Outside of the "simple" change to each variant instead of the entire weight class.... that's probably the best you can do.

Remember that some people play better in certain mechs, than others, even in the same weight class.

[Inveramsay]

What I wish they did was to do something like Forza does. They add up the various variables that decides how good a car is and give a score based on that. It'd be pretty easy to add up Armour + speed + damage/second + alpha potential + heat dissipation + homing weapons and possibly some multiple based on the size of the chassis or based on the chassis itself. This would give a nice elo modifier if something is off which means if you do very little damage in your spider with 4xMG and 1Mlas but still contribute by capping points etc you still gain ELO points despite your low match score.

The match score I think is a bit flawed as it is so heavily dependent on damage done. Much fairer would be to add more emphasis of things like capping resource points and destroying components among other things. Killing enemies where you get something like 125 xp for the kill should give you a lot more points than getting a lucky killsteal

[1985]

Deathlike, on 26 February 2013 - 10:54 AM, said:

It's not that "simple", because assigning some sort of ELO modifier to each variant is actually subjective for the most part.
...
Remember that some people play better in certain mechs, than others, even in the same weight class.

Great observations. Hence we should avoid using a fixed ELO modifier, but instead to track a different ELO score, per mech, per person.

[ohtochooseaname]

OP, you're spot on. Also, this way, when you try out a new variant, you start out midway in the ELO system, and will get (presumably) a few wins under your belt to unlock those basic proficiencies faster.

I've suggested previously that they do a weighting based on ELO per specific mech and your overall ELO, but I really think it should just be reset with a new mech variant.

Edited by ohtochooseaname, 26 February 2013 - 01:24 PM.

[1985]

ohtochooseaname, on 26 February 2013 - 01:23 PM, said:

I've suggested previously that they do a weighting based on ELO per specific mech and your overall ELO, but I really think it should just be reset with a new mech variant.

Feel free to include the link to your post, I am interested in reading it...

[Aym]

Argent Usher, on 25 February 2013 - 01:21 AM, said:

Uhmm well the current isn't only bad but it is "as usual" (sry PGI) not really good and again only half-baked.

Certainly is a KISS system to integrate easily and i would have no problems with it but honestly the current "only damage" is wrong. As you wrote we have different mech classes (and role) and mech variants to do their work well or not on the battlefield (your example a stronger (e.g. RVN-3L) than others (e.g. RVN-4X, RVN-2X).

I still think the lack of a battle value system in addition to an ELO score is another problem.

We don't need a complicated system here, imo you would need to take only the cbill value of the mech and it would be more balanced as yet.
I also think the current xp points system is more suitable for an ELO system as the current "dealt damage" ELO system.

It should be discussed in any case because i don't want to see one day only three mechs on the battlefield.



Cheers A.U.

Wrong on both points.
ELO is win-loss not Damage done.
Battle Value is too hard to balance and too easy to break.