27 replies to this topic

[Pater Mors]

Everyone is harping on about Elo lately, wanting it to be shown so that they can either flex their e-peen or cry quietly in a corner. The problem is that Elo in MWO and other team based games is not a direct measurement of skill. It's not even a direct measurement of strengths. Don't believe me? Let's take a look:

Elo ranking system as intended.

Originally the Elo ranking system was devised as a way to calculate the relative difference in skill between two players in a skill based game. Most notably Chess however, it will work in any competitor-vs-competitor environment. Checkers or Backgammon are two other examples.

Elo works by distributing points based on whether you win or lose and based on the Elo rating of your opponent. The amount of points gained or lost depend on which version of Elo used (there are many modified versions). The winner of the match gains Elo points and the loser loses some. This obviously translates well as a measure of skill as the most skilled player will inevitably rise to the top of the pile (there are, however, problems with Elo that newer ratings systems address but I won't go into them here). Your Elo rating will also tend to balance out around your skill level so that you will be able to see a direct measurement when you are improving.

Elo ranking system in a team based environment.

Many people have attempted to modify Elo to fit into a team based system but generally without much luck. Since Elo relies on individual scores it can only work if the entire teams are treated as individuals. If the teams are always this same, this is not much of a problem as the most skilled team will rise to the top (this translates in MWO to units who, I assume, commonly drop teams made up of the same players against other teams who do the same). In this case, it is possible to calculate a players relative skill compare to his team mates and thus, to any other player.

However if the teams are composed of different people for each game, then it is almost impossible to calculate an individual Elo score that actually matches a players skill due to the huge number of variables in random team play. A player who's never killed a single Mech can still rise a high bracket if he is carried enough (not likely, but possible). It is possible to still assign individual Elo ratings (MWO does this) but there is no way that they can reflect player skill under these conditions. The most that can be assumed about a high Elo score is that the player is average to above average.

Low Elo's reflect general strength in a system, high Elo's reflect actual skill.

In Chess, Elo at low levels doesn't tell you much about a players actual skill. Let's take some examples from my chess history.

My current Chess Elo sits between 1300 and 1400 and has averaged out over thousands of games, with my highest peak being 1455 some years ago. This reflects my general strength in chess. Strengths refer to pattern recognition, memorization of openings and positions, realisations of move types (pins, X-ray attacks, blocks etc), pawn structure and other technical things. These are fundamental concepts in Chess which are requisite to developing skill and at lower levels, this is what Elo measures.

For example I am quite strong when playing White and opening with the Kings Pawn Opening, which commonly develops into a move set called the Ruy Lopez. I am proficient in a large number of variations in this opening and know the 'best' moves to make accordingly for the opening phase of the game.

Similarly, when playing black, I am very strong when allowed to play The Sicilian defence. Again, I know a large number of variations of this defence including some of the more complicated ones used in high level play (such as the Najdorf variation, a favourite of chess great Garry Kasperov). I recently won a rated tournament (1200 - 1400, 64 players) by using obscure variations that I was very comfortable with but my opponents were not. This is a strength of mine. However, my Elo shows that I am not actually that skilled at chess... If I know so many openings and positions, why is that?

Skill in Chess (and any other skill based game) comes from taking all of the fundamental understanding of the above and applying it in the middle and endgame. I may have 10,000 openings and positions memorized, but if I cannot translate that into good middle and end game play, I am still going to lose a lot. This is my failing at Chess and the area that I am constantly working on. I am very good at overall strategy and picking variations that I know well to wrong foot my opponents, but if I come across a truly skilled middle/end game player I am boned.

At higher levels of play, is where Elo really counts skill, not strength. If you've ever watched games from some of the immortal players like Capablanca, Alekhine, Kasperov, Fischer or Morphy (my personal favourite) you will understand the difference between strength and skill.

In MWO, strengths begin as things like Chassis preference, perhaps a disposition towards good tactics or strategy, realisation of torso twist to spread damage, proficiencies with different weapons groups etc. Skill is when you can pull all those fundamentals together and consistently win games.

So how does all of this relate to MWO?

The simple fact is that at present, your Elo rating only matters because of your team mates. None of your strengths are measured and neither are your skills when you learn how to translate those strengths. All that is measured now is if you won or lost which is only really good for forming bands of players which Matchmaker can work with. There of course are higher tiers and it stands to reason that better players are within those tiers, but the current system gives only a anecdotal indication of skill at the very best.

If you constantly drop with the same people against teams who do similar and you are in the top bracket, congratulations. You are probably a very good player and your Elo is probably reasonably close to what it would be in a 1v1 system. But remember that even a few drops with different people mess up all that hard work.

The only true way pilot skill will be measured in this game is with the implementation of 1v1 with a separate Elo rank. Once that is in game everyone will be able to tell exactly how good or bad you are simply by looking at your 1v1 Elo (although Glicko is a much better system as it doesn't allow players to 'protect' their rating by refusing challenges).

So, in conclusion, PGI can make our Elo's visible. I'm fine with that. It will show who understands how the Elo system really works, and who just wants to flex their e-peen.

Edited by Pater Mors, 21 June 2013 - 09:19 PM.

[Soy]

Wait, so dueling is the only true skill?

Ok sounds like fun to me whatever

Edited by Soy, 21 June 2013 - 09:22 PM.

[Pater Mors]

Soy, on 21 June 2013 - 09:22 PM, said:

Wait, so dueling is the only true skill?

Ok sounds like fun to me whatever

No. That's not what my post says at all. It's the only way Elo as a system can give you a true reading of your skill inside a game. There are other systems which measure skill, but they're not implemented in this game so why would I talk about them?

[Deathlike]

http://mwomercs.com/...ost__p__2475383

I gave up trying to figure out ELO where it dumped me into a game where there was potentially 4-5 trial mechs on the field with me, and expecting me to magically LEAD THE TEAM TO VICTOLY.

And yes, I spelled it that way intentionally.

Edited by Deathlike, 21 June 2013 - 10:30 PM.

[Pater Mors]

Deathlike, on 21 June 2013 - 10:28 PM, said:

I gave up trying to figure out ELO where it dumped me into a game where there was potentially 4-5 trial mechs on the field with me, and expecting me to magically LEAD THE TEAM TO VICTOLY.

And yes, I spelled it that way intentionally.

That's because your Elo score forced matchmaker to put you in a losing game. It determined that to balance your Elo you needed to lose. You will often notice a streak of bad match ups or a streak of good. It's an unfortunate side effect of matchmaker using Elo to pair random teams using brackets because obviously it's got to keep those brackets, they can't get lop sided or the system doesn't work. Hence the 'bell curve' graph normally associated with Elo.

In a one on one, that's exactly what you want. It takes hard work to change your Elo bracket but hard work will do it and will move you up the curve. In random match ups it sucks because the computer makes the determination about what bracket you can fall into to fit it's needs. Skill isn't really considered, just W/L. Even if you do perform consistently better, you still lose because your team sucks. Thus, no actual measure of skill aside from magically beating the computers determinations occasionally.

[MisterPlanetarian]

Whats the point of having ELO if we don't get a ladder system?

Sometimes like the poster above me said I get absolutely awful teams with 4-5 trialmechs for some reason. This is only really a problem when I drop solo though, which is not that ofthen. I thought it was because of a lack of players simply breaking the matchmaking system.

[Deathlike]

Pater Mors, on 21 June 2013 - 10:41 PM, said:

That's because your Elo score forced matchmaker to put you in a losing game. It determined that to balance your Elo you needed to lose. You will often notice a streak of bad match ups or a streak of good. It's an unfortunate side effect of matchmaker using Elo to pair random teams using brackets because obviously it's got to keep those brackets, they can't get lop sided or the system doesn't work. Hence the 'bell curve' graph normally associated with Elo.

I'm pretty sure that most people would prefer that over time, they play against better players as they themselves become better.

The thing about "balancing out the ELOs" as it were, is that a loss to a better player or a better overall team is good to learn from. However, a needless loss due to simple sheer incompetence of your teammates that you have little or no connection to is not a cause every person is able to write off. If you've ever wondered why occasionally the solo PUGs vs premade debate comes up is that a premade that actually has a clue, tends to be matched up with "PUG six-pack".. the people you really don't know about. Once in a while, those premade/goon squads do die (and the lols are sweet), but it is automatically an unfair matchup that people prefer to avoid, unless you get your own premade/goon squad...

When the skill differences increase... it can certainly affect retention rates when these issues combine into the worse case scenario (which, really disappoints me at times).

I'm not saying that my ELO is anything worth talking about, but I would like to think that those that I fight generally is close to my level.. and not completely out of my league or a trial mech user, waiting to be feasted on because I'm on a losing streak.

Quote

In a one on one, that's exactly what you want. It takes hard work to change your Elo bracket but hard work will do it and will move you up the curve. In random match ups it sucks because the computer makes the determination about what bracket you can fall into to fit it's needs. Skill isn't really considered, just W/L. Even if you do perform consistently better, you still lose because your team sucks. Thus, no actual measure of skill aside from magically beating the computers determinations occasionally.

At this point, all I care about winning some bracket in the tourney. Everything else is kinda meh.

[Pater Mors]

Deathlike, on 21 June 2013 - 10:59 PM, said:

At this point, all I care about winning some bracket in the tourney. Everything else is kinda meh.

Yeah well the tourny is a slightly better indicator of skill because it takes into account a large number of variables. It's got kills/assists, damage, team damage, and tiebreakers. As I said in my other post, Elo just has W/L. See how the lack of variables makes it hard to get any information out of Elo in the current iteration of MWO?

[Keifomofutu]

Elo absolutely doesn't work when half the team can easily be premades on one team but not the other. (Ill leave out the horror story that is 2 4mans on one team vs 8 pugs because its just too silly a situation. Matchmaker shouldn't be so dumb as that.

[One Medic Army]

Keifomofutu, on 21 June 2013 - 11:18 PM, said:

Elo absolutely doesn't work when half the team can easily be premades on one team but not the other. (Ill leave out the horror story that is 2 4mans on one team vs 8 pugs because its just too silly a situation. Matchmaker shouldn't be so dumb as that.

All I have to say to this is that when I group I face other groups far more frequently.
It's actually a bit insane, play solo and I only see groups maybe 1 in 5 to 1 in 10 games.
Play with 1 other person and that's about the frequency of a game without an opposing group.

[Deathlike]

Pater Mors, on 21 June 2013 - 11:04 PM, said:

Yeah well the tourny is a slightly better indicator of skill because it takes into account a large number of variables. It's got kills/assists, damage, team damage, and tiebreakers. As I said in my other post, Elo just has W/L. See how the lack of variables makes it hard to get any information out of Elo in the current iteration of MWO?

There's a lack of ANYTHING that comes out of PGI that is known... not just ELO.

Keifomofutu, on 21 June 2013 - 11:18 PM, said:

Elo absolutely doesn't work when half the team can easily be premades on one team but not the other. (Ill leave out the horror story that is 2 4mans on one team vs 8 pugs because its just too silly a situation. Matchmaker shouldn't be so dumb as that.

Too bad it happens more than it should. I think I've played in enough premade where we actually get 3 premades on the same team... and tend to roll over the opposition...

[One Medic Army]

Pater Mors, on 21 June 2013 - 11:04 PM, said:

Yeah well the tourny is a slightly better indicator of skill because it takes into account a large number of variables. It's got kills/assists, damage, team damage, and tiebreakers. As I said in my other post, Elo just has W/L. See how the lack of variables makes it hard to get any information out of Elo in the current iteration of MWO?

The tourneys are a bit of a terrible indication of skill. If you're facing opponents determined by Elo than both the strong and weak players face roughly similar relative challenges in their matches. The difference is the good player is with 7 other good players and facing 8 good players (theoretically at least). The poor player is with a bunch of other poor players, and at the low end of Elo random luck matters quite a lot.

It's like if you looked at chess matches, and gave an award to the person who had the fastest match. It'd go to someone in a low enough Elo bracket he managed a fool's mate on his opponent.

What we really need is a personal metric averaged over a large number of matches, rather than your 10 best, but that's a discussion for another time.

Edited by One Medic Army, 21 June 2013 - 11:32 PM.

[Pater Mors]

One Medic Army, on 21 June 2013 - 11:32 PM, said:

The tourneys are a bit of a terrible indication of skill. If you're facing opponents determined by Elo than both the strong and weak players face roughly similar relative challenges in their matches. The difference is the good player is with 7 other good players and facing 8 good players (theoretically at least). The poor player is with a bunch of other poor players, and at the low end of Elo random luck matters quite a lot.

It's like if you looked at chess matches, and gave an award to the person who had the fastest match. It'd go to someone in a low enough Elo bracket he managed a fool's mate on his opponent.

What we really need is a personal metric averaged over a large number of matches, rather than your 10 best, but that's a discussion for another time.

Theoretically is exactly right, because that's not how it works in this game by evidence of all of our own experiences. Just because a tourny is on, it doesn't stop you getting dropped with players either far better or far worse than you.

On the chess example; no it wouldn't because Elo functions properly there, so you'd never hold a tournament like that which covered all Elo brackets. A better example is an open blitz chess tournament that anyone can enter. Also Elo is measured differently depending on the type of chess. A bullet game gets a different elo rank than a blitz game. You could be a fracking awesome bullet player and still have a blitz Elo hundreds of points lower.

Edited by Pater Mors, 21 June 2013 - 11:47 PM.

[One Medic Army]

Pater Mors, on 21 June 2013 - 11:42 PM, said:

Theoretically is exactly right, because that's not how it works in this game by evidence of all of our own experiences. Just because a tourny is on, it doesn't stop you getting dropped with players either far better or far worse than you.

On the chess example; no it wouldn't because Elo functions properly there, so you'd never hold a tournament like that which covered all Elo brackets. A better example is an open 10 minute chess tournament that anyone can enter. Also Elo is measured differently depending on the type of chess. A blitz game gets a different elo rank than a 10 minute game. You could be a fracking awesome blitz player and still have a 10 minute Elo hundreds of points lower.

I was mainly pointing out the scoring system for the tourneys here encourage those who play massive numbers of games, those who play far below their Elo, and those who get really lucky.
I do not see them as a good indicator of skill since the first one measured endurance, and those since measures your best 10 results rather than average result.

[Pater Mors]

One Medic Army, on 21 June 2013 - 11:45 PM, said:

I was mainly pointing out the scoring system for the tourneys here encourage those who play massive numbers of games, those who play far below their Elo, and those who get really lucky.
I do not see them as a good indicator of skill since the first one measured endurance, and those since measures your best 10 results rather than average result.

No you're right, they're not really, but my point was they're better than Elo is especially if someone performs consistently well in the same weight class in different tourny's.

Edited by Pater Mors, 21 June 2013 - 11:49 PM.

[One Medic Army]

Pater Mors, on 21 June 2013 - 11:49 PM, said:

No you're right, they're not really, but my point was they're better than Elo is especially if someone performs consistently well in the same weight class in different tourny's.

It's quite hard to qualify skill in a team game, especially one that's real-time and has many non-combat actions of worth.

The spider who runs through the enemy team and drags several of them off chasing him is being useful, but he's not doing anything quantifiable.

If anything was to be a determiner of skill, I'd probably go for two stats:
Damage done/XP (lower is better, XP is awarded generally for things which are useful, being useful with minimum firepower usually indicates a bit of skill)
Average fractional damage taken/death (higher is better, interpreted as percentage of mech's health lost per death, individual for each mech) A metric of how hard someone is to kill.

Edited by One Medic Army, 22 June 2013 - 01:08 PM.

[M e g a M a n X]

i think the OP has a point (though im not completely sure lol)

I've tried watching videos in youtube (typed keyword "MWO Kaos" since they seem to be popular)

players in the videos that appeared had realistic skills (not godlike).

But they have excellent comm so they win.

I think if you put some of them in PUGs they wont make a miracle or sumthing..


Premade average to above-average players with excellent teamwork >>

higher chance of winning >> "higher" ELO than what they would have if they were just in PUGs?

>> therefore not completely accurate?


Of course there are exceptional ones like Koreanese -- look up "Koreanese FTW"

in youtube hes REALLY good

Edited by M e g a M a n X, 22 June 2013 - 06:18 AM.

[Bishop Steiner]

Deathlike, on 21 June 2013 - 10:59 PM, said:

I'm pretty sure that most people would prefer that over time, they play against better players as they themselves become better.

You are a glass half full kinda guy, ain'tcha Bro?

Most of the people pounding their chest about their ELO on here ain't Pro-Gamers, ain't skilled, they are simply MetaRapers, using the path of least resistance to abuse any current Meta. Good players would favor a challenge, such as driving a sub-optimal Mech, because if you can excel while using Garbage, it will help make one Death on Wheels when the training wheels are knocked off. Right now, most of those who think their ELO is good are just FOTM, CookieCutterWarriors. Some might actually be decent, but you might also note a serious absence of certain players after the Meta shifts, only to return to play after the new Meta-Break is discovered.

If they Implement Solaris Dueling, ELO will have it's place, and to a lesser degree in 8 man (since MM is simply attempting to match the average of the ELOs of the 8 people on each team, it goes to reason, that 8 high ELO players on a Team will face 8 high ELO players. Regardless of if the same guys drop together or not (it will be more precise that way, though), as long as the 8 dropping are High Ranked, then their competition should be too.)

It's pretty much worthless in PUG/4Man though, and as the OP stated is not essentially indicative of skill. If they could base an ELO style system that took account things such as REAL accuracy (I say real, because Lasers give a false read, since if they touch for even a millisecond, it is counted as a hit. It should only count as a hit if the laser connects at the beginning of the shot, or if the beam stays on target 50% of the beam duration or more), and even hit location consistency, MAYBE, just maybe it would tell us something about actual skill. But even then, the hacks and exploits will of course skew who is what.

In short, in my experience, the real deal walk the walk and don't feel insecure enough to have to talk themselves up. You might have the occasional ego-maniac or low self esteemer, but usually "Pros" in any field are pretty comfortable in their own skins.

[Bishop Steiner]

One Medic Army, on 21 June 2013 - 11:58 PM, said:

It's quite hard to qualify skill in a team game, especially one that's real-time and has many non-combat actions of worth.

The spider who runs through the enemy team and drags several of them off chasing him is being useful, but he's not doing anything quantifiable.

If anything was to be a determiner of skill, I'd probably go for two stats:
Damage done/XP (XP is awarded generally for things which are useful, being useful with minimum firepower usually indicates a bit of skill)
Average fractional damage taken/death (interpreted as percentage of mech's health lost per death, individual for each mech) A metric of how hard someone is to kill.

Disagree with damage done. One can spread shots all over creation, and get massive damage, while taking forever to kill an enemy unit, or even impair it's combat efficiency. Truly skilled guy will have more kills, with medium-ish damage, as he consistently places his shots to make them count. Problem is there that you run the ragged edge of lumping Kill Stealer Guy in as actually being good as he might contribute next to nothing, then suddenly jump in and shoot the already cored mech (and usually equal opportunity on their teammates in their haste for that precious kill).

Which of course also leads to the question, what IS a kill steal? I've seen a guy in a mech with an open torso kill 3 of my teammates while they splashed damage everywhere else, lined up a shot, and killed the dog. Did I kill steal, or save my team's collective bacon?

*Shrugs*

I guess, in short, it's one reason why massive E-peen stroking has always seemed laughable to me. I love playing games as much as the next guy, but when it actually translates to a useful skill, or accomplishing good IRL, then get back to me. But if you are the bestest CoD-Warrior evah, and yet are just a no social skill, unemployed fat douche living in mom's basement, color me unimpressed, regardless. (Though oddly (sadly?), even gamers seem to have groupies, now. Weird world we live in)

[Hellen Wheels]

As a class C / class B chess player myself (USCF rating), I would add that in chess tournaments, one usually can elect whether to play within one's class, or in an open class (e.g., I could either elect to play against similarly rated opponents, in the 1300-1400 or 1300-1500 range, depending on the tourney, or elect to play in the "open" class where I would probably get paired with the highest ranked player in the first round.)

My point is, it is my CHOICE to play either within my class, or against the field that elects to play in the "open" category.

In MWO, it is not my choice. I don't get paired against similarly ranked opponents. I get paired against groups of 2x4man sync dropping thugs "elite" players, and that is hogwash.

If PGI is going to use ELO, then pair us accordingly and separate those of us who want to play within our class from the 2x4man sync dropping thugs "elite" players who can't be bothered with the minimal effort it would take them to form 8v8 against their "elite" comrades.