Jump to content

Why Elo Doesn't Work Here


633 replies to this topic

#81 MrMadguy

    Member

  • PipPipPipPipPipPipPipPipPip
  • 2,221 posts

Posted 23 January 2014 - 03:37 AM

It feels like my ELO is oscillating like pendulum: first it's low and I win many matches in a row, then it becomes high and matchmakes start to put me into completely dumb teams, which result in 100500 loses in a row, then my ELO falls again and I start winning... So it doesn't reflect my skill at all. And may be it will be much more fair, if your team will be chosen randomly and you'll have 50/50 chance to get bad or good team?

I guess, now my ELO is on peak, cuz I've been playing this game the whole morning and all of my mechs still have x2 EXP bonus, which meant, I haven't won even a single match today.

Edited by MrMadguy, 23 January 2014 - 03:50 AM.


#82 Abivard

    Member

  • PipPipPipPipPipPipPipPip
  • Shredder
  • 1,935 posts
  • LocationFree Rasalhague Republic

Posted 23 January 2014 - 03:46 AM

View PostMischiefSC, on 22 January 2014 - 05:17 PM, said:


You know I should probably not post this. Just got back from physical therapy and I'm on so many pain meds I am high as a ******* kite right now but I'm still going to respond.

I get that people want to believe that any time they lose it's not their fault. I get that. Totally get it. I get that when you see someone on your team do something stupid when you're spectating it's easy to draw the conclusion that 'if they'd just X, we'd have won!' That, however, has nothing to do with your Elo and I'll explain to you why.

Elo (not ELO, it's a last name) is not 'made for 1v1'. It's not 'made for' anything. It's a system for representing in a point value the results of a 1/0 (win/loss for example) contest. 1v1, 100 v 100, it doesn't matter. The only difference how many people involved in each match creates is how many total matches you need to play to get a comparatively accurate reading. To refine it more simply in a 12 v 12 environment you need to play 11 times as many matches to get the same results as 1 v 1.

The other 11 people on your team are irrelevant. Completely and totally and absolutely 100% irrelevant in the aggregate measure of win/loss as it relates to you.

Why you ask? That's impossible! They make you lose when otherwise you would win! It totally undercuts your performance! CUZ REASONS!

Every single other person playing MWO is in the same boat though. They are 100% as likely to be in the same boat as you. Over 100, 200, 300 matches those anecdotal events wash out because they happen to everyone.

What doesn't wash out is your performance. How much you help carry your team to a win. That's not reflected in damage or kills or assists, it can be good tactical recommendations. How well you support your other team instead of hanging back and kill-stealing. At the end of the day is your team more likely to win because of how you played. Every single other factor is inherently less reliable because it's contextual. Win/loss is not. You either help your team win or you don't.

Sometimes I take my Jag and throw on 2x LB10x and 4xSRM4s. When my team does well I can go in and pull a score of 120, 130 or so because I get a ton of component destructions, assists and the like and it's not hard to do 800+ damage. Yet I'm not really helping win, am I? I can take an M3 with 2 PPCs and lurk back sniping and get 4 or 5 kills with reasonable consistency but honestly most of those kills were people my team could have finished without my presence. Conversely I can play a D-DC and play human shield for my team, keep up and use my ECM to shelter them when they get attacked and always run back to help allies caught alone by lights. I can finish with ~300 damage, 0-2 kills and maybe 4 assists but have markedly improved my teams odds of winning.

Win/loss is all that matters. Everything else you do, damage, kills, it's ego-fapping.

Elo works, works correctly and with reasonable accuracy will reflect after a good 200 or 300 matches how likely your presence is to help your team win (or lose) a match against comparably skilled opponents.

Now, the matchmaker (which uses Elo to make matches) needs to have pug and premade Elo split and will provide better matches by matching everyone with the same range rather than matching high/low for a target but that's a separate issue.

Elo and measuring win/loss is the only worthwhile and reliable method of ranking players.

Here are some links on why math works and isn't an lie of THE MAN to just keep you down:
Statistics
Probability
Probability sampling

Here are some links on why you don't want to believe that it works:
Anecdotal experience

Confirmation bias

Self-serving bias

Persistence of discredited beliefs

Illusory Correlation

I totally understand that people want to believe that statistical mathematics are wrong or just somehow math doesn't count when it applies to this but it's not wrong and it does count. I could, with enough samples, model not just the odds of your behavior promoting a win or loss for your team but I could predict your political affiliation, religion, familial status (married/unmarried/living at home) and financial stability with enough samples of how you play MW:O.

Elo is accurate and about as correct as possible. Like with any sport it obviously can't predict in the immediacy your mood and likelihood of winning a specific game but taken as an average how likely having you there will promote a win over a loss against a given skill of adversary? Yes, it works like all other probability sampling and statistical analysis works.



The above is Intelligent, well-educated, ignorance.

#83 Joseph Mallan

    ForumWarrior

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 35,216 posts
  • Google+: Link
  • Facebook: Link
  • LocationMallanhold, Furillo

Posted 23 January 2014 - 04:13 AM

View PostBeliall, on 22 January 2014 - 02:01 PM, said:

a Win / Loss elo system is not suited well for any team based game. You can perform flawless on your end and then support your team the best you can and still end up in low elo ranking do to very inept team mates. Its irritating in other games as well in this one.

A team based Elo would work fine I think, IF the team was the same drop after drop. Same players same Mechs Same builds every drop. You would have a sample that has meaning. But, dropping with random players time after time increases our probabilities instead limiting them.

#84 Unleashed3k

    Member

  • PipPipPipPipPipPipPip
  • Death Star
  • Death Star
  • 525 posts
  • LocationGermany

Posted 23 January 2014 - 04:36 AM

View PostSandpit, on 22 January 2014 - 04:49 PM, said:

OR

More players can start taking up responsibility for their own skill level instead of looking to blame any and everything for losing.

Other than the occasional tonnage discrepancy (which incidentally will be resolved once tonnage limits are implemented) the MM does a pretty good job of matching players by skill. Not once have I seen a single player jumping up and down about MM, ELO, premades, weapon balance, etc. say "Yea I was in a match last night and we got stomped. We got stomped because we played like {Scrap} as a team"

I DO, however, see a lot of those players talking about how they got 3 kills and did 900 damage and still lost. That shows me they're less concerned about teamwork and more about their personal KDR. never mind that they got that 3 kills and 900 damage because they waited in the rear launching lrms at random mechs (instead of focusing on a target), then rolled up after the match has become 12v1 and took out a couple of heavily damaged mechs (that their teammates had incidentally chewed up in the earlier melee) and then claim they were great while the rest of the team "sucked"



Yeah, here's another 900dmg+ scenario: You drop with 1-2 friends, make a rush on enemy, your friends and you do 7 or more kills, over 1100dmg+ and 4 kills for you and after U died by the rest cause ur team sleapt somewhere else or ran away, U can watch the rest get shot by a bunch of damaged mechs simply cause u've been thrown together with complete fools.
Elor rating for this game as i understand the Elo would be: My friends / I / rest of team// ALL suck and get a - in Elo, that can't be the right way, as U can try what u want, 60-80% of the random matches ur team just won't listen to play a tactic together.

and on the other hand the noobs that got bashed at the begin of the round and only won because of few good on their team, they get a + for elo ratings, actually ppl that do 37dmg in a ddc and mostly shot the rest of their damage into allied backs get a positive elo...

k/d rates could only improve the system, if there would be a difference in case i killd highend elo or on the other side an absolute noob that just played his first games cause they don't have the chance to stand a mastered Ilya (for example) in any way with any of the trialmechs, face to face. (noob elo shouldnt drop much from high-elo players / 2 noobs shoot each other same rating like 2 pro kill each other/ pro kill low, less + for the pro, less - for the low/ Low kill pro the other way round of course)

dmg/minute; points/minute; (both class dependant) accuracy; etc this would be things that could help improve the matchmaking a lot, or with matchmaking we get the chance to kick bad players and assemble high-skilled lobbys somewhen, i prefer getting bashed from other pro's from time to time or have close wins rather than getting overrunned by premades on low skill pug-games or the other way just overrun the map in 1 walk cause u've been dropped with 2-3other premade groups on your side...

hope this will somewhen get fixed and also shown to the players so we can work on it and maybe improve the system further.

Edited by Battlestar3k, 23 January 2014 - 04:51 AM.


#85 Joseph Mallan

    ForumWarrior

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 35,216 posts
  • Google+: Link
  • Facebook: Link
  • LocationMallanhold, Furillo

Posted 23 January 2014 - 04:52 AM

View Postlockwoodx, on 22 January 2014 - 04:49 PM, said:



One of these days I'd love for you to make a second account just to experience how bad the average joe has it when it comes to how ELO currently works. PGI decided to cater to the elitists again by speeding up their queue times and dropping them in with low ELO players. It's a blunder that will cost them fresh blood if not corrected soon.

I have 4 Alts. All with varying degrees of limitations I personally placed on the account. The more flexible I was with my play style the more successful the Account is. Anton Shiningstar (a FedRat), Is the most successful, Followed by My DCMS Xando Parapasu, I use primarily mobile Mediums and Heavies... Got a good AC10 Pult and a Shadow Hawk I am not ashamed of! My Clanner Is modeled after a Scorpion MechWarrior so mostly Energy weapons and fast moving Lights and Mediums. I have mentioned a few times I am not a good light pilot right? :D

The last one I am not telling but its a Leaguer, and to new t know how it will turn out. :D

View PostAbivard, on 23 January 2014 - 03:46 AM, said:



The above is Intelligent, well-educated, ignorance.

And his first line explains why! :unsure:

Edited by Joseph Mallan, 23 January 2014 - 04:50 AM.


#86 Artgathan

    Member

  • PipPipPipPipPipPipPipPip
  • Knight Errant
  • Knight Errant
  • 1,764 posts

Posted 23 January 2014 - 05:53 AM

View PostJoseph Mallan, on 23 January 2014 - 04:13 AM, said:

A team based Elo would work fine I think, IF the team was the same drop after drop. Same players same Mechs Same builds every drop. You would have a sample that has meaning. But, dropping with random players time after time increases our probabilities instead limiting them.


This would actually make each individual's Elo impossible to determine - but it would allow you to accurately determine the Elo for the entire team.

I'm going to take a moment to explain how a W/L ranking system works.

Imagine you have a pool of 20 players, which contains a normal distribution of "skill". They will randomly be formed into teams of 10 and played against each other. Every time a team wins, every player on the team gets +1 to their personal score, every time a team loses, every player on the team gets -1 to their personal score. After every match the teams will be re-formed. They will play 100 matches against each other.

At the end of these 100 matches, we look at everyone's individual scores. What we'll see is that the "good" players will have high positive scores (since their teams tend to win when they are present), the average players will have scores centered around 0 (since they can't reliably make their team win or lose), and the "bad" players will have high negative scores (since their teams tend to lose when they are present).

Most of the players will be "average", with a smaller proportion of the group comprised of "good" and "bad" players.

This is why a W/L ratio can be used to determine "skill".

#87 Joseph Mallan

    ForumWarrior

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 35,216 posts
  • Google+: Link
  • Facebook: Link
  • LocationMallanhold, Furillo

Posted 23 January 2014 - 06:34 AM

View PostArtgathan, on 23 January 2014 - 05:53 AM, said:


This would actually make each individual's Elo impossible to determine - but it would allow you to accurately determine the Elo for the entire team.

I'm going to take a moment to explain how a W/L ranking system works.

Imagine you have a pool of 20 players, which contains a normal distribution of "skill". They will randomly be formed into teams of 10 and played against each other. Every time a team wins, every player on the team gets +1 to their personal score, every time a team loses, every player on the team gets -1 to their personal score. After every match the teams will be re-formed. They will play 100 matches against each other.

At the end of these 100 matches, we look at everyone's individual scores. What we'll see is that the "good" players will have high positive scores (since their teams tend to win when they are present), the average players will have scores centered around 0 (since they can't reliably make their team win or lose), and the "bad" players will have high negative scores (since their teams tend to lose when they are present).

Most of the players will be "average", with a smaller proportion of the group comprised of "good" and "bad" players.

This is why a W/L ratio can be used to determine "skill".

Well if I am part of a team I should not be so worried about my personal kill count... The teams victory should be most important... So teh results of the 3 high score players and the 3 low score players are more important than the 14 average Joe's? So like I said W/L is no real metric for an individual player. You do not create statistical control from the data of "fliers". Also the better TEAM will always win.

#88 Artgathan

    Member

  • PipPipPipPipPipPipPipPip
  • Knight Errant
  • Knight Errant
  • 1,764 posts

Posted 23 January 2014 - 07:10 AM

View PostJoseph Mallan, on 23 January 2014 - 06:34 AM, said:

Well if I am part of a team I should not be so worried about my personal kill count... The teams victory should be most important... So teh results of the 3 high score players and the 3 low score players are more important than the 14 average Joe's? So like I said W/L is no real metric for an individual player. You do not create statistical control from the data of "fliers". Also the better TEAM will always win.


No, everyone's results count. Each player's "score" gives an indication of their "skill" in a team environment. IE: If you stacked a team with the top scoring players, you could reasonably expect this team to win against a team stacked with players with low scores.

Each player's "score" in my example is indicative of how "well" they perform in a team environment. The W/L metric behaves similarly on MW:O - its indicative of your ability to fight in a team setting. It could very well be true that someone is a great 1v1 player, but has a terrible W/L because they can't perform well in a team environment (consider LeBron James as a Cavalier).

#89 Joseph Mallan

    ForumWarrior

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 35,216 posts
  • Google+: Link
  • Facebook: Link
  • LocationMallanhold, Furillo

Posted 23 January 2014 - 07:22 AM

View PostArtgathan, on 23 January 2014 - 07:10 AM, said:


No, everyone's results count. Each player's "score" gives an indication of their "skill" in a team environment. IE: If you stacked a team with the top scoring players, you could reasonably expect this team to win against a team stacked with players with low scores.

Each player's "score" in my example is indicative of how "well" they perform in a team environment. The W/L metric behaves similarly on MW:O - its indicative of your ability to fight in a team setting. It could very well be true that someone is a great 1v1 player, but has a terrible W/L because they can't perform well in a team environment (consider LeBron James as a Cavalier).

My (F)Atlas W/L 1.18 (100/85) , K/D 0.68 (86/127)

My D-DC W/L 1.10 (128/116) K/D 1.11 (165/148) So the Mech I do better in, has a lower win loss! So I don't think your premise holds up under scrutiny.

#90 Artgathan

    Member

  • PipPipPipPipPipPipPipPip
  • Knight Errant
  • Knight Errant
  • 1,764 posts

Posted 23 January 2014 - 07:27 AM

View PostJoseph Mallan, on 23 January 2014 - 07:22 AM, said:

My (F)Atlas W/L 1.18 (100/85) , K/D 0.68 (86/127)

My D-DC W/L 1.10 (128/116) K/D 1.11 (165/148) So the Mech I do better in, has a lower win loss! So I don't think your premise holds up under scrutiny.


But bear in mind that your personal success does not necessarily equate to your team's success. Elo is measuring your ability to fight as a team player. Sure, you could very well be an extremely dominant player - 11 kills every match, 1000+ damage - but if you do that at the total expense of your team (such as by only appearing at the end of the match to mop up) as the last enemy kills you, you're not performing well in a team environment.

#91 Werewolf486 ScorpS

    Member

  • PipPipPipPipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 1,271 posts
  • LocationSinsinnati Ohio

Posted 23 January 2014 - 07:33 AM

Wait, there's an ELO system at work here? I thought it was just some random drop system that puts you with bad players to punish how well you did the previous drop(s) and makes your game experience miserable. That or it's Russ sitting at his desk making you pay for not spending enough money to pad his pockets.

#92 Joseph Mallan

    ForumWarrior

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 35,216 posts
  • Google+: Link
  • Facebook: Link
  • LocationMallanhold, Furillo

Posted 23 January 2014 - 07:37 AM

View PostArtgathan, on 23 January 2014 - 07:27 AM, said:


But bear in mind that your personal success does not necessarily equate to your team's success. Elo is measuring your ability to fight as a team player. Sure, you could very well be an extremely dominant player - 11 kills every match, 1000+ damage - but if you do that at the total expense of your team (such as by only appearing at the end of the match to mop up) as the last enemy kills you, you're not performing well in a team environment.

Doesn't apply to me. A new example please. You are saying that good players will win more, my Win loss is not an indication of this.

I would be interested in seeing how many assists each Atlas has though to see which one has the better team player stats! :D

Edited by Joseph Mallan, 23 January 2014 - 07:38 AM.


#93 FunkyFritter

    Member

  • PipPipPipPipPipPip
  • 459 posts

Posted 23 January 2014 - 07:44 AM

ELO can work in team games. It doesn't work very well here because the population is too small to provide well balanced teams on a consistent basis.

#94 Artgathan

    Member

  • PipPipPipPipPipPipPipPip
  • Knight Errant
  • Knight Errant
  • 1,764 posts

Posted 23 January 2014 - 07:45 AM

I get the impression that there's a lot of confusion concerning what Elo actually does in MW:O. Since it is applied to each player individually people tend to think that it measure their individual performance - but this is not what it does. It measures how well you perform in a team setting. Consider this example:

Bob is terrible at MW:O. He cannot pilot a mech to save his life. BUT! Bob has a high Elo score. How is this possible? Well it turns out that Bob is a great commander (and charismatic enough to be able to get people to listen to him). Whenever Bob takes command he can guide his team to victory, and that's exactly what Bob does. Therefore, because Bob has a high W/L ratio (due to his commanding skills) he will have a high Elo rating as well.

In essence, Elo measures how much of a "force multiplier" you are on the field. If you have a "high" Elo you have something that allows you to make your team win more on average. This something can be anything: maybe you can get 12 kills blindfolded. Maybe you just make a great distraction that allows your team to get a few easy kills, which then lets them snowball. Maybe you're a great commander. You just need to have some skill that makes you become a "force multiplier".

#95 Joseph Mallan

    ForumWarrior

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 35,216 posts
  • Google+: Link
  • Facebook: Link
  • LocationMallanhold, Furillo

Posted 23 January 2014 - 07:45 AM

View PostFunkyFritter, on 23 January 2014 - 07:44 AM, said:

ELO can work in team games. It doesn't work very well here because the population is too small to provide well balanced teams on a consistent basis.
Your data would go a long way in proving your position sir. :unsure:

Where is Bob? I'd like to meet him! :D

Edited by Joseph Mallan, 23 January 2014 - 07:48 AM.


#96 Artgathan

    Member

  • PipPipPipPipPipPipPipPip
  • Knight Errant
  • Knight Errant
  • 1,764 posts

Posted 23 January 2014 - 07:47 AM

View PostJoseph Mallan, on 23 January 2014 - 07:37 AM, said:

Doesn't apply to me. A new example please. You are saying that good players will win more, my Win loss is not an indication of this.

I would be interested in seeing how many assists each Atlas has though to see which one has the better team player stats! :D


But it does apply to you - in one Atlas you help the team win more than the other. It doesn't matter what you're actually doing to help the team win more. Your individual performance doesn't matter - it's what you contribute to the team. Elo doesn't measure personal skill. It measures what you bring to the team.

#97 Unleashed3k

    Member

  • PipPipPipPipPipPipPip
  • Death Star
  • Death Star
  • 525 posts
  • LocationGermany

Posted 23 January 2014 - 07:50 AM

yeah, wtb all the thouand assists, are they counted? how many headshot? how many legged, why is it so hard to make a detailed statistics screen like its implemented in bf/cod/ps2/wg/aa/moh/eve/etc where u nearly can see any of ur actions? and in most common games u can see everybodys stats if u search their name. so there would be natural stop to many whiners when u can see why they've failed anyway and that's often not an op gun/mech that needs a nerf, in the most certain case a playerbuff would be the only chance for their success

#98 Joseph Mallan

    ForumWarrior

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 35,216 posts
  • Google+: Link
  • Facebook: Link
  • LocationMallanhold, Furillo

Posted 23 January 2014 - 07:50 AM

View PostArtgathan, on 23 January 2014 - 07:47 AM, said:


But it does apply to you - in one Atlas you help the team win more than the other. It doesn't matter what you're actually doing to help the team win more. Your individual performance doesn't matter - it's what you contribute to the team. Elo doesn't measure personal skill. It measures what you bring to the team.

The Atlas I am dead in almost twice as often, helps my team more... Are you reading that? :D

#99 Unleashed3k

    Member

  • PipPipPipPipPipPipPip
  • Death Star
  • Death Star
  • 525 posts
  • LocationGermany

Posted 23 January 2014 - 08:01 AM

View PostFunkyFritter, on 23 January 2014 - 07:44 AM, said:

ELO can work in team games. It doesn't work very well here because the population is too small to provide well balanced teams on a consistent basis.


that is the problem of random matchmaking, it screws too many people. if u had pro lobbys and lobbys for noobs only etc or could set own servers as a clan there would be much more constance. not to mention the stupid pings u have in europe, especially here in germany with 32k kb downstream and high-end pc never under 100-110pings, and u play vs canadians/us with under 40... what about that? very fair for cw, i remember times when >20ping+ was much, but here its 3 times higher then average north american... weren't we supposed to have the 21st century already started its 14th year? 20th? 19th?.. =(

#100 Artgathan

    Member

  • PipPipPipPipPipPipPipPip
  • Knight Errant
  • Knight Errant
  • 1,764 posts

Posted 23 January 2014 - 08:07 AM

View PostJoseph Mallan, on 23 January 2014 - 07:50 AM, said:

The Atlas I am dead in almost twice as often, helps my team more... Are you reading that? :D


Yes. I know it seems counter-intuitive, but it apparently is true (based on the stats you presented). Consider an infantry section attack:

Your section is taking enemy fire, but you can't locate the enemy. Your section commander orders you to take a bound so that they can determine the enemy's location. During your bound you get shot, but the rest of your section is able to locate and destroy the enemy. Did your section win? Yes. Does it suck to be you in this scenario? Yes.

Helping your team doesn't always mean "being a bada**". Sometimes the way you help is more subtle.





14 user(s) are reading this topic

0 members, 14 guests, 0 anonymous users