58 replies to this topic

[Trip Hammer]

I've read as many of the posts on ELO as I could find with the search tool and it has left me scratching my head.

Why is the PGI so fixated on Win/Loss, as opposed to a more balanced view that includes actual battle stats?

The reason that I ask is not for myself so much but for a friend (and no I'm not talking about myself here) in game that is a much better player than I am. I know that he is a better player than myself for the following reasons.

• better battlefield awareness
  
• consistent damage output and enemy mechs damaged
  
• better engagement skills
  
• longer time in matches (usually one of the last 2 people in match)
  
• Works with a group consistently
  
• doesn't make a lot of stupid mistakes (unlike myself)

So in general he is a better player and I have a great deal of respect for his skill.

With that said, his win loss has been going down consistently here lately. This would indicate that his ELO is likely dropping due to the large number of losses he is experiencing. How is this a reflection of what kind of player he is?

So again I ask, does anyone have any insight as to why PGI weighs their ELO system so heavily with win/loss stats?

[Egomane]

The Faceless, on 15 July 2014 - 06:06 AM, said:

Why is the PGI so fixated on Win/Loss, as opposed to a more balanced view that includes actual battle stats?

What kind of stats?

The Faceless, on 15 July 2014 - 06:06 AM, said:

better battlefield awareness

How would you track that?

The Faceless, on 15 July 2014 - 06:06 AM, said:

consistent damage output and enemy mechs damaged

Which in the end will contribute to the win.

The Faceless, on 15 July 2014 - 06:06 AM, said:

better engagement skills

Which again will contribute to the win.

The Faceless, on 15 July 2014 - 06:06 AM, said:

longer time in matches (usually one of the last 2 people in match)

Survivability might be usefull stat, but can be gamed.

The Faceless, on 15 July 2014 - 06:06 AM, said:

Works with a group consistently

How do you track that?

The Faceless, on 15 July 2014 - 06:06 AM, said:

doesn't make a lot of stupid mistakes (unlike myself)

How do you track that?

So what other stats could you mean?

Damage done - More accurate players tend to deal less damage then spray and pray players. I'd say the former is better. How do you discern a good shot with a spray and pray one who had bad luck and died early.

Number of Kills - Will lead to kill stealing or unwilligness to kill, dependent if you want to artificially heighten or lower your rating. In other words... it can be gamed. And what about those who are simply unlucky to seldom get a kill shot. I know I work my behind off some days only to be rewarded with assists only.

Accuracy - What is better? One shot that hits, for 100 % accuracy or 100 shots where only 10 % hit but deal overall more damage?

Before the next argument comes up, that says: "But if you combine some stats, then..." It only makes it more complicated to create a working formula.

In the end, everything you do during a match, or maybe during hundreds of matches, it all contributes to your wins and losses in the end. It is the one stat that all others are contributing to.

[Lynx7725]

The Faceless, on 15 July 2014 - 06:06 AM, said:

With that said, his win loss has been going down consistently here lately. This would indicate that his ELO is likely dropping due to the large number of losses he is experiencing. How is this a reflection of what kind of player he is?

Correction. Your friend's ELO will drop only if he loses games he is expected to win. If Maggie the Matchmaker sets your friend up in games he is expected to lose, and he loses, his ELO goes nowhere.

[Trip Hammer]

Egomane, on 15 July 2014 - 06:22 AM, said:

So what other stats could you mean?

Damage done - More accurate players tend to deal less damage then spray and pray players. I'd say the former is better. How do you discern a good shot with a spray and pray one who had bad luck and died early.

Number of Kills - Will lead to kill stealing or unwilligness to kill, dependent if you want to artificially heighten or lower your rating. In other words... it can be gamed. And what about those who are simply unlucky to seldom get a kill shot. I know I work my behind off some days only to be rewarded with assists only.

Accuracy - What is better? One shot that hits, for 100 % accuracy or 100 shots where only 10 % hit but deal overall more damage?

Before the next argument comes up, that says: "But if you combine some stats, then..." It only makes it more complicated to create a working formula.

In the end, everything you do during a match, or maybe during hundreds of matches, it all contributes to your wins and losses in the end. It is the one stat that all others are contributing to.

I agree that most of what I was praising him for would be difficult to track. I was really just putting that in to try to show why I believe he is a good player (I'm not trying to say he is a top tier player or even in the top 20%).

Looking at his overall stats for damage, weapon accuracy, kills and kill assists you might deduce that he is a good player. But if you look at his win/loss that's not the case. As I've seen posted many times there are 24 players on the map and they all contribute to the win loss, not just one person. (although I have seen a rare match where that was what happened) So how can win/loss alone count for that much of a person's ELO?

Lynx7725, on 15 July 2014 - 06:25 AM, said:

Correction. Your friend's ELO will drop only if he loses games he is expected to win. If Maggie the Matchmaker sets your friend up in games he is expected to lose, and he loses, his ELO goes nowhere.

Sorry, I miss stated that. But speaking to that point. Hypothetically speaking, if you have a player that is truly a good player that does well consistently but looses time and time again it could suggest that the matchmaker is putting him on teams it expects to loose an awful lot. Something that could lead to a lot of frustration for the player through no actual fault of his own. That could either just be bad luck of the draw or some other piece of the matchmaker that I'm missing.

Either way it doesn't really make a lot of difference as it seems unlikely that PGI is going to be making a lot of big changes to MM.

Thanks guys for the discussion though. Nice to see some civility here on the forums!

[Clit Beastwood]

The Faceless, on 15 July 2014 - 06:06 AM, said:

I've read as many of the posts on ELO as I could find with the search tool and it has left me scratching my head.

Why is the PGI so fixated on Win/Loss, as opposed to a more balanced view that includes actual battle stats?

The reason that I ask is not for myself so much but for a friend (and no I'm not talking about myself here) in game that is a much better player than I am. I know that he is a better player than myself for the following reasons.

• better battlefield awareness
  
• consistent damage output and enemy mechs damaged
  
• better engagement skills
  
• longer time in matches (usually one of the last 2 people in match)
  
• Works with a group consistently
  
• doesn't make a lot of stupid mistakes (unlike myself)

So in general he is a better player and I have a great deal of respect for his skill.


With that said, his win loss has been going down consistently here lately. This would indicate that his ELO is likely dropping due to the large number of losses he is experiencing. How is this a reflection of what kind of player he is?

So again I ask, does anyone have any insight as to why PGI weighs their ELO system so heavily with win/loss stats?

Don't think about ELO - you'll be happier in the long run. Just read the first page or two of posts to see what ELO obsession does to people.

[Clit Beastwood]

The Faceless, on 15 July 2014 - 07:54 AM, said:

Looking at his overall stats for damage, weapon accuracy, kills and kill assists you might deduce that he is a good player. But if you look at his win/loss that's not the case. As I've seen posted many times there are 24 players on the map and they all contribute to the win loss, not just one person. (although I have seen a rare match where that was what happened) So how can win/loss alone count for that much of a person's ELO?

The value of a teammate goes beyond damage, kills, assists, etc. Is this someone that will die in a light to protect their missile boats? Will this person go deep behind enemy lines, sacrificing doing damage, etc. to place a UAV? There are many factors that affect the outcome of a drop. Many many players fancy themselves "good", usually in posts complaining about how much damage they did while their team died. If you achieve huge numbers, but your team is dying, you are ineffective as a teammate. It does not make someone "good" to turn their blinders on and focus on one enemy at a time while your team is dying. To win often, it takes people working in concert toward the common goal of victory.

[Lynx7725]

The Faceless, on 15 July 2014 - 07:54 AM, said:

Sorry, I miss stated that. But speaking to that point. Hypothetically speaking, if you have a player that is truly a good player that does well consistently but looses time and time again it could suggest that the matchmaker is putting him on teams it expects to loose an awful lot. Something that could lead to a lot of frustration for the player through no actual fault of his own. That could either just be bad luck of the draw or some other piece of the matchmaker that I'm missing.

Either way it doesn't really make a lot of difference as it seems unlikely that PGI is going to be making a lot of big changes to MM.

Maggie's constrained by the pool of player logged in at any one time, so she's a bit handicapped too. Time zone difference does bring different levels of players, so if you are unlucky, you can end up with a whole bunch of.. not so skilled players. Yes, it is frustrating. That's why nowadays it's better to group up, but solo games are still possible.

On the contrary, PGI is tinkering a lot with the MM. It's just that there are multiple facets to matchmaking. ELO by itself is frankly, IMO a minor component and one that's only really relevant in the mid to long term.

[Heffay]

Winning isn't everything. It's the only thing.

If you win often enough, you'll end up fighting other people who win as much as you do. That will ensure better matches for everyone.

[ScarecrowES]

Suffice it to say, but Elo is completely useless as a skill system for individuals in team play. In fact, it has never been designed to account for anything but one individual (or complete team) against another individual (or complete team). Without a way to accurately track what effect each player is having on the outcome of a match, it's impossible to rate player skill. As it stands now, you could have the world's best MWO player on a team of rubes, which loses, and even if that player put out 11 kills and 1000 damage, his Elo rating will still go down.

As for how you would go about tracking how useful a player has been to a team, or how much he contributed to the outcome of a match, you need only look at pretty much every other modern shooter out there. The Battlefield series is particularly good at accurately predicting player skill, and accounts for literally hundreds of stats per player per match. It can, and has, been done.

[Heffay]

ScarecrowES, on 15 July 2014 - 08:25 AM, said:

Suffice it to say, but Elo is completely useless as a skill system for individuals in team play. In fact, it has never been designed to account for anything but one individual (or complete team) against another individual (or complete team).

This isn't true at all. While it was originally created for chess, the math behind it supports rating all kinds of activities, including team games.

Just because things were designed for a particular purpose doesn't mean it can be used in a variety of different ways. Look at lasers for example.

[Odins Fist]

Elo can be your best buddy, or your Nemesis..

Most people claim it's their Nemesis.

[Heffay]

Odins Fist, on 15 July 2014 - 08:44 AM, said:

Elo can be your best buddy, or your Nemesis..

Most people claim it's their Nemesis.

People who hate Elo don't understand Elo.

[Bhael Fire]

Egomane, on 15 July 2014 - 06:22 AM, said:

It is the one stat that all others are contributing to.

And that's exactly why it's inaccurate as far as measuring an individual's skill.

KDR is far more accurate of stat than W/L as far as determining player skill. I know you mentioned that it can be "gamed" but honestly, anyone that intentionally gets fewer kills in one match just so they can get some kills another is only hurting themselves. Spinning their wheels, as they say.

Between KDR, Assists, and W/L you get a far more accurate picture of the player's individual skill.

[Roadkill]

Lynx7725, on 15 July 2014 - 06:25 AM, said:

Correction. Your friend's ELO will drop only if he loses games he is expected to win. If Maggie the Matchmaker sets your friend up in games he is expected to lose, and he loses, his ELO goes nowhere.

Yep, and that's why PGI's rankings are broken.

That's not Elo. In a proper Elo system, your ranking changes after every match, win or lose, whether the expected result occurred or not. It changes less if the expected happens, but it's a linear progression and not a cliff as PGI has it set up.

The issue that PGI's system causes is that it takes longer for your ranking to reach your actual skill level. It also encourages streaks of wins/losses more than a proper Elo system would.

[nehebkau]

There are just so many factors that can affect one's win/loss that it's not good a good indicator of anything, neither is KDR nor is match damage. Win Loss, is probably going to be higher for people in dedicated groups when compared to pug-only players. (Though with the changes to group only and solo only queues that will change) Likewise, KDR can be easily tweaked by just using a high alpha mech, hiding until the end of the match and blasting damaged mechs. With players of equal skill, match damage is going to be dependent more on what type of mech used (i.e. how much firepower) and not on your skill. In fact, a more skilled pilot will often have lower damage. (Why? A skilled pilot is going to need to take fewer shots to kill a target since they are aiming for and hit their target's weak points).

I think ELO is kind of a 'best we can do' approach but its not going to be perfect and to expect that it will be perfect is unreasonable.

My opinion, ignore your stats. Anyone who 'brags' about their stats -- ignore them because the current stats are so easy to manipulate.

Edited by nehebkau, 15 July 2014 - 09:15 AM.

[Roadkill]

Bhael Fire, on 15 July 2014 - 08:56 AM, said:

And that's exactly why it's inaccurate as far as measuring an individual's skill.

Elo isn't a ranking of player skill. Whatever gave you the idea that it is?

If my Elo ranking is higher than yours, it doesn't mean I'm more skilled. It means I'm more likely to win. I realize that's a bit of a brain twister, but keep reading...

Quote

Between KDR, Assists, and W/L you get a far more accurate picture of the player's individual skill.

While arguably true, this is also irrelevant as the point of Elo rankings is not to rank players by skill. At least not "skill" in the way that most people use it.

Elo ranks how well you contribute to the win, not how well you pilot your Mech. Just as one example, you might not be able to hit the broad side of a Dire Wolf at 50 meters, but if you're very very good at directing and positioning your team you might win consistently without ever doing a single point of damage. In that case, you'd have a high Elo rating because you're good at winning.

KDR, Assists, damage done, all that crap... can't accurately measure your net contribution to the team. That's what Elo measures.

Because when you get right down to it, wins and losses are the only things that matter. KDR, Assists, damage, etc, are only the means to the end, and as such can only partially reflect on your total contribution. Elo boils it down to the simplest and ultimately the most accurate measurement of your net contribution - wins and losses vs opponents.

[DAYLEET]

ELO can't and never will work in an environment thats not 1on1 or at least has very few players. 12vs12 has too many factors.
While ELO makes lots of sense in chess, it just does not in a fps, it sounds cool, gives an imaginary goal to the kids on console and thats that. Why it went across to the PC fps i dont know. Have it in a dota game or starcraft game but not a 24 player game.

[Bhael Fire]

nehebkau, on 15 July 2014 - 09:14 AM, said:

Likewise, KDR can be easily tweaked by just using a high alpha mech, hiding until the end of the match and blasting damaged mechs.

You not seeing the obvious error in that logic; if you somehow manage to get your KDR up high enough...you are going to be facing others with an equally high KDR. There's no scenario where a high KDR comes out on top when you start ranking players on the KDR. The more you get kills the more you face others that get kills.

If you intentionally gimp yourself to get fewer kills in an attempt to face lower skill players, you're just hurting yourself and going to end up at the same place eventually, while at the same time getting crappy end of match rewards. There's no running from it. There's no bargaining or reasoning with it...when player skill is based on KDR.

Roadkill, on 15 July 2014 - 09:22 AM, said:

Elo isn't a ranking of player skill. Whatever gave you the idea that it is?

That's why I said it needs to go. Matchmaker needs to be taking individual player skill into account, not just their W/L record.

Roadkill, on 15 July 2014 - 09:22 AM, said:

Because when you get right down to it, wins and losses are the only things that matter.

And killing is often the best way to win in this game. The way this game is set up, the more efficient killers tend to win more.

[DAYLEET]

Bhael Fire, on 15 July 2014 - 09:25 AM, said:

Matchmaker needs to be taking individual player skill into account, not just their W/L record.

Individual player skills in a 1 on 1 scenario translate directly into win, how does it work in a pug game where you also depend on 11 other guy on your team?

[Koniks]

Heffay, on 15 July 2014 - 08:39 AM, said:

This isn't true at all. While it was originally created for chess, the math behind it supports rating all kinds of activities, including team games.

Right. Elo works because each player is the common variable in every match they play. The edge case where Elo potentially fails is actually when a group of players earned their rating by playing exclusively as a team. If one of the players was then grouped in a random team against random opponents, Elo might not properly account for them. Their rating will effectively be the aggregate of their original team but not individualized for the player's own likelihood to win.

Edited by Mizeur, 15 July 2014 - 09:35 AM.