277 replies to this topic

[Krzysztof z Bagien]

Hauser, on 20 April 2013 - 03:59 AM, said:

Looking at those graphs I conclude the following:

• The smallest variation between individual bars appears to be slightly less then one pixel. The number of pixels between the height markers is 86 so it would be reasonable to assume that the height markers are placed at the hundreds.
  
• Each elo value appears to be plotted as its own bucket. There are 58 pixels per 100 buckets.
  
• So for every 10.000 players we have 4988 pixels. Or ~2 players per pixel.

I applied a threshold to separate the graphs out and counted the blue pixels.





The total players graph covers ~50k pixels or ~100k players

The graph of players with more then 50 games covers ~25k pixels or ~50k players.

And that would match rough estimates of how big the playerbase is made by some of the guys.

I still have this concern and keep asking this, but nobody seems to bother answering:
how can Elo system properly rate individual players in a team game, when it only accounts wins/loses of entire team (which composition is random in most cases)? How can you extract any data about individual player's performance from that, without taking into account any other factors?
Or am I missing something really important? If that's the case please enlighten me.

Edited by Krzysztof z Bagien, 20 April 2013 - 07:11 AM.

[Stoicblitzer]

A friend of mine who finished 2nd in the first pgi tournament solo dropped for 2 hours last night and lost every single game. predictably, he rage quit. working as intended™.

[Imagine Dragons]

Ya know... before this fix, matches seemed wildly imbalance, were one side usually ended up curbstomping the other...

Now i've been in matches were the tables been turned up to 3 times...

I suppose thats much much better result lol!

[Redoxin]

Krzysztof z Bagien, on 20 April 2013 - 07:10 AM, said:

And that would match rough estimates of how big the playerbase is made by some of the guys.

I still have this concern and keep asking this, but nobody seems to bother answering:
how can Elo system properly rate individual players in a team game, when it only accounts wins/loses of entire team (which composition is random in most cases)? How can you extract any data about individual player's performance from that, without taking into account any other factors?
Or am I missing something really important? If that's the case please enlighten me.

The only thing that influences Elo is win or loss. If you are a good player (better than the average of payers on the filed) youn increase the chance of winning of your team. if you are worse, you decrease it. Of course you cant make your team win every game, but the more games you play the more accurate your Elo gets. It works well in other team games.
This is the best way to measure skill, because it it only measures your ability to increase the win chances of your team. And this is what really matters, and not how much kills you get or whatever.

The travesty of course is that we are not allowed to know the most important stat of all (our Elo), while we get all the fluff information like damage done etc.

Edited by Redoxin, 20 April 2013 - 09:37 AM.

[Redoxin]

Stoicblitzer, on 20 April 2013 - 07:15 AM, said:

A friend of mine who finished 2nd in the first pgi tournament solo dropped for 2 hours last night and lost every single game. predictably, he rage quit. working as intended™.

Working as intended would be if everybody wins 50% of his games on average. So if your friend won more than 50% of his games before, getting tougher opponents now is indeed working as intended.
That said, I am still winning significantly more than 50%. I guess because of weight class matching the system is still very flexibel with the Elo matching, so that you get opponents with too low Elo.

Edited by Redoxin, 20 April 2013 - 09:42 AM.

[Hauser]

Redoxin, on 20 April 2013 - 09:37 AM, said:

The travesty of course is that we are not allowed to know the most important stat of all (our Elo), while we get all the fluff information like damage done etc.

What do you need to know it for? What can you do with it?

[Krzysztof z Bagien]

Redoxin, on 20 April 2013 - 09:37 AM, said:

The only thing that influences Elo is win or loss. If you are a good player (better than the average of payers on the filed) youn increase the chance of winning of your team. if you are worse, you decrease it. Of course you cant make your team win every game, but the more games you play the more accurate your Elo gets. It works well in other team games.
This is the best way to measure skill, because it it only measures your ability to increase the win chances of your team. And this is what really matters, and not how much kills you get or whatever.

The travesty of course is that we are not allowed to know the most important stat of all (our Elo), while we get all the fluff information like damage done etc.

It works in games it was designed for (like chess), basically 1vs1 games. AFAIK the only multiplayer team game that uses Elo system are League of Legends and MWO.
Elo system is also used in team games, like football, but to rate whole teams not individual players, and team compisition doesn't change much or even at all during one season. How can you rate individual player's performance based solely on win or lose? I can do nothing at all and my team can still win, on the other hand I can be the best player in the universe, but when my team does nothing I'm unable to win by myself, even if I'm playing against weak players.

I used this analogy somewhere else:
An average football team won against weaker opponent, then lost against stronger opponent, and then won against stronger opponent. How should goalkeeper's rating change?

[Gideon Grey]

Something is still horribly wrong. In the last 6 matches, 5 of 6 have ended in a pretty unbalanced stomp (8-1, 8-2, 2-8, etc) one way or the other (we won 2, lost 3) and only one showed anything resembling a decent game.

Also, it is clear that weightmatching hasn't been properly adjusted yet at all? Last match the other side had an obvious premade of two 3Ls a DDC and Highlander plus another assorted lance, our side had no lights (closest was me in my Cicada, getting shredded by the tag team Ravens).

How hard is this, seriously?

[PhoenixFire55]

Elo as it is now is being stupid, because win/loss ratio it is based off means NOTHING. Your smart curve only shows that 90% of people win about 50% of their games, meaning that its just being a random factor of haveing good/bad team. If you want to really evaluate players skill you have to use their game score related to their team average score, thus better players will always gain Elo even if they end up on a horrible team.

But, in terms of equal matchmaking Elo alone won't work. You NEED mech types matching to be no.1 priority, because if you put 4 lights and 4 assaults vs 4 mediums and 4 heavies, even tho the total weights are equal, 4 lights + 4 assaults will always have a huge advantage, because it has both fast mechs and mechs that can take a lot of punishment, while other team has neither.

Skill matching in games right now is still bad. I see no difference in % of my games ending 8:0, 0:8 compared to closed beta. Uneven weight balance matches occur JUST THE SAME as they did before 'reroll'.

Also... not being able to see your Elo score makes no sense. People who 'farm' their Elo can do it without seeing it, people who don't won't start doing it if they see it. I'm yet to see a sensible answer from anyone in PGI on the question why we can't see our Elo score. How are your new players you care so much about are supposed to tell if they are doing smth right or smth wrong if they don't see their Elo go up/down?

Edited by PhoenixFire55, 20 April 2013 - 10:34 AM.

[Hauser]

Krzysztof z Bagien, on 20 April 2013 - 10:22 AM, said:

An average football team won against weaker opponent, then lost against stronger opponent, and then won against stronger opponent. How should goalkeeper's rating change?

It shouldn't. Both are correct predictions based on their rating.

Now put that keeper in 100 random teams for 100 games against teams constructed in the same fashion. If that keeper is good, he'll win more games, him being the only constant in those random teams.

Edited by Hauser, 20 April 2013 - 10:42 AM.

[Krzysztof z Bagien]

PhoenixFire55, on 20 April 2013 - 10:32 AM, said:

Elo as it is now is being stupid, because win/loss ratio it is based off means NOTHING. Your smart curve only shows that 90% of people win about 50% of their games, meaning that its just being a random factor of haveing good/bad team. If you want to really evaluate players skill you have to use their game score related to their team average score, thus better players will always gain Elo even if they end up on a horrible team.

But, in terms of equal matchmaking Elo alone won't work. You NEED mech types matching to be no.1 priority, because if you put 4 lights and 4 assaults vs 4 mediums and 4 heavies, even tho the total weights are equal, 4 lights + 4 assaults will always have a huge advantage, because it has both fast mechs and mechs that can take a lot of punishment, while other team has neither.

Skill matching in games right now is still bad. I see no difference in % of my games ending 8:0, 0:8 compared to closed beta. Uneven weight balance matches occur JUST THE SAME as they did before 'reroll'.

Also... not being able to see your Elo score makes no sense. People who 'farm' their Elo can do it without seeing it, people who don't won't start doing it if they see it. I'm yet to see a sensible answer from anyone in PGI on the question why we can't see our Elo score. How are your new players you care so much about are supposed to tell if they are doing smth right or smth wrong if they don't see their Elo go up/down?

^^ a beer for that guy for speaking the truth.

Edited by Krzysztof z Bagien, 20 April 2013 - 10:39 AM.

[Krzysztof z Bagien]

Hauser, on 20 April 2013 - 10:37 AM, said:

It shouldn't. Both are correct predictions based on their rating.

Now put that keeper in 100 random teams for 100 games against teams constructed in the same fashion. If that keeper is good, he'll win more games, him being the only constant in those random teams.

Think about his: keeper is horrible, and can't save even once, but the rest of the team is so good that they either won't let opponent score (so keeper has nothing to do during whole match) or they manage to score more goals than the opponent. They will still win, despite keeper is worthless. With current system his rating will go higher with every win, and it should not, as he sucks.
The flaw in using Elo system the way it is used in MWO is that when your performance is so significant that it influences outcome of the game it also means that the rest of your team didn't do anything significant to change the outcome - but their rating will change depending on win/lose they are not responsible for. If you do 90% of the job it means the rest of your team did only 10%. And even if every player contributes to the win (or lose) equally it still is only 1/8 of the whole "job". Moreover, 3 players can be terrible, other 3 can be as good as the previous 3 suck, and their "contribution" will cancel out, leaving 2 remaining players' performance to be deciding factor (of course it's a great simplification).

Edit: you said "both", but there were 3 games in my example. And the other thing is: it is not he goalkeeper who wins a game. It's the whole team.

Edit 2: I'm certain that if you put our goalkeeper in 100 random teams and make those teams play 100 matches against random teams the win/lose ratio would be... purely random, and given quite big sample size should be close to 50/50. Keeper's performance wouldn't be "constant" as he can have better or worse days, maybe can't comunicate with his team because he doesn't speak their language (teams are of random composition after all!) etc. And most important thing: his performance is not so significant it can influence the outcome of every single game. And this is what we see now. I'm certain that what we have here has no difference from random matchmaking.

Edited by Krzysztof z Bagien, 20 April 2013 - 11:28 AM.

[Hauser]

Gideon Grey, on 20 April 2013 - 10:29 AM, said:

Something is still horribly wrong. In the last 6 matches, 5 of 6 have ended in a pretty unbalanced stomp (8-1, 8-2, 2-8, etc) one way or the other (we won 2, lost 3) and only one showed anything resembling a decent game.

It's called combat loss grouping.

After a bit of a back and forth a few members on each team have a red torso and it just takes a pop to make them go. But until that happens they are still contributing fire power. The one team that gets two of those to go they get a big advantage in numbers. Big enough to keep their weakened guys in the rear so the other team can't make a comeback.

PhoenixFire55, on 20 April 2013 - 10:32 AM, said:

Elo as it is now is being stupid, because win/loss ratio it is based off means NOTHING. Your smart curve only shows that 90% of people win about 50% of their games, meaning that its just being a random factor of haveing good/bad team. If you want to really evaluate players skill you have to use their game score related to their team average score, thus better players will always gain Elo even if they end up on a horrible team.

Using wins and losses, not their ratio is fine for elo. It measures the one result that counts.

PhoenixFire55, on 20 April 2013 - 10:32 AM, said:

Also... not being able to see your Elo score makes no sense. People who 'farm' their Elo can do it without seeing it, people who don't won't start doing it if they see it. I'm yet to see a sensible answer from anyone in PGI on the question why we can't see our Elo score. How are your new players you care so much about are supposed to tell if they are doing smth right or smth wrong if they don't see their Elo go up/down?

Why do you want to know your elo? What are you going to do with it?

[Krzysztof z Bagien]

Hauser, on 20 April 2013 - 11:08 AM, said:

Why do you want to know your elo? What are you going to do with it?

The same thing you do with your kill/death ratio or damage dealt by small laser?

[PhoenixFire55]

Hauser, on 20 April 2013 - 11:08 AM, said:

Why do you want to know your elo? What are you going to do with it?

See if I'm going upwards (aka improving myself) or downwards (aka becoming worse)? D'oh!

Hauser, on 20 April 2013 - 11:08 AM, said:

Using wins and losses, not their ratio is fine for elo. It measures the one result that counts.

Win / loss only means your TEAM is better / worse then the other team. Has nothing to do with your skill.

[Hauser]

Krzysztof z Bagien, on 20 April 2013 - 10:53 AM, said:

Think about his: keeper is horrible, and can't save even once, but the rest of the team is so good that they either won't let opponent score (so keeper has nothing to do during whole match) or they manage to score more goals than the opponent. They will still win, despite keeper is worthless. With current system his rating will go higher with every win, and it should not, as he sucks.

The example you've described is that of a a consistent team with a single bad player. This is not the regular example we're considering but it is possible in MWO so I'll entertain it.
So yes if this one guy plays with a good team his rating will go higher and higher. At least up to a point where the match maker matches him and his team against another team that has been winning just as much. Now this poor guy will have to start to work. And as his team mates will find out, he can't play worth a damn.
But this isn't really anybodies problem with the exception of his team mates. In the long run they'll and their opponents be served with a 50/50 chance at winning. Or as close to that as possible in the given population. This satisfies the aims of the matchmaking system.

Krzysztof z Bagien, on 20 April 2013 - 10:53 AM, said:

The flaw in using Elo system the way it is used in MWO is that when your performance is so significant that it influences outcome of the game it also means that the rest of your team didn't do anything significant to change the outcome - but their rating will change depending on win/lose they are not responsible for. If you do 90% of the job it means the rest of your team did only 10%. And even if every player contributes to the win (or lose) equally it still is only 1/8 of the whole "job". Moreover, 3 players can be terrible, other 3 can be as good as the previous 3 suck, and their "contribution" will cancel out, leaving 2 remaining players' performance to be deciding factor (of course it's a great simplification).

Edit: you said "both", but there were 3 games in my example. And the other thing is: it is not he goalkeeper who wins a game. It's the whole team.

Now I get the feeling you're not familiar with statistics. You keep on mentioning rather specific examples that work towards your argument but you don't take into account that with random match making those examples can go either way. The better players can on either team. So I'll reduce this to a very small and simple example.

Imagine a game of tug of war. Each players skill in game can be represented by a number, the higher the number the better they are. This number is the players inherent aptitude at playing tug of war, you can not measure it, it is a hidden value. I do expose them however so I can simulate the game.

We have 3 players, A, B, C and D. They have a power of 1,2,2 and 3 respectively. You can add up these powers to see how strong a team is. The stronger team will win. If the scores are equal it's a 50/50 chance either way.

Given a random matchmaker the possible teams are:

AB | CD = 3 vs 5
BC | AD = 4 vs 4
AC | BD = 3 vs 5

Now if you look at all possible outcomes we get this:

A has a 16.6% chance of winning.
B has a 50% chance of winning.
C has a 50% chance of winning.
D has a 82.6% chance of winning.

B and C are both average players and as such have an average chance of winning and losing. Their elo won't change much.

A and D are a different story though. D is usually carrying the team, unless he is paired with A. A on the other hand is poor player and while he might occasionally be carried by D, he'll still lose most of his games.

As such when using elo, D increase in rank while A goes down. Atleast until the point where their scores are so extreme that winning from A or losing against D yields no change in elo.

Now keep in mind that while the BC | AD match gave both teams a 50% chance of winning, the match maker is not averaging out elo. Rather it searches for people that match a specific rating, which shows you need a good population for elo to work. As such this population is too small to provide good matches, but I'm trying to keep this example simple.

So in summary, if you consistently do the majority of the work for your team (like D does), then you will go up. Perhaps not in every match, but on average you will go up.

edit:

Split up quote, fixed percentages.

Edited by Hauser, 20 April 2013 - 12:01 PM.

[Hauser]

PhoenixFire55, on 20 April 2013 - 11:17 AM, said:

See if I'm going upwards (aka improving myself) or downwards (aka becoming worse)? D'oh!

Win / loss only means your TEAM is better / worse then the other team. Has nothing to do with your skill.

Now you've got me puzzled. On one hand you say that your skill has nothing with the performance of your team, yet on the other hand you want to know this statistic based on the result of your team.

[Krzysztof z Bagien]

So you say that the using Elo system in long term should make players win/lose ratio get closer and closer to 50/50, am I right? Isn't it supposed to be exactly that with purely random matchmaking? So how can you judge if it is working properely?
Why do we even need any rating system? The answer is: not for the sake of rating each and every player (as we can't see our Elo score), but to make player experience better - to make them play against opponents of simmilar skill. And this player experience would be ultimate proof that system works well or not, but it is difficult or even impossible to measure. Instead win/lose ratio is used to show everything works well, but there's no difference between Elo and random at the end.
Trust me, many people smarter than PGI devs spent lots of time and resources to create matchmaking system able to match people with similar skill level, and none of those systems are as simple as Elo's.

And you are not right, I'm quite familiar with how statistics work, I'm just using oversimplified examples to show what I mean, English is not my native language so I find it difficult to articulate my thoughts sometimes.

Edited by Krzysztof z Bagien, 20 April 2013 - 12:07 PM.

[Stoicblitzer]

Redoxin, on 20 April 2013 - 09:42 AM, said:

Working as intended would be if everybody wins 50% of his games on average. So if your friend won more than 50% of his games before, getting tougher opponents now is indeed working as intended.
That said, I am still winning significantly more than 50%. I guess because of weight class matching the system is still very flexibel with the Elo matching, so that you get opponents with too low Elo.

I would say he is one of the top players in the game. It is unfortunate that Elo has very little to do with personal skill and all to do with wins/losses. What is that match score stat they added a few months ago? Seems to me that would be a decent indicator of personal skill. Maybe the mean of the match score stat would be better than the current MM system? I'm just brainstorming. Maybe put (2 high elo + 6 bad elo) vs (2 high elo + 6 bad elo) instead of (4 magnificent elo + 4 horribad elo) vs (8 above average elo) which is what it seems to do now.

Edited by Stoicblitzer, 20 April 2013 - 12:17 PM.

[Exoth3rmic]

The point of introducing an Elo system was to remove the random drubbings to see whether a skill based approached to matchmaking would lead to more engaging "balanced" play. There are several things that flow from this that people are finding hard to understand, including myself.

The reason the new people drop in the middle of the curve is because that's the average point, obviously, and then they distribute up or down. If they dropped in the middle and the system wasn't working they'd fall to the left and we wouldn't see the bell curve we have with the light highlighted (i.e played less than 50 matches) people. You'd see that entire light bar slip left. Which it isn't.

However, I see only one graph - is this the "averaged" Elo for all people's 4 weight classes? Can we see the distribution of people for each weight class?

Also previously Bryan stated in the dev answers 35 that the Elo was a perfect bell curve at that time, but your second graph now shows that to be untrue and couldn't possibly have been the case. is that because the original curve was calculated using the eroneous bugged data? How could that be without it being forced to that position through fudging the numbers with population etc?

As a simple matchmaking mechanic I am finding the matches I have are better than the simple random rollings we saw previously to pugging, and we must remember that even without Elo "bads will be bad" and will still suffer the same fate.