149 replies to this topic

[Wintersdark]

BoseMensch, on 18 December 2015 - 11:05 AM, said:

So I'm new to this game, and have only anecdotal evidence, but it seems to me that the ranking is extremely biased toward overall win/loss for determination. I have only around 100 matches played (I think its around there), and of those only probably 4 were played in the faction war section before I realized I was severely outclassed and only doing a disservice to those playing there. The reason that I went there, however, I think may be relevant to the conversation. When I played my rookie matches, I was (obviously and unsurprisingly) not very good. I didn't start paying attention to the PSR until several matches in, which were all losses. I noticed that I was at tier 5, but the problem was that even though I started to improve my skills, my tier was not really increasing that much. I spent many hours learning tactics and strategies to improve my skills by reading forums and watching vids, and it wasnt long before I went from doing around 100 dmg per game to doing around 250-300 per game in my hunchback. I was still stuck in tier 5 for a long time, because although I would outperform the mediums in my matches, I was on the losing team many times because the other players hadn't gotten away from the COD/Leroy Jenkins style of play and were just charging into fire and getting picked off one by one. I had a day where I lost probably 15 matches in a row, but was still performing fairly well on an individual basis (for a noob), due to teamwork fails. That is what prompted me to go to the faction wars portion, as I was thinking that there would be better teamwork, which would improve the chances for a win. I was correct, but not skilled enough to really participate and help the team, so I stopped faction play.

Did you read the OP? Gaining and losing rating is directly related to match score, and whether you win or lose.

Anyways, you're not going to find improved chances to win in different environments or tiers, it's going to remain roughly 50% because everyone in the tier is going to be playing at roughly the same level. There are only two ways to get better odds to win reliably:
1) Play better yourself.
2) Play in a group. This is particularly relevant for faction play - Teamwork is what determines win/loss in Faction Play, and the only reliable way to get teamwork is by working in an actual team. Join a unit, and everything is gravy in Faction Play.

[BoseMensch]

Wintersdark, on 30 December 2015 - 10:32 AM, said:

Did you read the OP? Gaining and losing rating is directly related to match score, and whether you win or lose.

Yes, I did read the OP, which is why I posted my thoughts on this topic. And way to flame me for no reason.... Thanks! I guess the lesson here is that nobody wants anyone else's opinion or thoughts and no new players are welcome. I'm sure that's good for the community and exactly the kind of player base that the devs want. Instead of attacking me, you could have just, I dunno, not said anything...

The point I was trying to make (BTW, did you read MY post or just pick out the part that you wanted to flame me about?) is that individual tier ranking should really focus on the performance of the individual, not on whether you get lucky enough in a PUG to get teamed up with better players. The tier system may work well at higher levels, but it is skewed at lower levels. That was really my point. But thanks for making me feel totally unwelcome here!

[Spetulhu]

BoseMensch, on 31 December 2015 - 11:30 AM, said:

The point I was trying to make is that individual tier ranking should really focus on the performance of the individual, not on whether you get lucky enough in a PUG to get teamed up with better players. The tier system may work well at higher levels, but it is skewed at lower levels.

On the other hand, if it focused only on personal performance and cared nothing for team win/loss you'd have people with all the right FPS reflexes move up in rank very fast and then get demolished by those who have both the reflexes and the teamwork. Someone would be unhappy in any case.

[Tesunie]

Spetulhu, on 31 December 2015 - 12:19 PM, said:

On the other hand, if it focused only on personal performance and cared nothing for team win/loss you'd have people with all the right FPS reflexes move up in rank very fast and then get demolished by those who have both the reflexes and the teamwork. Someone would be unhappy in any case.

Win should be a factor, but it shouldn't at the same given time, be about 90% of the determination factor. Not in "PUG random team drops".

I, personally, would rather seem more degrees of "stay where you are" and any low result (win or lose) drop your PSR. The numbers could change slightly on a win compared to a loss, but overall it shouldn't be so drastic.

A score of 200-300 is considered rather average. If a player is achieving that score on an average (win or loss), then they are probably in the correct PSR, and should remain there. The current system would bring them up if their team wins, and lower them if they lose with that average performance. And, to top it off, you always gain more PSR than you can lose currently, making moving backwards if you are improperly ranked harder. This results in, even on a less than 1 to 1 win/loss, every player eventually rising upwards towards Tier 1.


As of current, the ranking system doesn't rank Player Rating, but "random" (for PUG groups) Team Rating, which will always be in flux. Team work skills are important, but if no one else is working as a team on your team, does that mean you should be penalized?

It's a great base of a system, but it just isn't there yet in my opinion. Still could use some work. (Certainly better than Elo, but not by all that much.)

[Wintersdark]

BoseMensch, on 31 December 2015 - 11:30 AM, said:

Yes, I did read the OP, which is why I posted my thoughts on this topic. And way to flame me for no reason.... Thanks! I guess the lesson here is that nobody wants anyone else's opinion or thoughts and no new players are welcome. I'm sure that's good for the community and exactly the kind of player base that the devs want. Instead of attacking me, you could have just, I dunno, not said anything...

The point I was trying to make (BTW, did you read MY post or just pick out the part that you wanted to flame me about?) is that individual tier ranking should really focus on the performance of the individual, not on whether you get lucky enough in a PUG to get teamed up with better players. The tier system may work well at higher levels, but it is skewed at lower levels. That was really my point. But thanks for making me feel totally unwelcome here!

How did I flame you? How did ibattack you? It's hardly unusual for a commenter to not read the OP, so its a legitimate question, and your post certainly led to that possibility.

Anyways.

Individual performance is only relevant as far as it contributes to victory, because winning matches is what matters. That is why win/loss is so important to final rating adjustments. And besides, if you dobwell in a loss, you don't lose rating. If you lost rating, its because you didn't do well.

Edited by Wintersdark, 31 December 2015 - 01:08 PM.

[maniacos]

Well the system is obviously bs if your personal skill is rated on your team's performance in a PUG game.

Edited by Jherek C, 01 January 2016 - 04:48 AM.

[Tesunie]

Jherek C, on 01 January 2016 - 04:30 AM, said:

Well the system is obviously bs if your personal skill is rated on your team's performance in a PUG game.

The system does base it's action on your performance, but how it rates that is based on how your team did (win/lose). If one performs good enough (read as extremely good), they can remain even or even go up (on a very above average score) on a loss. If one performs poorly (read as, basically does nothing) and your team wins, you can remain even on your PSR.

The "personal" performance it rates is so greatly altered on the team's abilities that it is essentially rating the team more than the individual. With randomly generated teams...

AKA: Unless you performed above average on a loss (not easy, considering you lost and probably: 1 Died, 2 Died early, 3. didn't kill or kill assists many opponents, all of which reduce your match score), your PSR is basically relevant to your team winning. Add to that the fact that on a loss you don't lose as much PSR as you gain on a single win... and you have a system based on rating the team, rating W/L (for the most part) and will force players upwards easier than keeping even or even downwards.

[maniacos]

Tesunie, on 01 January 2016 - 08:42 AM, said:

The "personal" performance it rates is so greatly altered on the team's abilities that it is essentially rating the team more than the individual. With randomly generated teams...

Yeah that's the point

[RAM]

Tesunie, on 31 December 2015 - 12:31 PM, said:

(Certainly better than Elo, but not by all that much.)

Mathematically, ELO was significantly better.

[Tesunie]

RAM, on 04 January 2016 - 02:49 PM, said:

Mathematically, ELO was significantly better.

W/L is too biased on the team. Elo also wanted to force players into a 1/1 win/lose ratio. This sometimes caused Elo to toss players into a match, knowing/predicting that they would lose the match. When the MM sets up matches that are basically "already a loss", it isn't exactly a very good system. (And yes, I know that there was still a chance to win.)

PSR was suppose to be biased towards the individual player's skill level, so that players would fight other players of similar skill level. However, it is still too bias towards winning for that to happen, so it is still very bias towards rating the team.

ELO was designed for 1v1 games, such as Chess. It doesn't work so well for multi-player games. If this was one player vs one player, than ELO would remain viable and a great system.

Elo was better than no match maker (though I did very much enjoy my matches back then anyway).
PSR is better than Elo.
PSR may be better, but it could still use some work.

Of course, no system will ever be "perfect".

[RAM]

Tesunie, on 04 January 2016 - 07:11 PM, said:

W/L is too biased on the team. Elo also wanted to force players into a 1/1 win/lose ratio. This sometimes caused Elo to toss players into a match, knowing/predicting that they would lose the match. When the MM sets up matches that are basically "already a loss", it isn't exactly a very good system. (And yes, I know that there was still a chance to win.)

PSR was suppose to be biased towards the individual player's skill level, so that players would fight other players of similar skill level. However, it is still too bias towards winning for that to happen, so it is still very bias towards rating the team.

ELO was designed for 1v1 games, such as Chess. It doesn't work so well for multi-player games. If this was one player vs one player, than ELO would remain viable and a great system.

Elo was better than no match maker (though I did very much enjoy my matches back then anyway).
PSR is better than Elo.
PSR may be better, but it could still use some work.

You have a number of misconceptions about ELO. Suffice it to say, there is a reason it is so widely used in skill based systems: it works very well.

Winning is the only non-gameable metric.

[Tesunie]

RAM, on 05 January 2016 - 10:23 AM, said:

Winning is the only non-gameable metric.

Wanna make a bet?
Cheats, hacks, game manipulation... All affect winning. Making it a gameable metric.
Logging in, playing matches, and throwing the match by not doing anything productive in the match.

As I said, Elo is great for 1v1. It starts to fail on multiple player vs multiple player situations. Elo could still work then when the teams are consistently the same people. However, for PUGs, it's all random teams. So, gauging a single player's "ranking" upon win/lose, which is determined not by one player but by (in MW:O's case) 23 other players, is not accurate. By any means for an individual player.

PSR stands for "Player Skill Rating". However, being based so much on win and loss, it isn't ranking the player so much as it is their team. When the teams are random, you can't rank the random team, because it's going to change next time they play. This was also a fault of Elo as well, as it ranked the team and not the individual.

In a 1v1, Elo is a good measure of skill, because it's one player vs one player. If one wins, it's because of their own skill over the other player, giving them a higher ranking. Once you start adding more players into the mix, it starts to get murky, and the system starts to fail.

Elo could place 2 high Elo ranked players onto a team, with 4 low Elo ranked players, the rest average, then pit them again another random team with 4 high Elo ranked players, and 2 very low Elo ranked players, and the rest average. Even though the Elo scores per team may be equal, the team with 4 high ranked players will probably win, because they have more skill in more places, and less dross. Then, next match, it's all random again...



No system will be perfect. Elo is not good for this style of match making. Even PGI said that when it was implemented, but that they were going to give it a try because it was better than nothing at all. Now we have PSR, which is a good step away from Elo, but still needs some more work. PGI knows this, and they have already stated that the system is not complete. For this style of game play, with random teams, PSR is a better system over Elo.

[L Y N X]

Elo system is a preferable mechanism in 1 on 1 matches, by far! In Fact, that was what it was intended for...

In recent years, many multiplayer team vs team online games began using Elo, only to subsequently move to a rating system that is weighted on W/L and player contribution. I think PSR in MWO is better then the old Elo system they used, with 20,000+ matches, it just feels better now, more competent players on both sides and closer matches with fewer stomps. Is PSR perfect? no it is not, but this thread is about explaining what is, and not about what should be...

Thanks for reading, and I hope those who read it benefited from doing so.

nearly half my pug match losses I reach match score of greater than > 250, consequently I'm taking fewer PSR steps back so that my forward moving PSR progress feels greater or faster. This is from improving my mech loadouts and my own choices tactically in matches, working the flanks, back up and supporting larger mechs, playing better and as a better team player. I should be Teir 1 before January is over.

[L Y N X]

Tesunie, on 05 January 2016 - 11:26 AM, said:

PSR stands for "Player Skill Rating". However, being based so much on win and loss, it isn't ranking the player so much as it is their team.

Tenusie, I agree with most of your post, all of it that I have not quoted, consider us in agreement. However, PGI has stated that PSR is NOT a ranking system. It is a rating system. To me, I think of a ranking system as a kind of leaderboard ranking players to some heuristic ruleset. A Rating system on the other hand does not rank players but rather groups them according to some performance metrics, it is less ordinal. The difference may seem like semantics, and it might also be subtle, but there is a difference in intent and outcome.

[Hexenhammer]

7ynx, on 25 September 2015 - 08:48 AM, said:

From Paul:

I'm thinking I might do some 0 score matches to get back to tier 5.

but I always assumed.
win equals no change or a positive change. And loosing is a negitive change, or at best no change.

[Tesunie]

7ynx, on 05 January 2016 - 06:22 PM, said:

Tenusie, I agree with most of your post, all of it that I have not quoted, consider us in agreement. However, PGI has stated that PSR is NOT a ranking system. It is a rating system. To me, I think of a ranking system as a kind of leaderboard ranking players to some heuristic ruleset. A Rating system on the other hand does not rank players but rather groups them according to some performance metrics, it is less ordinal. The difference may seem like semantics, and it might also be subtle, but there is a difference in intent and outcome.

The intent is to place players of similar skill together for matches. But, you are correct. It is rating, not ranking. My bad on the terminology.

And consider all this debate posts on PSR to be free bumps to get your thread noticed. You've got good information here.

[RAM]

Tesunie, on 05 January 2016 - 11:26 AM, said:

Wanna make a bet?
Cheats, hacks, game manipulation... All affect winning. Making it a gameable metric.
Logging in, playing matches, and throwing the match by not doing anything productive in the match.

Yes, I would take that bet. If your definition of winning is getting banned, good luck with that (not to mention those 'factors' also affect PSR ).

Ultimately your posts make it clear you do not understand how ELO works. You also confuse matchmaking with ELO. Your analysis is in error because your assumptions are fundamentally flawed.

[Tesunie]

RAM, on 06 January 2016 - 04:49 PM, said:

Yes, I would take that bet. If your definition of winning is getting banned, good luck with that (not to mention those 'factors' also affect PSR ).

Ultimately your posts make it clear you do not understand how ELO works. You also confuse matchmaking with ELO. Your analysis is in error because your assumptions are fundamentally flawed.

Then, I'm not the only one here...

7ynx, on 05 January 2016 - 06:13 PM, said:

Elo system is a preferable mechanism in 1 on 1 matches, by far! In Fact, that was what it was intended for...

In recent years, many multiplayer team vs team online games began using Elo, only to subsequently move to a rating system that is weighted on W/L and player contribution.

So... if we all have it wrong on how it works (as it was explained to us by PGI), care to enlighten us? Or you going to sit there and just continue to say how flawed everyone else is and never give any real response?


Elo as I understand it:
- Is based on W/L.
- It gives a rating/number to a player based on their individual W/L rate.
- It then tries to pair that player with another player (single player related) who has a similar if not exact rating. (Or for teams, tries to create teams that all player Elo scores equal a certain total per team.)
- If there is a disparity of their Elo rating, then the formula predicts (by what PGI said of their own formula) which player (or Team for MW:O) is likely to win.
- If the player who is predicted to win wins, then there is little change in their rating. If the player predicted to win losses though, then changes happen accordingly.
- As stated before, is a system designed best for solo PvP (as in, one player on each side, not a team of players per side) styled games, such as chess (or checkers, or boxing even, or Battleship, or...). In that area, it is very effective.
- Doesn't work so well for Multi-player, as each player could have vastly different actual skill levels within the (randomly) assigned team (for PUGs). As an example: This makes it so that, to balance Elo scores, a team could have a concentration of skill but an overall poor skilled team (Bad team/Carry harder), or a team of evenly distributed skill (decent team) to a team with an above average skill with a concentration of low skill (best team typically).

Did I miss anything major? Did I mess something up here?

[Wintersdark]

Hexenhammer, on 05 January 2016 - 06:25 PM, said:

I'm thinking I might do some 0 score matches to get back to tier 5.

but I always assumed.
win equals no change or a positive change. And loosing is a negitive change, or at best no change.

This is incorrect. The chart linked is correct.

You cannot lose rating on a win, but you can gain rating on a loss.

[Wintersdark]

Tesunie, on 06 January 2016 - 06:27 PM, said:

- Doesn't work so well for Multi-player, as each player could have vastly different actual skill levels within the (randomly) assigned team (for PUGs). As an example: This makes it so that, to balance Elo scores, a team could have a concentration of skill but an overall poor skilled team (Bad team/Carry harder), or a team of evenly distributed skill (decent team) to a team with an above average skill with a concentration of low skill (best team typically).

Did I miss anything major? Did I mess something up here?

When it's building a team, the Elo based MM follows this algorithm (as per the guy who coded the system, Karl Berg):

Take the oldest player in the queue. His Elo rating is now Team 1 seed rating.

Collect the next 11 players closest to that rating in the queue, and matching weight class restrictions.

Team 1 Elo rating is assigned the average of Team 1's players. This rating is the Team 2 seed.

Collect 12 players, matching the weight class restrictions, closest to the seed value.

Team 2 Elo rating is assigned the average of Team 2's players.

Compare Elo ratings and predict a winner (as you note in the post) and use this to determine how score moves after the match.



The important take-away's here as to how Elo works:

If there are a good number of players at a given Elo rating, then this creates a very balanced match. If there are not - such as at the extremes - the match may end up very screwed up. For example, if the oldest player in the queue has the highest Elo rating in the queue, it's quite problematic. The MM is going to grab him, then the next highest players (as there's nobody above him). This takes the 12 highest ranked players away from the queue. The next team, trying to match Team 1's average, has nothing but lower ranked players. Hopefully not dramatically, but the closer you get to the ends of the scale, the fewer players there are. In lower pop/off times, this could really be a problem.

This is resolved by the delays in opening the release valves. It'll initially only go a certain number of points away from the seed value when looking for players, and only pick players correctly conforming to the weight class restrictions. After a certain amount of time, those release valves open and it'll pick players further from the seed value. This is why long match making times are common at high/low Elo values. Note that matchmaking time is only relevant for the first player chosen on team 1, everything is calculated from that point.

Another take-away is that in building a team, at no point did the matchmaker deliberately chose a low or high ranked player to offset an existing high or low ranked player to maintain an average. Choice was always based on closest to target seed/within correct weight class.

This did mean that, for the vast bulk of players, Elo actually worked pretty well, only failing at the ends of the spectrum.