71 replies to this topic

[JernauM]

We all understand that PSR is closer to an XP bar than to a true rating of pilot skill. As players generally trend upward towards higher tiers that do not properly reflect their skill level, this flaw in PSR becomes a bigger and bigger problem for matchmaking. Players are being teamed up in matches with a high variance of skill, because the matchmaker is creating those matches with an imperfect understanding of players' "true" skill levels.

If the matchmaker is to depend on PSR, then PSR must be improved. I think the way to improve it is to make it a relative measure, not an absolute score. In other words, if there are going to be 5 tiers, then Tier 1 players should represent the top 20% of the player base, Tier 2 should represent the next 20%, and so on.

To say that a player is skilled is only meaningful if you compare that player's performance to the performance level of other players. PSR should reflect that fact, and so should the matchmaker. There must be some kind of a ranking.

[JernauM]

Bump. Matchmaker cannot function well if PSR does not function well.

[Lord0fHats]

I think the change that's really needed is that people should lose PSR even when their team wins if they do poorly enough. That's the real issue. The outcomes of a match are;

Win, Gain points
Win, = Outcome
Lose, = Outcome
Lose, Lose points

Nominally, I think people make too much of the 'exp bar' bit. If someone did well enough to not lose points even in a loss, then they did pretty well. But really if someone did really really badly, I think they should still lose points even if their team won. Likewise, maybe people should gain points even if they lose if they exceptionally well.

I think the bigger issue is the way match score is calculated, namely the massive weight derived from damage done. Want an easy way to climb the PSR ladder? Just load up LRMs and SSRMs (don't we have a dozen threads on why there's so many LRM boats now?). These weapons are bad (very bad), but they can put up high damage numbers while requiring little if any skill to get the damage. Because of the weight for damage in Matchscore, putting up 600 dmg in missiles pretty much assures you won't lose PSR. It's 'fluff' damage. It looks good on paper, and really brings up your match score while not really helping the team much.


The tabulation of Matchscore really needs adjustment for a number of things I think (torso twisting should be rewarded) and that would help solve the current PSR issues.

[Alistair Winter]

I'd be happy with this. I just don't know if PGI can do it. The elo system didn't work very well, at least not in the way it was implemented.

It may be a case of not trying enough variations of the elo system, but the current XP bar does accomplish the goal better than the previous system.

Personally, I always like the elo system as used in CounterStrike, but I'm not sure that could be adapted to MWO, because the games are so different.

[Ratpoison]

Alistair Winter, on 27 January 2016 - 11:34 PM, said:

I'd be happy with this. I just don't know if PGI can do it. The elo system didn't work very well, at least not in the way it was implemented.

It may be a case of not trying enough variations of the elo system, but the current XP bar does accomplish the goal better than the previous system.

Personally, I always like the elo system as used in CounterStrike, but I'm not sure that could be adapted to MWO, because the games are so different.

I kind of doubt it, the heavily team oriented aspects of MWO's metagame really seem to prevent an individual rating system like that from being effective. It can't track how often you screw your team or make great plays other than by secondary data like damage and kills. I'm fine with it being more experience based than skill based for the time being due to this, I don't need everyone around me trying to game a new elo system.

[XX Sulla XX]

It just needs more ways to go down in losses and wins. Also its normally so the top would not tend to be 20% of the player base.

Lord0fHats, on 27 January 2016 - 11:33 PM, said:

I think the change that's really needed is that people should lose PSR even when their team wins if they do poorly enough. That's the real issue. The outcomes of a match are;

Win, Gain points
Win, = Outcome
Lose, = Outcome
Lose, Lose points

Nominally, I think people make too much of the 'exp bar' bit. If someone did well enough to not lose points even in a loss, then they did pretty well. But really if someone did really really badly, I think they should still lose points even if their team won. Likewise, maybe people should gain points even if they lose if they exceptionally well.

I think the bigger issue is the way match score is calculated, namely the massive weight derived from damage done. Want an easy way to climb the PSR ladder? Just load up LRMs and SSRMs (don't we have a dozen threads on why there's so many LRM boats now?). These weapons are bad (very bad), but they can put up high damage numbers while requiring little if any skill to get the damage. Because of the weight for damage in Matchscore, putting up 600 dmg in missiles pretty much assures you won't lose PSR. It's 'fluff' damage. It looks good on paper, and really brings up your match score while not really helping the team much.


The tabulation of Matchscore really needs adjustment for a number of things I think (torso twisting should be rewarded) and that would help solve the current PSR issues.

You are exactly right. Being able to go down on a win is a must. For example having some one playing in a group and they make like 5 damage each game and the rest of the team is making 400-900 each game and they are winning. 5 damage person should be going down.

Alistair Winter, on 27 January 2016 - 11:34 PM, said:

I'd be happy with this. I just don't know if PGI can do it. The elo system didn't work very well, at least not in the way it was implemented.

It may be a case of not trying enough variations of the elo system, but the current XP bar does accomplish the goal better than the previous system.

Personally, I always like the elo system as used in CounterStrike, but I'm not sure that could be adapted to MWO, because the games are so different.

I agree the current system does work better than the last system.

[adamts01]

JernauM, on 27 January 2016 - 09:33 AM, said:

if there are going to be 5 tiers, then Tier 1 players should represent the top 20% of the player base, Tier 2 should represent the next 20%, and so on.

I agree this is the best way to do it. I personally love the W/L weight. It's too easy to find out what the computer classifies as "skill" and work that angle, even if it's detrimental to a win.

JernauM, on 27 January 2016 - 11:04 PM, said:

Bump. Matchmaker cannot function well if PSR does not function well.

We might have the best MM in the world, but we don't know because too many people have their participation award of tier 3-1.

[Bonger Bob]

PGI will continue to refuse to do anything different because they know better than every other player ranking system ever designed and any change would have to have more layers to the ratings which would break the match maker they've poorly developed.

[kesmai]

Although I will get tar+feathers for that:


reset every player to tier 3 every 3 or 6 months.

[adamts01]

Bonger Bob, on 27 January 2016 - 11:48 PM, said:

PGI will continue to refuse to do anything different because they know better than every other player ranking system ever designed and any change would have to have more layers to the ratings which would break the match maker they've poorly developed.

Well, they've completely re-done matchmaker once. And for all we know it's the best matchmaker ever, but we don't know because they keep handing out tier advancements like candy on Halloween.

[627]

kesmai, on 27 January 2016 - 11:48 PM, said:

Although I will get tar+feathers for that:


reset every player to tier 3 every 3 or 6 months.

I can see where you come from with this, but there are some problems. first, this would only result in unfair matches as there'd basically be no matchmaking until everyone is settled in their proper tier again.
And second, this would punish players that don't play as much as others because you don't really get up (or down) a tier very fast.

You'd first rewrite the whole PSR system to make something like seasonal skill levels.

Edited by 627, 27 January 2016 - 11:59 PM.

[C I L L I P U D D I]

No matter what the ranking system is that they implement players will whine about where they're placed and the players they match up against. People always think they should be a higher ran than they really are. If you play 500 games and you're still Tier 3, guess what, you should be Tier 3.

Watching people type "Thanks MM" in game chat makes me laugh because the only constant in all your games is yourself. Losing a lot of games consistently? More than likely you're part of the problem. It's not MM's fault you lose games, it's you. You have to carry hard 9/10. You can't count on your teammates, in solo queue, to do enough to win. Even in group queue, outside of your squad, you can't count on other people. You have to figure out how to do 700-800 damage and 5 kills every match. Will you do it every match, no. Will it get to the point where you get frustrated when you dont do over 400, yes. Am I perfect, no. xxSullaxx can attest to that. What I do though, and what Sulla and the other guys in our unit promote is retrospective gameplay. There will be times in matches, whether you do well or not, that you could've made better decisions and helped your team.

Looking at your own gameplay and shot selection retrospectively is the key to not caring about MM or your team or whatever other excuse you can make for losing games.

Edited by Cillipuddi, 28 January 2016 - 12:13 AM.

[Kilo 40]

I don't know why they don't just do something like have 6 players in a match with the highest scores go up in PSR, the lowest 6 go down in PSR, and everyone in the middle stay the same. or some variation of that. just so long as PSR rises or lowers based on a comparison with everyone else in the match, and not some arbitrary number or whether the team won or lost.

[NextGame]

Lord0fHats, on 27 January 2016 - 11:33 PM, said:

I think the change that's really needed is that people should lose PSR even when their team wins if they do poorly enough. That's the real issue. The outcomes of a match are;

Win, Gain points
Win, = Outcome
Lose, = Outcome
Lose, Lose points

Nominally, I think people make too much of the 'exp bar' bit. If someone did well enough to not lose points even in a loss, then they did pretty well. But really if someone did really really badly, I think they should still lose points even if their team won. Likewise, maybe people should gain points even if they lose if they exceptionally well.

I think the bigger issue is the way match score is calculated, namely the massive weight derived from damage done. Want an easy way to climb the PSR ladder? Just load up LRMs and SSRMs (don't we have a dozen threads on why there's so many LRM boats now?). These weapons are bad (very bad), but they can put up high damage numbers while requiring little if any skill to get the damage. Because of the weight for damage in Matchscore, putting up 600 dmg in missiles pretty much assures you won't lose PSR. It's 'fluff' damage. It looks good on paper, and really brings up your match score while not really helping the team much.


The tabulation of Matchscore really needs adjustment for a number of things I think (torso twisting should be rewarded) and that would help solve the current PSR issues.

The other thing it needs to do is only account for a players matches within the last say, 3 months or 6 months or so. You would find the tiering more fluid that way and players who have gone on breaks etc, or not so good with the current game balance to be playing at a more appropriate level.

kesmai, on 27 January 2016 - 11:48 PM, said:

Although I will get tar+feathers for that:


reset every player to tier 3 every 3 or 6 months.

heh, pretty much just wrote this, will teach me to read the entire thread first.

[Anunknownlurker]

From my point of view, I am 1 cm from tier 2 (Really? I am NOT that good!) there are two things I'd say:

1. PSR - The problem is in the name; it's too late now but if only PGI hadn't used the word "skill" how many threads would never had been started? This is quite simply a match making algorithm heavily based on win/loss and games played and that is all.

2. It's too easy to go up and too hard to go down - My own performance recently is an indicator of this. I am mainly a light/fast medium player, that's where I am a "decent" pilot. Over the last two months I have started playing heavies (Marauder + Warhammer) now, whilst I accept my advance up the tiers has slowed significantly, it is still rising whilst my KDR and average damage have fallen (not the best measures but they are pretty much all we've got). I am not surprised about my stats (basic mechs, learning how to pilot heavies, non-meta builds etc) but I am pretty sure that my tier rating should have fallen as I am, quite simply, not as skilful in these new, heavy mechs.

TL/DR - as a match maker PSR does its job. As a reflection of pilot skill it fails

[Duke Nedo]

tortuousGoddess, on 27 January 2016 - 11:41 PM, said:

I kind of doubt it, the heavily team oriented aspects of MWO's metagame really seem to prevent an individual rating system like that from being effective. It can't track how often you screw your team or make great plays other than by secondary data like damage and kills. I'm fine with it being more experience based than skill based for the time being due to this, I don't need everyone around me trying to game a new elo system.

This I think needs to be remembered. PSR is supposedly a personal rating in a team based game. Difficult (impossible) to sample properly, so whatever they do we are likely to get a gameable system. I am also quite content with the way it is now tbh, the "xp bar" filters out the worst and keeps matchmaking sort of "random" but within reasonable limits. My waiting times in the solo queue is mostly less than 30 seconds at more than half full xp bar in T1, so it may be possible to narrow the MM ranges a bit. I wouldn't mind waiting 2 minutes for using Tier +/- 1 instead of Tier +/- 2, worth a test at least.

The other thing about true Personal Skill Ratings in a team game is that if it is to measure the skill of the pilot and not the skill of the team/group/unit, then it needs to be based only on solo drops... by definition. If you can farm or destroy your PSR in the group queue by dropping with good or bad mates respectively, then it's no longer your Personal Skill Rating, it's something else. It should hence only be used for Matchmaking in the solo queue as well, it should be taken into account in exactly the same settings as it's built. Being able to adjust your PSR in one mode and apply it to another mode is begging to be broken...

XX Sulla XX, on 27 January 2016 - 11:45 PM, said:

It just needs more ways to go down in losses and wins. Also its normally so the top would not tend to be 20% of the player base.




You are exactly right. Being able to go down on a win is a must. For example having some one playing in a group and they make like 5 damage each game and the rest of the team is making 400-900 each game and they are winning. 5 damage person should be going down.

I agree the current system does work better than the last system.

It's not so simple though... some of my best games with the best teams on both sides I may only have scored 250 damage and 1 kill, and that's perfectly fine if your entire team does the same, nobody needs to break 300-400 damage in a solid win. On the other hand I may play half-sloppy skill-wise against bad opponents and bad mates and score 1500 damage and 10 kills. That is something that is not possible to do in a good game. How could PSR based on score tell the difference?

Cillipuddi, on 28 January 2016 - 12:10 AM, said:

No matter what the ranking system is that they implement players will whine about where they're placed and the players they match up against. People always think they should be a higher ran than they really are. If you play 500 games and you're still Tier 3, guess what, you should be Tier 3.

Watching people type "Thanks MM" in game chat makes me laugh because the only constant in all your games is yourself. Losing a lot of games consistently? More than likely you're part of the problem. It's not MM's fault you lose games, it's you. You have to carry hard 9/10. You can't count on your teammates, in solo queue, to do enough to win. Even in group queue, outside of your squad, you can't count on other people. You have to figure out how to do 700-800 damage and 5 kills every match. Will you do it every match, no. Will it get to the point where you get frustrated when you dont do over 400, yes. Am I perfect, no. xxSullaxx can attest to that. What I do though, and what Sulla and the other guys in our unit promote is retrospective gameplay. There will be times in matches, whether you do well or not, that you could've made better decisions and helped your team.

Looking at your own gameplay and shot selection retrospectively is the key to not caring about MM or your team or whatever other excuse you can make for losing games.

To be honest though, the MM sends you on some bad RNG streaks at times. I am not writing that because I'm losing now, rather the opposite. I'm on a positive streak right now where I'll win even if I crash and disco early in the game... that is just as boring as the opposite.

[Parnage Winters]

So long as my que times don't get even longer yeah sure but the thing is the que times get kinda lengthy so the matchmaker gives up seemingly half the time and just grabs anyone it can. I mean I'll suffer longer que times but only if matches are worth waiting for. Right now it's suffer for a grab bag of possibilities.

[Paigan]

Also see the link in my signature: PSR Needs Adjustment To Be Sane

Edited by Paigan, 28 January 2016 - 01:32 AM.

[TexAce]

I agree on this. In my last match I didnt contribute really well to the win, I only did 130 damage, one assist, died early and felt totaly worthless...still my PSR went up. This should not happen. I should have lost points, thats how bad I played this match (to be fair with myself I tried an unskilled Hunchy 4H with bad loadout and got distracted by boobs early).

Edited by TexAce, 28 January 2016 - 01:42 AM.

[Paigan]

Kilo 40, on 28 January 2016 - 12:15 AM, said:

I don't know why they don't just do something like have 6 players in a match with the highest scores go up in PSR, the lowest 6 go down in PSR, and everyone in the middle stay the same. or some variation of that. just so long as PSR rises or lowers based on a comparison with everyone else in the match, and not some arbitrary number or whether the team won or lost.

Highly unfair:
Say you have 12 super guys stomping 12 noobs:

The 6 super guys with score 500-550 get uprated, while the 6 super guys with score 450-500 get downrated.
Did the 6 "lower" guys really play so bad that they deserve a downrate?

Same on the other side:
The 6 noobs with score 20-30 get rewarded for having performed SO well.
The 6 noobs with score 0-20 get downrated because they suck so much.


My observation:
People are REALLY quick to come up with extremely short-sighted, half-baked solutions for complex problems.
For a proper solution, see the link provided in my post and sig.

Edited by Paigan, 28 January 2016 - 02:00 AM.