Jump to content

Is Psr / Tier Accurately Reflecting Skill Level?


56 replies to this topic

#21 DeathlyEyes

    Member

  • PipPipPipPipPipPipPip
  • The Messenger
  • 940 posts
  • Google+: Link
  • Facebook: Link
  • Twitter: Link
  • Twitch: Link
  • LocationMetaphorical Island somewhere in the Pacific

Posted 25 September 2015 - 04:48 AM

The bigger issue is, we're pre psr scores augmented. If you were stuck in an Elo range where when you played solo matches and the games were basically unwinnable, (your team is low Elo with you boosting the average against an above average elo team) do those games affect your psr differently?

#22 QuaxDerBruchpilot

    Member

  • PipPipPipPipPipPip
  • The 1 Percent
  • 319 posts
  • LocationAustria

Posted 25 September 2015 - 05:07 AM

LOL. I just snipped that from todays post by Paul Inoque regarding some "clarifications" around the PSR:

Quote

In the chart above, you can see that winning is the key to move UP in PSR


Fine. So they invented a system called Pilot skill rating, whis is to refelct the personel skill of the player, just to set it up that the main driver for boosting your personel skill level is the only thing that only barely can be influenced by the pilot himself.

Quote

Remember, PSR calculations reward team play more than 'Rambo' or 'I'm the hero and will carry my team to victory' play styles.


In other words, PSR befor anything else punishes me for beeing teamed up with non-team-playing-mind-setted fellows.


Wow. What a fail.

Edited by Quax1102, 25 September 2015 - 05:17 AM.


#23 rolly

    Member

  • PipPipPipPipPipPipPip
  • Guardian
  • Guardian
  • 995 posts
  • LocationDown the street from the MWO server

Posted 25 September 2015 - 05:55 AM

View PostKmieciu, on 24 September 2015 - 10:45 PM, said:

http://mwomercs.com/...37#entry4719837

The two factors to PSR are:
winning
match score


If this is true then how does my friend in the example with a consistently less match score and lower damage output get a hier tier rating than I who has a consistently higher match score and damage rating?

For example in the Territorial challenge; in the spare spotty time he had on a good connection he could not get the minimum requirement of 200 for the event. With casual play I got the 200-300 on half the maps. (mostly do to the randomness)

Edited by rolly, 25 September 2015 - 05:58 AM.


#24 Commissar Aku

    Member

  • PipPipPipPipPip
  • Urban Commando
  • Urban Commando
  • 195 posts

Posted 25 September 2015 - 06:00 AM

View Postrolly, on 25 September 2015 - 05:55 AM, said:


If this is true then how does my friend in the example with a consistently less match score and lower damage output get a hier tier rating than I who has a consistently higher match score and damage rating? For example in the Territorial challenge, in the spare spotty time he had on a good connection he could not get the minimum requirement of 200 for the event, on casual play I got the 200-300 on half the maps (mostly do to the randomness)

I didn't score on a single map that I played on less than a score of 400 during the challenge, it put me at the bottom of tier 5. On every map I play I rarely do less than a match score of 250 with equal or better damage. I have unit mates with half that score in tier 3, consistently do the same or worse than me, yet, they have a higher tier? Going with the tier system is a popularity contest, if PGI likes you you got a higher tier, if they don't like you and want you to quit playing their game they put you in the trouble maker tier. Neko BS all over again

Edited by Commissar Aku, 25 September 2015 - 06:00 AM.


#25 rolly

    Member

  • PipPipPipPipPipPipPip
  • Guardian
  • Guardian
  • 995 posts
  • LocationDown the street from the MWO server

Posted 25 September 2015 - 06:34 AM

View PostDeathlyEyes, on 25 September 2015 - 04:48 AM, said:

The bigger issue is, we're pre psr scores augmented. If you were stuck in an Elo range where when you played solo matches and the games were basically unwinnable, (your team is low Elo with you boosting the average against an above average elo team) do those games affect your psr differently?


For that matter why do we even have TWO systems for ranking? (Because lets face it, the lead dev can call something "Not a Ranking system" but it really for all intents and purposes is.)
This one seems to just to keep us busy and distract more players while they push fluff content? How much does this tier system actually even add to the depth of this game or its universe aside from feeding ego? Or is there an actual deeper element here? Why not just release ELO stats instead of creating an entirely new system that's horribly inconsistent?

Edited by rolly, 29 September 2015 - 07:02 AM.


#26 Ironwithin

    Member

  • PipPipPipPipPipPipPipPipPip
  • Overlord
  • Overlord
  • 2,613 posts
  • LocationGermany

Posted 25 September 2015 - 07:19 AM

View Postrolly, on 25 September 2015 - 06:34 AM, said:


For that matter why do we even have TWO systems for ranking?
...


We don't. Elo was led out back, shot and buried.

Also: the system is not meant to add any depth. It is just a means for the matchmaker to put together, theoretically, fair teams.
It's not there to build any leaderboards or give e-peen enlargements to anyone.

#27 QuaxDerBruchpilot

    Member

  • PipPipPipPipPipPip
  • The 1 Percent
  • 319 posts
  • LocationAustria

Posted 25 September 2015 - 07:26 AM

View Postrolly, on 25 September 2015 - 05:55 AM, said:


If this is true then how does my friend in the example with a consistently less match score and lower damage output get a hier tier rating than I who has a consistently higher match score and damage rating?

For example in the Territorial challenge; in the spare spotty time he had on a good connection he could not get the minimum requirement of 200 for the event. With casual play I got the 200-300 on half the maps. (mostly do to the randomness)


Maybe just count your number of wins against his. I think the system works as follows: the better your personal score is (ELO,PSR, doesn't matter in my opinion), the more likely you are teamed up with players worse than you. Thus, the more likely you will lose tha match. Thus ("win" beeing the main factor in PSR), you will end up with a lower PSR value than your friend.

It would help if they would show the actual details of your PSR change in the after-battle info screen like you see your C-Bills and damage detail (Killing blow - 500 PSR points, spotting assist - 300 PSR points, victory - 5000 PSR points ..)

The fact they decide to keep that hidden makes me suspicious about how well this new thinggy is working out actually.

#28 QuaxDerBruchpilot

    Member

  • PipPipPipPipPipPip
  • The 1 Percent
  • 319 posts
  • LocationAustria

Posted 25 September 2015 - 07:41 AM

View PostNomadicCanuck, on 24 September 2015 - 04:24 AM, said:

the steering wheel brigade.


Great :-)

#29 Tesunie

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The Seeker
  • The Seeker
  • 8,586 posts
  • LocationSeraphim HQ: Asuncion

Posted 25 September 2015 - 08:18 AM

View PostInflatable Fish, on 25 September 2015 - 04:04 AM, said:

the major flaw in the system is that it doesn't reward good play, instead it rewards primarily doing damage.


It's been rewarding rather selfish "worry about my own match score, and hope the team wins" game play, if you asked me.

View PostCommissar Aku, on 25 September 2015 - 06:00 AM, said:

I didn't score on a single map that I played on less than a score of 400 during the challenge, it put me at the bottom of tier 5. On every map I play I rarely do less than a match score of 250 with equal or better damage. I have unit mates with half that score in tier 3, consistently do the same or worse than me, yet, they have a higher tier? Going with the tier system is a popularity contest, if PGI likes you you got a higher tier, if they don't like you and want you to quit playing their game they put you in the trouble maker tier. Neko BS all over again


I highly doubt that the system as such clauses in place to "downtrodden" on disfavored players. I'm fairly certain that it's just a mathimatical and uncaring formula.

View PostIronwithin, on 25 September 2015 - 07:19 AM, said:


We don't. Elo was led out back, shot and buried.

Also: the system is not meant to add any depth. It is just a means for the matchmaker to put together, theoretically, fair teams.
It's not there to build any leaderboards or give e-peen enlargements to anyone.


Elo wasn't shot and buried. It was taken out back one door, redressed and some minor alterations were made, and then escorted out another door renamed PSR. Elo was all about winning a match to go up. PSR is "almost" all about winning a match to go up. I see little difference between the two at this time.

I don't mind what tier I am in, but I want a system that I feel is accurately placing me. If I'm T4, then I do want the system to find ways to keep me there. If I'm T3, I don't want to fall into T5 just because my team happens to lose often for some reason. If I'm a T5 player, I don't want to be placed in T1 because my team seems to win without me. (1/24 is not a very good percentage of influence on a win or a loss.)

#30 Inflatable Fish

    Member

  • PipPipPipPipPipPipPip
  • Rage
  • Rage
  • 563 posts

Posted 25 September 2015 - 08:46 AM

View PostTesunie, on 25 September 2015 - 08:18 AM, said:


It's been rewarding rather selfish "worry about my own match score, and hope the team wins" game play, if you asked me.


that's... exactly what I'm saying? (because match score depends 80% on damage dealt)

#31 Tesunie

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The Seeker
  • The Seeker
  • 8,586 posts
  • LocationSeraphim HQ: Asuncion

Posted 25 September 2015 - 08:59 AM

View PostInflatable Fish, on 25 September 2015 - 08:46 AM, said:


that's... exactly what I'm saying? (because match score depends 80% on damage dealt)


Match score is your damage divide by half with "other stuff added in" (which wasn't how the new match score system was pitched to us, but that's a different subject). There is no reward for pushing a charge, being a lead man, taking fire for a teammate (which is hard to reward all of those anyway without rewarding ramo and his great death run), almost none for scouting, staying with your team to protect them with ECM/AMS.

I've been raising my PSR by becoming a sniper. I've had to load in a meta *shivers* Cheetah to go behind the enemy team, and snipe them. I'm not near my team (most of the time) and I'm dealing higher damage because if someone sees me, I typically run. (Though, I am not a selfish person, so I tend to still run to someone's aid. Now I just do so if I'm not being shot at.)

The play style I've had to adopt if I want to go up in rank, which I only did as an experiment and to level the mech, is a cowardly and selfish play style which becomes concerned mostly about myself (even if I do try to help the team still when I can). I hate it. (But, I don't tend to stay in that role. I start out as selfish sniper, and then see a teammate in trouble and instinct kicks in and I rush to their aid if I can... then I think of it, and sometimes I'll back off again... but... ARG!)


The system also rewards winning too much. Not actual skill. (And it can't counter for bugs that occur either. Yesterday I had a match where I wanted something different. I jumped into my Jeagermech 6-A. So many LRMs visibly hit their target. So many UAC5 shells landing with a rewarding red crosshair. Died anyway by being shot visibly through terrain (HSR or "somehow"). Did 75 points of damage and a horrible match score.)

*As a note: I seem to have the luck that, if a bug can hit me, it will. I've hovered in mid air long enough for an enemy to shoot me, return fire and I'm teleported to the ground. I run around a corner... three times later I finally get around and stay around that corner. Too bad the enemy mech I was running away from got to shoot me 3 times longer... Etc.

View PostInflatable Fish, on 25 September 2015 - 08:46 AM, said:


that's... exactly what I'm saying? (because match score depends 80% on damage dealt)


Felt it worth requoting...

80% does sound about right.

50% is actual damage. 30% is damage related rewards (kills, assists, components destruction, etc).

#32 i am a bore

    Member

  • Pip
  • Legendary Founder
  • Legendary Founder
  • 17 posts

Posted 25 September 2015 - 09:41 AM

The problem with PSR is that it exists overall.

There is no, good definable, reason that a player needs PSR to be visible, or necessary, at all. Other then d**k-waving contests, "Muh PSR is 1, you need PSR 2 to play with me or ur s**t." PSR falls into the same place that ELO/MMR/Every other rating that every game has, and every other game falls into the category that a win boosts it and a loss lowers it.

It's standard across the board, and yet when you ask any dev from any of the other games about it, you get the same rote response. "PSR/MMR/ELO/Skill is this super-secret, transparent-to-the-player highly magical equation that balances games for fun, fair play across all skill levels, from beginners to the pros."

There is no good way to do PSR/MMR/ELO/Skill for anything, simply because it will always come down to winning/losing.

You cannot have a system that rates players on singular skill and push forward that it 'promotes teamwork and cooperation' while limiting players abilities to play as a team. The fact that you can't get 3-11 of your buddies and play a game against 12 random kids and blow them out of the water, because yes, "teamwork is OP", is good. But it hurts it at the same time. PSR would be a better thing, if, Lances were able to queue together and get put along with the teams of 8 randoms. against 1 lance and 8 randoms as well.

So, no. I do not feel that PSR accurately reflects "Player Skill Rating" simply for the fact that...

Quote

In the chart above, you can see that winning is the key to move UP in PSR. Your match score determines HOW FAR you move up (or down in case of a loss). As I mentioned above, there is one difference in that getting a very high match score but you and your team lose the match, will still result in a small climb in PSR.


If winning is the key to moving up in PSR, why not just call it that a Win/Loss Ratio and leave it at that, since that what it seems to be to be, or just bash your head against a wall since...

Quote

[color=orange]Clarification 2: The more games you play the higher Tier you're going to get.[/color]
That's partly true. The more games you play, the better you should become. The better you become, the higher your match scores will be. But you will plateau somewhere. Our top players have hit the PSR point ceiling. Even at that level their PSR values bounce up and down off the ceiling. They may go down 1 point on a loss but spring back up on their next victory.


Or, you know. You can take the Win/Loss part out of the equation so it actually takes player's additions to the game into higher account then the other things. The current system is HIGHLY biased towards Winning, as evidenced by their explanation of it to begin with.

Instead, if you swap these things into a system that PROMOTES players to do things that BENEFIT the team versus just winning, you get a better system overall.

The current system, as far as I can tell, grades like this...
1: Winning
2: Kills*
3: Assists**
4: Various Other Actions

And instead swap it...
1: Assists**
2: Kills*
3: Various Other Actions
4: Winning

*Divided up into Solo/Most Damage Done/Last Hit
**Divided up into Scouting/Target/Tag/Narc

If you promote players to work as a team, versus just winning overall with, lets say 1 lance to be fair, carrying the weight of the team then you have a more accurate description of Player Skill Rating and not an arbitrary Team Skill (That Changes Based on The Current, Randomly Assigned Team You Get) Rating. Winning should not be the deciding factor of a PLAYERS skill rating, that's what W/L Ratio is for, but the icing on the cake for bonus points on top of what they, should have already, earned.

But this is all just me bitching about the same, broken system that is in every game today.

tl:dr PSR should reflect Player Skill Rating and not Win-Loss Ratio.

#33 NomadicCanuck

    Member

  • PipPipPip
  • The 1 Percent
  • 53 posts

Posted 25 September 2015 - 10:26 AM

I previously mentioned my pilot that has two accounts; His solo-queue account has a claimed 1.44 W/L ratio and he has a higher KDR than his regular account (and mine)... yet he's still listed in Tier 3 below both his regular account (and mine.)

Apparently it's not enough to JUST win, you must win and do well... The results I just mentioned implies that being alive at the end does not always equate to a good match.
*assuming PSR works as advertised.

#34 NomadicCanuck

    Member

  • PipPipPip
  • The 1 Percent
  • 53 posts

Posted 25 September 2015 - 12:54 PM

I read the update on the PSR and got to thinking... See if there's merit to this theory:

If you're a Tier2 pilot, and you play with a friend who is Tier4. Your average is Tier3 and you would play with other tier3's. Lets assume your W/L is 1:1 ... when you win, the Tier2 pilot (in theory) does well and he moves up. The tier4 pilot has a hard time competing against the tier3's and may end up equal. When you lose, the Tier2 pilot likely continues to excel and the tier4 pilot loses points. The combined 'friendship average' remains similar in Tier3 while the actual PSR widens... eventually, It's Tier 1 and Tier 5 playing in an average Tier3 drop.

The fundamental argument is that the Tier4 pilot *should* improve, breaking the cycle...I'm thinking that players plateau at a certain point and that improvement may never come. The option? Stop playing with your high-scoring friends.

Just a theory...

#35 RolfS

    Member

  • PipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 134 posts

Posted 25 September 2015 - 03:15 PM

I heard Tier-1 is supposed to be top 20% of the game supposedly 10.000 player base. I guess since I was around place 4000 in the last event I should expect to be low tier 2 or high tier 3 but I am low tier 4.
I play rubbish mechs like black knight and shadow cats quite a bit which means whatever that "method" is supposed to be it will never be accurate when you play bad mechs.
The tier system needs unranked games to allow you to play weak mechs....

The tier system is simple making things worse..... it is that bad

#36 Greyboots

    Member

  • PipPipPipPipPipPip
  • Liquid Metal
  • Liquid Metal
  • 396 posts

Posted 25 September 2015 - 03:26 PM

View PostNomadicCanuck, on 24 September 2015 - 04:24 AM, said:

Hi All,
My PSR/Tier seems to be accurately reflecting my skill level. However, I took a quick survey of my unit and the Tiers seemed "off" by a little for some pilots and "way out of whack" for one or two pilots in particular.

The end result is that I have some skilled veterans playing in Tier 4 against new pilots.
The end result is that I have a skilled veteran who has been on hiatus / infrequent play who is in Tier 5 with the steering wheel brigade.


The first reason for this is because of the arbitrary system used move from ELO to PSR. It wasn't a straight swap, they used server data to calculate PSR. Because they are not the same in terms of variables some players will be shifted from where they used to be to places that might not make sense.

This was explained somewhere or other. If I happen across the link I'll come back with it.

Quote

The end result is that I have one pilot who plays often in the group under one account and is Tier 2, but his alter-account (he plays infrequently when he wants to play alone) is Tier 3. Same pilot skill, same mechs... different result.


Ok, what you need to understand about this one:

Win/loss matters. On a loss you need a match score of about 400 to go up in PSR. For a win I think it's about 120.

If you get a 400 match score on a win you go up in PSR more than 400 in a loss. You can play CONSISTENTLY and the berks around you still make a large difference to what happens to your PSR.

A noob who plays with an elite team that always wins, shoots a bit, kills something occasionally etc will go up in PSR EVERY TIME on a match score of about 150.

A skilled player dumped in with yahoos, on the other hand, can constantly HOLD STEADY on a match score of 390 if the yahoos always lose.

This is in spite of the fact that it's actually harder to get a match score of 390 when teamed with yahoos than it is to get 150 when with a skilled team.

The idea that this system rewards a player based on their personal skill is simply not true. The system naturally accounts for the servers they play on, the time of day they play, whether they drop in teams or solo and so on.

For example, at certain times of day on certain servers my match score is, on average, approximately 50% higher than other times in spite of my win/loss ratio remaining about the same. This will impact on my PSR because on a win I will go up MORE while on a loss I will go up more frequently or go down less.

Who a player plays with and against will naturally vary any given player's PSR.

Quote

We are drawing a conclusion that your PSR seems to be linked to the frequency of play as much as skill.


Sort of. The more you play a given account the more opportunity you have to go up. You start at X so a tier 1 player playing 2 different accounts, the one plays most frequently will reach tier 1 first. In THEORY, it will take about the same number of MATCHES to reach the top tier on both accounts provided teamed with players that maintain their win/loss ratio.

Quote

These Tier misalignments have lead to some of my middle-of-the-road pilots serving out some whopping scores against Tier5's... no fun for either party there.


Just to restate this one: The changeover was expected to create these kinds of problems and it will take time for things to even out.

Quote

My question to you guys, what have you found in your units? Are your infrequent pilots accurately ranked/tiered? Are your most active pilots your highest rank? Is this reasonable?


We are a small unit but it played out mostly as expected.

Quote

Since the PSR algorithm is hidden and PSR values are not public... it is really difficult to accurately tell if PSR is broken already.

-- NomadicCanuck.


Well, not "broken" but it's not going to make a lot of sense to people until they understand it's got little to do with player skill and everything to do with match performance + time. Someone who only ever pugs and attains tier 3 may very well actually be a better "player" than someone who only ever drops in 12 mans and is at tier 1 but only because they are riding the other 11 players.

It should be remembered that this is to balance matches, not measure an individual player's actual skill. These are not one and the same. .

Edited by Greyboots, 25 September 2015 - 03:29 PM.


#37 Ironwithin

    Member

  • PipPipPipPipPipPipPipPipPip
  • Overlord
  • Overlord
  • 2,613 posts
  • LocationGermany

Posted 25 September 2015 - 05:27 PM

View PostRolfS, on 25 September 2015 - 03:15 PM, said:

I heard Tier-1 is supposed to be top 20% of the game supposedly 10.000 player base.
...


Where'd you get that number from, your magic 8-ball ?

In the territorial challenge event I played one match, on the last day of the event and had a ****** score that barely qualified in public queue (232 points). I got ranked at 22.725th place.

#38 Tesunie

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The Seeker
  • The Seeker
  • 8,586 posts
  • LocationSeraphim HQ: Asuncion

Posted 25 September 2015 - 07:59 PM

View PostIronwithin, on 25 September 2015 - 05:27 PM, said:


Where'd you get that number from, your magic 8-ball ?

In the territorial challenge event I played one match, on the last day of the event and had a ****** score that barely qualified in public queue (232 points). I got ranked at 22.725th place.


I did a bit more than that, and was placed in 14,731 place. Unless I'm ranked in the "invisible ghostly brigade", there are more than 10,000 players.

And 20% is not correct either. PSR ranks are not set to a specific percentage of players per tier. It's based on a specific number interval. If your PSR is high enough, then you are rank 1 no matter how many other players are also rank 1.

They have not released exact PSR ranking numbers, nor what numbers are required to fill in a specific rank.

#39 Wayreth

    Member

  • PipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 109 posts

Posted 25 September 2015 - 08:50 PM

View PostTesunie, on 25 September 2015 - 07:59 PM, said:

They have not released exact PSR ranking numbers, nor what numbers are required to fill in a specific rank.


They already stated they had no intentions of doing so. Probably due to all of the bad math they have to do before they get to their absurd rating systems. Oh and btw people it has been proven that PSR does not take into effect for team fragging, just damage done. Period.

#40 Tesunie

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The Seeker
  • The Seeker
  • 8,586 posts
  • LocationSeraphim HQ: Asuncion

Posted 25 September 2015 - 09:22 PM

View PostWayreth, on 25 September 2015 - 08:50 PM, said:


They already stated they had no intentions of doing so. Probably due to all of the bad math they have to do before they get to their absurd rating systems. Oh and btw people it has been proven that PSR does not take into effect for team fragging, just damage done. Period.


Damage done to teammates reduces your match score. You can see how much damage you've caused to an ally. Killing a teammate also will drop your match score as well. Not to mention a team kill and team damage's affect on a possible match win, which reduces your chances of gaining PSR...

So in a way, it does. In a way, it doesn't.





15 user(s) are reading this topic

0 members, 15 guests, 0 anonymous users