Jump to content

[Guide] Pilot Skill Rating (PSR) - How it works

guide psr

62 replies to this topic

#1 Catalina Steiner

    Member

  • PipPipPipPipPipPipPipPipPip
  • Stone Cold
  • Stone Cold
  • 2,119 posts
  • Google+: Link
  • LocationNagelring Academy

Posted 23 September 2015 - 05:34 AM

The Pilot Skill Rating (PSR) is in effect since 18th of August 2015. But since 22nd of September, the PSR is also shown to the user and that's new. We never knew about our Elo data but the PSR is viewable now.

This thread is about how this PSR works. Even if we know our value of PSR, we don't know about the formula, we don't know anything about the dispersion of players (I think of a pyramid where Tier 1 players are on top and Tier 5 players are the broad basement). These information would be very interesting to make PSR a real valid data.

I was disappointed in the beginning that everyone discussed about the question of showing the PSR or not. This question is totally uninteresting for me compared to the question how it really works. PSR don't say much about you if you don't know the formula or the dispersion. For example:
Tier 1 means 1% of the player base or 10%?
Tier 5 means 20% of the player base or 50%?
What has more weight: kills or damage or something else?
It's possible to reach Tier 1 with a well team-supporting light mech or it's easier to run a stupid constant firing assault mech instead?

These are the real exciting questions. At least for me.

Let's see what kind of information we got right now:
Spoiler



Here are the most important information:
  • PSR is made for 12v12 team based game. Elo was made for 1vs1 games.
  • Elo was calculated with help of win/loss ratio only. PSR is based primarily on the individual performance.
  • Actions that trigger rewards (C-Bills or XP) now apply to PSR: kills, assists, damage done etc.
  • PSR allows to put players in five Skill Tiers. Tier 1 is high and Tier 5 is low.
  • The Skill Tiers will care for Tier 1 players will never play against a Tier 4 or Tier 5 player, for example.
  • New players are put in the mid-to-high range of Tier 4.
  • Current players will be put in whichever Skill Tier best aligns with their historical data since January 2015.
  • The formula and the action triggers are secret.

How does this new system work in terms or raising or lowering skill ratings?

Posted Image

With the new PSR system:
  • If a players' team wins, and the player did well during the match (achieving a high Match Score), the player will rise in skill rating.
  • If a players' team wins, but the player did not perform well (achieving a low Match Score), the player will not move in terms of skill rating.
  • If a players' team loses, but the player does exceptionally well (achieving a very high Match Score), the player will go up slightly in skill rating.
  • If a players' team loses, but the player performed well (achieving a high Match Score), the player will not move in skill rating.
  • If the players' team loses, and the player performed poorly (achieving a low Match Score), they will drop in skill rating.
You can watch your progress at the second page of the mission summary at the end of the match. Don't watch the team stats but your player stats.

Here are examples for the display:

PSR increased
Spoiler


PSR stays the same
Spoiler


PSR decreased
Spoiler



Another interesting point is the skill bar:
How long does it take to increase your tier from 5 to 4. And it's the same duration for the distance from tier 2 to tier 1?
How many "green arrow matches" we need for increasing our tier?
We also need answers here.


One last thought about the dispersion of the tiers.
This information would be interesting for further reflections about the PSR.
If there are only 1% Tier 1 players and only 5% Tier 2 players and 15% Tier 3 players, it's hard to promise that Tier 1 players will never meet Tier 4 or Tier 5 players.
But if the dispersion allows the high skill tier players to play with other high skill tier players only because there are 10% Tier 1 players, 20% Tier 2 players and 30% Tier 3 players, it's really possible that the matchmaker don't need to build a match with Tier 1 and Tier 4 players.
It's all about the dispersion and that's why we need this information.

Posted Image




Posted Image
We need the formula!
We need information about the dispersion of the tiers!!
We need to know about the skill bar and the duration to raise tiers!!!









For more information about the old Elo system. Unfortunately it's written in german but you can find the original links and the formula.
matchmaking-und-elo/

Source of patch notes:
patch-notes-18aug2015

#2 Chados

    Member

  • PipPipPipPipPipPipPipPip
  • 1,948 posts
  • LocationSomewhere...over the Rainbow

Posted 23 September 2015 - 06:49 AM

Nice work, Catalina. The Rocket Grrl strikes again! One of my favorite gurus.

It's good to see other women working the game.

#3 Karl Streiger

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Wrath
  • Wrath
  • 20,369 posts
  • LocationBlack Dot in a Sea of Blue

Posted 23 September 2015 - 07:22 AM

Nice topic.

Give me formulas or give me death (don't dare)

a summary about the tier population levels would be helpful either.

Edited by Karl Streiger, 23 September 2015 - 07:23 AM.


#4 Odanan

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 8,198 posts
  • LocationBrazil

Posted 23 September 2015 - 08:25 AM

Very good Catalina! This system is really much better than the previous one. Now I'm looking forward for info about the population of each tier...

#5 Dettmam

    Member

  • PipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 115 posts
  • LocationRio de Janeiro - Brazil

Posted 23 September 2015 - 12:28 PM

Agreed, Catalina!

We need to know if it pays to pilot a NARCer light mech!

#6 Sky Hawk

    Member

  • PipPipPipPipPipPipPip
  • 700 posts
  • LocationDeep Periphery, aka Hungary

Posted 23 September 2015 - 01:55 PM

Hello Lina!

If you really want to know the answers for your questions.. you will need datas... many-many-many datas, and some players, with free time and "math-power"!

1. Match Score Formula
Match Scores (with high possibility) are calculated from the Match Performance Datas and probably addicion to that from win/lose and dead/alive infos. (There was a Thread somewhere here, with the title of "For Science..."... you will surely find it.. that could be a good starting point).. So you will need many of those Full Datablocks (MS, MP (full!), W/L, D/A). It will be easier, if you can find some low skilled (T4/T5) players, because, their Datablocks are significantly shorter, then those from high skilled players.. That would be shorten the decoding process.. Than you can do some math..

2. Skill Bar
For that, first you must find out, what PGI exactly means, with "Very High/High/Low Match Score".. For that, you will need Match Score, W/L, and Drop/Unchanged/Rise datalines... (Many of them!)
If you have those six numbers, with their help you can exam the Skill Bar... For that, you will need a bunch of player, who have time and can somehow record ALL changes in the moving of their Skill Bar between two Tier-border.. (You will need several player-datas, from least the three middle Tier).. Than you can do some math..

3. Player "Pyamid"
That is the easiest point.. You should just start a player-poll, with the question "What is Your PSR Tier Number? (Vote for one number: 1-5)"... And after some hundred answer you will have a very nice estimation... (Actually, I think, it has a rombic form... so, we have just few T1/T5 players and lot of T3...) The more answer you have, the better the accuracy.

I know, you love such long-time projects.. so, I wish for you luck and lot of patience for it..

#7 RagingOyster

    Member

  • PipPipPipPipPipPip
  • Bad Company
  • Bad Company
  • 462 posts
  • LocationBaltimore, Maryland

Posted 23 September 2015 - 02:06 PM

Excellent, pertinent information presented very well.

This post should be stickied so everyone can view it easily.

#8 Catalina Steiner

    Member

  • PipPipPipPipPipPipPipPipPip
  • Stone Cold
  • Stone Cold
  • 2,119 posts
  • Google+: Link
  • LocationNagelring Academy

Posted 23 September 2015 - 02:45 PM

Thanks for all your feedback, it's really appreciated.



View PostCatalina Steiner, on 23 September 2015 - 05:34 AM, said:

Another interesting point is the skill bar:
How long does it take to increase your tier from 5 to 4. And it's the same duration for the distance from tier 2 to tier 1?
How many "green arrow matches" we need for increasing our tier?


At least I found something in another PSR thread. I can't say if it's valid information but it's better than nothing.

Posted Image

#9 DeepSky

    Rookie

  • Bad Company
  • 1 posts

Posted 23 September 2015 - 03:01 PM

Is it right, that a bad player how always drops with a good 12 player team (which wins most times of course) gets a better tier than a good player who drops in an bad team?

That is how I have understood the system.

#10 Peiper

    Member

  • PipPipPipPipPipPipPipPip
  • The Dragoon
  • The Dragoon
  • 1,444 posts
  • Google+: Link
  • LocationA fog where no one notices the contrast of white on white

Posted 23 September 2015 - 05:08 PM

Catalina, you're my hero!

Posted Image

#11 Catalina Steiner

    Member

  • PipPipPipPipPipPipPipPipPip
  • Stone Cold
  • Stone Cold
  • 2,119 posts
  • Google+: Link
  • LocationNagelring Academy

Posted 23 September 2015 - 06:56 PM

View PostDeepSky, on 23 September 2015 - 03:01 PM, said:

Is it right, that a bad player how always drops with a good 12 player team (which wins most times of course) gets a better tier than a good player who drops in an bad team?

That is how I have understood the system.


That was true for the old Elo system. It was win/loss based.
But PSR is (theoretically) quite the opposite as you can see in the graphic above.

If your team wins and you perform bad, nothing happens.
If your team loses and you perform (very) good, you have even a light chance to increase your rating.

Edited by Catalina Steiner, 23 September 2015 - 06:57 PM.


#12 Karl Streiger

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Wrath
  • Wrath
  • 20,369 posts
  • LocationBlack Dot in a Sea of Blue

Posted 23 September 2015 - 09:21 PM

Do we know if the outcome of a battle is also modified by the enemy tier.
For example getting lucky shot as tier 1 facing mostly players from lower tiers is worst case
While playing exceptional facing higher tiers is best case.

#13 Sky Hawk

    Member

  • PipPipPipPipPipPipPip
  • 700 posts
  • LocationDeep Periphery, aka Hungary

Posted 25 September 2015 - 06:53 AM

Lol.. Paul try to explain PSR again...

http://mwomercs.com/...37#entry4719837

#14 Karl Streiger

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Wrath
  • Wrath
  • 20,369 posts
  • LocationBlack Dot in a Sea of Blue

Posted 28 September 2015 - 01:07 AM

View PostSky Hawk, on 25 September 2015 - 06:53 AM, said:

Lol.. Paul try to explain PSR again...

http://mwomercs.com/...37#entry4719837


and still only "pictures" and vague formulation - not a single number
SO:

View PostCatalina Steiner, on 23 September 2015 - 05:34 AM, said:

We need the formula!
We need information about the dispersion of the tiers!!
We need to know about the skill bar and the duration to raise tiers!!!

Just a reminder

#15 Torezu

    Member

  • PipPipPipPipPipPip
  • Philanthropist
  • 329 posts

Posted 28 September 2015 - 08:58 AM

PGI has said they are not giving out the formula, nor the dispersion of the tiers, other than a vague "banded set of tiers" and "hundreds of games to move between, unless you're extremely good, then only many games" type thing.

#16 Sky Hawk

    Member

  • PipPipPipPipPipPipPip
  • 700 posts
  • LocationDeep Periphery, aka Hungary

Posted 28 September 2015 - 01:18 PM

It's not accurate (and not english), but least a start...

http://mwomercs.com/...-tier-seid-ihr/

And.. .http://mwomercs.com/...g/page__st__680


Ye... that is not even graphical, but looks like, with lots of data... (I wish, someone had a time to count it...)

#17 Luscious Dan

    Member

  • PipPipPipPipPipPipPipPip
  • The Privateer
  • The Privateer
  • 1,146 posts
  • Twitter: Link
  • LocationEdmonton, AB

Posted 30 September 2015 - 10:50 AM

Do we have an idea of the threshold for getting a positive PSR score when you lose? I haven't been paying close attention, but I believe I hit 350 score in a loss and PSR still went down. I might not be remembering correctly, but it'll be interesting to see what PGI considers "a very high score" or whatever in a match.

Especially during events, people seem to think 200+ is pretty darn amazing (which it isn't).

Edited by Luscious Dan, 30 September 2015 - 10:51 AM.


#18 Tim East

    Member

  • PipPipPipPipPipPipPipPip
  • Shredder
  • Shredder
  • 1,422 posts

Posted 30 September 2015 - 11:42 AM

View PostLuscious Dan, on 30 September 2015 - 10:50 AM, said:

Do we have an idea of the threshold for getting a positive PSR score when you lose? I haven't been paying close attention, but I believe I hit 350 score in a loss and PSR still went down. I might not be remembering correctly, but it'll be interesting to see what PGI considers "a very high score" or whatever in a match.

Especially during events, people seem to think 200+ is pretty darn amazing (which it isn't).

I've had it come up equal between 300 and 350. Haven't had any plus losses yet.

#19 Not A Real RAbbi

    Member

  • PipPipPipPipPipPipPipPip
  • Survivor
  • Survivor
  • 1,688 posts
  • LocationDeath to Aladeen Cafe

Posted 30 September 2015 - 09:17 PM

View PostLuscious Dan, on 30 September 2015 - 10:50 AM, said:

Do we have an idea of the threshold for getting a positive PSR score when you lose? I haven't been paying close attention, but I believe I hit 350 score in a loss and PSR still went down. I might not be remembering correctly, but it'll be interesting to see what PGI considers "a very high score" or whatever in a match.

Especially during events, people seem to think 200+ is pretty darn amazing (which it isn't).


I've had losses as low as 264 come up equal at Tier 4. I had a 413 in a loss that came up as a PSR increase. Been keeping track of that (Match Score, W/L, and net PSR effect) on the team's forum. So, I'm guessing the threshold for that would be 250 and 400. Just a hunch. Such seemingly arbitrary numbers wouldn't surprise me at all. Lowest WIN score I've had since this madness started was a 139, and that was still a net PSR increase. So, I'm guessing that threshold would be 125 or 100, and you'd have to REALLY dork up to drop in a winning match.

I just decided that I've got plenty of C-Bills, and so I can afford to expend the occasional consumable module. I've been scraping the extra XP off of MASTERed mechs into GXP for unlocking modules and efficiencies, so I'm now on the IMPROVED Artillery Strike, and will be rocking IMPROVED Air Strike soon too (already have the IMPROVED UAV, as well). Got my first H-H-Headshot achievement today with an Air Strike, so that paid for itself with interest.

Fret not, though. Soon the Crab will be released, and it'll be the MEDIUM queue that's 40-50% all day every day. THEN we'll have our fun, running up our PSRs in heavies against those mediocre IS mediums (except the early adopter variant with the JJs, which might be an alright mech). Followed by the sure-to-also-be-mediocre Wolfhound, which will leave the LIGHT queue saturated for at least five days.

So CHEER UP, fellow underhive dwellers! Our dawn is coming. The time will soon be at hand to step into the relative light of Tier 3, by the blood and tears and butthurt of so many fresh R2 mechs.

(And then the Marauder will drop, and we'll be back to a 20-minute wait for a heavy drop. At least it won't be full lances of TBRs, though.)

#20 Khereg

    Member

  • PipPipPipPipPipPipPip
  • Knight Errant
  • Knight Errant
  • 919 posts
  • LocationDenver, CO

Posted 02 October 2015 - 08:49 AM

View PostCatalina Steiner, on 23 September 2015 - 05:34 AM, said:


We need the formula!
We need information about the dispersion of the tiers!!
We need to know about the skill bar and the duration to raise tiers!!!



I have my suspicions/hypotheses on some of this. From another post, relevant portions bolded:

Quote

I think there's another issue in play here, though that could be easy to fix: It appears PGI set the tier cutoffs using hard PSR scores that aren't changing. This means as more players improve their PSR's, the number of players in each tier is shifting. This would be expected if players generally improve in skill over time.

PGI likely divided up the player base using a target % for each slice (which they've never discussed publicly) and seem to have left it alone after that initial pass.

What they could do instead is set each tier at the desired target fraction of the player base and periodically recalculate the PSR's needed to reach that tier. They could recalculate those PSR values daily if they wanted.

If they did this, rising PSR's by themselves wouldn't be a huge issue. To advance in tiers, your PSR must rise faster than the average of the player base. Otherwise you'll stay put or lose ground. Under the current system, eventually most people will be able to make it to Tier 1 and I don't think that was ever the intent.






1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users