The Pilot Skill Rating (PSR) is in effect since 18th of August 2015. But since 22nd of September, the PSR is also shown to the user and that's new. We never knew about our Elo data but the PSR is viewable now.
This thread is about how this PSR works. Even if we know our value of PSR, we don't know about the formula, we don't know anything about the dispersion of players (I think of a pyramid where Tier 1 players are on top and Tier 5 players are the broad basement). These information would be very interesting to make PSR a real valid data.
I was disappointed in the beginning that everyone discussed about the question of showing the PSR or not. This question is totally uninteresting for me compared to the question how it really works. PSR don't say much about you if you don't know the formula or the dispersion. For example:
Tier 1 means 1% of the player base or 10%?
Tier 5 means 20% of the player base or 50%?
What has more weight: kills or damage or something else?
It's possible to reach Tier 1 with a well team-supporting light mech or it's easier to run a stupid constant firing assault mech instead?
These are the real exciting questions. At least for me.
Let's see what kind of information we got right now:
Spoiler
Pilot Skill Rating - The Elo Replacement
The purpose of the new Pilot Skill Rating (PSR) system is to make your pilot skill values based more on individual performance, rather than the binary win or loss outcome of a match. This new PSR system will effectively remove the current Elo system from the game and the matchmaker.
The best way to explain how the PSR system works and why it is being implemented is to first explain the shortcomings of the Elo system in relation to its use for MechWarrior Online.
Elo was originally developed for measuring the relative skill levels of competitive chess players, but over time its use has expanded to an array of other competitive games from football to collectable card games. A players’ Elo score is derived from their win/loss rate, and is compared against their opponents' Elo score to determine which of the two scores should rise and which should fall after the match has reached a conclusion.
Therein lies the critical failure point when using the Elo scoring system for MWO.
MWO is not a 1v1 game. It is, currently, a 12v12 team-based game. Beyond your own performance, the outcome of a match is fundamentally reliant on the performance of your teammates. Their level of skill or experience would have a significant impact on determining whether your Elo would rise or fall after a match had concluded.
One of the impacts of this was that organized and effective teams could potentially carry a player to an Elo score well beyond what may befit their actual skill level. In this way and others, the Elo system caused all sorts of issues with the matchmaker in both solo and group queues, and would never accurately reflect how a player may perform on a personal basis.
To correct this problem, we have implemented a new scoring system based primarily on your individual performance within a match.
Any player action in the game that triggers a C-Bill or XP reward (such as kills, assists, or damage done) now applies to this scoring system. Skill rating in MWO is no longer determined solely by your win/loss rate. The Match Score displayed at the End of Round screen is now the primary metric that will determine a player’s skill rating.
We use the action triggers in a weighted formula that employs multipliers to place greater emphasis on certain actions over others. We have also added more action triggers to the tracking system to take into account other types of gameplay aside from just kills and damage done. Actions such as flanking an enemy, using a UAV to spot targets, or using TAG/NARC/BAP to mark targets, are now all part of this formula. These and other actions are weighted in such a way that promotes effective team play.
Essentially, the weighting system and new action triggers allow for a player who may excel in a scouting or support role to be just as well-rated as a player who excels at securing kills and dealing heavy damage.
How does this new system work in terms or raising or lowering skill ratings?
First off, the win/loss condition plays a much smaller role compared to the current Elo system.
With the new PSR system:
If a players' team wins, and the player did well during the match (achieving a high Match Score), the player will rise in skill rating.
If a players' team wins, but the player did not perform well (achieving a low Match Score), the player will not move in terms of skill rating.
If a players' team loses, but the player does exceptionally well (achieving a very high Match Score), the player will go up slightly in skill rating.
If a players' team loses, but the player performed well (achieving a high Match Score), the player will not move in skill rating.
If the players' team loses, and the player performed poorly (achieving a low Match Score), they will drop in skill rating.
PSR and Skill Tiers
With the new PSR system in place we are now able to put players into Skill Tiers.
There are 5 Skill Tiers in which a pilot can be ranked: Tier 1 being comprised of the best-rated players in the game, and Tier 5 being comprised of pilots who are in dire need of additional training. That being said, we now have much more control over who plays against whom, without the presence of the ‘release valves’ that were in place under the original system.
The original Elo system for MWO employed these release valves to allow the matchmaker to relax restrictions in an effort to start games faster. This could result in top-tier players being matched with newer or lower-skilled players. With the release of PSR, that system is now gone.
A Tier 1 player will never play against a Tier 4 or Tier 5 player.
New players are automatically seeded into the mid-to-high range of Tier 4.
All current players will be seeded into whichever Skill Tier best aligns with the historical data already present in their account. The historical data pool used for generating your Tier placement goes back until January 2015.
The weighted formula and action triggers will not be released publicly. However, we are currently debating the merits of making all players’ respective Skill Tier levels publicly accessible. Part of this potential public Skill Tier listing is the possibility for opting-in or opting-out of listing your Skill Tier publicly. Tina will likely be putting up a poll to gather feedback about which option the community feels would be the best choice for handling public Skill Tiers.
Here are the most important information:
PSR is made for 12v12 team based game. Elo was made for 1vs1 games.
Elo was calculated with help of win/loss ratio only. PSR is based primarily on the individual performance.
Actions that trigger rewards (C-Bills or XP) now apply to PSR: kills, assists, damage done etc.
PSR allows to put players in five Skill Tiers. Tier 1 is high and Tier 5 is low.
The Skill Tiers will care for Tier 1 players will never play against a Tier 4 or Tier 5 player, for example.
New players are put in the mid-to-high range of Tier 4.
Current players will be put in whichever Skill Tier best aligns with their historical data since January 2015.
The formula and the action triggers are secret.
How does this new system work in terms or raising or lowering skill ratings?
With the new PSR system:
If a players' team wins, and the player did well during the match (achieving a high Match Score), the player will rise in skill rating.
If a players' team wins, but the player did not perform well (achieving a low Match Score), the player will not move in terms of skill rating.
If a players' team loses, but the player does exceptionally well (achieving a very high Match Score), the player will go up slightly in skill rating.
If a players' team loses, but the player performed well (achieving a high Match Score), the player will not move in skill rating.
If the players' team loses, and the player performed poorly (achieving a low Match Score), they will drop in skill rating.
You can watch your progress at the second page of the mission summary at the end of the match. Don't watch the team stats but your player stats.
Here are examples for the display:
PSR increased
Spoiler
PSR stays the same
Spoiler
PSR decreased
Spoiler
Another interesting point is the skill bar:
How long does it take to increase your tier from 5 to 4. And it's the same duration for the distance from tier 2 to tier 1?
How many "green arrow matches" we need for increasing our tier?
We also need answers here.
One last thought about the dispersion of the tiers.
This information would be interesting for further reflections about the PSR.
If there are only 1% Tier 1 players and only 5% Tier 2 players and 15% Tier 3 players, it's hard to promise that Tier 1 players will never meet Tier 4 or Tier 5 players.
But if the dispersion allows the high skill tier players to play with other high skill tier players only because there are 10% Tier 1 players, 20% Tier 2 players and 30% Tier 3 players, it's really possible that the matchmaker don't need to build a match with Tier 1 and Tier 4 players.
It's all about the dispersion and that's why we need this information.
We need the formula!
We need information about the dispersion of the tiers!!
We need to know about the skill bar and the duration to raise tiers!!!
For more information about the old Elo system. Unfortunately it's written in german but you can find the original links and the formula. matchmaking-und-elo/
If you really want to know the answers for your questions.. you will need datas... many-many-many datas, and some players, with free time and "math-power"!
1. Match Score Formula
Match Scores (with high possibility) are calculated from the Match Performance Datas and probably addicion to that from win/lose and dead/alive infos. (There was a Thread somewhere here, with the title of "For Science..."... you will surely find it.. that could be a good starting point).. So you will need many of those Full Datablocks (MS, MP (full!), W/L, D/A). It will be easier, if you can find some low skilled (T4/T5) players, because, their Datablocks are significantly shorter, then those from high skilled players.. That would be shorten the decoding process.. Than you can do some math..
2. Skill Bar
For that, first you must find out, what PGI exactly means, with "Very High/High/Low Match Score".. For that, you will need Match Score, W/L, and Drop/Unchanged/Rise datalines... (Many of them!)
If you have those six numbers, with their help you can exam the Skill Bar... For that, you will need a bunch of player, who have time and can somehow record ALL changes in the moving of their Skill Bar between two Tier-border.. (You will need several player-datas, from least the three middle Tier).. Than you can do some math..
3. Player "Pyamid"
That is the easiest point.. You should just start a player-poll, with the question "What is Your PSR Tier Number? (Vote for one number: 1-5)"... And after some hundred answer you will have a very nice estimation... (Actually, I think, it has a rombic form... so, we have just few T1/T5 players and lot of T3...) The more answer you have, the better the accuracy.
I know, you love such long-time projects.. so, I wish for you luck and lot of patience for it..
Thanks for all your feedback, it's really appreciated.
Catalina Steiner, on 23 September 2015 - 05:34 AM, said:
Another interesting point is the skill bar:
How long does it take to increase your tier from 5 to 4. And it's the same duration for the distance from tier 2 to tier 1?
How many "green arrow matches" we need for increasing our tier?
At least I found something in another PSR thread. I can't say if it's valid information but it's better than nothing.
Is it right, that a bad player how always drops with a good 12 player team (which wins most times of course) gets a better tier than a good player who drops in an bad team?
Is it right, that a bad player how always drops with a good 12 player team (which wins most times of course) gets a better tier than a good player who drops in an bad team?
That is how I have understood the system.
That was true for the old Elo system. It was win/loss based. But PSR is (theoretically) quite the opposite as you can see in the graphic above.
If your team wins and you perform bad, nothing happens. If your team loses and you perform (very) good, you have even a light chance to increase your rating.
Edited by Catalina Steiner, 23 September 2015 - 06:57 PM.
Do we know if the outcome of a battle is also modified by the enemy tier.
For example getting lucky shot as tier 1 facing mostly players from lower tiers is worst case
While playing exceptional facing higher tiers is best case.
PGI has said they are not giving out the formula, nor the dispersion of the tiers, other than a vague "banded set of tiers" and "hundreds of games to move between, unless you're extremely good, then only many games" type thing.
Do we have an idea of the threshold for getting a positive PSR score when you lose? I haven't been paying close attention, but I believe I hit 350 score in a loss and PSR still went down. I might not be remembering correctly, but it'll be interesting to see what PGI considers "a very high score" or whatever in a match.
Especially during events, people seem to think 200+ is pretty darn amazing (which it isn't).
Edited by Luscious Dan, 30 September 2015 - 10:51 AM.
Luscious Dan, on 30 September 2015 - 10:50 AM, said:
Do we have an idea of the threshold for getting a positive PSR score when you lose? I haven't been paying close attention, but I believe I hit 350 score in a loss and PSR still went down. I might not be remembering correctly, but it'll be interesting to see what PGI considers "a very high score" or whatever in a match.
Especially during events, people seem to think 200+ is pretty darn amazing (which it isn't).
I've had it come up equal between 300 and 350. Haven't had any plus losses yet.
Luscious Dan, on 30 September 2015 - 10:50 AM, said:
Do we have an idea of the threshold for getting a positive PSR score when you lose? I haven't been paying close attention, but I believe I hit 350 score in a loss and PSR still went down. I might not be remembering correctly, but it'll be interesting to see what PGI considers "a very high score" or whatever in a match.
Especially during events, people seem to think 200+ is pretty darn amazing (which it isn't).
I've had losses as low as 264 come up equal at Tier 4. I had a 413 in a loss that came up as a PSR increase. Been keeping track of that (Match Score, W/L, and net PSR effect) on the team's forum. So, I'm guessing the threshold for that would be 250 and 400. Just a hunch. Such seemingly arbitrary numbers wouldn't surprise me at all. Lowest WIN score I've had since this madness started was a 139, and that was still a net PSR increase. So, I'm guessing that threshold would be 125 or 100, and you'd have to REALLY dork up to drop in a winning match.
I just decided that I've got plenty of C-Bills, and so I can afford to expend the occasional consumable module. I've been scraping the extra XP off of MASTERed mechs into GXP for unlocking modules and efficiencies, so I'm now on the IMPROVED Artillery Strike, and will be rocking IMPROVED Air Strike soon too (already have the IMPROVED UAV, as well). Got my first H-H-Headshot achievement today with an Air Strike, so that paid for itself with interest.
Fret not, though. Soon the Crab will be released, and it'll be the MEDIUM queue that's 40-50% all day every day. THEN we'll have our fun, running up our PSRs in heavies against those mediocre IS mediums (except the early adopter variant with the JJs, which might be an alright mech). Followed by the sure-to-also-be-mediocre Wolfhound, which will leave the LIGHT queue saturated for at least five days.
So CHEER UP, fellow underhive dwellers! Our dawn is coming. The time will soon be at hand to step into the relative light of Tier 3, by the blood and tears and butthurt of so many fresh R2 mechs.
(And then the Marauder will drop, and we'll be back to a 20-minute wait for a heavy drop. At least it won't be full lances of TBRs, though.)
Catalina Steiner, on 23 September 2015 - 05:34 AM, said:
We need the formula!
We need information about the dispersion of the tiers!!
We need to know about the skill bar and the duration to raise tiers!!!
I have my suspicions/hypotheses on some of this. From another post, relevant portions bolded:
Quote
I think there's another issue in play here, though that could be easy to fix: It appears PGI set the tier cutoffs using hard PSR scores that aren't changing. This means as more players improve their PSR's, the number of players in each tier is shifting. This would be expected if players generally improve in skill over time.
PGI likely divided up the player base using a target % for each slice (which they've never discussed publicly) and seem to have left it alone after that initial pass.
What they could do instead is set each tier at the desired target fraction of the player base and periodically recalculate the PSR's needed to reach that tier. They could recalculate those PSR values daily if they wanted.
If they did this, rising PSR's by themselves wouldn't be a huge issue. To advance in tiers, your PSR must rise faster than the average of the player base. Otherwise you'll stay put or lose ground. Under the current system, eventually most people will be able to make it to Tier 1 and I don't think that was ever the intent.
6
This topic is locked
