57 replies to this topic

[Siriothrax]

Tarogato, on 29 September 2015 - 11:22 PM, said:

While outliers gonna lie out, they do at least reveal one of the weaknesses of the system. It takes many many games (I think Paul said it could be as much as a thousand) for a player to level out in the tier they deserve. In my opinion, a player should be properly rated well within his first 100 games.

Siri, have you probably played 100 games since PSR in January? Maybe more than 50? Or literally zero?

Mmm, probably in the order of 60ish? 75 would likely be a high bound. After PSR patch dropped I was at about 90% tier 4 and had to play nine or ten games to get out of it, so those numbers sound about right. In any case, it's fair to say that it would take me a prettttty long time to get to tier 1 when guys like Ex Machiina, Gman, and the others trying to get a fresh account up aren't even close and I don't have the benefit of the cadet tier acceleration.

[Tarogato]

Siriothrax, on 29 September 2015 - 11:59 PM, said:

Mmm, probably in the order of 60ish? 75 would likely be a high bound.

Yeah, in my opinion, a good matchmaking system should already have you mostly where you belong, even with only 60-75 matches. You should at least be at the top of Tier 2 for that performance record.

Think about it... out of the 200-odd players I surveyed, the average pilot had 3,903 matches played. I wish I had that dev quote handy, but I think it said something like 1,000 for a player to "grind" to their appropriate tier. That's 25% of all the games the average player actually played. One quarter of your time spent essentially... seeding. And if you improve as a player, your progress through the tiers will be glacial.

117 matches would be only 3% of that "total matches" figure, which I think is a more reasonable amount of time for a pilots performance record to stabilise and be placed in the appropriate tier for their skill level. For comparison, when I started tracking my personal stats in an external spreadsheet, it took about 150 matches for my WLR to level out and stabilise at a number that agreed with my actual game stats. By that time, I should already be properly tier-sorted.

I'm curious how many games indeed it will take for the streamers to reach their tier goals from fresh accounts, even being among the best players in the game. I haven't been keeping track of their progress.

[Siriothrax]

Tarogato, on 30 September 2015 - 12:33 AM, said:

Yeah, in my opinion, a good matchmaking system should already have you mostly where you belong, even with only 60-75 matches. You should at least be at the top of Tier 2 for that performance record.

Think about it... out of the 200-odd players I surveyed, the average pilot had 3,903 matches played. I wish I had that dev quote handy, but I think it said something like 1,000 for a player to "grind" to their appropriate tier.

Here ya go: https://www.reddit.c..._to_settle_out/

Silly Taro, you know there are filters on the sub for things like news, right?

That actually made me quite interested in the outlier at the bottom of tier 1 - just shy of 2k games played - so I went and had a look at the data. 1770 played, 3.0 kdr, 1.83 W/L. A pretty good record, and possibly legit if all of those matches were from this year. I think that might give a bit of a lie to Paul's statement that you would have to "magically win every game you played", and illustrates that the PSR score adjustment is not performing to even PGI's stated parameters.

Proof in the pudding, we'll have to wait and see how these fresh alt runs turn out before we can call it out for real.

...dammit, now I want Bubble Tea at 2 am. Thanks, Taro. Grr.

Edited by Siriothrax, 30 September 2015 - 12:54 AM.

[The Great Unwashed]

All the alt accounts are (presumably) by Tier 1 players who'll probably drop in teams, so you'd expect that the stats will not reflect the learning curve inherently present for newer players and lower W/L of PUGs. So, they'll push the system a bit. And even if they require a thousand matches to get to Tier 1, that still does not prove that the PSR is entirely an XP collection game.



Even when you do modestly and have a W/L of around 1.0, you will not get far ahead with modest scores (changes of a large drop are 50%) while you'll do well with good matches in any case. I think it's in the medium scores where the positives and the negatives appear to favor the increase in PSR and you'll trickle into the higher tiers.

Anyway, the results from the poll show a bias towards a W/L>1 while the population should average out to W/L=1.0; so careful with the conclusions.

[Tarogato]

The Great Unwashed, on 30 September 2015 - 01:06 AM, said:

Anyway, the results from the poll show a bias towards a W/L>1 while the population should average out to W/L=1.0; so careful with the conclusions.

That's the bit that bothers me. Theoretically, if matchmaking were perfect, all the Tier 1 pilots would only fight other Tier 1 pilots. Among themselves in their own little island of tryhard, they'd all average out to a 1.0 W/L. But they'd still have more wins than losses on their record because of the leftovers from previous tiers. Once upon a time they were tier 4, 3, and 2, and they had to win more games than lose to get up to Tier 1. The question is, are the WLRs we're seeing at tier 1 indicative of those leftovers? Even among the "mostly" and "only" solo queue players, a number of whom have 1.20 to exceptional 1.60+ W/L? Also, many of the very highest WLRs overall don't necessarily belong to people who spend a lot of time stomping around in large groups - plenty are people who attest to only playing in groups of 2-4. So individual skill factor/influence is really coming into play here - having at least as much an effect as large group play does on people's W/L records.

Still doesn't answer the question of where all the neg ratio players are. Technically, if Tiers 1, 2, and 3 are fighting among themselves, half of them altogether should be negative. Unless those neg players get kicked down to Tier 4. If all the neg players fell down to Tier 4 and below, then who all are the Tier 1, 2, and 3 players still actually defeating to maintain their high WLRs? It's all really odd.

Edited by Tarogato, 30 September 2015 - 01:25 AM.

[The Great Unwashed]

I suppose Tier 1 can consist of players who all have a positive (W/L-1) ratio, as they compensate by beating all the Tier 2 players, who in their turn compensate using Tier 3 players until you are left people in Tier 5 who have a negative (W/L-1) ratio. This won't happen as many Tiers are put against each other so this is why the tiers overlap as they have to... But the question remains: who are these people who enjoy loosing all the time? Are they the newer players that do not stick around? Do they enjoy the game even when they loose?

I play both PUG and group and I do not mind loosing occasionally, but really dislike loosing all the time.

Let's assume that people in Tier 5 mainly compensate to absorb all the losses, and are newer players that missed your data request or, didn't really feel like sharing, introducing a strong bias in the Tier 2/3 people (because Tier 1 people are looking down from outreach, right?)

What would be nice if PGI repeats your analysis with their numbers and plots the results as smooth graphs... like sand people, hiding their numbers.

Edited by The Great Unwashed, 30 September 2015 - 01:54 AM.

[Karl Marlow]

Lol. Why exactly are you needing to collect stats to find the average wl ratio?

As to the rest. How much are you willing to pay for my stats?

[Duke Nedo]

Tarogato, on 29 September 2015 - 10:22 PM, said:

Although many did, not everybody gave me their exact number of wins and losses each. I was indeed planning on going back and grabbing that information later, though. Will probably be a few days before I get around to it, though - if I do at all.

No need to go back, you have the numbers in your excel file, just need to calculate it from the W/L ratio and total matches.

Best correlation so far I think. Since it's an absolute number I removed the guys that have saturated their rating at 1.0 from the line fit. In order to improve the fit I think you'd need access to everyones average score per match... perhaps KDR could be factored in as a substitute for that.


Excess wins plotted versus "exact" tier. (excess wins = nr of wins - nr of losses)



All in all, very interesting read! Thanks for putting in the work and for sharing the raw data!!

The outliers in this plot would be people who either plays with a good group and wins a lot, but contributes really poorly themselves, or the reverse, people who plays with small/horrible group and lose a lot but carries like a champ.

Edit: Interesting fun fact here if you decide to trust this graph is that you can put your own nr of excess wins and your current tier and read out if you are ahead of your surrounding average in your progression (below the dotted line) or if you are generally pulling your team down (above the dotted line).

Edited by Duke Nedo, 30 September 2015 - 02:26 AM.

[Lefteye Falconeer]

How can someone with:

- 5150 played matches
- 1.83 KDR
- 1.10 W/L ratio

be stuck in tier 4? Even if the system wasn't biased the way it is, I don't understand this data entry.

Either there are some serious bugs in the system or this data is false/mistyped?

Same for:

- 3764 played matches
- 2.06 KDR
- 1.57 W/L ratio

still swamped in tier 3?

What is going on here? Theories?

Another question:

Do "Archived stats" count, or only "Current", after the reset?

[Duke Nedo]

Lefteye Falconeer, on 30 September 2015 - 02:45 AM, said:

How can someone with:

- 5150 played matches
- 1.83 KDR
- 1.10 W/L ratio

be stuck in tier 4? Even if the system wasn't biased the way it is, I don't understand this data entry.

Either there are some serious bugs in the system or this data is false/mistyped?

Same for:

- 3764 played matches
- 2.06 KDR
- 1.57 W/L ratio

still swamped in tier 3?

What is going on here? Theories?

Another question:

Do "Archived stats" count, or only "Current", after the reset?

Easy, stat reset was long before PSR started to track data so if you stopped playing and dropped very few matches in 2015, then your PSR will be based on very very few games...

[Davegt27]

I am probably skewing the results

Kills / Death 7,107 / 7,949
C-Bills 113,327,802
Experience Points 9,025,269
Wins / Losses 5,450 / 5,396
Kill / Death Ratio 0.89


In 1 year and 3 months I have 10,846 matches

My K/D was .92 in June it has been going down like crazy

I wonder how many matches I would need to win to get me where the average tier 3 players K/D ratio is?

Oh wait I am already tier 3 with a .89 K/D

[Thorqemada]

Now what is missing is IS Performance vs Clan Performance.

[Duke Nedo]

Another observation about PSR and how it relates to personal skill.

Average Win/Loss ratio for:
Mostly Group drops: 1.46
Mostly Solo drops: 1.28
Only solo drops: 1.16

[Daelen Rottiger]

Thx OP - great job!

[TheStrider]

Until they split our stats for group games from solo games, bot stats and rating systems will be skewed.

[Mazzyplz]

Duke Nedo, on 30 September 2015 - 02:16 AM, said:

No need to go back, you have the numbers in your excel file, just need to calculate it from the W/L ratio and total matches.

Best correlation so far I think. Since it's an absolute number I removed the guys that have saturated their rating at 1.0 from the line fit. In order to improve the fit I think you'd need access to everyones average score per match... perhaps KDR could be factored in as a substitute for that.


Excess wins plotted versus "exact" tier. (excess wins = nr of wins - nr of losses)



All in all, very interesting read! Thanks for putting in the work and for sharing the raw data!!

The outliers in this plot would be people who either plays with a good group and wins a lot, but contributes really poorly themselves, or the reverse, people who plays with small/horrible group and lose a lot but carries like a champ.

Edit: Interesting fun fact here if you decide to trust this graph is that you can put your own nr of excess wins and your current tier and read out if you are ahead of your surrounding average in your progression (below the dotted line) or if you are generally pulling your team down (above the dotted line).

with your graph here we can finally see the "steps" on the ladder;
we can (i think!) make some predictions and read between the lines and see the missing data

[Duke Nedo]

Mazzyplz, on 30 September 2015 - 05:37 AM, said:

with your graph here we can finally see the "steps" on the ladder;
we can (i think!) make some predictions and read between the lines and see the missing data

The equation for the line is: y = -229.4x + 1037.1, so if you follow that, for a fresh player to reach the lower limit of Tier 1 it would take in the order of ~ (4.2-2.0) x 230 = 506 excess wins.

For someone starting now, this value will be off by a percentage depending on when they started to record PSR-data relative to the stat-reset. I don't remember either of these dates, but if we would just assume about 50% of the time since stat reset was used to seed PSR, then a fresh cadet would need to gather some 250 more wins than losses to reach the boundry between Tier 1 and Tier 2.

For a good solo player at a 1.2 W/L ratio, that corresponds to some 2700 games.
For a new player that learns along the way with a 1.1 W/L ratio it would take some 5250 games.
For a player playing with a good group with a 2.0 W/L ratio it could take less than 750 games.

[jay35]

Tarogato, on 30 September 2015 - 12:33 AM, said:

Yeah, in my opinion, a good matchmaking system should already have you mostly where you belong, even with only 60-75 matches. You should at least be at the top of Tier 2 for that performance record.

It all depends on the greater objective. Some people enjoy the journey more than the destination. Providing a longer path to get to the top means a gentler ascent and time to actually bring one's skill up along the way. Shoving people right to the top quickly means they get burned out quicker if all they're fighting is stiff competition every single match.

So if the goal is simply to label people, then sure, get them to Tier 1 as quickly as possible. If the goal is to create a journey and a grind, then it's better that it takes longer to get there and it feels like more of an accomplishment to arrive there.

Edited by jay35, 30 September 2015 - 07:52 AM.

[Noober]

Lefteye Falconeer, on 30 September 2015 - 02:45 AM, said:

- 3764 played matches
- 2.06 KDR
- 1.57 W/L ratio

still swamped in tier 3?

What is going on here? Theories?

*Shifty eyes*

Can confirm this is due to "that player" not playing much this year.

I'd love to see the comparison of last known Elo to tier.

BTW: The bonus of being seeded into tier 3 is that you can get away with utter nonsense:

http://imgur.com/Plqpwog

http://imgur.com/2dV1M7I

http://imgur.com/z1ydIBq

http://imgur.com/5mx5tRu

http://imgur.com/t0iK4JK

Full Album:
http://imgur.com/a/JHVk4

Its great to play locust again not feeling like I'm hurting my team!

[Death Proof]

Tarogato, on 30 September 2015 - 12:33 AM, said:

Think about it... out of the 200-odd players I surveyed, the average pilot had 3,903 matches played. I wish I had that dev quote handy, but I think it said something like 1,000 for a player to "grind" to their appropriate tier.

As of today I have 652 matches logged. About 40 of those matches I've played since tiers went public.

I was seeded about 40% into T4. After comparing screenshots from last week with my current progress, it appears I've been increasing at a rate of approximately 0.75% per match on average. So I estimate that I'll be T3 within 30-35 more matches. So about 70-75 matches total to clear T4 from 40%.

I'm not much of a Mathy McMatherson like y'all, but I reckon that's far less than a 1000 matches to get to my appropriate tier...unless my appropriate tier isn't T3. I'm curious to see how drastically my progress decreases once I hit T3 (which I expect will be substantial).

Current Stats:

652 Matches
0.87 W/L (303/349)
1.22 K/DR (573/468)