57 replies to this topic

[C E Dwyer]

N.P.E is more likely to be bad, because they can advance to quickly, again pointing to PSR as the culprit, unless they are truly bad players.

P.G.I have provided what is a comprehensive package for the new player, it's better than any other F.P.S I play by a long way.

One thing P.G.I can't control are the people not using it, thinking they can pick things up as they go along and then not, because they become fodder, hen they should be learning to control their mech

[Nik Reaper]

The part about vets being not above the "low" average while using non meta builds and strategy is people not competing , at least not for real, and we would need a well incentivized "ranked" queue for those who want to always play to the best of there ability when in that queue , no test builds, no unmastered mechs and no Leroy plays, where you expect the other team doing the same, more accent put on bonus Cbills and xp based on performance and a lower base payout, so that those who want easy money stay clear.

[NighthawK1337]

How about we add overall population as a deciding factor for tier placement? T1 gets 10%, T2 gets 20%, T3 gets 40% T4 gets 20%, T5 gets 10%. This should at least reduce the variance of skill for T1. Also a decaying PSR with respect to time would also help. This would keep hardcore win-hungry meta players at T1 while casual scrubs who likes to have fun (like me) at T2.

This would also make sure that only the higher percentile people who keeps playing gets to T1 instead of everybody who just plays a lot because correlation =/= causation.

Edited by NighthawK1337, 17 January 2017 - 03:20 PM.

[Bud Crue]

Tarogato, on 17 January 2017 - 01:32 PM, said:

Which unveils that only 5% of players are above ~290 average match score, which about where I'd consider a threshold for "actually good at the game". Similarly, 75% of players are ~230 match score or lower, which is what I would personally consider "bad." And the average player is actually only 200 average match score. Anything at or below that is really atrociously bad in my opinion. Like... I question if you're even having fun playing the game if you consistently score that poorly. Yet that's actually 50% of people that play this game...

50% of the player population consistently does "atrociously bad" and most of the rest not doing a whole lot better. Hmm...almost makes one think that perhaps there are people who are simply not physically capable of scoring high in this game and/or whose sense of fun is not totally dependent on winning or doing particularly well...or maybe they just suck at this (raises hand).

Seems to me that if your distribution model is remotely accurate then perhaps PGI ought to think about a two tier match maker rather than our current five tier system. With the above average players in one and the below average players in the other, with the two crossing into each other only after several minutes of wait; or maybe even a two tier system with your top 5% in their own and the rest in the other?

[Nik Reaper]

NighthawK1337, on 17 January 2017 - 03:09 PM, said:

How about we add overall population as a deciding factor for tier placement? T1 gets 10%, T2 gets 20%, T3 gets 40% T4 gets 20%, T5 gets 10%. This should at least reduce the variance of skill for T1. Also a decaying PSR with respect to time would also help. This would keep hardcore win-hungry meta players at T1 while casual scrubs who likes to have fun (like me) at T2.

This would also make sure that only the higher percentile people who keeps playing gets to T1 instead of everybody who just plays a lot because correlation =/= causation.

This kinda sounds good, but the problem remains... safety valves that open like a night walkers legs X) , population is still a problem and even if only the truly good few were in tier 1 they would still end up in matches with us under this MM... witch is also a problem of having a dedicated try hard queue , not sure if there are 24 good players on at all times X) .

Edited by Nik Reaper, 17 January 2017 - 03:30 PM.

[Tarogato]

Bud Crue, on 17 January 2017 - 03:21 PM, said:

Seems to me that if your distribution model is remotely accurate then perhaps PGI ought to think about a two tier match maker rather than our current five tier system. With the above average players in one and the below average players in the other, with the two crossing into each other only after several minutes of wait; or maybe even a two tier system with your top 5% in their own and the rest in the other?

Actually, because the middle tiers seem kinda redundant (I'd have to also check WLR, kill rates, and such though), I'd think 4 tiers would be enough. I'd divide it roughly like this:

Tier 4 - bottom 20%

Tier 3 - mid-low 30%

Tier 2 - mid-high 30%

Tier 1 - top 20%

And then make it so that matchmaker can only find players +/- 1 tier.

[Snazzy Dragon]

Tarogato, on 17 January 2017 - 03:30 PM, said:

Actually, because the middle tiers seem kinda redundant (I'd have to also check WLR, kill rates, and such though), I'd think 4 tiers would be enough. I'd divide it roughly like this:

Tier 4 - bottom 20%

Tier 3 - mid-low 30%

Tier 2 - mid-high 30%

Tier 1 - top 20%

And then make it so that matchmaker can only find players +/- 1 tier.

As long as it isn't upwards biased.

[Bud Crue]

Tarogato, on 17 January 2017 - 03:30 PM, said:

Actually, because the middle tiers seem kinda redundant (I'd have to also check WLR, kill rates, and such though), I'd think 4 tiers would be enough. I'd divide it roughly like this:

Tier 4 - bottom 20%

Tier 3 - mid-low 30%

Tier 2 - mid-high 30%

Tier 1 - top 20%

And then make it so that matchmaker can only find players +/- 1 tier.

And the $64000 question is would the MM handle the +/-1 aspect or would the valves be constantly open?

[Tina Benoit just posted something on this and I velieve she said right now the the +/-3 tiers (such that T1s are playing with T4s) have been opened in the past after 3 minutes ( could be getting this way wrong I'll try to find the thread). Have to come up with a similar mechanism with less tiers I suppose so as to not make wait times intolerable.]

https://mwomercs.com...ng-q-and-tiers/

Edited by Bud Crue, 17 January 2017 - 03:47 PM.

[VorpalAnvil]

Bud Crue, on 17 January 2017 - 11:26 AM, said:

Tarogato has the data and data doesn't lie, but I think his raw charts and your observation above are more indicative of the state of the game, as opposed the conclusion that we have too few good players. I think it that the majority of players are pretty good to relatively great (55% of the response pool being tier 1 at the time of this post), That majority, may not be, for the most part, vastly superior in their overall performance in game than the rest of the population (see your point above), but it is enough that the better players still consistently dominate the fewer players of lesser skill in terms of match outcome. That snowballs. That is to say: good players stay on top with the "bad" players never getting any better. Thus, bringing us to Tarogato's final point:



Good players need to be willing to teach and bads like me (no matter how insignificant that difference is statistically speaking is) need to be willing to listen and make an effort to learn.

More so than any other game I've ever played, MWO has an overabundance of people and resources willing to help. But in stark contrast, it also has a shockingly large portion of the population who is not just unwilling to improve, they revel in their lack of skill and wear it like a religious icon on their collective sleeve.

[Deathlike]

VorpalAnvil, on 17 January 2017 - 05:13 PM, said:

More so than any other game I've ever played, MWO has an overabundance of people and resources willing to help. But in stark contrast, it also has a shockingly large portion of the population who is not just unwilling to improve, they revel in their lack of skill and wear it like a religious icon on their collective sleeve.

Welcome to MWO. Would you like a potato?

[MacClearly]

Tarogato, on 17 January 2017 - 01:32 PM, said:

The distribution of respondents in my survey is not indicative of the actual in-game distribution. It's only a sample after all, and is subject to the predispositions of higher tier and higher skill players to more frequently read and participate in the forums, reddit, etc. Thus, the actual distribution may be wildly different. Only PGI knows the actual distribution, because PSR tiers are not publicly available (except by opt-in), and they are not referenced in any databases we have access to.

What do we have access to however is a database of 60,000+ players, with their average match scores, what weight classes they play, their kills, deaths, wins, and losses. From that we have this: https://docs.google....#gid=1246543983

Which unveils that only 5% of players are above ~290 average match score, which about where I'd consider a threshold for "actually good at the game". Similarly, 75% of players are ~230 match score or lower, which is what I would personally consider "bad." And the average player is actually only 200 average match score. Anything at or below that is really atrociously bad in my opinion. Like... I question if you're even having fun playing the game if you consistently score that poorly. Yet that's actually 50% of people that play this game...

What I find interesting is that I thought that my average score would have been higher in the lower tiers as I was working my way through. It is not. There is a steady progression from 195 to just barely over 230 the last two seasons. Which I find amusing because I have only just started to feel barely adequate in this game.

I also find it sort of weirdly consistent using these quick play stats when my focus is my unit and faction. I use quick play like it is practise and am prone to stupidity there. Considering how heavily weighted psr is to winning I think my wlr of .82 is particularly bad especially up against where my other stats put me.

[justcallme A S H]

Completed.

[justcallme A S H]

Just looking at results, seems the match score avg is too high, given that the average score to move up is a bit lower (than the average).

I still reckon a match "WIN" should count for 50% less match score than it does currently. Thus relying far more on personal performance. Or raise the match score requirements for increases.

[VorpalAnvil]

Deathlike, on 17 January 2017 - 06:00 PM, said:

Welcome to MWO. Would you like a potato?

[jss78]

Survey completed.

A couple of thoughts (boring rumination ahead):

First, I feel like it's a false expectation that Tier 1 should be filled with people with super high match score averages. Even in a perfect, evenly matched high-tier games, despite how great you are, you cannot ALL be doing these "acceptable for top player" 400-500 match scores. You'll run out of things to shoot! Or put other way, for each person owning the opposition, someone is necessarily getting wrecked, even if everyone is good.

In a perfect tier system, you'd expect match score averages to be roughly similar in all tiers. Now we see this steady trend towards higher average match score in higher tiers. SOME of this I'd expect as a legacy of the players' past upward movement.

What I'm driving at is that IMO T1 littered with people with ~average match score is not an indication of a faulty system. What I WOULD be concerned with is people with significantly LOWER than average match scores in high tiers.

As such I'd pay special attention to the left tails of those match score normal distributions: not the guys who are doing average, but those really doing badly. There's maybe something going on, as in Tier 1 you have that small population with <225 average. (Though there's a missing piece of information in the data, and that's the current direction of PSR change. Maybe those 175 match score guys in Tier 1 used to be good, but started to play while drunk, or bought Vindicators.)

Second, one thing I'd bear in mind that "most people moving up over time" isn't really either an indication of a faulty tier system. The thing is, if we old-timers here on forums talk about our PSR histories, there's a kind of a survivorship bias going on. We're the guys who've been playing for a long time.

On the other hand, you have people continuously cycling in and out of the player base. If you play for 1, 2, 3 years, you're gaining an increasing experience advantage over the average player. Since most people get better at doing things with experience, most people SHOULD go up in time. You'd expect the guy who doesn't move up with experience to be a rare exception. In effect, he's the guy who cannot beat a "noob" despite having years of experience.

Anyway, I'm not saying there isn't an upward bias, I just feel it's maybe not as bad as people think. Interested to see what Russ presents to us.

[Cato Phoenix]

Cathy, on 17 January 2017 - 02:12 PM, said:

What has, amused, bemused, made me lift an eyebrow, is the disparity between people that claim to be competitive and the amount of solo dropping.

Surely the genuinely competitive people are in group play at every opportunity and not farming pugs, which the pie charts seem to indicate

Solo queue is not 'pug farming', since you are usually at the mercy of the matchmaker gods.

Bud Crue, on 17 January 2017 - 03:21 PM, said:

50% of the player population consistently does "atrociously bad" and most of the rest not doing a whole lot better. Hmm...almost makes one think that perhaps there are people who are simply not physically capable of scoring high in this game and/or whose sense of fun is not totally dependent on winning or doing particularly well...or maybe they just suck at this (raises hand).

I also think this ignores that average match score is probably much more variable. I would wager most players who are getting 200 have 100 match scores and 300 match scores, for instance, depending on their win/loss. So it's not really a 'physically incapable of scoring high', it's a 'only rarely capable of scoring high while losing'.

Edited by Cato Phoenix, 17 January 2017 - 08:23 PM.

[Tarogato]

jss78, on 17 January 2017 - 07:04 PM, said:

First, I feel like it's a false expectation that Tier 1 should be filled with people with super high match score averages. Even in a perfect, evenly matched high-tier games, despite how great you are, you cannot ALL be doing these "acceptable for top player" 400-500 match scores. You'll run out of things to shoot! Or put other way, for each person owning the opposition, someone is necessarily getting wrecked, even if everyone is good.

Actually this wouldn't be so. Remember, matchmaking for Tier 1 players isn't just other Tier 1 players. It's a pool of all Tier 1, 2, and 3. And recently also Tier 4 in the quiet hours. So Tier 1 players have plenty of opportunity to farm high scores off of lower tier players, driving the averages in divergent directions. Also keep in mind Tier 5 players can be dragged into group queue Tier 1 level, and vice-versa.

Quote

Maybe those 175 match score guys in Tier 1 used to be good, but started to play while drunk, or bought Vindicators.)

That trend would haff^_<sic> to have persisted through June to December though, as that's the window of data I'm pulling these stats out of. That's a significant enough timeframe to accumulate a reliable running average of performance. I should at some point figure out a way to visualise the performance over time. I have the per-season matchscores of each player available and can look at them, I just need to think of a proper way to analyse it.

Quote

Second, one thing I'd bear in mind that "most people moving up over time" isn't really either an indication of a faulty tier system. The thing is, if we old-timers here on forums talk about our PSR histories, there's a kind of a survivorship bias going on. We're the guys who've been playing for a long time.

Except... if you are moving up in rank over time without improving in skill relative to the rest of the player base, then it definite *is* an indication of a faulty system. Right now, a Tier 3 player can be just good enough to keep climbing and climbing until they max out tier 1, and they don't have to get any better as a player to reach that point, nor do they have to owe their acclivity to the dying out of older higher tier players making way for the new. The system is simply designed to assume that the more they play, the better they will get. Trevelyas explains this in his post here, which also links to one of his older posts as well if you're interested: https://www.reddit.c...and_how_to_fix/

[jss78]

Tarogato, on 17 January 2017 - 09:22 PM, said:

Actually this wouldn't be so. Remember, matchmaking for Tier 1 players isn't just other Tier 1 players. It's a pool of all Tier 1, 2, and 3. And recently also Tier 4 in the quiet hours. So Tier 1 players have plenty of opportunity to farm high scores off of lower tier players, driving the averages in divergent directions. Also keep in mind Tier 5 players can be dragged into group queue Tier 1 level, and vice-versa.

This is true, thought about putting this in but didn't want to write a full-length essay ... I agree a kind of a feedback going on. The further you are towards the end of the tier system, the more likely you're to face people ranked towards the middle. Of course when you're towards the top end of T1, really nothing can be done, it's either private matches or accepting that most are worse than you.

At the other end, it might explain why we have (in my experience) a surprising number of highly experienced players "permalocked" in T4/T5. They're just systematically going against better opposition, and the games are tough.

In a perfect MM where we don't mix tiers this effect should be eliminated, but oh well.

Tarogato, on 17 January 2017 - 09:22 PM, said:

Except... if you are moving up in rank over time without improving in skill relative to the rest of the player base, then it definite *is* an indication of a faulty system. Right now, a Tier 3 player can be just good enough to keep climbing and climbing until they max out tier 1, and they don't have to get any better as a player to reach that point, nor do they have to owe their acclivity to the dying out of older higher tier players making way for the new. The system is simply designed to assume that the more they play, the better they will get. Trevelyas explains this in his post here, which also links to one of his older posts as well if you're interested: https://www.reddit.c...and_how_to_fix/

It's true the system shouldn't allow a person to rise up without getting better. All I was saying that I'd expect MOST people to get better compared to average player, as they gain experience. You'd need to have some unusual handicap (playing while drunk, physical handicap, run stock 'mechs) to not be able to achieve this.

Anyway, I feel like it's a wonderful thing you're doing, and I love looking at the data on the Google sheets. It's tricky and hard work, in part because we're missing the best diagnostic, and that's the currect rate and direction of change in each player's PSR. That'd really be the bottom line, which way he's going right this moment with his current performance, but we don't have those numbers.

If I had all the data PGI has, I'd love to look at stuff like:
- Average match score rank among team (1-12) vs. direction of PSR change. Disregarding absolute match score values, if you persistently rank higher than 6.5 in your team, the system should ideally be moving you up towards better opponents.
- Same with W/L vs. direction of PSR change, at least in random solo queue teams this is kind of the bottom line. If you're persistently able to carry, you'd ideally be given better opposition.

Edited by jss78, 18 January 2017 - 01:44 AM.