Jump to content

Stats Study: Matchmaker Is Unfair

Balance

344 replies to this topic

#121 BLOOD WOLF

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The Jaws
  • The Jaws
  • 6,368 posts
  • Locationnowhere

Posted 21 April 2017 - 03:04 PM

Needs more comprehensive data, with comparative studies. Data needs to also be falsifiable. Science ain't easy.

#122 Too Much Love

    Member

  • PipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 787 posts

Posted 22 April 2017 - 01:28 AM

View PostGhogiel, on 21 April 2017 - 07:46 AM, said:

If you wanted to find out if the teams are equal according to the MM, then all the players PSR would be entirely essential and the only info you would need. The stats have almost nothing to do with PSR and MMing. Just because there is a tier spread in the MMing doesn't mean the MM isn't doing a good job of matching the tier spread between teams, I personally doubt it but it could be working as intended and within threshold (which could also be loose af but you have no idea whether the teams are equal or not and never will know unless you know all the players tiers in a given match)


You will probably suppose wrong to think that the MM is taking into account stats other than PSR, to think otherwise would be contray to the dozen of explaination from karl berg threads, town halls and command chairs on the subject. That sort of thing is nearly tinfoil assertions

There are 2 possibilities:

1) The MM does use only tier system.

2) The MM doesn't use tiers system or uses something along with tier system (other variables like K\D and W\L).

In either way the proposed study managed to show something.

If 1) is correct and MM uses only tier system, it means it works awfully wrong and assembles unequal teams.

If 2) is correct and MM doesn't use tiers system or uses something along with it, it means that MM deliberately makes unequal teams.

I'm not 100% sure wich variant is true. I'm not all into, as you said, "tinfoil plot government hides 9\11 truth theory".

But I'm far from being confident in that PGI told us, at least, the whole truth about MM. For example, they were already cought on "not telling the whole story", when players found out thet tier 1 was matched with tier 4, despite the fact they claimed that it can be macthed with tier 3 only. The MM is not that transparent at all. They don't even publish online numbers, why you expect they will reveal you the core mechanics of their business project?

#123 LordNothing

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 17,228 posts

Posted 22 April 2017 - 01:53 AM

View PostAppogee, on 19 April 2017 - 09:27 AM, said:

I know it's weird, but it feels to me like win:loss ratio is still being used in the background to allocate people to teams.

I say that because my win:loss ratio cycles up to 1.1 then down to 1.01. Up to the top of that range, then down to the bottom again.

And it's been doing that for two years. It's as if when I get to 1.1 some kind of switch kicks in and I get a run of awful teams that not even Proton could carry.

I have no proof for this, other than that my win:loss stat seems like it hasn't behaved in a random fashion over the course of thousands of matches.


yea i see my win/loss hovering close to 1 in qp but its actually closer to 0.7 in fp. since fp is more random (more or less). i think its actively fixing games so that everyone gets a w/l close to 1. this is one of the reasons i actually prefer fp, it doesn't lie to me about how good i am.

#124 Too Much Love

    Member

  • PipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 787 posts

Posted 22 April 2017 - 01:54 AM

Posted Image

#125 Escef

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Overlord
  • Overlord
  • 8,530 posts
  • Twitter: Link
  • Twitch: Link
  • LocationNew England

Posted 22 April 2017 - 02:02 AM

View Postdrunkblackstar, on 22 April 2017 - 01:54 AM, said:

Posted Image


Pretty much everyone with a better than 3rd grade knowledge of statistics.

#126 Xetelian

    Member

  • PipPipPipPipPipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 4,397 posts

Posted 22 April 2017 - 02:04 AM

View PostEscef, on 22 April 2017 - 02:02 AM, said:


Pretty much everyone with a better than 3rd grade knowledge of statistics.


What about the other one that used 100? Both show the same results.

#127 Too Much Love

    Member

  • PipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 787 posts

Posted 22 April 2017 - 02:05 AM

View PostXetelian, on 22 April 2017 - 02:04 AM, said:


What about the other one that used 100? Both show the same results.

Posted Image

#128 Escef

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Overlord
  • Overlord
  • 8,530 posts
  • Twitter: Link
  • Twitch: Link
  • LocationNew England

Posted 22 April 2017 - 02:45 AM

View PostXetelian, on 22 April 2017 - 02:04 AM, said:


What about the other one that used 100? Both show the same results.


100 is still a questionably small sample. Especially when you don't have access to all of the relevant data to do analysis upon.

#129 Ghogiel

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • CS 2021 Gold Champ
  • CS 2021 Gold Champ
  • 6,852 posts

Posted 22 April 2017 - 02:51 AM

View Postdrunkblackstar, on 22 April 2017 - 01:28 AM, said:

If 1) is correct and MM uses only tier system, it means it works awfully wrong and assembles unequal teams.



Prove your bald assertion that the teams are unequal already or just resign yourself to the fact you haven't a clue if MM is assembling unequal teams or not.

#130 Too Much Love

    Member

  • PipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 787 posts

Posted 22 April 2017 - 02:59 AM

View PostGhogiel, on 22 April 2017 - 02:51 AM, said:


Prove your bald assertion that the teams are unequal already or just resign yourself to the fact you haven't a clue if MM is assembling unequal teams or not.
Read the original post.

#131 Widowmaker1981

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The Widow Maker
  • The Widow Maker
  • 5,031 posts
  • LocationAt the other end of the pretty lights.

Posted 22 April 2017 - 03:37 AM

View PostGhogiel, on 22 April 2017 - 02:51 AM, said:


Prove your bald assertion that the teams are unequal already or just resign yourself to the fact you haven't a clue if MM is assembling unequal teams or not.

View Postdrunkblackstar, on 22 April 2017 - 02:59 AM, said:

Read the original post.


You have not proved that the matchmaker is making unequal teams *when only tier of pilot, and nothing else* are considered.

1) PGI have explained how the matchmaker works, and said that it looks at the Tier of the pilot only (that may be more granular than just tier levels 1-5, we dont know)
2) PGI have explained in great detail how the PSR system works, and we know that it is upward biased and therefore a poor means of determining actual skill; you can prove this to yourself by looking up a selection of T1s and seeing the huge variation in stats. We also know that since its an upward biased system, this skill disparity in T1 will increase over time as more people arrive in T1.

Due to 1) and 2) we can confidently predict that, especially at T1, MM will be assembling teams of unequal skill - because it simply doesnt know the skill levels of the pilots and as such it will be random.

Since high WL, KD and MS will tend to correlate, all your (OPs) data shows is that the team with higher skilled players tends to win. SHOCKING! It does absolutely nothing to prove that it is deliberately set up that way.

As someone said earlier, the existence of win/lose streaks is expected in a system like this, and not a reason to disbelieve it.

Edited by Widowmaker1981, 22 April 2017 - 03:40 AM.


#132 Too Much Love

    Member

  • PipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 787 posts

Posted 22 April 2017 - 03:46 AM

View PostWidowmaker1981, on 22 April 2017 - 03:37 AM, said:


You have not proved that the matchmaker is making unequal teams *when only tier of pilot, and nothing else* are considered.

1) PGI have explained how the matchmaker works, and said that it looks at the Tier of the pilot only (that may be more granular than just tier levels 1-5, we dont know)
2) PGI have explained in great detail how the PSR system works, and we know that it is upward biased and therefore a poor means of determining actual skill; you can prove this to yourself by looking up a selection of T1s and seeing the huge variation in stats. We also know that since its an upward biased system, this skill disparity in T1 will increase over time as more people arrive in T1.

Due to 1) and 2) we can confidently predict that, especially at T1, MM will be assembling teams of unequal skill - because it simply doesnt know the skill levels of the pilots and as such it will be random.

Since high WL, KD and MS will tend to correlate, all your (OPs) data shows is that the team with higher skilled players tends to win. SHOCKING! It does absolutely nothing to prove that it is deliberately set up that way.

It is what I said, namely variant number 1:

View Postdrunkblackstar, on 22 April 2017 - 01:28 AM, said:

There are 2 possibilities:

1) The MM does use only tier system.

2) The MM doesn't use tiers system or uses something along with tier system (other variables like K\D and W\L).

In either way the proposed study managed to show something.

If 1) is correct and MM uses only tier system, it means it works awfully wrong and assembles unequal teams.

It may be like that, but, once again, it doesn't explain the loopside streak wins or loses.

The position "PGI told us how MM works, this is all we need to know" is legitimate, but I personally don't buy it.

I've been always interested in what MM does as a result, and this small study helped me to uderstand it.

Edited by drunkblackstar, 22 April 2017 - 03:50 AM.


#133 Widowmaker1981

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The Widow Maker
  • The Widow Maker
  • 5,031 posts
  • LocationAt the other end of the pretty lights.

Posted 22 April 2017 - 03:50 AM

View Postdrunkblackstar, on 22 April 2017 - 03:46 AM, said:

It is what I said:

It may like that, but, once again, it doesn't explain the loopside streak wins or loses.

The position "PGI told us how MM works, this is all we need to know" is legitimate, but I personally don't buy it.

I've been always interested what MM does as a result, and this small study helped me to uderstand it.


The results we see match the results we would predict based on what PGI has told us. Occams razor says to accept the simplest explanation that fits the facts..

Assuming that PGI went to extra effort to program a more intelligent matchmaker that could make better games, then forced it to make worse games and lied about it existing.. is very tin foil hat. WHY would they do that?

edit: Its NOT MM thats 'working awfully wrong', its PSR. MM bases its info off PSR, so the effect is MM having no chance to work well.

edit 2: Also, by the way, nothing in your data explains why streaks would happen even if the MM was making deliberately lopsided games

Edited by Widowmaker1981, 22 April 2017 - 03:58 AM.


#134 Too Much Love

    Member

  • PipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 787 posts

Posted 22 April 2017 - 03:57 AM

View PostWidowmaker1981, on 22 April 2017 - 03:50 AM, said:


The results we see match the results we would predict based on what PGI has told us. Occams razor says to accept the simplest explanation that fits the facts..

Assuming that PGI went to extra effort to program a more intelligent matchmaker that could make better games, then forced it to make worse games and lied about it existing.. is very tin foil hat. WHY would they do that?

edit: Its NOT MM thats 'working awfully wrong', its PSR. MM bases its info off PSR, so the effect is MM having no chance to work well.


View Postdrunkblackstar, on 20 April 2017 - 01:27 AM, said:

I agree that at first it seems absurd and the sort of a conspiracy theory.

But on a second approach there could be some logic used in online gaming marketing. This system gives the opportunity to win to everybody. Maybe the guys in marketing think that "bad players" would lose constantly and, as a result, they will leave the game. This system gives them the chance to be paired with the "winning team" and to win. As a drawback it spoils the game for everybody else time after time.


#135 BLOOD WOLF

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The Jaws
  • The Jaws
  • 6,368 posts
  • Locationnowhere

Posted 22 April 2017 - 04:00 AM

View PostWidowmaker1981, on 22 April 2017 - 03:50 AM, said:


The results we see match the results we would predict based on what PGI has told us. Occams razor says to accept the simplest explanation that fits the facts..

Assuming that PGI went to extra effort to program a more intelligent matchmaker that could make better games, then forced it to make worse games and lied about it existing.. is very tin foil hat. WHY would they do that?

edit: Its NOT MM thats 'working awfully wrong', its PSR. MM bases its info off PSR, so the effect is MM having no chance to work well.

edit 2: Also, by the way, nothing in your data explains why streaks would happen even if the MM was making deliberately lopsided games

yea, I see what your saying. Certain variables are not accounted for with drunk's analysis or that meta data that was collected. If the MM was indeed pairing people for the chance to win, how is it I can win or anybody can win 10 in a row?

It's also possible that the MM can have failings, after all its not a perfect system. He should have set up peer review, would have made the data a bit more sturdy

Edited by BLOOD WOLF, 22 April 2017 - 04:03 AM.


#136 Widowmaker1981

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The Widow Maker
  • The Widow Maker
  • 5,031 posts
  • LocationAt the other end of the pretty lights.

Posted 22 April 2017 - 04:03 AM

View Postdrunkblackstar, on 22 April 2017 - 03:57 AM, said:

I agree that at first it seems absurd and the sort of a conspiracy theory.

But on a second approach there could be some logic used in online gaming marketing. This system gives the opportunity to win to everybody. Maybe the guys in marketing think that "bad players" would lose constantly and, as a result, they will leave the game. This system gives them the chance to be paired with the "winning team" and to win. As a drawback it spoils the game for everybody else time after time.


It IS an absurd conspiracy theory.

Said marketing guy would rapidly be told by people with brains that the effect they want (everyone winning sometimes) will happen randomly anyway, since the allocation of people to teams is random, in that it doesnt care about who you are, just your Tier.

#137 Xetelian

    Member

  • PipPipPipPipPipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 4,397 posts

Posted 22 April 2017 - 04:03 AM

View PostEscef, on 22 April 2017 - 02:45 AM, said:


100 is still a questionably small sample. Especially when you don't have access to all of the relevant data to do analysis upon.



A 100 matches is still looking up 2400 people, not exactly the easiest task but they did it. I think the sample size is adequate for the purposes.

#138 BLOOD WOLF

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The Jaws
  • The Jaws
  • 6,368 posts
  • Locationnowhere

Posted 22 April 2017 - 04:05 AM

View PostXetelian, on 22 April 2017 - 04:03 AM, said:



A 100 matches is still looking up 2400 people, not exactly the easiest task but they did it. I think the sample size is adequate for the purposes.

yea, but he has no comparative studies. Also his data has holes. You can't simply say "I like the guy" therefore It's adequate. Not for the purpose of proving the MM is unfair.

#139 Tarogato

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Civil Servant
  • Civil Servant
  • 6,558 posts
  • LocationUSA

Posted 22 April 2017 - 04:08 AM

View Postvandalhooch, on 21 April 2017 - 03:00 PM, said:

You need access to the results from thousands and thousands of matches to show that long streaks are more or less common than predicted by chance. Without that data, any individual streaks you observe, including Taragato's cherry picked nonsense, is simply an observation.


Pardon, but what was cherry picked about my study?

I studied extreme match results (stomp matches, like 12-0) in a consistent environment (solo queue), and included all of the data I could get my hands on over the course of 2-3 months. I didn't cherry pick any of my data, and I even provided a link to the data I collected if anybody else wanted to look into it.

#140 Tarogato

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Civil Servant
  • Civil Servant
  • 6,558 posts
  • LocationUSA

Posted 22 April 2017 - 04:23 AM

View PostBLOOD WOLF, on 22 April 2017 - 04:05 AM, said:

yea, but he has no comparative studies. Also his data has holes. You can't simply say "I like the guy" therefore It's adequate. Not for the purpose of proving the MM is unfair.


Comparing what?

If you're talking about the relationship between the various stats (WLR, AvgMatchScore, KDR, etc), I've done a little work on that here:

https://mwomercs.com...ire-playerbase/


And some stuff related to Tier rating here: https://mwomercs.com...-stats-results/

And more here: https://mwomercs.com...chmaking-survey





5 user(s) are reading this topic

0 members, 5 guests, 0 anonymous users