Jump to content

Stats Study: Matchmaker Is Unfair

Balance

344 replies to this topic

#281 vandalhooch

    Member

  • PipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 891 posts

Posted 24 April 2017 - 09:56 PM

View PostDimento Graven, on 23 April 2017 - 09:52 PM, said:

Define that please.


Oooohhhh poor baby. Did you get your feel feels hurt?

Quote

So you're stating for the record that MM is doing it intentionally? And what was the number from Taragato's data then? Was that before or after he added more matches?


You are terrible at this. Why do you insist on showing everyone how very bad you are at this?

Taragato's data doesn't include the rate of unbalanced matches because he didn't record the results of every match.

Quote

If the intent is to created balanced teams, and one team ends up with more players (we'll start at 3 or more) with higher W/L ratios, avg. match score, avg. damage per match, than the other team, I'd define that as a fail.


How does one player have a higher WLR than an entire other team? Do you mean a higher WLR than the player on the other team with the highest WLR? What if that player is the only one that has a WLR higher than the other team but all his teammates have WLR lower than the lowest on the opposing force? What if you have two more players showing the same pattern for match score and damage? Is the match unbalanced? If so, which team has the skill advantage? How did you determine that?

What if player A (blue team) has the highest WLR of both teams and Player B (blue team) has the highest match score average of both teams and Player C (blue team) has the highest damage average of both teams. But, Player D (red team) has the second highest WLR and it is higher than B and C while Player E (red team) has the second highest match score and it is higher than A and C while Player F (red team) has the second highest average damage and it's higher than A and B.

Is the match unbalanced? Why or why not?

What if the highest WLR is only .01 higher than the second highest and the opposing force has three such players? Does the higher WLR count as one of your three?

Quote

Since we have the data from the stomps with these numbers, conclusion: For the matches resulting in stomps MM failed.

It's probably failing elsewhere but because the matches weren't 'stomps'...

How much is enough? Define 'systematic manner'.


20% of the weaker teams STOMPED the stronger team. Does that sound like you accurately determined who the weaker team was?

EDIT: With the cherry picked data removed, the weaker team ended up STOMPING the stronger team 24% of the time.

Quote

You're claiming the conclusions are wrong because you don't like the number of samples or how it was gathered.


No. Read very carefully. You can't draw any conclusions, right or wrong, with this data.

The conclusions you are drawing could very well turn out to be right, but you don't have any evidence to support that claim right now.

Quote

So get the numbers in the manner most acceptable to you and prove to us the MM is always doing the job of creating balanced teams.


Listen you deranged fruitbat, I never claimed that the matchmaker is doing a good job of creating balanced teams. I don't have the data to make such a claim! Neither do you have the data to claim that it is doing a poor job.

WE DON'T HAVE THE DATA TO DRAW ANY CONCLUSIONS ABOUT HOW THE MATCHMAKER IS DOING IN TERMS OF BALANCED TEAMS!

Quote

So because no one can agree, you're arguing to do nothing?


Most people with even the most meager of math skills has long since understood what the problem with these analyses are. Now, I'm mostly here to continue to show how much of a clueless blowhard you are.

Quote

No, this is an argument for argument's sake, pure and simple.

Nope, objectivity would be bad, I have to be more... well call it 'empathic' than anything. Being able to see the problem from the customer's point of view goes a long way to providing a solution the customer is 'happy' to accept.

And it's not scientific, it's 'practical and human' something science sometimes forgets. If I force the customer to bring me a thousand samples of whatever it is that's the problem, because... SCIENCE... well, FAIL.

No it's not science, but it pays the bills.


I'll just skip over the embedded anti-intellectualism and get to the point. Nowhere did I claim that science must be used at all times and in all situations. That's just you realizing you've lost the discussion and lashing out with straw men and red herrings.

Determining if the matchmaker is failing to make balanced teams for matches according to what we feel indicates a player's skill is a question that CAN be answered through statistical (scientific) analysis. However, that analysis requires data be collected in an unbiased manner and that as many extraneous factors as possible be accounted for.

Quote

And you're forgetting the fact that it doesn't matter if it's smoke or dust when the end result is: "KABOOM!!!"


Except if it's dust, there is no kaboom. There's just you running around with your tin foil hat screaming about the impending kaboom that isn't actually going to happen. Really, it's quite a sight to behold.

Quote

You don't have to perfectly define a problem to know that there's a problem.

Just because a problem isn't perfectly defined doesn't mean it's imaginary.


And humans are very good at tricking themselves into seeing problems that aren't actually there. How many of your irate customers turn out to be completely mistaken in their beliefs about your products?

Science is the best tool we have for eliminating the biases from our humanness in order to identify what is really going on.

Quote

Yeah usually in the real world, what they do is get statements from multiple "humans" about a specific event and when there's a lot of similar statements, they start their work in the direction of the common consistencies.

It's not perfect, but it's what we got.


Did it get better or worse when forensic science tools became available? Any comment on the 349 death row exonerations completed due to modern DNA science?

Quote

If I got a room full of people telling me they're smelling smoke, I don't ask them for a chromatagraph of the air, or to define "smell"...


Irrelevant analogy remains irrelevant.

Quote

Again, provide your definition. I've attempted and failed, I want to see how your definition is better than pretty much everyone else's including PGI's.


You want to claim that someone has measured the imbalance in matches but refuse to explain how you measure imbalance?

You are a maroon.

Quote

I see, I can't say that the MM is broken until I have already programmed its replacement.

In other words, we can't say man is causing the Earth to heat up until we find a way for man to cool the planet down.

If not Exxon, you work for BP then?


Just stop. You just look, dumber and dumber every time you go to this well.

Quote

Agreed.

This is good to know, although I should reply with a snarky, "Define rapidly"...

Sorry, I didn't know I wasn't allowed to say that the current system wasn't functioning as intended without having a fully designed ready to implement replacement ready.

Is this a new forum rule that I missed or something?


Hyperbole is boring.

Quote

Didn't Taragato include 12-2 matches as well? Thought he came back later and added them...


This is how bad you are at this. You don't even know what's actually going on.

Taragato originally included SOME 12-2 matches in his data set but after I explained the bias that introduces he went back and removed them.

Not only do you not understand statistics but your understanding of what Taragato did is completely backward to what actually happened!

Quote

Most people seem to respond that a stomp occurred when the match ended quickly (under 6 minutes) and the result was less than a lance of the enemy's forces were destroyed.


That was not Taragato's definition of a stomp. If that's your definition, then you can't reference Taragato's data supporting any of your claims because he didn't collect data on what you call stomps.

Broken to get it posted.

Quote

A lance or more dies and the end of game replies are typically, "Good brawl", which I won't translate as 'stomp' because it implies there was more of an actual two way fight taking place.

You wouldn't, you don't think MM is broken at all, so any data that might show examples of a broken MM are subjected to unreasonable criticism.


Listen you syphilitic camel, I NEVER CLAIMED THAT THE MATCHMAKER IS NOT BROKEN! You just assumed I said that because I seem to be in opposition to your position. That would be the human cognitive bias I keep warning you about. Your perception of what I've said doesn't match at all what I've actually said.

Go ahead. Check to see if I ever made that claim.

Quote

You stated reasons for including a wider range of PSR rankings and time limitations on MM as the lack of player population. So, no it's not a separate issue, it's probably the original root cause of all this...


A small player pool will not allow your idea of a "fixed" matchmaker to create balanced matches in a timely matter either.

Quote

The PSR metric is a value separated from all the afore mentioned values.


It most certainly is NOT separated (the word you are looking for is independent). How much the PSR moves depends on the final match score. Match score depends on damage done and whether the pilot won or lost the match.

PSR is a derived metric.

Quote

It increases based on each winning match result DISPRAPORTIONATELY to the amount it is decreased for a losing match result.

A system where there was no "win bias" a player winning 10 matches with receiving the highest possible PSR bump, then losing 10 matches receiving the highest possible PSR penalty should have a net gain of zero.

In the current system, as documented by PGI, that won't happen.

Because it's possible to lose more matches than win, and still go up in rank, PSR won't work for creating balanced teams in a match.


The second does not necessarily follow from the first. You are assuming it's true but you have still failed to provide any actual evidence that it is true.

Quote

It is happening, we play with these people every day.

You don't like my definition, provide one of your own. If it's good, and less than 3 pages long and doesn't include several large matrices, I might even accept it.


You are the one claiming that matches are unbalanced and yet you can't give an objective definition of what that means? That's your problem to solve, not mine. I'm not making any claims as to balanced or unbalanced.

BTW: That anti-intellectualism you suffer from seems to be flaring up again. I'd suggest you see a doctor for some ointment but you would only reject it "'cause of all that fancy schoolin' the doctor got up to."

Quote

In that particular sentence, where did I speak for Taragato? I simply commended him on his patience and amenability.

No where in that statement did I put any words in his mouth. I'm not sure where you got that...


What makes this statement all the more delicious for me is that my interaction with Taragato is just up thread for everyone to see.

Quote

Hmmm... If you're not a professional climat denier, are you a recall specialist for the automotive industry?

No, you've just the numbers were gathered incorrectly, aren't complete, and therefore any conclusions based on them are wrong.


"All right chorus, once more with feeling!"

I NEVER SAID YOUR CONCLUSION WAS WRONG. I SAID YOU HAD NO EVIDENCE TO CLAIM THAT IT IS RIGHT.

The fact that you can't tell the difference between being demonstrably wrong and lacking evidence of being right is your own ignorant cross to bear.

Quote

Though the majority of matches that ended in a stomp, and had one sided with more players with higher stats than the other can't be ignored. EACH ONE of those is an example of MM not creating balanced teams. Since there were A LOT of those examples,


71 matches is a lot? Let me check my dictionary . . . nope. Can't seem to find a single dictionary that defines a lot as 71 things. Perhaps you can link me to the dictionary you used? Oh, wait. Not all 71 of those matches were actually unbalanced according to his criteria . . . Hmmmmm "a lot" is starting to sound like it's not a very big number at all. I wonder if there is a more precise means for determining a minimum number of samples necessary in order to be able to express our confidence in our conclusions in the form of a probability of being in error . . . . If only there were some way . . .

Naaaaahhhhh. That sounds to much like some sort of fancy book learnin' and you don' want no part of that hooey!

Quote

even in such a small data set, it can be reasoned that there are many, MANY more matches out there where this is happening, AND, since we can see that in only 20% of the cases was the team with the lower overall stats able to win,


Ahem. Those "overall lower stats teams" STOMPED the higher stats teams. They didn't just win. They STOMPED!

Quote

there appears to be an 80% likelihood of the team with higher stats winning.


Nope. Nope. Nope.

Taragato did not record how often higher stats team won. He did not record how often lower stats team won. Taragato did not record the results of just wins and losses, he only collected data from STOMPS. Since the data is for STOMPS only, you can't draw any conclusions about any of the other matches he played in.

Quote

You're saying you didn't make accusations of cherry picking and the like?


Accusation? His original data was cherry picked. Once it was explained to him, he realized that it was cherry picked and to his credit admitted it and removed the cherry picked data from his analysis.

Quote

Are you SURE you're not pissed off then?


Hardly. Mostly I'm sitting here, laughing my fanny off.

Quote

Again, I admire Taragato's ability to be so amenable in the face of such purposeful obstinance.

He's a really nice guy, isn't he?

Back to insisting there be a ready solution before we can point out that there's a problem...

So I wasn't up to date on PGI's new starting point for new players. Sue me.


So you feel it's okay to go around making declarative statements about the states of things for which you don't even know the basics of?

No. Not pompous at all.

Quote

I haven't started a new character since closed beta... Being admittedly ignorant on one minor point doesn't mean I'm ignorant, or incorrect about everything.


No. It just means you are a pompous blowhard.

Quote

The fact that you have to harp on this one minor thing so hard is illuminating as to the confidence you have in your position.


Harp on? I made one reference to it. You've said far more about it in your response than me.

Quote

I'm not going to fully define a solution for you in order to be allowed to point out there's a problem with the current system.


And I'm not going to let your anti-intellectualism deter me from pointing out that you have absolutely no evidence to back up your claims.

Quote

In the sample set most of the stomps were in matches with unbalanced teams, correct?

As I recall from the data (and I admit it's been a while since I looked) once the percentage difference in avg. match score gets beyond 10% it's a near certainty the team with the average higher match score will win.


Not sure it still shows that with the cherry picked data removed and with nearly 40% of the data gone your statistical confidence in such a statement just fell through the floor.

BTW: With the cherry picked data removed, the weaker team ends up STOMPING the stronger team 24% of the time. Is a system that fails to pick the stronger team 24% of the time really that good at identifying skill?

Quote

That 10% disparity should be avoided by MM.

Once or twice a week now, in the stomps I've participated in, I've always seen one or two do outstanding damage and kills on winning team.

We disagree.

Hence the call to advertise to get more players in the game, and the statement WAY up there (or in the other post, can't remember now) that low population is probably the at the root of all this.


A "fixed" matchmaker will still have to assemble matches out of the small player pool we have now.

Quote

Because even with a million new players, the win biased PSR system will do the same thing to them. Push everyone upwards in rank.

The PSR system should have more drastic penalties for losing.


So, do you want a new matchmaker or do you want to overhaul how PSR is calculated? Those are two separate things.

Edited by vandalhooch, 24 April 2017 - 10:02 PM.


#282 Ghogiel

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • CS 2021 Gold Champ
  • CS 2021 Gold Champ
  • 6,852 posts

Posted 25 April 2017 - 12:47 AM

View PostTarogato, on 24 April 2017 - 04:44 PM, said:

To me, that's like auditioning people for a choir based on how well they play an instrument. Sure, these are two extremely related and intertwined skills, and you may find a correlation, but it's still not fair.

Or matching casual pickup basketball by BMI. Sure, maybe the fit people would beat the overweight people, but what does BMI tell you about how good two people are at basketball if they have the same BMI, even if we know that playing basketball regularly can "improve" your BMI?


Point is, PSR does not equal actual skill level. It equals experience. Better players can fill their experience bars faster, and worse players can fill their experience bars by playing more, but at the end of the day both players have a full experience bar, they achieved the same goal by different means, and their actual skill differs greatly. The MM could put together the perfect match, where all 24 players are max Tier 1... identical PSR. And it can still be outrageously imbalanced. So checking PSR means nothing when it comes to evaluating the effectiveness or "fairness" of the matchmaker.
PSR means pretty much everything according to the MM and to how it is assembling teams.

However if this thread were about PSR, the OP would be interested in PSR and not completely blanking it's existance entirely.

Secondly that example you gave would be a perfectly balanced match according to the MMer, but the OP would never know because he doesn't understand the MMer and thinks it's looking people up on the leaderboard and feeding potatoes winning teams, even though it can't even do that because PSR is borked so wouldn't even know what a winning team is so as to feed a player with it.

Lastly we still have to know PSR whether it is working or not to know if the MMer is assembling equal teams anyway. Without knowing, it's just pretending to know if the MM is actually working or not. Or worse, end up with some crackpot ideas about the MM ******* with players W/L instead of assembling teams .

#283 Shifty McSwift

    Member

  • PipPipPipPipPipPipPipPipPip
  • 2,889 posts

Posted 25 April 2017 - 12:59 AM

View PostSteel Claws, on 24 April 2017 - 05:39 PM, said:

What in life is fair.


Well a coinflip for one is about as fair as it gets.

Then you have scenarios where people have an equal playing field that can only be overcome by skill/experience or luck, like a game of chess, so I guess lots of things are fair in the relative sense.

View PostSteel Claws, on 24 April 2017 - 05:39 PM, said:

Should we hand out participation trophies.


You mean like being payed in game currency and XP for participating in a game no matter what the result is? You mean getting rewards when even on the losing team?

That would be so weird right? What an upside down topsy-turvy world that would be, plus all the horses would eat eachother, what a nightmare /s

Edited by Shifty McSwift, 25 April 2017 - 01:00 AM.


#284 Mazzyplz

    Member

  • PipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • 3,292 posts

Posted 25 April 2017 - 01:30 AM

i proposed this was the case and a fix for it like over 2 years ago.

i suspected the matchmaker was making a winning and a losing team, then checking to see if the predictions were correct. if you won a losing game your psr would go up a lot.

the solution for this is quite simple. after all 24 players are in the match, rebalance both teams. so there's an equal ammount of braindead players on both sides. or at least try to do that. taking into account the kdr, wlr and tonnage.
yes this is needed.

this would improve the experience A LOT.

but i believe pgi has read my suggestion, and decided it was better to spend their time including superfluous stuff to the game, and modes that nobody want. including the whole of CW.

in other words; they couldn't care less if they tried.

edit: oh and this wouldn't increase waiting times at all. that's the thing. the rebalance of teams would be done in match. so no excuses for them to hand us a loss before the match begins because they don't want to make us wait so long.

Edited by Mazzyplz, 25 April 2017 - 01:40 AM.


#285 Savage Wolf

    Member

  • PipPipPipPipPipPipPipPip
  • The Wolf
  • The Wolf
  • 1,323 posts
  • LocationÅrhus, Denmark

Posted 25 April 2017 - 04:02 AM

View PostBLOOD WOLF, on 24 April 2017 - 10:43 AM, said:

that's division. your thinking of 5/5. I said 5:5, which is a ratio.

And all ratios are fractions. You can either just use the fraction to express the ratio or you can solve it into a decimal number. Usually ratios like 4:3 or 16:9 solve as some very non-sexy numbers for marketing so they just keep it as a fraction. However, 5:5 solves to a rather simple number. The number 1.
So essentially, you agree with me.

#286 BLOOD WOLF

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The Jaws
  • The Jaws
  • 6,368 posts
  • Locationnowhere

Posted 25 April 2017 - 05:46 AM

View PostBLOOD WOLF, on 24 April 2017 - 05:35 AM, said:

so It doesn't follow from matching 12 people against an equally rated 12 people means if they played a series of 10 matches that the score would be 5:5

The W/L would be whatever it happens to be.


View PostSavage Wolf, on 24 April 2017 - 05:41 AM, said:

What is 5 divided by 5?


Sorry, I meant that the premise doesn't follow that 12v12 who play 10 matches would be equal. That is what I meant.

even if teams had equal skill, the score could be 4:6 or 9:1.

View PostSavage Wolf, on 25 April 2017 - 04:02 AM, said:


So essentially, you agree with me.

No, because that was an example and not necessarily the case with what is actually happening.

Edited by BLOOD WOLF, 25 April 2017 - 06:19 AM.


#287 Rhaezor

    Member

  • PipPipPipPipPip
  • Knight Errant
  • 122 posts
  • LocationPT

Posted 25 April 2017 - 06:46 AM

Thank you for taking the time to look into this.

I too notice that my first games are always teamed up with the worst possible players, and only after 10 games or more I start getting decent teams. The matchmaking might have worked but now (with less people playing) its just scrapping everyone they can.

#288 Alan Hicks

    Member

  • PipPipPipPipPipPip
  • The 1 Percent
  • 414 posts

Posted 30 April 2017 - 03:31 PM

The current MM system still does not work. It is an obvious deduction given the terrible experience this game sometimes delivers. It is not you, it is not your chosen battlemech, it is the disastrous team they set you in by a broken matchmaker. For example, at times you see some assaults mechs players delivering under 200 or not even around 100 damage.

Uneven teams and the experience this brings may be one of the main reasons people flee from MWO. if this does not get fixed to offer well balanced matches, there is a high probability the gaming population will shrink even more than it is now.

Edited by Alan Hicks, 30 April 2017 - 03:43 PM.


#289 Xetelian

    Member

  • PipPipPipPipPipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 4,371 posts

Posted 01 May 2017 - 04:43 AM

Between Taragoto(SP?) and this post I think the match making needs tightening around what it uses to allocate teams.

#290 Exilyth

    Member

  • PipPipPipPipPipPipPipPipPip
  • Bridesmaid
  • 2,100 posts
  • LocationTerra

Posted 02 May 2017 - 02:42 PM

Matchmaking has been getting worse lately.

#291 Scyther

    Member

  • PipPipPipPipPipPipPipPip
  • 1,271 posts
  • LocationOntario, Canada

Posted 02 May 2017 - 04:11 PM

Just a small thought to toss in here (can't read all 15 pages, sorry if duplicating):

I play 5 different games that use a matchmaker to balance/create teams. Some are very large (WoT), some are very small (Prime World MOBA).

Every single one of those games has a long list of player complaints about bad MM, roflstomps, MM 'enforcing' a specific win/loss pattern or ratio, MM creating unfair matches by design, etc etc.

Just saying that if none of them seem to be getting it 'right', perhaps it is an inherently intractable problem from some angles. Not saying it couldn't be better but it does not seem prone to 'simple' fixes.

#292 Exilyth

    Member

  • PipPipPipPipPipPipPipPipPip
  • Bridesmaid
  • 2,100 posts
  • LocationTerra

Posted 03 May 2017 - 12:34 PM

"For a set of players, find two disjoint subsets of players with equal skill" is a variation of the generalised assignment problem: https://en.wikipedia...ignment_problem - except that each team is an agent, each player is a job/task and the skill of a player is the cost of the task (and the constraint is for both teams to have equal cost/skill and each team should have exactly 12 people). As the wiki article states, while the problem itself is hard, approximations of the solution are (relatively) easy to find.

#293 Scyther

    Member

  • PipPipPipPipPipPipPipPip
  • 1,271 posts
  • LocationOntario, Canada

Posted 03 May 2017 - 01:13 PM

That's cool that you have a link to a theoretical model of a mathematical representation of a generic algorithm to solve a somewhat related problem.

Please explain why so many companies find it impossible to find a 'player satisfying' solution to this (relatively) easy problem. Or even an approximation of one.

Please note where the formulae linked assess 'degree of user satisfaction'. Or where they deal with the 'uneven distribution on a randomly changing schedule of revolving-door availability of agents/tasks'.

Or possibly, consider that real-world examples trump mathematical models when you are evaluating real-world results.

Edited by MadBadger, 03 May 2017 - 01:14 PM.


#294 MechWarrior5152251

    Member

  • PipPipPipPipPipPipPipPip
  • Bad Company
  • Bad Company
  • 1,461 posts

Posted 03 May 2017 - 03:25 PM

Who is the idiot who wrote the code for the Matchmaker?

#295 Mawai

    Member

  • PipPipPipPipPipPipPipPipPip
  • Legendary Founder
  • Legendary Founder
  • 3,495 posts

Posted 04 May 2017 - 05:21 AM

Ever since the beginning of MWO ... very few folks seem to have understood what the matchmaker is doing and tend to just blame it for bad matches.

In quickplay, the matchmaker ONLY does two things.

1) Makes sure that there are an equal number of mechs in each weight class on each team.
2) Makes sure that the pilot ratings on each side are close to balanced.

... and it that is what it does. Based on the limited metrics that have been mentioned by PGI it actually does a reasonable job.

BUT >>>

Unfortunately, those conditions are necessary but not sufficient to provide consistent matches with close outcomes (if that is even possible). Close outcomes (i.e. not stomps) are not solely the result of good players on one team and bad players on the other. (As someone mentioned, even the statistics in this thread showed something like 24% of the "less skilled" teams based on W/L,KDR,etc still stomped the team with greater "skill".

Uneven results can be caused by a mismatch in mechs (3 x KDK-3 vs 3xVictor), they can be caused by a mismatch in skill trees (one side with more skilled up mechs than the other), they can be most frequently caused by tactical errors (one lance on one team encounters two lances on the other and is wiped out giving the other team a four mech advantage) ... not sticking together and using focused fire is probably THE BIGGEST reason for stomps. However, sticking together, moving as a coherent group, firing at the same targets as your team mates .. are player SKILLS .. that are unfortunately difficult to measure.

Finally, we come to the topic of PSR. This is supposedly a player skill metric ... it is based on damage done, actions performed in a match, KMDD, Kills, Assists ... most of the contributions to match score ... and it is used to determine how the player skill rating changes based on match outcome. Elo was the previous rating system which depended on the statistical nature of match outcomes and the fact that the common element in any match YOU play in is YOU. After enough matches, it is actually possible to determine your individual contribution to match outcome. A reasonable rough estimate doesn't even take that many matches. However, even with Elo, when you start a new mech, if you've had a bit to drink, if you are just playing for giggles, if you are playing a new weight class ... your "skill" varies ALL the time. The myth is believing that there is ONE NUMBER that will absolutely reflect the player skill at the moment the match is formed.

What all this means is that EVEN with a "perfect" system ... you will still see stomps. They are NOT by themselves a symptom of bad matchmaking. They will always occur even with perfect matchmaking as long as 12 randoms are dropped against 12 randoms.

As for this stupid remark ...

"Who is the idiot who wrote the code for the Matchmaker?"

If you want details of how the matchmaking system was built it is in the following thread (it is old and refers to Elo but the matchmaker is the same):

http://mwomercs.com/...courtesy-phone/

The developer who wrote the matchmaker was one of the most involved PGI devs I have EVER run across. He actually interacted with the community and provided answers. That thread is one of the best ever posted on these forums .. unfortunately for PGI and likely good for Karl he left PGI to take a position with Amazon (though I have no idea where he might be now).

The bottom line is that ... Yes, the matchmaker could be improved
- add tonnage matching or even better some sort of battle value estimate that factors in mech, loadout and skill tree
- come up with a better player skill rating system .. preferably with more granularity for mech class/specific mech and solo vs group play (though since there is no matchmaker in group play these days it doesn't really matter there)

However, these take effort and investment and have little or no direct revenue impact (even if improved matches might increase player retention) so they would seem to be very low on the priority list for PGI if they are even on the list at all.


Some key myths to keep in mind:
- the matchmaker doesn't care about YOU ... whether you won your last 10 games or lost them. The extremely small change in your PSR that results from winning or losing 10 games in a row is negligible in terms of matchmaking ... you are seeing the same PSR range of players either way.
- Folks seem to think that if they are losing then the matchmaker is giving them bad teams. IF a player rating system is working and if the matchmaker is working then you SHOULD win and lose equal numbers of matches. This is reinforced by stomps ... but stomps are NOT solely the result of an imbalance in player skill. (Stomps can and do occur even if the team skills are balanced). The bottom line is that if you win and lose about equally then you are actually being matched up about right ... the matchmaker does NOT know or care what the match outcomes are and it does NOT create matches that intentionally result in stomps. It isn't built that way.

Edited by Mawai, 04 May 2017 - 05:22 AM.


#296 Mawai

    Member

  • PipPipPipPipPipPipPipPipPip
  • Legendary Founder
  • Legendary Founder
  • 3,495 posts

Posted 04 May 2017 - 06:10 AM

View PostExilyth, on 03 May 2017 - 12:34 PM, said:

"For a set of players, find two disjoint subsets of players with equal skill" is a variation of the generalised assignment problem: https://en.wikipedia...ignment_problem - except that each team is an agent, each player is a job/task and the skill of a player is the cost of the task (and the constraint is for both teams to have equal cost/skill and each team should have exactly 12 people). As the wiki article states, while the problem itself is hard, approximations of the solution are (relatively) easy to find.


Interesting :) ... but I see two problems with the comparison to this mathematical problem.

1) The task cost is not well known. (Where the task cost is the individual player skill ... to get an accurate measure requires an accurate player rating system .. which PSR is not).

2) The algorithm is designed to combine tasks for each agent to maximize profit. However, game matchmaking has an additional constraint in that it also wants to minimize the standard deviation of task costs (skill) for each of the two agents (teams). Why? So we don't get the "carry harder" problem. The problem as outlined in the link does not exclude solutions with a large task cost combined with many low task costs compared to one with all average task costs. Although this might give two agents with comparable "profit" (balance) ... it does not produce an actually balanced match from a gaming perspective.

#297 mad kat

    Member

  • PipPipPipPipPipPipPipPip
  • Shredder
  • Shredder
  • 1,907 posts
  • LocationFracking the third toaster.

Posted 25 September 2018 - 03:22 AM

View Postdrunkblackstar, on 19 April 2017 - 09:06 AM, said:


**SNIP**



This is truly a great find but quite a concerning one as that means this actually compounds the imbalance not reduces it. It definitely backs up my experience of MWO ever since the steaming pile of crap that is PSR was introduced.

I for one will say the old ELO was a much much MUCH better system and usually resulted in very close tense games which are now a thing of the past bar the odd rare occasion.

Edited by mad kat, 25 September 2018 - 03:37 AM.


#298 Too Much Love

    Member

  • PipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 787 posts

Posted 16 October 2018 - 09:58 PM

View Postmad kat, on 25 September 2018 - 03:22 AM, said:


This is truly a great find but quite a concerning one as that means this actually compounds the imbalance not reduces it. It definitely backs up my experience of MWO ever since the steaming pile of crap that is PSR was introduced.

I for one will say the old ELO was a much much MUCH better system and usually resulted in very close tense games which are now a thing of the past bar the odd rare occasion.


Thanks for the appreciation.

IMHO, ELO was equally bad as PSR, in fact switching from former to the later didn't change anything.

Which proves that there is some other system behind both ELO and PSR we don't know about, or that PGI can't divide 24 persons into 2 teams with the equal skill.

#299 Tiewolf

    Member

  • PipPipPipPipPipPip
  • Bad Company
  • 408 posts
  • LocationHessen

Posted 16 October 2018 - 11:25 PM

I think we can all agree that in 70% of the time we can predict the outcome of a game by looking at player/unit/cmd signatures. That w/l ratio always pushes towards the mean (1) is not pgi or the coding of the MM but the mathematical phenomenon of regression to the mean. That is a strong indicator that MM is purely random.

Personally I wonder more if MM considers secondary variables like premium time? Would be smart of pgi to give paying costumers advantages without the unpleasant p2w feeling.

#300 Variant1

    Member

  • PipPipPipPipPipPipPipPip
  • 1,148 posts

Posted 17 October 2018 - 08:18 AM

thank you OP weve been tellin others that 12-0 means mm don goofed not both sides were equal thats why it lead to a stomp Posted Image

mm needs to balance by loadouts and stats. Have both teams have meta build and even stats.
Infact id prefer if mm just paired meta cookie cutter builds against each other, that way us filthy casuals with non meta bracked builds could enjoy slugging it out while not being torn to shreds by high alpha strikers. just my 2 centurions

Edited by Variant1, 17 October 2018 - 08:19 AM.






1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users