344 replies to this topic

[Mawai]

Deathlike, on 19 April 2017 - 09:51 AM, said:

I don't think the MM is actually trying to construct the teams the way they do, but it's just the consequence of really bad metrics and telemetry that causes that to happen.

My basic example is pretty much this...

A Tier 1 max bar Lurm user CAN have the same "PSR rating" than some random player from Emp/SJR. Without even trying to figure out metrics and stuff, you know these two people ARE NOT OF THE SAME CALIBER OF SKILL. YET BY THE MAGIC OF THE MM'S ALGORITHM, they are considered ONE AND THE SAME.

That is all you need to really know about how bad the MM is. It doesn't consider stats... they consider the almighty PSR score as one and the same, even though anyone with a brain can tell you they are not.

I am sure that you have heard the famous statement "Garbage In, Garbage Out".

I could design the most perfect matchmaker in the world ... and it would still produce the most awful results if the numerical values used to represent the expected contributions from each player to the match were garbage.

By this point in time, just using your example, I think most folks agree that PSR is that garbage.

All the current matchmaker does is two things:
1) Make sure that each team has a matching number of each weight class. It starts aiming for a 3/3/3/3 distribution but relaxes this depending on the weight class contents of the queue.
2) Tries to have the average PSR for each team as close as possible while minimizing the spread as much as possible given the available pilots and mechs currently in the queue.

It doesn't factor in tonnage, particular mech, loadouts, skills unlocked or the pilot skill in the particular weight class (driving a light is NOT the same as driving an assault ... and being good at one does not guarantee being good at the other for example).

So .. there are several things that PGI can do to make a better matchmaker.
1) Improve the pilot rating system ... perhaps add granularity by weight class
2) Add a factor for skill unlocks on the mech you are driving
3) Add a factor for tonnage so that a 100 ton is not considered the same as an 80 ton.

and
4) Consider adding a factor based on loadout (this would need some work on weapon balance)


However, they could start by just improving the pilot rating system as a first step.

[Deathlike]

Mawai, on 23 April 2017 - 05:05 PM, said:

I am sure that you have heard the famous statement "Garbage In, Garbage Out".

I could design the most perfect matchmaker in the world ... and it would still produce the most awful results if the numerical values used to represent the expected contributions from each player to the match were garbage.

By this point in time, just using your example, I think most folks agree that PSR is that garbage.

All the current matchmaker does is two things:
1) Make sure that each team has a matching number of each weight class. It starts aiming for a 3/3/3/3 distribution but relaxes this depending on the weight class contents of the queue.
2) Tries to have the average PSR for each team as close as possible while minimizing the spread as much as possible given the available pilots and mechs currently in the queue.

It doesn't factor in tonnage, particular mech, loadouts, skills unlocked or the pilot skill in the particular weight class (driving a light is NOT the same as driving an assault ... and being good at one does not guarantee being good at the other for example).

So .. there are several things that PGI can do to make a better matchmaker.
1) Improve the pilot rating system ... perhaps add granularity by weight class
2) Add a factor for skill unlocks on the mech you are driving
3) Add a factor for tonnage so that a 100 ton is not considered the same as an 80 ton.

and
4) Consider adding a factor based on loadout (this would need some work on weapon balance)


However, they could start by just improving the pilot rating system as a first step.

It doesn't even do 3/3/3/3 anymore.

[Kin3ticX]

Tier 1 is such a broad brush

I think its more about making people feel good than it is about matchmaking

[BLOOD WOLF]

Does the MM still match by an ELO rating or was that changed?

looking back at the notes for the MM.

Edited by BLOOD WOLF, 23 April 2017 - 05:44 PM.

[Deathlike]

Kin3ticX, on 23 April 2017 - 05:16 PM, said:

Tier 1 is such a broad brush

I think its more about making people feel good than it is about matchmaking

Secret PGI MM Memo said:

Making sure everyone in Tier 1 feel like they are just as good as EmP and SJR should be a good thing.

[Dimento Graven]

vandalhooch, on 23 April 2017 - 12:58 PM, said:

I don't have to have analysis of my own to point out the weakness in the analysis of others. For someone who earlier tried to lecture me about how science works, you seem to be awfully ignorant about the process of peer review.

You've repeatedly told everyone who doesn't agree with you that their wrong, their numbers are somehow wrong, even though the numbers are gathered from the system, therefore their analysis are wrong, they don't have enough numbers, blah, blah, blah, blah, blah, blah, blah, BLAH...

At the very least, the numbers from the stomps show that AT A MINIMUM, for those matches, MORE OFTEN THAN NOT, the winning side has higher performing players than the other. In these instances MM fails to provide truly balanced teams.

Even with the hundred or so samplings provided by Tarogato and the OP, it was a very obvious thing to observe from the numbers.

I don't see how you can keep denying that, yet you do.

Quote

No it doesn't. That's the whole point. The sample sizes are too small to draw any conclusions about. The sample sizes are too small to claim that they have told you anything meaningful. You attaching meaning to them is just you expressing your own confirmation bias on what is in essence, random noise.

Gee, do I have to stick my hand in a thousand pots on stoves to understand that stoves can make things hot?

Or can do I get to stop when my hand is only mildly scalded and get to conclude, "Y'know, there might be something going on here..."

Quote

That isn't an objective definition. What units are "skill levels" measured in? How is "skill level" calculated? How far apart do these "skill levels" have to be to be defined as unbalanced?

Without any actual numbers you haven't defined anything. This just makes your earlier attempts to lecture others about how science works all the more laughable.

Ok Mr Webster, please define it for us. I've used the most common definitions bandied about on the forums by just about everyone, up to and including, PGI employees when they speak on the subject, so, YOU define it.

I'm tired of attempting to get past the fingers in your ears and the loud "NYAH NYAH NYAH NYAH" coming from your mouth...

Quote

You need actual numbers to claim that one team is "significantly more players" than the other. You are starting to sound like many politicians who tried to pass anti-porn legislation. "I know it when I see it."

So you're a 'porn scientist' now? Cool!

But seriously the numbers were there, gathered from the leader boards. In a 12v12 I would say any number above 2 is significant, ie: 3 or more.

I could have sworn that's what we were seeing from the numbers.

Quote

No they aren't. They are absolutely confounding factors. They must be accounted for in any analysis. Or, are you going to try and claim that low skill players in multiple LRM boats definitely don't have a greater chance of victory on Polar Highlands than HPG? What analysis have you done to show that maps play no role in determining the outcome of matches? Did the OP get a random sample of maps? Did the OP even record which maps the matches took place on?

Don't know for sure, so we need to properly analyze the data to see if there is no effect. If there isn't any effect, then any further studies can safely ignore map variety. You can't just ignore it now because you can't imagine there is an effect. That's biased data collection.

You being confident that it's true and the data showing that it's true are two completely different things. One is pseudo-scientific nonsense and the other is how science is actually done.

Greater number of low skill/noobs in LRM boats, on PH against a number of greater skilled/experienced players in whatever 'mechs they happen to choose? You're not seriously asking that question are you?

That's damn near a dream seal clubbing scenario, the noobs/low skills won't stand a chance. You are of course assuming that all the experience players only brought flamer LOLbuilds or something, I take it.

Maps have little bearing on how players apply their skills. Players can have favorite and least favorite maps, may know the terrain better, or what have you, but as far as affecting their ability to aim, pilot a 'mech, utilize situational awareness, get the most out of their 'mech, so on and so forth, the things that experienced/skilled players do oh so much better than noobs/low skill players, again, it'll be a difference that makes no difference.

Ignore maps as irrelevant because a good player doesn't suddenly forget how to aim, pilot his 'mech, or press the R key to target what he's shooting at just because he's on a certain map, while a bad player won't be able to consistently do these things on ANY map...

Quote

In other words, you have absolutely no data as to how quickly the average player moves through the Tiers. You could have just admitted you had no data. I already knew you didn't and so did everyone else.

Don't need it, the speed at which an average player moves up in rank is not actually relevant. The fact is PGI has admitted that you play enough games, unless you're doing unreasonably badly consistently, you're going to go up in rank.

Quote

Trend? What trend? Trends are patterns in time series data. Taragato did not do a time series analysis. There is no trend to be discovered.

The numbers appeared to show a general direction in which stomps occurred, where MM has created teams of lopsided skills.

That's the way the data was 'leaning', I dunno, what word do you want to use then?

Quote

Why? Taragato's data has absolutely no bearing on the question of "is the matchmaker failing to make adequately evenly matched teams?"

He only recorded data for matches that were STOMPS!

And his data appears to show that when stomps occur MM has created teams of lopsided skill, ergo, it failed to make evenly matched teams.

Quote

Scroll up to what? Your completely subjective definitions of evenly matched and skill level? Do you even comprehend what the term objective means?

Again Mr. Webster, you define it then.

I was using generally accepted terms, terms even PGI uses.

Y'know, kind of like how we define stars: "a self-luminous gaseous spheroidal celestial body of great mass which produces energy by means of nuclear fusion reactions"

We don't have to count the number of hydrogen atoms before we can declare, "Yes, it's a star."...

Quote

Do you have the slightest clue how they do that? Hint: It does not involve their ability to accurately predict the outcomes of sporting events. Vegas casinos do not gamble!

No, they don't. They figure the odds based on past performance. They have all sorts of statistics on the teams, the individual players, home vs away, temperature variances, time of day of the game, whether it's a weekday game vs. a weekend game. Sometimes the numbers they have makes NASA look like they're playing bingo...

So yeah, Vegas has what they feel is a 'reasonable certainty' of how a game is going to turn out and offers odds based on their outcome. As Vegas is more often right than wrong, and most gamblers ate betting their 'gut', Vegas makes pretty good money on it.

Me? When I go to Vegas, I stick to the best odds in the house: Blackjack and craps for me, sir.

Quote

Bwaaaa, haaaa, haaaaa. You couldn't be more wrong.

That's exactly how a professional client denier would be paid to respond.

Quote

Which match outcomes? Stomps only? All matches? All maps? Group only? Solo only? Group and solo? One mode only? All modes pooled together?

Which metrics (they aren't statistics until they are pooled or compared to others)? Why those? How are they to be weighted? Why? What about metrics that are derivations of the other metrics?

Averaging? How exactly do you mathematically average a win loss RATIO with a damage PER MATCH? What does that even mean? What units is your final number in?

Do I just take my 1.27 WLR and add it to my 325 damage per match score and divide by 2?

'Yeah, which CO2 sources are raising the temperature of the earth? Is the coal based ones, or the ones from burning petroleum? Do we care about burning wood? Should we dived it all by the number of cows farting methane?'

[Zergling]

Kin3ticX, on 23 April 2017 - 05:16 PM, said:

Tier 1 is such a broad brush

I think its more about making people feel good than it is about matchmaking

Given there were people complaining about the AMS Match Score nerf making it harder for them to gain PSR Tier, it sure seems a lot of people think it is something worth aiming for.

[Dimento Graven]

vandalhooch, on 23 April 2017 - 01:03 PM, said:

It's biased because you seem to be mathematically illiterate.

Define that please.

Quote

Was it inadvertent? How do you know? What is the frequency of actual matches having unbalanced teams according to Taragato's ranking system? I'll give you a hint, it isn't 80%.

So you're stating for the record that MM is doing it intentionally? And what was the number from Taragato's data then? Was that before or after he added more matches?

Quote

You still haven't actually measured this to draw such a conclusion.

If the intent is to created balanced teams, and one team ends up with more players (we'll start at 3 or more) with higher W/L ratios, avg. match score, avg. damage per match, than the other team, I'd define that as a fail.

Since we have the data from the stomps with these numbers, conclusion: For the matches resulting in stomps MM failed.

It's probably failing elsewhere but because the matches weren't 'stomps'...

Quote

This could be true, but you still don't have any actual evidence that it is. Neither the OP nor Taragato collected enough data in a systematic manner in order to provide that evidence.

How much is enough? Define 'systematic manner'.

Quote

That's not how science works. You made the claim, you provide the evidence to support your claim. I can't search for counter-evidence to something that only exists in your imagination.

You're claiming the conclusions are wrong because you don't like the number of samples or how it was gathered. So get the numbers in the manner most acceptable to you and prove to us the MM is always doing the job of creating balanced teams.

Quote

What should be the appropriate wait time to open valves in the current system? Should that time be calculated from the time the first pilot was slotted into the match or the latest? I think you'll find that there is absolutely no such thing as universal agreement on what those values should be.

So because no one can agree, you're arguing to do nothing?

No, this is an argument for argument's sake, pure and simple.

Quote

And objectivity doesn't factor in? I thought you were trying to lecture me about how science works and yet objectivity is nowhere included in the process of defining things.

Nope, objectivity would be bad, I have to be more... well call it 'empathic' than anything. Being able to see the problem from the customer's point of view goes a long way to providing a solution the customer is 'happy' to accept.

Quote

Sounds incredibly useful. It's not scientific.

And it's not scientific, it's 'practical and human' something science sometimes forgets. If I force the customer to bring me a thousand samples of whatever it is that's the problem, because... SCIENCE... well, FAIL.

No it's not science, but it pays the bills.

Quote

The problem is that you are confusing dust in the air for smoke. You don't actually understand how to tell the difference between dust and smoke.

And you're forgetting the fact that it doesn't matter if it's smoke or dust when the end result is: "KABOOM!!!"

You don't have to perfectly define a problem to know that there's a problem.

Just because a problem isn't perfectly defined doesn't mean it's imaginary.

Quote

And humans are incredibly terrible at reporting what actually happened instead of what they saw happen. Those two things are NOT the same thing. What you "see" happening is an experience in your brain but it is not a recording of what actually happened.

That's why science has rigorous protocols to eliminate the biases of human experience from any deliberations about what is actually happening.

Yeah usually in the real world, what they do is get statements from multiple "humans" about a specific event and when there's a lot of similar statements, they start their work in the direction of the common consistencies.

It's not perfect, but it's what we got.

If I got a room full of people telling me they're smelling smoke, I don't ask them for a chromatagraph of the air, or to define "smell"...

Quote

Because you have continually failed to give an objective definition for your terms.

Again, provide your definition. I've attempted and failed, I want to see how your definition is better than pretty much everyone else's including PGI's.

Quote

Is your system going to adjust players' rankings based on the results of their matches? How much will players move in your system? What about players who get pulled into higher or lower level matches due to the small player pool?

I thought you were getting rid of the current system! I'm asking how these situations will be handled in your newer, "better" system.

I see, I can't say that the MM is broken until I have already programmed its replacement.

In other words, we can't say man is causing the Earth to heat up until we find a way for man to cool the planet down.

If not Exxon, you work for BP then?

Quote

Nah, you got it mostly right. Although, I think ELO's were also used to initially seed the matches to begin with in addition to calculating movements after matches. Either way, it was an inappropriate system for essentially randomly assembled teams.

Agreed.

Quote

BTW: New accounts start mid-level Tier 5. The first few matches have PSR gain multipliers applied to move smurf accounts out of Tiers 4 and 5 rapidly.

This is good to know, although I should reply with a snarky, "Define rapidly"...

Quote

Seems you don't know a lot about your newer, "better" system.

But if you are designing a "better" system you have to speak to everyone's opinion on it. You have to program the valve release times. If you aren't willing to even address that simple tool, why should we care that you think you have a better system?

Sorry, I didn't know I wasn't allowed to say that the current system wasn't functioning as intended without having a fully designed ready to implement replacement ready.

Is this a new forum rule that I missed or something?

Quote

People reporting something is not the same thing as it actually happening. Confirmation bias is a real thing.

So now 12-2 and 12-3 matches are considered stomps? Why? Does everyone agree with that? Does this match Taragato's data you were citing in support of your position?

Didn't Taragato include 12-2 matches as well? Thought he came back later and added them...

Most people seem to respond that a stomp occurred when the match ended quickly (under 6 minutes) and the result was less than a lance of the enemy's forces were destroyed.

A lance or more dies and the end of game replies are typically, "Good brawl", which I won't translate as 'stomp' because it implies there was more of an actual two way fight taking place.

Quote

Why should I listen to you when you cite data that doesn't even match your definition of what constitutes a "stomp?"

You wouldn't, you don't think MM is broken at all, so any data that might show examples of a broken MM are subjected to unreasonable criticism.

Quote

A completely separate issue from "is the matchmaker failing to create evenly matched teams?"

You stated reasons for including a wider range of PSR rankings and time limitations on MM as the lack of player population. So, no it's not a separate issue, it's probably the original root cause of all this...

Quote

Still waiting on your proposed system. You keep saying that everything would be better if we had such a system but you don't seem capable of even describing how such a system would actually work.

Win and lose are part of match score. Damage dealt is part of match score. PSR moves are based on match scores, therefore PSR incorporates WLR and damage per match.

PSR is not an independent metric.

Theoretically possible is not the same as actually happening in reality. The matchmaker does not work off of what pilots are theoretically capable of having for metrics.

The PSR metric is a value separated from all the afore mentioned values. It increases based on each winning match result DISPRAPORTIONATELY to the amount it is decreased for a losing match result.

A system where there was no "win bias" a player winning 10 matches with receiving the highest possible PSR bump, then losing 10 matches receiving the highest possible PSR penalty should have a net gain of zero.

In the current system, as documented by PGI, that won't happen.

Because it's possible to lose more matches than win, and still go up in rank, PSR won't work for creating balanced teams in a match.

It is happening, we play with these people every day.

Quote

And I'll stop just as soon as you actually define your terminology in such a way that we can both understand what it is we are discussing.

You don't like my definition, provide one of your own. If it's good, and less than 3 pages long and doesn't include several large matrices, I might even accept it.

Quote

Are you trying to speak for him now? Why not just let him speak for himself?

In that particular sentence, where did I speak for Taragato? I simply commended him on his patience and amenability.

No where in that statement did I put any words in his mouth. I'm not sure where you got that...

Quote

No it isn't. The fact that you think it's indicative of anything at all just shows how ignorant you are of statistical analysis.

Hmmm... If you're not a professional climat denier, are you a recall specialist for the automotive industry?

Quote

I have in fact explained why "the numbers" don't really show what you claim they do. I don't need my own numbers to do that.

No, you've just the numbers were gathered incorrectly, aren't complete, and therefore any conclusions based on them are wrong.

Though the majority of matches that ended in a stomp, and had one sided with more players with higher stats than the other can't be ignored. EACH ONE of those is an example of MM not creating balanced teams. Since there were A LOT of those examples, even in such a small data set, it can be reasoned that there are many, MANY more matches out there where this is happening, AND, since we can see that in only 20% of the cases was the team with the lower overall stats able to win, there appears to be an 80% likelihood of the team with higher stats winning.

Quote

Now who sounds like a climate denier?

You're saying you didn't make accusations of cherry picking and the like?

Quote

Bwaaaa haaaa haaaaa. Nothing you have said has "pissed me off" in the slightest. When I come across someone who repeatedly refuses to admit they are only pretending to understand something, I find a great deal of satisfaction in showing how clueless they are.
...

Nothing passive about it.

Are you SURE you're not pissed off then?

Quote

Taragato admitted that he learned a great deal from his experience. He has earned my respect for both initial hard work and his willingness to listen and learn from criticism. You on the other hand are a pompous know-nothing.

Again, I admire Taragato's ability to be so amenable in the face of such purposeful obstinance.

He's a really nice guy, isn't he?

Quote

I'm not the one claiming to have statistics on my side . . .

If you are going to use them in a matchmaking algorithm, yes.

So new players will instantly jump in with the best of the best because they had one good match? Sound like a recipe for good new player experience?

Back to insisting there be a ready solution before we can point out that there's a problem...

Quote

Nice to see you feel confident enough to argue about a system that you don't even know the basics of.

So I wasn't up to date on PGI's new starting point for new players. Sue me.

I haven't started a new character since closed beta... Being admittedly ignorant on one minor point doesn't mean I'm ignorant, or incorrect about everything.

The fact that you have to harp on this one minor thing so hard is illuminating as to the confidence you have in your position.

Quote

Bwaaaaaa haaaaa haaaaaa.

For someone who was complaining about bad players being able to rise to Tier 1 in the current system you seem particularly dumb when it comes to your proposed alternative.

Do they do better in groups or in solo? Do you have any actual data to support such an assertion?

That's not how zero-sum systems work. You can't have a match of 24 killers and have all of them get high damage numbers. They can't all maintain high win-loss ratios.

I'm not going to fully define a solution for you in order to be allowed to point out there's a problem with the current system.

Quote

You have zero evidence that stomps are the result of unevenly balanced teams. Taragato's analysis does not address that claim. All Taragato's data shows is that if a stomp does occur, the "stronger team" usually wins. Well, duh. It does not show that unbalanced teams result in stomps more often than balanced teams do.

In the sample set most of the stomps were in matches with unbalanced teams, correct?

As I recall from the data (and I admit it's been a while since I looked) once the percentage difference in avg. match score gets beyond 10% it's a near certainty the team with the average higher match score will win.

That 10% disparity should be avoided by MM.

Quote

Do you even play the game anymore? Most of the stomps I've observed result in damage being spread more evenly. Of course, I could be wrong and we'd need to go back through Taragato's data to check. That's something that his data set could actually address already in fact.

Once or twice a week now, in the stomps I've participated in, I've always seen one or two do outstanding damage and kills on winning team.

Quote

The data you are referring to provide no evidence as to the actual question at hand.

We disagree.

Quote

In your new system, highly ranked players will eventually be pooled in with less skilled players because there won't be enough of them to fill out matches on demand. That is exactly the thing the OP and you are complaining about right now. Your new system will end up with the same results we already have.

Hence the call to advertise to get more players in the game, and the statement WAY up there (or in the other post, can't remember now) that low population is probably the at the root of all this.

Quote

You admit that the fundamental issue is small player pool and a new matchmaker won't change that. Then what's with all the crying about the current matchmaker?

Because even with a million new players, the win biased PSR system will do the same thing to them. Push everyone upwards in rank.

The PSR system should have more drastic penalties for losing.

[Khobai]

honestly if you take 24 random people and put 12 on each team, odds are the average skill level is going to be pretty close. the current tier system prevents the worst and best players from being mixed so what you get is a good average.

in my experience its not skill level of players where matchmaker fails the most. its tonnage and quality of mechs.

where matchmaker fails is when it does crap like put 4 kodiaks on one team but 0 kodiaks on the other team. in my experience the team that wins the most is the team thats given the better mechs or straight up more tonnage. games were far better balanced when we had tonnage matching or 3/3/3/3, unfortunately the wait times were also longer because no one wanted to play lights and mediums.

Edited by Khobai, 23 April 2017 - 10:03 PM.

[Savage Wolf]

Appogee, on 20 April 2017 - 12:58 AM, said:

No, it's the opposite of perfect.

A perfect matchmaker would have me hovering around 1:1 Win:Loss ratio all the time ... not ramping up to 1.1 with a run of easy wins, only to then have a run of impossible carry missions forcing me back down to 1.0.

So close to perfect is now the opposite of perfect. Got it. Yes, never leaving exactly 1.0000 ratio would be perfect, but also virtually impossible to achieve.
People are talking about as if the ratio was like 2.0 or 0.1. Face it, it's doing pretty well actually.

[Appogee]

Savage Wolf, on 24 April 2017 - 12:29 AM, said:

So close to perfect is now the opposite of perfect. Got it. Yes, never leaving exactly 1.0000 ratio would be perfect, but also virtually impossible to achieve.

No, you didn't get it. In fact, you missed the point entirely.

Let me try to help you understand, by using an example in extremis... Achieving a "perfect" 1:1 ratio is "bad" if that 1:1 ratio is achieved a "bad way". For example, if the 1:1 ratio is achieved by rigging match results so that a player first wins 10 easy matches in a row, then loses 10 impossibly hard matches in a row - repeated over and over - then that would be a "bad" way* to achieve what might seem like a "perfect" result. It's bad because it means the player's performance of no consequence, so why should they bother even trying in the 10 matches where they have been pre-determined to lose?

A "perfect" matchmaker would consistently make evenly-matched teams, and over thousands of matches, luck and statistics would lead to close to a 1:1 ratio, with fair odds of winning and losing throughout.

[Widowmaker1981]

Appogee, on 24 April 2017 - 02:41 AM, said:

A "perfect" matchmaker would consistently make evenly-matched teams, and over thousands of matches, luck and statistics would lead to close to a 1:1 ratio, with fair odds of winning and losing throughout.

how can it possibly do that, or even get close, when it thinks you (for example) are exactly as good as Proton (for example)?

It cant, because PSR isnt really very good.

Its not creating bad matches on purpose, it just doesnt have the information required to create better ones.

[Savage Wolf]

Appogee, on 24 April 2017 - 02:41 AM, said:

No, you didn't get it. In fact, you missed the point entirely.

Let me try to help you understand, by using an example in extremis... Achieving a "perfect" 1:1 ratio is "bad" if that 1:1 ratio is achieved a "bad way". For example, if the 1:1 ratio is achieved by rigging match results so that a player first wins 10 easy matches in a row, then loses 10 impossibly hard matches in a row - repeated over and over - then that would be a "bad" way* to achieve what might seem like a "perfect" result. It's bad because it means the player's performance of no consequence, so why should they bother even trying in the 10 matches where they have been pre-determined to lose?

A "perfect" matchmaker would consistently make evenly-matched teams, and over thousands of matches, luck and statistics would lead to close to a 1:1 ratio, with fair odds of winning and losing throughout.

But then I'm not wrong. A perfect matchmaker would have a W/L ratio close to 1. People are literally saying that the MM is bad BECAUSE it produces a W/L ratio of 1. What you explained is that it matters HOW it becomes 1.0 to which I agree, but that still doesn't make 1,0 bad because it's still part of the goal.

In short: The statistic itself IS close to perfect. The means by which this is achieved is possibly bad.

[BlueStrat]

Clydewinder, on 19 April 2017 - 03:21 PM, said:

when i can take a joke build vindicator and score in the top 3 for damage and match score, i know the MM is gone full on spud cannon because of a positive w/l metric.

the tier system is not workable for MM balance because it is an XP bar and only by running Victors and/or Vindicators can you make that bar go back down again

I'm a living exception to that 'rule'. Note, I play QP single-drop PUG matches only. Back around November/December of 2016 I was working my way steadily up Tier 3 anticipating T2 and then something changed, either MM was changed or a lot of old players quit and a flood of new players joined, or something else major happened, as suddenly teams went braindead and my Tier and stats have dropped like a rock and my PSR dropped all the way back to almost 2/3rds of a bar in T4.

I think that LordNothing makes an excellent point. The MatchMaker drops matching criteria when the available pool of players in que is low. I'm not sure if it's a 'hard' cutoff at a certain population point where all criteria are dropped at once, or if criteria are dropped in increments as the population varies.

Depending on when the matches in the OP's study were played, and therefor the number of players in que, possibly only the very broadest of matchmaking filtering may have been occurring because there were not enough players in que to reach the minimum number upon which MM was programmed to trigger more/better/any? filtering/selection.

[EekaBlitzer]

Thanks to the OP. The continued failure of MM, which I have been all too horribly reminded of after returning to QP recently after solely playing FW for a couple of months, needs to be kept as a source of complaint.
The pseudo statistical objections to the OP keep referring to a minimum sample size requirement, but those sample sizes needed, vary hugely according to the statistical correlation, in this case as to how one-sided the team set up is compared to the match result. If the correlation is VERY strong, then even small samples can be entirely appropriate.

[Savage Wolf]

EekaBlitzer, on 24 April 2017 - 04:15 AM, said:

Thanks to the OP. The continued failure of MM, which I have been all too horribly reminded of after returning to QP recently after solely playing FW for a couple of months, needs to be kept as a source of complaint.
The pseudo statistical objections to the OP keep referring to a minimum sample size requirement, but those sample sizes needed, vary hugely according to the statistical correlation, in this case as to how one-sided the team set up is compared to the match result. If the correlation is VERY strong, then even small samples can be entirely appropriate.

The strength of the correlation does not have any impact on the appropriate size of the samle whatsoever. That makes no logical sense whatsoever. What we are trying to determine is the strength of the correlation, so if you already know this, why are you making a test in the first place?
It would be more likely that you are talking about the strength of your bias which would fit the description.
I don't believe that we need science levels of accuracy, which would be 1000 samples. 100 would do fine to show some indication, but we only have 12.

[BLOOD WOLF]

Savage Wolf, on 24 April 2017 - 02:50 AM, said:

But then I'm not wrong. A perfect matchmaker would have a W/L ratio close to 1.

A perfect MM would not have a W/L ratio of one. IT would just simply pair people perfectly

[Savage Wolf]

BLOOD WOLF, on 24 April 2017 - 05:20 AM, said:

A perfect MM would not have a W/L ratio of one. IT would just simply pair people perfectly

Which would naturally result in a W/L ratio of 1. Otherwise, what would you think the ideal W/L would look like?

[Appogee]

Savage Wolf, on 24 April 2017 - 02:50 AM, said:

In short: The statistic itself IS close to perfect. The means by which this is achieved is possibly bad.

Yes I think we are understanding each other now.

[Appogee]

Widowmaker1981, on 24 April 2017 - 02:49 AM, said:

how can it possibly do that, or even get close, when it thinks you (for example) are exactly as good as Proton (for example)?

It cant, because PSR isnt really very good.

Its not creating bad matches on purpose, it just doesnt have the information required to create better ones.

Yes, I agree.

Matchmaker needs a more granular skill number than can be provided by five broad tiers, and that number should be equally able to go up or down, based on performance.

The matchmaker could then allocate 24 people across the teams in order, from highest score to lowest. Just like we did at school creating ad hoc football teams at lunchtime.

Edited by Appogee, 24 April 2017 - 05:29 AM.