248 replies to this topic

[The6thMessenger]

Der Geisterbaer, on 11 June 2019 - 08:04 AM, said:

I guess you going ad hominem there is a testimony to your ability of reflecting over your own confirmation bias and dealing with systematic errors?! ~laugh~

...

He doesn't seem to be trying to insult you. I don't know to tell you, but IQ does fall in Normal Distribution. It's just a standard of distribution for a population, no need to be snippy about it. Such as you are well within reason to assume that average people where IQ is between 70 and 130 is around 68% of the population.

As you said it, Jarls List AMS falls at a normal distribution, he also just used Normal Distribution as heuristics, to represent the skill derived from the normal distribution of AMS in the Jarls. His point is specifically that people could be normally distributed, from Height, IQ, and because AMS could also be normally-distributed, it stands to reason that him using Normal Distribution is representative of players.

Fun fact, it's called Laplace-Gaussian Curve, and I didn't have to google it.

Nightbird, on 11 June 2019 - 02:51 PM, said:

I've found that farming wins and farming match score are opposing goals. When I want to farm match score, I take high dps dakka mechs and spray mechs down from arm to arm for maximum damage before killing them. You can do this also with LRMs and ATMs. When I want to farm wins, I take laser vomit and drill through the CT without wasting damage on anything else.

My concern is that, well Dakka could both farm damage and wins, maybe even the ATMs and the LRMs (with DF). You could also bring an AMS (Anti-Missile-System), to just pad MS anyways. They might be opposite goals, but with certain weapons, they have near same result.

AMS should only have given CBills, not MS. AMS would have been an acceptable measure of skill, if only PGI didn't made it so that it's easy to pad it.

Edited by The6thMessenger, 11 June 2019 - 03:28 PM.

[Nightbird]

The6thMessenger, on 11 June 2019 - 03:19 PM, said:

My concern is that, well Dakka could both farm damage and wins, maybe even the ATMs and the LRMs (with DF). You could also bring an AMS (Anti-Missile-System), to just pad MS anyways. They might be opposite goals, but with certain weapons, they have near same result.

AMS should only have given CBills, not MS. AMS would have been an acceptable measure of skill, if only PGI didn't made it so that it's easy to pad it.

Consider this thought problem then: Two players contribute equally to their teams, and over the course of thousands of games, both have 1.5 WLR as a result. However, one player is obsessed with increasing his MS for Jarl's list, and the other doesn't even know the list exists. As a result, one has 300 average match score, and the other has 400 avgMS. How do you build a match maker that recognizes these two players are the same skill?

[The6thMessenger]

Nightbird, on 11 June 2019 - 04:00 PM, said:

Consider this thought problem then: Two players contribute equally to their teams, and over the course of thousands of games, both have 1.5 WLR as a result. However, one player is obsessed with increasing his MS for Jarl's list, and the other doesn't even know the list exists. As a result, one has 300 average match score, and the other has 400 avgMS. How do you build a match maker that recognizes these two players are the same skill?

Honestly, I don't know. Define skill? And how is it tied to contribution?

Because if it's all about contribution to the win, you don't have to be completely skilled to contribute when you're bringing something like an AMS that could nullify enemy missile boats. Of course that doesn't happen a lot, but it still contributes to the win.

[Nightbird]

The6thMessenger, on 11 June 2019 - 04:16 PM, said:

Honestly, I don't know. Define skill? And how is it tied to contribution?

Because if it's all about contribution to the win, you don't have to be completely skilled to contribute when you're bringing something like an AMS that could nullify enemy missile boats. Of course that doesn't happen a lot, but it still contributes to the win.

Skill is your ability to help your team win, the more you have, the more your team will win, and vice versa if less. If you are completely average, your team will win 50% of the time.

The specific actions you take in helping your team win doesn't matter. You can drop call targets and strategies, and be charismatic enough that people listen, just as an example.

Edited by Nightbird, 11 June 2019 - 04:24 PM.

[East Indy]

This is an awesome thread and it pains me to suspect that PGI won't know about it, or what to do in response.

The6thMessenger, on 11 June 2019 - 03:19 PM, said:

My concern is that, well Dakka could both farm damage and wins, maybe even the ATMs and the LRMs (with DF). You could also bring an AMS (Anti-Missile-System), to just pad MS anyways. They might be opposite goals, but with certain weapons, they have near same result.

AMS should only have given CBills, not MS. AMS would have been an acceptable measure of skill, if only PGI didn't made it so that it's easy to pad it.

I don't see performance coloration as an issue. Competition does an excellent job at self-correcting -- below-average players can pad at a relative level but not against better players, whereas those who can really milk points are exceptional anyway. I don't think I've seen a player behave very far outside his Jarl's List ranking. Of course, PSR overrides that and we get what we have today.

[Wolfos31]

I would hope they post one more pass on MM before they consider MWO feature complete. I'm still in tier 2, but when I play matches it's often stomps, one way or another. Even if they don't drastically improve MM I'd like it if they just made the consequences equal, instead of having the tier system be an escalator to the top. I'd be fine sitting in tier 3-2 forever. I honestly don't believe I'm a tier 1 player. But I know I'll get there in the current system.

[Der Geisterbaer]

The6thMessenger, on 11 June 2019 - 03:19 PM, said:

...

He doesn't seem to be trying to insult you.

~laugh~ Yeah, his answers were totally free of ad hominem, were they?

The6thMessenger, on 11 June 2019 - 03:19 PM, said:

I don't know to tell you, but IQ does fall in Normal Distribution.

And my question to you is: What part of my initial comment or what came after indicated that I actually needed any "education" on that - be it from Nightbird or now you?

The6thMessenger, on 11 June 2019 - 03:19 PM, said:

It's just a standard of distribution for a population, no need to be snippy about it.

Now the question is: Who was "snippy" about it? ~giggle~

Quote

As you said it, Jarls List AMS falls at a normal distribution, he also just used Normal Distribution as heuristics, to represent the skill derived from the normal distribution of AMS in the Jarls.

The more interesting parts about that are

• the question whether or not the normal distribution in Jarl's List does cast a doubt onto Nightbird's premise that a Match Maker in general or the current one in particular indeed does not have access to a metric that represents "skill"
  
• the question if some of his claims about the short comings of average MS - like players deliberately tampering with it - are indeed true

The6thMessenger, on 11 June 2019 - 03:19 PM, said:

His point is specifically that people could be normally distributed, from Height, IQ, and because AMS could also be normally-distributed, it stands to reason that him using Normal Distribution is representative of players.

See, it was never about a normal distribution not being a suitable premise but straw manning just happens to be one of the employed fallacies there

The6thMessenger, on 11 June 2019 - 03:19 PM, said:

Fun fact, it's called Laplace-Gaussian Curve, and I didn't have to google it.

Congrats ... so we could now pat our backs on as to why the central limit theorem and approximate normality would be center pieces of the discussion at hand ... provided that an actual discussion were the goal here?!

Edited by Der Geisterbaer, 11 June 2019 - 05:24 PM.

[The6thMessenger]

Der Geisterbaer, on 11 June 2019 - 05:23 PM, said:

~laugh~ Yeah, his answers were totally free of ad hominem, were they?

The "rag on your gray-matter" is for sure. But merely referencing IQ and Height as part of Normal Distribution isn't.

Der Geisterbaer, on 11 June 2019 - 05:23 PM, said:

And my question to you is: What part of my initial comment or what came after indicated that I actually needed any "education" on that - be it from Nightbird or now you?

Probably part where you over reacted.

Der Geisterbaer, on 11 June 2019 - 05:23 PM, said:

Now the question is: Who was "snippy" about it? ~giggle~

Both of you for sure, but there's no reason to go to his level. (On the assumption that you think of him as low for being the one snippy)

Der Geisterbaer, on 11 June 2019 - 05:23 PM, said:

The question whether or not the normal distribution in Jarl's List does cast a doubt onto Nightbird's premise that a Match Maker in general or the current one in particular indeed does not have access to a metric that represents "skill"

So what you're implying is that the current MM already has a system that pushes the people into a Normal Distribution?

Der Geisterbaer, on 11 June 2019 - 05:23 PM, said:

the question if some of his claims about the short comings of average MS - like players deliberately tampering with it - are indeed true

I see, so your point is that with AMS falling on Normal Distribution as well, then it's just as representative of WLR system?

I don't know about him, but while I don't have confidence with AMS being indicative of skill, well it's indicative of something.

If it's about figuring out the skill, maybe we could derive a new representation based on the ratio of WLR and AMS. I mean if you are doing 400 AMS for 1.5 WLR, and the other is doing it at 300 AMS, maybe the one with less AMS for the same WLR is just better?

Der Geisterbaer, on 11 June 2019 - 05:23 PM, said:

See, it was never about a normal distribution not being a suitable premise but straw manning just happens to be one of the employed fallacies there

Right okay. Like what though?

Der Geisterbaer, on 11 June 2019 - 05:23 PM, said:

Congrats ... so we could now pat our backs on as to why the central limit theorem and approximate normality would be center pieces of the discussion at hand ... provided that an actual discussion were the goal here?!

I was just trying to lighten the mood.

[Nightbird]

The6thMessenger, on 11 June 2019 - 05:36 PM, said:

I see, so your point is that with AMS falling on Normal Distribution as well, then it's just as representative of WLR system?

I don't know about him, but while I don't have confidence with AMS being indicative of skill, well it's indicative of something.

If it's about figuring out the skill, maybe we could derive a new representation based on the ratio of WLR and AMS. I mean if you are doing 400 AMS for 1.5 WLR, and the other is doing it at 300 AMS, maybe the one with less AMS for the same WLR is just better?

You're almost there. Both players help their teams win the same, but the player with 400 MS is more skilled. The question is, more skilled at what? Both players are equally skilled at winning. Therefore the only direction left is, the person with higher match score is more skilled at getting match score.

In other words, when we have both WLR and MS available, WLR tells us how valuable a player is in helping the team win, and MS tells us how much MS he will earn while doing it, sort of like if we had a stat called average C-bills earned. Just like MS, more C-bills earned usually means a better player, but like MS it's not going to be very accurate.

Edited by Nightbird, 11 June 2019 - 07:10 PM.

[Dee Eight]

Nightbird, on 10 June 2019 - 04:37 AM, said:

It's also not my policy to be nice to someone who starts out rude.

I have a policy that I don't play well with stupid people.

[The6thMessenger]

Nightbird, on 11 June 2019 - 06:59 PM, said:

You're almost there. Both players help their teams win the same, but the player with 400 MS is more skilled. The question is, more skilled at what? Both players are equally skilled at winning. Therefore the only direction left is, the person with higher match score is more skilled at getting match score.

In other words, when we have both WLR and MS available, WLR tells us how valuable a player is in helping the team win, and MS tells us how much MS he will earn while doing it, sort of like if we had a stat called average C-bills earned. Just like MS, more C-bills earned usually means a better player, but like MS it's not going to be very accurate.

So if WLR is how valuable a player, and the MS is how much he earned, how is skill relevant again? No really, it's not a rhetorical question.

So WLR is where we could assume the potential of contribution, but AVG is the actual result of the contribution.

Can I assume that your MM isn't really about skill, but about potential of contribution? But if that's the case, why couldn't we just get on the actual result of the contribution because it's more indicative of better player?

That being said, if it was distributed both by Normal Distribution, would it mean that it doesn't matter, because using the metric of AMS would have simmilar result as with WLR?

So with that, here's an Idea.

What if we develop Tiers based on average match-score since it's supposed to be indicative of what it means to be a "better player", we gather players in a bucket based on the closest AMS but distribute the players within the bucket towards the team by WLR?

Edited by The6thMessenger, 11 June 2019 - 07:56 PM.

[W4R GOD]

Awesome thread Nightbird, thanks for looking at all of this and putting it into an understandable presentation.

PGI needs to take a read.

[Nightbird]

The6thMessenger, on 11 June 2019 - 07:52 PM, said:

So if WLR is how valuable a player, and the MS is how much he earned, how is skill relevant again? No really, it's not a rhetorical question.

So WLR is where we could assume the potential of contribution, but AVG is the actual result of the contribution.

Can I assume that your MM isn't really about skill, but about potential of contribution? But if that's the case, why couldn't we just get on the actual result of the contribution because it's more indicative of better player?

That being said, if it was distributed both by Normal Distribution, would it mean that it doesn't matter, because using the metric of AMS would have simmilar result as with WLR?

So with that, here's an Idea.

What if we develop Tiers based on average match-score since it's supposed to be indicative of what it means to be a "better player", we gather players in a bucket based on the closest AMS but distribute the players within the bucket towards the team by WLR?

Suppose you're dying from a serious injury, you're at the hospital, and two doctors (Doctor A and Doctor B ) can perform the procedure. You have to choose who will perform your surgical procedure. Doctor A tells you he has a 75% success rate with this procedure, the doctor B tells you he earns 150k $ on average from doing the procedure. You only have strength left to ask one question. Do you ask 1) Doctor A what he earns from his procedures on average, or 2) Doctor B what his success rate is?

Edited by Nightbird, 11 June 2019 - 08:25 PM.

[The6thMessenger]

Nightbird, on 11 June 2019 - 08:11 PM, said:

Suppose you're dying from a serious injury, you're at the hospital, and two doctor can perform the procedure. Doctor A tells you he has a 75% success rate with this procedure, the doctor B tells you he earns 150k $ on average from doing the procedure. You only have strength left to ask one question. Do you ask 1) Doctor A what he earns from his procedures on average, or 2) Doctor B what his success rate is?

I really don't know. I mean maybe 75% is the lower one? What if the guy that is being paid by a lot more (assuming is a lot more) has better success rate, that is a result of being paid by a lot more.

I don't get it, you're sending mix messages.

[Nightbird]

The6thMessenger, on 11 June 2019 - 08:15 PM, said:

I really don't know. I mean maybe 75% is the lower one? What if the guy that is being paid by a lot more (assuming is a lot more) has better success rate, that is a result of being paid by a lot more.

I don't get it, you're sending mix messages.

Ask the question (1 or 2), I'll give you the doctor's answer

[The6thMessenger]

Nightbird, on 11 June 2019 - 08:17 PM, said:

Ask the question (1 or 2), I'll give you the doctor's answer

Oookay, so which one is better?

What does the 75%'s earnings?
What is the 150$K's success rate?

Edited by The6thMessenger, 11 June 2019 - 08:52 PM.

[Nightbird]

The6thMessenger, on 11 June 2019 - 08:51 PM, said:

Oookay, so which one is better?

What does the 75%'s earnings?
What is the 150$K's success rate?

LOL, you are supposed to choose to ask one question to either doctor. Do you want to ask Doctor A his earnings, or Doctor B his past success rate?

Anyways, bed time for me, so here are the results

Choose you own adventure!

Nightbird, on 11 June 2019 - 08:11 PM, said:

Suppose you're dying from a serious injury, you're at the hospital, and two doctors (Doctor A and Doctor B ) can perform the procedure. You have to choose who will perform your surgical procedure. Doctor A tells you he has a 75% success rate with this procedure, the doctor B tells you he earns 150k $ on average from doing the procedure. You only have strength left to ask one question. Do you ask 1) Doctor A what he earns from his procedures on average, or 2) Doctor B what his success rate is?

1) Gasping for breath as blood oozed from the gunshot wounds in your stomach. You turn to Doctor A and with your last strength, ask, 'what do you earn on average from your procedures?'. He looks shocked, but responds, '100k'. With your last remaining strength, you point towards Doctor B and lose consciousness.... The sounds of a child and a woman weeping can be heard, grasping the hands of a pale man. The shrill tone and flat line on the monitor nearby finally gave way to silence as the nurse disconnected the wires. After a long while, the woman finally collected herself and began speaking with the nurse. The topic came to your odds of surviving the surgery, when the nurse said, "I wish he'd picked Doctor A. It's true he makes less per procedure, but he is new and doesn't understand the system. Doctor B has been here for a long time and knows how to game the system, that's why he makes more but his success rate is only 50%"


2) Gasping for breath as blood oozed from the gunshot wounds in your stomach. You turn to Doctor B and with your last strength, ask, 'what is your success rate?'. He looks embrassed, but responds, '50%'. With your last remaining strength, you point towards Doctor A and lose consciousness.... The sounds of a child and a woman weeping can be heard, grasping the hands of a pale man. The sound of drips from an IV blends together with a steady beep from a machine in the background. "I'll be fine" a weak voice can be heard repeatingly reassuring.

Edited by Nightbird, 11 June 2019 - 09:03 PM.

[The6thMessenger]

Nightbird, on 11 June 2019 - 08:58 PM, said:

LOL, you are supposed to choose to ask one question to either doctor. Do you want to ask Doctor A his earnings, or Doctor B his past success rate?

Anyways, bed time for me, so here are the results

Choose you own adventure!

Spoiler

1) Gasping for breath as blood oozed from the gunshot wounds in your stomach. You turn to Doctor A and with your last strength, ask, 'what do you earn on average from your procedures?'. He looks shocked, but responds, '100k'. With your last remaining strength, you point towards Doctor B and lose consciousness.... The sounds of a child and a woman weeping can be heard, grasping the hands of a pale man. The shrill tone and flat line on the monitor nearby finally gave way to silence as the nurse disconnected the wires. After a long while, the woman finally collected herself and began speaking with the nurse. The topic came to your odds of surviving the surgery, when the nurse said, "I wish he'd picked Doctor A. It's true he makes less per procedure, but he is new and doesn't understand the system. Doctor B has been here for a long time and knows how to game the system, that's why he makes more but his success rate is only 50%"


2) Gasping for breath as blood oozed from the gunshot wounds in your stomach. You turn to Doctor B and with your last strength, ask, 'what is your success rate?'. He looks embrassed, but responds, '50%'. With your last remaining strength, you point towards Doctor A and lose consciousness.... The sounds of a child and a woman weeping can be heard, grasping the hands of a pale man. The sound of drips from an IV blends together with a steady beep from a machine in the background. "I'll be fine" a weak voice can be heard repeatingly reassuring.

Okay, I get it.

If that's the case, then we could just change it up. Gather players based on their WLR, and then distribute them base on AMS.

[Wil McCullough]

The6thMessenger, on 11 June 2019 - 09:08 PM, said:

Okay, I get it.

If that's the case, then we could just change it up. Gather players based on their WLR, and then distribute them base on AMS.

No point doing the latter. Just do the first and then separate them into teams as per usual (distributed by mech weight).

If it works (reduces stomp occurance), then you can think of whether to change distribution criteria to further optimize the formula via testing. This will probably take up to two years for reliable results (quarterly performance comparisons etc.).

Point is to minimize new variables so you can attribute an effect to the ones you implement. If not you'll end up with a lot of variables and can't tell what works and what doesn't. Attempting to change.anything after that will be like throwing darts at a dartboard you can't see.

[MrMadguy]

Nightbird, on 11 June 2019 - 06:59 PM, said:

You're almost there. Both players help their teams win the same, but the player with 400 MS is more skilled. The question is, more skilled at what? Both players are equally skilled at winning. Therefore the only direction left is, the person with higher match score is more skilled at getting match score.

In other words, when we have both WLR and MS available, WLR tells us how valuable a player is in helping the team win, and MS tells us how much MS he will earn while doing it, sort of like if we had a stat called average C-bills earned. Just like MS, more C-bills earned usually means a better player, but like MS it's not going to be very accurate.

You forget about carrying. If you're good, but your team is bad - you lose. If you're bad, but your team is good - you win. Therefore WLR doesn't fully represent your PERSONAL skill. This idea isn't just assumption. It comes from my personal experience. When teams are balanced, but have, lets say, 2 good players, 4 mediocre and 4 bad players, it's that 2 good players, WHO DEFINE RESULT OF MATCH. No matter, how 4 bad players play - they will NEVER AFFECT RESULT OF MATCH. Result? May be bad players have WLR = 1. But their MMR rating isn't actually their. It's actually rating of players, who carry them.

Edited by MrMadguy, 12 June 2019 - 12:44 AM.