155 replies to this topic

[SFC174]

DaZur, on 11 April 2022 - 07:08 PM, said:

Correction: This team game resolves in three ways... Win, lose draw. 24 individual players contribute to that binary outcome. The point of my examples was to infer you can have a great game and still loss the match. Conversely, you can contribute very little to a team win yet still net a statistical win.

A win/loss binary outcome is a dependent variable as it relates to the other metrics that contribute to that binary outcome.

Et all, I'm mealy stating opinion and responding to questions and propositions laid out in front of me... I'm to old for a pissing match but do believe any discussion is good discussion so long as all parties play nice and ""someone/anyone" might glean something useful from it.

Please go and read some of Nightbird's older threads about win loss ranking. You're completely correct there are many variables that go into winning a match. But with a large enough sample size, the impact of those variables becomes statistically neutral and the only variable that matters for your aggregate WLR is your contribution. Thus it serves as a much better ranking tool than PSR which is flawed and can be gamed.

I understand that some of this can be a bit thick to wade through, and I do have math background and have had to use basic statistical analyses in some of my work. But I don't have the command of the topic like Nightbird who seems to do it everyday for a living. That said, some of the stuff he's posted is pretty simplified so that most people can understand it without too much trouble.

WLR ranking/matchmaking won't fix most of what's wrong with the system, but it would be an improvement for what _seems_ like little effort.

[SFC174]

pbiggz, on 11 April 2022 - 07:39 PM, said:

The point im making is that taking a huge dookie on the existing match maker because you made some graphs that say WLR is "n% better" is misleading. WLR is not a silver bullet that will solve all our problems. All it will do is push different people into the margins. Your comments regarding player retention are all assumptions, and without a window into the actual source code you have no grounds to speak authoritatively. If you're honest about the assumptions you're making, thats fine, be honest about them, but don't talk like an authority because you figured out how to use microsoft excel.

And further to that point, taking a huge dookie on everyone who suggests ways to tune up the existing matchmaker is non-constructive and just makes you look like a ****.

I get that Nightbird may not be the most sociable fellow, but please go look at what he's posted in the past. The analysis of retention rates used PGI data to tease out how likely players that fell into different skill categories would stay with the game each month. Then by looking at predicted WLR changes after implementing a WLR based MM (basically the WLR for the entire population would be tightened up to be much closer to 1.0), he used that to predict changes in player retention.

Sure, you can question whether the correlation between WLR and player retention equates to causation, but the numbers look good statistically. You can question whether WLR ranking would make the improvement that Nightbird claims it would, but the modelling he used for that prediction was extremely good at predicting the results of the current matchmaker after it was implemented.

All told the numbers look good and the accuracy of the models has also been good. Given that the actual implementation of a WLR MM seems as simple (or moreso) as the current MM was, it would be nice to try.

[Nightbird]

pbiggz, on 12 April 2022 - 10:34 AM, said:

If you can't understand why ******** on everyone who thinks differently from you isn't an effective form of communication then you should stop talking at all. It doesn't matter how qualified you think you are. It doesn't matter how qualified you actually are. You're being a ****.

Welp sorry I'm not suited for being a kindergarten teacher. If you're so very wrong and have such significant real life consequences for being wrong, you can only expect the proverbial ruler or paddle from me.

I'm not just hard on the community, I'm also hard on PGI. If MWO had a WLR MM from the beginning, due to the effects of compounding on retention, you could have made 2x to 5x more revenue over the life of the game to date. This is a fun game, messed up because of the MM. If you made 100 million $ over the life of the game (I don't know, just a random guess), you could have made 200-500 million $ instead. Look at the match stomp rates for new players (which I predicted to be high), look at the attrition rates for them. After you do so, you'll be able to finally open your eyes and see the hole you've put in your own foot.

Edited by Nightbird, 12 April 2022 - 10:45 AM.

[pbiggz]

Nightbird, on 12 April 2022 - 10:43 AM, said:

Welp sorry I'm not suited for being a kindergarten teacher. If you're so very wrong and have such significant real life consequences for being wrong, you can only expect the proverbial ruler or paddle from me.

You think you're being firm, you're being an ***. It doesn't matter what you say, if you can't communicate it effectively nobody is going to listen to you, and a smug "i told you so" attitude is how you get ignored.

Nightbird, on 12 April 2022 - 10:43 AM, said:

I'm not just hard on the community, I'm also hard on PGI.

The people who have had the most positive effect on the game are people who have been able to communicate meaningful critiques without being smug self-satisfied *******. I.E. cauldron types. If you want to offer criticisms, you better learn to do it without spitting in peoples' faces or nobody will have the time of day. Its not about being a kindergarten teacher, its about communicating effectively with your fellow man, which you are unable to do.

Nightbird, on 12 April 2022 - 10:43 AM, said:

If MWO had a WLR MM from the beginning, due to the effects of compounding on retention, you could have made 2x to 5x more revenue over the life of the game to date. This is a fun game, messed up because of the MM. If you made 100 million $ over the life of the game (I don't know, just a random guess), you could have made 200-500 million $ instead.

If you think experience with statistics makes you a prophet you are grossly overconfident.

Nightbird, on 12 April 2022 - 10:43 AM, said:

Look at the match stomp rates for new players (which I predicted to be high), look at the attrition rates for them. After you do so, you'll be able to finally open your eyes and see the hole you've put in your own foot.

And did your scrying take into account externalities like the basic design of the game, the open-endedness of the mech lab, the lack of lobbies, the lack of respawns, the age of the game, the relatively low profile of the IP, or the general thorniness of the mechdad community? Because all of those things impact everything from stomp-rates, to player retention.

I don't have a problem with the substance of what you're saying if you're honest about the limitations of your own numbers, but you aren't, and you're smug about it too, so if you want to dig a pit, sit in it, and scream about how right you were, and how wrong everyone else is, you're welcome to, nobody's going to listen.

[Nightbird]

Why do you keep crying, I already said I'm not a kindergarten teacher. I have no clue on what to do with you lol.

Edited by Nightbird, 12 April 2022 - 11:27 AM.

[pbiggz]

perhaps stop being a ****

[Nightbird]

Nothing you say matters, you're not Jay Z, Dubstep, Fox or any other people who has at least input into the discussion, and yes, I criticized. You got nothing.

[pbiggz]

Nightbird, on 12 April 2022 - 11:30 AM, said:

Nothing you say matters, you're not Jay Z, Dubstep, Fox or any other people who has at least input into the discussion, and yes, I criticized. You got nothing.

Way to prove my point.

[Nightbird]

pbiggz, on 12 April 2022 - 11:36 AM, said:

Way to prove my point.

As long as my point that the current PSR and Match Maker is hot garbage wasn't refuted, I'm fine with that.

Better to be a jerk and right than nice and dumb.

[martian]

pbiggz, on 12 April 2022 - 11:36 AM, said:

Way to prove my point.

Is Nightbird's math correct or incorrect?

Edited by martian, 12 April 2022 - 11:46 AM.

[DaZur]

martian, on 12 April 2022 - 11:41 AM, said:

Is Nightbird's math correct or incorrect?

NB's data appears to be very much correct and at face value is great example of hypothesis testing.

While I have no way of proving or disproving (I honestly wouldn't even know where to start) It "feels" like he's kind'a ignoring statistical MWO outliers that directly affect his conclusions... correlation does not necessarily imply causation, as some would say.

That said, "Yes" Happy players is a desirable outcome... Doesn't take math to draw that conclusion (But is does look great in chart form). So to that end, NB's math proofs out.

[Nightbird]

DaZur, on 12 April 2022 - 12:28 PM, said:

While I have no way of proving or disproving (I honestly wouldn't even know where to start) It "feels" like he's kind'a ignoring statistical MWO outliers that directly affect his conclusions... correlation does not necessarily imply causation, as some would say.

Actually, in my simulation I used player stats from the population then. This includes very high and low skilled players, players who play a lot or a little. When I select players to form a simulated match, I even use the number of games they played as the chance to select that person. As a result, stomps still happen in my simulated matches, as the graphs show. It's just lower, which is what we want.

The results between the simulation and reality wouldn't be so close if I excluded those outliers.

Edited by Nightbird, 12 April 2022 - 01:32 PM.

[Knownswift]

DaZur, on 11 April 2022 - 07:08 PM, said:

Correction: This team game resolves in three ways... Win, lose draw. 24 individual players contribute to that binary outcome. The point of my examples was to infer you can have a great game and still loss the match. Conversely, you can contribute very little to a team win yet still net a statistical win.

Whether or not it is a team game is completely irrelevant.

If you perform well you win more than you lose, and this is shown by the public data EVERY registered user of this game can access, in team sportsball, auto racing, or any other activity that revolves around competition.

A player's performance, their impact, every emergent in game event, and every intrinsic value is represented by the win loss ratio, especially over a large sample size.
Your assumptions are absurd, your cherry picking is weak. That isn't even an original strawman.

Edited by Knownswift, 12 April 2022 - 01:47 PM.

[Nightbird]

Knownswift, on 12 April 2022 - 01:32 PM, said:

A player's performance, their impact, every emergent in game event, and every intrinsic value is represented by the win loss ratio, especially over a large sample size.

Yep. Personally, I word this as: Over many games, your teammates will average out, your opponents will average out, the only person that won't is you. Therefore if you are above average, you will have >1 WLR, and if you are below, you will have <1 WLR.

That been said, a good match maker would ensure you're only slightly above or below 1 WLR, and a bad match maker will make you soar like a bird or sink like a stone.

[Horseman]

The problem with WLR based matchmaking is simple: speed by which it converges and accuracy (because as discussed it's fairly noisy). MWO has too many players who don't play enough games to get accurately positioned by such a system, and players who are simply lucky to get teamed up with better players would increase in ranking for no contribution of their own. Matchscore based matchmaking tries to use your individual contribution to the match as a base, which means that players who stand out will shift substantially higher up/down than the middle of the pack guys whose change will be minuscule if any.

[Nightbird]

Horseman, on 12 April 2022 - 09:57 PM, said:

The problem with WLR based matchmaking is simple: speed by which it converges and accuracy (because as discussed it's fairly noisy). MWO has too many players who don't play enough games to get accurately positioned by such a system, and players who are simply lucky to get teamed up with better players would increase in ranking for no contribution of their own. Matchscore based matchmaking tries to use your individual contribution to the match as a base, which means that players who stand out will shift substantially higher up/down than the middle of the pack guys whose change will be minuscule if any.

Nothing you said is true, as most statements not backed up by math tend to be. In actuality, there is no time period you can pick where the current match maker is not vastly inferior to WLR.

https://mwomercs.com...th-in-business/

Jay Z's PSR after settling/converging for 2 years will perform worse than WLR match maker given just 2 weeks, and that's how bad it is.

In terms of accuracy, Jay Z's PSR has none to speak of. It may be good enough to divide the pop into two halves, but it doesn't have the resolution to arrange the two teams after you get 24 players together. It's basically blind, vision 1/20.

The bottom line is WLR would outperform Jay Z's PSR today with just 1 week of settling in. After 1 month, it'll be 2x as good.

Edited by Nightbird, 12 April 2022 - 11:59 PM.

[Horseman]

You're making assumptions about player activity. The average player runs ~100 matches a month, but some do a lot more - and many a lot less. Taking myself as an example, there are months where I've played less than fifty matches on my main and barely two dozen on my alt. Are you saying that your system would be able to place me accurately after twelve to twenty-five matches? How much variance in player performance does your simulation account for, if it does at all?

And of course, with your system an utterly crap player who consistently groups with 2-3 actually competent people would rise up and up despite the fact that in practice they're carried by their teammates and contribute little to nothing to the match.

Edited by Horseman, 13 April 2022 - 12:10 AM.

[Nightbird]

Horseman, on 13 April 2022 - 12:09 AM, said:

You're making assumptions about player activity. The average player runs ~100 matches a month, but some do a lot more - and many a lot less. Taking myself as an example, there are months where I've played less than fifty matches on my main and barely two dozen on my alt. Are you saying that your system would be able to place me accurately after twelve to twenty-five matches? How much variance in player performance does your simulation account for, if it does at all?

And of course, with your system an utterly crap player who consistently groups with 2-3 actually competent people would rise up and up despite the fact that in practice they're carried by their teammates and contribute little to nothing to the match.

1. It only takes 80 matches average across the population for WLR to be better than Jay Z's PSR today per the link I provided. I don't have a cut-off at 80 matches but it's somewhere between 60 and 120. That's less than 1 month.

2. If you create an alt account, after 8 matches it will place you in T1 assuming new players are seeded something like 4 wins 6 losses and as a result you carry your team with your experience 6/8 times. The odds of true newbie being carried up like this is <14%, and he would bounce down soon enough.

3. If you exclusively group an unskilled player with skilled players, you can lift him higher than he deserves, but Jay Z's PSR can be gamed as well. You can do team damage within a group, or use weapons such as flamers/narcs which allows a skilled player maintain Tier 5 to lower a group's average PSR and game the match maker.

In the end, WLR is 76/100 in quality, and Jay Z's PSR is 26/100, making WLR 3x better. If you want a method that is 100/100 in quality, pay me or another statistician 200k USD. You get what you pay for, 3x a brown pile is still a brown pile.

Edited by Nightbird, 13 April 2022 - 12:43 AM.

[feeWAIVER]

Question,
Do you think emphasizing W/L over match score will change player behavior?
Specifically Assault Game Mode being played for cap more than kills?

I ask because honestly that last assault event created some really fun games when both teams were on the same page about what the objective was.

[DaZur]

Knownswift, on 12 April 2022 - 01:32 PM, said:

Whether or not it is a team game is completely irrelevant.

If you perform well you win more than you lose, and this is shown by the public data EVERY registered user of this game can access, in team sportsball, auto racing, or any other activity that revolves around competition.

A player's performance, their impact, every emergent in game event, and every intrinsic value is represented by the win loss ratio, especially over a large sample size.
Your assumptions are absurd, your cherry picking is weak. That isn't even an original strawman.

Our discussion is essentially useless if you truly believe the peripheral impact of the team in a team game is irrelevant...

Of course an individuals quality of play contributes to the probability of a win. That said, the composite quality of a teams play has a much stronger correlation to that same win.

This is by definition correlation =/= causation.

Edited by DaZur, 13 April 2022 - 05:51 AM.