248 replies to this topic

[Nightbird]

PhoenixFire55, on 10 June 2019 - 03:34 AM, said:

Please do, because they are.

I wasn't going to explain the strengths and flaws of each to you, luckily here is a brain dead simple article that explains it
https://towardsdatas...ss-d46ee57c1314

[PhoenixFire55]

Nightbird, on 10 June 2019 - 03:41 AM, said:

I wasn't going to explain the strengths and flaws of each to you, luckily here is a brain dead simple article that explains it
https://towardsdatas...ss-d46ee57c1314

Heh ... explains it and comes to a conclusion that Elo and W/L are basically the exact same given you optimize the parameters of both. Which is kinda expected since both models are dependant solely on wins and losses as the only factor determining the rating used in MM. But hey, as I've said, don't mind me, if you want to think otherwise then be my guest.

[Nightbird]

PhoenixFire55, on 10 June 2019 - 03:52 AM, said:

Heh ... explains it and comes to a conclusion that Elo and W/L are basically the exact same given you optimize the parameters of both. Which is kinda expected since both models are dependant solely on wins and losses as the only factor determining the rating used in MM. But hey, as I've said, don't mind me, if you want to think otherwise then be my guest.

Ok, I'll try to find a simpler article for you but I probably won't succeed.

[PhoenixFire55]

Nightbird, on 10 June 2019 - 04:02 AM, said:

Ok, I'll try to find a simpler article for you but I probably won't succeed.

And here is the problem ... You think yourself really smart and that you can educate people in this glorious grand thread of yours. While in fact you apparently can't, coz you are not. Otherwise you will be arguing points people make in your thread, instead of telling them to stop posting, doing childish name calling and providing us with the first link from google search on "difference between Elo and Win/Loss". But hey, your thread's funeral.

[Nightbird]

PhoenixFire55, on 10 June 2019 - 04:28 AM, said:

And here is the problem ... You think yourself really smart and that you can educate people in this glorious grand thread of yours. While in fact you apparently can't, coz you are not. Otherwise you will be arguing points people make in your thread, instead of telling them to stop posting, doing childish name calling and providing us with the first link from google search on "difference between Elo and Win/Loss". But hey, your thread's funeral.

I knew the difference, didn't want to type it out. Don't complain when you obviously haven't read to the conclusion of that article.

It's also not my policy to be nice to someone who starts out rude.

Edited by Nightbird, 10 June 2019 - 04:41 AM.

[PhoenixFire55]

Nightbird, on 10 June 2019 - 04:37 AM, said:

I knew the difference, didn't want to type it out. Don't complain when you obviously haven't read to the conclusion of that article.

LOL? ... Complain? Hmm, I'm really wondering which part of what I've said is "complaining".

Its kinda funny tho. You claim that you knew the difference, yet instead of typing it out you've just googled the first random article close to the subject. Somehow that doesn't really correlate with you typing out your first long and supposdely highly educational post.

So ... once again, avoiding arguing the point.

Nightbird, on 10 June 2019 - 04:37 AM, said:

It's also not my policy to be nice to someone who starts out rude.

It seems we have a very rude forum audience coz apparently half the people repliying to this thread started out rude.

[General Solo]

I think currently win or loss is over emphasised alreadly

Different PSR change conditions for win and if u lose. aka U cant drop PSR on a loss win :doh edit
PSR is supposed to be PSR Personal skill rating, Winning and losing is more of a team effort rating, more but not totally.
Guys who get carried on a win get to T1, horcrux of the problem in my view




Win whilst afk and increase PSR personal skill rating coz ur TEAM WON.
So wot is PSR exactly?

Edited by OZHomerOZ, 10 June 2019 - 06:13 PM.

[Nightbird]

PhoenixFire55, on 10 June 2019 - 04:51 AM, said:

It seems we have a very rude forum audience coz apparently half the people repliying to this thread started out rude.

Yep, it's the internet, it's full of people who make nebulous claims and try to pass it off as fact.

For my simulation, I offered everything needed to reproduce its results: the initial seeding of players, the randomization method into teams, the formula I used to calculate win chance and stomp chance, and how the final result is determined. Anyone with any understanding of the topic can criticize a specific point to which I can reply.

As for you, I'm glad I farmed so many bumps from you. You may leave, unless you'd like to bump some more.

OZHomerOZ, on 10 June 2019 - 05:10 AM, said:

So wot is PSR exactly?

PSR is a measure of your experience, the total number of matches you've played. It is not a indication of skill. Win or loss does not matter to PSR.

[K O Z A K]

This thread was a fun read. So many posts having absolutely no point at all, just various versions of "you're right but it doesnt matter", "you're right but I disagree", "I dont understand all tha stuff you wrote, but I disagree"

[Peter2k]

Nightbird, on 10 June 2019 - 05:17 AM, said:

Yep, it's the internet, it's full of people who make nebulous claims and try to pass it off as fact.

For my simulation, I offered everything needed to reproduce its results: the initial seeding of players, the randomization method into teams, the formula I used to calculate win chance and stomp chance, and how the final result is determined. Anyone with any understanding of the topic can criticize a specific point to which I can reply.

As for you, I'm glad I farmed so many bumps from you. You may leave, unless you'd like to bump some more.



PSR is a measure of your experience, the total number of matches you've played. It is not a indication of skill. Win or loss does not matter to PSR.

And yet it still weighs a loss or win into your gain or loss (or rather stay, as loss in PSR is hard to do)

Edited by Peter2k, 10 June 2019 - 11:12 AM.

[Peter2k]

Btw

I think I would already be happy if the MM can do the following:

Sort:
- newbs evenly among both teams
- slow assaults evenly among teams
- ecm mechs evenly among teams
- Lurm Mechs evenly among teams

When I say evenly, of course as good as possible.

That way I wouldn't be stuck with 3 timid 100t mechs with no ecm support, on Alpine or Polar, against a team with 3 or 4 Lurm boats and 3 ecm mechs of which 1 which Narcs.

Not saying it's perfect, but given PGI's resources I hope that should be more achievable then convincing them to redo the tier system.

Edited by Peter2k, 10 June 2019 - 11:19 AM.

[Nightbird]

Peter2k, on 10 June 2019 - 11:11 AM, said:

And yet it still weighs a loss or win into your gain or loss (or rather stay, as loss in PSR is hard to do)

It's basically a broken pedometer, sometimes it goes up by 2 per step, sometimes 0 per step, but on average it tracks how far you walked rather than how skillfully you walked.

[Peter2k]

Nightbird, on 10 June 2019 - 11:18 AM, said:

It's basically a broken pedometer, sometimes it goes up by 2 per step, sometimes 0 per step, but on average it tracks how far you walked rather than how skillfully you walked.

Did not say it's useful or actually right to include the win condition.
Also I'm aware it's a XP bar.

Just baffled how you can call that a MM at all.

[Nightbird]

Peter2k, on 10 June 2019 - 11:17 AM, said:

Sort:
- newbs evenly among both teams
- slow assaults evenly among teams
- ecm mechs evenly among teams
- Lurm Mechs evenly among teams

Dividing 24 people into what is at best 16 buckets to then be split into 2 each will be challenging task for sure

[Feral Clown]

I was kinda shocked that average match score wasn't the main thing that a tweaked matchmaker should be based on. It makes sense though considering a vet such as myself can average over 300 match score and not necessarily influence the outcome of a game and why I don't reach a wlr of 2.0.

This is speaking in quick play however since in group queue and CW I do much better and contribute a good deal more. Ashamed to say it but I am guessing in part it has a lot to do with my approach and attitude in quick play and my focus on one metric (doing a minimum 500 damage) and my disregard and almost hostile attitude in quick play (not communicating targets, ignoring caps almost completely despite mostly playing fast mediums or lights a good deal of the time).

To defend myself a little I do have some positive traits in quick play as I will often run back and help assaults (killing lights is either easy cause most players are terrible at piloting them, or a challenge when I run into the guys that are good at it), and I do get those lock on folks their locks a lot, which I benefit from as well.

My point though is that your thread and work taught me some things. Most importantly is that if I want to use the time left we have in this game productively and improving, I should shift towards not emphasizing pure damage output and consider doing/behaving more along the lines of playing with teammates and focusing on doing what it takes to win. Now I am curious if things I do all the time (share information on comms) will have an impact on my wlr in quick play.

[The6thMessenger]

OZHomerOZ, on 10 June 2019 - 05:10 AM, said:

I think currently win or loss is over emphasised alreadly

Different PSR change conditions for win and if u lose. aka U cant drop PSR on a loss win :doh edit
PSR is supposed to be PSR Personal skill rating, Winning and losing is more of a team effort rating, more but not totally.
Guys who get carried on a win get to T1, horcrux of the problem in my view


Win whilst afk and increase PSR personal skill rating coz ur TEAM WON.
So wot is PSR exactly?

Well, I agree, that seems to be a better way for PSR to work. But considering that PGI's Scoring isn't exactly Ideal, it's probably just not going to be the best either.

I mean, you could easily pad MS score, just bring AMS and neuter poor missile-boats by your kit-fox while getting that sweet-sweet MS, or during pre-rework LRM boats padding so much damage they could easily rack up PSR score -- damn you could do that right now with ATM boats.

PhoenixFire55, on 10 June 2019 - 03:34 AM, said:

Nightbird, on 10 June 2019 - 03:33 AM, said:

LOL, you think Elo and WLR are the same, OK... I'm gonna walk away now

Please do, because they are.

I think ELO is a specific type of scoring, they have specific processes in calculating those scores. WLR is merely a statistical representation relative to your wins and losses.

I see the worth of WLR for the MM, I mean that was the point of this thread. But like I've said before, hopefully it does have some sort of reset, or some form of mechanic similar to that of an ELO rating that is normalized overtime.

I get that WLR may not be completely an indication of skill, but hey it is at least a viable component in improving the quality of games. And honestly, that's all I want from an MM.

PhoenixFire55, on 10 June 2019 - 03:52 AM, said:

Heh ... explains it and comes to a conclusion that Elo and W/L are basically the exact same given you optimize the parameters of both. Which is kinda expected since both models are dependant solely on wins and losses as the only factor determining the rating used in MM.

I've read the article, and it says which is which is better at predicting games. Such as the ELO was over reacting it even becomes inaccurate at very long seasons, and lowering the K-Value to 0.5 and the Beta to 400 would make it predict better than WLR at a certain degree.

I didn't see anything that says WLR and ELO is basically the same, it just said that they could predict the win-rate using both, and it was to be used to get the Brier score. They come close in terms of predicting, but having near-similar result is a different matter when it comes to the mechanics.

It's basically saying that an atomic bomb is basically the same as an average lump of TNT, because both go boom.

Edited by The6thMessenger, 11 June 2019 - 12:10 AM.

[PhoenixFire55]

The6thMessenger, on 10 June 2019 - 11:48 PM, said:

I think ELO is a specific type of scoring, they have specific processes in calculating those scores. WLR is merely a statistical representation relative to your wins and losses.

On one hand we have pure W/L, on the other a value that is calculated based on same W/L, i.e. single exact same input parameter, thus they are essentially the same. Of course some weighting values can be tweaked back and forth to improve predictions for specific requirements, but it changes nothing.

The6thMessenger, on 10 June 2019 - 11:48 PM, said:

I see the worth of WLR for the MM, I mean that was the point of this thread. But like I've said before, hopefully it does have some sort of reset, or some form of mechanic similar to that of an ELO rating that is normalized overtime.

Again, we already had W/L as the sole input parameter of the MM. It was proven both by it failing in-game and by people on the forum that this simply doesn't work in MWO due to several things, including for example the fact that combined team W/L quite frankly have nothing to do with the sum of individual W/L's of its players.

The6thMessenger, on 10 June 2019 - 11:48 PM, said:

I get that WLR may not be completely an indication of skill, but hey it is at least a viable component in improving the quality of games. And honestly, that's all I want from an MM.

My point was that W/L is largely a product of the MM, hence it can't be used as an input parameter of MM. If W/L depends on the quality of MM and not just on skill, then you are matching by "luck" just as much as by "skill", which clearly achieves nothing and only further affects W/L towards it being the indication of MM mechanics instead of a player skill.

Besides, I think most people agree that W/L=1.0 is what a quality MM will produce for every player, and if an input MM parameter for every player is exactly or nearly the same, then what is the point of such MM? Yes, it will balance itself out by bouncing back and forth near 1.0 for everyone, but it'll do it statistically, i.e. overall a large number of matches will be balanced, but it doesn't in any way mean that each and single one of those matches will be anywhere near balanced. And that quite literally means nothing good in terms of match quality improvement.

It is an example of a MM that is at its worst, i.e. most unbalanced, near the state of player W/L distribution that it actually should produce, i.e. bad. A proper MM will actually be at its most stable near the uniform W/L~1.0 distribution across playerbase.

The6thMessenger, on 10 June 2019 - 11:48 PM, said:

I've read the article, and it says which is which is better at predicting games. Such as the ELO was over reacting it even becomes inaccurate at very long seasons., and lowering the K-Value to 0.5 and the Beta to 400 would make it predict better than WLR.

Inaccuracy in long seasons was mostly due to the limitations of his model. He even specifically talks about it closer to the end. And over/under-reacting is just a matter of choosing MM parameters, and you can achieve pretty much any result by changing those.

The6thMessenger, on 10 June 2019 - 11:48 PM, said:

I didn't see anything that says WLR and ELO is basically the same. They come close in terms of predicting, but having near-similar result is a different matter when it comes to the mechanics.

Again, input parameter is the same. Output result is the same. Anything in between becomes irrelevant. An example if you like ... (2+2)*2 = 8 = 84/21+sqrt(10+(2^5-2^3)/(3^2-5)). Exact same result calculated with very different methods. Sure, balancing is done differently for Elo vs W/L, but balancing can be done differently for any model, and given no limitations for balancing coefficients you can achieve pretty much any output.

[The6thMessenger]

PhoenixFire55, on 11 June 2019 - 12:32 AM, said:

On one hand we have pure W/L, on the other a value that is calculated based on same W/L, i.e. single exact same input parameter, thus they are essentially the same. Of course some weighting values can be tweaked back and forth to improve predictions for specific requirements, but it changes nothing.

That Atom-Bomb and TNT analogy still works.

PhoenixFire55, on 11 June 2019 - 12:32 AM, said:

My point was that W/L is largely a product of the MM, hence it can't be used as an input parameter of MM. If W/L depends on the quality of MM and not just on skill, then you are matching by "luck" just as much as by "skill", which clearly achieves nothing and only further affects W/L towards it being the indication of MM mechanics instead of a player skill.

Like I've said, I have no interest in defining "player skill", but rather the quality of matches. So if this auto-fellating system does that, I have no qualms if it's still matching by luck.

Now does this increase the quality of matches? I don't know. Maybe?

PhoenixFire55, on 11 June 2019 - 12:32 AM, said:

Besides, I think most people agree that W/L=1.0 is what a quality MM will produce for every player, and if an input MM parameter for every player is exactly or nearly the same, then what is the point of such MM? Yes, it will balance itself out by bouncing back and forth near 1.0 for everyone, but it'll do it statistically, i.e. overall a large number of matches will be balanced, but it doesn't in any way mean that each and single one of those matches will be anywhere near balanced. And that quite literally means nothing good in terms of match quality improvement.

I agree, 1 WLR is ideal, and I actually see what you mean -- that if the WLR becomes high enough, they would be in an enviroment that they would fight people with equally high WLR and would reduce the WLR back to 1, and back where they started -- it's self-stabilizing.

That being said, scores we get are ultimately derived from the MM -- the current or otherwise, it's inevitable that it's auto-fellatious.

I still agree with that thought experiment, yet I still see the worth of WLR in properly distributing matches, Nightbird did the math to prove it.

I think ELO is specifically for this, so I would wager that it's probably a good idea to separate Environments by ELO or something in the lines, but distribute players in matches by pure seasonal WLR. IDK though.

You think we could get better quality of matches if players were reduced to 16 from 24?

Edited by The6thMessenger, 11 June 2019 - 01:06 AM.

[mad kat]

Firstly really great job on the simulation. It made for some interesting reading and only highlights why there are so many threads complaining about the broken MM.

Although switching to a WLR based match maker makes me nervous as that means your individual chance of winning is entirely dependant on the other people in the teams.

In other words you can try to carry the team get five kills, do over 1000 damage, 10 components destroyed arty strikes and UAV's everywhere several saviour kills etc etc and have 11 of the enemy killed but one fresh spider (after doing 80 damage) capping your base means you've lost. Or you have an 11/11 match and the last guy is a DC (it happens!)

The match maker MUST be based around an individual players performance not that as a result of the team. Reason i say this is that i find my stomp chance is a lot higher than 1/5.

I think part of the problem is the mechs performance isn't calculated into the formula you can have an utter potato drive a meta mech like a marauder and perform the same as an elite player driving a Spider 5V. Flip this around and put a potato in a crap mech and the match outcome is already set.
Somehow (like battle value) the players skill needs to be combined with a performance rating of the mech they're in. Maybe rated more this way than the tonnage. We've all seen skilled locust pilots utterly decimate the enemy team and a direwolf pilot with 6 x ERLL or a cyclops with 7 x LRM10's hide at the back and just scavenge kills from people that have done all the hard work.

I may of misunderstood the wording however...

Edited by mad kat, 11 June 2019 - 01:30 AM.

[PhoenixFire55]

The6thMessenger, on 11 June 2019 - 12:58 AM, said:

I still agree with that thought experiment, yet I still see the worth of WLR in properly distributing matches, Nightbird did the math to prove it.

Tbh its not that hard to prove that pretty much any MM is going to be better than what we currently have, but sure. The problem is, that just like I've already mentioned, PGI and other people had similar thoughts when they were implementing Elo MM way back when. Exact same motivation, nearly exact same math that proved that Elo system will improve matches. It was implemented and yet it failed. And as I've pointed out, essnetially Elo and W/L are same since they use the same supposedly self-balancing factor - wins vs losses. So it is, lets put it politely, strange to expect W/L MM to fare any better in actual game. Because none of the problems that Elo MM had in MWO were at any point addressed by anyone, including the OP in this thread.

The6thMessenger, on 11 June 2019 - 12:58 AM, said:

You think we could get better quality of matches if players were reduced to 16 from 24?

With a properly working MM it won't matter. With a partially faulty one it will actually worsen the situation. The explanation here is simple, due to the LLN (Law of Large Numbers) a summ of many similarly distributed values (players skill in this case) is close to the normal distribution. In turn it means that the total skill of players you pick for the team will be very close to the middle of that normal distribution with high probability even if you pick players at random. The more players per team there are, the better it works and the closer to the expected value team skill is going to be. Even if MM does nothing and just picks people randomly, teams will be matched closer and closer with the increasing number of player per team. And visa versa, if you decrease the amount of players on a team this natural balancing mechanism suffers.

P.S. Also please stop typing ELO, its not an abbrevation, its named after a person - Arpad Elo.