Jump to content

Why Is Win/loss Ratio The Definition Of A Good Matchmaker?


42 replies to this topic

#21 crazytimes

    Member

  • PipPipPipPipPipPipPipPip
  • Ace Of Spades
  • 1,356 posts

Posted 18 November 2022 - 06:18 PM

View PostNightbird, on 18 November 2022 - 06:07 PM, said:


You just have to ensure there is the same number of those +4 W/L people are on each team. MWO's match maker will happily put 8 of them on one team, 0 on the other, because there are 8 "T1" people with 1.0 W/L to put on the other team lol.


They can't even factor weight into MM anymore and are still struggling to balance groups. Now you want to balance by W/L?

I'm just happy if I can get into matches without overly long wait times. Team composition balances itself out in the long run after that, I don't feel vicitimised because I sometimes drop against 99.9 percentile 4 manners. Sometimes they're on my team instead.

#22 VeeOt Dragon

    Member

  • PipPipPipPipPipPipPipPip
  • Survivor
  • Survivor
  • 1,212 posts
  • LocationHell

Posted 18 November 2022 - 06:41 PM

there are also roles that lead to victory but do not lend themselves to getting kills themselves. take the light scout/spotter mech. it runs around perhaps doing a bit of damage here and there but mostly it is using TAG/Narc to gain targets for LRM or capping points. they might snag a kill now and again but their primary role is to help the rest of the team do well.

you also have those few mechs that can carry more than 2 AMS (i think there are two if i remember right, the Corsair 7A (4 AMS) and a clan light that can carry 3 AMS, there may be more but i can't remember off hand.) yeah they can dish out damage but they also cut down the damage the other team can do to you with missile weapons.

the glory of MWO is that there are so many options for how to play it. i think the best indicator of a good MM is not so much the W/L but the quality of the matches. if you have say a dozen matches and all of them are stomps one way or the other but you W/L is 50/50 then the MM isn't doing its job right. now if you play those same dozen matches and lose say 9 of them but all the matches were close 10-12 or better then i would say the MM was doing its job right. there are many factors that are completely out of your control in a match seeing as it is a multiplayer game that your own performance can be fantastic but your team does **** like scatter to the winds or rotate into oblivion.

#23 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 18 November 2022 - 07:14 PM

View Postcrazytimes, on 18 November 2022 - 06:18 PM, said:


They can't even factor weight into MM anymore and are still struggling to balance groups. Now you want to balance by W/L?

I'm just happy if I can get into matches without overly long wait times. Team composition balances itself out in the long run after that, I don't feel vicitimised because I sometimes drop against 99.9 percentile 4 manners. Sometimes they're on my team instead.


Why not move those 1 WL T1 people to T3 where they belong, and then the current MM will work?

The problem with the current MM doesn't work is that people are in wrong tiers.

>2 WLR = Tier 1, >1.3 WLR = Tier 2, etc

#24 VeeOt Dragon

    Member

  • PipPipPipPipPipPipPipPip
  • Survivor
  • Survivor
  • 1,212 posts
  • LocationHell

Posted 18 November 2022 - 08:04 PM

well if the match maker worked and we had the population then every tier would be 1 W/L, that is if we are using W/L as the metric. if you are playing against players of equal skill you W/L should even out to 1 or perhaps a little over as you have all those matches that got you there to go on.

again W/L isn't the metric we should be using but more quality of said of said matches. the fewer stomps either way the better the experience and the better the MM.

Edited by VeeOt Dragon, 18 November 2022 - 08:04 PM.


#25 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 19 November 2022 - 06:25 AM

View PostVeeOt Dragon, on 18 November 2022 - 08:04 PM, said:

well if the match maker worked and we had the population then every tier would be 1 W/L, that is if we are using W/L as the metric. if you are playing against players of equal skill you W/L should even out to 1 or perhaps a little over as you have all those matches that got you there to go on.

again W/L isn't the metric we should be using but more quality of said of said matches. the fewer stomps either way the better the experience and the better the MM.


A good MM is good because it makes the best quality matches possible with the population available at the time. A bad MM is bad because it doesn't create the best matches possible from the people in the queue. The population itself doesn't matter.

#26 Gagis

    Member

  • PipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 1,731 posts

Posted 19 November 2022 - 10:12 AM

View PostDing Toast is Ready, on 19 November 2022 - 09:56 AM, said:

Is WLR the best predictor of individual player skill/performance (as opposed to, say, KDR or MS average)?

Yes, once they have played long enough that their w/l record is based on games played at their actual skill level.

#27 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 19 November 2022 - 10:16 AM

View PostDing Toast is Ready, on 19 November 2022 - 09:56 AM, said:

Is WLR the best predictor of individual player skill/performance (as opposed to, say, KDR or MS average)?


Yes, your past WLR includes everything you do that helps your team, from kills to scouting to tanking damage to distracting... etc. Using WLR instead of avgMS would move the quality slider enormously.

KDR is not as good as Kills per Match. Someone who averages 1 enemy kill per match and survives 50% of matches helps his team much less than some who kills 2 per match but always dies. (Aggressive playing, distracting, tanking damage, versus passive playing)

I compared how good past WL, past MS, past KPM are for predicting future WL in this thread: https://mwomercs.com...and-suggestion/

#28 VeeOt Dragon

    Member

  • PipPipPipPipPipPipPipPip
  • Survivor
  • Survivor
  • 1,212 posts
  • LocationHell

Posted 19 November 2022 - 02:21 PM

View PostDing Toast is Ready, on 19 November 2022 - 09:56 AM, said:

Is WLR the best predictor of individual player skill/performance (as opposed to, say, KDR or MS average)?


no its not. think about it, you are on a team with 11 other players. you can be one of the best players in the game but if the MM sets you up with a bunch of nimrods then you are still going to lose. i have had games where i do 1000+ damage get 3 or more kills/KMDD (more if you count the KMDDs as i seem to get more of those than actual kills) and my team still loses. my personal performance was fantastic but we still lost. now this is not my usual showing as i am average or perhaps a bit above (high tier 3) but you still get my point. W/L is not just dependent on the individual but on the team. you can have a whole team of good players against slightly less skilled players but if the good players scatter to the winds or just rotate endlessly while the less individually skilled players actually communicate and use tactics the more skilled players are likely to lose.

TLDR: no as its a team game and who you get matched with is a huge factor. not to mention a Low population and a MM that is all screwy

#29 Gagis

    Member

  • PipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 1,731 posts

Posted 19 November 2022 - 04:09 PM

View PostVeeOt Dragon, on 19 November 2022 - 02:21 PM, said:


no its not. think about it, you are on a team with 11 other players. you can be one of the best players in the game but if the MM sets you up with a bunch of nimrods then you are still going to lose. i have had games where i do 1000+ damage get 3 or more kills/KMDD (more if you count the KMDDs as i seem to get more of those than actual kills) and my team still loses. my personal performance was fantastic but we still lost. now this is not my usual showing as i am average or perhaps a bit above (high tier 3) but you still get my point. W/L is not just dependent on the individual but on the team. you can have a whole team of good players against slightly less skilled players but if the good players scatter to the winds or just rotate endlessly while the less individually skilled players actually communicate and use tactics the more skilled players are likely to lose.

TLDR: no as its a team game and who you get matched with is a huge factor. not to mention a Low population and a MM that is all screwy

You fundamentally misunderstand how statistics works.

What team you get is the major random component of MWO matches. The random part is the same for all players and equally likely to put you in the weak team as to the strong team. This means the random component cancels out when you average the statistics. Games with large luck based components can be statistically analyzed just fine.

There are games with even larger luck based components than MWO, such as Magic the Gathering, where w/l ratios of players are extremely consistent and predict their tournament performance accurately.

#30 PocketYoda

    Member

  • PipPipPipPipPipPipPipPipPip
  • Shredder
  • Shredder
  • 4,138 posts
  • LocationAustralia

Posted 20 November 2022 - 06:13 AM

View PostfeeWAIVER, on 17 November 2022 - 01:00 PM, said:

You can kill someone with 1pt of damage.
A kill is meaningless.


It should be KMDDs not kills.

#31 VeeOt Dragon

    Member

  • PipPipPipPipPipPipPipPip
  • Survivor
  • Survivor
  • 1,212 posts
  • LocationHell

Posted 20 November 2022 - 06:51 AM

View PostGagis, on 19 November 2022 - 04:09 PM, said:

You fundamentally misunderstand how statistics works.

What team you get is the major random component of MWO matches. The random part is the same for all players and equally likely to put you in the weak team as to the strong team. This means the random component cancels out when you average the statistics. Games with large luck based components can be statistically analyzed just fine.

There are games with even larger luck based components than MWO, such as Magic the Gathering, where w/l ratios of players are extremely consistent and predict their tournament performance accurately.


i do know statistics and probability, hell i did one of my highschool science fair projects on Luck (mind you that was many many moons ago). keep in mind my sample size was small (100 people). i had each person role a pair of dice (standard 6 siders) 300 times breaking each person's rolls into 100, 200 and 300 data blocks (i had concluded before the experiment that after about 300 rolls everyone would have the same spread, i was wrong). in any statistic there are outliers. my experiment was geared as a game where the higher the roll the better (i think the person who got the highest average roll overall got a candy bar or something stupid like that, it doesn't matter). i found that after 300 rolls each very few of those 100 people had the same exact array of numbers rolled. there were indeed those that rolled low more consistently and those that rolled high. (to take any kind of rolling method out of the equation i used a hand made dice tower that you dropped the dice into and everyone dropped the dice in at the same orientation, this was before portable computers were as common as they are now so i didn't have access to something like say a Random number generator). the one thing that turned out as i expected was the more rolls one made the closer the differences in the arrays were but it never became exactly the same, there were always those that deviated considerably from the expected average.

now to take this into the perspective of the discussion you can conclude that there will be people who get the luck of the draw on the MM more often than others. also factor in groups that throw a wrench into any such equation and you have have my point. now for those people that play constantly (say playing every day) the numbers might even out more but it will never be the same. now for those people who play say a dozen or so games a day but only on say the weekend it is far more probably that you will get more outliers. (hell i have had entire days where all i get are teams that nascar or scatter with the inevitable constant losses no matter how well i play individually. but rarely do i get win streaks (though honestly this could be the mental bias of one remembering negative results more readily than positive))

now the current system the MM uses if i remember right to determine what tier you are in is based at its core on a scale that is based on how well you did in the match (using MS) compared to the other players on the team (mind you this in itself is flawed as in the current system damage and kills far outweigh more tactical contributions like scouting, TAG, objective capture, AMS and other non damage factors). the MM then takes where you are on this scale (going up if your performance is better than average on your team going down if you are worse) and then trying to place you in a group of as closely ranked players as possible. now what the game does not factor in is your mech selection (hence matches where one team has 5 assaults and the other has zero) and it doesn't seem to know how to properly deal with groups in QP. now with the low population the MM has to reach outside your Tier to fill a match sometimes considerably. now if we didn't have groups it would try to put an equal number from each tier on each team with as close a distribution of mech classes as possible but with groups this gets tossed out the window as the MM doesn't to calculate groups properly.

so the end goal of a good MM is to put people up against equally skilled players with equal team composition (as best it can since there is no way to factor builds into the mix). the end goal being as many "enjoyable" matches as possible. the trouble with using W/L as your main factor in determining a "good" match is in itself flawed as you can have a loss that was still a fun match and a win that is boring as hell. a match that ends in a 0-12 stomp isn't all that fun for either team really (unless you are one of those that enjoys seal clubbing) where an 11-12 match even if you are on the losing side is more often than not a fun match. ok its gonna suck for those first couple of people killed no matter what but someone has to die first.

EDIT: i forgot to even mention your own build selection and how it works on the map that ends up selected (say picking your LRM mech only to get Solaris, or a brawler and getting say Polar (though Polar is much better in that regard than it once was). there are certain maps that come up more than others but you can never predict what maps you can pick from when you select your mech (this is one of the reason i prefer mid-range builds and back-up weapons on LRM boats but thats for another conversation)

Edited by VeeOt Dragon, 20 November 2022 - 07:04 AM.


#32 Gagis

    Member

  • PipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 1,731 posts

Posted 20 November 2022 - 07:07 AM

Soooo. the argument against statistical measures is the superstitious notion that luck is magic?

Who gives people luck? Gods? Spirits? Demons? Virtue in past lives? Perhaps we can sacrifice a lamb to make the spirits be fair when applying luck to MWO players. Or perhaps we need to track peoples win/loss statistics to know who are the lucky ones and pair them against tougher opponents.

Edited by Gagis, 20 November 2022 - 07:09 AM.


#33 VeeOt Dragon

    Member

  • PipPipPipPipPipPipPipPip
  • Survivor
  • Survivor
  • 1,212 posts
  • LocationHell

Posted 20 November 2022 - 08:03 AM

belief or non-belief in luck aside i stand by W/L not being the determining factor for a if a MM is good or not

lets go as simple as possible which of these two would determine if a MM is working properly best (going to use some relatively low numbers for simplicity's sake)

play 10 matches in a day with a 50/50 W/L but every game is a stomp one way or the other

or

play the same 10 matches being on the losing side say 7 out of 10 matches but each match was 11-12 or 12-11 (or vice versa i just went with losing because everyone likes to win)

honestly in an ideal world we would have both a 50/50 W/L and each match being close. both factors should be considered in a MM's defectiveness but i personally see quality of match being greater than straight W/L. i personally don't mind losing if i had fun. then again i am not an overly competitive sort and just want to relax with some stompy robot fun. (i would go back to playing more MW5 honestly if it wasn't for my go to mod still not working since the last DLC)

Edited by VeeOt Dragon, 20 November 2022 - 08:04 AM.


#34 Gagis

    Member

  • PipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 1,731 posts

Posted 20 November 2022 - 08:22 AM

Matches being consistently close is mostly a sign of a low average skill level of all players in a match. A lot of people need to fail to seize a large number of opportunities to crush their opponents for that to happen often. There's usually a lot of opportunities to take decisive victory in a match if there are players actively looking for those opportunities. The higher the average skill level in the game, or in a tier, the fewer 12-11 crutches you will see.

#35 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 20 November 2022 - 08:49 AM

View PostGagis, on 20 November 2022 - 08:22 AM, said:

Matches being consistently close is mostly a sign of a low average skill level of all players in a match. A lot of people need to fail to seize a large number of opportunities to crush their opponents for that to happen often. There's usually a lot of opportunities to take decisive victory in a match if there are players actively looking for those opportunities. The higher the average skill level in the game, or in a tier, the fewer 12-11 crutches you will see.


Or, the people in T1 are a mix of people who should be in T1, T2, T3, but were all shoeboxed into T1 because of avgMS.

Edited by Nightbird, 20 November 2022 - 09:30 AM.


#36 Gagis

    Member

  • PipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 1,731 posts

Posted 20 November 2022 - 11:12 AM

View PostDing Toast is Ready, on 20 November 2022 - 10:20 AM, said:

Wouldn't match length be a factor here? An 11-12 that dragged on for a while would be indicative of the condition you describe, but a quick frantic match with the same or similar might not tell the same story (though I imagine it's much rarer)

Hmm, yeah. Thats a reasonable point.

#37 Tarl Cabot

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Tai-sho
  • Tai-sho
  • 7,748 posts
  • LocationImperial City, Luthien - Draconis Combine

Posted 21 November 2022 - 05:44 PM

View PostNightbird, on 20 November 2022 - 08:49 AM, said:


Or, the people in T1 are a mix of people who should be in T1, T2, T3, but were all shoeboxed into T1 because of avgMS.


At least tis no longer every tier and their dog moving into Tier 1, and reaching Tier 1 by brute force with the number of games played !!!



For the newer players, after PGI changed the MM from utilizing an Elo setup to Tier/PSR, belong was how Tier/PSR was setup with a flat number. A player today with a 171 MS would definitely be in Tier 5, but that same player had been able to reach tier 2 and Tier 1 simply due to brute force, having played over 25K games and with a 0.78 W/L ratio....essentially it had been nearly impossible to drop Tiers back then for most players.

https://mwomercs.com...anges-jun-2020/

Quote

Player LOSES:
Match Score: 0-100 goes down in PSR by -2
Match Score: 101-250 goes down in PSR by -1
Match Score: 251-400 does not move.
Match Score: 401+ goes up in PSR by +1

Player WINS:
Match Score: 0-100 does not move.
Match Score: 101-250 goes up in PSR by +1
Match Score: 251-400 goes up in PSR by +3
Match Score: 401+ goes up in PSR by +5


Could the MM be improved on? Definitely... but even if PGI could muster up the effort and dev/engineer(s), I do not have faith that it would be close to what would be expected... We have all been there and seen that what happens across the board for actual items that are more than simple xml changes... Posted Image

Edited by Tarl Cabot, 21 November 2022 - 05:45 PM.


#38 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 21 November 2022 - 08:13 PM

MS is a very imperfect indicator of skill, which is why having it drive the movement of tiers results in the terrible MM we have today.

#39 Gagis

    Member

  • PipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 1,731 posts

Posted 21 November 2022 - 09:44 PM

The Elo rating based system honestly was pretty good. Perhaps tuned a bit strict in how long it made some top tier players wait for an equal opponent, but thats tunable.

#40 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 21 November 2022 - 10:31 PM

View PostGagis, on 21 November 2022 - 09:44 PM, said:

The Elo rating based system honestly was pretty good. Perhaps tuned a bit strict in how long it made some top tier players wait for an equal opponent, but thats tunable.


If you are talking about MWO's first MM using the Elo system, they messed it up. Elo is supposed to stabilize, but the formula MWO was using was incorrect and sent good players to infinity, and bad players to minus infinity.





2 user(s) are reading this topic

0 members, 2 guests, 0 anonymous users