713 replies to this topic

[Zanotam]

The goal is winning the game, right? Or at least killing all the enemy while staying alive in your own stompy robot. And that wins the game coincidentally. So to maximize overall fun in the long run just maximize the effect of fun things - e.g. winning. WIn/Loss should be the only thing that matters - I can matchscore farm ezpz in my Catapult for example but if I wanted just to win I would probably spam my Dragon Flame as I am terrible at this game but I'm still better at it when I play my stupid brawly LBX20 boi than when I play basically any other mech I'm pretty sure. In the end, the variable that most correlates with your chance to win your 10,000th game is roughly your last 9,999 games maybe to keep the stat ideal you can even just use your last 500-100games although if you assume all games are found using matchmaking looking for balanced games than you might only need the last 100-200 matches to give you an idea of the player's 'win velocity' so to speak e.g. a climbing player should go up against tougher opponents and someone who is on tilt and has a 'negative' velocity of say 40% wins over the last 100 games but say 49% over the last 1000 games or whatever should be going down against weaker opponents most likely.

[LordNothing]

Andrzej Lechrenski, on 10 June 2020 - 05:15 PM, said:

I don't know if I agree that is how standard deviations work. Using standard deviations also won't help very much with such a low population, as it essentially means that either the extremes will never get matches, or else the MM will be loosened so much that it was a wasted effort to begin with. Might as well just go with a percentage based system where top W/L is 99.9999+% and bottom is 0.0001% and just let them loosen valves from there.

Considering using only the most recent hundred or so matches is a fine idea, though.

i kind of just skimmed the article on wikipedia, so im probably misusing terminology. ive made posts about it elsewhere, but your average players are going to make up about 68% of the total population, and this would be t3. t1 and t5 are exclusive clubs in comparison less than 3% of the population in each. t2 and 4 would be around 13%. i think nightbird is a professional statistician or something so he might be able to clarify.

also i dont think there will be pure tier matchmaking and teams are always going to be mixed tiers, just because the total population is so low. mixed teams is also one of the reasons i prefer to use ms.

even if w/l is used i would most definately want it limited to the last 100 or so games, as players do change performance over time. i peaked a very long time ago. i figure this was why i was in t3 so long before shooting to t1 like a rocket when psr was introduced.

Edited by LordNothing, 10 June 2020 - 05:38 PM.

[Nightbird]

VerKa, on 10 June 2020 - 05:25 PM, said:

I'll probably get flamed for this. Just take your original patch you were going to implement on Tuesday and install it. Run for a week, tweak it . Run it another week , do the same. You have to start somewhere. Thanks.

After the PSR reset it will take months of horrible match making before the MM recovers to how it is today. A weekly reset will do so much damage that you might as well turn the MM off and make teams completely randomly.

[Andrzej Lechrenski]

LordNothing, on 10 June 2020 - 05:29 PM, said:

i kind of just skimmed the article on wikipedia, so im probably misusing terminology. ive made posts about it elsewhere, but your average players are going to make up about 68% of the total population, and this would be t3. t1 and t5 are exclusive clubs in comparison less than 3% of the population in each.

Absolutely true. But MM still needs to be able to find matches for the top and bottom percentages. If we had hundreds of thousands of players, that might not be so hard. But with as few players as we have, we would end up with those people sitting and waiting for unacceptably long periods of time in order to get a match if we are trying to get same-Tier against same-Tier. As a result, MM would loosen the constraints more and more until the effort of sorting a realistic bell curve is wasted.

Of course, some sort of sorting is needed, but it may as well be linear to cut steps.

Just to be clear here, I'm essentially advocating against "Tiers" entirely for the purpose of match making. However, if we absolutely *must* apply labels to people, and those labels don't actually mean anything to the MM, then what you are saying is fine.

Edited by Andrzej Lechrenski, 10 June 2020 - 05:41 PM.

[LordNothing]

VerKa, on 10 June 2020 - 05:25 PM, said:

I'll probably get flamed for this. Just take your original patch you were going to implement on Tuesday and install it. Run for a week, tweak it . Run it another week , do the same. You have to start somewhere. Thanks.

yea no mater how tiers are decided, zero sum is probibly the most important thing that needs to be done. were essentially debating the fine details.

Andrzej Lechrenski, on 10 June 2020 - 05:38 PM, said:

Absolutely true. But MM still needs to be able to find matches for the top and bottom percentages. If we had hundreds of thousands of players, that might not be so hard. But with as few players as we have, we would end up with those people sitting and waiting for unacceptably long periods of time in order to get a match if we are trying to get same-Tier against same-Tier. As a result, MM would loosen the constraints more and more until the effort of sorting a realistic bell curve is wasted.

Of course, some sort of sorting is needed, but it may as well be linear to cut steps.

Just to be clear here, I'm essentially advocating against "Tiers" entirely for the purpose of match making. However, if we absolutely *must* apply labels to people, and those labels don't actually mean anything to the MM, then what you are saying is fine.

it might just be a situation where if you have a t1, you might also have to include a t5 to balance it out. the mm wants the team average to be 3 (or whatever is the average of all players in the queue).

Edited by LordNothing, 10 June 2020 - 05:46 PM.

[Nightbird]

LordNothing, on 10 June 2020 - 05:43 PM, said:

yea no mater how tiers are decided, zero sum is probibly the most important thing that needs to be done. were essentially debating the fine details.

zero sum does nothing in simulations, best players are scattered T1 thru T3, average players are scattered through T1 to T5, bad players scattered T3 thru T5, with no systemic additional information available to MM to make better quality matches

scientific equivalent of throwing an egg against a wall, you get a mess that is skewed a little more towards T5 than T1.

Edited by Nightbird, 10 June 2020 - 06:00 PM.

[Andrzej Lechrenski]

LordNothing, on 10 June 2020 - 05:43 PM, said:

yea no mater how tiers are decided, zero sum is probibly the most important thing that needs to be done. were essentially debating the fine details.



it might just be a situation where if you have a t1, you might also have to include a t5 to balance it out. the mm wants the team average to be 3 (or whatever is the average of all players in the queue).

I think it might be best if, whatever we do, we make sure that T1 and T5 never mix, even in equal ratios.

[ESC 907]

How about a poll? Post suggestions here, compile suggestions into another post for reference (maybe even lock it to prevent comments?), put suggestions into an online poll, and then have community-members vote on the poll?

[Andrzej Lechrenski]

Nightbird, on 10 June 2020 - 05:54 PM, said:

zero sum does nothing in simulations

W/L ratio is just another kind of zero sum (well, zero multiplicative, but whatever).

[Nightbird]

Andrzej Lechrenski, on 10 June 2020 - 06:00 PM, said:

W/L ratio is just another kind of zero sum (well, zero multiplicative, but whatever).

I amended that post to be more informative

ESC 907, on 10 June 2020 - 05:56 PM, said:

How about a poll? Post suggestions here, compile suggestions into another post for reference (maybe even lock it to prevent comments?), put suggestions into an online poll, and then have community-members vote on the poll?

We will go into a poll. However, the result will be the same as deciding how to perform surgery with a poll. A dead patient.

Edited by Nightbird, 10 June 2020 - 06:03 PM.

[Too Much Love]

I generally like the idea to make new PSR based the on performance ranked from 1 to 24 player in each match.

Win/loss doesn't really make any sense, because mathematically speaking your contribution to victory or defeat is 1/24, so it shouldn't be applied to judging player's performance

[Tyler Valentine]

Eatit, on 09 June 2020 - 11:29 AM, said:

Problem -

Matches are unbalanced.

Causes of Problem -

PSR is basically an XP bar. Low population MM is pulling from all tiers.

Proposed Solution -

Change the way the tiers are decided.

Outcome -

All matches will still be unbalanced due to Low population MM pulling from all tiers. Change the tiers all you want it will still be the same as the MM uses all of the tiers to create a match. The valves are wide open and will remain wide open.


Alternate Solution -

Others have posted a better solution that is to change the MM to create teams for the matches in an A-B-A-B... manner. Highest tier player gets added to team A, next highest tier player gets added to team B, and so on until you have two equally matched teams.

This will create two evenly matched teams of people from all of the tiers. Each of the teams will contain the same number of players from each of the tiers.

*edited for spelling

eatit is onit (he has a few follow-up posts on pp 11-12 that are also spot on). Everybody is getting matched with everybody else. 'fixing PSR' alone will not fix the problem of evenly matched games. We need to come up with a PSR solution and we need to fix how matchmaker assigns teams

something like:

Team A PSR Ranks: 1,4,5,8,9,12,13,16,17,20,21,24 (150 total)
Team B PSR Ranks: 2,3,6,7,10,11,14,15,18,19,22,23 (150 total)

in order to see any real change in match quality.

Edited by Tyler Valentine, 10 June 2020 - 08:21 PM.

[Nightbird]

Tyler Valentine, on 10 June 2020 - 07:41 PM, said:

eatit is onit (he has a few follow-up posts on pp 11-12 that are also spot on). Everybody is getting matched with everybody else. 'fixing PSR' alone will not fix the problem of evenly matched games. We need to come up with a PSR solution and we need to fix how matchmaker assigns teams

something like:

Team A PSR Ranks: 1,4,5,8,9,12,13,16,17,20,21,24 (150 total)
Team B PSR Ranks: 2,3,6,7,10,11,14,15,18,19,22,23 (150 total)

in order to see any real change in match quality.

How would you handle teams of 2, 3, 4?

As far as how the MM organizes players from tiers, for all solo players matches this is already how it does it. The reason it doesn't work is because past MS is a poor predictor of performance on the team. (For the people that haven't hit the T1 cap, total past MS earned is their PSR bar location today.) It doesn't go by the 5 tiers but rather by a more granular number in the background. (The EXP bar line)

This is also why all the proposals based on MS won't work. Past MS is only loosely related to odds of winning future matches, whereas past WLR is strongly related. Picked any match score on Jarl's list (not adjusted) and look up and down the list at the player's WLR. You'll see a huge variance. This is clearer when you take all the data and graph it, showing a clear pattern that MS is inferior to WLR.

Edited by Nightbird, 10 June 2020 - 08:37 PM.

[Scooter Libby]

There should be no MS penalty for dying.

Those who lead the charge, or push when it's time to push are valuable players. Those who camp, hide, and solo... less so. Match score and PSR should not incentivize the latter.

[Dauntless Blint]

Tyler Valentine, on 10 June 2020 - 07:41 PM, said:

eatit is onit (he has a few follow-up posts on pp 11-12 that are also spot on). Everybody is getting matched with everybody else. 'fixing PSR' alone will not fix the problem of evenly matched games. We need to come up with a PSR solution and we need to fix how matchmaker assigns teams.

something like:

Team A PSR Ranks: 1,4,5,8,9,12,13,16,17,20,21,24 (150 total)
Team B PSR Ranks: 2,3,6,7,10,11,14,15,18,19,22,23 (150 total)

in order to see any real change in match quality.

True but this is still a low risk place to get it right for future projects.

They need to get match performance and average chassis performance other wise the second you jump in your worst chassis verses your best chassis you break the MM for that one match. MM needs to know your effective PSR so that you don't sand bag a team in off meta builds while, carry it in your best brain synergy meta build/archetype.

Likewise if you join a match in an "obsolete" chassis highly competitive guys don't really want you there, which is fine so the chassis needs a PSR modifier to recognize some chassis are more equal then others. You could call it a CSR.(side effect becomes a tool to buff or nerf a chassis)

This way comp level guys don't get "just for fun" builds ruining their contest, and like wise guys who can start to compete in meta chassis don't crush in lower PSR tiers but can add to player pool of higher tiers.

Further more win loss becomes nearly neural when you reach your true PSR because you should be winning and loosing an equal amount of match's because the MM is putting you in your correct bracket. So you need to know if your above or below average for your chassis to move up or down per match.

In the far future the pilot should have an avatar who holds PSR, and the Chassis should hold a PSR modifier.
A Mad-Cat should have a higher PSR (CSR) modifier than a Zeus for instance.

Edited by Dauntless Blint, 10 June 2020 - 09:08 PM.

[Tyler Valentine]

Nightbird, on 10 June 2020 - 08:29 PM, said:

How would you handle teams of 2, 3, 4?

As far as how the MM organizes players from tiers, for all solo players matches this is already how it does it. The reason it doesn't work is because past MS is a poor predictor of performance on the team. (For the people that haven't hit the T1 cap, total past MS earned is their PSR bar location today.) It doesn't go by the 5 tiers but rather by a more granular number in the background. (The EXP bar line)

This is also why all the proposals based on MS won't work. Past MS is only loosely related to odds of winning future matches, whereas past WLR is strongly related. Picked any match score on Jarl's list (not adjusted) and look up and down the list at the player's WLR. You'll see a huge variance. This is clearer when you take all the data and graph it, showing a clear pattern that MS is inferior to WLR.

excellent points nightbird.

Reset PSR. I have no strong opinion of how new PSR is calculated. my gut says base it on match score but if evidence suggests W/L is a better metric than im fine with that. once PSR is calculated more accurately, 'staggered PSR rank' team building will work [more] effectively. as far as groups, they are a problem. They will have to be given a modifier with best player PSR as a starting point. the specifics of how they will be incorporated requires more thought. the main point of my post is to highlight the fact that PSR and matchmaking need to be updated in order to see effective change given the low player population

[Nightbird]

Tyler Valentine, on 10 June 2020 - 08:46 PM, said:

excellent points nightbird.

Reset PSR. I have no strong opinion of how new PSR is calculated. my gut says base it on match score but if evidence suggests W/L is a better metric than im fine with that. once PSR is calculated more accurately, 'staggered PSR rank' team building will work [more] effectively. as far as groups, they are a problem. They will have to be given a modifier with best player PSR as a starting point. the specifics of how they will be incorporated requires more thought. the main point of my post is to highlight the fact that PSR and matchmaking need to be updated in order to see effective change given the low player population

Agreed on both needing an update. I actually did a simulation with the exact Match Maker you listed, take a look at the link if you're interested. Scroll down to section "Simulation of Win-Loss Ratio (WLR) based MM"

https://mwomercs.com...thread-we-need/

[Tyler Valentine]

Nightbird, on 10 June 2020 - 08:59 PM, said:

Agreed on both needing an update. I actually did a simulation with the exact Match Maker you listed, take a look at the link if you're interested. Scroll down to section "Simulation of Win-Loss Ratio (WLR) based MM"

https://mwomercs.com...thread-we-need/

That's a really in-depth analysis... it's like im back in data science class The results don't lie, a better matchmaker leads to better matches. Hope PGI has taken a look at that.

[Scooter Libby]

gruntnflush, on 09 June 2020 - 08:00 AM, said:

People in my guild as well as those I've randomly chatted with in game were really looking forward to a new system based on win loss, as this would have served the role of encouraging people to focus more on winning than on personal stats. Doubt any of these people I've talked to even ever post on the forums at all.

I know a guy who predominately plays assaults and calls games. He sacrifices his own armor to lead pushes playing the role of tank. Even if he goes down without any kills he still is a huge factor on whether his team loses or not.

My own opinion is that Win/Loss should be a huge factor on PSR. Huge! Don't further promote those who wait in safety while their team strips armor only to pounce out near game's end to get easy kills regardless of win/loss.

Please encourage a culture of healthy team play, not hero ball.

Seen the idea mentioned in this thread a few times, but I'll reiterate it. Because it has merit. And it is intriguing in its simplicity:

No more PSR. Ranking is nudged up/down based on win/loss, period.

Over the course of many matches tiers emerge organically, as skilled players contribute more to winning than unskilled players. Plus this has the added benefit of encouraging team play, playing to the game mode, and equally weighting all roles in a team (i.e. scouting, spotting, capping, ECM and AMS umbrella, diversion, etc.) rather than emphasizing dmg/kill-counting.

Edited by Scooter Libby, 10 June 2020 - 09:23 PM.

[Scooter Libby]

Eatit, on 09 June 2020 - 11:29 AM, said:

Others have posted a better solution that is to change the MM to create teams for the matches in an A-B-A-B... manner. Highest tier player gets added to team A, next highest tier player gets added to team B, and so on until you have two equally matched teams.

This will create two evenly matched teams of people from all of the tiers. Each of the teams will contain the same number of players from each of the tiers.

This idea also deserves attention. The player pool is small. Looking forward, the player pool might shrink further (because new games keep coming out, not because MWO is "broken").

So effective match balancing now and in the future will have to consider a mechanism for deliberately mixing tiers in balanced teams. This is hard because teams must also be sorted for mech tonnage. Hard, but not impossible. Player ranking could be applied as a tonnage multiplier, for example. So a skilled player in a medium would equal an unskilled player in an assault.

Regardless, player skill should take precedence over mech tonnage in team balancing. The mech classes are already well balanced against each other in the game. So long as there is some minimum diversity of mech classes on each team, a tonnage mismatch will work fine if the top players are distributed evenly.

Edited by Scooter Libby, 10 June 2020 - 09:40 PM.