357 replies to this topic

[TheFourthAlly]

Lacking in depth knowledge, 2B sounds ok to me in principle.

I think a win or a loss should count. Actual good players should be able to influence outcomes over the long run and ride out the occasional flub or bad luck match-up.

[RCore]

+1 vote for Core 1A.

Or alternatively Core 1B with zero-sum (I think Jay Z was trying to explain how that can be done with his formulas).

[Gozuri]

While I am against the use of match score at all, it is evident that PGI thinks otherwise, and so I no longer have much else to input. If I had to choose, I guess it'll be 2A. Or 2B if made to be zero sum.

And since someone brought up Solaris, it does have a psr system. Anyone wanna argue if we should be using matchscore for solaris? If yes, I'd like to hear it. If no, translate your answer to QP.

[kapusta11]

I'm not an expert number cruncher but lets assume we have a perfect matchmaker. In every game both teams have X amount of veterans, Y amount of mid level players and Z amount of low skill pilots. In the long run W/L ratio for everyone would be close to 1 because teams are perfectly balanced and as someone mentioned on page 1 of this thread, a side benefit of it is better player retention. But how do you distinguish between players of different skill? Obviously you can't use W/L ratio because it should be 1 for everyone. If you're good and you're still have high W/L ratio it means you don't see equally skilled players on the opposing side. If you have low W/L ratio it means matchmaker is stacking better player for the opposite team. Therefore matchmaker CANNOT use W/L ratio to balance teams BUT making sure it stays as close as possible to 1 as a result of other measures is the goal and adjustments to the system should be made if it's not.

If you were to balance someone with any W/L ratio with someone with equal W/L ratio on the opposing side over time it will result in a W/L drop/increase that moves it closer to 1. Lets say you always put proton and bows3r on opposite teams and other players are equaly balanced by their W/L ratios too, over time everyone's W/L ratios will drop/increase to 1 but now how do you distinguish them from one another? High W/L ratios that we see right now is a product of bad matchmaking. It means top 1% players are "balanced" with top 50% because both are in "Tier 1" allowing former to farm the latter along with lower tiers. There was a guy with W/L and K/D ratios below 1 who complained about losing his tier 2. The fact that people with stats like that are in tier 2 tells you everything you need to know about how broken current system is and how biased everyone's stats are. Hell, I should not be in tier 1, probably not even t2.

So what metrics exactly should matchmaker use? How the hell do I know? I don't have the data. What's worse is the existing data is not objective because of the broken matchmaker we have right now.

What good players do that bad ones do not? Get more kills? If matchmaker stacks nobodies on the opposing side, maybe. Staying alive while being engaged in a fight with multiple enemy mechs (brawling iic)? I guess it makes the enemy team outnumbered for a period of time increasing the odds of winning. I've done that in a light. Getting more KMDD? This one is interesting. It can be considered a measure of how accurate you are, not in terms of just hitting but hitting where it counts. Think about it, if you manage do get two good alphas or dps down and enemy CT with accurate fire chances are you'll get KMDD for that mech IF it dies because whoever fires at it again will likely kill it. Of course you can just LRM someone for a while and get KMDD that way in which case it doesn't reflect skill all that much so it could be tweaked to count CT damage alone and kill requirement could be removed. A similar reward could be added for ST damage. Securing a kill should be important too. I've had a ton of games where half of the enemy team was cored while stomping and peope failed to just shoot the right mechs. This is why survive rate is also important. Can you stay alive and still contribute on low %? Taking some damage for a team should probably be rewarded more as well.

Reducing reward for damage is a mistake. Dealing a ton of damage is not something that everyone can do especially if you can do it over and over. You have to know how to stay alive, how to trade favorably, how to share armor to prevent teammates from dying without taking too much damage yourself. I guess it could be tweaked to count LRM damage at 2/3 strength. Like I said, I'd rather buff "brawling" rewards to make light mechs that use diversion to win games have higher scores.

Edited by kapusta11, 19 June 2020 - 07:52 AM.

[Gozuri]

Psr in solaris doesnt look like WLR to me. Either way, no point arguing now. I'll just have to shelve my flamers or ignore the scores.

[Gagis]

Solaris uses Elo ratings, which go up on a win and down on a loss and the amount depends on the difference between yours and your opponents ratings.

Elo ratings are a great system. I wish we could have them back for Quick Play.

[Nightbird]

Elo works for teams with FIXED rosters, but do not work for teams with random rosters. PGI started with Elo and failed to recognize this fact, so their implementation caused people's Elo scores to diverge rofl.

[WarmasterRaptor]

So, in regards of the proposal in the OP,

1A : I Support, removes the groups effects and win/loss, you do what you can do in the chaotic mixed queue.

1B : not zero sum, therefore no support,

2A : Can't support as is.
A player that didn't contribute it's fair share to the win shouldn't be carried by the win of the team.
Therefore negative PSR adjustments should happen even if you win!
Reverse also applies.
If this is changed to consider the above, I would support,

2B : not zero sum, no support.

[Gagis]

Nightbird, on 19 June 2020 - 09:14 AM, said:

Elo works for teams with FIXED rosters, but do not work for teams with random rosters. PGI started with Elo and failed to recognize this fact, so their implementation caused people's Elo scores to diverge rofl.

It should work for teams too if carefully implemented, but yeah. Does sound plausible the original implementation may not have been convergent. In team games it should converge, but due to 24 players worth of variance, it will take a much larger number of games to find a stable rating.

Other than the convergence vs divergence question, the original MWO matchmaker seemed to not give enough weight to how hard an above the norm pilot can carry their team.

Like, if you have a lance with 3 players at Elo 1333 and one player at Elo 2000 playing against a 4-man team where all players are at Elo 1500. The first team would actually win more often than not, even though both teams average at 1500. Some sort of weighting based on how far the rating of a player is from the match mean or median might be able to give proper emphasis on outliers.

Edited by Gagis, 19 June 2020 - 09:46 AM.

[Gagis]

WarmasterRaptor, on 19 June 2020 - 09:43 AM, said:

A player that didn't contribute it's fair share to the win shouldn't be carried by the win of the team.
Therefore negative PSR adjustments should happen even if you win!
Reverse also applies.
If this is changed to consider the above, I would support,

Why do you think this? It makes no difference for matchmaking, since all players are influenced equally much by luck and other uncontrollable factors, and with time, skill is what makes the difference in if you win more often than not.

Any attempt to adjust numbers on a match-by-match basis will lead to less accurate long-term statistics and thus less accurate matchmaking.

[Nightbird]

Gagis, on 19 June 2020 - 09:44 AM, said:

It should work for teams too if carefully implemented

Yes, though at that point it's not really Elo anymore. Maybe, inspired by Elo? The math gets a whole lot different and harder, and we know what PGI says to math... "no thanks"

[Gagis]

Nightbird, on 19 June 2020 - 09:57 AM, said:

Yes, though at that point it's not really Elo anymore. Maybe, inspired by Elo? The math gets a whole lot different and harder, and we know what PGI says to math... "no thanks"

Actually, I just saw a proposal online of using Elo as is with its 1v1 definition but with your opponent's rating being the sum of the ratings of the enemy team minus the ratings of other players on your team. Something like that would be extremely simple mathematically.

Might in practice require some sort of weighted sums or normalization to make it behave nicely, but still not very mathy at all.

[SilentScreamer]

Paul Inouye, on 16 June 2020 - 04:30 PM, said:

Hey folks,

First off, thank you to everyone participating in this. There are some ideas here that will work with some fine tuning and we can work on those to get something that works for the game and you the community.
...
Core 2A: Win/Loss Team Compare
In this system, win/loss still dictates direction of PSR movement. However, instead of an individual measure against Match Score, the player's match score is compared to the match score of all players on their team. For example, the winning team will have a neutral to positive movement in PSR. The losing team will have neutral to negative movement in PSR.

In the following image you can see how this scenario works out:



Again, to maintain zero sum, the distribution of points in both positive and negative directions need to be equal.
...
Thanks again for all the suggestions and discussion in the other thread. I hope to see it continue here as well.

-Paul

2A is my favorite of the choices presented. It is zero sum and doesn't penalize two important groups:
a) better performing players on the loosing team
b) lowest performing players on the winning team

I would actually like to see a small increase in PSR for the top two to three players on the winning team; awards amounts could easily be adjusted so the system remains zero sum by the formula:

4X + 6Y = 5Z

+X applied to Top 4 performance winning team
+Y applied to MIddle 4 performance winning team and Top 2 on loosing team
-Z applied to Lowest 5 performance on loosing team
Lowest 4 (#8-12) on winning team and middile 5 (#3-7) on loosing team receive no PSR adjustments.

Edited by SilentScreamer, 19 June 2020 - 11:57 AM.

[Nightbird]

Gagis, on 19 June 2020 - 10:47 AM, said:

Actually, I just saw a proposal online of using Elo as is with its 1v1 definition but with your opponent's rating being the sum of the ratings of the enemy team minus the ratings of other players on your team. Something like that would be extremely simple mathematically.

Might in practice require some sort of weighted sums or normalization to make it behave nicely, but still not very mathy at all.

That's what PGI did and it diverges for obvious reasons:

If for every match your and your opponents team always have the same Elo, unless you average 1 WLR, you will diverge. If you are >1 WLR your Elo will go up infinitely and if you are <1 WLR you will go down infinitely. This is made worse when the teams have the same Elo, Elo gains and losses are maximized, even though the individuals in the teams may have very high or low ratings.

Another case of hopeful opinions not being a replacement for boring math. At the beginning of this process PGI stated wanting to reduce stomps. After all this time, where is math that plugs in the values proposed and shows a result of less stomps?

Edited by Nightbird, 19 June 2020 - 12:35 PM.

[MEH0W]

The starting point should be MS Kickers. This must account for player's involvement in team play but also in objective participation, use of terrain to avoid LRM DMG, ECM cover to teammates and so on. Great example on what needs improving is winning a conquest game in a light mech and getting miserable MS due to low DMG while capped all objective .
Current games are mainly skirmish type-like due to little incentive to do anything else than farm DMG.

I can see that many of you put a lot of effort to evaluate what the best possible PSR adjustment should be, but no matter which one will be used, game play style will remain the same due to MS Kickers.

I agree that Tier status may look like an accomplishment to many, but it is used to make sure you play with other 23 similarly performing players whether grouped or not. Maybe Tier status should be readable from the user profile and instead players should be shown Current Rank in game to give them something to pursue?

I don't expect best PSR mechanism in the world to work well if MS inputs are not right...

[Cluster Fox]

From those provided, I can only support core 2 options, 2A or 2B with heavy W/L factored in, JayZ or not.
Like, JayZ's 2C with X = 9, Y = 10, C = 1 for instance, which creates a strong shift between win and loss.

I'd like 1A supporters to show data or a simulation that, at least, suggests it will give better matches over time, aka less stomps. Like, IDK, run it for 10000 games. Please provide assumptions and show work.

We've seen data that heavily weighting W/L should do that. I haven't seen long term simulations for the flip side of the argument. Yeah the MM is flawed but PSR and MM work hand in hand.

Decision shouldn't be based on matchscore screenshots and feeling good/bad about a match.

PSR isn't there to make you feel good/bad about PSR, it should give you good matches and ideally reduce stomps. Then you'll feel good cause the matches are better.

My Tier best Tier, change my mind.

Edited by Cluster Fox, 19 June 2020 - 02:44 PM.

[C64 Warrior]

Because your personal skill has nothing to do with winning or loosing (there are 23 other people in any given match that effect win/loss) I vote 1A

[JohnOSpencer]

I'm pretty new overall so most of this is a bit beyond me. I did just hit tier 4 and, while proud of it, it's not going to impact my play that much. I really just want to have fun stomping around in a giant robot blasting things. So any of the options seem fine.

That being said, the teamwork is occasionally tragic so I'm for anything that would increase teamwork. I'd choose 2B (preferably with JayZ's changes).

[DontStandBehindMe]

Winning shouldn't be the only thing that matter or the more people loose the less they will want to play. This isn't comp play, we aren't on teams of our own choosing (meaning we didn't select our teammates). If your performance in in the top 1-4 players of the 24 people in your match you shouldn't be held back because your side lost.

my vote: 1A

if I could make a modification to the 1A system it would to also compare your result with your average match result (baest on all the games youve played to date) if you score below your average then you should drop PSR (or not increase as much based on the match) and if you score above then your PSR should go up a little more

[Blechreiz]

Cluster Fox, on 19 June 2020 - 02:30 PM, said:

I'd like 1A supporters to show data or a simulation that, at least, suggests it will give better matches over time, aka less stomps. Like, IDK, run it for 10000 games. Please provide assumptions and show work.

We've seen data that heavily weighting W/L should do that. I haven't seen long term simulations for the flip side of the argument. Yeah the MM is flawed but PSR and MM work hand in hand.

Decision shouldn't be based on matchscore screenshots and feeling good/bad about a match.

This may seem like a non-substantial post, but please, think about what I'm trying to explain (sorry, english has only been my 2nd lanuage):

It's really nice to see so many people participating and developing their own ideas of how matchmaking, PSR and matchscore could be reworked. Now, I think using just historical data is something that's just not close to reality (which these models are based on).

Sorry to say, but Jay Z's models seem to try to only remedy the errors in the assumptions of the initial model. The error was, that the model didn't include the change in the average matchscore of the player... Furthermore it completely diregarded the current matchscore system (of which we didn't know the values). Overall, there were too many signficant unknown variables in the system...

Problem is, with the give data from Jarl's list and without the current matchscore kickers, it's not really easy to come up with a good model to resolve these issues...

Hence I still think 1A is a good starting point, and most of the other issues can be addressed by chaneging the matchkicker values.