713 replies to this topic

[Nightbird]

Zanotam, on 12 June 2020 - 03:39 PM, said:

To everyone complaining about groups:

A group can generally expected to be maybe 4 slots, 1/3 of your team. In League of Legends however the 'solo queue' lets people queue in pairs which makes them 2/5ths of the team (~6.6666% repeating more!) and yet nobody is bitchign about how it's impossible to accurately rank people in LoL.... in fact, matchmaking in LoL is pretty damn good.

There are exactly 3 meaningful things for the future MM:

1. player population,
2. using something meaningful to predict winrate
3. Effectively composing teams in the most fair fashion to adjust for top-tier and bottom-tier players skewing things.


GROUP QUEUE IS NOT RUINING THINGS AND SMALL GROUPS AS PART OF A LARGER TEAM ARE NOT AN ISSUE FOR A MATCHMAKER!

For a working* matchmaker. LoL uses a heavily modified Elo system. It's basically W/L but with point movements adjusted for the relative strength of teams and players. PGI did ELO at first but they didn't use the right formula and so above average players went to infinity and below average players went to 0 lol. The current system is slightly less broken.

Edited by Nightbird, 12 June 2020 - 03:52 PM.

[VonBruinwald]

Horseman, on 12 June 2020 - 03:07 PM, said:

It has been advertised... you just missed when and where.

Got a link for that advert?

Horseman, on 12 June 2020 - 03:07 PM, said:

They're working on their proposals. In an environment that's more conducive to actual work as opposed to getting constantly interrupted by armchair mechwarriors who think they have a constitutional right to be in Tier 1.

You do realise that's most of T1 you're talking about.

[OneTeamPlayer]

Zanotam, on 12 June 2020 - 03:39 PM, said:

To everyone complaining about groups:

A group can generally expected to be maybe 4 slots, 1/3 of your team. In League of Legends however the 'solo queue' lets people queue in pairs which makes them 2/5ths of the team (~6.6666% repeating more!) and yet nobody is bitchign about how it's impossible to accurately rank people in LoL.... in fact, matchmaking in LoL is pretty damn good.

There are exactly 3 meaningful things for the future MM:

1. player population,
2. using something meaningful to predict winrate
3. Effectively composing teams in the most fair fashion to adjust for top-tier and bottom-tier players skewing things.


GROUP QUEUE IS NOT RUINING THINGS AND SMALL GROUPS AS PART OF A LARGER TEAM ARE NOT AN ISSUE FOR A MATCHMAKER!

Has it occurred to you that people aren't complaining about groups in LoL because LoL isn't putting it's players into unwinnable matches one out of every three games. Not just unwinnable but so one sided as to be not enjoyable for both teams?

The rotating stomp fest started after group queue merge which is why this community is wary of the merge.

Pre-merge there were losses and bad tactics, heck even a stomp on occasion, but it wasn't so often that players could go an entire session seeing mostly stomps one way or another.

The most obvious solution would be to remove the group queue from solo then tweak a solution, but what's actually happening is keeping the two mixed, watching as players get more and more frustrated, and making a community based solution from externally based input sources after asking for community input on their own official forums.

In short, everything about the merge is a mess and MWO is obviously not LoL.

That's like saying "well zuckerberg can afford to buy a ferrari and not go into debt, why can't I?" Well, for starters, you're not Mark Zuckerberg rich, that's why.

When MWO starts getting down basics as well as LoL has we can start comparing the two, until then its an apples to dishwashers comparison.

[Eatit]

After reading many of these posts it occurs to me that there are some that think their skill is being devalued by allowing less skilled players to hold the same skill title, Tier 1.

There are also some that want the tiers to be better sorted so that they can have a more balanced experience when playing.

The current proposal will only work for the people that want the title T1 to mean something. Matches will still be unbalanced.
It seems as though this is an ego stroke and not an actual attempt to fix an unbalance MM.

[Zanotam]

Nightbird, on 12 June 2020 - 03:50 PM, said:

For a working* matchmaker. LoL uses a heavily modified Elo system. It's basically W/L but with point movements adjusted for the relative strength of teams and players. PGI did ELO at first but they didn't use the right formula and so above average players went to infinity and below average players went to 0 lol. The current system is slightly less broken.

I mean, isn't W/L but with point movement adjusted for relative strength of teams and players even likely better than your pure W/L suggestion? Although I think your W/L suggestion is probably the best one I've seen though so far.

OneTeamPlayer, on 12 June 2020 - 04:01 PM, said:

Has it occurred to you that people aren't complaining about groups in LoL because LoL isn't putting it's players into unwinnable matches one out of every three games. Not just unwinnable but so one sided as to be not enjoyable for both teams?

The rotating stomp fest started after group queue merge which is why this community is wary of the merge.

Pre-merge there were losses and bad tactics, heck even a stomp on occasion, but it wasn't so often that players could go an entire session seeing mostly stomps one way or another.

The most obvious solution would be to remove the group queue from solo then tweak a solution, but what's actually happening is keeping the two mixed, watching as players get more and more frustrated, and making a community based solution from externally based input sources after asking for community input on their own official forums.

In short, everything about the merge is a mess and MWO is obviously not LoL.

That's like saying "well zuckerberg can afford to buy a ferrari and not go into debt, why can't I?" Well, for starters, you're not Mark Zuckerberg rich, that's why.

When MWO starts getting down basics as well as LoL has we can start comparing the two, until then its an apples to dishwashers comparison.

I would point out that the statistics say stomps were happening 25% of the time pre-merge and you must not be aware of how common stomps are in LoL because 30% or so of the time sounds about right for stomps one way or another.

[Paul Inouye]

After reading and trying to keep everything organized, I will be back Tuesday after combining like minded thoughts with one another and go over the main proposals laid out here.

As mentioned on twitter (yes... I tweeted something and feel dirty doing it.), it's imporant that we keep focused on:

Match Score movement calculations - The +5, +3, +1 stuff when you reach a Match Score threshold. This includes those diagrams of overlapping movement on win/loss and even the discussions on removing the win/loss component and comparing your match score vs the 23 other players and assigning movement based on that.
Match Score thresholds - The 0-100, 101-300, 301-400+
Match Score event kicker calculations - The you should get X for a kill, Y for an assist etc.

I can't respond to every single post made here, but I'll do my best (as mentioned above) to combine any ideas that are the same but just have different variable settings into one response and I'll do my best to address any outliers outside of those groupings.

For Match Score event kickers, I'd like to release a set of data that you could run your formulas against. I don't know if you'd like that as a Google doc or an Excel spreadsheet. Let me know.

I've also been asked a number of times for current values of those Match Score kickers. At this time I can't provide you with that due to possible exploitation/trolling behaviors. I can however provide you a sample set of numbers to show you how the calculations work in general that reveals what would be your end of round match score and you could tweak from there.

-Paul

p.s. Forgot to mention something about the population numbers and some of you are aware... but no matter what system is put in place, the match maker is still going to have to open up to have matches kick off in timely manner.. so yes, while a new match score formula and PSR tier movement numbers will make a tighter skill based distribution, the numbers in the actual queue will dictate how far apart those skill numbers will be in a given match. This is why Tier 1 will continue to play against Tier 2 and Tier 3 players.

[Nightbird]

Zanotam, on 12 June 2020 - 04:19 PM, said:

I mean, isn't W/L but with point movement adjusted for relative strength of teams and players even likely better than your pure W/L suggestion? Although I think your W/L suggestion is probably the best one I've seen though so far.

Absolutely, but data is needed to fine tune and we have none. I'm not asking for PGI to release data either, since other companies don't release it either and do all of their math through hiring analysts.

[OneTeamPlayer]

Nightbird, on 12 June 2020 - 04:29 PM, said:

Absolutely, but data is needed to fine tune and we have none. I'm not asking for PGI to release data either, since other companies don't release it either and do all of their math through hiring analysts.

Except the "hiring analysts" part is missing here.

Also if the whole system is due for an imminent overhaul why would Paul care about "exploitative/trolling behavior" from data release since the system shouldn't be around long enough for any trolling behavior to matter in the short or long term?

Is this admitting that changes aren't coming any time soon?

[Gagis]

https://docs.google....t#gid=414734865
Details of my proposal are in the spreadsheet. Feel free to play with the values to see how it works.

PSR Based on Wins and Losses Evaluated by Your Share of Match Score

EXPLANATION
The goal here is to create a PSR that is convergent and does not diverge to infinities over a large number of games. In order to get rid of the ''XP Bar'' phenomenon, we need something other than a straight sum of match results. The playerbase however seems to be unhappy with tracking Win/Loss Record, so I made a model that gives your wins and losses a value based on your Match Score. I call this the WinScore/LossScore Ratio.

I even added a Feelgood Factor to make it possible to gain PSR on a loss trough ''negative LossScore'' or lose PSR on a win trough ''negative WinScore'' if your performance is too far above or below what it should be. If Feelgood Factor is 0, you won and earned the highest Match Score, you would get 1.0 Winscore or if you both lost and got the lowest Match Score, you would get a LossScore of 1.0.

HOW?
WinScore = (your_score - lowest_score - FgF) / (highest_score - lowest_score)
LossScore = 1 - (your_score - lowest_score + FgF) / (highest_score - lowest_score)

PSR approximates PSR = sum(WinScore) / sum(LossScore) over the last X games but since we don't want to store WinScores and LossScores or track multiple results between matches, it is actually updated as
new_PSR_if_winning = old_PSR + WinScore / X
new_PSR_if_losing = old_PSR / (1 + LossScore / X)
where X is whatever we want Length of Pseudo Match History to be.

I want to stress again that only information we need is the Match Scores, winners/losers and existing PSR and the only variable that needs to be saved is PSR.

Please check the accompanying google docs spreadsheet for more details.

Edited by Gagis, 12 June 2020 - 05:11 PM.

[Tamerlin]

Paul Inouye, on 12 June 2020 - 04:20 PM, said:

For Match Score event kickers, I'd like to release a set of data that you could run your formulas against. I don't know if you'd like that as a Google doc or an Excel spreadsheet. Let me know.

I've also been asked a number of times for current values of those Match Score kickers. At this time I can't provide you with that due to possible exploitation/trolling behaviors. I can however provide you a sample set of numbers to show you how the calculations work in general that reveals what would be your end of round match score and you could tweak from there.

Excel would be good

[Poor-Life-Choices]

Nightbird, on 12 June 2020 - 03:01 PM, said:

You have to analyze the data for all players to see trends. A single person's data isn't useful.

Just look at the jarls list global stats. Match score improves with more games played. 99% of players have WLR below 2, 50% are below 1, you can’t slice that small enough to tell good from bad. I looked at 3 sets of data, and all of the WLR numbers varied greatly month to month.

[thievingmagpi]

Any kind of "drift", as may occur in the proposed systems might be an issue with the assumption that the game is going to last another 8 years.

Or that any kind of ranking system couldn't just be reset periodically.

Edited by thievingmagpi, 12 June 2020 - 06:50 PM.

[Mr D One]

This caught my eye, and made me log in to comment.

The issue is not the formula or the player base. It's in the balance of weapons and tech, and experience.

For example - If the game was locked to 1 mech, with the same tech base, same weapons, across the 24 players, then you would get a real sense of who knows what to do and the skill level involved.

So for instance with have 24 Commando's on the field, all the same, and I would expect the account named Mycrus to dominate most of the matches.

Then what happens is people remember that name as being a threat, so the other players start to focus Mycrus. Now Mycrus may make an alt account, or change his account name to keep the focus of him.

So then PGI introduces mech X and weapon X, and then suddenly, the meta changes, and a great player in a Commando is very average in a Hunchback.

Your punishing a players account based on the fact that he hasn't got the right mech, the right weapons, despite having the skill in another chassis?

If it were me, and I have said this to you in the past PGI, you would have the Tiers renamed as Elite, Veteran, Rookie, and Green.

You would have a mode where the mechs have to be the same builds, and then sort the best from the worst in that mode.

Currently, your trying to assess a player's worth on shifting sands, and if your building a system on that, it will fail.

Good luck with that. Thanks again for MW5.

[Dogmeat1]

I'm posting this on behalf of kamikaze viking.

Kami's Lookup Table PSR Shift Proposal.

the reasoning behind My proposal is

a) To fit within the scope given. Minimal work for Best effort gains.

A compromise to cater to both camps of "everything is Win/loss" and "everything is personal skill"

I realised early in the discussion regarding how Match Score affect PSR movement that due to score increase based on the WIN The current system is 'double dipping'. Hence the Solution appeared to me to be to neutralise the Raw Match score by using a relative system separated by team. This would seem to be to cancel out the Match Score granted for winning, so that the total MS for each player reflects better on their skill within the team.

The result is that this system grants a factor for the WIN/loss as well as your personal skill relative to your team, and these should be easily tweakable within this system (values within a lookup table).

Method
==============

This graphic may be familiar to some as ive posted it a few times in recent weeks.

https://i.imgur.com/nSl4rQ9.jpg

PSR Shift Values
================
These are base shift values - totalled as zero sum.
Win = -1 +0 +1 +2 = +1
Loss = -2 -1 +0 +1 = -1
Win + loss = +1 -1 = 0

The higher the number the faster people will move through the psr range per match. initial Simulations they found that 0, 12 & 24 worked best (and fit with Pauls proposed range).

Applying this across the match. Many systems suggested will sort all 24 players and then rank them, mine MUST keep the two teams separate and rank them by MS within their team.
==================
When applied to each team in a 3/3/3/3 team split

Win = -1x3 +0x3 +1x3 +2x3 = -3 +0 +3 +6= +6
Loss = -2x3 -1x3 +0x3 +1x3 = -6 -3 +0 +3= -6
Win + loss = +6 -6 = 0

Example Match using 3/3/3/3 split
https://i.imgur.com/FJoNrQO.jpg

==================

As Jay Z was testing in his simulator (as seen in this post https://mwomercs.com...66#entry6337266), he implemented this using a lookup table. Which showed once tweaked, a more granular version seemed a better distribution and still remains zero sum. I derived my own version as can be seen below;

Winning team PSR Shift

Base version	Tweaked version
24	24
24	20
24	16
12	12
12	8
12	4
0	0
0	0
0	0
-12	-4
-12	-8
-12	-12

Losing team PSR Shift

Base version	Tweaked version
12	12
12	8
12	4
0	0
0	0
0	0
-12	-4
-12	-8
-12	-12
-24	-16
-24	-20
-24	-24

Sum = 0

Edited by Dogmeat1, 13 June 2020 - 02:55 AM.

[MischiefSC]

I don't play enough to be a real contributor to this discussion but I want to point this out -

Match Score doesn't equate to how effective someone is at winning matches. It's skewed by damage farming, kill stealing, etc. While top tier players often have a high average match score that's a product of a lot more than just damage numbers and component destruction.

The only useful metric for determining how effective someone is at winning matches is.... how often what they do, how they play, helps their team win matches.

So the question very honestly becomes:

Do you want a matchmaker that is as effective as possible at building teams who are evenly matched at winning

OR

Do you want a matchmaker that rewards the average players perception of 'I dun good' and punishes the average players perception of 'I dun bad'.

Maybe the game lacks the population to make the first super effective but the second is largely worthless as a matchmaker anyway. If the purpose of the new MM/PSR system is to just give player a sense of validation for 'I dun good' or 'I dun bad' that needs established. That's not necessarily a bad thing even; the community is a lot more 'stompy robbits EXPLOSIONS boom whee' than 'LEARN GIT GUD'.

[Nightbird]

Kurlon, on 08 June 2020 - 04:30 PM, said:

The community driven PSR project

In response to PGI’s announcement on this thread, the discord and reddit communities have decided to work together to try to find the best solution to this issue. At first this was a JGx led initiative but quickly expanded to include dozens of representatives from most of the major units, as well as a large number of voices from the solo community. After many hours exchanging and refining ideas, we believe we have come up with a solution that both meets PGI’s requirements and that will improve the gaming experience of the community as a whole.

Arbitrary fixed PSR gains/losses present a number of problems in terms of balance and fairness to all players involved in an individual match. Therefore our solution is to use scaling PSR gains/losses based on an individual player’s match score matchscore as compared to the team's average. The system uses the Win/Loss status to compute a 0 sum PSR shift that is scaled fairly to the pilot’s contribution in that match. The maths involved is no more complex than the current XP, C-bill and Matchscore calculations requiring minimal development time.

This system was developed by Jay Z with heavy input from the veteran community. A link to a full explanation is below;

https://docs.google....3LgMBQJ4hE/edit

And a spreadsheet simulating the system can be found here;

https://docs.google....dit?usp=sharing

In summary, this proposal uses simple arithmetic similar to existing calculations to assign fair and reasonable PSR shifts in accordance to pilot contribution to the match result. The aim of PSR is to segregate the playerbase on their respective match impact to give the Matchmaker the best possible information to create fair and interesting matches.

The version of this proposal you see presented is simplified from a more complicated model we have been testing which has more adjusting and weighting factors. Within the parameters set for this task, we believe this to be a straightforward implementation which will have community support. Finally, given there are only two values that control the maths, the long term management is simple and easy to hand over. If you have any questions, or concerns, please let us know as we have investigated this and numerous other solutions in great detail.

The matchscore formula itself does need quite a bit of adjusting. We plan on posting an update regarding that soon.

Finally during our discussions we also came up with 2 other proposals we strongly believe would be worth considering;

Kamikazi viking’s PSR lookup table https://mwomercs.com...05#entry6338105

Gagis’ PSR gains based on matchscore ratios https://mwomercs.com...st__p__6338091

Dogmeat1, on 12 June 2020 - 07:17 PM, said:

I'm posting this on behalf of kamikaze viking.

Kami's Lookup Table PSR Shift Proposal.

the reasoning behind My proposal is

a) To fit within the scope given. Minimal work for Best effort gains.

A compromise to cater to both camps of "everything is Win/loss" and "everything is personal skill"

I realised early in the discussion regarding how Match Score affect PSR movement that due to score increase based on the WIN The current system is 'double dipping'. Hence the Solution appeared to me to be to neutralise the Raw Match score by using a relative system separated by team. This would seem to be to cancel out the Match Score granted for winning, so that the total MS for each player reflects better on their skill within the team.

The result is that this system grants a factor for the WIN/loss as well as your personal skill relative to your team, and these should be easily tweakable within this system (values within a lookup table).

Method
==============

This graphic may be familiar to some as ive posted it a few times in recent weeks.

https://i.imgur.com/nSl4rQ9.jpg

PSR Shift Values
================
These are base shift values - totalled as zero sum.
Win = -1 +0 +1 +2 = +1
Loss = -2 -1 +0 +1 = -1
Win + loss = +1 -1 = 0

The higher the number the faster people will move through the psr range per match. initial Simulations they found that 0, 12 & 24 worked best (and fit with Pauls proposed range).

Applying this across the match. Many systems suggested will sort all 24 players and then rank them, mine MUST keep the two teams separate and rank them by MS within their team.
==================
When applied to each team in a 3/3/3/3 team split

Win = -1x3 +0x3 +1x3 +2x3 = -3 +0 +3 +6= +6
Loss = -2x3 -1x3 +0x3 +1x3 = -6 -3 +0 +3= -6
Win + loss = +6 -6 = 0

Example Match using 3/3/3/3 split
https://i.imgur.com/FJoNrQO.jpg

==================

As Jay Z was testing in the simulator (as seen in this post https://mwomercs.com...66#entry6337266), he implemented this using a lookup table. Which showed once tweaked, a more granular version seemed a better distribution and still remains zero sum.

Winning team PSR Shift

Base version	Tweaked version
24	24
24	20
24	16
12	12
12	8
12	4
0	0
0	0
0	0
-12	-4
-12	-8
-12	-12

Sum = 0

Losing team PSR Shift

Base version	Tweaked version
12	12
12	8
12	4
0	0
0	0
0	0
-12	-4
-12	-8
-12	-12
-24	-16
-24	-20
-24	-24

Sum = 0

The issue with the current PSR + MM is that you can calculate an average PSR movement per match for player, call it X, then the player's "skill" = # matches_played * X.

After 100 games, PSR=100X. After 2000 games, PSR=2000X. Whether the PSR really represents skill is convoluted with the number of matches.

It means you can have two players with PSR = 1000 and 3000, and the PSR = 1000 player reached it with 10 * 100matches, and the PSR = 3000 player reached it with 3 * 1000matches. It is clear the PSR = 3000 player is not as skilled. Such mix ups across the population means the Match Maker is blind when using this PSR.


With either PSR shift proposal above, we're changing X for Y. Is it obvious how much of an impact this will have on match quality?

Edited by Nightbird, 12 June 2020 - 07:42 PM.

[Jochi Kondur]

Kurlon, on 08 June 2020 - 04:30 PM, said:

- s n i p -

This system was developed by Jay Z with heavy input from the veteran community. A link to a full explanation is below;

https://docs.google....3LgMBQJ4hE/edit

And a spreadsheet simulating the system can be found here;

https://docs.google....dit?usp=sharing


- s n i p -

For Case 3 (Stomp), although prefaced with "hopefully become less common", players 13 - 16 on the Losing Team in the cited example were assessed to have a far greater JZ PSR Shift because they “valiantly fought on a sinking ship” under the proposed system.

Was this scenario tested by usage of actual stomp data, could they be shown? Ideally, a video analysis from the Losing Team POV to show that those who stood to valiantly fight and fell as a result are indeed deserving to be in the company of players 3 - 5 (from Case 3 Stomp example) on the Winning Team.

Edited by Jochi Kondur, 12 June 2020 - 09:08 PM.

[Dogmeat1]

Paul Inouye, on 12 June 2020 - 04:20 PM, said:

I've also been asked a number of times for current values of those Match Score kickers. At this time I can't provide you with that due to possible exploitation/trolling behaviors. I can however provide you a sample set of numbers to show you how the calculations work in general that reveals what would be your end of round match score and you could tweak from there.

That would be much appreciated. We understand the reservations PGI has with giving out the exact formula. Many of us already have a fair idea of how the formula works ,however, any data you can give us will be very helpful.

Is there any timeline yet on when we might be able to receive that data, as well as the deadline for submitting matchscore proposals? I have been working with many in the comp and casual communities and we have already submitted some PSR ideas and now are focusing on improving the matchscore balance so that it better reflects key values that contribute strongly towards wins. We will probably submit our matchscore formula ideas soon but it would be good to know when the cutoff for that will be.

[Horseman]

Nightbird, on 12 June 2020 - 02:09 PM, said:

Don't worry about it. I was in that group but my idea was rejected because of the reason... and I paraphrase "The community is too stupid to understand it". I have a screenshot lol

You forget that some of us in this thread also have access to the discord. Your idea has been applauded by quite a few people. The problem they had with it was that it would be extremely difficult to convince most of the playerbase of its' value.

Edited by Horseman, 12 June 2020 - 10:56 PM.

[Gagis]

Jochi Kondur, on 12 June 2020 - 05:57 PM, said:

I can't reconcile with the fact that the worst performing player (i.e. worst of 24) has no negative PSR movement just because the person happens to be on the winning team. I can't cast my support behind such a system.

If their score is low enough, the Feelgood Factor will push their WinScore to the negatives. The Feelgood Factor was added so you would feel good about this.