248 replies to this topic

[MrMadguy]

Wil McCullough, on 12 June 2019 - 12:59 AM, said:

I don't think you understand how statistics work.

Your statistics would work only in All-vs-All situation. I.e. when I will be able to play against both good and bad players. But when I always end in team, where we have 2-3 Tier 1 players, I will never play match, where I will affect result of this match. This match will always be Tier 1 vs Tier 1 with other players, just padding teams and serving as free frags and dummy targets. And my MS - is only thing, that will show my REAL performance.

So yeah, we really need last thread about matchmaker. WLR based matchmaker.

[Wil McCullough]

MrMadguy, on 17 June 2019 - 08:02 PM, said:

Your statistics would work only in All-vs-All situation. I.e. when I will be able to play against both good and bad players. But when I always end in team, where we have 2-3 Tier 1 players, I will never play match, where I will affect result of this match. This match will always be Tier 1 vs Tier 1 with other players, just padding teams and serving as free frags and dummy targets. And my MS - is only thing, that will show my REAL performance.

So yeah, we really need last thread about matchmaker. WLR based matchmaker.

Lol all-vs-all. You get matches with absolute potatoes, matches with top players and everything in between. What all-vs-all are you on abouy? How much you affect your team's ability to win isn't visible on a per-match basis. It's however very obvious in the long run. Your match score can be gamed. Your win loss record can't. You want fairer games with less stomps, wlr is obviously better than ms, which is practically what the mm uses now that results in stomps.

Thanks for proving you do indeed have no idea how statistics work lol.

[Nightbird]

TheArisen, on 17 June 2019 - 04:49 PM, said:

Well every problem will have various nuance and details to be dealt with but imo I can't see how PGI at least trying out NB's idea here could hurt the game and there's a reasonable chance it'd at least make things better even if it's not perfect.

Obviously there's no reason to blindly go for it but just compare NB's idea to what we have now. IMO it's at least worth further/official PGI testing.

Thanks man. I'm not expecting PGI to give this idea a try, but I do appreciate your comment.

[Brown_Sea_Kraken]

Bumping the single most useful thread on this forum.
If you are reading it for the first time, I suggest to pay attention to OP's post and skip criticism by some members of community cause its inconsistent and derailing. OP has some great ideas.

[MrXanthios]

everything looks fantastic, wlr might work if solo q and group q stats are separate

otherwise if you want to keep things like they are, you have to take in account also avg ms and k/d ratio to have a broader view of one's skills, but that complicates things, I assume, in terms of coding and would probably be less accurate

and also someone already mentioned it, it would probably be better taking in account one's last six months of gameplay and not the entire history

Edited by MrXanthios, 27 June 2019 - 02:51 PM.

[Nightbird]

MrXanthios, on 27 June 2019 - 02:49 PM, said:

everything looks fantastic, wlr might work if solo q and group q stats are separate

otherwise if you want to keep things like they are, you have to take in account also avg ms and k/d ratio to have a broader view of one's skills, but that complicates things, I assume, in terms of coding and would probably be less accurate

and also someone already mentioned it, it would probably be better taking in account one's last six months of gameplay and not the entire history

There are ways to improve upon WLR, absolutely, but it takes more data than PGI releases to simulate it. That been said, with such a dumb, simplified simulation, it's insane much improvement is already possible.

[Nightbird]

Someone said the match maker is better now during the event, is this right? Did they fix it?

[Nightbird]

https://mwomercs.com...u-do-to-the-mm/

No more MM threads

[memorandum]

i just had match in qp, a guy on my team was asking about missiles, this was his 5th game (fresh new player), i'm in tier 1.

i wasn't really paying attention on how long it took to find the match but the matchmaker really shouldn't be putting fresh new players in with tier 1's, like at all.

[Nightbird]

Bump since we haven't had a MM thread for a week

[Nightbird]

People are blaming their team again, when really based on the current match maker... it is them

[TheArisen]

Nightbird, on 14 August 2019 - 04:39 PM, said:

People are blaming their team again, when really based on the current match maker... it is them

Hahaha it's never my fault! I don't make mistakes! Why can't you understand that NB?!

[MischiefSC]

Nightbird, on 15 June 2019 - 03:15 PM, said:

Bad example because regions are not intentionally fixed like divisions are. Some people in region A plays against those in B against those in C against those in A. Another reason it's a bad example is the skill seen in comp play should not be viewed as a systematic difference in skills like you are seeing. The main difference is in the population, the top 12 players in a region with 10,000 players will average a lot higher than a region with 1,000, simply because with a normal distribution of skills, the top will be more standard deviations away from the mean.



Elo is only proven to do what you say in a controlled environment with fixed teams or individuals, fixed match schedules, with fixed opponents. In a random team environment, there is no proof for these claims. If you'd to make some proof, I look forward to reading it,

My suggestion is still to not bother starting with Elo, the assumptions that lead to it working simply don't exist in MWO, better to start from scratch.

How about that Elo is the core of TrueSkill, for a reason. Elo is proven to work in more matchmakers than you can easily count. Like, for example, Overwatch uses an Elo based system for their matchmaking and so does pretty much every competitive online FPS. They may have their own take on a K factor or have additional tweaks based on their unique setting but it's the core of almost every matchmaking system out there. Even stuff like Glicko-2 is based on Elo but has an additional factor to try and measure how reliable someones score is.

At their core though it's all an Elo system. For a reason.

Also there really isn't a 'start from scratch'. All Elo does is exactly what you have to do; it creates a value to identify a players value, then it modifies that values based on wins or losses and it attempts to make that adustment bigger or smaller based on the relative value of who they played against. This is just as relevant in a pug queue as a group queue, for an individual or a team. Like any and all matchmakers it assumes that the value it has currently is incorrect and that the match is NOT perfectly balanced - which is why your score adjusts by varying degrees after the match.

There isn't some new version of Elo you're going to create that does something different, because there isn't something different to do. Every serious matchmaker out there has the Elo equation in some form at its core for the same reason the mean = sum/n. Because if you want to get the mean, you're going to divide sum by n. You want a matchmaker to try and create balanced matches it needs to value each player an arbitrary value then adjust the value up/down based on win/loss and the relative value of who they played. Solo or team or whatever the goal is the same and it starts with exactly what the Elo equation does.

Find a better way to create a matchmaking process and sell it to Microsoft to replace TrueSkill and Overwatch and, well, everyone and you'll be rich and famous.

Until then start with what is already proven to work (as well as anything we know of is going to) and that's Elo. Tweak the result, have variables for how much you trust this or that factor, but the core of it is still exactly the same.

[Nightbird]

MischiefSC, on 14 August 2019 - 06:51 PM, said:

How about that Elo is the core of TrueSkill, for a reason. Elo is proven to work in more matchmakers than you can easily count. Like, for example, Overwatch uses an Elo based system for their matchmaking and so does pretty much every competitive online FPS. They may have their own take on a K factor or have additional tweaks based on their unique setting but it's the core of almost every matchmaking system out there. Even stuff like Glicko-2 is based on Elo but has an additional factor to try and measure how reliable someones score is.

At their core though it's all an Elo system. For a reason.

Also there really isn't a 'start from scratch'. All Elo does is exactly what you have to do; it creates a value to identify a players value, then it modifies that values based on wins or losses and it attempts to make that adustment bigger or smaller based on the relative value of who they played against. This is just as relevant in a pug queue as a group queue, for an individual or a team. Like any and all matchmakers it assumes that the value it has currently is incorrect and that the match is NOT perfectly balanced - which is why your score adjusts by varying degrees after the match.

There isn't some new version of Elo you're going to create that does something different, because there isn't something different to do. Every serious matchmaker out there has the Elo equation in some form at its core for the same reason the mean = sum/n. Because if you want to get the mean, you're going to divide sum by n. You want a matchmaker to try and create balanced matches it needs to value each player an arbitrary value then adjust the value up/down based on win/loss and the relative value of who they played. Solo or team or whatever the goal is the same and it starts with exactly what the Elo equation does.

Find a better way to create a matchmaking process and sell it to Microsoft to replace TrueSkill and Overwatch and, well, everyone and you'll be rich and famous.

Until then start with what is already proven to work (as well as anything we know of is going to) and that's Elo. Tweak the result, have variables for how much you trust this or that factor, but the core of it is still exactly the same.

Then why did PGI abandon Elo?

[Nightbird]

MischiefSC, on 14 August 2019 - 06:51 PM, said:

Overwatch

I did check out Overwatch's leaderboards, and it's no better than MWO's in terms of lopsided WLR for top players. I don't play it though, can anyone say that stomps happen less often in Overwatch, and most matches feel close?

Results usually say all I need to know about the MM though, if people are maintaining 3/1 WLR across thousands of games, the MM isn't taking their skill into consideration when making matches. As an analogy, if a car breaks down every 5 miles, any argument about how reliability the engine is designed to be is rendered moot.

[MischiefSC]

Nightbird, on 14 August 2019 - 08:02 PM, said:

Then why did PGI abandon Elo?

Nightbird, on 15 August 2019 - 08:57 PM, said:

.... Spl needs?

I did check out Overwatch's leaderboards, and it's no better than MWO's in terms of lopsided WLR for top players. I don't play it though, can anyone say that stomps happen less often in Overwatch, and most matches feel close?

Results usually say all I need to know about the MM though, if people are maintaining 3/1 WLR across thousands of games, the MM isn't taking their skill into consideration when making matches. As an analogy, if a car breaks down every 5 miles, any argument about how reliability the engine is designed to be is rendered moot.

You're conflating issues with human skill curve in a constantly active environment with matchmaker issues.

If the Olympics ran 24x7 and people dropped in and out as they wanted you'd have the same issue.

In a PvP computer game environment there is inevitably going to be outliers. That's a product of the human skill curve - it's more of a mesa. Vast majority are average, tiny percentage good, handful excellent. Yet the good and excellent players drop just as much or more often than the sea of mediocrity. There is no effective way to guarantee that in any given match creation window you have the means to create a balanced match. The idea that stacking bads with 1 good player gives a team that's equal to all mediocre. Skill doesn't work that way, because the bads are on as uneven a slope as the goods are.

Also, human development curve. If the vast majority of players are mediocre and you're only a bit above mediocre, that vast majority doesn't actually improve at the same rate because people who put in the effort to get above average will improve far faster rate than the average players, the majority of whom won't improve at all.

In a online he environment w/l is as much about "rate of improvement/speed of adaption to change" as it is raw skill.

The environment is one that will always inherently he unbalanced. So the goal is to mitigate that as much as possible but, again, there's only so much you can do to account for outliers and still have them able to play matches in a timely manner.

[Nightbird]

MischiefSC, on 16 August 2019 - 09:51 AM, said:

You're conflating issues with human skill curve in a constantly active environment with matchmaker issues.

If the Olympics ran 24x7 and people dropped in and out as they wanted you'd have the same issue.

In a PvP computer game environment there is inevitably going to be outliers. That's a product of the human skill curve - it's more of a mesa. Vast majority are average, tiny percentage good, handful excellent. Yet the good and excellent players drop just as much or more often than the sea of mediocrity. There is no effective way to guarantee that in any given match creation window you have the means to create a balanced match. The idea that stacking bads with 1 good player gives a team that's equal to all mediocre. Skill doesn't work that way, because the bads are on as uneven a slope as the goods are.

Also, human development curve. If the vast majority of players are mediocre and you're only a bit above mediocre, that vast majority doesn't actually improve at the same rate because people who put in the effort to get above average will improve far faster rate than the average players, the majority of whom won't improve at all.

In a online he environment w/l is as much about "rate of improvement/speed of adaption to change" as it is raw skill.

The environment is one that will always inherently he unbalanced. So the goal is to mitigate that as much as possible but, again, there's only so much you can do to account for outliers and still have them able to play matches in a timely manner.

This is something I disproved with the first post. I even created players with skills based on the bell curve so only a few people are outliers etc etc

[Koniving]

The6thMessenger, on 08 June 2019 - 08:59 PM, said:

If only PGI actually listens to the community.

You have to remember, we "don't know anything about game development."
Yet we have amateur animators that animate better than PGI does..
Stat-magicians that do (as far as I can tell) pretty good analysis.
People that can design better looking levels with a lot of potential all over the map that aren't using the "we must force everyone to congregate in 2 to 3 areas in order to create the most monotonous play possible" method of PGI.
People that can work out actual scales without using magic to determine mech sizes.
People that can tweak the cfg file to create a far more beautiful game (Lordred of Perfect Screenshot fame; after page 30 or so it was pure cfg file best the game could actually look without ENBs/post-editing and the game still looks far better than MWO does natively).
People that can design far better hitboxes to make mechs feel unique and work with the unpleasant geometry to create survivable mechs even when grossly oversized (CarrionCrow).

But yeah.. We don't know jack, apparently.

[Koniving]

Spoiler

Quote

We should be able to extend our arms.

Koniving, on 16 August 2019 - 10:44 AM, said:

I worked out a concept for it a couple years ago, even exactly how to control it, and how to have the extended arms work with aiming extreme left/right and how 'mixed' actuator arrangements would work.

Typical mech position.



Press Q, the mech head (if moveable; mind you this was for a Battletech simulator) and both arms aim left, as far as possible and as applicable. This assumes full actuators.

If the opposing arm cannot reach the intended targeted area, the weapons on that arm simply disable to prevent wasted heat. The torso weapons however would not lock out, as being able to use both the torso and arm crosshairs independently to attack multiple targets at once is a valuable, if niche, skill.


Then here's Press E, but to save time this one demonstrates the opposing arm is only upper actuator as PGI has defined them.

Both head and applicable arms aim to the right. The opposing arm in this case would be arm-locked to the torso crosshair, so as to bolster the firepower potential of the torso weapons should the pilot prefer to use them. It would be ideal if the player modified the firing groups accordingly.

Press both Q and E together, and both arms do this. (Watch the Atlas)

Simple, straight forward design.
For MWO, remove head movement of the mech and just put in head movement of the pilot instead.
Example of how it might look.

Just add in being able to see the arms raised up to shoulder height.

Yep.
We have no idea what we're doing.

[MischiefSC]

Nightbird, on 16 August 2019 - 10:06 AM, said:

This is something I disproved with the first post. I even created players with skills based on the bell curve so only a few people are outliers etc etc

I get the logic of what you're trying to do but bell curves for human skill don't work super well. It helps, sure, but it doesn't work super well.

This is my wheelhouse; it's what I do in analytics, analyze both employee and customer performance and associated behaviors. Ironically that's where I got into gaming matchmakers because the challenges are extremely similar.

Think of what drives win/loss as being a 200 piece jigsaw puzzle. Most people only get like 50 pieces, ever. As someone collects more pieces he can make more of the puzzle. The best players may have 150 pieces -

The problem is that the vast majority of people have the same 50 pieces. 100 players each with a 1.1 W/L probably all have almost the exact same strengths and weaknesses, with tiny variation. The idea that you can build a low skill team with a couple high skill players to make a team that matches a bunch of mediocre players isn't really accurate because there's no guarantee the skill deficiencies in the lowbies are offset by that good players strengths.

What you're talking about really falls under a StrengthFinders umbrella; trying to slice performance up into specific strengths and weaknesses to then build teams toward a set performance goal based on stacking strengths and weaknesses. It's possible but hugely time intensive and in a setting like this not that reliable.

Your matchmaker idea is still great. I just want to be clear that any matchmaker for this sort of environment is going to have outliers, because humans and everyone needs to still play matches even when a good matchup isn't available.

However your idea is still going to get us better results than we have currently, and likely by a good margin. Just need to set the expectation that both ends of the curve are going to be skewed.