138 replies to this topic

[Nightbird]

Continued from post 16,

I claimed that the teams will be worse balanced in 8v8, what is my defense of this claim? There are two ways to go about this, the intuitive explanation and the mathematical explanation. I'll do the intuitive way first, and the math afterwards.

Suppose a match is made within a tier, and in this tier the players are organized into skill levels 1 thru 100 averaging 50. Because the tiers are pretty wide, a skill 1 player stands no chance against a skill 100 player despite being in the same tier. I hope this part is obvious. Now we want to create two teams to play against each other. For the match to be fun, let's say the average skill of team A needs to be within 10 points of team B. For example, if team A averages 50 skill, team B needs to be 40-60 on average for a good match to happen.

Ok, now let's see how team size plays into this. Suppose we do teams of 1 player, 1vs1, and the first player has skill 55. The MM doesn't look at skill, so it randomly picks a player from skill 1-100. The odds of a good match, picking a player that is 45-65, is about 20%.

Now, suppose we have teams of 1,000 vs 1,000. The chances of a team averaging over 55 or under 45 is slim to none. Just as flipping a coin and getting 3 heads is not rare, but 100 heads in a row is not likely. For the point of this argument, the chances of a good match is 100%.

I hope with these two extremes of team size that it is intuitively clear the more players in a team, the closer their average skill is to 50, and the likelier the chances of a good match.

To be continued...

[Nightbird]

...continued from last post.

The mathematical way of showing why 8v8 will result in worse matches with the current match maker: We have a distribution of the true player skill within the MWO population (we don't know what it is) and this distribution has a variance (an expression of how wide the skill gap is, we don't know this either).

In 12 versus 12, team A vs team B, the difference in the skill between the two teams can be expressed as Squareroot(variance/12 +variance/12) = 1/Quality_of_matchs_12v12_today

If you want to know where the formula comes from, take any intro stats course. The relation is inverse because greater the difference in skill between the two teams, the worse the quality of matches.

We don't have the variance to plug into this equation, but we can use this to find what the quality of matches in 8v8 will be:

1/Quality_of_matchs_8v8=
Squareroot(variance/8 +variance/8) =Squareroot[3/2*(variance/12 +variance/12) ]=
Squareroot(3/2)*Squareroot(variance/12 +variance/12) =Squareroot(3/2)*1/Quality_of_matchs_12v12_today
>>
Quality_of_matchs_8v8=0.82*Quality_of_matchs_12v12_today

In other words, in 8v8, there is going to be 20% more variance between average team scores, or 20% worse matches than today. If today you find 50% of matches to be good, and 50% to be bad, in 8v8 you'll find 40% of matches to be good (20% less), and 60% to be bad.

The conclusion is, if you want 8v8 to get better frame rates at the expense of worse matches, go for it. If you want 8v8 for better matches though, take your finger off the gun pointed at your foot

to be continued...

Edited by Nightbird, 08 June 2018 - 06:49 PM.

[Nightbird]

...continued from last post

Why is it clear from Solaris that people want balanced play more than having a 'mode' of play? For one, Solaris is a new mode. For another, think of what people say about Faction Play, and the reality that by comparing the Faction Play war log and the Solaris queue numbers, it is clear there are fewer people playing Solaris than Faction Play.

So what are the problems with Solaris? From reading the various threads regarding it, the two main ones are 1) there are too many buckets (14 divisions, for solo and group) that split players meaning there isn't enough people in one bucket to make good matches based on Elo, and 2) the mechs in each division offer only a few mechs and load-outs that are competitive, if you don't choose those you're playing with two arms tied behind your back. Elo assumes that both players start with the same pieces, it doesn't apply to MW.

This is because Mechwarrior has always been about the pilot and the mech, not just the pilot. There are no pilots skilled enough that can make any mech stand a chance against any other mech. Ball-parking mechs into 7 divisions doesn't change this, and suggestions about making divisions based on mech weight class wouldn't change this either. The reality is every mech and every loadout has a significant modifier on the effectiveness of a pilot... depending on the opponent they are facing. Any MM that doesn't take this into consideration will not create equitable matchs and therefore forces everyone into the few top mechs and loadouts and otherwise quit Solaris altogether.

How would a new MM benefit Solaris? (Obviously dynamic team numbers doesn't apply here) Quite simply because in addition to pilot skill, not only can a statistical model assign a value to his mech and loadout, the model can also assign a modifier based on his opponent's mech and loadout. A Streak Maddog will be given a tremendous value against a laser/MG Firestarter, and a pitiful value against an AC20/SRM Atlas. The performance values of all the pilots with their mechs in queue can be evaluated based on all the possible opponents and their mechs, and fair matches made. A less skilled pilot with a stronger mech can be matched against a better pilot with a weaker mech. The best pilots in the best mechs that do not have an appropriate opponent will simply sit in queue until a worthy opponent arrives, or until they switch to a less strong mech choice to intentionally lower their total value.

The benefits to this MM are: 1) all the queues can be combined into one (one solo and one 2-man group) for better wait times AND better matches, and 2) you can bring any mech and any loadout you want, and be paired with a fair opponent.

to be continued...

Edited by Nightbird, 09 June 2018 - 11:51 AM.

[Nightbird]

Well, would this improve MWO for everyone more or is some other feature more beneficial?

Does anyone want to disagree with my Solaris or 8v8 analysis?

[LordNothing]

it seems to me psr is more to cater to snowflakes than anything else. its there to artificially stroke your ego and make you feel special. problem is everyone bubbles up to t1 when they finally put away their childish things and bring builds that are good. once your there the game is defacto random, because t1s seem to be the bulk of the player base.

the only thing psr really does is buffer new players from the experienced. and by new players what i really mean is alt acounts made by t1s who really didnt want to be there. if we really want to buffer new players we should just give them their safe space while they are getting their cadet bonuses and throw them into the shark pit they are going to end up in anyway. now if psr actually works as it appears it should work (actual skill levels with no free unearned bubble up), then people in general would have better games, players who enjoy competitive play can have t1, players who just want to have fun can have t3 and taters and noobs can have t5. even number tiers become transitionary for players who are looking to cross the gap. the important thing is it actually represents skill and not the harder to quantify experience it tries to be.

even then its still kind of crappy. population isnt there to really justify 5 skill levels. bringing it down to 3 might help, casual, skilled and elite tiers. either with no bubble up tendency or a bubble towards middle tier.

[El Bandito]

Nightbird, on 09 June 2018 - 06:34 PM, said:

Well, would this improve MWO for everyone more or is some other feature more beneficial?

Does anyone want to disagree with my Solaris or 8v8 analysis?

I disagree. Assuming MM doesn't completely disregard tiers, it is far easier to group 16 people together from the currently limited player pool, than to group 24 players. More players per team = more wait time for MM to gather players = more chance of MM loosening its parameters and fill in one side with players far below/above the curve. Which will result in more lopsided play.

There are tons of other variables such as loadouts (of which there are many thousands), to completely screw over matchmaking so I doubt 8v8 MM will be in anyway worse than 12v12 MM. In fact, I have not experienced any positive MM improvement ever since they switched to 12v12--with the exception of groups no longer dropping in SQ. I'll take my reduced wait time, and increased frame rate instead, thank you very much. Especially during off hours.

Edited by El Bandito, 09 June 2018 - 08:36 PM.

[Mystere]

El Bandito, on 06 June 2018 - 10:16 PM, said:

Dynamic team number is a nightmare to balance. Not even big multiplayer arena game companies dare to do it. PGI certainly can't.

War Thunder does not seem to have a problem deploying still "incomplete" teams when its matchmaker decides to do so. Heck, they don't even seem to have much of a problem deploying asymmetric teams during special events (i.e. Battle of Britain commemoration).

Thus, I am sure such a thing is nowhere close to being rocket science.

El Bandito, on 06 June 2018 - 10:16 PM, said:

As for reverting back to 8v8 in QP, I'm all for it. It has more pros than cons.

And I am against it. Regression is not a good thing. We might as well just shut down the servers.

[El Bandito]

Mystere, on 09 June 2018 - 09:01 PM, said:

And I am against it. Regression is not a good thing. We might as well just shut down the servers.

If you wanna talk about regression, then the implementation of 12v12 had already regressed framerates, MM wait times, and in game graphics, compared to 8v8 days. Ha!

Mystere, on 09 June 2018 - 09:01 PM, said:

War Thunder does not seem to have a problem deploying still "incomplete" teams when its matchmaker decides to do so. Heck, they don't even seem to have much of a problem deploying asymmetric teams during special events (i.e. Battle of Britain commemoration).

Thus, I am sure such a thing is nowhere close to being rocket science.

Nah, there is no way PGI can pull it off. They can't even pull off balanced 12v12 matches.

Edited by El Bandito, 09 June 2018 - 09:10 PM.

[50 50]

El Bandito, on 06 June 2018 - 10:16 PM, said:

Dynamic team number is a nightmare to balance. Not even big multiplayer arena game companies dare to do it. PGI certainly can't. As for reverting back to 8v8 in QP, I'm all for it. It has more pros than cons.

Why would it be a nightmare to balance?
I would have thought that dynamic team sizes for Faction Play would work spectacularly well as it doesn't try to balance by tier at all and the drop decks means there is no limitation on the mechs by weight class (no 1/1/1/1 etc).
For QP Group queue, the balance is on the group as a whole to meet a tonnage limit and it has the tier matching as well. In this queue it would seem that the biggest benefit would be reduced wait times by allowing smaller team sizes and then potentially more even matching between the teams as it should be easier to match by tier with less numbers.
Solo QP I would seriously doubt it has any impact on, but should there be a bit of fluctuation in population in that queue, matching smaller teams together should work exactly as it does now.

The added bonus to doing something about wait times is that smaller team sizes create a different game experience.
It's instant variety to every single mode.

Can a team get rolled easier? Maybe. But that happens now in all modes so what does it matter if you can click to play and get into a new game in 30 seconds.

I'm not going to clog up your thread here but I can see some massive benefits to moving to dynamic team sizes for Faction Play beyond the wait time issue.

SFC174, on 07 June 2018 - 04:55 AM, said:

A good post OP, and logical. It's hard to disagree with anything you wrote. There are some semantics to hash out maybe, but....

Anyways, the only issue here is PGI. And while dynamic team sizing does make sense in times of low pop, like others, I just don't see it happening. Not only do I see it being a major technical hurdle for PGI to do right, but going to smaller team sizes is going to require mode/map changes in concert to make things work right.

For example, if you get stuck with Escort, how much do you reduce the health of the VIP? (shouldn't be that hard, but how long will it take to figure it out?)

Or if you go down to 6v6 do you just scrap certain modes like Conquest where 6v6 might just result in no combat at all?

That might actually be the answer - if you dynamically downsize, the smaller the teams the fewer maps/modes available. Skirmish or Domination work with almost any team size. Other modes less so.

Surely it can't be that difficult to implement.
The functionality is there in the private lobby to allow any number of players on either side.
Wouldn't it just be an adaption of that?.
I would seriously avoid odd team sizes however and focus directly on lances of 4 players as the building block.

As for the maps and modes, I don't really see the need to change these, at least not right away.
Given that in quick play it generally boils down to beating the snot out of each other and then going for objectives it may not make much difference.
It is good to work through the different match modes and find potential issues and probably from a 4v4 perspective as the lowest possible match size.

This does mean limiting group size to 4

The reason group size has to be limited is to make sure you don't get left behind if there are not enough players for an opposition.
But there is a bit of a side benefit to doing so.
We could get Scouting merged into the QP and FP queues.
This instantly adds a new mode for QP which is nice.
For FP it means removing a queue and thus helps to combine the player base.
It may even mean that recombining solo and group queue in quick play could be a consideration or at least an option to select there.

The reason to base it around 4 is the whole Lance structure that makes up the composition of a full company.
We drop in lances.
The drop ship holds a lance.
etc.

[El Bandito]

50 50, on 09 June 2018 - 10:31 PM, said:

Why would it be a nightmare to balance?
I would have thought that dynamic team sizes for Faction Play would work spectacularly well as it doesn't try to balance by tier at all and the drop decks means there is no limitation on the mechs by weight class (no 1/1/1/1 etc).
For QP Group queue, the balance is on the group as a whole to meet a tonnage limit and it has the tier matching as well. In this queue it would seem that the biggest benefit would be reduced wait times by allowing smaller team sizes and then potentially more even matching between the teams as it should be easier to match by tier with less numbers.
Solo QP I would seriously doubt it has any impact on, but should there be a bit of fluctuation in population in that queue, matching smaller teams together should work exactly as it does now.

The added bonus to doing something about wait times is that smaller team sizes create a different game experience.
It's instant variety to every single mode.

Can a team get rolled easier? Maybe. But that happens now in all modes so what does it matter if you can click to play and get into a new game in 30 seconds.

I'm not going to clog up your thread here but I can see some massive benefits to moving to dynamic team sizes for Faction Play beyond the wait time issue.

If you want smaller teams to play faster, then sure bringing the queue down to 8v8 in FP/GQ is fine (in fact PGI already set up a poll regarding 8v8 implementation already), but the number of participants MUST be the same for both sides for it to have a semblance of balance. Any lower than that, you might as well play Scouting. Of course, Invasion maps in particular are heavily balanced towards defenders, but that's map issue that can be fixed without having different number of participants per team.

Most importantly, dynamic sized teams means PGI will have to fiddle with dynamic match reward numbers--something they REALLY do not wish to get themselves tangled with.

[Nightbird]

El Bandito, on 09 June 2018 - 11:04 PM, said:

but the number of participants MUST be the same for both sides for it to have a semblance of balance.

Keep in mind that asymmetric team numbers would only be created if a group is too strong for any equal numbered team to face that's currently in queue. When this is the case, the numbers advantage would only be enough to give both teams a fair chance at winning. This can be computed mathematically.

[El Bandito]

Nightbird, on 09 June 2018 - 11:09 PM, said:

Keep in mind that asymmetric team numbers would only be created if a group is too strong for any equal numbered team to face that's currently in queue. When this is the case, the numbers advantage would only be enough to give both teams a fair chance at winning. This can be computed mathematically.

How would the calculation be done then? With just how much confidence can you say the calculations would be anywhere close to accuracy, when factoring in things such as mechs, builds, individual skill--which in itself is arbitrary, and unit competence? There are units that are very strong when together, but individually they are nothing special. There are also units where skill gap between players are very wide. Mercstar doing their Crab rush in GQ/FP is totally different beast than when they bring in regular mechs. What are the chance of a company like PGI to come up with such dynamic MM, algorithm that is actually guaranteed to work, when they are busy with other projects?

Edited by El Bandito, 09 June 2018 - 11:23 PM.

[Nightbird]

So if it could be done right, you wouldn't have a problem with it? I don't want to go down the PGI bashing road here, I get that it's hard, but if it is done it would make the game a lot better and more so than a new map or new mech pack would.

As for whether it can be done, of course it can! I do it for a living, sort of, since I don't work in the gaming industry. I analyze how medication like cancer drugs affect the patient. The data analyzed has factors like age, race, sex, bmi, hundreds of lab tests, on top of tumor size over time, and things you don't want such as adverse events, all across hundreds or thousands of test subjects. I have to make a simple determination, is Drug A better, equal, or worse than Drug B? For a game, it would be the opposite in that you can to assign people to team A and B such that on average the two are equal.

PGI needs to find someone who can do that within the gaming industry, and crunch their data. It's not hard, and actually implementing it shouldn't take a programmer more than 2 months... after they have the formulas. Afterwards, just re-crunch the data once a week to account for meta changes.

Edited by Nightbird, 09 June 2018 - 11:46 PM.

[El Bandito]

Nightbird, on 09 June 2018 - 11:41 PM, said:

So if it could be done right, you wouldn't have a problem with it? I don't want to go down the PGI bashing road here, I get that it's hard, but if it is done it would make the game a lot better and more so than a new map or new mech pack would.

As for whether it can be done, of course it can! I do it for a living, sort of, since I don't work in the gaming industry. I analyze how medication like cancer drugs affect the patient. The data analyzed has factors like age, race, sex, bmi, hundreds of lab tests, on top of tumor size over time, and things you don't want such as adverse events, all across hundreds or thousands of test subjects. I have to make a simple determination, is Drug A better, equal, or worse than Drug B? For a game, it would be the opposite in that you can to assign people to team A and B such that on average the two are equal.

PGI needs to find someone who can do that within the gaming industry, and crunch their data. It's not hard, and actually implementing it shouldn't take a programmer more than 2 months... after they have the formulas. Afterwards, just re-crunch the data once a week to account for meta changes.

Still easier said than done, otherwise there would have been plenty of MP only games that have different players per team--which is not the case. Judging by the fact that PGI is still willing to retain this upwards biased PSR system, even after all the feedbacks about it, what makes you think they would be even interested in spending time and resources on it in the first place, rather than the next mech pack? Can you actually quantify the benefits PGI get from such a system vs. the cost, so that there are actually presentable numbers Russ can look at?

Edited by El Bandito, 09 June 2018 - 11:52 PM.

[Nightbird]

El Bandito, on 09 June 2018 - 11:50 PM, said:

what makes you think they would be even interested in spending time and resources on it in the first place, rather than the next mech pack?

I know I said that, but it wouldn't be the art team, but rather the team that put together Solaris.

El Bandito, on 09 June 2018 - 11:50 PM, said:

Can you actually quantify the benefits PGI get from such a system vs. the cost, so that there are actually presentable numbers Russ can look at?

I would assume they know this is a major problem, they made the effort to move to the PSR system after all. As such, I don't feel any need to convince them that it is beneficial, I only need to convince them that it is possible. It's more a, can't choose a path that you don't know exists deal. Once they become aware of it, and do their own research into it, it'll be implemented because it's a better solution to a problem they desperately want to solve.

Any comments on one queue Solaris?

Edited by Nightbird, 10 June 2018 - 07:19 PM.

[El Bandito]

Nightbird, on 10 June 2018 - 12:05 AM, said:

I know I said that, but it wouldn't be the art team, but rather the team that put together Solaris.

PGI will still need to allocate resources for them.

Nightbird, on 10 June 2018 - 12:05 AM, said:

I would assume they know this is a major problem, they made the effort to move to the PSR system after all. As such, I feel the need to convince them that it is beneficial, I only need to convince them that it is possible. It's more a, can't choose a path that you don't know exists deal. Once they become aware of it, and do their own research into it, it'll be implemented because it's a better solution to a problem they desperately want to solve.

Not sure about that, since they wouldn't even be bothered to remove the upwards bias this PSR system has, which could have improved match quality with minimal cost.

[Nightbird]

El Bandito, on 10 June 2018 - 12:08 AM, said:

Not sure about that, since they wouldn't even be bothered to remove the upwards bias this PSR system has, which could have improved match quality with minimal cost.

It's because player skill has a normal distribution, you can see it from the Jarl's average match score bar chart: https://leaderboard.isengrim.org/stats

PGI made a mistake seeking to push 20% of the population into each Tier, when statistically about 50% of the population should be in T3, 20% in T2, T4 respectively, and 5% in T1 and T5. This is a consequence of the MM they selected, which the suggested MM does not need.

[50 50]

El Bandito, on 09 June 2018 - 11:04 PM, said:

If you want smaller teams to play faster, then sure bringing the queue down to 8v8 in FP/GQ is fine (in fact PGI already set up a poll regarding 8v8 implementation already), but the number of participants MUST be the same for both sides for it to have a semblance of balance. Any lower than that, you might as well play Scouting. Of course, Invasion maps in particular are heavily balanced towards defenders, but that's map issue that can be fixed without having different number of participants per team.

Most importantly, dynamic sized teams means PGI will have to fiddle with dynamic match reward numbers--something they REALLY do not wish to get themselves tangled with.

Ah, ok. Missed that bit.
Given all the effort gone to 'balancing' the mechs and tech I wouldn't agree with having teams of different sizes.
I think that even trying to work out some sort of balance to cover it, it's just too much to expect.

I'd be really interested to see how the modes and maps play out just as they are now with a smaller team size.
It would change the dynamics a fair bit and in my mind, that variety in the game play is good.

[Nightbird]

50 50, on 10 June 2018 - 02:59 AM, said:

Given all the effort gone to 'balancing' the mechs and tech I wouldn't agree with having teams of different sizes.
I think that even trying to work out some sort of balance to cover it, it's just too much to expect.

Without dynamic team sizes, balance between teams is simply impossible because a group might not only be too strong for the second best players in queue to face, but they are made to face worse and even novice teams because they're a better size to fit together for a 12v12. That much stomping is bad for MWO.

Edited by Nightbird, 10 June 2018 - 08:37 AM.

[Nightbird]

Maybe if I asked a different question

a ) Suppose you're dropping in a strong team of 8 winning most of your matches in 12vs12 by rolling your opponents 12 to 0/1/2. Would you enjoy a scenario where the MM created a 10vs14 match to challenge you?

b ) Suppose you're in a group of intermediate pilots, losing many matches 0/1/2-12 against the strong teams that roam QP GQ. Would you enjoy a scenario where the MM took that into consideration and put your group into a reinforced group of 15 if your opponent is a strong team of 9?

Edited by Nightbird, 10 June 2018 - 07:07 PM.