76 replies to this topic

[ScrapIron Prime]

I have two basic questions at this point.

1 - whether the matchmaker was better before they rolled groups into the solo queue. To my recollection, it was about the same as it is now with regard to wins/losses, but I recall distribution of weight classes being better.

2 - whether there are a significantly higher percentage of the population with alt-accounts playing at a higher proficiency than the tier would indicate. That... I only have gut feeling on, and gut feeling is a bad way to write an algorithm.

You can resolve the first with a formula change. Build the match around two groups and balance weight classes after seeding the groups, assuming you can find enough light mech pilots in queue and assuming that this doesn't make the weight time for one class astronomically long (like everyone piloting a Timby during the free mech event).

The second... yeah. How to get data on alts? You could do it by tracking the source IP of the player and building a table out of that, which would work for everything except shared internet connections (like gaming cafes and the like). Not a good assumption, though.

[Nightbird]

Vlad Ward, on 04 June 2021 - 09:09 AM, said:

What would be next on your list if PGI is unable to hire one of those people?

There are plenty of other methods, but they all involve some math. The Cauldron can find somebody I guess.

[Vlad Ward]

Nightbird, on 04 June 2021 - 09:41 AM, said:

There are plenty of other methods, but they all involve some math. The Cauldron can find somebody I guess.

I mean, if we're talking about using ML to build out a predictive MMR that's able to accurately identify potential smurfs with high recall and precision it's a little more involved than 'some math.' Technically possible but far and away beyond anything The Cauldron could provide on their own or that PGI has the resources to outsource unless there are secret Data Scientists on the team that are willing to work for free.

[Nightbird]

Vlad Ward, on 04 June 2021 - 01:42 PM, said:

I mean, if we're talking about using ML to build out a predictive MMR that's able to accurately identify potential smurfs with high recall and precision it's a little more involved than 'some math.' Technically possible but far and away beyond anything The Cauldron could provide on their own or that PGI has the resources to outsource unless there are secret Data Scientists on the team that are willing to work for free.

To me it's just some math. If the Cauldron can't fix it then we just accept the high attrition rate as we have for years.

[Vlad Ward]

Nightbird, on 04 June 2021 - 01:45 PM, said:

To me it's just some math. If the Cauldron can't fix it then we just accept the high attrition rate as we have for years.

I mean, are you volunteering? I'm not saying DS don't exist. I'm saying they're expensive. There are surely going to be less expensive solutions that can still have a significant positive impact.

[Nightbird]

Vlad Ward, on 04 June 2021 - 01:53 PM, said:

I mean, are you volunteering? I'm not saying DS don't exist. I'm saying they're expensive. There are surely going to be less expensive solutions that can still have a significant positive impact.

My solutions will not be accepted by the Cauldron, look at the first post.

[Vlad Ward]

Nightbird, on 04 June 2021 - 01:56 PM, said:

My solutions will not be accepted by the Cauldron, look at the first post.

Maybe not. Idk. I'm not one of them.

I might be missing it, but I don't see a link or a summary of your proposal in the OP. I remember reading it at one point, but I have no idea where it is in the forum now.

[Nightbird]

It's nothing impressive, it was just using WLR to determine PSR as compared to Jay Z's formula we're using today. It was designed to be both easy to understand and more effective, as was shown in the simulated results.

[Nightbird]

Going on vacation tomorrow so dropping in this month's analysis quickly.



It looks like retention rate took a dip in June, so what was the cause? Are people happy with MW5, or unhappy with MWO, or something else?



Before trying to attribute micro causes, I first look at the macro (global) numbers, which peak concurrent steam users provides a good picture of. From Jan 2020 (pre-Covid) to Apr 2021, steam numbers almost doubled. In May, June (not in chart yet), perhaps due to COVID lockdown ending, Steam numbers dipped hard.



As you can see, MWO numbers have moved in lock step with Steam numbers the whole time. There's not sufficient evidence that people loved MW5 or hated Cauldron changes to a degree that impacted the game pop.

Over and out.

Edited by Nightbird, 01 July 2021 - 06:33 AM.

[Dogstar]

Well that's kind of a relief, fewer people playing games overall means fewer people playing MWO, and there's no major reason other than that for any drop in population.

Question - is there any way to gather any other statistics? e.g. number of mechs owned from the profile screen?

Edited by Dogstar, 02 July 2021 - 11:52 AM.

[Nightbird]

There isn't really any pressure need to explore other sources of population. Most people who play MWO play QP, with only a small group in FP, Solaris, or in private lobby exclusively.

If the Cauldron's updates improve play experience, we may yet see MWO's numbers break away from the global trend of fewer players, but I don't expect it personally as my analysis continue to show matchmaking being far more important than weapon or mech balance to retention. The Cauldron driven changes are well intended but not expected to be impactful.

[w0qj]

Oh wait, so are you suggesting that perhaps folks doing work-from-home sometimes goes on Steam occasionally?

Nightbird, on 01 July 2021 - 06:33 AM, said:

...Before trying to attribute micro causes, I first look at the macro (global) numbers, which peak concurrent steam users provides a good picture of. From Jan 2020 (pre-Covid) to Apr 2021, steam numbers almost doubled. In May, June (not in chart yet), perhaps due to COVID lockdown ending, Steam numbers dipped hard.

[Nightbird]

Or laid off and living on unemployment benefits.

[Nightbird]

Still no changes to retention rates, from new events, Cauldron updates, or streamer events. I'll still argue that while some people have benefited from these, most people don't care and that is why results are very muted.

[VeeOt Dragon]

nice to see all the hard work OP

honestly i do think we ned an improvement to the MM or at least remove Groups from QP since the Current MM doesn't take into account the Tier or mechs hence we get our current situation. i'm no programmer so i couldn't tell you how to do it.

as for cash i can name one quick way to get some cash influx from new players (or old players for that matter). all they would need to do is give a nice big 50% or more discount on older mech packs. if you look all packs are mostly C-bill mechs with 2 or 3 that are MC only. if you cut the cost of the older packs down to just what it would cost for those MC only mechs and just let the others be a bonus for spending real money they would be more likely to sell. the cost of mech packs is ridiculous, you pay the cost of a full AAA title (ok calling most modern AAA titles full games is a bit ridiculous in itself). even most DLC only cost $10-$20. yeah they wont get as much per pack but something is better than nothing at all. (then again i am one of those folks that have never paid a dime for the game just because of those insane prices)

[Nightbird]

Thanks! While I would like to see groups gone from QP as well, that is pretty unlikely. I would at least like the unfair advantage given to groups of good pilots removed from the matchmaker.

As for the cost of the game, it seems that PGI understands the prices are too high given the new packs are just 20$. It remains to be seen if they understand the business benefit of droping the prices of existing packs by 50% permanently (and going to 75% off for sales).

[Nightbird]

Added the last season to the graph:



I also figured out why the WLR > 1.3 group has the second worst retention rate. There is no match maker for this group due to the PSR cap of 5000, let me explain:

To illustrate, suppose in queue there are 2 groups of 4=8 players with 5000 PSR, 8 players with 4000 PSR, and 8 players with 3000 PSR. Suppose that one group of 4 with 5000 PSR has 1.3 WLR, and the other group of 4 with 5000 PSR has 5 WLR. Then the current match maker will create:

Team 1: 4 players with 5000 PSR (1.3 WLR), 4 players with 4000 PSR, 4 players with 3000 PSR
Team 2: 4 players with 5000 PSR (5.0 WLR), 4 players with 4000 PSR, 4 players with 3000 PSR

The 5000 PSR cap means that the match maker can't separate the barely T1 players against great T1 players. When these two teams meet, Team 2 will win 95% of the time, and 50% will be stomps. The group of players with 1.3 WLR and 5000 PSR repeatedly faces this situation and become frustrated with the game.

Instead, if the match maker were to do this:
Team 1: 4 players with 5000 PSR (1.3 WLR), 8 players with 4000 PSR
Team 2: 4 players with 5000 PSR (5.0 WLR), 8 players with 3000 PSR
It would create a more even match. even though Team 2 would still win most of the time, it would be closer to 60% of the time, and only 10% of games would be stomps.

In other words, people who win more are suddenly given impossible matches by the match maker and stop playing.

Edited by Nightbird, 03 September 2021 - 08:46 AM.

[Black Caiman]

If only matchmaker had that capability. Teams of all 99%ers will continuously be the great destabilizer unless it is modified to understand the differences between player skill within a given tier, and be able to balance team composition to account for this. I understand the argument of "I just want to play with my friends," and am totally for it. Im just always amazed some people apparently have 0 in game "friends" that arent in the top 1-2%. Almost my entire unit falls outside of that category, but I drop with them because "I just want to play with my friends." I do drop with some very, very good players as well, but I have "friends" in game that range from Tier 5 all the way to the top of Tier 1, and will drop with any of them who want to roll.

Also, I fall into that WLR > 1.3 category, and agree that it can be frustrating falling into the situation you have mentioned above. The way I handle it is to get myself and my group to take up the challenge of trying to hand a loss to said WLR > 5.0 group. Does it always happen? Nope. Does it happen sometimes though? Sure, and it is significantly more rewarding when it does.

Edited by Black Caiman, 03 September 2021 - 08:46 AM.

[Nightbird]

Black Caiman, on 03 September 2021 - 08:41 AM, said:

Also, I fall into that WLR > 1.3 category, and agree that it can be frustrating falling into the situation you have mentioned above. The way I handle it is to get myself and my group to take up the challenge of trying to hand a loss to said WLR > 5.0 group. Does it always happen? Nope. Does it happen sometimes though? Sure, and it is significantly more rewarding when it does.

As a die-hard player in this game, I am in the same boat. However the graphs show without the bias of anecdotes or personal opinions what is actually happening within the larger population. It's not really an option to talk to all these people leaving, to say, "hey stick around, it's more rewarding if you win". The easy solution remains to fix the PSR match maker.

[Davegt27]

just copied this from Jarl's

not sure how this translates to KDR but I look at retired players



so what happened when player numbers started to drop it started with players with the low match score, they started leaving the game



then it moved on to players with higher match score

Edited by Davegt27, 03 September 2021 - 06:15 PM.