37 replies to this topic

[Nightbird]

There was some confusion the last time I posted this, so to define: here, the retention rate is the % of players in a season's leaderboard that appears again in the next leaderboard. It can never be over 100%, since at most every player comes back.

The reason this number is important is because it breaks down the change in every season's population into the number of new players versus the number of departing players. If one season had 100 players and the next season had 105 players, a retention rate of 80% tells us that we lost 20 old players and gained 25 new players.

Edited by Nightbird, 02 April 2021 - 06:48 PM.

[CFC Conky]

Interesting to see how each group's lines follow a similar curve.

[Vorpal Puppy]

I'm going to make a broad generalization here, and maybe I am not understanding your chart, but what your data tells me is that this game loses approximately 1/3rd of its player base every month. That seems really high, but I am not up on player retention trends in online gaming.

If so, have you tried to compensate in your data analysis for players who play more sporadically, or alternate between multiple accounts? You would just have to open up your comparison from a one-month window to say a 3-month window.

[Nightbird]

Vorpal Puppy, on 01 April 2021 - 04:25 PM, said:

If so, have you tried to compensate in your data analysis for players who play more sporadically, or alternate between multiple accounts? You would just have to open up your comparison from a one-month window to say a 3-month window.

Here's for 3 months. It removes 3 season's worth of data instead of one, for only 3-5% better retention. Most people when they leave don't come back. I think it means that people juggling multiple accounts are the exception, not the norm.



But yes, this game loses 25-35% of players month-to-month on average. Unless a lot of people change their names every month for fun... 20-30% if looking at periods of 4 month. (Just eyeballing here)

Edited by Nightbird, 01 April 2021 - 04:50 PM.

[Vorpal Puppy]

Thanks. That's really interesting - a lot more player churn than I would ever have expected. Assuming PGI is doing this sort of analysis, they would really want to focus on improving the new player experience.

I know this is an entirely different analysis, and I may not be explaining it well, but I wonder if there is a threshold number of games that once a new player sticks around for N games, their likelihood of still being playing 1 year later suddenly increases dramatically. PGI uses a threshold of 25 games for the CADET bonuses, but maybe the incentives to stick around should be longer to get new players to that magical N number of games, if there is such a thing.

[PocketYoda]

But how does this compare to previous years?

[Nightbird]

Vorpal Puppy, on 01 April 2021 - 05:07 PM, said:

Thanks. That's really interesting - a lot more player churn than I would ever have expected. Assuming PGI is doing this sort of analysis, they would really want to focus on improving the new player experience.

I know this is an entirely different analysis, and I may not be explaining it well, but I wonder if there is a threshold number of games that once a new player sticks around for N games, their likelihood of still being playing 1 year later suddenly increases dramatically. PGI uses a threshold of 25 games for the CADET bonuses, but maybe the incentives to stick around should be longer to get new players to that magical N number of games, if there is such a thing.

There's nothing wrong with further improving new player experience, but what I hoped to show with this graph is that the issue is not only with newer players. You don't see retention improving as WLR increases, instead the worst (new player) WLR bracket and the best (experienced player) WLR bracket are the two worst performing lines. Basically the happiest players are the ~1 WLR bracket, they are challenged just right by the matchmaker. On the other hand, the matchmaker is giving too hard a match to new players, who get frustrated, and giving too easy a match to experienced players, who get bored. My conclusion, which some people disagree with, is that if we put 100% of players in the ~1 WLR bracket with a different matchmaker, we'd instantly see much better player retention.

Edited by Nightbird, 01 April 2021 - 06:15 PM.

[pbiggz]

it needs to be pointed out that any player retention data for really any game these days is going to be completely useless in 3 years because player numbers for pretty much all games are heavily skewed right now by the various pandemic restrictions that are in place.

[Nightbird]

Samial, on 01 April 2021 - 05:15 PM, said:

But how does this compare to previous years?

pbiggz, on 01 April 2021 - 06:15 PM, said:

it needs to be pointed out that any player retention data for really any game these days is going to be completely useless in 3 years because player numbers for pretty much all games are heavily skewed right now by the various pandemic restrictions that are in place.

There's 3.5 years of data here, and it shows retention hasn't changed much since before covid hit? There's more new players thanks to the restrictions, but the game's ability to retain them has been extremely consistent.

[PocketYoda]

Nightbird, on 01 April 2021 - 06:17 PM, said:

There's 3.5 years of data here, and it shows retention hasn't changed much since before covid hit? There's more new players thanks to the restrictions, but the game's ability to retain them has been extremely consistent.

So losing nearly 30% of the playerbase every three months is consistent O.o

[Aleksandr Sergeyevich Kerensky]

Over my full career in MWO, I have introduced Mechwarrior Online to about 30+ some people, and of those people, only 2 stayed long enough to exit cadet.

Of those that left, most played under 5 matches and then pretty much quit.

While I am sure many others will have vastly different results as I did... One thing I can say for sure is... MWO has a steep learning curve for players new to the Intellectual Property.

The New Player Experience for MWO really needs to be looked at if PGI wants to grow the player base. (Adjusting the Default Game settings could be a good start... heck, just having the default Mouse Sensitivity set a bit lower could help a lot for New Player Retention).

Maybe give New Players a "Starter" free mech... maybe let them keep the Training Academy Thunderbolt if they actually go through the tutorial? (as opposed to "Skipping").

Or maybe give the Trial mechs a Trial Skill Tree that is already pre-loaded? (Like maybe ECM trials have all the ECM skill tree nodes)...(or maybe light trial mechs have radar deprevation)?

Anyway, thank you Nightbird for compiling this data!

[Doctor Fission]

I tried to introduce a real-life friend to the game, and he quit after 3 matches because he couldn't strafe. *shrug*

[Nightbird]

Samial, on 01 April 2021 - 08:14 PM, said:

So losing nearly 30% of the playerbase every three months is consistent O.o

I wish it was every 3 months LOL

[GARION26]

Night bird thanks for the really cool data analysis.
I forget if you have done this previously but have you/can you do the following?
1) Show overall retention of the player base ignoring WLD breakdowns
2) Show retention by quartile of the player base (divided into quartiles on whatever measure you pick)

The reason I ask is I suspect the majority of the player base clusters around WLD ratio of 1.0. The people who lose or win at a very high rate are I would suspect a fairly small group of the overall player base. The graphs you've show aren't easy to interpret in terms of which lines most represent the player base as a whole (though clearly the trends are mirrored at all WLD levels.)

One point I've made when you've posted this in the past is people who win a lot or lose a lot both tend to have the lowest retention rate. Despite some believing highly skilled players 'want to farm' others, the folks who win at the highest rate in MWO tend to drop out of the game almost as much as the ones who lose at the highest rate. Some of that may be opposite sides of the experience spectrum "This new game is really hard I give up," vs "I've solved this game and it's no longer a challenge."

Separate question I wonder if there is a way to show % of players with a certain number of months accrued. For example in any given season what percent of the players have >6 months or so lifetime accrued months of play (or you could do lines based on multiple cutpoints <1month, >12 months etc). Not sure how that helps us visualize data but I was thinking about it when looking at the graphs -is any given season's total population a lot of older players returning or newer players.

Edited by GARION26, 02 April 2021 - 09:00 AM.

[w0qj]

There's a lot of older players returning, if you take time to play Tier 4/5 you regularly see premium mechs like TBR-Prime(I), BLR-1G(P), HBR-Prime(I), WHK-Prime(I), HBK-IIC(O), etc.

Talked to a few of them, and they invariably said that "they heard that MWO development has restarted" and returned to play!

Here's to hoping that they would continue to play in a few months time

Edited by w0qj, 02 April 2021 - 06:54 AM.

[Nightbird]

w0qj, on 02 April 2021 - 06:54 AM, said:

Here's to hoping that they would continue to play in a few months time

I would describe game pop as being like water in a rusty bucket. You pour in water from the top, and it leaks out through holes in the bottom. All actions fit in one of two categories, putting in more water, or fixing the holes. If you focus only on putting more water from the top, the total pop will increase momentarily, but the water pressure increases and the leaks get larger. If you only fix the holes in the bottom, the pop will decrease as you're not putting in effort to put in more water.

The retention rate graph I've posted shows that we've never made any fixes to the holes. All pop changes have been based on forcefully putting in more water into the same leaky bucket.

The only successful way to permanently boost the population is the fix some holes, mainly because efforts to put in more water is limited in duration and budget, but leaks work tirelessly.

Edited by Nightbird, 02 April 2021 - 08:11 AM.

[Grus]

I think the recent uptick was due to the rumor that PGI was going to be under new management.

[John Bronco]

Any idea if this is good or bad vis a vis other F2P competing titles?

[GARION26]

For what it's worth though retention is a real issue in other FTP games i.e. a quick google search pulls up the following
https://www.reddit.c..._players_there/

http://forum.worldof...layer-turnover/

MWO isn't alone in that amongst the FTP market.

[Nightbird]

GARION26, on 02 April 2021 - 09:06 AM, said:

For what it's worth though retention is a real issue in other FTP games i.e. a quick google search pulls up the following
https://www.reddit.c..._players_there/

http://forum.worldof...layer-turnover/

MWO isn't alone in that amongst the FTP market.

Can't really compare as there's no monthly or X-monthly retention % calculation in those links.

It's not enough to just find the word retention being used.

Edited by Nightbird, 02 April 2021 - 09:11 AM.