Jump to content

Test Your Matchmaking Algorithm Queue Times With Parsed Server Log Data


5 replies to this topic

#1 Mawai

    Member

  • PipPipPipPipPipPipPipPipPip
  • Legendary Founder
  • Legendary Founder
  • 3,495 posts

Posted 18 March 2014 - 07:29 AM

Dear PGI,

One possible problem that has been brought up with the planned 3/3/3/3 mech class and 1 group/team matchmaking is the issue of queue times that will develop. In my opinion, anything longer than a 5 minute wait time for a match that typically lasts less than 10 minutes will drive players away from the game.

-----------
SECTION 1: Weight class queue times may be unacceptably long with the planned implementation.


Simple calculations indicate that even small discrepancies in the preferred choice of player mechs will result in a wait queue.

Three examples:

Assume 15,000 active players with 6 drops/hour (this could be way off but it is a starting point).
= 90,000 drops/hour

Case 1) weak preference for assaults over lights

20,000 lights, 22,500 medium and heavies, 25,000 assaults

Number of lights constrains number of matches formed = 20,000/3 = 6,667 matches/hour

At the end of one hour there will be 2500 mediums and heavies and 5000 assaults queued up for a match.

To get to the last one in each queue will take 2500/3 = 833 matches for mediums and heavies
and 5000/3 = 1667 matches for assaults.

So after 1 hour of uptime the queue times will be:
Heavy and medium = 833/6667 hours = ~ 7.5 minutes to get a match
Assault 1667/6667 = ~ 15 minutes to get a match

For every hour this imbalance continues the queues will only get longer. After 4 hours, assaults will be waiting an hour for a match.

Case 2) strong preference for heavies and assaults over lights and mediums

18000 lights, 17000 mediums, 26000 heavies, 29000 assaults

Number of mediums constrains the number of matches/hour = 17000/3 = 5667

After an hour every other class has a queue:
Lights = 1000 in queue = 333/5667 hours = 3.5 minutes
Heavies = 9000 in queue = 3000/5667 hours = ~32 minutes
Assaults = 12000 in queue = 4000/5667 hours = ~42 minutes

As long as the imbalance continues queues grow at this rate every hour.

Case 3) very weak preference for assaults

21,500 lights, 22,000 mediums, 23,000 heavies, 23,500 assaults (I think it is unlikely the numbers are this close but PGI HAS THE DATA to check).

Lights: Number of matches/hour = 21,500/3 = 7167

After an hour every other class has a queue
Mediums: 500 in queue = 167/7167 hours = ~1.5 minutes
Heavies: 1500 in queue = 500/7167 hours = ~ 4 minutes
Assaults: 2000 in queue = 667/7167 hours = ~ 5.5 minutes

As long as the imbalance continues the queues grow at that amount every hour.

----------

So ... even in the most favourable case outlined above with about a 2% variation between most and least common (2000 mechs out of 90000) ... queue times start to grow out of the acceptable 5 minute range in less than 1 hour.

THIS IS A PROBLEM. I hope you can see this issue.

PGI has access to the server logs. It would take less than a day with that data to simulate queue sizes based on actual player drops (assuming you have data on player numbers and mechs from the logs).

However, based on the brief analysis above. The 3/3/3/3 algorithm will NOT work with any reasonable level of variation in the player selection of mechs.

If you do not want a PR issue to develop I would suggest that whatever changes you decide to make to address the queue sizes be announced publicly. (Alternatively, you could put your fingers in your ears and ignore the issue and address it after the queues become enormous ... but that would not be a wise business decision since you will likely lose too many players by that point in time).

------------------------------------------
SECTION 2: Group queue issues.

Recently there were some statistics published by Paul regarding solo and group drops. 84% of launches are solo and 16% are groups. The groups split into 8% 2man, 4% 3man and 4% 4man.

If this data refers to players then out of 100 players there are 84 solo and 16 grouped split into between 4 and 8 groups - 5,8 groups on average.

With 4 matches being formed from 100 players this allows for 3 matches with one team each and one match with no teams giving no queue for teams - ideal! ;)

However, IF there are more than 8 teams in every 96 players then a group queue will develop.

If Paul's data actually referred to LAUNCHES and not players (where a launch is considered a single or group together) then there are actually 66% of players dropping solo and 33% in groups.

Based on launches then out of every 96 players there will be 32 in groups with an average of 11.6 groups in every 96 players.

Only 8 groups/ 4 matches can be accommodated based on the planned matchmaker - leaving 3.6 groups queued every 4 matches.

Taking the best case data from case 3 above with 7167 matches/hour.

7167/4 * 3.6 = ~6450 groups queued at the end of an hour.
With 2 groups/match this is a queue time of (6450/2)/7167 = ~27 minutes

So ... if the number of players in groups is closer to 30% than 16% then the group queue times will get extremely long as well.

However, PGI HAS the DATA. They can and hopefully will repeat these calculations using real data and will have a VERY GOOD IDEA of the expected queue times before they release the new matchmaker.

--------------------------------

Anyway, the purpose of this post was to hopefully bring the possible issue to the attention of someone who can do something about it before it becomes another PR disaster.



P.S. If PGI simply does not have the manpower to put a day into averting a problem then I am sure there are many in the community who would sign an NDA and do the analysis for you given the log data. It just isn't that difficult. The matchmaking itself is challenging and the analysis here does not factor in the time required to actually form the match ... but figuring out the queue times assuming a constant wait time for the matchmaking process to be added on top is very straight forward.

Edited by Mawai, 18 March 2014 - 07:36 AM.


#2 Supersmacky

    Member

  • PipPipPipPipPipPip
  • The Messenger
  • The Messenger
  • 239 posts
  • Facebook: Link
  • Twitter: Link
  • LocationGeorgia

Posted 18 March 2014 - 08:56 AM

+1 - - Agreed!

#3 Aym

    Member

  • PipPipPipPipPipPipPipPipPip
  • Bad Company
  • Bad Company
  • 3,041 posts
  • LocationLos Angeles

Posted 15 April 2014 - 05:28 PM

That was the first problem I thought of with the 3/3/3/3, because I know tournaments based around that such as the first Run Hot or Die and the Marik Civil War were great fun with some varied game-play, but in reality it'll make mediums and lights the new tanks-and-healers from WoW's Looking-for-dungeon queue. On the one hand, as a life-long tank, instant queues are nice. On the other, it sure sucks for the dps'ers. So Paul mentioned concern over tonnage limits with two friends playing Atlai together, and I can sympathize, I got many players into the game running my Founders Atlas alongside whatever mech they were playing, and tonnage limits would severely punish that in a 2-player group, but consider the queue times for 2 Atlai playing new players in a group and you have effectively removed their ability to do this anyway.

#4 Kmieciu

    Member

  • PipPipPipPipPipPipPipPipPip
  • Urban Commando
  • Urban Commando
  • 3,437 posts
  • LocationPoland

Posted 24 April 2014 - 11:05 PM

Mawai is probably right.

I logged onto the public test server about 3 hours after the test had started. Tried to get into a match in a medium mech - waited 30 minutes with no effect. Switched to a light mech and I found a match within 2 minutes.

This class balancing will surely turn into another PR disaster, but it can be somewhat mitigated by introducing wait timers, like in Warthunder.

Edited by Kmieciu, 24 April 2014 - 11:10 PM.


#5 Rushin Roulette

    Member

  • PipPipPipPipPipPipPipPipPip
  • WC 2018 Top 12 Qualifier
  • WC 2018 Top 12 Qualifier
  • 3,514 posts
  • LocationGermany

Posted 25 April 2014 - 12:02 AM

Actually, there is a very quick and easy fix which should pose absolutely no problem in implementing while still giving the players the chance to pick their mechs freely (except for 4 or more player groups still being limited to no more than 3 mechs of one weight class).

In the closed Beta we had a number displying the number of players online at the time... just re-implement this with the change of splitting it into 4 groups:

Number of players online Light Mech
Number of players online Meduim Mech
Number of players online Heavy Mech
Number of players online Assault Mech

Players can then determine for themselves which weight class has less players and use that mech for a shorter wait time or chosse a different weight class in the full knowledge that wait times will be longer.

Which player is logged into which mech at one time is already in the game, as can be seen in the pre match group window tracking that data along with where they are in the UI..

#6 Bobzilla

    Member

  • PipPipPipPipPipPipPipPipPip
  • The Determined
  • The Determined
  • 2,003 posts
  • LocationEarth

Posted 25 April 2014 - 03:05 AM

No current information can really be used, a medium vs less heavy and assaults will be more viable.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users