1911 replies to this topic

[Karl Berg]

TKSax, on 06 September 2014 - 09:15 AM, said:

Karl,
How much have you been working on the new PGI game.

Thanks

Sorry guys, no comment here. This is being handled by Bryan Ekman and Niko, so I direct you to their thread for a response. I would appreciate keeping this thread on-topic with MWO if possible. Thanks!

[Karl Berg]

9erRed, on 06 September 2014 - 12:17 AM, said:

Technical question about a single item in the game,
- TAG -
Is there a specific reason that the described infrared beam is actually visible in normal vision for all to see?
- Is there some coding issues that do not allow for it's function to only be seen in thermal?

Currently it rather destroys the stealth of spotters if the beam can just be followed back to the 'shooter'.Is it 'on the list' of items to look at in the future? Designing a beam that is invisible to all unless in thermal? Possibly adding a filtered view mode to those that are using it so they can see their TAG.

Nope, no technical reason. I'm guessing this was an explicit design choice, but it's worth bringing up with design directly if you think it's a big issue.

[Dimento Graven]

Karl Berg, on 06 September 2014 - 10:36 AM, said:

Sorry guys, no comment here. This is being handled by Bryan Ekman and Niko, so I direct you to their thread for a response. I would appreciate keeping this thread on-topic with MWO if possible. Thanks!

Karl, Karl, Karl...

You poor naïve SOB...

The proper response was, "What new game?"

YOUR response just screams, "Yes, I'm working on the new game, but I'm not allowed to admit it or discuss it in any way..."

<sigh>

Edited by Dimento Graven, 06 September 2014 - 10:43 AM.

[Karl Berg]

p4r4g0n, on 02 September 2014 - 03:47 PM, said:

Following from a discussion with some friends after a recent series of such matches, I wonder if there is any merit in putting in some additional filters on the matchmaker to prevent the formation big high Elo groups unless said groups are prepared to accept unlimited or much longer wait times to find a closely matching team. The benefits of this could be:-

• elimination of the need to use group size modifiers (penalties) which does not actually take into consideration skill / experience;
  
• reducing the disincentive to lower Elo rated players to form bigger groups;
  
• creating larger numbers small high Elo groups which in turn would would make it easier for the matchmaker to balance teams; and
  
• reduced wait times for high Elo players as there are more groups of equivalent Elo ratings for matching.

Example:


Average Group Elo : Max group size
2,300 to 2,500 : 2
2,100 to 2,299 : 3
1,800 to 2,099 : 4

The only downside that comes to mind is that high Elo players may object to limitations in the number of friends they could play with at any one time in the public group queue. However, given the amount of time high Elo players already spend with similar rated players in practice, scrimmages, etc this may not really be an issue in most cases.

That would be a very extreme measure indeed. I don't think anyone has discussed Elo based group size caps before. Something this restrictive would need to be very carefully considered by design and product owners, since it restricts a subset of players in non-obvious ways. Elo is currently a hidden variable after all.

Dimento Graven, on 06 September 2014 - 10:42 AM, said:

Karl, Karl, Karl...

You poor naïve SOB...

The proper response was, "What new game?"

YOUR response just screams, "Yes, I'm working on the new game, but I'm not allowed to admit it or discuss it in any way..."

<sigh>

Heh; well I'm not going to pretend I don't know about the new game. I don't think it would be constructive in the least to feign ignorance.

[Karl Berg]

Mawai, on 02 September 2014 - 11:17 AM, said:

Hi Karl,

Since you are the matchmaker guru

There seems to be a lot of complaints about the matchmaker on the forums at the moment. Some people are complaining of streaks of losses and unbalanced matches. I haven't experienced the issue myself but I was wondering if you have any statistics on win/loss streaks for players and whether there may be a subset of the player base that is experiencing some sort of bias from the matchmaker? Perhaps depending on weight class or Elo? ... which results in a streaky win/loss result. Maybe the player is near an Elo bin boundary in match formation?

Also, could a match balance issue be related to Elo bin expansion as the timer for match creation increases? Does the matchmaker form each team in an alternating manner so that when release valves are opened both teams are equally affected? If the matchmaker works on one team at a time and then can't find enough to form the other team and then opens the release valves ... there is the possibility for uneven matches to be formed.

Anyway, I think the player base could benefit from a detailed explanation of the algorithms used in the matchmaker, the nature of the release valves and the points at which they kick in.

I was also wondering if it would be possible to list the difference in Elo between the two teams and the spread of Elo on each team as part of the end game screen? This would not reveal individual player rank but it would let the players know that the MM is working correctly and give them an idea of the player skill balance in the match.

Keep in mind the matchmaker plays a sort of tetris game in the group queue; fitting together available groups into teams of 12 with equal numbers of weight classes, and limiting the total number of any one weight class as much as possible. That results in some occasional weird matchups.

The release valves are global to the game that it is trying to assemble. Those release valves are calculated using the age of the oldest request in the game. The process of adding new players to the game is iterative, and is always done to the smaller team so that the matchmaker essentially builds the two teams in an alternating manner.

In fact, ever since Elo was added, the matchmaker has assembled games in this manner, even prior to my rewrite.

[Dimento Graven]

Karl Berg, on 06 September 2014 - 10:45 AM, said:

Heh; well I'm not going to pretend I don't know about the new game. I don't think it would be constructive in the least to feign ignorance.

Right, but what we needed to hear from you was that you're not in any way working on that new title.

For those of us in the 'spin' industry, the answer you gave seems to indicate that you actually are.

With your reputation among this forum's followers we'd have believed you if you had come out with flat out denial of something like, "That new title? No, while it's a really neat project, my time is totally dedicated to MWO, maybe someday I'll get the chance to work on it, but not until MWO is where we, and you the players, want it."

You're never going to make it in politics brudda!

[Karl Berg]

SnagaDance, on 26 August 2014 - 10:16 AM, said:

He's been on crunch for weeks, he'll be one of the undead by now, shuffling through the office going "PUUUUUUUUUUUUUUUUUGSSS!!".

And Niko took some unfortunate pics of me apparently..

[Alaskan Nobody]

Karl Berg, on 06 September 2014 - 10:51 AM, said:

And Niko took some unfortunate pics of me apparently..

Could be worse, I know several people who do not photograph well.

[Karl Berg]

Dimento Graven, on 06 September 2014 - 10:50 AM, said:

Right, but what we needed to hear from you was that you're not in any way working on that new title.

For those of us in the 'spin' industry, the answer you gave seems to indicate that you actually are.

With your reputation among this forum's followers we'd have believed you if you had come out with flat out denial of something like, "That new title? No, while it's a really neat project, my time is totally dedicated to MWO, maybe someday I'll get the chance to work on it, but not until MWO is where we, and you the players, want it."

You're never going to make it in politics brudda!

Agreed, I'd make a terrible politician. I'm also not about to lie to anyone.

I've done a very large amount of work on the new data center over the last few weeks. The majority of that work involved transferring over all the required services for Mechwarrior Online. The new projects website is indeed running in that same datacenter. I have *not* however touched any gameplay code outside of MWO at Piranha since Duke. Hopefully that's enough of an answer for you.

Thanks

[Dimento Graven]

Karl Berg, on 06 September 2014 - 10:56 AM, said:

Agreed, I'd make a terrible politician. I'm also not about to lie to anyone.

I've done a very large amount of work on the new data center over the last few weeks. The majority of that work involved transferring over all the required services for Mechwarrior Online. The new projects website is indeed running in that same datacenter. I have *not* however touched any gameplay code outside of MWO at Piranha since Duke. Hopefully that's enough of an answer for you.

Thanks

LOL! I'm still left with an impression there's more to the story but this answer is close enough to what we all probably want to hear I'm good with it.

[Karl Berg]

Mawai, on 02 September 2014 - 11:17 AM, said:

I was also wondering if it would be possible to list the difference in Elo between the two teams and the spread of Elo on each team as part of the end game screen? This would not reveal individual player rank but it would let the players know that the MM is working correctly and give them an idea of the player skill balance in the match.

That's an interesting suggestion. I've been looking very closely at the ranges and standard deviations of Elo within matches myself, with a definite goal towards reducing standard deviation as much as I possibly can without adversely affecting the other matching criteria. The good news is I do have a set of pending changes that appear to help a great deal when tested against current production data. I'm hoping to get these tested and released as soon as I can.

Sorry guys, I need to step out for a bit.

[Chronojam]

Karl Berg, on 06 September 2014 - 10:36 AM, said:

Sorry guys, no comment here. This is being handled by Bryan Ekman and Niko, so I direct you to their thread for a response. I would appreciate keeping this thread on-topic with MWO if possible. Thanks!

Hi Karl; it's great to see you're still answering fan concerns here. I tried asking Bryan Ekman a few questions directly the other night in that thread since they were a bit related to both MWO and PGI's other projects, but as of this morning,


So focusing on MechWarrior Online, I'd like to check into some of the statements we've been given, just to help set the record straight or at least clear up some discrepancies.

In 12/2013 we were told by Russ that MWO had a staff of 45 dedicated developers; we'll use this as our baseline.

In 5/2014 fans were concerned over slow progress, caused by understaffing forcing the need to shift e.g. the entire environment team to work on clan mech design.

Later in 5/2014, Russ bragged that PGI had grown 25% (from 11/2013 to 5/2014) so MWO fans should not worry about continued staffing problems.

In 9/2014 Bryan explained that MWO currently has reduced a staff of 35-40, which now includes external contractors.

Hopefully, as these are public statements, you won't have any trouble filling in the gaps for us. Despite bragging of 25% growth over six months, four months later MWO's team appears to have shrunk by 10-20% compared to baseline -- in conjunction with a reliance on external contracted labor instead of dedicated in-house developers.

Were these statements true? Is a reduced MWO workforce, dependent on outside labor, now struggling to keep up with old development timelines?

[Goose]

Karl Berg, on 06 September 2014 - 10:50 AM, said:

… the matchmaker plays a sort of tetris game in the group queue …

"Official" Tetris reference = Dooms Day Clock advancing one tick

[Chronojam]

It's a little bit concerning, especially if any current MWO support staff are also being tapped to work new projects. They were clearly stretched thin in the past, and it wouldn't help matters to stretch them worse.

[Gyrok]

Karl did you see my question about the matchmaker results for the Clan vs IS matches?

[Darth Futuza]

Karl Berg, on 06 September 2014 - 10:39 AM, said:

Nope, no technical reason. I'm guessing this was an explicit design choice, but it's worth bringing up with design directly if you think it's a big issue.

Yes, please do.

[MischiefSC]

Karl Berg, on 06 September 2014 - 10:32 AM, said:

The matchmaker 'seeds' with an oldest player or group. Once it's decided on it's seed, it then proceeds to iteratively add groups / players to the smallest team until the game is full and ready for launch. The criteria for how it adds players is complex and deserves a fairly in-depth post; but beyond the initial seed, there is no consideration of age.

That's what I was trying to get at. So it does effectively alternate adds to each team for new population. In balancing Elo does it shoot for an initial 'range' target or does it grab a 'bundle' of high/low to match target for the match?

So if I get this right it grabs a couple of the oldest waiting players in the queue and they effectively set the Elo target for the match as well as pre-seed themselves in the 4x3 matchup. Then it looks through the whole queue for fits based on criteria of exact tonnage match > weight class and matching Elo > high/low to target?

For all the talk of nobody taking Lights I'll get matches with 4 lights on each side. Does it prioritize weight class over Elo or vice versa?

Karl Berg, on 06 September 2014 - 10:32 AM, said:

Depends. For all public games, the selection of map is chosen randomly by the server when it first boots. Long before any players connect to it. When the game ends, that entire dedicated server process also ends, and a new process is immediately started up to take its place. Obviously for private matches, the map can be overridden by the players in their lobby, so we inform the dedicated server to throw away it's random map selection to use the player lobby selection instead.

Is it pattern generated, as in when I finish a Terra Therma match another TT match starts or just scrolling down a menu? As in if I finish that TT match and immediately drop another am I likely to get caught in that same new TT match or is this just random chance? I find that if I drop again immediately after the last I'm more likely to catch the same map or is that just anomalous?

[Karl Berg]

Chronojam, on 06 September 2014 - 12:02 PM, said:

Hi Karl; it's great to see you're still answering fan concerns here. I tried asking Bryan Ekman a few questions directly the other night in that thread since they were a bit related to both MWO and PGI's other projects, but as of this morning,


So focusing on MechWarrior Online, I'd like to check into some of the statements we've been given, just to help set the record straight or at least clear up some discrepancies.

Hi Chrono,

Apparently Bryan has decided to delete the old thread and start over. I suggest you repost your questions in the new thread that they've opened. I hope you understand that I can neither correct nor call out any statements made by Bryan or Russ. That said I highly doubt there was any intent to deceive. We're certainly not one of the largest studios by any means, but simply adding bodies to a project can easily do more harm than good. So I strongly prefer HR take their time and really seek out the best talent that we can find when we find ourselves short staffed.

People have frequently asked me what the size of the studio is, and I'm certainly guilty of simply guestimating, rather than spending time counting heads. It certainly sounds like some rounding has been done to the figures you've been provided at least.

Staffing levels, both internal and external, change all the time as the needs of the project change. There is also some amount of turnover as employees personal situations change. After all, we started this project nearly three years ago now.

Gyrok, on 06 September 2014 - 12:10 PM, said:

Karl did you see my question about the matchmaker results for the Clan vs IS matches?

Would you mind reposting your question for me Gyrok? I'm going to continue working my way back through the question backlog, but a repost would help me answer your question sooner. Thanks!

[Karl Berg]

MischiefSC, on 06 September 2014 - 12:39 PM, said:

That's what I was trying to get at. So it does effectively alternate adds to each team for new population. In balancing Elo does it shoot for an initial 'range' target or does it grab a 'bundle' of high/low to match target for the match?

So if I get this right it grabs a couple of the oldest waiting players in the queue and they effectively set the Elo target for the match as well as pre-seed themselves in the 4x3 matchup. Then it looks through the whole queue for fits based on criteria of exact tonnage match > weight class and matching Elo > high/low to target?

For all the talk of nobody taking Lights I'll get matches with 4 lights on each side. Does it prioritize weight class over Elo or vice versa?

Yes, it effectively alternates adding to each team, as long as the size of the groups being added are roughly equivalent. In the case where an 8 man is added to team 1, and a 5 to team 2, then the next team to receive a group would be team 2 since it is still the smaller.

And now this is exactly where it gets complicated.. Yes, the average Elo of the seed group is saved off. This become the baseline skill level around which the entire game is built up on. There are also obviously multiple competing constraints that the matchmaker is attempting to simultaneously satisfy, including group size, Elo, weight class, game modes, region, and factions when we enable that feature. Most of those constraints require some form of tolerance decay, or release valve as I describe them to design. The decay functions are exponentials, with constant start and stop inputs, that vary with respect to time (age). That evaluation of a given decay function is quantized onto an output range, and these are fed into a search algorithm that locates potential matches and returns the most desirable. All of these parameters are tunable on the fly from our matchmaking command center.

This somewhat analog approach to balancing simultaneous constraints seems to work quite well in practice. We can easily make small tweaks to inputs to subtly change our desired biases in match outputs. There are a few constraints that always take priority. These are what we refer to as hard-constraints, as opposed to the softer constraints like Elo, or the number of heavy mechs in a match. Those hard constraints are currently team size, balanced weight class matching (although this switches off after 4 minutes searching in group queue), region, and game mode.

MischiefSC, on 06 September 2014 - 12:39 PM, said:

Is it pattern generated, as in when I finish a Terra Therma match another TT match starts or just scrolling down a menu? As in if I finish that TT match and immediately drop another am I likely to get caught in that same new TT match or is this just random chance? I find that if I drop again immediately after the last I'm more likely to catch the same map or is that just anomalous?

It's a random dice roll, using the c standard random function, seeded with system time, every time a server boots. There should be no magical stride that sees you getting placed on the same server on the same map repeatedly. There have been times where I could swear there was some selection bias in the system myself though. One of the engineers went so far as to write up a quick simulation of distributed RNG's like we're using just to see if there was a loss of entropy when compared to a single random number generator. We found that the distribution using a distributed system was less random. Not terribly surprising I guess. Anyways, all of this has reminded me to go back and change this so that it uses a centralized RNG; so thanks!

[Karl Berg]

Cimarb, on 12 August 2014 - 04:40 PM, said:

Karl, if you read this, I would just like to know what the variance is within the teams, and what you think of that number (good/bad/needs improvement)?

I think it needs improvement. Many of the games that are kicked off have standard deviations within 200 Elo for each team; but certain less matchable inputs frequently cause standard deviations within teams that I don't like; and I don't think the players like it either. I've certainly seen certain players claim that this is the result of the matchmaker intentionally hobbling them on more than one occasion. Extremely high Elo players, dropping very popular weight classes, with restricted game modes are a perfect example of what really gives the matchmaker a hard time.

It's a problem that I've been putting a lot of thought towards for the past while, to the point that I've been adding additional analytics in behind the scenes since the new matchmaker went live, as well as experimenting with various potential solutions by replaying production telemetry in my test environments wherever I can find time. I hope to launch some improvements here soon.