187 replies to this topic

[DAYLEET]

Tarogato, on 30 January 2017 - 01:43 PM, said:

Ever wonder what is the driving force behind those really lopsided matches?

I really like your works Taro, all of them and above all, statistics about everything. And i wish PGI shared more numbers. Since this thread tries to make sense of match results, i will /ramble, a tiny bit...

I think people are chasing after the secret geometry of the perfect Player Rating System and i believe it to be a myth if not an illusion. In a 12v12 theres too much variables going on. I appreciate that you chose 12-0 in an effort to remove dirty variables but are 12-2 not valid when it can take so little to finish a mech? Wheres the difference between 12-0 and 12-1 12-2? Im not going against your 12-0 assessement, please dont focus on that. Hard Numbers shouldnt take precedence over logic in a chaotic 12v12. Should newbies be mixed with vet? ofc not. Should pure win(elo) be used? ofc not, what about matchscore? well the game isnt near balanced enough to consider that.

Battle Value? Some mech really cant compete with other mech(on paper with equal pilot) so should they be divided evenly in each team or never see better mech? We all know a guy who only use a crap mech and always do good in it and can play with and versus the best mech. Are outliers allowed in the equation of left to roam freely? Then theres the underlying to all of that. Should the matchmaker use the player Chassis/Variant stats rather than general player stats? Should it take those stats per game versus the other team before further analysis?

In the end we can only confirm that there isnt enough players to consistently, if ever, create fair match for everyone in a timely manner. PGI might create rules for their matchmaker so that they can say, our matchmaking is fair. By the numbers/stats and predetermined rules it might be fair but in reality it isnt if there's a "valve" regualting matchmaking.

Tarogato, on 30 January 2017 - 02:20 PM, said:

and see if there's any patterns.

Don't, you're looking for an answer by finding the "culprit", we need an equation.

We should agree on a system that encompass everything the game has rather than try to find that perfect value and focus on it to balance. We always tried to find that thing that changed this or that in a game and that has to be the wrong way. We shouldnt look for an answer but an equation and accept that there will be a X at the end, no hard answer can be at the end of that equation. The answer will never be the same in our chaotic system. Make a list of everything that can affect a match outcome rather than funel it like when you troubleshoot problem.

This, and other threads, are symptomatic of how we are thought to solve problems by eliminating either good/bad/neutral values for the purpose of finding an answer, a number or the cause of a problem. I don't think it's gona work in MWO. Anyways, thanks for everyone who read through that. Keep fighting the good fight and do it for you, remember that you are at the mercy of pgi.

/rambling off

Edited by DAYLEET, 30 January 2017 - 05:33 PM.

[Tarl Cabot]

LT Satisfactory, on 30 January 2017 - 02:16 PM, said:

You take a while to get there, but your last point is really the one that matters - MM doesn't split players correctly between teams.

Most of us that have been playing a long time can look at the names on each side and probably have an 80% chance of guessing the match outcome before we even fall out of the dropship because we know who the good players are and when it's shifted to one side like your example, it's patently obvious what will happen in the next 6 minutes.

That is because the players are not gathered together then split into teams.

Seeded Player to determine initial setup
Next player selected goes to the opposite team then back and forth for each team then the person selected who is in a different tier determines direction.

Or LRM Kodiak vs pew pew Awesome.

That and once a team is down a mech or two vs their opponent, they tend to become more defensive while the team who is ahead becomes more aggressive, the blood in the water effect.

PSR really needs to be reworked to make it closer to a zero balance with more variables, where the losing team could have 1-2 people whose PSR increases while the winning team could have one person whose PSR decreases, or if not that a higher ceiling on no PSR change. If a player is not in the top 10% of their tier, drop them back down to 75% (I want more cushion for myself! ) of the next lower tier.

[Lostdragon]

Tristan Winter, on 30 January 2017 - 02:04 PM, said:

Nice job.

I'm sure we've all had matches where some guy on your team was upset and said something like "That's why you should never push saddle" or "That's why you have to protect your assaults" or "That's why your team should never ever split up!" or some other infallible truth. Meanwhile, you're pretty sure that both teams did basically exactly the same thing, and that one side got roflstomped because the other side just had better players.

I know I have.

I had this happen last night on Canyon in Domination mode. At the end some guy who did about 100 damage was griping and the conversation went something like this:

Him: You idiots don't have to go to the middle to fight, you always lose if you do.
Me: Uh, do you understand how Domination mode works? That is literally what you have to do.
Him: No, it never works
Me: The team that beat us just did it...

There were a couple of guys from each team sniping from the flanks and stuff but 85% of the fighting took place around the central cap area. We didn't lose because we had a bad strategy, we lost because we got out traded and didn't rotate the vanguard when we pushed, then we fell back to the top of the ramp and they pincered us.

[Champion of Khorne Lord of Blood]

Would be nice if we based match maker off average match score instead of tier. We could even do average match score per mech and have it be rather volatile for the first few matches until it stabilizes.

[Tarogato]

Rampage, on 30 January 2017 - 05:08 PM, said:

I assume that you were not able to obtain the actual PSR levels of the players in the matches so perhaps not all were Tier 1 or Tier 2.

Correct, this is impossibru. I can't possibly know enough of the two-thousand-odd player's PSRs to make any use of that information.

And certainly not all were Tier 1 or Tier 2. The matchscore spread is very wide, I was seeing players in my matches who average 150 or below, and the global average is just above 200. So that implies that they were either Tier 4 or 5, or that they were Tier 3 (or higher... *shudder*) and PSR isn't working properly. I could devote an entire thread/post on the analysis of this sample of players, actually, given that it represents as close to "tier 1 matchmaking" as we can get in this game.

Quote

Another factor that may influence the outcome of a game is a players proficiency in a specific weight class. Did you look up each players Leaderboard stats in each weight class or did yo use the global stats? In solo QP it is often the case that players are leveling Mechs they may be unfamiliar with and not very skilled at using. For example, if they normally favor Lights but just recently got the Marauder IIC and they are trying to complete the basic skill they will probably not perform up to their Global stats that have been achieved by driving the Mechs they normally use.

I always use global stats - because doing vlookups on multiple seasons worth of leaderboard data, with multiple criteria... and then splitting it up into four extra datasets per season... that really starts to get unwieldy and I actually started running into the hardcoded limit of cells that you can have in Google Sheets. So I had to make concessions and just go with each players global stats, and the percentage of each weight class played per season. I haven't done any work with this data using my weight class formula, but I plan to. Just to ... see what comes out the other end.

Maybe I should just get unlazy and try to figure out how to transfer MS Office from my old computer to my new one. Never thought I'd have a need for mathy-nerdy stuff like Excel until I got addicted to this bloody game, lol.

Quote

I love looking at your spreadsheets and appreciate all the work you put into them but I wonder if you can truly draw any solid conclusions from them. I have watched many MSRB and MWOWC matches that ended in 8-0 or 12-0 stomps. Nearly everyone in those matches were Tier 1. Everyone was using the best Mechs available. All are very skilled players. Yet, stomps happen. People make mistakes, someone is in the right place at the wrong time, communication is lacking, someone gets a lucky shot and suddenly the snowball starts rolling downhill and takes the whole unlucky team with it. This is the case in comp and it is much more often the case in solo QP.

I feel like comp is a bit of a different ball game. When you have two roughly equal teams that are firing on all cylinders butting heads, strateegery becomes a lot more of a critical element. For instance, you might see a stomp where one team decided on brawl, but didn't have the plan completely ironed out or maybe it was a bad plan in the first place, and they get wiped out 8-0. Similarly, a brawl team getting the jump on a range team. But rarely will you see an 8-0 match that is range vs. range or brawl vs. brawl. I can't think of any examples off hand that feature roughly equal skill teams - if you know any, lemme know. =]

Ted Wayz, on 30 January 2017 - 05:25 PM, said:

Things missing I would like to see:

Time to first kill

% team with disconnect

I understand that you are talking about stomps and I am wondering how much is attributed to teams playing from behind early. I have seen stomps result from a team going up 1-0 and waiting for the other team to press.

Discos, especially valuable discos, can also play a factor. Your stats will hide a team with 100 ton disco as they will most likely lose yet have an inflated tonnage. Need to know tonnage active during the match.

Time to first kill would be highly dependent upon which map and possibly also which gamemode (ie, Conquest caps could delay first contact on most maps). It would be extremely hard to both measure this statistic and correct for variables.


I entirely excluded matches with any disconnects because a player that contributes nothing to the match is essentially guaranteed to be not representing what their stats say they are worth. I believe I only included two matches with disconnects, and that was because they were on the side that ended up winning anyways. But my memory is hazy and we're going back on three months here. xD

In retrospect, I should probably have excluded those two(?) matches as well.

[jjm1]

Its no surprise clan wins more, but its still because of the KDK-3s. Occasionally you will see a couple on one team just roflstomp the opposition. Then there is the ACH, they can still back-stab their way through a lance while taking AC-20s to the face.

[chucklesMuch]

Thanks for doing this

Every time I see analysis on MWO stats I just want more information released for us to look at!

Just recently I have been thinking about maps and drop/spawn locations by game type. We know our win/loss by map and by game mode... but does the side of the map we drop on matter? I suspect that I, personally, do much better on several of the maps depending on which side I drop on (and maybe the specific spawn locations too)... what about by weight class (or mech speed) and game mode where the spawns change.

And especially wish more information to look at when I hear the SHC is an over achiever... must be all the non stompy games where the SHC over achieves though... as in your file the 84 games for the SHC has an average score of 217.5 ... This doesn't seem over the top to me

[MadIrish]

You analyzed Quick Play you wasted your time. The issue is in faction play those numbers would be way clan lopsided if you did it in faction.

Edited by MadIrish, 30 January 2017 - 08:27 PM.

[Ex Atlas Overlord]

xTrident, on 30 January 2017 - 04:06 PM, said:

58%? That really doesn't seem like all that much more for Clans. Not when I see so many posts about them being so OP. 58% is nothing really.

8% is huge.

The entire casino industry operates on less.

[Lostdragon]

Tarogato, on 30 January 2017 - 06:10 PM, said:

I always use global stats - because doing vlookups on multiple seasons worth of leaderboard data, with multiple criteria... and then splitting it up into four extra datasets per season... that really starts to get unwieldy and I actually started running into the hardcoded limit of cells that you can have in Google Sheets. So I had to make concessions and just go with each players global stats, and the percentage of each weight class played per season. I haven't done any work with this data using my

You should definitely be doing this in Excel, a few array formulas would make it a lot easier to slice and dice the data.

[Snazzy Dragon]

Davegt27, on 30 January 2017 - 02:37 PM, said:

Need avg dmg output per match and avg kill per match

Match score is a good indicator of that already - avg match score of 230 is considered "pulling your own weight" as far as I know

[MadIrish]

Ex Atlas Overlord, on 30 January 2017 - 08:29 PM, said:

8% is huge.

The entire casino industry operates on less.

Like I said he did his numbers in quick play. Read it carefully and you see he/she mentions both teams having a mix of clan and IS mechs.

[Ex Atlas Overlord]

MadIrish, on 30 January 2017 - 08:35 PM, said:

Like I said he did his numbers in quick play. Read it carefully and you see he/she mentions both teams having a mix of clan and IS mechs.

Are you saying that it is a low value because of the fact that techs were mixed, and that it would be much more obvious and exaggerated if all gathered data was from faction play?

I just want to make sure I'm following you.

[MadIrish]

Ex Atlas Overlord, on 30 January 2017 - 08:37 PM, said:

Are you saying that it is a low value because of the fact that techs were mixed, and that it would be much more obvious and exaggerated if all gathered data was from faction play?

I just want to make sure I'm following you.

Yes they should have ran there analysis on Faction Play, doing this on Quickplay does not provide any valid data on Clan or IS OPeeness. Had they ran these numbers in faction play the percentage would be like 90% Clan Wins 10%IS but I'm sure PGI knows this already and that's where there is no faction stats on the leader boards. If they posted those stats on the leaderboard they would have to admit there is major balance problem and all the IS players like me would say "See we told you so, Clan OP - IS Gimped"

Edited by MadIrish, 30 January 2017 - 08:45 PM.

[Ex Atlas Overlord]

MadIrish, on 30 January 2017 - 08:44 PM, said:

Yes they should have ran there analysis on Faction Play, doing this on Quickplay does not provide any valid data on Clan or IS OPeeness. Had they ran these numbers in faction play the percentage would be like 90% Clan Wins 10%IS but I'm sure PGI knows this already and that's where there is no faction stats on the leader boards. If they posted those stats on the leaderboard they would have to admit there is major balance problem and all the IS players like me would say "See we told you so, Clan OP - IS Gimped"

IIRC the last time Russ mentioned the clan vs IS stats in FP... it was something around 60% wins to clans.

Edit: Maybe it was 65%.... can't remember, it was a while ago after the second Tukkaid invasion or however that's spelled.

But I DO remember him saying "So they're pretty balanced"... to which I LOL'd and kept not playing FP.

Edited by Ex Atlas Overlord, 30 January 2017 - 08:56 PM.

[a gaijin]

Tarogato, on 30 January 2017 - 01:43 PM, said:

{Excellent thread and info}
Now it looks much more balanced! PGI, why can't your matchmaker do something like this?

Taro, this is for you:


Junior Software Developer
[color=#222222]Piranha[/color] Games has an exciting opportunity for a Junior Software Engineer/Developer!
As a technical generalist, you are passionate about coding, solving challenging problems and improving your software engineering toolbox. In this position, your knowledge and hands-on experience in C++ programming will be put to the test! You will contribute to all areas of development, implementation & integration of online in-game features and systems.
REQUIREMENTS:

• Degree in Computer Science, Computer Engineering or Engineering Physics
  
• 0-2 years year of programming related work experience in a software or gaming environment
  
• Experience and proficiency in C/C++
  
• Proven ability of debugging skills
  
• Ability to write clear, maintainable and optimized code
  
• Good verbal and written English skills
  
• Ability to work within a team-oriented environment

ADDITIONAL QUALIFICATIONS:

• A passion for games!
  
• Knowledge of TCP/IP, UDP protocols
  
• Knowledgeable about cheating/anti-cheating in multiplayer games

If you've got a BS degree in IT+ 5yrs programming experience then this is for you. You'd be doing all of us a favor




Edited cause my computer at works sucks

Edited by Star Commander Horse, 30 January 2017 - 09:00 PM.

[InspectorG]

Tarogato, on 30 January 2017 - 01:43 PM, said:

...

You need to look into the snowball effect.

Im willing to bet Solo matches are more chaotic events than more organized play.

Initial actions in a zero-sum, no respawn environment HAVE(my opinion) to have larger effects on outcomes than in matches where coordination(and skill) can help deter the first snowball.

Like in Chess, where opening moves that concentrate on the center 4 spaces of the board are more optimal(unless you know of a counter-play suited to your opponent's tendencies), MWO in Solo favors that initial lead.

So, in my experience, the factors that lead to snowballs:

Teams not moving as a cohesive unit. (as base as it is, getting to center and grabbing valuable firing lines -in Solo mind you, has to be brainstem level skill demand to be viable unless a skilled leader is present)

Also, Rabbit Runs(essentially a charity match for the other team)

Blindly following a random light(seed-event for NASCAR?, why usually to the team's right side?)

Less skilled or unlucky Light pilot who gets merc'd early.(reduced target choices for enemy team resulting in more focused incoming fire).

Slow Assault left behind(same as above but usually @1-2 minute into match, map depending)

Camping(my favorite sin to chastise).

Cowardice(refusal to 'share' armor).

Improper LRM use or Sniping(subtracts team armor).

Poor early trades.

How to gather info and statistics on this? I have no idea. Just my experience in @3 years of play.

[Jungle Rhino]

Great study other than one glaring omission - clearly all the losing teams had 'mechs carrying LRMs

[Hydrocarbon]

Tarogato, on 30 January 2017 - 01:43 PM, said:

Now that we've revealed each player's average match score, we see a massive imbalance! But if we just shuffle a few players from one side to the other:






Now it looks much more balanced! PGI, why can't your matchmaker do something like this?

THIS!!!!

I've said this since clan invasion. Right now I believe it builds 2 teams separate, but w/o this final adjustment you end up with an imbalance much of the time. Now with definite stats being tracked, it would take a simple spreadsheet 0.02 sec on a PIII laptop to calculate the best mix.

[Dimento Graven]

Not to side track ANYTHING here, BUT, I just can't help but post something that struck me out right odd...

Horse, y'know what intrigues me most about that PGI job posting you've quoted?

This right here:

Star Commander Horse, on 30 January 2017 - 08:56 PM, said:

...

• Knowledgeable about cheating/anti-cheating in multiplayer games

Now, if the game had as little cheating, and PGI knew exactly as much as Russ has continued to claim over the years, I find it absolutely odd that experience with cheating/anti-cheating would even be bothered to be listed as any sort of requirement.

Me thinks there's quite a bit of... obfuscation, coming from Russ on the subject.