187 replies to this topic

[Lostdragon]

Tremendous Upside, on 31 January 2017 - 08:36 AM, said:

I think the biggest issue with looking at data like this comes down to players individual "expectations" of what it is a matchmaking system is or should be doing. If you're looking at this to prove that the matchmaker isn't providing "equal" teams in games -- you're probably right. It's also highly possible, if not probable that it's doing this on purpose. A matchmaking system that creates the best possible game every single time would be closer to the ELO system we used to have. People generally didn't like that - and PGI scrapped it (likely because it wasn't doing what they wanted it to do). If I had to guess the matchmaker we have is designed to try and keep all players closer to the 50/50 line for wins and losses over time rather than creating the very best matches possible every time out. It's much better for player retention. QP is entry point for this game for everyone. It's there so players can jump in, stomp around and have fun. It's not there to provide some sort of Tier 1 uber-player subculture - no matter how bad some seem to want this to be the case...

As for lopsided blowout games... I fail to see why people get so obsessed by these. In every single sport on earth you have professional teams facing each other and blowing each other out. It's not always because one team is 1000x better than the other. Sometimes it's just a matter of how things break over the course of the match. Other times you just have a complete lack of chemistry/communication between the members of your team. It is what it is. If there's ever a silver lining, those sorts of games tend to end quickly - and then you're back at the menu screen ready to drop again. I'd rather be doing "something" in-game even if frustrating than sitting around watching the stupid circle search for matches instead.

I have been saying for a long time players have unrealistic expectations for match outcomes. In a no respawn game mode you will always have stomps because once you are down 2 or 3 mechs the weight of fire from 12 enemies is simply too overwhelming and the odds of a comeback are extremely slim. In my experience 12-0 matches are actually not as common as the crying about them would lead one to believe. Matchmaking can definitely be improved and going by average matchscore would seem to be a better system, but no system is ever going to ensure precious snowflakes don't get crushed 12-0.

Edit: I would be very interested to see if frequency of 12-0 losses is inversely proportional to average match score. I suspect it probably is, the better you play the less likely you are to get blown out would seem to be reasonable.

Edited by Lostdragon, 31 January 2017 - 09:23 AM.

[Tarogato]

Star Commander Horse, on 30 January 2017 - 08:56 PM, said:

*PGI job description*

NighthawK1337, on 31 January 2017 - 03:44 AM, said:

Tarogato, are you a statistician by trade? those are damn good observations.

No degree, no programming experience, no knowledge of most of the things listed, etc, etc. I would be mostly dead weight if employed by PGI. I'm just a dude who plays the game a lot, has a passion for it, and learned just enough about statistics and spreadsheets to do something semi-productive with my spare time.

I'd love it if people more qualified and experienced than me would more often come along and point out things I could do/learn, or maybe point out fallacies I might have fallen victim to, but such people crop up so rarely. =[

InspectorG, on 30 January 2017 - 09:07 PM, said:

Like in Chess, where opening moves that concentrate on the center 4 spaces of the board are more optimal(unless you know of a counter-play suited to your opponent's tendencies), MWO in Solo favors that initial lead.

I feel like I'm being annoying and intentionally contradictory in making this reply... but many of the things you listed are things that I've seen go both ways in my own matches. For instance, I've seen teams (still talking solo queue) go to what I'd consider an absolute worst part of the map to engage from, and still win handily.

Quote

Teams not moving as a cohesive unit. (as base as it is, getting to center and grabbing valuable firing lines -in Solo mind you, has to be brainstem level skill demand to be viable unless a skilled leader is present)

I've seen teams split up across the map, surround the enemy on all flanks, box them in little by little, and crush them in a lopsided victory. It makes sense too, because this is a tactic that works in competitive matches. Balling up is bad, because a team that spreads out can surround you and shoot the fish in a barrel. It usually doesn't work in solo queue because solo queue is inundated with bads that can't shoot. But sometimes it just works anyways by a fluke of nature.

Quote

Also, Rabbit Runs(essentially a charity match for the other team)
Blindly following a random light(seed-event for NASCAR?, why usually to the team's right side?)

Depends on where that light takes you. Could lead in to a gg nascar win, or could lead you into a gg nascar loss - I've seen it go both ways. And why does NASCAR rotate around the right side happen? I suspect because it was ingrained early on - the River City 500 and such, where lights spawning on the right side just simply going forward would catch out the enemy's assault lance, and both teams would do this simultaneously. Plus, most mechs were built for (and still are) for right-side peeking. Think Cataphract, Centurion, Dragon, Hunchback... who were forced to build that way. The right-side meta took hold, and NASCAR became a permanent feature.

Quote

Less skilled or unlucky Light pilot who gets merc'd early.(reduced target choices for enemy team resulting in more focused incoming fire).

Chances are they still wouldn't have contributed much if they hadn't died there, and lights typically aren't focused until the end of a match anyways, unless they are literally the only thing to shoot at.

Quote

Slow Assault left behind(same as above but usually @1-2 minute into match, map depending)

Often happens to both teams at the same time.

Quote

Camping(my favorite sin to chastise).

Is sometimes a winning strategy.

Quote

Cowardice(refusal to 'share' armor)
Improper LRM use or Sniping(subtracts team armor).
Poor early trades.

These are signs of a bad player, which is something that would easily be reflected in their stats.

Quote

How to gather info and statistics on this? I have no idea. Just my experience in @3 years of play.

I don't mean to detract from your experience and observations, even though I'm being a bit of a dоuchecanoe and doing it anyways - because really a lot of these things can go both ways.

Also keep in mind that a lot of their bad behaviours are simply exhibited more often by low scoring players than high scoring players. Or at least... I would be extraordinarily surprised if that weren't the case. So a team comprised of lower scoring individuals is more likely to exhibit these mistakes and not be able to overcome them.

MadIrish, on 31 January 2017 - 08:27 AM, said:

I'd like to see this done in faction play not quick play!

Unfortunately, premade sizes is one of the biggest factors in the outcome of FP matches. Also, since there is no matchmaker there at all, the skill disparity is even wider. And we already have a bit of a metric for overall Clan vs IS performance in that realm - it's the battle log or whatever it's called, as well as the CW map itself. Which to be honest, I never pay attention to anymore because I don't play CW.

s0da72, on 31 January 2017 - 05:52 AM, said:

I wish PGI would share these types of statistics with the community. Seems a shame you have to put so much 'manual' work to get these numbers when its probably all residing in some database somewhere.

Karl Streiger, on 31 January 2017 - 06:06 AM, said:

I'm pretty sure Tarogato had asked.
I know that I had - and the answer by Tina*) came promt - "NOPE" ok she did use different words
or Patience????

I know that Bandit and the MRBC staff have been pressing hard for some private match API so that screenshotting for competitive league matches becomes a thing of the past. It follows that if a private match API can be done, then it can easily be applied to quickplay and factionplay as well with little or no fuss. Last I heard, it sounded like it was still an idea that's on the table, and PGI isn't completely completely opposed to it.

But just bulk statistics shared by PGI? Nah, probably not ever happening. I think the Tukayyid statistics is the best we ever got and will ever get. But I wish PGI would just fess up and make everything public. It makes my mouth water when I look at League of Legends or World of Tanks and they have sites that show you the winrates of each champion/tank, and you can see (especially for Riot/LoL) in the patch notes as they address these outlying performers with nerfs/buffs as necessary throughout the seasons. I really wish PGI would take that approach and that level of transparency with balance. But instead... we just mill about on the forums making conjectures until somebody like me comes along with data analysis, and even then it's still not the whole story and conjecture still plays a large factor.

Just disappointing, ya'know?

Edited by Tarogato, 31 January 2017 - 09:42 AM.

[a s s a i L]

The sheer volume of work here hurt's my head.

I say this personally thank you for taking the time to do this, and from what it looks like so does the community

Amazing work Taro.

[MrMadguy]

You forget about one of most important factors - maps. Some maps are very Meta-biased. Sometimes when I play certain 'Mechs/Builds and some specific maps win voting - I'm being effectively removed from match, i.e. -1 player for my team. And as many players prefer good maps, like Canyon or HPG - many other players feel themselves exactly the same. So no wonder, that sometimes half of team simply can't play on this map -> stomp.

[Karl Streiger]

Don't do the generalgeneral mistake to create cases where stats might be incomplete
Of course, with more data stuff like maps, game modes and loadout became more and more unimportant.

About the lobby data API this was exactly what I've asked for and the "No" seemed to be as solid as mountain

[Amsro]

Khereg, on 30 January 2017 - 02:26 PM, said:

Agreed, but I view it as a corollary to what I said - we know the "good" players in the sense that those players both have individual ability and also aren't likely to do something boneheaded from a teamplay perspective.

I've been surprised, though. I recall a particular match a couple of weeks ago where virtually every player on both teams had name recognition for being unspud-like. I thought, 'hey. this should be interesting".

Domination on Crimson. My team literally failed to move to the circle until I raced in with less than 5 sec on the clock with a PPC HBK (which is decidedly NOT what you do with a PPC HBK). To no one's surprise, I got killed in 30 seconds by a couple of light brawlers while our other team members trickled in and got killed one-by-one. It was the most talented Tier 5 match I had ever seen.

I chalked it up to the idea that everyone was used to hanging on the outside during domination while the lower tier players held the circle and it backfired when literally EVERYONE tried to do it. Scrubs. All of us.

I find most of the better players tend to avoid the circle on most maps, this is because the game mode sucks and derping into the circle is a good chance of death.

[lpmagic]

while I am a HUGE fan of statistical analysis, I think this would be more accurate if you were to ask, say 50 active players (a mix of comp and non) and ask them to play a set amount of matches and report back results. for instance, my stats suck bald face truth, however, fully half (or more) of my matches are specifically to level mechs, or learn a new position in hopes of getting a chance to get a drop in a comp match. I digress,

TL/DR a true killer spread sheet would have a number of players, playing for keeps, run your best and don't test, and then compile some statistics, I spend entire evenings testing mechs etc...I know many do as well, since many people could care less about their stats (some do of course) this might lead to interesting datum....I suspect that it would unbalance this quite a bit.

[MrMadguy]

There are stubborn players in this game, who intentionally vote for terrible maps, because they want to make their game more "diverse". They don't even understand, that we hate this maps for reason. They don't understand, that via doing it - they simply gimp themselves. If you want balanced games - vote for Canyon, HPG and Mining. Remember! Not all players play ER-LL/Gauss vomit builds! Some players know, that MechLab exists in this game.

P.S. I'm really curious, if PGI will ever fix matchmaker, will they compensate hundreds of stomps, I have to suffer due to being unable to drop my rating to appropriate level?

Edited by MrMadguy, 31 January 2017 - 10:22 AM.

[Khereg]

MadIrish, on 31 January 2017 - 08:27 AM, said:

I'd like to see this done in faction play not quick play!

i saw this a couple times on this thread, so not picking on you, but I have to ask, "What's the point of that analysis?"

The whole reason Taro did this work was to evaluate the source of stomps and whether it was a failing on the part of the matchmaker or some other variable. He chose solo queue b/c that is the most "pure" set of data we have available - random assortments of players unlikely to be using coordinated builds and tactics.

In FP there simply is no matchmaker and the population is so low there isn't likely to ever be one. That means you're down to the other variables that Taro looked at, like:

* Individual/Cumulative Team skill
* Effectiveness of builds
* Clan v IS mechs
* Coordinated builds and tactics/general teamplay

A number of people have looked at this in the past and the general conclusion is that for FP individual and cumulative team skill combined with coordinated builds and effective teamplay tactics dwarfs the other factors, although there does appear to a slight advantage for clan tech over IS tech, all else being equal.

Build effectiveness obviously has an impact but it's generally correlated with individual skill (i.e. "good" players don't bring bad builds).

Since the advantages in FP are related in very large part to the individuals playing and not in-game balance issues, there's really nothing PGI can do short of forcing a matchmaker in FP. Unless the population that regularly plays FP increases by a quantum margin (like a factor of 10x - 20x), that just won't happen.

Edited by Khereg, 31 January 2017 - 10:18 AM.

[Khereg]

lpmagic, on 31 January 2017 - 09:55 AM, said:

TL/DR a true killer spread sheet would have a number of players, playing for keeps, run your best and don't test, and then compile some statistics, I spend entire evenings testing mechs etc...I know many do as well, since many people could care less about their stats (some do of course) this might lead to interesting datum....I suspect that it would unbalance this quite a bit.

So, he should take all the MRBC match results and look at the stomps that happened there? Sounds like an lpmagic project if I ever heard one.

[prox]

Jman5, on 30 January 2017 - 03:35 PM, said:

The team with more clan mechs won 58% of the time.

I hope PGI doesn’t draw any false conclusion based on that. There are 2 overperformers among clan mechs, Gyr and Kodiak (maaaybe TBR and Marauder IIC), that make up a significant proportion of clan mechs in “real” top tier matches. These specific mechs should be brought in line, not blanket nerfs of clantech again.

Edited by prox, 31 January 2017 - 10:24 AM.

[PhoenixFire55]

Jman5, on 30 January 2017 - 03:35 PM, said:

The team with more clan mechs won 58% of the time.

Which clearly has nothing to do with your typical PUG play of hiding and taking timid shots from 600m+ ...

[xTrident]

prox, on 31 January 2017 - 10:24 AM, said:

I hope PGI doesn’t draw any false conclusion based on that. There are 2 overperformers among clan mechs, Gyr and Kodiak (maaaybe TBR and Marauder IIC), that make up a significant proportion of clan mechs in “real” top tier matches. These specific mechs should be brought in line, not blanket nerfs of clantech again.

Agreed. Although I don't know if I'd still say the Kodiak is an over-performer right now. I believe I bought the mech just before it was changed/nerfed and it was nothing short of awesome for me. High damage/multiple kill - KMDD matches over and over. Since though I feel like a shell of what I was once able to do with it. It's still nothing to mess with, but it doesn't feel OP to me now. Hell I brought my Atlas back out a couple nights again to try it again and I ran it just as well as my Kodiak. Before the nerf it wasn't close, Kodiak was way better. I see the Gyr doing some great things as well as the Marauder, like you mentioned. But I also still think the IS Marauder is still pretty good, at least in sponging damage. Always seemed like it would take damage as well as the Atlas to me.

But I'm with you, I don't want to see any big reactions, at least in quickplay where mechs are mixed together. I still have my IS favorites that perform very well when compared to my clans.

[DaZur]

Nice effort on the testing and data gathering... kudos.

My premise has always been that stomps were the outcome of a statistical norm. I.e. compounding force strength over attrition...

You've added a granular break out to it I wasn't paying attention to and it's pretty intriguing!

[4rcs1ne]

Great post Tarogato!

I think the reason PGI doesn't share this kind of data is because it would reveal the flaws in their matchmaker algorithm.

[MrMadguy]

Lol, noobs simply don't know, how to play this map. Most of this guys have even worse stats, than me. 95 avg match score in 249 matches???

Edited by MrMadguy, 31 January 2017 - 11:05 AM.

[ProfessorD]

@Taro

Do you have any plans to get some Tier 5 potatoes to collect an equivalent dataset and see what happens? I'm betting we could find some volunteers for the sake of science!

Edited by ProfessorD, 31 January 2017 - 11:29 AM.

[rook]

Is there any correlation between between lower mech weights doing (slightly better) and more clan mech doing better? That is, can we assume that clan mechs are lighter but care as much firepower and that's an advantage if you want to have a stomp?

[GabrielSun]

It's an interesting post, but you can't draw any kind of conclusions from the data. It's all correlative and doesn't mean a thing to the results in general other than that people with higher w/l tend to win more often (which is really a circular analysis). I think to find causation you would need to consider weapon types, firing groups, component damage, overheat events, pinpoint versus spread damage types, and the amounts of each of those types of damage. That's all in addition to the types of mechs doing them, whether IS or clan or assault or medium.

[Star Dust]

I admit I haven't read through this topic and do not intend to so due to its length and as a result I may be repeating what others have already mentioned.

With respect to the explanation of the tonnage effect, in addition to your proposal for the nascar effect, I also find it not uncommon for a good, or couple of good lights to flank attention from the other team, especially on maps like grim where the chasers totally expose their backs to incoming direction of fire and are out of the match quickly as a result.