187 replies to this topic

[Tarogato]

GotShotALot, on 02 February 2017 - 10:19 AM, said:

The thing that strikes me about this (very interesting) analysis is that the difference between the stompers and the stompees is in fact fairly small.

In World of Tanks, where very similar MM issues occur, there was a great post a few years back showing how the first few losses of the match often led to the stomp-roll. Basically, when 1 mech/tank happens to encounter 2-4 foes and goes down quickly doing little damage, the scales start to tip. When the 2nd vehicle does the same, it's now 10 guns against 12. This leads to a vicious cycle where mech after mech gets seriously outgunned and taken down before it can deal proportional damage.

Obviously, this does not always happen, and good gameplay and strategy can come back from some losses. But it is the way things generally trend and thus the frequent roflstomps.

In which case, stomps are more of a 'chaos of the battlefield' result and less of a 'MM team selection' issue. Yes, on average, slightly better teams will cause those first couple early deaths to happen somewhat more often. But 'better' MM won't significantly cut down on them.

In that game, in regards to stomps, has anybody ever done studies via XVM or however else you fetch stats? It seems pretty interesting at first that the first player or two dead tips the scales and increases the likelyhood of a stomp, but that way I'm looking at it is... what incited those first few kills in the first place? Experienced players preying on inexperienced players, systematically removing tonnage from the field to the point where the good players on the losing team can't pull it back? Because if that is what is happening, then matchmaking is to blame (and I know WoT doesn't even have a skill based matchmaker... I don't know how people handle it. Potatoes... potatoes everywhere.)

[MischiefSC]

I think it's funny that clan/IS is one of the biggest single variances.

Correlation doesn't equate to causation, it's likely that better players just gravitate to better mechs and Clans currently have the best mechs in each weight class.

The irony however is not lost.

[Dimento Graven]

MischiefSC, on 02 February 2017 - 11:27 AM, said:

... it's likely that better players just gravitate to better mechs and Clans currently have the best mechs in each weight class.

...

I wouldn't put it quite that way, maybe: "...it's likely that better players earn more money having access to the better MC purchasable 'mechs more quickly than the average or below average player would..."

I mean, I love to play in my KGC, and while it's fairly decent IS assault, it is far from being truly comparable to most of the Clan assaults. Heck, my NTG can out perform my KGC simply because of the 10kph speed, JJ, high weapon mount advantages it has over my KGC.

I really like my NTG and tend to score better in it, but I love playing in my KGC.

[GotShotALot]

Replies to Dimento Graven and Tarogato: (quote button never works for me, sorry)

Dimento: Yes,those small percents do add up, in the statistical way. Meaning, in any given battle, 5% better stats on one side won't matter much (player piloting choices will make much more difference). But in the long run, the side with the 5% will start the steamroll a disproportionate amount of times by getting in that first kill or two. Leading to the results as shown.

Tarogato: Yes, people have done XVM analyses in WoT many times, lots of interesting results. The XVM mod even has (or used to, I haven't loaded it in ages) a calculator for win chance prior to each battle. Which shows just how often having a 'stats based' calculation of team makeup doesn't 'predict' anything - plenty of battles with a 65%+ win chance for one side go to the underdogs.

What it amounts to is, if 4 mechs on one team go to point A and 2 mechs on the other team do, most likely result is 2 dead mechs for one team and a couple lightly-damaged mechs for the other.

Even further, if 4 mechs go on one side, and only 3 on the other, but the 3 are faster mechs, in position and waiting as the opposed 4 come in one by one, boom, mech imbalance begins.

Other causes: One match recently our side had a moderate number of LRMs, but we also had a light who was absolutely amazing at running around and both NARC and TAGing enemy mechs one by one. We just LRMed the current target while a couple fast mechs strafed and they went down like dominoes. 12-zip.

So, positioning opportunites, weapon or speed imbalances, having that one player on your team who's always gotta suiscout, etc, these things get the stomp rolling.

My point was, the MM can match stats to 100% balance and it won't prevent these stomps from happening due to battlefield choices. In which case, how much MM fiddling is worthwhile given that we don't exactly have an extensive player queue to draw from?

[Dimento Graven]

GotShotALot, on 02 February 2017 - 12:36 PM, said:

Dimento: Yes,those small percents do add up, in the statistical way. Meaning, in any given battle, 5% better stats on one side won't matter much (player piloting choices will make much more difference). But in the long run, the side with the 5% will start the steamroll a disproportionate amount of times by getting in that first kill or two. Leading to the results as shown.

Considering that the 5% adds up to entire 'mech, if one side is has a cumulative 5% performance start, that's like having an extra 'mech on their side, or like the enemy is starting with only 11 'mechs.

Yes, that means they are definitely more likely to get that first kill and start that stomp ball a rollin'.

Quote

What it amounts to is, if 4 mechs on one team go to point A and 2 mechs on the other team do, most likely result is 2 dead mechs for one team and a couple lightly-damaged mechs for the other.

Even further, if 4 mechs go on one side, and only 3 on the other, but the 3 are faster mechs, in position and waiting as the opposed 4 come in one by one, boom, mech imbalance begins.

BUT, if those 4 'mechs are shooting 5% more accurately (or rather, are doing 5% more damage) than any other 4 'mechs on the opposing side, it's more than likely they'll come away with AT LEAST causing significant damage to the enemy team before dying, if not outright taking down multiple enemy mechs, allowing the rest of their side to take advantage of the disparity.

In other words, you can make the smartest tactical/strategic decisions in the world, but if you can't shoot worth a flying F-bomb, it's all for not.

Quote

Other causes: One match recently our side had a moderate number of LRMs, but we also had a light who was absolutely amazing at running around and both NARC and TAGing enemy mechs one by one. We just LRMed the current target while a couple fast mechs strafed and they went down like dominoes. 12-zip.

So, positioning opportunites, weapon or speed imbalances, having that one player on your team who's always gotta suiscout, etc, these things get the stomp rolling.

I would argue your light's performance was significantly better than the enemy team could counter. NOW, had the enemy team had enough players who could aim and shoot, those lights would have been killed off early, and the stomp would have more than probably gone the other way...

Quote

My point was, the MM can match stats to 100% balance and it won't prevent these stomps from happening due to battlefield choices. In which case, how much MM fiddling is worthwhile given that we don't exactly have an extensive player queue to draw from?

Battlefield choices are only as much a factor as skill and performance.

People doing seemingly stupid things can often end up with an unexpected win:

While not a stomp, it was a half-assed strategy done rather haphazardly, but our team performed significantly better, on average, than the enemy even though, for the most part the enemy had the strategically important high ground.

[InspectorG]

Tarogato, on 31 January 2017 - 09:31 AM, said:

I don't mean to detract from your experience and observations, even though I'm being a bit of a dоuchecanoe and doing it anyways - because really a lot of these things can go both ways.

Also keep in mind that a lot of their bad behaviours are simply exhibited more often by low scoring players than high scoring players. Or at least... I would be extraordinarily surprised if that weren't the case. So a team comprised of lower scoring individuals is more likely to exhibit these mistakes and not be able to overcome them.

Dont sweat it, good criticism is good.

Yes, much of the above can go 'both ways', but, i think the conditions i mentioned offer the enemy an opportunity IF they can capitalize on it, like you said. Very much the case.

Ive had many matches where my team came back from an impressive deficit for the win. Were we that skilled? Or did the enemy lose because perhaps the goods died or they ran dry, or gave up position? Very hard to say, but im guessing that in more organized play between more skilled players, the margin for error closes because skilled players will take advantage and be less prone to making tactical errors themselves.

Eg.

Camping CAN work IF blues act as a team and camp a good spot on the map and punish, via concentrated fire, anyone who pokes on them.

Is it safe to assume the above in a Solo match with no drop commander? I lean towards 'no'.

Is there even a way to measure this?

Another interesting facet would be to track my W/L if i started calling drops? For Science!?!?!?

[GoatHILL]

I may have missed it but I'd like to see battle times for these matches. I've noticed that most of the 12-0 games I'm in go very fast 1 side gets the momentum and just steamrolls the other team.

[Almond Brown]

Quote

"or maybe matchmaker just dropped the ball and one side had better players.

It is quite silly really that the MM does not ask each player, at the start of each match, some simple but vital questions.

"Do you Suck at MWO?" Y/N (depending on the answer, you are either assigned, as excess baggage, or to the appropriate Winning Team)

"Are you a Potato?" Y/N (depending on the answer, you are either assigned to the appropriate Losing Team or you are expected "try to carry" all the true Potatoes on the Team to its inevitable Loss.

Because everyone knows if you don't "Suck at MWO" you will Win and if you are a "Potato", you will Lose. MMing issues solved. LOL!

Edited by Almond Brown, 03 February 2017 - 08:54 AM.

[Tiger Dad]

Taro, I was just linked to this page and I read it through. I don't know you except for the times I see you and chat with you a bit in game. I don't know what you do in RL. I don't know what you look like in RL, nor your style. But the passion I see you put into the numbers and analyzing this game and the effort and work. Man, I so admire you and respect the heck outta you. If you ever, ever, EVER left MWO, the community and even PGI would have lost a great resource and champion for this game. Thinking man's game was what they call MWO. And you definitely are that.

[Xetelian]

While every game strives to perfect that match making and every game seems to have issues with that, this game seems to have significant issues.

[KodiakGW]

Tarogato, on 30 January 2017 - 01:43 PM, said:

Now it looks much more balanced PGI, why can't your matchmaker do something like this?

Nice work. Thanks for doing that. Problem is, unless you are one of the white knight NDA holders, they will never listen to you. The direction of the game and all changes are influenced only by them and the whales.

[Tarogato]

This is a re-post of the numbers in the OP, but with all 12-2 matches removed, to compare the results and see how much of an impact the 12-2 matches had on the findings. This is out of concern that since I did not include ALL 12-2 matches (I only included ones that felt like "stomps"), that the data could be considered cherry picked. Also, any findings that significantly differ between these two versions could render both versions invalid, due to the fact that neither is more correct than the other, and it is unfair to choose one over the other after already knowing the results of both (data peeking).

Because the formatting/layout is identical, you could compare the versions by viewing them side by side in two browser windows.




THE FINDINGS:


Total matches collected: 71
Total unique players: 1,533
Total unique chassis: ?? too lazy to recount
Total unique variants: ?? too lazy to recount


Average match score:
This is each pilot's average match score on the QP Leaderboard, not how they actually performed inside the match itself.

- Winning team average: 240
- Losing team average: 226
6.5% advantage to the teams that won.

- The team that won had the advantage 52 times (average of a 10% advantage)
- The team that lost had the advantage 19 times (average of a 3% advantage)



Win/Loss Ratio:
This is each pilot's cumulative WLR on the QP Leaderboard

- Winning team average: 1.17
- Losing team average: 1.07
9.3% advantage to the teams that won.

- The team that won had the advantage 55 times (average of a 15% advantage)
- The team that lost had the advantage 16 times (average of a 7% advantage)



Kill rate:
This is each pilot's cumulative kills per match (not KDR!) on the QP Leaderboard

- Winning team average: 0.87
- Losing team average: 0.78
12% advantage to the teams that won.

- The team that won had the advantage 51 times (average of a 18% advantage)
- The team that lost had the advantage 20 times (average of a 6% advantage)



Death rate:
This is each pilot's cumulative deaths per match on the QP Leaderboard

- Winning team average: 65%
- Losing team average: 67%
2.9% advantage to the teams that won.



Matches played:
This is each pilot's cumulative matches on the QP Leaderboard

- Winning team total: 1,197,050
- Losing team average: 1,081,288
Teams that won play an average of 10.7% more QP matches



Tonnage:

- Winning team average: 65.2
- Losing team average: 65.1
0.1% advantage to the teams that won.

- The team that won had the advantage 34 times (average of a 3.5% advantage)
- The team that lost had the advantage 37 times (average of a 3.2% advantage)



GMan Tier Rating
Lower is better. This is each mech's tier rating according to GMan129's site, MetaMechs. These tier lists are the opinion of only one player, so they are not absolute, but they are widely regarded as "pretty close."

- Winning team average: 2.1
- Losing team average: 2.2
5% advantage to the teams that won.

- The team that won had the advantage 38 times (average of a 18% advantage)
- The team that lost had the advantage 33 times (average of a 15% advantage)



GManTonnage
Higher is better. This is mech tonnage adjusted by GMan tier rating. A tier 3 mech will be worth its facevalue, while a tier 1 mech will be worth more tons and a tier 5 mech will be worth fewer tons. The formula I decided on is [ Tonnage + (12 * (3 - Tier)) ]

- Winning team average: 75.5
- Losing team average: 74.4
1.5% advantage to the teams that won.

- The team that won had the advantage 41 times (average of a 8.6% advantage)
- The team that lost had the advantage 30 times (average of a 7.7% advantage)



Clan vs. Inner Sphere
Number of each tech base per side.

- Winning team: 40% IS, 60% Clan.
- Losing team: 49% IS, 51% Clan
The winning team had 18% more clan mechs on average.

[Tarogato]

And it's now that I realise I failed to point out one of the most important statistics here - how often the hypothesis was true. And here's comparing the two versions:

	12-2 included	12-2 excluded
Average match score:	75.9%	73.2%
Win/Loss ratio:	80.1%	77.5%
Kill rate:	75.9%	71.8%
Tonnage:	41.4%	53.5%
GManTonnage:	56.0%	57.8%

[Pr8Dator2]

Yes, average match score is the only indicator of player "level" in my opinion... PGI should just scrap the Tier system and matchmake based on average matchh score. Otherwise, simply use the average match score as a huge determinant in the PSR index.

[MW Waldorf Statler]

-what is with One Day Players? second Accounts ? not all Guys in all classes have the same Performance ,Weapons builds ...have a T1 Player in a Light the same WLR as in a Assault, with 3 xT5 new Players or 5 xT2 Players, with LRMs same like gauss or Lasers? all play only the same Mech with the same builds ?thats a Good Statistic for only Metaplayers in Comp, not for Quickplays with guys thats never before played a PC Game or a MW Game

Edited by Old MW4 Ranger, 23 April 2017 - 01:20 AM.

[nehebkau]

So, it's been a Year since Taro put this up and, the question is:


What has PGI done with this excellent infromation?


How bout we talk about how it feels like MM is still just "phoning it in"?

[Brain Cancer]

Tarogato, on 30 January 2017 - 01:43 PM, said:

The team with higher collective WLR was on the winning side of a stomp 80% of the time. This was the strongest correlation of all the variables I examined.

The next strongest factors were QP average match score and kill rate.

Here's your proper way to determine tiering. W/L, match score, kill rate. Figure a formula that makes all three relevant, apply it to the last year or so of matches per player, and divide them 1-5 accordingly per 20% of the player base in each tier.

Then do it again every month. Or even split it 10 times if you wanted to.

[thievingmagpi]

forgive me if I missed the math on this, but can't average score can be sort of difficult and not always transparent?

a high scoring player and a low scoring player may equal each other out statswise- a guy with 100 avg score and a guy with 500 is the same as two dudes with 300 for purposes of "average".

I would what something like the standard dev or some other metric on this figures would be?

[Rededevil]

Thank you for your thoughtful work on this - and although it is on a limited amount of matches, I feel you have highlighted what may be something that PGI could use to tweak the matchmaker system a bit - making teams a little more balanced. Would it work every time? Probably not... but keep in mind that they have access to a lot more information about game breakdowns.

I feel this might be something for them to look at in order to make the game better. Even in the case of smurf accounts, the system you described would actually help to level out the skill levels of the teams overall. I feel that this could be a plus.

It is observations like this that (should PGI investigate) could result in a more balanced experience for the players.

IMHO

[Dr Cara Carcass]

Tarogato, on 30 January 2017 - 02:41 PM, said:

I agree. I'd like to do that follow up. Though... I might have to request some crowd-sourcing for the screenshots and data entry. Entering in all those names and mechs gets old after a while (and I'm not sure I'd trust an OCR output, nor do I have such a resource myself).




Oh I agree completely. Basing something off of Metamechs is immediately dubious. I would have only put stock into it if there was a very strong correlation to winning. But... it was rather inconclusive, so better to just ignore it for now and save it for later.

how do i send you the screenshots? any particular mail adress or something like that?
If you still care to increase the numbers after a year

Edited by Cara Carcass, 17 January 2018 - 05:36 AM.