187 replies to this topic

[MrMadguy]

When stomp happens, what I usually see - is one team being very active, performing coordinated rush at other and other being too passive, being unable to defend itself, just standing and allowing enemies to simply come and stomp them. It's not about communication - players simply know, what they should do in order to win, i.e. how to play on certain map, what route to choose for example. Players in losing team simply look like much less skilled - they prefer stupid "stand behind cover and wait for enemies to come, instead of supporting team mates, who attack/defend" tactic, or even worse ones, like picking suicidal route on map. I won't list all kinds of noob behavior - it's very long.

So, Tier 1 vs Tier 1 or Tier 2 vs Tier 2 - doesn't really mean anything. What MM fails to do - measure real skill level. One Tier 1 players may be extremely skilled and other can be - complete noob. For example: I'm in Tier 3 and MM expects me to play at Tier 3 level skill. But I don't have Tier 3 skill, sorry. That's why when you see me in your team - be ready to lose.

For example: I played 5-7 matches yesterday. Won 1 only - last one. Killed 3 enemies due to pure luck. I was in Battlemaster with unfinished build - not leveled, no modules, STD engine, some weapons and heatsinks missing. Map was good (HPG) and enemies were noobish - Battlemaster didn't support his EBJ team mate, who almost killed me (cored CT), noticed me due to getting my pulse laser shots into his back, started to pursue, but, lol, allowed me to backstab him via going around pillar, like I was in Light 'Mech. Bad thing - I don't record videos. His death was spectacular - Battlemaster died mid air, while trying to jump down from edge of platform, due to thinking, that I jumped too. Third 'Mech - was last one. I simply was lucky enough to finish him. Without this match my stats would be complete disaster:

W/L = 12:12 and K/D = 8:19, yeah.

Edited by MrMadguy, 30 January 2017 - 11:01 PM.

[Nik Reaper]

Well, as usual , the question is there any kind of metric that is available to any company that describes your current state to the matchmaker in any level of completeness?

A bad day, a good day, chemical stimulants, sickness and so on, also one thing that can't be accounted for is how well you gel with the teammates you get, unless you get to review every team mate after the game as how likely you would like to have them on your team again... not a terrible idea there X) .

Though averages are the only real thing one can work with until out AI overlords take over everything and can pay attention to and take everything in to consideration. It is hard to tell what even small changes to the formula of an MM would ultimately do but I'm all for more often attempts to try new things that make sense, though with a small sample size of players, unevenly distributed over a skill curve, appearing in different time zones that seems all the more difficult.

[Pineapple Salad]

This is exactly the kind of statistical analysis I needed so I wouldn't have had to throw random numbers to my own solution.

But I think this goes even further, maybe the differences would be even clearer if you included any kind of matches. As we know, a single good pilot may turn a 12-1 loss to a 12-5, but that doesn't save the team mates from doing sub 200 damage games. The average skill in the winning team may still be significantly higher, even if the end score does not indicate an outright stomp.

[MrMadguy]

Nik Reaper, on 30 January 2017 - 11:06 PM, said:

Well, as usual , the question is there any kind of metric that is available to any company that describes your current state to the matchmaker in any level of completeness?

A bad day, a good day, chemical stimulants, sickness and so on, also one thing that can't be accounted for is how well you gel with the teammates you get, unless you get to review every team mate after the game as how likely you would like to have them on your team again... not a terrible idea there X) .

Though averages are the only real thing one can work with until out AI overlords take over everything and can pay attention to and take everything in to consideration. It is hard to tell what even small changes to the formula of an MM would ultimately do but I'm all for more often attempts to try new things that make sense, though with a small sample size of players, unevenly distributed over a skill curve, appearing in different time zones that seems all the more difficult.

Implementing precise measure - is second task. More skill-related things should be rewarded, instead of pure dmg - such as accuracy, dmg efficiency, i.e. dmg dealt/whole dmg ratio for example. But first task - to make MM at least work properly with current skill measurement system. As I've said many times already - your expected average performance is W/L = 1 and avg Match Score = 200..250. My average Match Score = 100..150. It's 100 points below expected. PSR rating refuses to drop. And IT SHOULD.

Edited by MrMadguy, 30 January 2017 - 11:41 PM.

[Lily from animove]

thats interetsing as it shows MM isn'T too bad doing the job and only slight differnces (on the aveage) lead to big differences in the result.

However I would like you to make another statistic:

take of each match the 3 pilots doing the least damage and the 3 doing the most damage.
analyse in which team they are by %. and also what those peoples QP W/L K/D ratio is.

I would like to see how the extremely well performing or the extremely bad performing pilots affected the chance in stomping.

Further I would like to see how the below 100 damage pilots distributed between the stompers and getting stomped ones.

I know it's not all about damage, but surely damage still reflects to some degree how these people distributed and how much of "carry hard" had influence on the stomps.



I am not sure how you did gmans tier rating because he mostly rates on specific builds, so you can not very accurately define his rating accuracy by not knowing if people used those builds or just that chassis/vasiant with a bad loadout.

Edited by Lily from animove, 31 January 2017 - 01:44 AM.

[NighthawK1337]

Tarogato, are you a statistician by trade? those are damn good observations.

[Duke Nedo]

Didn't read all comments, but did you try to merge some of these variables into one "meta-score"?

For example, you have the tiers of mechs and the average score of the pilot, have you tried to merge these two so that a skilled pilot in a bad mech is worth about the same as an average player in an average mech etc?

[Raggedyman]

Tarogato, on 30 January 2017 - 01:43 PM, said:

Players that tend to win more will more often be on the winning side of a stomp. (I KNOW, GROUNDBREAKING SCIENCE, RIGHT??!!) The team with higher collective WLR was on the winning side of a stomp 80% of the time. This was the strongest correlation of all the variables I examined.

First up, thank you. That's some pretty nice work you've done there.

The only question i have on this though is "how often do stomps happen?", because although they are annoying if they are rare then they aren't really an issue, and I would love to know if you have any data on that. I'd also be curious to know if you have any data on what the factors are for a close matches (12 to 6 and 12 to 11), as finding out what is working can often result in a simpler fix.

[xTrident]

Dakota1000, on 30 January 2017 - 06:07 PM, said:

Would be nice if we based match maker off average match score instead of tier. We could even do average match score per mech and have it be rather volatile for the first few matches until it stabilizes.

Yes, exactly. Average match score can always be changing so I do believe it'd be better than, essentially, an experience bar. Frankly... If PSR would actually fluctuate rather than basically constantly increasing it would just our average match score.

Edited by xTrident, 31 January 2017 - 05:53 AM.

[s0da72]

I wish PGI would share these types of statistics with the community. Seems a shame you have to put so much 'manual' work to get these numbers when its probably all residing in some database somewhere.

[Karl Streiger]

s0da72, on 31 January 2017 - 05:52 AM, said:

I wish PGI would share these types of statistics with the community. Seems a shame you have to put so much 'manual' work to get these numbers when its probably all residing in some database somewhere.

I'm pretty sure Tarogato had asked.
I know that I had - and the answer by Tina*) came promt - "NOPE" ok she did use different words

or Patience????

Edited by Karl Streiger, 31 January 2017 - 06:07 AM.

[xWiredx]

Tarogato, thank you for the analysis.

Yes, that is all I had to say about this.

[Yanlowen Cage]

Sifting through old screen shots of my own. I have a few odds and ends to throw in there.

Missiles boats are not useless if they are used correctly. For example a atlas lurm boat (to me) is not a good use of a lurm boat. too much armor and firepower wasted standing back. now a 5 lrm 5 3 med lasers Kintaro golden boy (if used properly) is a good missile boat. How To use you ask? The aforementioned atlas is a brawler. So as the Golden boy I support the atlas and suppress his target as he closes into range. Saving his armor and damaging up the enemy for him to finish. Also a lurm pilot with skill can help a team mate disengage from a bad fight. I consider a lurm boat when used properly a massive plus to that team. I play in tier 2 btw. So it is not just unskilled players that get frustrated by lurms. So Lurms can not be counted as a factor for a teams loss or win.

Secondly Many assume that when a team plays together that they will do better. And that is true. But! Often times when just part of a team works together then it can also swing the game (sometimes wildly) in favor of that team. For example I recently played in a game where I (in a raven) and a locust pilot worked as a team. The only communication I heard was between me and him. we successfully trapped individual mechs in the enemy rear and destroyed or distracted a large portion of the enemy team. effectively giving our team a numerical advantage in the main fight. Which led to a big win for our side. 12-4 or some such.

Thirdly a single player can swing the game while not providing a huge damage or match score. Example I was playing a Commando on polar highland and got behind the enemy. For 80% of the game I just stood there spotting enemy targets in the open for our lrms and calling enemy movements. It was a 12-0 stomp. I personally got a crappy score. but the enemy had no idea I was back there and was there fore unable to counter the superior intelligence of our team. Which led to their stomp.

So there are many factors that match maker can not extrapolate (intangibles). And they can have a huge impact on the game.

A few more examples.
An assault that will sacrifice himself to get a push into the enemy to break the enemy. Team mates will often follow an assault that will "soak" damage for them so that they can get into range.

Map type and team composition. One team has many more effective weapons loadouts for the map. Like one team has a larger number of brawler builds on polar highland.

People Speccing after death that actually give useful advice or help the pilot spot targets or point out weak spots on the enemy mech/team/formation.

People calling targets weak spots. (huge!) why shoot the legs off an Timberwolf when his has a missing torso and the other side is crit.

These are not primarily highly skill players abilities, but higher level players do tend to do these things.

Oh and one last thing is mindset. when you realize this is a team game and that sometimes the team wins even if you don't personally win. big score, high damage, and such. Sometimes you DO have to take one for the team.

[Paigan]

Tarogato, on 30 January 2017 - 01:43 PM, said:

[...]
These findings support my theory that either the PSR system or Matchmaker does not adequately assess the skill level of individual players. I fear that the Matchmaker only sees PSR Tier and nothing else.
[...]

This is the first complaint/suspicion I read about MM not working properly that is competent enough to be believable.
Well done and thank you for the work.

(Now ... who's gonna tell PGI? )

Edited by Paigan, 31 January 2017 - 06:51 AM.

[Mawai]

A couple of comments.

As far as we know (based on information in the Karl Berg thread and elsewhere) the matchmaker uses the actual PSR values and doesn't use the tier at all except in excluding certain players from playing against other players.

It was an interesting piece of work. One question would be: What is a good metric for player skill?

The numbers you have could potentially be used to figure out what factors contribute to balanced matches and perhaps design a metric for assessing player skill based on the analysis of available metrics. The sad aspect to this is that PGI has access to FAR more data, more easily obtained and yet has apparently done NOTHING towards resolving this issue.

Finally, having one value for player skill is likely not effective since a player may not be equally skilled with each weight class or with different mechs within a weight class depending on loadout and preferred playstyle. The player skill ranking system needs more granularity to capture these differences.

The hilarious thing about this is that PGI could probably issue a contract for a very modest amount of money (5 to 10k?) to someone in the community to sign an NDA and perform this analysis. Writing the data mining scripts involved would probably take at most a few weeks and in the end they would have the data they need at a fraction of the cost of hiring someone.

[Nik Reaper]

Interesting point, if players could add an attribute to a mech via a check box describing a mech , like brawler, sniper, support or as many or as few as needed, could the MM do a better job then? The idea is like how now in DotA games you get to select 2 preferences, a main and a secondary, of witch lane you want to play.

A team would be much less likely to end up with more than needed number of players per position, now MWO is not a game with a rigid structure like that, but mby matchers could be more interesting if they were ether balanced as a balanced composition, ei. similar number of mech profiles per side, or matching specialists against specialists in a number of combinations.

[Lostdragon]

InspectorG, on 30 January 2017 - 09:07 PM, said:

You need to look into the snowball effect.

Im willing to bet Solo matches are more chaotic events than more organized play.

Initial actions in a zero-sum, no respawn environment HAVE(my opinion) to have larger effects on outcomes than in matches where coordination(and skill) can help deter the first snowball.

Like in Chess, where opening moves that concentrate on the center 4 spaces of the board are more optimal(unless you know of a counter-play suited to your opponent's tendencies), MWO in Solo favors that initial lead.

So, in my experience, the factors that lead to snowballs:

Teams not moving as a cohesive unit. (as base as it is, getting to center and grabbing valuable firing lines -in Solo mind you, has to be brainstem level skill demand to be viable unless a skilled leader is present)

Also, Rabbit Runs(essentially a charity match for the other team)

Blindly following a random light(seed-event for NASCAR?, why usually to the team's right side?)

Less skilled or unlucky Light pilot who gets merc'd early.(reduced target choices for enemy team resulting in more focused incoming fire).

Slow Assault left behind(same as above but usually @1-2 minute into match, map depending)

Camping(my favorite sin to chastise).

Cowardice(refusal to 'share' armor).

Improper LRM use or Sniping(subtracts team armor).

Poor early trades.

How to gather info and statistics on this? I have no idea. Just my experience in @3 years of play.

You could gather this data but you would have to watch and analyze a lot of recorded/streamed matches on Youtube or Twitch. I believe you would probably find a strong correlation between getting the first kill and winning the match. I don't think there would be much actionable data about "stomps", I think they are inherent in the nature of a no respawn game mode. Once you are down by 2 or 3 mechs I would guess the chances of you winning are pretty low, comebacks happen but they are rare and that seems to have been exacerbated by 12v12.

[MadIrish]

I'd like to see this done in faction play not quick play!

[Tremendous Upside]

I think the biggest issue with looking at data like this comes down to players individual "expectations" of what it is a matchmaking system is or should be doing. If you're looking at this to prove that the matchmaker isn't providing "equal" teams in games -- you're probably right. It's also highly possible, if not probable that it's doing this on purpose. A matchmaking system that creates the best possible game every single time would be closer to the ELO system we used to have. People generally didn't like that - and PGI scrapped it (likely because it wasn't doing what they wanted it to do). If I had to guess the matchmaker we have is designed to try and keep all players closer to the 50/50 line for wins and losses over time rather than creating the very best matches possible every time out. It's much better for player retention. QP is entry point for this game for everyone. It's there so players can jump in, stomp around and have fun. It's not there to provide some sort of Tier 1 uber-player subculture - no matter how bad some seem to want this to be the case...

As for lopsided blowout games... I fail to see why people get so obsessed by these. In every single sport on earth you have professional teams facing each other and blowing each other out. It's not always because one team is 1000x better than the other. Sometimes it's just a matter of how things break over the course of the match. Other times you just have a complete lack of chemistry/communication between the members of your team. It is what it is. If there's ever a silver lining, those sorts of games tend to end quickly - and then you're back at the menu screen ready to drop again. I'd rather be doing "something" in-game even if frustrating than sitting around watching the stupid circle search for matches instead.

[Lostdragon]

MadIrish, on 31 January 2017 - 08:27 AM, said:

I'd like to see this done in faction play not quick play!

What would you hope to get put of it? FP doesn't have matchmaking, so it seems kind of pointless to me. We already know what the win/loss rate for Clan vs IS because Russ told us, and FP uses tonnage as a balance lever so it wouldn't be apples to apples in that regard either.