152 replies to this topic

[Spetulhu]

But sadly it all is broken. I just spent a FP match (kk, might not have been too effective) on keeping our guys defending and telling them where I saw lots of red. We won, and I was last on any points. Wow, good way to make anyone even try to lead if he's not already in a group!

[EgoSlayer]

Mister Blastman, on 09 July 2016 - 07:37 AM, said:

I expect this to be a polarizing thread. But I hope it can prove to be a useful one and ultimately one other folks are willing to share and contribute too.

This is a thread built on data and statistics, not blind accusations.

This is a thread built on concrete numbers provided to us by the leaderboards.

Premise:
There was a time many moons ago where we had to accept the results given to us by the great matchmaker overlords. Often there would be 12-0, 12-1 and 12-2 stomps and often folks would chalk it up as bad luck or whatever. Sometimes accusations would be thrown around, things such as, "I got a team full of terribads," or "The matchmaker stacked me with awful players and put nothing but average players on the other team!" or, "Screw you all, I just lost 20 games in a row and I'm uninstalling!"

Whatever the case was, the variety was immense. But there was speculation. There was speculation as to what was really happening.
<snip>

While the premise is flawed - the stomps are due to no-respawns and combat loss grouping, the data does appear to be interesting in this case as a measure of the PSR system and how it works with the match maker.

But like the premise being flawed, so is using that data as a measure of the match maker because the match maker isn't using skill to create matches anymore. That requires an elo or elo like skill based system which the masses screamed for to get rid of. The match maker is handicapped because PSR treats people who are not equal skill as the same because they have the same or near same PSR.

Now the match maker is using experience (PSR) and the match maker doesn't know anything about skills. It's a step up from completely random, but it's not using a skill measure to create the teams so match making has got worse since it's implementation. And it can't get better until there is a valid way to create skill based matches, which ironically leads to a higher percentage of matches that end in stomps. The whole reason people screamed for getting rid of the skill based match maker.
Rock, meet hard place.

EDIT:
So really what this is showing is what a joke PSR is. Because the match making criteria is creating equal or near equal PSR teams (Note - really only applies to solo queue, group queue averages throw everything off) and so it was a "good" match by the criteria of the match maker.

Edited by EgoSlayer, 09 July 2016 - 01:19 PM.

[KodiakGW]

Agree completely. For me, MM has been screwy for a while. Going to start off with a game I did some research on July 2nd. I only figured out the player's standing on the leaderboards, and put their numbers next to the names.



The most disparaging part of this is that more of the most highest ranked players all got put on one team. And, guess what? They won.

Now for a most recent match, with data you requested. Now, keep in mind this game took me less than 20 seconds to find. So this should be all Tier 1-3 players.



Winning Team
Match Rank AVG K/D W/L Games
11,160 229 1.06 1.1 376
17,132 207 1.48 .76 30
19,828 198 1.32 .70 85
17,088 207 1.00 .89 67
3,191 285 1.89 1.74 96
12,890 222 1.39 1.16 165
27,481 173 .64 .85 37
29,217 166 .63 1.16 98
2,127 303 2.23 1.60 65
28,544 169 .90 1.17 13
16,436 209 1.26 .88 128
7,853 246 .77 .79 75

Losing Team
Match Rank AVG K/D W/L Games
27,875 171 .55 1.03 255
22,436 190 .50 .58 19
24,763 182 .64 1.34 97
15,211 214 .68 .81 30
22,938 188 .71 1.02 115
4,030 275 2.00 1.32 124
10,097 234 1.02 1.36 200
10,736 231 1.44 1.13 469
4,421 271 1.23 1.13 640
14,279 217 1.15 1.04 101
22,260 190 1.04 .92 266
29,484 165 .77 1.00 115

Some will say it looks pretty even. But, look at the numbers. There is a difference of up to 138 on the average match score of the lowest player to the highest one. These players should not be in the same game. If they think it is working to find players of equal skill level for a match, they are wrong. PSR is broken, and I won't be buying any new mechs until I don't see this anymore. Too painful to level unless they are the meta creep.

And before anybody jumps in to say "CARRY HARDER"...



I cored and killed the HBK-IIC from behind while the rest of my team was ignoring him hiding among us between two pillars. Then I ran into the EXE and solo'ed him. Not too much left to help carry after going toe-to-toe with a ERMPL spam EXE.

[Blue Pheonix]

Mister Blastman,

Great post. I see you are a numbers cruncher like me. This is exactly the type of thing I do as well (look at stats and statistics and crunch numbers).

I hear a number of people saying "who cares about the leaderboard and stats" and stuff like that but this is exactly why tracking each players stats is important! It not only gives each player input into how they are doing compared to their peers but it helps in pairing teams together to get each team as evenly matched against each other as reasonably possible. At least there should be an algorithm (or two) that does this.

This is why I am asking for changes to the leaderboard in an upcoming patch. It will ultimately help player matching and teamplay. Mathmatically, the more accurate the stats, the better the pairing. Randomly pairing people together, while making for a shorter que time, will not make for the most entertaining match. No one likes to get dominated regardless of skill. No one likes feeling like they have to carry a lot. No one likes to get put into games where there just is no chance because the pairing system made one side disproportionately better then the other side. While even with good pairing algorithms there will just be bad games and one team out of two always has to lose, tracking statistics, good stats and good algorithms will significantly decrease "stompings" of one team over another. While it wouldn't be perfect in every single match, it really will make for better team play and gameplay experience for all of us overall.

Here are a few things I would like to see regarding the leaderboards and stats tracked: what do you think regarding each idea and how it will benefit team play?
http://mwomercs.com/...th-leaderboard/

Edited by Blue Pheonix, 09 July 2016 - 01:25 PM.

[Trauglodyte]

The only hole that I'd poke in what you're trying to say, Blastman, is that, because the leader board only started with the June patch, we don't have an idea how good those players actually are. If you looked at my stats, for example, you'd probably conclude that I'm an average player. But, my stats are skewed because I've played only 117 games, which seems like a lot, since the leader board was added and the bulk of those matches have been me leveling the Rifleman and toying around with my Locust. So, are my stats indicative of how good I am or are they tainted because 1) there are a lot of bad players in this game, thus making it hard to actually win and kill other mechs and 2) because I was in a mech that wasn't leveled along with other mechs that are, quite frankly, sub-optimal? For that matter, all of my games are solo games in Quick Play - that, alone, makes the data hard to judge.

We all know that most people that play this game are bad. Those bad players force other players to press harder which leaves less room for error. That pressure ultimately results in land slide losses and, sadly, ends up making the "skill" of the players on the bad team non-representative of how good they are.

Ultimately, I like where you're going. If nothing else, it should show, without a shadow of a doubt, how bad the PSR is for grouping players.

[M T]

There's no need for any of this.

The only thing that needs to be acknowledged is that the match maker still fails horribly at the most obvious instance.

Two competitive teams get queued up as a 10/12~ man against total scrubs.

Yeah, No further discussion required.

[Trauglodyte]

Every game's match maker fails miserably. I saw a legendary rank 7 player get pitted against a rank 21 player the other day in Hearthstone. In WoT, I'm in a tier 5 mech playing against tier 7 mechs with a tier 3 on my team. MMs fail constantly because they're all coded horribly with the intent of getting games played faster while HOPING that they're actually balanced. ELO was better for MW:O way back when because it was based within weight classes. So, you could be a badass in a Heavy but suck in everything else. The only problem with ELO was we were still playing with Groups matched against Solos.

[Tyler Valentine]

We still don't know what tier anyone is (barring forum warriors) or how often they drop with KCom etc. in group queue, so trying to determine how skilled someone is based solely on these statistics is alchemy at best. Still, a fun exercise I'll admit.

Unrelated: FBJ and I were ranked right next to each other on the leaderboard last night. Thought it was funny to see another forum warrior in the next slot.

[MauttyKoray]

You know what the problem with matchmaker is?

MWO has a small playerbase and matchmaker can only do as well as what it is given. The amount of people queueing are what are used for MM and it constantly opens 'gates' loosening the restrictions because there aren't enough people to fill the original.

[Ex Atlas Overlord]

Mister Blastman, on 09 July 2016 - 07:37 AM, said:

The average Win/Loss ratio was 1.3 versus 0.96

Let that sink in. The players on the winning team on average win 35.4% MORE GAMES than the losers.

That means they have a 35.4% higher chance of winning, every single time.

Win / loss.... wins for every loss.

So if player wins N games...they played N+1 total games.

So a player with a W/L of 1.3 -> 1.3 / 2.3 -> .565 -> They win 56.6% of their games.
A player with W/L of 0.96 -> 0.96/1.96 -> .489 -> They win 48.9%

Difference of basically 7.5%

Mister Blastman, on 09 July 2016 - 07:37 AM, said:

The average KDR was 1.57 versus 1.12

The KDR illustrates a 40% differential. 40% more likely to kill something BEFORE THEY DIE.

For KD/R... I would assume it's N deaths caused by player.... and total deaths....aka N+1 (N + the player)

So KD/R of 1.57 -> 1.57 / 2.57 -> 61% chance to kill enemy
and KR/R of 1.12 -> 1.12 /2.12 -> 52% chance to kill enmey

Difference of Basically 9%

Edit: I'm pretty confident in this assumption b/c a kill rate of X enemy kills over zero player deaths yields a 100% chance to kill. (N / N+0 = 1)

Edited by Ex Atlas Overlord, 09 July 2016 - 07:22 PM.

[VictoriaSeymore]

VictoriaSeymore 634 [Average Match Score] 725 [Kills] 74 [Deaths] 9.80 [K/D] 207 [Wins] 52 [Losses] 3.98 [W/L] 260 [Matches]

^ Example of Pugstar Hero Try-Hard Carry. 100% Solo Queue. It's difficult to maintain a good W/L carrying alone. I find myself frequently in matches where the 2nd highest damage player on team is 1000 damage below my own. If I die, chances of losing the match increase drastically, even if I've already done 1000+ damage.

[Mister Blastman]

Here's a couple more:



Bad team 3
Score KDR W/L Games
238 1.4 1.61 87
228 1.41 .98 294
193 .72 1.04 137
177 .89 .79 70
239 1.54 1.06 145
282 1.24 1.40 127
206 1.03 1.15 390
232 1.08 1.16 54
269 1.68 1.47 107
182 .73 .81 471
258 1 1.03 80
195 .75 .86 67

225 1.12 1.11 169 (2029)

Winners
Score KDR W/L Games
191 .96 .76 132
270 1.71 1.43 165
164 .52 .82 203
212 .82 1.38 95
227 1.39 1.09 394
193 .63 1.8 14
207 .73 .90 138
196 .63 .91 109
343 3.22 1.59 171
245 1.09 .85 63
166 .58 1.10 231
242 1.18 1.08 226

221.3 1.121 1.14 161.7 (1941)



Bad team 2
Score KDR W/L Games
252 1.77 1.03 603
229 1.43 .99 288
205 1.15 .77 205
223 .83 .62 113
259 1.16 1.2 207
205 1.17 1.19 306
290 2.31 1.53 139
207 1.03 1.04 96
343 3.22 1.59 171
188 .70 1.01 357
180 .66 1.05 375
216 .73 .80 237
233.08 1.35 1.07 258.08 (3097)

Winners
Score KDR W/L Games
384 3.86 3.0 168
231 1.17 .87 167
212 .90 .88 339
285 1.06 .92 173
287 1.35 2.61 65
312 2.29 1.37 166
220 1.38 .92 352
216 .94 1.13 220
312 2.75 .98 234
250 1.23 1.23 244
204 .73 .95 107
166 .95 .98 99

256.58 1.55 1.32 194.5 (2334)


The second game above is what we'd expect as the winning team has superior stats but strangely significantly less games played.

The first, however, is a bit strange as the game was very close in stats but the opposite in results. I'll chalk that one up to drop commanding and teammates listening and moving tightly together.

[Cy Mitchell]

The first game that Yellonet posted was similar where the team with worse stats was the winner. I do not think you will be able to get any definitive results or conclusions from this data.

For example. if youo have a guy that normally plays with his unit and therefore has a good win/loss and KDR and then he drops with a bunch of PUGs that do not carry or cover for him like his team mates then his stats could badly skew your results. As youo stated the inclusion of Group queue data kind of mucks up the data and therefore can somewhat invalidate the conclusions you try to draw from the results.

I personally think the MM does a passable job of matching teams. It only takes one guy who is willing to call the shots and a few guys who will follow his lead to turn any group of PUG players into a workable team that will stomp a team where everyone is doing their own thing.

When I say "Hi" on VOIP at the beginning of the match and no one answers, I pretty much know the odds are good that we are going to lose.

[Hit the Deck]

VictoriaSeymore, on 09 July 2016 - 07:43 PM, said:

VictoriaSeymore 634 [Average Match Score] 725 [Kills] 74 [Deaths] 9.80 [K/D] 207 [Wins] 52 [Losses] 3.98 [W/L] 260 [Matches]

^ Example of Pugstar Hero Try-Hard Carry. 100% Solo Queue. It's difficult to maintain a good W/L carrying alone. I find myself frequently in matches where the 2nd highest damage player on team is 1000 damage below my own. If I die, chances of losing the match increase drastically, even if I've already done 1000+ damage.

What 'Mechs do you bring into the QP?

[Sagamore]

Hit the Deck, on 09 July 2016 - 08:23 PM, said:

What 'Mechs do you bring into the QP?

Going to venture a guess - Kodiak?

[Davegt27]

waste of time the game is controlled by the computer

[Weeny Machine]

Trauglodyte, on 09 July 2016 - 02:14 PM, said:

Every game's match maker fails miserably. I saw a legendary rank 7 player get pitted against a rank 21 player the other day in Hearthstone. In WoT, I'm in a tier 5 mech playing against tier 7 mechs with a tier 3 on my team. MMs fail constantly because they're all coded horribly with the intent of getting games played faster while HOPING that they're actually balanced. ELO was better for MW:O way back when because it was based within weight classes. So, you could be a badass in a Heavy but suck in everything else. The only problem with ELO was we were still playing with Groups matched against Solos.

I totally agree. I may be really good in a light or medium but put me in an assault...oh boy The whole thing should be coded for classes

[Orbit Rain]

Hey guys, some anecdotal evidence for ya. I believe/ have observed that the newer people in my unit play more than the vets. They still have that wonder of the game, and many challenges and goals ahead of them. So in essence, I think the new and the bad, play more than the old and the good. More anecdote, back in the poptart era, I didn't do it much, but whenever it did, it seemed to put me on a team full of them, whereas if I wasn't there would be none of them lol.

[Blue Pheonix]

VictoriaSeymore, on 09 July 2016 - 07:43 PM, said:

VictoriaSeymore 634 [Average Match Score] 725 [Kills] 74 [Deaths] 9.80 [K/D] 207 [Wins] 52 [Losses] 3.98 [W/L] 260 [Matches]

^ Example of Pugstar Hero Try-Hard Carry. 100% Solo Queue. It's difficult to maintain a good W/L carrying alone. I find myself frequently in matches where the 2nd highest damage player on team is 1000 damage below my own. If I die, chances of losing the match increase drastically, even if I've already done 1000+ damage.

I was going to call this player out earlier but did not, but since you already did....

This player regularly pilots the Kodiak. Especially the Kodiak 3. The "elited" and "mastered" Kodiak with AC's is absolutely the most devastating and powerful mech in this game bar none right now. Power creep. Its armor, ability to carry the weapons it does, its speed and turn rate make this mech an absolute powerhouse. I pilot all weight classes in the game. Trust me, I know.

Unfortunately, since it was just recently released, the Kodiak is only available for purchase with regular cash. If this is not a glaring example of unbalance in this game for recently released mechs (before the enevitable nerf after a patch) in the game that have not gone up for C-Bills, I dont know what is.

This player (and other reasons) is exactly the reason why I made these two threads (see the two links):
http://mwomercs.com/...upcoming-patch/

http://mwomercs.com/...th-leaderboard/

PGI, please keep MWO from becoming "pay to win". Please implement my ideas into the next patch regarding the leaderboard. Especially idea #6.

Edited by Blue Pheonix, 10 July 2016 - 07:48 AM.

[Mister Blastman]

Sagamore, on 10 July 2016 - 07:07 AM, said:

Going to venture a guess - Kodiak?

The KDK-3 is exceptionally deadly in talented hands. Victoria is talented to begin with. Add KDK-3 and you have a nuclear arsenal without equal.