Jump to content

The Curious Case Of The Broken Matchmaker: Bads, Terribads, Chronicbads And Pugstarheroes


152 replies to this topic

#41 MadcatX

    Member

  • PipPipPipPipPipPipPipPip
  • Big Brother
  • Big Brother
  • 1,026 posts

Posted 10 July 2016 - 07:53 AM

I really dig the constructive thread, good work OP.

The only issue I find with the data based on the dataset, is it doesn't taken into account teams that have good callers, which in my experience usually will lead a less-then-average team to victory.

Also, are the good games, the ones in which the groups are statically close, accounted for in this test?

I believe maybe autumn would be a better time for those in the community to undertake this endeavor. As mentioned earlier, Summer is IMHO the worst time to collect matchmaking data for a low population game since you have even less people for matchmaker to match, could skew the results.

#42 wanderer

    Member

  • PipPipPipPipPipPipPipPipPipPipPip
  • Civil Servant
  • Civil Servant
  • 11,152 posts
  • LocationStomping around in a giant robot, of course.

Posted 10 July 2016 - 07:53 AM

The silly thing is everyone calling out "NERF THE KDK-3" fails to realize the pattern that has held for years will be repeated.

It'll get nerfed once it's no longer a cash shop only chassis. Like every single other "OP" design that gets complained about, it'll go through a series of downgrades. Paul is happy to do so, as there's always a new one down the line to repeat the cycle for profit, and the frustration the nerfs cause can only be even more satisfying.

#43 Blue Pheonix

    Member

  • PipPipPipPipPipPip
  • Ace Of Spades
  • 229 posts

Posted 10 July 2016 - 07:55 AM

View PostMister Blastman, on 10 July 2016 - 07:48 AM, said:


The KDK-3 is exceptionally deadly in talented hands. Victoria is talented to begin with. Add KDK-3 and you have a nuclear arsenal without equal.


Yes, you absolutely have to be a good player to get those kind of scores in PUG matches. You put a great player with a great (very arguably overpowered) mech and you get scores like that.

I am so confident that if PGI were to implement my idea #6 regarding the leader boards and start pairing people up based on their match scores (idea #10), this players stats would drop significantly if this player plays more games.
http://mwomercs.com/...th-leaderboard/

I am so confident of this that I would be willing to bet any mech I have on this.

View Postwanderer, on 10 July 2016 - 07:53 AM, said:

The silly thing is everyone calling out "NERF THE KDK-3" fails to realize the pattern that has held for years will be repeated.

It'll get nerfed once it's no longer a cash shop only chassis. Like every single other "OP" design that gets complained about, it'll go through a series of downgrades. Paul is happy to do so, as there's always a new one down the line to repeat the cycle for profit, and the frustration the nerfs cause can only be even more satisfying.


I agree.

Just because the Kodiak will get nerfed when it goes up for C-Bills does not mean this problem will not persist in the future with new cash only mechs. There will always be new mechs. These new mechs statistically have a problem of "power creep" until the inevitable nerf when the chassis goes up for C-bills.

That is why it is so important for PGI to implement my idea #6 in the thread I linked to above. Let cash players have their fun dominating with their mech in its new "cash only" state in matches but keep MWO from being pay to win in regards to the leaderboards.

Edited by Blue Pheonix, 10 July 2016 - 08:34 AM.


#44 wanderer

    Member

  • PipPipPipPipPipPipPipPipPipPipPip
  • Civil Servant
  • Civil Servant
  • 11,152 posts
  • LocationStomping around in a giant robot, of course.

Posted 10 July 2016 - 08:28 AM

View PostBlue Pheonix, on 10 July 2016 - 07:55 AM, said:

I agree, just because the Kodiak will get nerfed doesnt mean this problem will not persist. There will always be new mechs. These mechs statistically have a problem of "power creep" until the inevitable nerf when the chassis goes up for C-bills. That is why it is so important for PGI to implement my idea #6 in the thread I linked to above. Keep MWO from being pay to win (at least in regards to the leaderboards).


You assume that PGI doesn't care about revenue loss. Which is one of the few things we can be assured they do care about.

#45 Mcgral18

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPip
  • CS 2019 Top 8 Qualifier
  • CS 2019 Top 8 Qualifier
  • 17,987 posts
  • LocationSnow

Posted 10 July 2016 - 08:39 AM

Well, I tried one and 5 players aren't on the leaderboard

I'll remove a 3rd on our team (lowest) to balance that out
Posted Image

10 "Terribad" plays on our side, to 2 on their side

Team 1

Score KDR W/L Games
301 1.52 1.08 77
255 2.25 1.11 85
224 1.1 .77 39
119 .24 .76 72
227 1 0.8 36
166 .72 .95 492
430 5.93 4.03 402
478 4.27 1.67 24
243 1.61 1.19 426

avg
271 2.1 1.37 161 (1453)


team 2
Score KDR W/L Games
251 1.05 0.87 58
230 1.79 1.27 250
276 1.59 1 122
232 0.95 0.75 126
262 1.11 1.68 102
264 1.95 2.05 330
357 3.62 2.46 83
181 .42 .35 31
207 .77 .8 235

avg 2
284 1.47 1.24 148(1337)

lol, l33t games


So,

avg1
271 2.1 1.37 161 (1453)

avg 2
284 1.47 1.24 148(1337)


but with 2 outliers in the first team, and possibly one on the second...along with any of the 6 players removed (one had 3 matches, so I took him out to balance)


Something tells me missing 25% of the data might not be entirely accurate...but why are 5 people not on the leaderboards?

#46 C E Dwyer

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Legendary Founder
  • Legendary Founder
  • 9,274 posts
  • LocationHiding in the periphery, from Bounty Hunters

Posted 10 July 2016 - 08:41 AM

I don't need stats to work out in the first few minutes whether I'm on a team that will win or lose.

I look at the people and think he's got a tag lurm boat, she's pure meta, he'll scream useless scrubs in viop before anyone has fired a shot, he/she is a great caller.

That idiot has run off on his own, that bigger idiot has tried to follow him in a far slower mech, we're two down, no three down because that fool is a.f.k at the spawn, checking he has ping, yup as I suspected, a.f.k

Are we grouped but not so tightly we get up everyone's arse if someone stops, she's aggressive he's a chicken, never seen or heard of him etc etc.

We got this so much, or Crap I hope to hell the other team is even worse, aww bollocks, its got him and her in their team, hold on to your arse this is going to be a wasted 7 minutes of my life.

#47 Blue Pheonix

    Member

  • PipPipPipPipPipPip
  • Ace Of Spades
  • 229 posts

Posted 10 July 2016 - 08:42 AM

View Postwanderer, on 10 July 2016 - 08:28 AM, said:

You assume that PGI doesn't care about revenue loss. Which is one of the few things we can be assured they do care about.


This is a good point. Its also why I have my doubts PGI would actually implement idea #6. I still hope that they do.

As per multiple posts on this forum, people like the leaderboards. Sure there are some who do not but a lot do. Who of us has played Mario Brothers, Centipede, Donkey Kong or any other game multiple times just to a higher level and get a higher score? I believe most of us have.

I am one of the ones who cares about the leader boards and the integrity of them. I never bought a chassis or anything else (except for the Oxide) but I am a person who has spent real money on the game (only for mech bays though) and I continue to do so. I went to purchase MC again last night but the site was down. I need more mech bays.

If MWO is "pay to win" in regards to the leaderboards this will be a significant turn off for me. What is the point in trying to get a higher score if I am not on equal footing regarding mechs available to me? Why shouldn't all pilots be on an equal footing in regards to the tools available to them? Not eveyone has the cash to spend when that new shiny mech comes out. I'm not saying I would stop playing MWO but its something I would consider. If the word got out that MWO was in fact not free but pay to win, I am sure there are many others who would feel the same way. PGI should consider this aspect of it as well.

Edited by Blue Pheonix, 10 July 2016 - 08:50 AM.


#48 Weeny Machine

    Member

  • PipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 4,014 posts
  • LocationAiming for the flat top (B. Murray)

Posted 10 July 2016 - 08:49 AM

View PostCathy, on 10 July 2016 - 08:41 AM, said:

I don't need stats to work out in the first few minutes whether I'm on a team that will win or lose.

I look at the people and think he's got a tag lurm boat, she's pure meta, he'll scream useless scrubs in viop before anyone has fired a shot, he/she is a great caller.

That idiot has run off on his own, that bigger idiot has tried to follow him in a far slower mech, we're two down, no three down because that fool is a.f.k at the spawn, checking he has ping, yup as I suspected, a.f.k

Are we grouped but not so tightly we get up everyone's arse if someone stops, she's aggressive he's a chicken, never seen or heard of him etc etc.

We got this so much, or Crap I hope to hell the other team is even worse, aww bollocks, its got him and her in their team, hold on to your arse this is going to be a wasted 7 minutes of my life.

...or when the glorious MM gives the enemy team 150t more. I just had that. How fun...

#49 Screech

    Member

  • PipPipPipPipPipPipPipPipPip
  • Knight Errant
  • 2,290 posts

Posted 10 July 2016 - 08:57 AM

Until the leaderboards stats are split by queue all this effort is largely wasted.

#50 Sader325

    Member

  • PipPipPipPipPipPipPipPip
  • 1,181 posts

Posted 10 July 2016 - 09:01 AM

Matchmaker only takes into account tier, I'm not sure why you find your "findings" to be so shocking.

#51 Mister Blastman

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Survivor
  • Survivor
  • 8,444 posts
  • LocationIn my Mech (Atlanta, GA)

Posted 10 July 2016 - 09:23 AM

View PostSader325, on 10 July 2016 - 09:01 AM, said:

Matchmaker only takes into account tier, I'm not sure why you find your "findings" to be so shocking.


Have no findings of significance yet. Data collection is required. Feel free to contribute!

#52 Timber Ghost

    Member

  • PipPipPip
  • The 1 Percent
  • The 1 Percent
  • 58 posts

Posted 10 July 2016 - 11:06 AM

I have always thought matchmaker does pretty well. My issue is the PSR. I have a K:D ratio of 1.1, and a win loss record that hangs right around 1:1. Somdays it dips below, some days it is above. Yet, within a couple days, I will be tier 1.

Edited by Timber Ghost, 10 July 2016 - 11:06 AM.


#53 Ano

    Member

  • PipPipPipPipPipPipPip
  • Bridesmaid
  • Bridesmaid
  • 637 posts
  • LocationLondon

Posted 10 July 2016 - 12:32 PM

This is an interesting project, but a bloody nightmare to manage on a personal level.

I wonder if it would be possible to have PGI enable "logging" in the client, by which I mean the automatic export of end-of-match screens as text files at the end of the match. Presumably the data is transmitted back to the client anyway to create the ingame screen -- writing out a textfile to the screenshot folder would be neither cumbersome to code nor resource-intensive in the client. And it would make both projects like this and just simple personal performance logging much simpler.

I don't doubt for a moment that if PGI enabled such a feature, a member of the community would build a stats analysis tool or service that would allow you to upload/process your logfiles to create a picture of your performance over time -- something many people would be interesting in, I suspect.

TBH I find it fairly impressive that we have tools like Smurfy, LSML and others given the relatively small size of the playerbase -- clearly there are enough people with the technical chops and willingness, given available data...

#54 Mudhutwarrior

    Member

  • PipPipPipPipPipPipPipPipPip
  • The 1 Percent
  • The 1 Percent
  • 4,183 posts
  • LocationThe perimieter, out here there are no stars.

Posted 10 July 2016 - 12:44 PM

Awesome thread and idea but just for that reason alone PGI will never even consider it. History is way on my side on this one.

#55 Andi Nagasia

    Volunteer Moderator

  • PipPipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 5,982 posts

Posted 10 July 2016 - 04:35 PM

Great Topic, but i think its abit more Balanced than that,
Not saying your Math is Wrong, but how we are Preserving it is wrong,
as in it looks like a Large Margin but its really no, type of Way,

For Instance,
if the Diffrence between the Two Sides was 7.5%
Cut that in Half you have 3.75%(lets say 4%) Up or Down in a Percentage Scale,
so the Winning Team had a 54% Chance to Win, where as the Losing Team had a 46% Chance to win,
-
and we also Dont know how the Two Teams Coordinated in the Match,
Perhaps If the Losing Team Coordinated more you could have Won,

Also the KDR i dont think matters as Much,
as Anyone can get 200Damage into a Single Target then have that Target Stolen by another,
It matters but perhaps not as much as some Seem to think it does(COD Fans) ;)


All in all though Great Topic, ;)
like the Science behind it and would love to see PGIs thoughts on this,

#56 GreenHell

    Member

  • PipPipPipPipPipPipPip
  • 543 posts
  • LocationGrandmas House

Posted 10 July 2016 - 04:43 PM

I've been on both ends of the 12-0 spectrum, and I can tell you neither end feels good.

You either feel bad for losing so hard, or you feel bad for stomping so hard.

I'll be sure to keep an eye on my end match screenies in the future and see if I can lend some data to the thread.

#57 9thDeathscream

    Member

  • PipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 563 posts
  • LocationDown Under. 260 pinging.

Posted 10 July 2016 - 05:03 PM

Calling someone out as good or bad due to their stats isn't right.

People play the game for different reasons. I like to skill up every mech I have. Thus I have lower stats than others as I'm not using mastered meta mechs. Do I care. No. I have a broad mix of games. Sometimes 5+ kills 1000+ damage.
Somethings 0 Kills and sub 100 damage.

I don't tend to hide b hind my team to pad my stats. I stay usually on one flank and move with the team.

Always remember your interpretation of how the game should be played is not always what another persons is and the stats will reflect that.

This reason with the low population at certain hours and various skill mixes is why the Match maker has such a hard time.

Not everyone cares about others epeen.

Edited by Akulla1980, 10 July 2016 - 05:06 PM.


#58 Naelbis

    Member

  • PipPipPipPipPip
  • Big Brother
  • 127 posts

Posted 10 July 2016 - 07:06 PM

Well I had the misfortune to drop against Victoria today...it was a slaughter. My team did pretty much everything they could to lose and she walked away with 4 kills (including me) and 1k+ damage. Only 2 of us scored higher than 200 match score...I should have taken a screenshot.

Edited by Naelbis, 10 July 2016 - 07:06 PM.


#59 Blue Pheonix

    Member

  • PipPipPipPipPipPip
  • Ace Of Spades
  • 229 posts

Posted 10 July 2016 - 07:59 PM

View PostNaelbis, on 10 July 2016 - 07:06 PM, said:

Well I had the misfortune to drop against Victoria today...it was a slaughter. My team did pretty much everything they could to lose and she walked away with 4 kills (including me) and 1k+ damage. Only 2 of us scored higher than 200 match score...I should have taken a screenshot.


This is a perfect example on why matchmaking should not be random. It is also a perfect example on why "cash only whole chassis" should not contribute towards stats on the leader board.

#60 Ex Atlas Overlord

    Member

  • PipPipPipPipPipPipPipPip
  • Survivor
  • 1,018 posts

Posted 10 July 2016 - 09:52 PM

Winning team
Match score average 240
KD/R average 1.359
W/L average 1.19

Losing team
Match score average 190
KD/R average 0.78
W/L average 0.89

Don't know how to link the screenshot, but it was a 12/3 win.... so yeah, that game was over before it started.

Edit:
Match score difference -> 20.8%
Chance of kill difference -> 13.7%
Chance of winning diff -> 7.24%

Edit 2: To be fair I called targets, and people actually listened so that undoubtedly didn't hurt our chances of winning.

Edited by Ex Atlas Overlord, 10 July 2016 - 10:23 PM.






1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users