182 replies to this topic

[SecretMantis]

So I haven't read this entire thread and I don't know the formula that MM works with but I can give you my experiences from tonight. Played 22 matches in group cue with one other friend of mine. Ended up with a win-loss of 4-18. Every single match, except for one, was a complete stomp. It wasn't a fun experience. Its clear to me that the MM is not producing even matches.

[Duke Nedo]

Davers, on 07 December 2014 - 01:19 PM, said:

See, The Gambler's Fallacy. http://en.wikipedia....ler%27s_fallacy

Ofcourse that is not what I am talking about. I am in touch with probabilities and statistics, work with it every day for what its worth on a gaming forum.

Davers, on 07 December 2014 - 01:19 PM, said:

How Elo works is pretty open content. Lots of sites talk about it, and even give the percentage chance of winning based on the difference. For example, when the Clans won 65% of their matches with approximately 100 point Elo advantage, it matches up perfectly with their predicted chance to win. Thus proving Elo works in MW:O.

Then you assume that clan mechs and IS mechs are balanced? I would challenge that by saying that clan mechs are still a bit better than IS on average because of the cXL engine and that Elo in this case may not have had a significant impact. At least not been alone responsible for the outcome.

Davers, on 07 December 2014 - 01:19 PM, said:

Elo is the foundation of every MM out there. If there is a problem with the MM it is more likely due to the issues that PGI has brought up to fix things, things that the players demanded they not do. Which is iron out tonnage discrepancies and have players be flexible in game mode options. Basically players told PGI "We would rather have more imbalanced matches as long as we can drop in the mech we want, in the mode we want." PGI flat out said that for better MM to be possible one of those has to go.

Something I would like to see from PGI is an analysis of streakyness from their data. It would be a simple thing to prove or disprove. If there are on average more 10-match-streaks than 1 per ~2000 drops per player there is a systematic thing happening in there somewhere. I can only keep track of my own results, and I have more streaks that expected if there was a 50% to win each match before it starts. Even as little as a 5-match loss-streak only has ~3% probability per try. Over a typical gaming evening of say 20 drops, the chance is ~15-25% to have one. Everyone can judge for themselves if that is what they are seeing or if they have more streaks than that.

The other thing that is not talked about much when discussing matchmaking is the imbalanced maps. Most of them I'd say have one side that has a clear advantage, that must add a lot of noise to the personal Elo changes in addition to all the noise we already get from the performance of our 11 team mates.

[Sandpit]

Duke Nedo, on 08 December 2014 - 12:00 AM, said:

Ofcourse that is not what I am talking about. I am in touch with probabilities and statistics, work with it every day for what its worth on a gaming forum.



Then you assume that clan mechs and IS mechs are balanced? I would challenge that by saying that clan mechs are still a bit better than IS on average because of the cXL engine and that Elo in this case may not have had a significant impact. At least not been alone responsible for the outcome.



Something I would like to see from PGI is an analysis of streakyness from their data. It would be a simple thing to prove or disprove. If there are on average more 10-match-streaks than 1 per ~2000 drops per player there is a systematic thing happening in there somewhere. I can only keep track of my own results, and I have more streaks that expected if there was a 50% to win each match before it starts. Even as little as a 5-match loss-streak only has ~3% probability per try. Over a typical gaming evening of say 20 drops, the chance is ~15-25% to have one. Everyone can judge for themselves if that is what they are seeing or if they have more streaks than that.

The other thing that is not talked about much when discussing matchmaking is the imbalanced maps. Most of them I'd say have one side that has a clear advantage, that must add a lot of noise to the personal Elo changes in addition to all the noise we already get from the performance of our 11 team mates.

Most of what you talk about here has nothing to do with the mm though
Map balance
Tech balance
Etc.
That has nothing to do with mm

[Duke Nedo]

No exactly, but it affects the observed match result which is then blamed on the MM/Elo system... especially map balance is hardly ever discussed in this context and I find that a bit strange. Since map balance is not considered when the Elo system predicts the winner, it will 50% of the time on a badly balanced map help the worse team to beat the good team, thus it can make it so that Elo points are awarded not because the players performed above their expected skill level, but because they had an advantage from external factors. That could make Elo jump up and down much more than it should and hide the actual individual player contributions that the system is supposed to fish out from that 12 v 12 mess....

[SgtKinCaiD]

I must agree with Sandpit about the MM and the streaks of wins or losses : the MM does not work like that.
It tries to setup match where both team have a 50% chance of winnning (at least it tries to be as close as possible).

Now the real problem is how the MM assembles teams to get a match. Right now, it only considers the ELO sum up of each team to create a "fair" match, or it is not a good indicator of the actual skill of the team, especially for teamplay.
Plus, the rule of mixing high ELO players with low ELO players in order to have an average ELO easier to work with, is not working : right from the start, the team is already missing firepower from one (if not more) player which is a huge disavantage given the nature of the game and the snowball effect.
Another problem is that a player ELO is based on W/L only, or past a certain point, a low ELO player won't decrease anymore because basicaly he's caried by his team every time. When you see the last post of Kiiyor about (C) trial mech, it's terrible, i don't understand why there are still put in regular match while they don't know how to drive a mech and won't understand why there are getting killed so easily.

That's why i asked several times to reintroduce ELO buckets : use the player ELO score to determine in which bucket he is and put him with and against players of the same bucket. While the ELO sum up of each team will not be as close as before, the match will be better balance because every player in both team will have roughly the same skill. The real problem is that PGI has to rewrite the MM nearly entirely for that.

@Sandpit : the fact that you recommand to cancel the MM search given a certain period of time, is a good indicator that the MM is not working correctly. And the supposedly low population is a false excuse : actually, even in a TBR and a Heavy queue greater than 50%, you still find a match under 3 minutes.

@Mallan : trying to give advice to your team is helpful (it's a polite way to say guiding the herd) but what can you expect from a player asking "how do i save the game ?", another player asking "why do i power off with a red screen ?", or a player bringing a Victor LRM-boat ?

[Duke Nedo]

When you say that the MM doesn't work like that, you mean "the MM isn't supposed to work like that", right? A lot of people are seeing streaks, that may or may not be the MM/Elo implementation not doing what it's supposed to do, or something else. As far as I know we have no information about exactly how the MM builds the teams... which order are slots filled, in which order are valves released for elo, 3-3-3-3, etc. Are slots filled at all, or are 24 players selected and then put into teams? Could any of this give strange results when one queue is overpopulated, or when the seeding elo is an outlier etc etc. In my mind there are many unknowns which makes it hard to say if strange behavior is caused by the MM, or Elo, or map imbalance, or sync drops or jupiter and mars being in conjunction... we can only observe the end result, and a lot of people are unhappy about that.

All I know is that when I am on a bad team, there is not much I can do about it. Pug-bossing has little effect since half the team either has team chat turned off or can't drive their mech and read chat at the same time, nor look at the radar or spot that we are losing mechs. Not until after they are dead, then they see all this and loudly state that the team sucks because we lost...

Edited by Duke Nedo, 08 December 2014 - 02:22 AM.

[STEF_]

Just made a drop in Puglandia, with my Huncie wubby.
12-4.
3 kills for me, cored and killed completely by myself a cent and a summoner; a kitty was already damaged.
Average team dmg around 200.
I chatted "carry hard. I'm having fun".....

Pgi.... just throw away that MM elo based, please.....

[MoonfireSpam]

Hang on, it's nice to blame MM for bad matches, but consider something else for a minute.....

In Solo Queue Land:
Remember how often you have 1-2 AFK / disconnects on one team most games? So one team starts at a disadvantage from the get go.

Then remember how 1-2 people nearly always get caught out of position / suicide into enemy team?

By 5 minutes in, before any furball / team fight happens one team is generally 2-4 people down.

Combine this with the piss poor situational awareness, bad coordination and poor judgement if your average human (how often have you seen people ignore a cored mech, or chase a disarmed mech just for the kill, or all chase a lone ECM spider, or fail to notice big swathes of red and take firing positions in advance).

Then you factor in the size of the teams. The numbers game makes the lesser team so much more likely to be focused down by pure weight of fire.

Boom stomp has happened - nothing to do with MM.

Stomps are a combination of badness from people, AFK / disconnects, large amounts of firepower due to previous points + team numbers.

[Duke Nedo]

There are two types of stomps, one that you describe where team balance is ok but the snowball starts rolling. Those are unavoidable and are quite often good fun anyways.

Then there is the other type where the teams are just incredibly unbalanced, one team pushing and the other running around trying to find some teammates to hide behind... in best case. In worst case they don't even know we have already lost the game, still running full speed counter-clockwise looking for the enemy, team chat turned off, not reading casualties, not looking at the radar... those should be possible to avoid or at least tone down.

[Joseph Mallan]

Duke Nedo, on 08 December 2014 - 03:50 AM, said:

There are two types of stomps, one that you describe where team balance is ok but the snowball starts rolling. Those are unavoidable and are quite often good fun anyways.

Then there is the other type where the teams are just incredibly unbalanced, one team pushing and the other running around trying to find some teammates to hide behind... in best case. In worst case they don't even know we have already lost the game, still running full speed counter-clockwise looking for the enemy, team chat turned off, not reading casualties, not looking at the radar... those should be possible to avoid or at least tone down.

So you are complaining about human nature and fight or flight reactions? Sorry man but MM does not calculate for courage under fire, just win/loss.

[SgtKinCaiD]

Duke Nedo, on 08 December 2014 - 03:50 AM, said:

There are two types of stomps, one that you describe where team balance is ok but the snowball starts rolling. Those are unavoidable and are quite often good fun anyways.

Then there is the other type where the teams are just incredibly unbalanced, one team pushing and the other running around trying to find some teammates to hide behind... in best case. In worst case they don't even know we have already lost the game, still running full speed counter-clockwise looking for the enemy, team chat turned off, not reading casualties, not looking at the radar... those should be possible to avoid or at least tone down.

I can not agree more than that.

The first one, let's call it the "snowball stomp", usually lasts past 8-9 minutes, the survivors are pretty torn up and it's been a good fight.

The second one, the roflstomp, ends in 6 minutes because less than 180 seconds after the start, 3 of your teammates are already dead and you end up outdamaging and outscoring more than a full lance of your team.

[speleomaniac]

My only problem with MM it is building time decks like following...

Team A - Assaults, Awesome, Victor-9, Banshee, ..

Team B - Direwhale, Direwhale, Direwhale,....

When there are Direwhale's waiting in the queue why both team does not get one, is beyond me....

[Duke Nedo]

Joseph Mallan, on 08 December 2014 - 05:11 AM, said:

So you are complaining about human nature and fight or flight reactions? Sorry man but MM does not calculate for courage under fire, just win/loss.

Imo win/loss should reflect the ability to play as a team, which is what I am talking about. All experienced/good players have awareness and knows when to commit and stand and fight.

[-Natural Selection-]

speleomaniac, on 08 December 2014 - 06:09 AM, said:

My only problem with MM it is building time decks like following...

Team A - Assaults, Awesome, Victor-9, Banshee, ..

Team B - Direwhale, Direwhale, Direwhale,....

When there are Direwhale's waiting in the queue why both team does not get one, is beyond me....

Did you know it wasn't to long ago it was random as to what you got on your team? You could end up with 12 lights against 12 assaults. You actually had to get a scope of the other team before you went in. And actually think about how you could make the best of what you had. Hell, you could have though you were in the good with 4 assaults and run in to find they had 6. To be honest, in most ways it was more fun and challenging.

[Smokeyjedi]

Its sad to say the least, specially when there are dozens of good players being cancelled out by bad ones.

[Joseph Mallan]

Duke Nedo, on 08 December 2014 - 06:19 AM, said:

Imo win/loss should reflect the ability to play as a team, which is what I am talking about. All experienced/good players have awareness and knows when to commit and stand and fight.

Still doesn't mean they will stand and fight. I see Dire Wolves and Timber Wolves always walking backwards I go around a corner and the threat can be a Single Blackjack or a Heavy Lance.

Also not all Good players are good Team players.

[Almond Brown]

Duke Nedo, on 06 December 2014 - 04:19 AM, said:

I actually think the MM offsets Elo a bit to be able to predict the outcome, in order to know if it should award Elo or not. If Elo is perfect, prediction will be impossible, thus unnecessary... and from what I have read, Elo uses prediction of result to award Elo, thus it "needs" imperfect matchmaking to work.

If this is anywhere near the truth the really must scrap the whole prediction thing and just make sure average elo is within reasonable limits and then count elo for every match. Hope I am wrong, but something is seriously wrong with the frequency of streaks we are getting in the solo queue.

The prediction is based on the values obtained after building the Teams. The odds of an "exact" elo match between 24 players is likely "zero". The prediction for a "WIN" can be placed against one Team based on a difference of 1 elo point between the 2 teams.

It doesn't predict a "winner", "red" or "blue", then build the teams if that is your concern.

Edited by Almond Brown, 08 December 2014 - 07:42 AM.

[Jetfire]

LordKnightFandragon, on 06 December 2014 - 12:57 AM, said:

What MMs need to stop doing is using w/e mechanic it is that stacks the deck. WoT uses your win rate, if you win like 10 in a row, you will be assured that your next 10 games will be absurdly bad. If you win like 5, you will lose 5. I really hate that rigging of the MM. drop elo, and simply use Mech class and you need to fill a slot........3/3/3/3 go.....forget the elo and all that......then it wont feel the need to stick all the 3.5 KD, 60% WR players against a team of 0.75kd 45% WR players, which it does...

Yes, basically the ELO moving average is far too short sighted and your MM ELO seems to move like a square wave with a 5 match period. It should be like a 30 day moving average.

[Duke Nedo]

Almond Brown, on 08 December 2014 - 07:42 AM, said:

The prediction is based on the values obtained after building the Teams. The odds of an "exact" elo match between 24 players is likely "zero". The prediction for a "WIN" can be placed against one Team based on a difference of 1 elo point between the 2 teams.

It doesn't predict a "winner", "red" or "blue", then build the teams if that is your concern.

My concern would perhaps be that there is likely a sweet spot for how uneven matches should be for Elo to work the best, and that too good matchmaking could be unwanted in the way Elo is implemented right now. Perhaps... just a thought.

If the average Elos are perfect matches, then the Elo change (if any) awarded after each match would be very small. The opposite, if they are very uneven, Elo would not change often, but by the max allowed value every time it happens...

Put that together with the fact that you are very likely to be able to reshuffle players between the two built teams to achieve an almost perfect average Elo match every game... (you have 24 numbers you can combine in many different ways) it could very well be that PGI chooses not to do everything they could possibly have done to get good Elo match because that would make Elo changes very small per game played.

But, that being said, that is probably only of academical interest anyways since Elo is most likely such a bad prediction of a players contribution in any given game that reshuffling is perhaps only another flavor of the same random.

[Mystere]

Duke Nedo, on 08 December 2014 - 01:47 AM, said:

No exactly, but it affects the observed match result which is then blamed on the MM/Elo system...

Many people will blame anything and everything, but never themselves. That happens in real life and in MWO.