44 replies to this topic

[Evil Ash]

There's nothing I hate more than an unbalanced match, whether it's a 2/12 for the enemy team or a 12/2 for my team it takes away from any feeling of excitement to have have a guaranteed win or loss. The current ELO system seems to allow for these "horrendously bad" games and they make up a majority of games that I play. This is why i'll propose a few ideas for a new system that I believe will match players by their skill rather than what mech they play (because a lance of ECM Spiders can beat 4 lrm atlases any day).

The current ELO (according to my understanding) is as follows. ELO is based off of your wins and losses and that's it. Depending on who you win against is what changes your ELO. Now this spurs a problem because you have games that are 12/2 that increase your ELO by a ridiculous amount (as ridiculous as up to 50 gets anyways) and then is shot down again by a 2/12.

A better way to calculate elo is as follows. We'll use a scale between 0-10000 as an example.

For a new chassis or variant: start its ELO near the bottom of the scale, maybe near 100. This will allow a player to enjoy a new mech instead of getting destroyed repeatably game after game because he hasn't found a good build for it yet.

Calculate ELO by their energy, ballistic, and missile accuracy. It makes no sense to have someone playing with a 20% PPC accuracy to play against someone with a 100% PPC accuracy. the accuracy for their weapon is their ELO number, 100% accuracy = 10 points, 50% 5.

Look at the mech they pilot, if the mech has 7 medium lasers with 100% accuracy, he gets 700 points from his accuracy, now this may seem ridiculous, but seeing how he has 100% accuracy he probably can play with other experienced players.

Different weapons also can have different point values. A medium laser might get a maximum of 100 points each, but a small laser might have a maximum of 75 points, this is so you don't have your locust with nothing but small lasers in a game with a Jenner running nothing but mediums.

While looking at the weapons on a chassis, missile boats can also be separated more equivalently as well. Dakka and Laser boats can also be separated better to make more balanced groups.

Groups should get a temporary ELO bonus based on how many people are in the group. 2 players could each get a 50 point bonus, 3 players could get a 75 point bonus each, and 4 players could get a 100 point bonus each. This is because a four man tends to have much higher cooperation than their pug teammates, they will be able to get grouped with better players.

Lastly wins and losses should have negligible if any outcome of your Elo because they are almost entirely based on your teammates and opponents abilities, your personal statistics should determine who you play against.

Credit to LT for a great idea!

LT Satisfactory, on 13 January 2014 - 04:09 PM, said:

If you truly believe that our current implementation of elo is as good as it gets, or only small tweaks needed, then you should be a strong advocate of making the elo score per mech chassis (or even per variant basis) than by class so you have more than 4 elo scores.

I don't see how a single person could say a locust=jenner, treb=shd, qkd=phract, aws=hgn.

Edited by Evil Ash, 14 January 2014 - 02:25 PM.

[Wintersdark]

Your wins and losses, over time, are affected by you a lot more than people like to admit. You may be only 1/12 of the team, but you're the one constant in all your statistics.

Accuracy is worthless as a scoring metric. So is damage. All your stats are, in fact, pretty much worthless. Why? Because they are too easy to game.

For example; accuracy? Well, then, I'll just chainfire my ML's into the dirt to push my ML accuracy down to 5%. Woohoo! Now I'm matched against the worst of the worst players. I can stomp their ***** with impunity, because I can hold my accuracy down even though I'm murdering them left and right.

Damage? Damage is worthless. So many people get all excited about damage numbers, but high damage numbers are very often a result of poor play.

Kills? Extremely problematic. You do all the work, random guy kills your target. Or, conversely, you get lots of kills because you play, lets say, an LRM boat firing continuously at enemy mechs who are engaged. You're actually doing little damage, and what you do inflict is spread out (see: useless damage), but because of the high frequency and spread you get a bunch of kills.

Mech loadouts? Again, terrible. You'd have to chase game balance, and it'd still be very easy to game to push yourself into lower brackets. A good player with a weaker build will utterly dominate a poor player with a highly optimized build.

Wins and losses, modified by the relative team elo scores, is about the best way we currently have. Yes, you can still "smurf" it; deliberately losing repeatedly, but there's little gain in that - you can't do it to "farm" c-bills, because you'll earn so little in the time you're deliberately losing, you'd have been better off just playing at your skill level... and, of course, because climbing back from low Elo would be a really unpleasant process.

As to unbalanced matches: There's always going to be some. It's inevitable, because of how the game plays. If a team makes a successful early push, crushes 2-3 mechs quickly, the firepower disparity will usually snowball. Likewise, a minor mistake by a team, say a lance that started far to one side being isolated early... The losing team didn't necessarily play badly, after the initial loss they'd have had to play far better in order to claw their way back into the running.

You see results that look grossly one sided quite frequently vs. very well matched teams all the time.

[Roadbeer]

Who are you and what have you done with Wintersdark?

[Evil Ash]

Winters, accuracy is more relevant than wins/losses, because if you are only 1/12, you are not the most constant statistic. Your skill level cannot be accurately determined through the current ELO system because you can float through victories without having to do anything useful. Although there will have to be countermeasures against "smurfing", the system we have now is garbage and a game that's not down to the wire isn't worth the good games said at the end.

And no you don't see 1 sided vs well matched often because you have 1 good match for every 4 horrible ones.

Edited by Evil Ash, 11 January 2014 - 06:19 PM.

[focuspark]

Evil Ash, on 11 January 2014 - 06:17 PM, said:

Winters, accuracy is more relevant than wins/losses, because if you are only 1/12, you are not the most constant statistic. Your skill level cannot be accurately determined through the current ELO system because you can float through victories without having to do anything useful. Although there will have to be countermeasures against "smurfing", the system we have now is garbage and a game that's not down to the wire isn't worth the good games said at the end.

And no you don't see 1 sided vs well matched often because you have 1 good match for every 4 horrible ones.

Wintersdark is correct, pretty much any stat except win/loss can be completely and too easily gamed.

What most people miss about ELO is that it doesn't offer fair matches, it offers fair scoring of matches. WAT?! The ELO system will attempt to match teams of average ELO value but when it can't it''ll predict a winner. If that prediction comes true, neither team is affected (score wise); if that prediction comes false the losers are more harshly punished and the winners more favorably scored.

[Wintersdark]

Evil Ash, on 11 January 2014 - 06:17 PM, said:

Winters, accuracy is more relevant than wins/losses, because if you are only 1/12, you are not the most constant statistic. Your skill level cannot be accurately determined through the current ELO system because you can float through victories without having to do anything useful. Although there will have to be countermeasures against "smurfing", the system we have now is garbage and a game that's not down to the wire isn't worth the good games said at the end.

And no you don't see 1 sided vs well matched often because you have 1 good match for every 4 horrible ones.

You do indeed see it a lot. Look at 12v12 matches - it's not at all uncommon for the result between reasonably matched teams to be grossly one sided. As I said above, a good choice by one team, a risk that pays off well, just a good strategy - anything that gives you a early decisive advantage and the match is almost certainly going to end up looking very one sided. Not always, of course - but if you get a 2-3 mech lead early, it's extremely difficult for the other team to catch up.

The reality is there's no system that can give fair matches all the time. There's just no way to do it. The current system is absolutely not perfect - there's lots to complain about - but it's very difficult to game the system. It's possible to game the system, but it's purely disadvantageous to do so.

Gaming the system is something that's very important for our Lords And Masters to avoid. People think Elo is here to give you fair matches.

It's not.

Elo is here first and foremost to protect newbies from veterans, and after that to match players against opponents that are vaguely near their skill level. That doesn't mean the match will be close.

Any calculation relying on player-visible and manipulable stats is easy to game. When the stats are easy to game, players absolutely will do so. Why? Because then they can be gods amongst men instead of just another mechwarrior. And then they can boost their other stats. For example, I could take my Assault mechs, blow all my accuracy stats to {Scrap}. Use this to fight poor players, and crank my KDR stupidly high.... while maintaining a low accuracy stat. And while I'm doing this, having a grand old time smashing newbies, I'm ensuring those new players (who already suffer a very newbie unfriendly game) are having an even worse time facing an opponent with several thousands of drops of experience.

Countermeasures against smurfing are extremely difficult to implement. Look at Ghost Heat - it was instituted as a countermeasure for heavy alpha striking, but it doesn't actually achieve that at all in high level play, because the moment players understand the parameters, it's trivially easy to work around it. The same applies to smurfing countermeasures.

[Wintersdark]

If you ACTUALLY don't contribute anything useful... You're going to lose more matches than you win. A lot more. You can't really float through many victories without doing anything useful. Sure, sometimes things work out that way (see my comments above re: early decisive actions deciding matches at the start), but in the long run, how much you contribute to a match matters. Not how much damage you do. Not even how many kills. But how much you contribute overall.

Maybe you do 100 damage, kill nobody. You can still have a significant role in the match - hell, you can be the linchpin to victory.

Maybe you're playing squirrel, you draw the enemies attention and pull a lance out of position so it can be crushed by other players. You'll probably die, do little damage, but if your death costs the enemy team a full lance? Your team is quite likely to win that match. You were instrumental to that victory, even though by stats you did practically nothing.

Maybe you're piloting a Jag, and you're watching over a flank. A couple shots keep enemy mechs in cover, and delays a flanking rush from the opposing team. Still, little damage done, very poor accuracy... But delaying that flank attack saves the day for your team.

Maybe you're just terrible. Maybe you can't hit the broad side of a Awesome. But, maybe you've got a good eye for tactics, and a good way with people. You take command in matches, and guide teams to victory. Or hell, maybe you simply make a point of relaying information - enemy contacts, etc. Even one person relaying information to the team in a PUG otherwise lacking in communication can be critical. (Incidentally, I feel this is a major influencing factor for really one-sided PUG matches: One side actually communicates. Even just one player on one side. If just one guy does, that added organization and situational awareness is a huge advantage. Teamwork is OP)

[z00med]

Main problem while considering "Elo"-Systems for a MMO: Its not chess or go. So as far as I understood good ol' Arpak, besides the fact that there are 23 Pugs and you (and not 2 players) in a match you simply have too many variables (as u discussed above). And changing the influence of one or two of these factors will change the outcome of any equation massively. Not to mention factors like "communication"...
Math never was my favorite but I talked about this with a friend who is an astrophysicist yesterday... He watched a couple of matches (sry buddy, daily victorys... ) and then claimed he would prefer calculating the next dimension (11th, I think... All I can say is that this guy was saving my *** in the last three years of school...)
Have fun balancing that...

By now I still maintain that some system that considers tonnage and classes in both teams (and g*dd***n ECMs) would suffice.
Plus a playground for the fresh meat (first 25 matches, eventually expanded depending on how hard you fail. Simply fill their ranks with bots that follow waypoints and miss with 75% of their shots if neccessary).

edit: sorry for written mistakes. my english starts to fail when it comes to maths (mainly my whole brain starts to fail when numbers exceed my fingers :/)

Edited by z00med, 11 January 2014 - 07:43 PM.

[Wintersdark]

z00med, on 11 January 2014 - 07:38 PM, said:

Main problem while considering "Elo"-Systems for a MMO: Its not chess or go. So as far as I understood good ol' Arpak, besides the fact that there are 23 Pugs and you (and not 2 players) in a match you simply have too many variables (as u discussed above). And changing the influence of one or two of these factors will change the outcome of any equation massively. Not to mention factors like "communication"...
Math never was my favorite but I talked about this with a friend who is an astrophysicist yesterday... He watched a couple of matches (sry buddy, daily victorys... ) and then claimed he would prefer calculating the next dimension (11th, I think... All I can say is that this guy was saving my *** in the last three years of school...)
Have fun balancing that...

This isn't news. The Elo system is somewhat weak, but it does work to an extent. I know that I never see new players in matches, for example. The point, though, is that there's no better way (that we've found thus far, anyways). And the means we have ensures that more experienced players don't face new players. This improves the new players experience.

Quote

By now I still maintain that some system that considers tonnage and classes in both teams (and g*dd***n ECMs) would suffice.
Plus a playground for the fresh meat (first 25 matches, eventually expanded depending on how hard you fail. Simply fill their ranks with bots that follow waypoints and miss with 75% of their shots if neccessary).

There already is a Fresh Meat playground: Players have an artificially lowered Elo score for their first 25 matches. After that, they are switched to their "proper" scores.

We had a system that just counted weight classes. That's what existed before Elo. To be absolutely clear: IT WAS NOT BETTER. It was awful for new players. It ensured that skilled, veteran players had absurdly huge KDR's. That the vast bulk of players KDR/WLR are much tighter these days is a clear indication that Elo is worker at least somewhat.


Complaining about the Elo system is useless. Until we have some viable option (and those based on stats are not) we're stuck with it. We've already had just weight class matching, and it was terrible. Well, it was awesome for vets, but terrible for everyone else.

[GRiPSViGiL]

The answer is match score. The more useful things you do for your team the higher your match score is plain and simple.

Do it PGI!!!

Edited by GRiPSViGiL, 11 January 2014 - 08:25 PM.

[Evil Ash]

Winters, you've put up many good points, but saying that nothing should be changed is absurd, for example, the temporary group ELO boost I proposed would be and still is a good idea in my opinion, hell it could already exist in a form for all I know, but the fact we know barely anything about ELO besides it doesn't give ideal matches (which as several people have said is not feasible) doesn't give much hope to the system. We can't have a strong matchmaking program off nothing but Win/Lose (Apparently it's based off of Win/Lose in each of the 4 weight classes from looking into it more), at least adding more into the equation could match players of similar skill level.

GRiPSViGiL, on 11 January 2014 - 08:24 PM, said:

The answer is match score. The more useful things you do for your team the higher your match score is plain and simple.

Do it PGI!!!

Match score doesn't actually help that much, as winters said in his various and numerous examples your statistics (and this includes match score) don't actually sum up what people's abilities are. If only a player based ranking existed, too bad that's impossible because of meta and etc.

Edited by Evil Ash, 11 January 2014 - 08:40 PM.

[Wintersdark]

GRiPSViGiL, on 11 January 2014 - 08:24 PM, said:

The answer is match score. The more useful things you do for your team the higher your match score is plain and simple.

Do it PGI!!!

Nope. Match score is weighted very heavily to damage, and doesn't track all sorts of things that the game can't score - see communication and such above in my most recent lengthy post Even if you "fixed" match score to not be based primarily on damage (which as has been established is a terrible metric of player skill) there's just too many things that can't be scored.

When you play in premades regularly, it's different. Communication is a given. But in an open PUG, having one guy seriously scout, or actually take command can have an absolutely enormous impact on your team's "winningness".

[Wintersdark]

Evil Ash, on 11 January 2014 - 08:33 PM, said:

Winters, you've put up many good points, but saying that nothing should be changed is absurd, for example, the temporary group ELO boost I proposed would be and still is a good idea in my opinion, hell it could already exist in a form for all I know, but the fact we know barely anything about ELO besides it doesn't give ideal matches (which as several people have said is not feasible) doesn't give much hope to the system. We can't have a strong matchmaking program off nothing but Win/Lose (Apparently it's based off of Win/Lose in each of the 4 weight classes from looking into it more), at least adding more into the equation could match players of similar skill level.

I didn't say nothing should be changed, I said we didn't have good changes to make. That's a very different thing, and it's an important difference.

A premade Elo boost is unnecessary. There are basically two situations here:
A) Player primarily plays in premades. In this case, he's going to win more often due to added communication, awareness, and familiarity. Winning more often is going to result in a higher Elo, effectively placing him against stronger opposition than he would be otherwise.
Player rarely plays in premades. In this case, his Elo will more reflect solo play, so his opponents will be somewhat weaker.

This is more effective than a "grouping boost" because it will accurately (* as the system can) reflect where the player ends up in whatever play modes he uses most frequently. Thus, it's reasonably accurate most of the time. A static number would be immensely difficult to tune, and wouldn't be accurate for anyone - Does this player use voice comms in his premades? If so, he'll be adding more to the overall team than if he doesn't, for example.

Finally, the Elo system isn't based entirely off Win/Loss ratio. In fact, it's not based off WLR at all. It's based off whether or not you win or lose a battle relative to whether the system predicted you'd win or lose the battle. That's a very important difference, and crucial to how the system works. You could lose 3/4 games and have your Elo go up (though this would only happen in edge cases).

And, finally, we actually do know how the MM system works, pretty well. PGI has been quite open about it. The matchmaker uses three things when building a match:

It uses Elo rankings (you have 4 scores, one for each weight class) and your mech's weight class. The matchmaker attempts to get players of comparable rankings in each game.
It attempts to get an even distribution of premades on each team.
It uses weight classes to ensure an even distribution of weight classes between the teams. This fails pretty badly when premades are involved, however, because the MM will match premade against premade first - you could have a 4man D-DC premade on one team, and a 4man Trebuchet group on the other, for example.

Finally, and this is a VERY IMPORTANT POINT: All the above restrictions are increasingly relaxed the longer the matchmaker spends trying to find a match. So you may end up with outlier Elo players or bizarre weight class issues, even if premades are not involved. This of course is most likely the more extreme the Elo rankings of players involved is, because due to the nature of the system Elo scores will end up distributed on a bell curve (and this does happen, PGI has shown the distribution). So, the vast majority of players are "average" Elo, despite thinking they are all High Elo folks.

It then calculates how likely it feels each team is to win. If you do lose and were expected to, your rating may not drop at all, and if it does it'll only drop a tiny bit. If you win, it'll go up substantially.

The actual math is posted in the CC post.

[GRiPSViGiL]

Winters makes good points and arguments but that is all they are because he isn't a insider or Dev who truly knows. I do fully agree that damage is an overrated stat and shouldn't be a basis for any matching system so long as it was a very small percentage.

Are we 100% sure match score isn't effected by target assist, kill assists, TAG, capping, etc.?

If not then that should be the adjustment. Match score should be a culmination of all the helpful things you can perform in a match for your team.

Edited by GRiPSViGiL, 11 January 2014 - 11:39 PM.

[Wintersdark]

GRiPSViGiL, on 11 January 2014 - 11:08 PM, said:

Winters makes good points and arguments but that is all they are because he isn't a insider or Dev who truly knows. I do fully agree that damage is an overrated stat and shouldn't be a basis for any matching system so long as it was a very small percentage.

Well, everything I stated above isn't educated guesses, it's been explained by the devs. Sure, they could be lying, or could have changed things, but there's no point whatsoever in pursuing that line of reasoning.

Quote

Are we 100% sure match score isn't effected by target assist, kill assists, TAG, capping, etc.?

I didn't say it wasn't affected by those things. It is. But it's primarily impacted by damage. It's not hard to test this - track match scores and your performance stats. I often play a spotter Jenner (because I'm terrible at lights, and spotting is easy, and sometimes I just like to run really fast) and have matches where I pull massive amounts of XP and CBills through lots of assists, tag, spotting etc. Crushingly low Match Scores, though.

Quote

If not then that should be the adjustment. Match score should be a culmination of all the helpful things you can perform in a match for your team.

As I said above, it can't be. They simply can't score "He took command and lead the team to victory" - that's not even hard to do. In a PUG vs. PUG situation, one team with leadership vs. one without is going to win practically every time.

All the things I noted in my post above too, they can't be scored. Simply scouting and relaying enemy mech movements to the team vastly increases team situational awareness. That's a huge advantage for your team, but it can't be scored.

Suppression fire keeping a number of players in cover? Low damage done, but if it prevents a flank attack it's easy for that to be the deciding factor in who wins a battle.

There are countless ways a good player can contribute to victory, particularly in PUG matches, that involve completely non-scoring actions... and ones they couldn't score.

Many of these methods are overlooked by players who play in premades or 12v12 frequently, because those are very different animals. Communication is the norm, not the exception.

[Effectz]

I can't even solo drop in my assault or heavies anymore,takes me 40 minutes to find a game.When I do on the rare occasion I get into a game I get dropped with 3 trial mechs who do under 100 damage against a team with no trials.

[Wintersdark]

Effectz, on 12 January 2014 - 09:54 AM, said:

I can't even solo drop in my assault or heavies anymore,takes me 40 minutes to find a game.When I do on the rare occasion I get into a game I get dropped with 3 trial mechs who do under 100 damage against a team with no trials.

Sorry, I call ********.

I haven't seen a failed to find match screen on a solo drop in any weight class in months and months. Sometimes it takes a while - a minute or three - but 40 minutes to find a game? No.

Now that there's only Champion trial mechs, if you're a swaft of newbies in your matches, it's because you're in scary-low Elo. Trial mechs on one side and not on the other doesn't matter anymore, because while not ideal builds the Champion builds aren't terrible either. If they're not newbies, then it doesn't matter.

No, this really seems like random exaggerated (and thus entirely useless) complaining.

I've seen lots of failed matches when running 12's, and very rarely in premades, but it's been a very, very long time since I've seen a failed match in PUG play - and even then, it's only happened after patches with MM changes that where, without fail, reverted after failed matches started happening.

I play at all times (yay, rotating shiftwork is just so awesome!) so I see player pools at all sorts of different times. At really oddball times (say, 3am) yes, matches are pretty sketchy: You see more tonnage disparity and more Elo disparity, but I still always get matches within 3 mins tops.

ALWAYS.

[focuspark]

Wintersdark, on 12 January 2014 - 10:12 AM, said:

Sorry, I call ********.

I haven't seen a failed to find match screen on a solo drop in any weight class in months and months. Sometimes it takes a while - a minute or three - but 40 minutes to find a game? No.

Now that there's only Champion trial mechs, if you're a swaft of newbies in your matches, it's because you're in scary-low Elo. Trial mechs on one side and not on the other doesn't matter anymore, because while not ideal builds the Champion builds aren't terrible either. If they're not newbies, then it doesn't matter.

No, this really seems like random exaggerated (and thus entirely useless) complaining.

I've seen lots of failed matches when running 12's, and very rarely in premades, but it's been a very, very long time since I've seen a failed match in PUG play - and even then, it's only happened after patches with MM changes that where, without fail, reverted after failed matches started happening.

I play at all times (yay, rotating shiftwork is just so awesome!) so I see player pools at all sorts of different times. At really oddball times (say, 3am) yes, matches are pretty sketchy: You see more tonnage disparity and more Elo disparity, but I still always get matches within 3 mins tops.

ALWAYS.

Actually, this is possible at the far ends of the ELO spectrum. Just think about the math: Find pilots +/- 10% of your ELO score, if you're at the bottom it's only the pilots 10% above you. To make things worse ELO maths tend to create a normal distribution curve making the number of pilots near the ends of spectrum few and far between.

While I agree the poster is liking telling stories out of school, it is possible.

That said: ELO doesn't create far matches, it creates fair scoring AND since we don't see or use our actual ELO modified scores it's fairly unsatisfying.

[Effectz]

I have waited as long as 40 minutes to drop,I should of rephrased that.And I get multiple failed to find matches one after another for 20 minutes easy in heavies and assaults so I just get fed up and drop to mediums or lights,and I'm not bullshitting.And it does happen,heres Jager and Siri waiting for 82 minutes.

And im not in low ELO lol

Edited by Effectz, 12 January 2014 - 11:49 AM.

[Skyfaller]

ELO is the worst system ever implemented.

Its ABSURD to consider win:loss rate as a measure of player skill or performance. Most maps are lost in the first 10 seconds when the team-mates scatter all over and noobrush the first contact they see..and die. How then is the skill of a player who sees his entire team die like that in 3 minutes measured?

The matchmaker should balance the hardware only.

Match the mech tonnage (not exactly but in the sense that if a team has two assaults the other team needs to have 2 assaults as well) and limit ECM mechs to one light/medium ecm and one heavy/assault ECM tops.

Finally, pre-made teams need to cease being allowed in PUG games. All it does is open the door for exploit and throwing off the MM system into chaos. A premade of four dual ac20 jagers pitted against the randomly chosen by MM on the other team of 4 heavies of very different configurations means the team with the premade jagers has a massive advantage.