88 replies to this topic

[Grumbling Coot]

OP must not have been around before ELO as implemented. I don't think the debs ever claimed that it was the best system possible, but it has clearly improved the quality of most matches. Before ELO it would be pretty common to have streaks of 8-0 rolfstomps on end. Not every match is close, but most matches are CLOSER.

[MWHawke]

Give ELO some time to work. Most of your teammates might have been those DC-to-prevent-death-count players. They THINK they are uber, but in reality ELO is just filtering them out?

[BigBadHarv]

ELO is a system designed for 1 versus 1 games of chess.
it has no relevance to 8 versus 8 games of Mechwarrior Online

winning a solo drop = luck
you either get the team, or you get the morons. that isnt a stable pattern to establish a players skill, let alone number it and put it as a statistic.
it doesnt matter if you play out of your skin and destroy 5 enemy mechs, if your team has decided to lose, you will lose.

telling the players on your team where to go and what to do is fruitless. if they are baddies, they will wander aimlessly, die, leave and queue up again. the fact is that mechwarrior online is a complicated game and the general population are terrible at it. if you are solo dropping, your elo will be determined by the luck you have in landing with like minded players who know wtf to do.
Ive lost count of how many games which are completely farcical because one team has the right players and the other has complete lemons. any stats gained out of those are completely pointless and are of no help to anyone. elo is a bolted on system which has no use whatsoever in this game. The only positives from PGI's point of view are that from a marketing standpoint the game seems similar to league of legends and the like and they are trying to throw their hat in with that crowd (as LoL is an extremely profitable game).

oh, and another thing.
So you have this rating system, elo, in which you claim to use to create fairer matchups... but then you have consumable items which give you a direct pay-to-win advantage in a game. how the diddlyf**k does that make sense PGI?
oh no wait you just want money lol

[Ialti]

freak, on 12 March 2013 - 01:52 PM, said:

Before I begin I want to make something very clear, I am a 24 year veteran of the Battletech Table Top game, a registered Catalyst Demo Agent and have played every Mechwarrior game since Mechwarrior 2, anyone who thinks I don't know what I'm talking about stop now and leave because odds are I've probably forgotten more about this Universe/Game then you know.

Recently I've been seeing a lot of commentary on the ELO matchmaking system, many people have commented on individual matches but I would like to present some more concrete data. I have been working on unlocking the Master Level on my Awesome-9M and decided to kill two birds with one stone by grinding the necessary XP and recording the data from a series of matches to illustrate several points.

Over the course of several hours I played a total of 20 straight matches, all but 1 was in my custom Awesome-9M, for the benefit of completeness the mech is equipped with 3xERPPC's that I have set to Chainfire and 3X Streak SRM2 racks for periods when cooling is necessary as well as for dealing with light mechs. Finally it carries a Beagle Active Probe to aid sensors and because I had tonnage left over and was at Elite level with the Sensor Module for the entire test. The only match not fought in this mech was a single game (Game two on the list) in the Trial Trebuchet-7M.

The Win/Lose results were as follows,

Wins = 6
Loses = 14

That's a lose rate of 70%, if the ELO system is supposed to provide me with balanced matches, why isn't closer to 50%???

Most of the matches took place on the Frozen City/Frozen City Night and Forest Colony/forest Colony Snow maps and were a mix of Conquest and Assault.

The actual Win/Lose ratio is not my only point however and I want to draw attention to the Casualty figures for the 20 Matches, specifically the number of mechs destroyed on each side.

Casualty Figures.

8/2
8/2
5/4
8/0
8/2
8/3
8/2
8/4 *
6/2 (Base Cap Win)
8/2
5/3 (Base Cap Win)
8/3
7/1 (Base Cap Win)
8/2
8/1
8/4
7/4 **
8/0
8/3
8/4

The two Asterisk marked matches I will get to in a minute but a casual look at the figures illustrates a disturbing trend. Under the ELO system of matchmaking 65% of the matches ended in a casualty rate of 8/3 or worse for one side or the other, what I would reasonably describe as a Landslide Victory for one side, 50% had a casualty rate of 8/2 or worse, if that's not a Landslide, I don't know what is.

* This particular match looks close initially until you factor in the fact that the winning side had 3x Atlas D-DC mechs, mechs which are not only the largest, most heavily armed and armoured monsters in the game but that are also fundamentally invisible and invulnerable to LRM and Streak SRM fire and furthermore were fighting on the River City Night map.

**Another match that looks close on paper but was a Conquest game on Frozen City were one team had a Cicada, a Jenner and a Spider while the opposing sides fastest mech was a Yen Lo Wang. The only reason they didn't simply CAP their way to a victory was they wanted the kills, even then the points at the end had over 400 points in the difference.

From the various comments and posts here on the forums there is a clear indicator that ELO is producing more Landslide wins and they are not fun for anyone, if as the DEV's say, the sytem is supposed to produce more even matches then the rests should be showing at least half the games with 8/4 or closer casualty rates. Further I would make the following contention,

ELO is fundamentally flawed because it conflicts with a primary principle of Mechwarrior Online.

Time and again PGI have stated that MWO is a game that requires teamwork, yet they have introduced a system for generating matches that is based on the skill of the INDIVIDUAL, a quality that is variable at best, some pilots are better in light mechs then heavies. Some groups drop as teams while others have to try and rely on their spatial awareness to guess what the team is going to do. TeamSpeak can help but only if the bulk of the people you drop with are using it. So if Team Co-operation is the key to victory, why are you matching up people based on INDIVIDUAL skill??????

This also means the system is ignoring the differences between the different chassis and believe me this can make a huge difference, for example, my Assault mech is an Awesome. I like it, it suits my style and since I am a Marik player I feel it is appropriate but, an Awesome of any type will struggle to bring down a Stalker as the Stalker typically mounts more weapons and armour, not much but it gives it an edge that a pilot must be careful of. Against an Atlas though an Awesome is lunch, the lighter mech may hurt it, even cripple it but the Atlas is simply too large and well armoured to deal with.

This is not as big a problem in TableTop play as the ranges of weapons are fixed values and certain tactics can level the playing field but, again as has been repeatedly stated by PGI this game is not TableTOP, as such weight of fire and weight of ARMOUR make a huge difference that ELO completely ignores.

Most of the time the solution to a problem is the simplest, I would suggest PGI return to a simple tonnage based system for now (I am not going to go into things like ECM and Weapons here and now, this post is long enough and is for a specific problem). Later a look at a BV balanced system may allow the DEVS to narrow the gap even more but ELO needs to go, if you want more proof of this then I would heartily encourage more and more players to repeat my experiment and post the results to illustrate the problem, give PGI all the data they can handle, if necessary until they choke on it.

You do not know that you are talking about. I'm STOPPING AND LEAVING NOW!!111

[FerretGR]

BigBadHarv, on 12 March 2013 - 03:24 PM, said:

ELO is a system designed for 1 versus 1 games of chess.
it has no relevance to 8 versus 8 games of Mechwarrior Online

http://en.wikipedia....gs_beyond_chess

BigBadHarv, on 12 March 2013 - 03:24 PM, said:

So you have this rating system, elo, in which you claim to use to create fairer matchups... but then you have consumable items which give you a direct pay-to-win advantage in a game. how the diddlyf**k does that make sense PGI?
oh no wait you just want money lol

http://mwomercs.com/...-drawing-board/

trolling as usual.

Edited by FerretGR, 12 March 2013 - 03:37 PM.

[Tickdoff Tank]

ThomasMarik, on 12 March 2013 - 02:20 PM, said:

ELO also does not account for the rules changing. In chess you start the game with the exact same pieces in the exact same setup. In MWO the only thing that is the same is that there are 8 pieces n each side. Those pieces change wildly from match to match. The rules also change from game to game as we have assault maps and conquest maps.

You can't rate skill in this fashion. They would be better served by implementing a BV system and keeping two teams equal in that way. Of course BV is also subjective.

Actually, what they need to do is have an Elo system that works hand-in-hand with a BV system. Neither an Elo rating nor a BV rating is sufficient to gauge a players performance in an MWO match (IMO), there are too many variables. But if you can develop a system that can (attempt to) account for a players skill (Elo) +/- the effectiveness of the mech that he is in for a particular match then you might have a better system.

[BigBadHarv]

FerretGR, on 12 March 2013 - 03:34 PM, said:

http://en.wikipedia....gs_beyond_chess




http://mwomercs.com/...-drawing-board/

trolling as usual.

oh right, so the examples there that use established teams with a very limited stable of players are relevant to random 8 versus 8 games of mechwarrior where the pool of players is in the tens of thousands.
but yeah, say its trolling... that way it can't be right. so extremely typical of the masses

[Nidhoggr]

zmeul, on 12 March 2013 - 03:07 PM, said:

I had my own mischief with the ELO rating until it settled me to the correct bracket: http://mwomercs.com/...86#entry1983686



but, I see 2 major issues that affect one's ELO rating and matchmaking quality

• rating based on class; should be per chassis, or even per variant
  
• solo vs grouping

I play exclusively my D-DC, if I decide to play something else assault class, my rating and matchmaking quality will go to new places

My original forum post, made before the dawn of Elo, was about factoring in variant. I now suspect that this will be implemented, despite not hearing the devs say anything about it. I am now starting to suspect that either they are geniuses and had planned on implementing it all along or, if not, then at least accidental geniuses if the implement what I suggest below. Allow me to explain my theory, the pre-seeding was a very limited window and it would've been unreasonable to imagine everyone would play every mech they own during that time period. As such, had they implemented a by-variant, or even a by-chassis, version of Elo at the start, they would have been forced to fill in all the blanks of the mechs the entire player base didn't play during pre-seeding (that's a lot of mechs especially when you count the ones people don't own) with essentially the n00b Elo rating because there would be no statistics to go off of. This would have resulted in these kind of threads for months on end from the original player base as the original experienced players would keep falling into the newb brackets every time they bought a new mech and the ebb and flow of victories/defeat would have been even more jarring than what we are currently experiencing during the infancy of Elo. Given the way they currently did it, they can branch, and thus transition the original player base much more smoothly to the Elo system. By basing it on the weight class to start, the next step could be to base it on chassis, and now instead of filling in all the blanks with the n00b rating they can fill the blanks with a value more closely relating to your skill in a similar mech (i.e. just fill in all the blanks with your current weight class Elo rating at the time of the switch to a per chassis rating.) Then just do the same sort of branching from per chassis to per variant. Much smoother this way. If this was the secret plan all along -- clever, devs. Clever.

Edited by Nidhoggr, 12 March 2013 - 03:45 PM.

[Exoth3rmic]

freak, on 12 March 2013 - 01:52 PM, said:

That's a lose rate of 70%, if the ELO system is supposed to provide me with balanced matches, why isn't closer to 50%???

It isn't. Its meant to not overly penalise you for losing to teams which are judged by the system to be "better", and not overly reward you for rolling "easier" teams. The only time you'll see large swings is when quite the opposite occurs.

As for the rest of your post I simply disagree and will leave it at that.

[MechGorilla]

Matthew Craig, on 12 March 2013 - 02:42 PM, said:

If you take two teams of identical Mechs so zero tonnage disparity the outcome is likely very one sided if one team has a vast difference in skill over the other. In the end creating balanced matches is a complex problem, we have every confidence that the new system will create great matches given time to tune/balance it.

But isn't that the point of ELO, to balance out the skill levels on each side?

[FerretGR]

BigBadHarv, on 12 March 2013 - 03:39 PM, said:

oh right, so the examples there that use established teams with a very limited stable of players are relevant to random 8 versus 8 games of mechwarrior where the pool of players is in the tens of thousands.
but yeah, say its trolling... that way it can't be right. so extremely typical of the masses

I thought Elo wasn't relevant because it's only useful for the 1 on 1 game of chess? Now it's because those non-chess games that use it are established teams with limited stables? Guild wars isn't limited to pros playing for established teams IIRC:

wiki said:

Various online games use Elo ratings for player-versus-player rankings. In Guild Wars, Elo ratings are used to record guild rating gained and lost through Guild versus Guild battles, which are two-team fights. The initial K-value was 30, but was changed to 5 in January 2007, then changed to 15 in July 2009.[33] World of Warcraft formerly used the Elo Rating system when teaming up and comparing Arena players, but now uses a system similar to Microsoft's TrueSkill.[34] The MOBA game League of Legends uses an elo rating system as well.[35] The game Puzzle Pirates uses the Elo rating system to determine the standings in the various puzzles. Roblox introduced the Elo rating in 2010.[36] Counter-Strike: Global Offensive also uses the Elo rating.

But please, keep shifting the goalposts to suit your argument.

And yes, continuing to make claims like this, that Elo is only or chess or for whatever situation you claim it's limited to that's not MWO, or that the consumables announced are P2W despite PGI making the changes necessary to ensure they're not, you're probably trolling. But of course, I disagree with your troll-like points, and it's because your a forward-thinking genius and I'm one of the drooling masses, obviously. Can't be because you're not right.

Edited by FerretGR, 12 March 2013 - 03:52 PM.

[BigBadHarv]

FerretGR, on 12 March 2013 - 03:49 PM, said:

I thought Elo wasn't relevant because it's only useful for the 1 on 1 game of chess? Now it's because those non-chess games that use it are established teams with limited stables? Guild wars isn't limited to pros playing for established teams IIRC:



But please, keep shifting the goalposts to suit your argument.

And yes, continuing to make claims like this, that Elo is only or chess or for whatever situation you claim it's limited to that's not MWO, or that the consumables annmounced are P2W despite PGI making the changes necessary to ensure they're not, you're probably trolling. But of course, I disagree with your troll-like points, and it's because your a forward-thinking genius and I'm one of the drooling masses, obviously. Can't be because you're not right.

pretty much that. a complete lack of depth or substance to your understanding of the subject but you are so willing to defend extremely obvious wrongs in some weird effort to carry favour with a computer game developer. you never know, they might make you a mod someday
dont worry mate, i have a very very clear picture of you and how you think in my head. you are extremely common on teh internetz and what you think or say has absolutely no relevance on anything substantial whatsoever. good to see you hold grudges though, paints an even clearer picture.

yes, it has been used in video games and thats on wikipedia.... doesn't mean its any use though does it?
was it used for ladders for established teams with a limited somewhat exclusive stable? if so, then yeah, elo can be used in that way. the point you are missing is as follows: 1v1 chess is light years away from 8v8 games of mechwarrior online. the variables are too wild for it to be in any way accurate. if elo was used for a limited amount of players on established teams with limited transfers of players between teams, then yes, it could be accurate.
but in its current form, it is used for a pool of thousands of players on no established teams (premades being forced to have randomers with them keep in mind). the difference of its current use in MWO and the source you are quoting are light years apart. do you understand? no you don't because you don't want to because it doesn't suit your current agenda. stop being typical pls

oh, and for the record, i have been part of many ladders that have used an elo system in team based games.. and it worked. but if the same system was to be bolted on to public games of the ones i played, then no.. it would have been horribly inaccurate and a clusterf**k

Edited by BigBadHarv, 12 March 2013 - 04:04 PM.

[Vrekgar]

ELO is not designed for more than 1v1. It can be adapted but sacrifices must be made for speed in matchmaking, as well as trying to take into account any factor groups make.

However your also complaining about an unbalanced record when you have less than a thousand games in the record.

Think of it like flipping a coin. You can easily flip the coin 10 times and get the same result, despite the long statistical odds. Your sample size is too small to be an accurate measurement of an Elo matchmaking system. You need hundreds if not thousands of samples to determine if its giving you a balanced match, given that its trying to ostensibly match you with 50% losses.

[Nidhoggr]

BigBadHarv, on 12 March 2013 - 04:01 PM, said:

pretty much that. a complete lack of depth or substance to your understanding of the subject but you are so willing to defend extremely obvious wrongs in some weird effort to carry favour with a computer game developer. you never know, they might make you a mod someday
dont worry mate, i have a very very clear picture of you and how you think in my head. you are extremely common on teh internetz and what you think or say has absolutely no relevance on anything substantial whatsoever. good to see you hold grudges though, paints an even clearer picture.

yes, it has been used in video games and thats on wikipedia.... doesn't mean its any use though does it?
was it used for ladders for established teams with a limited somewhat exclusive stable? if so, then yeah, elo can be used in that way. the point you are missing is as follows: 1v1 chess is light years away from 8v8 games of mechwarrior online. the variables are too wild for it to be in any way accurate. if elo was used for a limited amount of players on established teams with limited transfers of players between teams, then yes, it could be accurate.
but in its current form, it is used for a pool of thousands of players on no established teams (premades being forced to have randomers with them keep in mind). the difference of its current use in MWO and the source you are quoting are light years apart. do you understand? no you don't because you don't want to because it doesn't suit your current agenda. stop being typical pls

oh, and for the record, i have been part of many ladders that have used an elo system in team based games.. and it worked. but if the same system was to be bolted on to public games of the ones i played, then no.. it would have been horribly inaccurate and a clusterf**k

Do you have an alternative? None of your arguments really 'say' anything. Given everything you have said, Elo still might be the 'optimal' course of action for pairing up teams. What optimal strategy do you suggest for matching thousands of players together at any given time? I think some people just don't understand the concept of an optimal strategy. In complex systems that's the best you can hope for. There are no perfect solutions for this sort of thing.

[BigBadHarv]

Nidhoggr, on 12 March 2013 - 04:11 PM, said:

Do you have an alternative? None of your arguments really 'say' anything. Given everything you have said, Elo still might be the 'optimal' course of action for pairing up teams. What optimal strategy do you suggest for matching thousands of players together at any given time? I think some people just don't understand the concept of an optimal strategy. In complex systems that's the best you can hope for. There are no perfect solutions for this sort of thing.

tbh, no system. let it be random.
have one queue for public games/grind fests where winning and losing simply dictates how much xp/cbills you earn. Another queue for premade teams of players. then have Community Warfare as the place where games actually mean something beyond xp/cbills.

[Inveramsay]

I'm beginning to wonder as well how well the ELO system is working at the moment. My perspective is the opposite though, I recently started playing assaults and have put in 84 matches in my two awesomes so far. Out of those 54 have been wins and 28 losses which gives me a ratio of almost 2:1 wins losses. KDR is very good for both of them. My overall win/loss ratio since the new tracking came into place for all mechs has been 3:2 and that is generally not because the teams have been unbalanced though occasionally it happens. Possibly the match maker seems to stack one team and I am mostly on it. All pugging so I would have expected my win ratio be closer to 1:1 rather than 3:2 but I suppose I haven't played more than a 160 matches so far and it might even out a bit more once my assault ELO normalises though I frequently drop with or against four man premades and very high percentage founders teams.

There is no reason why an ELO system wouldn't work, obviously a raw win/loss ratio wouldn't be very helpful as it is a team game and you don't carry your team that often though it certainly happens. I wonder whether the available playerbase at any given time looking for a match simply isn't big enough to have a similar enough players. I still see noobs in trials but they are far less than before ELO came in. I'm a decent enough player but would not expect that my presence will make such a big difference to the outcome of the match though I suppose my 3.7 KDR in the awesome I've played most with will skew it a bit.

[Deathlike]

Matthew Craig, on 12 March 2013 - 02:13 PM, said:

The current iteration of Elo on production is quite loose in the games it creates with respect to Elo rating and tonnage, this was intentional to keep the time to find a match down.

We're actively gathering data from production and working towards tuning the match making system to create better matches without significantly increasing the time to find a match. We haven't even had the first balancing pass yet for Elo so I think it's premature to speculate on how well Elo does or doesn't work for MWO.

As the tuning work progresses we will be monitoring and considering adding an additional rating for Group play so players would have a separate rating for lone wolf vs. group play if the data shows that it is necessary.

It's evident from how MM is working that it's optimized to speed. It does need to wait a bit longer for people to join teams.

Group ELO in premades are necessary... because like the current version of ECM, teamwork is literally a force multiplier, which has a greater impact in a match. I'm not saying solo players aren't good... but people already working together in a team game is exactly what this game requires to succeed.

[N0MAD]

Matthew Craig, on 12 March 2013 - 02:13 PM, said:

The current iteration of Elo on production is quite loose in the games it creates with respect to Elo rating and tonnage, this was intentional to keep the time to find a match down.

We're actively gathering data from production and working towards tuning the match making system to create better matches without significantly increasing the time to find a match. We haven't even had the first balancing pass yet for Elo so I think it's premature to speculate on how well Elo does or doesn't work for MWO.

Until you introduce weight matching into the game your data gathering is flawed and inacurate. How can you gather data when the game can match say 400 tons vs 600 ? lets say that the players are of similar skill lvl the 600 team will in most cases win.
Until weight matching is introduced all data gathered is useless.

[Nidhoggr]

BigBadHarv, on 12 March 2013 - 04:16 PM, said:

tbh, no system. let it be random.
have one queue for public games/grind fests where winning and losing simply dictates how much xp/cbills you earn. Another queue for premade teams of players. then have Community Warfare as the place where games actually mean something beyond xp/cbills.

First, please "see" my first post in this thread... if you are capable of such a thing. Next, realize that using statistics to make an educated decision is better than just making a random decision. Particularly when large numbers are involved as, you know, that's the whole reason the discipline of statistics was created... That discipline still exists and is taught, because, oh I don't know -- it works.

[SuomiWarder]

175 matches since Elo with 10 different chassis. 95 wins. That's .54 or a 54% win ratio. Matches about 50-50 solo pugging and drops with one to three allies in a group.