1911 replies to this topic

[TheCaptainJZ]

Cimarb, on 12 August 2014 - 01:12 PM, said:

When a team of very high and very low Elo players is matched against a team of average players, THAT is when you are likely to get a bad match....

A very legitimate concern. But without a clear solution.

[Goose]

Cimarb, on 12 August 2014 - 08:02 AM, said:

I would like more info about that test too. For instance, I think the average Elo of individual players, as opposed to a team, is very important. All we keep seeing is "the average Elo was 40-50 points", which is great until you realize that Team A had an individual Elo range of 1500 (all very high and very low Elo players) and Team B had an individual Elo range of 5 (all mid-Elo players).

Really: The use of the phrase "average" basically negates any sentence it's placed in, as it can mean anything you want.

If they'd said "one standard deviation of any given match-up went from 50 to 150," I'd have understood, and believed them.

[Kageru Ikazuchi]

Cimarb, on 12 August 2014 - 01:12 PM, said:

... snip ...

Cimarb, we've been over this a few times, and you may remember that while we share some ideas and perspectives, we differ on others.

I agree that the variance of individual Elo scores within a team can be a problem ... in the solo queue, the main culprit is likely more related to size of the player base in the queue and the desire to get create decent matches faster (rather than great matches slowly).

In the group queue, however, the origins of the problem of individual Elo variance are more diverse ...
- a team of skilled players, particularly one that plays together often, will often have a hard time finding a good team to go up against (which means they get matched with a handful of Elo "rocks" to balance out the average) ... but unless the high Elo group is the only one in the queue, it is likely that this will happen for both sides
- a couple of highly skilled players with a couple of new players (or average players) "along for the ride" (to borrow a phrase from JagerXII) will end up with a artificially low Elo score, underrepresenting their actual effectiveness ... again, two such teams could be matched against each other
- the group queue Elo imbalance can be exacerbated by the group size matching and weight class "relief valves" ... with group size, weight class and average team Elo as competing priorities, getting a great match will continue to be difficult
- a group of mid-to-high Elo players either goofing off in ineffective (troll?) 'mechs or rolling full meta ... in the former case, a great match is unlikely ... in the latter, this is the high-Elo arms race that occurs almost every NA prime time I play in ... the good groups keep tonning up and running their min-maxed builds to keep up with each other.

However - the group queue is not really in the "Clan vs. IS 90% win rate" discussion, since the "switch was flipped" for the solo queue only. It is possible, and even likely, that there were some matches that were a dozen mid-to-high Elo players in Clan 'mechs against a few Elite players and a bunch of new players in IS 'mechs. It is also similarly possible that the converse was true for some matches.

In general, I agree that it is possible (and maybe even likely) that most players who bought clan packages have a couple of thousand matches under their belt. As more of the clan 'mechs become available for C-Bills, this will become less of a problem.

PGI is not perfect, but we should have a little faith that they want to make a great game, and that they are working to ensure that (at least in the solo queue, where they have the most control) the "problem matches" are outside of the standard deviation ... that most matches are good matches, and some matches are great. Bad matches will always exist, unless we want 15 minute wait times in the solo queue.

Other than having a much, much larger player base (probably not going to happen before CW is fully live and IGP restarts their ad campaign), there is only one idea that I can think of (that is simple and elegant) that would help alleviate some of the problems. That idea is a dynamic Elo modifier for each variant. PGI has the win-loss ratio for each variant ... they could apply that to a player's Elo score, but different builds with the same 'mech would skew the results somewhat. However, only PGI hs the data that could possibly show whether or not this would make any difference.

Edited by Kageru Ikazuchi, 12 August 2014 - 05:20 PM.

[Cimarb]

Thank you, Kageru, for your response, as at least it was a response and not an attack. I also agree with you on a lot of it.

All I wanted to do was ask a question of Karl, since this thread has meandered quite a bit in his absence, but apparently that is not ok with Heffay the attack hound. It took me a while, but I have came to realize why so many longtime members of this community pounce on him as soon as he gets in a conversation. It is quite ridiculous.

Karl, if you read this, I would just like to know what the variance is within the teams, and what you think of that number (good/bad/needs improvement)?

[Gyrok]

Just for the sake of visibility, because I would really like to know...

Gyrok, on 12 August 2014 - 07:40 AM, said:

Karl, I have a question about the IS versus the Clans data...if you could oblige me that would be great.

1.) What was the ELO predicted win % for the Clans in those matches?

2.) What is the acceptable variance of a standard match for prediction versus outcome in your ELO algorithm?

I would like to know this, because if ELO predicted a 90% win ratio for the Clans based on ELO alone, then all is right in the world, and clan mechs are well balanced.

[Void Angel]

Cimarb, on 12 August 2014 - 12:13 PM, said:

Heffay, on 12 August 2014 - 01:47 PM, said:

Goose, on 12 August 2014 - 02:38 PM, said:

Kageru Ikazuchi, on 12 August 2014 - 03:17 PM, said:

Hijacked thread is hijacked - can we move the Elo (which is not an acronym) debate to another thread? Any of the circular threads refusing to acknowledge the validity of the system would be fine, but I'm trying to keep track of this thread, and arguments flood my notifications.

[Cimarb]

Void Angel, on 12 August 2014 - 05:59 PM, said:

Hijacked thread is hijacked - can we move the Elo (which is not an acronym) debate to another thread? Any of the circular threads refusing to acknowledge the validity of the system would be fine, but I'm trying to keep track of this thread, and arguments flood my notifications.

I was asking a question about Elo to the person handling Elo. I never wanted to get into an argument about it

[Hammerhai]

Just to answer for Karl in part: Apparently one of the reasons they use an ELO by weight class and not by chassis is the good old "computational explosion" phenomenon. IOW it simply is too much data to handle to be practical. Should even be fairly early in the thread, but I am old and decrepit and the memory fails.

[Kageru Ikazuchi]

Hammerhai, on 13 August 2014 - 04:07 AM, said:

Just to answer for Karl in part: Apparently one of the reasons they use an ELO by weight class and not by chassis is the good old "computational explosion" phenomenon. IOW it simply is too much data to handle to be practical. Should even be fairly early in the thread, but I am old and decrepit and the memory fails.

I figure the reason for Elo per player by weight class (rather than by chassis or variant) is so an experienced player in a new mech doesn't end up gong against scrubs for his first handful of matches.

That said, I do think that a dynamic variable for each chassis would make sense. A well-built Timber Wolf is not equal in "chance to affect a win" to a well-built Dragon.

[Jody Von Jedi]

Karl!!!

Save us from this endless Elo debate, please!!

J/K, I know you're busy.

Jody

[CyclonerM]

Kageru Ikazuchi, on 13 August 2014 - 05:41 AM, said:

That said, I do think that a dynamic variable for each chassis would make sense. A well-built Timber Wolf is not equal in "chance to affect a win" to a well-built Dragon.

That variable is the pilot

[Goose]

Kageru Ikazuchi, on 13 August 2014 - 05:41 AM, said:

I figure the reason for Elo per player by weight class (rather than by chassis or variant) is so an experienced player in a new mech doesn't end up gong against scrubs for his first handful of matches.

Non-average players will always go though rapid Elo shifts when driving a new 'Mech for the first couple of games …

[TheCaptainJZ]

An elo score per weight class and a chassis modifier might work. Would probably require more time to investigate than they currently have though.

[Goose]

I don't see the problem: More HDD space, some more work by the virtual server running MM, game starts to grow with the improvement in match quality, requiring more HDD space, some more work by the virtual server running MM, etc., etc., etc., drek, etc. …

[Shlkt]

I already consider this thread to be completely derailed, otherwise I would not pollute its content with the following observations.

1) Chassis modifiers would hide real imbalances that ought to be fixed via game mechanics rather than matchmaking. If the Locust is not viable at high levels of play then the game designers ought to give the chassis advantages or tweak game modes until the Locust is viable at high levels of play. Otherwise half the mechs in our collections become worthless piles of scrap that you only bring you when you want to fight easier opponents. You absolutely, positively should not use matchmaking to balance game mechanics.

2) Chassis modifiers could never be representative of the variety of builds on the field anyway. Just because the CTF-3D is often used by competitive players doesn't mean that every CTF-3D is running a good build; there are plenty of terrible builds for the CTF-3D. There are way, way too many variables to take into account. New players should not be penalized by the matchmaker just because they want to try a "competitive" chassis; they're not going to turn into amazing pilots just by jumping into a different cockpit.

3) Several posters here seem to regard a 12-2 match as imbalanced because the result was lopsided. That is simply not the case. Lopsided victories are common even when both teams are evenly matched within a narrow band of skill. Small tactical advantages, positional advantages, and other variables (e.g. one team has ECM and the other does not) compound until one team starts dropping like flies.

Lopsided victories also occur due to a lack of organization and discipline. In PUGs our pilots generally do not group up and fight to the bitter end. Instead they flee or play passively when taking fire. This allows the team with an advantage to gain an even greater advantage as they defeat in the enemy in detail. Mechs which are fleeing do not hold target locks. Mech which are fleeing do not return fire effectively. Pilots who are fleeing tend to panic and miss out on the few opportunities available to them to turn the tide.

Edited by Shlkt, 14 August 2014 - 06:18 AM.

[Modo44]

A battle value system can help with tracking more Elo values. Calculate average Elo for [chassis X] and use it as a modifier when a player starts using [chassis x] for the first time. That way you can get faster convergence despite having more Elo values.

As explained before, using battle values in addition to Elo has all kinds of abuse and **** up potential. Both systems try to do the same thing, and they would inevitably clash.

[TheCaptainJZ]

If there was a chassis modifier, it should be small and variable depending on the player. For instance, a player may be 1300 in heavies but suck in dragons. if the win/loss is lower, it should slightly drop the players elo when he plays dragons.

But your point about not balancing mechs with the MM makes sense. They shouldn't.

[Cimarb]

Modo44, on 14 August 2014 - 06:44 AM, said:

A battle value system can help with tracking more Elo values. Calculate average Elo for [chassis X] and use it as a modifier when a player starts using [chassis x] for the first time. That way you can get faster convergence despite having more Elo values.

As explained before, using battle values in addition to Elo has all kinds of abuse and **** up potential. Both systems try to do the same thing, and they would inevitably clash.

I was going to agree with you, but you have one thing that I completely disagree with: battle values and Elo are not the same thing and there is no reason for them to ever clash.

Elo is about determining player skill. I am not going to get into this one, as I do not feel like being attacked again.

Battle Value is about determining the capabilities of the mech chassis and loadout being used. It is completely separate from Elo, just like the current "generic battle value" of weight class is used. Currently, there are four generic Battle Values for MWO: light, medium, heavy and assault. If PGI has implemented the tonnage factor to matchmaker, there are then 17 generic Battle Values for MWO, which is better, but still no where near accurate compared to a true Battle Value system.

A true Battle Value system, such as that at www.heavymetalpro.com, is still not perfect, but the granular nature of it makes it the most accurate way to determine the capabilities of a mech available. It is also a very easy system to manage and update once it is set up, as a particular weapon can be tweaked and it adjusts the Battle Value of every mech that equips it equally. It is also not biased, like a "popularity value" system would be (such as Roland's), and cannot be gamed well at all (and that would be easy to adjust even if it did happen).

If a true Battle Value system is in place, it would then take the place of weight class and/or tonnage in the current system, which again has nothing to do with Elo.

[Modo44]

We had this circlejerk 30 pages ago. Adjust battle values, players adjust, Elo changes, mech usage changes, and you are back to adjusting battle values. WT has tried to combine the systems, and they routinely end up with edge cases that break matchmaking. And they have (to have) a BV update every 2 patches.

[Heffay]

Cimarb, on 14 August 2014 - 12:53 PM, said:

I was going to agree with you, but you have one thing that I completely disagree with: battle values and Elo are not the same thing and there is no reason for them to ever clash.

Elo is about determining player skill. I am not going to get into this one, as I do not feel like being attacked again.

Battle Value is about determining the capabilities of the mech chassis and loadout being used. It is completely separate from Elo, just like the current "generic battle value" of weight class is used. Currently, there are four generic Battle Values for MWO: light, medium, heavy and assault. If PGI has implemented the tonnage factor to matchmaker, there are then 17 generic Battle Values for MWO, which is better, but still no where near accurate compared to a true Battle Value system.

A true Battle Value system, such as that at www.heavymetalpro.com, is still not perfect, but the granular nature of it makes it the most accurate way to determine the capabilities of a mech available. It is also a very easy system to manage and update once it is set up, as a particular weapon can be tweaked and it adjusts the Battle Value of every mech that equips it equally. It is also not biased, like a "popularity value" system would be (such as Roland's), and cannot be gamed well at all (and that would be easy to adjust even if it did happen).

If a true Battle Value system is in place, it would then take the place of weight class and/or tonnage in the current system, which again has nothing to do with Elo.

Which is more useful? An LRM Stalker in River City, or an LRM Stalker on Caustic?

Which would you rather have on your team? The 2000 Elo pilot in an LRM Stalker in River City, or the 1000 Elo pilot in an LRM Stalker in Caustic?

Battle Value is an unnecessarily complex and gameable method of creating matches that end up being far worse than Elo.