Heffay, on 12 August 2014 - 12:33 PM, said:
It's a leading question. Taken in context we all know you're just looking for evidence that Elo is flawed and you want to bolster your case against it. Taken in context with everything you've said about Elo, it's clear what you are looking for.
How about this: The variance within the team will fall within normal statistical ranges for any game based off of an Elo matching system. The larger the player base, the closer it gets to zero. The existence of statistically negligible deviations from that are irrelevant, since they'll apply to *any* system you implement.
It's math dude.
Since you WANT to argue, I am happy to oblige.
You cannot take a large variance within a match - which is where quality happens - and average it to say that everything is ok because eventually things even out.
For example, the quality of a car. If you have one bad part, it can ruin the entire car and kill the person driving it. Just because you have 8,000 other cars that did not kill anyone does not mean that there is not a problem that needs fixed. The recent huge auto recalls were not because every car was breaking down. It was because a handful of people died or had serious accidents that led to the recall of millions upon millions of vehicles. That individual quality gap was so immense that the whole line of cars involved had to be repaired, just like Elo and matchmaker can be improved by finding the "bad part".
If you average everything, everything will eventually average. It is the law of averages. Only by looking at the individual match and variance within the two teams of that match, will you see the issue related to match quality.
You may be ok with winning a match horribly, then losing a match horribly, but I would rather have two very close fights that could have went either way up until the last person fell - THAT is a good match. That happens when the variance in skill between the individual pilots is very low. That means a team of inexperienced players is matched against a team of inexperienced players, a team of average players is matched against a team of average players, and a team of good players is matched against a team of good players. When a team of very high and very low Elo players is matched against a team of average players, THAT is when you are likely to get a bad match, and is happening quite often I would bet.
That all being said, while I highly dislike the entire Elo system, this was not a leading question, at least how you took it. This was trying to get info so we can have the issue of "high+low=/=avg" addressed within the current Elo system. If they want to revamp the whole system afterwards, great, but if we are going to use Elo, I just want them to be looking at the correct data, instead of an average that means very little.