58 replies to this topic

[Bhael Fire]

DAYLEET, on 15 July 2014 - 09:30 AM, said:

Individual player skills in a 1 on 1 scenario translate directly into win, how does it work in a pug game where you also depend on 11 other guy on your team?

If you're all similar skill level, that's how.

[Roadkill]

Bhael Fire, on 15 July 2014 - 09:27 AM, said:

And killing is often the best way to win in this game. The way this game is set up, the more efficient killers tend to win more.

Often, but not always.

Winning more often (what Elo ratings measure) is ALWAYS the best way to win in this game, and as such is the best way to rate players for matchmaking purposes.

It sounds like you want a skill ladder. Those are fine, but they're not actually very good at putting together balanced teams because they ignore the more subtle skills like cooperation and leadership. Elo takes those into account because it only looks at the end result.

[DAYLEET]

Bhael Fire, on 15 July 2014 - 09:33 AM, said:

If you're all similar skill level, that's how.

Well ELO can't tell if you are all similar level because it takes in account 11 other dude on your team.
And every time someone on your team screw up, or you get 2 locust guys or whatever else... your skill goes down? Why did you get worse? Will you try to emulate them those guys?

Edited by DAYLEET, 15 July 2014 - 09:46 AM.

[Bhael Fire]

Roadkill, on 15 July 2014 - 09:37 AM, said:

It sounds like you want a skill ladder. Those are fine, but they're not actually very good at putting together balanced teams because they ignore the more subtle skills like cooperation and leadership. Elo takes those into account because it only looks at the end result.

I get what you're saying and I agree to an extent. Now that solo players and groups have their own queues, I'm hoping that Elo will start working a little better in this game.

What would make it even better would be if they gave players separate Elo scores for their solo drops and their grouped drops.

DAYLEET, on 15 July 2014 - 09:44 AM, said:

Well ELO can't tell if you are all similar level.
And every time someone on your team screw up, or you get 2 locust guys or whatever else... your skill goes down? Why did you get worse? Will you try to emulate them those guys?

I don't think you read my other posts; I'm saying that using KDR and Assists along with W/L would be more accurate of assessing individual player skill than Elo.

[Egomane]

Bhael Fire, on 15 July 2014 - 09:27 AM, said:

That's why I said it needs to go. Matchmaker needs to be taking individual player skill into account, not just their W/L record.

What is player skill? How do you meassure it?

[Koniks]

Bhael Fire, on 15 July 2014 - 09:46 AM, said:

I don't think you read my other posts; I'm saying that using KDR and Assists along with W/L would be more accurate of assessing individual player skill than Elo.

Except we don't know the correlation of those stats to winning. So yes, they're good for measuring a certain set of skills (causing damage, killing more than dying) but they're potentially uncorrelated proxies for others.

If you want to create teams with a near-even probability of winning, then win loss record, opponents' win loss record, and opponents' opponents' win loss record are the most strongly correlated.

You can use things like margin of victory and player performance only in circumstances where schedules are somewhat normalized and every team or player will play each other.

If that's not possible, then you need to account for differences in likelihood to win by factoring in opponents in common. Which is basically what Elo does.

Edited by Mizeur, 15 July 2014 - 09:55 AM.

[DAYLEET]

Bhael Fire, on 15 July 2014 - 09:46 AM, said:

I don't think you read my other posts; I'm saying that using KDR and Assists along with W/L would be more accurate of assessing individual player skill than Elo.

Except maybe elo already take those into consideration, we dont know, we had a vague enumeration on a vlog last month with age of a player and other things, for all we know it takes every single stats of every mech into consideration.

I agree that now that solo queue exist it should help, i already see much better game and more fun over all. Playing pug and as a group requires different set of "skills".

Edited by DAYLEET, 15 July 2014 - 09:55 AM.

[Roadkill]

Bhael Fire, on 15 July 2014 - 09:46 AM, said:

What would make it even better would be if they gave players separate Elo scores for their solo drops and their grouped drops.

Yep. And also separate ratings for each Mech that they pilot.

You could get really crazy with Elo, and the crazier you got the more accurate it would eventually be. But it would also take longer to reach accurate ratings, so it's a trade-off.

The biggest problem with PGI's implementation right now is the whole "expected result" fiasco. Fix that and their ratings for new players would rapidly become more accurate. The ratings would also adjust more quickly when players switched from grouping to solo play. They've effectively crippled their rating system by including that clause, which no proper Elo system uses.

[Bhael Fire]

Egomane, on 15 July 2014 - 09:53 AM, said:

What is player skill? How do you meassure it?

KDR+Assists+W/L trumps just W/L alone when getting a better picture of player skill.

In this game, killing is far more indicative of player skill than winning, since winning alone is dependent on too many other factors.

[Roadkill]

DAYLEET, on 15 July 2014 - 09:55 AM, said:

Except maybe elo already take those into consideration, we dont know, we had a vague enumeration on a vlog last month with age of a player and other things, for all we know it takes every single stats of every mech into consideration.

We do know. They posted exactly how they implemented Elo, and it's a very standard system aside from one critical difference that essentially means it isn't an Elo system at all. PGI decided that your Elo rating shouldn't change if the team that's expected to win does in fact win (or loses when it's expected to lose). That dramatically slows down how quickly the ratings will converge on their correct values.

The only "stats" used to calculate Elo ratings are wins and losses, and the Elo ratings of your opponents.

[Adiuvo]

Bhael Fire, on 15 July 2014 - 10:18 AM, said:

KDR+Assists+W/L trumps just W/L alone when getting a better picture of player skill.

In this game, killing is far more indicative of player skill than winning, since winning alone is dependent on too many other factors.

I'd agree with this to a point. It is entirely possible, even if you're playing light mechs, to get your W/L to at least 3 from just the solo queue. Assault mechs can get it even higher. Past that it's iffy but if someone is really a top player, they're winning at least 75% of their matches.

[Roadkill]

Bhael Fire, on 15 July 2014 - 10:18 AM, said:

KDR+Assists+W/L trumps just W/L alone when getting a better picture of player skill.

In this game, killing is far more indicative of player skill than winning, since winning alone is dependent on too many other factors.

Assists in particular are VERY easy to game, and so aren't an accurate indication of anything at all.

KDR varies wildly by play style. Snipers tend to have higher KDR than brawlers because they rarely die during victories, yet without the brawlers to keep the enemy off the snipers the snipers wouldn't be able to compile those high KDRs.

Damage (which you haven't suggested, but others have) is also easy to manipulate. Strip the enemy to score extra damage before killing them, when it'd be better for the team to headshot every enemy. Or if low damage/kill is the goal, stand there while some newb pummels the enemy until you can kill it with one 50-pt alpha. Easily manipulated either way.

Anything other than wins a losses is too easy to manipulate.

[Heffay]

Mizeur, on 15 July 2014 - 09:33 AM, said:

Right. Elo works because each player is the common variable in every match they play. The edge case where Elo potentially fails is actually when a group of players earned their rating by playing exclusively as a team. If one of the players was then grouped in a random team against random opponents, Elo might not properly account for them. Their rating will effectively be the aggregate of their original team but not individualized for the player's own likelihood to win.

And likewise you're a common variable in every match you play. So if you lead your team to victory through whatever means necessary, your Elo increases. And if you are such a drag on your team that they can't compensate for your incompetence, your Elo drops until all of a sudden you're playing with people who are just as likely to win a match as you.

Elo works. Plain and simple.

Edited by Heffay, 15 July 2014 - 10:35 AM.

[Bhael Fire]

Roadkill, on 15 July 2014 - 10:24 AM, said:

Assists in particular are VERY easy to game, and so aren't an accurate indication of anything at all.

But as I pointed out before, the more kills/assists/wins you get the more you'll be matched against others that get high kills/assists/wins...there's no "gaming" it.


Regardless if you attempt to manipulate your stats, you're going up with others with similar stats. So if you gimp yourself to try to get easier kills/assists/wins, you're only hurting yourself in the end and will spend your entire time in the game spinning your wheels.

[Roadkill]

Bhael Fire, on 15 July 2014 - 10:33 AM, said:

But as I pointed out before, the more kills/assists/wins you get the more you'll be matched against others that get high kills/assists/wins...there's no "gaming" it.

The point is that you can game your assists (or other stats) in a way that is unrelated to actual performance. You can't game win/loss because that's a 1st order stat.

Assists are at best 2nd order - people who tend to win a lot tend to have a lot of assists, but it's not causal.

Quote

Regardless if you attempt to manipulate your stats, you're going up with others with similar stats. So if you gimp yourself to try to get easier kills/assists/wins, you're only hurting yourself in the end and will spend your entire time in the game spinning your wheels.

Not at all. I can run up my Assists whether my team wins or loses. The point is that Assists don't necessarily lead to winning.

It's correlation, not causation.

[Heffay]

Roadkill, on 15 July 2014 - 10:49 AM, said:

The point is that you can game your assists (or other stats) in a way that is unrelated to actual performance. You can't game win/loss because that's a 1st order stat.

Assists are at best 2nd order - people who tend to win a lot tend to have a lot of assists, but it's not causal.


Not at all. I can run up my Assists whether my team wins or loses. The point is that Assists don't necessarily lead to winning.

It's correlation, not causation.

I don't get why so many people have such a hard time understanding why W/L is really the only metric that matters. A 0 damage 0 kill 0 assist game you win where you played area defense makes you a far better player than a 1200 damage 8 kill match where you lost.

[Roadkill]

Heffay, on 15 July 2014 - 10:52 AM, said:

I don't get why so many people have such a hard time understanding why W/L is really the only metric that matters.

Because it can't be gamed, so they can't artificially inflate their e-peens. A lot of people like KDR because it feels like a valid metric, but it's also easy to manipulate against (and with) lesser skilled players so they can inflate their KDR and make it look like they're better than they actually are.

Elo speaks the truth. They can't handle the truth.

[Bhael Fire]

Heffay, on 15 July 2014 - 10:52 AM, said:

I don't get why so many people have such a hard time understanding why W/L is really the only metric that matters.

I get why Elo advocates think winning is the only metric; I just disagree.

Heffay, on 15 July 2014 - 10:52 AM, said:

A 0 damage 0 kill 0 assist game you win where you played area defense makes you a far better player than a 1200 damage 8 kill match where you lost.

See, that is COMPLETELY up for debate.

If you scored 1200 damage and 8 kills you did more for your team than the guy that did 0/0/0, regardless if you won.

Edited by Bhael Fire, 15 July 2014 - 11:03 AM.

[Roadkill]

Bhael Fire, on 15 July 2014 - 11:00 AM, said:

I get why Elo advocates think winning is the only metric; I just disagree.

For the purpose it's designed for, it's the only metric that matters.

For other purposes, I completely agree that other metrics are better. And in fact the matchmaker could use more than just Elo when creating matches, but it's important to make that distinction.

Elo shouldn't use anything but wins and losses. The matchmaker could use other metrics and/or other ratings to help it create balanced matches. But it makes no sense to modify Elo to incorporate other metrics because doing so makes it less suitable for its intended purpose.

It's a clarity of purpose issue. Fix the matchmaker, not Elo. Elo doesn't need fixing.

[Koniks]

Bhael Fire, on 15 July 2014 - 10:18 AM, said:

In this game, killing is far more indicative of player skill than winning, since winning alone is dependent on too many other factors.

Except there's more that goes into winning than just damage caused, getting the final tick of damage on a killed mech, or whether any ticks of damage were made on a killed mech. So at best, those stats are measures of a particular skillset that are 1) proxies for the underlying piloting skills 2) not a complete reflection of player ability 3) not necessarily strongly correlated with wins, and 4) if they are correlated, not necessarily the direct cause.

Those stats are also earned in an environment where players are already pre-sorted. So they have to be normalized for Elo rating, anyway.

Winning is the stat most strongly correlated with predicting future wins. And that's all that Elo is designed to do--match you with and against players who are equally likely to win.

It's not a measure of player skills.

Edited by Mizeur, 15 July 2014 - 11:09 AM.