Jump to content

Why Elo Doesn't Work Here


633 replies to this topic

#361 MischiefSC

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPip
  • The Benefactor
  • The Benefactor
  • 16,697 posts

Posted 24 January 2014 - 07:14 PM

View PostRichAC, on 24 January 2014 - 06:46 PM, said:



Well it should be tracked. I assumed it was before recent revelations, silly me.


Oh they definitely care about stats friend. Its also what makes sports popular to most fans. STATS. Especially fans of losing teams...lol

The general manager and owner of the teams care about wins. But The players themselves usually just care about money and their personal STATS which get them more money.

Noone uses fancy math equations that dont' even take stats into account, except a bunch of nerds who dont' even follow sports.

And yes we know, you think a better indicator of skill is an individuals constantly changing random teammates..... we shouldn't worry though, because if a player is rated higher after performing poorer in a match then players who performed better, which we judge immediately by stats, we just have to feel assured that it will all eventually "correct" itself in the end. Whenever that is.... but thats ok it will happen one day... you've done the math.

But Once again, noone cares about wins.

1. its a random team game.

2. the game is based on cbills.


Here's what you're missing Rich. It's why your Elo is hidden.

You're exactly right about stats like damage, kills, etc. They're what people care about. They're what you should focus on and enjoy, they're what you should get paid on.

They are not, in truth, what actually denotes 'skill' though, not in the context of truly winning or losing. Not the only thing anyway.

The point of Elo for the matchmaker is to try and ensure that when you get those kills, earn those cbills and XP, it's against similarly skilled opponents. One could almost say the true nature of the matchmaker is to *punish you* for doing well and *reward you* for doing poorly - it wants you to have a 1.0 win/loss.

The point of that however is to ensure that in every match as you earn cbills and do damage and get kills and assists and component destruction you're doing so against comparably difficult opponents. So that no matter how good or how bad you are your rate of advancement is relatively similar, otherwise some people would outstrip everyone else at a literally exponential rate - the better you do the more money and XP you get, the better gear you get so the more you win.... soon there's no way for a new player to bridge the gap. It's impossible to catch up unless you're one of the first people to excel.

It's a little more complex but that's the gist of it.

You keep mistaking high Elo for 'winning at the game'. The point of Elo is just to keep you in approximately equal matches as you earn cbills, get kills and play games and to ensure that you're always challenged (but not too much) when doing so.

Which is why win/loss is all that matters to the matchmaker. I had an 8 kill 4 assist game in a Battlemaster the other day - on Terra Therma. I made close to 400k cbills and 2500 xp. One of the best games I've had in a long time. It was like I just couldn't miss. I worked incredibly hard for that victory and I enjoyed the **** out of it.

Back in the days before Elo I'd get 5 kills, 6 kills in 8v8 reasonably often and when I dropped with friends in 4mans we would literally win 19 out of 22 straight matches. Didn't even have to worry much about tactics; line up, rush the set point on whatever map it was, kill the other team starting biggest to smallest. We'd only bother calling targets if we recognized a 4man on the other team.

Elo has done more to KEEP people in MW:O than any other change they've made in the last year. It's got a lot of improvements to be made in it to refine it even more and make matchmaking even better but not only does it work but it works remarkably well. My base KDR has plummeted all the way from almost 3.0 to 1.6 and my win/loss has dropped from around 2.5 to 1.2, painfully, over the last 2.5k matches I've had since Elo came out.

It's done the same for pretty much everyone. The point of Elo is to push you towards 1.0 win/loss. It does that. It does it very well in fact. It's not there to rank you, or reward you, or compare you to other players. It's there to ensure that your matches are as even as game population and other factors allow so that the kills/damage/assists/etc. that you earn are earned with the same effort everyone else puts forward.

Elo works. Time to move on.

#362 RichAC

    Member

  • PipPipPipPipPipPipPip
  • 661 posts

Posted 24 January 2014 - 07:47 PM

@Mishief
The problem with that system, is there can be an example of a player, playing poorly, not getting any kills or dmg, feeling like he is out of his league, but still be on the winning team. And then according to your system, get rated higher.

Then your answer is going to be, "it will even out eventually. It will auto correct itself, Don't worry its math, its perfect" Not a good enough answer friend. In how many games exactly will it be evened out? By that time the players true skill might already be different compared to the playerbase. Your system is always playing catchup to a players skill, but never actually getting there.

We have different definitions of what "skill" is apparently. You want to use a system because most other game companies use them. WHen even someone without a math degree knows it is not designed for team games. Ive also explained how I don't know of any other FPS game that has a rating system, let alone a matchamaker, besides quakelive. The Godfather of computer e-sports. And they only use an ELO for duels not for team games. I've explained to you how scoring lowest on a winning team, or server can still rate you lower in quakelive clan arena. They have failed for other reasons besides their MM, so now its PGI's chance to lead this industry back into popularity.

At least I'm glad you agree people care more about their match score. Or so roadkill doesn't have a heart attack, let me rephrase. their match stats.

But You just seem very condescending or demeaning when you say so, like us common folk care about such things simply because we don't know any better. Well I feel the same way about nerds who don't even follow or have played on athletic teams work in the sports industry. A billion dollar industry. And the only reason video games have never gotten to that level, is not only because of the anonymity, but they are not respected enough due to issues like we are discussing.

Roadkill wants to deny stats aren't what make athletic sports popular or what determines a players worth. But thats not even debateable for most people. Video games should be treated no differently.

But If you agree this is true stats are more important to people, and we should play for them, then how can you not agree that is what should be the criteria for matching people up? If that is wy people are playing the game, no matter how even their win loss ratio is, they will still complain and not be happy with the game otherwise.

Edited by RichAC, 24 January 2014 - 08:12 PM.


#363 MischiefSC

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPip
  • The Benefactor
  • The Benefactor
  • 16,697 posts

Posted 24 January 2014 - 08:10 PM

View PostRichAC, on 24 January 2014 - 07:47 PM, said:

@Mishief
The problem with that system, is there can be an example of a player, playing poorly, not getting any kills or dmg, feeling like he is out of his league, but still be on the winning team. And then according to your system, get rated higher.

Then your answer is going to be, "it will even out eventually. It will auto correct itself, Don't worry its math, its perfect" Not a good enough answer friend. In how many games exactly will it be evened out? By that time the players true skill might already be different compared to the playerbase. Your system is always playing catchup to a players skill, but never actually getting there.

We have different definitions of what "skill" is apparently. You want to use a system because most other game companies use them. WHen even someone without a math degree knows it is not designed for team games. Ive also explained how I don't know of any other FPS game that has a rating system, let alone a matchamaker, besides quakelive. The Godfather of computer e-sports. And they only use an ELO for duels not for team games. I've explained to you how scoring lowest on a winning team, or server can still rate you lower in quakelive clan arena. They have failed for other reasons besides their MM, so now its PGI's chance to lead this industry back into popularity.

At least I'm glad you agree people care more about their match score. Or so roadkill doesn't have a heart attack, let me rephrase. their match stats.

But You just seem very condescending or demeaning when you say so, like us common folk care about such things simply because we don't know any better. Well I feel the same way about nerds who don't even follow or have played on athletic teams work in the sports industry. A billion dollar industry. And the only reason video games have never gotten to that level, is not only because of the anonymity, but they are not respected enough due to issues like we are discussing.

Roadkill wants to deny stats aren't what make athletic sports popular or what determines a players worth. But thats not even debateable for most people. Video games should be treated no differently.

But If you agree this is true stats are more important to people, and we should play for them, then how can you not agree that is what should be the criteria for matching people up? If that is wy people are playing the game, no matter how even their win loss ratio is, they will still complain and not be happy with the game.


Let me help you with that -

The answer is, depending on the players skill, probably 200 to 500 matches. It could be way less then that depending on how well balanced the matches he plays are. The higher or lower your skill the more quickly you'll seat correctly.

If you flip a quarter 1,000 times Rich 500 of them will be heads and 500 will be tails. However, the odds are 100% that at least once in that 1000 flips you'll get heads 10 times in a row. The odds of you flipping a coin and getting heads 10 times in a row is 0.0009765625. You'll have gotten tails more often then heads 10 times by about the time you hit 1,000 flips and it'll even out.

What you're again failing to understand though is that your kills/damage/etc only matter TO YOU. They have NOTHING TO DO WITH HOW LIKELY YOU ARE TO HELP YOUR TEAM WIN.

If you get matched with people based on damage or the like then you're going to create an inherently biased system.

I don't know how else to explain this to you Rich. Damage, kills, KDR, assists, it can all be gamed. Those metrics have nothing, absolutely nothing what so ever to do with how likely you are to help your team win. Kill stealing, hiding while powered down, using LRMs for max damage instead of dual gauss and raking in cockpit kills, sprinting through matches, lasing everyone and DCing, there's as many ways to game those metrics as there are players.

You can't game win/loss to make yourself win more often than you should.

Also, tactical performance, coordination skills, drawing fire, taking fire to protect teammates, good scouting, competent flanking, pugbossing, tactical awareness, there are a whole array of skills and behaviors that can't be measured in points that are hugely impactful to your teams success. What it does affect?

Your win/loss, as an average, over hundreds of games.

Which is why anything other than win/loss, anything at all, is irrelevant in determining how likely you are to win against similarly ranked opponents. Not your damage. Not your kills. Not your assists.

You may like kills and assists. They may give you a warm fuzzy feeling, like getting an award for participating or being the kid to collected all the towels after the game or whatever 'everyone gets an award' sort of goodie is popular these days. You certainly get cbills and xp for them. You should! They're a good thing to reward people for as they have a sense of immediacy.

Yet that's there to pat people like you on the head and make you feel accomplished in the moment at the end of the match.

The matchmaker however can't be based on giving everyone GOOD FEELZ. It needs to actually be accurate in matching people based on how likely they are to win, not how likely they are to get a warm fuzzy.

The matchmaker would be a study in abject failure if it was based around giving you nice warm fuzzy feels and not accurately placing people based on their odds of winning vs each other for an overall balanced match.

So, Elo. It works. It's the smart choice and really the only choice. There's modifiers you can add to it based on how likely someone is to win in particular mechs, loadouts, etc. At no point should it be modified by damage, kills, etc. That'd be statistically unreliable, gameable and 'double-dipping' the metric.

#364 Deathlike

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • Littlest Helper
  • Littlest Helper
  • 29,240 posts
  • Location#NOToTaterBalance #BadBalanceOverlordIsBad

Posted 24 January 2014 - 08:23 PM

View PostRichAC, on 24 January 2014 - 07:47 PM, said:

@Mishief
The problem with that system, is there can be an example of a player, playing poorly, not getting any kills or dmg, feeling like he is out of his league, but still be on the winning team. And then according to your system, get rated higher.


That's not accurate.

I can win a bunch of games even if I feel I made little to no contribution to it. Chances are, my ELO isn't going to go up too much.

I can lose a bunch of games even if feel I made lots of contribution to it, but it didn't prevent the loss. Chances are, my ELO isn't going down too much.

In the games were it is "competitive" where your existence dictates the fate of the match, chances are, my ELO will consistently grow or shrink depending on MY OWN CONSISTENCY towards the match.

If you keep losing to cap matches, you'll probably have teammates that'll cap outright to win matches.

If you keep stopping the enemy from capping consistently, chances are you'll win the match more than you lose (especially if that is their only goal).

Quote

Then your answer is going to be, "it will even out eventually. It will auto correct itself, Don't worry its math, its perfect" Not a good enough answer friend. In how many games exactly will it be evened out? By that time the players true skill might already be different compared to the playerbase. Your system is always playing catchup to a players skill, but never actually getting there.


I think this most resembles the "now" generation where things must be done quickly or instantly, or it's considered a failure. You know... if things were that simple, we wouldn't have to do all these "slower" things. Deal with it until the situation improves (if ever).

Quote

We have different definitions of what "skill" is apparently. You want to use a system because most other game companies use them. WHen even someone without a math degree knows it is not designed for team games. Ive also explained how I don't know of any other FPS game that has a rating system, let alone a matchamaker, besides quakelive. The Godfather of computer e-sports. And they only use an ELO for duels not for team games. I've explained to you how scoring lowest on a winning team, or server can still rate you lower in quakelive clan arena. They have failed for other reasons besides their MM, so now its PGI's chance to lead this industry back into popularity.


Many games use some form of ELO concepts to form the matchmaking. It will continue to be used whether you understand them or not.

Quote

At least I'm glad you agree people care more about their match score. Or so roadkill doesn't have a heart attack, let me rephrase. their match stats.


You can still get an epic match score... except if you still lose, it's kinda irrelevant in the end. Can you produce it every game AND win? That's the feat that has to be undertaken.

Quote

But You just seem very condescending or demeaning when you say so, like us common folk care about such things simply because we don't know any better. Well I feel the same way about nerds who don't even follow or have played on athletic teams work in the sports industry. A billion dollar industry. And the only reason video games have never gotten to that level, is not only because of the anonymity, but they are not respected enough due to issues like we are discussing.


You need to get the basic ideas understood before you can properly disagree with them. You may not need "perfect math" to solve all woes, but there's a lot more to stuff than "just the numbers". There's a lot more analysis that can be done that isn't just specific numbers.

Quote

Roadkill wants to deny stats aren't what make athletic sports popular or what determines a players worth. But thats not even debateable for most people. Video games should be treated no differently.


In football, a great example. It is hard to "get useful numbers" or "telemetry" over an offensive lineman. I guess the "easy stat" people can pick out is "# of sacks caused by X player". However, in football.. you may not necessarily be the problem. Is it the guy next to you that didn't do his job? Did the QB hold on to the ball too long (either being indecisive or simply had noone to throw to)? Numbers don't tell the whole story. That is why people keep trying hard to find metrics, but let's be honest... you still have to use your eyes and understand there's a lot more to football than "# of sacks given up by a player".

Quote

But If you agree this is true stats are more important to people, and we should play for them, then how can you not agree that is what should be the criteria for matching people up? If that is wy people are playing the game, no matter how even their win loss ratio is, they will still complain and not be happy with the game.


There will be complaints no matter the reasoning. The question is... whether the match generated will be competitive or not. Right now, it's kinda not as competitive as it could be due to how the numbers are used... not ELO specifically.

#365 RichAC

    Member

  • PipPipPipPipPipPipPip
  • 661 posts

Posted 24 January 2014 - 08:27 PM

View PostMischiefSC, on 24 January 2014 - 08:10 PM, said:


Let me help you with that -

The answer is, depending on the players skill, probably 200 to 500 matches. It could be way less then that depending on how well balanced the matches he plays are. The higher or lower your skill the more quickly you'll seat correctly.


200 - 500 matches? haha so like months for most.. Ridiculous... by that time people already quit the game.

Quote

If you flip a quarter 1,000 times Rich 500 of them will be heads and 500 will be tails. However, the odds are 100% that at least once in that 1000 flips you'll get heads 10 times in a row. The odds of you flipping a coin and getting heads 10 times in a row is 0.0009765625. You'll have gotten tails more often then heads 10 times by about the time you hit 1,000 flips and it'll even out.


So now determining ones skill in sports is like flipping a quarter? haha more like ones skill at the roulette table. The more I talk to you the more this pains me.

Quote

What you're again failing to understand though is that your kills/damage/etc only matter TO YOU. They have NOTHING TO DO WITH HOW LIKELY YOU ARE TO HELP YOUR TEAM WIN.


of course they matter. I guarantee you people that have higher cbill averages, also have higher win/loss ratios. And not because of the win bonus obviously. Are you being serious man?

Quote

If you get matched with people based on damage or the like then you're going to create an inherently biased system.


based on match score plus wins is my suggestion. If you mean biased in the fact that only people with higher scores will get rated higher. Then of course. Thats the point.

Quote

I don't know how else to explain this to you Rich. Damage, kills, KDR, assists, it can all be gamed. Those metrics have nothing, absolutely nothing what so ever to do with how likely you are to help your team win. Kill stealing, hiding while powered down, using LRMs for max damage instead of dual gauss and raking in cockpit kills, sprinting through matches, lasing everyone and DCing, there's as many ways to game those metrics as there are players.

You can't game win/loss to make yourself win more often than you should.


Is this what it boils down to, your worried about sandbaggers? People can just create a new account in an f2p game bud. That is what ruined quakelives system no matter what they did. We call them alt accounts. In LoL they call them "smurfs" Riot bans for that and their community is very vocal against it. ID felt they should do nothing about it, and now they are going out of business.

I have always suggested you get a trial period before getting stats or ratings to stop smurfs/tier slummers. Also make stats public.

Also, people already know now that if they win, they will get ranked higher and reap no rewards for it. So now just add this to the list of many reasons why noone gives a dam about trying to win!!!

Quote

Also, tactical performance, coordination skills, drawing fire, taking fire to protect teammates, good scouting, competent flanking, pugbossing, tactical awareness, there are a whole array of skills and behaviors that can't be measured in points that are hugely impactful to your teams success. What it does affect?


all that leads to a higher score bud. And i'm not sayiing take win/loss out of the equation. What i'm saying is add match score to it.

Quote

Your win/loss, as an average, over hundreds of games.

Which is why anything other than win/loss, anything at all, is irrelevant in determining how likely you are to win against similarly ranked opponents. Not your damage. Not your kills. Not your assists.

You may like kills and assists. They may give you a warm fuzzy feeling, like getting an award for participating or being the kid to collected all the towels after the game or whatever 'everyone gets an award' sort of goodie is popular these days. You certainly get cbills and xp for them. You should! They're a good thing to reward people for as they have a sense of immediacy.


Well immediately we are going to judge how well we were matched in a game by those things that give us a warm fuzzy feeling. Otherwise PGI might as well pack it in. Immediacy is needed, we don't need a match maker to correctly rate us 500 games later. Absolutely ludicrous.

Quote

Yet that's there to pat people like you on the head and make you feel accomplished in the moment at the end of the match.

The matchmaker however can't be based on giving everyone GOOD FEELZ. It needs to actually be accurate in matching people based on how likely they are to win, not how likely they are to get a warm fuzzy.


If you think someone who avgs 1000 dmg a match is not one of the most likely people to win in this game. Then you come from another planet buddy. Ya you don't sound condescending at all...

Quote

The matchmaker would be a study in abject failure if it was based around giving you nice warm fuzzy feels and not accurately placing people based on their odds of winning vs each other for an overall balanced match.


no. you would just be out of a job and pro sports would still be a billion dollar industry.

Quote

So, Elo. It works. It's the smart choice and really the only choice. There's modifiers you can add to it based on how likely someone is to win in particular mechs, loadouts, etc. At no point should it be modified by damage, kills, etc. That'd be statistically unreliable, gameable and 'double-dipping' the metric.


i feel completely the opposite. And I thought you felt the same way which you now seem to be contradicting. Some people are great with the same loadouts, others are horrible at them. It comes down to personal preference and playstyle. It definitely should not be in the equation at all, only the end result should be.

And again, Noone cares about wins bud, sorry to say.

1. its a random team game.

2. its a game based on cbills.


get your resume ready.

Edited by RichAC, 24 January 2014 - 08:34 PM.


#366 Abivard

    Member

  • PipPipPipPipPipPipPipPip
  • Shredder
  • 1,935 posts
  • LocationFree Rasalhague Republic

Posted 24 January 2014 - 08:40 PM

Do any of you actually play chess?
If so follow this, Player A is very good, Player B is average.

player A spots his QUEEN!, Player B manages to win, someone else decides to adjust their Elo rating just like Player A had his queen, even though Player A was down a queen, this happens several times and Player B's skill has not really gone up, but his Elo skyrocketed.

He is now matched against a players at his new Elo, he loses, and loses and loses, but his score goes down slowly, he then goes back and plays against people who spot him pieces and it shoots up.

There is not a thing wrong with the Elo system, but it is being applied wrongly in these cases.

Elo is not the BE ALL AND END ALL. It is a formulae designed to rate chess players, it can be adapted to similar games.
MWO is not in the least bit like Chess... except they are both games.

#367 RichAC

    Member

  • PipPipPipPipPipPipPip
  • 661 posts

Posted 24 January 2014 - 08:46 PM

View PostDeathlike, on 24 January 2014 - 08:23 PM, said:


That's not accurate.

I can win a bunch of games even if I feel I made little to no contribution to it. Chances are, my ELO isn't going to go up too much.

I can lose a bunch of games even if feel I made lots of contribution to it, but it didn't prevent the loss. Chances are, my ELO isn't going down too much.

In the games were it is "competitive" where your existence dictates the fate of the match, chances are, my ELO will consistently grow or shrink depending on MY OWN CONSISTENCY towards the match.

If you keep losing to cap matches, you'll probably have teammates that'll cap outright to win matches.

If you keep stopping the enemy from capping consistently, chances are you'll win the match more than you lose (especially if that is their only goal).


Apparently its all based on the teams avg ELO. You have some good strategies for matches... Thats nice.



Quote

I think this most resembles the "now" generation where things must be done quickly or instantly, or it's considered a failure. You know... if things were that simple, we wouldn't have to do all these "slower" things. Deal with it until the situation improves (if ever).


Its just confirmation of how absolutely inaccurate this system is. People will be rated correctly month later for most people? That is an acceptable timeframe for you? And if thats true how can it ever be accurate? Can you imagine if stockbrokers worked this way?



Quote

Many games use some form of ELO concepts to form the matchmaking. It will continue to be used whether you understand them or not.


Oh so that makes it ok. First of all this is a team game. And second the teams are not static. And this industry will continue to die. Oh wait. I'm sure the out of work developers will be posting to prove me wrong now.



Quote

You can still get an epic match score... except if you still lose, it's kinda irrelevant in the end. Can you produce it every game AND win? That's the feat that has to be undertaken.


you don't need a math degree to know that guys who avg higher match scores, naturally have better win ratios. And besides who cares about wins in this game?


Quote

You need to get the basic ideas understood before you can properly disagree with them. You may not need "perfect math" to solve all woes, but there's a lot more to stuff than "just the numbers". There's a lot more analysis that can be done that isn't just specific numbers.


The only math that matters is on the scoreboard at the end of the match. If PGI wants to cater to a couple guys who want to use a fancy math formula and ELO to rank players based on wins, when noone cares about wins. They might as well start packing.



Quote

In football, a great example. It is hard to "get useful numbers" or "telemetry" over an offensive lineman. I guess the "easy stat" people can pick out is "# of sacks caused by X player". However, in football.. you may not necessarily be the problem. Is it the guy next to you that didn't do his job? Did the QB hold on to the ball too long (either being indecisive or simply had noone to throw to)? Numbers don't tell the whole story. That is why people keep trying hard to find metrics, but let's be honest... you still have to use your eyes and understand there's a lot more to football than "# of sacks given up by a player".


I've linked to what metrics are used in baseball. What it boils down to, is they still take stats into the equation. Not just wins. No agent is negotiating his clients salaries based on wins alone, cause after all, you already said it could of been due to his teamates.. This might be the reason guys get traded, but Stats are what determines their salary.



Quote

There will be complaints no matter the reasoning. The question is... whether the match generated will be competitive or not. Right now, it's kinda not as competitive as it could be due to how the numbers are used... not ELO specifically.


OH? so how would you use the numbers better? I gave my suggestions. IMO win/loss ratios for most players means ELO's are perfect. The problem, as I think your implying here, is not nescessarily w/l ratios. But how competitive the matches are, or i would say individual performance.

Tell me how do you judge how competitive a match is> I'm taking a wild guess. but could it be the scoreboard at the end of a match?

Edited by RichAC, 24 January 2014 - 08:49 PM.


#368 MischiefSC

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPip
  • The Benefactor
  • The Benefactor
  • 16,697 posts

Posted 24 January 2014 - 10:29 PM

View PostAbivard, on 24 January 2014 - 08:40 PM, said:

Do any of you actually play chess?
If so follow this, Player A is very good, Player B is average.

player A spots his QUEEN!, Player B manages to win, someone else decides to adjust their Elo rating just like Player A had his queen, even though Player A was down a queen, this happens several times and Player B's skill has not really gone up, but his Elo skyrocketed.

He is now matched against a players at his new Elo, he loses, and loses and loses, but his score goes down slowly, he then goes back and plays against people who spot him pieces and it shoots up.

There is not a thing wrong with the Elo system, but it is being applied wrongly in these cases.

Elo is not the BE ALL AND END ALL. It is a formulae designed to rate chess players, it can be adapted to similar games.
MWO is not in the least bit like Chess... except they are both games.


That you don't understand why you're wrong, even when you can look up Elo on Wikipedia and see how and why it's used by other games....

Elo is a system used to rate players of games by the most critical and least gameable factor - wins and losses. This allows for an overall ranking system that's not skewed by specific behaviors but in fact rewards alternative approaches to victory the same as standard approaches. It awards winning, however that's done.

That it was originally used for Chess is as relevant to the value of Elo as the fact that instant replay was first used in Canada for Hockey is to how useful it is in all other sports.

It's unfortunate that you're unhappy with the fact that Elo is the best way to rate players in a matchmaker in a game. Your opinion, your feelings, don't change the nature of that fact. That you don't understand the underlying math or statistics behind it is also not relevant in any way other than you are, in fact, dissatisfied by it.

View PostRichAC, on 24 January 2014 - 08:27 PM, said:


200 - 500 matches? haha so like months for most.. Ridiculous... by that time people already quit the game.

And again, Noone cares about wins bud, sorry to say.

1. its a random team game.

2. its a game based on cbills.

get your resume ready.


Thanks for the reminder as to why I quit trying to discuss this with you.

First, people will start seeing a shift towards their correct Elo within about 40 matches and a significant change within about 100 matches - unless they are already pretty average. I agree completely that it takes a painful number of games to seat correctly. This isn't a product of Elo however but of both A) size of the available player pool broadening the margin for error and B ) mix of premade and pug Elo causing a wider variance on skilled premade players when pugging, mixing with a high/low matchmaking to a target method.

The matchmaker in MW:O has nothing to do with my job. I have a great job that pays well and I'd have plenty of prospects if I wanted to look. Someone who actually understands statistics and their use and can put that to work in Excel can find work pretty easily. The average MLB team is worth about $750 million. The company I currently work for literally makes almost 2x that every month. The industry I work in, the 4 major players in it, could literally buy all 30 teams of the MLB in about 5 months.

It gets this rich by effective application of statistics. Oh the bitter irony.

Which is all beside the point.

I get that you either don't get or don't want to get why Elo is relevant. Okay, it doesn't matter. It's also not going to change nor is gaming going to move away from Elo - it is, in fact, drilling down and building on Elo to make something more comprehensive. At no point though is anyone using scores aside from win/loss to determine a players value in generating a win or loss for their team.

You are right in your signature though. The only math that matters is the math on the scoreboard at the end of the match - who won, who lost. Not how many yards one player rushed or interceptions or 3 point shots, but win vs loss. That you use that for a tag line and still don't get it though.... okay.

Best of luck to you Rich. The world must be a frustrating place to you and seem terribly unfair sometimes full of people and decisions that seem irrational or defy your 'common sense'. Maybe you should develop a multiplayer game ranking system and get famous with it. Shine on you crazy diamond.

#369 Deathlike

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • Littlest Helper
  • Littlest Helper
  • 29,240 posts
  • Location#NOToTaterBalance #BadBalanceOverlordIsBad

Posted 24 January 2014 - 11:01 PM

View PostRichAC, on 24 January 2014 - 08:46 PM, said:

Its just confirmation of how absolutely inaccurate this system is. People will be rated correctly month later for most people? That is an acceptable timeframe for you? And if thats true how can it ever be accurate? Can you imagine if stockbrokers worked this way?


I'll put it this way... you will still end up grinding about 100 matches to master a mech variant. By that time, ELO would have correctly put you in the "correct" ELO ranking by that time. It would take more to master an entire set (3 variants min), so that's 300 matches. That's more than enough, not that it will be perfect, but pretty much "good enough".



Quote

Oh so that makes it ok. First of all this is a team game. And second the teams are not static. And this industry will continue to die. Oh wait. I'm sure the out of work developers will be posting to prove me wrong now.


You know, if you keep using this excuse, you clearly have no argument.

Quote

you don't need a math degree to know that guys who avg higher match scores, naturally have better win ratios. And besides who cares about wins in this game?


I care, because I make more money doing so.

Quote

The only math that matters is on the scoreboard at the end of the match. If PGI wants to cater to a couple guys who want to use a fancy math formula and ELO to rank players based on wins, when noone cares about wins. They might as well start packing.


The scoreboard is useless in a cap loss. Sorry about that.


Quote

I've linked to what metrics are used in baseball. What it boils down to, is they still take stats into the equation. Not just wins. No agent is negotiating his clients salaries based on wins alone, cause after all, you already said it could of been due to his teamates.. This might be the reason guys get traded, but Stats are what determines their salary.


Stats do dictate salary. That's not what's at issue. For instance, Prince Fielder gets awesome numbers in the regular season. He's a terrible choke artist (to date) in the postseason. He's not going to get a great additional payout for his postseason numbers, but he'll sure cash in with what he has.

The thing is there is MORE TO THE NUMBERS than just salary. Some players are not reliable when you need them the most. Sometimes you can show that through stats (they track stats per inning, past the 7th inning, etc.). There's something about a player that is most effective at a particular ballpark (if he's a pull hitter, and the park favors him, like the short right porch for left handed hitters @ Yankee Stadium). To say "numbers dictate all" does not tell the entire story...

Quote

OH? so how would you use the numbers better? I gave my suggestions. IMO win/loss ratios for most players means ELO's are perfect. The problem, as I think your implying here, is not nescessarily w/l ratios. But how competitive the matches are, or i would say individual performance.


Individual performance doesn't always mean "epic damage numbers" and "kills" and "assists". If anyone typed out info like letting the teammates know that the mechs left are crit on a side/CT/legs, whatever, that can be the difference between winning or losing. Not everything is stat trackworthy, as much as you'd like to claim.

Quote

Tell me how do you judge how competitive a match is> I'm taking a wild guess. but could it be the scoreboard at the end of a match?


It's not. Sorry. It's like giving credit to the kill stealer, where he does like little damage, but collects the easy kill "because, IT'S EVERYTHING".

In sports, IT'S ALL ABOUT THE W(IN).

Edited by Deathlike, 24 January 2014 - 11:03 PM.


#370 Roland

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • 8,260 posts

Posted 24 January 2014 - 11:11 PM

Quote

And this industry will continue to die. Oh wait. I'm sure the out of work developers will be posting to prove me wrong now.

Why do you keep saying this?
You realize that the video game industry is the largest entertainment industry in the world now, right? GTA V pulled in $800 million in ONE DAY.

Honestly dude, where do you get these crazy ideas that the video game industry "is dying"?

Or are you suggesting that PC gaming is dying? Despite things like Steam seeing a 76% year over year growth in sales last year?

What exactly are you saying?

#371 Deathlike

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • Littlest Helper
  • Littlest Helper
  • 29,240 posts
  • Location#NOToTaterBalance #BadBalanceOverlordIsBad

Posted 24 January 2014 - 11:12 PM

View PostRoland, on 24 January 2014 - 11:11 PM, said:

Why do you keep saying this?
You realize that the video game industry is the largest entertainment industry in the world now, right? GTA V pulled in $800 million in ONE DAY.

Honestly dude, where do you get these crazy ideas that the video game industry "is dying"?

Or are you suggesting that PC gaming is dying? Despite things like Steam seeing a 76% year over year growth in sales last year?

What exactly are you saying?


He likes reading what he wrote. Because, reasons.

#372 Bhael Fire

    Banned - Cheating

  • PipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 4,002 posts
  • Twitter: Link
  • Twitch: Link
  • LocationThe Outback wastes of planet Outreach.

Posted 24 January 2014 - 11:32 PM

View PostMischiefSC, on 24 January 2014 - 10:29 PM, said:

Best of luck to you Rich. The world must be a frustrating place to you and seem terribly unfair sometimes full of people and decisions that seem irrational or defy your 'common sense'. Maybe you should develop a multiplayer game ranking system and get famous with it. Shine on you crazy diamond.


I get that you would probably refer to yourself in the third person if you thought you could get away with it without being laughed at. I get it, man... No, really I do.

But being a flat-out snarky dickhole all the time will only undermine your efforts to communicate.

Bhael Fire apologizes for being ironically snarky in this post.

#373 MischiefSC

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPip
  • The Benefactor
  • The Benefactor
  • 16,697 posts

Posted 24 January 2014 - 11:42 PM

View PostDeathlike, on 24 January 2014 - 11:12 PM, said:


He likes reading what he wrote. Because, reasons.


Confirmation bias, which is almost always present with belief perserverance, means that you're going to have to stop responding to him if you don't want to watch it repeat and repeat and repeat. It's not about facts, in some cases facts are viewed with great skepticism. Did you catch the bit where Rich talked about how all those people at MIT 'are just wrong'? We've all presented him with facts and links which were promptly ignored. It's not about the facts or reality or truth. For some people it's about feeling right.

It's things like this that remind me exactly why game developers tend to just ignore players. You know if I got a response to my comments on what I'd like to see changed or at least considered for the matchmaker and Paul says 'We looked at it, the telemetry shows that the improvement from switching to a Gaussian distribution and matching within a range instead of high/low to target would produce almost the exact same results' I'd be okay with that. They've got the telemetry, I don't.

Yet that's not the result they get. They get people like this telling them that well obviously then their record of game data is wrong, or they are not using enough cow bell.

The big mistake is getting roped back into the debate. It's not a debate. A debate requires two reasonable people. If one side is saying 1 + 1 = 2 and the other side says they feel better with 1 + 1 = 3, there's nothing to debate or discuss. You've put forward your 1 + 1 = 2, you've shown your work and demonstrated your point. Time to end the discussion and not get led off into an argument about how 3 looks prettier when next to 1s when you use colored pencils and how 3 is a lucky number in China and how 3 is the fourth number in the Fibonacci sequence and four is two sets of two and two is two sets of one, so one plus one probably equals two and everyone else is just wrong - and if they're not wrong they will be when science realizes how important that is.

That conversation goes nowhere. If you don't want to take the ride to crazytown you don't get in the crazytown cab. You've made your point, shown your work, demonstrated the math, facts and data behind it. People who can get it will get it. Those who can't, won't. Some people could get it but still won't. When someone has a vested interest in not understanding something (be that why Elo is in fact correct for a game like MW:O or why their bill is higher than they expected because they didn't actually pay the last bill in full) they will aggressively not get it.

1 + 1 = 2. Points been made (again, not the first time at the debate) and the only thing keeping it going is us, arguing with the cab driver to crazytown about why we're not going to pay in Danish tulips and ride strapped to the roof.

#374 MischiefSC

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPip
  • The Benefactor
  • The Benefactor
  • 16,697 posts

Posted 24 January 2014 - 11:48 PM

View PostBhael Fire, on 24 January 2014 - 11:32 PM, said:


I get that you would probably refer to yourself in the third person if you thought you could get away with it without being laughed at. I get it, man... No, really I do.

But being a flat-out snarky dickhole all the time will only undermine your efforts to communicate.

Bhael Fire apologizes for being ironically snarky in this post.


I've argued the point, including links and references and in some cases copies of the mathematical formula that confirm the point, for a dozen posts in this thread and literally more than 80 posts in 3 other threads a few months ago. When someones position is, literally, that the facts are not entertaining and people don't like them and so the mathematics behind how reliable matchmaking work should change then I reserve the right to shut the rational part of the debate down. I'd also point you to the references in his prior post to how somehow Elo as a matchmaking concept is going to 'fail' across the gaming industry and that this would in some way cost me my job or that statistical analysis was, in fact, somehow 'wrong' or some sort of trick and since I work in the field I would be needing a new job.

My response was the gentle version.

#375 Deathlike

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • Littlest Helper
  • Littlest Helper
  • 29,240 posts
  • Location#NOToTaterBalance #BadBalanceOverlordIsBad

Posted 24 January 2014 - 11:52 PM

View PostMischiefSC, on 24 January 2014 - 11:48 PM, said:


I've argued the point, including links and references and in some cases copies of the mathematical formula that confirm the point, for a dozen posts in this thread and literally more than 80 posts in 3 other threads a few months ago. When someones position is, literally, that the facts are not entertaining and people don't like them and so the mathematics behind how reliable matchmaking work should change then I reserve the right to shut the rational part of the debate down. I'd also point you to the references in his prior post to how somehow Elo as a matchmaking concept is going to 'fail' across the gaming industry and that this would in some way cost me my job or that statistical analysis was, in fact, somehow 'wrong' or some sort of trick and since I work in the field I would be needing a new job.

My response was the gentle version.


You're only "crazy" for countering his "logic".

Sometimes, I wonder why people don't question or at least reevaluate their own values. Then again, some people only "do what they are told" and don't think about it.

People don't usually critically think about why they go about things a certain way... so... you can't help them all.

#376 Bhael Fire

    Banned - Cheating

  • PipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 4,002 posts
  • Twitter: Link
  • Twitch: Link
  • LocationThe Outback wastes of planet Outreach.

Posted 24 January 2014 - 11:59 PM

View PostMischiefSC, on 24 January 2014 - 11:48 PM, said:


I've argued the point, including links and references and in some cases copies of the mathematical formula that confirm the point, for a dozen posts in this thread and literally more than 80 posts in 3 other threads a few months ago. When someones position is, literally, that the facts are not entertaining and people don't like them and so the mathematics behind how reliable matchmaking work should change then I reserve the right to shut the rational part of the debate down. I'd also point you to the references in his prior post to how somehow Elo as a matchmaking concept is going to 'fail' across the gaming industry and that this would in some way cost me my job or that statistical analysis was, in fact, somehow 'wrong' or some sort of trick and since I work in the field I would be needing a new job.

My response was the gentle version.


It's ok to walk away if you come to an impasse.

It's no slight on anyone's intellect if you don't have the last word in a pointless debate.

Trust me, nobody is keeping track of points or Elo for forum debates.

#377 Abivard

    Member

  • PipPipPipPipPipPipPipPip
  • Shredder
  • 1,935 posts
  • LocationFree Rasalhague Republic

Posted 25 January 2014 - 12:36 AM

Intelligent, well educated, ignorance.
You are parroting ideas without understanding them.

You keep confusing correlation with causation.

BTW, M.I.T. has no connection to Mr Elo whatsoever! In fact, M.I.T. has very little to do with theoretical mathematics, it is an engineering school, not a math school.

#378 Nightfire

    Member

  • PipPipPipPipPipPip
  • Legendary Founder
  • Legendary Founder
  • 226 posts
  • LocationAustralia

Posted 25 January 2014 - 01:21 AM

View PostRichAC, on 24 January 2014 - 09:01 AM, said:

So you hate sports, but have the nerve to post in a thread about sportsmanship. And then people wonder about the sad state of this industry.... No way was tiger woods only viable on the console. A Golf Game in 1986 is what started the pc revolution and proved that pc's were just as viable for sports games. It was EA sports before they were called EA sports. I've posted about it on these forums before. That same game was built upon and went through many evolutions, until 2008 when it was pulled from the PC. Golf Games were always the games to first showcase the new graphics engines. People were golfing with people from around the world on the pc buddy.



Oh! My! God! Can you straw-man any harder!? I hate sports games and yet you somehow manage to translate that into my mental conduct and psychological state. You have no idea what I am like and furthermore, sports games not the pinnacle of online sportsmanship and respect of ones adversary to the exclusion of any other team game. To infer as much is pure arrogance. To infer that I lack these qualities because of my dislike for sports games boarders on liable. Restrain yourself Sir!

The PC was the ONLY viable platform for sports games pre-PS1. Some sports games existed on early consoles but the PC is where the bastion of processing and graphic resources lay. As this changed, so did where Sports games were published. It is far more sociable to play in the lounge on a TV screen with people than in front of a PC alone. I have never said Sports Gaming on the PC wasn't viable, simply:

Quote

. It's a poor argument to say that games made for the only viable platform at the time have moved to a better suited platform that wasn't around when the game genre began


and I stand by that statement! The PC was the only VIABLE platform at the time. It ceased to be the ONLY viable platform and sports games IN GENERAL moved to the better suited platform. To infer that I stated PCs were not viable platforms for Sports Gaming means you are either intellectually dishonest or you lack critical reading skills. Your entire rant to this point lies on the premise of the straw-man you have built to vilify me.

Quote

MWO will be soon [be dead] too at this rate.


The brackets are mine to aid clarity of point without the preceding quote.

And unfortunately I agree with that perception.

Quote

A dozen large conquest servers in all of North America, hardly screams big online playerbase for me. And we are only talking a year after release. The country that always has the largest playerbase is Germany. They would have 100 servers in comparison. But that is because they are the preferred host site for all of Europe. And they do have the biggest gaming communities by far. Gotta love those Germans.


Your experience sucks. I'd have some empathy for you but you seem like a rather angry person so ... sucks to be you.

My FPS scene on the PC is booming! I play BF4 on a number of servers every night, I even have local servers for BF4 (You may not know it but this is a big deal). I even play a bit of Natural Selection 2 now and again.

Let's agree to disagree here on personal experiences and just say your individual mileage may vary.

Quote

I'd be surprised if it [world of Warcraft] went F2P to be honest. I still don't see them [Blizzard] having any competition, or any MMO with a 1/10th of their playerbase.


Blizzard has plenty of competition, WoW is aging and dying and Blizzard knows both of these things. Blizzard's own financial reports and Project Titan are evidence of that!

True! WoW has a phenomenal player base, possibly the likes no game will see again in this generation. Even given that, it's dying and Blizzard knows it!

Agreed. WoW is unlikely to go F2P. More likely they will scale it down and close WoW in favor of their next product. That might be F2P though!

Quote

Counterstrike still has more players then TF2 imo. A 15 year old game. Other fps games are dead in comparison. Including battlefield unfortunately. I agree FPS gaming on the console sucks without mouse and keyboard. But doesn't change the fact they are now more popular on the console. I also hear the new consoles let you use mouse/kb for BF3? Not sure if its true yet, but it will be eventually. I keep hearing about this star citizen lmao from these forums. Ya we'll see...


Yes CS is still strong and I have no idea why. It was fun up until about 1.1 (original, not source) and then it kinda lost it. You keep saying BF is dead but I don't see it, I'm having a blast in it (I still can't snipe for **** though!).

FPS is indeed popular on the console, it's still a poor platform for twitch gaming though a keyboard and mouse do considerably increase performance. Yes consoles now can support standard USB devices.

Elite: Dangerous and Star Citizen are both looking incredibly promising. The resurgence of the space sim is something akin to a frog in the rainforest. That is, they are both usually the first things to go.

Quote

This is the most quoted stat. Have you played any of those games on steam? I downloaded Age of Conan after it went free. Guess what....its empty.... Again these companies posting their sales stats only reminds me of Enron. I only care about how much of a playerbase is playing online. Start posting those stats... You use to have xfire, steam and gamespot. Gamspot no longer posts such stats for some mysterious reason. Xfire and steam are all we have to go on, but not everyione uses xfire, and steam player could be playing singleplayer for all we know.


Played any of what games on steam!? I apparently have 129 games in my steam library, I've played many games on Steam so you're going to have to be a little more specific. AoC is dead? And? There are many games that fail, that doesn't mean gaming is dead or that gaming on platform X is dead.

Quote

I go by what I see in the games I play, there is no other reason to look anywhere else. Especially since I've played most of them.


We get it, your gaming world sucks! It does indeed suck to be you!

Quote

It started dying in 2004. So in reality, you've been hearing about it for 15 years. Before that it was gaining in poularity for 20 years.


No, I've been hearing it for 30+ years just like I've lost track of how many times the world has ended! Just because in your world it's only been in decline for the last 10 odd years doesn't mean you can tell me how long I've been hearing it for. I've been a gamer since the days of the early PC through the C64 years and on. I know what I've heard and I know the reality. I've seen the ebbs and flows of the PC gaming industry and you don't get to redefine my experiences to conform to yours.

Quote

we have this debate constanlty on these forums, because people want to know why MWO has a very small playerbase. I bet according to you its thriving.....


Oh God you are amusing.


  • I've been on the forums for a while, I'm not a newbie to be lectured. The argument this quote was responding to was not in reference to player-base but rather why PC gaming was dying. You stated: "Also PC Sales in general have been doing down and so have their stocks. But Hey Microsoft stocks went up today, maybe there is hope...lol The only real debate, is why? Some blame consoles, some blame android phones. I blame hackers and viruses so I'm in the minority lol But hey, what do I know." So again you take a quote out of context, attribute it to a different conversation and re-frame it as you respond. Nice Straw-maning! In this context again I say, why have this debate!?
  • You bring up a new point of debate, decreasing player-base. You believe I think it's thriving!? You haven't done your homework on me very well have you? It is evedent you haven't even read any other posts of mine in any other thread! No, I'm one of the proponents that the Player Base is rapidly shrinking but hey, it's far easier to pigeon-hole me and then vilify the construct you've created, isn't it?


  • Finally, this is an Elo thread. Ask MischiefMC, we've butted heads before. We both have (I'm assuming from what he's said in the past about his employment) some decent experience in Statistics. I have multiple problems with how Elo (and the matchmaker, you can't divorce Elo's operation from the mechanism that implements and influences it) works and isn't directly tied to player performance and he thinks it is just fine. I will add at this point that even if I held everything he says as valid, the sample pool for Elo to find a new players skill level is so high as most new players won't play through it, in essence making it useless. Just count this as my attempt to bring this a little back on topic.

"The truth hurts. But when that time comes, it will be a happy time for gamers. You should only be worried about that if your malicious hacking cheater."

(I apologise for not using quote tags but PGI's forum can only handle so many before it just gives up!)

And no man has anything to fear if he just tells the truth, right?
All I will say is 1984 was a warning, not an instruction manual!

Now if you are going to respond could you please leave the Straw-man's and quote mining alone and respond as an intellectual.

Edited by Nightfire, 25 January 2014 - 01:36 AM.


#379 RAM

    Member

  • PipPipPipPipPipPipPipPipPip
  • The Resolute
  • The Resolute
  • 2,019 posts
  • Google+: Link
  • LocationVancouver, BC

Posted 25 January 2014 - 02:27 AM

After 20 pages, the only thing that is clear is that AC stands for Absolutely Clueless.

#380 RichAC

    Member

  • PipPipPipPipPipPipPip
  • 661 posts

Posted 25 January 2014 - 08:57 AM

View PostMischiefSC, on 24 January 2014 - 10:29 PM, said:


Thanks for the reminder as to why I quit trying to discuss this with you.

First, people will start seeing a shift towards their correct Elo within about 40 matches and a significant change within about 100 matches - unless they are already pretty average. I agree completely that it takes a painful number of games to seat correctly. This isn't a product of Elo however but of both A) size of the available player pool broadening the margin for error and B ) mix of premade and pug Elo causing a wider variance on skilled premade players when pugging, mixing with a high/low matchmaking to a target method.

The matchmaker in MW:O has nothing to do with my job. I have a great job that pays well and I'd have plenty of prospects if I wanted to look. Someone who actually understands statistics and their use and can put that to work in Excel can find work pretty easily. The average MLB team is worth about $750 million. The company I currently work for literally makes almost 2x that every month. The industry I work in, the 4 major players in it, could literally buy all 30 teams of the MLB in about 5 months.

It gets this rich by effective application of statistics. Oh the bitter irony.

Which is all beside the point.

I get that you either don't get or don't want to get why Elo is relevant. Okay, it doesn't matter. It's also not going to change nor is gaming going to move away from Elo - it is, in fact, drilling down and building on Elo to make something more comprehensive. At no point though is anyone using scores aside from win/loss to determine a players value in generating a win or loss for their team.

You are right in your signature though. The only math that matters is the math on the scoreboard at the end of the match - who won, who lost. Not how many yards one player rushed or interceptions or 3 point shots, but win vs loss. That you use that for a tag line and still don't get it though.... okay.

Best of luck to you Rich. The world must be a frustrating place to you and seem terribly unfair sometimes full of people and decisions that seem irrational or defy your 'common sense'. Maybe you should develop a multiplayer game ranking system and get famous with it. Shine on you crazy diamond.


You said 200-500 matches, now your trying to backtrack and say only 40. haha

SO your not only a math Genius, Your also Wealthier then all of us. I guess you know better then all of us commonfolk then when it comes to sports.

WHich is what MWO is. Its a sport friend. And the reason why video games will never be as popular, and why society can never respect them the same. Is not only because of anonymity and hackers, but because it is not treated like a sport by people in the industry.

How many yards one player rushed, how many interception or 3 point shots, determines his individual skill. More then if his team won or not. Period.

Ya, continue to get that money like Enron friend. good luck.

Edited by RichAC, 25 January 2014 - 08:58 AM.






16 user(s) are reading this topic

0 members, 16 guests, 0 anonymous users