Jump to content

Question For People With A Good Grasp Of Statistical Signifigance


83 replies to this topic

#21 Karl Streiger

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Wrath
  • Wrath
  • 20,369 posts
  • LocationBlack Dot in a Sea of Blue

Posted 26 January 2015 - 07:07 AM

View PostMercules, on 26 January 2015 - 05:39 AM, said:


Well.... supposedly even though you performed well your team may have been "intended" to lose. So the impact on your Elo was 0. In games where you did nothing but your teammates won the game you may have been slated to win anyway so there was hardly any impact on your Elo.

Now that in itself is frustrating but if Elo is working correctly then you are not losing rank because you have 11 derps and you, because Elo stopped and went, "Wait, he has 11 derps and so should lose to this team."

I think this if the more important answer:
when you are supposed to win - your ELO didn't rise either.

A good piloted YLW can have impact on the game - but you can't win a battle on your own. You can make the same moves - with the same success - but there is no determination if you win or loose - depends on the other players in your team.

Of course you can increase your impact. For example becoming "drop leader" in PUG Hell - and you don't get killed by a team member.
With a team that uses your "orders" because worse orders are still better than none - they will win more often then loose.
So maybe 33% of your "wins" could be count as a "rise" in ELO - the games you lost where ELO said you should win.

#22 Joseph Mallan

    ForumWarrior

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 35,216 posts
  • Google+: Link
  • Facebook: Link
  • LocationMallanhold, Furillo

Posted 26 January 2015 - 07:13 AM

View PostHeffay, on 26 January 2015 - 06:54 AM, said:


Yes, it should be just W/L. Winning isn't everything. It's the *only* thing that matters.

The Direwolf who does 0 damage sitting on a cap point during conquest to secure it and help his team win is more valuable than the one who ran off, did 1600 points of damage with 8 kills while ignoring the objectives.

Damage doesn't mean squat if you lose the war.
LOL. Man I used to think like this...25+ years ago!

Securing the win this way is a personal achievement only if you alone accomplished it. But If your team dies while you are sitting on Cap and you end up losing... You didn't do a thing to help. ;)

That is a gamble, and if more folks thought this way we would not be forced to fight it out cause "Capping is a lame tactic". :lol:

#23 Lily from animove

    Member

  • PipPipPipPipPipPipPipPipPipPipPip
  • The Devoted
  • The Devoted
  • 13,891 posts
  • LocationOn a dropship to Terra

Posted 26 January 2015 - 07:14 AM

View PostRedDragon, on 26 January 2015 - 07:03 AM, said:

Why? It doesn't give a perfect view on how you perform in matches, but it is way way way better than W/L. Because it at least measures your personal skill, either in capturing points or in dealing damage/helping your team. Basing it off W/L just doesn't make any sense (except maybe for teams that play a lot together in the same formation). You can be the best player on your team and it won't matter. Likewise you can be the worst player and it also won't matter.

You basically can't be a bad player and reach a high match score, and if you always have low match scores, you probably are not a good pilot. On the other side, it is perfectly possible that the best pilot here on the forums loses every single game because of bad team mates.

your logic is flawed by having the pro-matchscore biased view. Yes you can't be a bad player if your make high scores, no matter if you lose or win.

but automatically in reverse, you are not always a bad player of you have low scores. Because currently some assiting stuff like tagging and narcs get often rated way too low. And they are very luck dependend too. Got no lrm put in your team in your pug and then your worth drops a lot. and with this automatically comes lower score and lower elo judgement.
Same for the situation of 2 players, So a good guy A is having good aim, vs a player B being bad. player A kills player B in 150dmg and gets one kill.
While player C is just a averagely aiming guy, yet better than player B. and so hishald baked aim causes him to do 250dmg and achieves one kill. Now matchscore judges this guy better than the other.

You system by logic is not the better system ist just differently flawed. Yes the high skilled high scoring guy that get the bad team by luck will be judged downwards incorrectly. Just by randomness of MM.

But as much will the high skilled support guy be judged downwards when the randomness of MM does not give him tammates that synergises with his build. Even if they win.

#24 Heffay

    Rum Runner

  • PipPipPipPipPipPipPipPipPipPip
  • The Referee
  • The Referee
  • 6,458 posts
  • LocationPHX

Posted 26 January 2015 - 07:16 AM

View PostLily from animove, on 26 January 2015 - 07:00 AM, said:


but then we know its not about winning anymore or maybe it is?, becuse the direwolf making those 8 kills may have prevented 8 opponents from capping, or killing that single direwaolf camping the spot. So we can not entirely say how much everyone distributed to the kill within this entire scenario.

Actually a good distributed score to the matchscore depending on what one does would be the best way, but this now includes so many different variables, that it is very hard to judge how much truly distributed how much to the win scenario.


If you get 8 kills and lose, you may still have been the problem. Out of position, not supporting your team, etc.

You lost. You deserve the hit to your Elo. There was probably something you could have done aside from killing mechs, like focusing on the objectives.

#25 Shlkt

    Member

  • PipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 319 posts

Posted 26 January 2015 - 07:17 AM

View PostErrinovar, on 26 January 2015 - 03:29 AM, said:

at what point does a chassis win loss ratio become useful information?


I'll give you the answer you want and then explain why it's totally wrong and cannot be reliably determined with any measure of confidence :)

First of all, instead of W/L ratio it makes more sense to use Win % for this type of calculation. Your 1.3 W/L ratio translates into a 56.52% win rate.

If we assume that all other variables are constant (quality of opponents, your own performance in each match, unvarying meta, what you had for dinner last night, etc...) then after your 790 matches we can say with 90% confidence that your "real" win rate is between 53.54% and 59.36%. That corresponds to W/L ratio between 1.15 and 1.46.

If you had only played 100 matches then the range grows considerably: anywhere between 48% and 65%, corresponding to W/L ratios of 0.92 and 1.86, respectively. That's actually a huge gap and it's caused by nothing but dumb luck.

The above numbers, however, are not correct because none of our assumptions are really valid. Matchmaking is not constant - you will face tougher or easier opponents as your Elo fluctuates (or even due to a lack of suitable opponents). The meta changes over time, making your build more or less effective. Your own performance changes over time as you adapt to new play styles. What you had for dinner last night might make a difference :) Weekend events, tournaments, anything can change your performance and skew the results depending on how much you played during that period.

But the basic jist stays the same: you need to play a lot of games to get a reasonable estimate of your performance.

BTW: That's a pretty good win rate. You are only 1/12th of the team, after all, so if you can consistently tip the win rate by 6% by yourself then you're doing well.

Edited by Shlkt, 26 January 2015 - 07:23 AM.


#26 Lily from animove

    Member

  • PipPipPipPipPipPipPipPipPipPipPip
  • The Devoted
  • The Devoted
  • 13,891 posts
  • LocationOn a dropship to Terra

Posted 26 January 2015 - 07:19 AM

View PostHeffay, on 26 January 2015 - 07:16 AM, said:


If you get 8 kills and lose, you may still have been the problem. Out of position, not supporting your team, etc.

You lost. You deserve the hit to your Elo. There was probably something you could have done aside from killing mechs, like focusing on the objectives.


yes probably, probably not. as much as the guy on the spot probably may have helped to win, or PROBABLY NOT. if your team was leading 740:10 and you lost because getting killed entirely maybe the guy on the spot would have been better off helping the other guy kill the remaining 2 people who then came shredding the last guy on the spot.

Situations are way too complicated to say on a paper desing what really is how important. And especially in every matches specific situation this may entirely vary from the on paper designed judgement.

#27 Heffay

    Rum Runner

  • PipPipPipPipPipPipPipPipPipPip
  • The Referee
  • The Referee
  • 6,458 posts
  • LocationPHX

Posted 26 January 2015 - 07:19 AM

View PostShlkt, on 26 January 2015 - 07:17 AM, said:

I'll give you the answer you want and then explain why it's totally wrong and cannot be reliably determined with any measure of confidence :) First of all, instead of W/L ratio it makes more sense to use Win % for this type of calculation..


Elo doesn't use W/L ratio or Win % at all in its calculation.

#28 RedDragon

    Member

  • PipPipPipPipPipPipPipPip
  • 1,942 posts
  • LocationKurpfalz, Germany

Posted 26 January 2015 - 07:19 AM

View PostLily from animove, on 26 January 2015 - 07:14 AM, said:

your logic is flawed by having the pro-matchscore biased view. Yes you can't be a bad player if your make high scores, no matter if you lose or win.

but automatically in reverse, you are not always a bad player of you have low scores. Because currently some assiting stuff like tagging and narcs get often rated way too low. And they are very luck dependend too. Got no lrm put in your team in your pug and then your worth drops a lot. and with this automatically comes lower score and lower elo judgement.
Same for the situation of 2 players, So a good guy A is having good aim, vs a player B being bad. player A kills player B in 150dmg and gets one kill.
While player C is just a averagely aiming guy, yet better than player B. and so hishald baked aim causes him to do 250dmg and achieves one kill. Now matchscore judges this guy better than the other.

You system by logic is not the better system ist just differently flawed. Yes the high skilled high scoring guy that get the bad team by luck will be judged downwards incorrectly. Just by randomness of MM.

But as much will the high skilled support guy be judged downwards when the randomness of MM does not give him tammates that synergises with his build. Even if they win.

While I totally am with you on assists giving too few points (which should have long been changed by PGI), I am sure the system would be better. Right now, it kind of is a lottery if your team wins or loses. But as you yourself conceded, if you are a good player, you will have good match scores. In how many cases does a good player get so few points because he's only narcing/tagging? A neglectable amount, I'd guess, and the positive aspects of basing it on match score would far outweigh that. The system wouldn't be perfect, but it would be way better than the current one.

#29 Joseph Mallan

    ForumWarrior

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 35,216 posts
  • Google+: Link
  • Facebook: Link
  • LocationMallanhold, Furillo

Posted 26 January 2015 - 07:21 AM

View PostHeffay, on 26 January 2015 - 07:16 AM, said:


If you get 8 kills and lose, you may still have been the problem. Out of position, not supporting your team, etc.

You WE lost. You deserve the hit to your Elo. There was probably something you could have done aside from killing mechs, like focusing on the objectives.
Now its correct. And since when has focusing on THE objective been given anything more than Scorn from the peanut gallery Heffay?

#30 Davers

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Legendary Founder
  • Legendary Founder
  • 9,886 posts
  • Facebook: Link
  • LocationCanada

Posted 26 January 2015 - 07:24 AM

View PostJoseph Mallan, on 26 January 2015 - 03:44 AM, said:

I've always questioned Using W/L as teh metric for a player's Elo in a team based game.

For instance I can have 4 kills and 6 assists, The Team loses. I get points taken from my Elo cause of teh team's failure to win.

I can have no kills 1 assist the team wins and I get point Added to my Elo ...for being carried.

Our Elo system does not judge MY performance properly this way.


Winning is more important that getting kills. 2/3 game modes can be won without killing a single opponent (theoretically at least). Elo doesn't look at just one match- it looks at your entire history. If you are getting 4 kills and 6 assists every match, you will win a lot of games and your Elo would go up. But most people have good matches and bad mtches- and a lot of those factors are still within a player's control. For example, I've been running my YLW a lot lately- I use an XL. The YLW is not a 'top tier' mech and while I have great matches, I also have poor matches where I get dual gaussed from behind and die early. My mech choice effects my Elo, as it should, because mech building is part of the game as well. That is why good players prefer to run good mechs and good builds. People can claim that Elo doesn't work, but I have more good games than bad (win or lose) so I have to say Elo is working for me.

#31 Heffay

    Rum Runner

  • PipPipPipPipPipPipPipPipPipPip
  • The Referee
  • The Referee
  • 6,458 posts
  • LocationPHX

Posted 26 January 2015 - 07:25 AM

View PostLily from animove, on 26 January 2015 - 07:19 AM, said:

Situations are way too complicated to say on a paper desing what really is how important. And especially in every matches specific situation this may entirely vary from the on paper designed judgement.


Which is why winning is how you determine whether or not you're good, since winning is the only thing that matters.

#32 Lily from animove

    Member

  • PipPipPipPipPipPipPipPipPipPipPip
  • The Devoted
  • The Devoted
  • 13,891 posts
  • LocationOn a dropship to Terra

Posted 26 January 2015 - 07:25 AM

View PostJoseph Mallan, on 26 January 2015 - 07:21 AM, said:

Now its correct. And since when has focusing on THE objective been given anything more than Scorn from the peanut gallery Heffay?


heffay just wants to stand on the objective and twerk with his atlas.

#33 Errinovar

    Member

  • PipPipPipPipPip
  • Knight Errant
  • Knight Errant
  • 159 posts

Posted 26 January 2015 - 07:26 AM

What I'm trying to figure out is how many matches one has to play to make the win/loss ratio a significant indicator of how well that person plays a given chassis and how high or low the ratio needs to be in relation to 1 for it to be significant.

Lets say Bob drops 1000 times in a HBK-4G, all solo, and manages to have a 2.0 w/l ratio. Now to me I think at 1k matches a 2.0 w/l ratio says that if Bob solo drops in his HBK-4G he is giving his team a significant advantage because he plays the G well. That is statistically the simple fact of Bob dropping in the G makes his team statistically more likely to win.

That is what I am shooting for.

#34 Shlkt

    Member

  • PipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 319 posts

Posted 26 January 2015 - 07:28 AM

View PostHeffay, on 26 January 2015 - 07:19 AM, said:

Elo doesn't use W/L ratio or Win % at all in its calculation.


Elo influences quality of opponents and therefore it also influences your win rate, making it relevant to the calculation and the question at hand. That is why I mentioned it.

#35 Heffay

    Rum Runner

  • PipPipPipPipPipPipPipPipPipPip
  • The Referee
  • The Referee
  • 6,458 posts
  • LocationPHX

Posted 26 January 2015 - 07:29 AM

View PostErrinovar, on 26 January 2015 - 07:26 AM, said:

What I'm trying to figure out is how many matches one has to play to make the win/loss ratio a significant indicator of how well that person plays a given chassis and how high or low the ratio needs to be in relation to 1 for it to be significant.

Lets say Bob drops 1000 times in a HBK-4G, all solo, and manages to have a 2.0 w/l ratio. Now to me I think at 1k matches a 2.0 w/l ratio says that if Bob solo drops in his HBK-4G he is giving his team a significant advantage because he plays the G well. That is statistically the simple fact of Bob dropping in the G makes his team statistically more likely to win.

That is what I am shooting for.


That's tough to say, but it's directly reflected in the Elo score. Which, unfortunately, we can't see. There is some fancy math that will determine your probability of winning a match based on your team's combined Elo vs the enemy team's combined Elo, but how much you will be a factor in it (to a reasonably accurate percentage) depends on what every other Elo is on your team as well.

#36 Joseph Mallan

    ForumWarrior

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 35,216 posts
  • Google+: Link
  • Facebook: Link
  • LocationMallanhold, Furillo

Posted 26 January 2015 - 07:30 AM

View PostErrinovar, on 26 January 2015 - 07:26 AM, said:

What I'm trying to figure out is how many matches one has to play to make the win/loss ratio a significant indicator of how well that person plays a given chassis and how high or low the ratio needs to be in relation to 1 for it to be significant.

Lets say Bob drops 1000 times in a HBK-4G, all solo, and manages to have a 2.0 w/l ratio. Now to me I think at 1k matches a 2.0 w/l ratio says that if Bob solo drops in his HBK-4G he is giving his team a significant advantage because he plays the G well. That is statistically the simple fact of Bob dropping in the G makes his team statistically more likely to win.

That is what I am shooting for.

Bob plays 1000 games in his Hunchback, gets a 2.0 W/L, But Bob hides behind other team mates, shuts down in a hideyhole and is carried to that W/L.

I know it is an exaggeration, but the info I wanna see along with W/L is K/D and I would like to see Assists as well. No single stat is a good indication of how good a player is.

#37 Davers

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Legendary Founder
  • Legendary Founder
  • 9,886 posts
  • Facebook: Link
  • LocationCanada

Posted 26 January 2015 - 07:31 AM

View PostErrinovar, on 26 January 2015 - 07:26 AM, said:

What I'm trying to figure out is how many matches one has to play to make the win/loss ratio a significant indicator of how well that person plays a given chassis and how high or low the ratio needs to be in relation to 1 for it to be significant.

Lets say Bob drops 1000 times in a HBK-4G, all solo, and manages to have a 2.0 w/l ratio. Now to me I think at 1k matches a 2.0 w/l ratio says that if Bob solo drops in his HBK-4G he is giving his team a significant advantage because he plays the G well. That is statistically the simple fact of Bob dropping in the G makes his team statistically more likely to win.

That is what I am shooting for.

You seem to make it sound like Bob is getting better, but his team is still the same. What would happen, ideally, is Bob would be dropping with people with a similar Elo, so he will not be this stand out player who is personally responsible for his team's victory. He will just be a good player on a team of good players, facing a team of good player.

#38 Joseph Mallan

    ForumWarrior

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 35,216 posts
  • Google+: Link
  • Facebook: Link
  • LocationMallanhold, Furillo

Posted 26 January 2015 - 07:31 AM

View PostLily from animove, on 26 January 2015 - 07:25 AM, said:


heffay just wants to stand on the objective and twerk with his atlas.

I would too if I could get it to do so in game!!! :huh: :D

#39 Errinovar

    Member

  • PipPipPipPipPip
  • Knight Errant
  • Knight Errant
  • 159 posts

Posted 26 January 2015 - 07:32 AM

Hey I want to say a big Thanks to Solar and Shlkt. That was pretty much the answer to the question I was asking.

#40 Heffay

    Rum Runner

  • PipPipPipPipPipPipPipPipPipPip
  • The Referee
  • The Referee
  • 6,458 posts
  • LocationPHX

Posted 26 January 2015 - 07:33 AM

View PostJoseph Mallan, on 26 January 2015 - 07:30 AM, said:

Bob plays 1000 games in his Hunchback, gets a 2.0 W/L, But Bob hides behind other team mates, shuts down in a hideyhole and is carried to that W/L. I know it is an exaggeration, but the info I wanna see along with W/L is K/D and I would like to see Assists as well. No single stat is a good indication of how good a player is.


Statistically, this is almost impossible. With a win/loss rate of 50% based on a well balanced Elo score, the odds of having 333 more wins than average by being carried is... yeah. No. 2^333? The math escapes me at the moment.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users