15 replies to this topic

[David Sumner]

Stats capture (possibly including Elo calculation) is still invalid


Having started a new mech variant since the last patch, I can verify that the stats page is still failing to correctly display Win/Loss.


I can also demonstrate from the stats that it is failing to record losses.


If these same failure are occurring for Elo, then it is artificially inflating my Elo score, and there by causing the effect to amplify


HBK-4H 8 3 2
HBK-4P 5 1 2
HBK-4J 3 2 0

So, according to the W/L recorded, I have W/L between 0.5 and 2.0

Well, I can assure you that I have lost FAR more than I have won in these 16 matches

In fact, 10:6 would seem pretty much exactly right to me.

[David Sumner]

From today, 2 matches played:

Starting:
HBK-4P 7 2 2
HBK-4J 8 4 3

Ending:
HBK-4P 8 2 3 I stayed after dying, yes I did damage
HBK-4J 9 4 3 I DC'd after dying, yes I did damage

[Kamenjar]

Hmm good catch. It would help if you at least explained the numbers, but after reading multiple times, I got it.
Maybe that would explain my streaks of losses. Took me maybe 8+ hours to get 10 points during the weekend's challenge and matchmaking was just plain horrible. I was grouped with guys playing in third person, ppl using 12 ML Novas etc.

[Willard Phule]

Heh...he thinks it matters.

It might mean something in the group queue, but when you're playing solo....dude, the better you are, the more you carry. If your Elo got high in the group queue, don't expect anyone in the solo queue to be even close to you. You have one Elo per weight class....it's not broken down into group/solo Elos.

[David Sumner]

@William, I don't even know what you think you are saying.

What I'm saying is that if the Elo win/loss calculation is triggered at the same time as it is recorded for stats (and no other way makes sense to me)

Then the behaviour I'm showing above is artificially inflating my Elo. Which in turn is exacerbating long runs of losses, a severe loss of fun, and further inflating my Elo.

Welcome to recursion.

[David Sumner]

Chassis, Games, Wins, Losses
HBK-4J 10 5 3

Loss - stayed connected after death
HBK-4J 11 5 4

Win - lived, did 0 damage (blasted phone calls), won 12:1 - bad match maker
HBK-4J 12 6 4


New mech
HBK-4P 12 3 6

Lost - Died, Disconnected before end, no new match started
HBK-4P 13 3 7

Win - Died, stayed connected
HBK-4P 14 4 7


New mech
HBK-4H 9 4 2

Lost - Died, Disconnected before end, probably started new match but long wait
HBK-4H 10 4 3

Edited by David Sumner, 29 January 2015 - 02:54 AM.

[David Sumner]

Had a bright idea, and checked XP as well


Chassis Matches Win Loss XP
HBK-4J 12 6 4 795

- Died, Disconnected before end
HBK-4J 13 6 4 1125

So XP is awarded, but W/L is not.

So if Elo goes with XP, all good
If Elo goes with stats, all not good.

Hey PGI, you want to say which way it goes?

[Willard Phule]

David Sumner, on 27 January 2015 - 09:58 PM, said:

@William, I don't even know what you think you are saying.

What I'm saying is that if the Elo win/loss calculation is triggered at the same time as it is recorded for stats (and no other way makes sense to me)

Then the behaviour I'm showing above is artificially inflating my Elo. Which in turn is exacerbating long runs of losses, a severe loss of fun, and further inflating my Elo.

Welcome to recursion.

Show me where you're getting this information. Link some stuff to posts about how the MM and Elo works.

But, remember, it was re-written in June when the clans come out....so all your information needs to be current as of June 2014.

Guess what? You can't find anything. That's sort of the point.

And it's Willard, not William. Reading is fundamental.

Edited by Willard Phule, 30 January 2015 - 07:22 AM.

[David Sumner]

My apologies Willard.
I guess my eyesight's a little blurry at 1 am.

And I still don't see how anything you've said is relevant to what I'm saying.

At some point post match, Elo has to be updated, because you know, "not magic".

I'm a programmer, so I've got a pretty good idea how you generally go about writing these sorts of things.

And there are at least 3 things that happen post match.
Elo, XP, and stats.

There are 2 to 4 different ways you could implement recording them, depending on your consistency/transactional/performance/timeliness requirements.

Given that W/L stats are provably NOT updated while XP is, it implies that consistency is not a requirement for PGI across them.

So that in turn leaves an open question about when/how Elo is updated.

If I can trigger a stats failure in a consistent fashion, that's a reproducible bug.
One that should be fixed.
It's likely but not certain the Elo and XP updates use the same or similar mechanism as the stats, in which case they are subject to the same failure conditions.

If Elo is failing to update under the conditions I indicate for state, it means your Elo will end up artificially high.

What Elo does in detail is irrelevant.
How Elo is calculated is irrelevant.
When Elo is updated matters.
How the update is implemented matters.

[Willard Phule]

David Sumner, on 30 January 2015 - 10:35 PM, said:

My apologies Willard.
I guess my eyesight's a little blurry at 1 am.

And I still don't see how anything you've said is relevant to what I'm saying.

At some point post match, Elo has to be updated, because you know, "not magic".

I'm a programmer, so I've got a pretty good idea how you generally go about writing these sorts of things.

And there are at least 3 things that happen post match.
Elo, XP, and stats.

There are 2 to 4 different ways you could implement recording them, depending on your consistency/transactional/performance/timeliness requirements.

Given that W/L stats are provably NOT updated while XP is, it implies that consistency is not a requirement for PGI across them.

So that in turn leaves an open question about when/how Elo is updated.

If I can trigger a stats failure in a consistent fashion, that's a reproducible bug.
One that should be fixed.
It's likely but not certain the Elo and XP updates use the same or similar mechanism as the stats, in which case they are subject to the same failure conditions.

If Elo is failing to update under the conditions I indicate for state, it means your Elo will end up artificially high.

What Elo does in detail is irrelevant.
How Elo is calculated is irrelevant.
When Elo is updated matters.
How the update is implemented matters.

Here's my point.

We (Egomane and I) have asked Karl Berg repeatedly to explain how Elo/MM works now that everything has been rewritten (the MM was rewritten in June 2014). He refuses to tell us anything.

So, anything you find out there that explains how it works is dated and inaccurate.

Things that we DO know are:

- you have an Elo for each weight class...it is NOT separated into group/solo brackets. If you play in the group queue and do well, you'll have a high Elo. When you come to the solo queue, it is still high....meaning you'll be forced to carry several lower ranking players to even out your score.

- Elo calculation is completely messed up. We know you are GIVEN an 1800 as soon as you're done with your first 25 matches, regardless of how well you actually do. That places you squarely in the middle of the Elo range.

- The way the MM was rewritten takes into account a team's average Elo...that means if you have a high Elo, expect lower ones to be placed on your team to even you out with everyone else.

Other than that, we don't know squat.

[David Sumner]

But none of that is relevant to what I am saying.

As long as a win is supposed to increase Elo, and a loss reduce Elo, then if Elo is not updated for some losses, you will end up with an inflated Elo.

My data indicates that disconnecting (which you are more likely to do after a loss) causes at least some end of match functions not to be done.

And high Elo should not directly lead to worse people being assigned to your team. If that was the case they would be driving all their best users away in frustration.

More likely, MM tries to match the first player or group on each side, then "balances" from there.

[Willard Phule]

David Sumner, on 02 February 2015 - 02:08 AM, said:

But none of that is relevant to what I am saying.

As long as a win is supposed to increase Elo, and a loss reduce Elo, then if Elo is not updated for some losses, you will end up with an inflated Elo.

Where do you get that information from? Are you POSITIVE that winning increases your Elo? Are you absolutely certain of that?

Before the re-write, if you were on the team that was predicted to lose by the Matchmaker and you actually won, your Elo could increase by as much as 50 points. That's the way it was prior to June. Noone knows what it is now, they're not putting out that kind of information. With a spread of 1700 points available to put teams together, 50 points doesn't matter.

David Sumner, on 02 February 2015 - 02:08 AM, said:

My data indicates that disconnecting (which you are more likely to do after a loss) causes at least some end of match functions not to be done.

No offense, man, but your data is just that....your data. Without any input from PGI as to how the whole thing works, that's all it is.....data.

My data says that in the solo queue, the better I am...the more new players I'm expected to carry. Granted, that's just my data. I can't prove crap with it.

David Sumner, on 02 February 2015 - 02:08 AM, said:

And high Elo should not directly lead to worse people being assigned to your team. If that was the case they would be driving all their best users away in frustration.

Really.

The information we DO have came from the rewrite at the time of the Clan release. At that time, the matchmaker was designed to put teams together based on "team average" Elo......which should work in the Group queue, as they're using the average of every premade group to put teams together.

In the Solo queue, that equates to having lower Elo players being used to balance out the higher Elos....which normally come over from the Group queue, then ***** about how terrible the solo players are.

David Sumner, on 02 February 2015 - 02:08 AM, said:

More likely, MM tries to match the first player or group on each side, then "balances" from there.

Again, have you even bothered to read what little bit actually HAS been put out about the matchmaker since June 2014? You really should look it up. Team Average Elo is what everything is based on.

[warner2]

The main point being made by the OP is that based on what he is seeing the game is apparently either not displaying or is losing track of wins and loses. If it can't record those basic statistics accurately, that's a pretty big bug, and could impact other statistics that build on those basics.

[Ironwithin]

warner2, on 02 February 2015 - 08:13 AM, said:

The main point being made by the OP is that based on what he is seeing the game is apparently either not displaying or is losing track of wins and loses. If it can't record those basic statistics accurately, that's a pretty big bug, and could impact other statistics that build on those basics.

That is incorrect.

What the OP is seeing is that the WEBSITE does not keep track of wins and losses correctly.
We have no way of knowing if that is purely a website-problem or an actual game-problem.
That combined with Willard Phule's point of us not knowing ANYTHING about Elo and its calculation makes this whole discussion utterly pointless.

[warner2]

Given the only place that displays these statistics is the website, thank you Captain Obvious. The OP has reported what appears to be a bug. Is this not a valid thing to do? Is it not reasonable to expect the website (game) to display these basic statistics correctly? Of course we don't know whether it is a display bug or a database bug. How could we, and how could the OP? He's just reporting a ******* bug!

Anyway, I will leave you to your internet argument, and hope that PGI ignores the nose in this thread and addresses the bug.

[David Sumner]

@Willard, yes "Team Average Elo". "Team Average" does not Equal "All Players Average"

So if Team A starts* with 8 people and an average of 1000, and Team B starts with 12 people and an average of 1200, then the last 4 people added to Team A would be expected to have an average Elo of 1400.

If the team sizes stay the same, and the starting averages are reversed, then the last 4 would be expected to have an average Elo of ... 800.

* -Start being the point at which the MM tries to "balance" the Elo

So if you are in the starting group, it's random(ish) if you carry (I say ish because I'd expected some range limits by default [yes that's supposition on my part]).

If you are in the last group, then you will either carry or be carried.
Human ego and perceptual bias being what it is, you are much more likely to notice if you are carrying.

Warner2 is right. I'm reporting a bug. That bug is that the stats we can see are wrong, and that a bias is evident in how they are wrong.

That data provides places for PGI to look.

Logical inference indicates likely correlations between the bug and other important functions.
That correlation is dependent on implementation details, but to avoid those correlations, you would need to take non-intuitive steps.

This again provides places for PGI to look.

I have provided hard numbers for my data which PGI can verify I would think. I have not seen any numbers from you.

The POTENTIAL inferences (notice I used the word IF up the top there?) I have reported have no requirement on Elo or MM beyond:

MM - tries to balance team based on their Elo (which you yourself have stated is accurate)
Elo - attempts to update your skill level based on match outcome. (If this is not the case, we might as well have random matches)

This is because the bug would induce a systematic bias in results.