17 replies to this topic

[CFC Conky]

Hello all,

So I just watched an epic epeen measuring contest on YT where two experienced players discuss the merits of the various metrics in the Leaderboard for determining skill. On player favored overall wins in a given season, the other W/L and K/D ratios.

Since both players play a different number of games in a given season, how significant are the metrics?

Most, if not all sports use a fixed number of games to measure a season. I wonder what the Leaderboard would look like if only a certain number of games were counted. My stats don't change much after 100 games so would that be a reasonable number? 200? 50?

Good hunting,
CFC Conky

P.S. My apologies if this has been discussed to death before.

Edited by CFC Conky, 03 March 2021 - 04:39 PM.

[East Indy]

Wholeheartedly agree -- the number of accounts with <20 games parked for the month with high stats is surprising. I doubt it's even intentional, but it is distracting.

Jarl's uses weighting but Scurro is in a league of his own.

Even 50 games, in order to appear without toggling a filter, would be fine.

[Heavy Money]

I support KMDD being more important than KBs. They should at least show KMDD too.

[PocketYoda]

Does anyone know why when i check myself on the leader boards i get this?

YOU ARE NOT RANKED ON THIS BOARD

CURRENT RANK: N/A in Lights, Heavies and Assaults..

Mediums i'm ranked 299 and my overall rank is 1436

1436 Samial 20 23 0.87 28 28 1.00 43 212

which i assume is suck.. But i play every day on all mechs mostly Heavies and Assaults? which i do far better in, WTF..

Edit I'm not sure how it says i only have 28 kills some matches i get 5+ kills others i get 1 or 2 some i get none but lately i've gotten way more than 28 kills. Season 56 And i've played more than 43 matches.

I think this is busted..

Edited by Samial, 03 March 2021 - 10:18 PM.

[Aidan Crenshaw]

Samial, the current season started on the 1st of March. You have to make at least 10 matches to show up on the general/light/medium/heavy/assault leaderboard. To see the former season stats, you'll have to change the filter on the top right. Or you go directly to the jarl's list: https://leaderboard....search?u=Samial

Edited by Aidan Crenshaw, 03 March 2021 - 10:24 PM.

[Elizander]

Why not just base 50% of it on win/loss and 50% of it on kill/death? If you're killing the enemy early enough and fast enough, then you should end up winning more. If you wait until the last minute after your team softens them up for you, you can get lots of kills and still lose because you're just shooting banged up mechs before you go down.

As for ranking, I believe 20 games is too few and 100 is a bit too high. Maybe just add a filter to the list by games played since people might have different ideas on how many games makes it valid.

[PocketYoda]

Aidan Crenshaw, on 03 March 2021 - 10:24 PM, said:

Samial, the current season started on the 1st of March. You have to make at least 10 matches to show up on the general/light/medium/heavy/assault leaderboard. To see the former season stats, you'll have to change the filter on the top right. Or you go directly to the jarl's list: https://leaderboard....search?u=Samial

I play every night/day with many many matches a day.. I'd say a lot more than 43 games, even in 5 days.

Edited by Samial, 04 March 2021 - 08:19 AM.

[Aidan Crenshaw]

the leader board lists 50 matches for your account. Can't tell you any more.

[CFC Conky]

Elizander, on 04 March 2021 - 02:38 AM, said:

Why not just base 50% of it on win/loss and 50% of it on kill/death? If you're killing the enemy early enough and fast enough, then you should end up winning more. If you wait until the last minute after your team softens them up for you, you can get lots of kills and still lose because you're just shooting banged up mechs before you go down.

As for ranking, I believe 20 games is too few and 100 is a bit too high. Maybe just add a filter to the list by games played since people might have different ideas on how many games makes it valid.

I'm not a statistician so I don't really know how many matches constitute a useful sample size. All I know is that in my case the movement in stats becomes very small after 100 matches. Right now I'm averaging ~280MS per season but that could be skewed because I'm mostly running fully skilled BAS-A/B and MC-IIB dakka rides these days.

Good hunting,
CFC Conky

Edited by CFC Conky, 04 March 2021 - 11:32 AM.

[Theodore]

To be honest, there are very big glaring problems with the Leaderboards.

First off, let me stat that currently I am a T3 player. (I just recently found out what Tiers mean in this game). This is very important to take note of, and will be important for what I am about to say.

I honestly have a very small set of games played and am not experienced... at all... I know how good I am, and I honestly consider myself a noob.

I am going to post my stats here (and trust me they are bad... and I really should not call attention to them. but its important for the subject matter at hand).

Jarl's List (something I just found out about the other night):
The Jarl's List - Player Search Results For: Theodore (isengrim.org)

and mwomercs.com leaderboards: (also I just found out about).
MWO: Profile (mwomercs.com)
for the mwo leaderboards, I posted a link to Season 55 K D Ratio.

ok so now, I'll give a breakdown of what these list (so people do not actually have to click the links).
Jarls list:
overall percentile rank 21845, percentile 37%, games played 216, with a W/L ratio of 1.72, and a K/D ratio of 1.67
Season 55 (Feb 21) : Rank 8875. percentile 58%, games played 175, with a W/L ratio of 1.95, and a K/D ratio of 1.78

MWO Leaderboards:
Rank 1670, W/L ratio of 1.96 and K/D ratio of 1.78

first off, we can see that I am not a skilled player.
Lets look at the Jarl's List. My season 55 percentile is 58% but my overall percentile is 37%.
My overall percentile shows me as I was as a player who has not joined a unit. And then my season 55 shows a dramatic jump in stats. That is because I joined a unit this month and they immediately raised my stats.

Obviously, you can infer that I was carried by my team mates. Yes this is true for me. I was. I am definitely not as good as my season 55 stats indicate. (my overall stats are definitely indicative of my true performance).

But also, its important to note that those stats are also played within the Tier 3 bracket. I was able to obtain this stat line because I was playing against players of equal skill level.

I am definitely, without a doubt, not at the skill level appropriate of a Tier 2 or Tier 1 player. Heck, I'm probably more along the lines of a Tier 4 or Tier 5 player, and was carried to Tier 3 by my new unit mates.

Now that I have moved up in tiers and started noticing the huge gaps in skill level between tier 5 and tier 2+ players... I can honestly say that... this skill gap is immense.

So, using the MWO leaderboards... how can I be ranked 1670? I am sure there a multitude of Tier 3, 2 and Tier 1 players way better then I am.

The leaderboards honestly, should be divided into Tiers. T1 players should have their own leaderboards. A Tier 1 player scored against other Tier 1 players should be the real bar to measure Leaderboard stats.

And of course for progress tracking, T2 should be scored against T2. T3 versus T3, T4 versus T4 and of course T5 versus T5. Progress tracking is very important for players who want to gauge their improvement over time so definitely keep these statistics in place.

(now briefly perusing the forums, I have randomly read things about Tiers being an EXP bar or something like that. I do not know exactly how that works, but regardless of how it is, no one can convince me other wise, that the average T1 player is not absolutely better then average T5 players. I do not care how long a player has grinded... T1 players definitely play at a completely different level then a T5.)

Anyway, that is at least my view on the subject matter. (of course I must state that my view is extremely limited to a short amount of play time).

[Vlad Ward]

Theodore, on 04 March 2021 - 01:15 PM, said:

[color=#959595]So, using the MWO leaderboards... how can I be ranked 1670? I am sure there a multitude of Tier 3, 2 and Tier 1 players way better then I am.[/color]

You have the default sort on, which is # of wins, so your ranking is inflated based on your number of games played that month. Switch the drop down to Average Match Score to see something closer to Jarl's. Jarl's uses average match score weighted by Mech class.

For what it's worth, Jarl's is just pulling its data directly from the MWO leaderboards, converting raw match score to adjusted match score based on Mech usage distribution, and ranking by adjusted match score.

Edited by Vlad Ward, 04 March 2021 - 03:33 PM.

[Theodore]

Vlad Ward, on 04 March 2021 - 03:33 PM, said:

You have the default sort on, which is # of wins, so your ranking is inflated based on your number of games played that month. Switch the drop down to Average Match Score to see something closer to Jarl's. Jarl's uses average match score weighted by Mech class.

For what it's worth, Jarl's is just pulling its data directly from the MWO leaderboards, converting raw match score to adjusted match score based on Mech usage distribution, and ranking by adjusted match score.

Sorry, I probably should have prefaced that I was discussing the topic creators post about W/L and K/D ratios on the leaderboards for determining skill.

Be that as it may, what ever score I got-

(I am Tier 3 currently) Do you think I would achieve roughly the same score, if I matched up against Tier 1 or Tier 2 players consistently?

I am of the opinion, that what ever my current score is, (match score, K/D, or W/L) it would be drastically lower, had I been in playing in the T1 bracket using my current skill level as a barometer. (granted I most likely would not be in T1, but hypothetically, lets pretend that I somehow managed to reach T1 for the sake of argument).

--

While, please correct me if I am wrong, since I am not actually T1. But is it safe to assume that T1 players are generally speaking, better players then those who are actually and consistently play within the T5 bracket? I know I am painting with a broad brush, and I'm highly generalizing, but that has been my arm-chair observation in my career.

--

As my closing statement, please understand that I am not trying to attack you in anyway. But just looking for your perspective of the situation, since you do actually play in the T1 bracket.

[Vlad Ward]

Yes, I agree with the points you've made. Average match score is going to be a bit inflated whenever you're playing in lower tiers. Practically, this effect will resolve itself over time. If you have a high average match score, you will gain PSR. If you start losing PSR to remain below T1-max, your average match score will go down. That's why I prefer using Avg Match Score as an indicator.

W/L and K/D can't account for tier and are thus more vulnerable to abuse.

[CFC Conky]

Another thing that comes to mind, how much does a fully skilled vs unskilled (skill tree) mech influence the objective measurement of 'pilot' skill?

Good hunting,
CFC Conky

[Vlad Ward]

CFC Conky, on 05 March 2021 - 12:35 PM, said:

Another thing that comes to mind, how much does a fully skilled vs unskilled (skill tree) mech influence the objective measurement of 'pilot' skill?

Good hunting,
CFC Conky

I think it's perfectly fair to include "Mechlab" as a component in pilot skill. In the absence of a stock-only game mode, all we can do is measure outcomes. Whether or not a player could have performed better if only their Mech were stronger is immeasurable and therefore immaterial.

[PocketYoda]

CFC Conky, on 05 March 2021 - 12:35 PM, said:

Another thing that comes to mind, how much does a fully skilled vs unskilled (skill tree) mech influence the objective measurement of 'pilot' skill?

Good hunting,
CFC Conky

A lot.. Especially survival, heat and consumables wise.

[Dozer6]

Vlad Ward, on 05 March 2021 - 12:43 PM, said:

I think it's perfectly fair to include "Mechlab" as a component in pilot skill. In the absence of a stock-only game mode, all we can do is measure outcomes. Whether or not a player could have performed better if only their Mech were stronger is immeasurable and therefore immaterial.

Just like every other game out there with weapon and vehicle customization and perks.

[Horseman]

CFC Conky, on 03 March 2021 - 12:31 PM, said:

So I just watched an epic epeen measuring contest on YT where two experienced players discuss the merits of the various metrics in the Leaderboard for determining skill. On player favored overall wins in a given season, the other W/L and K/D ratios.

Since both players play a different number of games in a given season, how significant are the metrics?

Anyone trying to use the raw number of wins as a metric has no clue what they're talking about - following that logic, if you play 400 games and win 80 (win rate 20%), you're supposed to be "better" than someone who played 100 games and won 75 (win rate 75%). Which is clearly complete ********. I've ran into people like that on these forums and on Outreach before, usually they just did not realize the leaderboard's default sort order is by raw wins (which increase just by, you know, playing and can easily come from your team rather than any contribution of your own) rather than any meaningful value.
WLR is THE most accurate statistic over a large sample size, as everything you do that statistically affects your team's chances of success will find its' way there.
Average match score is less accurate but converges faster, usually making it "good enough" for comparison purposes.
KDR is next to useless as a metric by itself, as it is extremely gameable - there's quite a few players who believe preserving their KDR is worth giving up the match, or who outright refuse to engage until literally everyone else on their side is dead. Seriously, **** those guys.

Quote

Most, if not all sports use a fixed number of games to measure a season. I wonder what the Leaderboard would look like if only a certain number of games were counted. My stats don't change much after 100 games so would that be a reasonable number? 200? 50?

Most, if not all, sports only have a fixed number of games in the season to begin with. In MWO, we have a fixed time period instead, with a variable number of matches per player. It makes more sense to normalize the values by number of matches and work from there.