262 replies to this topic

[LordLeto]

Thirdstar, on 23 January 2013 - 08:56 PM, said:

OP says he understand that correlation =/= causation, then proceeds to say there IS a relationship. And you're telling ME that I don't have reading comprehension?

Saying something is related doesn't mean something causes the other(or vice versa) rather that there is a statistically significant correlations between the two. The best example for this that probably comes up in every intro stats class is the "relationship" between Ice Cream sales and Crime. Now no one in their right mind thinks that ice cream causes crime, but the relationship exists. The causal relationship for both those items is generally presumed to be the weather, not each other.

Again, please stop misconstruing the OPs words.

[Thirdstar]

Buzzkillin, on 23 January 2013 - 09:07 PM, said:

Should take account of time into the chart and what does people do with the MC. When I started, I only had a commando and had a 0.20 kdr. When I finally got the cbills to get a Dragon and outfitted the way I wanted, my kdr went up, I am at a 1.20 now and continues to go. I only bought some MC to paint my mechs.So lets assume a long term player will put down money for MC if they like the game enough and a short term player puts down money to buy mechs right away. and compare their kdr and their progress over time.

To get a good picture you'd have to actually ask questions. You have to find out the motivation behind buying the MC, the numbers by themselves mean nothing without actual human input. You can make an assumption but without surveys the assumptions will remain just that.

[Sug]

I appreciate your effort. It seems like there are just too many variables to get any meaningful stats out of the data. I'd love to see a dev chime in with some of the data they've collected as they've stated they keep track of, literally, almost everything.

[Thirdstar]

LordLeto, on 23 January 2013 - 09:10 PM, said:

Saying something is related doesn't mean something causes the other(or vice versa) rather that there is a statistically significant correlations between the two. The best example for this that probably comes up in every intro stats class is the "relationship" between Ice Cream sales and Crime. Now no one in their right mind thinks that ice cream causes crime, but the relationship exists. The causal relationship for both those items is generally presumed to be the weather, not each other.

Again, please stop misconstruing the OPs words.

Then why make a chart? Why write a paragraph about something that has no causal relationship. Unless of course the OP think thinks there IS one.

"Furthermore, if you are a pug and are looking to improve your win/loss ratio, there is a relationship between having MC in your account and your kill/death ratio."

So what exactly are you defending?

Edited by Thirdstar, 23 January 2013 - 09:25 PM.

[Murku]

Murku, on 23 January 2013 - 09:04 PM, said:

We've seen KDRs posted at 10 or more. We've heard about the possible exploits to pad ratings. And yes, we've heard a lot of complaints about PUGstomping. Are all these things pure fiction, or are they all absolutely true? At the end of the day we could use a little more transparency in team makeups, together with a better matchmaking system. And apparently this is on it's way.

So why exactly is a particular point being made here? Are things fine as they are, and matchmaking a bad idea?

OP?

[Bren McGuire]

abloobloo, on 23 January 2013 - 07:46 PM, said:

You often see pugs on these forums cry foul of premades. They claim that premades ruin the game, and that pugs are constantly being wiped off the map, round after round. Eventually, after hearing the anti-premade mantra over and over again, I thought to myself "is this really the case?" Surely, if it were true that pugs were systematically losing matches, that would suggest a deeply flawed matchmaker that was systematically pitting pugs against premades more than against other pugs.

So I started a thread on this forum asking anybody who has never dropped as a premade to post a screenshot of their stats. Lots of people responded by posting their stats, and in total, I had gathered 31 data points (sorry, if you typed your stats, they weren't included, l2 read instructions ).

Cool.

Hypothesis
I went into this with the hypothesis that the win/loss ratio of pugs would cluster at around 1:1 indicating that they are, on average, giving as good as they get.

Findings




In this chart we can clearly see the distribution of both kill/death ratio and win/loss ratio. As I predicted, the win/loss ratio appears distributed around the 1 line, indicating a near 1:1 win/loss ratio for pugs. The average win/loss ratio is 0.92:1 with a standard deviation of 0.31. This is very close to 1 and represents a strong rebuke to pugs who claim they are constantly being steamrolled due to the fact that they are pugging. Some pugs are losing more than they are winning, some are losing and winning about the same number of matches, and some pugs are winning more than losing. This data suggests that a total steamroll of all pugs always, is unlikely.

On the other hand, win loss ratios are comparatively high at 1.49:1 with a standard deviation of 1.11. My guess is that kill/death ratios are higher because this stat is more dependent on personal skill, and mech loadout. If you're a good player with a good mech, on a bad team, you might well take out 2 or more enemy mechs before losing.

With my hypothesis proven correct, I started to look at other things I could do with the data so I tried to correlate some of the stats.



What I found was that the only thing that really correlated strongly were kill/death ratio and win/loss ratio. However, I was disappointed to see no strong relationship between player performance and the amount of MC in their accounts.



So, I decided to split up the data into one group of participants who had MC in their accounts and one group that didn't have MC and compare their average performance. In this chart, we can see that people with MC in their accounts experienced a 51% improvement in kill/death ratio over non-paying pugs. However, in terms of win/loss ratio, the numbers did show a slight advantage for paying pugs, but after running a t test, the p value exceeded 0.05, so take that result with a grain of salt.

Conclusion
Based on the sample, the pug player experience is not uniformly bad; as a group, they are very close to a 1:1 win/loss ratio and a 1.49:1 kill/death ratio. While personal experience for pugs varies by individual, it is not accurate to state that all pugs are experiencing catastrophic losses. If you are constantly being steamrolled as a pug, you should be looking towards personal improvement, rather than looking towards the developers to elevate pugs as a group, because as a group, they're already very close to a 1:1 win/loss ratio.

Furthermore, if you are a pug and are looking to improve your win/loss ratio, there is a relationship between having MC in your account and your kill/death ratio, there is also a relationship between between kill/death ratio and win/loss ratio. And before anyone says it, yes, I know correlation is not causation. I make no claims of the existence of causal links here.

Raw data

If anyone more experienced with statistics than I am would like to poke around the raw data, here's the excel file.

https://www.dropbox....fw8e/TXdkfNMzOP


If the devs would like to weigh in with the actual pug performance stats, I would be really interested to see how well it lines up with my small sample.

Your sample group is flawed, if you queued solo for thirty games and played like a newbie with a trial mech you'd know so.

[MajorBorris]

I dont understand how so many blame the other team with "unfair" teamwork when its almost always the lack of teamwork that gets you a loss in a pug match.

Edited by MajorBorris, 23 January 2013 - 09:24 PM.

[Brilig]

Thanks for the effort Abloobloo. It would be nice to see a higher sample. I think PGI has said before that things were getting closer to their desired 1:1 win loss ratio. We could definitely use a comment, and a statistic from a Dev on this.

Actually I would really like to see the number of Pre-Made on PUG games vs PUG on PUG games.

[LordLeto]

Thirdstar, on 23 January 2013 - 09:14 PM, said:

Then why make a chart? Why write a paragraph about something that has no causal relationship. Unless of course the OP think thinks there IS one. So what exactly are you defending?

Hes a statistics nerd? Hes got a hard on for graphs? I honestly don't know.

What I'm defending is the OPs right to not be misquoted or misconstrued. Which you knowingly or unknowingly have done.

[abloobloo]

Murku, on 23 January 2013 - 09:04 PM, said:

So why exactly is a particular point being made here? Are things fine as they are, and matchmaking a bad idea?

The point I am trying to make is that an average win/loss of 1:1 is probably the best that pugs, as a group, should hope for (individuals can expect win/loss ratios above or below). Right now, the--admittedly small--sample seems to indicate that there is a trend towards a 1:1 win/loss ratio. It's not quite 1:1, but it's close. So there is still a little bit of room for PGI to improve matchmaking, but I think that they are closing in on the highest win/loss ratio possible for pugs short of giving them an artificial advantage.

I'd also like to add that skill based matchmaking, in my uneducated opinion, is far more convoluted than simply ensuring that both teams drop with an equal number of premades.

Edited by abloobloo, 23 January 2013 - 09:44 PM.

[Thirdstar]

LordLeto, on 23 January 2013 - 09:25 PM, said:

Hes a statistics nerd? Hes got a hard on for graphs? I honestly don't know.

What I'm defending is the OPs right to not be misquoted or misconstrued. Which you knowingly or unknowingly have done.

What? So because he said that he understands causation =/= correlation and then proceeds to prove that he does not, that I should shut up?

Hey man I know 1+1 isn't 4 but clearly 1+1 is equal to 4, but you don't get to call me out because clearly I've already demonstrated I don't know what I'm talking about.

Whelp, okay then.

[LordLeto]

Thirdstar, on 23 January 2013 - 09:29 PM, said:

What? So because he said that he understands causation =/= correlation and then proceeds to prove that he does not, that I should shut up?

Hey man I know 1+1 isn't 4 but clearly 1+1 is equal to 4, but you don't get to call me out because clearly I've already demonstrated I don't know what I'm talking about.

Whelp, okay then.

Saying there is a relationship doesn't imply that there us a casual relationship. Period.

Edited by LordLeto, 23 January 2013 - 09:32 PM.

[Thirdstar]

LordLeto, on 23 January 2013 - 09:32 PM, said:

Saying there is a relationship doesn't imply that there us a casual relationship. Period.

Should have said initially that you're just in it to argue semantics. I'm out, you can't debate with someone who narrowly argues on semantics while ignoring context and nuance.

[abloobloo]

Thirdstar, on 23 January 2013 - 09:29 PM, said:

What? So because he said that he understands causation =/= correlation and then proceeds to prove that he does not, that I should shut up?

Hey man I know 1+1 isn't 4 but clearly 1+1 is equal to 4, but you don't get to call me out because clearly I've already demonstrated I don't know what I'm talking about.

Whelp, okay then.

Why don't you go read the excel file in the OP? It has all the numbers there. The extent of my claim was that group A experienced a higher average kill/death ratio than group b. Why? Nobody knows. Maybe you should do a study and find out?

[Thirdstar]

abloobloo, on 23 January 2013 - 09:35 PM, said:

Why don't you go read the excel file in the OP? It has all the numbers there. The extent of my claim was that group A experienced a higher average kill/death ratio than group b. Why? Nobody knows. Maybe you should do a study and find out?

Why don't you reply to Murku's post upthread. Here, i'll helpfully repost it.

"We've seen KDRs posted at 10 or more. We've heard about the possible exploits to pad ratings. And yes, we've heard a lot of complaints about PUGstomping. Are all these things pure fiction, or are they all absolutely true? At the end of the day we could use a little more transparency in team makeups, together with a better matchmaking system. And apparently this is on it's way.

So why exactly is a particular point being made here? Are things fine as they are, and matchmaking a bad idea?"

[LordLeto]

Thirdstar, on 23 January 2013 - 09:34 PM, said:

Should have said initially that you're just in it to argue semantics. I'm out, you can't debate with someone who narrowly argues on semantics while ignoring context and nuance.

The distinction between a relationship and a casual relationship isn't semantics. Its the crux of your argument that the OP doesn't understand correlation =/= causation. Or rather the missing crux as you've yet to show that in any way, shape, or form.

Edited by LordLeto, 23 January 2013 - 09:42 PM.

[Thirdstar]

LordLeto, on 23 January 2013 - 09:40 PM, said:

That you fail to understand that the 2nd paragraph of the conclusion was more than likely a joke(or do you really think the OP wants to imply there's a casual relationship between MC and win rate?) lends credence to my first question.

The distinction between a relationship and a casual relationship also isn't semantics. Its the crux of your argument that the OP doesn't understand correlation =/= causation. Or rather the missing crux as you've yet to show that in any way, shape, or form.

Oh it's a joke now. Ah, right.

Friends with the OP are we? Because the post implies no joke. So surely you have insight into the mind of the OP.

Edited by Thirdstar, 23 January 2013 - 09:42 PM.

[abloobloo]

Thirdstar, on 23 January 2013 - 09:40 PM, said:

Why don't you reply to Murku's post upthread. Here, i'll helpfully repost it.

"We've seen KDRs posted at 10 or more. We've heard about the possible exploits to pad ratings. And yes, we've heard a lot of complaints about PUGstomping. Are all these things pure fiction, or are they all absolutely true? At the end of the day we could use a little more transparency in team makeups, together with a better matchmaking system. And apparently this is on it's way.

So why exactly is a particular point being made here? Are things fine as they are, and matchmaking a bad idea?"

I did reply to it. Look up.

[LordLeto]

Thirdstar, on 23 January 2013 - 09:41 PM, said:

Oh it's a joke now. Ah, right.

Friends with the OP are we? Because the post implies no joke. So surely you have insight into the mind of the OP.

As much insight as yourself has. And some how I came to wildly different conclusions, both about the intent of that paragraph and the distinction between relationships and casual relationships a cursory knowledge of statistics bestows.

[Snowhawk]

Well... I think there is a little problem in your statistic because:

abloobloo, on 23 January 2013 - 07:46 PM, said:

So I started a thread on this forum asking anybody who has never dropped as a premade to post a screenshot of their stats.

You asked the people from this forum..... And that makes a huge difference. Please keep in mind that gamers who are often reading in a forum are "sophisticated players". They evolve much faster then the real casual gamers who not even know the damage values of the weapons.