Wraith05, on 24 January 2013 - 01:04 PM, said:
If you were testing for a variable you'd be right. But we aren't testing a variable to answer why something happens we are just testing to see what the average pug win loss ratio is with the current matchmaker. To do that all we need is data from random people solo queuing. And lots of it.
Using control groups would be if you were trying to answer why something is happening. Such as why do pugs have a win loss ratio below one? Then you'd set up a control group of pure pug on pug and test the variables such as trial mechs mc groups in queue etc to find which is actually causing the loss.
Here's the thing, though, his question was:
abloobloo, on 23 January 2013 - 07:46 PM, said:
You often see pugs on these forums cry foul of premades. They claim that premades ruin the game, and that pugs are constantly being wiped off the map, round after round. Eventually, after hearing the anti-premade mantra over and over again, I thought to myself "is this really the case?"
He then went on to say:
Quote
I went into this with the hypothesis that the win/loss ratio of pugs would cluster at around 1:1 indicating that they are, on average, giving as good as they get.
I can largely agree that this is the case, I've mentioned that when MM decides to endow one side with a better team or simply better players than the other, there is only a one in eight chance less that you would benefit rather than lose out from that. However, his sample was:
A- pitifully small, not really his fault, but someone else did some work recently and concluded there were 6,000 players on in one given day. That number may be high, but keep in mind that to make 31 even one percent of the active playerbase there would have to be only 3,100 active players. When you take a bite out of a sandwich, some bites have more sauce than other bites, but bigger bites are more likely to have an average amount.
B- selective, again not really his fault. Well, ok, maybe kinda his fault, but it would have been a pain in the *** to get a non-selective sample. For instance, if you're trying to figure out the average income per household in your area, short of getting you hands on tax information, you need to go set up shop to gather info in a high traffic area. But you have to be careful where you set up shop - if you take information in a pricey shopping mall, you'll get far different statistic than if you set up in front of a low-end grocery store. I can't say definitively that the where his sample is from makes it faulted because I don't have statistics on it, either; but it does keep me from taking it on faith.
Because of the above two, its much more reliable to do an actual scientific test with a control. For all any of us know, the 31 people in the test could be better than the average player, and their W:L and K:D might skyrocket without the influence of premade teams, or they might be horrible and are actually just being carried by premade teams on their side; but we can't actually know unless we actually tested it.
Sidebar - the W:L K:D of your average player is not really the issue that concerns me about this game. The success of the game depends on it being fun, which is not derived from having at least a 1:1 W:L, or everyone who has less than that would leave, and then the players who's W:L dropped would leave, et cetera, until one dude was left. It may seem strange, but I have more fun losing a close match than winning in a landslide.