Rifter, on 23 January 2013 - 08:03 PM, said:
Sorry to break this to you but you are going to need 2000+ samples to get reliable stats.
not exactly.
presidential polls are often done on sample sizes as low as 500 and that's considered scientifically accurate +- 3-5% for a universe of more than 130 million likely voters.
the size isn't so much important as making sure all demographics are accurately and proportionally represented in that sample. if he's only getting 2000 people who voluntarily submit their scores from this forum only, without any kind of controls for other factors of those gamers... it doesn't matter if he gets 30 or 3,000, its still not going to be REALLY trustworthy in terms of results.
I think there are a lot of different factors you would have to take into account in order to figure this out really, and it might just be too difficult to do.. i mean just based on the top of my head you would have to control and make sure you're getting the right distribution of stats from people in different time zones represented, different ping levels, what mech class they piloted during that time, what types of configs and play styles are represented.
then I think you have to question whether or not it's a good idea to poll only puggers. even though you're getting stats from people who ONLY pug (if you believe the players) you don't have any stats about who they were playing against. it's a factor that we really just don't know