140 replies to this topic

[Kyll Long]

Statistics in general is a pain in the rear quarter Having said that for endeavors regarding population spread you can actually get a statistically significant result from as few as 32 people regardless of population size. Your confidence level is good the only other wavering figure is that it was a voluntary survey online and on a message board so the respondants may not represent A. People with no internet access (a given) B. People with no interest in BT( also a given) and C. People who do not frequent MBs (the only statistically significant factor for our purposes). Theres probably a few other quirks but overall they don't apply unless you skewed the results by PMing friends of like mind specifically . Bottom line awesome job you could use it for a college paper

Mr. Smiles, on 29 November 2011 - 08:26 PM, said:

Statistics is a bit wonky and hard to understand sometimes. I'll try to explain it as best I understand it--someone with actual professional statistics knowledge, please correct me here.

Take a population of 28,000 people (the MechWarrior Online registered user count). You have a true-false question you want to ask them, but have no idea just how skewed the answers will be toward true or false. Worst case scenario, a 50% "response distribution" as it's called.

Now, there is a "true" answer out there. Suppose your yes:no question has, if you asked all 28,000 people, a ratio of let's say 60:40. You have to set a "margin of error", which means how close to the "true" answer you're willing to get: a typical margin of error is 5%. A margin of error of 5% in this example would mean that your ratio would be anywhere from 65:35 to 55:45.

Obviously, the amount of skew you have makes larger margins of error more acceptable. That is, if the true answer was 90:10, a margin of error of even 25% would still only make the answers range from 100:0 to 65:35. Still a massive majority, and if majority's all you care about, who cares if it's wrong in the specific numbers?

Then, you also have to set a "confidence level". Confidence level is how often you're willing to permit your answer to be beyond your margin of error. Suppose we had 20 questions, true answers are all 60:40, and we set the confidence level at 95%. That means that for 19 of our questions, we would get an answer between 65:35 to 55:45... and for one of those questions, we would get something even farther off, like 70:30 or 40:60.

* * * *

Now, statistics thankfully takes all of these values, and has a way to plug them into a formula to churn out a single number: how many people you have to question to get the values you input.

Of course, you're absolutely right that getting thousands of responses would be the absolute best thing. If I wanted to have only 1 in 100 questions be off my margin of error of 5%, I would only need a sample of 649 people. If I had 1,000, well, you can't get much better than 99% confidence level, so 1,000 people would get me a margin of error of 4%. Meaning, if I had a yes:no ratio of 55:45 on a question, I would be 99% sure that the yeses would still have the majority, since I'm 99% sure that the worst case scenario is the "true" answer is a 51:49 ratio.

Fortunately for me, I still have some pretty good statistics with only 300 people. That gets me a confidence level of 95%, and a margin of error of 5.63%. That's something I can live with...

...except with the really, really narrow questions. Anything with a difference of less than 5.63% between the answers I'm not sure about, and I'm only 95% confident about the rest of them.

...and that assumes that there's absolutely no skew with my answers. But look at them. Most of them are in the 60%'s, some are at 99.9%. Therefore, most of my questions, I could've gotten away with far less respondents when writing about my answers. But, I don't know enough about statistics to change my "response distribution", so I'll leave it the worst case scenario, 50%.

NOTE: Everything I just wrote is based on my best knowledge based on Google and Wikipedia and what makes sense to me. I'm an English person first, a Calculus person second, and a Statistics person like... 74th.

[Cyote13]

Great run down. I think it really shows that we all want to have an active role in the universe not simply match after match for the sake of stats.

Like the man said the numbers are actually a good sample, pay attention to the total sample size of "national" political polls sometimes, they can also be around 500 for a representation of 300 million Americans. Yes, this one was voluntary and you had to go do it on your own, but it shows in the data that we really want this game to be great, not simply another shooter in a mech skin.

[John Clavell]

Mr. Smiles, your a legend thanks for all your hard work planning, collecting, and analysing this data.

[Haeso]

Mr. Smiles, on 29 November 2011 - 06:09 PM, said:

I was thinking of something similar, originally I had two categories: fake products and real products. But I didn't want it to get confusing, so I condensed them. Still.

Basically, think of Nuka Cola from Fallout. It's a cola drink with a red label in a very iconic-looking bottle with white letters and a white swoosh underneath it. It's exactly the same product as Coca Cola, except it's slightly more radioactive/addictive, and it says "Nuka", not "Coca".

What if you went into a Solaris VII map and were advertised things like "Budsteiner" or "Capella Cola" or "The Terran Texts: Spacerim" or some silly stuff like that. That's off the top of my head, imagine those names a bit more polished Anyway. By not directly naming the advertised product, they keep like 80% of the immersion in the game--but you know what? After playing Fallout, I usually want a Coca-Cola. That's good advertising, though I don't know if <current maker of Fallout of the year> gets any money from Coca-Cola or not.

I don't drink beer (mm, "Captain Morgan Kell's Spiced Rum" goes great with "Capella Cola" ) but if I did, I bet my desire for a Budweiser would up after a few Budsteiner ads on the side of a building while fighting over <random planet name>.

That's good advertising. And you know what? If PGI can get those advertising contracts, where they post fake products on the side of buildings and get payed for it? I encourage it. Good for them. Increase your revenue! Go make a tidy profit without charging your customers a dime!

...but when decals come out to show your 'Mech's "sponsor" in Solaris VII matches? Send my account a free Captain Morgan Kell's Spiced Rum decal as payment for that idea



Hehee

As I was alluding to in my post: It must be done with an in universe feel to be acceptable. That's all.


By the way regarding survey size/sample, I question if the battletech/oldschool mech fans are significantly more likely to have found this forum sooner than the masses, I think that's the case. I regularly look for news on Mechwarrior/battletech, many people only vaguely know what it is. I think the appeal of this game will extend, even if they don't cater to the drooling masses, to a large group of people outside of our fandom.

Edited by Haeso, 30 November 2011 - 01:54 AM.

[Thorqemada]

Great Survey - you sum up the expectations i share much.

For the RPG part - i hope for a strong World feeling in the sense of being in a parallel universe and not only in a combat simulator.
Character progression is a sidekick and i feel it best the way the survey already states.

[Thorqemada]

Oh...and forgive me but i think that translation into foreign languages isnt the most necessary thing.
In Europe, especially middle and northern Europe, everone has good english skills and the money is better spend into other parts of the game.

[Xhaleon]

Haeso, on 30 November 2011 - 01:50 AM, said:

As I was alluding to in my post: It must be done with an in universe feel to be acceptable. That's all.

By the way regarding survey size/sample, I question if the battletech/oldschool mech fans are significantly more likely to have found this forum sooner than the masses, I think that's the case. I regularly look for news on Mechwarrior/battletech, many people only vaguely know what it is. I think the appeal of this game will extend, even if they don't cater to the drooling masses, to a large group of people outside of our fandom.

There are a number of very vocal people who prefer the game to diverge greatly from the tabletop and go in its own direction. It only makes me wonder why those tabletoppers remain so silent in general on the forums, if the results of the survey is to be believed.

[Haeso]

Xhaleon, on 30 November 2011 - 03:22 AM, said:

There are a number of very vocal people who prefer the game to diverge greatly from the tabletop and go in its own direction. It only makes me wonder why those tabletoppers remain so silent in general on the forums, if the results of the survey is to be believed.

I'm personally part of the old guard myself, I still have all my TROs and such. I can't speak for others, but if I had to wager a guess, the generation more likely to enjoy BT/Simulation over action games is older, less likely to bother arguing on a forum for example. Note: I'm in my 20s, the only reason I got into BT at a very young age is my father, I inherited and cherish most of my BT books/figures etc.

I also believe the survey is an accurate depiction of our current community, definitely. I'm only wondering if that community changes proportionally as the game gets more publicity.

Edited by Haeso, 30 November 2011 - 03:34 AM.

[FiveDigits]

Thorqemada, on 30 November 2011 - 02:18 AM, said:

Oh...and forgive me but i think that translation into foreign languages isnt the most necessary thing.
In Europe, especially middle and northern Europe, everone has good english skills and the money is better spend into other parts of the game.

I second that. In fact, I am German myself and do NOT want a German localisation. I even went so far as to pick English as my native language when taking the survey (though it's German). I haven't seen a single localisation done right ... ever. And by "right" I mean leaving all IP-related terms (proper names, technology terms etc.) in their original form and just translating the remaining text. I much prefer to play the original (English) version. Sadly it's hard to avoid localisation and movie dubbing here in Germany. You have to go to extra lengths obtaining (importing) an original version. Sometimes it's even outright impossible as with Steam's geoloc(k)ation for example.

[Mr Smiles]

Fair enough I wonder if a future survey should ask "Which languages would you prefer this game be localized into?" or something.

[Mr Smiles]

Here's an updated PDF: http://plotscaper.fi...mary-report.pdf

I'm still slightly disappointed with the PDF print, since what I see when I look at the website is color-coded backgrounds for the various options based on how strong the pick was--that is, if 90% of people pick option A and 10% of people pick option B, there's an orange background with 90% opacity for option B and 10% opacity for option A. Very useful for just glancing at a list of choices (especially the big grids!) and saying "Okay. Aesthetic items have the darkest coloring for payed-for items, therefore, people wanted them more."

At least this PDF is a bit cleaner though, and not an awkward "Print to doPDF 7"

Edited by Mr. Smiles, 30 November 2011 - 07:20 AM.

[Dihm]

Very interesting results. Thanks for taking all this time.

I'm surprised that I agree with the majority on the majority of things, though I still think they're wrong on some stuff (stupid majority!).

[Caballo]

Great, so you've made a survey on 300 people, and you have the truth... Not gonna say what i'm thinking about your survey, because i got better manners than you.

Come again when you got 2000 people, and then we'll talk

[Adridos]

Perfect results. You really made a perfect and informative survey and the biography part was so good.

It even helped me to make me even more proud of being Liao - not the average Joe.

Is the survey going to be available after this to get more precise results? Also, will you make another, just Clan survey? Or a survey translated to German and Russian?

I am so excited to see the develloper response to your survey, really good work.



To Caballo: People asked for this, it was not his decision to do it so early.

Edited by Adridos, 30 November 2011 - 07:46 AM.

[Mr Smiles]

Caballo, on 30 November 2011 - 07:44 AM, said:

Great, so you've made a survey on 300 people, and you have the truth... Not gonna say what i'm thinking about your survey, because i got better manners than you. Come again when you got 2000 people, and then we'll talk

If you're going to have this terrible attitude and be this rude, I'd prefer it if we never talked Also, as we've been discussing, you don't need 2,000 people to get accurate survey results--you don't even need 1,000--so, you can take your unreasonable expectations and go make your own survey with them

Adridos, on 30 November 2011 - 07:45 AM, said:

Perfect results. You really made a perfect and informative survey and the biography part was so good. It even helped me to make me even more proud of being Liao - not the average Joe. Is the survey going to be available after this to get more precise results? Also, will you make another, just Clan survey? Or a survey translated to German and Russian? I am so excited to see the develloper response to your survey, really good work. To Caballo: People asked for this, it was not his decision to do it so early.

I see the website is no longer including line spaces when I use a multiquote *eyeroll* Oh well.

I might find someone to do a Russian translation, I've found that most Russians I meet online are the kind of people who would be happy to help with that sort of thing (though maybe I've been lucky, who knows?). Germans, apparently, take offense at the implication they don't understand English perfectly well ( ). As a language-focused person, that idea gives me a moment of hesitation, because even the most fluent people don't usually pick up on 100% of the subtext. But I'll give them the benefit of the doubt, and save myself time, by not bothering with a German translation

The survey is still up indefinitely, as are the forum threads on various forums. If people want to continue taking it, by all means I'll look back at it at 400, 500, or whatever results and see if anything's changed. For reasons discussed above, however, I seriously doubt any of the strong statistics will change, and while the weaker statistics might waver, they're probably not going to do a 360-degree 180-degree turnaround.

I hope I get a good dev response too I sent an e-mail to the general Piranha Games address, no reply yet, but who knows, right?

I might do another, more refined survey at some point, focusing on areas that this survey left blank--note to future me reading this thread, that includes Clan preferences, localization options, race and gender options for any pilot/avatar images, and specifics on weapon spread, maybe achievement-related questions--but if I did it pre-launch it would have to be very small, because I don't know if people would have patience for another huge survey. Post-launch, however, we're going to have so much information--and so many arguments--that it'll be a ripe environment for another survey

Personally though, I'm worn out from thinking about and typing about numbers for a while. Like, seriously, when I was explaining my best knowledge about how sample sizes work before, I had the need to pull out a spreadsheet to quickly punch some small numbers and I was like "UGH I DO NOT WANT TO!" and just glazed over them instead

That, and school. Yay, now it's finals season! I have to write a seminar paper! And then it's working all winter break, then graduating! ...yeah, I might not have time to create another focused survey for a while

Edited by Mr. Smiles, 30 November 2011 - 08:06 AM.

[Mr Smiles]

I just wrote 1/12th of a novel in a single post, and it went through just fine.

But there's an emoticon limit? 10 emoticons max? 8,000 words, fine, 11 emoticons, HELL NO! Talking about the genocide and constant warfare and racism of the BattleTech universe? Fine. Clinical terms for things that only make Puritans blush? CENSOR IT!

...this forum makes me laugh.

[Adridos]

Thanks for response.
About the translations, it is just about people, for instance all people can be trolls, stupids, kind ones, etc.

If you ask Slavics (I am one myself), they will tell how kind we are, yet half of them can think about you as someone you really arent. Same goes for the other part of the world, just take those jokes aimed at Russians based on Cold war facts, which were just clear propaganda of the side you lived in.

I would be happy to make a russian translation for you, but I am only learning their language now and I know English a lot better, but I hope, you will find someone to do it for you.

Edited by Adridos, 30 November 2011 - 08:16 AM.

[BarHaid]

I'd like to retract my dislike of in-game ads. I now want to destroy billboards! Perhaps get an Achievement for totally clearing a map of all ads in the middle of battle?

[Kudzu]

Xhaleon, on 30 November 2011 - 03:22 AM, said:

There are a number of very vocal people who prefer the game to diverge greatly from the tabletop and go in its own direction. It only makes me wonder why those tabletoppers remain so silent in general on the forums, if the results of the survey is to be believed.

Arguing with mouthbreathers who think that mechassault or mw4 are the greatest implementations of the BTU gets tiring after awhile-- they often have no idea how the TT works at and thus have no basis for comparison. I don't blame this silent majority whatsoever.

[MagnusEffect]

Mr. Smiles, on 29 November 2011 - 06:09 PM, said:

I don't drink beer (mm, "Captain Morgan Kell's Spiced Rum" goes great with "Capella Cola" ) but if I did, I bet my desire for a Budweiser would up after a few Budsteiner ads on the side of a building while fighting over <random planet name>.

At the risk of getting too off topic, no amount of ads for Budweiser will ever make me want one... it is among the crappiest of crap beers... watered down **** and not even cheap. It can't even claim to be owned by Americans anymore (was sold to a Belgian conglomorate... one would hope that improved the quality, but sadly, no). If i'm going to drink ****, I reach for a Steel Reserve... at least it gets you ****** up without having to drink a 12-pack and is easily found almost anywhere
http://blog.winemag....-lager-no-more/


Ok... back to on topic.

AWESOME POLL! (still need to read in detail when i have time)

Edited by MagnusEffect, 30 November 2011 - 09:09 AM.