Tweaks, on 01 December 2011 - 06:42 PM, said:
Just to get something clear before I respond, I'm not here to troll in any way and my post wasn't purposely doing that either. However since you responded so violently, I have to at least explain myself a little more, and then I'll stop.
I'm sorry about getting mad before, truly. But you should work on being a bit diplomatic: how would you feel if someone approached you while you were doing a project that you've spent 20-30 hours of your time on just the
typing portion of, not even the research and thinking portion of, and told you that what you're doing is meaningless?
And then compounded that statement by saying it's meaningless because of <reasons which are untrue and assumptive>?
Well, in case you're a saint who wouldn't be at all irritated, for the rest of us, having people snidely tell you that your hard work which you've been lovingly nurturing to fruition for weeks, is meaningless, is very angering.
Quote
You didn't "get" 300+ people per say, they came to you when they saw your survey link. That means that you didn't actually get a "random assortment" of people either. There's nothing random when only the interested and most subject to answer voluntarily fill out your survey. Typical statistics gathering techniques usually involves direct solicitation where people are selected randomly from a large list and contacted directly to answer a survey. That is how actual statistics agencies do it when they want to truly get a random assortment of people.
How would you characterize the typical interaction between a statistics agency and a potential respondent? I imagine it like this:
"Would you like to take this survey, sir?"
"No thanks."
"Would you like to take this survey, madam?"
"No thanks."
"Would you like to take this survey, sir?"
"No thanks."
"Would you like to take this survey, sir?"
"Sure, I have some time for that."
All surveys are voluntary, except for census bureau ones, and even those people skip out on (though it costs them in the end). There's no true difference between a statistics agency's "random selection" and my own.
And additionally, you make assumptions about
how statistics agencies get random selections. You think calling people is random? When the phone first became a major middle-class thing, there was one president candidate--I forget who--whose successful election was announced in the paper, because the news company called thousands of people and 90% of them had voted for that candidate. As it turned out, that candidate had been
defeated in a landslide loss. Why the huge difference? ...because only
rich people had phones at that point, and rich people were primarily voting for that failed Republican candidate, but everyone else wasn't.
Think it's so different today? Stand on the street corner with a survey, nobody will stop when you ask, except those who don't have anywhere to go (unemployed, or rich enough to not need a job). Cold call people, and you'll realize that there's a huge 'do not call' list, less and less people are using land-lines, and the only people who respond to you are those who are unemployed (they're the only ones with the time for a survey) or upset about something (they're the only ones who have the motivation for a survey).
Mass e-mail people, and everyone tech savvy enough to set up a spam filter doesn't receive it. You'll have a lot of work e-mails on your mass e-mail list, and those people won't take those surveys because they can't access the site at work. Everyone whose
not tech-savvy will
also be nervous about taking it, because it might be a virus or a scam (did you know that you can modify the "From" address on your e-mail so that it claims you're someone else? I've received e-mails from Blizzard Entertainment Customer Support, @blizzard.com and everything about needing my password to solve a problem with my account. The "From" address means nothing, even if it says Piranha Games Inc.)
I might be oversimplifying to prove a point, but not by much. The point is,
no matter how you perform a survey, you will be eliminating large portions of the population who, for some reason or another, either never
hear about your survey or refuse to
respond.
A good survey taker isn't someone who can overcome that obstacle, because it can't be overcome. A good survey-taker is someone who can write up their analysis or report
with those obstacles in mind. I keep them in mind.
Quote
That is not at all what I meant. I meant your choices are too restrictive. If you want people to vote, then there must be at least a "none of the above" option in each multiple-choice question, for if none of the choices apply then you are forced to pick the "closest" thing that fits your true opinion when it's not fully representing it. Afterwards, you'll be saying "Hey, 200 people said they want to have this and that this way" when in fact, it's not exactly that. They just couldn't pick anything else.
Not what you said, don't lie.
In any case, I'm not too worried. I agree, I should've had more 'indifferent' options, but in some cases it was impossible (like in the 'Mech lists) due to code, and in some cases I didn't think it was necessary, and in some cases I forgot. But I'm not worried: when you have a question that generates false responses like that, the false responses tend to just show up as static. That is, either everyone says 'yes' and it doesn't matter because you're only paying attention to which answers people said yes to the
most, or the reverse for 'no', or people say a mix of 'yes' and 'no' (as they did with the 'Mechs)... and you
still only have to pay attention to which answers people said yes/no to the
most....Again, I agree, but it's still not something I can't just mentally compensate for with extreme ease.
Also in any case, I compensated for this in other ways. The order of answers is randomized for each question. If people hit the first option on the list because they don't have an opinion, their opinion becomes static (because if there's three possible answers, 100 indifferent people will show up as voting 33% for one, 33% for another and 33% for the third).
Quote
You had to give your email to PG when you created your account to reserve your pilot name. My point was that your survey is currently not visible to all players because:
1. It's not a sticky
2. It wasn't announced per say
3. It's buried in a lot of other posts
If PG sends a mass email about an official survey of their making, then a lot more people will see it (it will literally be in their face when they login to the site or check the Home page, AND be in their inbox) and if more people see it, then more people might answer it. It's closer to actual statistics gathering techniques than just posting a survey somewhere and waiting for people to come. At least that way, they are solicited to do so.
See above. There are a number of flaws with mass-email, just as there are a number of flaws with my technique--but at least I have experience with overcoming the flaws in my technique, and most of the flaws are irrelevant to my situation (if I'm mostly trying to get people from the general forum to give me their opinion, posting on the general forum makes sense.)
But you're making two very, very,
VERY wrong arguments here. Extremely wrong.
First, by e-mailing people, you remove anonymity. This completely warps people's ability to answer honestly, if they know that their survey has their name attached to it. Maybe not consciously, but there is an extremely well-documented ability for the loss of anonymity to corrupt answers in seemingly uncorruptable surveys. (Yes, I have studied how to write surveys and study their results, I'm not just some random 'kid').
The second fail-argument here is that by e-mailing people directly, Piranha Games would lose
their neutral-party standpoint. People were able to answer my survey honestly--or more honestly, in any case--because I reassured them repeatedly that while I would e-mail the results to PGI, there were no guarantees they'd do more than pat me on the head, say 'good job', and throw it in the trash. I'm a neutral party, a nobody, an unknown--nobody cares about me. So having my name on the byline of the survey
does not influence the survey.
Maybe taking it from another tact would explain it better. You know what subtext is, right? It's the meaning behind words. "Her eyes were beautiful, like glass spheres filled with blue rosebuds." has its literal meaning, and its metaphorical meaning, but it also has subtext: "I, the narrator, am falling in love with this woman." This is commonly called
show don't tell in the writing world.
Well, when
I write the question "Would you be willing to accept in-game advertising in MechWarrior Online?", there's no subtext. I'm simply asking a direct question, with no possible other interpretation because
I am an unknown.
What if PGI asked that question? "Would you be willing to accept in-game advertising in MechWarrior Online?", then there's a
huge amount of subtext. The biggest bit being "We at PGI are seriously considering adding in-game advertising. We probably will. We're sort of curious whether the backlash will be big enough to renege though.
And that'll
completely warp the answers. To my question, people respond "Yeah, that might not be bad. Might not be good either. Not sure." but to
Piranha's question people would respond "Wait, they're looking for backlash? Then we'd better
give it to them! NO NO NO!"
There's one third problem with mass e-mail. Only 65% of my respondents came from
MWOMercs.com. I had a
ton from a variety of different forums who I posted to. Mass e-mail might get the general forums here just fine... but they completely filter out any other groups.
Quote
By the way just out of curiosity, how do you know ~10% of the people who filed your survey are not forum goers if the survey is anonymous? You're tracking the requester URL or something like that?
Didn't you take the survey? One of the final--and mandatory--questions was telling me how you arrived at the survey. In the words of Winston Churchill: "Self-report is the worst form of survey-taking, except all others that have been tried." Or maybe Churchill didn't say it quite like that, I'm not sure
That is to say, I forced people to tell me how they arrived at the survey, which I would've preferred to do with an advanced technique like URL tracking or something, but that's not possible for a huge number of reasons related to both how HTML works, how browsers work, and how the particular software I'm using works.
Quote
I don't need to be a statistics expert to write what I wrote, it's just logic.
Nope. Statistics defies logic more often than you might think.
For example: gut instinct tells you that every time it rains, there is also lightning. Therefore, rain causes lightning, right? Nope. Because correlation does not imply causation: they have a central
root cause (clouds), but they are not
related to each other in a direct way.
Or: two children from the same parents both develop schizophrenia. The root cause is
logically something genetic, or to do with their household, right? Logic! ...except, maybe they developed a chemical imbalance in the womb, or ingested something questionable at school, or maybe it was just random with no root cause whatsoever.
In any case. That's probably the most offensive thing to say to any expert in any field--"Meh, you don't have to be an expert to make strong, blanket claims about your expertise--after all, it's not like you need an
education to do it! Ha!"
Quote
That said, could you enlighten us all about your knowledge of statistics methods and techniques? Do you have a degree in statistics or better? (that's a honest question, not sarcasm by the way).
"Us all" being you, the only person whose asked. I believe the reason being I already answered this elsewhere, but maybe I didn't.
My knowledge of statistics comes from the following sources:
My attempted psychology minor (I have the credits, just, our school doesn't have one even though when I started school I had sworn it did. Maybe they closed the program.) My sociology courses. My political science courses. My business courses. All of which involved studying how surveys are constructed, worded, and analysed. I've also done a fistful of similarly large surveys and analyses over the years, and have been refining my techniques to suit this online context since. And every time, I give myself a refresher course using borrowed statistics texts, sites, and Wikipedia.
Augmented by my years and years of Calculus courses allowing me to easily read the complex formulas which underly common statistical... er... formulas.
Quote
I don't expect that much experience from a 23 years old, but we could all be surprised...You talk down to me as if you knew more than I did on the subject (which may be true, but I'm still 10 years older than you are), with a condescending tone I might add...
You should know that saying your opinion is probably better because you're older has caused at least 40 heart-attacks amongst people who know the various logical fallacies.
In my experience, you can meet perfectly intelligent and knowledgeable people at a young age, and complete idiots at an older age. The reverse is also true. After you graduate college, your age has almost nothing to do with how much knowledge you have--after college, the biggest difference between those with a particular bit of knowledge and those without are your willingness to say "there are things I might not know". Your willingness to learn.
It's the people who make a habit of saying "I don't
need to have
knowledge about an area, I use my logic and it's just as good as your
education!" who end up having less knowledge as people a decade their junior.
I'm not saying my knowledge of statistics is perfect, nor that I am a professional at it. Just that I've been studying survey-taking for my entire college career in at least one course per semester, and you just told me you wrote your statement using only 'logic'.
Edited by Mr. Smiles, 01 December 2011 - 08:45 PM.
9
This topic is locked
.
