Doktor Totenkopf, on 11 January 2013 - 03:10 AM, said:
Tolkien first of all let me say I concur that ECM is, in its current iteration, too powerful for all the reasons you named. I appreciated the work your doing to gather data on your own since the Devs seem not to be willing or allowed, so far to provide their own findings.
Nevertheless I feel your "experimental setup" has some unknown variables, eg. have you controlled for pre-mades on both sides, useage of coms...just to name two. Sorry if I missed that somewhere.
The reason I ask is that in my personal experience is that communication and not ECM is the deciding factor. I am pretty sure the numbers for that would look even more impressive.
Well to be fair his "data set" so far is also an anecdote. I dont wanna discredit him but he could in theory just writen all this down. If you wanna play the scientific evidence game you have to adhere to the rules, scientific experiments follow. He hasnt proven anything, yet. He has provided us with first evidence, very strong first evidence.
What we would need would be logs in the form of videos, multiple people taking "measurements" and repetitions and more repetitions.
Hello Death's Head,
Put simply, I have not controlled for any variables I do not have control over. Stated another way my team will never be an 8 man, nor a 4+4 man, nor a 4+3 man. As long as a player in the game has to collect the data that player will be part of the data set and I get the impression that there will always be people who are unsatisfied with results that come from this method.
In terms of why didn't I ask other players what team compositions they were in, what voice comms they had, or even how many ECM are on the other team, the problem is that people on the internet are generally jerks:
Basically I can't trust my own team, let alone the enemy to even answer accurately how many ECM they have, so there's no hope that they will accurately inform me of the order of battle comms. Another factor is that by getting enough games in uncorrelated factors should drop out in terms of influence so rather than attempt to control for them I simply tried to collect a good sized chunk of data. Once again someone will always say the results will be different if I had just collected 10x more data... then 100x more, etc.
Another interesting suggestion that I had to consider was that "Tolkien is the best pilot evar so him bringing ECM means his team will often have more ECMs and his good looks and skill will always win the match".... I'm paraphrasing here, but this has to be responded to. In short, we won a little less than 2/3rds of our conclusive matches, and had ECM superiority slightly less than 2/3 of the conclusive matches as well. If my ultimate badassery was the deciding factor this would push the expectation to 2:1 and
not let it go higher.
You'll note that in the post I already explained that the difference between the data as measured and the 2:1 win ratio is 2.23 sigmas lower than the measured set, or >97% that you will be above that 2:1 ratio when your team has an extra ECM on it.
An argument can probably be made for 2nd order effects, but with this margin of certainty between the expected value of this argument and the actual data outcomes, I'm willing to declare victory and get out.
Those who aren't willing to accept this data already will always come up with another factor that can't be tested in practice like how much sleep each pilot had the night before or whether their parents hugged them enough growing up. Personally I just want some raw match data released to the community so we can pick it apart with a fine toothed comb and see if the fellow who
made offered up data showing machine guns decide match was actually blowing smoke
Edited by Tolkien, 13 January 2013 - 04:50 AM.