344 replies to this topic

[Dimento Graven]

Zergling, on 22 April 2017 - 03:42 PM, said:

Ok fair enough, I'm talking more about personal W/L than predicting individual battle results.

It certainly is possible to predict match results to some extent if skill levels of players on each team is known, but personal W/L doesn't come down to 'pre-determined matchmaker results', as each player is able to influence the battles to cause their W/L to differ from 1.00.

I mean, if the matchmaker was pre-determining battle results, each battle would have a 50/50 chance of a win/loss, which makes players with W/L substantially different from 1.00 over a substantial number of battles impossible.

The actual point being, though, that if your team has MORE people with sub 1.0 W/L ratios, than the other side, you are more likely to lose.

[Vanguard319]

Look at it another way:

If the MM was fair and balanced as the white knights claim it to be, there would not be so many one sided stomps and rampant seal clubbing that seems to be so prevalent in public matches these days. You would see way more matches that were close instead, because the teams would be relatively even in terms of skill, and average team tonnage.

[Zergling]

Dimento Graven, on 22 April 2017 - 03:44 PM, said:

The actual point being, though, that if your team has MORE people with sub 1.0 W/L ratios, than the other side, you are more likely to lose.

Yeah, I just don't want bad players to interpret this as W/L meaning nothing because 'battle results are pre-determined!'.

That excuse gets thrown around enough as it is.

[Tarogato]

SFC174, on 22 April 2017 - 03:24 PM, said:

Taro and Blackstar, I really don't think you're going to get anywhere with Vandal. Unless you have full access to all match data and player data from PGI and can run a full analysis on all games played over a sample period, he's going to pick holes in anything you put out there (and we all know PGI will never give us that data).

He's certainly not looking to help you do a better job of figuring out whether or not the matchmaker does a good job of balancing teams. I appreciate the efforts you've made to analyze what data you have, but don't waste your time on this argument. Keep collecting data if you can and do what analysis is possible.

No, actually, he helped point out a scientific flaw in my method. One that I don't consider to be valid, but certainly one that *could* be and perhaps *should* be considered valid. As a result, I've run the numbers again to see how much of an impact this potential error could have had.

vandalhooch, on 22 April 2017 - 02:33 PM, said:

Cherry picking. Now tell me why a 12-3 would never be considered a stomp. Be sure to back it up with something more than "I just feel that way." Statistics don't work off of what you "feel" about things no matter how clever you think your use of the term boolean is.

You can't possibly know that because you didn't actually run those calculations did you?

I've reposted new numbers here: https://mwomercs.com...ost__p__5706010

You actually caught me surprise here, a huge chunk of my matches were actually 12-2. I had no idea so many of them were. It was 45 out of 116, or nearly 40%. Despite this, most of the stats didn't change by much (which makes sense, since I noticed strong patterns very early on while collecting data.)

Also, the next post I added a table that just compares how often the hypothesis is correct between the two versions (ie., a team with higher cumulative WLR is expected to be on the winning side of a stomp match, which turned out to be true around 80% of the time. This is really the most interesting stat to me.)

Three of my "major" correlations/hypotheses went down, average of 3%. Not very significant, but slightly lower nonetheless.

Two of my "minor" correlations went up, average up 7%. Significant enough that I'd consider their measurement to be dubious, or at least requiring a much larger sample before making conclusions. But these were both stats that I wasn't concerned with too much to begin with (tonnage, and GmanTons), so I have no qualms with throwing those out.



Overall, I'm still confident in the study. My goal was to analyse "stomp" matches, and for me the qualification for a "stomp" match was pretty clear cut. For instance, a 12-2 match might not have been considered a "stomp" if it was a long belaboured trading match that at least felt good for a while. Meanwhile the majority of 12-2 were just steamroll fests that were over very quickly and left you thinking "wtf even just happened."

[Dimento Graven]

Tarogato, on 22 April 2017 - 05:51 PM, said:

...My goal was to analyse "stomp" matches, and for me the qualification for a "stomp" match was pretty clear cut. For instance, a 12-2 match might not have been considered a "stomp" if it was a long belaboured trading match that at least felt good for a while. Meanwhile the majority of 12-2 were just steamroll fests that were over very quickly and left you thinking "wtf even just happened."

Yeah, stomps are, typically, numbers at the end, plus, how long it took for the match to end.

Most matches where it's 12-3 or less at the end, and also end in under 5 minutes were PROBABLY stomps.

It's possible to have a 12-0 "stomp" end in 12 minutes, but those are absolutely rare and usually because it took so long to happen feel less "stompish" to the receiving team (at least whenever it's happened to me, it didn't feel so much like a stomp).

[Dimento Graven]

Zergling, on 22 April 2017 - 04:33 PM, said:

Yeah, I just don't want bad players to interpret this as W/L meaning nothing because 'battle results are pre-determined!'.

That excuse gets thrown around enough as it is.

No, I get it. It's not a consciously programmed requirement to pre-build the winning and losing sides.

I'm sure MM is just putting together teams of T1-T4's and trying to match the same weight and average Tier on the other side of the match. Unfortunately it appears that there is a large tendency to regularly build matches where one side has a higher than average W/L, match score, and average damage than the other side.

It's an unintended consequence of how MM operates and builds teams so quickly. I would not be surprised if MM has a mechanism to rebuild the same teams if all 24 players becomes available (drops) again within a certain amount of time.

We've all seen the situation where we just finished a match and then the very next match end up having a lot of the same people on our team and facing up against a lot of the same people on the other side, and having that happen match after match after match.

I know that it's something that when I notice it happening, and I happen to be on one of those losing streaks, I'll stop playing for 15 to 20 minutes to take myself out of that particular pool.

Anyway, the Tier system isn't working right, which has broken MM.

In my mind this is what all this boils down to.

[vandalhooch]

Dimento Graven, on 22 April 2017 - 02:44 PM, said:

Except I know how to read and comprehend a point not specifically stated.

You're wrong, I'm right. I'm using PGI's own words. "as evenly matched as possible".

Quote mining for fun and profit! What was the context of "evenly matched?" Was player skill supposed to be evenly matched or Tier levels?

Quote

Yes, there's a time limiting factor of the MM that is a portion of the calculation, however, the goal isn't just slap some people together as fast as possible.

So if the matchmaker has 24 Tier 1's in two sets of 3-3-3-3, it won't immediately launch the match? That's the very definition of making matches as fast a possible.

Quote

We originally started that way in MWO, and there's a very loud and vocal minority who did nothing but ceaselessly post on the forums about it until we ended up with the original elo-based MM abortion.

ELO was a mess because ELO can't be used for individuals that make up a team. You can use ELO in games that are strictly 1 v. 1 or for teams that remain the same from match to match. MWO was never appropriate.

Quote

Then we all bitched about that for years until they came out with the Tier system, which was supposed to do a better job at factoring actual individual performance, which due to its "win weighted" scoring methodology, it couldn't help but fail, too.

Ahhhh, selective memory. The Tier system was implemented to reduce the chances of new players being thrown into matches with veteran's. The fact that we have only five Tiers shows that it was never designed to differentiate between skill levels within Tier's.

Quote

Yeah, now it looks like you're just here to argue, not actually "discuss" anything with an intent to glean some sort of solution.

No. I'm here to point out, yet again, that just because somebody "did some math" doesn't mean they have any actual evidence to support their position. I'm not here to defend the current matchmaker as great, but if we want to discuss it's weaknesses then let's use actual evidence instead of made up nonsense.

Quote

BUT I'll restate it: The Tier scoring system is slanted, GREATLY, towards heavily scoring wins much, much, MUCH more so than it penalizes losing. It makes it such that even a low-average player can, EVENTUALLY, make it to Tier 1, and be grouped with people who are SIGNIFICANTLY better than himself.

I say again, so what? Do you see me arguing against that statement? It seems you've joined the conversation without paying attention to what the OP actually claimed. He said he had done a "study" that showed that the matchmaker purposefully builds lopsided teams. Are you supporting his assertion or not?

Quote

If the points lost for losing were scaled to be, at least, on par to the points awarded for winning, the skills in the various Tiers would be more stratified. People who consistently play at a Tier 4 level would probably still be Tier 4.

And wait times at the higher Tiers would either increase due to lack of enough pilots or the release valves would open and Tiers would be mixed. In other words we would have exactly the same situation we have now. Changing how Tiers are determined does not overcome the fundamental problem that this game has a relatively small player pool from which to build matches.

Quote

As it is now, you have to be incredibly bad, or have a horribly unreliable ISP, or completely craptastic computer, or some disability, to NOT go up in rank. There are people playing under these conditions, hence we have a small subset of players who have never, and probably never will (without some dramatic change) be Tier 1.

If the Tier-ing system is failing, there's no possible way for MM to do its job. MM can't do its job using Tiers, so it should be using other data, W/L, MS, etc. all seem like a good place to start.

Then put your money where your mouth is, give us your algorithm for determining player skill and explain how it should be implemented in the matchmaker.

Quote

You mean besides win/loss ratio, plus average damage, plus average match score?

You mean using THOSE 3 numbers might not be better than say, just ONE number of "1","2","3","4", or "5"?

As was mentioned, there was another thread by someone else I'd read a long while back, and if I remember right he took the people dropped in the match, rearranged them with 'mech weight, and those 3 numbers and came out with sides that were absolutely MORE balanced than they were when MM originally reassembled them.

And did this rearrangement produce more balanced matches? Oh wait, you and he can't possibly know that.

Quote

Uh huh, and yet a potato is very easy to spot when observing his play.

Observation bias is a thing. And statistical illiteracy appears to also be a thing here.

[vandalhooch]

Vanguard319, on 22 April 2017 - 04:01 PM, said:

Look at it another way:

If the MM was fair and balanced as the white knights claim it to be, there would not be so many one sided stomps

Show me your calculation supporting the assertion that there are "so many one sided stomps." Given the mechanics of a no-respawn, accumulated damage game, what frequency of stomps is expected between evenly matched teams? What frequency of stomps indicates uneven teams? Even more basic, what is the current frequency of stomps experienced by the player base right now?

Observation bias isn't data.

Quote

and rampant seal clubbing that seems to be so prevalent in public matches these days. You would see way more matches that were close instead, because the teams would be relatively even in terms of skill, and average team tonnage.

That isn't how no-respawn games work.

[BLOOD WOLF]

This still going on? Would have been over with if the guy who made the data accepted that it was flawed. Could have saved some pages

[vandalhooch]

Tarogato, on 22 April 2017 - 05:51 PM, said:

No, actually, he helped point out a scientific flaw in my method. One that I don't consider to be valid, but certainly one that *could* be and perhaps *should* be considered valid. As a result, I've run the numbers again to see how much of an impact this potential error could have had.

I've reposted new numbers here: https://mwomercs.com...ost__p__5706010

Better, but now there are other problems to address. But still, better.

Quote

You actually caught me surprise here, a huge chunk of my matches were actually 12-2. I had no idea so many of them were. It was 45 out of 116, or nearly 40%.

And you were personally making the call of whether to include or not include them in your data set based on how you "felt" about the match.

Quote

Despite this, most of the stats didn't change by much (which makes sense, since I noticed strong patterns very early on while collecting data.)

Okay, now for the next set of problems with your data.

1 - Sample size of 71 is likely not large enough for a an alpha of 5% given the inherent variance of the data.

2 - Your sample data does not control for other factors that might increase or decrease frequency of stomps.

3 - What is your control for map effects? Mode effects? Group vs. solo effects? Initial drop site effects?

4 - What is the baseline rate of difference between teams for all matches? Are stomps common? Are they rare? How well does your model predict the rate of stomps?

Without all this additional data, you nor I can not really say anything at all about the strength or weakness of the matchmaker. All you have possibly shown is that unbalanced teams (according to your metric) rarely result in a reverse stomp.

Quote

Also, the next post I added a table that just compares how often the hypothesis is correct between the two versions (ie., a team with higher cumulative WLR is expected to be on the winning side of a stomp match, which turned out to be true around 80% of the time. This is really the most interesting stat to me.)

But it has absolutely no relevance to the question of if the matchmaker is failing to generally make evenly matched teams. For that, you need to collect the results of thousands of matches so that you can run a proper ANOVA to account for all the factors that might affect the outcome of any particular match.

BTW: Dimento just tried to tell me that true pilot skill is a combination of WLR, KDR, and MS. Gosh, it seems like there's no universal agreement on what metrics truly indicate an individual pilot's skill.

Quote

Three of my "major" correlations/hypotheses went down, average of 3%. Not very significant, but slightly lower nonetheless.

Not sure what you mean by "went down, average of 3%." Significance is something that can be calculated given the proper data. With a sample size of 71 matches however, your conclusions are unlikely to prove significant at an alpha of 5%.

Quote

Two of my "minor" correlations went up, average up 7%. Significant enough that I'd consider their measurement to be dubious, or at least requiring a much larger sample before making conclusions. But these were both stats that I wasn't concerned with too much to begin with (tonnage, and GmanTons), so I have no qualms with throwing those out.

They should never have been included in the first place. They are just as much cherry picking as your "feelings" about stomps. A proper data set, properly analyzed, might be capable of supporting the metamechs opinions but without that, their inclusion in your data set is useless. Garbage in, garbage out.

Quote

Overall, I'm still confident in the study. My goal was to analyse "stomp" matches, and for me the qualification for a "stomp" match was pretty clear cut. For instance, a 12-2 match might not have been considered a "stomp" if it was a long belaboured trading match that at least felt good for a while. Meanwhile the majority of 12-2 were just steamroll fests that were over very quickly and left you thinking "wtf even just happened."

Not a study (see my previous posts). Your analysis, now, is more reliable in indicating that 12-0 and 12-1 matches often have unbalanced teams (as defined by your WLR metric). Your analysis says absolutely nothing about whether or not the matchmaker is doing a "good job" or not.

Edited by vandalhooch, 22 April 2017 - 08:09 PM.

[Too Much Love]

All those claims about "insufficient data" are plain ridiculous.

It's obvious that the more data you have, the better, but the point of any research is to draw conclusions from the part of the picture. If you have the whole picture, if you know the formula of MM, you don't need to study it.

So all this "until you have 1 mill. samples with all the tonnage, tier ratings, time of the day, disconnects etc, every research is useless" claims are absurd and missing the point.

If I have all this info, it means I'm the PGI's server admin or game developer and I already know how it works. "The most precise map of Great Britain is Great Britain itself".

It's the same as to claim that it's a waste of time and money to build hadron collider until the properties of all the elementary particles is known. It's absurd, because the very meaning of building hadron collider is to research this particles.

Even most of the modern science works with insufficient data. All the weather forecasts, space exploration, physics. You wouldn't disregard all the opinion polls and statistics itself upon the fact that they don't know opinion of every citizen on every question.

Not to add, as I underlined already, that the MM is not a complex natural phenomena or social process. In fact, it's a human-built algorithm, relatively simple formula, that stamps the same product every time as assembly line. Its mark is well seen on everything it touches.

Edited by drunkblackstar, 22 April 2017 - 09:08 PM.

[Dimento Graven]

vandalhooch, on 22 April 2017 - 07:39 PM, said:

Quote mining for fun and profit! What was the context of "evenly matched?" Was player skill supposed to be evenly matched or Tier levels?

Yes, you're only here to argue, very obvious with that.

The Tier system was originally intended for more balanced matches, and yes to also protect the noobs, who originally came it near the top end of Tier 4, avoid playing against vets, or being put on the side of vets who don't typically like having to carry 'potatos'.

Quote

So if the matchmaker has 24 Tier 1's in two sets of 3-3-3-3, it won't immediately launch the match? That's the very definition of making matches as fast a possible.

Sure, but due to the Tiering system being slanted to favor winning over losing, only the worst of the worst will NOT be able to move rank.

So the longer it functions this way the more meaningless Tiers 1-3 are (BUT, interestingly enough, it makes Tier 5 really mean something, because man you have to play REALLY sh!tty, consistently, to deserve a spot in Tier 5).

So while, from a Tier and tonnage perspective it's "balanced" and quickly, as soon as someone actually looks at the performance of the players, they find that the MM is regularly setting up sides where one side has more higher performing players than the other side, resulting in an upwards of 80% accuracy in predicting which side is probably going to win.

So the match was fast, but it wasn't actually balanced.

Quote

ELO was a mess because ELO can't be used for individuals that make up a team. You can use ELO in games that are strictly 1 v. 1 or for teams that remain the same from match to match. MWO was never appropriate.

Totally agree, it was doomed to failure before it began, they wasted a LOT of effort on someone misapplying elo.

Quote

Ahhhh, selective memory. The Tier system was implemented to reduce the chances of new players being thrown into matches with veteran's. The fact that we have only five Tiers shows that it was never designed to differentiate between skill levels within Tier's.

That was part of it yes, didn't think it merited mentioning because that's obvious. No one wants brand new players being butt ***** by people who have been playing for years. While it might be fun to club seals, it's no fun being the seal. BUT ALSO, it was an attempt at producing a mechanism that worked better than elo, and would create more interesting and fun matches by ensuring that both sides had as close to equal skill as reasonably possible.

Quote

No. I'm here to point out, yet again, that just because somebody "did some math" doesn't mean they have any actual evidence to support their position. I'm not here to defend the current matchmaker as great, but if we want to discuss it's weaknesses then let's use actual evidence instead of made up nonsense.

Sorry, but that's kind of how science works. You observe, you gather data, you try and understand what the data is telling you.

The limited sample supports what it appears a majority post regularly about in the forums, MM is not doing its intended job.

If you just want to make matches as quickly as possible, you don't need Tiers, or rankings, you just start slopping people together as soon as they show up.

We had that, we didn't like it, we got elo, then we got Tiers.

Quote

I say again, so what? Do you see me arguing against that statement? It seems you've joined the conversation without paying attention to what the OP actually claimed. He said he had done a "study" that showed that the matchmaker purposefully builds lopsided teams. Are you supporting his assertion or not?

I'm supporting these sentences:

"In fact matchmaker doesn't assemble the equal teams. It makes teams to be unequal. "

This from experience and two different people doing analysis independently, appears to be proven.

Yes, yes, he goes on to say:

"The one team is determined to win, the other - to lose."

Poorly worded, perhaps a bit hyperbolic if the syntax was intended, but it doesn't invalidate the fact that it really does appear from the data, and reported experiences, and my own personal experience, that MM is NOT creating balanced teams and if/when it does, it's more accident than intent, primarily because the key measure of each pilot is insufficient, the player's Tier ranking is not sufficient to gauge skill.

After all, I'm fairly certain that in most circumstances any player with a W/L ratio of less than 1.0 should be going DOWN in ranking at some point. HOWEVER, with the Tier system as slanted to wins as it is, you can have a very low W/L ratio and STILL grind into Tier 1.

Quote

And wait times at the higher Tiers would either increase due to lack of enough pilots or the release valves would open and Tiers would be mixed. In other words we would have exactly the same situation we have now. Changing how Tiers are determined does not overcome the fundamental problem that this game has a relatively small player pool from which to build matches.

Or, we Tier 1's could suck it up and let MM churn for another minute or two...

AND/OR BETTER YET: PGI could actually spend for some WoT level TV and magazine ads to maybe attract more players so that we could actually HAVE a decently sized player base to avoid the wait time...

Quote

Then put your money where your mouth is, give us your algorithm for determining player skill and explain how it should be implemented in the matchmaker.

I guess I wasn't clear:

"...so it should be using other data, W/L, MS, etc. all seem like a good place to start."

PGI has this information already, surely it can make sure that the average W/L, match score, and damage per match amongst the two sides is closer to even.

You want me to write the code for that? Soon as PGI hires me away from my current employer, and trust me, in Canadian dollars, that's gonna be a LOT more than I bet they're willing to spend, I'll be happy to.

Quote

And did this rearrangement produce more balanced matches? Oh wait, you and he can't possibly know that.

Doesn't invalidate that, at least at that level, IT CAN BE DONE.

Since we're fairly certain we're not getting balanced matches now, it'd sure be nice to at least attempt it, no?

Or are you just here to argue?

Quote

Observation bias is a thing. And statistical illiteracy appears to also be a thing here.

Yeah, yeah, yeah... "lies, damned lies, and statistics..."

<yawn>

Quote

BTW: Dimento just tried to tell me that true pilot skill is a combination of WLR, KDR, and MS. Gosh, it seems like there's no universal agreement on what metrics truly indicate an individual pilot's skill.

I didn't mention KDR.

I mentioned W/L ratio, match score, and damage per match.

I think those are pretty good places to start measuring 'true' pilot skill.

Are we not seeing people that generally considered "very skilled" at the top of the leaderboards month after month? What are their W/L ratios, avg. match score, and avg. damage per match?

Typically they're MUCH better than people lower down the leaderboards...

It seams a not unreasonable thing to try...

Edited by Dimento Graven, 22 April 2017 - 09:18 PM.

[Too Much Love]

Dimento Graven, on 22 April 2017 - 09:12 PM, said:

Yes, you're only here to argue, very obvious with that.

To argue with the people whose only intent is to argue is a wate of time.

[EgoSlayer]

Dimento Graven, on 22 April 2017 - 09:12 PM, said:

<snip>
I'm supporting these sentences:

"In fact matchmaker doesn't assemble the equal teams. It makes teams to be unequal. "

<snip>

Except that statement is not supported, and 100% incorrect based on the information we know.

The statement is incorrect because it's not looking at the criteria the match maker used to make the match. It's looking at other criteria that are unavailable to the match maker.

Based on data the PGI has released, the matches *are* balanced when looking only at the PSR ranking used by the match maker. Something we already know isn't an accurate match making factor.

So really, the only thing proven here and at other looks into matches, is that PSR is flawed and not adequate for being the sole match making skill criteria. Something most people already knew as soon as PGI posted how PSR works.

[Dimento Graven]

EgoSlayer, on 22 April 2017 - 09:38 PM, said:

Except that statement is not supported, and 100% incorrect based on the information we know.

The statement is incorrect because it's not looking at the criteria the match maker used to make the match. It's looking at other criteria that are unavailable to the match maker.

Based on data the PGI has released, the matches *are* balanced when looking only at the PSR ranking used by the match maker. Something we already know isn't an accurate match making factor.

So really, the only thing proven here and at other looks into matches, is that PSR is flawed and not adequate for being the sole match making skill criteria. Something most people already knew as soon as PGI posted how PSR works.

From the raw context of your sentences, we actually agree.

PSR in its current form is not the best tool to use to create balanced matches.
MM uses PSR as its primary 'balance' mechanism.

Ergo: MM is not making balanced matches, because the criteria it's using is effectively meaningless for the purpose intended.

If you fix PSR, you fix MM.

Or you could change MM to not use PSR (Then what is PSR good for? Good question.)

Either/or, we actually agree: MM could do better than it is currently doing.

Edited by Dimento Graven, 22 April 2017 - 09:52 PM.

[Too Much Love]

EgoSlayer, on 22 April 2017 - 09:38 PM, said:

Except that statement is not supported, and 100% incorrect based on the information we know.

The statement is incorrect because it's not looking at the criteria the match maker used to make the match. It's looking at other criteria that are unavailable to the match maker.

This topic has been discussed in this thread already.

I can explain it one more time: my purpose was to examine how MM actually works, not how it should work "based on the information we know".

I didn't make the discovery that "MM is bad", I tried to show how it is bad.

P.S. People lived on the flat Earth for thousands years, "based on the information they knew".

[EgoSlayer]

drunkblackstar, on 22 April 2017 - 09:56 PM, said:

This topic has been discussed in this thread already.

I can explain it one more time: my purpose was to examine how MM actually works, not how it should work "based on the information we know".

I didn't make the discovery that "MM is bad", I tried to show how it is bad.

P.S. People lived on the flat Earth for thousands years, "based on the information they knew".

And my point and some other people's as well, is that until you can actually look at the data the MM used to create the match - you can't make any assertions about "how MM actually works" you can only analyze the end result.

And that data the MM used is the exact PSR ratings of each player in the match not just tier, because PSR is a scale and there are hidden numeric values behind PSR tiers.so even if you had everyone honestly telling you their tier we still don't know that hidden value. To the match maker it's making balanced matches with the data it has - the match maker is the flat earth believer, not the players you are arguing with.

Edited by EgoSlayer, 23 April 2017 - 07:26 AM.

[Slowth]

I'm from a unit, and I suck at quick (I mean selfish) play. Sorry to wreck your selected view.
Maybe w should just get rid of quick play all together.

Edited by Slowth, 22 April 2017 - 10:34 PM.

[Too Much Love]

I guess, everybody who's got at least the intellect of 10 y.o. understood each other and there is no need to repeat themselvs.

[vandalhooch]

drunkblackstar, on 22 April 2017 - 09:04 PM, said:

All those claims about "insufficient data" are plain ridiculous.

Tell that to your statistics professor, or graduate student adviser, or reviewing committee.

Quote

It's obvious that the more data you have, the better, but the point of any research is to draw conclusions from the part of the picture. If you have the whole picture, if you know the formula of MM, you don't need to study it.

If you collect biased data or insufficient data, then you don't learn anything about your subject. There really is a minimum sample necessary for analysis. They teach that in every basic statistics course.

Quote

So all this "until you have 1 mill. samples with all the tonnage, tier ratings, time of the day, disconnects etc, every research is useless" claims are absurd and missing the point.

No one said millions. That's just you being defensively hyperbolic. You do need all of that other information because you want to claim that pilot skill is a determining factor in match outcomes. In order to show that that is true you must first eliminate all other possible factors that might play a role in determining match outcomes. If you don't eliminate other possible sources of effect then YOU CAN NOT DRAW A CONCLUSION YOU WANTED TO BE TRUE IN THE FIRST PLACE. That's just being downright dishonest.

Quote

If I have all this info, it means I'm the PGI's server admin or game developer and I already know how it works. "The most precise map of Great Britain is Great Britain itself".

Yes. In order to analyze how effective matchmaker is doing you need to have the relevant data. We don't have it. We can't claim to be drawing any conclusions based on data that we don't in fact have.

Quote

It's the same as to claim that it's a waste of time and money to build hadron collider until the properties of all the elementary particles is known. It's absurd, because the very meaning of building hadron collider is to research this particles.

Oh please. Don't try to now pretend you are going to lecture us about how science is done at the LHC. Do you even have the slightest clue how we detected Higgs particles in the first place? I'll give you a hint, it required that we have a complete working model of how we thought everything already works. It's called the Standard Model.

Quote

Even most of the modern science works with insufficient data. All the weather forecasts, space exploration, physics.

No scientist ever proposes drawing a conclusion based on insufficient data. There are statistic rules for determining what is sufficient and for determining a mathematical expression in your confidence of your conclusion based on the data you have. You didn't do any of that.

Quote

You wouldn't disregard all the opinion polls and statistics itself upon the fact that they don't know opinion of every citizen on every question.

Ever notice that all those opinion polls are always based on asking at least 1,100 people and conclusions are expressed with numerical error margins? Do you have the slightest understanding as to why that is?

Quote

Not to add, as I underlined already, that the MM is not a complex natural phenomena or social process. In fact, it's a human-built algorithm, relatively simple formula, that stamps the same product every time as assembly line. Its mark is well seen on everything it touches.

And if we had the appropriate data, we could determine if it was working as the programmers intended. We don't have that data.