1911 replies to this topic

[Hammerhai]

I joined end of closed beta, but before matchmaker, and it is difficult to argue by analogy but it seems in the nature of the game dynamics that from about 3 down it becomes a stomp pretty much regardless of skill, as the firepower is easier and easier to focus. In the days of 8 vs 8 it was an excellent predictor of defeat. So be aware game mechanics may play a role, not just skill

Yes, it is easy to blame the matchmaker and the pugs for everything, but the truth is a different shade of grey IMHO. Just saying. However thank you Karl for revisiting the metric and investigating the issue. In fact the whole idea of quantifying expertise and skills (think knowledge systems) is actually hugely fascinating.

In fact statistics arose because Mathematician Blaise Pascal was asked by a nobleman to tell him why he was always losing a certain bet he was fond of making with friends. Just imagine the brainsweat that went into conceptualising that chance can be measured and used in forecasting.

Karl, there is a little known gem on the internet called free allegiance. These guys in the end wrote their own skill measuring system for that game, after the original microsoft trueskill (?) system was determined to be broken. Unfortunately I myself cannot access those forums anymore. But maybe the guys would not mind sharing their research into that aspect. I talked a bit to a chap called MrChaos about it. I know this was mentioned by someone else on the forums as well a long time ago, but that would likely have gotten lost in the shuffle.

Just hoping I am not mentioning a blind alley here

Edited by Hammerhai, 26 May 2014 - 02:41 PM.

[Kageru Ikazuchi]

Karl Berg, on 26 May 2014 - 01:07 PM, said:

- Yes, an Elo is calculated for groups. We use this calculated value (currently the average Elo of all team members, but I want to run analytics to determine whether or not this is actually a good metric to use) to match skill levels between groups, the same as we match skill levels between solo players.

This might be the best news I've heard since you were pegged as "the guy" working on the match maker.

[Cimarb]

Karl Berg, on 26 May 2014 - 01:07 PM, said:

- Yes, an Elo is calculated for groups. We use this calculated value (currently the average Elo of all team members, but I want to run analytics to determine whether or not this is actually a good metric to use) to match skill levels between groups, the same as we match skill levels between solo players.

I agree with Kageru. Please do look into better ways to do this.

For example, maybe you could calculate for outliers:

• In a group of four, with 3 high Elo and 1 low, you ignore the low and average the high
  
  • ​2400, 2100, 1900, 800 - ignore 800 and average becomes 2133, instead of 1800 if you had included the outlier
• In a group of two, with 1 high and 1 low, you skew towards the higher by doubling it and dividing by 3
  
  • ​2100 and 800 - becomes (2100*2+800)/3 = 1667, instead of 1460

Or, maybe ignore the averages altogether and just match based upon Elo as if they were individual players

Also, Elo really needs to be determined by more than JUST win/loss. WLR is a great stat, but it has little to nothing to do with an individual's contribution to a team. I won't go into detail, as I'm sure you have heard all sorts of arguments about this, but determining an individual's value based off of a team-based stat is odd.

[9erRed]

Greetings all,

Reference:
Q: Can you please escalate to 3D modellers issue with missing back and top sides of cockpit in many Mechs? The problem is, that when you have lightning source behind you Mech - you get all the reflections and lighting effects inside the cockpit as if your 'Mech was transparent.
A: From Karl - Missing cockpit geo. This is on the art side of course, so the best I can do is bring it up with the artist lead. They're pretty busy with Clan work right now, so I don't know if anyone will have time to go back and look at this currently.

This was brought up quite a while ago when there were issues with light and glare directed into the cockpit from behind, and some talented people (the MWO community) actually displayed what the physical cockpit looked like outside of the Mech's. But the OP is correct, there is no back on any of the cockpit design's. Basically any light or weapons fire directed at the rear of the Mech shows light or flares into the cockpit. The cockpit is not a closed box or component, and the MWO artists have known about this from the initial design stage.

Additionally, any of the smaller Mech's. as they pass through any water that is 3/4 the depth of the Mech.
- If the Pilot looks down (free view) while in water, the cockpit is filled with water, "inside"! This kind of destroys any sense of being in a vehicle of any kind, let alone an Armoured Mech.
- From Karl's statement, this may be the same for the Clan Mech's unless they have changed the model design.
- At one time the Pilot was able to 'free look' behind himself, and was looking at an open hole in the back of the Mech cockpit. PGI locked the view somewhat so the Pilot cannot actually see behind anymore. But that open hole is still there.

Was there a reason for the lack of cockpit closure? Is that needed to be able to 'slot' that section into the Mech? Is this an CryEngine issue of placing the Pilot inside a structure?

Karl, when you do get an answer, could you or one of the artists place a response here? Advising the players just what is going on, or the issues that this design has.

Thanks,
9erRed

Edited by 9erRed, 26 May 2014 - 07:17 PM.

[Wintersdark]

Cimarb, on 26 May 2014 - 04:39 PM, said:

Also, Elo really needs to be determined by more than JUST win/loss. WLR is a great stat, but it has little to nothing to do with an individual's contribution to a team. I won't go into detail, as I'm sure you have heard all sorts of arguments about this, but determining an individual's value based off of a team-based stat is odd.

Elo is determined by wins and losses, but not by win loss ratio. This is a critically important point, and it is why there are no stats for determining Elo. Elo is not calculated by stats at all, and can't be. The whole system is based on calculating the likelihood of your team winning the match by using past results, and then adjusting scores if the calculation ends up being incorrect. It's an adaptive system that does work reasonably well (no, not perfectly, but there is no perfect system).

You can't incorporate stats into it and still have it work. You'd need an entirely different system for that.

Now, here's the thing. Everyone wants to think other stats will better reflect your skill, but ultimately that's wrong in this context.

The only thing that matters is: Did you contribute positively to your teams victory?

How can we determine that? How much damage you did? Damage done is irrelevant: I can score 600+ damage in a battle and contribute nothing noteable to my teams' success, and I can score 200 damage, get 3 kills, and be a linchpin of the team's success. In many cases, in fact, high damage games are a result of poor play, not good play.

Assists? Again, that's not a reflection of how well you did. I'd rather have a player focus down a target or two rather than splatter a little damage over multiple foes uselessly.

Kills? Still meaningless. You're usually a better contributor to your team if you switch from a crippled target to a threatening one, giving the kill to whoever's nearest when mopping up time comes around. Also, while "kill stealing" isn't a thing (you SHOULD be focussing fire, after all), likewise if you push that HGN to a red CT then someone else finishes him, it's not a reflection on your performance.

KDR? As kills isn't particularly relevant, neither are deaths. If you play a Squirrel light/medium, and do it well, you'll likely end up with a rather poor KDR but could be strongly contributing to your team's success.

And then, what about all the immeasurable factors? Maybe you're a terrible shot, but a decent leader skilled at organizing PUG teams? Just being a good communicator can have a massive impact on your teams likelyhood to succeed.


See, here's why Elo is so commonly used. On a match-by-match basis, it's generally useless. However, when provided with many matches, it's the best system to actually gauge your real contribution to a match. No matter what people like to say, they are always 1/12th of the match. Sometimes, that's not enough to make a difference that changes the outcome (either because your team would win or lose regardless of what you did, but sometimes it is. As the "auto-win" and "auto-lose" are equally likely (yeah, everyone likes to think they lose more because of terrible players, but that's just stupid confirmation bias working - the other team has the same factors at play), statistically speaking over many matches then the differentiating factor is you.

Thus, Elo is the only system that can account for the immeasurable factors.

Again, it's not perfect. It fails primarily in two ways:
1) Small sample size. Without lots of matches, individual matches have too much weight. This is why, incidentally, we don't have per-chassis Elo scores - the more scores we have, the more random they are. We could have a better overall score if we only have one Elo rating rather than 4, but there's a strong argument to be made that play between weight classes is very different, and players can be very good at one and terrible at another (I'm terrible at lights, excellent in assaults, for example). Have to pick your poison with this.

2) Player pool availability. This cannot be fixed. The hard truth is that even with a seemingly large pool of players playing concurrently, at the extremes of the scale there will be very, very few players searching for a match at any given time. As such, if you're very high or very low Elo, you will get a larger spread in your matches. It's simply unavoidable. The only solution to this is extending matchmaking time, but that obviously has it's own failings. I've shown math for this elsewhere - even with 50000 concurrent players(which would be indicative of 6 digits worth of players playing daily, a very good number), having 300 of them searching for a launch at any given time is extremely generous, and of that only a tiny portion will be high or low rated.

Any other rating system will fail in these same ways as well. If the pool of players at your skill level actively searching for a match at exactly the same time as you is low, then you're going to either fail at searching, have really long waits, or have more random matches. There's simply no other options there.

Designing a matchmaking system is hard, and it can't work perfectly unless you've got both an extremely large player pool and a vast sample size. MWO will never have the former (it'll always be a niche game), and the later only comes with time.

[Cimarb]

Wintersdark, on 26 May 2014 - 07:18 PM, said:

Spoiler

Elo is determined by wins and losses, but not by win loss ratio. This is a critically important point, and it is why there are no stats for determining Elo. Elo is not calculated by stats at all, and can't be. The whole system is based on calculating the likelihood of your team winning the match by using past results, and then adjusting scores if the calculation ends up being incorrect. It's an adaptive system that does work reasonably well (no, not perfectly, but there is no perfect system).

You can't incorporate stats into it and still have it work. You'd need an entirely different system for that.

Now, here's the thing. Everyone wants to think other stats will better reflect your skill, but ultimately that's wrong in this context.

The only thing that matters is: Did you contribute positively to your teams victory?

How can we determine that? How much damage you did? Damage done is irrelevant: I can score 600+ damage in a battle and contribute nothing noteable to my teams' success, and I can score 200 damage, get 3 kills, and be a linchpin of the team's success. In many cases, in fact, high damage games are a result of poor play, not good play.

Assists? Again, that's not a reflection of how well you did. I'd rather have a player focus down a target or two rather than splatter a little damage over multiple foes uselessly.

Kills? Still meaningless. You're usually a better contributor to your team if you switch from a crippled target to a threatening one, giving the kill to whoever's nearest when mopping up time comes around. Also, while "kill stealing" isn't a thing (you SHOULD be focussing fire, after all), likewise if you push that HGN to a red CT then someone else finishes him, it's not a reflection on your performance.

KDR? As kills isn't particularly relevant, neither are deaths. If you play a Squirrel light/medium, and do it well, you'll likely end up with a rather poor KDR but could be strongly contributing to your team's success.

And then, what about all the immeasurable factors? Maybe you're a terrible shot, but a decent leader skilled at organizing PUG teams? Just being a good communicator can have a massive impact on your teams likelyhood to succeed.


See, here's why Elo is so commonly used. On a match-by-match basis, it's generally useless. However, when provided with many matches, it's the best system to actually gauge your real contribution to a match. No matter what people like to say, they are always 1/12th of the match. Sometimes, that's not enough to make a difference that changes the outcome (either because your team would win or lose regardless of what you did, but sometimes it is. As the "auto-win" and "auto-lose" are equally likely (yeah, everyone likes to think they lose more because of terrible players, but that's just stupid confirmation bias working - the other team has the same factors at play), statistically speaking over many matches then the differentiating factor is you.

Thus, Elo is the only system that can account for the immeasurable factors.

Again, it's not perfect. It fails primarily in two ways:
1) Small sample size. Without lots of matches, individual matches have too much weight. This is why, incidentally, we don't have per-chassis Elo scores - the more scores we have, the more random they are. We could have a better overall score if we only have one Elo rating rather than 4, but there's a strong argument to be made that play between weight classes is very different, and players can be very good at one and terrible at another (I'm terrible at lights, excellent in assaults, for example). Have to pick your poison with this.

2) Player pool availability. This cannot be fixed. The hard truth is that even with a seemingly large pool of players playing concurrently, at the extremes of the scale there will be very, very few players searching for a match at any given time. As such, if you're very high or very low Elo, you will get a larger spread in your matches. It's simply unavoidable. The only solution to this is extending matchmaking time, but that obviously has it's own failings. I've shown math for this elsewhere - even with 50000 concurrent players(which would be indicative of 6 digits worth of players playing daily, a very good number), having 300 of them searching for a launch at any given time is extremely generous, and of that only a tiny portion will be high or low rated.

Any other rating system will fail in these same ways as well. If the pool of players at your skill level actively searching for a match at exactly the same time as you is low, then you're going to either fail at searching, have really long waits, or have more random matches. There's simply no other options there.

Designing a matchmaking system is hard, and it can't work perfectly unless you've got both an extremely large player pool and a vast sample size. MWO will never have the former (it'll always be a niche game), and the later only comes with time.

All very good points.

Elo should not be based upon JUST wins and losses, or WLR, or whatever you want to call it. Instead, it should be adjusted by any other factors that are appropriate. Win/Loss can essentially be the equivalent of your leadership and/or following skills, but I don't even think it has much to do with that. How much you contribute through averages of cap time, damage, kills, ECM counters, TAG assists, and tons of other statistics would make for a much better estimation of your Piloting Skill than just your wins and losses.

Elo (Piloting Skill or PS) should not be used by itself, though, in a match that involves so many other factors. Elo should be an adjustment on top of a true Battle Value (BV) system, which should take into account the combined total of it's component factors, such as weapons, engine, armor, etc. I don't care how good you are, if you bring a Battlemaster with 8 flamers, you are NOT going to contribute much to the match.

[Wintersdark]

Cimarb, on 26 May 2014 - 08:14 PM, said:

All very good points.

Elo should not be based upon JUST wins and losses, or WLR, or whatever you want to call it. Instead, it should be adjusted by any other factors that are appropriate. Win/Loss can essentially be the equivalent of your leadership and/or following skills, but I don't even think it has much to do with that. How much you contribute through averages of cap time, damage, kills, ECM counters, TAG assists, and tons of other statistics would make for a much better estimation of your Piloting Skill than just your wins and losses.

All good in theory, useless in practice. As I said in my post, all those stats are not necessarily (or even usually) reflective of useful contribution. Its virtually impossible to quantify - more on this below.

Quote

Elo (Piloting Skill or PS) should not be used by itself, though, in a match that involves so many other factors. Elo should be an adjustment on top of a true Battle Value (BV) system, which should take into account the combined total of it's component factors, such as weapons, engine, armor, etc. I don't care how good you are, if you bring a Battlemaster with 8 flamers, you are NOT going to contribute much to the match.

Elo is thought of as pilot skill, but it is not pilot skill. Elo is a measure of how much you contribute to victory on average.

That was a major part of my post above thou seem to have missed. Elo is largely irrelevant to individual matches - you may normally do really well, but today you're piloting some ridiculous Flamer Battlemasters. If you do that frequently, your rating will alter to take that into account. Its an adaptive system, and that's its greatest strength. It doesn't require constant tuning or balance, and its effectively impossible to game.

What about a battle value and stat based pilot skill combined rating? In theory craft, it'd be perfect. The best system, hands down.

In MWO, its impossible to such a degree its useless to even discuss. It would be a massively complex balancing nightmare, and would be very easy to exploit.

Why? Because its not good enough to just come up with values for stats, you need the relationships between those values to work too. You need to have the relationship between the value you place on a PPC to make sense relative to the value you place on an assist (remember, too, random splattered assists in a match that don't actually contribute, etc).

You can't use tabletop values. Weapons in MWO have no relation to their tabletop counterparts because the game mechanics are so different, and wrong values would be counterproductive.

Basically, if your values in BV(and PS for that matter, but I'll just focus on BV here) are incorrect, then you get what amounts to a matchmaker failure in every single match always... But one controllable and manipulatable by enterprising players.

So, you'd need vast devtime to build up good BV starting points... Then, you'd need to constantly invest substantial dev time into maintaining those values as the game mechanics and weapon/mech stats change. An online game like MWO is very much a moving target. And that's all assuming whoever was in charge of developing and maintaining those values actually does a good job.

Its awesome in theory, and impossible in practice.

[Kageru Ikazuchi]

Snipped, because you and I have had this discussion before (several times, I think) ...

Cimarb, on 26 May 2014 - 08:14 PM, said:

Elo should not be based upon JUST wins and losses, or WLR, or whatever you want to call it. Instead, it should be adjusted by any other factors that are appropriate.

The thing is, all of those other factors contribute to one end result ... one team wins, one team loses (and very rarely, there's a tie ... which I think doesn't affect Elo at all).

And, as I understand it, if based on your current Elo, you're ...

• Greatly favored to win, and you win ... no change; lose ... down hard
  
• Slightly favored to win, and you win ... no change; lose ... down slightly
  
• Even match, and you win ... slight change up; lose ... slight change down
  
• etc.

Over the very long term, it takes into account your skill in building 'mechs, choosing the right 'mech for you, how hard your past opposition has been, how idiotic your past teammates have been, how well you use your weapons, etc., because, in the long term, all of these things contribute to the ultimate result ... one team wins.

While I agree that this is not chess, I choose whether to bring a Meta-Victor or a 6xFlamer/1xTAG Awesome or anything in between (in chess, you don't have this choice). I pay (or benefit) from that choice.

[Wintersdark]

And after all that? Say you make the system, miraculously get the correct values and can maintain them in an ever changing environment? The matchmaker would still fail in exactly the same way it does now, particularly in Case #2 of my earlier post.

No... In a world of imperfect options and realistic resources, Elo is the best option to adaptively quantify your overall average contribution to matches. No stats provide a reliable indication of this. None of them
Not even WLR. In every case, I can provide multiple reasons why a "poor" stat may be an indication of strong play, and a high stat an indication of weak play.

[DarkonFullPower]

Wintersdark, on 26 May 2014 - 07:18 PM, said:

*snip*

So much this. I helped win a match once because I decided to shoulder check a double Gauss mech. He couldn't hit anything or anyone other then my right arm. Idk if I got an assist on the kill, or if I even got any notable personal damage on it, but my fellow 2 Lights sure did. That one mech was the only thing stopping us from flanking. A lost AC 5 arm for a dead double Gauss mech and flanking position? I more then helped earn that victory, even though I did little more then tank for my fellow teammates.

No "Stat" in the game will show that I did well, other then my ELO going up. Comparing Win/Loss directly will lead to even more situations of low skill players being jumped into games above their skill level just because of either a lucky winning streak or crushing many low skill players. ELO, at the very least, only moves you up when you defeat a team of similar or higher ELO players.

However ELO was never deigned for team play use, as least not on the individual level. It works perfectly for rating a consistent team as a whole, but even then it cannot rate each individual player's skill on that team, they would all be the same ELO. In public games, you can get into "ELO Hell" easily because of this.
A high skill level player may be stuck with a low ELO because his fellow teammates are constantly actual low skill players. Vise versa, a low skill player can be bumped up to such a high ELO bracket, that even his death does not effect the outcome of the battle. (I was there. It took a month and a lucky 10 game LOSING streak to get be back to my bracket.) The only way out of "ELO Hell" is pure luck, and that luck is harder to find the more players are on a team.

ELO isn't perfect, but it is sadly the best system that exist right now. Like it or not, we have to use MATH to code the matchmaker. If the computer cannot math it, then it isn't an option.
How would you calculate the "Getting shot at on purpose" stat?

Edited by DarkonFullPower, 26 May 2014 - 10:06 PM.

[Nothing Whatsoever]

DarkonFullPower, on 26 May 2014 - 09:03 PM, said:

How would you calculate the "Getting shot at on purpose" stat?

I wonder if could be like the Savior Kill, or the Liquid Metal achievement?

I happen to have a thread in the Feature Suggestion area, related to this idea too.

[Goose]

Karl Berg, on 26 May 2014 - 01:07 PM, said:

- Yes, an Elo is calculated for groups. We use this calculated value (currently the average Elo of all team members, but I want to run analytics to determine whether or not this is actually a good metric to use) to match skill levels between groups, the same as we match skill levels between solo players.

I ashore you, Sir, that Averaging is Bad.

You could square the Elo of each player on the team, then root the sum with an index equal to the head count (+1? -1?), or you could just use the highest Elo in the group for all members. Boosting a team Elo by something like 25% would probably not work, ever: Too high or too low, it'd just be off.

The last time Battle Value was worked on by Teh Powers That Be, they figured out how averaging was a source of Shenanigans in that system …

… I trust I'm not repeating a point in this massively informative thread, and think you for it.



Er: Maybe just cube, then cube-root them?

Edited by Goose, 27 May 2014 - 05:37 PM.

[Hammerhai]

All of the posters above. Thanks for giving some more insight into the maths of ELO. It is actually very informative. And may save Karl a post ....
Particularly the posts by Wintersdark made things a lot clearer.

[Modo44]

Kageru Ikazuchi, on 26 May 2014 - 08:51 PM, said:

And, as I understand it, if based on your current Elo, you're ...

• Greatly favored to win, and you win ... minimal change up; lose ... down hard
  
• Slightly favored to win, and you win ... small change up; lose ... down slightly
  
• Even match, and you win ... normal change up; lose ... normal change down
  
• etc.

FTFY. Not adding or dropping values for consecutive predicted wins/losses would skew future results by making everyone seem closer in skill than they actually are. You can only stop moving after literally reaching Elo_max or Elo_min. And if those are reached by many players, then you have a balance problem (obvious winning strategy breaking the system from the outside).

Edited by Modo44, 27 May 2014 - 04:07 AM.

[Kageru Ikazuchi]

Modo44, on 27 May 2014 - 04:04 AM, said:

[/list] FTFY. Not adding or dropping values for consecutive predicted wins/losses would skew future results by making everyone seem closer in skill than they actually are. You can only stop moving after literally reaching Elo_max or Elo_min. And if those are reached by many players, then you have a balance problem (obvious winning strategy breaking the system from the outside).

Thanks ... I'm certainly not an expert (I've just been mostly paying attention for the past year or so).

I'm pretty happy only having a very general idea how it works, since it is one of the things that could actually be used to game the system in the right circumstances and in the wrong hands.

[Modo44]

You pretty much described the algorithm. The only note was that there is almost always a move -- if small in certain conditions.

[Heffay]

Dear Mr Berg,

Recently you stated that the game creates "several tens of thousands" of matches a day. There is some debate on another forum that you are referring to person-matches, so 1 match really means 24 people in that number you referenced (i.e. divide that range by 24 to get the actual number of matches you spawn a day). Can you clarify if you meant matches, or person-matches when you gave that number? Thank you!

Love,
Heffay

Edited by Heffay, 27 May 2014 - 10:48 AM.

[Goose]

Wintersdark, on 26 May 2014 - 07:18 PM, said:

1) Small sample size. Without lots of matches, individual matches have too much weight. This is why, incidentally, we don't have per-chassis Elo scores - the more scores we have, the more random they are. We could have a better overall score if we only have one Elo rating rather than 4, but there's a strong argument to be made that play between weight classes is very different, and players can be very good at one and terrible at another (I'm terrible at lights, excellent in assaults, for example). Have to pick your poison with this.

I suspect there is a mistake, here, in worrying about sample size: There are builds and chassis that work better for a given meta, and the Elo for such player-build combos will be clarified with use. I believe commentary regarding the Raven-3L vs. her sisters covers this …

I wonder what Teh Devs know about Smurfing that we don't, seeing as how one starts the game at mid-Elo, with a 200-point crutch.

Elo is de jure: A trailing indicator …

[Modo44]

The basic idea behind Elo is not to start near the correct rating, but to converge on it quickly. Karl said himself that values converge pretty fast in MWO. On the other hand, we have multiple examples of too general Elo tracking breaking the matchmaker (switch from a [insert your best mech in class] to [insert the worst in class] to see it first hand, multiply that random -- from the current matchmaker's point of view -- error by the number 24, understand that this happens in every public match, cry). We know we need more value separation, and it appears that the Elo system has spare capability. It is high time to use it.

Yes, tracking more values also has potential issues, but those are precisely what Elo is set up to deal with. Every new chassis/variant would start potentially far from its correct Elo value, but we can make it converge very fast. This is superior to the current setup which guarantees incorrect values for all but the most often played mechs in each weight class.

Edited by Modo44, 27 May 2014 - 12:46 PM.

[Cimarb]

Wintersdark, on 26 May 2014 - 08:47 PM, said:

Spoiler

All good in theory, useless in practice. As I said in my post, all those stats are not necessarily (or even usually) reflective of useful contribution. Its virtually impossible to quantify - more on this below.

Elo is thought of as pilot skill, but it is not pilot skill. Elo is a measure of how much you contribute to victory on average.

That was a major part of my post above thou seem to have missed. Elo is largely irrelevant to individual matches - you may normally do really well, but today you're piloting some ridiculous Flamer Battlemasters. If you do that frequently, your rating will alter to take that into account. Its an adaptive system, and that's its greatest strength. It doesn't require constant tuning or balance, and its effectively impossible to game.

What about a battle value and stat based pilot skill combined rating? In theory craft, it'd be perfect. The best system, hands down.

In MWO, its impossible to such a degree its useless to even discuss. It would be a massively complex balancing nightmare, and would be very easy to exploit.

Why? Because its not good enough to just come up with values for stats, you need the relationships between those values to work too. You need to have the relationship between the value you place on a PPC to make sense relative to the value you place on an assist (remember, too, random splattered assists in a match that don't actually contribute, etc).

You can't use tabletop values. Weapons in MWO have no relation to their tabletop counterparts because the game mechanics are so different, and wrong values would be counterproductive.

Basically, if your values in BV(and PS for that matter, but I'll just focus on BV here) are incorrect, then you get what amounts to a matchmaker failure in every single match always... But one controllable and manipulatable by enterprising players.

So, you'd need vast devtime to build up good BV starting points... Then, you'd need to constantly invest substantial dev time into maintaining those values as the game mechanics and weapon/mech stats change. An online game like MWO is very much a moving target. And that's all assuming whoever was in charge of developing and maintaining those values actually does a good job.

Its awesome in theory, and impossible in practice.

"Elo is thought of as pilot skill, but it is not pilot skill. Elo is a measure of how much YOU contribute to victory on average." (caps mine)
- True, but Elo doesn't show that. It shows how good or poor your luck with groups has been, on average, across your lifetime. Being hated by the RNG Gods, I can tell you that my luck is HORRIBLE. I can contribute in every way possible, but if I get teamed up with horrible teams over and over and over again, my Elo is going to be abominable despite everything I contributed (or not).

It has nothing to do with my skill, nor my contribution. If I am a member of a 4-man team for my first six months and sit on the spawn point while they win over and over and over again, despite my complete lack of contribution, my Elo will be exactly the same as theirs. That is something that is happening all over the place currently, btw (just not to that extreme).

As far as BV, it is far less complex than 98% of what Karl does. BV is simply assigning a value to the combination of weapons and equipment on the mech. Since all mechs use the same equipment in different quantities, even though there are immense numbers of combinations, it is a very simple calculation once you get initial values, and it would only need changed when a weapon or piece of equipment was changed, or for fine-tuning balance (if that was even needed).

You could use TT values to begin with, actually, since the stats for weapons are relatively the same. You could do some tinkering to take into account FLD and other MWO-only issues, but those would be minor factors.

Much like the heat scale, using the system that has been used for decades is a good start and should be used as a benchmark for any system built off of it. Only when that system has been tried and found unusable should a completely new system be attempted.

What you mention about "manipulable by enterprising players" is not a fair argument, as you can currently do the exact same thing with Elo - just go into a match and disconnect or, even worse, TK a team member or two, and you have now lowered your Elo. Do that often enough (but not often enough to be obvious), and you can keep your Elo in the bottom bucket quite easily. I'm not sure why you would do that, but same thing as what you are saying people will do with BV.