56 replies to this topic

[Project_Mercy]

MM is one professional Baseball player + 11 T-Ball children vs a University Ball Team. It's amusing once or twice, but afterwards it's just a waste of everyone's time.

Getting rid of it couldn't really do any worse than it currently does. At a minimum it wouldn't feel like you were constantly being set up to fail.

Given all the statistics kept though, I can come up with 2 or 3 algos that rely on PERSONAL stats that would work infinitely better.

Edited by Project_Mercy, 27 February 2015 - 09:31 PM.

[Spr1ggan]

I'm doing the challenge while mastering Ice Ferrets. The pain... The pain!

[Nimbus Captain Brannigan]

hmm if they do just add and divide by 12, elo is bit meh.

Here is a quick table I cooked up. Lets say elo is rated 0 - 10.

|------------player ELO-------| |Team elo| |My personal power level multiplier|
10-10-10-5-4-4-4-3-3-3-3-2 --- 60 ------- 34560000
6 - 6 - 6 - 6-5-5-5-5-5-4-4-3 --- 60 ------- 162000000

This was my cooked up table of adding up. The last numbers I multiplied them, because in Team, I think the skill levels of people should be multiplied, since they enhance everyone's skill not just add to it.

So in this made up example, top lance would win against bottom one, in front, and other 2 bottom lances would win over 2 top ones. Both suffered damage. now its 4 v 8. So you have 2(players)x5(elo)=10 there is 5 of them. so its 10v10 elo (except more guns on bottom) for first 2 10 elo players and 9v10 on last 10 elo player. The 5 elo on top is facing 8 elo combined might. This I think goes favorably in multiple people 2 lances which are still alive.

Edited by ValheruThunar, 27 February 2015 - 09:46 PM.

[Lynx7725]

Heh. You've set up a fairly extreme case to prove your point. I've looked at it in the past and the ELO-based matchmaking isn't that terrible under a particular assumption.

Let's take your pool of players first:

Skill 10: 3
Skill 6: 4
Skill 5: 6
Skill 4: 5
Skill 3: 5
Skill 2: 1

My thinking is that the MM adds people to the teams one by one, by the following routine:

1. Sort players by skill level
2. Add the top player into Team A, at the start.
3. Take the next most skilled player, and then check which team has the lower total ELO. Add into the team with the lower ELO.
4. Repeat from 3 until all players are in a team, or you hit a player limit.
5. Put all remaining players into the "short" team.

From the above, what you would get is:

Team A: 10, 10, 6, 5, 5, 5, 4, 4, 4, 3, 3, 1
Team B: 10, 6, 6, 6, 5, 5, 5, 4, 4, 3, 3, 3

That's taking the same players as your pool, but the end result isn't as bad. The algo is surprisingly robust, and the matchups aren't terrible. This is particularly true of the Solo queue; Group queue has more potential to be lopsided, but group cohesion goes a long way to balance things.

So what's the fail point with the above? Two particular things. One, is that even with a perfect distribution of skillset, there's still going to variables that cannot be controlled. A particular team of random PUGs may end up being particularly good at working together, or they pick a particular route that gave them an advantage under that particular game's circumstances. These can't be easily accounted for in ELO calculations.

The second fail point is related, in that whether ELO actually has any accuracy. ELO may not be an accurate reflection of an individual pilot's skill, since his ELO is affected by his games, both Solo and Group. If a pilot consistently drops in Groups, his Elo is likely to be higher than his actual, individual skill when he drops alone. Those two 10s in Team A? Maybe they always drop with their super-group and was just slumming that day. Say their actual skill level is 8. That creates an imbalance in the teams immediately.

Since ELO is IMO easier to ramp up than to thin down -- you need to consistently lose when you are expected to win in order to reduce ELO -- over-inflated ELO is a problem. In theory, over time your ELO should average out, but for that one instance in time in PUGalandia, you might just be the most elite Mechwarrior in the Inner Sphere...

[SethAbercromby]

I made some code to simulate how easly even perfectly balanced teams can have one side ending with a vastly superior result.

The first version was to simulate an "underhive" pattern. Everyone manages to do damage, but they strike at random targets and the is no cohesion or even commitment to a single target. The field considered perfect balance between the 'Mechs, assuming each having an HP pool of 15 and doing 5 damage each round to a slected target. The results of that I posted in a different thread:

Quote

Okay, the first results are in. First, let me explain the fundamental rules that the simulation followed:

1) Any unit can hit any other unit
2) A unit is guaranteed to do damage in a turn it begins alive
3) Each unit selects a random target each turn

Spoiler

While the numbers showed a very balanced result, they also showed one very important thing: The first kill advantage is decisive. Not a single game was won by a team that ended up one or more kills down after the first round of casulties, the best result being a draw.

For those interested in how round 21 went down (puls how the simulation operated), have a look for yourself

Spoiler

Excuse me for not providing proper statistics this time, but I will take the effort of compiling them again if you guys really want me to though.

The second simulation had the following rules:
1) Any 'Mech can hit any other 'Mech
2) Every 'Mech chooses a random target
3) Every 'Mech remain commited to said target until it has been eliminated
4) Once the target is eliminated, the 'Mech will randomly select a new target
5) Every 'Mech is guaranteed to do damage in a turn it begins alive

The results of this simulation ended up varying greatly. Some runs had 0-1 (11-12 in MWO) results, sometimes I'd end up with 8-0 (12-4 in MWO). The likelyhood of getting results of 5-0 was pretty large. Without statistics, I'd guess the probability of having close results (0-1 to 0-4) were about equal to one side dominating the other (0-5 and higher).

Of course this is a very limited representation of how the game works, but if this can happen in a perfectly sterile environment, even miniscule differences to one teams's advangtage can throw off the scales completely.

[Norbaer HALL]

Tarogato, on 27 February 2015 - 06:46 PM, said:

... The system is working fine under the constraints its given, it's the method of actually determining/affecting individual players' Elos that needs to be reconsidered in my opinion, because I don't think it's working. ...

NO it is the Matchmaker which is faulty !

For example, in the Adder up event I had a streak of 19 losses in a row !
And often people complain about more than 20 losses in a row, which happens much more often than it should, accourding to mathematic.

On the other hand I also had a streak of 11 wins in a row during the Adder up event.
And the most frustrating about this streaks is that they seem to alternate between streaks of losses and streaks of wins !

Edited by Norbaer HALL, 03 March 2015 - 12:52 AM.

[Duke Nedo]

I am guessing that the MM works OK, I mean how hard can it be to divide 24 players between two teams so that a single parameter (Elo) is evenly distributed between the two teams?

However, I have a strong feeling that Elo in its current implementation is a very, very, very poor descriptor of the players ability to drive a win in MWO. Elo in a 12 v 12 team game with different mechs, loadouts, maps, map sides, disconnects, crashes, hit reg issues etc etc, must consist of mainly noise really. So, if Elo is half as bad as I would expect, the MM can't win unless it starts taking more factors into account, like Mech Tiers and ECM. I would very much prefer if they would base the MM on "Average-Reward-2.0-for-the-last-100-games-in-that-class" instead of Elo.

[Brother MEX]

In the Adder up event I didnt play much :

Quote

• ADDER UP EVENT

Starts: FRIDAY FEBRUARY 27^th 10:00:00 AM (PST)
Ends: MONDAY MARCH 2^nd 10:00:00 AM (PST)
Game ID Mech' Win Kills Deaths Kill Assists Qualified Time
318136814679404265 ENF-5DR 0 1 3 2015-02-27 18:07:20
318418865181733218 CTF-3DC 0 1 3 2015-02-27 18:12:36
317292316734829327 CTF-1X 0 1 0 2015-02-27 18:21:23
317292389749274038 KGC-0000 0 1 2 2015-02-27 18:28:24
317292471353653290 KGC-0000 0 3 5 2015-02-27 18:53:58
339247927558979405 KGC-0000 0 4 2 2015-02-27 19:29:35
339248107947606879 CTF-1X 1 1 1 2015-02-27 19:30:14
339248228206691898 CTF-1X 0 1 6 2015-02-27 19:44:37
317856400559621871 KGC-0000 0 1 1 2015-02-27 19:46:23
318419784304740852 CTF-1X 1 0 1 2015-02-27 19:56:38
339248481609763812 CTF-1X 0 1 3 2015-02-27 20:05:32
339248584688979457 CTF-3DC 0 1 1 2015-02-27 20:14:55
317856791401648023 CTF-1X 1 1 1 2015-02-27 20:20:52

I asume you too dont like to play with worse than 75% loss ratio

Edited by Brother MEX, 03 March 2015 - 11:33 AM.

[Raggedyman]

One thing to remember: ELO operates across the board, rather than on an individual players level. So as long as the average player is on a 1:1 win/lose ratio then it's technically "working". What this means is that you can have 5 players on 10:0 and 5 players on 0:10 and that's still technically "balanced". It's also why this



Would be an equal match, even thought it could very rapidly be a 12vs5 / 870 Elo vs 600 Elo fight (and that's before you further complicate it with weight class issues).

So Elo does kinda work overall, however it can at any moment mean you are having a very crappy day.

[Brother MEX]

Added data from my profile of all 3 Like a Champion events to the first post, where I replaced the with a X and the with a J ... maybe I add color to the X and J later to make it easier to read, or replace them with Emoticons like or ( too bad that there are no Emoticons with a red X or a green 1 ) ... or add more data from my profile

PS: I also didnt play much in this event because of my bad win/loss ratio :

Quote

• TO THE VICTORS

Starts: FRIDAY NOVEMBER 21^st 10:00:00 AM (PST)
Ends: MONDAY NOVEMBER 24^th 10:00:00 AM (PST)

Game ID Mech' Win Kills Deaths Kill Assists Qualified Time
487281916801815031 AS7-S 0 1 1 2014-11-23 08:01:51
487282054240769542 CPLT-K2 0 1 7 2014-11-23 08:14:52
487282153025018118 CN9-AH 0 1 1 2014-11-23 08:27:54
486437740979788516 LCT-1E 0 1 0 2014-11-23 08:39:18
487297477468422297 AS7-S 0 1 5 2014-11-24 10:03:51
486454551481894907 AS7-S 1 0 3 2014-11-24 15:11:35

Edited by Brother MEX, 03 March 2015 - 11:22 AM.

[STEF_]

Brother MEX, on 26 February 2015 - 05:55 AM, said:

If I look at the results of the games played in the last challenges ( https://mwomercs.com/tournaments ) it seems that the matchmaker is not fair to me !

Sorry man, but... getting an assist and a kill, as nothing to do with MM, really.

That's why PGI has removed the win requirement: a win "could" be conditioned by elo of players.
But "a scratch and a kill" requirements are all up to you.

[Apnu]

I really don't understand the MM and Elo.

My beloved HBK-4G has 111 matches. 64 wins, 42 losses. 1.52 W/L ratio. Also 123 kills, 66 deaths. A 1.86 KDR. This mech feels "lucky" when I drop in it and 2/3 of the time my team is pretty good.

My HBK-4J has 180 matches (I use it in challenges), 92 wins, 72 losses. 1.16 W/L ratio. 212 kills 102 deaths. A 2.08 KDR. I have to carry like a mofo in this thing, but it works. Half the time my team just stinks.

My Protector has 100 matches. 60 wins, 37 losses. 1.62 W/L ratio. It has 89 kills, 51 deaths. A 1.75 KDR. Like the HBK-4G it feels lucky despite the fact that I run it as a LRM boat half the time and a brawler the other half.

My TDR-5S has 140 matches. 66 wins, 73 losses. 0.90 W/L ratio. It has 118 kills, 81 deaths. A 1.46 KDR. I mostly run it like a LRM boating CPLT-C1 because I didn't get the Legendary Founders pack and so I use the TDR-5S to fill that role and its a key c-bill grinder for me. This mech feels "cursed" No matter what I do, how hard I carry, the MM puts me on Team Derp.

I know, because of luck, my Elo rating is descent in two of the 4 mechs, and the other two it just stinks. No matter what I do, I can't seem to climb out of the Elo hole on those machines.

[STEF_]

Tarogato, on 26 February 2015 - 07:16 AM, said:

I've whined about the matchmaker many times, so here we go, I'll have a go at it again.

Disclaimer: I love and enjoy this game, I play it a lot and my criticism is not mindless QQ, I just really think it could be made to be better.

According to various sources coming from PGI and restated time and time again, this is what I understand:

• Your own Elo rating is combined with your team to formulate a sum or average. It is assumed that the enemy team has roughly the equivalent same sum or average Elo when compared to yours.
  
• Only wins and losses affect Elo.
  
• One team will always have a slightly higher Elo compared to the other and is thus predicted to win.
  
• If the team predicted to win actually wins, nothing happens and nobody's Elo changes.
  
• If the team predicted to lose manages to win, every member of that team gains some Elo, while the team that was supposed to win but actually lost will lose some Elo.

---

What are my specific personal experiences with matchmaker?

It's a joke, or it doesn't exist. I say this very harshly because I know it's a system that PGI has put a lot of effort into and some really brilliant folk have worked on it and PGI continually insists that it's doing its job very well. They can actually watch the matches in progress and it'll show them the Elo of individual players on each team, as well as the combined Elo of both teams, and they are seeing numbers that are very close to the ideal - very evenly matched teams comprising of players all in a very close Elo band. Yes, I'm insulting all of that because I honestly don't believe the metrics are reflecting the gameplay experience.

Time and time again I get put in a team and "oh, hey look, it's ZuFFuLuZ, or rageagainstthedyingofthelight, or The Flying Gecko... Mister D... Marc von der Heide... Tahribator... Aliisa White..." all people who I know are either members of competitive teams or have made it onto leaderboard events, sometimes multiple times. People I see a lot, who have been playing the game for well over a year, and are well invested. Then six minutes later I die for whatever reason and I get to spectate somebody on the team, and WOW are they new. Sometimes they're even in a trial mech, sometimes they're in third person, sometimes they have no situational awareness, sometimes they don't lock targets and have no idea on how to target specific components, like... I mean, both his legs are cored red and you still shoot him in the face and miss? Please. That kind of new. How does this person get placed on the same team as me ... on the same team as aforementioned top-level players? How? Something's broken.

It's esactly how elo MM works, and I hate it.

Because for elo based MM, it's the esact damn thing to put 3-4 comp players along with noobs vs "band of players".
The only thing that matters for such a MM, is that the sums of elos between the 2 teams are similar.
But it's not a balance game in this way!
Let's make a 4vs4 example: 2200+2200+500+500=5400
1350+1350+1350+1350= 5400

does it seem to you a balance 2 lance match?

For me this causes a lot of bad matches, where skilled players are forced to play with noobs and have a frustrating "carry harder" experience, while watching those bunch of incredible noobs for 15 minutes. And noobs thierselves may have a frustrating and disappointing experience while being eated alive by average enemies (1350 in the example) and by good players in team chat.

That's why I always say: MM should be reworked, and MM should match players with very similar elo.
That is noobs team vs noobs team. Average players vs average players, and comp. players vs comp. players.

[STEF_]

Apnu, on 03 March 2015 - 11:28 AM, said:

I really don't understand the MM and Elo.

My beloved HBK-4G has 111 matches. 64 wins, 42 losses. 1.52 W/L ratio. Also 123 kills, 66 deaths. A 1.86 KDR. This mech feels "lucky" when I drop in it and 2/3 of the time my team is pretty good.

My HBK-4J has 180 matches (I use it in challenges), 92 wins, 72 losses. 1.16 W/L ratio. 212 kills 102 deaths. A 2.08 KDR. I have to carry like a mofo in this thing, but it works. Half the time my team just stinks.

Because you are using them well.
BUT elo works putting together all your W and L in your medium class mechs.

Do you have medium in which you do bad?

What matters is W/L inside medium mechs; sum it up. If your w/l ratio of all medium mechs is high, then you have an high elo in that class.

Edited by Stefka Kerensky, 03 March 2015 - 11:49 AM.

[Norbaer HALL]

It is even worse since the last patch !

I have played some games since the patch in mechs from different classes and DIDNT WIN A SINGLE GAME !

[Apnu]

Stefka Kerensky, on 03 March 2015 - 11:49 AM, said:

Because you are using them well.
BUT elo works putting together all your W and L in your medium class mechs.

Do you have medium in which you do bad?

What matters is W/L inside medium mechs; sum it up. If your w/l ratio of all medium mechs is high, then you have an high elo in that class.

I picked those because they are the mechs I've used the most, the rest of them have a low count of games that can be written off as not having enough sample data.

So the Elo is rated on the class of mech? I thought each mech had its own Elo.

Anyway, other mediums I have that have more than 50 matches per.

CN9-AH: 78 games. 33 wins, 44 losses. .75 W/L ratio. 76 kills, 61 deaths. 1.25 KDR
Grid Iron: 77 games. 30 wins, 39 losses. .77 W/L ratio. 63 kills, 55 deaths. 1.15 KDR
TBT-5N: 70 games. 37 wins, 32 losses. 1.16 W/L ratio. 83 kills, 33 deaths. 2.52 KDR
WVR-6R: 57 games. 30 wins, 24 losses. 1.25 W/L ratio. 65 kills, 39 deaths. 1.67 KDR
GRF-1N: 47 games. 30 wins, 16 losses. 1.88 W/L ratio. 51 kills, 25 deaths. 2.04 KDR

All other mediums (17 chassis): 216 games. 101 wins, 108 losses. 0.94 W/L ratio. 159 kills, 149 deaths. 1.07 KDR.

Grand total for mediums (24 chassis): 836 games. 417 wins, 384 losses. 1.09 W/L ratio. 832 kills, 530 deaths. 1.57 KDR.

Therefore, in mediums my Elo is essentially unchanged if Elo only counts for the weight class.

[STEF_]

Apnu, on 03 March 2015 - 01:19 PM, said:

I picked those because they are the mechs I've used the most, the rest of them have a low count of games that can be written off as not having enough sample data.

So the Elo is rated on the class of mech? I thought each mech had its own Elo.

Anyway, other mediums I have that have more than 50 matches per.

CN9-AH: 78 games. 33 wins, 44 losses. .75 W/L ratio. 76 kills, 61 deaths. 1.25 KDR
Grid Iron: 77 games. 30 wins, 39 losses. .77 W/L ratio. 63 kills, 55 deaths. 1.15 KDR
TBT-5N: 70 games. 37 wins, 32 losses. 1.16 W/L ratio. 83 kills, 33 deaths. 2.52 KDR
WVR-6R: 57 games. 30 wins, 24 losses. 1.25 W/L ratio. 65 kills, 39 deaths. 1.67 KDR
GRF-1N: 47 games. 30 wins, 16 losses. 1.88 W/L ratio. 51 kills, 25 deaths. 2.04 KDR

All other mediums (17 chassis): 216 games. 101 wins, 108 losses. 0.94 W/L ratio. 159 kills, 149 deaths. 1.07 KDR.

Grand total for mediums (24 chassis): 836 games. 417 wins, 384 losses. 1.09 W/L ratio. 832 kills, 530 deaths. 1.57 KDR.

Therefore, in mediums my Elo is essentially unchanged if Elo only counts for the weight class.

Then, your medium mech w/l ratio is combined with the other 11 ones, and summed up.

But, here is the trap: you can join a team with average 1.50-1.60 kdr vs a team with similar k/d ratio... OR having 1-2-3-4 mates with 0.50 in it... or even a bunch with 3.0 w/l ratio in a mech class. MM just sums up average elo in both teams, and that's all.

Which is weird , imo.

[warner2]

Tarogato, on 26 February 2015 - 07:16 AM, said:

Time and time again I get put in a team and "oh, hey look, it's ZuFFuLuZ, or rageagainstthedyingofthelight, or The Flying Gecko... Mister D... Marc von der Heide... Tahribator... Aliisa White..." all people who I know are either members of competitive teams or have made it onto leaderboard events, sometimes multiple times. People I see a lot, who have been playing the game for well over a year, and are well invested.

Part of the problem is perception. We don't have the numbers. I don't recognise all of those players but I recognise some of them. Based on what I've seen there is a significant disparity in ability on that list alone, so the fact that you are lumping all of those players into some bracket labelled "top-players" indicates that what you are working on is perception not the actual facts.

The other thing is that actually we can all prove that Elo does work by creating a new account and running some trial 'mechs. In one recent event weekend I was so annoyed by the quality of the matches (which I firmly believe to be down to people being people, not the matchmaker) that I did that for fits and giggles and the stock Nova in the hands of a component player does really well at low (beginners) Elo.

So it works to some degree. I believe it's struggling for a sufficient population to draw from, at mid and above Elos, especially since CW came along.

[Apnu]

Stefka Kerensky, on 03 March 2015 - 02:05 PM, said:

Then, your medium mech w/l ratio is combined with the other 11 ones, and summed up.

But, here is the trap: you can join a team with average 1.50-1.60 kdr vs a team with similar k/d ratio... OR having 1-2-3-4 mates with 0.50 in it... or even a bunch with 3.0 w/l ratio in a mech class. MM just sums up average elo in both teams, and that's all.

Which is weird , imo.

Agreed. I think the MM and Elo are too simple for what players are doing. It doesn't consider mechs or builds. If you have a two teams with roughly the same Elo, but one is stacked with, say, ECM and the other isn't (this happens pretty often to me), the ECM team is "stronger"

Granted I've seen ECM teams get rolled due to bad group decisions, so ECM, alone doesn't mean auto-win, but it can tip the balance if all other things are even. Elo should consider that.

Either that or the MM needs to consider special tech like ECM in the pub. solo queue and distribute it more evenly.

Personally, I'd love some kind of lobby system (we'll see what the LFG is when we get it next month) where players form ad-hoc lobbies then look for other players to round it out to 12 players, then that group is sent off to the MM for a match. By enforcing weight and/or class restrictions on the lobby, it could take a lot of work off the MM's plate and put it on the players.

Or do some kind of hybrid, say groups of 4, 8 or 12 go looking for other groups to glob together and/or head off to the MM for a 4v4 or 8v8 match. In any of those cases, the players are forming groups before the matches, so the social aspect of the game increases and there's less work for the MM and there's better evening of the weight classes.

[GeneralArmchair]

The matchmaker could start recording "Mech elo's" for different chassis. It would act as a rudimentary self-callibrating BV system of sorts as the metagame evolves and the most effective mechs rise to the fore-front.