277 replies to this topic

[PhoenixFire55]

Deathlike, on 25 April 2013 - 05:04 PM, said:

You're saying that a player's contribution to teamwork is different from a player's contribution through skill? What an understatement!

Here is an example. An absolutely no-skilled player in an Atlas goes into the middle of the enemy team right away and dies horribly but ... his teammates are smart enough to use it and while everyone is shooting the Atlas they are significantly softening up the enemy. Mr Atlases skill is zero, same time his contribution to the team is great.

Another one. 7 skilled players from one team are killed by 8 skilled players from the other team in assault mode on a big map like Alpine. 8th no-skill player from the first team goes for cap and successfully does it. His skill is zero, his contribution to his team is great.

Teamwork and pure skill are different things. Two average players can be far better in terms of teamwork just because they understand their weaknesses and agree to work together to compensate them while same time very skilled players can fail to work effectively together just because 'I am so awesome I am gonna do it my way'.

[Hauser]

Krzysztof z Bagien, on 25 April 2013 - 04:26 PM, said:

I meant skill of individual player - the more people plays in single match the less relevant individual contribution is on average, I think that we can agree on that.

Fair enough. I was mostly just picking on that because you were separating a player individual skill from the gear he was brining. I can agree that in a larger team a players individual contribution is less important.

What bothers me about this is that some people (not you actually) jump from the idea of their contribution being less effective to the idea that their contribution is irrelevant to the win. They have the idea that because their team is always so bad they can't improve their rating and are unjustly forced to play with bad players.

This is still wrong though.

Krzysztof z Bagien, on 25 April 2013 - 04:26 PM, said:

I really can't fully articulate my concerns with Elo system in English as it would require some mathematical terminology I only know in Polish and I'm not sure I can properly translate that without creating more confusion.

Can you try looking up the terms in Polish wikipedia and then switch to the English version? I've read the paper and I can see what you might be going after but it'd help if you can point it out.

Krzysztof z Bagien, on 25 April 2013 - 04:26 PM, said:

Here you can read about TrueSkill (Bayesian skill rating system which can be viewed as a generalisation of the Elo system used in Chess) developed by Micro$oft for their multiplayer games. Not like I'm great fan of them, but that work looks really solid from mathematical point of view.

It says:

Multiplayer online games provide the following challenges:
1. Game outcomes often refer to teams of players yet a skill rating for individual players is needed for future matchmaking. (and that's the main problem I see with Elo system being used in multiplayer games)
(...)
TrueSkil (...) addresses both these challenges in a principled Bayesian framework.


There's also a comparison to Elo system (they run tests in Halo 2 beta) - for 8vs8 games Elo system was inacurate in almost 40% of the games.
In fact Elo system was less acurate than Bayesian even in 1vs1 games!

As I understand it PGI implemented Elo system pretty straightforward, without any significant modifications. That's why I belive it can't do any significant change to matchmaking quality (even if it was implemented properly, and we can't be sure about that). Also, we have quite small playerbase (100k? maybe slightly more players total, and no more than 30k - 40k players at any given moment; and no, I don't have any data to prove it, it's just an educated guess) and in many cases there won't be enough players to be matched properly (skillwise) as tonnage would be (or is it already?) one of the factors matchmaker uses.

I hope I made myself clear

Edit: "I belive it can't", not "it can". One shouldn't think of solving complicated mathematical problems at 3AM.

Edit 2: also, funny fact - TrueSkill is patented. You can actually patent mathematical formula

That was an interesting read. Thank you.

For anybody else that doesn't want to chew through a paper, here are a few friendly explanations:

• https://research.mic...ll/details.aspx
  
• http://www.moserware...your-skill.html
  
• http://forums.epicga...Skill-Explained

The major things I took away from it is that:

• TrueSkill is based on Glicko which is based on Elo.
  
• Like Elo the TrueSkill rating is only based on wining or losing the game. Skill in this context is also defined as the probability of winning.
  
• The major improvements are in the win estimation factor. When estimating teams with an equal skill mean and variance the system actually reduces to elo.

I think #3 is especially important because the win estimation factor determines how much elo you your are awarded for a win or loss. Improving this would make the system converge faster.

The results in the paper are interesting but I wonder how they'll compare. The paper mentions that for 4vs4 the TrueSkill system did not do significantly better then Elo. The researchers bring up the argument that the most played gamemode is capture the flag which violates the additive performance model used. I think the same can be said about MWO as most games are a form of capture the flag(s).

Looking at the results also shows something far more important. The matches on which TrueSkill and Elo were evaluated were produced by a match maker. So they are already quite equal teams. Both TrueSkill and Elo correctly predict the outcome in 70% of all these matches. Only when given matches that the other system thinks would be equal TrueSkill seems to be better. This is somewhat expected as it takes more data into account then Elo.

So the results regardless of which is better at least validates the general principle of using a match maker and ranking system based on wins and losses as an effective solution. That was my starting argument and I'll consider it sufficiently argued by now.

edit:

Going by the command chair posts, the matchmaker takes tonnage and elo into account. When it can't find players it will relax those requirements. I don't know if it relaxes them both equally or favours the more tonnage less elo and less tonnage more elo directions.

Edited by Hauser, 26 April 2013 - 06:16 AM.

[Deathlike]

PhoenixFire55, on 26 April 2013 - 12:05 AM, said:

Here is an example. An absolutely no-skilled player in an Atlas goes into the middle of the enemy team right away and dies horribly but ... his teammates are smart enough to use it and while everyone is shooting the Atlas they are significantly softening up the enemy. Mr Atlases skill is zero, same time his contribution to the team is great.

Another one. 7 skilled players from one team are killed by 8 skilled players from the other team in assault mode on a big map like Alpine. 8th no-skill player from the first team goes for cap and successfully does it. His skill is zero, his contribution to his team is great.

Teamwork and pure skill are different things. Two average players can be far better in terms of teamwork just because they understand their weaknesses and agree to work together to compensate them while same time very skilled players can fail to work effectively together just because 'I am so awesome I am gonna do it my way'.

For the record, I wasn't hating on the idea.. it depends on the circumstances and to some degree, the end result.

It's understandable when a crippled mech (no weapons, but still mobile enough) goes capping or use themselves to be a shield for a time. These don't go unnoticed..

Although, I don't know why a defender heading back to base to "stop/slow the cap" gets no bonus for doing so (especially on their death).

[Devil Fox]

So PGI when are you going to open your eyes?! ELO matchmaking still isn't working... I've either had stomping or as is more the case, getting the nub players and getting stomped by pre-mades. I've only had 5 close games since this tournament begun, most of these stomps are resulting from putting newer players into the 1300 elo bracket where they are out-classed... I love a dragon stareing at me with arm lock on because their easy meat. If it isn't the nub players then it's the balance, I've had some one-sided battles...

So when can we expect weight matchmaking for the pug queue to be implemented... we didn't have it for 8man queue before when we had it for pugs, so separating the 2 systems should still be applicable... so when are we actually going to see an improvement in the ELO?

[MasterErrant]

I gotta syy that while I see the skill matching and shorter wait times I'm not seeing much inprovement in the weight matching. though I imagine that sime light mechs lost a big chunk of their invunerability, lost of players have moved away a little in conquest especially weight balance it important.

[Hauser]

MasterErrant, on 29 April 2013 - 12:03 PM, said:

I gotta syy that while I see the skill matching and shorter wait times I'm not seeing much inprovement in the weight matching. though I imagine that sime light mechs lost a big chunk of their invunerability, lost of players have moved away a little in conquest especially weight balance it important.

The the overall weight balance has been pretty good actually. What does happen occasionally though is that one team gets Atlases and Jenner while the other has Centurions and Stalkers. Or something similar with equal tonnage.

In this scenario the team without the lights has to realize the don't have any lights and should push to crush the slow guys fast. After that they can shut up and defend points.

[Dukarriope]

I thought elo was supposed to result in equal percentage of win and losses? I'm seeing 90% lose ratio now with my score topping the board 95% of the time. And I've seen several cases of gross out-tonnage.

Edited by Dukarriope, 30 April 2013 - 11:58 AM.

[PhoenixFire55]

Hauser, on 30 April 2013 - 02:09 AM, said:

In this scenario the team without the lights has to realize the don't have any lights and should push to crush the slow guys fast. After that they can shut up and defend points.

You mean whats left of them (if smth left at all) is gonna spread out and die one by one because they are all cored and that jenner is fully fresh?

[Hauser]

PhoenixFire55, on 30 April 2013 - 02:22 PM, said:

You mean whats left of them (if smth left at all) is gonna spread out and die one by one because they are all cored and that jenner is fully fresh?

If you can't take out 4 Atlases with 4 Stalkers and 4 Centurions you have another problem.

Dukarriope, on 30 April 2013 - 11:50 AM, said:

I thought elo was supposed to result in equal percentage of win and losses? I'm seeing 90% lose ratio now with my score topping the board 95% of the time. And I've seen several cases of gross out-tonnage.

Are you losing 90% of all your games or is your win/loss ratio 0.9?

When and what are you playing? What are you playing in? There can be a few reasons.

Edited by Hauser, 30 April 2013 - 02:40 PM.

[Devil Fox]

Well without weight based matchmaking ELO is still giving me 1-2min wait's followed by such an unbalanced team of both skill level and tonnage. Had 4 pug drops, 1 was 120+ mismatch to the enemy, the 2nd was 220t to the enemy, 3rd was unskilled players (2.5min wait on a game), then my 4th was 175t difference.

Honestly why am I bothering to play? I just keep getting long queue's for a game which gives me little return but dc'ing trial mechs or players that I have to carry. Or it gives such heavy bias of tonnage ranging from 100-200t in weight difference... one or 2 players might be able to prop 'average' players but when half your team don't effectively do anything it makes you wonder how this ELO system is ACTUALLY improving a players games (e.g 2 HGN that did 'ZERO' damage and 3 more that sub-par 100dmg, whilst propped by 2 400+ dmg players)... I don't see any benefit of it towards the spectrum's of the 'bell-curve' because it keeps giving shoddy games.

So when will 'weight-based' matching be introduced to the PUG queue? Not the 8's or 12's! but the queue that actually needs some sort of internal balance mechanic. Because right now gameplay experience is mediocre outside of 4man's, due to the multiple bugs, crashes, and the poor implementation of any match balancing mechanism to date.

[Dukarriope]

Hauser, on 30 April 2013 - 02:28 PM, said:

Are you losing 90% of all your games or is your win/loss ratio 0.9? When and what are you playing? What are you playing in? There can be a few reasons.

My W/L for the last week is probably only 0.2. And basically it's no matter what I take I end being the last survivor or something.

Edited by Dukarriope, 01 May 2013 - 09:41 AM.

[Dasein]

I like that is now Gaussian as it should be and not that freakish anomaly distribution. And this means you doing it right.

[Kahoumono]

I don't fault their math about these statistics but how can we trust those numbers when there are disconnects and bugged players every game. I would say I see 8v8 matches about 10% of the time. I guess you could say disconnects are random white noise but I am sure they aren't because having a better connection or better hardware one should disconnect less. A statistic I'd like to see is the number of players with >0 dmg per game. At the moment I think thats way more relevant.

[Symbiodinium]

Still seeing some matches that are wildly unbalanced in terms of weight. Teams A/B in my last match:

Assault: 3/0
Heavy: 3/2
Medium: 2/1
Light: 0/5

Total weight differential was 250-300 tons. I was pugging, don't know about the rest.

[Ari Dian]

Thontor, on 20 April 2013 - 01:01 PM, said:

Know what's not cool? dropping into Alpine Peaks in Conquest and having only heavies and assaults on your team, when the other team has three lights.

It seems like this happens every time I get that map/game mode combo... and every time, the other team is the only one that has light mechs....

there's pretty much no chance of winning against that. Would really like weight class matching to be back...

No chance in PUG without communication maybe.

2 left, 2 right, 1 left at base. Last 3 in the middle. You dont have to hold all 5 bases, only 3. And with the 3 Heavy/Assault in the middle, you can react fast enough to move wherever their assaults are going. After all you outweight their team.

BUT this will never ever work in a PUG game. Because you cant coordinate the team well and fast enough for it.

[Duncan Aravain]

Well, ELO still borked even after I have done 5.1k matches, admittedly only a part of this large number under the newest ELO program. Just finished the match losing 8 to 2. I recognized 3 players on the other side as tourney/good players; no one on mine. No obvious signs of premades except for a couple mech pairs. Match starts and ends in 5.5 minutes. On my side, 6 out of 8 have double digit damage and 0 kills. I do much better, but obviously not enough. Other team has anywhere from 175-650. How in the heck can the darn program have a great 8/7 match one time and then present an 8/2 or worse slaughter the next? Has the player pool shrunk so badly that the computer has to just grab any available mech to make a match? *sigh*, the Beta test continues.

[Wieland]

The pure weightclass matchmaking before ELO definitely was more balanced.

[PhoenixFire55]

Elo matchmaker ... really close game for you ...

[BFett]

I was wondering what specifically affects my ELO?

Is it the numbers of kills per match?
If my team wins?
If I do an action that grants me C-Bills/XP in the match?

Also, does the match maker look something like this? Where the game weighs the mech Battle Value and pilot skill (ELO) together for a more balanced game.

         ELO    BV   Mech    Player score
Player 1 1200    708  RVN-3L    1908
Player 2 900    1130 CN9-D      2030
Player 3 2100   1325 CTF-3D     3425
Player 4 950    1344 CTF-2X     2294
Player 5 1450	1812 AWE 9M	3262
Player 6  800    708 RVN-3L     1508
Player 7 1600   1850 Atlas D-DC 3450
Player 8 1800   1399 CPLT-C1	3199
Average  1350    1284.5         2634.5

Edited by BFett, 14 May 2013 - 04:49 PM.

[PhoenixFire55]

BFett, on 14 May 2013 - 04:33 PM, said:

I was wondering what specifically affects my ELO?

Winning against stronger opposition / losing to weaker opposition (in theory) strongly affects your Elo score (and everyone on your team), regardless of how well you do in a game damage-wise, kills-wise etc.

BFett, on 14 May 2013 - 04:33 PM, said:

Where the game weighs the mech Battle Value and pilot skill (ELO) together for a more balanced game.

		 ELO	BV   Mech	Player score
Player 1 1200	708  RVN-3L	1908
Player 2 900	1130 CN9-D	  2030
Player 3 2100   1325 CTF-3D	 3425
Player 4 950	1344 CTF-2X	 2294
Player 5 1450	1812 AWE 9M	3262
Player 6  800	708 RVN-3L	 1508
Player 7 1600   1850 Atlas D-DC 3450
Player 8 1800   1399 CPLT-C1	3199
Average  1350	1284.5		 2634.5

These BV's are for stock configs, thus irrelevant for non-trial mechs.