91 replies to this topic

[Xyroc]

Can we let something happen before it is complained about ? please!

[Fabe]

Znail, on 01 February 2013 - 09:43 PM, said:

This was a large problem in HoN, that has a lot more players then MWO as well. The thing is that they few at the top of ranking couldn't get random matches at all as the odds that a two full random teams of that level tried to match up at the same time was too low. The only way to get a "random" high level match in the end was to do a 'sync drop' of two full teams. This of course ment that most of the time so would high level players be playing smurf accounts, wich made one wonder what the idea behind ELO is?

Another killer issue is one that would most likely have made me stop playing the first day, if they had an ELO back then. I understand that lots of people are looking forward to not having to play in the same team with as many bad players in the future. But consider those new people who just join the game. They have to look forward to first playing the game with Trial mechs, watching their ranting plumet and then slowly try and crawl their way back up again to eventually, maybe get a match where not everyone on their team sucks. How is that for 'fun' introduction to the game?

Well if the ELO accounts for individual performance like it should then I think that last part will not be too much of a issue really. New players will be match with new players at the start but as a individual player gets better the elo/mm should advance them to games with better players.

[Fabe]

Also I think the ELO system is not going to be prefect right from the start . We're still going to get some mismatches at first but I think the more we play under the new system the more info PGI will have to better adjust the numbers . Over time the ELO will get more accurate and we'll all get better matches.

Edited by Fabe, 01 February 2013 - 10:12 PM.

[p4r4g0n]

Fabe, on 01 February 2013 - 10:05 PM, said:

Well if the ELO accounts for individual performance like it should then I think that last part will not be too much of a issue really. New players will be match with new players at the start but as a individual player gets better the elo/mm should advance them to games with better players.

Actually, since the intent is to start everyone with a 1300 rating including new players, Znail may be right in saying that they may initially plummet. However, this very much depends on the matchmaking parameters (e.g. near equal number of trial mechs on each team) that PGI may build into Phase 3 MM.

So unless PGI makes a huge oversight, a new player should have a reasonable chance of dropping with experienced players in relatively balanced teams as opposed to 8 new players on one team vs experienced 1300 rated players on the other.

On the other hand, if PGI overlooks this then pugstomping (personally I prefer the term roflstomping) i.e. wipeouts between hugely mismatched teams will still happen at this level.

[Fabe]

p4r4g0n, on 01 February 2013 - 10:16 PM, said:

Actually, since the intent is to start everyone with a 1300 rating including new players, Znail may be right in saying that they may initially plummet. However, this very much depends on the matchmaking parameters (e.g. near equal number of trial mechs on each team) that PGI may build into Phase 3 MM.

So unless PGI makes a huge oversight, a new player should have a reasonable chance of dropping with experienced players in relatively balanced teams as opposed to 8 new players on one team vs experienced 1300 rated players on the other.

On the other hand, if PGI overlooks this then pugstomping (personally I prefer the term roflstomping) i.e. wipeouts between hugely mismatched teams will still happen at this level.

True,if every one start a 1300 then a new player might find them selves dropping in the ELO rankings , The good news is that they should quickly find them selves facing players of equal or lesser skill so their ELO should start to level out to the right spot from there.

[Karl Berg]

By the time we launch the ELO system, all the various games you guys will have been playing on production will have already seeded your ELO values, so the system won't be going live with everyone at 1300.

Also, right now you guys are playing with a matchmaking system that has no concept of skill ratings at all, it's basically random. We're certainly hoping this new system does better than random, so it should be a significant improvement for you guys.

The ELO ratings themselves are very unbiased, it's concerned only with your probability of winning based on your past history, and is recalculated based on the result of every game you play. The new matchmaker however cares about more than just ELO. It's examining mech weight classes like the old system did; game mode preferences, how long you've been waiting for a match, your ELO, and a few other factors to try and create balanced teams. We'll still be tweaking this whole system even after it's been launched to improve it even further.

The combat score which has been talked about elsewhere is an entirely separate concept from your ELO. The combat score is kind of like a 'match score' for a single game to try and give you an idea of how well you did. We're not currently persisting the match scores, and I suspect it will get tweaked quite a bit over the next few patches.

[Thirdstar]

Karl Berg, on 01 February 2013 - 10:43 PM, said:

By the time we launch the ELO system, all the various games you guys will have been playing on production will have already seeded your ELO values, so the system won't be going live with everyone at 1300.

Also, right now you guys are playing with a matchmaking system that has no concept of skill ratings at all, it's basically random. We're certainly hoping this new system does better than random, so it should be a significant improvement for you guys.

The ELO ratings themselves are very unbiased, it's concerned only with your probability of winning based on your past history, and is recalculated based on the result of every game you play. The new matchmaker however cares about more than just ELO. It's examining mech weight classes like the old system did; game mode preferences, how long you've been waiting for a match, your ELO, and a few other factors to try and create balanced teams. We'll still be tweaking this whole system even after it's been launched to improve it even further.

The combat score which has been talked about elsewhere is an entirely separate concept from your ELO. The combat score is kind of like a 'match score' for a single game to try and give you an idea of how well you did. We're not currently persisting the match scores, and I suspect it will get tweaked quite a bit over the next few patches.

Crap. I sure picked a terrible time to stop playing then.

Love the explanation BTW, but I don't suppose you could let us know when you started tracking stats for ELO?

[p4r4g0n]

Thank you very much Karl for the explanation, I really appreciate it. May I suggest including this explanation / clarification in the Command Chair thread on MM?

Otherwise, it will get lost pretty fast on the forums like many other valuable / informative comments posted by Devs in the past in response to other discussions.

[Mackman]

The main thing I'm seeing in this thread is

Some Random Guy said:

Maaaaan, if PGI institutes ELO without thinking about it even a little bit, there's going to be all these super-obvious problems with it!

A common objection is: Newbies with Trial Mechs tanking their rating. This is reeeeaaaaally easily solved with two options right off the top of my head:

• A "newbie island" approach, where players with less than <X> matches (let's call it 25, since that's when the Cadet Bonus runs out) are only matched with other players with less than <X> matches.
  
• Or just not letting people with fewer than <X> matches have an Elo!

Either of these, with a bit of tweaking, would completely eliminate this problem.


Then, I admit it, there are other problems that aren't so easily solved. Take Douchey McD*ckbag, who decides that today he's going to create a brand new account and stomp all over the newbies. There's one really important thing we need to realize, though:

That's already happening right now, and Douchey McD*ckbag doesn't even have to create a new account to do it.

But there are ways to mitigate this once Elo appears.

• Have some sort of system in place to detect when players in Newbie Island are performing much better than the average newbie (should be fairly easy to calculate). If that threshold is crossed two or three games in a row, they get kicked out of Newbie Island and immediately placed at their appropriate Elo.

That's just one example, but it has the potential to be extremely effective. If Douchey actually pubstomps, then he get's placed immediately at his appropriate Elo. If he holds back, then sure, he's doing a bit better than the average newbie, but not enough to dramatically ruin the new player experience for everyone else.



Will there be problems with the Elo system? Almost definitely, at first. But when you're suggesting a problem, suggest a solution as well. Don't just use it as a reason why PGI should just abandon Elo entirely.

Edited by Mackman, 02 February 2013 - 08:57 AM.

[Karl Berg]

Thirdstar, on 01 February 2013 - 10:47 PM, said:

Love the explanation BTW, but I don't suppose you could let us know when you started tracking stats for ELO?

With the patch going out this Tuesday we'll start saving ELO values on the production servers.

p4r4g0n, on 01 February 2013 - 11:06 PM, said:

May I suggest including this explanation / clarification in the Command Chair thread on MM?

Otherwise, it will get lost pretty fast on the forums like many other valuable / informative comments posted by Devs in the past in response to other discussions.

It's a decent suggestion, I'll talk with Bryan about it when I get a chance. A lot of the systems work doesn't get much coverage, so there could be some value in communicating a bit more about what we're doing and why.

[Vaneshi SnowCrash]

Smegmw, on 31 January 2013 - 02:32 PM, said:

High ranked players generating low ranked characters/accounts to engage in newb stomping.

Smurfing is the word you were looking for.

ParasiteX, on 31 January 2013 - 02:39 PM, said:

New Stats system.

I agree that they'll have to rejig the metrics being used to generate ELO for optimal placement of people... what if they don't? How would that one play out?

SinnerX, on 31 January 2013 - 02:56 PM, said:

Does anyone else think "future problem" topics are worse than "I'm quitting" topics?

No. In fact even if PGI were able to achieve Valve like productivity-per-person there would be one group that could out generate and out permeate them: us.
Running the numbers and stomping the ground to get a rough consensus of what feels 'right' is something PGI can't do at the speed the player base can.

If you don't like it: Can I haz your stuff?

[Znail]

Karl Berg, on 01 February 2013 - 11:26 PM, said:

With the patch going out this Tuesday we'll start saving ELO values on the production servers.

It's a decent suggestion, I'll talk with Bryan about it when I get a chance. A lot of the systems work doesn't get much coverage, so there could be some value in communicating a bit more about what we're doing and why.

Comunication is nice. I do hope that you can point Bryan to this thread as while I don't need a reply from him so would it be nice to know that he have considered the potential problems that can show up. Some of them may seem obvious, but they are not obvious enough that several other companies have ELO systems that fails to consider them.

Lets add one more to the list of potential issues. If it's possible to get a worse rating then one starts with, then expect some people to intentionally tank their rating starting tuesday, just to get to play easy matches later. And the easy way to tank your rating is to suicide lots of games and I hope I don't have to remind people of the plague that used to be.

[Thirdstar]

Bumping this thread cause the information from Karl is far more important than 99% of the bickering on general.

[Mackman]

Znail, on 02 February 2013 - 12:15 AM, said:

Lets add one more to the list of potential issues. If it's possible to get a worse rating then one starts with, then expect some people to intentionally tank their rating starting tuesday, just to get to play easy matches later. And the easy way to tank your rating is to suicide lots of games and I hope I don't have to remind people of the plague that used to be.

I'm assuming PGI will have a system in place to catch things like this. Is Player X showing a distinct pattern of really poor performance followed by really great performances? If they simply suicide, it will be even easier for PGI to catch them using a purely automated system.

It all comes down to numbers eventually, and with all the numbers the new matchmaker is going to take into account, it should be fairly easy to spot anomalies. The pubstompers will stand out both in their losing and in their winning.

Now: Will it still happen? Yes, as long as their are D*ckbags playing this game, there will be people trying to drop their Elo down to pubstomp. That will happen with any kind of actual matchmaking PGI puts in, though, and I think we can agree that a few people trying to pubstomp is better than the purely randomized stomp-fests we have every other match here. I also think that PGI will be able to minimize the impact these people have on the game as a whole.

Znail, on 02 February 2013 - 12:15 AM, said:

Some of them may seem obvious, but they are not obvious enough that several other companies have ELO systems that fails to consider them.

My first post was too harsh, and that was my bad. I've played LoL for three years, and I've seen those developers put this kind of thing into play. I can only assume that PGI is taking several pages out of Riot's playbook, considering that LoL's Elo system certainly hasn't harmed the popularity of the game.

I'm annoyed not by the suggestions themselves, but by certain posters implying that because such a potential problem exists, it would be better to abandon Elo altogether, And when the problem itself is so simple to address, that kind of attitude bothers me.

[Hekalite]

I'm not sure how easy it will be to intentionally tank your ELO considering this (from Paul's thread):

Quote

A player’s rating will only go down if they are beaten by a player who has a lower rating than theirs.

The OP's initial assumption that 52 losses will take you from 1300 to 0 assuming an average rate of change of 25 is incorrect because the odds of you always being matched against a lower ranked opponent, which exposes you to a potential ELO loss, are unlikely to be constant. I'm sure there will still be people who persist in griefing others, but the matchmaker determines somewhat randomly (or maybe not) if you even have a chance to lose ELO in a given match. I think that should make it enough of a chore that it is unlikely to be common.

In any case, speculation thread is just that. Speculation.

[Karl Berg]

Mackman, on 02 February 2013 - 08:44 AM, said:

I'm assuming PGI will have a system in place to catch things like this. Is Player X showing a distinct pattern of really poor performance followed by really great performances? If they simply suicide, it will be even easier for PGI to catch them using a purely automated system.

The CSR team already has access to general match telemetry within about 24 hours of a person playing a game. They can also view your current ELO. By the time final matchmaking goes live they should also have access to a players ELO history as well.

Purely automated systems are always a bit risky as they might accidentally trip and punish honest players, which would be a pretty frustrating thing to have happen to you. More likely, if community reporting turns out to be insufficient, we'll set up data mining systems to generate lists of suspicious players for the CSR's to audit.

[Znail]

Mackman, on 02 February 2013 - 08:44 AM, said:

My first post was too harsh, and that was my bad. I've played LoL for three years, and I've seen those developers put this kind of thing into play. I can only assume that PGI is taking several pages out of Riot's playbook, considering that LoL's Elo system certainly hasn't harmed the popularity of the game.

I'm annoyed not by the suggestions themselves, but by certain posters implying that because such a potential problem exists, it would be better to abandon Elo altogether, And when the problem itself is so simple to address, that kind of attitude bothers me.

Your post is a bit amusing as LoL just anounced that they are changing their ELO system into a league system similar to S2, so it can't have been working perfectly or else they wouldn't have changed it.

I can see several advantages with the new system that incidentally deals with some of the issues I have mentioned. The main ones are that you can't fall below the bottom of the lowest league, wich equals a limit to how far down you can get. And that the leagues will even the spread out and make it so the top levels have a healthy amount of people, rather then a bell curve with few at the top.

Just because you like the idea of an ELO system doesn't make someone who doesn't into a person with some kind of criminal attitude. People can disagree with you without being fundamentallly evil people.

Now myself while I would prefeer not to have the ELO system so is my main motive behind this thread to point out the pitfalls of ELO and suggest ways to avoid them. My main reason not to want the ELO is that I quite frankly don't trust the devs to manage to avoid all those pitfalls. It's not really more then what can be expected as Devs are essentially designers or artists and math is not a required skill. There is a reason why it was a Physics Proffessor and Chess master that invented the ELO system and not a Football player.

[Mackman]

Karl Berg, on 02 February 2013 - 12:37 PM, said:

The CSR team already has access to general match telemetry within about 24 hours of a person playing a game. They can also view your current ELO. By the time final matchmaking goes live they should also have access to a players ELO history as well.

Purely automated systems are always a bit risky as they might accidentally trip and punish honest players, which would be a pretty frustrating thing to have happen to you. More likely, if community reporting turns out to be insufficient, we'll set up data mining systems to generate lists of suspicious players for the CSR's to audit.

I figured as much. But how would the community report someone who intentionally tanks their Elo to stomp low-level players? I assume that's where your data-mining comes in.

Znail, on 02 February 2013 - 02:16 PM, said:

Your post is a bit amusing as LoL just anounced that they are changing their ELO system into a league system similar to S2, so it can't have been working perfectly or else they wouldn't have changed it.

I can see several advantages with the new system that incidentally deals with some of the issues I have mentioned. The main ones are that you can't fall below the bottom of the lowest league, wich equals a limit to how far down you can get. And that the leagues will even the spread out and make it so the top levels have a healthy amount of people, rather then a bell curve with few at the top.

Just because you like the idea of an ELO system doesn't make someone who doesn't into a person with some kind of criminal attitude. People can disagree with you without being fundamentallly evil people.

Now myself while I would prefeer not to have the ELO system so is my main motive behind this thread to point out the pitfalls of ELO and suggest ways to avoid them. My main reason not to want the ELO is that I quite frankly don't trust the devs to manage to avoid all those pitfalls. It's not really more then what can be expected as Devs are essentially designers or artists and math is not a required skill. There is a reason why it was a Physics Proffessor and Chess master that invented the ELO system and not a Football player.

Perfectly? No system is perfect: Not the previous Elo system, not their current Elo system. I merely said that many of the "problems" people are bringing up are easily avoided using an Elo System, and that the solutions are evident in LoL's previous system as well as their new one.

Also, they aren't getting rid of Elo. Each player still has an Elo rating, it just isn't shown. I think you have a fundamental misunderstanding of how the Leagues work. It just divided the playerbase into Leagues and removed the visible number, to make it easier and more fulfilling to see whether you were making progress. They're still using Elo to determine where players are placed and how far they move up or down.

In as much as someone tanking their rating can be prevented, Elo can do so. And if I may ask: What is your proposed solution? How should they match people up for the most even/fun matches possible? What possible system can you come up with that doesn't have the same problems?

[Bguk]

Karl Berg, on 01 February 2013 - 10:43 PM, said:

By the time we launch the ELO system, all the various games you guys will have been playing on production will have already seeded your ELO values, so the system won't be going live with everyone at 1300.

Also, right now you guys are playing with a matchmaking system that has no concept of skill ratings at all, it's basically random. We're certainly hoping this new system does better than random, so it should be a significant improvement for you guys.

The ELO ratings themselves are very unbiased, it's concerned only with your probability of winning based on your past history, and is recalculated based on the result of every game you play. The new matchmaker however cares about more than just ELO. It's examining mech weight classes like the old system did; game mode preferences, how long you've been waiting for a match, your ELO, and a few other factors to try and create balanced teams. We'll still be tweaking this whole system even after it's been launched to improve it even further.

The combat score which has been talked about elsewhere is an entirely separate concept from your ELO. The combat score is kind of like a 'match score' for a single game to try and give you an idea of how well you did. We're not currently persisting the match scores, and I suspect it will get tweaked quite a bit over the next few patches.

Karl Berg, on 01 February 2013 - 11:26 PM, said:

With the patch going out this Tuesday we'll start saving ELO values on the production servers.

Thirdstar, on 02 February 2013 - 02:41 AM, said:

Bumping this thread cause the information from Karl is far more important than 99% of the bickering on general.

Hear, hear!!

[Taizan]

Thanks for the "ELO preview" info Karl, we nourish ourselves from such breadcrumbs. Although ELO ranking may have difficulties at a certain critical point (depending on active player base) etc. I think it's still the right approach and may help against the random totally frustrating matches players get (Those that remain memorized for longer times than the good one's).
Changing it to a bracketed league system can always be done if necessary - as seen in LoL and other products with ELO or similar ranking systems.