717 replies to this topic

[Gozuri]

Zerex, on 14 June 2020 - 11:04 PM, said:

As all players on the losing team all scored 0 MS they all lost the maximum PSR under this system just like they would in Nighthawks

Now in our current system where everyone is comparing matchscores or dmg done, you run a locust but the rest of the team is far behind. Do you attempt to hold the point alone to stop an enemy cap and risk dying early, or do you just let enemy cap and move on? The 2nd option is what you got. The 1st option would leave you potentially eliminated early but with the rest of the team having a fighting chance. Scoreboard wise, you will look like you sucked relative to the rest of the team.

Also, this discussion distracts from the actual point. This game mode is unaffected whether you use MS or W/L. Or are you saying that using a W/L system will result in this type of game more often? Because your original point was about rushing objectives

Edited by Gozuri, 14 June 2020 - 11:49 PM.

[MrMadguy]

Gozuri, on 14 June 2020 - 07:25 PM, said:

Wait...isn't that winning the game? Wasn't that the objective? Why put those objectives there if it was never meant to be achieved? How about developing new team strat to prevent someone stealing a win?

In PVP games objectives are usually just points of interest on a map to prevent players from spreading too much and to use all parts of map, not just players' favorite one. Killing enemies - is still main goal.

I personally remember that moment, when Skirmish was implemented. All Assault matches were just NASCAR base rushes prior to that.

Edited by MrMadguy, 14 June 2020 - 11:50 PM.

[crazytimes]

We still arguing about how to sort the last few hundred people playing into different tiers so they can all play in the same match still?

By the looks of it, it doesn't matter how it's done, after the reset enough people will be butthurt and leave that it will have to be a full tier spread to get matches.

Just get the reset over with. The salt will flow whatever method is used thereafter, let's just get there.

[Kodyn]

crazytimes, on 15 June 2020 - 01:32 AM, said:

We still arguing about how to sort the last few hundred people playing into different tiers so they can all play in the same match still?

By the looks of it, it doesn't matter how it's done, after the reset enough people will be butthurt and leave that it will have to be a full tier spread to get matches.

Just get the reset over with. The salt will flow whatever method is used thereafter, let's just get there.

Seriously. What this guy said.

I keep coming back to check on when the reset will occur, since it might prove interesting to play again for a bit. Played through the last match score event, but that was super easy and only took 2 days to finish, so I had hoped there'd be a more interesting event tied to the reset patch - or at the very least, get to enjoy a month or so of clawing my way back to T1, to see what the T1 pool actually feels like without valves opened for a short window - until they realize player count dictates opening them wide once more.

You know, those valves that PGI openly admitted to, that certain members of this forum swore never got opened despite others of us swearing for months and years that we'd been seeing potatoes in our matches...(Dead horse by now I'm sure, but I still find that last bit amusing. Especially considering the same community that didn't understand how the current PSR system worked is now designing the new one. The irony.)

[technopredator]

I think the game shouldn't care as the most weight criterion the win or loss of a match, just 1 more in a criteria, to determine how the player did, according to a good definition of a Mechwarrior. Win/lose criteria should have a big effect but not way bigger than the other criteria.

So, I think enough ideas has been given, it's time for You PGI, to round up the top 5 most popular because evaluate personal performance the best and put them in a poll for vote, offer easy 100,000 CB in game for the vote, and let's get over with this already. We need to progress, we have refined the idea s much as I think it could be for now; future improvements or replacements would be suggested for another discussion run in 3 months, this would be a great way to improve the game without derailing too much from it and would be what most of us find important for a fun and sustainable game-play hopefully over many years to come

[Mat Sorkas]

Kodyn, on 15 June 2020 - 02:31 AM, said:

Seriously. What this guy said.

I keep coming back to check on when the reset will occur, since it might prove interesting to play again for a bit. Played through the last match score event, but that was super easy and only took 2 days to finish, so I had hoped there'd be a more interesting event tied to the reset patch - or at the very least, get to enjoy a month or so of clawing my way back to T1, to see what the T1 pool actually feels like without valves opened for a short window - until they realize player count dictates opening them wide once more.

You know, those valves that PGI openly admitted to, that certain members of this forum swore never got opened despite others of us swearing for months and years that we'd been seeing potatoes in our matches...(Dead horse by now I'm sure, but I still find that last bit amusing. Especially considering the same community that didn't understand how the current PSR system worked is now designing the new one. The irony.)

You need to understand that with the current Tier system you have a lot of potatoes in T1. If someone played for a long time and never improved they still end up in T1 regardless of their actual skill level due to the upward bias of the PSR system. The matchmaker just does not have a tool to distinguish an ancient potato pilot from an actual battlefield god.
The whole conversation in this thread is about how to divide the playerbase into meaningful tiers. This will still not prevent "true T1s" from playing against weaker pilots. The number of concurrent players just is not there for it to be feasible. The valves would need to remain open to some degee.
However with a properly categorized players we could see teams that are more balanced so for every potato in your team there would be one in the opposing force.

[Eatit]

Mat Sorkas, on 15 June 2020 - 03:41 AM, said:

You need to understand that with the current Tier system you have a lot of potatoes in T1. If someone played for a long time and never improved they still end up in T1 regardless of their actual skill level due to the upward bias of the PSR system. The matchmaker just does not have a tool to distinguish an ancient potato pilot from an actual battlefield god.
The whole conversation in this thread is about how to divide the playerbase into meaningful tiers. This will still not prevent "true T1s" from playing against weaker pilots. The number of concurrent players just is not there for it to be feasible. The valves would need to remain open to some degee.
However with a properly categorized players we could see teams that are more balanced so for every potato in your team there would be one in the opposing force.

Changing the criterion the MM uses to sort teams is not on the table. The metric used to assign tier is what is being changed. The MM will continue to sort by size not tier.

The current MM combines Tier1, Tier 2, and Tier 3 for a normal match. It opens valves when that's not enough.

If you move people around in those tiers it will not make a difference. The same people will still be in each match and will still be sorted by size.

Balance will not change!!!

This is an attempt to placate the people who want to be seen as superior to others. They will rise to Tier 1 and be rewarded with that title. The title of Tier 1 will not be given to people who are less skilled but play a lot of matches.

This is an ego stroke for those that need to have their egos stroked. PERIOD

[OneTeamPlayer]

Zerex, on 14 June 2020 - 10:00 PM, said:

This is what happens in a team based PvP game when 100% f the rewards are based on the win.

How long will the player base hang round if this starts happening even in 10% of matches?

Also it doesn't matter if on paper Nightbirds system is perfect or not they haven't took the human factor into account. They will do anything to game the system.

This is entirely misleading as if this video was released in 2010 it either shows:

1. A battlefield populated with NPCs that took literally over a day in multiple instances to complete which was then tuned downward in difficulty due to player complaints, while simultaneously being overshadowed by advances in the ingame "tech" (higher levels, more legendary equipment, better consumables).

or

2. A private server, which is neither tweaked to the same standards as official servers nor the same item progression (characters would have gear that most players wouldn't attain by default or payment).

If you don't play wow in short this comment is like showing a 2013 release day unskilled Cataphract losing handily to a modern day fully skilled recently made clan mech (say a Corsair or whatever is most popular) and saying that represents a fair fight and/or the standard of the game.

Arguing in bad faith and terribly disappointing to see.

Ironically what the video does show is an example of how badly gameplay can suffer when a developer caves to player demands rather than using their own team to properly analyze and design with a side of the perils of not watching balance when new weapons are introduced thereby invalidating a large amount of previous content (sound familiar?).

Edited by OneTeamPlayer, 15 June 2020 - 04:41 AM.

[Xiphias]

Jochi Kondur, on 13 June 2020 - 06:25 PM, said:

Am I right in your assessment that in terms of the benefit to the player experience, Nightbird's solution >= Gagis' solution > Jay Z's solution. However, only Jay Z's solution fits within the scope of changes and therefore has the best chance of being implemented?

I'm not an expert in statistics, so I defer to those in the thread like Nightbird who are more knowledgeable, but generally yes, I think that's correct. Gagis' might be within the scope, haven't checked.

OZHomerOZ, on 13 June 2020 - 08:10 PM, said:

I read that other contenders who have remained within the scope/limits have the same problem of divergence which imo is easily fixed with regular reset's like some other games do, apparently, I read that somewhere on the forums.

Correct me if I am wrong.

Sure, divergence could be fixed with resets, however then you'd have bad matchmaking every time the reset happens. A better setup would be to have something like Solaris seasons (reset every three months) and just reseed (instead of reset) using the global stats for that period to place players in their appropriate tier. If that's on the table I would be all for using those numbers to get better placement.

Quote

What or who's goal is to win, match maker, solo players or groups?

A solo pilot does not have the same chance at winning as a group pilot.

If you make winning a large part of the formula it will favour grouped pilots and disadvantage solo pilots in terms of PSR movement. Is this really measuring a pilots skill is what I am saying.
Edit: Even Handsome Proton mentioned that maintaining a positive win loss as a solo was hard work or some thing close to it. Correct me if I'm wrong beautiful?

Groups don't need more advantages.

Match making is used to match pilots and groups with similarish skill to avoid hugh skillgaps between two teams to my knowledge.

Again correct me if I am wrong. amirite?

The goal is to balance matches, not to make players feel good about their ranking. If groups win more then the matchmaker would make it harder for them, if anything that disadvantages groups rather than advantaging them. I don't see why giving groups higher PSR is an advantage/bad. If they win more, they go up, it gets harder, so either:

1) They always play in a group and MM should consider them better (because groups advantageous to winning)
2) They inflate in a group and play solo. MM puts them higher than their skill and they lose, this drives down their record to where it should be.

Groups mixed with solos obviously messes with any match maker, but if getting close matches is the goal, groups that win more should be given a higher rank and be forced to play against either other groups or better players. People are getting two attached to the "player skill" term and putting player worth into the equation. Yes, a solo might be a better player than someone who always plays in a group, yet the solo might have a lower WLR, however when grouping for matches the group should absolutely be considered higher skill because they are more likely to win than the solo. What's better, 4 good solos or a group of 4 average pilots? If the solos are better they will win and more up, if the group is better then it will win and move up and be put against even better solos or other groups.

Moving up in tier isn't an advantage, it causes the MM to make things harder for you.

East Indy, on 14 June 2020 - 09:22 AM, said:

How many is "a bunch"? I'm curious how long it takes to get that divergence.

Also — while I acknowledge that a feature of Jay Z's proposal is a "fair" approach to contribution, what happens to long-term simulations when a coefficient (or whatever) is added to make significant PSR shifts for, say, the top 4 and bottom 4 only, and very tiny shifts to the other 16 players? In other words, can the system assume that two-thirds of the players it's being fed average 175-250 matchscore and work to keep them in the middle, sending only the really good and really subpar folks to the extremes?

Specifically, that image was with 100,000 total matches (matches per player 2240-2571) at smaller numbers the divergence is obviously less bad. Here are examples at
10,000 (193-293)

Spoiler

20,000 (415-554)

Spoiler

30,000 (633-807)

Spoiler

Basically, it's not too bad within a few hundred matches (several months for most players) and just progressively starts to get worse.

That said, the basic model that I put together isn't really something robust enough to be pulling hard statistical numbers about the game. It has a number of simplifying assumptions so it's more for illustrations purposes rather than anything else.

Edit: To your second question East Indy. I could play around with things and generate something like that, but as I said I don't consider the model robust enough for hard numbers. I'm making a variety of assumptions (such as how a player's actual match score varies around their average) that I don't really have the data for. To get good numbers you would need to pull in the real data and make fewer arbitrary assumptions (e.g. chances of winning being based on "secret skill" numbers).

I think yes though, if you move fewer people or reduce the total amount of PSR movement, the divergence will be slower. The downside is that slower divergence will also lead to accurate MM being slower if everyone starts in the same spot.

Edited by Xiphias, 15 June 2020 - 05:40 AM.

[Nightbird]

Xiphias, on 15 June 2020 - 05:34 AM, said:

Great sim, I have one suggestion though, take the number of matches each player plays based on the Jarl's data, and randomize the number of matches played in the simulation using that instead of saying everyone has an equal chance of playing matches.

Take one look on Jarl's on any page and you can see that the # games played can go from 100-500 to 10,000+. This has a major impact when PSR=skill*games played.

I would program this into the chance of being picked for a match, and make sure it gives a final matches played variance comparable to true data.

I didn't do in my sim because WLR isn't impacted by matches played, but MS random walks are severely impacted. I predict you'll see more of a of cloud between skill and PSR than the nice line you have right now.

This is intended to demonstrate that resets don't do anything.

Edited by Nightbird, 15 June 2020 - 07:52 AM.

[Surn]

While I like the evaluation of teams based on average skill, it does not address the definition of skill.

That is my focus, once a reasonable skill is agreed upon, the rest is easy.

[Zerex]

OneTeamPlayer, on 15 June 2020 - 04:40 AM, said:

This is entirely misleading as if this video was released in 2010

WoW Burning crusade was released in 2007, pretty sure by 2010 AV had been guttered of its NPC's and was just a base rush.

The point is if you make winning the only goal don't be surprised when players on play to win

Edited by Zerex, 15 June 2020 - 09:46 AM.

[Xiphias]

Nightbird, on 15 June 2020 - 07:33 AM, said:

Great sim, I have one suggestion though, take the number of matches each player plays based on the Jarl's data, and randomize the number of matches played in the simulation using that instead of saying everyone has an equal chance of playing matches.

Take one look on Jarl's on any page and you can see that the # games played can go from 100-500 to 10,000+. This has a major impact when PSR=skill*games played.

I would program this into the chance of being picked for a match, and make sure it gives a final matches played variance comparable to true data.

I didn't do in my sim because WLR isn't impacted by matches played, but MS random walks are severely impacted. I predict you'll see more of a of cloud between skill and PSR than the nice line you have right now.

First, thanks for the feedback.

There are a lot of things that could be done to improve the sim. It's a really basic model that I threw together to illustrate the point, rather than something that is going to be able to generate quantitative values. I'm using simulated players and then just picking from that list randomly to generate the overall matches, which I understand is a naive and not realistic assumption, but it made the code easy.

I suppose I could pull in real player data and the use that to weight the chances of playing a match or something similar to generate matches, but it would be more complicated than the simply model I'm using.

Another problem with my sim is that it's built on the assumption that matchscore accurately reflects skill. I based the skill values off of match scores from Jarls and I base the wins/scores in each match based this underlying skill value. Since they are directly linked, obviously matchscore will end up being a much better predictor than it actually is in game.

Another problem with that specific graph is that I used a generated random number that was evenly distributed rather than normally distributed to scale a player's "skill" into a match score for each match. You end up getting a much wider spread if you use the standard normal distribution.

Perhaps I'll go back in a few weeks when I have more free time and build a more robust simulator off the real game data. Could be a fun project and useful practice. If I did that would you be open to providing some quick feedback/input if I PM'd you on the forums?

[Nightbird]

Xiphias, on 15 June 2020 - 08:08 AM, said:

First, thanks for the feedback.

There are a lot of things that could be done to improve the sim. It's a really basic model that I threw together to illustrate the point, rather than something that is going to be able to generate quantitative values. I'm using simulated players and then just picking from that list randomly to generate the overall matches, which I understand is a naive and not realistic assumption, but it made the code easy.

I suppose I could pull in real player data and the use that to weight the chances of playing a match or something similar to generate matches, but it would be more complicated than the simply model I'm using.

Another problem with my sim is that it's built on the assumption that matchscore accurately reflects skill. I based the skill values off of match scores from Jarls and I base the wins/scores in each match based this underlying skill value. Since they are directly linked, obviously matchscore will end up being a much better predictor than it actually is in game.

Another problem with that specific graph is that I used a generated random number that was evenly distributed rather than normally distributed to scale a player's "skill" into a match score for each match. You end up getting a much wider spread if you use the standard normal distribution.

Perhaps I'll go back in a few weeks when I have more free time and build a more robust simulator off the real game data. Could be a fun project and useful practice. If I did that would you be open to providing some quick feedback/input if I PM'd you on the forums?

Add me on Discord and PMing me there will probably be better, after the MM is decided I'll likely hop off the forums.

I do have a suggestion on how to quickly program this without real data. For every player generate a number between 1 and 100 to represent their play frequency. When you are picking players randomly, do a double randomization. When one player is selected by the first randomizer, do a second randomization with the play frequency as a % to see if they truly join the team. Keep going until you have 24 players selected and this should give you the necessary impact.

As for whether avgMS = true skill, if you want to simulate that, you'd have to 1) use a bell curve to create a random population with true skill values, and 2) use a second randomization on each people to give their avgMS. (This avgMS will have 11% R-square predictive value compared to true skill). -- it's getting hard to describe so I'd skip this for now.

Edited by Nightbird, 15 June 2020 - 08:23 AM.

[Xiphias]

Surn, on 15 June 2020 - 07:38 AM, said:

While I like the evaluation of teams based on average skill, it does not address the definition of skill.

That is my focus, once a reasonable skill is agreed upon, the rest is easy.

For the purpose of matchmaking isn't the goal to balance wins/losses? If that's the case isn't skill the ability to win? Match maker doesn't care how you win as long as you do.

[Wraith of Shadow]

Xiphias, on 15 June 2020 - 08:18 AM, said:

For the purpose of matchmaking isn't the goal to balance wins/losses? If that's the case isn't skill the ability to win? Match maker doesn't care how you win as long as you do.

I thought the 'goal' was to populate both sides with players of approximately the same skill level, so that they can have a fairly even match up and not have one side dominate the other.

[Capt Deadpool]

Agree W/L should be only effective measurement of skill. Everyone gets unlucky and has bad teams creating losses, but because we know this happens to everyone, any subjective feeling of, "I am so unlucky, don't punish me because of the potatoes on my team." is irrelevant.

Grouping reduces statistical likelihood of drawing a full team of taters, which is an option available to everyone, but there are also many people who enjoy playing solo. Hopefully phase two after, hopefully, most of the community and devs realize obviousness of basing PSR/MM around W/L will be balancing solos against groups via use of modifiers or multiple PSRs based on if someone is grouped or not.

[Capt Deadpool]

Wraith of Shadow, on 15 June 2020 - 11:02 AM, said:

I thought the 'goal' was to populate both sides with players of approximately the same skill level, so that they can have a fairly even match up and not have one side dominate the other.

Well, the 'goal' is to find the most effective metric to base PSR/MM on, which is W/L (data/simulations should not even be necessary to grasp this), for the purpose of 'populating both sides with players of approximately the same skill level, so that they can have a fairly even match up and not have one side dominate the other.'

[Nightbird]

Friendly question to people, what do you think a good MM is supposed to do?

Is it to bring unskilled players and skilled players as close to 1 WLR as possible? Or punish unskilled players with a very low WLR and reward skilled players with a high WLR?

Here's a simulation of the current MM graphing skill versus WLR





Here's a simulation of the WLR MM graphing skill versus WLR:




All suggestions that doesn't use WLR or avgMS will preserve the status quo. Vote what you want to see.

Edited by Nightbird, 15 June 2020 - 11:45 AM.

[Capt Deadpool]

^To rephrase, I'd say the purpose of an effective MM is to reward every player regardless of skill-level with competitive matches, with any sense of 'punishment' only being experienced by those who prefer stomps

Ideal world (which will never happen due to low population): top-tier comp world class Olympian 4-mans play against each other and have as close to 1:1 W/L as possible, and Day-1 Tier 5 headless chickens cadets who still have not discovered the mini-map should also be 1:1.

Maybe... current population of players being drawn from determines size of teams to better achieve competitive parity? 100 players online: 12v12, 75 players: 8v8, 50 players: 6v6...