Jump to content

Data-Driven Decision-Making


34 replies to this topic

#1 arcana75

    Member

  • PipPipPipPipPipPipPipPip
  • Shredder
  • Shredder
  • 1,161 posts

Posted 24 October 2017 - 03:00 AM

I was in another forum asking if others there played MWO and someone replied that he and his friends were interested but read that the game had gone down the drain over the years, which to me was a surprising reply considering that they haven't played the game yet and were discouraged from playing it by other avenues, possibly word of mouth, possibly from this forum.

The biggest "beef" of angry players players in MWO forums from near as I can tell, is the perception that the players know MWO well, and PGI doesn't and some allege PGI doesn't even play MWO, citing issues like FW and tech imbalance and their handling of FW Events like the Battle of Luthien.

I'm thinking, PGI, why not collect opinions from your entire active player base than the vocal minority? One way is, each time a player logs in, ask them a few questions in-game about the game in general eg favourite map/mode, opinions about current events, etc; each time a match ends, ask them about the match itself, eg was MM fair, any lag, overall experience, which mech they used etc; each time they log out, ask them about their experience playing MWO since logging in and if their view changed or not, their most memorable event, and other closing thoughts. All the feedback would be tied to player ID so there's no uncontrollable poll spamming.

Do this for 3 months then analyse the data, then combine it with your server data to make changes accordingly. Then keep doing it, as more data = better decisions.

FYI, each time someone ends a beta match in HareBrainedScheme's BattleTech beta, a survey comes up asking questions about their overall experience, and the questions vary.

Edited by arcana75, 24 October 2017 - 03:02 AM.


#2 El Bandito

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • Big Daddy
  • Big Daddy
  • 26,736 posts
  • LocationStill doing ungodly amount of damage, but with more accuracy.

Posted 24 October 2017 - 03:04 AM

And watch as LRMs get nerfed even further than it already is.

#3 arcana75

    Member

  • PipPipPipPipPipPipPipPip
  • Shredder
  • Shredder
  • 1,161 posts

Posted 24 October 2017 - 03:12 AM

View PostEl Bandito, on 24 October 2017 - 03:04 AM, said:

And watch as LRMs get nerfed even further than it already is.

Other than opinion, what data do you have that substantiates that claim?

#4 The Mysterious Fox

    Member

  • PipPipPipPipPipPip
  • The Altruist
  • The Altruist
  • 381 posts
  • LocationUsing your bathroom

Posted 24 October 2017 - 03:25 AM

how do you quantify fun and have it be applicable to the widest base? youve got crazies spewing anecdotals on these forums all day long with wishlists that would kill the game. data mining from a population is usually a risk that pisses off players. a focus group would be good though the selection of the group extremely precarious on how it represents the different groups of the playerbase. and of course this all costs money

#5 El Bandito

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • Big Daddy
  • Big Daddy
  • 26,736 posts
  • LocationStill doing ungodly amount of damage, but with more accuracy.

Posted 24 October 2017 - 03:29 AM

View Postarcana75, on 24 October 2017 - 03:12 AM, said:

Other than opinion, what data do you have that substantiates that claim?


Drop into any lower tier match and you will hear people bitching about LRMs being OP, and un-fun. No doubt they will extend that negativity into any feedback PGI asks of them.

IMHO: Keep a niche game a niche game if you want it to survive. Appealing to the masses, and attempting to go mainstream had killed plenty of franchises. The whole e-sports scene is just non-sense, and waste of money.

Edited by El Bandito, 24 October 2017 - 03:33 AM.


#6 C E Dwyer

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Legendary Founder
  • Legendary Founder
  • 9,274 posts
  • LocationHiding in the periphery, from Bounty Hunters

Posted 24 October 2017 - 03:32 AM

H.B.S survey isn't relevant to what happens in M.W.O.

They already have a set plan and a clue, they won't change certain aspects of it no matter how many negative survey results they receive, unless it's over whelming things like Ac10 damage is a prime example.

P.G.I has never had a clue about which direction they were going other than they wanted this to be E-sport, and it was going to have clans.

They change the way they balance the ame almost at the drop of a hat, and rather than use a scalpel, on one aspect of tuning, they use a wrecking ball.

As a couple examples. the

Kodiak -3 there were also grounds for the SB and 4 needing to be pulled back, the rest were not that good, they hammered all of them.

Skill tree introduction.

While it does nothing positive for the game, adds a new screen and more clicking (P.G.I love to give us RSI) it was claimed this would end the need for major offensive quirks.

Fail

The self professed clanboi in the balance department, gave the nova prime a mech that didn't need them, some rather impressive offensive quirks.


Further more the clanboi (yes he's another that's on record for his preferences) in the modelling department, had he put the second pair of ballistic hard points on the kodiak lower than the cockpit (until P.G.I's model the only weapon slightly above cockpit height, on either drawing or model, had been two missile launchers) rather than higher the Kodiak issue would never have arisen.

This games current balance problems are being caused by unprofessional people, allowing personal desires get in the way of the games development.

Either that or they are so bad at their job, they wouldn't be heads of their department in any competent company, because they make so many bad choices.

#7 Bud Crue

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Rage
  • Rage
  • 9,943 posts
  • LocationOn the farm in central Minnesota

Posted 24 October 2017 - 03:32 AM

Well, if you believe what PGI via Russ, Chris, Paul, Tina, etc. have posted here, and stated in town halls and round tables and NGNG streams over the years, they have all the data they need and it is their absolute confidence in that data and their exclusive understanding of what that data shows that drives EVERY change they make to the game. Moreover, they have stated that they are "as communicative as possible" with the community as it is. So, in light of that perspective, why on earth would they need, let alone want, to ask players questions about those players opinions regarding the game? Finally, one must also keep in mind that Russ is on record saying that this is his game and that he and PGI know what is best for it, so again why would they care about our opinions at this point? Hell, given their absolute confidence in their development history of this game, asking for our opinion now would be as crazy as having a round table (or 2) and asking us to tell them how to fix CW while insisting that it is "fine". I mean come on. No one is that crazy.


Edited by Bud Crue, 24 October 2017 - 03:32 AM.


#8 arcana75

    Member

  • PipPipPipPipPipPipPipPip
  • Shredder
  • Shredder
  • 1,161 posts

Posted 24 October 2017 - 03:35 AM

View PostThe Mysterious Fox, on 24 October 2017 - 03:25 AM, said:

how do you quantify fun and have it be applicable to the widest base? youve got crazies spewing anecdotals on these forums all day long with wishlists that would kill the game. data mining from a population is usually a risk that pisses off players. a focus group would be good though the selection of the group extremely precarious on how it represents the different groups of the playerbase. and of course this all costs money

You can't please everyone, but from pure data rather than anecdotal evidence, you derive insights, objective views. It is this that should be driving changes, not what a bunch of forummers are yelling for, or what a single dev's idea of change is, even if it's Russ Bullock, for example.

Focus groups don't work, because as you said it's difficult to find reps that can adequately represent the entire player base, and more often than not, these reps are self-serving.

View PostEl Bandito, on 24 October 2017 - 03:29 AM, said:


Drop into any lower tier match and you will hear people bitching about LRMs being OP, and un-fun. No doubt they will extend that negativity into any feedback PGI asks of them.

The problem with this anecdotal view is that it's just that anecdotal. For all the 50 people who have the same view as you, there could be literally 500 thinking differently, and without collecting data, no one will actually know what the entire player base is thinking. We know what the vocal players are saying, but it's always the vocal minority. Always.

FWIW other than the forum, I've never once heard a player on VOIP ***** about LRMs. ***** about pushing yes, potatoes yes, terribad teammates yes, but for me never LRMs. So this does not invalidate either of our views, cuz they're both valid and anecdotal. You can say this bitching happens only in T3/2/1 which I'm not in yet, but the point would be missed if so, as the topic isn't about who's bitching about LRMs where.

#9 The Mysterious Fox

    Member

  • PipPipPipPipPipPip
  • The Altruist
  • The Altruist
  • 381 posts
  • LocationUsing your bathroom

Posted 24 October 2017 - 03:49 AM

View Postarcana75, on 24 October 2017 - 03:35 AM, said:

You can't please everyone, but from pure data rather than anecdotal evidence, you derive insights, objective views. It is this that should be driving changes, not what a bunch of forummers are yelling for, or what a single dev's idea of change is, even if it's Russ Bullock, for example.

Focus groups don't work, because as you said it's difficult to find reps that can adequately represent the entire player base, and more often than not, these reps are self-serving.


The problem with this anecdotal view is that it's just that anecdotal. For all the 50 people who have the same view as you, there could be literally 500 thinking differently, and without collecting data, no one will actually know what the entire player base is thinking. We know what the vocal players are saying, but it's always the vocal minority. Always.

FWIW other than the forum, I've never once heard a player on VOIP ***** about LRMs. ***** about pushing yes, potatoes yes, terribad teammates yes, but for me never LRMs. So this does not invalidate either of our views, cuz they're both valid and anecdotal. You can say this bitching happens only in T3/2/1 which I'm not in yet, but the point would be missed if so, as the topic isn't about who's bitching about LRMs where.


this has already been the year of spreadsheets and balancing to the curves and its been incredibly downhill with each patch. Even all the goodwill from the skill tree and civil war has pretty much evaporated in an instant so there is more to design than just a textbook. unfortunately for a game running into 5 years and on, its pretty crazy how little finesse there is at handling these things

#10 Trissila

    Member

  • PipPipPipPipPipPip
  • Survivor
  • 439 posts

Posted 24 October 2017 - 04:20 AM

Blizzard tried collecting in-game feedback from Overwatch players once. There used to be a "Rate This Match" interface at the end of every match where you could rate the quality of the match on a scale of one to three.

They very quickly found that the rubrik for the overwhelming majority of players was "If I won, Rate 3, If I lost, Rate 1". So they abandoned the idea and removed the interface.

You do not ask the average player what their thoughts are in-game.

#11 Asym

    Member

  • PipPipPipPipPipPipPipPipPip
  • Nova Captain
  • 2,186 posts

Posted 24 October 2017 - 05:05 AM

View Postarcana75, on 24 October 2017 - 03:12 AM, said:

Other than opinion, what data do you have that substantiates that claim?

That's simple: anything that changes status quo equals hate...... Read the forums and you find there are "core" players who do not want anything that is "in-direct" or a better description, "not aimed as a direct fire weapon".... Many of us newer players refer to them as "the Brawling" community based on their comments.... Arcade FPS gamers. Up close, personal game play.

If the only "voice" in a game is a forum and who spends what money where, opinion reigns supreme and is a mteric. As said before, many gaming companies require their staff to play often and competently. LoL is an example and you can Google many good articles about what they do... Does PGI do that? Who really knows because I kind of doubt recent nerf's would have happened if they were.

#12 Lily from animove

    Member

  • PipPipPipPipPipPipPipPipPipPipPip
  • The Devoted
  • The Devoted
  • 13,891 posts
  • LocationOn a dropship to Terra

Posted 24 October 2017 - 06:07 AM

View Postarcana75, on 24 October 2017 - 03:12 AM, said:

Other than opinion, what data do you have that substantiates that claim?



dunno, the efficiency of lrm in T1 games? where they are inefficient, unlike in T4/T5 where they are godlike weapons because people don't know what cover is and other basics?

LRM's are pure trash within proper skilled pilots games. At best a minor annoyance, but they spread damage, they are easy to counter and made pointless. What data? well go for the tournament and tell us how many higher teams there play lrm's. You know people who play to win stuff at all costs, those that play the game by: "if it's inferior I am not gonna use it" mentality.

#13 Trissila

    Member

  • PipPipPipPipPipPip
  • Survivor
  • 439 posts

Posted 24 October 2017 - 06:32 AM

View PostAsym, on 24 October 2017 - 05:05 AM, said:

That's simple: anything that changes status quo equals hate...... Read the forums and you find there are "core" players who do not want anything that is "in-direct" or a better description, "not aimed as a direct fire weapon".... Many of us newer players refer to them as "the Brawling" community based on their comments.... Arcade FPS gamers. Up close, personal game play.


People don't hate arty/strikes because they're indirect. They hate those things because they are zero-ton, zero-crit, zero-heat, zero-risk damage which flies in the face of everything Battletech/Mechwarrior has ever been balanced around.

#14 TLBFestus

    Member

  • PipPipPipPipPipPipPipPipPip
  • The 1 Percent
  • The 1 Percent
  • 3,519 posts

Posted 24 October 2017 - 07:21 AM

View Postarcana75, on 24 October 2017 - 03:00 AM, said:


I'm thinking, PGI, why not collect opinions from your entire active player base than the vocal minority?


You missed a layer of complexity here. We are never sure exactly WHICH vocal minority PGI listens to at any given time;

The White knights?
The Black knights?
The Comp/eSport crowd?
The Casuals?
The little voice in RBs head?
The Whales?
pick any 2?
pick any 3?

You get the picture, right?

#15 Khobai

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 23,969 posts

Posted 24 October 2017 - 07:23 AM

Quote

Drop into any lower tier match and you will hear people bitching about LRMs being OP, and un-fun.


yeah and higher tiers think lrms are useless and unfun

but everyone still agrees theyre unfun

so regardless of whether youre a tierzerospudtato or metaproplayerextraordinaire, everyone agrees pgi needs to make LRMs more fun

yet this most recent patch did the opposite...

Edited by Khobai, 24 October 2017 - 07:24 AM.


#16 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 24 October 2017 - 07:53 AM

Data driven doesn't mean what you think it does... for one the data needs to be good quality - unbiased and accurate. Otherwise, we call it garbage in garbage out.

#17 Jun Watarase

    Member

  • PipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 3,504 posts

Posted 24 October 2017 - 08:33 AM

View Postarcana75, on 24 October 2017 - 03:00 AM, said:

I was in another forum asking if others there played MWO and someone replied that he and his friends were interested but read that the game had gone down the drain over the years, which to me was a surprising reply considering that they haven't played the game yet and were discouraged from playing it by other avenues, possibly word of mouth, possibly from this forum.

The biggest "beef" of angry players players in MWO forums from near as I can tell, is the perception that the players know MWO well, and PGI doesn't and some allege PGI doesn't even play MWO, citing issues like FW and tech imbalance and their handling of FW Events like the Battle of Luthien.

I'm thinking, PGI, why not collect opinions from your entire active player base than the vocal minority? One way is, each time a player logs in, ask them a few questions in-game about the game in general eg favourite map/mode, opinions about current events, etc; each time a match ends, ask them about the match itself, eg was MM fair, any lag, overall experience, which mech they used etc; each time they log out, ask them about their experience playing MWO since logging in and if their view changed or not, their most memorable event, and other closing thoughts. All the feedback would be tied to player ID so there's no uncontrollable poll spamming.

Do this for 3 months then analyse the data, then combine it with your server data to make changes accordingly. Then keep doing it, as more data = better decisions.

FYI, each time someone ends a beta match in HareBrainedScheme's BattleTech beta, a survey comes up asking questions about their overall experience, and the questions vary.


PGI has consistently shown that they do whatever they want instead of what their customers want.

For example, when they decided that artemis missiles should be nerfed even though not a single person complained about artemis on the forums.

Edited by Jun Watarase, 24 October 2017 - 08:33 AM.


#18 Valdarion Silarius

    Member

  • PipPipPipPipPipPipPipPip
  • Giant Helper
  • Giant Helper
  • 1,680 posts
  • LocationWubbing and dakkaing everyone in best jellyfish mech

Posted 24 October 2017 - 08:52 AM

View Postarcana75, on 24 October 2017 - 03:00 AM, said:


Do this for 3 months then analyse the data, then combine it with your server data to make changes accordingly. Then keep doing it, as more data = better decisions.


From what I heard from someone else on the forums, it sounds like Chris (the current balance guy) is trying to make balance decisions based on collecting in game data but then Paul (who has the final say on game play balance) says yes or no based on what changes with what Chris has collected. Paul then does balance changes which I think puts Chris in a bad spot.

I'm not entirely sure if there is a severe miscommunication happening inside the company (we can't tell unless more insiders come out and tell), if PGI is being openly ignorant to the community and disregards our say since we are apparently a bunch of cheapskates, or if they are so blindly oblivious with their vision of MWO being some kind of massive E-Sport that they make these balance changes solely just on this idea. I'm going to go with 2 and 3 here, since their nerfs defy logic and rational reasoning.

View PostEl Bandito, on 24 October 2017 - 03:29 AM, said:


IMHO: Keep a niche game a niche game if you want it to survive. Appealing to the masses, and attempting to go mainstream had killed plenty of franchises. The whole e-sports scene is just non-sense, and waste of money.

It should stay niche, but at this point it time if PGI will renew the license I'm almost certain their future goals for the game won't be. There is not enough profit draining whatever loyal whales are left that are keeping their lights on. They are most likely banking that MW:5 Mercs will be a success. If it isn't, then I bet there will be an announcement a few months after declaring that they want to withdraw from the franchise.

Edited by Arnold The Governator, 24 October 2017 - 09:04 AM.


#19 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 24 October 2017 - 08:57 AM

PGI doesn't care about customers! What customers want is what I want!

:D

Really, all you can do is express your opinions and PGI makes what they think is the right decision.

Thread gonna get locked because of personal attacks, good going

#20 Mystere

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Bad Company
  • Bad Company
  • 22,783 posts
  • LocationClassified

Posted 24 October 2017 - 09:03 AM

View PostEl Bandito, on 24 October 2017 - 03:29 AM, said:


Drop into any lower tier match and you will hear people bitching about LRMs being OP, and un-fun. No doubt they will extend that negativity into any feedback PGI asks of them.

IMHO: Keep a niche game a niche game if you want it to survive. Appealing to the masses, and attempting to go mainstream had killed plenty of franchises. The whole e-sports scene is just non-sense, and waste of money.


Well, eSports seems to be the femme fatale of online gaming. Posted Image

View PostDewd, on 24 October 2017 - 03:31 AM, said:

The best data-driven decision making they could create now would have to come from the matches themselves, like the world championships, and clearly they are satisfied with their decisions so far, otherwise they wouldn't keep making similar decisions.


MWOWC (i.e. eSports) is in no way representative of the player base.





7 user(s) are reading this topic

0 members, 7 guests, 0 anonymous users