7 replies to this topic

[Riddler9884]

Whenever a Public Test is conducted a couple of the following things happen.

- Servers have to be set aside to for matches to happen.

- Bandwidth is dedicated to provide the download of the entire game to whoever in the player base decides to test it.

- Players have to coordinate to be able to download and play during the limited window.

My suggestion is make the public test say 1 map with a scripted battle (think 3dmark) and software essential resources for it to happen. The test will run and play out like any garden variety match. As the match progresses players will spectate like they do after their mech is destroyed. As they are spectating the client can record computer details such as fps, memory usage, gpu/cpu information and other telemetry that can be submitted after completion. After the match is over the client can ask to provide any observations or report any bugs.

This method requires no server resources to oversee the match, less bandwidth for download/play and can be made available to players for the entire day or longer. You will also be able to catalog and track bugs more accurately when reported.

Since this is less interactive you could even make this incentive based.

[Sandpit]

I think the only problem with being so scripted is that it doesn't allow for the "chaos" caused by random people doing random things and playing the game in ways that produce bugs and the devs didn't expect.

[Riddler9884]

That is true but the goal of the above approach is to report and track render,display issues and sound issues (Too much smoke from auto cannons, the fact you cant hear foot steps - in the recent dx11 public test) and to help possibly track things that contribute to drops in FPS which they said they are working on with DX11.

The current method is better to track "Chaos" you pointed out, netcode, server side changes, stuff like HSR and hit boxes.

Edited by Riddler9884, 14 November 2013 - 09:46 AM.

[focuspark]

There's a problem with this logic: standard MWO isn't scripted, therefore any scripted environment is contrived and therefore not a valid test of standard MWO.

Scripted would make for a good marketing ploy, but not a good test of new features.

[Riddler9884]

I guess I didn't make the goal of the suggestion understood properly.

I am recommending what they did here to a larger scale
http://mwomercs.com/...23#entry2348823

and I quote:

"Eventually we did manage to reproduce one of these issues with a debug build, and what this effectively confirmed for me was that there was almost certainly an issue somewhere in CryEngine itself."

Granted they were working on hud bugs, but who is to say this wont work on other things that might be hindering performance in some other way.


The Goal is not Gameplay testing, its feedback in terms of Graphical Performance

Edited by Riddler9884, 14 November 2013 - 10:15 AM.

[Sandpit]

That is definitely a more specific test you're looking for there, in which case your idea would probably be a better testing environment but I would imagine that the dedicated private testing area would do exactly this once they get feedback from the public tests

[Riddler9884]

The idea came to me recently. I had participated in the dx11 test and all i could do was 1 drop and I missed the rest of the afternoon. Then the following day I was trying out MWO on a friends laptop only to average 10 fps on Alpine. Meanwhile battlefield 4 performed reasonably well....

I know the devs continue to work on improving graphical performance, but with such narrow windows to test windows I don't see how they could get much telemetry or feedback from the community. At the same time I hear of workarounds to try and force MWO to better utilize computer resources and give better performance.

It's not my intention to take their attention away of what seems to be the 3 projects they are working on like C.W., DX11 and that Thong style hit box tweaking they are doing. I don't have anything against the current style of community test, I am just pointing out somethings that might save them money and give them needed feedback to improve our visual experience.

[Firewuff]

They will already be doing this. That said you need people to do stupid **** to really test. It also is used to see ifnplayers find exploits that designers would never see otherwise.

Also there are millions of players.. You only need to get a small percentage to play the test to have a huge dataset to run with