Jump to content

System and GPU Benchmarking Basics


6 replies to this topic

#1 RenegadeMaster

    Member

  • PipPipPipPipPipPipPip
  • 587 posts
  • LocationUSA's Caustic Valley: Arizona

Posted 30 June 2012 - 10:55 AM

After getting some recommendations here for a new video card (GPU), researching GPUs on my own, buying one, and doing research here & online about benchmarking, I started to realize that I could not find any threads here about benchmarking (gasp!). Benchmarking is the process of figuring out how fast or how well one or multiple pieces of hardware perform. There are some references to benchmark scores that web sites have obtained using various software, and people have occasionally referred to various programs that can be used for benchmarking, but I don't remember seeing anything that gave benchmarking tips/advise or even other people's scores.
So that is my hope for this thread, which happens to be the first original one I've started. I'm not as well versed on hardware performance people like Vulpes, but I just want to create a helpful place to share & consolidate what we all know about benchmarking. :(


[TIPS BEFORE YOU INSTALL YOUR HARDWARE:]
1) Back that thing (computer) up like Ginuwine! If your drivers or OS somehow chokes while detecting new hardware, and the system has a hard time coming back from it, this could save you a lot of headache.
2) Want to do a before & after comparison? Run or download benchmarking software BEFOREhand, and see the Software section below for different things you can use to do that with. If a GPU is being replaced/upgraded, make sure that the software is compatible with both your current and new hardware. For example: My integrated GPU was not Direct X 11 compatible, so I made sure to run DX 10 instead before AND after installing a new GPU so that I'd have an apples to apples comparison instead of an apples-with-tessellation (DX 11 only) to oranges.
If you have a smaller case like mine and you have to play music hardware to get your new GPU, RAM, or what not to fit, nothing sucks more than to have to bust the tools out again and try to take the new hardware out because its bugging you not known how much your system improved after you spent a bunch of money on new hardware.
3) If you are going to overclock anything, make sure that you have software that you can run during the benchmark that tells you what the current CPU or GPU temperature is. I am NOT responsible for any damage to any of your hardware or software, but I try to ensure that my CPU temp doesn't go much above 50 Celcius and that my GPU doesn't go over 65 Celcius.


[BENCHMARKING SOFTWARE:]
Recommended! CryEngine 3 SDK = http://www.crydev.net
This isn't benchmarking software per se, but it is the same exact graphics & game engine used to develop MWO, which means it will be the closest thing you can get to seeing how your system would handle the MWO's graphics & environments. It unzips to a whopping 1.27 GB (right click the ZIP file first and click "Unlock" if you have Win 7 w/UAC) - Plan ahead for some download and decompression time. Also, you will be required to obtain a log in for their web site, as this is used to sign massive disclosure statements and is required every time you launch an SDK app.
As for actually using it...
  • There will be a Bin32 and Bin64 folder, so open the one that matches the bit processing of your CPU & OS.
  • Run either Launcher or Editor. I have never gotten Launcher's menu to display properly for me, so I can't use it.
  • After running the Editor & logging in, you can load a demo level by going to File > Open, navigate to a folder where a ".cry" level file exists, such as \Game\Levels\Singleplayer\Forest\
  • The textures and everything will attempt to load. If not, and it goes full screen & black, be ready to press ALT+TAB, ESC, OR pull up task manager to kill it. If it loads, FPS will be in the upper right, and you can go into the Game menu to try and go in-game full screen.
Anyone have some good maps and/or scripts that could be used for benchmarking with this bad boy?



3DMark = www.3dmark.com
There are three different versions of this for DX 9, 10, and 11. It is free to download and run, but the free version does not let you configure ANY settings. So if your current system has a really weak GPU, don't be surprised if the program tells you after the test that it couldn't run "properly" because the GPU was really slow. It will still run, but obviously getting an FPS of less than a few Frames Per Second isn't going to be a good benchmark. This program is so easy to run, that I'm not going to post steps for it, but just know that it requires an internet connection in order to view the results. If you want to save your results for later:
  • Open the results page/URL in IE
  • Open the File menu, select Save As, and choose the file type of HTML single file.
  • This will allow you to easily see all the stats just the way you see them now, except for the detailed stats (e.g. FPS per test, etc). I recommend copying & pasting that text elsewhere or taking a screenshot of that.
One thing that is nice about 3DMark, is that the results automatically compare your scores to other people with similar hardware configurations. Since I'm guessing that it compares people with similar or the same CPU, GPU, and amount of RAM, my theory is that this comparison helps you see how much faster or slower your system is given the GPU optimization, software optimization, and overclocking (or lack thereof) that has been done.


Furmark = Google it at your own risk.
I was morbidly curious what software people could use to perform burn-in of their GPUs, and I found this app that sounds like a method of branding Furries, but it takes the video card's processing power to the limit to ensure that the hardware doesn't fail or cause a system reboot. It uses OpenGL (as opposed to DirectX?). I haven't done it yet based on the fact that MWO doesn't use OpenGL and I'm happy with my GPU temp results during the other benchmarks, but this may be good for anyone who wants to play OpenGL games or is a ********* to their system.

Windows Experience Index (WEI) = Included by default with all Vista or later non-server Microsoft Operating Systems
Although the single giant index number that it provides isn't of much use unless a game that you buy refers to min & recommended WEI scores, it also provides 5 different sub-scores that may be useful: Processor, RAM, Graphics (OS only), Gaming graphics, primary hard drive. This has the potential to point out where any possible bottlenecks might be in your hardware configuration.
For example: Before I installed a new GPU, my "Graphics" score was the lowest (4.1) followed by my Gaming Graphics (5.2). After I installed the new GPU and overclocked my CPU (as well as RAM a little), all my scores were in the 7s except for hard disk. That indicates to me that my hard drive will be a bottleneck if a game:
  • Does a lot of loading while actively playing, excluding cutscenes (see GTA 3, Morrowwind, countless others)
  • Uses all my RAM, causing the game to use virtual disk memory.
  • Does large saving of data while actively playing (not sure what games do this, but its possible)
FRAPS = http://www.fraps.com/

This can be used to get FPS, capture video, or easily take screenshots in games. The free version does not seem to have any restrictions on showing FPS during gameplay. After installing FRAPS, open the FPS tab, and just have MinMaxAvg selected for simplicity sake. This seems to work pretty well. I personally can't get the other modes to work.


[My BENCHMARKS/TESTIMONIAL]
I needed a new GPU because I want to make sure MWO (and in the interim Max Payne 3) would run great on my home PC, which was a Black Friday BYO (Build Your Own) deal from Newegg.com that I bought 2 years ago.
CPU: AMD Athlon II X4 640 3.36GHz (overclocked from 3.0GHz)
Motherboard & RAM: MSI 880GM-E41, 4 GB DDR3 1300
GPU: Integrated ATI Radeon HD 4250 with 256 MB (up to 1 GB more from shared RAM)

Benchmarks before:
3D Mark Vantage (DX10) on default settings: Score P275. Graphics Score: 208. CPU Score: 10462.
CryEngine 3 SDK: ~5FPS at 1128x469 windowed resolution, within the Level Editor
FRAPS with Max Payne 3: ~20 fps, 800x600, Low/Medium settings

The upgrade: After getting advise here and doing some of my own research, I bought a HIS IceQ X (AMD) Radeon HD 6870 with 1GB video RAM.

Benchmarks after:
3D Mark Vantage (DX10) on default settings: Score P13614. Graphics Score: 14899. CPU Score: 10815.
CryEngine 3 SDK: It hates my latest AMD drivers and won't render graphics :(
FRAPS with Max Payne 3: 30 fps all day, 720p (16:9) @ 60 Hz, High settings except for medium water & reflection, 4x Anti Aliasing


[REQUEST FOR FEEDBACK:]
So that is my breakdown of everything I have learned about benchmarking basics, and I hope it helps some of you that are trying to figure out how your system will perform with MWO whenever you get access. Please feel free to reply with things to add or corrections. Thanks!

Edited by RenegadeMaster, 03 August 2012 - 11:15 PM.


#2 Aznpersuasion89

    Member

  • PipPipPipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 614 posts
  • Locationca

Posted 30 June 2012 - 11:24 AM

What was the other benchmark program? Heaven or haven something?

Found it, heaven dx11

Edited by Aznpersuasion89, 30 June 2012 - 11:26 AM.


#3 RenegadeMaster

    Member

  • PipPipPipPipPipPipPip
  • 587 posts
  • LocationUSA's Caustic Valley: Arizona

Posted 30 June 2012 - 11:44 AM

View PostAznpersuasion89, on 30 June 2012 - 11:24 AM, said:

What was the other benchmark program? Heaven or haven something?

Found it, heaven dx11


Good to know - Sure beats the benchmarking Hell I was in with my crappy on-board GPU :)

#4 Vulpesveritas

    Member

  • PipPipPipPipPipPipPipPipPip
  • 3,003 posts
  • LocationWinsconsin, USA

Posted 30 June 2012 - 02:44 PM

Hey look I was mentioned O.O

also, +1 on the guide for being a decently good guide.

#5 RenegadeMaster

    Member

  • PipPipPipPipPipPipPip
  • 587 posts
  • LocationUSA's Caustic Valley: Arizona

Posted 30 June 2012 - 03:06 PM

View PostVulpesveritas, on 30 June 2012 - 02:44 PM, said:

Hey look I was mentioned O.O

also, +1 on the guide for being a decently good guide.

Thanks - I wanted to leave something for you all to remember me by while I'm away for a week :lol:

But seriously, part of my last two jobs was writing documentation. So if I ever wrote something noobish and any of my countless current or former co-workers saw it, I'd be laughed at and possibly shot on-site. The call centers I've been in have lots of MMO players & gun nuts. Also, those two types of people are not mutually exclusive here in the wild west ^_^

#6 Cyllan

    Member

  • PipPip
  • 34 posts

Posted 02 August 2012 - 11:15 AM

Good start. Something that might be useful to include is a separate/pulled-out section on your specs. I realize that the info is in there, but it makes it easier to read/compare results.

I also find CPU-Z (http://www.cpuid.com...ares/cpu-z.html) & GPU-Z (http://www.techpowerup.com/gpuz/) to be handy for people who don't know their CPU & GPU specs off-hand.

Something like (in my case):

System Specs
CPU: AMD Phenom II X4 965 running @3.4 GHz
GPU: ATI 5870 2 GB RAM
Resolution: 5760x1080

(I realize that most benchmarking suites won't be using this resolution, but it would be useful to figure out whatever resolution you intend to game at, and eventually when people have access to MWO, this data will be useful to compare expected FPS.)

Edited by Cyllan, 02 August 2012 - 11:16 AM.


#7 RenegadeMaster

    Member

  • PipPipPipPipPipPipPip
  • 587 posts
  • LocationUSA's Caustic Valley: Arizona

Posted 03 August 2012 - 11:22 PM

View PostCyllan, on 02 August 2012 - 11:15 AM, said:

Good start. Something that might be useful to include is a separate/pulled-out section on your specs. I realize that the info is in there, but it makes it easier to read/compare results.

I also find CPU-Z (http://www.cpuid.com...ares/cpu-z.html) & GPU-Z (http://www.techpowerup.com/gpuz/) to be handy for people who don't know their CPU & GPU specs off-hand.

Something like (in my case):

System Specs
CPU: AMD Phenom II X4 965 running @3.4 GHz
GPU: ATI 5870 2 GB RAM
Resolution: 5760x1080

(I realize that most benchmarking suites won't be using this resolution, but it would be useful to figure out whatever resolution you intend to game at, and eventually when people have access to MWO, this data will be useful to compare expected FPS.)


Thanks for the extra links! If people want to post their benchmarks here for comparison, then that's cool. I just happened to focus on giving people tips and wasn't really thinking about comparing specs since hardware components & such can vary so widely.

However, I will be posting updated benchmarks after I receive an GTX 680 from NVIDIA - It may provide an interesting case study for when a CPU might be a performance bottleneck before a GPU.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users