If any of both programs drop your FPS by more than 15 on average you are trying to record at a resolution too high for your PCIE-BUS and CPU to handle.
Only solution to this either is to step by step reduce the resolution you play at (which of course impacts your playing experience) or record at "1/2 Frame" in MSI Afterburner repectively "Half Size" in Fraps. This way you get playable FPS and don't sacrifice too much visual quality in the end product.
Apart from that the least performance impact in both is when recording uncompressed. Which is the only option in Fraps anyhow. In MSI Afterburner both internal compression codecs (MJPEG, RTV1) even when set to 100% quality create too much artifacts for the size decrease to be worth it. The best adjustable tradeoff I could find when using VFW Compression and configuring XVid MPEG 4 Codec for a single pass compression using the HighDef/Realtime profiles. You can then calculate the target bitrate by setting a target size for a 10 minute video. I found the bigger the target bitrate/target size the less performance impact. Obviously because less compression is needed. Additionally I set MSI Afterburner to use four of my six cores so that two cores are there only for the game.
Conclusion: Hands down at the moment I will convert over to MSI Afterburner for recording. Much better options. Especially when it is about recording sound. As far as I have overlooked things by the help of "Virtual Audio Cable" and Stereo Mixer it should be possible to record ingame sound and microphone at the same time quite easily which is nice for tutorials. And most improtantly MSI Afterburner comes for free while Fraps has a pretty inflated pricetag nowadays.
Edited by Jason Parker, 02 March 2013 - 09:41 AM.