I've gotten a few eMails complaining that it takes a huge tool on some users computer performance while recording pc game-play. I cannot speak for any other recording software except Fraps because that's all I use.
Fraps records video and audio unencoded, uncompressed, raw. That means the files are huge. 30 minutes of recorded gameplay footage are about 50-70 gigabyte large, and there's absolutely no way around that. Which brings me to my first point of this how-to: get a large HDD/SSD. I'm talking about 1 single HDD/SDD, specifically just for Fraps. 250 Gigabyte ought to do it. Recording video and audio uncompressed also means a huge tool on your computer initially.
Now, the trick is to set up your system in a way that Fraps, while recording, takes as little a tool as possible on your running game.
I have 3 HDDs. 1st HDD is partitioned in C, F, G. 2nd HDD is partitioned in E, H and 3rd HDD is unpartitioned, D. C is my System partition, H is my games partition and D is my Fraps HDD.
You see, 1 HDD for my system, 1 HDD for my games, 1 HDD for Fraps. They are all physically separated, which is imperative. Why am I doing it this way? When a game runs the read/write heads goes back and forth reading the game and deploying the data to your CPU/RAM/GPU. When Fraps records it causes the read/write heads to move around and write data on the HDD.
Now imagine you have a game running and Fraps recording on the same HDD/partition. That means the read/write head has to do double the work namely reading the game and writing the recording (in simplified terms, realistically it's more than double the work) on the HDD which in turn translates into FPS drops and "lags" in the game. And remember, recording uncompressed with Fraps already takes a huge tool on your computer.
So ideally you'd want your game running on one HDD and Fraps recording on another, physically separated, (unpartitioned) HDD. Another important point is the swap/page file. Make absolutely sure your swap/page file is not on the same HDD as Fraps or your games are. I've my swap/page file stowed on C: .
It took me a bit experimenting back then but all this lead to a more stable, even increased, FPS while a massive drop in "lags". It may not work for each and every one of you, but it did for me.
On a side note: Perhaps some of you are using SSDs and can share your experiences? I don't have a SSD, yet, so I can't say how well it works on SSDs.
The biggest problem most of us face while recording gameplay clips with Fraps is its disk space usage. Fraps records everything uncompressed, continuously (free version is limited to 30 sec.) and sequential. Meaning from start till stop the "whole" clip is split into single 4GB files (For FAT32 filesystem user convenience).
The length of a single 4 GB clip depends on the resolution and the in-game scene you record it in. For example, for me a 4 GB clip has about 90 sec. worth of gameplay in 1920x1200 resolution.
Because of this uncompressed real-time recording your frame-rates drops significantly while playing. There is NO way to keep up 40+fps with Fraps running unless you have one hell of a monster system.
Imagine what happens when you want to capture a 30+ min gameplay clip. Either you buy yourself a Terabyte HDD or you use a very simple trick: recording in half-size, upscaling and sharpening. Read the How-To guide!
I often record gameplay clips from video games I'm playing with FRAPS. Usually in 1920x1080 resolution depending on the game and the FPS I get with FRAPS running. For transcoding the raw footage to something bearable to upload I usually use only VirtualDub.
Lately I've experimented with settings, and I think I found some good ones. Good video and sound quality with small file size were my goals. I think roughly 100 mb for a 5 min. clip and about 300mb for the full 10 min. in full 720p HD quality is awesome.
I'm still experimenting with sound compression but I think with 192 kb/s AC-3 ACM I have a good start.
Please bear in mind that the higher your resolution is the bigger your file size will be and vice versa. Those 100 mb respectively 300 mb are based on my 1920x1080 resolution!
This guide is intended for 16:10 and 16:9 resolutions. If you're playing on a 4:3 resolution, do not resize down to 1280*720p without maintaining aspect ratio, because the end-result will otherwise look over-stretched/-squeezed.
My settings for VirtualDub are in the picture below. If you want a very detailed explanation what each option does, check this out!
By the way, you have to have the H.264 codec installed to get it working. Download and install the latest VFW-version build here. PAY attention which version you're downloading, x86 (=32bit) or x86_64 (=64bit).
A warning about the Option "Multithreading" -> "Threads" don't set it higher than [# of your CPU Cores * 1.5] because then your whole system will start chugging. For example a Core 2 Duo would be 2 Cores * 1.5 = 3, a Q9550 would be 4 Cores * 1.5 = 6 and an AMD Phenom II X3 would be 3 Cores * 1.5 = 4.5 4.
This is by no means a guarantee that it will or will not work on your system. Each system, its components and its software are unique. You'll probably have to experiment a bit on your own. But those mentioned numbers are a good start.
It also might be necessary for you to adjust the audio volume to your liking. I have -10 dB, because the sound I record is terribad. My onboard sound sucks, period. Some people also experience asynchronous sound, here's how you can fix it.
Edit 30.05.2012 - FFDSHOW Resize Filter:
There's an external filter, a better one in my opinion, available to use with VirtualDub. It's calle ffdshow and it provides amongst other things resize, watermark and sharpen functions. Here's the guide.
On a side note: When you have uploaded your clip on YouTube and YouTube processed it the quality then will be lowered again. There's nothing you or I can do about that, it's up to YouTube.
The whole tutorial is also available on YouTube as a video:
A few links to my own HD clips:
Addendum 16.September 2009: Enabling GPU acceleration
Since version 1.9.4 it is possible to use the GPUs processing power (if your GPU supports it, which most modern GPUs do) as part of whole the transcoding process. This way you can speed up the transcoding process significantly (depending on your compression settings!).
On my computer this cut my transcoding time from an average 1h for a 10 min. clip down to some 40-ish minutes. Under "Options" -> "Preferences" enable "3D accel".
Addendum 23.10.2009: Crashes during encoding
There's currently a problem causing VirtualDub 1.9.4/5/6/7/8/9 to crash under Windows 7 while encoding sometimes. The culprit is 3D accel! Turn it off under "Preferences" -> "3D accel" and enjoy VirtualDub once again!
Addendum 16.05.2010: Option "3d accel" slowing down encoding
I've been getting mixed messages. Apparently using "3d accel" causes it to render slower than usual on some systems. I can confirm that. With 3d accel turned on, a 10min video takes about 1 hour 10-20 minutes, while turned off it takes about 1 hour max. You'll have to experiment on your system!
And if you are running on a multicore cpu, don't forget to enable threading via "Option" -> "Preferences" -> set "Threading" to 1!
Added link to ffdshow filter guide.