Fraps & VirtualDub: How to save disk space and enjoy good quality, small file size HD video clips

The biggest problem most of us face while recording gameplay clips with Fraps is its disk space usage. Fraps records everything uncompressed, continuously (free version is limited to 30 sec.) and sequential. Meaning from start till stop the "whole" clip is split into single 4GB files (For FAT32 filesystem user convenience).
The length of a single 4 GB clip depends on the resolution and the in-game scene you record it in. For example, for me a 4 GB clip has about 90 sec. worth of gameplay in 1920x1200 resolution.
Because of this uncompressed real-time recording your frame-rates drops significantly while playing. There is NO way to keep up 40+fps with Fraps running unless you have one hell of a monster system.
Imagine what happens when you want to capture a 30+ min gameplay clip. Either you buy yourself a Terabyte HDD or you use a very simple trick: recording in half-size, upscaling and sharpening. Read the How-To guide!
This guide is intended for 16:10 and 16:9 resolutions. It should not be used for 4:3 resolutions. One cannot upscale a 4:3 resolution into a 16:9, i.e. true 720p or 1080p, without the end result either having black bars left and right or being over-stretched/-squeezed.
Recording in Half-screen/size
Before we start with VirtualDub transcoding, upscaling and sharpening, we first have to record our gameplay. We do not record our gameplay in full-screen/size mode, as usual, but in half-screen/size:

This means if you have a game running at 1920x1200 resolution, Fraps will record it at 960x600, or if you run it at 1280x1024, Fraps will record 640x512, etc. Which in turn translates into much more gameplay time in each 4 GB file.
Another example: While recording in 1920x1200, I only got about 90 sec. (1:30 min) per 4 GB file, recording in half the resolution (i.e. 960x600) resulted in about 360 sec. (6 min) per 4 GB file. Whole 270 sec. more!
Upscaling and Sharpening
We want HD Quality, so we're going to upscale and sharpen our recorded clip. Start up VirtualDub, open the uncompressed file, change in "Video" -> "Full Processing Mode" and "Audio" -> "Full Processing Mode", then set your VirtualDub according to this screenshot:
Save your settings via "File" -> "Save processing settings" or Ctrl+S, so you can simply load them in the future. Saves a lot of time!
Addendum: Regarding "Threads"; QuadCores = 6, DualCores = 3.
You'll need VirtualDub and the H.264 codec, which you can get here. Get the VFW-version, and pay attention if you're downloading x86(32bit) or x86_64(64bit).
The settings in the screenshot are the same ones like in "VirtualDub 720p HD settings for YouTube! [Good quality, small file size]", what differs, however, is that we are also going to use the "sharpen" filter. If you want a very detailed explanation what each option in the x264 codec does, check this out!
We call the method to resize a lower resolution (960x600) to a higher resolution (1280x720, in this case), UPSCALING (sometimes also refereed to as up-converting). For your information, 1280x720 is the Standard HD (720p) resolution. The other is Ultra-HD or Full-HD (1080p, 1920x1080).
However due the upscaling method we lose quality, which is where the filter "sharpen" comes in. Upscaling from lower to higher resolutions causes the image to "blur" (A more detailed explanation). Sharpen counteract this.
You'll have to experiment with the slider in preview mode around a bit, until you find the result best to your liking. I found myself settling down between 23-30, depending on each clip.
When everything is set, start transcoding via "File" -> "Save as AVI".
Now on the other hand, if you UPSCALE from a lower resolution to HD resolution the video is not really HD material anymore, because you lose quality during upscaling. Just saying. It's at best pseudo HD.
Two important VirtualDub settings you should not forget:
"Options" -> "Preferences" -> "Threading": Set the "Video Compression Threads" to 1. This is important if you have a multi-core CPU to make use of the multiple cores and in turn speed up the transcoding process.
Since VirtualDub Version 1.9.4 there's an option to use your GPU as part of the transcoding process. Make use of it to additionally speed up your transcoding process. "Options" -> "Preferences" -> "3D accel", hack in the box.
Addendum 16.05.2010:
I've been getting mixed messages. Apparently using "3d accel" causes it to render slower than usual on some systems. I can confirm that. With 3d accel turned on, a 10min video takes about 1 hour 10-20 minutes, while turned off it takes about 1 hour max. You'll have to experiment on your system!
Addendum 23.10.2009:
There’s currently a problem causing VirtualDub 1.9.4 - 1.9.9, crashing under Windows 7. The culprit is 3D accel! Turn it off under “Preferences” -> “3D accel” and enjoy VirtualDub once again!
Update 27.01.2011:
Updated Setting screenshot






March 5th, 2012 - 19:42
Hi,
I’ve done exactly what you’ve said in this tutorial. But if I upload it to youtube the quality becomes bad. No 720p option whatsoever. Only 380p/480p with bad quality.
Any ideas?
March 5th, 2012 - 20:29
If you don’t have the option 720p then your video is not at least 1280×720. That the quality suffers AFTER uploading on YouTube is normal:
April 10th, 2011 - 16:07
Thanks you so much for this post! Working on a batch / script system to process all the files from Fraps for YouTube instead of sitting on a computer for hours doing it, and these settings worked nicely for the compression part!
As a note for anyone else who might run into error like with virtualdub:
–
Error initializing audio stream compression:
The audio codec cannot compress the source audio to the desired format.
–
Apparently with input from Fraps, AC-3 ACM Codec with 44100 Hz does not work, but 48000 Hz does.
April 6th, 2011 - 04:48
Wow, this hasn’t been posted on for a long time, but anyways OT: I’ve been using a newer version of the H.264 codec with some settings missing, and when I compressed my video, it turned out a lot larger than the original (900mb for original, 8.7gb for compressed) the video looks fine but why would it be so BIG?!
April 6th, 2011 - 04:51
Also, the audio seems to be bad quality and lags behind the video.
April 6th, 2011 - 05:03
I mean it goes ahead of the video |:P
April 6th, 2011 - 11:41
Because you’re using a different h264vfw codec. I’m always using http://komisar.gin.by/ . I guess you’re using Masternobodys.
And I don’t know why your audio is asynchronous.
September 26th, 2010 - 17:35
I am using FRAPs and recording at full resolution (1920×1080). I combined the many clips into one single video with VDub and used your x264 settings from this article. Previously I simply chose Xvid MPEG-4 settings at default which is a little over twice the file size (which I was not concerned about). I noticed that upon uploading to YouTube the text in the video (Starcraft 2 in particular) was hard to read at 1080p. Obviously YT did some re-encode on it, because it is more legible on my local copy.
I found that doing exactly what you did in this guide (except for resizing), while getting a smaller file size, the quality on YouTube looks no better (but certainly not worse). Any ideas on how to fix this?
My attempt with Xvid: http://www.youtube.com/watch?v=xu9fglEKPqA
My attempt with x264: http://www.youtube.com/watch?v=a9nxJJbhy6g
For both videos, my local copy looks much crisper. Thanks!
September 26th, 2010 - 17:45
I don’t see any difference in text quality either in xvid or x264. Looks like it’s a SC2 problem. Your local copy will always looks better. Mine do too. YouTube, like you said, always re-encodes which results in quality loss. There’s no fix for that, except maybe initially encoding it in way, way higher quality, but that’ll make the point of this tutorial moot.
September 26th, 2010 - 18:52
I just uploaded an upscaled video (from half-size 1080p to full 1080p) using your instructions here. The text looks less butchered by YouTube.
http://www.youtube.com/watch?v=wz2hvck1MSU
Many other people have perfectly crisp, readable text in their SC2 videos (HD Starcraft, Husky, a few others that do 1080p).
I just don’t get it. :/
September 26th, 2010 - 22:17
Yea, I have to pass. I have no clue what’s up
. Sorry…
September 27th, 2010 - 02:39
Just wanted to let you know that I heard back from another HD YouTuber. Resizing to 1920×1072 solves everything. Strange, huh?
Thanks again for this guide and replying to me!
September 27th, 2010 - 09:52
That makes absolutely no sense. Anyway, glad you could resolve the issue.
September 28th, 2010 - 18:41
Of course it makes sense. If you scale an object from a smaller to a higher resolution, it is usually blurred. But if you scale down from a higher to a lower resolution, then the result can only be sharper. But that does not solve the initial problem with the reference videos. I have already sent a team of experts to investigate the matter…
September 28th, 2010 - 18:48
Not really, no. “I am using FRAPs and recording at full resolution (1920×1080)”, meaning he records in 1080p.
August 28th, 2010 - 14:07
Hi.
This seems a nice guide, I’m trying to follow it but I have a problem just from the beginning. When I try to open the .avi file from fraps, Virtualdb shows the message:
“Could not locate decompressor for format ‘FPS1′ (unknown)”
I’m doing anything wrong?
(I use Win7 x64)
H.264 codec configuration seems to be fine, with the same values you use.
Also in the Virtualdb Audio options cant find the AC3 ACM codec (have downloaded and installed it, but still doesn’t appear on the list)
August 28th, 2010 - 14:22
Huh, I have no experience with Win7 64Bit. But the error you get, that’s due Fraps codec not being installed (properly). You should solve it by uninstalling and reinstalling Fraps.
Don’t know about your AC3 ACM codec, but the one I’m using is this one. I’m also not sure if it runs on 64Bit Win7. But according to the AC3 ACM changelog, it should.
Otherwise use the Lame MP3 codec.
August 28th, 2010 - 18:59
Finally it works!
Thank you so much for the help