[Edit 2/12/2010: Now that the tool-chain has matured some, its easier to convert that High-Def movie into a number of formats to work with your PS3, iPhone or even iPad!]
[Edit 5/02/2010: Redid the types of encoding, acknowledged the PS3 3.30 firmware that supports AC3 through the PS3 in MP4 files and added other tweaks for the iPad.]

The first and last steps for both Windows and Mac are basically the same – use MakeMKV to rip the high-def movie to your hard drive and then encode it using Handbrake. The only difference is how to convert the DTS audio to AC3 audio.

Windows

Preparation: Install the following software: MakeMKV for Windows, AudioConverter for Popcorn Hour (and any additional apps AudioConverter needs, the program will download and install them on first run), Handbrake

Workflow:

1. Put your high-def disc in the BR drive and run MakeMKV. Be sure to note where it is going to put the resulting MKV file. After an initial scan, you can uncheck all the unwanted materials – the cast interviews, blooper reel, etc. Then find the main title (usually with a number of chapters specified, and the largest movie file) and expand it. In the main title you will find a number of audio files, usually DTS or DD, as well as subtitles. Make sure you have only one audio soundtrack checked (either DTS 5.1 or DD 5.1, do not include “Master Audio” or “TrueHD” audio tracks). On some titles, you might see both. You’ll have to guess at which one is the correct one – in one case the DTS 5.1 audio was the correct one and the DD 5.1 was a mix of the audio for the visually impaired (a narrator would describe the scene initially). The resultant MKV file will be between 15-30GB.

2. Use AudioConverter to convert any DTS tracks to AC3. Select “Convert DTS Tracks” and then choose “Convert to Dolby Digital” and check the remove box to remove the old DTS track. Select your input MKV file from the first step and go!

3. From here, we will create two copies of the movie. The first one will play on all devices (iPad, iPhone 3GS, and the PS3), the second one is a “better” version for the PS3.

3a. Using Handbrake, specify the MKV file from steps 1 and 2, then select the AppleTV preset. Check the Audio tab to make sure there is both an AC3 passthrough as well as a mixed down version using AAC audio. The AAC audio version will play on your iPhone and iPad, while the AC3 version will play on AppleTV and PS3 units (the PS3 3.30 software supposedly enables AC3 surround sound – I haven’t tested this myself). Specify a file name and then either hit start or add it to the queue if you’re going to make a second encode.

3b. With the previous MKV file still as the input file, select “High Profile” and check that there is still an AC3 passthrough and AAC mix down tracks under the audio tab. Set the Picture Settings to 1280 width for 720p, or leave it at 1920 for the full 1080p picture. Specify a different output file name or location and add to queue, then start the queue in the queue window!

Mac

Preparation: Since there aren’t any Macs (as of this writing) that come with Blu-ray, you’ll need an external Blu-ray drive – it will need to have some writing capability (DVD or BR) and not a Read-Only drive otherwise OSX will just ignore it. Install MakeMKV, Handbrake, and the following tools as described in the steps below… (cribbed from here). Note: you will need to have XCode installed from your Leopard or Snow Leopard install DVD to compile some of these programs below. Its not that big of a deal, so just go install it.

1. Create a directory in your home directory.

mkdir ~/ConverterTemp

2. Install CMake from here (choose Previous Releases (2.6.4) > Darwin-Universial for the Mac).

3. Compile and install Aften (this encodes the audio to AC3)

cd ~/ConverterTemp
svn co http://aften.svn.sourceforge.net/svnroot/aften Aften
cd Aften
mkdir default
cd default
cmake ~/ConverterTemp/Aften/
make
sudo make install

4. Install MKVInfo

cd ~/ConverterTemp
curl -O http://iamthekiller.net/downloads/mkvinfo.zip
unzip mkvinfo.zip
sudo chmod 755 mkvinfo
sudo cp mkvinfo /usr/local/bin

5. Compile and install libdca

cd ~/ConverterTemp
svn co svn://svn.videolan.org/libdca/trunk libdca
cd libdca
sudo ./bootstrap
sudo ./configure
sudo make
sudo make install

6.Install MKVDTS2AC3

cd ~/ConverterTemp
curl -O http://iamthekiller.net/downloads/mkvdts2ac3.zip
unzip mkvdts2ac3.zip
cd mkvdts2ac3
sudo chmod 755 mkvdts2ac3.sh
cp ~/ConverterTemp/mkvdts2ac3/mkvdts2ac3.sh ~/ConverterTemp/mkvdts2ac3/mkvdts2ac3
sudo cp ~/ConverterTemp/mkvdts2ac3/mkvdts2ac3 /usr/local/bin/

7. Install MKVMerge

cd ~/ConverterTemp
curl -O http://iamthekiller.net/downloads/mkvmerge
sudo chmod 755 mkvmerge
sudo cp mkvmerge /usr/local/bin/

8. Install MKVExtract

cd ~/ConverterTemp
curl -O http://iamthekiller.net/downloads/mkvextract
sudo chmod 755 mkvextract
sudo cp mkvextract /usr/local/bin/

Then you can remove the temp directory in your home directory.

Workflow:

1. Put your high-def disc in the BR drive and run MakeMKV. Be sure to note where it is going to put the resulting MKV file. After an initial scan, you can uncheck all the unwanted materials – the cast interviews, blooper reel, etc. Then find the main title (usually with a number of chapters specified, and the largest movie file) and expand it. In the main title you will find a number of audio files, usually DTS or DD, as well as subtitles. Make sure you have only one audio soundtrack checked (either DTS 5.1 or DD 5.1, do not include “Master Audio” or “TrueHD” audio tracks). On some titles, you might see both. You’ll have to guess at which one is the correct one – in one case the DTS 5.1 audio was the correct one and the DD 5.1 was a mix of the audio for the visually impaired (a narrator would describe the scene initially). The resultant MKV file will be between 15-30GB.

2. If the file contains a DTS audio track (identifiable by “DTS” in MakeMKV), you will need to convert it to AC3 so the AC3 portion can be preserved in the result file for your PS3. Run the following at the command line:

mkvdts2ac3 -n <name of MKV file from step 1>.mkv

If you aren’t sure if the movie has a DTS soundtrack, don’t worry, you can always run that command on the file and it will tell you if it didn’t find a DTS track to convert to AC3. The “-n” parameter tells the app to remove the DTS track instead of leaving it in the file with the AC3 audio.

3. From here, we will create two copies of the movie. The first one will play on all devices (iPad, iPhone 3GS, and the PS3), the second one is a “better” version for the PS3.

3a. Using Handbrake, specify the MKV file from steps 1 and 2, then select the AppleTV preset. Check the Audio tab to make sure there is both an AC3 passthrough as well as a mixed down version using AAC audio. The AAC audio version will play on your iPhone and iPad, while the AC3 version will play on AppleTV and PS3 units (the PS3 3.30 software supposedly enables AC3 surround sound – I haven’t tested this myself). Specify a file name and then either hit start or add it to the queue if you’re going to make a second encode.

3b. With the previous MKV file still as the input file, select “High Profile” and check that there is still an AC3 passthrough and AAC mix down tracks under the audio tab. Set the Picture Settings to 1280 width for 720p, or leave it at 1920 for the full 1080p picture. Specify a different output file name or location and add to queue, then start the queue in the queue window!

And that should be it! Whether you’ve got a Mac or PC you should be able to convert those movies. Be wary though, converting down 1080p video can take a long time, especially if you’re not running a Core i5 or i7-based quad-core CPU!

The Improvements

The goal of Handbrake is to encode videos into the H.264 codec using the x264 engine, so we can start with those improvements. These improvements are done by a different group of developers (the x264 team) and the HB developers leverage x264 to create Handbrake. The first big boost is the instructions optimized for the Intel Nehalem architecture – up to 40% over a similarly clocked Penryn, and further improvements that have been subsequently committed to the codebase. Unfortunately I don’t have a Nehalem-based CPU to test them on (I’m waiting for motherboards that will have USB 3.0). Other non-performance code improvements have been made as well – one improvement is known as MB-Tree (macroblock tree). This optimization will track the progression of data from the current frame and how its used in future frames, so that data in the current frame that is wiped off screen very soon doesn’t get a whole lot of bits spent on it, and data that will be persisted for many frames to come will get more bits invested. Likewise weighted-prediction will increase the compression and quality of the video, but require extra processing and time.

The Handbrake software also has numerous improvements, most notably the addition of soft subtitles to allow people to turn on/off subtitles on their video instead of having them burned into the video frame. DVDs typically come with closed captioned text available, and you can choose to encode those into the video and turn them on and off through the video playback software if supported – VLC, iTunes, QuickTime X and the iPhone/iPod Touch all support it. I tested it on some raw high-def content (MPEG2 in a m2ts container and H.264 AVC in an m2ts container), it was able to handle converting it to a MP4 container (maintaining the 1080 resolution) as well as scaling it down to DVD resolution and iPhone resolution. Lastly, the video preview option will have HandBrake create a 5-60 second clip from anywhere in the video so you can check for things like proper deinterlacing as well as cropping without having to wait for the entire encode to finish (or just picking out one chapter).

Whats Been Cut

Much like the few months before your high school reunion, Handbrake developers cut the fat from the software by removing AVI container and Xvid and OGM codec support. As Handbrake is primarily for encoding video to H.264 in an MPEG-4 container, its appropriate to keep the focus on those aspects. It can still encode to MKV and supports features through MKV that aren’t supported using the MP4 container format (DTS audio passthrough).

Support was also removed for CQP (constant quantization parameter) encoding in lieu of CRF (constant rate factor). I could retype this entire page in a paragraph, but just go read it and understand that the HB developers made a sound decision. In one sentence, its another human perception vs computer analysis question that came down on the side of human perception. I would have liked to see some sort of feedback indicator on the CRF setting for novices – red for either too low or too high, yellow for the boundary area, and green for good RF values (18-22) – it was discussed at some point but not yet implemented.

What Isn’t Included

One of the top hits to my website from Google searches has to do with Handbrake and OpenCL or Grand Central technologies for the Mac. First, its inappropriate to even lay this at the feet of the Handbrake developer team because they aren’t in a position to implement them. As I mentioned above, the x264 team would be the ones responsible for implementing those technologies, and they haven’t shown much interest – they did post a Google Summer of Code opening for someone to write a Motion Estimator in CUDA (which would be restricted to Nvidia cards). The SoC is over now, and it doesn’t appear anything came of it so we’ll continue to wait. I believe it will eventually come, but it might not be for another year or two as the features of the underlying GPU hardware and languages like OpenCL and DirectCompute mature.

Another item not included is the support of WMA Pro as an input audio codec. The source library (ffmpeg) that would provide the decoding for WMA Pro had the feature added too late for the update to be included in 0.9.4. This means decoding of WMA Pro audio tracks wont be included in Handbrake until 0.9.5 (hopefully sooner than November 2010). Likewise, multi-threaded decoding is also not included (ffmpeg-mt). This isn’t quite needed yet, since the ffmpeg can currently decode input video faster in one thread than x264 can encode video in 8 threads, but as CPUs move towards 6 and 8 cores per CPU plus hyper-threading (12-16 threads) the need for multi-threaded decoding will increase – I would expect that by the time the next release hits sometime in 2010, ffmpeg-mt will be standard.

Benchmarks

Normal or iPhone presets at CRF: 20 (source: Juno DVD, all chapters; 1080i source from TiVo HD STB w/ deinterlace-slow)

Handbrake 0.9.4 Benchmark

I tested the 0.9.3 release against the 0.9.4 32- and 64-bit releases on a 4-core 2.66Ghz Penryn-based OSX computer. The source material for the iPhone and Normal tests were the Juno DVD (copied to HDD first), and the 1080i-source was an episode of The Big Bang Theory from my TiVo HD with the commercials removed. On the 1080i-source, the material was de-interlaced during the encode. You can see that the 64-bit version has better performance that the 32-bit version. The caveat is that the 64-bit version of VLC is (at the time of this post) not very stable. So if you do use the 64b version of HB, I’d recommend you pick another piece of software to remove the DVD encryption. The quality of the output was comparable for each version. The file size for the iPhone version was surprisingly larger (521MB vs 440MB), but the standard resolution version was smaller (929MB vs 1.1GB) and the 1080i version was much smaller in 0.9.4 (1.1GB vs 1.6GB).

There could be even larger gains for those who have Intel Core i5/i7-based CPUs, as x264 can take advantage of new SSE instructions in those CPUs – benchmarks from around the web indicate an additional boost of 10-25%. For those with the advanced chips, moving from 0.9.3 to 0.9.4 64-bit could result in performance increases between 40-50%.

Handbrake 0.9.4 is a much needed upgrade – the new features can really put a polish on the video for those who are in to ripping all their media to a home server, plus speed improvement from the x264 library will cut 20-30% or more off the time to encode video, which can be a while when you’re encoding in high definition. Get it, and if you can, get the Intel-64 bit version!

Since 0.9.3 (almost a year old at this point), performance has improved 20% for 32bit and 30% for 64-bit on a Core 2-generation CPU. Higher performance increases will be seen on Core i5/i7 chips due to advances in the underlying x264 library that Handbrake uses to convert the input video into H.264.

I’ll have a full review of Handbrake 0.9.4 once its released, so be sure to check back.

[Edit 11/6: The fifth snapshot has been released]

The x264 encoding engine is the basis for handbrake – it does all the heavy lifting in the video conversion from whatever source to H.264. A revision made to x264 back in August changed the way the encoder will use its bits to encode the image, and results in a higher quality picture for the same bitrate. One fantastic example of this is an animated movie from Japan streaming at 67kb/s (thats slightly faster than what a 56k modem could deliver in the late 90s), the quality isn’t the best, but the video holds together (no macroblocking or other major artifacts) at such a low bitrate.

Unfortunately, there have been a lot of posts in the HB forum since 10.6 came out about the inclusion of OpenCL or GrandCentral technologies. First, Handbrake is not the appropriate place for them – as I said above x264 is where the heavy lifting is done and that’s where these types of technologies would be implemented. While there are GPU assisted video encoding technologies out there, the problem is that they don’t produce quality output. You might get a result faster but it doesn’t look as good as what you’ll get from x264 (and x264 keeps getting faster with better output).

You can get a list of the other major changes at the forum announcement, and download it here.

[Edit 11/24: The final version has been released]

The new download page is here, and there are some new features that have landed since the last snapshot. First is better support for subtitles (importing from SRT files), as well as closed captioning support. There are many minor fixes as well, including subtle option reductions like the removal of the output extension on the Windows GUI (so you don’t have to know what the difference between MP4 and M4V extensions are).

What isn’t a part of this release was the recent enhancements to x264 – the MB-tree enhancement. The Handbrake devs rolled back to a previous version of x264 until more testing can be done and the enhancement can be refined some. Hopefully it makes it into the final 0.9.4 release.

Remember that these snapshot releases aren’t supported, they’re only for you to play with and find bugs (and report them).

[Edit 11/6: The fifth snapshot has been released]