Introduction to Xuggler for Video Manipulation

With the explosion of video in the internet, developers frequently need to manipulate video content in their applications. Xuggler is a free open-source library for Java developers which can be used to uncompress, manipulate, and compress recorded or live video in real time. Xuggler uses the very powerful FFmpeg media handling libraries under the hood, essentially playing the role of a java wrapper around them. It is the easy way to uncompress, modify, and re-compress any media file (or stream) from Java.

FFmpeg is a complete, cross-platform solution to record, convert and stream audio and video, supporting numerous formats. It is probable that you are already using it on your computer, even without knowing it. However, Xuggler’s use is not limited to just providing an easy access to the complex FFmpeg native libraries. The Xuggler dev team also constantly adds improvements to FFmpeg. You can find the latest news on the Xuggle blog, where a number of tutorials is also published. Don’t miss the Overly Simplistic Guide to Internet Video from those guys.

Let’s continue by getting FFmpeg. Note that Xuggler comes with its own (improved) FFmpeg version to avoid misconfiguration problems, so you do NOT have to manually obtain FFmpeg. Along the way, we are going to test some things directly using FFmpeg before passing control to Xuggler, so you might prefer to have the original version as a separate executable.

Go to FFmpeg downloads page and get the latest distribution, version being 0.6.1 at the time. For Linux, you download the source code from the tarball and proceed with the compilation. In Windows though, you should probably get a precompiled binary. I used the one provided by Mplayer-win32 and can be obtained from here. You will find a ffmpeg.exe executable there. Copy that in a specific folder, I chose “C:\programs\ffmpeg” and optionally add the ffmpeg.exe in your system’s path if you don’t want to write the whole path every time.

To test that the executable works correctly open a terminal and just run it with no arguments. You should see an output similar to this:

FFmpeg version SVN-r21231-Sherpya, Copyright (c) 2000-2010 Fabrice Bellard, et al.
built on Jan 16 2010 05:42:31 with gcc 4.2.5 20080919 (prerelease) [Sherpya]
libavutil 50. 7. 0 / 50. 7. 0
libavcodec 52.47. 0 / 52.47. 0
libavformat 52.47. 0 / 52.47. 0
libavdevice 52. 2. 0 / 52. 2. 0
libavfilter 1.15. 0 / 1.15. 0
libswscale 0. 8. 0 / 0. 8. 0
libpostproc 51. 2. 0 / 51. 2. 0
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]… {[outfile options] outfile}…

Use -h to get full help or, even better, run ‘man ffmpeg’

You can also use the “-h” switch, as suggested by the output, to receive a ridiculously long list of arguments and options. Better check out the online FFmpeg documentation.

The next step would be to transcode your first video, probably from a prerecorded file. My input file is a MP4 video, 4min 20sec long, with a size of 18.1MB called “myvideo.mp4”. I would like to convert that to a Flash Video, considerably lowering its quality. This is accomplished with FFmpeg very easily by issuing the following command (note the use of forward slashes for the paths):

ffmpeg.exe -i C:/myvideo.mp4 C:/myvideo.flv

Here is what the console output looks like:

FFmpeg version SVN-r21231-Sherpya, Copyright (c) 2000-2010 Fabrice Bellard, et al.
built on Jan 16 2010 05:42:31 with gcc 4.2.5 20080919 (prerelease) [Sherpya]
libavutil 50. 7. 0 / 50. 7. 0
libavcodec 52.47. 0 / 52.47. 0
libavformat 52.47. 0 / 52.47. 0
libavdevice 52. 2. 0 / 52. 2. 0
libavfilter 1.15. 0 / 1.15. 0
libswscale 0. 8. 0 / 0. 8. 0
libpostproc 51. 2. 0 / 51. 2. 0

Seems stream 1 codec frame rate differs from container frame rate: 59.92 (14981/250) -> 29.96 (14981/500)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from ‘C:/myvideo.mp4’:
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: isomavc1mp42
Duration: 00:04:20.96, start: 0.000000, bitrate: 582 kb/s
Stream #0.0(und): Audio: aac, 44100 Hz, stereo, s16, 115 kb/s
Stream #0.1(und): Video: h264, yuv420p, 480×270 [PAR 1:1 DAR 16:9], 464 kb/s, 29.96 fps, 29.96 tbr, 29962 tbn, 59.92 tbc
Output #0, flv, to ‘C:/myvideo.flv’:
Stream #0.0(und): Video: flv, yuv420p, 480×270 [PAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 1k tbn, 29.96 tbc
Stream #0.1(und): Audio: libmp3lame, 44100 Hz, stereo, s16, 64 kb/s
Stream mapping:
Stream #0.1 -> #0.0
Stream #0.0 -> #0.1
Press [q] to stop encoding
[libmp3lame @ 0038f3a0]lame: output buffer too small (buffer index: 9404, free bytes: 388)
Audio encoding failed

Ignore the “Audio encoding failed” message, there is no error. The outcome is a nice, playable FLV file named “myvideo.flv” with a size of 10.1MB.

Cool, let’s proceed with installing Xuggler. First we get the latest version from the Xuggler downloads page. As stated there:

The Xuggler is composed of two main components; a set of Java jar files, and a set of native shared libraries (.dll files on Windows, .so files on Linux, or .dylib files on Mac). To use you need to first install the native libraries, and then you can write programs that use the Xuggler.

Make sure you download the correct file matching your OS’s architecture and your Java version. For example I downloaded the 32-Bit (no 64-bit for Windows), Java 1.5 or later version for Windows, which is basically an installer.

You can find instructions on how to install the native libraries in the downloads page. For Windows, you uninstall any previous versions, run the installer and of course you reboot your computer. There is also a video describing how to install Xuggler on Microsoft Windows. After the reboot, let’s test if the installation was successful. First we check if the Xuggle path variable has been set:

C:\>echo %XUGGLE_HOME%
C:\Program Files (x86)\Xuggle

The path has been correctly set. Note that the Xuggler FFmpeg executable resides in the “%XUGGLE_HOME%/bin” folder.

Let’s play our first video by issuing the following command (replace the “c:/myvideo.mp4” with a file of yours):

java -cp “%XUGGLE_HOME%\share\java\jars\xuggle-xuggler.jar” com.xuggle.xuggler.demos.DecodeAndPlayVideo c:/myvideo.mp4

Time to write our first code with Xuggler. We are going to inspect a video file, find out its media container, and print out a summary of the contents. Meanwhile, bookmark the Xuggle API Javadocs for future reference.

Fire up your favorite IDE, create a new project and import all the JAR files found in the “%XUGGLE_HOME%/share/java/jars” folder. The example is similar to the one provided in the How To Write Your First Xuggler Application In Eclipse post.

package com.javacodegeeks.xuggler.intro;

import com.xuggle.xuggler.ICodec;
import com.xuggle.xuggler.IContainer;
import com.xuggle.xuggler.IStream;
import com.xuggle.xuggler.IStreamCoder;

public class VideoInfo {
    
    private static final String filename = "c:/myvideo.mp4";
    
    public static void main(String[] args) {
        
        // first we create a Xuggler container object
        IContainer container = IContainer.make();
        
        // we attempt to open up the container
        int result = container.open(filename, IContainer.Type.READ, null);
        
        // check if the operation was successful
        if (result<0)
            throw new RuntimeException("Failed to open media file");
        
        // query how many streams the call to open found
        int numStreams = container.getNumStreams();
        
        // query for the total duration
        long duration = container.getDuration();
        
        // query for the file size
        long fileSize = container.getFileSize();

        // query for the bit rate
        long bitRate = container.getBitRate();
        
        System.out.println("Number of streams: " + numStreams);
        System.out.println("Duration (ms): " + duration);
        System.out.println("File Size (bytes): " + fileSize);
        System.out.println("Bit Rate: " + bitRate);
        
        // iterate through the streams to print their meta data
        for (int i=0; i<numStreams; i++) {
            
            // find the stream object
            IStream stream = container.getStream(i);
            
            // get the pre-configured decoder that can decode this stream;
            IStreamCoder coder = stream.getStreamCoder();
            
            System.out.println("*** Start of Stream Info ***");
            
            System.out.printf("stream %d: ", i);
            System.out.printf("type: %s; ", coder.getCodecType());
            System.out.printf("codec: %s; ", coder.getCodecID());
            System.out.printf("duration: %s; ", stream.getDuration());
            System.out.printf("start time: %s; ", container.getStartTime());
            System.out.printf("timebase: %d/%d; ",
                 stream.getTimeBase().getNumerator(),
                 stream.getTimeBase().getDenominator());
            System.out.printf("coder tb: %d/%d; ",
                 coder.getTimeBase().getNumerator(),
                 coder.getTimeBase().getDenominator());
            System.out.println();
            
            if (coder.getCodecType() == ICodec.Type.CODEC_TYPE_AUDIO) {
                System.out.printf("sample rate: %d; ", coder.getSampleRate());
                System.out.printf("channels: %d; ", coder.getChannels());
                System.out.printf("format: %s", coder.getSampleFormat());
            } 
            else if (coder.getCodecType() == ICodec.Type.CODEC_TYPE_VIDEO) {
                System.out.printf("width: %d; ", coder.getWidth());
                System.out.printf("height: %d; ", coder.getHeight());
                System.out.printf("format: %s; ", coder.getPixelType());
                System.out.printf("frame-rate: %5.2f; ", coder.getFrameRate().getDouble());
            }
            
            System.out.println();
            System.out.println("*** End of Stream Info ***");
            
        }
        
    }

}

We start by obtaining an IContainer instance, which is a data source that contains one or more streams of audio and video data. We then attempt to open the media file and make it ready for reading. If the operation is successful, we can easily retrieve information such as the number of streams contained, the total duration, the file size and the bit rate.

Note that this information responds to the container itself. However, we can obtain meta-data regarding the individual streams that the container consists of. We use the getStream method to take reference of the corresponding stream and then an IstreamCoder which is the decoder that can decode the particular stream. From that object, we can find the stream’s codec type, codec ID and a bunch of other information.

Finally, we are able to distinguish among audio and video streams. For audio streams, we can find the sample rate used, the number of channels and the audio sample format. Likewise, for video streams, we can get dimensions (width and height), the pixel format and the frame rate.

Here is what a sample output will look like:

Number of streams: 2
Duration (ms): 260963888
File Size (bytes): 19007074
Bit Rate: 582672
*** Start of Stream Info ***
stream 0: type: CODEC_TYPE_AUDIO; codec: CODEC_ID_AAC; duration: 11507712; start time: 0; timebase: 1/44100; coder tb: 1/44100;
sample rate: 44100; channels: 2; format: FMT_S16
*** End of Stream Info ***
*** Start of Stream Info ***
stream 1: type: CODEC_TYPE_VIDEO; codec: CODEC_ID_H264; duration: 7819000; start time: 0; timebase: 1/29962; coder tb: 250/14981;
width: 480; height: 270; format: YUV420P; frame-rate: 29.96;
*** End of Stream Info ***

That’s all guys. A soft introduction to Xuggler for video manipulation. As always, you can download the Eclipse project created for this tutorial.

In the next tutorials I will show you some cooler stuff that can be done with Xuggler and FFmpeg, like video conversion and modification. So, stay tuned here at JavaCodeGeeks! And don’t forget to share!

Related Articles:

• RTMP To RTSP re-stream using wowza and xuggler
• Xuggler Tutorial: Transcoding and Media Modification
• Xuggler Tutorial: Frames Capture and Video Creation
• Install Android OS on your PC with VirtualBox