JavaFX and WEBM on Linux

Google recently announced that it has open sourced the WEBM container and the VP8 video codec (WebM Project). Not to be left behind, I wanted to explore what would it take to enable WEBM video within the JavaFX platform. But first, a word about the JavaFX Video/Audio architecture.

The JavaFX 1.3 Video/Audio architecture is made of 2 layers. On top, is a layer that presents a common interface to the rest of the JavaFX platform, and underneath is a platform specific layer. On Windows, the platform specific layer calls out to DirectShow, on Mac it calls out the CoreVideo, and on Linux and OpenSolaris it calls out to the GStreamer library. Because GStreamer is open source and it is easy to get the latest bits, I decided to look to GStreamer on Linux to see if I can make the WEBM media container work.

It so happens that when Google announced that WEBM and VP8 would be open sourced, Collabora Mutlimedia and Entropy Wave simultaneously announced that they were open sourcing their GStreamer work that supports WEBM. That announcement is here. Since then, this work has been incorporated in the main source repositories at GStreamer.

To start with, I am using the latest version of Ubuntu, 10.04 LTU, with kernel version 2.6.32-22. For GStreamer, I downloaded the latest stable builds that include the WEBM support. Specifically,

• gstreamer-0.10.29
• gst-plugins-base-0.10.29
• gst-plugins-good.0.10.23
• gst-plugins-bad-0.10.19

I started with building the gstreamer core engine. First, from the initial install of Ubuntu, I had to install a few ubuntu packages required to build the gstreamer libraries.

• g++-4.4
• libglib2.0-dev
• liboil0.3-dev
• build-essential
• bison
• flex
• yasm
• libvorbis-dev
• libtheora-dev

Then I went into the gstreamer-0.10.29 directory, and did the following:

1. ./configure
2. make
3. sudo make install

This made the basic gstreamer package, libraries and executables and installed them in /usr/local/.

I then repeated the steps for the gstreamer plugins contained in the gst-plugins-base-0.10.29, gst-plugins-good.0.10.23, and gst-plugins-bad-0.10.19 directories. (For an explanation of good, bad, and yes ugly, check out this link at GStreamer.)

I then installed the latest Java JDK for Linux from http://java.sun.com and the latest JavaFX SDK for Linux from the JavaFX site.

Next, I needed a WEBM video to test. Luckily, Google has started to convert the YouTube videos to WEBM, and this blog explains how to gain access to these videos. Because, YouTube WEBM support is still "experimental", you have to log into Google and opt-in to get access to the videos. To actually get to the video stream for use in JavaFX, I had to download the video directly to disk, because the video URL would not work outside of the browser from which I had logged into Google. When you do the search for the WEBM videos, not all of the returned listings are converted to WEBM yet. To find one that has been converted, select the listed item and if it is a WEBM video, "HTML5" will initially appear in the middle of the screen. To get to the actual video bits, I opened up the HTML source and looked for "webm", until I found the likely URL candidate. I then used that URL in the browser, and the bits were saved to the local disk in a file I named "videoplaypack.webm"

To play this in JavaFX, I wrote a simple application utilizing the JavaFX standard media classes.

package webmmovie;

import javafx.stage.Stage;
import javafx.scene.Scene;
import javafx.scene.media.MediaView;
import javafx.scene.media.MediaPlayer;
import javafx.scene.media.Media;
import javafx.scene.control.Button;
import java.io.File;

/**
 * @author jclarke
 */
var file = new File("/media/NONAME/videoplayback.webm");
var scene: Scene;

def player = MediaPlayer {
    media: Media {
        source: file.toURI().toString();
    }
}

Stage {
    title: "Application title"
    scene: scene = Scene {
        width: 500
        height: 500
        content: [
            MediaView {
                fitHeight: bind scene.height
                fitWidth: bind scene.width
                preserveRatio: true
                mediaPlayer: player
            }
            Button {
                text: "Play"
                action: function () {
                    player.stop();
                    player.play();
                }
            }
        ]
    }
}

However, when I went to run this, GStreamer presented me with an error stating it did not know how to handle the media type, "video/x-vp8". Running "gst-inspect" showed no such mapping to the vp8 plugin, though the vp8 plugin was listed. To rectify this, I had to modify the vp8 plugin source to register a "type_find" object that mapped the "video/x-vp8" mime type to the vp8 plugin. The code I modified was in "gst-pugins-bad-0.10.19/ext/vp8/plugin.c" and is shown in the following listing.

#ifdef HAVE_CONFIG_H
#include "config.h"
#endif

#include 

/*** video/x-vp8 typefinder (temporary) ***/
static GstStaticCaps vp8_caps = GST_STATIC_CAPS ("video/x-vp8");

#define VP8_CAPS (gst_static_caps_get(&vp8_caps))


GType gst_vp8_dec_get_type (void);
GType gst_vp8_enc_get_type (void);
static void gst_vp8_typefind(GstTypeFind * tf, gpointer ignore);

static gboolean
plugin_init (GstPlugin * plugin)
{
#ifdef HAVE_VP8_DECODER
  gst_element_register (plugin, "vp8dec", GST_RANK_PRIMARY,
      gst_vp8_dec_get_type ());
  gst_type_find_register(plugin, "video/x-vp8", GST_RANK_PRIMARY,
      gst_vp8_typefind, NULL, VP8_CAPS, NULL, NULL);
      
      
#endif

#ifdef HAVE_VP8_ENCODER
  gst_element_register (plugin, "vp8enc", GST_RANK_PRIMARY,
      gst_vp8_enc_get_type ());
#endif

  return TRUE;
}

#ifdef HAVE_VP8_ENCODER
static void
gst_vp8_typefind(GstTypeFind * tf, gpointer ignore) {
    /* Magic number (DKIF) + version, + length + codec-FourCC */
    guint8 *data = gst_type_find_peek (tf, 0, 12);
    if(!data) return;

    if(data[0] !='D' || data[1] !='K' || data[2] != 'I' || data[3] != 'F')
         return;
    
    // TODO Check FourCC?
    gst_type_find_suggest(tf, GST_TYPE_FIND_MAXIMUM, VP8_CAPS);
}
#endif

GST_PLUGIN_DEFINE (GST_VERSION_MAJOR,
    GST_VERSION_MINOR,
    "vp8",
    "VP8 plugin",
    plugin_init, VERSION, "LGPL", GST_PACKAGE_NAME, GST_PACKAGE_ORIGIN)

Once I remade and installed the gstreamer plugins, I re-ran my JavaFX application and all is good in the world again. The following is a screen image of the trailer for Avatar, found at this link (assuming you have signed up for the WEBM experimental access).

CAVEAT: This is using internal behavior of the JavaFX runtime and there is no guarantee that future versions of JavaFX will work in this way. 
With JavaFX 1.3,  it should be possible to add WEBM codecs to Windows DirectShow and JavaFX should then be able to play WEBM movies. However, I have not personally tried this. The WEBM project does provide a set of DirectShow Filters, found here.