GraphUI MediaPlayer – Feature Complete

After showing our GLMediaPlayer & GraphUI MediaPlayer progress, I can now conclude this demonstration being feature complete with text/ASS & bitmap subtitles including international fallback-font (Asia etc), mouse-over time-code & still-images on the progress-slider, proper aspect-ratio cropping, chapter visibility & hopping as well as displaying full stream information. Another milestone in our Graph Type Rendering and UI saga. (Updated w/ impl. details at the bottom)

While completing an almost unrelated work item (VTK), I finished bits & pieces for the GraphUI MediaPlayer over the last weekend and took the time to make a blog post covering the new features.

Let’s start with the full stream information, also showing centered English text/ASS subtitles, both above a blending box.

(orig image link – resolution 1920 x 1080)

Below the whole video grid is shown, exposing text/ASS subtitles in many languages, including the international fallback font Droid Sans Fallback Full visible for the Chinese subtitle in the Jimmy Dore tile 3rd row 4th column. In row 4 column 1 an English subtitle split in 3 lines is also visible, as it was streamed in one line and GraphUI MediaPlayer now re-layouts text subtitles to better use up the transparent overlay area.

(orig image link – resolution 1920 x 1080)

Bitmap subtitles are supported as well, here a HDMV PGS subtitle is shown as well as mouse-over time-code & still-images on the progress-slider, also showing chapter marks which are appended to the title in the info box. Since the video is in 4:3 aspect-ratio (1.33) it can be cropped (removed letterbox and zoom, dropping top and bottom parts), as shown below in 16:9 (1.77) as well. The original 4:3 ratio (1.33) is visible in the still-image above the progress-slider.

(orig image link – resolution 1920 x 1080)

Here another cropped aspect-ratio screenshot from 2.39 to 16:9 (1.77), having the mouse-over still-image above the progress-slider in its original aspect-ratio.

(orig image link – resolution 1920 x 1080)

Since screenshots are not telling the whole story, here the 2 minute video-recording exposing all of the above

(orig video link – resolution 1920 x 1080)

***

Adding a few notes regarding implementation details of above features (post update).

The complete application is using JogAmp and rendered with GraphUI, the UI toolkit utilizing our resolution independent GPU curve rendering Graph. TTF Font support is provided via the migrated and updated typecast module.

The text/ASS subtitles are rendered w/ Graph as well using different fonts, additional layout is also performed on the Java side. GraphUI’s MediaPlayer choses the fallback font (Chinese language) if its codepoint coverage of the subtitle is higher than with the chosen font.

Bitmap’ed subtitles are transported via textures directly from the native FFmpeg layer to the GPU, similar like the actual video frames. The latter are kept in their original format, e.g. YUV2, to reduce footprint and converted into RGBA via our own texture shader. All this is handled via our FFmpeg variant of GLMediaPlayer, its Android and OMX implementations are not yet covering these features.

The cropping or letterbox clipping + background color setting is also performed in our shader, simply via texture coordinates and the optional background color.

As shown in an earlier blog, the full clipping (culling) of shapes is done in the GraphUI group while the pixel accurate clipping for not fully outside shapes is performed in the shader.

Bottom line, most if not all performance intensive tasks are performed on the GPU – the underlying goal to enable embedded devices and handhelds as well.

***

Interested companies & organization desiring to adopt & license this technology, as well as to support the work are more than welcome to contact us at Göthel Software e.K. We are also open to other contracting opportunities across C++/Java from biz-application, medical device application to the embedded space.