3D & Multimedia Across Platforms and Devices Using JOGL

Tuesday, 27 July | 4:00 PM – 6:00 PM | Los Angeles Convention Center Room 507

This session discusses the features, contributions, and future of
OpenGL, OpenCL, and OpenMax across devices and OS
exposed on top of Java using the JogAmp open-source libraries.

Sven Gothel
sgothel (at) jausoft.com

JogAmp

Until then, we work hard to have at least the following platforms working stable:

• linux-x86_64
• linux-x86
• window-x86_64
• window-x86
• macosx-x86_64
• macosx-x86

utilizing deployment methods

• Applet: applet-loader (fallback)
• Applet: JNLP (where available)
• Webstart: JNLP (where available)
• Pre-Installed JARs (static last-resort (mobile))

where Applet shall work with

• Firefox 3.x
• IE x.y
• Safari z.y

Further more a mobile device running JOGL on top of it shall be done, if time allows
and the ordered beagleboard, an OMAP 3530 device arrives.

Hope to see you there, if you are there

However, all our results will be visible as usual via jogamp.org.

However, with the great help of Michael Bien, we are close to something here:

• Main http://jogamp.org
• Wiki http://jogamp.org/wiki/
• SCM http://jogamp.org/git/
• Bugzilla http://jogamp.org/bugzilla/
• Our Forum/Mailinglist http://jogamp.762907.n3.nabble.com/
• Gathered News and Blogs http://jogamp.org/blog/
• Gathered Video Streams http://jogamp.org/stream/
• Automated build and test http://jogamp.org/chuck/
• The next JNLP base http://jogamp.org/deployment/webstart-next/

So again .. we are not dead yet

Our first intermediate-term goals are

• reproduce automated builds
• recreated all web resources on free server space
• receive a signature for the JAR files

Then we wish to continue with the actual development and deployment

Contributions are very welcome, and a new sponsor might be needed.

Further details about the project changes will be posted soon.

Thank you very much .. stay tuned.

Fixes: Swing/Java2D OpenGL usage, PBuffer usage

Adds: OpenGL 3.2 instantiation with WGL/GLX and incl. their extensions in GL3

Changes: Proper extension promotion/aliasing for OpenGL 3.0,3.1 and 3.2

The new repository homepages are:

http://kenai.com/projects/gluegen/
http://kenai.com/projects/jogl/

Create a local copy/branch of the git repository,
either anonymous:

git clone git://kenai.com/gluegen~gluegen-git gluegen
git clone git://kenai.com/jogl~jogl-git jogl
git clone git://kenai.com/jogl~jogl-demos-git jogl-demos

or via SSH and your user credential, so you can easily push back your
changes to the kenai server:

git clone ssh://username@git.kenai.com/gluegen~gluegen-git gluegen
git clone ssh://username@git.kenai.com/jogl~jogl-git jogl
git clone ssh://username@git.kenai.com/jogl~jogl-demos-git jogl-demos

TODO:

• autobuild git repository checkout (WIP)
• webpages / manual automatically checked-out by autobuild and hosted on download.java.net
• …

Abandoning SVN in our projects (gluegen, jogl and jogl-demos) for the various reasons, http://git.or.cz/gitwiki/GitSvnComparsion,
we had to choose a new SCM.

Here are some reasons why git wins over mercurial/hg,

• http://whygitisbetterthanx.com/
• http://importantshock.wordpress.com/2008/08/07/git-vs-mercurial/,
• http://news.ycombinator.com/item?id=577976

especially the branch/merge handling, maturity and wide growing user base favors git.
However, it may also be just a matter of taste ..

Some git resources:

• git wiki
• git tutorial
• git everyday
• git cheat sheet
• git crash course for svn users
• git documentation
• msysgit – git for MS Windows

SVN to git Conversion

The conversion demonstrated here is using the dev.java.net gluegen repository as an example, we have converted the jogl and jogl-demos the same way.

We have used git version 1.6.3.3.

Instead of using git-svn directly, we utilize the tool svn-all-fast-export, http://repo.or.cz/w/svn-all-fast-export.git,
which not only uses a fast local SVN repository, but also migrates all local branches and tags to git branches.
The latter is true, SVN tags converted to git branches, since SVN tags are actually branches.

Update: It turns out that the tool http://repo.or.cz/w/svn-all-fast-export.git
is not able to find the proper tag and branch points within the original repository,
hence we will use git-svn.

Directory Layout

We assume the following layout ..

/../JOGL
/../JOGL/svn-server-sync
/../JOGL/svn
/../JOGL/git-svn

All subsequent action start from the root directory JOGL.

SVN Preperation

Clone Repository

Update: This task is optional. However it is highly recommended since it dramatically speeds up the SVN checkout process.

First we create a local clone of the SVN repository. This is already tricky, since the svn versions are not backward compatible,
i.e. dev.java.net SVN repository cannot be synchronized with svn version 1.6. The latest compatible version is svn 1.5.2.
Here we used an installed svn version 1.4.4 from a MacOSX machine!

1: mkdir svn-server-sync
2: cd svn-server-sync
3: svnadmin create gluegen
4: cp gluegen/hooks/pre-revprop-change.tmpl gluegen/hooks/pre-revprop-change
5: vi gluegen/hooks/pre-revprop-change
6: chmod ugo+x gluegen/hooks/pre-revprop-change
7: svnsync --username userid --password pwd init file://`pwd`/gluegen https://gluegen.dev.java.net/svn/gluegen
8: svnsync sync file://`pwd`/gluegen

Step [1..3] create the local SVN repository.

Steps [4..6] create the svn critical hook file and makes it writeable, while step 5 edits it as follows:

diff -Nur gluegen/hooks/pre-revprop-change.tmpl gluegen/hooks/pre-revprop-change
--- gluegen/hooks/pre-revprop-change.tmpl       2009-07-08 09:18:55.000000000 -0700
+++ gluegen/hooks/pre-revprop-change    2009-07-08 09:20:06.000000000 -0700
@@ -60,6 +60,8 @@
 PROPNAME="$4"
 ACTION="$5"

+exit 0
+
 if [ "$ACTION" = "M" -a "$PROPNAME" = "svn:log" ]; then exit 0; fi

 echo "Changing revision properties other than svn:log is prohibited" >&2

Step 7 links the local SVN repository  to it’s remote master.

Step 8 finally pulls it down to our local copy.

This is done for the jogl and jogl-demos repositories as well.

Create author mapping file

Create a svn.authors file, which maps the svn author names to git author names:

#!/bin/bash
authors=$(svn log -q | grep -e '^r' | awk 'BEGIN { FS = "|" } ; { print $2 }' | sort | uniq)
for author in ${authors}; do
  echo "${author} = ${author} <${author}@dev.java.net>" >> svn.authors ;
done

Use the above script in all of your original checked-out svn repository directories (gluegen, jogl and jogl-demos) to catch all author names.
Fix the svn.authors file’s email addresses if appropriate.
Add the mandatory author names:

(no author) = First Last <no.author@domain.net>
gfxadmin = gfxadmin <gfxadmin@domain.net>
root = root <root@domain.net>
httpd = httpd <httpd@domain.net>

Move old local SVN repositories

We move away the old SVN checkouts ..

mkdir svn
mv gluegen jogl jogl-demos svn/

Converting to git

Preperation

Get the tool svn-all-fast-export, http://repo.or.cz/w/svn-all-fast-export.git, via git clone git://repo.or.cz/svn-all-fast-export.git, build and install it.

Go our parent directory JOGL, where svn-server-sync resides.

mkdir git-svn
cd git-svn

Copy the previously created svn.authors file to this directory git-svn.

Copy the file merged-branches-tags.rules out of the checked out svn-all-fast-export.git
repository, svn-all-fast-export/samples/merged-branches-tags.rules, to this directory git-svn.

Converting

cd git-svn

We customize the svn-all-fast-export configuration file, here for gluegen:
sed ’s/myproject/gluegen/g’ merged-branches-tags.rules > gluegen-merged-branches-tags.rules

git svn --authors-file=./svn.authors clone -s file://`pwd`/../svn-server-sync/gluegen gluegen 2>&1 | tee gluegen.log

This task migrates the SVN repository to git, note that all tags of the SVN repository are represented as branches within git,
which we will fix within the next step. Don’t forget to check the log file, here gluegen.log.

In case somethings goes wrong, ie the task is incompete, e.g. due to a missing author-name mapping,
fix that error (add author-name to svn.authors) and continue:

cd gluegen
git svn --authors-file=../svn.authors fetch  2>&1 | tee ../gluegen-2.log

Pick remote branches & manual tagging

cd git-svn/gluegen

The converted repository has referenced branches to the remote SVN origin,
we have to pick these and make them local.

Also, as mentioned above, SVN tags are actually branches, so we have to converted them to git tags.

git branch -a
* master
  remotes/1.0b06-maint
  remotes/1.0b06a
  remotes/JOGL_2_SANDBOX
  remotes/tags/1.0b06
  remotes/tags/1.0b06a
  remotes/trunk
git tag -l

The above shows us a few remote branches and no tags.

We have to distinguish the remote branches from the tags,
which is easy at looking at the SVN branches page.  Valid remote branches are 1.0b06-maint and the JOGL_2_SANDBOX.
The below commands create local branches from the remote ones.

git checkout --track -b 1.0b06-maint remotes/1.0b06-maint
git checkout --track -b JOGL_2_SANDBOX remotes/JOGL_2_SANDBOX

Last but not least, we have to manually create git tags from the fake SVN tags, which are visible as remotes/tags/* branches.

1: git checkout --track -b tag_1.0b06 remotes/tags/1.0b06
2: git checkout --track -b tag_1.0b06a remotes/tags/1.0b06a
3: git checkout master
4: git tag 1.0b06 tag_1.0b06
5: git tag 1.0b06a tag_1.0b06a
6: git branch -D tag_1.0b06 tag_1.0b06a
7: git branch -a
  1.0b06-maint
  JOGL_2_SANDBOX
* master
  remotes/1.0b06-maint
  remotes/1.0b06a
  remotes/JOGL_2_SANDBOX
  remotes/tags/1.0b06
  remotes/tags/1.0b06a
  remotes/trunk
8: git tag -l
  1.0b06
  1.0b06a
9: git branch -r -D 1.0b06-maint 1.0b06a JOGL_2_SANDBOX tags/1.0b06 tags/1.0b06a trunk
X: git branch -a
  1.0b06-maint
  JOGL_2_SANDBOX
* master

Step [1..2] creates local branches with the prefix tag_ from the remote ones.
Step 3 switches back to the master branch
Step [4..5] creates tags from the local branches
Step 6 deletes the local branches, which we just have tagged.
Step 7 shows the branches
Step 8 shows the tags
Step 9 removes the remote branches, which disconnects us from the svn repository
Step X: shows the branches

In case you have many branches, you may use a simple shell script, fed with a file containing the branch names,
without the ref-path (remotes/)

#! /bin/sh

branchfile=$1
shift

if [ -z "$branchfile" ] ; then
    echo Usage $0 branchfile containing branchnames without ref path
    exit 1
fi

for i in `cat $branchfile` ; do
    git checkout --track -b $i remotes/$i
    git checkout master
    git branch -r -D $i
done

In case you have many tags, you may use a simple shell script, fed with a file containing the tag names,
without the ref-path (remotes/tags/)

#! /bin/sh

tagfile=$1
shift

if [ -z "$tagfile" ] ; then
    echo Usage $0 tagfile containing tagnames without ref path
    exit 1
fi

for i in `cat $tagfile` ; do
    git checkout --track -b tag_"$i" remotes/tags/$i
    git tag $i tag_"$i"
    git checkout master
    git branch -D tag_"$i"
    git branch -r -D tags/$i
done

Cleanup

1: cd git-svn/gluegen
2: git gc
3: cd ..
4: git clone --mirror gluegen gluegen.git

Since the conversion left a lot of dirt in the git repository, and we have joggled around with a lot branches,
we want to manually garbage collect (gc) our folder with the Step 2.
Step 4 creates a bare mirror clone of the cleaned up git repository, you may like to use this as a backup.

Pre cleanup: 15MByte, post cleanup 6.6MByte, bare mirror 2.7M.

Publish git repository to the server

cd git-svn/gluegen.git

Now we push our conversion to the git server’s repository:

git push --mirror ssh://username@git.kenai.com/gluegen~gluegen-git

The mirror option ensures a 1:1 copy including all branches and tags.

git Usage

We assume to be in our root directory JOGL.

No more gluegen, jogl or jogl-demos directory exist anymore, since we moved it away to the new subdirectory svn.

Clone

1a: git clone git://kenai.com/gluegen~gluegen-git  gluegen
1b: git clone ssh://username@git.kenai.com/gluegen~gluegen-git gluegen
2: cd gluegen
3: git branch -a
4: git remote show origin

Step 1a or 1b creates the local gluegen git clone in the directory gluegen,
where 1a is the anonymous checkout and 1b uses the SSH login shell remote connection,
the latter is recommended for putbacks.

Step 2 cd’s into it.

Step 3 lists all branches.

Step 4 shows remote tracked branches

Checkout a remote tracked branch

In case you like to browse around within some branch, which is not yet local, ie remote,
you have to checkout the remote one to become local, ie the JOGL_2_SANDBOX.

git remote show origin
git checkout --track -b JOGL_2_SANDBOX origin/JOGL_2_SANDBOX
git checkout JOGL_2_SANDBOX

Then you can switch between local branches as usual ..

Switch local branches

git checkout JOGL_2_SANDBOX
git checkout master

Remove a remote branch on server

You may have created and published the new branch my_branch derived from master:

git branch --track my_branch origin/master
git checkout my_branch
.. changes ..
pit push --all

.. or you were using a remote branch ..

git checkout --track -b my_branch origin/my_branch
git checkout my_branch
.. changes ..
git push --all

Now we like to remove the branch my_branch,
and remove it on the server as well.

1: git branch -D my_branch
2: git push origin :refs/heads/my_branch

Step 1 removes the branch my_branch in the local repository.
Step 2 removes the branch my_branch on the server repository.

Another user may want to clean up the local repository, ie remove a tracked remote branch:

1: git branch -r -D origin/my_branch
2: git fetch

Last but not least …

.. participation and contributions.

Git’s model is not really layed out for central SCM,
sure it can be abused as such, as you have seen above.

It’s real powers are it’s distribution qualities!

Either a contributor has it’s own repository online via a webserver or ssh access,
or she can send the patch via email.
However it is offered to an integrator within the chain of trust (in a project),
she asks the integrator to pull the branch via the mentioned online or email content.
The integrator then verifies the patch somehow and merges it to her own branch, etc etc.
These are the key features of git,
a very reliable high performance branching and merging.

At least we shall maintain the FAQ and Wiki page, would be great.

The next thing would be to pull over the svn repository for gluegen, jogl and jogl-demos.
I personally would prefer using git from now on ..

Anybody likes to maintain jogl-utils ?

• Homepage: http://kenai.com/projects/jogl/
• Wiki: http://kenai.com/projects/jogl/pages/Home
• FAQ: http://kenai.com/projects/jogl/pages/FAQ

Actually I would say, we shall setup a WIKI page where we all could write the book and FAQ ..

http://blog.jausoft.com/jogl/

http://blog.jausoft.com/jogl-faq/

• New  JNLP Applet
• Applet-Launcher

Standard JNLP Applets are supported starting with Java 6u14, however, if you run MacOSX or an older version of  Java,
you may need to use the Applet-Launcher.

Below we utilize both, the standard JNLP mechanism of 6u14 as the default,
or falling back to our  Applet-Launcher.

You will find the JNLP file applet-gears.jnlp further below, which is being used in case of a JNLP launch,
otherwise it is ignored and the archive tags are being used, hence the applet-launcher will be started.

Note that it is important for the startup time to have the same JVM arguments in the applet tags,
as well as within the JNLP applet description, here see property sun.java2d.noddraw.
Only if JVM arguments of the JNLP applet description are satisfied by the applet tag’s JVM,
the plugin will not need to start a new JVM. OF course, the applet tag’s JVM spec may exceed the JNLP applet’s one.

<applet code="org.jdesktop.applet.util.JNLPAppletLauncher"
      width=600
      height=400
      archive="http://download.java.net/media/applet-launcher/applet-launcher.jar,
               http://download.java.net/media/jogl/jsr-231-2.x-webstart/nativewindow.all.jar,
               http://download.java.net/media/jogl/jsr-231-2.x-webstart/jogl.all.jar,
               http://download.java.net/media/gluegen/webstart-2.x/gluegen-rt.jar,
               http://download.java.net/media/jogl/jsr-231-2.x-demos-webstart/jogl-demos.jar">
   <param name="codebase_lookup" value="false">
   <param name="subapplet.classname" value="demos.applets.GearsApplet">
   <param name="subapplet.displayname" value="JOGL Gears Applet">
   <param name="noddraw.check" value="true">
   <param name="progressbar" value="true">
   <param name="jnlpNumExtensions" value="1">
   <param name="jnlpExtension1"
          value="http://download.java.net/media/jogl/jsr-231-2.x-webstart/jogl-core.jnlp">
   <param name="java_arguments" value="-Dsun.java2d.noddraw=true">
   <param name="jnlp_href" value="applet-gears.jnlp">
</applet>

Where the referenced JNLP file applet-gears.jnlp looks as follow:

<?xml version="1.0" encoding="utf-8"?>
<jnlp href="applet-gears.jnlp">
  <information>
    <title>JOGL JNLP Applet Gears Demo</title>
    <vendor>Sun Microsystems, Inc.</vendor>
    <homepage href="http://jogl-demos.dev.java.net/"/>
    <description>Gears Demo</description>
    <description kind="short">Brian Paul's Gears demo ported to Java and JOGL.</description>
    <offline-allowed/>
  </information>

    <resources>
      <j2se href="http://java.sun.com/products/autodl/j2se" version="1.4+"/>
      <property name="sun.java2d.noddraw" value="true"/>
      <jar href="http://download.java.net/media/jogl/jsr-231-2.x-demos-webstart/jogl-demos.jar" main="true"/>
      <jar href="http://download.java.net/media/jogl/jsr-231-2.x-demos-webstart/jogl-demos-util.jar"/>
      <extension name="jogl-all-awt" href="http://download.java.net/media/jogl/jsr-231-2.x-webstart/jogl-all-awt.jnlp" />
    </resources>

  <applet-desc
      name="Gears-Applet"
      main-class="demos.applets.GearsApplet"
      width="640"
      height="480">
  </applet-desc>
</jnlp>

The above demo is online here https://jogl-demos.dev.java.net/applettest-jnlp.html.

In the next part I will explain the partitioning of the new JOGL, JAR and JNLP files.

Note: A bug on MacOSX is known, launching 2 JOGL applets in 2 tabs.

The general nightly download sites are

• http://download.java.net/media/jogl/builds/nightly/

Let’s assume we are in the test directory ‘test’, from which we operate from this point.

• Get http://download.java.net/media/jogl/builds/nightly/jogl-demos.zip
  Extract the archive, i.e. unzip jogl-demos.zip
• Get http://download.java.net/media/jogl/builds/nightly/jogl-2.0-pre-20090702-linux-amd64.zip
  In case we have build a reference implementation (RI) the substring ‘pre-yyyymmdd-’ will be dropped.
  Replace the version 2.0 with the latest available and the OS/arch linux-amd64 with your test platform.
  Extract the archive, ie unzip jogl-2.0-pre-20090702-linux-amd64.zip
  Create a symbolic link or rename the archive directory from jogl-2.0-pre-20090702-linux-amd64 to jogl.
• If you like to test the binding to NV’s Cg, download and install http://developer.nvidia.com/object/cg_download.html

We assume java is in your binary search path.

On X11/Unix and MaxOSX you can test the build as follows:

sh java-run-newt.sh demos.es2.RedSquare -GL2 -GL2 -GL2
sh java-run.sh demos.gears.Gears

and with debug output

sh java-dbg-newt.sh demos.es2.RedSquare -GL2 -GL2 -GL2
sh java-dbg.sh demos.gears.Gears


On Windows you shall be able to run:

java-win32.bat demos.es2.RedSquare -GL2 -GL2 -GL2
java-win32.bat demos.gears.Gears

and with debug output

java-win32-dbg.bat demos.es2.RedSquare -GL2 -GL2 -GL2
java-win32-dbg.bat demos.gears.Gears

The windows scripts are pretty simple and flat.

The magic unix scripts offer more features and can be used either in the autobuild environment or in your development one.

• setenv-jogl.sh <JOGL-PROFILE> [<jogl-build-dir>]
  Looks up and invokes profile.jogl, finds gluegen, sets the environment variables (CLASSPATH, LD_LIBRARY_PATH & PATH).
• jogl/etc/profile.jogl <JOGL-PROFILE> [<jogl-build-dir>]
  JOGL profiles are one of JOGL_ALL, JOGL_ES1_MIN, JOGL_ES1_MAX, JOGL_ES2_MIN, JOGL_ES2_MAX, JOGL_GL2ES12_MIN, JOGL_GL2ES12_MAX, JOGL_GL2_MIN, JOGL_GL2_MAX.
  Looks up the set of JAR files necessary to satisfy the chosen JOGL-PROFILE.

This allows you to test a specific environment, ie ES2 without GL2 and AWT, using JOGL_ES2_MIN.
For this case I would recommend the native ES2 implementation from imageon SDK_OGLES2_LINUX_PCEMULATION_2.02.22.0756.