Friday, October 22, 2010

Gems in a Jar with RedBridge

Using JRuby embed API (RedBridge), it is easy to use Ruby gems from Java. However, packaging might not be simple, so people occasionally struggle to create a "portable" package. Being portable is important for a Java app. All stuffs of the Java app should be packaged in a jar archive, which should work on another PC, or even different OS. This is the good side of Java. This blog illustrates one solution for packaging.

I intentionally didn't use jruby command since the command hides what are going on behind. You might be exhausted by lines of "java -jar ....," sorry. But, it helps to understand a packaging process.

Directories


Since I used bundler to install gems, I created directories below to fit them to bundler.

Linden -+- lib -+- jruby -+- 1.8
+- src -+- linden
+- build

"Linden" is a top directory and the name doesn't have any special meaning. You can name it whatever you like. "lib/jruby/1.8" is the directory gems will be installed. "src" is for Java code, and "build" is for compiled *.class files.

Bundler Installation


I intentionally used jruby-complete.jar to assure gems won't be installed in the default location. The path to jruby-complete.jar can be anything since typing a full path to jruby-complete.jar works. But, I like a short path to type, so I copied jruby-complete.jar under "Linden/lib." Now, the directories/files were as in below:

Linden -+- lib -+- jruby -+- 1.8
+- jruby-complete.jar
+- src -+- linden
+- build


Then, the bundler installation went:

cd lib
java -jar jruby-complete.jar -S gem install bundler jruby-openssl --no-ri --no-rdoc -i jruby/1.8

Bundler and jruby-openssl gems were installed and the directories became as in below:

Linden -+- lib -+- jruby -+- 1.8 -+- bin -+- bundle
| +- cache -+- ...
| +- doc
| +- gems -+- bouncy-castle-java-1.5.0145.2 -+- ...
| | +- bundler-1.0.3 -+- ...
| | +- jruby-openssl-0.7.1 -+- ...

| +- specifications -+- ...
+- jruby-complete.jar
+- src -+- linden
+- build

Let's see whether gems are really installed.

export GEM_PATH=`pwd`/jruby/1.8
java -jar jruby-complete.jar -S gem list

This command should print out the installed three gems and pre-installed gems.

*** LOCAL GEMS ***

bouncy-castle-java (1.5.0145.2)
bundler (1.0.3)
columnize (0.3.1)
jruby-openssl (0.7.1)
rake (0.8.7)
rspec (1.3.0)
ruby-debug (0.10.3)
ruby-debug-base (0.10.3.2)
sources (0.0.1)

OK. Bundler was installed successfully.

Installation of other gems


Next step is to install other gems using bundler, so I used bundle command. Theoretically, PATH environment variable should work to find bundle command by ruby. Unfortunately, this didn't work in my "java -jar ..." usage. Instead, I typed a path to bundle command after -S option.

java -jar jruby-complete.jar -S jruby/1.8/bin/bundle init

Then, I added "twitter" gem to Gemfile.

# A sample Gemfile
source "http://rubygems.org"

# gem "rails"

gem "twitter"

The installation went on:

java -Xmx500m -jar jruby-complete.jar -S jruby/1.8/bin/bundle install --path=.

The java command option "-Xmx500m" is for performance and to avoid memory outage. Now, 6 gems were installed and the directories/files became:

Linden -+- lib -+- jruby -+- 1.8 -+- bin -+- bundle
| | +- httparty
| | +- oauth

| +- cache -+- ...
| +- doc
| +- gems -+- bouncy-castle-java-1.5.0145.2 -+- ...
| | +- bundler-1.0.3 -+- ...
| | +- crack-0.1.8 -+- ...
| | +- hashie-0.4.0 -+- ...
| | +- httparty-0.6.1 -+- ...

| | +- jruby-openssl-0.7.1 -+- ...
| | +- multi_json-0.0.4 -+- ...
| | +- oauth-0.4.3 -+- ...
| | +- twitter-0.9.12 -+- ...

| +- specifications -+- ...
+- jruby-complete.jar
+- Gemfile
+- Gemfile.lock

+- src -+- linden
+- build


Java code to use twitter gem


Since all gems were ready, I wrote Java code to see gems worked. This time, I relied on GEM_PATH environment variable to find gems. The fist code had Java package, linden, and class name, SearchSample. So, I created SearchSample.java file under "src/linden" directory.

package linden;

import org.jruby.embed.LocalContextScope;
import org.jruby.embed.ScriptingContainer;

public class SearchSample {
private String jarname = "sample.jar";

private SearchSample() {
String basepath = System.getProperty("user.dir");
System.out.println("basepath: " + basepath);
ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
System.out.println("jrubyhome: " + container.getHomeDirectory());
container.runScriptlet("ENV['GEM_PATH']='" + basepath + "/lib/jruby/1.8'");

String script =
"require 'rubygems'\n" +
"require 'twitter'\n" +
"require 'pp'\n" +
"pp Twitter::Search.new('#jruby').fetch.results.first";
container.runScriptlet(script);
}

public static void main(String[] args) {
new SearchSample();
}
}

Linden -+- lib -+- jruby -+- 1.8 -+- bin -+- bundle
| | +- httparty
| | +- oauth
| +- cache -+- ...
| +- doc
| +- gems -+- bouncy-castle-java-1.5.0145.2 -+- ...
| | +- bundler-1.0.3 -+- ...
| | +- crack-0.1.8 -+- ...
| | +- hashie-0.4.0 -+- ...
| | +- httparty-0.6.1 -+- ...
| | +- jruby-openssl-0.7.1 -+- ...
| | +- multi_json-0.0.4 -+- ...
| | +- oauth-0.4.3 -+- ...
| | +- twitter-0.9.12 -+- ...
| +- specifications -+- ...
+- jruby-complete.jar
+- Gemfile
+- Gemfile.lock
+- src -+- linden -+- SearchSample.java
+- build



Compile and run using rake-ant integration



Everything was ready. OK, how do I compile and run it? Of course, classic "javac" and "java" command were the options. But, these commands are not very convenient to repeat compile/run. So, I used JRuby's rake-ant integration. Yes, I wrote "Rakefile" to compile and run Java code. Isn't it nice? Here's Rakefile:

require 'ant'

namespace :ant do
task :compile => :clean do
ant.javac :srcdir => "src", :destdir => "build"
end
end

namespace :ant do
task :java => :compile do
ant.java :classname => "linden.SearchSample" do
classpath do
pathelement :location => "lib/jruby-complete.jar"
pathelement :path => "build"
end
end
end
end

require 'rake/clean'

CLEAN.include '*.class', '*.jar'

I created Rakefile under the Linden directory, so now the directories/files were:

Linden -+- lib -+- jruby -+- 1.8 -+- bin -+- bundle
| | +- httparty
| | +- oauth
| +- cache -+- ...
| +- doc
| +- gems -+- bouncy-castle-java-1.5.0145.2 -+- ...
| | +- bundler-1.0.3 -+- ...
| | +- crack-0.1.8 -+- ...
| | +- hashie-0.4.0 -+- ...
| | +- httparty-0.6.1 -+- ...
| | +- jruby-openssl-0.7.1 -+- ...
| | +- multi_json-0.0.4 -+- ...
| | +- oauth-0.4.3 -+- ...
| | +- twitter-0.9.12 -+- ...
| +- specifications -+- ...
+- jruby-complete.jar
+- Gemfile
+- Gemfile.lock
+- src -+- linden -+- SearchSample.java
+- build
+- Rakefile


When I typed "rake ant:java", the Java code above worked and printed out one tweet.

java -jar lib/jruby-complete.jar -S rake ant:java
(in /Users/yoko/Works/tmp/Linden)
basepath: /Users/yoko/Works/tmp/Linden
jrubyhome: file:/Users/yoko/Works/tmp/Linden/lib/jruby-complete.jar!/META-INF/jruby.home
{"profile_image_url"=>
"http://a3.twimg.com/profile_images/76084835/web-profile_normal.jpg",
"created_at"=>"Fri, 22 Oct 2010 15:38:09 +0000",
"from_user"=>"mccrory",
"metadata"=>{"result_type"=>"recent"},
"to_user_id"=>nil,
"text"=>
"RT @carlosqt: #IronRuby 1.1.1 a released! download from: http://ironruby.codeplex.com/ #programming #JRuby #Ruby #Rails #dotnet #code",
"id"=>28415816774,
"from_user_id"=>1757577,
"geo"=>nil,
"iso_language_code"=>"en",
"source"=>"<a href="http://twitter.com/">web</a>"}


Packaging


This time I can't rely on GEM_PATH environment variable. If GEM_PATH had worked also for the path in a jar, unfortunately, it didn't. Instead of GEM_PATH, I set all path to gems to ScriptingContainer. This was not complicated since all gems were installed in the same directory. I fixed the jar name, "sample.jar," so this name appeared in the Java code. This name could have been given via a command line argument. Edited SearchSample.java is in below:

package linden;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import java.util.zip.ZipEntry;

import org.jruby.embed.LocalContextScope;
import org.jruby.embed.ScriptingContainer;

public class SearchSample {
private String jarname = "sample.jar";

private SearchSample() throws IOException {
String basepath = System.getProperty("user.dir");
System.out.println("basepath: " + basepath);
ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
System.out.println("jrubyhome: " + container.getHomeDirectory());
container.setLoadPaths(getGemPaths(jarname, basepath));

String script =
"require 'rubygems'\n" +
"require 'twitter'\n" +
"require 'pp'\n" +
"pp Twitter::Search.new('#jruby').fetch.results.first";
container.runScriptlet(script);
}

private List<String> getGemPaths(String jarname, String basepath) throws IOException {
JarFile jarFile = new JarFile(basepath + "/" + jarname);
Enumeration<JarEntry> entries = jarFile.entries();
String gempath = "lib/jruby/1.8/gems/";
Set<String> gemnames = new HashSet<String>();
while (entries.hasMoreElements()) {
ZipEntry entry = (ZipEntry) entries.nextElement();
String entryName = entry.getName();
if (entryName.startsWith(gempath) && entryName.length() > gempath.length()) {
String n = entryName.substring(gempath.length());
String m = n.substring(0, n.indexOf("/"));
gemnames.add(m);
}
}
List<String> gemPaths = new ArrayList<String>();
for (String gem : gemnames) {
gemPaths.add("file:" + basepath + "/" + jarname + "!/lib/jruby/1.8/gems/" + gem + "/lib");
}
return gemPaths;
}


public static void main(String[] args) throws IOException {
new SearchSample();
}
}

To make jar archive, I added jar task to my Rakefile.

require 'ant'

namespace :ant do
task :compile => :clean do
ant.javac :srcdir => "src", :destdir => "build"
end
end

namespace :ant do
task :java => :compile do
ant.java :classname => "linden.SearchSample" do
classpath do
pathelement :location => "lib/jruby-complete.jar"
pathelement :path => "build"
end
end
end
end

namespace :ant do
task :jar => :compile do
ant.jar :basedir => ".", :destfile => "sample.jar" do
fileset :dir => "build" do
include :name => "**/*.class"
end
include :name => "lib/jruby/1.8/gems/**/*"
manifest do
attribute :name => "Main-Class", :value => "linden.SearchSample"
end
end
end
end


require 'rake/clean'

CLEAN.include '*.class', '*.jar'

All right, let's create jar archive.

java -jar lib/jruby-complete.jar -S rake ant:jar

The ant:jar task created the sample.jar archive in Linden directory. To test this archive was really gems in a jar, I unset GEM_PATH environment variable first. Then, I typed java command and got one tweet, Yay!

unset GEM_PATH; echo $GEM_PATH
java -cp lib/jruby-complete.jar:sample.jar linden.SearchSample

To ensure that the jar archive had gems in the jar, I moved sample.jar to a different directory and typed java command with the full path to jruby-complete.jar. It worked.

cp sample.jar ../.
cd ..
java -cp Linden/lib/jruby-complete.jar:sample.jar linden.SearchSample



This might be one answer of packaging gems in a jar and using gems from RedBridge.

3 comments:

Sunil Pradhan said...

Thanks a lot Yoko ...this article helped me a lot.But i have some doubts creating the jar.as m not familiar with ant m not able to understand how to make jar using that.

Can we make the jar manually from the command prompt.will it work?

Could you pls help me to make a jar manually without using ant?

yokolet said...

Sunil,
Of course, you can make a jar file using "jar" command. If you have JDK (not JRE), you already have jar command. Go to http://download.oracle.com/javase/tutorial/deployment/jar/build.html how to use it. It's quite similar to tar command of linux/bsd/solaris...

walt said...

if you are using Windows you cannot export with cygwin, ruby gets confused by the cygpath, but other than that this guide was helpful to me. thanks