Thursday, November 10, 2011

ClojureScript on Rails Asset Pipeline

This post is the second part subsequent to Tilt Template for ClojureScript. Now, Tilt template for ClojureScript has worked, so the template should work with Rails asset pipeline. Though it is brief, Ruby on Rails Guides: Asset Pipeline mentions "Registering gems on Tilt enabling Sprockets to find them." So, I tried that..


To make it work, I added just one simple class, which is clementine_rails.rb below:

module Clementine
class ClementineRails < Rails::Engine
initializer :register_clojurescript do |app|
app.assets.register_engine '.cljs', ClojureScriptTemplate
app.assets.register_engine '.clj', ClojureScriptTemplate
end
end
end

That's it. Then Sprockets finds ClojureScriptTemplate.


Let's try ClojureScript on asset pipeline.


I need Rails app in any case, so I created it. Undoubtedly, my Ruby is JRuby, and is bundler/rails gems installed.
rails new rouge

Then, I added "clementine" gem (a name of ClojureScript template gem) to Gemfile. Since the gem is under development, I specified the directory of the gem.

gem 'clementine', :path => "/Users/yoko/Projects/clementine"

Then, typed bundle install so that clementine gem is recognized. I need at least one controller, so, next, I created "Greetings" controller.

rails g controller Greetings index



Well, finally, Clojure stuff comes in. I put the file below in the directory, app/assets/javascripts. The filename is hello.js.clj . This is because Rails asset pipeline strips extensions off. The file, hello.js.clj, will be used as hello.js. Perhaps, the name, "hello" , works, but it doesn't look like javascript.

(ns hello)
(defn ^:export greet [name]
(str "Hellooo " name))

Finishing touch is to add a javascript snippet to use the function defined by ClojureScript. I added three lines in app/views/greetings/index.html.erb:

<script>
alert(hello.greet("ClojureScript"));
</script>
<h1>Greetings#index</h1>
<p>Find me in app/views/greetings/index.html.erb</p>



When I started Rails and requested http://localhost:3000/greetings/index, I saw the alert box successfully!


Tentatively, I set ClojureScript's option, {:optimizations :advanced}, as default. When I looked the contents of hello.js, I confirmed it was the one ClojureScript compiled.


Although the clementine gem should be improved in various ways, ClojureScript is on Rails asset pipeline. If you are interested, watch https://github.com/yokolet/clementine.

Wednesday, November 09, 2011

Tilt Template for ClojureScript

As a JVM language lover, I've written code mixed with more than one language. Ruby gems from Clojure, Clojure from Ruby, or other combinations. This blog is about using ClojureScript from JRuby. ClojureScript (https://github.com/clojure/clojurescript/) is "a new compiler for Clojure that targets JavaScript." ClojureScript uses Google Closure ( don't be confused! ) for optimization. So, instead of JavaScript, we can use Clojure's succinct syntax to write a JavaScript part.


As you know, Clojure is a JVM language. Clojure from JRuby is not so complicated. Many people know what's like that. My attempt here is using ClojureScript from Tilt (https://github.com/rtomayko/tilt). Tilt is a well known generic wrapper for Ruby template engines. It covers more than 20 template engines such as ERB, Haml, or CoffeeScript. Not only those, Tilt allows us to write other template wrappers. Thus, Tilt template for ClojureScript is possible.


Here's what I did. Whole code is on github, https://github.com/yokolet/clementine.

Firstly, I copied jars and Clojure code for ClojureScript under vendor/assets directory. The directory can be lib or other. An important point is to set Java's classpath to those jars and directories. See, clementine.rb below:

require 'rubygems'
require "clementine/version"
if defined?(RUBY_ENGINE) && RUBY_ENGINE == "jruby"
require "java"

CLOJURESCRIPT_HOME = File.dirname(__FILE__) + "/../vendor/assets"
$: << CLOJURESCRIPT_HOME + "/lib"
require 'clojure'

%w{compiler.jar goog.jar js.jar}.each {|name| $CLASSPATH << CLOJURESCRIPT_HOME + "/lib/" + name}
%w{clj cljs}.each {|path| $CLASSPATH << CLOJURESCRIPT_HOME + "/src/" + path}

require "clementine/clojurescript_engine"
require "clementine/clojurescript_template"
end

Line 6 sets CLOJURESCRIPT_HOME directory relative to clementine.rb. CLOJURESCRIPT_HOME + "/lib" directory has jars, so I set this directory to $LOAD_PATH so that JRuby can load clojure.jar. Line 8, "require 'clojure'" loads clojure.jar. You can also write "require 'clojure.jar'" instead, but extension doesn't matter for JRuby. Then, I added paths to $CLASSPATH. We can use $CLASSPATH global variable after "require 'java'". $CLASSPATH is Ruby Array and equivalent to Java's classpath. Line 10 and 11 set all paths to run ClojureScript.


The code below is a ClojureScriptEngine class, clojurescript_engine.rb:

require 'java'
%w{RT Keyword PersistentHashMap}.each do |name|
java_import "clojure.lang.#{name}"
end

module Clementine
class ClojureScriptEngine
def initialize(file, options)
@file = file
@options = options
end

def compile
cl_opts = PersistentHashMap.create(convert_options())
RT.loadResourceScript("cljs/closure.clj")
builder = RT.var("cljs.closure", "build")
builder.invoke(@file, cl_opts)
end

def convert_options()
opts = {}
@options.each do |k, v|
cl_key = Keyword.intern(k.to_s)
case
when (v.kind_of? Symbol)
cl_value = Keyword.intern(v.to_s)
else
cl_value = v
end
opts[cl_key] = cl_value
end
opts
end
end
end

This class basically performs ClojureScript REPL, (cljsc/build "hello.cljs" {:optimizations :advanced :output-to "hello.js"}). The function, cljsc/build, is defined in cljs/closure.clj file, so the code loads cljs/closure.clj by clojure.lang.RT.oadResourceScript("cljs/closure.clj"). Line 16 gets a reference to the method, then, line 17 invokes the method with arguments.
This gist https://gist.github.com/1350398 might be more understandable since it is straightforward.


Lastly, Tilt templete code, clojurescript_template.rb, became as in below:

require 'tilt/template'

module Clementine
class ClojureScriptTemplate < Tilt::Template
self.default_mime_type = 'application/javascript'

def self.engine_initialized?
true
end

def initialize_engine; end

def prepare
@engine = ClojureScriptEngine.new(@file, options)
end

def evaluate(scope, locals, &block)
@output ||= @engine.compile
end
end
end

Probably, you'd better to go to Tilt sites rather than to read my how-to about this code. So, I won't comment anything about this code.


Let's try ClojureScript Tilt template.

$ cd clementine
$ jruby -Ilib -S irb
jruby-1.6.5 :001 > require 'clementine'
=> true
jruby-1.6.5 :002 > require 'tilt'
=> true
jruby-1.6.5 :003 > Tilt.register(Clementine::ClojureScriptTemplate, 'cljs', 'clj')
=> ["cljs", "clj"]
jruby-1.6.5 :004 > template = Tilt.new('/Users/yoko/Works/tmp/clojurescript/hello.clj', 1, {:optimizations => :advanced})
=> #<Clementine::ClojureScriptTemplate:0x5513dd59 @compiled_method={}, @reader=#<Proc:0x517667bd@/usr/local/rvm/gems/jruby-1.6.5@jror/gems/tilt-1.3.3/lib/tilt/template.rb:67>, @engine=#<Clementine::ClojureScriptEngine:0x3494d313 @file="/Users/yoko/Works/tmp/clojurescript/hello.clj", @options={:optimizations=>:advanced}>, @options={:optimizations=>:advanced}, @line=1, @file="/Users/yoko/Works/tmp/clojurescript/hello.clj", @default_encoding=nil, @data="(ns hello)\n(defn ^:export greet []\n (str \"Hello \" n))">
jruby-1.6.5 :005 > template.render
=> "function b(a){throw a;}var f=true,h=null,j=false;function aa(){return function(a){return a}}function k(a){return function(){return this[a]}}function l(a){return function(){return
(snip)



It worked!


This is a preliminary implementation and far from a release at this moment. But, I could confirm ClojureScript is available from Ruby. ClojureScript will be added to Ruby's template engines. My next goal will be ClojureScript from Ruby web application frameworks. It'll be more practical.

Tuesday, September 27, 2011

RedBridge Presentaion Slides

I'll put the links to my presentation slides together here. Since I've written slides using Rails and jQuery instead of PowerPoint or OpenOffice, my slides are not on slideshare.

  • StrangeLoop 2011 - "Embedding Ruby and RubyGems Over RedBridge", http://redbridge-at-strangeloop2011.herokuapp.com/slideshow

  • When you put the cursor on the page, forward/backward arrows will appear on both sides of a main area. If you click the right arrow, you can see next page. Or, you can click bullets on the bottom.
    I used MobilySlider jQuery plugin.


  • RubyConf 2010 - "RubyGems To ALL JVM Languages", http://servletgarden-point.appspot.com/slideshow

  • When you click right and left arrows on the both side of a main area, you can go forward and backward. Also, you can click page numbers.
    I used Sudo Slider jQuery plugin.


  • JRubyKaigi 2010 - "Want To Use Ruby From Java? RedBridge Makes It Pretty Easy!", http://latest.1.servletgarden-point.appspot.com/slideshow

  • Click menus on the left side, then right side of contents will be changed. The right side is an accordion. If you click bottom/head tabs, other pages will show up. Basically, the slide is in English. But, a little amount of Japanese are in.

Sinatra on Scala

I rewrote the Servlet in my former post, Sinatra on RedBridge using Scala. Below is the code:

package chestnut

import java.io.{File, IOException}
import java.util.{ArrayList, List}

import javax.servlet.{ServletConfig, ServletException}
import javax.servlet.annotation.WebServlet
import javax.servlet.http.{HttpServlet, HttpServletRequest => HSReq, HttpServletResponse => HSResp}

import org.jruby.embed.{LocalContextScope, ScriptingContainer => Container}

import scala.collection.JavaConversions._
import scala.collection.mutable.ListBuffer


class HelloSinatra extends HttpServlet() {
private val gemPaths = new ListBuffer[String]
private var config: ServletConfig = _
private val container = new Container(LocalContextScope.THREADSAFE)
private var config_ru_path: String = _

def addGemPaths(gem_path: String) {
val gem_dir = new File(gem_path)
val gems = gem_dir.listFiles
for (gem <- gems) {
val path = gem + "/lib"
gemPaths += path
}
}

override def init(c: ServletConfig) {
config = c
val path = config.getServletContext().getRealPath("/WEB-INF/lib/vendor/jruby/1.8/gems")
addGemPaths(path)
gemPaths += config.getServletContext().getRealPath("/WEB-INF/lib/app")
config_ru_path = config.getServletContext().getRealPath("/WEB-INF/lib/app/config.ru")
}

override def doGet(request: HSReq, response: HSResp) = processHttpRequest(request, response)

override def doPost(request: HSReq, response: HSResp) = processHttpRequest(request, response)

private def processHttpRequest(request: HSReq, response: HSResp) {
container.setLoadPaths(gemPaths)
container.runScriptlet("require 'rubygems'")
container.put("config_ru_path", config_ru_path);
container.put("request", request);
container.put("config", config);
val script =
"""require 'rack'
rack_app, options = Rack::Builder.parse_file config_ru_path
require 'jruby-rack'
require 'rack/handler/servlet'
handler = Rack::Handler::Servlet.new rack_app
request.instance_variable_set(:@context, config.getServletContext)
class << request
def to_io
self.getInputStream.to_io
end
def getScriptName
self.getPathTranslated
end
def context
@context
end
end
handler.call(request)"""
var rack_response = container.runScriptlet(script);
var body : String = container.callMethod(rack_response, "getBody").asInstanceOf[String]
body += "\n\n\nand Hello from Scala Servlet!!"
response.getWriter().print(body);
container.clear();
}

}

Also, you can see the code, https://gist.github.com/1245405.


In the above Scala version, I added a small message to the Scala servlet. When you look at the result on your browser, you'll see two lines from Sinatra and Scala. The first line is from Sinatra, and the second line is from Scala servlet.

Saturday, September 24, 2011

Clojure's PersistentHashMap on JRuby

Clojure is an impressive language. Not just succinct syntax, Clojure has immutable data types for concurrency. Such Clojure's persistent data types might be useful in some cases in other languages. A question is whether we can use Clojure data types from JRuby. The answer is yes. Below is what I tried on irb:

$ rvm jruby-1.6.4
$ irb
jruby-1.6.4 :001 > require 'java'
=> true
jruby-1.6.4 :002 > $CLASSPATH << "/Users/yoko/Tools/clojure-1.3.0"
=> ["file:/Users/yoko/Tools/clojure-1.3.0/"]
jruby-1.6.4 :003 > require 'clojure-1.3.0-slim'
=> true
jruby-1.6.4 :004 > h = {"a" => 100, "b" => 300, "c" => 200}
=> {"a"=>100, "b"=>300, "c"=>200}
jruby-1.6.4 :005 > pmap = Java::clojure.lang.PersistentHashMap.create(h)
=> {"a"=>100, "b"=>300, "c"=>200}
jruby-1.6.4 :006 > pmap.class
=> Java::ClojureLang::PersistentHashMap

OK. I could successfully create Clojure's PersistentHashMap object. PersistentHashMap implements java.util.Map interface, which means the object has all of Ruby's Hash methods.

jruby-1.6.4 :007 > pmap.each {|k, v| puts "#{k} is #{v}"}
a is 100
b is 300
c is 200
=> {"a"=>100, "b"=>300, "c"=>200}
jruby-1.6.4 :008 > pmap.select {|k, v| k > "a"}
=> [["b", 300], ["c", 200]]
jruby-1.6.4 :009 > pmap.has_value?(400)
=> false

PersistenHasMap is immutable. So, when the method tries to change its data, Clojure raises exception:

jruby-1.6.4 :012 > pmap.delete_if {|k, v| k >= "b"}
Java::JavaLang::UnsupportedOperationException:
from clojure.lang.APersistentMap.remove(APersistentMap.java:273)
from org.jruby.java.proxies.MapJavaProxy$RubyHashMap.internalDelete(MapJavaProxy.java:157)
from org.jruby.RubyHash.delete(RubyHash.java:1407)
(snip)

This way, we can use Clojure's immutable data types on JRuby.

Thursday, September 08, 2011

Haml on Clojure Web App

I've wrote a couple of blog posts about making RubyGems work on JVM languages over RedBridge. Clojure is among them. So far, I could successfully make simple examples with DataMapper and UUID RubyGems. This time, I tackled a Clojure web app. The Rubygems to mix in was Haml.


To create a Clojure web app, I used Leiningen (https://github.com/technomancy/leiningen) and Ring (https://github.com/mmcgrana/ring) like I did when I wrote the blog post, JRuby on Heroku via Clojure.


I installed Leiningen, then followed the instruction:
$ lein new helloworld
$ cd helloworld
$ vi project.clj


My project.clj became below in the end:
(defproject hello-world "0.0.1"
:dependencies
[[org.clojure/clojure "1.2.1"]
[org.clojure/clojure-contrib "1.2.0"]
[org.jruby/jruby-complete "1.6.4"]
[ring/ring-jetty-adapter "0.3.9"]])

There were the choices such that [ring "0.3.9"] or others, but this worked well enough. Then, I ran:
$ lein deps

This downloaded all jar archives with dependencies and put them in lib directory. Leiningen deletes all entries in lib directory and downloaded jar archives when project.clj is updated. So, I kept in mind to stay away from lib directory.


Next, I wrote the Clojure web app. I needed to think about was how to load Haml gem. There were two choices, setting a load path and putting a gem to some directory that was listed in a classpath. The former is easier but needs to be careful not to see L/Ruby GEM_HOME that might be already there. So, I chose the latter. For just in case, I also wrote the former way commented out.


I used clojure.contrib.clsaspath to know what directories are in the classpath. As far as I checked, I could use classes and src directories. I chose src, but perhaps, there's no difference between them. I copied haml-3.1.2 and whole stuff under that direcrory from I installed the gem in Ruby way. The very important thing is the name, "haml-3.1.2/lib" should be renamed to something else. Leiningen or some other tool deletes all files under "lib" directory while deploying the app on Heroku. (This never happened on my local env) So, I changed the name from "haml-3.1.2/lib" to "haml-3.1.2/gem" . I really struggled to figure out this fact.


Usually, we write "require 'rubygems'; require 'haml'" , however, I wrote "require 'rubygems'; require 'haml-3.1.2/gem/haml'" in this case. JRuby loads Ruby code from classpath. On the classpath list, I have "src", and 'haml-3.1.2/gem/haml' is a relative path to "src" . The code is below (https://gist.github.com/1205198):

1 (ns demo.gemstoclojure
2 (:use ring.adapter.jetty)
3 (:use clojure.contrib.io)
4 (:use clojure.contrib.classpath))
5
6 (import '(org.jruby.embed ScriptingContainer LocalContextScope))
7 (def c (ScriptingContainer. LocalContextScope/THREADSAFE))
8
9 (println (classpath))
10 (println (pwd))
11
12 ;; Using $LOAD_PATH to load haml gem
13 ;(def gempath [(str (pwd) "/src/haml-3.1.2/gem")])
14 ;(. c setLoadPaths gempath)
15 ;(. c runScriptlet "require 'rubygems'; require 'haml'")
16
17 ;; Using classpath to load haml gem
18 (. c runScriptlet "require 'rubygems'; require 'haml-3.1.2/gem/haml'")
19
20 (def engineclass (. c runScriptlet "Haml::Engine"))
21 (def template
22 "%html
23 %head
24 %title
25 Hello Clojure!
26 %body
27 %h2
28 Hello Clojure from Haml!")
29 (def engine (. c callMethod engineclass "new" template Object))
30
31 (defn app [req]
32 {:status 200
33 :headers {"Content-Type" "text/html"}
34 :body (. c callMethod engine "render" String)})
35
36 (defn -main []
37 (let [port (Integer/parseInt (System/getenv "PORT"))]
38 (run-jetty app {:port port})))

The code above:

  • instantiates ScriptingContainer (line 6-7) (RedBridge!)

  • loads haml gem (line 18)

  • gets Haml::Engine class (line 20)

  • writes haml template (line 21-28)

  • instantiates Haml::Engine with template (line 29)

  • renders haml template (line 34)




Then, I wrote Procfile:

web: lein run -m demo.gemstoclojure



At last, I deployed this app on Heroku, and saw the result:

$ curl http://freezing-autumn-54.herokuapp.com
<html>
<head>
<title>
Hello Clojure!
</title>
</head>
<body>
<h2>
Hello Clojure from Haml!
</h2>
</body>
</html>



Yikes! RubyGems worked on the Clojure web app.

Tuesday, September 06, 2011

Sinatra on RedBridge

Trinidad (http://thinkincode.net/trinidad/), Kirk (https://github.com/strobecorp/kirk), TorqueBox (http://torquebox.org/), mizuno (https://github.com/matadon/mizuno ) and perhaps some more are out there. As you know, those hook up Rails and/or Sinatra app on Java web servers. They are all easy to use tools even for people don't have Java background. On the other hand, there are people who want to write Java Servlet and control Rails/Sinatra apps on a Servlet, like me. :)


In the past, to make Rails apps controllable on Servlet, I've attempted Rails on RedBridge a couple of times. For example, The second step to Rails on RedBridge and Rails on RedBridge, Scaffolded App to Work are. In those attempts, I tried to invoke rack middleware's method, call, directly using RedBrdige. It was possible because Rails' controllers have a "call(env)" method, the one of a rack middleware. The idea was simple; however, it revealed that creating "env" argument was not so simple. In my past attempts, I somehow created "env" argument on Servlet and succeeded to make just a simple app to work. But, for more complicated database models or complicated HTTP requests, that was not enough. Thus, in this attempt, I used JRuby-Rack to create "env", of course, on the Servlet. A web framework is now Sinatra. Since Sinatra is much simpler than Rails, it is easy to try my idea out.


My attempt has done in three steps. The first step is really simple and just checks whether it works. I created a Java Web Application project on Eclipse whose name is Walnut. Java web server is Tomcat 7.0.4. Nothing is special to create the project so far. After I created the project, I put jruby-complete.jar in WEB-INF/lib under the web app directory tree. On Eclipse, the directory is located in WebContent/WEB-INF/lib, which varies on IDEs. Then, set the build path on Eclipse so that jruby-complete.jar is used in both compiling and executing. Before writing a Servlet, I installed gems in the Web application directory tree.

cd WebContent/WEB-INF/lib
mkdir -p jruby/1.8
export GEM_HOME=`pwd`/jruby/1.8
java -jar jruby-complete.jar -S gem install bundler --no-ri --no-rdoc -i jruby/1.8
java -jar jruby-complete.jar -S jruby/1.8/bin/bundle init
vi Gemfile
java -jar jruby-complete.jar -S jruby/1.8/bin/bundle install --path=.

Gemfile is below:

# A sample Gemfile
source "http://rubygems.org"

gem "sinatra"
gem "jruby-rack"


The first Servlet I wrote was below, which can be seen at HelloJRuby.java:

1 package walnut;
2
3 import java.io.File;
4 import java.io.IOException;
5 import java.util.ArrayList;
6 import java.util.List;
7 import java.util.Map;
8 import java.util.concurrent.ConcurrentHashMap;
9
10 import javax.servlet.ServletConfig;
11 import javax.servlet.ServletException;
12 import javax.servlet.annotation.WebServlet;
13 import javax.servlet.http.HttpServlet;
14 import javax.servlet.http.HttpServletRequest;
15 import javax.servlet.http.HttpServletResponse;
16
17 import org.jruby.embed.LocalContextScope;
18 import org.jruby.embed.ScriptingContainer;
19
20 /**
21 * Servlet implementation class HelloJRuby
22 */
23 @WebServlet("/HelloJRuby")
24 public class HelloJRuby extends HttpServlet {
25 private static final long serialVersionUID = 1L;
26 private List<String> gemPaths;
27 private ServletConfig config;
28 private ScriptingContainer container;
29
30 /**
31 * @see HttpServlet#HttpServlet()
32 */
33 public HelloJRuby() {
34 super();
35 gemPaths = new ArrayList<String>();
36 container = new ScriptingContainer(LocalContextScope.THREADSAFE);
37 }
38
39 private void addGemPaths(String gem_path) {
40 File gem_dir = new File(gem_path);
41 File[] gems = gem_dir.listFiles();
42 for (File gem : gems) {
43 String path = gem + "/lib";
44 gemPaths.add(path);
45 }
46 }
47
48 public void init(ServletConfig config) {
49 this.config = config;
50 String path = config.getServletContext().getRealPath("/WEB-INF/lib/jruby/1.8/gems");
51 addGemPaths(path);
52 }
53
54 /**
55 * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
56 */
57 protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
58 processHttpRequest(request, response);
59 }
60
61 /**
62 * @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
63 */
64 protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
65 processHttpRequest(request, response);
66 }
67
68 private void processHttpRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
69 container.setLoadPaths(gemPaths);
70 String class_def =
71 "require 'rubygems'\n" +
72 "require 'sinatra/base'\n" +
73 "class MyApp < Sinatra::Base\n" +
74 " get '/' do\n" +
75 " 'Hello from Sinatra'\n" +
76 " end\n" +
77 "end\n" +
78 "MyApp.new";
79 Object myApp = container.runScriptlet(class_def);
80 Map<String, String> minimal_env = new ConcurrentHashMap<String, String>();
81 minimal_env.put("PATH_INFO", "/");
82 minimal_env.put("REQUEST_METHOD", "GET");
83 minimal_env.put("rack.input", "");
84 List rack_response = (List)container.callMethod(myApp, "call", minimal_env, List.class);
85 response.getWriter().print(rack_response.get(2));
86 container.clear();
87 }
88 }

The Sinatra app in HelloJRuby Servlet would be the simplest one with a minimal env argument. What I wanted to test in this Servlet was whether gems were correctly loaded or not. Since a web application must be portable, every path must be relative to a Servlet context. The line 50 gets the path to gems, which is relative to the context, then the paths are saved in a List, "gemPaths" . This gemPaths is set to the ScriptingContainer in line 69. I ran this Servlet on Eclipse and got the string "Hello from Sinatra" on a browser.


The second Servlet uses JRuby-Rack instead of creating rack request directly. This needs a little trick to make JRuby-Rack work. I didn't want to use whole lot of JRuby-Rack but just a part of it to create a rack request on a Servlet. The second Servlet is below, which is also HelloRack.java on Github:

1 package walnut;
2
3 import java.io.File;
4 import java.io.IOException;
5 import java.util.ArrayList;
6 import java.util.List;
7
8 import javax.servlet.ServletConfig;
9 import javax.servlet.ServletException;
10 import javax.servlet.annotation.WebServlet;
11 import javax.servlet.http.HttpServlet;
12 import javax.servlet.http.HttpServletRequest;
13 import javax.servlet.http.HttpServletResponse;
14
15 import org.jruby.embed.LocalContextScope;
16 import org.jruby.embed.ScriptingContainer;
17
18 /**
19 * Servlet implementation class HelloRack
20 */
21 @WebServlet("/HelloRack")
22 public class HelloRack extends HttpServlet {
23 private static final long serialVersionUID = 1L;
24 private List<String> gemPaths;
25 private ServletConfig config;
26 private ScriptingContainer container;
27
28 /**
29 * @see HttpServlet#HttpServlet()
30 */
31 public HelloRack() {
32 super();
33 gemPaths = new ArrayList<String>();
34 container = new ScriptingContainer(LocalContextScope.THREADSAFE);
35 }
36
37 private void addGemPaths(String gem_path) {
38 File gem_dir = new File(gem_path);
39 File[] gems = gem_dir.listFiles();
40 for (File gem : gems) {
41 String path = gem + "/lib";
42 gemPaths.add(path);
43 }
44 }
45
46 public void init(ServletConfig config) {
47 this.config = config;
48 String path = config.getServletContext().getRealPath("/WEB-INF/lib/jruby/1.8/gems");
49 addGemPaths(path);
50 }
51
52 /**
53 * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
54 */
55 protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
56 processHttpRequest(request, response);
57 }
58
59 /**
60 * @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
61 */
62 protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
63 processHttpRequest(request, response);
64 }
65
66 private void processHttpRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
67 container.setLoadPaths(gemPaths);
68 String class_def =
69 "require 'rubygems'\n" +
70 "require 'sinatra/base'\n" +
71 "class MyApp < Sinatra::Base\n" +
72 " get '/' do\n" +
73 " 'Hello from Sinatra over JRuby-Rack'\n" +
74 " end\n" +
75 "end\n" +
76 "MyApp.new";
77 Object myApp = container.runScriptlet(class_def);
78 container.put("rack_app", myApp);
79 String creates_handler =
80 "require 'jruby-rack'\n" +
81 "require 'rack/handler/servlet'\n" +
82 "Rack::Handler::Servlet.new rack_app";
83 Object handler = container.runScriptlet(creates_handler);
84 container.put("handler", handler);
85 container.put("request", request);
86 container.put("config", config);
87 String calls_app =
88 "request.instance_variable_set(:@context, config.getServletContext)\n" +
89 "class << request\n" +
90 " def to_io\n" +
91 " self.getInputStream.to_io\n" +
92 " end\n" +
93 " def getScriptName\n"+
94 " self.getPathTranslated\n" +
95 " end\n" +
96 " def context\n" +
97 " @context\n" +
98 " end\n" +
99 "end\n" +
100 "handler.call(request)";
101 Object rack_response = container.runScriptlet(calls_app);
102 String body = (String)container.callMethod(rack_response, "getBody", String.class);
103 response.getWriter().print(body);
104 container.clear();
105 }
106 }

Line 79-83 creates Rack Servlet handler. This is a part of JRuby-Rack and nothing special. Following part of line 84-101 is a little hack. Basically, JRuby-Rack uses an instance of HttpServletRequest as an argument of "call" method. However, some methods are lacked. The snippet here adds those methods in Ruby way, dynamically to the instance. The return value at line 101 is an instance of JRuby::Rack::Response type (response.rb). So, I called "getBody" method of the returned object. We can call other methods of JRuby::Rack::Response in the same way. When I ran this Servlet on Eclipse, I got the expected result. JRuby-Rack worked.


The third step is the one leads to a real Sinatra app. In HelloRack Servlet, I wrote whole Sinatra app in the Servlet as a string. Nobody does this to create a real app. So, the third Servlet, HelloSinatraApp, uses config.ru file, below:

require 'my_app'
run MyApp

"my_app.rb" is:

require 'sinatra/base'

class MyApp < Sinatra::Base
get '/' do
'Hello from Sinatra App over JRuby-Rack!'
end
end

I put both config.ru and my_app.rb files in WEB-INF/lib/app directory. This means I need to add WEB-INF/lib/app to a load path. The HelloSinatraApp Servlet is below, or HelloSinatraApp.java on Github:

1 package walnut;
2
3 import java.io.File;
4 import java.io.IOException;
5 import java.util.ArrayList;
6 import java.util.List;
7
8 import javax.servlet.ServletConfig;
9 import javax.servlet.ServletException;
10 import javax.servlet.annotation.WebServlet;
11 import javax.servlet.http.HttpServlet;
12 import javax.servlet.http.HttpServletRequest;
13 import javax.servlet.http.HttpServletResponse;
14
15 import org.jruby.embed.LocalContextScope;
16 import org.jruby.embed.ScriptingContainer;
17
18 /**
19 * Servlet implementation class HelloSinatraApp
20 */
21 @WebServlet("/HelloSinatraApp")
22 public class HelloSinatraApp extends HttpServlet {
23 private static final long serialVersionUID = 1L;
24 private List<String> gemPaths;
25 private ServletConfig config;
26 private ScriptingContainer container;
27 private String config_ru_path;
28
29 /**
30 * @see HttpServlet#HttpServlet()
31 */
32 public HelloSinatraApp() {
33 super();
34 gemPaths = new ArrayList<String>();
35 container = new ScriptingContainer(LocalContextScope.THREADSAFE);
36 }
37
38 private void addGemPaths(String gem_path) {
39 File gem_dir = new File(gem_path);
40 File[] gems = gem_dir.listFiles();
41 for (File gem : gems) {
42 String path = gem + "/lib";
43 gemPaths.add(path);
44 }
45 }
46
47 public void init(ServletConfig config) {
48 this.config = config;
49 String path = config.getServletContext().getRealPath("/WEB-INF/lib/jruby/1.8/gems");
50 addGemPaths(path);
51 gemPaths.add(config.getServletContext().getRealPath("/WEB-INF/lib/app"));
52 config_ru_path = config.getServletContext().getRealPath("/WEB-INF/lib/app/config.ru");
53 }
54
55 /**
56 * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
57 */
58 protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
59 processHttpRequest(request, response);
60 }
61
62 /**
63 * @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
64 */
65 protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
66 processHttpRequest(request, response);
67 }
68
69 private void processHttpRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
70 container.setLoadPaths(gemPaths);
71 container.runScriptlet("require 'rubygems'");
72 container.put("config_ru_path", config_ru_path);
73 String creates_handler =
74 "require 'rack'\n" +
75 "rack_app, options = Rack::Builder.parse_file config_ru_path\n" +
76 "require 'jruby-rack'\n" +
77 "require 'rack/handler/servlet'\n" +
78 "Rack::Handler::Servlet.new rack_app";
79 Object handler = container.runScriptlet(creates_handler);
80 container.put("handler", handler);
81 container.put("request", request);
82 container.put("config", config);
83 String calls_app =
84 "request.instance_variable_set(:@context, config.getServletContext)\n" +
85 "class << request\n" +
86 " def to_io\n" +
87 " self.getInputStream.to_io\n" +
88 " end\n" +
89 " def getScriptName\n"+
90 " self.getPathTranslated\n" +
91 " end\n" +
92 " def context\n" +
93 " @context\n" +
94 " end\n" +
95 "end\n" +
96 "handler.call(request)";
97 Object rack_response = container.runScriptlet(calls_app);
98 String body = (String)container.callMethod(rack_response, "getBody", String.class);
99 response.getWriter().print(body);
100 container.clear();
101 }
102
103 }

Let's see. The path to WEB-INF/lib/app is added in line 51. The path to config.ru is set in line 52. Line 73-79 creates Sinatra app instance and Rack Servlet handler. Other part is the same as the second Servlet, HelloRack. Again, I got the result I expected from this Servlet. I may write paths to gems and apps in web.xml to make this Servlet configurable.


Like in the above, Sinatra on RadBridge worked! This needs a knowledge of Servlet and Java web application. But, the good side is the app won't choose web servers. The app is an ordinary Java web application and works on any Servlet based web application. If you want to write a Servlet for some reasons, like me, this would be the choice.

Wednesday, July 13, 2011

JRuby on Heroku via Clojure

The big news about "Matz to Heroku" reminded me another news from Heroku. That's "Clojure on Heroku"!!! Yes, this news was for me, a JVM language lover. JVM has started running on Heroku, which means all JVM languages run on Heroku via Clojure. When I heard the news, I thought I should have tried that. So, today, I actually tried to run JRuby from Clojure. Happily, JRuby easily ran on Heroku.

I read these two, Clojure on Heroku and Getting Started With Clojure on Heroku/Cedar, and tried. These are good documents to get started. Following the documents, I wrote project.clj below:

(defproject hello-world "0.0.1"
:dependencies
[[org.clojure/clojure "1.2.1"]
[org.jruby/jruby-complete "1.6.3"]
[ring/ring-jetty-adapter "0.3.9"]])

This dependency is parsed by Leiningen(https://github.com/technomancy/leiningen). The format is [maven's groupId/ArtifactId version], so I added [org.jruby/jruby-complete "1.6.3"] in project.clj. Next, I edited web.clj as in below:

(comment "filename: web.clj")

(ns demo.web
(:use ring.adapter.jetty))

(import '(org.jruby.embed ScriptingContainer))
(def c (ScriptingContainer.))
(def version (. c runScriptlet "JRUBY_VERSION"))

(defn app [req]
{:status 200
:headers {"Content-Type" "text/plain"}
:body (str "Hello JRuby " version " from Clojure!")})

(defn -main []
(let [port (Integer/parseInt (System/getenv "PORT"))]
(run-jetty app {:port port})))

In this file, I imported org.jruby.embed.ScriptingContainer class. It is RedBridge. Then, I instantiated ScriptingContainer and evaluated JRUBY_VERSION constant which was assigned to "version" so that I could use later.

When I uploaded this project and requested from a browser, I could see the JRuby's version.
OK, JRuby worked!


I tried one more since just showing JRuby version is too simple. The second attempt was somehow proxy like code using Ruby's open-uri standard library. I wrote src/demo/jruby.clj as in below:

(comment "filename: jruby.clj")

(ns demo.jruby
(:use ring.adapter.jetty))

(import '(org.jruby.embed ScriptingContainer))
(def c (ScriptingContainer.))

(defn app [req]
{:status 200
:headers {"Content-Type" "text/html"}
:body (. c runScriptlet "require 'rubygems'; require 'open-uri'; f = open('http://www.ruby-lang.org/').read")})

(defn -main []
(let [port (Integer/parseInt (System/getenv "PORT"))]
(run-jetty app {:port port})))

In jruby.clj, I changed text/plain to text/html of response header and, in response body part, put a result of evaluating a short Ruby code. As you know, this tiny Ruby code reads HTML from http://www.ruby-lang.org/ and returns the contents. Then, changed Procfile as in below because the file is jruby.clj this time.

Procfile

web: lein run -m demo.jruby

Again, upload all to Heroku. Browsing the url, I got the output below:
Yes! It worked!



My next attempt will be Rubygems to Clojure on Heroku. I think Rubygems will possibly cover something missing in Clojure.



You might be interested about using Rubygems from Clojure. My presentation at RubyConf 2010, "Rubygems to All JVM Languages" (slide: http://servletgarden-point.appspot.com/slideshow, sample code: https://github.com/yokolet/rubyconf2010), might help you understand.

Wednesday, June 01, 2011

Extending JRuby, Compile and Jar Java Extension Code

I wrote about how to extend JRuby by Java in my blog post, Extending JRuby. At that time, I didn't compile Java code since Eclipse performed that automatically. To jar Java code, I used jar command and made an archive manually. However, this is definitely not nice. In general, Java people use maven or ant to package Java code into a single jar. Although maven and ant are among choices to package JRuby extension code, there is a more Ruby-like way of packaging. It is rake-compiler. Nokogiri uses rake-compiler for both CRuby and pure Java versions to compile and package.


Rake-compiler might be known as a cross-compiling tool, but it is also a compiling tool for Java code. Not like maven and ant, we don't write XML files. Instead, we write Rakefile. This means we compile and package Java code by a rake task on JRuby.


Let's get started. First, you need to install rake-compiler gem to your *JRuby* since it is Java code compilation. Make sure you are using JRuby.
jruby -S rake gem install rake-compiler

Next, make sure your Java code is under an *ext* directory. Rake-compiler assumes extension codes are under the *ext* directory in terms of Convention over Configuration. My Eclipse project created a *src* directory for Java sources, so I refactored the name from src to ext on Eclipse. Then, my Rakefile became as in below:
# -*- ruby -*-

require 'rubygems'
require 'rake/javaextensiontask'

Rake::JavaExtensionTask.new('commons/math/fraction') do |ext|
jruby_home = ENV['MY_RUBY_HOME'] # this is available of rvm
jars = ["#{jruby_home}/lib/jruby.jar"] + FileList['lib/*.jar']
ext.classpath = jars.map {|x| File.expand_path x}.join ':'
ext.name = 'commons/math/poplar'
end

To compile Java code for JRuby extension, requiring 'rake/javaextensiontask' and writing Rake::JavaExtensionTask are all you need. The question would be what should be in JavaExtensionTask. The constructor argument specifies the directory Java code resides. The jruby_home is to know where jruby.jar is. jars is just an array to set ext.classpath effectively. You may assign whole classpath directly to ext.classpath variable. If you are on Windows, you need to change classpath delimiter from ':' to ';' I also specified ext.name parameter. Without this, JavaExtensionTask creates an jar archive, "lib/commons/math/fraction.jar" from the constructor arguments. In my case, this name is confusing since I have lib/commons/math/fraction.rb, too. Once JRuby finds fraction.rb, JRuby happily quits searching loop. So, lib/commons/math/fraction.jar won't be loaded. You can also specify other parameters like Ruby exntesion, for example,
  • ext.gem_spec
  • ext.tmp_dir
  • ext.lib_dir
  • ext.platform
  • ext.config_options
  • ext.source_pattern
For Java extension, we can set
  • source_version
  • target_version
The default values of these parameters are 1.5.

Everything is ready. Let's compile and jar Java extension code. On your terminal, type
rake compile

Just this compiles and jars Java extension code. The created jar archive is lib/commons/math/poplar.jar. This directory is natural for JRuby extension gems. You can see other rake tasks from rake -T. On your terminal, rake clean, rake clobber and two compile tasks will show up. Next time, you might type
rake clean
rake compile


Like above, rake-compiler works to compile and jar JRuby Java extension code. This might be more familiar and better to J/Rubyists.


All of my code are on GitHub, https://github.com/yokolet/Poplar

Sunday, May 15, 2011

Rubinius on JRuby … ?

Of course, compiled Rubinius binary, *.rbc, file doesn’t work on JRuby. This post is about JRuby’s rubinius branch. I’m not sure how may people are aware that, but JRuby does have a rubinius branch. As far as I looked at that branch, it is not to be merged into master, at least, in near future. Maybe it is headius’ pet project at this moment, and is implemented as a JRuby extension. Yes, it is a JRuby extension. The branch attempts “extending JRuby,” like I wrote about in my blog post (http://yokolet.blogspot.com/2011/05/extending-jruby.html). Since this extension is still small, it would be a good practice to figure out how extension works. So, I’m going to write how you can decipher existing JRuby extension. Hopefully, this will help you to write your own extension.


To try the rubinius branch, you have two preparations to be done. The first one is to build JRuby of the rubinius branch. It is easy. Just clone out JRuby, checkout rubinius branch, and run ant command as in below:

git clone git://github.com/jruby/jruby.git
cd jruby
git checkout rubinius

ant clean-all
ant

Next, you need Rubinius source. You don’t need to build rubinius just to see how it works. But, the rubinius extension uses Rubinius’ kernal sources, which is included only in the source archive. That’s why you need the source. Rubinius source archive is available to download from http://rubini.us/. Get the archive and unzip it.


Up until now, you had rubinius branch JRuby and Rubinius source. Set the environment variables. Suppose the JRuby’s home directory is /Users/yoko/Projects/jruby, then, set JRUBY_HOME and PATH like in below. Adjust them to fit in to your system:

export JRUBY_HOME=/Users/yoko/Projects/jruby
PATH=$JRUBY_HOME/bin:$JRUBY

Suppose, Rubinius source are in /Users/yoko/Projects/rubinius-1.2.2, then set RBX_KERNEL as in below:

export RBX_KERNEL=/Users/yoko/Projects/rubinius-1.2.2/kernel



Everything should be ready. Let’s try this out.

bash-3.2 rubinius$ jruby -S irb
irb(main):001:0> require 'rubinius'
=> true

Yay! Rubinus was successfully loaded on JRuby. What’s next? Look at the RubiniusLibrary.java (https://github.com/jruby/jruby/blob/rubinius/src/org/jruby/ext/rubinius/RubiniusLibrary.java). At the line 51, “Rubinius” module is defined.

RubyModule rubinius = runtime.getOrCreateModule("Rubinius");

So, there should be a constant, Rubinius.

irb(main):002:0> Rubinius
=> Rubinius
irb(main):003:0> Rubinius.class
=> Module

So far, so good. Then, at the line 56, you can see

RubyTuple.createTupleClass(runtime);

This means you should go to RubyTuple.java (https://github.com/jruby/jruby/blob/rubinius/src/org/jruby/ext/rubinius/RubyTuple.java). On the line 55-62 of RubyTuple.java, createTupleClass method is defined. In this method, "Tuple" class is defined under the "Rubinius" module. Then, annotated methods, which have Java annotation @JRubyMethod, are defined. Looking at the rest of the code in RubyTuple.java, you can see three annotated methods (new, [], and []=), and one override method (dup) are there. Let’s try these.

irb(main):013:0> Rubinius::Tuple
=> Rubinius::Tuple
irb(main):014:0> tuple = Rubinius::Tuple.new 3
=> #<Rubinius::Tuple:0x3df89785>

This constructor needs one argument because rbNew method is defined as:

public static IRubyObject rbNew(ThreadContext context, IRubyObject tupleCls, IRubyObject cnt) {

Here's the rule. First two arguments are given internally, and the rest of the arguments are given from users. So, I typed “3” as an argument. Let’s keep going on.

irb(main):018:0> tuple[0]
=> nil
irb(main):019:0> tuple[0]=123
=> 123
irb(main):020:0> tuple[0]
=> 123
irb(main):021:0> tuple_dup = tuple.dup
=> #
irb(main):022:0> tuple_dup[0]
=> 123

All right, methods worked.


Next, get back to RubiniusLibrary.java and let’s look at the lines 84-88.

runtime.getObject().deleteConstant("Hash");
runtime.getLoadService().lockAndRequire(rbxHome + "/common/hash.rb");
RubyClass hash = (RubyClass)runtime.getClass("Hash");
hash.defineAnnotatedMethods(RubiniusHash.class);
runtime.setHash(hash);

Soooo interesting! Could you figure out what’s going on here? Hash is redefined using Rubinius code! JRuby’s Hash is entirely written in Java (https://github.com/jruby/jruby/blob/rubinius/src/org/jruby/RubyHash.java). But, once “require rubinius” is done, the Hash is totally replaced by Rubinius’ Hash, which IS written in Ruby. See? JRuby can be extended also in Ruby, not just Java. To do double check this, let’s add one line in initialize method of kernel/common/hash.rb:

def initialize(key, key_hash, value)
@key = key
@key_hash = key_hash
@value = value
@next = nil
puts "Rubinius Hash!!" # this line is added
end

Then, restart irb and re-request rubinius.

bash-3.2 rubinius$ jruby -S irb
irb(main):001:0> require 'rubinius'
=> true
irb(main):002:0> h = Hash.new
Rubinius Hash!!
=> {}

Yay, Hash is really Rubinius’ Hash. What an idea!


As I wrote, you have a lot of options for “extending JRuby.” You can extend using mature Java APIs and, also, cutting edge Ruby code. Why don’t you try this fantastic extension? It should be fun.

Monday, May 02, 2011

Extending JRuby

As pure Java Nokogiri does, we can extend JRuby writing a library backed by Java API. Other than Nokogiri, Weakling (https://github.com/headius/weakling), Warbler(https://github.com/nicksieger/warbler), JSON(https://github.com/flori/json) and more are examples of JRuby extension by Java. If you use google code search with a keyword, "BasicLibraryService," you'll find some more gems. This BasicLibraryService is a sign that the gem is implemented by Java. BasicLibraryService is an interface and has just one method, basicLoad(Ruby runtime). Simple. However, questions might come up in people's mind. What should I write in basicLoad method? How is it called? Not many answers are out there. The helpful answer I could find was a comment on LoadService.java (or LoadService19.java for 1.9 mode). But, it would be still short to write a JRuby extension for JRuby users who want to write their own. So, I wrote a sample code to see how JRuby can be extended. This sample is quite a simple one and far from real JRuby extensions such as pure Java Nokogiri or others. But, the first thing is to understand how it works. This sample will help to get started.


Before going deeper, let's look at a usage of Java API directly from JRuby. I chose Apache Commons Math API (http://commons.apache.org/math/). This API is interesting. It makes many mathematical calculations easy and natural. Among them, I picked up a fraction package. Everybody knows. In Japan, elementary school kids study how to add, subtract or common denominator, etc. It should be easy, but neither Java or Ruby doesn't have such API in a standard library.

Below is a JRuby code that uses fraction Java API directly. This code adds up reciprocals of 1 to 4, Harmonic series of n = 4. The answer is obvious, 1/1 + 1/2 + 1/3 + 1/4 = 25/12.
require 'java'
$: << '/Users/yoko/Tools/commons-math-2.2'
require 'commons-math-2.2'

java_import org.apache.commons.math.fraction.Fraction

f = Fraction.new(1, 1)
(2..4).each do |i|
f = f.add(Fraction.new(1, i))
end
puts f

My commons-math-2.2.jar is in /Users/yoko/Tools/commons-math-2.2 directory, so I added that path to $LOAD_PATH, then, required that jar archive. The .jar suffix is optional when requiring something on JRuby. JRuby searches from every possible paths adding .class, .rb, .jar or .bundle suffixes. Next, I imported Fraction class and calculated in a straightforward way.

Let's think how this code can be improved to more Ruby like one. Ruby programmer might like f.add!(something) rather than f = f.add(something). So, in this JRuby extension sample, I implemented "add!" method.

Firstly, I wrote FractionService class, which implements BasicLibraryService interface. But, wait. API design should come in before starting it because XXXService class works based on convention over configuration. Java's package name and Ruby's module structure must coincide. In my design, the Fraction class is Commons::Math::Fraction::Fraction in Ruby. This means FractionService class should be in a commons.math.fraction package, and require statement in Ruby should be "require 'commons/math/fraction/fraction.' This is how basicLoad() method is called.

Next would what we should write in basicLoad() method. In general, defining module structures/classes and object allocators are done in this method. Ola Bini's blog, "The JRuby Tutorial #4: Writing Java extensions for JRuby" (http://ola-bini.blogspot.com/2006/10/jruby-tutorial-4-writing-java.html) would be worth to read how to write the method. My simple FractionService became as in below:
package commons.math.fraction;

import java.io.IOException;

import org.jruby.Ruby;
import org.jruby.RubyClass;
import org.jruby.RubyModule;
import org.jruby.runtime.ObjectAllocator;
import org.jruby.runtime.builtin.IRubyObject;
import org.jruby.runtime.load.BasicLibraryService;

public class FractionService implements BasicLibraryService {

@Override
public boolean basicLoad(Ruby runtime) throws IOException {
RubyModule commons = runtime.defineModule("Commons");
RubyModule math = commons.defineModuleUnder("Math");
RubyModule fractionModule = math.defineModuleUnder("Fraction");
RubyClass fraction = fractionModule.defineClassUnder("Fraction", runtime.getObject(), FRACTION_ALLOCATOR);
fraction.defineAnnotatedMethods(Fraction.class);
return true;
}

private static ObjectAllocator FRACTION_ALLOCATOR = new ObjectAllocator() {
public IRubyObject allocate(Ruby runtime, RubyClass klazz) {
return new Fraction(runtime, klazz);
}
};
}


Then, I wrote commons.math.fraction.Fraction class. In this class, I defined "add!" and "to_s" methods. We can't use "!" in a method name in Java, so the Java method name is add_bang instead. Ruby method name is define in @JRubyMethod annotation. Also, I wrote Ruby's constructor method "new," which is "rbNew" method in Java. The "new" method should be a class method, so it is a static method in Java. Annotations of methods are important. Three annotated *JRubyMethods* in Fraction class get fired up by fraction.defineAnnotatedMethods(Fraction.class); in FractionService class. Because of this, we can use Java methods in Ruby. See my Fraction class below:
package commons.math.fraction;

import org.jruby.Ruby;
import org.jruby.RubyClass;
import org.jruby.RubyObject;
import org.jruby.anno.JRubyClass;
import org.jruby.anno.JRubyMethod;
import org.jruby.runtime.Arity;
import org.jruby.runtime.ThreadContext;
import org.jruby.runtime.builtin.IRubyObject;

@JRubyClass(name="Commons::Math::Fraction")
public class Fraction extends RubyObject {
private org.apache.commons.math.fraction.Fraction j_fraction = null;

@JRubyMethod(name="new", meta = true, rest = true)
public static IRubyObject rbNew(ThreadContext context, IRubyObject klazz, IRubyObject[] args) {
Fraction fraction = (Fraction) ((RubyClass)klazz).allocate();
fraction.init(context, args);
return fraction;
}

public Fraction(Ruby runtime, RubyClass klass) {
super(runtime, klass);
}

void init(ThreadContext context, IRubyObject[] args) {
Arity.checkArgumentCount(context.getRuntime(), args, 2, 2);
int numerator = (Integer) args[0].toJava(Integer.class);
int denominator = (Integer) args[1].toJava(Integer.class);
j_fraction = new org.apache.commons.math.fraction.Fraction(numerator, denominator);
}

org.apache.commons.math.fraction.Fraction getJFraction() {
return j_fraction;
}

@JRubyMethod(name = "add!")
public IRubyObject add_bang(ThreadContext context, IRubyObject other) {
if (other instanceof Fraction) {
org.apache.commons.math.fraction.Fraction other_fraction = ((Fraction)other).getJFraction();
j_fraction = j_fraction.add(other_fraction);
return this;
} else {
throw context.getRuntime().newArgumentError("argument should be Commons::Math::Fraction type");
}
}

@JRubyMethod
public IRubyObject to_s(ThreadContext context) {
return context.getRuntime().newString(j_fraction.toString());
}
}



I haven't written Rakefile for packaging at this moment, so I manually create jar archive of two Java classes.
jar -J-Duser.language=en -cvf ../lib/commons/math/poplar.jar commons



OK, all Java classes are ready for my simple JRuby extension, so let's work on Ruby code. We might have an option to require Java classes directory, but that is not nice. Since users themselves must require FractionService or other, internal change will affect users code. Besides, it doesn't look like Rubygems. So, I wrote commons_math_fraction.rb to hook up FractionService.
require 'commons-math-2.2'
require 'commons/math/fraction'

Surely, this code needs to be brush up, for example, paths. But, I kept simple since this sample is to understand the idea of extending JRuby.
Then, one more Ruby code, commons/math/fraction.rb:
require 'commons/math/poplar'
require 'commons/math/fraction/fraction'

module Commons
module Math
module Fraction
end
end
end

The first line requires poplar.jar archive, and the second does FractionService class.


Everything is ready, so let's write Ruby code using this tiny, shiny JRuby extension. The file name is fraction_sample.rb:
require 'java'

$: << '/Users/yoko/Documents/workspace/Poplar/lib'
require 'commons_math_fraction'
f = Commons::Math::Fraction::Fraction.new(1, 1)
(2..4).each do |i|
f.add!(Commons::Math::Fraction::Fraction.new(1, i))
end
puts f

Since my JRuby extension is not yet gem, GEM_PATH or other gem loading ways don't work. I set a load path to my *Poplar* project. Under the lib directory, I have jars and Ruby files.
jruby fraction_sample.rb

gave me the answer 25 / 12.


Like this, we can extend JRuby using Java API. Using Java API under the hood, we can create Rubygems. Some are re-implementation by Java like pure Java Nokgoiri. Others are Java originated Ruby API. JRuby extension is an example that Java effectively complements Ruby. So, add Ruby API to your favorite Java tools.


All codes of this sample are https://github.com/yokolet/Poplar.

Monday, April 25, 2011

Attempt to get Nokogiri work on Android


Conclusion


As a result, Nokogiri was loaded on Android successfully but didn't work on it. When I tried to parse XML document, I got tons of errors something like:
W/dalvikvm(  374): Unable to resolve superclass of Lorg/apache/xerces/dom/DeferredDocumentImpl; (2008)
W/dalvikvm( 374): Link of class 'Lorg/apache/xerces/dom/DeferredDocumentImpl;' failed

I'm pretty sure this sort of error messages complain there aren't enough interfaces of org.w3c packages defined in Android SDK. Actually, Android SDK's org.w3c API is a subset of JDK's. This is the problem. Xerces needs a full-set of org.w3c packages to work. Pure Java Nokogiri heavily relies on Xerces and nekoHTML/nekoDTD, which are built on top of Xerces. So, pure Java Nokogiri also needs the fullset of org.w3c packages to keep compatibility with libxml2 backed, CRuby version. This is why Nokogiri ended up in raising an exception as in below:
W/dalvikvm(  374): threadid=10: thread exiting with uncaught exception (group=0x40014760)
E/AndroidRuntime( 374): FATAL EXCEPTION: runWithLargeStack
E/AndroidRuntime( 374): java.lang.NoClassDefFoundError: org.apache.xerces.dom.DeferredDocumentImpl
E/AndroidRuntime( 374): at org.apache.xerces.parsers.AbstractDOMParser.startDocument(Unknown Source)
E/AndroidRuntime( 374): at org.apache.xerces.impl.dtd.XMLDTDValidator.startDocument(Unknown Source)
(snip)

Is this avoidable? Might be. Googling led me some discussions about replacing org.w3c and related other packages. If I can include Xerces' xml-apis.jar (this defines org.w3c/org.w3c.xxx, javax.xml.xxx, org.xml.xxx) in my Android app and override some of core packages, Nokogiri will start working exactly the same as a web app on Rails. But, it should not be a good workaround. Surgery on SDK might incur other applications that use replaced packages.


Probably, the best answer will create a subset of Nokogiri for Android. I'm not sure such limited version of Nokogiri still attracts users. But, I think it's better than nothing.



Thoughts on Ruboto and Android

Although my small Nokogiri app didn't work, I'm going to write about what I learned and did. This might help some poeple who want to make Ruby gems to work.

  • JDK should be 1.6.0_24 on OS X
Ruboto people might not develop JRuby on Rails on Google App Engine, but I do. Just before I tried Ruboto, I had to downgrade JDK version for Google App Engine gem. So, when I started, my JDK was 1.6.0_22. I spent pretty much time to figure out why ruboto didn't work on my PC at all. Once the JDK got back to the latest, ruboto worked like a magic. Make sure what version of JDK you are using.


  • Android API level should be 11
Not all Ruboto samples needs level 11 API. For example, samples of https://www.ibm.com/developerworks/web/library/wa-ruby/ worked on level 8. But, Nokogiri needs level 11. I'm not sure the reason, but, the activerecord (and jdbc) sample, https://github.com/ruboto/ruboto-core/wiki/Tutorial%3A-Using-an-SQLite-database-with-ActiveRecord-and-RubyGems, was also tested on level 11, which is Java backed rubygems like Nokogiri.


  • Jar archives should be moved to project's libs directory
This happens on an environment that uses custom classloader, for example, Google App Engine. So, I have all jars in my project's libs directory, https://github.com/yokolet/cranberry/tree/master/libs, so that custom classloader can load all jars. If those jars failed to be loaded, Nokogiri raises a mysterious, "undefined method `next_sibling' for class `Nokogiri::XML::Node'," error. I didn't get that error, so jars should be loaded.

Also, I commented line 18-24 out from nokogiri.rb (https://github.com/yokolet/cranberry/blob/master/assets/vendor/gems/1.8/gems/nokogiri-1.5.0.beta.4-java/lib/nokogiri.rb) so that Nokogiri doesn't try to load those jars again.


  • Configuration and setup are key to load gems
Loading gems on Ruboto was tricky. In the article, https://www.ibm.com/developerworks/web/library/wa-ruby/, the author rearranged all ruby files into single directory. This might work for small rubygems but never does for Nokogiri. For example, Nokogiri has nokogiri/html/document.rb and nokogiri/xml/document.rb. Instead, the way described in https://github.com/ruboto/ruboto-core/wiki/Tutorial%3A-Using-an-SQLite-database-with-ActiveRecord-and-RubyGems worked well. It looks complicated, but I realized that the thread based gem loading way was really necessary while I was trying other stuff. My config.rb is https://github.com/yokolet/cranberry/blob/master/assets/scripts/config.rb if you want look at it. Also, I edited src/irg/ruboto/Script.java (https://github.com/yokolet/cranberry/blob/master/src/org/ruboto/Script.java) and added "vendor" directory.

When I clicked on "Cranberry" Ruboto icon right after "rake install" said "Success," all Nokogiri files were copied to /data/data/.... directory. To cut down the time for copying, I deleted Nokogiri's test and ext directories, which are unnecessary to run Nokogiri.


  • Needs threads to become a nifty app
Android expects developers' "responsiveness" (http://developer.android.com/guide/practices/design/responsiveness.html). According the document, database or network access should not be performed on a main thread. In my Nokogiri sample, I tried to get rss feed, on the main thread firstly, so I got the error:
W/System.err(  343): org.jruby.exceptions.RaiseException: Native Exception: 'class android.os.NetworkOnMainThreadException'; Message: null; StackTrace: android.os.NetworkOnMainThreadException
W/System.err( 343): at android.os.StrictMode$AndroidBlockGuardPolicy.onNetwork(StrictMode.java:1077)
W/System.err( 343): at java.net.InetAddress.lookupHostByName(InetAddress.java:481)
(snip)

This is why config.rb uses threads to require rubygems.


  • No need to reinstall app when scripts are updated
"rake update_scripts" updates Ruby scripts of installed app. So, you don't need reinstall the app. This was a great help for me since an installing process took many many minutes.


  • ... but, it doesn't work. What's going on ???
As an Android newbie, I very often fell into troubles to get Android SDK and the app to work. Sometimes, app icons didn't show up, or rake install failed. The troubleshooting, https://github.com/ruboto/ruboto-core/wiki/Troubleshooting, was so helpful. Especially, "adb kill-server; adb start-server" commands were the best. Also, I made a rule to type "ruby -v" before I started something. As you know, rake tasks start working on CRuby. But, those won't complete tasks as you expect.

I'd like to add "uninstall" the app to the troubleshooting. You can uninstall the app on emulator as well as adb uninstall command. On the emulator, do the long-click on the icon you want to uninstall. Then, trash bin and the word "uninstall" appears. Dragging the icon on trash bin will delete the app. Or adb uninstall [package name of app] will delete the app. For example, my app's package name is com.servletgarden.ruboto.cranberry, so "adb uninstall com.servletgarden.ruboto.cranberry" deleted my app from emulator. In case you forget the package name, look at the path to XXXActivity.java file. That path corresponds to package layer.



How I made this app

In the end, I'm going to add how I made this app and how to start it. This app won't work, but for myself, to try this app in future again, I'll leave this memo.

1. install ruboto-core gem
  $ ruby -v    (double check I'm on JRuby)
$ gem install ruboto-core

2. set path to android tools
  $ cd path/to/android-sdk-mac_x86
$ PATH=`pwd`/tools:`pwd`/platform-tools:$PATH

3. create emulator image
  $ android -s create avd -f -n cranberry-11 -t android-11
This possibly creates it. Actually, I created my virtual image using Eclipse's ADT. It's way easy. Prior to using android command, I installed platforms. I also used Eclipse's ADT for that.


4. create ruboto app
  ruboto gen app --package com.servletgarden.ruboto.cranberry --target android-11
This generated cranberry directory and whole stuff under that.


5. add emulator task to Rakefile
  $ cd cranberry
$ [edit Rakefile]
(line 42-45 of https://github.com/yokolet/cranberry/blob/master/Rakefile)
Since my virutal image name is cranberry-11 (step 3) "-avd cranberry-11" is there. If the app is small, you don't need -partition-size option.


6. install nokogiri gem
  $ mkdir -p assets/vendor/gems/1.8
$ gem install --install-dir assets/vendor/gems/1.8 nokogiri -v 1.5.0.beta.4
$ rm -rf assets/vendor/gems/1.8/cache
$ rm -rf assets/vendor/gems/1.8/doc
$ pushd assets/vendor/gems/1.8/gems/nokogiri-1.5.0.beta.4-java
$ rm -rf ext test
$ popd

7. add config.rb file, one line in assets/scripts/cranberry_activity.rb and edit Script.java
line 2 of https://github.com/yokolet/cranberry/blob/master/assets/scripts/cranberry_activity.rb
line 186-188 of https://github.com/yokolet/cranberry/blob/master/src/org/ruboto/Script.java


8. start emulator
  $ rake emulator
The emulator took many minutes to boot up on my MacBook. Occasionally, it showed up without dark blue hexagons. In such case, emulator didn't work correctly. I tried a couple of times "adb kill-server" and "adb start-server." When that attempt didn't work, I shut the emulator down and did "adb kill-server," then restarted the emulator.


9. start log monitor
  $ adb logcat

This prints out verbose infos, errors, and others. It is a bit noisy, but a great help to figure out what's going on.


10. install app
  $ rake install
Be patient.


10. click Cranberry ruboto icon
Be patient again. JRuby needs long time to activate.
Ruboto default app, Figure 4 of https://www.ibm.com/developerworks/web/library/wa-ruby/ will show up.



11. edit ruby files and do "rake update_scripts"
Then, back to Apps view and clock ruboto icon. Updated version should work, or troubleshooting time starts.


Whew...!

Thursday, April 14, 2011

Nokogiri on Google App Engine

Nokogiri 1.5.0 is on its way right now. Sure, it should be soonish. This version is also the first release of pure Java Nokogiri. We call it *pure Java*, but the name might not express itself precisely. Since it is written half Ruby and half Java, so *pure JRuby* (pragdave called so) would be the best name. This pure JRuby version implements methods, which are implemented in C, using xerces, nekoHTML, jing and a couple more Java Tools, while CRuby version uses libxml and libxslt. When people use Nokogiri 1.5.0 on JRuby, they use pure Java version.
What's the beauty of pure Java Nokogiri? It works smoothly on various platforms if Java runs on them. On OS X, Linux, Windows, and even Google App Engine, Nokogiri starts working painlessly. Really frequently asked questions for Nokogiri are "I can't install Nokogiri," or "Nokogiri doesn't work." Definitely, pure Java Nokogiri doesn't have these problems.


To see pure Java Nokogiri works fine, I gave it a try on Google App Engine (GAE). As you know, GAE supports python or Java only. Using libxml is out of scope. In short, pure Java Nokogiri just worked. Easy. (Unexpectedly, I struggled to get GAE work, so I'll write how I made it.) Although I don't have many to write about, I'm going to note what I did for people who don't know they can use Nokogiri on GAE.


First, I installed gems following the instruction, https://gist.github.com/825451. The instruction says, "Do not use rvm," but, I used rvm. Using rvm is not the matter. Rubygems' version is the matter. After I installed Ruby 1.8.7 using rvm, I downgraded rubygems to 1.3.7. Don't forget, google-appengine gem needs version 1.3.7 (or before) of rubygems. Otherwise, bundler08 will fail to install gem command *bundle*. This will end up in raising an error when appengine gem tries to install gems in .gems/bundler_gems/jruby/1.8/gems directory. Make sure *bundle* is listed in there when you type "gem help commands." See http://groups.google.com/group/appengine-jruby/browse_thread/thread/2db62b1a51896098 for a detail.

You do need to have CRuby but don't need to install JRuby. Appengine gem will install jruby-jar gem when it is needed. The gem, jruby-jar, has JRuby's stdlib in a jar archive. JRuby gets stared using this jar archive. So, google-appengine gem mostly works on CRuby and uses jruby-jar gem when JRuby is needed. Therefore, all gems should be installed on CRuby. Below is what I did.

rvm 1.8.7
sudo gem install google-appengine (Since I installed rvm to /usr/local, I need *sudo*)
sudo gem install rails -v 2.3.11
sudo gem install rails_dm_datastore
sudo gem install activerecord-nulldb-adapter
mkdir rails_app; cd rails_app
curl -O http://appengine-jruby.googlecode.com/hg/demos/rails2/rails2311_appengine.rb
ruby rails2311_appengine.rb

Then, rails app is ready to run. To start app on a development server,

./script/server.sh

This should start Jetty and rails app on that.

However, I was among unlucky people. I got Segmentation fault because my Java was Java SE 6 Update 4 for Mac OS X. Googling, I followed "Comment 39" of http://code.google.com/p/googleappengine/issues/detail?id=4712. I didn't want to downgrade JDK, but there seemed no better choice. Anyways, rails app successfully worked on update 3.


Next, I added Nokogiri in Gemfile. Currently 1.5.0.beta.4 is the latest.

gem 'nokogiri', '1.5.0.beta.4'

One more. The latest version of jruby-jar gem is 1.6.1, but, sadly, the jar archive in the gem is too big to upload. JRuby 1.6.1 grew bigger. As far as I remember, 1.6.0 is also too big to upload. Again, downgrade came in. I used version 1.5.6, and my Gemfile became as in below:

# Critical default settings:
disable_system_gems
disable_rubygems
bundle_path '.gems/bundler_gems'

# List gems to bundle here:
gem 'rails_dm_datastore'
gem 'jruby-jars', '1.5.6'
gem 'jruby-openssl'
gem 'jruby-rack', '1.0.5'
gem 'rails', '2.3.11'
gem 'nokogiri', '1.5.0.beta.4'



OK, my platform has been ready. Let's create a simple Nokogiri sample. In this sample, I got the rss feed from cnn.com (http://rss.cnn.com/rss/cnn_topstories.rss), parsed it using Nokogiri, and displayed news list. Since this is just a simple sample of Nokogiri, I generated a controller only.

./script/generate controller newsfeeds index

The rss I used was like https://gist.github.com/921058. From this XML document, I collected item elements using xpath. Then, I extracted pubDate, title, link, and description children elements of item also using xpath.

# newsfeeds_controller.rb
require 'nokogiri'
require 'open-uri'

class Entry
attr_reader :title, :url, :description, :pubdate
def initialize(title, url, description, pubdate)
@title = title
@url = url
@description = description
@pubdate = pubdate
end
end

class NewsfeedsController < ApplicationController
def index
doc = Nokogiri::XML(open("http://rss.cnn.com/rss/cnn_topstories.rss"))
items = doc.xpath("//item")
@entries = []
items.each do |item|
title = item.xpath("title").text
url = item.xpath("link").text
description = item.xpath("description").text
pubdate = item.xpath("pubDate").text
@entries << Entry.new(title, url, description, pubdate)
end
end
end

# newsfeeds/index.html.erb
<h1>Newsfeeds#index</h1>
<% @entries.each do |entry| %>
<dl>
<dt><%= entry.pubdate %></dt>
<dt><b><%= entry.title %></b> [<%= link_to("Read", entry.url) %>]</dt>
<dt><%= entry.description %></dt>
</dl>
<% end %>

When I restarted the server./script/server.h and requested http://localhost:8080/newsfeeds/, I could see news list something like this.



The last thing I did was uploading. I set my application id on the line "application:" in WEB-INF/app.yaml, then uploaded it by ./script/publish.sh. Now my Nokogiri sample is working at http://4.latest.servletgarden-in-red.appspot.com/newsfeeds/.


In the end, I'm going to add a link to the blog talked about Nokogiri on Google App Engine. This would be helpful, too.

- Google App Engine, JRuby, Sinatra and some fun!


So far, pure Java Nokogiri worked just fine on Google App Engine. Give it a try!

Thursday, March 03, 2011

RedBrdige's Sharing Variables, How It Works

RedBridge's sharing variables feature is convenient to let objects back and forth between Java and Ruby. The feature makes things easy but might be hard to understand a bit. The idea of sharing variables itself is simple. On Java side, all variables to share are saved in an internal map. When Ruby code is parsed/evaluated, all variables in the map are injected to Ruby runtime. All variables (local vars are slightly different) used in Ruby code are eligible to be retrieved after the evaluation. When the variable is requested to be retrieved, the value is grabbed from Ruby runtime (lazy mode). Or, when the evaluation finishes, all variables used in Ruby code are retrieved and saved in the map on the Java side (non-lazy mode). However, it depends on choices of both local context type and local variable behavior. Also, it is Ruby's receiver and scope aware. Besides, there were bugs, which have made people confused. (I've fixed several bugs in this area in JRuby 1.6.0RC1/RC2/RC3) So, I'll write about how it should work for clarity.


1. Lazy or non-Lazy mode

This option works for retrieving variables from Ruby runtime and is independent from any local context type and local variable behavior. It specifies whether only requested variable is retrieved on demand or all variables are retrieved every time evaluation finishes. As you may realize easily, the former has better performance. Yes, this option has been added to improve performance. By default, lazy mode is on for embed core (ScriptingContainer), and off for JSR223 (I changed the default of JSR223 between RC2 and RC3. On RC3 and later, non-lazy is default for JSR223). JSR223 has javax.script.SimpleBindings/SimpleScriptContext for a variable holder, which makes it very hard to work on demand. For JSR223 users, it is important that JRubyEngine works in the same way as other ScriptEngines such as Rhino, BeanShell, etc. So, I took compatibility over performance. Still, JSR223 users can choose lazy mode, but they should know some trick makes available to get a variable value from Ruby runtime.

This figure illustrates how sharing variables of lazy mode works: The *BiVariableMap* in the figure is the internal map to save variables. The internal map is responsible for a type coercion from Java to Ruby and from Ruby to Java. Moreover, based on a given name, BiVariableMap chooses a right variable type such as a global, or instance variable and its logic to get from/set to Ruby runtime. As in the figure, only when a user want to get a variable, the variable is grabbed form Ruby runtime on demand. While the variable is returned to the user, it is also created in the BiVariableMap for a possible later use.



Next figure illustrates one of a non-lazy retrieving sequence on JSR223: This would be a typical way on JSR223. Users are allowed to instantiate JDK bundled classes javax.script.SimpleBindings/SimpleScriptContext. Unfortunatelly, RedBridge has no way to hook over these classes to make sharing variables work. Besides, there is no way to know what variable a user want to retrieve from Ruby runtime after the evaluation. So, when the evaluation finishes, JRubyEngine tries to get all variables/constants that look user defined ones. In some cases, BiVariableMap grows pretty fatty. But, when evaluation ends, all possible variables must exists in SimpleBindings/SimpleScriptContext. To implement JSR223 faithfully, JRubyEngine sacrificed performance and memory saving.




Well, how people can change lazy mode? I'll show just JSR223 example since ScriptingContainer users are happy with the default, lazy mode, and won't feel inconvenience.

For comparison, I'll put an example of default settings here:

// non-lazy mode; default
ScriptEngine engine = new ScriptEngineManager().getEngineByExtension("rb");
SimpleBindings bindings = new SimpleBindings();
engine.eval("$weather = 'freezing rain'; $temperature = '28F'", bindings);
System.out.println("It should be '28F': " + bindings.get("temperature"));

This code prints
"It should be '28F': 28F"

The following is a lazy setting example:

// lazy mode on
System.setProperty("org.jruby.embed.laziness", "true");
engine = new ScriptEngineManager().getEngineByExtension("rb");
bindings = new SimpleBindings();
engine.eval("$weather = 'snow'; $temperature = '17F'", bindings);
System.out.println("It should be null: " + bindings.get("temperature"));
System.out.println("It should be '17F': " + engine.get("temperature"));

Above prints:

It should be null: null
It should be '17F': 17F

So, when engine's get method is used, retrieving a variable on demand works. Or, more Ruby way would work. For example, Ruby can return more than one variables at the same time. Returned values are saved in an Array, which is converted to java.util.List:

List list = (List) engine.eval("$weather = 'sleet', $temperature = '32F'");
System.out.println("It should be 'sleet': " + list.get(0));
System.out.println("It should be '32F': " + list.get(1));

When this snippet gets run, it prints

It should be 'sleet': sleet
It should be '32F': 32F



2. Singleton Type

Before going forward, let's review local context types one by one. To make it clear, I wrote figures that illustrate the structures of each type in terms of sharing variables. The figures will help you to understand what are going on.


The first type is singleton. This is a default type for both ScriptingContainer and JRubyEngine. The singleton type has only one Ruby runtime on JVM, which is *singleton* as the name expresses. BiVariableMap ("Var Map" in the figure) is also only one on JVM. No matter how many instances of ScriptingContainer / JRubyEngine you create, there is only one set of the runtime and variable map. In this type, thread safety is users' responsibility. No synchronization in API is RedBridge's policy.


3. SingleThreaded Type

The second type is naive singlethread. This is the simplest type and good to test something simple. This singlethreaded model would be a typical one that other JSR223 engines adapt to. The singlethreaded type can have multiple sets of Ruby runtimes and BiVariableMaps on a single JVM. If you instantiate three ScriptingContainers / JRubyEngines, you'll have three sets of runtime and variable map on the JVM. Again, thread safety is users' responsibility.


4. Threadsafe Type

The third type is threadsafe. In this type, a set of runtime and variable map is a thread local value. Thus, each *thread* has its own set of runtime and variable map. This type allows us to isolate an internal state by creating a thread. Just one instance of ScriptingContainer / JRubyEngine creates multiple sets of runtime and variable map along with threads. Users don't need to worry about thread safety as long as the concerned threads are created in Java. The thread safety here doesn't mean Ruby threads.

Here's the example of threadsafe type that isolates the internal state. The code is here.

import java.util.Map;

import org.jruby.Ruby;
import org.jruby.embed.LocalContextScope;
import org.jruby.embed.LocalVariableBehavior;
import org.jruby.embed.ScriptingContainer;

public class TransientThreadsafe {

private TransientThreadsafe() {
ScriptingContainer container =
new ScriptingContainer(LocalContextScope.THREADSAFE, LocalVariableBehavior.TRANSIENT);
Runner runner1 = new Runner(container);
Runner runner2 = new Runner(container);
new Thread(runner1, "Runner-1").start();
new Thread(runner2, "Runner-2").start();
runner1.getVarMap().put("$tmp", "Atlanta");
runner2.getVarMap().put("$tmp", "Los Angeles");
}

public static void main(String[] args) {
new TransientThreadsafe();
}

class Runner implements Runnable {
ScriptingContainer container;
Map varMap = null;

Runner(ScriptingContainer container) {
this.container = container;
}

@Override
public void run() {
varMap = container.getProvider().getVarMap();
while (varMap == null || varMap.get("$tmp") == null) {
try {
Thread.currentThread().sleep(1000L);
} catch (InterruptedException e) {
// no-op
}
}
container.runScriptlet("puts \"" + Thread.currentThread().getName() + " ran in #{$tmp}\"");
}

Map getVarMap() {
while(varMap == null) {
try {
Thread.currentThread().sleep(1000L);
} catch (InterruptedException e) {
// no-op
}
}
return varMap;
}
}
}

From the output below, we can see two different sets of runtime and variable map are there:

Runner-2 ran in Los Angeles
Runner-1 ran in Atlanta


5. Concurrent Type

The last type is concurrent. This type is added in 1.6.0RC1 and mixture of singleton and threadsafe. Concurrent type has singleton runtime and thread local variable map. Probably, it is the most complicated type but works well in some cases. For example, gems are evaluated in Java Servlet's init() method, then, classes and methods of those gems can be used in doGet()/doPost()/etc methods. On a Servlet container, each HTTP request is on a thread, so each HTTP request can have an isolated state.


6. Transient Local Variable Behavior

OK, let's see local variable behavior types one by one. We have transient (default for ScriptingContainer), persistent, global (default for JSR223), and bsf. I'm not going to talk about bsf type. It is just for BSF engine, which is almost obsolete.

The first local variable behavior is transient. The transient type is natural to Ruby. When we assign value "hello" to a variable name "$message," it is $message = "hello" in Ruby. We can use Ruby's global, instance, local variables and constants to share between Java and Ruby.

When you use global variables to share, you need to care what local context type you are using. Because the global variables are global on a runtime, the variable becomes unique on the single runtime. When the runtime is singleton (singleton and concurrent types), a global variable, say $tmp, is shared globally. We have two more to care about. One is that embedding API doesn't see what type of variable is given. We can put a global variable with a receiver object, but the given receiver is ignored when the global variable gets pushed on to the runtime. Another is that the global variables persist in BiVariableMap unless they are deleted. As long as global variables are in BiVariableMap, they are re-injected to runtime. When the global variables are removed from BiVariableMap, nil is set as its value on Ruby side. (There's no way to delete global variables in JRuby)

How about instance variables? When you use instance variables, you need care about they are in top level or some object. If it is a top level instance variable, the variable will be unique on a single runtime. Because the "top self" object is the only one on the runtime, the instance variable of the top self object is the only one. When you put instance variables using embedding API without a receiver, they will be injected as top level variables. This behavior is the same on all four local context types. The persistence of instance variables on Java side is the same as global variables. Unless they are deleted from the map, they will keep being injected to the runtime.

Constants works exactly the same as instance variables, so I won't add anything about constants.

The behavior of transient local variables are remarkable. The local variables on Java side vanish from a variable map after each evaluation. Because they are the local variables, they should not survive over the evaluation. Suppose 'orange' gem is evaluated right after 'red' gem is evaluated. What if both RubyGems use a local variable name "tmp"? The 'red' gem might change the value of 'tmp' local variable. Ruby code never expects the value of 'tmp' is changed by another gem and given for evaluation. Thus, the local variables in the map are removed not to cause unexpcted results when evaluation finishes. If you want to use the same local variable again, put it to ScriptingContainer / JRubyEngine(or SimpleBindings/SimpleScriptContext) again. So far, sharing *local* variable is available only for a top level local variable.


7. Persistent Local Variable Behavior

Next local variable behavior is persistent. As the name shows, this type makes local variables persistent. The local variables survive over the evaluations like global and instance variables. This behavior was added mainly for ex-BSF users. BSF defines both local and global variables should persist on an engine. Ex-BSF users feel this behavior essential. This type might look useful, but users themselves need to avoid local variables collision.

Other than local variables, global variables, instance variables and constants work exactly the same as transient type.


8. Global Local Variable Behavior

The last local variable behavior is global. The name, global local variable, might sound weird, but it actually expresses the behavior. If we put the key-value pair, container.put("tmp", 100) in Java side, it will be $tmp = 100 in Ruby side. The variable looks like local or constants in Java, but everything is a global variable in Ruby. So, we can use only global variables and need to care what local context type is used. As I wrote in "6. Transient Local Variable Behavior," the global variables are shared globally on a single runtime.

This type is for JSR223. JSR223 reference implementation released from Sun was this type, so embedding API had the same local variable behavior for compatibility. Moreover, to fulfill JSR223 specification, values of local global type variables on Ruby side go to nil when the evaluation finishes but not on Java side.

This global local type is really weird in terms of Ruby programming. But, the design of Ruby language is very different from other JVM languages. Besides, not all JSR223 users are dedicated Ruby programmers. This type might be the answer for JSR223 to avoid unexpected results.


9. No Sharing Variables

In the end, you might not need sharing variables feature to work. You are probably a big fun of ScriptingContainer's callMethod() or JSR223's invokeMethod() / invokeFunction(). Every variable to get Ruby code work are given through method's arguments. In fact, this has better performance than repeating runScriptlet() or eval(). Embedding API does have the option to off the feature.

[ScriptingContainer]
container.setAttribute(AttributeName.SHARING_VARIABLES, false);

[JSR223]
System.setProperty("org.jruby.embed.sharing.variables", "false");




This is how RedBridge's sharing variables works. Various field of users have requested various kinds of features to embedding API. To cover those as much as possible, embedding API has become a bit complicated. If you understand how it works, you'll feel comfortable to use in the way that fits you the best. Have happy coding with embedding API.