Pub/Sub Pattern

Our first example is the pub/sub pattern. Our code will broadcast messages from Ruby which our Clojure process can subscribe to. One interesting capability of 0MQ is that we can subscribe to “channels” — that is, we’ll only receive messages which start with a given query.¹

#!/usr/bin/env ruby -wKU
# pub.rb
require 'rubygems'
require 'zmq'

zmq = ZMQ::Context.new
socket = zmq.socket(ZMQ::PUB)
socket.bind "tcp://127.0.0.1:5555"

# A REPL of sorts.  Messages entered here
# will get sent to our Clojure process.
while true
  '> '.display

  input = gets
  break unless input

  input.each do |msg|
    socket.send(msg)
  end
end

exit 0

; src/my-project/sub.clj
(ns my-project.sub
  (:use [org.zeromq.clojure :as zmq]))

(defn- string-to-bytes [s] (.getBytes s))
(defn- bytes-to-string [b] (String. b))
(defn- on-thread [f]
  (doto (Thread. #^Runnable f)
    (.start)))

(defn launch-subscriber [query]
  (on-thread
   #((let [ctx (zmq/make-context 1)
	   socket (zmq/make-socket ctx zmq/+sub+)]
       (zmq/connect socket "tcp://127.0.0.1:5555")
       ; Note the dot-syntax! clojure-zmq is a bit behind
       ; on the Java API, so you have to fall back to straight
       ; Java interop to subscribe to a pub "channel".
       (.subscribe socket (string-to-bytes query))
       (while true
	 (println (bytes-to-string (zmq/recv socket))))))))

; Subscribing to the "foo" channel.  We'll only see messages prefixed with foo.
(launch-subscriber "foo")

After launching the Clojure process, hop into the shell:²

$ ruby pub.rb
> Hello There
# No entry in the Clojure output
> foo Hello There
# prints "foo Hello There" in the Clojure output

Request/Response Pattern

The request/response technique allows us to delegate work from our master program to a worker process. In this particular example, we’re sending lists of values from our Ruby process to our Clojure process, which adds them and sends back the result.

#!/usr/bin/env ruby -wKU
# req.rb

require 'rubygems'
require 'zmq'

zmq = ZMQ::Context.new
socket = zmq.socket(ZMQ::REQ)
socket.bind "tcp://127.0.0.1:5556"

while true
  '> '.display

  input = gets
  break unless input

  input.each do |msg|
    socket.send(msg)
  end
  puts socket.recv
end

exit 0

; src/my-project/rep.clj
(ns my-project.rep
  (:use [org.zeromq.clojure :as zmq])
  (:use [clojure.contrib str-utils]))

(defn- string-to-bytes [s] (.getBytes s))
(defn- bytes-to-string [b] (String. b))
(defn- on-thread [f]
  (doto (Thread. #^Runnable f)
    (.start)))

(defn handler [request]
  (let [request (bytes-to-string request)
	values (map
		#(Integer/parseInt %)
		(re-seq #"\d+" request))
	output (str (apply + values))]
    (println (apply
	      str
	      (str-join " + " values)
	      " = "
	      output))
    output))

(defn make-adder []
  (on-thread
   #((let [ctx (zmq/make-context 1)
	   socket (zmq/make-socket ctx zmq/+rep+)]
       (zmq/connect socket "tcp://127.0.0.1:5556")
       (while true
	 (let [request (zmq/recv socket)
	       result (handler request)]
	   (zmq/send- socket (string-to-bytes result))))))))

(make-adder)

Fire up the Clojure process, and hop back into the shell:

$ ruby req.rb
> 2 2
4
# The Clojure process will print "2 + 2 = 4"
> 8 8
16
# The Clojure process will print "8 + 8 = 16"

You can find more ZeroMQ patterns on the official cookbook page.

I’ve been working on a pet project using Sinatra and Unicorn (doing some long-polling stuff, so I needed to support multiple connections even in dev). Unfortunately, this combination makes code reloading tricky to tackle, as it rules out shotgun entirely, and seems to also bust Sinatra::Reloader.

After a couple hours of HUPping unicorns manually, I decided that I needed a more robust solution. Some digging led me to the FSSM gem by ttilley. This library will watch your file system for events and runs a registered callback when they occur. A few minutes of hacking and my reloading woes are solved!

#!/usr/bin/env ruby -wKU

require 'rubygems'
require 'fssm'

puts "Monitoring #{File.expand_path(File.dirname(__FILE__))} for changes..."
FSSM.monitor(File.dirname(__FILE__), ['**/*.rb', '**/*.ru', '**/*.yml']) do
  update { system "kill -HUP `cat pids/unicorn.pid`" }
end

exit 0

Fun Stuff

Just like the original, this middleware downloads all the images on a page, transforms them with ImageMagick (-flip is the default transformation) and renders out the updated images. Pure. Simple. Fun.

Setup is just like any other middleware. Bonus: set your own image transformations!

# environment.rb
require 'rack_upside_down_ternet'
config.middleware.use Rack::UpsideDownTernet, '-blur 10 -flip'

Broken Stuff

• Uses curl instead of just grabbing the file out of the images directory
• Doesn’t even kinda work with remote/absolute URL images
• Requires unicorn (or any multi-worker setup). Make sure you have at least 2 workers running, as your first request will block curl from downloading images
• Requires that you create PROJECT_DIR/tmp and PROJECT_DIR/images/mod

I recently released a Japanese word search builder (see the discussion in the Hacker News post). This was my first foray into the world of multibyte characters, and I quickly discovered the pain involved in simply printing “ありがと” to the screen. Below are some of the details I found to help me through this trial.

Tips to make your Unicode+Ruby experience easier

0. Read James Edward Gray’s Understanding m17n series. Every word. Period.
1. If at all possible, use Ruby 1.9+. As James describes, Unicode support is far better than in 1.8.x.
2. In spite of Ruby 1.9′s superior Unicode support, it still doesn’t read files as UTF8 by default. This means that loading any files which include Unicode characters will cause Ruby to lash out at you with complaints about multi-byte characters. To get around this, set the file’s encoding type at the start of the file.

   # encoding: UTF-8
   ... unicode goes here ...
   

3. If you can’t use 1.9, make sure you use the Jcode library. It’s included with Ruby and adds lots of useful Unicode-oriented features to String and Regexp. The only disadvantage is that the original methods remain in place (e.g., you’ll have to make sure you call String#jsize rather than String#size).

   require 'jcode'
   "ありがと".size
   # => 12
   "ありがと".jsize
   # => 4
   

4. If you plan for your Ruby file to be run directly, make sure you add the -KU flag to your shabang line. This will tell Ruby to be open to UTF8 data.

   #!/usr/local/bin/env ruby -KU
   ... Ruby goes here ...
   

5. Familiarize yourself with iconv — both the Ruby library and the command line tool. This tool will allow you to translate data easily between various character encodings.
6. While not strictly required these days, it’s still recommended when serving HTML with Unicode data to set the content type both in a meta tag on the page and the in HTTP header.

   HTTP/1.1 200 OK
   Server: nginx/0.6.39
   Date: Wed, 24 Mar 2010 00:39:47 GMT
   Content-Type: text/html; charset=utf-8
   ...
   
   <html>
   <head>
   <meta http-equiv="Content-Type"  content="text/html; charset=utf-8" />
   ...
   

My project revolved around data collection, and I pretty quickly ran into the need to see what I was actually shoving into the database. After 10 minutes and only ~50 lines, I came up with a Sinatra backed Redis viewer. See the code below, or check out my gist on Github!

# app.rb
require 'rubygems'
require 'haml'
require 'sinatra'
require 'redis'

helpers do
  def redis
    @redis ||= Redis.new
  end
end

get "/" do
  @keys = redis.keys("*")
  haml :index
end

get "/:key" do
  @key = params[:key]
  @data = case redis.type(@key)
  when "string"
    Array(redis[@key])
  when "list"
    redis.lrange(@key, 0, -1)
  when "set"
    redis.set_members(@key)
  else
    []
  end
  haml :show
end

# views/index.haml
%html
%body
  %h1 Current Keys
  %ul
    - @keys.each do |key|
      %li
        %a{:href => "/#{key}"}= key

# views/show.haml
%html
%body
  %h1= "Data stored in '#{@key}'"
  %ul
    -@data.each do |data|
      %li
        %p= data

Like most of the developer blogosphere, I was thoroughly excited to read Vincent Driessen’s post on branch management in git. It captured a simple and repeatable methodology for extracting a great deal of power from git without having to really dive down into its belly. Given that Vincent’s methodology relies heavily on feature and bugfix branches, I began looking for techniques on combining Pivotal Tracker — which I’ve been using recently for all of my project management needs — with git. I found some advice in Rein Henrichs’ article which suggests using Pivotal Tracker story ids as branch identifiers, but not much else on how to really tighten the two together.

Moving towards smoother integration

Further Googling didn’t reveal any tools to aide my new objective, so I set out to create my own toolkit. After a few evenings digging through Pivotal Tracker’s API (which actually changed from v2 to v3 while I worked ), I’m ready to announce my work: Git Pivotal. It’s a very alpha release currently, but I have some work in development that will (hopefully) stabilize it soon.

Git Pick

The first (and currently only) tool is git pick. After some simple configuration, this will grab your top-most available feature story from Pivotal Tracker, display some simple info about it and offer to create a feature branch for it off of your current branch.

What’s coming

I plan to add some additional commands for marking current feature branches as finished, managing bugs and bugfix branches and possibly integrating release stories as well. If you’re a fan of git and Pivotal Tracker, keep an eye on this project!

So what do you think? Is this useful? Share your thoughts below!

# suppose isPrime(n) returns the primality of n
[x for x in range(1,10) if isPrime(x)]
# => [2, 3, 5, 7]

When I migrated from Python to Ruby recently, I was disappointed to find that there was no simple replacement for list comprehensions. Top that with everyone in the Ruby community saying “Ruby doesn’t do list comprehensions,” and I basically gave up on my hope of finding a suitable analogue. Fortunately though, I stumbled upon a thread in Ruby-Forum recently showing how to quickly add the capability onto any array!

The meat of the trick is this:

module Enumerable
  def comprehend( &block )
    block ? map( &block ).compact : self
  end
end

With this monkey patch, I’m now able to write:

# Likewise, is_prime?(n) returns primality of n
(1..9).comprehend { |x| x if is_prime?(x) }
# => [2, 3, 5, 7]

While this isn’t a direct replacement of Python’s implementation, it’s at least enough to prove that Ruby is surprisingly powerful!