Installing gRPC for Ruby on Mac OSX Yosemite
I've been looking at HTTP/2 recently along with the gRPC framework. One of the things I wanted to try out was cross language interchange. Herein some notes on getting things working for Java and Ruby on Mac OSX Yosemite, that might be of help to someone.
The gRPC project has not hit a final release yet, but is being worked on heavily. The range of language support is good - the Java project, grpc-java is standalone, as is Golang's grpc-go. Python, Ruby, Node, PHP, C#, Objective-C and C++ depend on shared code and live with the core and compiler codebase. To round it off, gRPC depends on Protocol Buffers 3, which itself is moving towards a final release.
I solve problems
As mentioned, grpc-java is the main project for JVM. It looks to be in good shape for a pre-1.0 project - the commits have the iterative feel and focus of code resulting from day to day use. It was easy to get going. There is Gradle build support in the form of the protobuf-gradle-plugin, which hooks into the standard 'build' task to generate code from the protobuf message file - it didn't take long to figure out the stanzas for sourceSets and protobuf declarations, and how to move the generated code into its own source root rather than the build area (personally I've always checked in protobuf gen code). This is a nice change from my experience in the past with pre-compile gen scripts. I expect direct build integration will be especially helpful if you're looking at gRPC for Android. There's a simple standalone project I put together here to get a feel for things, that uses the Hello Word example pretty much as is from the grpc-java project. I'm guessing Go will be simple as Java when I get to it.
I'm also interested in the state of gRPC for Ruby, Go and Python. I haven't gotten to Go and Python yet, but did manage to get basic exchanges running between Ruby and Java. That said, getting gRPC setup on Mac OSX with Ruby was a bit of a bear. Mostly the problems were mismatches between versions of gems, gRPC core and Protocol Buffers, pretty much the moving parts stuff you see in the run-up to a major release. Everything in gRPC is pre-release and evolving quickly right now, and while thing seem to work well, getting up and running against native dependencies was time-consuming compared to the JVM.
Didn't go into a Burger King
The first thing, after digging around the code in the main gRPC project and reading make errors, was to remove the existing installed protoc along with its header files. The previous version 3 protoc on the machine was gettting in the way of the new stuff. The includes were in '/usr/local/include/' (the Mac I use has a mismash of boxen and brew and previous make runs of protoc, so your details may vary).
The instructions to install gRPC via homebrew-grpc are very simple - a one line curl command. Unfortunately the targeted version, 0.10.0 wouldn't apply - it seems there were native signature bugs for Ruby in 0.10.0 which are long fixed, but haven't made it out a release yet. There is a 0.10.2 tarball available, but that also failed as it required a more recent protoc 3 version. The answer in the end was the patch the formula files up to more recent versions. There's a fork of the project here that uses versions gRPC 0.11.0 and google-protobuf 3.0.0-beta-1 as those two combined for me after some hours of digging.
To install them copy the two formula .rb files from Formula into "/usr/local/Library/Formula/" and then run -
brew install google-protobuf --without-python
brew install grpc
Check the installations with "brew list --versions", which should show -
google-protobuf 3.0.0-beta-1
grpc 0.11.0
The Python support in google-protobuf, which is on by default, didn't work for me due to signature mismatches. This is probably because the Python resources referenced in the brew formula are stale. You might be able to get protoc installed without excluding Python, but my patience digging into native code had run out at this point :) I'l go into that Burger King another time.
Version Fox Force Five
The last step for Ruby is to install the grpc gem. There were no published version that worked against the versions above or the previous 0.10.0. The answer was to patch the grpc.gemspec file in the grpc project's /src/ruby area. First, removing the "google-protobuf" dependency line entirely and then making sure "libgrpc" is set to "~> 0.11.0" (which it should be on a recent master). Then run -
gem build grpc.gemspec
gem install ./grpc-0.11.0.gem
The updated gem will appear in "gem --list". After all that, in your Gemfile you'll be able to declare -
gem 'google-protobuf', '~> 3.0.0.alpha.3.1.pre'
gem 'grpc', '0.11.0'
The 3.0.0.alpha.3.1.pre above is a cheat - it should really be 3.0.0-beta-1 to match up with the underlying protoc version installed by Brew, but as far as it goes it's been working well enough to test things out, which is all I need for the moment.
I haven't gotten round to publishing a patched gRPC gem or patching google-protobuf to get to 3.0.0-beta-1, but I'm hoping I won't have to as gRPC 0.11.0 just moved into beta (Today! I should have waited!). That should mean a round of packaging uplifts are coming which I hope will avoid the messing around described here.
Say interop one more time
In any case, Ruby/Java interop looks to work so far. This is what I've come to expect from Protocol Buffers over the years, but protoc 3 has some new features such as maps and streaming that I wanted to see in action and see how they look serialized. Also, even though HTTP/2 uptake is strong (there's very little doubt in my mind it will supersede HTTP over the years), you always want to see stuff working over the wire first hand. So far, the Ruby code is clean - creating a client stub is a one liner and making a call with it is another. A nice feature of the generated message code is that assignments try to honor their type - if you have a string field and try and set it to nil or another type, the code will raise. I've used protocol buffers in Java for half a decade and working with messages in version 3 is much like version 2, with the gRPC service code not unlike dealing with Thrift or a lightweight Netty service, and perhaps a bit leaner than either. Maps are very, very useful - I would upgrade for those alone.
Perhaps the interesting overall observation here is how gRPC dependencies ripple down into Protocol Buffers and up into language libraries. It's not what you'd want to deal with generally, and if you've been through the wringer with binary formats and RPC (as I have) this post is maybe setting off alarm bells - but I'm not overly concerned. I suspect things will harden once gRPC and Protocol Buffers hit their upcoming major releases. Protocol Buffers 2 has, at least historically for me, done a good job on stability, where 'good' means viable for storage and long term soft state. Also, other Google projects tend to take their version management seriously such that I suspect it's cultural in some parts of Google - Go, Guice and Guava come to mind. Finally, HTTP/2 is baked, the next phase is really about adoption of its core elements, and while it's understandable to be concerned about binary wire formats, it's going to interop as well as any ad-hoc text format you might put together, it'll be around for a long long time. and it will inevitably have good operational tools.