I’ve been considering starting a new project. Part of this will require a microservice REST API. I tend to be pretty language agnostic and tend to favor using the best tool for the job. That got me thinking that I should evaluate different ways of getting it done. To this end I put together a simple test to evaluate several languages for programming ease and performance. There is a lot of passion around many languages but I figured why not take a data driven approach when picking my implementation path.
One of my requirements is to run this as a microservice. I know it’s another buzz word, but I’ve had some very positive results with this as part of my day job. That meant that the API should run without an additional server layer such as Apache etc. That also removed things like Clojure from consideration since the JVM doesn’t really fit light or micro.
For this test I settled on the following 5 languages:
In the interest of full disclosure, I’m not an expert in any of these languages. I am pretty adept and proficient with programming and so I wasn’t scared of by the fact that I’ve never written any Go before. That said, there are likely better ways to write much of the code, which can be found on github.
For the comparison I decided to limit the API to a couple of pretty simple endpoints:
/– “Hello World!” … just because
/time– returns the current time in RFC3339 format, which I’ve long disliked since there is little in terms of great support for it out there
/sort– takes a JSON array of items via POST and sorts it
All of the code ran on a Vagrant machine using Ubuntu 14.04. I also used as many of the available packages as possible, but some special items were installed based on language specific tools. The things I installed and how can be found in the github README.
Version: go version go1.1.2 linux/amd64
Go has a nice intro on Writing Web Applications. That page got me moving pretty quickly.
I’m fond of the compiled binary that can easily be moved about. I also found that pretty much everything I needed was part of golang package. I also like the idea that go allows for the use of pointers. Yeah, you can shoot yourself in the foot with those, but when used right there is a lot of power. To me it has always seemed that the power of pointers should be taught much much more.
On the downside, the static typing is hard to adjust to after having spent much time with loosely/dynamic typed languages. One thing in particular that really felt awkward is that arrays are fixed length. The slice is recommended instead, but then there is lots of other work to do in order to sort a slice to get rid of the following error:
float64 does not implement sort.Interface (missing Len method)
Node also seems to offer the ability to build a REST API, but that seemed harder than it should be. Most of the internet seems to point at [express]((http://expressjs.com/) and the Hello Word Example is a nice way to start.
I was surprised that there wasn’t a built in way to deal RFC3339 timestamps and I had to add extra functions to obtain that functionality.
By default there isn’t a way to get the request body based on the req.body section in the express API docs. Installing a body parser is easy enough:
By default a curl call doesn’t set the
Content-Type header, which send the data provided via
application/x-www-form-urlencoded. I wasn’t able to get nodejs/express to grab the data correctly unless I set
Content-type: application/json. I ended up just using it by default for all tests against the
Version: ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]
Sinatra seems to be a framework that is popular. I’ve also used it in a previous project and figured I would use it again. By default it uses the the Webrick. There are a lot of references that this is not very performant. Switching that out for Thin is pretty easy and only one extra gem command and one package.
Writing that code was trivial and it looks readable and concise.
Version: Python 2.7.5+
I will admit upfront that I’m not a fan of the meaningful whitespace in python. That said it does offer a pretty clean syntax and I have some familiarity with it. There are a few choices for implementing a REST API in python. My choice is Flask, since it appears to be more bare bones. Since I’m shooting for a small service Django and Pyramid seemed like more than I wanted.
Version: This is perl 5, version 14, subversion 2 (v5.14.2) built for x86_64-linux-gnu-thread-multi
I used to spend a lot time writing perl, but have mostly left it behind after finding it to perform less well than ruby and python. None the less I still am very found of the Perl regular expressions and decided to add it into the mix after a colleague spoke highly of Mojolicious.
One thing the hit me as odd is the way you specify which port to listen on when running the Lite Mojolicious module. It’s a classic example of why people thing Perl is ugly:
@ARGV = qw(daemon --listen http://*:8080);
The other languages make that a bit cleaner and more obvious.
testing with siege
I used siege to run the performance tests, by pushing a JSON array to the
/sort end point.
On the first set of tests I found that Python, Perl, and Ruby were much slower than the Go and Node.js. I also realized that the first 3 all produced logs to stdout, while neither Go nor Node.js did. This seemed like a possible slowdown. Since I was interested in comparing as similiar as possible setups, I revised the code and disabled logging or set the logging level to errors only. This had a significant impact on performance and nearly doubled the siege benchmark results for all 3.
I then re-ran the tests with benchmark flag and 100 concurrent clients for 20s. The benchmark is pretty heavy and running longer tests produced a lot of failures and made the VM I was testing against unresponsive. Clearly there is a reason for the warning in the man page about using the benchmark.
sudo siege -b -t 20s -c 100 -T 'application/json' "http://localhost:8080/sort POST <test.json"
The results showed no failures for any of the languages, but the performance differences are pretty clear:
|Language||Transaction per second|
Clearly Go is the winner in performance and it’s also the only compiled language in the mix. The compile step adds a little extra effort, but that feels negligible given that the resulting binary can be moved around without much trouble. Go is also the only language without dynamic typing. That feels tedious to me and the code is also longer lending to tedium.
Ruby is a pretty distant second, but I will say that the syntax is easier on the eyes and to me is faster to code in.
Python came and Perl came in at the end of the field. Those results match my tests in the performance of different scripting languages: shell v. perl v. python v. ruby V2 I performed a while back. I used to write a lot of Perl and this result just makes me sad.
In the end it comes down to Go vs Ruby for me. Go clearly wins on performance, but Ruby takes the prize for code elegance.
I’m not sure I’m fully decided yet.