▲ 16 ▼ A tale of three kings - Python, Elixir and Go
How we tested Python, Elixir and Go’s abilities to rebuild some of Unbabel’s core services
Register to comment or vote on this story
How we tested Python, Elixir and Go’s abilities to rebuild some of Unbabel’s core services
My conclusion from all this seems to be that if you're running a likely scenario's (hello world is unlikely to be a production service) the benchmarks of frameworks showing how fast they are, are largely irrelevant. The second a database got involved the whole equation changed and I'd venture a guess that if a similar scenario were tried in which it had to query a 3rd service before it could respond you're get similar results as to the database setup.
However if you can answer everything directly from memory the Python-Japronto combination might prove interesting. I'm also curious how much of that standard deviation is caused by Gin vs. something that Go is doing. Would be interesting to rewrite that in Chi and FastHTTP too and see if you'd get similar results.
As interesting as the article is though, their conclusions section baffles me a bit. It seems to be disconnected from the rest of the article.
1. Where does the conclusion comes from that they should probably use Elixir for I/O intensive services?
HTTP is I/O for one and their simulation suggests Go achieves better throughput. If we're talking reading/writing files then there doesn't seem to be any data presented to support that conclusion.
2. I'm also failing to understand their general conclusion:
> In most scenarios, you can achieve the exact same end result with Python as you do with Go. With Python, you’ll probably be able to ship far faster, and have more developers available.
Looking at their benchmarks Go seems to achieve a much better result than Python in real world. So which "most scenarios" are we talking about then? The claims about being able to ship faster is one I find doubtful, especially based on how annoyingly complicated it can be to deploy a Python app because of how dependencies aren't packaged up. If this is about how easy it is to write Python, a lot of that is depending on those writing it. Having more developers available is possible but some kind of metric to back that up would be nice.
The first tests looked interesting, but I have suspicions about the test which takes out most of the work. They say "we parse the request params and return a fixed number." That sounds like they're returning just an arbitrary number (unrelated to params), if the compiler notices no real work is being done, it might in some circumstances just remove all the code, so that you're benchmarking nothing, or it might not. Real services tend to need to do lots of work which is not reflected in benchmarks - auth, logging, dbs etc.
As usual benchmarks are incredibly hard to do fairly and very hard to interpret correctly, so they should at most be one datapoint when making decisions between languages/software, and should also be considered in the context of the actual performance required, the existing team, experience etc. They are sometimes good for eliminating options though if they are nowhere near performance requirements. I did find it interesting to compare the different graphs of response time, including outliers (people often ignore the worst case response times).
Thanks for posting, this looks an interesting comparison of three languages very popular for web dev right now. I've edited the title to move some text into summary as it was too long.