Skip to main content

50.000 times too slow?

Posted by herkules on July 19, 2010 at 4:44 AM PDT

Recently I was in urged to do a web project with the latest and coolest web framework Ruby on Rails.

That's just one line of code grabbed from somewhere:

      next if entry =~ /^\./

Ruby is said to read like a natural language, but for my eyes this is just a cryptic sequence. Maybe I have the wrong natural language. So to clarify what it means I translated it to some assembly language:

      cmp (entryptr),'.'
      beq next

Yes: if the entry begins with '.', skip it.

The latter code might take 2 CPU cycles to execute.

How much will the Ruby code take? There is the line to be interpreted (syntax check, AST creation,... no idea), than a regexp engine is started which starts parsing the expression before it tries to find a match. All this is accompanied by lots of heap activity. My guess: 100.000 cycles. At least.

So it's 50.000 times slower than it needs to be to get the work done. Who needs a cluster of 50.000 machines to serve his application? Well, all others should be well of with a single server, even a poor one, if it only was programmed with care.

I know it's unfair and untrue, so please don't comment on that. But ... to some respect it is true.

Comments

you missed the point :)

This post is by no means about programming language performance. It is about developer habit, maybe about todays dogmas of application development.
It shows a real-world example on how far todays solutions can be away from what actually needed to perform a task.

The quoted code is no case study, no ruby presentation ... its production code. So any argument saying 'could be done faster' doesn't hold because it isn't actually done faster. But why?

Are we sure that we do not pile more and more on our technology stack while in the end things are so simple? Like in the example? And its marching on. Software using a regexp to ask for a '.' must be slow. But what is concluded? People ask for another piece of complex technology ... to make regexps faster!! This is a pattern observable in many areas.

The post is asking for developers not considering things to be atomic (from a CPUs point of view) just because they can be written in a single line. A regexp is not atomic. Applying a template is not atomic. Parsing a URL is not atomic.

I'd always agree to sacrifice some CPU cycles to achieve better abstraction, more developer security, better readability. But if 2 cycles are for the actual task and 499998 are just for comfort ... something might be wrong. And - as shown in the example - the comfort isn't even achieved.

My claim: don't add anything to make things fast in the first place. Instead, get rid of things that make it slow.
 

Bad Comparison Article

Terrible. First the Ruby code is poor. how about? next if entry[0] == "." That should save some cycles and make more sense. Comparing it then to assembly should result in a much closer match but obviously a high level language doesn't match assembly but if you would at least compare two things that are equal that would help. As long as you have been in programming I think you know better so perhaps you cranked this out too quickly and didn't proof read or you just wanted to generate a lot of comments? This doesn't even seem like an honest mistake. I think the point here should be that the Ruby programmer used overkill for a simple indexed match and you compared poorly.

A Bit Shortsighted

What you have chosen to make an example of is a bit shortsighted. Most modern languages have Regular Expression support. This does not diminish a language's 'naturalness'. By your definition, all languages which contain regular expressions are unreadable. Sure, someone used a regular expression when they didn't 'need' to, and that is just speculation. We don't know the requirement. This expression might need to be more complex in the future. So, the developer chose to use a certain API.

But, may i ask if it really mattered where they used it? How often is it invoked? Does it create a bottleneck in the application? Did it truly cause a performance issue? Did it really introduce a requirement of multiplying your server instances within the cluster by 50,000 (US format, or is that really 50)? If so, then fix the issue; don't complain about it. Not every application is a mission critical, real-time system. A true architect/developer should know the requirements of the application they are developing. Some applications require certain technologies and languages to fulfill these requirements. Was the choice of language/platform adequate for you to complete the task? If so, then the argument is pointless.

BTW, I don't use Ruby. It just irks me when people post performance issues on things that may not be an issue after all.

Code, Compile, Test, Profile. Then, tune only what needs to be tuned. The point is to get it done as quickly, and accurately, as possible. The goal is almost never how fast can it run.

Anyway, that was just my feeling when I read the post. No hard feelings, I hope. At least you got a conversation started. :-)

Extremingly Unfair

The most short-sighted and unfair comparison I have ever seen, I remember when Java was compared to other languages in the same f... up manner (please bear in mind that I am actually really mad about your post). In the least I can say your are a very bad programmer, using a regex to do something as simple as checking to see is a string starts with a dot. I do not know who is paying you for raising this bad publicity, bad I find your post to be very sad. Please do proper comparisons.

Apples and Oranges

The Ruby code is going to be several orders of magnitude slower, granted, but you could at least compare the right things. Whoever wrote the initial code used a regexp to check whether the first character in a string is a '.', which arguably doesn't make much sense. If your assembler code did the same, it would have to use a complete regexp implementation. next if entry[0] == '.' seems very readable to me, and there is no reason at all the compiled code couldn't be exactly the same as your assembler. So your point was ...? ;-)

Several factors mixed here

The Ruby code is slower in many dimensions and for many reasons, that must be separated:

- It uses a general-purpose regex package. General-purpose anything is always slower than custom code. In a fair language-to-language comparision you should write a regex library in hand-tuned Assembly, then compare that to Ruby's regex lib (which is written in C, but could be Ruby - Java's regex for example is written in Java, in traditional eating-own-dogfood fashion).
- Algorithmic quality of the regex package; Ruby's does not use advanced compilation tricks like those suggested by other poster. The regex engine is a full-blown interpreter over the input.
- High-level language (and platform: GC'd heap, etc.), versus low-level language and platform. Many tradeoffs in each side. (Even if you are an ASM guru, I challenge you to write the equivalent to a nontrivial, say 100-line Ruby program (that would become a several-KLOC program in ASM) that runs perfectly at first execution, without core dumps etc.)
- Interpreted versus native code. Yes the latest Ruby has a JIT compiler, but it's pretty raw tech at this time (doesn't hold a candle to mature JITs like HotSpot).
- Dynamic typing, that makes Ruby even slower than most high-level languages (although in such a simple example where you only have a call to regex which is a language-intrinsic operation, this won't probably be a factor... but add a trivial numeric arithmetic or function call, and you're again in the range of million-times-slower-than-ASM unless a sufficiently strong JIT - one that does not exist at this time, perhaps JRuby + HotSpot Server is what comes closer - optimizes it well enough).

Cycles

Completely agree with your point that this potentially kills many cycles for a simple byte comparison (assuming it's an ASCII string). It'd be advantageous to apply a little back-to-the-roots programming common sense in some cases like this, especially if you know it's hot code.
Regarding your assembly translation: It'd be interesting to see what modern Regexp JIT compilers come up with for the /^\./ expression, see [1]. Maybe such output is in fact very similar to your "compiler output"/assembly translation. After all, the complexity of the comparison could still be O(1) and you would run highly efficient assembly - but that's of course, *only after* all the cycles were burnt to do the Regexp JIT compilation :-)
Not so familar with the language itself - does anyone know whether Ruby compiles RegExp's to native instructions?
[1] http://trac.webkit.org/browser/trunk/JavaScriptCore/yarr