Archive for June, 2008

Pictures in a SLIME REPL

Saturday, June 21st, 2008

While working on some web analytics software a few weeks ago, I wrote a dirty hack to enable pictures in SLIME REPLs. The implementation is ugly, but I think the concept is neat:

You can grab the patch against CVS SLIME here.

Yanking pictures back into the REPL works fine too, so you can do something like (defparameter *my-chart* [picture-object]).

Sample usage:

(defparameter *flickr-url* "http://www.flickr.com/search/?q=~A&s=int")

(defun flickr-pic (search)
  (let ((html (drakma:http-request (format nil *flickr-url* search))))
    (cl-ppcre:register-groups-bind (url)
        ("photo_container.+img src=\"([^\"]+)\"" html)
      (swank:make-picture :url url :type 'jpeg))))

(flickr-pic "linux")

Update (22/06): To clarify in response to the reddit thread, this is a dirty proof-of-concept that resulted from an hour of hacking, and obviously shouldn’t be merged into mainline SLIME as it stands. See the slime-devel thread for the discussion surrounding it. With that said, it should be fairly straightforward to rework it as a saner contrib module.

Wikipedia-iPhone ported to the OLPC

Tuesday, June 3rd, 2008

Some sweet work by Chris Ball and Wade Brainerd has resulted in the iPhone app I wrote for offline browsing of Wikipedia being ported to the OLPC. From Chris’s email:

We’re going to be shipping the result to Peru on tens of thousands of laptops in the near future, and it should go up to hundreds of thousands if the other South American countries with OLPC deployments decide to include it in their builds too.

The source is available in the OLPC repository.

Awesome!

MagLev and language implementation

Monday, June 2nd, 2008

Sho Fukamachi has a spectacularly uninformed piece on MagLev and language implementation. He writes:

There are about 5 serious, credible, working Ruby implementations - MRI, YARV, JRuby, Rubinius, and IronRuby. They all have highly intelligent, experienced, dedicated staff who know a lot more about writing compilers and VMs than I could ever hope to learn. […] Do you seriously think that all these smart people, writing (and collaborating on) all these projects have somehow missed the magic technique that’s going to make Ruby run 60x faster?

Fukamachi distorts things (he goes from “up to 60x faster” to the unqualified “60x faster”), but to the underlying question—”despite all the work that’s been done on Ruby VMs, can MagLev really be hugely faster?”—the answer is an emphatic yes.

For a start, MRI is neither a compiler or a VM—it’s an interpreter. MagLev/GemStone compiles to bytecodes which are then JIT’ed to native code. MRI uses a non-compacting mark-and-sweep garbage collector; GemStone uses a generational collector. GemStone uses inline caches to speed message sends. And so on. There’s no “magic technique”. GemStone have been working on this implementation for decades. Like compound interest, incremental improvements over so many years tend to add up.

But a broad 10x speedup across the whole language beggars belief. It should be impossible. Actually, I’m going to come right out and say it is impossible, until conclusively proven otherwise, to make a fully compatible ruby implementation that’s more than two or three times faster than today’s best.

These kind of predictions are why we should have a site that stores quotes and holds people accountable. Look at Ruby’s performance on the language shootout. If Ruby was 10x faster, it’d still be slower than Python (Psyco) and Lua (LuaJIT).

As it happens, I write a lot of Common Lisp, which is both more flexible than Ruby, and (according to the shootout) runs 37x faster. I don’t know if MagLev will actually be 10x faster when it’s released, but it’s uninformed burbling to claim that such a speedup is impossible.

Benchmarks on a system which isn’t even partially compatible with Ruby are utterly worthless. I can write some routine which messes around with arrays in C which is a hundred times faster than Ruby. […] Who knows what they actually are, how tuned they are, whether they’re capable of doing anything other than running those benchmarks fast (I doubt it).

“Isn’t even partially compatible”? Maybe he missed the part where MagLev ran WEBrick.

And wow ..! A shared memory cache! Finally, Rails can cast off that shared-nothing millstone around its neck. Except, of course, that shared-nothing is one of its main selling points. If you want to share objects use the database! That’s what it’s there for!

A Rails deployment is not shared-nothing (unless each scaling unit has its own database, and each request contains enough information to route it to the correct node).

Except of course that OODBs have been around for decades, and the last time I checked, we’re all still using 3rd normal form. If OODBs were the solution to all scaling’s ills then Facebook would be using Caché, not MySQL. Guess which one they’re using.

Facebook scales through aggressive use of memcached and abandonment of 3NF.

This credulity and blind bandwagon-jumping is the single worst thing about the Rails community.

Mindless ranting is well up there too.

Disclaimer: Avi and the rest of the MagLev team are friends of mine.

Update (05/06): As expecting, preliminary benchmarking Antonio Cangiano shows that MagLev is indeed hugely faster than MRI (and only marginally slower than C++).