Patrick Collison » blog

While working on some web analytics software a few weeks ago, I wrote a dirty hack to enable pictures in SLIME REPLs. The implementation is ugly, but I think the concept is neat:

You can grab the patch against CVS SLIME here.

Yanking pictures back into the REPL works fine too, so you can do something like (defparameter *my-chart* [picture-object]).

Sample usage:

(defparameter *flickr-url* "http://www.flickr.com/search/?q=~A&s=int")

(defun flickr-pic (search)
  (let ((html (drakma:http-request (format nil *flickr-url* search))))
    (cl-ppcre:register-groups-bind (url)
        ("photo_container.+img src=\"([^\"]+)\"" html)
      (swank:make-picture :url url :type 'jpeg))))

(flickr-pic "linux")

Update (22/06): To clarify in response to the reddit thread, this is a dirty proof-of-concept that resulted from an hour of hacking, and obviously shouldn’t be merged into mainline SLIME as it stands. See the slime-devel thread for the discussion surrounding it. With that said, it should be fairly straightforward to rework it as a saner contrib module.

Some sweet work by Chris Ball and Wade Brainerd has resulted in the iPhone app I wrote for offline browsing of Wikipedia being ported to the OLPC. From Chris’s email:

We’re going to be shipping the result to Peru on tens of thousands of laptops in the near future, and it should go up to hundreds of thousands if the other South American countries with OLPC deployments decide to include it in their builds too.

The source is available in the OLPC repository.

Awesome!

Sho Fukamachi has a spectacularly uninformed piece on MagLev and language implementation. He writes:

There are about 5 serious, credible, working Ruby implementations - MRI, YARV, JRuby, Rubinius, and IronRuby. They all have highly intelligent, experienced, dedicated staff who know a lot more about writing compilers and VMs than I could ever hope to learn. […] Do you seriously think that all these smart people, writing (and collaborating on) all these projects have somehow missed the magic technique that’s going to make Ruby run 60x faster?

Fukamachi distorts things (he goes from “up to 60x faster” to the unqualified “60x faster”), but to the underlying question—”despite all the work that’s been done on Ruby VMs, can MagLev really be hugely faster?”—the answer is an emphatic yes.

For a start, MRI is neither a compiler or a VM—it’s an interpreter. MagLev/GemStone compiles to bytecodes which are then JIT’ed to native code. MRI uses a non-compacting mark-and-sweep garbage collector; GemStone uses a generational collector. GemStone uses inline caches to speed message sends. And so on. There’s no “magic technique”. GemStone have been working on this implementation for decades. Like compound interest, incremental improvements over so many years tend to add up.

But a broad 10x speedup across the whole language beggars belief. It should be impossible. Actually, I’m going to come right out and say it is impossible, until conclusively proven otherwise, to make a fully compatible ruby implementation that’s more than two or three times faster than today’s best.

These kind of predictions are why we should have a site that stores quotes and holds people accountable. Look at Ruby’s performance on the language shootout. If Ruby was 10x faster, it’d still be slower than Python (Psyco) and Lua (LuaJIT).

As it happens, I write a lot of Common Lisp, which is both more flexible than Ruby, and (according to the shootout) runs 37x faster. I don’t know if MagLev will actually be 10x faster when it’s released, but it’s uninformed burbling to claim that such a speedup is impossible.

Benchmarks on a system which isn’t even partially compatible with Ruby are utterly worthless. I can write some routine which messes around with arrays in C which is a hundred times faster than Ruby. […] Who knows what they actually are, how tuned they are, whether they’re capable of doing anything other than running those benchmarks fast (I doubt it).

“Isn’t even partially compatible”? Maybe he missed the part where MagLev ran WEBrick.

And wow ..! A shared memory cache! Finally, Rails can cast off that shared-nothing millstone around its neck. Except, of course, that shared-nothing is one of its main selling points. If you want to share objects use the database! That’s what it’s there for!

A Rails deployment is not shared-nothing (unless each scaling unit has its own database, and each request contains enough information to route it to the correct node).

Except of course that OODBs have been around for decades, and the last time I checked, we’re all still using 3rd normal form. If OODBs were the solution to all scaling’s ills then Facebook would be using Caché, not MySQL. Guess which one they’re using.

Facebook scales through aggressive use of memcached and abandonment of 3NF.

This credulity and blind bandwagon-jumping is the single worst thing about the Rails community.

Mindless ranting is well up there too.

Disclaimer: Avi and the rest of the MagLev team are friends of mine.

Update (05/06): As expecting, preliminary benchmarking Antonio Cangiano shows that MagLev is indeed hugely faster than MRI (and only marginally slower than C++).

(defun bracket-reader (stream char)
  (declare (ignore char))
  (let* ((lst (read-delimited-list #] stream t))
         (pos (position '|| lst)))
    (if pos
        `#'(lambda ,(mapcar #'intern (subseq lst 0 pos))
             ,@(nthcdr (1+ pos) lst))
        `#'(lambda (_) ,lst))))

(set-macro-character #[ #'bracket-reader)
(set-macro-character #] (get-macro-character #)))

Closures are beautiful, but the heaviness of CL’s lambda syntax kept jumping out at me as fairly ugly after a few months of writing Smalltalk. The above snippet allows you to write things like:

(mapcar [+ _ 1] '(1 2 3))

and:

(maphash [k v || (print v)] tbl)

Which, to my eyes at least, is nicer than:

(mapcar #'(lambda (x) (+ x 1)) '(1 2 3))

and:

(maphash #'(lambda (k v) (print v)) tbl)

Of course, to add syntax to Lisp is to wade into failure-littered territory. But although no-one agrees how it should be done, I really don’t think it’s a bad idea.

The Lisp Machines were easily the best environments ever created for developing in Lisp, but the companies that produced them died out quite a while ago (Dan Weinreb has described some of the reasons for this). They weren’t just great tools for writing Lisp—Genera is one of the best environments for writing code that’s been created for any language. Squeak is the only thing I’ve seen that comes close.

Although the current legal status of the IP around the Lisp Machines is unclear, you can now get your hands on the source, and run the environment pretty easily on a 64-bit Linux installation (or even OS X + VMWare + Linux)—credit for the Linux port goes to Brad Parker.

Setting it up isn’t exactly a point-and-click operation, and guides are non-existent (I think)—so here’s what I did to get it up and running on OS X using VMWare and Ubuntu 7.10 x86-64 (this may clobber other things; I’m assuming a dedicated Ubuntu installation):

Once you’re in the Genera environment, evaluate the following:

To test things out, you can evaluate something like Edit Definition read-from-string, which should pull the source for read-from-string from the Linux host over NFS. If you edit the definition, load it with M-x Compile Changed Definitions. If it all works, congratulations; you now have a fully-functioning Genera 8.5 environment.

Genera itself is excellently documented (just fire up the Document Examiner), but the emulator has very sparse docs. I’ll post some more soon with further details about day-to-day use—keybindings, etc.

Things I want my bank to have, in order of increasing wishful-thinking-ness:

• An RSS feed of my account activity (aggregated for all accounts). Especially desirable for credit card transactions.
• SMS alerts, if fishy activity is suspected.
• A Dashboard widget with a quick overview of my accounts.
• Spending analytics—I should be able to categorise the businesses on my credit card statement, and this data should be shared between all customers. 95% of businesses will be categorised very quickly.
• An API.
• Customisable actions—direct debits, standing orders, “Keep the Change”, and the like, are (relatively) expensive and awkward services that would be much better implemented in a customisable way with something akin to mail filters (as done in Gmail and Apple Mail).

Conrad Barski just posted a preview of his new book, Land of Lisp. It is distilled awesomeness. His other comics, like Casting SPELs in Lisp, are also great. Having him on board as an author looks like a coup for No Starch Press.

(Via Lemonodor)

Our company, Auctomatic, has been acquired by Live Current Media. The story has appeared in a few news outlets, including The Irish Times, the BBC, and VentureBeat, which has an interview with Harjeet.

I’ll post some longer reflections later; sleep beckons right now!

I recently switched from Meteor to Vodafone when in Ireland because Vodafone’s data rate (50MB for €1) is 3,000 times cheaper than Meteor’s (6c/KB).

Vodafone don’t have an EDGE network, but they do support GPRS—which works with the iPhone, despite appearances (GPRS coverage is indicated by an odd-looking blank square beside the carrier logo).

To get it to work, I edited my SystemPreferences plist file (/var/preferences/SystemConfiguration/preferences.plist) to contain the Vodafone APN details and a pointer to a PAC file describing the Vodafone proxy server (here’s the relevant section), and then created a simple proxy.pac in /var/root.

The Vodafone proxy doesn’t seem to allow much bar web traffic, but it’s enough for basic use on the move.

TeTeX isn’t maintained any more, but I’ve written a fair few scripts over the years that depend on it, and so I’ve yet to switch over to something like MacTeX or TeX Live.

TeTeX 3.0 doesn’t build out-of-the-box on Leopard. In case it’s useful to anyone, here is the patch I hacked together to get it to do so.