Patrick Collison » blog

Feel like calling your company something like Cashcoup, Feebany, Bunkapps, Morpone, Realance or Afative? Combining CrunchBase, Mathematica and stochastic matrices yields the Web 2.0 Company Name Generator:

My first attempt at automated name generation used a few Gutenberg books, which yielded appropriately Victorian-sounding names. CrunchBase seems to work better. If you want to experiment with the code, download the notebook.

Update: you can now use the name generator interactively.

Steven Troughton-Smith is a very talented iPhone developer from Dublin, with a bunch of cool apps under his belt. His blog is full of useful iPhone tidbits (Using Dynamic Library Injection with the iPhone Simulator, On Speed, Development & Design).

Anyway, the cool news is that he’s now going to be spending some time hacking on the offline Wikipedia iPhone app. Stay tuned to see what cool features he cooks up.

Back To My Mac seemed like a neat feature when Steve demoed it back at WWDC 07, but very little attention seems to have been paid to it since.

Remote NAT traversal for screen sharing and AFP is cool and all, but the most useful part is hardly mentioned anywhere: Back To My Mac can automatically establish on-the-fly tunnels to any machine with Back To My Mac enabled. You can just ssh foobar.joebloggs.members.mac.com, or curl something directly from the web server, or whatever. So long as you can make outgoing connections, it should work around any routers, firewalls, and other wrinkles in the network topology.

The catch is that it only works over IPv6. sshd on OS X has IPv6 enabled by default, as does Apache, but a lot of other stuff doesn’t.

I haven’t figured out how it works yet. It’s definitely not a straight IPv6 tunnel—the source IP of any connection is a private address (which kinda seems to defeat the purpose of using IPv6 in the first place). Any info or pointers appreciated.

Update: In the comments, JH points out that it’s not a private address, but an RFC 4193 unique local address.

John Gruber writes:

I’ve seen third-party iPhone developers complaining that this trick is only available to Apple; they want to use it too. The technical reason why they can’t is that because application bundles are cryptographically signed, you can’t modify the contents of the application bundle (by, in this case, changing the default.png resource file) without breaking the digital signature. Apple could enable this feature for signed applications by providing for a way to specify a dynamic default.png that exists outside the application bundle, somewhere in the application’s private Library folder.

With a bit of hackery, it turns out that you can actually create dynamic Default.png files that don’t cause problems. Here’s a demo of it in action:

This is possible because OS X’s codesign binary (I’ve had far too many run-ins with it while writing the offline Wikipedia browser), used to sign and verify bundles, doesn’t traverse symlinks:

$ codesign -vv Rememberer.app
Rememberer.app: valid on disk
$ touch Rememberer.app/test
$ codesign -vv Rememberer.app
Rememberer.app: a sealed resource is missing or invalid
/Users/patrick/Projects/Rememberer/build/Debug-iphoneos/Rememberer.app/test: resource added
$ rm Rememberer.app/test
$ codesign -vv Rememberer.app
Rememberer.app: valid on disk
$ ls -l Rememberer.app/randomfile
lrwxr-xr-x 1 patrick staff 24 8 Nov 17:21 Rememberer.app/randomfile -> ../Documents/randomfile
$ dd if=/dev/random of=Documents/randomfile count=1
1+0 records in
1+0 records out
512 bytes transferred in 0.000095 secs (5382165 bytes/sec)
$ codesign -vv Rememberer.app
Rememberer.app: valid on disk

This is somewhat understandable; the symlink itself doesn’t change. But if “randomfile” is instead something like “Default.png”, the OS will happily load it from the default path in the application bundle—and follow the symlink—even though the file is actually stored in an area (Documents) that’s dynamically modifiable.

I’m guessing that Apple will consider this a bug, and fix it in some future version of the OS. If that happens, though, the downside will probably be nothing worse than losing your dynamic Default.png.

To get it to work in Xcode, you can just add a Run Script phase to the Target:

ln -sf ../Documents/Default.png $TARGET_BUILD_DIR/$CONTENTS_FOLDER_PATH

Here’s the Xcode project for the above demo. (Code is public domain.)

Update (Nov 19): TechCrunch pointed out some wider implications of this vulnerability. Although the article was met with some skepticism, they’re basically right. There’s a good summary of the situation on the McAfee Avert Labs Blog.

I wrote the offline Wikipedia browser back before there was any official iPhone SDK documentation (or SDK, for that matter), and figuring out the APIs was a bit of a challenge. So in trying to get a handle on things, I wrote an API explorer for showing a rough outline of the system’s classes. It started out as a bare-bones script, and since then I’ve gradually bolted various bits on to it.

Unlike many compiled languages, Objective-C supports pretty powerful runtime introspection. The explorer uses this to present the implemented protocols, methods and instance variables of every loaded class. In addition, if the class responds to initWithFrame: (these are usually subclasses of UIView), you can draw and resize an instance, to get a basic feel for what it does.

It’s all more easily explained with a short screencast:

If you want to play around with it (it works in both the simulator and on the devices themselves), you can download the code.

Economic theory suggests that financial innovation must lead to failures. And, in particular, since successful innovations are hard to predict, the infrastructure necessary to support innovation needs to lag the innovations themselves, which increases the probability that controls will be insufficient at times to prevent breakdowns in governance mechanisms. Failures, however, do not lead to the conclusion that re-regulation will succeed in stemming future failures. Or that society will be better off with fewer freedoms. Although governments are able to regulate organisational forms, they are unable to regulate the services provided by competing entities, many yet to be born. Organisational forms change with financial innovations. Although functions of finance remain static and are similar in Africa, Asia, Europe and the United States, their provision is dynamic as entities attempt to profit by providing services at lower cost and greater benefit than competing alternatives.

—Myron Scholes (yeah, that Scholes), debating Joseph Stiglitz.

Back at the start of the year, I blogged about an app I wrote that allows you to store a complete copy of Wikipedia on an iPhone/iPod Touch.

The app got more attention than I expected, with tens of thousands of downloads in the first month, which I think made it one of the more popular apps for the jailbroken iPhone. (Not anticipating any of this, the non-existent documentation and installer ensured many were confused, and so someone made a YouTube installation tutorial that has over 57,000 views at time of writing. I’m not sure if that’s good or bad.)

I also released the app’s source code, and it’s been pretty fun to work with a lot of talented people in improving it. The OLPC crew took an interest in it, and thanks to some cool work from Chris Ball and Wade Brainerd, the iPhone application was ported to the XO laptop. Chris announced in June that:

We’re going to be shipping the result to Peru on tens of thousands of laptops in the near future, and it should go up to hundreds of thousands if the other South American countries with OLPC deployments decide to include it in their builds too.

When the iPhone 3G was announced, I didn’t originally intend to port the application to the new version of the OS. The original app was a short Christmas project, and now that I’m working at Live Current, I don’t have much spare time to hack. But after a few hundred emails enquiring about a new version, I eventually felt too guilty not to. So I spent a weekend porting it to iPhone OS 2.0, added a handful of new features, and I’m happy to say that the end result is now available in the App Store.

While working on some web analytics software a few weeks ago, I wrote a dirty hack to enable pictures in SLIME REPLs. The implementation is ugly, but I think the concept is neat:

You can grab the patch against CVS SLIME here.

Yanking pictures back into the REPL works fine too, so you can do something like (defparameter *my-chart* [picture-object]).

Sample usage:

(defparameter *flickr-url* "http://www.flickr.com/search/?q=~A&s=int")

(defun flickr-pic (search)
  (let ((html (drakma:http-request (format nil *flickr-url* search))))
    (cl-ppcre:register-groups-bind (url)
        ("photo_container.+img src=\"([^\"]+)\"" html)
      (swank:make-picture :url url :type 'jpeg))))

(flickr-pic "linux")

Update (22/06): To clarify in response to the reddit thread, this is a dirty proof-of-concept that resulted from an hour of hacking, and obviously shouldn’t be merged into mainline SLIME as it stands. See the slime-devel thread for the discussion surrounding it. With that said, it should be fairly straightforward to rework it as a saner contrib module.

Some sweet work by Chris Ball and Wade Brainerd has resulted in the iPhone app I wrote for offline browsing of Wikipedia being ported to the OLPC. From Chris’s email:

We’re going to be shipping the result to Peru on tens of thousands of laptops in the near future, and it should go up to hundreds of thousands if the other South American countries with OLPC deployments decide to include it in their builds too.

The source is available in the OLPC repository.

Awesome!

Sho Fukamachi has a spectacularly uninformed piece on MagLev and language implementation. He writes:

There are about 5 serious, credible, working Ruby implementations - MRI, YARV, JRuby, Rubinius, and IronRuby. They all have highly intelligent, experienced, dedicated staff who know a lot more about writing compilers and VMs than I could ever hope to learn. […] Do you seriously think that all these smart people, writing (and collaborating on) all these projects have somehow missed the magic technique that’s going to make Ruby run 60x faster?

Fukamachi distorts things (he goes from “up to 60x faster” to the unqualified “60x faster”), but to the underlying question—”despite all the work that’s been done on Ruby VMs, can MagLev really be hugely faster?”—the answer is an emphatic yes.

For a start, MRI is neither a compiler or a VM—it’s an interpreter. MagLev/GemStone compiles to bytecodes which are then JIT’ed to native code. MRI uses a non-compacting mark-and-sweep garbage collector; GemStone uses a generational collector. GemStone uses inline caches to speed message sends. And so on. There’s no “magic technique”. GemStone have been working on this implementation for decades. Like compound interest, incremental improvements over so many years tend to add up.

But a broad 10x speedup across the whole language beggars belief. It should be impossible. Actually, I’m going to come right out and say it is impossible, until conclusively proven otherwise, to make a fully compatible ruby implementation that’s more than two or three times faster than today’s best.

These kind of predictions are why we should have a site that stores quotes and holds people accountable. Look at Ruby’s performance on the language shootout. If Ruby was 10x faster, it’d still be slower than Python (Psyco) and Lua (LuaJIT).

As it happens, I write a lot of Common Lisp, which is both more flexible than Ruby, and (according to the shootout) runs 37x faster. I don’t know if MagLev will actually be 10x faster when it’s released, but it’s uninformed burbling to claim that such a speedup is impossible.

Benchmarks on a system which isn’t even partially compatible with Ruby are utterly worthless. I can write some routine which messes around with arrays in C which is a hundred times faster than Ruby. […] Who knows what they actually are, how tuned they are, whether they’re capable of doing anything other than running those benchmarks fast (I doubt it).

“Isn’t even partially compatible”? Maybe he missed the part where MagLev ran WEBrick.

And wow ..! A shared memory cache! Finally, Rails can cast off that shared-nothing millstone around its neck. Except, of course, that shared-nothing is one of its main selling points. If you want to share objects use the database! That’s what it’s there for!

A Rails deployment is not shared-nothing (unless each scaling unit has its own database, and each request contains enough information to route it to the correct node).

Except of course that OODBs have been around for decades, and the last time I checked, we’re all still using 3rd normal form. If OODBs were the solution to all scaling’s ills then Facebook would be using Caché, not MySQL. Guess which one they’re using.

Facebook scales through aggressive use of memcached and abandonment of 3NF.

This credulity and blind bandwagon-jumping is the single worst thing about the Rails community.

Mindless ranting is well up there too.

Disclaimer: Avi and the rest of the MagLev team are friends of mine.

Update (05/06): As expecting, preliminary benchmarking Antonio Cangiano shows that MagLev is indeed hugely faster than MRI (and only marginally slower than C++).