1. 64-bit Gnu/Linux, 2.6.25 kernel or later. I don’t much care which distro, although I personally use openSUSE 11.1. Ubuntu Intrepid Ibex or later, Fedora 9 or later will also work. Hardy Heron’s kernel is too old. If Gentoo ever gets their release engineering act together, I’d consider it. CentOS / RHEL 5 kernel is too old. Debian Lenny might work, but I’ve never used it and Ubuntu seems to be more tuned to a workstation than Debian, which is mostly a server distro.

2. R 2.8.1-patched or later. This has access to all the repositories; older versions don’t know about RForge. 2.9.0 is almost ready for release; I’ll be alpha testing it daily.

3. All of the CRAN task views! That includes the dependencies. For example, if you have Rgraphviz, you need graphviz and graphviz-devel (on openSUSE).

4. ggobi 2.1.8. or later.

5. PostgreSQL 8.3.5 — sorry, dolphins. You can have MySQL but I won’t use it. pgadmin3 is also required.

6. Lyx 1.6.1 or later. This integrates with R for publication-quality graphics, “literate programming” and “reproducible research”.

That’s what I’ve got on my workstation, and that’s what I’d expect as a minimum for software.

I can give you openSUSE 11.1 build scripts; the standard openSUSE 11.1 and even 11.2 (”Factory”) packages are behind the state of the art for R, GGobi and LyX, so I build them from upstream source. I’ve got most of this working on two different machines, and the scripts are up on Github, but I don’t have an Amazon AMI to test with yet. I could probably build an image here locally with the tools openSUSE has (Kiwi, Xen).

Bonus points if there’s a way to build the ATLAS Automatically Tuned Linear Algebra Subroutines on a virtual machine. :)

We can put in volume images that have “smaller” but related datasets — so, the BSD etc/words, the panoply of spellchecking programs, the BNC corpus word frequency list, the gutenberg.org word lists and dictionaries.

NLTK’s corpora will probably live on their own.

Packages: Every spell checking package you can put in there (for the word lists!)

Data: Wikipedia catagories dump, freebase, dmoz category dump. NIST text data if you can get it.

@Pete, your list of Python packages was indispensible! And thanks for the info about startup scripts — looks like a good way to manage fast-changing resources (like the Infinite Monkeywrench…).

Check http://machetec2.org for updates (right now it just points at this blog entry but soon will be its own wiki).

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1403

If the list of installs grows fairly large and you are building things from source on new versions of the ami, you can hit issues with unavailable repos or mirrors. A good strategy there is to build those non apt packages from an infochimps src repo just in case they move in the future.

I ended up using puppet for a lot of this package management stuff at work, but that might be a pain for public AMIs. A simple ami installtion bash script often does the job.

http://code.google.com/p/elasticwulf/

My image is way out of date at this point, and doesn’t do things like automatically load EBS volumes. I’ve been planning on updating it, but maybe I’ll hijack the infochimps image if you publish build scripts.

One thing I would suggest is to make sure that you have a 64 bit version of the ami, and that you use the corresponding 64 bit version of python before building numpy/scipy – otherwise you will hit a 2GB memory limit when processing data with scipy.