Some projects I've worked on at D. E. Shaw Research

D. E. Shaw Research does scientific research in computational biochemistry and molecular dynamics. (See an up-to-date list of D. E. Shaw Research papers and publications in various leading science and engineering journals and conferences)

David Shaw's 2006 talk at Stanford University's Computer Systems Colloquium has both video and slides which provide background, context and elaboration for our lab's work.

We have designed and built Anton 2 and previously, Anton, the fastest biomolecular simulation machines in the world. Some papers describing science done on Anton:

• How Fast-Folding Proteins Fold (Science, 28 Oct 2011) reports the results of many long atomic-level molecular dynamics simulations on Anton, each run for periods ranging between 100 µs and 1 ms of simulated time, that reveal a a set of common principles underlying the folding of 12 structurally diverse proteins. The proteins range in size from 10 to 80 residues, and include members of all three majoral structural classes (α-helical, β sheet and mixed α/β). Each protein was simulated near the melting temperature (so both folding and unfolding events were likely). For each protein, between one and four simulations were run for long enough to observe at least 10 folding and 10 unfolding events. In aggregate, this represents ˜8 ms of simulation with more than 400 folding and unfolding events. The proteins were all observed to fold close to their experimentally determined native structures.
• How Does a Drug Molecule Find Its Target Binding Site?(JACS, 5 May 2011) provides a continuous, atomic-level view of binding process of a ligand (the cancer drug dasatinib or the kinase inhibitor PP1) to a protein (Src kinase). In long, unguided simulations on Anton, the ligand, initially placed at a random location within a box that also contained the protein, correctly identified its target binding site. Watch the movie: the ligand is the orange-brown molecule.
• Atomic-Level Characterization of the Structural Dynamics of Proteins (Science, 15 Oct 2010) describes results of various record-length molecular dynamics simulations on Anton, including a millisecond-long simulation of BPTI (bovine pancreatic trypsin inhibitor) and hundreds of microseconds of WW domain and villin headpiece, including complete folding simulations from a fully extended state]

Papers describing Anton 2:

• Anton 2: Raising the bar for performance and programmability in a special-purpose molecular dynamics supercomputer (SC14, Gordon Bell Prize), describes the overall architecture, algorithms and performance of Anton 2. Highlight: getting to meet Gordon Bell.
• Hot Chips 26, described chip-level implementation details. This received some coverage by the technical press, e.g. an Tech Design Forums focusing on the thermal design challenges for high power chips like Anton 2, and an EETimes used a colorful "rare birds" metaphor for our "monster chip"!.
• Unifying on-chip and inter-node switching within the Anton 2 network (ISCA 2014)
• Hardware support for fine-grained event-driven computation in Anton 2 (ASPLOS 2013)

Papers describing Anton:

• Extending the Generality of Molecular Dynamics Simulations on a Special-Purpose Machine (IPDPS 2013, Boston, Best Paper, Applications) describes how we achieved high performance on several advanced molecular dynamics features.
• Millisecond-scale molecular dynamics simulations on Anton (SuperComputing 2009, Best Paper, Gordon Bell Prize for Special Achievement) describes Anton's algorithms and reports measured performance running the longest molecular dynamics simulations that anyone has run (as far as we know).
• Anton, a special-purpose machine for molecular dynamics simulation (Communications of the ACM, July 2008)
• Exploiting 162-Nanosecond End-to_end Communication Latency on Anton describes the combination of hardware mechanisms and software constructs that put Anton's communication network ahead of what's been achieved on most other clusters and supercomputers.
• Anton, a special-purpose machine for molecular dynamics simulation, (ISCA 2007).
• High-Throughput Pairwise Point Interactions in Anton, a Specialized Machine for Molecular Dynamics Simulation (HPCA 2008).
• Incorporating Flexibility in Anton, a Specialized Machine for Molecular Dynamics Simulation (HPCA 2008).

A side-project that emerged from Anton is a class of "counter-based" random number generators (CBRNGs), which we presented at SC11 (awarded Best Paper). CBRNGs are fast, stateless functions that are great for modern multi-core or distributed applications -- the source code is available for download. It was interesting getting them to work in C, C++0X as well as OpenCL (on an AMD HD6970) and CUDA (on an NVIDIA GTX580). NVIDIA includes one of these generators in their CURAND library, starting with version 6, they've been described in books like Numerical Computation with GPUS, and papers like Random number generators for massively parallel simulations on GPU. There's also a Haskell port of Random123, and a Julia implementation. There's a version of Random123 in Fujitsu's Open Petascale library. GitHub has a Boost-compatible implementation of Random123.

Another of our projects is Desmond, a scalable molecular dynamics (MD) program for commodity clusters (aka Beowulfs). We first described the performance of Desmond on a 2005-vintage Infiniband+Opteron cluster in Scalable Algorithms for Molecular Dynamics Simulations on Commodity Clusters, (SuperComputing 2006, Best Paper). We also released a tech report with updated Desmond performance numbers on our 2nd generation (2008-vintage) Infiniband+Xeon cluster, and Desmond performance numbers on modern NVIDIA GPUs.

I gave a talk on D. E. Shaw Research's experience with deploying our first generation (vintage 2005) Infiniband+Opteron cluster at the 2007 OpenFabrics Sonoma Workshop

The D. E. Shaw Research publications page.

More general articles about D. E. Shaw Research:

• A news article on a paper from our researchers in Science, 15 Oct 2010, about long-timescale folding
• Nature also did a news article on the 15-Oct-2010 Science paper, which got slashdotted.
• The American Chemical Society's Chemical & Engineering News also had an article concurrent with the 15-Oct-2010 Science paper
• D. E. Shaw Research is loaning an Anton machine to the Pittsburgh Supercomputer Center for use by US academic researchers. They have highlights on some of the research done with that machine in their annual reports, Epic Microseconds (2012) and Protein Research Leaps Forward (2011).
• Nature discussed some aspects of the challenge and some of those tackling it in a news feature called Chemistry: Power Play
• The space for the first Anton supercomputer was mentioned in artdaily.org. Pictures of the space with our first 512-node Anton in it can be seen in more recent articles
• The New York Times: John Markoff wrote a brief piece mentioning our CACM paper on Anton

My home page.