Getting the best performance today relies on deploying high performance clusters, rather than single unit supercomputers. But building clusters can be expensive, but using Linux can be both a cheaper alternative and make it easy to develop and deploy software across the cluster. I interview Joseph D Sloan, author of High Performance Linux Clusters about what makes a cluster, how Linux cluster competes with Grid and proprietary solutions and how he got into clustering technology in the first place.

Clustering with Linux is a current hot topic – can you tell me a bit about how you got into the technology?In graduate school in the 1980s I did a lot of computer intensive modeling. I can recall one simulation that required 8 days of CPU time on what was then a state-of-the art ($50K) workstation. So I’ve had a longtime interest in computer performance. In the early 1990s I shifted over to networking as my primary interest. Along the way I set up a networking laboratory. One day a student came in and asked about putting together a cluster. At that point I already had everything I needed. So I began building clusters.The book covers a lot of material – I felt like the book was a complete guide, from design through to implementation of a cluster – is there anything you weren’t able to cover?Lots! It’s my experience that you can write a book for beginners, for intermediate users, or advanced users. At times you may be able to span the needs of two of these groups. But it is a mistake to try to write for all three. This book was written to help folks build their first cluster. So I focused on the approach that I thought would be most useful for that audience.First, there is a lot of clustering software that is available but that isn’t discussed in my book. I tried to pick the most basic and useful tools for someone starting out.Second, when building your first cluster, there are things you don’t need to worry about right away. For example, while I provide a brief description of some benchmarking software along with URLs, the book does not provide a comprehensive description of how to run and interpret benchmarks. While benchmarks are great when comparing clusters, if you are building your first cluster, to what are you going to compare it? In general, most beginners are better off testing their cluster using the software they are actually going to use on the cluster. If the cluster is adequate, then there is little reason to run a benchmark. If not, benchmarks can help. But before you can interpret benchmarks, you’ll first need to know the characteristics of the software you are using-is it I/O intensive, CPU intensive, etc. So I recommend looking at your software first.What do you think the major contributing factor to the increase of clusters has been; better software or more accessible hardware?Both. The ubiquitous PC made it possible. I really think a lot of first-time cluster builders start off looking at a pile of old PCs wondering what they can do with them. But, I think the availability of good software allowed clusters to really take off. Packages like OSCAR make the task much easier. An awful lot of folks have put in Herculean efforts creating the software we use with very little thought to personal gain. Anyone involved with clusters owes them a huge debt.Grids are a hot topic at the moment, how do grids – particularly the larger toolkits like Globus and the Sun Grid Engine – fit into the world of clusters?I’d describe them as the next evolutionary stage. They are certainly more complex and require a greater commitment, but they are evolvingrapidly. And for really big, extended problems, they can be a godsend.How do you feel Linux clusters compare to some of the commercially-sourced, but otherwise free cluster technology like Xgrid from Apple?First, the general answer: While I often order the same dishes when I go to a restaurant, I still like a lot of choices on the menu. So I’m happy to see lots of alternatives. Ultimately, you’ll need to make a choice and stick to it. You can’t eat everything on the menu. But the more you learn about cooking, the better all your meals will be. And the more we learn about cluster technology, the better our clusters will be.Second, the even more evasive answer: Designing and building a cluster requires a lot of time and effort. It can have a very steep learning curve. If you are already familiar with Linux and have lots of Linux boxes, I wouldn’t recommend Xgrid. If you are a die-hard Mac fan, have lots of Mac users and systems, Xgrid may be the best choice. It all depends on where you are coming from.The programming side of a grid has always seemed to be the most complex, although I like the straightforward approach you demonstrated in the book. Do you think this is an area that could be made easier still?Thanks for the kind words. Cluster programming is now much easier than it was a decade ago. I’m a big fan of MPI. And while software often lags behind hardware, I expect we’ll continue to see steady improvement. Of course, I’m also a big fan of the transparent approach taken by openMosix and think there is a lot of unrealized potential here. For example, if the transparent exchange of processes could be matched by transparent process creation through compiler redesign, then a lot more explicit parallel programming might be avoided.What do you think of the recent innovations that puts a 96-node cluster into a deskside case?The six-digit price tag is keeping me out of that market. But if you can afford it and need it …Go on, you can tell me, do you have your own mini cluster at home?Nope-just an old laptop. I used to be a 24/7 kind ‘a computer scientist, but now I try to leaving computing behind when I go home.Like the cobbler’s kid that go without shoes, my family has to put up with old technology and a husband/father that is very slow to respond to their computing crises.When not building clusters, what do you like to do to relax?Relax? Well my wife says …I spend time with my family. I enjoy reading, walking, cooking, playing classical guitar, foreign films, and particularly Asian films. I tried learning Chinese last year but have pretty much given up on that. Oh! And I do have a day job.This is your second book – any plans for any more?It seems to take me a couple of years to pull a book together, and I need a year or so to recover between books. You put so many things on hold when writing. And after a couple of years of not going for a walk, my dog has gotten pretty antsy. So right now I’m between projects.Author BioJoseph D. Sloan has been working with computers since the mid-1970s. He began using Unix as a graduate student in 1981, first as an applications programmer and later as a system programmer and system administrator. Since 1988 he has taught computer science, first at Lander University and more recently at Wofford College where he can be found using the software described in this book.You can find out more on the author’s website. More information on the book, including sample chapters, is available at O’Reilly.