Linux clusters, InfiniBand and 64-bit processors -- particularly AMD Opteron -- are changing the face and fortunes of enterprise computing and Linux clusters, according to Donald Becker, co-founder of the Beowulf Project, which develops scalable, open source Linux clustering software. In this wide-ranging interview, he discusses the importance of those technologies and why Linux clusters are ousting supercomputers, among other things....
Becker is also chief technology officer at San Francisco-based Penguin Computing.
What do you expect to happen with clustering technologies in 2005?
Donald Becker: We are going to see more packaged commercial applications out there for clusters. Today, we see few applications packages software for clusters, and many of them are somewhat labor intensive to install.
I also think that we will see that managing large sets of machines as clusters, even when they are not serving a single application, will become the preferred approach. Administering machines one at a time, as we have done in the past, is an old-fashioned way of doing things. Anytime you have a collection of servers that would be more effective to administer as a single machine, clustering is an option. It is feasible to administer tens or hundreds of machines with the same effort that it takes to maintain and do updates on a single machine.
Cluster technologies in the future will be for running sets of machines that are synchronized and running the same applications. That is a much broader set of machines than traditional compute clusters and traditional supercomputers have addressed.
But, I think it is always difficult to predict the future … even a year into the future.
Will Linux clusters continue to steal users from supercomputers?
Becker: Linux clusters have been used to replace supercomputers to do jobs that were traditionally handled on specialized machines. For example, they've been used for crash simulation applications and in life sciences. Most of the algorithms and the applications in those areas work very well in clusters.
Commodity clusters bring the cost of computations down. So, simulations, for example, which before would have been cost prohibitive to do as a routine part of production and design can [now] be done cost effectively. Just as importantly, using clusters means that you don't have to have specialized machines and that you can scale up with commodity machines as your application needs grow.
What's happening with processors and interconnects in the clustering arena?
Becker: There is and will be a big shift to AMD Opteron 64-bit processors. We will see 64-bit systems running on AMD Opteron processors being used more widely, but that will certainly not be the dominate processor in the overall marketplace.
InfiniBand is becoming a popular high performance interconnect. InfiniBand does not yet dominate the marketplace, but there is clearly a major shift that has happened this past year to that interconnect.
What are the advantages of InfiniBand?
Becker: I think that InfiniBand will be the obvious programming model to work to, even if InfiniBand hardware is not used. Ethernet has always been the standard interconnect technology. InfiniBand's advantages over Ethernet are primarily latency and a higher performance interconnect. Latency and bandwidth are easy quantitative numbers, but qualitatively, InfiniBand makes remote DMA [direct memory access] operating system bypass much more approachable. It provides a standard way to do operating network operations that don't directly involve the operating system and that will likely change the way that clusters are programmed. Also, there have been several cluster interconnects that have been single-vendor solutions. With InfiniBand, there are a number of different vendors, and that I think opens up the market. It makes customers much more comfortable going with InfiniBand technology, if they have more vendors from which to choose.
Clusters have tended not to depend on or be locked into single-vendor interconnects to deliver the capabilities people expect. Now, I think we will start seeing a larger number of cluster applications that can take advantage of cluster-specific interconnects.
Why is Opteron becoming a compelling choice?
Becker: In the past, we have had 64-bit architecture, but it tended to be significantly more expensive than the best-priced performance commodity parts out there. So, a year or two years ago, dual processor [Intel] Xeon systems were the most common cluster compute node. The AMD Opteron processor are now cost competitive with 32-bit processors, and sometimes are even a better price performance choice.
One of the things that I am looking forward to seeing is 64-bit instruction set machines that people can address in much denser packaging. Today, the best packaging that we have is 32-bit processors. I think that we are going to see layer-based systems and densely packaged systems move to 64-bit processors.
What is your advice for IT managers who are considering consolidating servers and see clustering on Linux as an option?
Becker: The key element in selecting a system is long-term maintenance, being able to understand the system and to keep it updated, and keeping it consistent over a long period of time. To put clusters into operational use, as opposed to just experimental use, means that you have to start with a system that can be maintained. It also needs to be easy for end users to understand without them [having to] learn a whole new set of applications and a way to interact with the system. Having a single-system image approach for administration and for end users is the best way to minimize the new things that administrators and end users need to know.
What's developments in the Linux 2.6 kernel have piqued your interest?
Becker:What is going to change inside of Linux is largely about supporting new hardware devices. Some interesting work is being done with the user interface for Linux and the application level programs for Linux. I think that we are going to see a big transformation there. The developers can become a little bit more standardized on advanced, practical user interfaces. That is the hot area for Linux.
What developments in the Linux distribution area do you find interesting?
Becker:The thing that I have seen in the past year is that SuSE has become an equal player with Red Hat in the marketplace, especially when it comes to being the A86 64-bit instruction set distribution.
Several managers think that Debian is a good option as an enterprise Linux distribution. What do you think?
Becker: I really haven't seen Debian compete directly in that commercial marketplace. Debian tends to not be focused on naÏve end users, but more of a sophisticated system administrator. It is difficult to set up, especially during installation. Both Red Hat and SuSE distributions are much more oriented to people who don't know the answer to the questions yet.