Learning Guide

Linux clusters: High-performance computing

  

Linux clusters are not a one-trick pony, so don't dismiss them as scientific lab tools only. This SearchEnterpriseLinux.com guide explores the many uses of Linux clusters in the enterprise. Find out why Linux clusters can be your company's best

    Requires Free Membership to View

bet for handling storage, applications, high-volume transactions and more. Take a spin though these articles, tutorials, tips, tools, white papers and expert Q&As. Drop me us an e-mail to let me know what other learning guides you'd like to see on SearchEnterpriseLinux.com.

 

TABLE OF CONTENTS
   Basics
   High-performance computing clusters
   Application clusters
   Storage clusters
   Load-balancing clusters
   High availability clusters
   Clustering on blade servers
   Cluster configuration, management and administration
   Virtualization and clusters
  Other open source cluster projects

 

 

Clustering basics

Return to Table of Contents

 

Glossary (SearchEnterpriseLinux.com, powered by Whatis.com)

Other basic resources

 

High-performance computing clusters

Return to Table of Contents

 

General information

  • Cluster how-to
    How to set up a high performance Linux cluster.

  • Linux Clusters Institute
    The LCI site provides links to education and advanced technical training for the deployment and use of Linux-based computing clusters to the high-performance computing community worldwide.

  • Article:  Driving Linux cluster performance (Bio-IT World)
    Planning early and choosing the right tools can boost performance by 27%, according to experts cited here.

  • Linux HPC cluster how-to
    This is a specific description of how a lab's cluster is set up. It details the data center aspects and the desktop, laptop and public server aspects of building a cluster.

  • Article: The Linux Business Case: Clusters (CIO Information Network)
    This article discusses the characteristics of high-performance applications that make them a good fit for Linux clusters.

  • Q&A:  Tips for maximizing HPC cluster productivity (HPC Wire)
    Tom Quinn, Director of Government Business Development at Linux Networx, provides tips on maximizing cluster productivity.

  • Build a heterogeneous cluster with coLinux and openMosix (IBM.com)
    An experimental case study combines coLinux with an openMosix-enabled kernel to build a hybrid cluster.

  • SGI Partners to Launch 'Bright' Linux Clusters (IT Jungle)
    Clusters that behave like, are preconfigured like, and sold like monolithic systems are called "bright clusters," and supercomputer maker Silicon Graphics is partnering to target this fast-growing market with its Linux-based Altix machines.

  • Article: Dell inks HPC cluster deal with Cray
    This story describes how Dell plans to bolster its HPC Linux cluster business by partnering with Cray Inc., the Seattle-based supercomputer company.

Beowulf

  • Beowulf.org
    Project page for the open source project. Beowulf.org is a collection of resources for the expanding universe of users and designers of Beowulf class cluster computers. These enterprise systems are built on commodity hardware deploying Linux OS and open source software.

  • Beowulf cluster FAQ (Aspen Systems)

  • Beowulf how-to
    This document describes step by step instructions on building a Beowulf cluster.

  • Q&A: Beowulf founder: Linux holds its own in high-performance computing
    Beowulf founder Don Becker reflects on the legacy of Beowulf and its future in this interview. He also takes a hard look at Cray Canada CTO Paul Terry's claims that Linux clusters can't hold their own in HPC situations.

  • Article: Scyld's new cluster has Linux distro, Ganglia inside (SearchEnterpriseLinux.com)
    Scyld Software has added performance monitoring capabilities and hardware and customer support in the newest version of its Scyld Beowulf Series 29.

Scientific applications

Whether or not to use HPC clusters

  • Q&A: Linux clusters for HPC – Why and why not?
    What's keeping high-performance computing users away from Linux clusters? Should they reconsider?

  • Expert advice: What are the advantages of moving to a Linux HPC cluster? (Ken Milberg, SearchEnterpriseLinux.com expert)

  • Q&A: Cray CTO: Supercomputers outshine Linux clusters in HPC, Part 1 (SearchEnterpriseLinux.com)
    Linux clusters can not offer the same price-performance as supercomputers, according to Paul Terry, chief technology officer of Burnaby, British Columbia-based Cray Canada. In this interview, Terry explains that assertion and describes Cray's new Linux-based XD1 system, which will be priced competitively with other types of high-end Linux clusters.

  • Q&A: Cray Canada CTO answers IT pros' HPC questions (SearchEnterpriseLinux.com)
    Is high-performance computing only used in scientific labs? In this interview, Cray Canada chief technology officer Paul Terry answers this question, offers advice on how to choose the right HPC system responds to questions from SearchEnterpriseLinux.com's readers.

  • Q&A: Linux clusters give HPC price-performance (SearchEnterpriseLinux.com)
    High-performance computing (HPC) users are switching from supercomputers to Linux clusters, and enterprise IT shops are getting keen on that idea, too. In this interview, PathScale exec Alex Rublowsky explained why HPC on Linux makes sense. Also, find out which types of enterprise IT shops are checking out and what's ahead for HPC Linux clusters.

  • Linux clusters can have their day vs. supercomputers (SearchEnterpriseLinux.com)
    Linux Networx marketing executive Eric Pitcher makes the case for Linux clusters against supercomputers.

  • Filling in the gaps among Linux clusters (SearchEnterpriseLinux.com)
    Is the price right for a Linux cluster? Would it meet all of your company's needs? Get the scoop in part two of this interview with Linux Networx executive Eric Pitcher.

 

Application clusters

Return to Table of Contents

 

 

Storage clusters

Return to Table of Contents

 

 

Load-balancing clusters

Return to Table of Contents

 

  • Home page: Linux Virtual Server Project
    The Linux Virtual Server is a highly scalable and highly available server built on a cluster of real servers, with the load balancer running on the Linux operating system. The architecture of the server cluster is fully transparent to end users, and the users interact as if it were a single high-performance virtual server.

  • Home page: Ultra Monkey
    Ultra Monkey is a project to create load-balanced and highly available network services, such as a cluster of Web servers that appears as a single Web server to end users.

  • White paper: Craft a load-balancing cluster with ClusterKnoppix (IBM)
    Using Knoppix-based LiveCDs, build your own supercomputing Linux cluster.
  • Article: Load balancing gives Web traffic a green light (SearchCIO.com)

  • Resource: Load balancing Linux clusters (LCIC.org)

 

High availability clusters

Return to Table of Contents

 

  • Home page: High-Availability Linux Project (Linux-ha.org)
    The basic goal of the High Availability Linux project is to provide a high-availability (clustering) solution for Linux which promotes reliability, availability and serviceability through a community development effort.

  • White Paper: Introduction to clustering on IBM eServer OpenPower 710 (IBM)
    This article first introduces some general clustering concepts and software and then provides examples of how the OpenPower 710 can be used in the two most common types of clusters: high performance clusters and high availability clusters.

  • Home page:  Kimberlite HA cluster project site
    The collaboration site for Kimberlite cluster development. Kimberlite is an open source project for high availability clustering on Linux. The page offers links to documentation and more information.
  • Resource: SGI Linux FailSafe FAQ (SGI)
    Linux FailSafe provides a clustered HA application platform in software.

 

 

Clustering on blade servers

Return to Table of Contents


 

Cluster configuration, management and administration

Return to Table of Contents

 

  • Q&A: Beowulf founder: Virtualization hot on Linux clusters
    Virtualization is creating a buzz in the IT industry today, but that buzz fizzles when virtualization on Linux clusters is mentioned. That's a shame, says Beowulf Project co-founder Donald Becker, because clusters offer a no-clutter virtualization option.

  • White paper: What is CSM? (IBM)
    Find out more about Cluster Systems Management (CSM) software, which provides a distributed system management solution that allows a system administrator to set up and maintain a cluster of nodes that run the Linux operating system.

  • Home page: System Imager
    This is the open source project site for SystemImager, software that automates Linux installs, software distribution, and production deployment for Internet server farms, database server farms, high performance clusters, computer labs, and corporate desktop environments.

  • Article: Scali releases new products for multi-platform clusters (Linux HPC.org)
    This is an update on Scali's high performance clustering management product, called Scali Manage.

  • Home page: Maui Scheduler
    The Maui Scheduler helps admins manage large-system resources. Maui was designed to batch schedule and launch MPI-based parallel jobs, and can also schedule and run your own custom message-passing jobs.

  • Home page: The Cluster Command and Control (C3) tool suite
    This is the project page for The Cluster Command and Control (C3) tool suite, which implements a number of command line based tools that have been shown to increase system manager scalability by reducing time and effort to operate and manage the cluster.

  • Tip: Distributed computing, clusters and high-availability solutions
    In this tip, clusters and servers expert Don Becker describes basic rules for making network boot servers more reliable -- like avoiding multicasts -- and explains the Mosix approach to cluster architecture. He also suggests ways to avoid data loss when consolidating servers.

  • Expert Response:  Consolidation and common sense on the server side
    A Unix-to-Linux expert describes educational resources about distributed computing, clusters and high-availability solutions for a user who is considering a move to Linux from Unix.

  • Tip: Mastering MySQL: Drivers, clustering and remote access
    MySQL expert Scott Noyes describes the tricky aspects of clustering with MySQL and upgrading from MySQL 4.0 to MySQL 5.0.

  • Tip: Setting up MySQL replication for Linux server clusters
    MySQL facilitates Linux server clustering for load balancing and fault tolerance. Create a master/slave configuration with full data replication using the commands in this tip.

 

Virtualization and clusters

Return to Table of Contents

 

 

Other open source cluster projects

Return to Table of Contents

 

  • Home page: OpenSSI (Single System Image) Clusters for Linux
    The OpenSSI project is a comprehensive clustering solution offering a full, highly available SSI environment for Linux.

  • Home page: Cluster infrastructure for Linux
    This project is developing a common infrastructure for Linux clustering by extending the Cluster Membership and Internode Communication Subsystems from HP's NonStop Clusters for Unixware code base.

  • Resource: Cluster Project Page
    This page provides links to a number of Linux cluster development projects.

  • Home page: Open Cluster Group
    The Open Cluster Group is an informal group of people dedicated to making cluster-computing practical for high performance computing.

  • Home page: OSCAR: Open Source Cluster Application Resources
    OSCAR is an open source project for the development of a fully integrated, easy to install, small-to-midsized high performance clustering solution.

 

This was first published in February 2008

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: