So the Oakland, Calif.,-based company did something that Whitehead never anticipated; it abandoned Linux for Sun's Solaris 10 operating system for many of its production systems.
Solaris 10 has 50 times the throughput
Real Time Matrix provides content delivery technology and aggregation services for websites. The company has developed a matching engine that processes millions of articles and blogs every day, sending customers snippets of content that match their preferences. But the ballooning information requirements had gotten unwieldy. To handle its ever-growing processing needs, Real Time Matrix needed to upgrade its hardware.
As part of Sun Microsystems Inc.'s hardware trial program, at the beginning of 2007 Real Matrix tested out a T1000 Server that arrived pre-loaded with the Solaris 10 operating system prior to making a purchase. Whitehead liked some features of the Sun box, including the advanced lights-out manager that enables Real Time Matrix to manage the server without the need to shut it down first. Solaris 10 features that impressed Whitehead were the Zones partitioning technology and ZFS file system. "We adopted Zones to create separate environments on the same machine," Whitehead said. "We also implemented ZFS to facilitate our programs that perform massive file caching."
But what really sold Real Time Matrix on Solaris 10 was the OS's throughput when compared with Linux. "We process up to 2 million new RSS and XML articles per day through our data service," Whitehead said. The company also processes another 15 million Web requests per month for other services. To determine the best operating system, Real Time conducted a head-to-head competition pitting the Fedora Linux distribution (which the company uses for production) against Solaris 10. And it turned out that Solaris 10 had a throughput that was 50 times better.
"On a 64-bit AMD processor and Fedora, we could process approximately 200 matches per second of RSS," Whitehead said. "With Solaris 10 on the T1000, this match rate jumped to 10,000 per second."Within a month of the trial, Real Time Matrix decided to purchase the T1000. In order to run Solaris 10, Whitehead said it was necessary to hire an administrator with hands-on Solaris experience, not an insignificant investment for a company with a handful of IT staff. Real Time Matrix outsources system administration to LinuxCertified Inc., a training and services firm. LinuxCertified "sourced and hired the Solaris expert and added that support to our plan," Whitehead said.
A mixed shop
After the head-to-head comparison, Real Time Matrix recompiled its software to run on Solaris, and the switch became official. "We were nervous about moving to Solaris," Whitehead said, an unease that gave way to pleasant surprise "at how easy Solaris was to use."
The move to Solaris 10 wasn't without some adjustments. "Developers needed to learn some new shell commands, Whitehead said. "They also needed to clean up some loose practices with security." On the programming side, Real Time Matrix has been able to leverage the Zettabyte File System, or ZFS to dramatically improve our file caching, and Whitehead has found that MySQL scales well on Solaris. "The benefits have been worth the learning curve," he said. "We invest most of the time [in] tuning and preventive maintenance."
Real Time Matrix has continued to expand the number of transactions processed daily. "In order to keep pace, we have moved all of our transaction processing to a [SunFire] T2000, and we have just purchased a 5220 [Sun StorageTek Network attached storage appliance]," Whitehead said. Currently, Real Time Matrix uses one T1000, a pair of T200s, and one 5220 for all the company's data processing, which includes its custom matching service and re-broadcasting engine, according to Whitehead.
Real Time Matrix still runs Linux on standalone boxes. The Solaris boxes integrate with the company's front-end Web servers – a Linux cluster on Advanced Micro Devices (AMD)-based machines. Real Time Matrix also has a Linux-based storage area network and a database running MySQL. Up next, Whitehead anticipates moving the company's Linux Tomcat cluster from separate AMD machines to a zoned T2000. So while Whitehead never would have anticipated that he would run a mixed shop, he's convinced that it was the right choice for his company.