When you're in the business of providing customer service for other businesses, you don't want to be caught with your plants down. So, continuing CPU failures and maintenance-related server downtime at customer service centers posed a costly headache for $295 million Precision Response Corp. (PRC), a global service provider for such major corporations as American Express, AT&T, British Airways, DirecTV, FedEx, and Priceline.com.
"We run a 24/7 workshop," said Gopal Devalaraju, PRC's senior director of application technology services. "These problems had a direct impact on our customer service reps and our customers, impacting revenue for our organization."
In this report, Devalaraju describes PRC's journey from the downtime dumps to an always-on
For seven years, PRC ran its outsourced customer service operations on more than a dozen "big box," proprietary Unix-based symmetric multiprocessor (SMP) servers and islands of Oracle databases, according to Devalaraju. Computer resource utilization was uneven because the servers, applications and databases were siloed along customer lines. The system evolved into islands of computing environments because of PRC's rapid growth and its widely dispersed locations: 25 call centers in the United States, India and the Philippines. PRC employs more than 8,000 people in the United States alone.
Tired of CPU failures and the need for scheduled downtime for database updates, PRC's IT team began looking at ways to increase utilization, provide higher availability for customers, and scale seamlessly. At the same time, the company wanted lower upfront costs and total cost of ownership, higher availability and easier system management.
After determining that clustering would reap the benefits on PRC's wish list, the IT team tested its applications on Windows and Linux clusters. Unix was ruled out before tests began because it's proprietary and because of hardware costs, Devalaraju said.
During the testing, the system that came out ahead of the pack was Linux, running on commodity Dell severs and an Oracle 9i database with Real Application Clusters (Oracle 9i RAC). The Linux price was right -- because of lower licensing fees, primarily -- and using Linux would also require less IT training, because of the staff's in-depth knowledge of Unix. Also, the IT staff was overwhelmingly in favor of moving to an open source operating system.
"We have used other open source software," said Devalaraju. "For building CTI interfaces, we knew that open source software would be easier to work with."
Even so, Linux wasn't chosen without some qualms. Service and support for Unix is more comprehensive than for Linux. "Integration with other products is still evolving," Devalaraju said.
That decision made, the team stress-tested PRC's application environment on Oracle 9i RAC running on Dell-Intel-Red Hat Linux servers with an EMC storage solution. Vendors provided support teams, while PRC had one full-time Linux administrator, a database administrator working on the project. Meanwhile, two systems administrators trained on Linux.
Deployment day went off without a hitch, but support issues with vendor and kernel problems came up in the next couple of weeks.
"We ran into an issue with a tainted kernel that took a while to get resolution," Devalaraju said. "If we had known earlier, regarding issues and bugs with the OS and hardware, we could have avoided some glitches that we encountered."
Vendor support issues with some application vendors were quickly resolved. Also, PRC is currently paying an annual support fee of $2,499 to Red Hat.
After the initial glitches were overcome, Oracle 9i RAC on Linux ran like a dream, Devalaraju said. Just to make sure the dream was true, a consulting firm -- Mainstay Partners LLC, of Redwood City, Calif. -- was called in to test the system. Mainstay gave the system a thumbs-up in the financial, operational and customer-satisfaction areas.
Comparing the costs of PRC's new environment with a comparable proprietary Unix system, Mainstay found that PRC will spend about $210,000 less with Linux hardware and software than with Unix in the first year. That's 37% in one year, and about 22% over five years.
The migration was a big success in reducing costs and providing a better and more robust infrastructure to customers "without compromising the quality of the product," said Devalaraju.
More Linux projects may be in store for PRC. "We are still using Unix and Windows in our back office," Devalaraju said. "Maybe next we would move more onto Linux."
FEEDBACK: Have you realized similar benefits from an Oracle-on-Linux combination? We want to hear your story.
Send your feedback to the SearchEnterpriseLinux.com news team.