Over the past year, NYSE has implemented some 200 new rack-based HP ProLiant DL585 servers, about 400 ProLiant
BL685c server blades and several HP Integrity NonStop fault-tolerant servers for its three East Coast data centers.In a rare glimpse into a mission-critical Wall Street IT operation, the New York Stock Exchange (NYSE) has confirmed that it recently purchased roughly 600 Hewlett-Packard Co. servers in support of its Linux-based online trading system.
NYSE introduced the electronic trading system, the NYSE Hybrid Market, in October 2006 to allow investors to buy and sell stocks on the trading room floor and via the Internet. The NYSE Hybrid Market now handles more than 500 million messages a day, requiring more servers and storage, said Steve Rubinow, the chief information officer at NYSE.
To accommodate the online trading system, NYSE commissioned HP to replace old server and storage infrastructure.
All the blade servers and the ProLiant DL585s are based on Advanced Micro Devices Inc. (AMD) dual-core processors, which are upgradeable to quad-core.The decision-making process
Back in May, NYSE also began migrating off a 1,600 million-instructions-per-second (MIPS) mainframe to IBM System p servers running AIX.
Before settling on HP, Rubinow considered Sun Microsystems Inc. and IBM Corp. But he went with HP because it offered the best hardware options and service, he said.
With the new HP servers, the average trade-execution turnaround time has been reduced from seconds to milliseconds. The Hybrid Market now allows up to 1 million shares to be traded in a single order – up from 1,099 – thanks to the new server implementations and faster interconnects, said Paul Miller, HP's VP of server and storage marketing.
The HP servers connect to 200 terabytes of HP StorageWorks XP12000 storage arrays, and the servers and storage are managed with HP's OpenView management tool, Miller said.Free OS handles billions in stock trades
NYSE's central applications include the trading systems that match buyers and sellers, transaction reporting systems, and regulatory surveillance systems, most of which run on Linux.
In addition to running Linux on its x86 boxes, for the past 30 years NYSE has been an HP NonStop server user which runs the proprietary NonStop Kernel OS and also uses Sun Microsystems' Solaris for legacy applications.
"We have thousands of servers, and we are trying to architect our applications to make them more efficient, which starts with the software," Rubinow said. "If you have inefficient software, you end up throwing more hardware at it."
NYSE uses Linux because it's a more flexible and cost-effective operating system than most other options, according to Rubinow.
"We favor Linux for what we do. We don't want to be beholden to any one [hardware or software] supplier, even if it is very good. We want the freedom to be vendor-independent, so Linux was a good choice," said Rubinow.
Solaris for x86 is also used because it can run on various servers, including HP's, he said.Open source no problem
NYSE adheres to the U.S. Securities and Exchange Commission (SEC)-governed Regulation National Markets System and SEC rules, so using an open source OS, with its perceived licensing and security risks, may be surprising. Rubinow isn't worried, though.
"There is always concern about open source, but those concerns are really minor," Rubinow said. "Up until we have a bad experience, it will be fine. You really never know. Some issue could come out of the woodwork. … But we are regulated by the SEC. If they had any issues with Linux operating systems, they would have certainly voiced them."
NYSE also uses a limited amount of virtualization in its quality assurance and testing regimen, but that's about it, Rubinow said.
"Virtualization is not free, and it uses up resources. It also has some latency that is not acceptable in trading. People notice that tiny slowdown," Rubinow said. "We have only a handful of underutilized servers, so virtualization could save us some money and hardware, but it isn't high up on our list."Big Apple or bust
Adding server capacity to NYSE's three East Coast data centers has its challenges, not least of which is cost, with New York City topping the list of the most expensive places to operate a data center in the U.S., according to Princeton, N.J.-based site selection specialists, the Boyd Company Inc.
In addition to the three on the East Coast, NYSE has another data center in the Midwest and two other "tiny" data centers in other parts of the country. NYSE is on the hunt for another data center site to accommodate growth, also on the East Coast, Rubinow said.
Because operating data centers in the eastern U.S. is costly, Rubinow and his team work with manufacturers to ensure the hardware choices fit within the limits of the data center power availability. For example, when densely packed, blade servers are known to generate excessive heat, which can create cooling issues. To avoid this, Rubinow obtained heat profile projections to determine how many blades can be packed into each rack and in which data center to place the servers.
That said, being close to Wall Street, and the majority of NYSE customers, is worth the cost, he said.
"We are being as creative as we can be with the space we have now, but we are looking for a new space, which will be near our highest customer area," Rubinow said. "Being close to our customers is more important to us than finding a low-cost place to operate. Otherwise, we'd be following Google around."
Let us know what you think about the story; email Bridget Botelho, News Writer.
Also, check out our news blog at serverspecs.blogs.techtarget.com.