Entertainment Asset Management EA Maximizes Server Power For Online Games

If you've never heard of The Sims, it's time to turn in your joystick. The "people simulation" series from Electronic Arts is the most popular gaming franchise ever, having shipped over 24 million units since its launch over three years ago.

Which is why game publishing giant EA had high hopes for its online version, The Sims Online, a persistent online world in which a subscriber can share their sim-ing experiences with hordes of others. Indeed, EA execs discussed reaping as many as one million active monthly subscribers globally, which would far outstrip the 470,000 subscribers to Sony's EverQuest, the most popular virtual world to date.

In anticipation of massive numbers of gamers putting a huge load on EA's computer resources, Mark Rizzo, the company's chief architect, worldwide technology services and operations, went shopping. On his shopping list were almost 1000 servers to be crammed into the smallest footprint possible to maximize the total cost of operation.

Until two years ago, EA had been "a Dell shop," according to Rizzo. Dell supplied all the servers for online gaming applications. And so, when it was determined in the final eight months of TSO's development exactly what sort of processing power EA required, Rizzo went to Dell to price the 2.4-GHz Xenon processors Intel had just released. "But Dell couldn't provide them in the price-competitive way we were seeking, so we started searching for other solutions," Rizzo noted.

One of the "other solutions" was San Jose, CA-based Rackable Systems, which specializes in large-scale, rack-mounted servers and storage.

"I'm afraid I have no wonderful story about how I got in there," recounted Rackable senior accounts manager Bryan Hanson. "Frankly, Mark had heard about us from a co-worker, and EA contacted us through a website request. We hadn't been pursuing their business. They dropped by, took a little tour, did some evaluations, and the deal was done."

"Not only were they able to give us the density and CPU power we'd been seeking," said EA's Rizzo, "but it became obvious that the Rackable people did a great job of designing their cases and that they understood our problem." That problem involved a data center where space was at a premium. "We determined in our dollar-for-dollar cost analysis that even if Rackable's servers cost the same as Dell's, if you added in the ongoing per-rack unit cost of space, there was a huge detriment on the Dell side."

According to Rackable's Hanson, the company uses a half-depth chassis that enables it to mount back-to-back servers. "In the amount of space it takes Dell to put in two CPUs, we can put in four," he explained. "We also have a proprietary thermal management system that allows us to exhaust the air at the top of our cabinet to cool the system."

EA reportedly evaluated several different memory configurations of Rackable's dual 2.4 GHz CPUs--from 2GB to 6GB--and eventually decided to purchase slightly more than 950 servers, 70 percent of which were 4GB units and 30 percent 2GB units. Each unit contained an Intel Westville motherboard, dual Intel 2.4-GHz CPUs, 2GB or 4GB configurations of DDR RAM, dual Seagate 40GB IDE disk drives, and 2-port 3Ware IDE RAID cards, all housed in a 2U chassis. Each system was installed in Rackable's custom 44U cabinets which can be fully populated at Rackable Systems and shipped to EA in a large crate. They are then rolled into the data center and merely have to be plugged in.

Once the order was placed, everything moved quickly. "EA first talked to us around late July," recalled Hanson, "and they said they needed to have beta testers playing the game live in late September (2002), with the game officially launching in late November. We didn't ship all our servers at once but over a period from September to November. As soon as they got them, they tested them, got them ready, and installed them--within 30-60 days of their receipt." Was Hanson surprised by the size of the order, Rackable's largest to-date? "They had planned on a ton of people subscribing to TSO," he said,, "and they wanted to make sure they had the horsepower in case the game became oversubscribed."

Rizzo admitted it's always a challenge to anticipate how many gamers will want to play any particular game. "Fortunately, we've done these games before and can base our projections on the previous history of past games, like Ultima Online.

The marketing people make the forecasts and we take those numbers, tweak them, throw in some overhead, and determine what computing power we'll need. We try to overbuild to the point where we know we'll be covered in the worst case scenario. Building too much isn't a problem; we can repurpose it later for other projects. Fortunately, if the game is wildly successful and goes over our numbers, we can also slow down shipments of the game into the retail channel--to the chagrin of the salespeople. If we see a train wreck coming, if we see ourselves running short of computing power, we can stall registrations."

But EA's "train wreck" was of a different sort. According to news reports, gamers' reactions to TSO were underwhelming to say the least. While the initial projection was for 400,000 subscribers by the end of this year, EA execs trimmed their expectations to 200,000 shortly after the game came out last year. The most recent report, in April, was that there were just 97,000 subscribers, and analysts say those numbers may have dropped since then. Game critics blamed the non-enthusiasm on the nature of the online version of the game; if a player wants their Sim to, say, take a half-hour nap to refresh itself, he needs to watch it snooze in tedious real time. Gamers found it excruciating.

Fortunately, EA has other irons in the fire that will easily absorb much of the excess computing power. One is its online Pogo environment, which allows gamers to download small games and compete for cash prizes.

"We're looking to migrate Pogo from a 100 percent Sun architecture to an Intel architecture," explained EA's Rizzo. "Because we overbuilt on TSO, we'll be repurposing some of that, enabling us to replace 500 servers. We may even have to buy more for Pogo. And, at the same time, we're adding another massive multiplayer game that, like TSO, will require lots of servers in a very small space."

That makes Rackable very happy.

"I can't predict how many more servers they'll buy from us," said Rackable's Hanson. "The TSO situation was extraordinary in terms of the quantity they bought."

But Rackable's president and CEO, Tom Barton, sees a growing relationship with EA--his biggest gaming customer--and with other game publishers. "We expect to add another one this quarter and possibly a second by year-end," he said. "I mean, if there's one thing we've learned from this experience, it's that massive multiplayer games are a huge opportunity. Who else buys servers a thousand at a time?"

Paul Hyman was most recently the editor-in-chief of CMP Media's GamePower.

Thomson Launches MAM Service

Consumer electronics and media services firm Thomson announced last month the launch of a media asset management (MAM) service under its Technicolor unit. The new service, powered by a content management system from IBM and its strategic partner Ancept, provides an integrated platform for the digital storage, management, access, processing, and distribution of all forms of entertainment content.

By creating a digital master file for content (including feature films, broadcast and cable programming, commercial advertising, studio theatrical trailers and visual effect shots) and storing all associated elements (audio tracks, formatting instructions, metadata and editorial decisions), Technicolor MAM is able to create, manage, and re-create all versions derived from a title with consistent quality. The system further allows customers to place processing and fulfillment orders online after reviewing information about the specific content they need.

Components of the Technicolor MAM service include: storage of creative elements in a central, common storage area network; an ingestion and retrieval system, based on IBM DB2 Content Manager middleware and the Ancept Media Server; an internal, secure, high-speed network enabling distribution of production assets to Technicolor facilities in North America, Europe and Asia; and access for authorized users to securely search, manage, view and listen to lower bit-rate proxies and make decisions that require both picture and sound from their desktops.

"This initiative also eases the transition of Technicolor's customers from a strictly physical world (using film, videotape, and legacy machinery) to a robust and secure data-centric environment closely integrated with Technicolor's traditional post-production capabilities, service offerings, and worldwide facilities," added Technicolor chairman Lanny Raimondo.