Reducing Big Energy Cost for Big Data

This post has already been read 2390 times!
0 Flares Twitter 0 Facebook 0 0 Flares ×

If you or your CFO is concerned about the high energy costs of your Big Data Center, the following article should catch your attention.

The age of Big Data has brought along with it the age of big energy bills for companies and server farm operators. Indeed, cost concerns are always a motivation for finding solutions. The following article is a reprint from the Life Boat Foundation, a nonprofit organization set up to help provide information that supports technical innovation.

The new technology discussed in the following article can become an important factor in helping to bring down the high energy cost of big data. According to sources at MIT, Flash memory — the type of memory used by most portable devices — could provide an alternative to conventional RAM for big-data applications. It’s about a tenth as expensive, and it consumes about a tenth as much power. (www.mit.edu)

 

Cheap, power-efficient flash memory for big data without sacrificing speed

The following article is a reprint from the Lifeboat Foundation, August 6, 2015

Reducing Big Energy Cost for Big Data

A 20-node BlueDBM Cluster (credit: Sang-Woo Jun et al./ISCA 2015)

There’s a big problem with big data: the huge RAM memory required. Now MIT researchers have developed a new system called “BlueDBM” that should make servers using flash memory as efficient as those using conventional RAM for several common big-data applications, while preserving their power and cost savings.

Here’s the context: Data sets in areas such as genomics, geological data, and daily twitter feeds can be as large as 5TB to 20 TB. Complex data queries in such data sets require high-speed random-access memory (RAM). But that would require a huge cluster with up to 100 servers, each with 128GB to 256GBs of DRAM (dynamic random access memory).

Flash memory (used in smart phones and other portable devices) could provide an alternative to conventional RAM for such applications. It’s about a tenth as expensive, and it consumes about a tenth as much power. The problem: it’s also a tenth as fast.

But at the International Symposium on Computer Architecture in June, the MIT researchers, with colleagues at Quanta Computer, presented experimental evidence showing that if conventional servers executing a distributed computation have to go to disk for data even 5 percent of the time, their performance falls to a level that’s comparable with flash anyway.

In fact, they found that for a 10.5-terabyte computation, just 20 servers with 20 terabytes’ worth of flash memory each could do as well as 40 servers with 10 terabytes’ worth of RAM, and could consume only a fraction as much power. This was even without the researchers’ new techniques for accelerating data retrieval from flash memory.

“This is not a replacement for DRAM [dynamic RAM] or anything like that,” says Arvind, the Johnson Professor of Computer Science and Engineering at MIT, whose group performed the new work. “But there may be many applications that can take advantage of this new style of architecture. Which companies recognize — everybody’s experimenting with different aspects of flash. We’re just trying to establish another point in the design space.”

 

Additional Reading

Big Data Analytics for Inclusive Growth

Big Data Adds Real Value for the Oil and Gas Industry

If you liked this article, we'll be happy to send you one email a month to let you know the newest edition of the MetaOps/MetaExperts MegEzine has been published. Just fill the form below.

Related Post