Google Compute Engine: For $2 million/day, your company can run the third fastest supercomputer in the world

At the Google I/O conference in San Francisco, Google has announced the immediate availability of Compute Engine, an infrastructure-as-a-service (IAAS) product that directly competes with Amazon EC2 and Microsoft Azure. Citing more than a decade of running and optimizing its own data centers and network infrastructure, Google is claiming that the Compute Engine is more scalable, more stable, and cheaper than the competition.

For this story, we’ll focus on scalability and cost (I’m sure that Compute Engine is stable, but Google just hasn’t given us any figures to work with). Google says that Compute Engine has access to 770,000 cores — a figure that will surely grow over time. In one demo at Google I/O, a genomics app (it analyzed the human genome) was shown to use 600,000 cores. These cores are made available as Linux virtual machines (VMs), with 1, 2, 4, or 8 cores each. Each core apparently has access to 3.75GB of RAM each — and, of course, each VM is connected together using Google’s advanced networking technologies and topologies.

777,000 cores, assuming the entire Compute Engine cluster consists of 8-core CPUs, equates to 96,250 computers. This is a huge number — probably equal to the total number of servers operated by Intel, or data centers such as The Planet or Rackspace, but considering Google is estimated to have more than 1 million servers in total, it’s not that huge. Amazon EC2, by comparison, is estimated to have around 450,000 physical machines. Still, almost 100,000 servers on your opening day is rather impressive; Google is obviously starting as it means to go on.

For an interesting reference point, Sequoia, the world’s fastest supercomputer, has 1.78 million cores (clocking in 16 petaflops). Number two, the K computer, has 700,000 cores (10.5 petaflops). During that Google I/O demo, by using 600,000 cores, that genomics app was probably hitting the Compute Engine for somewhere between 1 and 10 petaflops of computational power — between 1 and 10 quadrillion calculations per second. For a few moments, the Compute Engine was probably the third fastest computer in the world.

But at what cost? The Google Compute Engine website says that an 8-core VM with 30GB of RAM costs $1.16 per hour. For 600,000 cores, you need 75,000 VMs — so, $87,000 per hour, or $2 million per day (and that’s before bandwidth and I/O costs). Even so, that price is actually quite reasonable, when you consider that supercomputers generally cost tens of millions to install and millions per year in running costs — well, as long as you don’t need the Compute Engine for more than a few days per year…

Finally, compared to “other leading cloud providers,” Google is claiming that Compute Engine gives you 50% more bang for your buck — though the company didn’t go as far as calling Amazon or Microsoft out. A quick glance at the Amazon EC2 pricing page suggests that Google’s $1.16/hour is actually more expensive, but perhaps it isn’t an apples-apples comparison.

Read more about the size of the cloud — or the size of porn sites

Source

Larrabee lives! 50-core Intel Xeon Phi unveiled, lures supercomputers away from Nvidia Tesla

Intel has finally unveiled Xeon Phi (codenamed Knights Corner), a range of more-than-50-core 22nm coprocessors built with the new Many Integrated Core (MIC) architecture. According to industry sources, Xeon Phi made such an impression at last week’s International Supercomputing Conference (ISC) that it has stolen numerous upcoming 100-petaflops supercomputer installations away from Nvidia’s Tesla coprocessor.

Xeon Phi is the end result of a project that began with the Larrabee architecture in 2006. Larrabee wasinitially meant to be a GCGPU, much like Nvidia’s Fermi and Kepler cores, but based on the x86 instruction set. Larrabee was eventually scrapped in 2010, but Many Integrated Core emerged from its ashes — and this time, MIC would simply be a high-performance computing accelerator. The graphics guts were stripped out, and all that remained were 50+ Pentium 1 (P54C) cores, with the addition of some juicy floating-point and vector processors. Intel has confirmed that each MIC core, like Larrabee, has a monstrous 16-wide ALU capable of 512-bit SIMD.

These Xeon Phi coprocessors (which come in a PCIe add-in card form factor) will be available in a few flavors, probably starting at 50 cores and with 8-16GB of GDDR5 RAM. Intel is targeting real-world performance of 1 teraflops per coprocessor, which is well above the Tesla M2090 (a Fermi-based card) and AMD’s HD 7970. The key difference, though, is that Xeon Phi uses the mature and very-well-understood x86 architecture, and is supported by Intel’s best-in-class compiler toolchain. Nvidia’s Kepler-based Tesla cards might be faster than 1 teraflops — but that’s theoretical performance. The fact of the matter is that writing and compiling software to effectively use hundreds of CUDA cores is incredibly hard.

Knights Corner/Xeon Phi die. Count the cores!

And therein lies the crux: Xeon Phi might not have the edge on raw performance, but it’s infinitely easier to deploy. The vast majority of current HPC installations use Intel or AMD x86 chips and software. Moving to CUDA or OpenCL is hard, expensive, and time-consuming work. According to VR-Zone, Xeon Phi is apparently so desirable that it has replaced Tesla as the coprocessor of choice in many upcoming 100-petaflops supercomputers, due for completion in the next few years. The same sources told VR-Zone that porting code to Intel’s MIC architecture took days; while Nvidia’s CUDA took months.

A 100-petaflops x86 supercomputer would have somewhere in the region of 80,000 Ivy Bridge Xeons and 80,000 MIC Xeon Phis, for a total of around 5 million x86 processor cores — and, assuming the Phi has a TDP of around 200 watts, a power draw of 25 megawatts. Compared to the current world’s fastest computer, IBM Blue Gene/Q Sequoia, which draws 7.9 megawatts at 16 petaflops, a Xeon Phi installation would be incredibly power efficient.

The Intel Xeon Phi is expected to be commercially available towards the end of the year. Technically, you’ll be able to plug one into your desktop’s PCIe slot — but it will be incredibly expensive, and probably not worthwhile unless you’re into modeling nuclear explosions at home.

Source

Canonical explains decision to ditch GRUB 2 on UEFI systems

Microsoft’s insistence that Windows 8 machines ship with Secure Boot technology in the Unified Extensible Firmware Interface (UEFI), a specification that essentially replaces and overcomes the limitations of the traditional BIOS, has Linux developers scrambling to ensure their distros will work on tomorrow’s systems. Secure Boot isn’t a bad thing, or at least it isn’t intended to be. Rather, Secure Boot technology allows Windows 8 systems to detect rootkits and other harmful software at the outset, preventing them from ever being loaded in the first place.

Embedded into the UEFI firmware is a security key, and the OS must know the key in order to boot. Where this becomes a problem is when users want to dual-boot Linux with Windows on a Windows 8 machine.

Red Hat earlier this month announced plans to pay Verisign a $99 fee to sign its software, which will allow it to sign an unlimited number of binaries and ensure that Fedora 18, which is scheduled to ship around the same time as Windows 8, will have no problem booting with the new Secure Boot system in place.

Quantal Quetzal

Canonical, the company that develops the popular Ubuntu distro, doesn’t want to take this same approach, and instead wants to use its own signing key. The good news is, Canonical has already “generated an Ubuntu key” and is “in active discussions with partners to implement simple ways for enterprises and consumers to use this key.” Ready for the bad news?

Future versions of Ubuntu, starting with Ubuntu 12.10, will not use GRUB 2 by default on systems with Secure Boot enabled, which amounts to all Windows 8 systems. In its place, Ubuntu will use a modified version of Intel’s efilinux loader. Wondering why Canonical doesn’t just license a relatively inexpensive key via Verisign like Fedora? Canonical explains it like this:

Ubuntu has a rather extensive base of preinstalled systems. Microsoft’s Windows 8 logo requirements do say that there must be a way for users to disable secure boot or to install their own keys, and we strongly support this in our own firmware guidelines; but in the event that a manufacturer makes a mistake and delivers a locked-down system with a GRUB 2 image signed by the Ubuntu key, we have not been able to find legal guidance that we wouldn’t then be required by the terms of the GPLv3 to disclose our private key in order that users can install a modified boot loader. At that point our certificates would of course be revoked and everyone would be worse off.

And therein lies the real issue. Canonical’s decision is more legal in nature than technical, and the company doesn’t want to end up in a situation where it’s forced to fork over its private key.

It remains to be seen how other major Linux players will cope with Microsoft’s Secure Boot requirements. In the meantime, you can read plenty of technical details regarding Canonical’s decision in a post on the Ubuntu development mail list.

Source

US retakes supercomputing crown with 16-petaflops Sequoia; China promises 100 petaflops by 2015

Almost 2,500 high-performance computing boffins have descended upon the German city of Hamburg to attend the International Supercomputing Conference 2012 (ISC) this week to discuss the latest and greatest innovations and applications in the realm of supercomputing. The biggest news of ISC, by far, is that the USA has beaten back the Chinese and Japanese to reclaim pole position on the 39th Top500 list — the list of the world’s fastest supercomputers. In retaliation, China is now promising to deliver a 100-petaflops by 2015 — some two years before the rest of the world is expected to reach such lofty computational heights.

The world’s fastest supercomputer is now Sequoia, an IBM Blue Gene/Q installation at the Lawrence Livermore National Laboratory in California, primarily for use by the National Nuclear Security Administration. The system’s Linpack benchmark performance is 16.32 petaflops (16 quadrillion floating point operations per second), some 55% faster than the Japanese K, the previous record holder. The Sequoia’s theoretical max performance is just over 20 petaflops. Sequoia will be used to simulate the safety and efficacy of the US nuclear weapons stockpile, without the need for actual underground testing — and other scientific research.

Under the hood, Sequoia boasts 98,304 18-core processors and 1.6 petabytes of RAM, spread out amongst 96 racks that occupy a total floorspace of 3,000 square feet (280 square meters). Compute Node Linux powers around 97,500 of the processors, and Red Hat Enterprise Linux runs on 768 I/O (filesystem) nodes. The Blue Gene/Q chip itself is a terrifying 1.6GHz 64-bit PowerPC A2, which is 4-way simultaneously multithreaded, with each core harboring a SIMD quad-vector double precision floating point unit. There are 18 cores in total, but one core is a backup in case another is permanently damaged, and another handles interrupts, asynchronous I/O, and other operating system helper functions. The chip is theoretically capable of 205 gigaflops while drawing 55 watts — just a wee bit more efficient than desktop PC chips.

The chip’s frugal power consumption means that Sequoia uses only 7.9 megawatts of power while running Linpack, or around two gigaflops per watt. This is enough to put Sequoia at the top of the Green500 list of most efficient supercomputers. In comparison,the K supercomputer uses 12.6 megawatts to perform 10.5 petaflops — or only 800 megaflops per watt. An Ivy Bridge Core i7-3770K achieves around 50 gigaflops while consuming around 100 watts — or 500 megaflops per watt.

Ni hao

Back in 2010, the 2.5-petaflops Tianhe-1A became the first Chinese installation to reach the top of the Top500, knocking the 1.7-petaflops Cray XT5 Jaguar supercomputer at the US Oak Ridge National Laboratory down into second place. This in itself was enough to get US alarm bells ringing — the US has basically been the undisputed king of supercomputing, except for a short period in the ’90s — and then Japan turned on the K computer at the RIKEN institute. K was four times faster than Tianhe and 6 times faster than Jaguar. The US has been in second place a few times throughout the years, but third place? Unthinkable!

Judging by the rumblings being made by the director of China’s National Supercomputing Center, though, perhaps America should get used to this being a three horse race. Speaking to The Register, Liu Guangming says that China will power up a 100-petaflops supercomputer sometime in 2015. This is rather impressive, as previous estimates have centered around the 2017-2018 mark.

Guangming says his team is currently evaluating what architecture the supercomputer will use. It could use Intel Xeons (like the Tianhe-1A), AMD Opterons, or its own homegrown CPUs. As we’ve reported previously, China has already built a 1-petaflops computer from its domestic 16-core 64-bit ShenWei CPUs. The Chinese also have to decide on whether the 100-petaflops computer will be a CPU-GPU hybrid (like Tianhe, which uses 7,168 Nvidia Tesla GPUs) or a straight-up CPU-powered beast. Again, Guangming says that they might use homegrown GPUs — though he didn’t share any details on their specs.

Back at home, the next big American supercomputer will be the Cray XK6 Titan, Jaguar’s successor. Titan, built out of AMD Opterons and Nvidia Kepler GPUs, should have a peak performance of 24 petaflops. You can see on Top500 that the upgrade process has already begun!

In other news, IBM’s SuperMUC, installed in Munich, Germany took fourth place on the Top500 and became the fastest supercomputer in Europe. SuperMUC is novel because it uses a hot water cooling system to reduce its power consumption by 40%.

Read about the history of supercomputers, or what supercomputers are actually used for

Source

Japanese team boosts NAND flash durability and performance with ReRAM buffer

A group of Japanese researchers led by Ken Takeuchi, a professor at Chuo University, has announced a hybrid SSD architecture that combines a ReRAM buffer with standard NAND flash. The resulting hybrid offers significantly improved performance and longevity, and could be a model for manufacturers trying to overcome some of flash’s scaling problems.

At present, long-term NAND scaling faces a number of problems. Flash built on smaller process nodes is less durable. Attempts to move from MLC (two bits per cell) to TLC (three bits per cell) have been slow thanks to the drastically reduced number of write cycles TLC flash can perform; QLC flash (four bits per cell) is wishful thinking. In theory, a ReRAM hybrid could solve a number of these problems.

ReRAM (Resistive Random-Access Memory) is one of a number of technologies under development by companies looking for next-generation storage mediums to replace NAND flash. In this context, its chief advantage over flash is that it’s both more durable and requires far less energy per write.

The research team proposes a hybrid drive that combines 256GB of NAND with ~1GB of ReRAM for both cache and storage. Instead of writing directly to the NAND, drive writes would be cached within ReRAM until a certain threshold was reached. These would then be written in bulk, thereby lowering data fragmentation and drastically reducing the number of write cycles the NAND flash performs.

Data is buffered and handled according to three algorithms. First, an anti-fragmentation filter ensures that writes are stored until they reach at least 60% of 16K depending on the ReRAM’s capacity. Second, data that’s overwritten in NAND is copied back to the buffer (to prevent fragmentation and additional re-write cycles). Finally, the buffer keeps a record of the most recently accessed addresses. If data is being written and re-written rapidly, the updates occur in ReRAM, not the NAND.

I’m normally extremely dubious of watershed innovations in technology, but one of the most striking features of Takeuchi’s proposal is that the amount of ReRAM required is quite small compared to the storage capacity of the drive. That’s important when considering the impact of economies of scale. Manufacturers who are currently building out NAND fab capacity aren’t going to fall all over themselves to retool for ReRAM (or anything else) unless they absolutely have to. Incorporating a small amount of ReRAM around high-capacity NAND makes much more sense and preserves the value of the current investment.

Even better, there’s the chance that the ReRAM buffer’s cost could be offset by the use of TLC flash at some point in the future. The researchers designed their prototype by assuming the use of TSVs (through-silicon vias) but believe the design could be adapted to other approaches currently in mass production. TSVs have been demoed and are under development, but aren’t yet shipping in widespread numbers.

The ReRAM buffer technique doesn’t solve the long-term need for a NAND replacement, but it would be a way to implement early manufacturing while research continues. The team has yet to build a physical prototype of their design, so current conclusions, while positive, should be taken with a grain of salt.

Even if the ReRAM approach doesn’t bear fruit, we expect to see more of these sort of buffers in the future. In theory, drive manufacturers could create specialized blocks of reliable (and expensive) SLC flash cache surrounded by an MLC or TLC main drive. While the entire drive would still use NAND, rapid writes would first be performed on the SLC block, where its much higher write endurance would provide a similar benefit. The lower-durability TLC/MLC flash would be treated as a “Read/Write Occasionally” data storage area.

The types of solutions we’ll see in the next 3-5 years will depend on how rapidly NAND alternatives mature and how successfully companies can deploy future process nodes. Such drives could take a few years to trickle down into consumer hands depending on cost scaling and enterprise suitability. Buffer drives could conceivably form a fourth tier of storage alongside cache drive SSDs, standalone SSDs, and HHD (hybrid hard drives). At present, analysts believe that the cache drive + HDD solutions will skyrocket in popularity over the next 12-18 months. Buffered drives could offset the reliability concerns that drive the HDD + cache drive deployments and reassure customers that they only need one product — not two.

Source

Microsoft tries to sex-up Azure by likening CSS to LSD, XML to ecstasy (video)

It’s a hard life, being Microsoft. For more than 30 years, you produce the operating system that retains a virtual monopoly in the desktop space. You’re one of the most valuable companies in the world, and the largest and most profitable software company in the world. You’ve even broken the Japanese stranglehold on the console market, and created the fastest-selling gadget of all time — and yet, despite all that, you’re undeniably unsexy.

Now, nun-like lasciviousness isn’t so much of an issue when you’re marketing a new iteration of Windows or Xbox, but when it comes to frigid products such as Azure, Microsoft’s cloud computing service, the company has a big problem. You see, while Apple and Google’s inherent sexiness can make even the most dull products sound exciting, Microsoft has to try really, really hard to get customers and consumers excited. Which brings me neatly onto this video, from the Norwegian Developers Conference in Oslo, which shows a euro-trance dance troupe bouncing along to quite the most ludicrous lyrics I’ve ever heard, ostensibly to promote Microsoft Azure.

[Warning: You probably shouldn't watch this at work -- certainly not without headphones.]

Marvel, as some Norwegian songwriter wows you with poetic lyrics such as:

• The words ‘micro’ and ‘soft’ don’t apply to my penis / I do CSS as my LSD, and XML is my ecstasy
• I’m a software developer / I’m developing / and I’m here to party
• Program people! Code fast, code hard, it’s the ND Conference / We’re here to talk software, we’re here to talk bugs / tonight we’re gonna party and coding is our drugs!

In Microsoft’s defense, this spectacle was apparently arranged by a third-party marketing company. Frank Shaw, Microsoft’s head of corporate communications, has since apologizedfor the “vulgar language” and said the show was “inappropriate.” Curiously, Microsoft hasn’t disclaimed its involvement — in all likelihood, this was probably arranged (and approved) by Microsoft Norway, without corporate HQ’s knowledge.

Personally, I don’t think this is much crazier than Microsoft’s use of Usher to advertise Kinect at E3 last week. The lyrics of the song are obviously obscene — but then again, this is Scandinavia. When it comes to sensibilities and social acceptability, you would be hard pushed to find a group of people that have more diametrically opposed sensibilities than the USA and Scandinavia. If Microsoft had tried to use Usher in Norway, the company would probably still be facing a PR snafu, just for different reasons.

Read more about the size of cloud computing

Source

New Bulldozer deep dive sheds fresh light on AMD’s troubled CPU

Of all the CPUs launched in the past decade, none has been as puzzling as AMD’s Bulldozer. While it’s been widely derided as AMD’s version of Intel’s Pentium 4 Prescott, even early analysis indicated that such comparisons were fundamentally flawed. Bulldozer was an ambitious design to fuse chip logic and improve performance, but teasing out why it didn’t work has been a long, difficult process.

Thanks to Johan De Gelas over at Anandtech, we now have considerably more information on what doesn’t work, and some insight into how AMD can fix it. De Gelas used analysis tools from AMD and Intel to gather data on how Sandy Bridge, Magny-Cours, and Bulldozer/Interlagos execute various workloads and where the performance bottlenecks lie. Conventional wisdom, including my own articles, implied that Bulldozer’s high L2 cache latency was a major factor in the chip’s disappointing performance. His findings indicate that cache latency isn’t the problem I thought it was, at least not in server workloads.

Death by a thousand cuts

There are two major problems undermining Bulldozer’s performance in server workloads. First, there’s the issue of branch prediction. Bulldozer’s branch predictor is more accurate than Magny Cours’, but Interlagos takes a 20-cycle penalty in the event of a missed branch, whereas MC’s penalty is 12 cycles. As Johan explains, this is where comparisons to Prescott miss the mark; Prescott’s branch prediction penalty could be as high as 100 cycles.

Sandy Bridge’s branch prediction penalty is actually also fairly high, at 17 cycles, but Intel uses a 6K decoded µop cache that brings the penalty down to 14 cycles if the instruction is found there. This is one avenue AMD could potentially explore to improve the chip’s performance.

L1 cache associativity is the other issue discussed in-depth. We’ve known for quite some time that Interlagos’ efficiency suffers when both modules are enabled; additional evidence suggests that flipping on both modules in an Interlagos core doubles the number of L1 cache misses in certain workloads. Increasing cache associativity could help counter this.

Anandtech hints at (but does not disclose) a fourth “showstopper,” which leaves clock speed as the final factor. Here, there’s evidence of improvements in the latest Piledriver core at the heart of AMD’s Trinity. Piledriver uses what are known as “hard flops” and a resonant clock mesh to reduce power consumption and improve clock speeds. AMD claims Piledriver delivers a 10% reduction in dynamic power consumption compared to Bulldozer, with certain workloads improving by as much as 20%.

Putting it all together

The data suggests that Interlagos’ future in the datacenter is more hopeful than some have thought. The clock speed improvements built into Piledriver should allow that CPU to compete more effectively against Intel’s Xeons when it launches later this year, even if the other improvements to the CPU’s branch prediction and IPC have only a small net impact.

The client roadmap is a bit murkier. Here, the high L2 latency is a much greater factor and there are precious few desktop apps that scale well beyond four cores. Higher clock speeds will still improve the competitive situation, but high cache latencies are the architectural equivalent of millstones around the architecture’s neck.

It’s entirely possible that we won’t see these problems truly addressed until Kaveri, the 28nm successor to Piledriver scheduled for 2013.

AMD’s roadmap still shows Vishera (aka Piledriver) holding the top of the performance spectrum, but its Kabini that integrates a third-generation Steamroller core. None of AMD’s roadmaps currently show a server/high-end desktop Steamroller variant; it’s not clear how the next-gen core transitions into the product line. Kabini will be the first AMD APU to integrate a GPU based on the company’s current 28nm hardware, but we suspect it’ll be a few more years before servers are able to consistently leverage the graphics cores in every day workloads.

Still, there’s reason to be marginally more optimistic about AMD’s long-term ability to scale the Bulldozer architecture. We don’t expect it to challenge Intel’s performance any time in the near future, but some of the more pessimistic appraisals of its scalability may also turn out to be misguided.

Source

Canonical teams up with Humble Indie Bundle, Ubuntu installations streamlined

As if there wasn’t reason enough to be a fan of Canonical and its Ubuntu flavor of Linux (Unity aside), the company announced that it has joined forces with the Humble Indie Bundle(HIB) to both raise awareness of the ability to play high quality games on the free platform as well as raise money for some awesome causes. Starting with this latest bundle of independent DRM-free games, Ubuntu users have a streamlined installation process that utilizes the Ubuntu Software Center in conjunction with the HIB purchasing engine to make it easier than ever before to download and install the games being offered. For those of you who are avid fans of supporting both the open-source movement and independent development houses this is a win-win scenario!

Purchasing the current HIB for play on Ubuntu is indeed a simple process. If you have bought a previous bundle, you’ll be used to how things work on the HIB website: you pay what you think is fair for the titles being offered which gives you the right to download and install the games. Before this new partnership with Canonical, Linux users who bought the bundle would have to download the individual files and install the games themselves. Canonical has done away with that process by allowing users to authenticate with the Ubuntu Software Center site, after paying onHumbleBundle.com. It’s a pretty simple procedure when compared to the past mechanic.

After talking with David Pitkin, Canonical’s Director of Consumer Applications, the full scope of the partnership became apparent. He noted, “We’re excited about the future of gaming on Linux and on Ubuntu devices. With the upcoming Ubuntu for Android and Ubuntu TV, there’s a lot of possibility for independent developers to find a good outlet for their titles. With our pricing structure being less expensive than both Apple and Android, smaller developers can still generate strong revenues while enjoying an established software deployment structure.”

Simply put, Canonical benefits from this partnership by creating exposure to the fact that Ubuntu users can play a lot of the same games that their Windows and OS X brethren have access to, and for a lower total cost. This benefit goes both ways as the HIB is also looking to increase its user base so that it can raise more funds for both charity and developers. Currently the Electronic Frontier Foundation and Child’s play are the charities that are being supported through the revenue generated by bundle sales, both of which are worthwhile organizations doing some amazing work.

This new partnership can be seen as yet another great decision in a string of moves that Canonical has made this year. Whether or not you’re a fan of the latest version of Ubuntu, there’s some exciting stuff coming down the pipe in terms of Ubuntu for Android and Ubuntu TV (both of which will support the Humble Indie Bundle games when they release officially). It seems like Canonical is maturing in its understanding of where it stands in the computing industry, and has decided to leverage partnerships that will help with penetration into an overcrowded market without overreaching as well as allowing innovation that expands the horizons of what’s possible.

The Humble Indie Bundle V is currently available for purchase, and has some awesome titles such as Psychonauts and Bastion for you to enjoy. The sale will continue for the next two-weeks, so if you can’t jump on it right now you have a bit of time. (So far we’ve only been able to locate Limbo, Amnesia, and Sword and Sworcery on Ubuntu’s apps site.)

Read more at the Canonical Blog

Source

How big is the cloud?

Last month, ExtremeTech revealed to youthe true scale of internet porn. At any one time, streaming adult videos probably utilize around 30% of the internet’s total bandwidth, which equates to around 6 terabytes of porn being consumed every second. But what about the other 70%? Netflix, YouTube, and other non-adult video sites are huge bandwidth hogs, possibly accounting for as much as 40% of internet traffic. Digital file lockers, such as Rapidshare and Megaupload, account for around 10% of traffic worldwide. Web surfing and email (and spam!) are another 15%. And then there’s cloud computing.

Today, the vast majority of web services and sites are hosted in the cloud. By this I mean that, instead of companies (such as Ziff Davis/ExtremeTech) managing their own hardware, third-party cloud storage and computing services are used. Amazon Web Services (AWS), Microsoft Azure, and Google are three prominent examples of huge cloud clusters, but there are hundreds of smaller operations that range in size from a whole data center down to a few racks.

The power of the cloud is vested in the fact that it can be coerced and shoehorned into tasks as disparate as a cloud-based supercomputer, to webmail, to simple document storage. On a single cloud cluster, Google can host and serve petabytes of YouTube videos and store all of your email and documents. Of all the facets of the cloud, though, today we’re going to focus on cloud storage.

A Microsoft data center

While storage might not be as sexy as terabytes of RAM and thousands of CPU cores, it is the most reliable way of measuring the size of the cloud, especially when we factor in bandwidth usage. From the total amount of storage we can also work out the cost of cloud storage — and from there, we can finally work out why the likes of Google, Microsoft, and Dropbox are falling over themselves to provide cloud storage services.

Like the porn story, we’ll first start with some theoretical numbers, and then move onto some real-world figures (and hardware) from Backblaze, a cloud backup provider.

Petabytes

For the most part, real numbers from the big companies, such as Google, Facebook, Amazon, and Microsoft, are few and far between. If you scour the web, though, some rough ballpark figures emerge:

• Facebook, in its IPO filing, said it stores over 100 petabytes (PB) of media (photos and videos). It’s not unrealistic to say that Facebook probably has a total storage of capacity well beyond that, once you factor in backups and other data (status updates, likes, and so on), possibly in the 300PB range.
• Microsoft recently admitted that Hotmail stores over 100 petabytes, and that SkyDrive, with “17 million customers,” stores 10PB of data. Like Facebook, Microsoft’s total capacity, once we factor in the rest of Azure and its web properties, is probably well over 300 petabytes.
• Megaupload is relatively tiny in comparison, apparently storing just 25 petabytes.
• Amazon, rather than giving us a nice, easy number of petabytes, instead announces the total number of objects stored by its S3 cloud storage service. As of April 2012, Amazon S3 stored 905 billion objects. If we assume an average size of 100KB, that’s around 90 petabytes; if the average size is 1MB, that’s 900 petabytes — almost an exabyte!
• Dropbox, a year ago, stored “10+ petabytes” of data. It had 25 million users then, and 100 million users today, so all things being equal the company now stores around 40PB of data.

To put these figures into perspective, an average computer probably has a 500GB or 1TB hard drive, and a petabyte is 1024TB. At the very least, then, Microsoft and Facebook data centers play host to more than 100,000 hard drives. Without building custom hardware, you can squeeze 48 drives into a 4U enclosure. After accounting for networking gear, that means you’re probably looking at around 400 hard drives per 40U rack — or 250 racks, each of which occupies around one square meter of floor space. This might sound like a lot, but when you consider that Google, Amazon, Facebook, and Microsoft regularly roll out data centers with floor plans of over 30,000 square meters (300,000+ square feet), it’s really not that much. In the grand scale of things, a lot more space is dedicated to servers (i.e. CPUs) and networking gear.

Bandwidth

Bandwidth-wise, we have even less data from the big boys. We know that, as of last year, one million files were being saved every five minutes — so today, with four times as many users, that’s 800,000 files per minute. Amazon S3, which is significantly larger than Dropbox, handles “650,000 requests per second.”

If we assume that the average file stored on Dropbox is 500KB (a mix of photos, videos, and documents) then Dropbox stores a total of 400,000 megabytes (0.4TB) per minute — or 6.7GB per second (54Gbps). We don’t have any data on how much data Dropbox sends per minute (i.e. people downloading files from their Dropbox), but it’s probably in the region of 10 to 20Gbps.

Amazon S3, which is mainly used to store static files for websites (images, style sheets, videos), probably has a lower average file size than Dropbox. If we assume an average size of 100KB per file, then 650,000 requests per second comes to a grand total of 61 gigabytes of data transferred per second, or 488Gbps. This is very close to the 800Gbps figure that we estimated for a large porn site, which equates to around 2% of total internet traffic — Amazon is pretty darn big!

Facebook and Microsoft, with between 100 and 300PB of storage each, probably fall somewhere between Dropbox and Amazon in terms of bandwidth usage — maybe 200Gbps a piece.

But enough theory! Let’s discuss some real-world numbers and real-world hardware!

Backblaze

Try as we might, every cloud storage provider rebuffed our requests for information on their hardware and software setups — except Backblaze, one of the few remaining unlimited cloud backup providers. Backblaze is relatively small, but the real numbers will still astound you.

First things first: Backblaze stores a total of 32 petabytes of user data, mostly fashioned out of 3TB hard drives. For redundancy, groups of 15 drives are banded together using RAID 6, which places two parity blocks on each of the 15 disks. RAID 6 can rebuild data from the parity blocks as long as there is no more than two concurrent disk failures in a group of 15 — and yes, replacing faulty drives is by far the most important task of whoever is on call at the Backblaze data center, so that redundancy can be restored as soon as possible.

Backblaze adds 3 petabytes to its storage cluster every month, in the form of 135-terabyte “pods” — custom-designed 4U rack mounted computer cases that accommodate 45 hard drives each (pictured below). 3 petabytes per month means that Backblaze is adding roughly 22 pods per month — which equates to almost two standard 48U racks.

While you might think that the pods themselves are a major operating cost (they cost $7,384 each), hardware is really the least of Backblaze’s concerns; power, data center space, and bandwidth are far more expensive over time.

At $0.20 per kilowatt hour (kWh) each pod costs around $100 per month. Data center power is incredibly expensive, which is why many web companies are building data centers in locations with cheap power, such as Oregon. Backblaze currently hosts its servers in a third-party data center, which means it has to pay for the floorspace occupied by its cabinets; again, this works out at around $100 per pod (~$1,000 per rack). At the time of writing, Backblaze has a total of around 300 pods stored in 40 rack cabinets — so around $30,000 is spent per month on electricity, and around $40,000 is spent on floor space.

And then, of course, there’s bandwidth. At any one time, Backblaze’s routers have 12Gbps of data inbound (about 1.5 gigabytes per second), and 500Mbps (62.5MB/s) outbound. The disparity is because Backblaze is a backup service — but even so, that 500Mbps figure would suggest that hard drives fail (or data is lost) much more than I would expect. At the moment, Backblaze receives around 100 million files from its users per day, or 36.5 billion files per year — and in 2011, it restored a total of 600 million files. Unfortunately we don’t know how exactly how many restored computers that equates to, but at a conservative estimate of 5,000 files per computer (excluding Windows and program files) that comes to 120,000 very relieved backuppers.

Back in its salad days, Backblaze paid $30 per Mbps per month — now, thankfully, that figure’s down to $2/Mbps/month, which comes to a total of around $25,000 per month. Backblaze does have cheaper offers on the table from other ISPs, and again the company would save considerable money if it built its own data center.

All told, then, the total operating cost (before employees!) of a 40-petabyte cloud storage cluster is $100,000 per month, split fairly evenly between floor space, bandwidth, and power consumption.

Why cloud storage is a cut-throat business

You may have noticed that, in recent months, Microsoft, Google, Dropbox, and others are fervently fighting to be your cloud storage provider of choice. At first, I couldn’t work this out — but now, after looking at the numbers, it becomes a little clearer.

Backblaze charges $5 per month for unlimited backup — so with operational costs of $100k per month (before employees), it needs just 20,000 users to break even. Backblaze didn’t provide us with its customer count, but its CEO did tell us that the company is profitable (Backblaze has 13 employees, incidentally, and it operates as a startup: i.e. everyone gets a tiny salary).

Another more prominent example is Dropbox, which charges $100 for 50GB of storage. Unlike Backblaze which runs its own hardware, Dropbox uses Amazon S3 for storage, which charges $0.055 per gigabyte per month for clients using more than 5 petabytes (Dropbox stores around 40PB). If the average Dropbox customer uses 25GB of his 50GB allotment, Dropbox pays Amazon $1.38 per month — and yet the service costs between $9 and $10 per month.

For cloud storage providers who operate their own data centers, such as Microsoft SkyDrive, Google Drive, and Amazon S3, the profit margins will be significantly fatter than Dropbox.

The caveat, though, is that all of these services have many more free users than paid. Dropbox, with 100 million users, is by far the largest cloud storage provider — but the vast majority of those are freeloaders. The number of paid users is probably quite small, and thus the profit margins are probably quite slim too.

Strategic dominance

The more likely reason for such feverish competition in the cloud space is strategic dominance; for Google and Microsoft, cloud computing is where the next platform war will be pitched. With Windows 8, you will log in with your Live ID and all of your documents and settings will automatically sync with the cloud, allowing you to freely roam between devices (or access your files from any web browser). Chrome OS is entirely web- and cloud-based, each successive version of Android develops deeper ties to the cloud — and of course, with Google+, the company is trying to weave as many of its web properties together as possible.

Ultimately, these companies are vying for control of your files. If you use Google Docs/Drive for word processing, then you’re much more likely to choose Chrome OS or Android, both of which are strongly tied into Google’s cloud — but if you use SkyDrive and Office Web Apps, Windows 8 and Windows Phone 7 make much more sense. In this regard, I’m not entirely sure what the future holds for “standalone” cloud storage services, such as Dropbox. If Microsoft, Google, and Apple can build cloud backup and file sharing into the operating system, what purpose does Dropbox serve?

Source

IBM concerned about Siri’s loose lips

IBM is worried about Siri and will prevent iPhones from accessing the virtual assistant while connected to the company’s internal network. Chief Technical Officer Jeanette Horan says IBM blocks the servers that power the software, fearing a security risk or information disclosure of corporate data.

When asking Siri a question, recordings of the question are sent to Apple’s servers where it is analyzed. Apple has not been too forthcoming on what is done with these files, if those files are stored ,and for how long if they are. This makes IBM uncomfortable, enough that it decided to ban the app.

“We’re just extraordinarily conservative. It’s the nature of our business,” Horan explains.

While the company’s move seems extreme, it stems from a legitimate concern. Companies big and small are struggling with employees’ desire to use their own devices in the workplace. The reasons for doing so vary: some prefer to use a platform they’re familiar with — say OS X over Windows, or Windows over Linux — or they want personal freedom, such as the right to access social networks or instant messaging clients while at work.

This presents a security problem for IT administrators. Admins can’t control what we’re doing on these devices quite like they can on company owned equipment, so it results in actions like the one IBM has taken.

Is Siri truly a threat to corporate security? That’s questionable. It’s hard to imagine what IBM employees would be asking or telling their phones that would truly put the company at risk. It seems more an issue of employee discipline than anything that Apple would do.

Above and beyond that, what would Apple truly gain from spying in on the actions of IBM employees? Probably not much. The moral of the story here seems to be that if you value your job, and are working on some pretty secret stuff, it’s your own responsibility to make sure it doesn’t slip out.

Source