Raising the dead: Can a regular person repair a damaged hard drive?

Edit: This story was written to detail the options a user has when a hard drive with important data dies unexpectedly. Many of you have left comments advocating the freezer trick, stating that used as a Hail Mary, you’ve had good results. I do not dispute that the freezer trick *can* work. If you have a hard drive you don’t really care about, and you’re curious to see if you can get the data, by all means, freeze it.

If you do need your data back to the point that you’re considering paying someone to retrieve it, do not freeze the drive, even as a Hail Mary. You may very well make the problem worse and end up costing yourself money.

On June 22, 2012, my primary hard drive, a Samsung HD103SI, quietly passed away. There was no warning — no grinding, no clicks, clacks, or sudden bang. One moment, I was working on a story, the next, I wasn’t.

It quickly became clear that something more serious than a simple system lock had happened. Post-reboot, the HDD would spin up smoothly, beep 12 times, and then spin down. The drive was never recognized in BIOS, which nixed any chance of using disk recovery software to extract data.

This is a story of my efforts to repair the drive myself, my research into the question of whether or not users can repair modern hard drives, and the results of my efforts. If your drive is still detected in BIOS, you may be able to use software tools to retrieve your data. Here, we’re going to focus exclusively on hardware-related failures, and what your options are.

Part of the reason for writing this story is that data recovery is difficult to accurately research unless you’re fairly versed in it to start with. There are dozens of data recovery firms, all promising clean rooms, the latest tools, and highly trained professional staff. Many firms refuse to publish their prices online, which makes comparisons difficult, and it’s apparently common for small companies to farm tough jobs out to larger ones.

This one, for example, is pretty much toast

Step 1: Broadly identify the type of problem

There are two broad categories of problems that can nuke a drive: PCB issues and internal component failures. If the problem is inside the drive, skip down to Step 3. If the problem is on the PCB, there is a glimmer of hope.

Original image courtesy of mdproductions.ca

The best kind of PCB problem to have is a blown Transient Voltage Supressor (TVS) diode, as shown above. According to Seagate’s FAQ, a TVS diode “protects a sensitive circuit by diverting damaging overvoltages and spikes away from the load.” When a spike occurs, the diode blows. Because the diode is no longer functional, the drive won’t power up. Snip the diode off, and the drive will function normally, albeit in an unprotected fashion. Copy your data over to a functional unit, toss the old one, and count yourself lucky.

My hard drive unfortunately didn’t die this way. There was no visible damage to the PCB but when I removed the board and flipped it over I found a burned-out contact point.

Step 2: Understand your options

From here, you’ve got two choices. You can opt for a replacement PCB, or you can buy an entire donor drive. It’s important to secure as close a match as possible between the original HDD and the donor drive/board. In my case, that meant finding an HD103SI PCB that matched my drive’s make/model number, PCB number, board revision, and drive family (Trinity, in this case).

The drive controller model number and PCB codes are outlined in red.

Understand this: Simply replacing the PCB almost certainly won’t fix a dead drive. If it does, you’ve gotten lucky. Each drive ROM contains parameters and data unique to that particular device — if the parameters for your dead drive are different from those contained on the donor PCB, it won’t function.

In my case, I opted for just a PCB. In retrospect, a full drive might have been the better option, but the purchase only set me back $22 and ~14 days while my part took the slowboat from China. Swapping out the PCB eliminated the 12 beeps, but accomplished nothing else. Beep-less, my drive sat mute — spinning, but unrecognized in BIOS.

Your options at this point depend on what sort of HDD you have. Some hard drives have an externally mounted ROM/NVRAM chip that can be removed and soldered on to a new PCB. Other drives, like mine, incorporate the ROM into the controller. The only way to find out is to go digging for information online, and you’ve got to apply a strong sort filter to estimate the value of what you’ll find. A number of dubious websites advertise a “PC3000 PCI” card, for example, but this is almost always a Chinese clone of the original product, and is far too old to handle modern drives in any case. The real PC3000 UDMA test kit runs over $4000 — far more than the typical cost of a data recovery.

After my simple PCB swap didn’t work, I decided to try to repair the burned contacts on the original board. Here, your options are to either buy a conductive ink pen or to use something a bit more humble. I took the humble option, trotted over to an auto parts store, and picked up a rear window defroster repair kit. I taped off the damaged contact, applied the conductive ink, gave it 12 hours of drying time, fired up the drive…

My repaired PCB. The burned contact is at the far lower right of the group

And nothing had changed. The drive still spun up, emitted twelve beeps, and spun back down.

That was my second major disappointment and it leads directly to the next step…

Step 3: Resist the urge to do something stupid

Surf the internet for more than two minutes, and you’ll find people who recommend you do one of the following things:

• Stick your hard drive in the freezer
• Pop your hard drive into the oven
• Give it a few taps with a hammer or rubber mallet

People will swear by these options and promise you that they’ve revived 15 drives just this way. Don’t listen. I’m not claiming that no person ever brought a disk back to life by jamming in amongst the frozen peas, but this strategy is far more likely to cause irreparable damage than it is to miraculously affect repairs. Keeping the drive in a ziplock bag while in the freezer won’t help; condensation will form on the drive when you remove it from the bag prior to firing it up again.

This is what happens when you take a drive out of a freezer — whether you bagged it or not

Leaving the drive in the bag until it’s returned to ambient air temperature might prevent condensation from forming on the platters or heads, but the point of the freezer trick is to run the drive at a lower temperature.

Every repair attempt you make should be balanced against the chance of doing additional damage. To that end, never open the enclosure. If putting your hard drive in the freezer is a bad idea, opening it is infinitely worse. You are not qualified to adjust the alignment of heads or platters that normally spin at 75 mph and are aligned to tolerances measured in micrometers.

Step 4: Hire a professional

This is where I ended up. I talked to a number of data retrieval companies, including Datacent and Secure Data Recovery Services. Datacent quoted me a $750 rate for drive head replacement; Secure Data Recovery was less certain of the cause, but believed repair would likely run between $1,200-$1,500. As of this writing, I haven’t decided what to do. Even after extensive research, my objective visibility on Datacent (or any other data recovery firm) is just about nil. If I opt for repair, I’ll certainly report back on my experience, but ultimately I have no way of knowing how difficult it will be for a properly equipped facility to recover my files, how long it will take, or what constitutes a fair market rate.

With all of that said, here’s a few tips on what to look for (and what to avoid).

• No data, no fee: Avoid companies that insist on charging you for the privilege of failing to provide a useful service.
• Low-cost evaluation + return fee: Many firms offer a free diagnosis but will charge relatively high postage to send the drive back. $25-$30 for an evaluation+postage seems fairly reasonable, we’d be cautious of companies charging $50 or more for the two services combined.
• Avoid broad estimates: No firm can completely diagnose a hard drive by remote, but that doesn’t mean they can’t get an idea for what’s wrong. Beware of companies that offer estimates broad enough to buy a car with. At the very least, ask for estimates that fit the typical cost of specific problems
• Check the price of parts: The quotes we received typically included the cost of replacement parts. Most companies also note, however, that difficult-to-find hardware may still add additional cost. It doesn’t hurt to see if you can self-source the part, particularly when the cost of a donor drive or PCB is a very small fraction of the total recovery fee.

Step 5: Get a (better) backup solution

I’ve learned two important lessons from this failure. The first is that different models of hard drive are more and less user-friendly; careful selection on my part when I bought the drive four years ago would’ve made it much easier to recover my data. The second is that a better backup strategy would’ve made the first point moot. Because I had a basic, episodic backup solution, much of my older data is safe. The only data I lost… was all the data related to ongoing projects that hadn’t been published yet.

That stung. The difficulty of recovering my information in the intervening month only made the situation that much more frustrating.

If you don’t have a backup solution and you actually care about your data, get one. There’s no guaranteed solution once the drive has failed and, unless you get lucky with a TVS burnout, no easy fixes. A burned out drive is your ticket to a whole lot of website surfing, information sifting, and one enormous headache from trying to separate “My cousin’s uncle’s friend’s neighbor’s dog knew a dude who threw his hard drive in a pool and it worked great” from real, reliable, data.

The forums and information at HDDGuru are a good place to start checking for information, but don’t be surprised if you have to check other places just to get an idea of what the problem might be. These waters are largely uncharted, and dragons lurk in the deep.

PS. Don’t do the freezer thing. Really.

Read: How ExtremeTech’s authors back up, how to make your own NAS, and how to make your own Windows Time Capsule

Source

Black Hat hacker gains access to 4 million hotel rooms with Arduino microcontroller

Bad news: With less than $50 of off-the-shelf hardware and a little bit of programming, it’s possible for a hacker to gain instant, untraceable access to millions of key card-protected hotel rooms.

This hack was demonstrated by Cody Brocious, a Mozilla software developer, at the Black Hat security conference in Las Vegas. At risk are four million hotel rooms secured by Onity programmable key card locks. According to Brocious, who should be scolded for not disclosing the hack to Onity before going public, there is no easy fix: There isn’t a firmware upgrade — if hotels want to secure their guests, every single lock will have to be changed.

The hack in its entirety is detailed on Brocious’s website, but in short: At the base of every Onity lock is a small barrel-type DC power socket (just like on your old-school Nokia phone). This socket is used to charge up the lock’s battery, and to program the lock with a the hotel’s “sitecode” — a 32-bit key that identifies the hotel. By plugging an Arduino microcontroller into the DC socket, Brocious found that he could simply read this 32-bit key out of the lock’s memory. No authentication is required — and the key is stored in the same memory location on every Onity lock.

The best bit: By playing this 32-bit code back to the lock… it opens. According to Brocious, it takes just 200 milliseconds to read the sitecode and open the lock. “I plug it in, power it up, and the lock opens,” Brocious says. His current implementation doesn’t work with every lock, and he doesn’t intend to take his work any further, but his slides and research paper make it very clear that Onity locks, rather ironically, lack even the most basic security.

I wish I could say that Brocious spent months on this hack, painstakingly reverse-engineering the Onity lock protocol, but the truth is far more depressing. “With how stupidly simple this is, it wouldn’t surprise me if a thousand other people have found this same vulnerability and sold it to other governments,” says Brocious, in an interview with Forbes. “An intern at the NSA could find this in five minutes.”

That is how he justifies his public disclosure of the vulnerability: If security agencies and private militias already have access to millions of hotel rooms, then this is Brocious’s way of forcing Onity to clean up its act. By informing the public, it also means that we can seek out other methods of securing our rooms — such as chain- or dead-locks on the inside of the room.

As for how Onity justifies such a stupendously disgusting lack of security, who knows. Generally, as far as managerial types go, securing a system seems like a frivolous expense — until someone hacks you. In non-high-tech circles, hacks like this are par for the course — usually, a company doesn’t hire a security specialist until after its first high-profile hack. For a company that is tasked with securing millions of humans every night, though, it would’ve been nice if Onity had shown slightly more foresight.

Read more about last year’s Black Hat hacks: Opening car doors via SMS and hacking wireless insulin pumps

[Image credit]

Source

The camera that can see through frosted glass and skin, and around corners

Scientists in Israel have created a camera that can see around corners, or through solid objects such as frosted glass, and skin. The most exciting facet of this innovation is that the camera uses natural light to perform the imaging — such as a lamp, or the Sun — and not lasers or X-rays.

Ori Katz, Eran Small, and Yaron Silberberg of the Weizmann Institute have shown that they can accurately resolve an object that’s hiding behind nearly opaque obstacles, or around a corner (or in another room, as long as the door’s open. In both cases, the light is scattered by the obstacle (the frosted glass, the corner wall), creating what appears to be white noise — but their camera can take these speckles of noise and enhance them “1000-fold” (the scientists’ words) to recreate the image with surprising accuracy.

The approach is surprisingly simple, and relies on a device called a spatial light modulator(SLM). Basically, when light bounces off an object, each part of that object changes the phase of light differently. An SLM is an array of pixels that can alter the phase of light passing through it, depending on the electrical current passing through each pixel. In this case, the scientists used a genetic optimization algorithm to modulate each pixel of the SLM until a sharp image is extracted from the white noise.

In the image below, the first two images show the genetic algorithm’s ability to discern a single point of light in a mess of white noise; ‘c’ shows what the human eye would see, looking at the obstacle between the camera and object; and ‘d’ shows what this new camera can see through the obstacle.

Now, if you’re an avid follower of bleeding edge tech, you will probably remember a previous project by MIT — a femtosecond laser that could photograph objects at a trillion frames per second, or see around corners. The Israeli camera is significant because it uses incoherent light — a light bulb, daylight — and off-the-shelf hardware, while MIT’s laser approach is huge, unwieldy, and expensive.

Moving forward, the scientists hope that their SLM-based approach can play a significant role in medical imaging, where it is currently very hard to resolve details that are inside the brain or other organs. It’s also worth noting that MIT’s around-corners camera is laboratory-sized — but it sounds like the technology behind the Israeli design might fit into a conventional camera. If that doesn’t raise the eyebrows of wannabe superheroes and privacy advocates, I don’t know what will.

Read more at the research paper: Looking around corners and through thin turbid layers in real time with scattered incoherent light

Source

Fifty Years after Telstar: Why the first commercial satellite was so important

Today, July 12, marks the 50th anniversary of the first broadcast from the Telstar satellite, the first commercial satellite in orbit. These broadcasts heralded a sea change in the way we communicated, and laid the groundwork for many of the services we take for granted today.

Telstar was one of the first satellites with a usable purpose. Up until its launch, the world’s two superpowers at the time — the US and the USSR — were more interested in one-upping each other by just proving they could put something in orbit. The Soviets were first, launching Sputnik in October 1957, which included a radio transmitter that emitted a short beep every few seconds. The US responded in January 1958, launching Explorer 1, which sent back measurements of the magnetic belt that surrounds the earth.

There were other major accomplishments: the first living organism in space, Laika the Dog, aboard Sputnik 2 in November 1957; and the first telephone and TV transmissions via satellite carried by Echo 1 in 1962 shortly before the launch of Telstar. But it was this satellite, built by Bell Telephone Laboratories for AT&T that changed everything.

On July 10, 1962, Telstar launched into orbit atop a Thor-Delta rocket and communicated between two ground stations, one at the Andover Earth Station in Andover, Maine, and the other at the Goonhilly Satellite Earth Station in Cornwall, England. Once in orbit, the satellite took what is called a “non-geosynchronous” orbit, which means the satellite orbited the earth at a different speed than the Earth’s rotation.

This resulted in only a 20 minute window of use every 2.5 hours as Telstar passed over the Atlantic Ocean. Those first few transmissions were fairly simple: the first attempt was live video of the American flag from the Andover Earth Station. Telstar’s big test though came on July 23, when viewers in both North America and Europe were treated to the first live television satellite pictures.

At 3:00pm ET, Telstar delivered portions of a press conference by President John F Kennedy, a clip of a baseball game between the Philadelphia Phillies and the Chicago Cubs, and video images from Cape Canaveral, Florida, Quebec, and Stratford, Ontario. Before its transmitters failed due to high-altitude radiation in November 1962 and then permanently in Feburary of the following year, it had carried more than 400 telephone, telegraph, facsimile and television transmissions in total.

Even though it had spent not much more than six months in operation, Telstar’s effect on society was immense. The world suddenly became a whole lot smaller. The era of the 24-hour news cycle arguably got its start right here: instead of the old news reels which moved only as fast as the couriers that delivered them, pictures of world events could be transmitted worldwide within minutes.

Communications also changed. Undersea cables were the go-to method for overseas communications. One break in these lines, however, and those communications could be cut off. Satellites in the mode of Telstar made possible a much more reliable method to communicate over long distances.

“50 years ago transmissions enabled by Telstar captured the attention and imaginations of people everywhere,” Secretary of the Smithsonian Museum Wayne Clough says of the occasion. “The 50th anniversary reminds us how far we have come, and how much potential there is the new era of digital communications.”

Clough is certainly correct, but those first few transmissions in the Summer of 1962 are what changed everything.

Source

I’ve seen the future: Hands-on with Google Glass

Like most of you, I’ve heard plenty about Google Glass and seen some interesting demos — including the amazingly cool one at the Google I/O keynote this morning that featured skydivers and stunt bike riders. It wasn’t until Sergey Brin put a prototype pair on my face this afternoon, though, that I realized just how cool they are and how important they may become.

To start with the glasses really are featherlight. I know we’ve heard that before, but I didn’t really believe it until now. They may not be lighter than my plastic reading glasses, but they are definitely feel lighter than my sunglasses — perhaps because the weight is nicely balanced in front of and behind one ear, instead of being bulked up in front and weighing down on the bridge of the nose.

The screen is surprisingly small and out of the way — a key design goal for the team. Brin and the rest of the team were very clear that a big piece of the project’s mission is to provide technology that does not get in the way of everyday life. At the same time, the image on the screen — mostly demo videos provided for us by Google — was razor sharp. Glass is designed to project the image at far-focus, so if you have good eyesight or corrective lenses (like me) that allow you to see in the distance, there is no need to re-focus to see the image. This is a big improvement over trying to look down at a dashboard or a phone, which requires some serious effort or reading glasses for anyone who is eyesight-challenged.

Brin and his product managers were very clear that they don’t see Glass as a phone replacement, or as a device that’ll be used for heavy web-surfing. Instead they envision it as a great way to share the moment with others, and to find information that might be relevant right then and there. Glass makes it simple to capture video, stills, and even time-lapse photos — one product manager gave the example of capturing images every ten seconds on a bike ride with no effort on his part, and without distracting from the ride or his chatting with his fellow riders.

Clearly the team has thought through many of the usability issues inherent in sticking a screen in front of your face. By placing the image outside your normal field of view, they made sure it doesn’t get in the way. When you receive a priority message for example — the default is to only alert the user of priority messages — Glass gently beeps. It only shows you the message if you tilt your head up like I am in this photo — which it figures out using its built-in sensors.

To help make the most of this amazing new hardware platform, Google is signing up thousands of developers to get an “Explorer” version of Glass early in 2013, for $1500. It expects to have a product for consumers — at a lower price point, but still premium-priced — perhaps by the end of 2013. For one, I’m incredibly excited about the possibilities. Sure, for awhile you’ll look like a dork walking around with a pair, but that was true of Bluetooth earpieces until they became a fashion item and are as commonplace as briefcases now.

Read more about the Utopian augmented reality dream, powered by Google Glass

[Image Credits: David Cardinal, Sergey Brin]

Source

Piezoelectric speaker will let researchers talk with dolphins

A team of Japanese researchers from the Tokyo University of Marine Science and Technology may become the first people to communicate with dolphins using their own language.

Dolphins, with many human-like social and sexual traits, are often regarded as one of the most intelligent animals on the planet. The problem is, despite our own vast intelligence, we don’t actually know how intelligent dolphins (or monkeys or pigs or crows) are because we can’t understand their language. Encoded in those whistles, burst-pulse sounds, and clicks, dolphins might regularly discuss calculus and astrophysics — we just don’t know.

Now, however, Japanese researchers have developed a “dolphin speaker,” which seems to be the first underwater device that is capable of producing the full range of dolphin sounds (pictured right). While the human voice generally ranges from 300Hz to 3KHz, and human hearing ranges from 10Hz to 20KHz, dolphins can produce and hear sounds up to 150KHz. Dolphins can also vocalize a number of frequencies simultaneously — the clicks it uses for echolocation are generally an ultra-broadband emission of sound ranging all the way from 1KHz to 150KHz (different objects attenuate different frequencies, so this no doubt gives dolphins a very accurate sense of its surroundings). This dolphin speaker uses four piezoelectric elements (which apparently have never been used underwater before), and one silver element, to reproduce a dolphin’s “voice” almost perfectly (pictured below).

In theory, the researchers can now head out to the ocean (or visit SeaWorld) and begin conversing with dolphins — or at least start down the very long path to understanding dolphin communication. At first they will simply record sounds and play them back to see what response they get — but it won’t be long before they cut and splice various clicks and whistles to try and understand the dolphin language.

It’s worth pointing out that humans have been communicating with dolphins for decades, much in the same way that we communicate with (and train) dogs. Beyond training them to jump through hoops and stuff, though, while researching this story I came across the US Navy’s efforts to train dolphins (and sea lions!), apparently to rescue swimmers and locate underwater mines. There also seems to be an (unsubstantiated) theory that the US and Russia have also trained dolphins to attack enemy combatants, or destroy submarines (presumably by swimming into the propeller aseembly?)

Suffice it to say, if this Japanese project is a success and they can actually talk with dolphins, and World War 3 kicks off, we might have to think twice before enjoining a naval assault on the Land of the Rising Sun.

Read more at Acoustical Society of America

Source

MIT wants you to print your own paper robots

According to researchers at MIT and the University of Pennsylvania, one day in the not too distant future you’ll be able to design, print and build your own robot. The researchers have so far received a $10 million grant from the National Science Foundation to develop the necessary technology.

The multi-disciplinary team says they want to make the process simple enough so that anyone could build a paper robot. Penn team leader Professor Vijay Kumar believes the technology could revolutionize the way science and technology is taught in school, and allow for rapid production of customized goods.

Paper robots are nothing new: A company called ReaDIYmate recently debuted on Kickstarter, looking for funding to build paper robots and that are controlled via WiFi. MIT’s paper robots seem a little more advanced in what researchers want them to do, and its more than just educating about robotics.

Imagine a time and place where building a robot for a specific need would be as easy as heading to your local Lowe’s or Home Depot, then have the robot you need designed, printed and programmed in as little as a day. Two prototypes have been built so far: an insect-like robot for exploring dangerous areas, and a gripper device for the handicapped.

“Our vision is to develop an end-to-end process; specifically, a compiler for building physical machines that starts with a high level of specification of function, and delivers a programmable machine for that function using simple printing processes,” MIT Professor Daniela Rus says.

The team points to the high expense currently to design and produce functioning robots. By simplifying the process, it would bring robotics to a much wider audience. With an automated process, more time could be spent on teaching the intricacies of robotics or getting to the task at hand rather than the laborious process of building the robot itself.

Certainly the prospect of robotics for the masses is something we’ve been told is coming for a long time now. Look at Asimo, Roomba, and other consumer-oriented robots. In all these cases it was the manufacturer building the robot for what they think we need it for.

Here, MIT is promising a future where we choose what we want in a robot, and the possibilities sure do seem endless. Robot slaves, anyone? I don’t know about you, but I’m getting visions of SkyNet and Judgment Day, where robots become self-aware and kill us all.

Read more at MIT

Source

Mac botnet grows to 600,000, 274 of them in Cupertino

Russian security firm Dr Web warns that at least 600,000 Macs are infected and part of a growing botnet, further disputing the notionthat Mac OS X is free of malware. 76% of these Macs are located in the US and Canada, with another 13% in the UK.

Possibly more embarrassing for Apple is the fact that 274 infected computers are located in Cupertino, California, which may indicate Macs belonging to Apple employees or even on the company’s campus might be infected. Mac users are advised to ensure their Macs are up-to-date to prevent infection, and some four million compromised web pages are believed to exist, including portions of DLink’s website, Dr Web claims.

The Flashback Trojan is the culprit here, but is nothing new. The Trojan first appeared disguised as a Flash installer last September, and disabled Mac OS X’s built in malware protections. This version makes its way into Macs through a Java vulnerability, and is loaded onto unpatched Macs without interaction from the user.

Apple could have prevented this from becoming a major issue, but for some reason it didn’t distribute the patch until this week (Oracle issued a fix in February). The update, “Java for OS X 2012-001,” is now available through Software Update. The description says it “delivers improved compatibility, security, and reliability by updating Java SE 6 to 1.6.0_31,” which is the Java version that closes the hole.

If you want to check if your Mac is infected, F-Secure has a fairly easy-to-follow guide. If your system is clean, your results will look something like the picture on the right.

Back to the sneakiness of Flashback: Apple enthusiasts are quick to point out malware doesn’t do well on Mac because the operating system always asks for a password before install. Here there is no such warning, and may signal a new front in Mac malware, where hackers are spending the time figuring out ways to get around Apple’s security protections.

Before, when Mac market share was in the pits, this was a lot of work for little benefit. That changed with the uptick in popularity of Apple — now Macs are a much more attractive target. Expect a lot more of this in the future, and probably the same excuses from the Apple faithful on why there’s nothing wrong.

In the meantime, head over to Software Update and apply that patch.

Source

Just how big are porn sites?

It is a truth universally acknowledged, that a person in possession of a fast internet connection must be in want of some porn.

While it’s difficult domain to penetrate — hard numbers are few and far between — we know for a fact that porn sites are some of the most trafficked parts of the internet. According to Google’s DoubleClick Ad Planner, which tracks users across the web with a cookie, dozens of adult destinations populate the top 500 websites. Xvideos, the largest porn site on the web with 4.4 billion page views per month, is three times the size of CNN or ESPN, and twice the size of Reddit. LiveJasmin isn’t much smaller. YouPorn, Tube8, and Pornhub — they’re all vast, vast sites that dwarf almost everything except the Googles and Facebooks of the internet.

While page views are a fine starting point, they only tell you that X porn site is more popular than Y non-porn site. Four billion page views sure sounds like a lot, but it’s only when you factor in what those porn surfers are actually doing that the size and scale of adult websites truly comes into focus.

We’ll start by laying the ground work, and then on the second page we have some real world figures from YouPorn, the second largest porn site on the web. If you like, take a moment to try and estimate the amount of traffic that YouPorn handles every second. Let us know in the comments if your guess is anywhere near.

Scale

The main difference between porn and non-porn sites is the average duration of a visit: For a news site like Engadget or ExtremeTech, an average visit is usually between three and six minutes; enough time to read one or two stories. The average time spent on a porn site, however, is between 15 and 20 minutes.

Then you need to factor in that most websites are predominantly text and images, while the largest porn sites push streaming video. When you load the ExtremeTech home page, you’re talking about a couple of megabytes, and then maybe 500 kilobytes if you load an article. When you stream porn, assuming a low resolution of 480×200, you’re looking at around 100 kilobytes per second — which, over 15 minutes, is around 90 megabytes.

Then you need to multiply 90 megabytes by the number of monthly visits — which is around 350 million for Xvideos. This comes to around 29 petabytes of data transferred every month, or 50 gigabytes per second. To put this into comparison, your home internet connection is probably capable of transferring a couple of megabytes per second, which is about 25,000 times smaller.

In short, porn sites cope with astronomical amounts of data. The only sites that really come close in term of raw bandwidth are YouTube or Hulu, but even then YouPorn is something like six times larger than Hulu.

Infrastructure

Serving up videos requires a lot more resources than plain text and images, in terms of storage, CPU cycles, internal I/O, and bandwidth.

While it obviously varies from site to site, most adult sites will probably store in the region of 50 to 200 terabytes of porn. This is quite a lot for a website (only something like Google, Facebook, Blogger, or YouTube would store more data), but in a world where 2TB drives are cheap and plentiful, this isn’t ultimately a very large amount. Last year we wrote about a Backblaze storage pod that can store 135TB in a 4U case, for just $7,400.

CPU cycles and I/O will be a function of the bitrate of the streaming video and the number of page views. First the porn site has to serve up a dynamic, searchable database of thousands of videos, and then, when someone clicks on a video, that file needs to be read from a hard disk and streamed over the internet. If you’ve ever transferred a lot of big files over a local network (i.e. stressed both your hard drive and Ethernet port) you will know how taxing this is.

Actual hardware requirements are almost impossible to derive (they’re not publicized), but in the case of a large porn site we’re probably talking about racks of quad-CPU servers, gigabit switches, and load balancers. Software-wise, most large porn sites will use a very-high-throughput database such as Redis to store and serve videos, and a light-weight HTTP server like Nginx to serve up the web pages.

Finally, bandwidth. Referring back to our Xvideos example (based on an Ad Planner estimate), a large porn site will have to have enough connectivity to serve up 50 gigabytes per second, or 400Gbps. Bear in mind this is an average data rate, too: At peak time, Xvideos might burst to 1,000Gbps (1Tbps) or more. To put this into perspective, there’s only about 15Tbps of connectivity between London and New York.

There are only so many ways of coping with this much traffic: You set up your own data center, rent a few racks in a very large data center, or use a cloud provider like Amazon AWS or Microsoft Azure.

A real-world example

The second largest porn site on the web, YouPorn, was kind enough to furnish us with some real-world facts and figures. You’ll be glad (or scared) to know that the estimated DoubleClick Ad Planner figures are actually quite a lot lower than reality.

YouPorn hosts “over 100TB of porn”, and serves “over 100 million” page views per day. All told, this equates to an average of 950 terabytes of data transfer per day, almost all of which is streaming video. This is around 28 petabytes per month, which means our 29PB estimate for Xvideos is on the low side; it probably serves 35 to 40PB per month.

It gets better! At peak time, YouPorn serves 4000 pages per second, equating to burst traffic in the region of 100 gigabytes per second, or 800Gbps. This is equivalent to transferring more than 10 dual-layer DVDs every second.

On the software-side of things, YouPorn’s primary data store is 100% Redis, with MySQL used as an admin tool to manage and add data to the Redis cluster. The site used to be primarily programmed in Perl with a MySQL backend, but in 2011 Perl was switched out for PHP and MySQL replaced with Redis. Nginx acts as the HTTP server, with both HAProxy and Varnish both used to load balance.

The Redis server deals with 300,000 queries per second, and between 8-15GB of data is logged every hour (visitor logs, behavior data, and so on). We’re told that this software stack should be capable of scaling up to 200 million views per day.

Sadly, YouPorn couldn’t tell us about its hardware infrastructure. Judging by the IP addresses of the YouPorn content delivery network (CDN), it’s probably not hosted by a cloud provider like Amazon, but rather in a large data center somewhere, with peering provided by Level 3.

To put that 800Gbps figure into perspective, the internet only handles around half an exabyte of traffic every day, which equates to around 50Tbps — in other words, a single porn site accounts for almost 2% of the internet’s total traffic. And there dozens of porn sites on the scale of YouPorn, and hundreds that are the size of ExtremeTech or your favorite news site. It’s probably not unrealistic to say that porn makes up 30% of the total data transferred across the internet.

The internet really is for porn.

Read more about the world of submarine fiber optic cables (which carry all of that porn)

Source

Lumia 900: Redefining the smartphone?

After almost five years of manic one-upmanship following the initial release of the iPhone, one OEM has finally realized that the future of smartphones lays in the arms of everyday consumers. The Nokia Lumia 900, available today and priced at $450 off-contract (or between $0 and $99 on-contract), is the cheapest, high-performance smartphone that the world has ever seen, and a strong indicator that it’s high time for early adopters to step aside before they’re washed away by mom-and-pop consumers. In comparison, the 16GB iPhone 4S is $650, and the 16GB Galaxy Nexus is around $600.

The key word here is “high-performance,” rather than high-end. The Lumia 900 has a thoroughly middle-of-the-road spec — a 1.4GHz single-core Snapdragon S2 Scorpion (circa 2010!), 512MB of RAM, a 800×480 display, and 16GB of storage — but for Windows Phone 7, that’s ample. A penta-band MDM9200 Qualcomm radio provides LTE coverage in the US, and good worldwide 3G coverage. The only stand-out piece of hardware is the 8-megapixel, Carl Zeiss, capable-of-720p-video camera on the back — but to be honest, this is the one bit that Nokia couldn’t skimp on; WP7 might be able to give an old CPU a new lease of life, but what good is that if the phone has a crappy camera?

Of course there are trade-offs, too. By using older, cheaper hardware, the Lumia 900 is larger and heavier than the competition, and it also has a shorter battery life (7 hours of talk time vs. 8 on the iPhone 4S and 12 on the Galaxy Nexus). And, of course, the biggest hurdle of all is still Windows Phone 7.

I know this is a very tired argument, but you can’t escape the fact that WP7 has just a 4% share of the US smartphone market (and probably much less worldwide). The third-party developer ecosystem is simply incomparable to Android or iOS. While WP7 recently crossed the 50k mark, Android and iOS both have at least 500,000. As a result, WP7 misses official apps for services like HBO Go, Hulu, Sonos, Tweetdeck, and Dropbox.

There are major architectural absences, too. WP7 doesn’t support IPsec VPN at all, and doesn’t have built-in support for VoIP or video calling, or USB mass storage. Windows Phone 8, due out later this year, will fix most of these issues, though, and cross-compatibility with Windows 8 should resolve any app ecosystem issues.

In reality, though, many of these factors are the kind of thing that would put off upgrades or sidegrades — they’re not the kind of thing that a first-time smartphone buyer balks at. The Lumia 900 feels good in the hand, looks attractive, is very responsive, and has most of the features that a smartphone should have.

That’s the key point here: The Lumia 900 isn’t targeting iPhone or Galaxy users; it’s targeting the 41% of the US public who still own a feature phone. If you’re already firmly set on getting an iPhone, the Lumia won’t stop you — but for walk-in customers, the $200 price difference is really quite significant. For customers on the fence — if saving $200 isn’t enough — the Lumia has a bigger screen than the iPhone and a better camera than most Android phones — two factors that could easily sway people towards the Nokia device.

Other tech sites are calling the Lumia 900 a flagship phone, and thus comparing it to other flagships like the iPhone 4S — but that’s like comparing the latest Hyundai with a Ferrari. They’re both excellent cars/phones, but they appeal to wholly different categories of consumer. The Lumia 900 is $200 cheaper than the iPhone 4S!

In actual fact, there isn’t really anything that we can compare the Lumia 900 to. It’s a whole new class of phone — a commodity smartphone that’s priced and marketed for the masses. It has all the trappings of a smartphone, but a price closer to that of a dumbphone. In short, as long as Nokia and Microsoft can convince the masses that this is a smartphone, and yet not directly comparable to theiPhone or high-end Androids — and the “Beautifully Different” ad campaign does just that — the Lumia 900 should fly off the shelves.

On a related note, the $450 Lumia 900 all but confirms my suspicion that a $200 Windows 8 tablet will be available at launch. Much in the same way that the Lumia could redefine the smartphone, the Nokia Windows 8 tablet should redefine tablet computing.

Read the full Lumia 900 review at PC Mag, or read about the dumbphone revolution

Source