So it’s time for me to eat crow, or depending on your taste, humble pie. Not so long ago I was confidently predicting a drawn out battle between two high definition disk formats; HD-DVD and Blu-ray. HD-DVD, backed by Toshiba and the DVD standards body offered cheaper players but only a limited range of titles; Sony’s Blu-ray was more expensive but had a larger library of movies. It seemed likely that the two formats would continue to co-exist, confusing purchasers who would continue to vote with their feet and carry on buying DVDs. Suddenly everything changed; HD-DVD is dead. On February 19th, Toshiba announced that it would immediately stop manufacturing HD-DVD players and recorders. In little over a month, HD-DVD had gone from a viable format for the future of movies to a technological cul-de-sac. What had gone wrong?

Two factors conspired to kill HD-DVD. The first was Sony’s technological wonder, the Playstation 3, which comes with a built-in Blu-ray player. After a troubled launch, the PS3 has begun to notch up impressive sales with more than 10 million sold to date (more than a million in the UK alone) – ten times the number of HD-DVD players that were sold during its lifetime. People may not have bought PS3 for movies, but they have certainly been experimenting with Blu-ray disks whose sales have been steadily climbing. The second nail in the coffin of HD-DVD was the decision by Warner Brothers to cease issuing new movies on the format. This left only Universal Studios and Dreamworks as committed to HD-DVD exclusivity for their movies. Within days of the Warner announcement; Woolworths in the UK and the colossal American chain WalMart said they were abandoning HD-DVD; the writing was on the wall for the format.

In the US and Japan, many retailers are compensating purchasers of HD-DVD players, either with cold, hard cash or with credits against the purchase of a Blu-ray player. On the software front, things are grim for HD-DVDs with most studios cancelling future releases; but HD-DVD users are enjoying a fire sale of existing titles as retailers dump their stocks, at the moment disks can be had for as little as £6 apiece – cheaper than DVDs! Existing HD-DVD players will continue to play regular DVDs, and in the event of one failing, Toshiba has stockpiled at least 8 years worth of spare parts. If, like me, you bought into HD-DVD, you will be able to enjoy it for many years to come.

The backers of Blu-ray, most notably Sony and the movie studios, are the victors. Toshiba might have lost this war, but it can easily afford to write off the costs of HD-DVD. The real losers in this war may well be consumers, and that is because we’re going to have fewer ways of enjoying our entertainment.

Region coding is part of a trend in media and computer software known as digital rights management (DRM) that aims to control how media can be used. DRM allows the publisher of a title to say where it can be played, on what machines, if it can be copied to another device, even whether the recording will evaporate after being played. You’ve almost certainly encountered DRM every time you play a DVD. When you first put a DVD into a player you will see a number of warnings about where the disk can be used and the consequences for pirating its contents. You may have noticed you cannot skip past these and get on with the movie – the DRM on the disk temporarily disables the functions that allow you to fast forward and go to the next track. The DRM on the disk also prevents you from copying its contents and from playing disks bought in one part of the world from working inside the UK. You can find a map of these ‘region codes’ on Wikipedia.

The DRM on DVDs was introduced as a reaction to the threat of piracy. Older, analogue technologies – such as audio and videotape can be easily copied, but the process is slow and the number of copies that can be created is very small. Crucially, as tape is repeatedly duplicated, the quality of the copies decreases – effectively limiting the number of pirated copies that could be circulated. However, when information is stored in digital form – such as on a DVD – it can be perfectly replicated an infinite number of times. These perfect copies can then either be written on to a blank disk, or distributed across the World over the Internet.

DVD’s DRM is a fairly elderly technology known as the Content Scrambling System (CSS). It was broken long ago and pirated versions of DVDs, stripped of all their DRM (often without the infuriating anti-piracy adverts found on genuine disks), can be found in most towns and cities and circulating on the Internet. When movie studios began to plan the move to high definition disks, they chose more powerful forms of DRM. Blu-ray’s protection is called BD+ and is generally thought to be superior to the AACS system found on HD-DVD as it allows manufacturers to continually upgrade their DRM against attacks. Unlike HD-DVD, Blu-ray also uses region coding to stop users playing and importing disks from other parts of the world. It is widely believed that the movie studios put their weight behind Blu-ray because of its stronger DRM, both because it offered better protection from piracy, and because it served to block the traffic in cheap disks from places such as the United States and Hong Kong.

DRM is not only found on DVDs, it is used by games companies to protect their products, in satellite and cable set-top boxes, on most music bought from online stores, even in the expensive HDMI cables needed to connect to high-definition television sets. Although there are a huge number of DRM technologies being used by various companies, most of them share a common technological root known as encryption; a field of mathematics concerned with scrambling information to shield it from prying eyes. Crucially, encryption is always reversible – that is the scrambled material can be restored to its original state by performing a decryption. Media documents controlled by DRM are distributed in an encrypted form and can only be decrypted by a user if they own both a decryption program and a second piece of information known as the key. The decryption is performed by dedicated microprocessors in the player using decryption keys stored in the player’s memory. If you use a software application such as Apple’s iTunes to play media files, your computer’s processor performs the decryption and the keys are stored in hidden files on your computer’s hard disk.

Some DRM schemes such as Apple’s FairPlay and Microsoft’s PlaysForSure tie media files to particular authorised computers. When you authorise a computer, the player software extracts information from that machine which might include information including your name, your registered email address, the unique serial number of machine’s CPU, the serial number of the operating system and so on. This information is used to generate the key needed to decrypt the media file. For both FairPlay and PlaysForSure users are restricted to playing a file on no more than five computers; if you try to play a file on an unauthorised computer or to authorise a sixth computer you will be unable to do so.

In Microsoft’s PlaysForSure scheme, machines need to be reauthorized when users upgrade from one version of Windows to another. Information about the machine is gathered, sent across the Internet to the PlaysForSure servers and a new key issued. And this process is now causing a problem because Microsoft is abandoning PlaysForSure in favour of its own Zune music player.

PlaysForSure was an attempt by Microsoft to eat into Apple’s dominance of the music player market. Rather than build a single device to compete against the iPod, Microsoft produced the PlaysForSure standard. Any manufacturer could then build devices PlaysForSure compliant devices with Microsoft collecting a small fee for each machine sold. Music, wrapped in DRM, could then be bought from a number of online stores that supplied music in Microsoft’s Windows Media Format. The idea was that competition between manufactures would quickly drive the price of their players below that of the iPod and users would gradually switch to the more affordable product, allowing Microsoft to steal Apple’s crown.

For any number of reasons, PlaysForSure was a failure and have Microsoft switched to copying Apple’s business model. They designed their own music player, the Zune (so far only available in America) that plays music bought through the dedicated Zune Marketplace online store. Rather than have Zune compete against iPod and PlaysForSure, Microsoft has chosen to kill PlaysForSure by simply switching off the authorisation servers. As soon as the servers are switched off it will no longer be possible to get new PlaysForSure keys from Microsoft. Anyone buying a new machine or upgrading their version of Windows after that date will find they can’t play music purchased from Microsoft’s old MSN Music store. Microsoft originally intended to switch the servers off on August 31st 2008, but after a furious customer reaction, have since extended the scheme, now excitingly branded 'Certified for Windows Vista', until 2011. Hot on Microsoft’s heels, the troubled Yahoo! corporation announced that it would be shutting down the DRM servers that authenticated its Unlimited Music Store from September 2008. This time, following angry complaints from customers, Yahoo! agreed to refund the purchase cost of any music bought through Unlimited.

DRM is turning into a public-relations disaster for media companies. Customers are increasingly chaffing against the artificial restraints placed on their use of products, and more and more of them are realising that DRM only affects law-abiding customers. Pay for a DVD or Blu-ray and you can’t make a copy for your laptop or your iPod, you’re forced to sit through adverts, you can’t even buy a cheaper version of the disk from another country. Pirate copies of TV programmes, music, movies and video games are available on the Internet; they look identical but have none of the restrictions. Piracy exists because it serves consumers’ desires and until the movie studios make their official disks every bit as attractive as the illegal copies, piracy will continue to thrive.

But it is possible that DRM is nothing more than a passing phase in the media industry. Amazon in the US and iTunes now allow people to purchase MP3 versions of music which can be freely copied between devices. As a testament to its popularity, it took just a few months for Amazon to become the second-largest retailer of music in the United States after the iTunes Music Store and it is growing at a much faster rate than the Apple Store. Despite fears from the industry, there has been no explosion in piracy; just many more satisfied customers and an expanding marketplace. Perhaps it’s time for Hollywood to wake up and begin treating the people who pay its wages like responsible adults.

About the author

Mike Richards joined the Open University in 1996 to help trial teaching over the Internet. Since then he has taught courses ranging from an introduction to robots to the engineering works of Leonardo da Vinci; but has spent most of his time writing about security - everything from the Enigma machines to e-shopping. He is currently working on a new course exploring the world of ubiquitous computers; imagine a world where computers so small and cheap they can be put in everyday objects - smartphones today, smartclothes tomorrow.

Subscribe to Mike Richards's posts

 

Despite what you might think, being in the Internet business isn’t a guaranteed road to riches. In fact Britain’s Internet Service Providers (ISPs) are in a brutal fight for market share; companies have repeatedly cut their monthly charges to customers, but the ISPs’ own costs have been falling much more slowly. So whilst we might be enjoying low cost Internet access, it has been at the expense of profits that could have been invested in customer service, the speed of connections and the necessary investment in new technologies. Price cutting has reached such levels that many ISPs are only marginally profitable and many have been bought up by their larger rivals. Every company is desperate to find a new source of income and they’ve been looking to the almighty Google for inspiration.

Google is fabulously rich, in fact it is one of the richest companies in history, and much of that wealth is built on advertising. When you use Google to search for information, say ‘PlayStation games’; you not only get pages telling you about games for the PlayStation; but also on the right-hand side of the page, a set of ‘sponsored links’ advertising games and game stores. Companies pay Google to place these links on the search results page. Every time a user clicks on one of these links, Google receives a small amount of money from the advertiser. Although each click only brings in a tiny amount of money and only a small fraction of users bother to click on the sponsored links, Google handles hundreds of millions of searches every day – and earns millions of dollars in the process. Part of Google’s success lies in the relevance of the adverts it shows users. When we search for ‘PlayStation games’, Google responds with adverts for games for the PlayStation – not for the Nintendo Wii, Sega Dreamcast or other types of game entirely such as football. Because the results are so precise, we’re more likely to click one of the adverts and Google will get its money. The more precisely the adverts are targeted the more likely people are to click on the links.

And here’s where your ISP comes into the picture. Recently, the three largest UK ISPs; BT Broadband, Virgin Media and Carphone Warehouse’s TalkTalk; entered into agreements with a small company called Phorm which is in the targeted advertising business. Phorm promises ISPs a share of advertising revenues by adding targeted advertising to web pages belonging to their partners in the Open Internet Exchange (OIX) advertising network; every time the user clicks on one of these adverts, a small amount of money goes straight to the ISP.

Phorm’s software uses a feature of your Internet browser known as cookies. These are nothing more than small files placed on your computer by web sites as you browse the Internet. Cookies are not like worms and viruses and cannot harm your computer. They have hundreds of uses, for example, a web site might use a cookie to store your name so when you revisit a site it can greet you personally, or they might be used to hold the contents of a shopping trolley as you browse an Internet store.

When a customer of a Phorm-related ISP first opens their browser, they will see a pop-up window from a so-called ‘Layer 7 Switch’ at their ISP asking if they want to use the Phorm system. If they say yes, a cookie will be placed on their machine containing a unique random 16-figure user number. From then on, all of their requests for pages will first pass through special Phorm computers installed at the ISP. If the user declines to use Phorm, a different ‘opt-out’ cookie is placed on their computer which tells the Layer 7 Switch to direct their browsing straight to the requested sites.

Supposing the user opts in to Phorm; the switch directs their request to a second computer called the Data Mirror which fetches the requested page and sends it to the user’s computer. Simultaneously, the data mirror passes a copy of the page to yet another machine known as the Profiler, and here’s where Phorm really gets to work.

The purpose of the Profiler is analyse the contents of the page. After removing information that could identify the user – such as names, titles, postcodes, email addresses and the like; the Profiler extracts the most frequent ‘interesting’ words describing the content of the article.  These words are used to assign the page to one of a number of advertising categories. This information is then stored in Phorm’s database along with the unique user number taken from the cookie.

When the user next visits a site belonging to OIX (such as the Daily Telegraph, the Financial Times or iVillage), the Phorm system extracts the user number from their cookie and matches it against the record held in the Phorm database. It reads the user’s record to see which categories they have previously browsed; then selects appropriate adverts for those interests. Finally, the targeted adverts are pasted into the webpage seen by the user. If two users with different interests visit the same page they will each see different adverts tailored to their tastes that will hopefully encourage them to click for more information. More advertising clicks, more money for the ISP.

Phorm cannot have expected the uproar their proposal has caused amongst Internet users. The response has been almost universally hostile and although some of the user reaction was sensationalist and paranoid, the situation was not helped by Phorm’s secrecy and the incompetent public relations employed by the ISPs – especially BT, of which more later. In the last few weeks’ Phorm has become more open about its system and it is now possible to say what the system can, and cannot do.

Firstly, Phorm cannot read the content of any page protected by encryption. You use these pages when shopping or banking online and reading email on some sites. These pages always have an address beginning https:// and your browser window will show a small padlock. Phorm also does not read the content of some online forms and it will not read any pages from the most popular web mail sites. Phorm has also promised not to advertise sensitive materials that may offend some users. Phorm does attempt to preserve the anonymity of users by using random user IDs, rather than names or emails. It does not keep the addresses of web sites users have visited, nor does it keep information on those people who do not wish to use the service.

So, the user community was wrong and Phorm was right?

Ummm…

I’m not so sure.

I have a couple of issues with Phorm and the ISPs. The first regards their so-called ‘common carrier’ status which is nothing more than a fancy way of saying the ISPs don’t read everything that passes through their system. ISPs have been vociferous in claiming that they are common carriers and cannot be expected to police the Internet for pirated software, video and music; and nor can they be expected to check every email message for libellous or illegal content. Their policy is so strong that the police must obtain permission to look for material relating to terrorism or child pornography. However, Phorm shows the hollowness of this argument; the ISPs are prepared to examine Internet traffic and categorise it. If they can do it for advertising, why can’t they intercept pirated movies or child pornography? In allowing Phorm, the ISPs might just open the floodgates to ambitious politicians who have long craved the mass surveillance of the Internet. Such surveillance might prevent or detect crimes, but it could also endanger our freedom of speech and expression.

The second issue I have with Phorm is that it may well be illegal under United Kingdom law. The turgidly-named Regulation of Investigatory Powers Act – RIPA - (2000) was passed by the present government in an attempt to formalise the procedure of obtaining material needed in civil or criminal cases. RIPA makes it illegal to intercept a communication without a legal warrant; or the explicit permission of the person sending or receiving the message. Some experts  in Internet law believe that Phorm is performing an interception by diverting users’ browsing through their computers, and if this is the case then the law may well have been broken. Certainly, the growing consensus from government bodies is that users must deliberately ‘opt-in’ and grant permission for Phorm to read their data.

And this opt-in might well be the single biggest blow to Phorm’s ambitions. By default, all users of a participating ISP are opted-in to Phorm; they must choose to opt-out. However, it has recently been revealed that over the last two years BT has conducted secretive trials of Phorm technology on up to 108,000 BT Internet customers. All of these customers were automatically opted-in to the trials with no ability to opt-out; nor were they informed about the trials. If (and it is still only an ‘if), Phorm’s operation only complies with RIPA when customers opt-in, then BT may be in very hot legal water and facing potentially unlimited fines. BT appears to realise it is in murky waters as it plans to change the legal terms and conditions of its Internet service before Phorm (which they call WebWise) can be deployed to all users later this year.

Certainly, Phorm hasn’t benefitted from the controversy. Its share price has halved since the announcement of its tie-up with the ISPs. A number of advertisers in OIX have refused to take part in the scheme, and TalkTalk has said it will make its use of Phorm an opt-in service. It is entirely possible that Phorm will not survive, but it’s not the only company trialling this type of technology, only the best known. In the United States, a number of similar companies have been secretly tracking the browsing habits of more than 100,000 Internet users for some time now and see no reason to stop doing so. Clearly targeted advertising is an attractive proposition to ISPs and unless it is deemed illegal in the UK, we’re going to see a lot more of it.

If Phorm, and Phorm-like systems are not deployed, then the underlying problem for ISPs remain – the current consumer charge for broadband is too low. If we want good quality, high-speed Internet access and investment in new technologies then we will have to pay for it; either by higher monthly charges, a charge per megabyte we use, or through advertising revenues.

At the end of the day, we each have to make a choice – do we want truly anonymous, more expensive Internet; or cheaper surfing where the advertisers know all about us?

About the author

Mike Richards joined the Open University in 1996 to help trial teaching over the Internet. Since then he has taught courses ranging from an introduction to robots to the engineering works of Leonardo da Vinci; but has spent most of his time writing about security - everything from the Enigma machines to e-shopping. He is currently working on a new course exploring the world of ubiquitous computers; imagine a world where computers so small and cheap they can be put in everyday objects - smartphones today, smartclothes tomorrow.

Subscribe to Mike Richards's posts