Worldwide internet blackout was caused by ONE Fastly customer

Worldwide internet blackout that saw sites including Amazon, UK government and Spotify crash was caused by ONE Fastly customer changing their settings

  • Yesterday’s huge internet outage has been blamed on a single Fastly customer
  • US cloud-computing firm responsible for outage said a software bug caused it 
  • It said the software bug was triggered when a customer changed their settings
  • Outage saw hundreds of websites crash including Amazon, Spotify and Netflix

The massive internet blackout which brought down hundreds of websites across the world has been blamed on a single unnamed IT customer.

It left millions of people unable to access a host of major sites including Amazon, Spotify and Netflix, as well as the BBC, UK government and the White House. 

Yesterday’s outage was caused by a software bug triggered when a customer for Fastly – the US cloud-computing company responsible for the problems – changed their settings, the firm said.

Scroll down for video

Source: The internet blackout which brought down hundreds of websites across the world has been blamed on a single customer of Fastly, the US company responsible for the issue (stock)

Which sites were hit?

The outage saw visitors to sites including the UK government’s pages receive the message: ‘Error 503 Service Unavailable’

Many popular websites were affected by yesterday’s outage, including: 

– Amazon

– Spotify

– Reddit

– Netflix

– gov.uk

– PayPal 

– Twitch

– Stack Overflow

– GitHub

– Hulu

– HBO Max

– Quora 

– Vimeo

– Shopify

– Stripe

– CNN

– The Guardian

– The New York Times

– BBC 

– Financial Times

It has raised questions about the reliance of many of the world’s biggest websites on just a handful of firms who run the infrastructure underpinning the internet. 

More than half of the web’s traffic is served by a content delivery network (CDN) company such as Fastly, which helps users view website content more quickly. 

The aim of CDNs is to reduce latency – the delay from the moment a user makes a request to the exact instant they receive a response. The higher the latency, the worse the user experience. 

But if the service suffers a failure, as Fastly’s did, it prevents the companies that use it from operating on the net at all. 

In a blog post on its website, Fastly apologised for the outage – which lasted about an hour – and said it should have anticipated it.

Nick Rockwell, senior vice president of engineering and infrastructure, said: ‘We experienced a global outage due to an undiscovered software bug that surfaced on June 8 when it was triggered by a valid customer configuration change. 

‘We detected the disruption within one minute, then identified and isolated the cause, and disabled the configuration.’

He said within 49 minutes, 95 per cent of Fastly’s network was operating as normal.

The company has vowed to carry out a ‘post mortem of the processes and practices we followed during this incident’ and to ‘figure out why we didn’t detect the bug during our software quality assurance and testing processes’. 

‘This outage was broad and severe, and we’re truly sorry for the impact to our customers and everyone who relies on them,’ Mr Rockwell added.

MailOnline has contacted Fastly for further information on the ‘customer’ behind the outage.  

The blackout saw visitors to a vast array of sites receive error messages including ‘Error 503 Service Unavailable’ and ‘connection failure’.

Streaming sites Netflix, Twitch and Hulu were also hit by the problem. 

Some sites including the UK government website were offline entirely, while others such as Twitter had more specific errors, such as not showing emojis. 

Among those affected were travelling Britons trying to fill out locator forms on gov.uk to enter the UK from Portugal and abroad. 

Former deputy national security adviser for intelligence and security between 2014 and 2018, Paddy McGuinness, said yesterday’s incident should serve as ‘a wake-up call’, and the UK government must expand its current security approach.  

‘We need resilience as an explicit policy goal, especially on the new networks we are building to deliver services to the citizen,’ he said.

‘A ”secure by design and default” mantra is welcome but it isn’t enough in itself.’

Many of Fastly’s customers are news sites that use its technology to update their websites with breaking news. 

Buzzfeed, for example, used Fastly to cut the time its users took to reach the site by half. 

Fastly had $290.9 million in revenue last year. 

Issues: A number of major websites crashed yesterday following a massive internet blackout. The outage tracker site DownDetector picked up the problems users experienced (pictured)

What is Fastly and why did an issue with it affect so many websites? 

Fastly is a content delivery network (CDN) that makes content transmission faster between websites and consumers. 

For CDNs, the goal is always to reduce latency – the delay from the moment a user makes a request to the exact instant they receive a response. The higher the latency, the worse the user experience.     

For example, when you load a page on a server on the other side of the world, it will take hundreds of milliseconds to get the page. 

Over time, this latency adds up, resulting in a sluggish consumer experience. 

However, when sites use a CDN like Fastly, they can start sending the content of the page in less than 25 milliseconds. 

Fastly is used by a range of popular websites, including several media sites such as the Guardian, New York Times and Buzzfeed. 

This means that when Fastly suffers a ‘disruption’, it affects data centres worldwide.   

Jake Moore, a cybersecurity specialist at ESET, explains: ‘Web pages are located all over the world so content delivery networks are placed to distribute the data evenly by reducing the physical distance between where it’s actually held and the end user.

‘This helps users around the world view the same high-quality information and content without any lag or slow loading times.’ 

Source: Read Full Article