Bouncing back from cyber attack

NO ONE wants to have that sinking feeling as IT systems fail or freeze and you realise it’s isn’t just you or it isn’t just the computer ‘acting up’. But a sinking feeling does not have to leave you sunk.

What happened

A ransomware attack hit noted IT services provider Blue Yonder on November 21, just ahead of Peak. The attack centred on the IT company’s managed services hosted environment. This had ramifications for many of Blue Yonder’s customers, for example Morrisons.

How Morrisons’ logistics operations responded

Morrisons group director for logistics, supply chain and technology Ross Eggleton explains: “We lost WMS over fresh produce and bread - 21 warehouses, 7 geographical locations, with total network down and system wide paralysis. We had 1 million cases on the floor, and half a million on supplier vehicles waiting to come in…”

It’s a truly terrifying scenario, particularly as the UK retail sector was coming into Peak as the attack occurred. So, what next?

Ross takes up the story: “We pressed continue… we cut a plan.

Day 1 - we create a manual contingency, go back to the 90s, creating picking sheets.

Day 2 - we get the sites back on time and re-align the ops clock, we apportion stock to stores to claw back lost capacity. Storm Bert hits just to keep us on our toes.

Day 3 - we do a full stock take to re-set the depots but run to time.

In parallel we set about creating a new WMS, configure it and host it in-house from a standing start. New servers, new tin.

Day 4 - live in our first site, all three chambers on a new WMS. We build the WMS for site 2 by cloning and reconfiguring site 1.

Day 5-10 we repeat, converting a site per night. No gap between implementations.”

Alongside this, Morrisons re-counted all 500 stores, all categories affected, to re-set its stock record. It aligned inbound increasing demand daily and flowing stock to each store and item combination needed.

All in all, it was a massive effort from the entire team at Morrisons. How does Ross reflect on the effort?

"Was it perfect? No, but we re-set the business and replaced all systems in 10 days, maintaining business continuity throughout and stepped on availability every day from the lowest point.

“I am incredibly proud of all the people who made this happen. They bought into the plan on day 1, they believed in it. We had a clear objective for each day of the recovery and a milestone to hit. They didn’t loose faith (in me or the plan) but each day became stronger and more creative. Teamwork and resilience was off the charts."

“The comeback is always greater than the setback,” insists Ross.

What Blue Yonder said

In terms of the broader attack, how did Blue Yonder respond? Basically, the firm pulled out all the stops to minimise disruption and restore service. At the time of going to press, the incident was still under investigation so Blue Yonder did not comment other than to give a statement to customers saying: ‘The Blue Yonder team is continuing to work around the clock, together with our external cybersecurity firms, to safely restore systems, resulting in steady progress. Our investigation remains ongoing, but please know that our priority is to ensure a safe and secure recovery. At this point in time, we do not have a timeline for restoration.’

Expert view

The ransomware gang Termite has claimed responsibility for the attack. Blue Yonder has not confirmed this, but if they did carry out this attack, how did they get in?

Bridewell cyber defence technical lead Gavin Knapp says: “There is limited information regarding the initial access vector used in this attack. However, the most common initial access vectors being used by Termite in recent reporting are phishing and exploitation of external facing vulnerabilities.”

Recent reports indicate Termite exploited a zero-day vulnerability in Cleo's LexiCom, VLTransfer, and Harmony file transfer software. 

Gavin continues: “The zero-day relates to CVE-2024-50623 and the vulnerability is still being actively exploited in the wild. Systems included those fully patched running 5.8.0.21 are still exploitable.

This approach has been popular with ransomware affiliated threat actors. It is possible Termite used the same tactics to gain access to Blue Yonder’s environment. Potentially deploying web shells after initial exploitation of external facing devices.”

Conclusion

Ransomware attacks are common. According to Verizon’s 2024 Data Breach Investigations Report, roughly one third of all breaches involved ransomware or some other extortion technique. The Report says there was 180% growth last year in attacks exploiting vulnerabilities, with these attacks primarily leveraged by extortion threat actors. According to the FBI, the median loss associated with ransomware and other extortion breaches is $46,000.

But as Morrisons' logistics operations shows, hope prevails. It shows that due to the great resourcefulness of UK logistics teams companies can respond to a crisis and quickly re-launch operations with new IT frameworks.

For detailed recommendations on cyber attack prevention, response and resilience from Gavin Knapp, turn the page.