Derbyshire runs out of space

30 January 2014

Derbyshire County Council (DCC) works in partnership with nine district and borough  councils. It has in excess of 30,000 staff who help deliver a huge variety of services to a population of 750,000 people.

Given the complexity of its business, DCC generates enormous volumes of data. But this had led to problems as data centre manager Rob Skermer explains: “We first implemented our SAN in 2003 and it was scaled for 2TB. Today, for each environment, we’re  looking at 250TB, so growth is more than a hundred-fold. Keeping pace is difficult.”

By 2012, Derbyshire’s existing storage solution was running out of capacity. It didn’t provide file services directly and required many Windows Servers to be connected to deliver the bandwidth and performance for 8,000 users. Data replication and file server failover was complex and often found to be unreliable. When backing up  the file servers an entire weekend was required due to the many millions of files, many never being touched or changed.

Skermer says the council was about to exceed capacity and wanted to avoid both penalties within its existing contract and having to spend capital on an interim solution. “We had to implement fast, over four weeks,” he says.

DCC then turned to its IT partner Phoenix. Following consultations, it found that the council had a perfectly good storage array that excelled in some functions but failed in others. Its approach was to “right-size” the solution and implement functional storage tiering – moving a large quantity of low IO file storage to a more optimised unified platform.

A dual-site NetApp storage system was implemented. This consolidated multiple file servers onto a single unified platform, using high capacity SATA drives and with storage efficiency features including thin provisioning, data de-duplication and compression.

NetApp Snapshot technology meant millions of tiny files could be backed up in seconds and retained for four weeks, with tape used for long-term retention to meet governance requirements.

NetApp is also used to replicate data offsite to a secondary site. “We’ve enhanced failover, improving our resilience and recoverability options: using NetApp we have failover in 10-15 minutes from  one site to another and it’s invisible to users,” says Skermer.

He adds that the solution is modular which means scalability. With the NetApp controllers, we can swap-out without any service  interruption, attaching more storage. So it’s sustainable – one of our original requirements.”

All this has ultimately enabled DCC to free up 20TB of tier 1 storage. The new efficiencies have reduced storage outlay by up to 50 per cent, while the backup window has shrunk from 48 hours to seconds. Reduced tape handling has improved service levels and also cut the cost of IT operations. In addition, greater efficiency of IP LAN and WAN bandwidth has also been achieved by using the Network Data Management Protocol.