Another R3 Data Recovery Structured Data Lab and Security House combined effort in the battle to protect data. π
βοΈ “You guys have been sterling this weekend and I wouldn't have got through it without you.” βοΈ
Friday Morning - Another Business Critical Emergency call to 0800 999 3282.
The background story
A company supplying industry cannot process day to day tasks, although a storage hardware problem which was planned for replacement.
Staff fear it is a cyber attack amidst the news of cyber warfare escalation (there are more than one way to analyse a problem, find the cause - get the right solution - that goes for disasterous wars as well as disater recovery).
Having bought a new Synology SAN in preparation for migration the client ran into a problem with a Drobo Tiered Storage that took 10TB of VMs offline, crippling a company that was facing a difficult weekend and week ahead.
The Call
“Do you do data recovery?” Yes β
“Do you do Drobo, Inc. Data Recovery?” Yesβ
“Recover Western Digital SAS Drives?” Yes β
“Experience with ESXi VMware?” Yes β
“Recover data from within corrupt VMs?” Yesβ
“Can you recover data from failed SSD?” Yes β
Team R3 coordinated with the client to get the system to Security House Friday afternoon.
After arrival
6 year old WD SAS HDD Imaged with Zero Bad Sectors. π
256GB Crucial SSD Bad sectors and power management issues - RAID affected. π°
Diagnosis in first call confirmed soon after arrival at Security House - SSDs faulty / failing.
Data protection critical
Ordered 3 sets of 3 matching SSD from different suppliers paying extra for Saturday delivery - none arrived! Delivery tracking said one unable to deliver, the other out for delivery.
Site has security controlled and camera monitored access 24/7. π₯
All three sets fail arrive, ultimately two sets later arrive on Monday too late. β±
3rd set - still yet to arrive at time of writing. π
Client calls Scan Computers they had them in stock and collection made on a Saturday Afternoon. π
R3 dedicated driver brought them back to Security House for 5:30pm.
Images previously recovered from failing SSD written to new SSD and put back with WD drives into the drobo and sent back Saturday Evening to put onto network and brought back online. π€
It was processing the RAID rebuild. β
Decision made to leave overnight in the knowledge that all images were safe back at R3 if a manual recovery was needed.
By Sunday Lunchtime RAID had rebuilt, volume accessible systems coming back online. π
Sunday afternoon all virtual machines confirmed ready to bring back online except Domain Controller (Affected by Dirty Shutdown).
Using a backup DC brought back to fully operation Sunday afternoon. π
Mission accomplished
Migration to Synology rescheduled for next weekend.
Team R3 are on the frontline of the battle to defend data and those affected by data loss.
R3 = R3vived β R3covered β R3stored β