Servers are not normally subject to failure as servers. In general they will have experienced failures as arrays, usually having lost more than one member while the redundancy was still being reported as “healthy”. The immediate need to address the outage creates a sense of urgency to do something quickly. Most of the damage that we observe was done during the first sixty minutes after the outage, by someone who was attempting to assist. Shut it down, leave it alone, document the bay sequence, and call us. There is no cost associated with diagnostics, and the price that follows is fixed in writing.
The time frame of an outage creates the desire to take action immediately. Almost all irreparable mistakes made on server-related projects were completed in that first hour while an individual was attempting to be helpful.
If a server with a failing array is left running, then whatever protection that server still has is being used up. Shut it down properly (if possible) or pull the plug (if not), and then stop.
The bay sequence is part of the array's structure, and documenting it is required prior to removing any component(s). Photographing the chassis and labelling each caddy before removal takes only two minutes. This documentation will save hours in the long run.
Rebuilding onto a new disk puts every member that survived the previous failure under load across its full surface for hours. When this occurs on disks of the same age and production batch as those which have already failed, it is then that the next disk will fail.
Reinstallation of the operating system, a file system repair, or letting a controller initialise a foreign configuration will overwrite the needed structures.
The chassis itself is mostly irrelevant. What matters most about your server is the RAID contained within it, and how long this RAID has been running without any form of protection.
You are essentially looking at a metal box containing one or more drives and a controller. When a server fails, it nearly always fails due to issues contained within the RAID and not because of problems with the physical box itself. Also, the RAID has likely been in a degraded state for significantly longer than those responsible for maintaining it have realised: the initial drive failure occurred quietly, and then there was redundancy built in to absorb the failure. Additionally, alerts were generated from the array failure but sent to a mailbox that no longer exists.
Small enterprises also have another typical pattern of data loss which surpasses data loss due to hard drive failures: backups were written to shares located on the same RAID as the primary storage location. Therefore when the RAID failed, so did the backups. It is worth checking where your backups actually write to before you need to know.
Cache modules and battery packs. On HP ProLiant and similar servers, the controller will refuse to allow an array to be brought online when a cache battery fails and writes remain in volatile memory. The failure appears to be complete, but frequently it is not: the disks are fine and the array can be reconstructed.
Corporate/back-office servers come from Peterborough and Milton Keynes (insurance, claims, finance and payroll systems, and document management under statutory retention), while professional practice servers come from Chelmsford and Bury St Edmunds (case files and land record files). They are standard hardware, with standard maintenance, usually failing due to age of disks rather than an event.
The third source differs in its type. The process and production servers belonging to food manufacturers and packers across the Fens and along the A14 run their batch, recipe and traceability systems, which a supplier audit will suddenly require. These arrive with deadlines based upon a contract instead of a project, and often they are running on an operating system that has been out of support for several years.
Recovery of the server and/or arrays begins at £500 +VAT, independent of the number of members, RAID levels and controllers. Extraction of databases and mail stores from the recovered volume is also included as part of this service (and is not an additional charge).
In order to recover your data, take the disks out and send them. Please label them by bay. We cannot accept the server or the case. On the phone call, please provide us with information about what controller model your server uses. Knowing the controller generation reduces the time required to perform the reconstruction.
Be sure to leave the server powered down. After a power event, many times there will be a cache module holding write operations that were never written to the disk(s). As a result, the controller will refuse to assemble the array. In most cases, the disks will be intact and therefore the volume will be able to be reconstructed.
This is common if a chassis has moved or a backplane has failed, and it doesn't mean your data has been lost. When prompted by the system to import or wipe out your configuration, do not select either option. Importing or wiping your configuration will write new metadata over the existing metadata. Send the disks as-is.
More commonly occurring than anybody likes, which is the reason for total loss normally being due to a backup decision rather than hardware. The array is still recoverable; this simply indicates that the recovery option is now the only way back to your data instead of a convenience.
Yes, and this is often the quickest option back to production. Once the volume has been rebuilt, we can then extract each component (SQL, Exchange and file shares) so the most important part of the system comes back into operation before the others.
No, only the disks: we cannot take in the server itself. The disk(s) and their bay positions are very important, but the chassis and motherboard rarely are. If you have knowledge of the controller model, please let us know, because knowing what generation of controller you have reduces the rebuild time.
The diagnostic takes 48 hours. Reconstruction is typically 5-10 working days, depending on how long it takes to image larger disks. Please tell us if there is a business deadline we need to work to and we will be truthful regarding whether the deadline is achievable.
Clients are charged from £500 +VAT; enterprise SAN environments are quoted separately from £1,250. A named contact from diagnostic through to handover. We don't collect: drop the disks off or post them insured.