The user had a Netgear ReadyNAS with multiple disks showing 'bad sectors', and it was also running badly. In response to this, the customer added another drive hoping it would rebuild and stabilise itself. Once the expansion started, the array never got back online.
← All case files · from £500 + VAT
This is a scenario, not a case study. It illustrates a common failure type that we address on a regular basis, and how it is approached, written from the technical side (versus as part of a single customer project). Please visit our documented case files here.
When a NAS is running low on space, you replace one of the current hard drives with a bigger one, or add a whole new one. The NAS will then start expanding its volume. This expansion process, depending on how big the drives are, could take anywhere from hours to days. During the time that this expansion is taking place, something interrupts it, such as a loss of power, a problem with one of the disks, or a reboot. When this happens, the volume doesn't come back.
Adding a member to an existing RAID is not simply appending additional space to it. It is an entirely different type of operation. The stripe width changes when you add an additional disk; therefore, each stripe on the entire volume must be read again, recalculated and then rewritten in its new stripe layout. In essence, the physical configuration of the array is completely rearranged while still maintaining online access.
As this happens, both geometries are stored on the disks at once. The area of the disks which has been processed into the new geometry is indicated by a pointer, and the rest still uses the old one.
If that pointer is gone or the process ceases operation mid-stream, then there exists an array in an unbuildable configuration for any controller. One-half of the total volume is arranged in one manner while the remaining one-half is laid out in another, and nothing anywhere within the unit can identify where that boundary lies.
Nothing was lost; each of the stripes remains intact either in one arrangement or the other, and the position of that border can be determined through identifying where the pattern begins to vary.
To reconstruct the array is to determine both configurations, locate the transition area, and assemble the entire volume in two portions, namely the converted region that will be read using the new stripe dimensions, and the remaining portion that will be read using the original dimension. From the perspective of a file system located above this, once properly reconstructed, there appears to be a single contiguous volume.
While reconstruction may prove difficult, it is not uncertain, and it is not the complexity that causes failures. Rather, what results in failure is additional attempts to resume or restart the expansion, since these result in writes into a format that the unit has interpreted incorrectly.
In most cases, yes. The disks are removed and their contents are read into an image. This is performed read-only. The array is then reconstructed outside of the enclosure. As such, even a failed NAS controller or power supply is no longer the obstacle. Layouts based upon proprietary arrangements such as Drobo's BeyondRAID are derived from those images and do not require reading them from the appliance.
Take all of your drives out if possible, labelled by their bay number before you remove them; otherwise, send us the NAS. This is important because we will need to know the exact order they were in originally for our reconstruction. Please do not attempt to run a repair or rebuild through the NAS interface prior to sending this to us.
The cost for recovering your NAS/RAID/server begins at £500 plus VAT. The costs are confirmed within your written quote after a free 48-hour diagnostic. If your recovery requires physical work to be performed on your drives, we require a 50% deposit prior to starting work on those drives. The remaining 50% balance is only due upon successful completion of your recovery.
When attempting to resume an interrupted expansion, you are asking the system to start from a point that it may no longer be aware of. Simply power off the unit instead.
If you change a member (disk) during the middle of an expansion, this creates even more problems. Send the array exactly as it sits today, complete with the newly added disk.
As well as the original order of how the disks were initially configured, the disk that was added is equally important, because it only holds data within the converted region.