How I recovered my Orico NS500RC3 RAID 5 setup

Been trying to fix my Orico NS500RC3 Raid Box, JMicron JMS567 based USB RAID box, just sharing my notes here. I have 5 x 12TB drives installed as a RAID 5 setup. Been using it for about 5 years. About time to expect for hard disk failure. Been at the back of my mind for next upgrade, but just been lazy, so I had to pay the price to fix it in the past week or so.

“Failure is often unexpected and rarely explained in any manual.” 🙂

First serious attempt to rebuild failed after 15 days 17 hours, got to around 80%. Rebuilding RAID 5 is very very slow. Got about 5% a day, so it would have taken about 3 weeks, I think. Failed because of Automatic Update. Forgot to turn that off and Windows just did what it did.

Starting another attempt with my Mac. Which has a better uptime record. But the problem is, I can’t see the progress if it is running on the Mac, can just do it blindly (the RAID software doesn’t run on latest Mac). Will see if I can keep the system running for another 3 weeks or so. I might attempt to copy the data off too.

It’s probably faster to clone/backup the data from degraded RAID, do a fresh destroy and rebuild, then copy back.

What happened? I came home, noticed that hard disks were “unmounted” from my Mac. Quite a common to if you just unplugged the USB cable or any power failure. But when I plugged it back in again, there was a very loud (ear piercing) alarm sound. But all the lights are normal, all blue. So I didn’t suspect a hard disk failure. But the alarm sound was very loud. I had to find other ways to troubleshoot the issue.

Unfortunately, the HW RAID Manager software is no longer supported on current macOS. It claims to support up to Big Sur (macOS 11), which I don’t have any machine running on that handy. So I switched to my PC version for some testing.

Initially, when I connected to both Mac and PC, nothing showed up in the system. The whole RAID was unresponsive, it powers up and the hard disks are spinning, but no USB connection recognised by the system. Which was very scary. 48TB of data is quite a lot.

Then I took all the disk out, cleaned it up a bit, and plugged them all back in. Remember that disk order might matter, so label your disks if needed. Always plug it back in the same order. The system seems to recognise the RAID 5 device after this step. And then I can see in the PC software that Disk 4 failed. The lights on the front panel are still all blue, not indicating any failure of Disk 4. But the software clearly identifies the error and the RAID 5 is in the “Degraded” status. I tried to connect the device to my Mac, and the drives are still there, which was a relief.

But the alarm sound was still annoying me a lot. In the PC software, there is a button to mute the alarm sound. No Mac option to stop the sound.

Now that I know it is the disk failure. I quickly bought a replacement drive. Couldn’t find any brand new one of the same model, bought a second hand one instead, watched out for the total powered up time to be less than the ones I am using. I thought it would be simple, just plug in and leave it running. But that wasn’t so.

Kind of my fault. Initially, that’s how it was, I bought the “new” disk. Formatted it, plugged it in, the light on the front changed colour and the status in the PC software changed to “Rebuilding”. But as I tried to move the device back to the shelf so I can leave it running for a few days, it got disconnected, not sure if it was power or USB disconnect, but I can see the lights flashed like a restart of the RAID (I “swear” it wasn’t me, haha.). And then the drive didn’t start rebuilding.

In the PC software, I started seeing Page 0 Damage Error on the new disk. When I unplug the disk and tested it separately on other computers, it works just fine. But it doesn’t work in the RAID box. After searching around online, found someone using similar JMicron based RAID had similar issue. After playing around with it a bit, got the error fix, see below. So I got the disk ready for rebuild.

Good news is that, when the disk is rebuilding, the alarm sound is automatically muted, no software needed. Bad news is, if you disconnect or power off your RAID box during the rebuild, you will be back to the Page 0 Damage error stage again.

So, I unplugged the RAID from my PC for the last time. Fix the Page 0 Error again. Get ready to do it all on my Mac. With all the disks, including the newly prepared one, plugged in. I connected the USB to my Mac. Powered it up. After the initial alarm of a failed/degraded RAID, the light for Disk 4 changed colour and started flashing. Alarm sound is off. Volumes are mounted as before on my Mac. Now it is just a matter of keeping the RAID running for a few days, seeing how long it takes to fully rebuild. Reminder, turn off any energy saving features on your Mac. Make sure it won’t get turned off or disconnected at all for a few days. Good luck. Will report back when I know how many hours it takes to rebuild my 12TB disk.

Page 0 Damage Error Fix

If you get the Page 0 Damage Error on the new disk. It can be caused by residual RAID config data left on the disk. You can’t just format the disk to get rid of these, as they are not part of the partitions. On Mac/Linux, you can use DD to write zeros directly to the beginning and end of the disk. (I didn’t do enough testing, but I did zero out the first and last 70k blocks, default block size, then it worked.)

## Use diskutil to list all drives
diskutil list 

## Use diskutil to find out how many blocks your specific disk has, mine was disk10 (use your own disk number found above)
diskutil information /dev/disk10

## Unmount the disk before writing to it
unmountDisk /dev/disk10

## Use dd to write zeros to the first 70k blocks
sudo dd if=/dev/zero of=/dev/disk10 count=70000

## Use dd to write zeros to the last ~70k blocks, use the Disk Size information found above, should have the exact number of 512-Byte-Units, take that number and minus 70k. Mine, 12TB disk, was 23437770752 Units, so I am writing zeros from block 23437700000 till the end.
sudo dd if=/dev/zero of=/dev/disk10 seek=23437700000

## Mount the disk, it will then prompt for initialise, just format it before putting it back into the RAID
mountDisk /dev/disk10

https://forums.freebsd.org/threads/raid-how-to-fix-a-page-0-damage-error.52187

Other tips to share:

  • Remember to upgrade/replace your RAID or hard disk every 4 years or so, aim at 30k hours for the disk.
  • RAID (certainly RAID 5) is not backup. Should regularly backup data, every 6 months or so.
  • Use cloud for live/working data. Use your own storage for medium/long term data storage.
  • Rebuilding large RAID can take days. Just backing up of data can take hours and hours. USB 3 is often limited to just 5gbps, which is slow. Assuming perfect USB 3.0 connection, will still take 5.5 hours to copy 12TB of data. USB 3.1/3.2 should be faster, but then it will be limited by SATA of 6gbps. Generally, hard disks are “slow”. SSD drives will be faster, but expensive.
  • Can use this site to estimate your copy/rebuild speed, but that is assuming perfect condition, multiply it by 3-5x. https://hdd.userbenchmark.com/
  • Avoid buying RAID box with no jumper (hardware switch) setting of RAID mode. Relying on software to configure the setup can be additional risk. Software not being maintained by manufacturers, etc.
  • I know you can use bigger disk to replace faulty ones, but somehow I still prefer to go for the same make/model for a matching set. Harder to find these after a few years.
  • Do not rely on official customer support, they will just give you standard answers, usually just copy and paste info from the manual. They usually won’t go out of their way to solve your problem.
  • It’s not easy to keep a computer running for a few days, need a desktop computer and turn off all energy saving features. That’s why I like Mac Mini or even Mac Studio, they are relatively low power, using laptop like components for desktop/continuous usage. But usually, support for “cheaper” external hardware can be tricky on Macs.

Other References:

idVendor 0x152d JMicron Technology Corp. / JMicron USA Technology Corp.
idProduct 0x0567
iProduct: USB to ATA/ATAPI Bridge