The whole point of a two disk raid1 setup is that I should be able to lose one hard drive right?
With great confidence I removed the defective disk from my QNAP TS210 and ran the machine on only one hard drive.
Complete success - I could still access all the files and everything looked good!
Everything worked fine. So, time to fit the new hard drive.... or maybe not. In my excitement I forgot to power down the machine properly, and just pulled the power out of the back of the machine. I had yet to do the whole read only root magic and so the machine booted up with the dreaded file system errors, and fstab was still in the default remount=ro for the root partition so the machine failed to get a valid ip address from the router.
Argh!
I quickly removed the hard drive from the nas, put it into my usb sata drive adaptor on my desktop and ran fsck on the root drive there. After a bit of back and forth and reading the /var/log/daemon.log I could see that the dhclient was trying to get an ip address from the router but something was not quite right.
Onto the topic at hand - I quickly (i.e. asked my awesome partner to remind me syntax of awk) made a script to make sure that something like this should never happen again.
And here it is - I called it backupip and call it from the /etc/rc.local script:
#!/bin/bash
# Sleep for 100 seconds
sleep 100
# Parse ifconfig and see if we have a valid ip address
a=$(/sbin/ifconfig eth0 | grep "inet addr" | awk 'BEGIN {FS=":"}{print $2}' | awk '{print $1}')
# If we have no ip address give the nas this ipaddres
if [ -z $a ]
then
echo "no ip address"
ifconfig eth0 down
ifconfig eth0 192.168.1.2 netmask 255.255.255.0 up
route add default gw 192.168.1.1 eth0
fi
Not exactly the best script in the world - but it's simple and now I managed to ssh into my system and get to the bottom of the problem. (which was a failure of my separate /var drive starting up soon enough.)
Note: I'm pretty sure I can get the script to only use awk but I was in a bit of a rush!