Excuse the length of this message but I thought the following might
be of interest to other Varian users who are upgrading their workstations
to Sun Blades with IDE drives. Since Solaris 9 includes RAID support out
of the box now and drive quality is uncertain these days, we decided to
add a second identical IDE drive to each Sun Blade 1500 and set up a
RAID-1 mirror. The instructions below describe how this is done and how
one can replace a failed drive and rebuild the RAID-1 mirror. I've done
this with two workstations so far and it is working out quite well.
Jack Howarth
------------------------------------------------------------------------
Configuring a RAID-1 mirror on IDE drives under Solaris.
WARNING: Complete data loss can occur if mistakes are made while
setting up the RAID-1 on an existing drive. Make sure you have
a complete backup of your data. That said these instructions, if
properly followed, should allow you to convert an existing IDE drive
to a RAID-1 mirror with the addition of a second drive without
data loss.
The following procedure has been tested under Solaris 9, but should
also work for Solaris 2.6, 7 and 8 with the DiskSuite software.
Two IDE drives, each master of an independent IDE bus, are required
in order for the RAID-1 mirror to boot if either drive fails. Thus
the CDROM will then have to be moved to the slave position on one of
the IDE buses. The jumpers on the CDROM drive should be changed from
'master' to 'cable select'. The devalias for the cdrom will also need
to be corrected. To do this, hit stop-A during the boot process to
obtain the "ok" prompt and enter (assuming the CDROM is moved to
the slave position on the second IDE bus)...
nvalias cdrom /pci_at_1e,600000/ide_at_d/cdrom_at_3,0:f
Also execute....
nvalias disk1 /pci_at_1e,600000/ide_at_d/disk_at_2,0
from the OpenBoot prompt. This will make 'boot cdrom' functional again
as well as allow booting from the second RAID-1 mirror disk with
'boot disk1'. Note that exact pci address of your devices may differ
depending on your machine. Check the current devalias for cdrom and
disk first for comparison.
1) Assuming that Solaris has already been installed, a metadb slice
of ~15 Mb must be created on the each IDE drive using any free space
left in the partition map. Otherwise, we can steal the necessary space
from the swap slice using the method described below.
***** WARNING: The following procedure can cause complete data *****
***** loss if improperly executed! Backup your data! *****
a. Execute "swap -l" and note the swap devicename returned.
b. Detach the swap with "swap -d <swap_device_name>"
c. Execute "format" and select drive "0". Select the "partition" menu item
and then select the "print" menu item to view the current slices. Noting
the slice number for swap, enter it (e.g. "1") to edit the swap slice.
Carefully adjust the ending cylinders for the swap slice to reduce its size
by 15-20 Mb. For example, if the swap slice spans cylinders from 0 to 500,
change the ending cylinder to "490e" when prompted. After the swap slice
has been reduced in size, select one of the unassigned slices (one without
any cylinder assignments). Edit the chosen unassigned slice (eg "3") to
have a starting cylinder of 491 and the ending cylinder at "500e". At this
point, view the partition map with the "print" command. It is critical that
the adjusted swap slice and the metadb slice be contiguous and together start
and stop at the same cylinders as the original swap slice. No other slice
should have been touched. Use the "label" menu item in format to write out
the partition map to drive 0.
d. Now you should be able to add back the swap with...
"swap -a <swap_device_name>"
2) The second IDE drive, placed as master on the second IDE bus, can now be
partitioned identically to the first with...
prtvtoc /dev/rdsk/c0t0d0s2 | fmthard -s - /dev/rdsk/c0t2d0s2
3) Use the format program to verify that second drive is now identically
partitioned to the first drive.
4) Create the metadb replicas, 2 per disk, with the command...
metadb -a -f -c 2 c0t0d0s3 c0t2d0s3
....(assuming that slice 3 was allocated as the metadb slice in step 1)...
5) Create the concats for the mirrors. Since the notation for this is not
standardized, I will use the following convention. The concats created from
slices on each drive will be labeled as "d<drive_number><slice_number>"
where drive number starts at 1. The actual mirrors will be labeled as
"d<slice_number>". This approach allows you to track which slice of the
drives is being mirrored under RAID-1. Thus, to setup root, swap and home
concats (submirrors) from slices (s0, s1 and s7 respectively) on the first
IDE drive, we execute...
metainit -f d10 1 1 c0t0d0s0
metainit -f d11 1 1 c0t0d0s1
metainit -f d17 1 1 c0t0d0s7
...and for the second IDE drive we execute...
metainit -f d20 1 1 c0t2d0s0
metainit -f d21 1 1 c0t2d0s1
metainit -f d27 1 1 c0t2d0s7
6) To create the actual mirrors, using the submirrors from the first drive,
execute the following...
metainit d0 -m d10
metainit d1 -m d11
metainit d7 -m d17
7) Set the system to boot from the mirror with the command...
metaroot d0
8) Edit the /etc/vfstab file. Note that the root partition entry is already
properly set to use its mirror so it can be left untouched as...
/dev/md/dsk/d0 /dev/md/rdsk/d0 / ufs 1 no -
The vfstab entries for the swap and home slices should be adjusted to...
/dev/md/dsk/d1 - - swap - no -
/dev/md/dsk/d7 /dev/md/rdsk/d7 /export/home ufs 2 yes -
9) Save the corrected /etc/vfstab and reboot.
10) Now that the mirrors on the first drive are configured, we will attach
the submirrors from the second drive with the commands...
metattach d0 d20
metattach d1 d21
metattach d7 d27
11) The progress of the mirror synchronization can be monitored with the
"metastat" command.
Testing drive failure and recovery of the RAID-1 mirror
1) Power off the workstation with "init 5" and disconnect the IDE RAID-1
mirror drive on the first IDE bus (c0t0d0).
2) Power back on the workstation. The Solstice DiskSuite mirroring software
uses a metastate database of which there is at least one replica. While the
software will work with as little as one replica, four replicas (distributed
equally across the disks) are recommended for a two drive RAID-1 mirror. These
replicas are stored in the metadb slice we previously created (slice 3 in our
example). At least half of the replicas must be available for the system to
continue to run so the system can survive the loss of a drive. However, if more than half of the replicas are lost, the system will kernel panic upon booting.
Fixing this boot problem requires removal of the references to the metastate
database on the missing drive (c0t0d0). When removing metastate database
entries, one must be careful to only delete those from the missing drive.
Use "metadb" to examine the state of the metastate database if uncertain.
The metastat database entries for the missing drive will appear as "unknown"
Once the correct entry is identified (c0t0d0s3 in our example) execute...
metadb -f -d c0t0d0s3
WARNING: Never delete the metastate database entries for both disks as that
will make the system unbootable!
3) Reboot the system ignoring any errors about a read-only filesystem.
Now you can boot from the second drive with "boot disk1" and the system will
boot complete without a kernel panic. During the boot, non-fatal warnings
will appear that the submirrors (d10, d11 and d17) from the missing disk
are unavailable and their associated mirrors are marked for maintenance.
4) To simulate reconstruction of the RAID-1 after adding a replacement drive,
power off the system with "init 5" and reattach the first mirror drive as
master on the first IDE bus. Boot the system by powering it on, hit stop-A
and at the OpenBoot "ok" prompt execute "boot disk1".
NOTE: When actually replacing a failed drive, you will need to partition
it identically to the remaining RAID-1 mirror drive. In our example, this
would be done with...
prtvtoc /dev/rdsk/c0t2d0s2 | fmthard -s - /dev/rdsk/c0t0d0s2
5) After the system has successfully booted off from the master drive on
the second IDE bus (c0t2d0), recreate the metastat database entries for
the "replaced" drive on the first IDE bus (c0t0d0) with the command...
metadb -a -c 2 c0t0d0s3
6) Now re-attach the submirrors from the "replaced" drive with...
metareplace -e d0 /dev/dsk/c0t0d0s0
metareplace -e d1 /dev/dsk/c0t0d0s1
metareplace -e d7 /dev/dsk/c0t0d0s7
7) Use 'metastat' to monitor the syncing of the mirrors.
Notes: During the boot process of a RAID-1 mirror, warnings will appear
the certain md modules couldn't be forceloaded. This is an artifact of
the boot process with the root partition in RAID-1 and can be ignored.
Cabling: The IDE cables that ship with a SunBlade 1500 are too short to
reach both an IDE hard drive and the CDROM. A replacement cable will be
needed with a sufficient length between the master and slave connectors.
---------------------------------------------------------------------
------------------------------------------------------------------------------
Jack W. Howarth, Ph.D. 231 Albert Sabin Way
NMR Facility Director Cincinnati, Ohio 45267-0524
Dept. of Molecular Genetics phone: (513) 558-4420
Univ. of Cincinnati College of Medicine fax: (513) 558-8474
Received on Thu Sep 30 2004 - 16:42:05 MST