-----Original Message----- From: Robert Pagan Sent: Tuesday, October 07, 2003 5:38 PM To: xxxxx Cc: Robert Pagan Subject: SunSolve Document
Overview:
This document explains the s= teps necessary to unencapsulate the root disk from Volume Manager control. This document applies to both Sun Enterprise Volume Manager[TM] (SEVM) 2.x and Veritas Volume Manager (VxVM) 3.x.
This document is divided into two distinct sections. The first section describes full unencapsulation while booted from a Solaris[TM] CDROM. This procedure shoul= d be used any time it is necessary to completely remove the root disk from Volum= e Manager control and bring the disk back to a pre-encapsulation state includ= ing all partitions such as /export, and /opt.
The second section explains the steps to perform a Basic/Functional (BF) unencapsulati= on while booted from a Solaris CDROM. Basic/Functional unencapsulation tempora= rily unencapsulates the root disk so that troubleshooting of booting issues or o= ther issues can be done. BF unencapsulation gives you access to an unencapsulate= d /, swap, /usr, and /var but no access to non "big-4" partitions. =
Notes for Full Unencapsulation:
Under normal circumstances, if the system can be booted to at least single user m= ode, it is recommended that the vxunroot command be used to unencapsulate root. = A full unencapsulation should be performed if the vxunroot command is not wor= king for some reason, or if the system cannot be booted and we want to completel= y remove Volume Manager from having any control over the root disk.
You cannot perform a full unencapsulation and still maintain Volume Manager functionality if the root disk is the ONLY disk in the rootdg diskgroup. If the root disk is the only disk in rootdg, you can still unencapsulate, but Volume Manager will not work until another disk is initialized into rootdg using vxinstall after the system has been fully unencapsulated. Normally if root it is encapsulated, it is also mirrored wh= ich gives us another disk in rootdg, however it should always be verified that there is at least one other disk in rootdg before following this procedure = so that we know what to expect once root is unencapsulated.
= Also note that Volume Manager will allow you to create volumes using free s= pace on the root disk, after the root disk has been encapsulated. Volumes create= d post-encapsulation like this do not have underlying hard partitions and therefore are not recoverable with this procedure. If at all possible, make backups of any volumes created on the rootdisk post-encapsulation before following this procedure. Once you are unencapsulated, if you have free spa= ce and a free partition, you could newfs that partition and restore to it from your backup.
Steps for Full Unencapsulation: Bring the system to the OK prompt and insert a Solaris CD into the CDROM dr= ive. Then issue: boot cdrom -s Once booted to the cdrom, set your terminal type so that vi will work correctly. If TERM=3Dsun doesn't work, often times TERM=3Dvt100 will. TERM=3Dsun;export TERM Fsck your root filesystem: fsck -y /dev/rdsk/c#t#d#s0 If fsck comes back cleanly, mount slice 0 to /a. If fsck cannot repair the root file system, there are obviously a number of possibilities. This procedure does not att= empt to explain file system corruption or how to repair it beyond fsck. Fsck must come bac= k cleanly to continue and mount root. mount /dev/dsk/c#t#d#s0 /a Make a backup of /a/etc/system and then edit it: cp /a/etc/system /a/etc/system.orig vi /a/etc/system Completely remove the following lines from the system file. If you re-encapsulate in the future, these lines will be added back correct= ly so there is nothing to be lost by removing them. rootdev:/pseudo/vxio@0:0 set vxio:vol_rootdev_is_volume=3D1 Make a backup of /a/etc/vfstab and then edit it: cp /a/etc/vfstab /a/etc/vfstab.orig vi /a/etc/vfstab Edit the vfstab file back to it's original state, pointing /, swap, /usr, a= nd /var to hard partitions on the disk like /dev/dsk and /dev/rdsk rather than /dev= /vx/ entries. Temporarily comment out all other /dev/vx volumes from the /a/etc/vfstab fi= le using the # character. This includes filesystems like /opt and /export, if they exist. The original /etc/vfstab will look something like this, assuming root is c0t0d0: Note: Columns have been aligned and spaces added for clarity. --------------------------------------------------------------------------- /dev/vx/dsk/swapvol - - swap - no - /dev/vx/dsk/rootvol /dev/vx/rdsk/rootvol / ufs 1 no - /dev/vx/dsk/usr /dev/vx/rdsk/usr /usr ufs 1 no - /dev/vx/dsk/var /dev/vx/rdsk/var /var ufs 1 no - /dev/vx/dsk/export /dev/vx/rdsk/export /export ufs 2 yes - swap - /tmp tmpfs - yes - /dev/vx/dsk/datadg/somevol /dev/vx/rdsk/datadg/somevol /somevol ufs 2 y= es - #NOTE: volume rootvol (/) encapsulated partition c0t0d0s0 #NOTE: volume swapvol (swap) encapsulated partition c0t0d0s1 #NOTE: volume usr (/usr) encapsulated partition c0t0d0s5 #NOTE: volume var (/var) encapsulated partition c0t0d0s6 #NOTE: volume export (/export) encapsulated partition c0t0d0s7 --------------------------------------------------------------------------- Once edited, the vfstab should look something like this: --------------------------------------------------------------------------- /dev/dsk/c1t0d0s1 - - swap - no - /dev/dsk/c1t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 no - /dev/dsk/c1t0d0s5 /dev/rdsk/c0t0d0s5 /usr ufs 1 no - /dev/dsk/c1t0d0s6 /dev/rdsk/c0t0d0s6 /var ufs 1 no - #/dev/dsk/c1t0d0s7 /dev/rdsk/c0t0d0s7 /export ufs 2 yes -= swap - /tmp tmpfs - yes - #/dev/vx/dsk/datadg/somevol /dev/vx/rdsk/datadg/somevol /somevol ufs 2 = yes - #NOTE: volume rootvol (/) encapsulated partition c0t0d0s0 #NOTE: volume swapvol (swap) encapsulated partition c0t0d0s1 #NOTE: volume usr (/usr) encapsulated partition c0t0d0s5 #NOTE: volume var (/var) encapsulated partition c0t0d0s6 #NOTE: volume export (/export) encapsulated partition c0t0d0s7 --------------------------------------------------------------------------- Now make sure Volume Manager does not start on the next boot: touch /a/etc/vx/reconfig.d/state.d/install-db This is important because IF the root disk contains mirrors, and the system= boots up, the mirrors will get resynced, corrupting the changes we just mad= e. Remove the flag that tells Volume Manager that the root disk is encapsulate= d: rm /a/etc/vx/reconfig.d/state.d/root-done Reboot the system for changes to take effect:
reboot When we reboot we will come up in an partially unencapsulated state with /, /usr, /var, and swap mounted. Volume Manager will not start but we can start it manually once w= e are booted. To start Volume Manager, run the following commands: rm /etc/vx/reconfig.d/state.d/install-db vxiod set 10 vxconfigd -m disable vxdctl enable Now we can remove the volumes that existed on the encapsulated boot disk. = They will generally be rootvol, swapvol, usr, and var. They might also include home, opt, or o= ther non-standard root partitions. Use the command 'vxprint -htg rootdg' to list the volumes= in rootdg before removing them. Then, for each volume, run the command: /usr/sbin/vxedit -rf rm <volume name> Remove the rootdisk from rootdg now that it has no volumes, plexes or subdi= sks: The disk name is usually 'rootdisk' /usr/sbin/vxdg rmdisk <disk name> The final step is to re-write the vtoc of the disk so that hard partitions = are again defined for the root file systems. There are several ways to put the hard partitions b= ack, including using fmthard on a modified /etc/vx/reconfig.d/disk.d/c#t#d#/vtoc file, using for= mat to manually repartition the disk, or using the vxmksdpart command. The simplest method however is = to use the vxedvtoc command as explained below. When Volume Manager encapsulates a disk, it makes a record of the old vtoc = of the disk. This file is stored for each disk in /etc/vx/reconfig.d/disk.d/c#t#d#. Thi= s file is stored in a Volume Manager specific format, so it can't be used as an argument to fmthard unless it is modified. The 'vxedvtoc' command is similar to fmthard but knows how to read this vtoc file and write that vtoc= to a disk. The command takes the form: vxedvtoc -f <filename> <devicename> Assuming that the boot disk is c0t0d0 we would now run the command /etc/vx/bin/vxedvtoc -f /etc/vx/reconfig.d/disk.d/c0t0d0/vtoc /dev/rdsk/c0t0d0s2 # THE ORIGINAL PARTITIONING IS AS FOLLOWS : #SLICE TAG FLAGS START SIZE 0 0x0 0x200 0 0 1 0x0 0x200 0 0 2 0x5 0x201 0 8794112 3 0x0 0x200 0 0 4 0x0 0x200 0 0 5 0x0 0x200 0 0 6 0xe 0x201 0 8794112 7 0xf 0x201 8790016 4096 # THE NEW PARTITIONING WILL BE AS FOLLOWS : #SLICE TAG FLAGS START SIZE 0 0x0 0x200 0 2048000 1 0x0 0x200 2048000 2048000 2 0x5 0x201 0 8794112 3 0x0 0x201 4096000 2048000 4 0x0 0x201 6144000 2048000 5 0x0 0x200 0 0 6 0x0 0x200 0 0 7 0x0 0x200 0 0 DO YOU WANT TO WRITE THIS TO THE DISK ? [Y/N] :y WRITING THE NEW VTOC TO THE DISK This will partition the disk back to a pre-encapsulation state. Now we can uncomment the entries for any of the non Big-4 root partitions from /etc/vfstab, as well as any d= ata volumes. In this example we removed comments from /export and the data volume /somevol. vi /etc/vfstab /dev/dsk/c0t0d0s1 - - swap - no - /dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 no - /dev/dsk/c0t0d0s5 /dev/rdsk/c0t0d0s5 /usr ufs 1 no - /dev/dsk/c0t0d0s6 /dev/rdsk/c0t0d0s6 /var ufs 1 no - /dev/dsk/c0t0d0s7 /dev/rdsk/c0t0d0s7 /export ufs 2 yes - = swap - /tmp tmpfs - yes -
/dev/vx/dsk/datadg/somevol /dev/vx/rdsk/datadg/somevol /somevol ufs 2 y= es - Just to make sure, start all volumes =
/usr/sbin/vxvol startall Now issue a mountall to mount the now uncommented volumes mountall At this point the root disk is completely free of Volume Manager control, Volume Manager daemons are started, and all file systems / volumes should be mounted. =
Notes for Basic/Functional Unencapsulation: =
This section explains the steps necessary to temporarily unencapsulate the root disk from Volume Manager control. BF unencapsulation allows the system to be booted from the raw Solaris partitions while still leaving the root disk under Volume Manager control.
T= his is a good method for troubleshooting boot issues that appear to be due to t= he disk being encapsulated because it can be undone by reversing the steps. Th= is procedure can be used even if the root disk is the only disk in the rootdg diskgroup because, throughout the procedure, root will keep it's private an= d public region. This procedure will only allow you to mount /, /usr, /var, a= nd swap. Any non "Big-4" partitions will not be mounted. If you must have non "Big-4" partitions available, you should perform a full unencapsulation as outlined above.
Steps for Basic/Functional Unencapsulation: Bring the system to the OK prompt and insert a Solaris CD into the cdrom dr= ive. Then issue: boot cdrom -s Once booted to the cdrom, set your terminal type so that vi will work correctly. If TERM=3Dsun doesn't work, often times TERM=3Dvt100 will. TERM=3Dsun;export TERM Fsck your root filesystem: fsck -y /dev/rdsk/c#t#d#s0 If fsck comes back cleanly, mount slice 0 to /a. If fsck cannot repair the root file system, there are obviously a number of possibilities. This procedure does not att= empt to explain file system corruption or how to repair it. fsck must come back cleanly to continue and mount root. mount /dev/dsk/c#t#d#s0 /a Make a backup of /a/etc/system and then edit it: cp /a/etc/system /a/etc/system.orig vi /a/etc/system Comment out the following lines using double asterisks **. **rootdev:/pseudo/vxio@0:0 **set vxio:vol_rootdev_is_volume=3D1 Make a backup of /a/etc/vfstab and then edit it: cp /a/etc/vfstab /a/etc/vfstab.orig vi /a/etc/vfstab Edit the vfstab file back to it's original state, pointing /, swap, /usr, a= nd /var to hard partitions on the disk like /dev/dsk and /dev/rdsk rather than /dev= /vx/ entries. Temporarily comment out all other /dev/vx volumes from the /a/etc/vfstab fi= le using the # character. This includes filesystems like /opt and /export, if they exist. The original /etc/vfstab will look something like this, assuming root is c0t0d0: Note: Columns have been aligned and spaces added for clarity. --------------------------------------------------------------------------- /dev/vx/dsk/swapvol - - swap - no - /dev/vx/dsk/rootvol /dev/vx/rdsk/rootvol / ufs 1 no - /dev/vx/dsk/usr /dev/vx/rdsk/usr /usr ufs 1 no - /dev/vx/dsk/var /dev/vx/rdsk/var /var ufs 1 no - /dev/vx/dsk/export /dev/vx/rdsk/export /export ufs 2 yes - swap - /tmp tmpfs - yes - /dev/vx/dsk/datadg/somevol /dev/vx/rdsk/datadg/somevol /somevol ufs 2 y= es - #NOTE: volume rootvol (/) encapsulated partition c0t0d0s0 #NOTE: volume swapvol (swap) encapsulated partition c0t0d0s1 #NOTE: volume usr (/usr) encapsulated partition c0t0d0s5 #NOTE: volume var (/var) encapsulated partition c0t0d0s6 #NOTE: volume export (/export) encapsulated partition c0t0d0s7 --------------------------------------------------------------------------- Once edited, it should look something like this: --------------------------------------------------------------------------- /dev/dsk/c0t0d0s1 - - swap - no - /dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 no - /dev/dsk/c0t0d0s5 /dev/rdsk/c0t0d0s5 /usr ufs 1 no - /dev/dsk/c0t0d0s6 /dev/rdsk/c0t0d0s6 /var ufs 1 no - #/dev/dsk/c0t0d0s7 /dev/rdsk/c0t0d0s7 /export ufs 2 yes -= swap - /tmp tmpfs - yes - #/dev/vx/dsk/datadg/somevol /dev/vx/rdsk/datadg/somevol /somevol ufs 2 = yes - #NOTE: volume rootvol (/) encapsulated partition c0t0d0s0 #NOTE: volume swapvol (swap) encapsulated partition c0t0d0s1 #NOTE: volume usr (/usr) encapsulated partition c0t0d0s5 #NOTE: volume var (/var) encapsulated partition c0t0d0s6 #NOTE: volume export (/export) encapsulated partition c0t0d0s7 --------------------------------------------------------------------------- Now make sure Volume Manager does not start on the next boot: touch /a/etc/vx/reconfig.d/state.d/install-db The presence of install-db tells Volume Manager not to start at boot. This is important because IF the root disk contains mirrors, and the system= boots up, the mirrors will get resynced, corrupting the changes we just mad= e. Remove the flag that tells Volume Manager that the root disk is encapsulate= d: rm /a/etc/vx/reconfig.d/state.d/root-done Reboot the system for changes to take effect: reboot
When we reboot we will come up in an unencapsulated state with /= , /usr, /var, and swap mounted. At this point we have performed a Basic/Functional unencapsulation. This i= s a not a state that they system should be left in permanently. It is a state that is usef= ul for troubleshooting and system maintenance. If problems with the system are resolved and you are ready to re-encapsulat= e, perform the following:
touch /etc/vx/reconfig.d/state.d/root-done rm /etc/vx/reconfig.d/state.d/install-db cp /a/etc/vfstab.orig /a/etc/vfstab cp /a/etc/system.orig /a/etc/system vxconfigd -d (start volume manager daemon) vxdctl enable (enable transactions) vxplex dis <secondary plex> reboot The <secondary plex> is determined by looking at the rootvol, swapvol= , usr, and var volumes, and determining which plex is the one you are booted from. To find this out, y= ou can compare your df -k output with the vxprint output for each volume concerned. The primar= y plex will have a subdisk(sd) directly below it in the output whose device is listed in the d= f output.
For example:
v rootvol root = DISABLED ACTIVE 3073440 ROUND - pl rootvol-01 rootvol ENABLED ACTIVE 3073440 CONCAT - = RW sd rootdisk-B0 rootvol-01 rootdisk 4152639 1 0 c0t0d0 = ENA sd rootdisk-02 rootvol-01 rootdisk 0 3073439 1 c0t0d0 = ENA pl rootvol-02 rootvol ENABLED ACTIVE 3073440 CONCAT - RW sd disk01-01 rootvol-02 disk01 0 3073440 0 c1t2d0 = ENA
df -k Filesystem kbytes used avail capacity Mounted on /dev/dsk/c0t0d0s0 1488575 1182144 246888 83% / /proc 0 0 0 0% /proc fd 0 0 0 0% /dev/fd swap 563848 1232 562616 1% /tmp
Notice that / is currently mounted on c0t00ds0. This is = the active side of the mirror, so to prevent corruption on reboot, we want to disassociate the other side.
The other side in this c= ase is rootvol-02, because it contains c1t2d0.
Now, afte= r the reboot, you will be up on volumes. We need to resynch the mirrors. Do this by:
vxplex att <volume> <secondary_plex> =
=