-----Original Message-----
From: Robert Pagan 
Sent: Tuesday, October 07, 2003 5:38 PM
To: xxxxx
Cc: Robert Pagan
Subject: SunSolve Document
Full and Basic/Functional Unencapsulation of a Volu= me Manager Encapsulated Root Disk While ... Document ID: ID24663 Synopsis: Full and Basic/Functional Unencapsulation of a Volume Manager Encapsulated Root Disk While ...

<= p>Keyword(s):SEVM, VxVM, Volume Manager, encapsulation

Overview:

This document explains the s= teps necessary to unencapsulate the root disk from Volume Manager control. This document applies to both Sun Enterprise Volume Manager[TM] (SEVM) 2.x and Veritas Volume Manager (VxVM) 3.x.

This document is divided into two distinct sections. The first section describes full unencapsulation while booted from a Solaris[TM] CDROM. This procedure shoul= d be used any time it is necessary to completely remove the root disk from Volum= e Manager control and bring the disk back to a pre-encapsulation state includ= ing all partitions such as /export, and /opt.

The second section explains the steps to perform a Basic/Functional (BF) unencapsulati= on while booted from a Solaris CDROM. Basic/Functional unencapsulation tempora= rily unencapsulates the root disk so that troubleshooting of booting issues or o= ther issues can be done. BF unencapsulation gives you access to an unencapsulate= d /, swap, /usr, and /var but no access to non "big-4" partitions. =

Notes for Full Unencapsulation:

Under normal circumstances, if the system can be booted to at least single user m= ode, it is recommended that the vxunroot command be used to unencapsulate root. = A full unencapsulation should be performed if the vxunroot command is not wor= king for some reason, or if the system cannot be booted and we want to completel= y remove Volume Manager from having any control over the root disk.

You cannot perform a full unencapsulation and still maintain Volume Manager functionality if the root disk is the ONLY disk in the rootdg diskgroup. If the root disk is the only disk in rootdg, you can still unencapsulate, but Volume Manager will not work until another disk is initialized into rootdg using vxinstall after the system has been fully unencapsulated. Normally if root it is encapsulated, it is also mirrored wh= ich gives us another disk in rootdg, however it should always be verified that there is at least one other disk in rootdg before following this procedure = so that we know what to expect once root is unencapsulated.

= Also note that Volume Manager will allow you to create volumes using free s= pace on the root disk, after the root disk has been encapsulated. Volumes create= d post-encapsulation like this do not have underlying hard partitions and therefore are not recoverable with this procedure. If at all possible, make backups of any volumes created on the rootdisk post-encapsulation before following this procedure. Once you are unencapsulated, if you have free spa= ce and a free partition, you could newfs that partition and restore to it from your backup.

Steps for Full
Unencapsulation:

Bring the system to the OK prompt and insert a Solaris CD into the CDROM dr=
ive.
Then issue:

      boot cdrom -s

Once booted to the cdrom, set your terminal type so that vi will work
correctly.
If TERM=3Dsun doesn't work, often times TERM=3Dvt100 will.

      TERM=3Dsun;export TERM

Fsck your root filesystem:

      fsck -y /dev/rdsk/c#t#d#s0

If fsck comes back cleanly, mount slice 0 to /a.  If fsck cannot repair the
root file system,
there are obviously a number of possibilities.  This procedure does not att=
empt
to explain
file system corruption or how to repair it beyond fsck.  Fsck must come bac=
k
cleanly to continue
and mount root.

      mount /dev/dsk/c#t#d#s0 /a

Make a backup of /a/etc/system and then edit it:

      cp /a/etc/system /a/etc/system.orig
      vi /a/etc/system

Completely remove the following lines from the system file.
If you re-encapsulate in the future, these lines will be added back correct=
ly
so there
is nothing to be lost by removing them.

      rootdev:/pseudo/vxio@0:0
      set vxio:vol_rootdev_is_volume=3D1

Make a backup of /a/etc/vfstab and then edit it:

      cp /a/etc/vfstab /a/etc/vfstab.orig
      vi /a/etc/vfstab

Edit the vfstab file back to it's original state, pointing /, swap, /usr, a=
nd
/var
to hard partitions on the disk like /dev/dsk and /dev/rdsk rather than /dev=
/vx/
entries.
Temporarily comment out all other /dev/vx volumes from the /a/etc/vfstab fi=
le
using the #
character. This includes filesystems like /opt and /export, if they exist.

The original /etc/vfstab will look something like this, assuming root is
c0t0d0:
Note: Columns have been aligned and spaces added for clarity.
---------------------------------------------------------------------------
/dev/vx/dsk/swapvol         -                     -       swap    -   no  -
/dev/vx/dsk/rootvol         /dev/vx/rdsk/rootvol  /       ufs     1   no  -
/dev/vx/dsk/usr             /dev/vx/rdsk/usr      /usr    ufs     1   no  -
/dev/vx/dsk/var             /dev/vx/rdsk/var      /var    ufs     1   no  -
/dev/vx/dsk/export          /dev/vx/rdsk/export   /export ufs     2   yes -
swap                        -                     /tmp    tmpfs   -   yes -

/dev/vx/dsk/datadg/somevol  /dev/vx/rdsk/datadg/somevol  /somevol  ufs  2 y=
es -

#NOTE: volume rootvol (/) encapsulated partition c0t0d0s0
#NOTE: volume swapvol (swap) encapsulated partition c0t0d0s1
#NOTE: volume usr (/usr) encapsulated partition c0t0d0s5
#NOTE: volume var (/var) encapsulated partition c0t0d0s6
#NOTE: volume export (/export) encapsulated partition c0t0d0s7
---------------------------------------------------------------------------

Once edited, the vfstab should look something like this:
---------------------------------------------------------------------------
/dev/dsk/c1t0d0s1           -                    -       swap    -   no  -
/dev/dsk/c1t0d0s0           /dev/rdsk/c0t0d0s0   /       ufs     1   no  -
/dev/dsk/c1t0d0s5           /dev/rdsk/c0t0d0s5   /usr    ufs     1   no  -
/dev/dsk/c1t0d0s6           /dev/rdsk/c0t0d0s6   /var    ufs     1   no  -
#/dev/dsk/c1t0d0s7          /dev/rdsk/c0t0d0s7   /export ufs     2   yes -=

swap                        -                    /tmp    tmpfs   -   yes -

#/dev/vx/dsk/datadg/somevol  /dev/vx/rdsk/datadg/somevol  /somevol  ufs  2 =
yes
-

#NOTE: volume rootvol (/) encapsulated partition c0t0d0s0
#NOTE: volume swapvol (swap) encapsulated partition c0t0d0s1
#NOTE: volume usr (/usr) encapsulated partition c0t0d0s5
#NOTE: volume var (/var) encapsulated partition c0t0d0s6
#NOTE: volume export (/export) encapsulated partition c0t0d0s7
---------------------------------------------------------------------------

Now make sure Volume Manager does not start on the next boot:

      touch /a/etc/vx/reconfig.d/state.d/install-db

This is important because IF the root disk contains mirrors, and the system=

boots up, the mirrors will get resynced, corrupting the changes we just mad=
e.

Remove the flag that tells Volume Manager that the root disk is encapsulate=
d:

      rm /a/etc/vx/reconfig.d/state.d/root-done

Reboot the system for changes to take effect:

      reboot

When we reboot we will come up in an partially unencapsulated state with /,
/usr, /var, and swap
mounted.  Volume Manager will not start but we can start it manually once w=
e
are booted.

To start Volume Manager, run the following commands:

      rm /etc/vx/reconfig.d/state.d/install-db
      vxiod set 10
      vxconfigd -m disable
      vxdctl enable

Now we can remove the volumes that existed on the encapsulated boot disk.  =
They
will generally
be rootvol, swapvol, usr, and var.  They might also include home, opt, or o=
ther
non-standard
root partitions.  Use the command 'vxprint -htg rootdg' to list the volumes=
 in
rootdg before
removing them.  Then, for each volume, run the command:

      /usr/sbin/vxedit -rf rm <volume name>

Remove the rootdisk from rootdg now that it has no volumes, plexes or subdi=
sks:
The disk name is usually 'rootdisk'

      /usr/sbin/vxdg rmdisk <disk name>

The final step is to re-write the vtoc of the disk so that hard partitions =
are
again defined for
the root file systems.  There are several ways to put the hard partitions b=
ack,
including using
fmthard on a modified /etc/vx/reconfig.d/disk.d/c#t#d#/vtoc file, using for=
mat
to manually repartition
the disk, or using the vxmksdpart command.  The simplest method however is =
to
use the vxedvtoc command
as explained below.

When Volume Manager encapsulates a disk, it makes a record of the old vtoc =
of
the disk.
This file is stored for each disk in /etc/vx/reconfig.d/disk.d/c#t#d#.  Thi=
s
file is stored in a Volume Manager
specific format, so it can't be used as an argument to fmthard unless it is
modified.  The 'vxedvtoc' command is
similar to fmthard but knows how to read this vtoc file and write that vtoc=
 to
a disk.  The command takes the form:

      vxedvtoc -f <filename> <devicename>

Assuming that the boot disk is c0t0d0 we would now run the command

      /etc/vx/bin/vxedvtoc -f /etc/vx/reconfig.d/disk.d/c0t0d0/vtoc
/dev/rdsk/c0t0d0s2

# THE ORIGINAL PARTITIONING IS AS FOLLOWS :
#SLICE     TAG  FLAGS    START     SIZE
 0         0x0  0x200        0        0
 1         0x0  0x200        0        0
 2         0x5  0x201        0  8794112
 3         0x0  0x200        0        0
 4         0x0  0x200        0        0
 5         0x0  0x200        0        0
 6         0xe  0x201        0  8794112
 7         0xf  0x201  8790016     4096
# THE NEW PARTITIONING WILL BE AS FOLLOWS :
#SLICE     TAG  FLAGS    START     SIZE
 0         0x0  0x200        0  2048000
 1         0x0  0x200  2048000  2048000
 2         0x5  0x201        0  8794112
 3         0x0  0x201  4096000  2048000
 4         0x0  0x201  6144000  2048000
 5         0x0  0x200        0        0
 6         0x0  0x200        0        0
 7         0x0  0x200        0        0
DO YOU WANT TO WRITE THIS TO THE DISK ? [Y/N] :y
WRITING THE NEW VTOC TO THE DISK

This will partition the disk back to a pre-encapsulation state.  Now we can
uncomment the entries
for any of the non Big-4 root partitions from /etc/vfstab, as well as any d=
ata
volumes.  In this
example we removed comments from /export and the data volume /somevol.

      vi /etc/vfstab

/dev/dsk/c0t0d0s1           -                    -       swap    -   no  -
/dev/dsk/c0t0d0s0           /dev/rdsk/c0t0d0s0   /       ufs     1   no  -
/dev/dsk/c0t0d0s5           /dev/rdsk/c0t0d0s5   /usr    ufs     1   no  -
/dev/dsk/c0t0d0s6           /dev/rdsk/c0t0d0s6   /var    ufs     1   no  -
/dev/dsk/c0t0d0s7           /dev/rdsk/c0t0d0s7   /export ufs     2   yes - =

swap                        -                    /tmp    tmpfs   -   yes -
/dev/vx/dsk/datadg/somevol  /dev/vx/rdsk/datadg/somevol  /somevol  ufs  2 y=
es -


Just to make sure, start all volumes                 =

      /usr/sbin/vxvol startall

Now issue a mountall to mount the now uncommented volumes

      mountall

At this point the root disk is completely free of Volume Manager control,
Volume Manager daemons are started, and all file systems / volumes should be mounted.             =

Notes for Basic/Functional Unencapsulation: =

This section explains the steps necessary to temporarily unencapsulate the root disk from Volume Manager control. BF unencapsulation allows the system to be booted from the raw Solaris partitions while still leaving the root disk under Volume Manager control.

T= his is a good method for troubleshooting boot issues that appear to be due to t= he disk being encapsulated because it can be undone by reversing the steps. Th= is procedure can be used even if the root disk is the only disk in the rootdg diskgroup because, throughout the procedure, root will keep it's private an= d public region. This procedure will only allow you to mount /, /usr, /var, a= nd swap. Any non "Big-4" partitions will not be mounted. If you must have non "Big-4" partitions available, you should perform a full unencapsulation as outlined above.

Steps for
Basic/Functional Unencapsulation:

Bring the system to the OK prompt and insert a Solaris CD into the cdrom dr=
ive.
Then issue:

      boot cdrom -s

Once booted to the cdrom, set your terminal type so that vi will work
correctly.
If TERM=3Dsun doesn't work, often times TERM=3Dvt100 will.

      TERM=3Dsun;export TERM

Fsck your root filesystem:

      fsck -y /dev/rdsk/c#t#d#s0

If fsck comes back cleanly, mount slice 0 to /a.  If fsck cannot repair the
root file system,
there are obviously a number of possibilities.  This procedure does not att=
empt
to explain
file system corruption or how to repair it.  fsck must come back cleanly to
continue and mount root.

      mount /dev/dsk/c#t#d#s0 /a

Make a backup of /a/etc/system and then edit it:

      cp /a/etc/system /a/etc/system.orig
      vi /a/etc/system

Comment out the following lines using double asterisks **.

      **rootdev:/pseudo/vxio@0:0
      **set vxio:vol_rootdev_is_volume=3D1
Make a backup of /a/etc/vfstab and then edit it:

      cp /a/etc/vfstab /a/etc/vfstab.orig
      vi /a/etc/vfstab

Edit the vfstab file back to it's original state, pointing /, swap, /usr, a=
nd
/var
to hard partitions on the disk like /dev/dsk and /dev/rdsk rather than /dev=
/vx/
entries.
Temporarily comment out all other /dev/vx volumes from the /a/etc/vfstab fi=
le
using the #
character. This includes filesystems like /opt and /export, if they exist.

The original /etc/vfstab will look something like this, assuming root is
c0t0d0:
Note: Columns have been aligned and spaces added for clarity.
---------------------------------------------------------------------------
/dev/vx/dsk/swapvol         -                     -       swap    -   no  -
/dev/vx/dsk/rootvol         /dev/vx/rdsk/rootvol  /       ufs     1   no  -
/dev/vx/dsk/usr             /dev/vx/rdsk/usr      /usr    ufs     1   no  -
/dev/vx/dsk/var             /dev/vx/rdsk/var      /var    ufs     1   no  -
/dev/vx/dsk/export          /dev/vx/rdsk/export   /export ufs     2   yes -
swap                        -                     /tmp    tmpfs   -   yes -

/dev/vx/dsk/datadg/somevol  /dev/vx/rdsk/datadg/somevol  /somevol  ufs  2 y=
es -

#NOTE: volume rootvol (/) encapsulated partition c0t0d0s0
#NOTE: volume swapvol (swap) encapsulated partition c0t0d0s1
#NOTE: volume usr (/usr) encapsulated partition c0t0d0s5
#NOTE: volume var (/var) encapsulated partition c0t0d0s6
#NOTE: volume export (/export) encapsulated partition c0t0d0s7
---------------------------------------------------------------------------

Once edited, it should look something like this:
---------------------------------------------------------------------------
/dev/dsk/c0t0d0s1           -                    -       swap    -   no  -
/dev/dsk/c0t0d0s0           /dev/rdsk/c0t0d0s0   /       ufs     1   no  -
/dev/dsk/c0t0d0s5           /dev/rdsk/c0t0d0s5   /usr    ufs     1   no  -
/dev/dsk/c0t0d0s6           /dev/rdsk/c0t0d0s6   /var    ufs     1   no  -
#/dev/dsk/c0t0d0s7          /dev/rdsk/c0t0d0s7   /export ufs     2   yes -=

swap                        -                    /tmp    tmpfs   -   yes -

#/dev/vx/dsk/datadg/somevol  /dev/vx/rdsk/datadg/somevol  /somevol  ufs  2 =
yes
-
#NOTE: volume rootvol (/) encapsulated partition c0t0d0s0
#NOTE: volume swapvol (swap) encapsulated partition c0t0d0s1
#NOTE: volume usr (/usr) encapsulated partition c0t0d0s5
#NOTE: volume var (/var) encapsulated partition c0t0d0s6
#NOTE: volume export (/export) encapsulated partition c0t0d0s7
---------------------------------------------------------------------------

Now make sure Volume Manager does not start on the next boot:

      touch /a/etc/vx/reconfig.d/state.d/install-db

The presence of install-db tells Volume Manager not to start at boot.
This is important because IF the root disk contains mirrors, and the system=

boots up, the mirrors will get resynced, corrupting the changes we just mad=
e.

Remove the flag that tells Volume Manager that the root disk is encapsulate=
d:

      rm /a/etc/vx/reconfig.d/state.d/root-done

Reboot the system for changes to take effect:

      reboot

When we reboot we will come up in an unencapsulated state with /=
,
/usr, /var, and swap
mounted.

At this point we have performed a Basic/Functional unencapsulation.  This i=
s a
not a state
that they system should be left in permanently.  It is a state that is usef=
ul
for
troubleshooting and system maintenance.

If problems with the system are resolved and you are ready to re-encapsulat=
e,
perform the following:
      touch /etc/vx/reconfig.d/state.d/root-done
      rm /etc/vx/reconfig.d/state.d/install-db
      cp /a/etc/vfstab.orig /a/etc/vfstab
      cp /a/etc/system.orig /a/etc/system
      vxconfigd -d (start volume manager daemon)
      vxdctl enable (enable transactions)
      vxplex dis <secondary plex>
      reboot

The <secondary plex> is determined by looking at the rootvol, swapvol=
,
usr, and var volumes, and
determining which plex is the one you are booted from.  To find this out, y=
ou
can compare your
df -k output with the vxprint output for each volume concerned.  The primar=
y
plex will have a
subdisk(sd) directly below it in the output whose device is listed in the d=
f
output.

For example:

v  rootvol      root       =

DISABLED ACTIVE   3073440  ROUND     -
pl rootvol-01   rootvol      ENABLED  ACTIVE   3073440  CONCAT    -        =
RW
sd rootdisk-B0  rootvol-01   rootdisk 4152639  1        0         c0t0d0   =
ENA
sd rootdisk-02  rootvol-01   rootdisk 0        3073439  1         c0t0d0   =
ENA
pl rootvol-02	rootvol	     ENABLED  ACTIVE   3073440  CONCAT    -        RW
sd disk01-01    rootvol-02   disk01   0        3073440  0         c1t2d0   =
ENA

df -k
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/dsk/c0t0d0s0    1488575 1182144  246888    83%    /
/proc                      0       0       0     0%    /proc
fd                         0       0       0     0%    /dev/fd
swap		      563848	1232  562616     1%    /tmp

Notice that / is currently mounted on c0t00ds0. This is = the active side of the mirror, so to prevent corruption on reboot, we want to disassociate the other side.

The other side in this c= ase is rootvol-02, because it contains c1t2d0.

Now, afte= r the reboot, you will be up on volumes. We need to resynch the mirrors. Do this by:

	vxplex att <volume> <secondary_plex>  =

=