Volume Group

VOLUME GROUP
When you install a system, the first volume group (VG) is created. It is called the rootvg. Your rootvg volume group is a base set of logical volumes required to start the system. It includes paging space, the journal log, boot data, and dump storage, each on its own separate logical volume.

A normal VG is limited to 32512 physical partitions. (32 physical volumes, each with 1016 partitions)
you can change it with: chvg -t4 bbvg (the factor is 4, which means: maximum partitions:4064 (instead of 1016), max disks:8 (instead of 32))


How do I know if my volume group is normal, big, or scalable?
Run the lsvg command on the volume group and look at the value for MAX PVs. The value is 32 for normal, 128 for big, and 1024 for scalable volume group.

VG type     Maximum PVs    Maximum LVs    Maximum PPs per VG    Maximum PP size
Normal VG     32              256            32,512 (1016 * 32)      1 GB
Big VG        128             512            130,048 (1016 * 128)    1 GB
Scalable VG   1024            4096           2,097,152               128 GB


If a physical volume is part of a volume group, it contains 2 additional reserved areas. One area contains both the VGSA and the VGDA, and this area is started from the first 128 reserved sectors (blocks) on the disk. The other area is at the end of the disk, and is reserved as a relocation pool for bad blocks.

VGDA (Volume Group Descriptor Area)
It is an area on the hard disk (PV) that contains information about the entire volume group. There is at least one VGDA per physical volume, one or two copies per disk. It contains physical volume list (PVIDs), logical volume list (LVIDs), physical partition map (maps lps to pps)

# lqueryvg -tAp hdisk0                                <--look into the VGDA (-A:all info, -t: tagged, without it only numbers)
Max LVs:        256
PP Size:        27                                    <--exponent of 2:2 to 7=128MB
Free PPs:       698
LV count:       11
PV count:       2
Total VGDAs:    3
Conc Allowed:   0
MAX PPs per PV  2032
MAX PVs:        16
Quorum (disk):  0
Quorum (dd):    0
Auto Varyon ?:  1
Conc Autovaryo  0
Varied on Conc  0
Logical:        00cebffe00004c000000010363f50ac5.1   hd5 1       <--1: count of mirror copies (00cebff...c5 is the VGID)
                00cebffe00004c000000010363f50ac5.2   hd6 1
                00cebffe00004c000000010363f50ac5.3   hd8 1
                ...
Physical:       00cebffe63f500ee                2   0            <--2:VGDA count 0:code for its state (active, missing, removed)
                00cebffe63f50314                1   0            (The sum of VGDA count should be the same as the Total VGDAs)
Total PPs:      1092
LTG size:       128
...
Max PPs:        32512
-----------------------

VGSA (Volume Group Status Area)
The VGSAs are always present, but used with mirroring only. Needed to track the state of mirror copies, that means whether synchronized or stale. Per-disk stucure, but twice on each disk.


Quorum
Non-rootvg volume groups can be taken offline and brought online by a process called varying on and varying off a volume group. The system checks the availability of all VGDAs for a particular volume group to determine if a volume group is going to be varied on or off.
When attempting to vary on a volume group, the system checks for a quorum of the VGDA to be available. A quorum is equal to 51 percent or more of the VGDAs available. If it can find a quorum, the VG will be varied on; if not, it will not make the volume group available.
Turning off the quorum does not allow a varyonvg without a quorum, it just prevents the closing of an active vg when losing its quorum. (so forced varyon may needed: varyonvg -f VGname)

After turning it off (chvg -Qn VGname) it is in effect immediately. 


LTG
LTG is the maximum transfer size of a logical volume (volume group?).
At 5.3 and 6.1 AIX dynamically sets LTG size (calculated at each volume group activation).
LTG size can be changed with: varyonvg -M<LTG size>
(The chvg -L has no effect on volume groups created on 5.3 or later (it was used on 5.2)
To display the LTG size of a disk: /usr/sbin/lquerypv -M <hdisk#>


lsvg                      lists the volume groups that are on the system
lsvg -o                   lists all volume groups that are varied on
lsvg -o | lsvg -ip        lists pvs of online vgs
lsvg rootvg               gives details about the vg (lsvg -L <vgname>, will doe not wait for the lock release (useful during mirroring))
lsvg -l rootvg            info about all logical volumes that are part of a vg
lsvg -M rootvg            lists all PV, LV, PP deatils of a vg (PVname:PPnum LVname: LPnum :Copynum) 
lsvg -p rootvg            display all physical volumes that are part of the volume group
lsvg -n <hdisk>           shows vg infos, but it is read from the VGDA on the specified disk (it is useful to compare it with different disks)

mkvg -s 2 -y testvg hdisk13    creates a volume group
    -s                    specify the physical partition size
    -y                    indicate the name of the new vg

chvg                      changes the characteristics of a volume group
chvg -u <vgname>          unlocks the volume group (if a command core dumping, or the system crashed and vg is left in locked state)
                          (Many LVM commands place a lock into the ODM to prevent other commands working on the same time.)
extendvg rootvg hdisk7    adds hdisk7 to rootvg (-f forces it: extendvg -f ...)
reducevg rootvg hdisk7    tries to delete hdisk7 (the vg must be varied on) (reducevg -f ... :force it)
                          (it will fail if the vg contains open logical volumes)
reducevg datavg <pvid>    reducevg command can use pvid as well (it is useful, if disk already removed from ODM, but VGDA still exist)


syncvg                    synchronizes stale physical partitions (varyonvg better, becaue first it reestablis reservation then sync in backg.)
varyonvg rootvg           makes the vg available (-f force a vg to be online if it does not have the quorum of available disks)
                          (varyonvg acts as a self-repair program for VGDAs, it does a syncvg as well)
varyoffvg rootvg          deactivate a volume group
mirrorvg -S P01vg hdisk1  mirroring rootvg to hdisk1 (checking: lsvg P01vg | grep STALE) (-S: background sync)
                          (mirrorvg -m rootvg hdisk1 <--m makes an exact copy, pp mapping will be identical, it is advised this way)
unmirrorvg testvg1 hdisk0 hdisk1 remove mirrors on the vg from the specified disks 

exportvg avg              removes the VGs definition out of the ODM and /etc/filesystems (for ODM problems after importvg will fix it)
importvg -y avg hdisk8    makes the previously exported vg known to the system (hdisk8, is any disk belongs to the vg)

reorgvg                   rearranges physical partitions within the vg to conform with the placement policy (outer edge...) for the lv.
                          (For this 1 free pp is needed, and the relocatable flag for lvs must be set to 'y': chlv -r...)

getlvodm -j <hdisk>       get the vgid for the hdisk from the odm
getlvodm -t <vgid>        get the vg name for the vgid from the odm
getlvodm -v <vgname>      get the vgid for the vg name from the odm

getlvodm -p <hdisk>       get the pvid for the hdisk from the odm
getlvodm -g <pvid>        get the hdisk for the pvid from the odm
lqueryvg -tcAp <hdisk>    get all the vgid and pvid information for the vg from the vgda (directly from the disk)
                          (you can compare the disk with odm: getlvodm <-> lqueryvg)


synclvodm <vgname>        synchronizes or rebuilds the lvcb, the device configuration database, and the vgdas on the physical volumes
redefinevg                it helps regain the basic ODM informations if those are corrupted (redefinevg -d hdisk0 rootvg)
readvgda hdisk40          shows details from the disk

Physical Volume states (and quorum):
lsvg -p VGName            <--shows pv states (not devices states!)
    active                <--during varyonvg disk can be accessed
    missing               <--during varyonvg disk can not be accessed + quorum is available
                          (after disk repair varyonvg VGName will put in active state)
    removed               <--no disk access during varyonvg + quorum is not available --> you issue varyonvg -f VGName
                          (after force varyonvg in the above case, PV state will be removed, and it won't be used for quorum)
                          (to put back in active state, first we have to tell the system the failure is over:)
                          (chpv -va hdiskX, this defines the disk as active, and after that varyonvg will synchronize)


---------------------------------------

Mirroring rootvg (i.e after disk replacement):
1. disk replaced -> cfgmgr           <--it will find the new disk (i.e. hdisk1)
2. extendvg rootvg hdisk1            <--sometimes extendvg -f rootvg...
(3. chvg -Qn rootvg)                 <--only if quorum setting has not yet been disabled, because this needs a restart
4. mirrorvg -s rootvg                <--add mirror for rootvg (-s: synchronization will not be done)
5. syncvg -v rootvg                  <--synchronize the new copy (lsvg rootvg | grep STALE)
6. bosboot -a                        <--we changed the system so create boot image (-a: create complete boot image and device)
                                     (hd5 is mirrorred, no need to do it for each disk. ie. bosboot -ad hdisk0)
7. bootlist -m normal hdisk0 hdisk1  <--set normal bootlist
8. bootlist -m service hdisk0 hdisk1 <--set bootlist when we want to boot into service mode
(9. shutdown -Fr)                    <--this is needed if quorum has been disabled
10.bootinfo -b                       <--shows the disk  which was used for boot

---------------------------------------

Export/Import:
1. node A: umount all fs -> varyoffvg myvg
2. node A: exportvg myvg            <--ODM cleared
3. node B: importvg -y myvg hdisk3  <-- -y: vgname, if omitted a new vg will be created (if needed varyonvg -> mount fs)
if fs already exists:
    1. umount the old one and mount the imported one with this: mount -o log=/dev/loglv01 -V jfs2 /dev/lv24 /home/michael
    (these details have to added in the mount command, and these can be retreived from LVCB: getlvcb lv24 -At)

    2. vi /etc/filesystems, create a second stanza for the imported filesystems with a new mountpoint.

---------------------------------------

VG problems with ODM:

if varyoff possible:
1. write down VGs name, major number, a disk
2. exportvg VGname
3. importvg -V MajorNum -y VGname hdiskX

if varyoff not possible:
1. write down VGs name, major number, a disk
2. export the vg by the backdoor, using: odmdelete
3. re-import vg (may produce warnings, but works)
(it is not necessary to umount fs or stop processes)

---------------------------------------

Changing factor value (chvg -t) of a VG:

A normal or a big vg has the following limitations after creation: 
MAX PPs per VG = MAX PVs * MAX PPS per PV)

                   Normal       Big
MAX PPs per VG:    32512        130048
MAX PPs per PV:    1016         1016
MAX PVs:           32           128

If we want to extend the vg with a disk, which is so large that we would have more than 1016 PPs on that disk, we will receive:
root@bb_lpar: / # extendvg bbvg hdisk4
0516-1162 extendvg: Warning, The Physical Partition Size of 4 requires the
        creation of 1024 partitions for hdisk4.  The limitation for volume group
        bbvg is 1016 physical partitions per physical volume.  Use chvg command
        with -t option to attempt to change the maximum Physical Partitions per
        Physical volume for this volume group.

If we change the factor value of the VG, then extendvg will be possible:
root@bb_lpar: / # chvg -t 2 bbvg
0516-1164 chvg: Volume group bbvg changed.  With given characteristics bbvg
        can include up to 16 physical volumes with 2032 physical partitions each.

Calculation:
Normal VG: 32/factor = new value of MAX PVs
Big VG: 128/factor= new value of MAX PVs

-t   PPs per PV        MAX PV (Normal)    MAX PV (Big)
1    1016              32                 128
2    2032              16                 64
3    3048              10                 42
4    4064              8                  32
5    5080              6                  25
...

"chvg -t" can be used online either increasing or decreasing the value of the factor.

---------------------------------------

Changing Normal VG to Big VG:

If you reached the MAX PV limit of a Normal VG and playing with the factor (chvg -t) is not possible anymore you can convert it to Big VG.
It is an online activity, but there must be free PPs on each physical volume, because VGDA will be expanded on all disks:

root@bb_lpar: / # lsvg -p bbvg
bbvg:
PV_NAME           PV STATE          TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdisk2            active            511         2           02..00..00..00..00
hdisk3            active            511         23          00..00..00..00..23
hdisk4            active            1023        0           00..00..00..00..00

root@bb_lpar: / # chvg -B bbvg
0516-1214 chvg: Not enough free physical partitions exist on hdisk4 for the
        expansion of the volume group descriptor area.  Migrate/reorganize to free up
        2 partitions and run chvg again.

In this case we have to migrate 2 PPs from hdisk4 to hdsik3 (so 2 PPs will be freed up on hdisk4):

root@bb_lpar: / # lspv -M hdisk4
hdisk4:1        bblv:920
hdisk4:2        bblv:921
hdisk4:3        bblv:922
hdisk4:4        bblv:923
hdisk4:5        bblv:924
...

root@bb_lpar: / # lspv -M hdisk3
hdisk3:484      bblv:3040
hdisk3:485      bblv:3041
hdisk3:486      bblv:3042
hdisk3:487      bblv:1
hdisk3:488      bblv:2
hdisk3:489-511

root@bb_lpar: / # migratelp bblv/920 hdisk3/489
root@bb_lpar: / # migratelp bblv/921 hdisk3/490

root@bb_lpar: / # lsvg -p bbvg
bbvg:
PV_NAME           PV STATE          TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdisk2            active            511         2           02..00..00..00..00
hdisk3            active            511         21          00..00..00..00..21
hdisk4            active            1023        2           02..00..00..00..00

If we try again changing to Big VG, now it is successful:
root@bb_lpar: / # chvg -B bbvg
0516-1216 chvg: Physical partitions are being migrated for volume group
        descriptor area expansion.  Please wait.
0516-1164 chvg: Volume group bbvg2 changed.  With given characteristics bbvg2
        can include up to 128 physical volumes with 1016 physical partitions each.

If you check again, freed up PPs has been used:
root@bb_lpar: / # lsvg -p bbvg
bbvg:
PV_NAME           PV STATE          TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdisk2            active            509         0           00..00..00..00..00
hdisk3            active            509         17          00..00..00..00..17
hdisk4            active            1021        0           00..00..00..00..00

---------------------------------------

Changing Normal (or Big) VG to Scalable VG:

If you reached the MAX PV limit of a Normal or a Big VG and playing with the factor (chvg -t) is not possible anymore you can convert that VG to Scalable VG. A Scalable VG allows a maximum of 1024 PVs and 4096 LVs and a very big advantage that the maximum number of PPs applies to the entire VG and is no longer defined on a per disk basis.

!!!Converting to Scalable VG is an offline activity (varyoffvg), and there must be free PPs on each physical volume, because VGDA will be expanded on all disks.

root@bb_lpar: / # chvg -G bbvg
0516-1707 chvg: The volume group must be varied off during conversion to
        scalable volume group format.

root@bb_lpar: / # varyoffvg bbvg
root@bb_lpar: / # chvg -G bbvg
0516-1214 chvg: Not enough free physical partitions exist on hdisk2 for the
        expansion of the volume group descriptor area.  Migrate/reorganize to free up
        18 partitions and run chvg again.


After migrating some lps to free up required PPs (in this case it was 18), then changing to Scalable VG is successful:
root@bb_lpar: / # chvg -G bbvg
0516-1224 chvg: WARNING, once this operation is completed, volume group bbvg
        cannot be imported into AIX 5.2 or lower versions. Continue (y/n) ?
...
0516-1712 chvg: Volume group bbvg changed.  bbvg can include up to 1024 physical volumes with 2097152 total physical partitions in the volume group.

---------------------------------------

0516-008 varyonvg: LVM system call returned an unknown error code (2). 
solution: export LDR_CNTRL=MAXDATA=0x80000000@DSA (check /etc/environment if LDR_CNTRL has a value, which is causing the trouble)

---------------------------------------

If VG cannot be created:
root@aix21c: / # mkvg -y tvg hdisk29
0516-1376 mkvg: Physical volume contains a VERITAS volume group.
0516-1397 mkvg: The physical volume hdisk29, will not be added to
the volume group.
0516-862 mkvg: Unable to create volume group.
root@aixc: / # chpv -C hdisk29        <--clears owning volume manager from a disk, after this mkvg was successful

 ---------------------------------------

root@aix1: /root # importvg -L testvg -n hdiskpower12
0516-022 : Illegal parameter or structure value.
0516-780 importvg: Unable to import volume group from hdiskpower12.

For me the solution was:
there was no pvid on the disk, after adding it (chdev -l hdiskpower12 -a pv=yes) it was OK
--------------------------------------- 


Reorgvg log files, and how it is working:

reorgvg activity is logged in lvmcfg:
root@bb_lpar: / # alog -ot lvmcfg | tail -3
[S 17039512 6750244 10/23/11-12:39:05:781 reorgvg.sh 580] reorgvg bbvg bb1lv
[S 7405650 17039512 10/23/11-12:39:06:689 migfix.c 168] migfix /tmp/.workdir.9699494.17039512_1/freemap17039512 /tmp/.workdir.9699494.17039512_1/migrate17039512 /tmp/.workdir.9699494.17039512_1/lvm_moves17039512
[E 17039512 47:320 reorgvg.sh 23] reorgvg: exited with rc=0

Field of these lines:
S - Start, E - End; PID, PPID; TIMESTAMP

At E (end) line shows how long reorgvg was running (in second:milliseconds):
47:320 = 47s 320ms


for a long running reorgvg, you can see it's status:

1. check the working dir of reorgvg
root@aixdb2: /root # alog -ot lvmcfg | tail -3 | grep workdir
[S 5226546 5288122 10/22/11-13:55:11:001 migfix.c 165] migfix /tmp/.workdir.4935912.5288122_1/freemap5288122 /tmp/.workdir.4935912.5288122_1/migrate5288122 /tmp/.workdir.4935912.5288122_1/lvm_moves5288122


2. check lvm_moves file in that dir (we will need the path of this file):
root@aixdb2: /root # ls -l /tmp/.workdir.4935912.5288122_1 | grep lvm_moves
-rw-------    1 root     system      1341300 Oct 22 13:55 lvm_moves5288122

(it contains all the lp migartions, and reorgvg goes through on this file, line by line)


3. check the process of reorgvg:
root@aixdb2: /root # ps -ef | grep reorgvg
    root 5288122 5013742   0 13:52:16  pts/2  0:12 /bin/ksh /usr/sbin/reorgvg P_NAVISvg p_datlv

root@aixdb2: /root # ps -fT 5288122
 CMD
 /bin/ksh /usr/sbin/reorgvg P_NAVISvg p_datlv
 |\--lmigratepp -g 00c0ad0200004c000000012ce4ad7285 -p 00c80ef201f81fa6 -n 1183 -P 00c0ad021d62f017 -N 1565
  \--awk -F: {print "-p "$1" -n "$2" -P "$3" -N "$4 }

(lmigratepp shows: -g VGid -p SourcePVid -n SourcePPnumber -P DestinationPVid -N DestinationPPnumber)

lmigratepp shows the actual PP which is migrated at this moment
(if you ask few seconds later it will show the next PP which is migrated, and it uses the lvm_moves file)

4. check the line number of the PP which is being migrated at this moment:
(now the ps command in step 3 is extended with the content of the lvm_moves file)

root@aixdb2: /root # grep -n `ps -fT 5288122|grep migr|awk '{print $12":"$14}'` /tmp/.workdir.4935912.5288122_1/lvm_moves5288122
17943:00c0ad021d66f58b:205:00c0ad021d612cda:1259
17944:00c80ef24b619875:486:00c0ad021d66f58b:205

you can compare the above line numbers (17943, 17944) to how many lines lvm_moves file has.
root@aixdb2: /root # cat /tmp/.workdir.4935912.5288122_1/lvm_moves5288122 | wc -l
   31536

It shows that from 31536 lp migrations we are at this moment at 17943.

---------------------------------------

0516-304 : Unable to find device id 00080e82dfb5a427 in the Device
        Configuration Database.


If a disk has been deleted (rmdev) somehow, but from the vg it was not removed:
root@bb_lpar: / # lsvg -p newvg
newvg:
PV_NAME           PV STATE          TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdisk2            active            31          20          06..00..02..06..06
0516-304 : Unable to find device id 00080e82dfb5a427 in the Device
        Configuration Database.
00080e82dfb5a427  removed           31          31          07..06..06..06..06


VGDA still shows the missing disk is part of the vg:
root@bb_lpar: / # lqueryvg -tAp hdisk2
...
Physical:       00080e82dfab25bc                2   0
                00080e82dfb5a427                0   4

VGDA should be updated (on hdisk2) but it is possible only, if the PVID is used with reducevg:
root@bb_lpar: / # reducevg newvg 00080e82dfb5a427

---------------------------------------

If you run into not being able to access an hdiskpowerX disk, you may need to reset the reservation bit on it:

root@aix21: / # lqueryvg -tAp hdiskpower13
0516-024 lqueryvg: Unable to open physical volume.
        Either PV was not configured or could not be opened. Run diagnostics.

root@aix21: / # lsvg -p sapvg
PTAsapvg:
PV_NAME           PV STATE          TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdiskpower12      active            811         0           00..00..00..00..00
hdiskpower13      missing           811         0           00..00..00..00..00
hdiskpower14      active            811         0           00..00..00..00..00

root@aix21: / # bootinfo -s hdiskpower13
0


Possible solution could be emcpowerreset:
root@aix21: / # /usr/lpp/EMC/Symmetrix/bin/emcpowerreset fscsi0 hdiskpower13

(after this varyonvg will bring back the disk into active state)