Create mirror copy of OCR/VOTING DISK online

OCR:
OCR can be created at the time of Grid Installation. It’s store information to manage Oracle cluster-ware and it’s component such as RAC database, listener, VIP,Scan IP & Services.
Minimum 1 and maximum 5 copy of OCR is possible.


Some key points about OCR handling in 11gR2 ASM:

1. We can store OCR And Voting disk on ASM or certified cluster file system.
2. We can dynamically add or replace OCR.
3. OCR can be keep in same disk-group or different disk-group
4. OCR automatic backup kept together in a single file.
5. Automatic backup of OCR happen after every four hours, end of the day, end of the week
6. must have root or sudo privilege account to manage it. 

To find location of Current OCR:
[oracle@rac1]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version                  :          3
Total space (kbytes)     :     262120
Used space (kbytes)      :       2964
Available space (kbytes) :     259156
ID                       : 1390115973
Device/File Name         :      +DATA
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user


Create mirror copy of OCR online
[oracle@rac1]$ sudo ocrconfig -add +CRS
Password:

Check location of OCR after mirror copy creation:
[root@rac1]# /u01/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version                  :          3
Total space (kbytes)     :     262120
Used space (kbytes)      :       2964
Available space (kbytes) :     259156
ID                       : 1390115973
Device/File Name         :      +DATA
Device/File integrity check succeeded
Device/File Name         :       +CRS
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded

Another file to find location of OCR:
[root@rac1]# cat /etc/oracle/ocr.loc
#Device/file  getting replaced by device +CRS
ocrconfig_loc=+DATA
ocrmirrorconfig_loc=+CRS

Voting Disk: 

It manage information about node membership. Each voting disk must be accessible by all nodes in the cluster. If any node is not passing heat-beat across other note or voting disk, then that node will be evicted by Voting disk.
Minimum 1 and maximum 15 copy of voting disk is possible.

Some key points about VOTING DISK handling in 11gR2:

1. Voting disks will be kept in an ASM diskgroup
2. We can dynamically add or replace voting disk
3. Redundancy of the Voting Disk depends on the Redundancy of that diskgroup
4. Voting disk can be keep in same disk-group or different disk-group
5. Voting disk automatic backup kept together in a single file
6. Backup of Voting disk using “dd” command not supported
7. Voting Disks are mirrored across Failuregroups
8. Automatic backup of Voting disk and OCR happen after every four hours, end of the day, end of the week
9. must have root or sudo privilege account to manage it
10. vote file is not mirrored and stripped like any other files (data, redo, control, OCR etc…) and it has a fixed assignmentto a failure group in a disk. 
11. Vote disk can be accessed even if ASM is not up whereas another file cannot be accessed if ASM is not up and running.
12. we cannot mix vote files on ASM and vote files on non-ASM. Instead we can dedicate either one ASM DG for all you cluster vote disk files or we can use non-ASM but never both except for initial migration to 11.2 when previous versions were on non-ASM storage. 
13. We cannot use the ‘crsctl add css votedisk’ to add a vote disk on ASM disk group or ACFS file system

A common example would be the Diskgroup DATA with redundancy normal that has at least 3 Failuregroups and contains also the Database Area of a RAC database. In this case 3 Failuregoups would each store one voting disk unstriped on one disk.

To find current location of Voting disk:

SQL> set line 200
SQL> column path format a30
SQL> select name, path, header_status from v$asm_disk; 

[oracle@rac1]$ crsctl query css votedisk
##  STATE    File Universal Id File Name Disk group
–  —–    —— —— —–——— ———
1. ONLINE   6a60a2c3510c4fbfbff62dcdc279b247 (ORCL:DATA1) [DATA]

Relocate or multiplexing Voting disk to another disk-group (With normal redundancy)
[root@rac1]# /u01/app/11.2.0/grid/bin/crsctl replace votedisk +CRS
Successful addition of voting disk afb77b2693a24f1ebfe876784103e82a.
Successful addition of voting disk 3e2542c5b1154ffdbfc8b6dea7dce390.
Successful addition of voting disk 8e0f3c5921cc4f93bf223de1465d83cc.
Successful deletion of voting disk 6a60a2c3510c4fbfbff62dcdc279b247.
Successfully replaced voting disk group with +CRS.
CRS-4266: Voting file(s) successfully replaced

New location of Voting disk:
[root@rac1]# /u01/app/11.2.0/grid/bin/crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
–  —–    —————–                ——— ———
1. ONLINE   afb77b2693a24f1ebfe876784103e82a (ORCL:DATA2) [CRS]
2. ONLINE   3e2542c5b1154ffdbfc8b6dea7dce390 (ORCL:DATA3) [CRS]
3. ONLINE   8e0f3c5921cc4f93bf223de1465d83cc (ORCL:DATA4) [CRS]
Located 3 voting disk(s).

[root@rac1]#
[root@rac1]# /u01/app/11.2.0/grid/bin/crsctl add css votedisk +data
CRS-4671: This command is not supported for ASM diskgroups.
CRS-4000: Command Add failed, or completed with errors.

[root@rac1]# /u01/app/11.2.0/grid/bin/crsctl add css votedisk /u01/vote_acfs.dsk
CRS-4258: Addition and deletion of voting files are not allowed because the voting files are on ASM
[root@raclinux2 bin]#

How ohasd will know which disk contains voting disk? 

Following is the ASM disk header metadata (dumped using kfed).
ohasd use the markers between vfstart & vfend.
If the markers between vfstart & vfend are 0 then disk does *NOT* contain voting disk

[root@rac1] kfed read ‘/dev/sdi’ | grep vf
kfdhdb.vfstart: 130; 0x0ec: 0×00000060
kfdhdb.vfend: 168; 0x0f0: 0x000000d0

[root@rac1] kfed read ‘/dev/sdg’ | grep vf
kfdhdb.vfstart: 0 ; 0x0ec: 0×00000000
kfdhdb.vfend: 0 ; 0x0f0: 0×00000000

[root@rac1] kfed read ‘/dev/sdh’ | grep vf
kfdhdb.vfstart: 0 ; 0x0ec: 0×00000000
kfdhdb.vfend: 0 ; 0x0f0: 0×00000000

So from above output, disk ‘/dev/sdi’ contains voting disk.

Conclusion: The way to handle Voting Disk and OCR has changed significantly – they can be kept inside of an ASM Diskgroup especially.