1) Shutdown the CRS and DISABLE CRS auto start
2) Move the servers to new data center
3) Start the CRS – only on first node(./crsctl start crs)
4) Modify the IP details on OCR using oifcfg
5) Update the GPGNP profile in 11gR2
6) Now bounce CRS on rac1 and start the CRS on rac2
.........................................
note :
Now looking at 11gR2 with same scenario of moving RAC to new datacenter.
When we Start the CRS – we got below errors when we tried starting the CRS with server running on different subnet with new IP address.
” [crsd(21366)]CRS-0804:Cluster Ready Service aborted due to Oracle Cluster Registry error [PROC-44: Error in network address and interface operations Network address and interface operations error [7]]. Details at (:CRSD00111:) in /u01/app/grid/11.2.0/log/rac1/crsd/crsd.log.”
What is the difference between 10g/11gR1 and 11gR2??
As of 11.2 Grid Infrastructure, the CRS daemon (crsd.bin) now has a dependency on the private network configuration stored in the gpnp profile and OCR. If the private network is not available or its definition is incorrect, the CRSD process will not start and any subsequent changes to the OCR will be impossible NOTE:1094024.1
5) Update the GPGNP profile in 11gR2
STEPS UPDATE TO 11GR2 GRID GPGNP PROFILE
(run all the steps on rac1 as ROOT unless specified dont run on other nodes)
STEP1: Start CRS on all the nodes (though CRSD won’t come up we need gpnp deamon running on all the cluster nodes)
STEP 2: On rac1: As the from 11gR2 – CRS startup is based on GPNP profile we need to update profile to reflect new IP address (to view current profile infomation
STEP 3: on rac1: Take backup of existing GPNP profile before editing (we are going have temp profile XML file and do our modification there in that file)
STEP 4: on rac1: Get current profile sequence number – we need to increment it while editing profile
i.e current prf_sq +1
STEP 7: on rac1: Sign the XML file with GPNP tool and upload it to update the current profile
STEP 8: on All the nodes : Start the CRS and check Status
STEP 9: Now we run oifcfg getif — interesting piece of warning/error which ignorable as we edited GPNP profile and its not matching with OCR
STEP 10: update OCR configuration using oifcfg
STEP 11: Modifing SCAN IP is very simple compare to VIP/Interconnect changes.
2) Move the servers to new data center
3) Start the CRS – only on first node(./crsctl start crs)
4) Modify the IP details on OCR using oifcfg
5) Update the GPGNP profile in 11gR2
6) Now bounce CRS on rac1 and start the CRS on rac2
.........................................
note :
PUB.LI.CI.PP -- public ip address (eth1)
PRI.VA.TE.IP -- private ip address ( eth2)
4) Modify the IP details on OCR using oifcfg
root@rac1 bin]# ./oifcfg delif -global eth1 root@rac1 bin]# ./oifcfg setif -global eth1/PUB.LI.CI.PP:public -- New ip address netmask root@rac1 bin]# ./oifcfg delif -global eth2 root@rac1 bin]# ./oifcfg setif -global eth2/PRI.VA.TE.IP:cluster_interconnect --- New ip address netmask root@rac1 bin]# ./oifcfg getif eth1 PUB.LI.CI.PP global public eth2 PRI.VA.TE.IP global cluster_interconnect root@rac1 bin]# ./srvctl config nodeapps ---- VIP exists: /rac1-vip/<newip>/<old netmask>/255.255.255.128/bond0, hosting node rac1 VIP exists: /rac2-vip/<newip>/<old netmask>/255.255.255.128/bond0, hosting node rac2 ---- [root@rac1 bin]# ./srvctl modify nodeapps -n rac1 -A rac1-vip.us.oracle.com/255.255.255.0/eth1 [root@rac1 bin]# ./srvctl modify nodeapps -n rac2 -A rac2-vip.us.oracle.com/255.255.255.0/eth1 [root@rac1 bin]# ./srvctl config nodeapps ---- VIP exists: /rac1-vip/<New IP address>/<PUB.LI.CI.PP>/255.255.255.0/eth1, hosting node rac1 VIP exists: /rac2-vip/<New IP address>/<PUB.LI.CI.PP>/255.255.255.0/eth1, hosting node rac2 ----when try to bounce CRS on rac1 and start the CRS on rac2
Now looking at 11gR2 with same scenario of moving RAC to new datacenter.
When we Start the CRS – we got below errors when we tried starting the CRS with server running on different subnet with new IP address.
” [crsd(21366)]CRS-0804:Cluster Ready Service aborted due to Oracle Cluster Registry error [PROC-44: Error in network address and interface operations Network address and interface operations error [7]]. Details at (:CRSD00111:) in /u01/app/grid/11.2.0/log/rac1/crsd/crsd.log.”
What is the difference between 10g/11gR1 and 11gR2??
As of 11.2 Grid Infrastructure, the CRS daemon (crsd.bin) now has a dependency on the private network configuration stored in the gpnp profile and OCR. If the private network is not available or its definition is incorrect, the CRSD process will not start and any subsequent changes to the OCR will be impossible NOTE:1094024.1
5) Update the GPGNP profile in 11gR2
STEPS UPDATE TO 11GR2 GRID GPGNP PROFILE
(run all the steps on rac1 as ROOT unless specified dont run on other nodes)
STEP1: Start CRS on all the nodes (though CRSD won’t come up we need gpnp deamon running on all the cluster nodes)
[root@rac1 bin]# cd $GRID_HOME/bin/
[root@rac1 bin]#./crsctl start crs
[root@rac1 bin]# ps -ef | grep gpnp
STEP 2: On rac1: As the from 11gR2 – CRS startup is based on GPNP profile we need to update profile to reflect new IP address (to view current profile infomation
[root@rac1 bin]# cd $GRID_HOME/bin/
[root@rac1 bin]# ./gpnptool get >> we can notice the old ip and interface name
STEP 3: on rac1: Take backup of existing GPNP profile before editing (we are going have temp profile XML file and do our modification there in that file)
[root@rac1 bin]#mkdir /home/oracle/gpnp [root@rac1 bin]#export GPNPDIR=/home/oracle/gpnp [root@rac1 bin]# ./gpnptool get -o=$GPNPDIR/profile.original [root@rac1 bin]# cp $GPNPDIR/profile.original $GPNPDIR/p.xml
STEP 4: on rac1: Get current profile sequence number – we need to increment it while editing profile
i.e current prf_sq +1
[root@rac1 bin]# ./gpnptool getpval -p=$GPNPDIR/p.xml -prf_sq -o- ===> this output +1 will be used for prf_sq during edit commandSTEP 5: on rac1: Available network interfaces
[root@rac1 bin]#./oifcfg iflistSTEP 6: on rac1: Get the current NETWORK ID from profile and crosscheck the xml file
[root@rac1 bin]# ./gpnptool getpval -p=$GPNPDIR/p.xml -net -o- exampleOUTPUT : [root@rac1 bin]#./gpnptool getpval -p=$GPNPDIR/p.xml -net -o- net1 net2 [root@rac1 bin]#more $GPNPDIR/p.xml (compare the above net1/net2 with xml file) and append below command based on " (d), (e) and (f)" [root@rac1 bin]#./gpnptool edit -p=$GPNPDIR/p.xml -o=$GPNPDIR/p.xml -ovr -prf_sq=2 -net1:net_use=public -net1:net_ip=PUB.LI.CI.PP -net1:net_ada=eth3 [root@rac1 bin]#./gpnptool edit -p=$GPNPDIR/p.xml -o=$GPNPDIR/p.xml -ovr -prf_sq=2 -net2:net_use=cluster_interconnect -net2:net_ip=PRI.VA.TE.IP -net2:net_ada=eth4 [root@rac1 bin]#more $GPNPDIR/p.xml >> we should be seeing new ip and interface names here
STEP 7: on rac1: Sign the XML file with GPNP tool and upload it to update the current profile
[root@rac1 bin]#./gpnptool sign -p=$GPNPDIR/p.xml -o=$GPNPDIR/p.xml -ovr -w=cw-fs:peer [root@rac1 bin]#./gpnptool put -p=$GPNPDIR/p.xml >> At this step if GPNP deamon not running then profile won't get updated on that node so we need to make sure it running on all the cluster nodes
STEP 8: on All the nodes : Start the CRS and check Status
[root@rac1 bin]#./gpnptool get >> we will be seeing new IP and interface name [root@rac1 bin]#./crsctl start res ora.crsd –init [root@rac1 bin]#./crsctl status res -tAt this point all the CRS deamons will be up (make sure listener/DB not up)
STEP 9: Now we run oifcfg getif — interesting piece of warning/error which ignorable as we edited GPNP profile and its not matching with OCR
[root@rac1 bin]# ./oifcfg getif eth3 PUB.LI.CI.PP global public eth4 PRI.VA.TE.IP global cluster_interconnect Only in OCR: bond0 SCA.NV.IP.IP global public Only in OCR: bond2 YYY.YY.yy.yy global cluster_interconnect PRIF-30: Network information in OCR and GPnP profile differs
STEP 10: update OCR configuration using oifcfg
[root@rac1 bin]# ./oifcfg delif -global bond0 [root@rac1 bin]# ./oifcfg delif -global bond2 [root@rac1 bin]# ./oifcfg setif -global eth3/PUB.LI.CI.PP:public [root@rac1 bin]#./oifcfg setif -global eth4/PRI.VA.TE.IP:cluster_interconnect
STEP 11: Modifing SCAN IP is very simple compare to VIP/Interconnect changes.
[root@rac1 bin]# ./srvctl config scan -- Verifiying existing configuration SCAN name: node-scan, Network: 1/PUB.LI.CI.PP/255.255.255.0/eth3 SCAN VIP name: scan1, IP: /node-scan/SCA.NV.IP.IP SCAN VIP name: scan2, IP: /node-scan/SCA.NV.IP.IP SCAN VIP name: scan3, IP: /node-scan/SCA.NV.IP.IP [root@rac1 bin]# ./srvctl config scan_listener -- Verifiying existing configuration SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521 SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521 SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521 [root@rac1 bin]# ./srvctl status scan_listener -- Making sure scan listener is down SCAN Listener LISTENER_SCAN1 is enabled SCAN listener LISTENER_SCAN1 is not running SCAN Listener LISTENER_SCAN2 is enabled SCAN listener LISTENER_SCAN2 is not running SCAN Listener LISTENER_SCAN3 is enabled SCAN listener LISTENER_SCAN3 is not running [root@rac1 bin]# ./srvctl modify scan -n node-scan -- Modifying SCAN IP's on OCR, this command will pickup the new ip addres from DNS and updates OCR [root@rac1 bin]# ./srvctl config scan -- Verifying new ip's registered on OCR SCAN name: node-scan, Network: 1/PUB.LI.CI.PP/255.255.255.0/eth3 SCAN VIP name: scan1, IP: /node-scan/PUB.LI.CI.PP SCAN VIP name: scan2, IP: /node-scan/PUB.LI.CI.PP SCAN VIP name: scan3, IP: /node-scan/PUB.LI.CI.PPDo clean bounce of CRS on all the nodes using
- crsctl stop crs
- crsctl start crs
- crsctl enable crs
No comments:
Post a Comment