Today tried to install 11gr2 rac ... as i wanted to test some scenario, but came across couple of errors and thought of sharing it here
Error : -
/u01/grid/bin/srvctl start nodeapps -n krish1 ... failed
FirstNode configuration failed at /u01/grid/crs/install/crsconfig_lib.pm line 9379.
/u01/grid/perl/bin/perl -I/u01/grid/perl/lib -I/u01/grid/crs/install /u01/grid/crs/install/rootcrs.pl execution failed
Root.sh output on first node :-
[root@krish1 ~]# /u01/grid/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding Clusterware entries to upstart
CRS-2672: Attempting to start 'ora.mdnsd' on 'krish1'
CRS-2676: Start of 'ora.mdnsd' on 'krish1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'krish1'
CRS-2676: Start of 'ora.gpnpd' on 'krish1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'krish1'
CRS-2672: Attempting to start 'ora.gipcd' on 'krish1'
CRS-2676: Start of 'ora.cssdmonitor' on 'krish1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'krish1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'krish1'
CRS-2672: Attempting to start 'ora.diskmon' on 'krish1'
CRS-2676: Start of 'ora.diskmon' on 'krish1' succeeded
CRS-2676: Start of 'ora.cssd' on 'krish1' succeeded
ASM created and started successfully.
Disk Group DATA created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 6845b39c31064fc8bfa54c7bbaea4ffd.
Successfully replaced voting disk group with +DATA.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 6845b39c31064fc8bfa54c7bbaea4ffd (/dev/oracleasm/disks/VOL1) [DATA]
Located 1 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'krish1'
CRS-2676: Start of 'ora.asm' on 'krish1' succeeded
CRS-2672: Attempting to start 'ora.DATA.dg' on 'krish1'
CRS-2676: Start of 'ora.DATA.dg' on 'krish1' succeeded
/u01/grid/bin/srvctl start nodeapps -n krish1 ... failed
FirstNode configuration failed at /u01/grid/crs/install/crsconfig_lib.pm line 9379.
/u01/grid/perl/bin/perl -I/u01/grid/perl/lib -I/u01/grid/crs/install /u01/grid/crs/install/rootcrs.pl execution failed
Additional logs needs to be looked at, rootcrs_hostname.log
check the log and look for the errors, you will see below error
PRCR-1013 : Failed to start resource ora.ons
PRCR-1064 : Failed to start resource ora.ons on node krish1
cat rootcrs_krish1.log |grep CRS
Error desc :-
2015-05-10 22:07:49: output for start nodeapps is PRCR-1013 : Failed to start resource ora.ons PRCR-1064 : Failed to start resource ora.ons on node krish1 CRS-5016: Process "/u01/grid/opmn/bin/onsctli" spawned by agent "/u01/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/grid/log/krish1/agent/crsd/oraagent_oracle/oraagent_oracle.log" CRS-2674: Start of 'ora.ons' on 'krish1' failed
2015-05-10 22:07:49: output of startnodeapp after removing already started mesgs is PRCR-1013 : Failed to start resource ora.ons PRCR-1064 : Failed to start resource ora.ons on node krish1 CRS-5016: Process "/u01/grid/opmn/bin/onsctli" spawned by agent "/u01/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/grid/log/krish1/agent/crsd/oraagent_oracle/oraagent_oracle.log" CRS-2674: Start of 'ora.ons' on 'krish1' failed
2015-05-10 22:07:49: Running as user oracle: /u01/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_NODECONFIG -state FAIL
2015-05-10 22:07:49: s_run_as_user2: Running /bin/su oracle -c ' /u01/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_NODECONFIG -state FAIL '
2015-05-10 22:07:49: Succeeded in writing the checkpoint:'ROOTCRS_NODECONFIG' with status:FAIL
2015-05-10 22:07:49: 'ROOTCRS_NODECONFIG' checkpoint has failed
2015-05-10 22:07:49: Running as user oracle: /u01/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_NODECONFIG -state FAIL
2015-05-10 22:07:49: s_run_as_user2: Running /bin/su oracle -c ' /u01/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_NODECONFIG -state FAIL '
2015-05-10 22:07:50: Succeeded in writing the checkpoint:'ROOTCRS_NODECONFIG' with status:FAIL
----
Cause :
Incorrect value used for localhost in /etc/hosts
IP address 127.0.0.1 should only map to localhost and/or localhost.localdomain, not anything else
For now, you have to do fresh installation only.. by making the necessary changes to /etc/hosts file.
Error : -
/u01/grid/bin/srvctl start nodeapps -n krish1 ... failed
FirstNode configuration failed at /u01/grid/crs/install/crsconfig_lib.pm line 9379.
/u01/grid/perl/bin/perl -I/u01/grid/perl/lib -I/u01/grid/crs/install /u01/grid/crs/install/rootcrs.pl execution failed
Root.sh output on first node :-
[root@krish1 ~]# /u01/grid/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding Clusterware entries to upstart
CRS-2672: Attempting to start 'ora.mdnsd' on 'krish1'
CRS-2676: Start of 'ora.mdnsd' on 'krish1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'krish1'
CRS-2676: Start of 'ora.gpnpd' on 'krish1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'krish1'
CRS-2672: Attempting to start 'ora.gipcd' on 'krish1'
CRS-2676: Start of 'ora.cssdmonitor' on 'krish1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'krish1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'krish1'
CRS-2672: Attempting to start 'ora.diskmon' on 'krish1'
CRS-2676: Start of 'ora.diskmon' on 'krish1' succeeded
CRS-2676: Start of 'ora.cssd' on 'krish1' succeeded
ASM created and started successfully.
Disk Group DATA created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 6845b39c31064fc8bfa54c7bbaea4ffd.
Successfully replaced voting disk group with +DATA.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 6845b39c31064fc8bfa54c7bbaea4ffd (/dev/oracleasm/disks/VOL1) [DATA]
Located 1 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'krish1'
CRS-2676: Start of 'ora.asm' on 'krish1' succeeded
CRS-2672: Attempting to start 'ora.DATA.dg' on 'krish1'
CRS-2676: Start of 'ora.DATA.dg' on 'krish1' succeeded
/u01/grid/bin/srvctl start nodeapps -n krish1 ... failed
FirstNode configuration failed at /u01/grid/crs/install/crsconfig_lib.pm line 9379.
/u01/grid/perl/bin/perl -I/u01/grid/perl/lib -I/u01/grid/crs/install /u01/grid/crs/install/rootcrs.pl execution failed
Additional logs needs to be looked at, rootcrs_hostname.log
check the log and look for the errors, you will see below error
PRCR-1013 : Failed to start resource ora.ons
PRCR-1064 : Failed to start resource ora.ons on node krish1
cat rootcrs_krish1.log |grep CRS
Error desc :-
2015-05-10 22:07:49: output for start nodeapps is PRCR-1013 : Failed to start resource ora.ons PRCR-1064 : Failed to start resource ora.ons on node krish1 CRS-5016: Process "/u01/grid/opmn/bin/onsctli" spawned by agent "/u01/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/grid/log/krish1/agent/crsd/oraagent_oracle/oraagent_oracle.log" CRS-2674: Start of 'ora.ons' on 'krish1' failed
2015-05-10 22:07:49: output of startnodeapp after removing already started mesgs is PRCR-1013 : Failed to start resource ora.ons PRCR-1064 : Failed to start resource ora.ons on node krish1 CRS-5016: Process "/u01/grid/opmn/bin/onsctli" spawned by agent "/u01/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/grid/log/krish1/agent/crsd/oraagent_oracle/oraagent_oracle.log" CRS-2674: Start of 'ora.ons' on 'krish1' failed
2015-05-10 22:07:49: Running as user oracle: /u01/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_NODECONFIG -state FAIL
2015-05-10 22:07:49: s_run_as_user2: Running /bin/su oracle -c ' /u01/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_NODECONFIG -state FAIL '
2015-05-10 22:07:49: Succeeded in writing the checkpoint:'ROOTCRS_NODECONFIG' with status:FAIL
2015-05-10 22:07:49: 'ROOTCRS_NODECONFIG' checkpoint has failed
2015-05-10 22:07:49: Running as user oracle: /u01/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_NODECONFIG -state FAIL
2015-05-10 22:07:49: s_run_as_user2: Running /bin/su oracle -c ' /u01/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_NODECONFIG -state FAIL '
2015-05-10 22:07:50: Succeeded in writing the checkpoint:'ROOTCRS_NODECONFIG' with status:FAIL
Cause :
Incorrect value used for localhost in /etc/hosts
IP address 127.0.0.1 should only map to localhost and/or localhost.localdomain, not anything else
Solution :
Edit localhost and ensure that entry for localhost is
127.0.0.1 localhost.localdomain localhost
For now, you have to do fresh installation only.. by making the necessary changes to /etc/hosts file.
