root.sh script failed on first node

Today  tried to install 11gr2 rac ... as i wanted to test some scenario, but came across couple of errors and thought of sharing it here


Error : -

/u01/grid/bin/srvctl start nodeapps -n krish1 ... failed
FirstNode configuration failed at /u01/grid/crs/install/crsconfig_lib.pm line 9379.
/u01/grid/perl/bin/perl -I/u01/grid/perl/lib -I/u01/grid/crs/install /u01/grid/crs/install/rootcrs.pl execution failed



Root.sh output on first node :-

[root@krish1 ~]# /u01/grid/root.sh
Performing root user operation for Oracle 11g

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
Adding Clusterware entries to upstart
CRS-2672: Attempting to start 'ora.mdnsd' on 'krish1'
CRS-2676: Start of 'ora.mdnsd' on 'krish1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'krish1'
CRS-2676: Start of 'ora.gpnpd' on 'krish1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'krish1'
CRS-2672: Attempting to start 'ora.gipcd' on 'krish1'
CRS-2676: Start of 'ora.cssdmonitor' on 'krish1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'krish1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'krish1'
CRS-2672: Attempting to start 'ora.diskmon' on 'krish1'
CRS-2676: Start of 'ora.diskmon' on 'krish1' succeeded
CRS-2676: Start of 'ora.cssd' on 'krish1' succeeded

ASM created and started successfully.

Disk Group DATA created successfully.

clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 6845b39c31064fc8bfa54c7bbaea4ffd.
Successfully replaced voting disk group with +DATA.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   6845b39c31064fc8bfa54c7bbaea4ffd (/dev/oracleasm/disks/VOL1) [DATA]
Located 1 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'krish1'
CRS-2676: Start of 'ora.asm' on 'krish1' succeeded
CRS-2672: Attempting to start 'ora.DATA.dg' on 'krish1'
CRS-2676: Start of 'ora.DATA.dg' on 'krish1' succeeded
/u01/grid/bin/srvctl start nodeapps -n krish1 ... failed
FirstNode configuration failed at /u01/grid/crs/install/crsconfig_lib.pm line 9379.
/u01/grid/perl/bin/perl -I/u01/grid/perl/lib -I/u01/grid/crs/install /u01/grid/crs/install/rootcrs.pl execution failed

Additional logs needs to be looked at, rootcrs_hostname.log

check the log and look for the errors, you will see below error
PRCR-1013 : Failed to start resource ora.ons 
PRCR-1064 : Failed to start resource ora.ons on node krish1


 cat rootcrs_krish1.log |grep  CRS

Error desc :-
2015-05-10 22:07:49: output for start nodeapps is  PRCR-1013 : Failed to start resource ora.ons PRCR-1064 : Failed to start resource ora.ons on node krish1 CRS-5016: Process "/u01/grid/opmn/bin/onsctli" spawned by agent "/u01/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/grid/log/krish1/agent/crsd/oraagent_oracle/oraagent_oracle.log" CRS-2674: Start of 'ora.ons' on 'krish1' failed
2015-05-10 22:07:49: output of startnodeapp after removing already started mesgs is PRCR-1013 : Failed to start resource ora.ons PRCR-1064 : Failed to start resource ora.ons on node krish1 CRS-5016: Process "/u01/grid/opmn/bin/onsctli" spawned by agent "/u01/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/grid/log/krish1/agent/crsd/oraagent_oracle/oraagent_oracle.log" CRS-2674: Start of 'ora.ons' on 'krish1' failed
2015-05-10 22:07:49: Running as user oracle: /u01/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_NODECONFIG -state FAIL
2015-05-10 22:07:49: s_run_as_user2: Running /bin/su oracle -c ' /u01/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_NODECONFIG -state FAIL '
2015-05-10 22:07:49: Succeeded in writing the checkpoint:'ROOTCRS_NODECONFIG' with status:FAIL
2015-05-10 22:07:49: 'ROOTCRS_NODECONFIG' checkpoint has failed
2015-05-10 22:07:49: Running as user oracle: /u01/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_NODECONFIG -state FAIL
2015-05-10 22:07:49: s_run_as_user2: Running /bin/su oracle -c ' /u01/grid/bin/cluutil -ckpt -oraclebase /u01/app/oracle -writeckpt -name ROOTCRS_NODECONFIG -state FAIL '
2015-05-10 22:07:50: Succeeded in writing the checkpoint:'ROOTCRS_NODECONFIG' with status:FAIL

----
Cause :
Incorrect value used for localhost in /etc/hosts
IP address 127.0.0.1 should only map to localhost and/or localhost.localdomain, not anything else

Solution :

Edit localhost and ensure that entry for localhost is

127.0.0.1 localhost.localdomain localhost

For now, you have to do fresh installation only.. by making the necessary changes to /etc/hosts file.