Steps to Shutdown/Start RAC Database
1. Shutdown Oracle Home process accessing database.
2. Shutdown RAC Database Instances on all nodes.
3. Shutdown All ASM instances from all nodes.
4. Shutdown Node applications running on nodes.
5. Shut down the Oracle Cluster ware or CRS.
2. Shutdown RAC Database Instances on all nodes.
3. Shutdown All ASM instances from all nodes.
4. Shutdown Node applications running on nodes.
5. Shut down the Oracle Cluster ware or CRS.
Starting steps are from 5 to 1(means which stopped last should be started 1st).
1. Shutdown Oracle Home process accessing database: There could
be oracle and non-oracle application which access database, So
first step is to stop all the applications or DBA should
inform application owner and he should stop
all applications accessing Database. DBA should also stop
Oracle application like Enterprise Manager Grid Control
and Database Listener which access database for monitoring
and database connections.
[oracle@database ~]$ emctl stop dbconsole
Oracle Enterprise Manager 11g Database Control
Release 11.2.0.2.0
Copyright (c) 1996, 2010 Oracle Corporation.
All rights reserved.
https://database.example.com:5500/em/console/aboutApplication
Stopping Oracle Enterprise Manager 11g Database
Control ...
... Stopped.
[grid@skumar1 bin]$ srvctl stop listener -n skumar1
[grid@skumar1 bin]$ srvctl status listener -n skumar1
Listener LISTENER is enabled on node(s): skumar1
Listener LISTENER is not running on node(s): skumar1
2. Shutdown RAC Database Instances
on all nodes: Suppose DBA has two node RAC database,
So he has to stop all instances from all DB nodes. Here, I am taking
anexample of two node RAC. First, I am checking on which
server database is running and then stopping and verify for the same.
Syntax: srvctl stop database -d {databasename}
[oracle@skumar2 ~]$ srvctl status database -d oradb
Instance oradb1 is running on node skumar1
Instance oradb2 is running on node skumar2
[oracle@skumar2 ~]$ srvctl stop database -d oradb
[oracle@skumar2 ~]$ srvctl status database -d oradb
Instance oradb1 is not running on node skumar1
Instance oradb2 is not running on node skumar2
We just need to
execute one command from any one of the server having database and it
will stop all database instances on all servers. If you have more
than one database configured on Nodes,
then Database Administrator has to execute this command for
each database.
3. Shutdown All ASM instances from all nodes: Next DBA has to shut down an ASM instance which are used to acess database, enter the following command, where node is the
name of the node where the ASM instance is running
Syntax: srvctl stop asm -n {node}
[grid@skumar2 oracle]# srvctl stop asm -n skumar1 -f
[grid@skumar2 oracle]# srvctl stop asm -n skumar2 -f
[grid@skumar2 oracle]# srvctl status asm -n skumar1
ASM is not running on skumar1
[grid@skumar2 oracle]# srvctl status asm -n skumar2
ASM is not running on skumar2
Sometimes, Database administrator
face some issues in stopping ASM instance, In that case use "-f"
option to forcefully shutdown ASM instances.
4.Shutdown Node applications running
on nodes: To stop node applicationsrunning on a node, enter the
following command, where node is the name of the node where
theapplications are running
[grid@skumar2 oracle]# srvctl stop nodeapps
-n skumar1 -f
[grid@skumar2 oracle]# srvctl status nodeapps -n skumar1
VIP skumar1-vip is enabled
VIP skumar1-vip is running on node: skumar1
Network is enabled
Network is running on node: skumar1
GSD is disabled
GSD is not running on node: skumar1
ONS is enabled
ONS daemon is running on node: skumar1
Repeat same command
for all nodes one by one. If you face any issue in stopping
nodeapplications use "-f" as force option to
stop applications.
5. Shut down the Oracle Clusterware or CRS: In this command all CRS related process will be stopped. This is the only command which needs to be executed by "root" user on all database nodes.
[root@skumar1 bin]# crsctl check cluster -all
**************************************************************
skumar1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is
online
CRS-4533: Event Manager is online
*************************************************************
skumar2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is
online
CRS-4533: Event Manager is online
**************************************************************
[root@skumar1 bin]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High
Availability Services-managed resources on 'skumar1'
CRS-2673: Attempting to stop 'ora.crsd' on 'skumar1'
CRS-2790: Starting shutdown of Cluster Ready
Services-managed resources on 'skumar1'
CRS-2673: Attempting to stop
'ora.LISTENER_SCAN2.lsnr' on 'skumar1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on
'skumar1'
CRS-2673: Attempting to stop
'ora.LISTENER_SCAN3.lsnr' on 'skumar1'
CRS-2673: Attempting to stop 'ora.skumar2.vip' on 'skumar1'
-------------------------------------------------
-------------------------------------------------
-------------------------------------------------
CRS-2677: Stop of 'ora.cssd' on 'skumar1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'skumar1'
CRS-2677: Stop of 'ora.gipcd' on 'skumar1'
succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'skumar1'
CRS-2677: Stop of 'ora.gpnpd' on 'skumar1' succeeded
CRS-2793: Shutdown of Oracle High Availability
Services-managed resources on 'skumar1' has completed
CRS-4133: Oracle High Availability Services has
been stopped.
[root@skumar1 bin]# crsctl check cluster -all
CRS-4639: Could not contact Oracle High
Availability Services
CRS-4000: Command Check failed, or completed with
errors.
DBA can see Now RAC
is completely down. Now you can move ahead with your patching, maintenance
activity. Next let's see how to start Oracle
RAC cluster database.
Steps to start RAC Database ?
In Starting
Real Application Clusters Database, We will also follow a set of
sequence. The order will be just opposite to Shutting down the
Real Application Clusters Database.
1. Start Oracle Clusterware or CRS.
2. Start Node applications running on nodes.
3. Start All ASM instances from all nodes.
4. Start RAC Database Instances on all nodes.
5. Start Oracle Home process accessing database.
1. Start Oracle
Clusterware or CRS: CRS starts automatically when
you start or restart Server, but Here DBA has manually shutdown the CRS So, he
has to start it manually. This is the only command which needs to be executed
by "root" user. Database Administrator should
execute this command on all nodes.
[root@skumar1 bin]# crsctl start crs
CRS-4123: Oracle High Availability Services has
been started
[root@skumar2 bin]# crsctl check cluster -all
**************************************************************
skumar1:
CRS-4535: Cannot communicate with Cluster Ready
Services
CRS-4529: Cluster Synchronization Services is
online
CRS-4533: Event Manager is online
**************************************************************
skumar2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is
online
CRS-4533: Event Manager is online
**************************************************************
Here, DBA can see "CRS-4639: Could not contact Oracle High Availability Services" or "CRS-4535: Cannot communicate with Cluster Ready Services" messages. Wait 5 minutes and then again check with "crsctl check cluster -all" command. This time Database administrator will get "CRS-4537: Cluster Ready Services is online". If still same issue DBA can start ora.crsd process to resolve this issue. Below is the command
[root@skumar1 bin]# crsctl start res ora.crsd -init
[root@skumar1 bin]# crsctl check cluster -all
**************************************************************
skumar1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is
online
CRS-4533: Event Manager is online
**************************************************************
skumar2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is
online
CRS-4533: Event Manager is online
**************************************************************
Now, Cluster is up and running. DBA can also use "ps -ef | grep d.bin" to check cluster status.
2. Start Node
applications running on nodes: To start node
applications running on a node, enter the following command, where node is the
name of the node where the applications are running
[grid@skumar1 bin]$ srvctl start nodeapps -n skumar1
[grid@skumar1 bin]$ srvctl status nodeapps -n skumar1
VIP skumar1-vip is enabled
VIP skumar1-vip is running on node: skumar1
Network is enabled
Network is running on node: skumar1
GSD is disabled
GSD is not running on node: skumar1
ONS is enabled
ONS daemon is running on node: skumar1
DBA has to execute this command for each node to start Real Application Clusters Cluster database.
3. Start All ASM
instances from all nodes: Next DBA has to start all ASM
instances which are used to access database, enter the following command, where
node is the name of the node where the ASM instance has to start.
[grid@skumar1 bin]$ srvctl start asm -n skumar1
[grid@skumar1 bin]$ srvctl status asm -n skumar1
ASM is running on skumar1
DBA has to start ASM instance on all database nodes.
4. Start RAC
Database Instances on all nodes: Now, We will start database
instances on database nodes to access data.
[grid@skumar1 bin]$ srvctl start database -d oradb
[grid@skumar1 bin]$ srvctl status database -d oradb
Instance oradb1 is running on node skumar1
Instance oradb2 is running on node skumar2
Now database is up and running on both DB nodes.
5. Start Oracle
Home process accessing database: let's start listener and OEM to
access database and allow users to connect to the database.
[grid@skumar1 bin]$ srvctl start listener -n skumar1
[grid@skumar1 bin]$ srvctl status listener -n skumar1
Listener LISTENER is enabled on node(s): skumar1
Listener LISTENER is running on node(s): skumar1
[oracle@database ~]$ emctl start dbconsole
Oracle Enterprise Manager 11g Database Control
Release 11.2.0.2.0
Copyright (c) 1996, 2010 Oracle Corporation.
All rights reserved.
Starting Oracle Enterprise Manager 11g Database Control
..........................
. started.
------------------------------------------------------------------
Logs are generated in directory
/etc/oracle/oracle/database.example.com_orcl/sysman/log
