DBA Plus Workshop: opatchauto out-of-place patching Oracle Restart and Database home

Out of place (OOP) patching is a new feature introduced by Oracle 12.2 with the latest version of opatchauto. It is a patching method where patching is performed by following operations,

* Creating a copy of current oracle home

* Applying patches to the copied home

* Switching services to the copied home

This patching method helps on moving Oracle home and/or reducing the downtime of patching Oracle home and database. OPatchauto supports OOP patching for the following configurations,

* Grid Infrastructure (GI) and Real Application Cluster (RAC)

* Oracle Restart (Standalone Grid Infrastructure) (SIHA) and single instance database (SIDB)

I am going to out-of-place patch SIHA and SIDB from 12.2.0.1.20191015 (12.2 Release Update Oct 2019) to 12.2.0.1.20200414 (12.2 Release Update April 2020).

1. Download following Oracle utility/patch files from My Oracle Support

p6880880_122010_SOLARIS64.zip - Latest version (12.2.0.1.19) of OPatch for Solaris on SPARC (64-bit)

p30805580_122010_SOLARIS64.zip - Oracle JavaVM Component Release Update 12.2.0.1.200414

p30920127_122010_SOLARIS64.zip - GI Apr 2020 Release Update 12.2.0.1.200414

Including Database Apr 2020 Release Update 12.2.0.1.200414

p25087834_12201200414DBAPR2020RU_SOLARIS64.zip - ORACLE DATABASE Patch for Bug# 25087834

All these files are created for Solaris on SPARC (64-bit) Platform.

2. Upgrade OPatch to latest version

It is a best practice for most Oracle database patching process to upgrade OPatch utility to latest version before patching, though latest version is found issues in very a few circumstances where old version works fine.

Backup directory $ORACLE_HOME/OPatch for SIHA as user root,

[root@host01]# ls -ld $ORACLE_HOME

drwxr-x--- 98 root oinstall 115 Apr 24 15:58 /u01/app/grid/product/12.2.0/grid_1

[root@host01]# cd $ORACLE_HOME

[root@host01]# ls -ld OPatch *

drwxr-xr-x 14 grid oinstall 28 Nov 20 14:13 OPatch

[root@host01]# mv OPatch OPatch.bak

[root@host01]# mkdir OPatch

[root@host01]# chown grid:oinstall OPatch

[root@host01]# ls -ld OPatch*

drwxr-xr-x 2 grid oinstall 2 Apr 22 09:58 OPatch

drwxr-xr-x 14 grid oinstall 28 Nov 20 14:13 OPatch.bak

Although grid is home owner of SIHA home, it needs root to move and create directory under the home directory because grid does not have permission to move and create directory under home directory(root has the ownership of directory /u01/app/grid/product/12.2.0/grid_1).

Install new OPatch in SIHA home as home owner grid,

[grid@host01]$ echo $ORACLE_HOME

/u01/app/grid/product/12.2.0/grid_1

[grid@host01]$ unzip -q -d $ORACLE_HOME /u01/download/p6880880_122010_SOLARIS64.zip

[grid@host01]$ $ORACLE_HOME/OPatch/opatch version

OPatch Version: 12.2.0.1.19

OPatch succeeded.

Backup old OPatch and install new OPatch in SIDB home as home owner oracle

[oracle@host01]$ ls -ld $ORACLE_HOME/OPatch

drwxr-x--- 14 oracle oinstall 28 Apr 22 10:24 /u01/app/oracle/product/12.2.0/dbhome_1/OPatch

[oracle@host01]$ cd $ORACLE_HOME

[oracle@host01]$ ls -ld OPatch*

drwxr-xr-x 14 oracle oinstall 28 Nov 20 15:57 OPatch

[oracle@host01]$ mv OPatch OPatch.bak

[oracle@host01]$ unzip -q /u01/download/p6880880_122010_SOLARIS64.zip

[oracle@host01]$ $ORACLE_HOME/OPatch/opatch version

OPatch Version: 12.2.0.1.19

OPatch succeeded.

3. Validate patch level of Oracle Home

List patches of SIHA home as user grid

[grid@host01]$ $ORACLE_HOME/OPatch/opatch lspatches

30122828;ACFS OCT 2019 RELEASE UPDATE 12.2.0.1.0 (30122828)

30122814;OCW OCT 2019 RELEASE UPDATE 12.2.0.1.191015 (30122814)

30093408;TOMCAT RELEASE UPDATE 12.2.0.1.0(ID:190208.0920) (30093408)

30138470;Database Oct 2019 Release Update : 12.2.0.1.191015 (30138470)

26839277;DBWLM RELEASE UPDATE 12.2.0.1.0(ID:170913) (26839277)

OPatch succeeded.

Patch 30116802 was installed in Oracle Restart home. It is Grid Infrastructure Oct 2019 Release Update 12.2.0.1.191015 Bundle Patch and includes five patches 30122828, 30122814, 30093408, 30138470 and 26839277.

List patches of SIDB home as user oracle

[oracle@host01]$ $ORACLE_HOME/OPatch/opatch lspatches

30133625;OJVM RELEASE UPDATE: 12.2.0.1.191015 (30133625)

25087834;

30122814;OCW OCT 2019 RELEASE UPDATE 12.2.0.1.191015 (30122814)

30138470;Database Oct 2019 Release Update : 12.2.0.1.191015 (30138470)

OPatch succeeded.

Four patches are installed in database home. The patching process will upgrade these patches as following,

OJVM: Patch 30133625 (RU 12.2.0.1.191015) -> Patch 30805580 (RU 12.2.0.1.200414)

OCW: Patch 30122814 (RU 12.2.0.1.191015) -> Patch 30882603 (RU 12.2.0.1.200414)

DB: Patch 30138470 (RU 12.2.0.1.191015) -> Patch 30886680 (RU 12.2.0.1.200414)

Note: one-off patch 25087834 is specific to database RU version, current patch version conflicts with database patch 30886680. Therefore, patch 25087834 has to be rolled back before patch 30886680 can be applied. New version of patch 25087834 will be applied after patch 30886680.

4. Unzip the patches

Make sure you are using correct version of unzip utility. Use $ORACLE_HOME/bin/unzip if you do not know which unzip should be used.

As grid

[grid@host01]$ mkdir -p /u01/stage/grid

[grid@host01]$ unzip -q -d /u01/stage/grid /u01/download/p30920127_122010_SOLARIS64.zip

As oracle

[oracle@host01]$ mkdir -p /u01/stage/database

[oracle@host01]$ unzip -q -d /u01/stage/database /u01/download/p30920127_122010_SOLARIS64.zip

[oracle@host01]$ unzip -q -d /u01/stage/database /u01/download/p30805580_122010_SOLARIS64.zip

[oracle@host01]$ mkdir -p /u01/stage/one-off

[oracle@host01]$ unzip -q -d /u01/stage/one-off /u01/download/p25087834_12201200414DBAPR2020RU_SOLARIS64.zip

5. Run OPatch Conflict Check

For SIHA home, run following commands as home owner grid

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/stage/grid/30920127/30886680

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/stage/grid/30920127/30882603

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/stage/grid/30920127/30869447

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/stage/grid/30920127/26839277

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/stage/grid/30920127/30888810

Following is sample output of successful OPatch Conflict check, it means the patch can be applied on current home without conflict

Oracle Interim Patch Installer version 12.2.0.1.19

PREREQ session

Oracle Home : /u01/app/grid/product/12.2.0/grid_1

Central Inventory : /u01/app/oraInventory

from : /u01/app/grid/product/12.2.0/grid_1/oraInst.loc

OPatch version : 12.2.0.1.19

OUI version : 12.2.0.1.4

Log file location : /u01/app/grid/product/12.2.0/grid_1/cfgtoollogs/opatch/opatch2020-04-22_11-12-14AM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.

For database home, run following commands as home owner oracle

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/stage/database/30920127/30886680

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/stage/database/30920127/30882603

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/stage/database/30805580

The check succeeds with patch 30882603 and 30805580, but fails with 30886680 (Database Apr 2020 Release Update) because of conflict with patch 25087834. Following is sample output of failed OPatch Conflict check

[oracle@host01]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/stage/database/30920127/30886680

Oracle Interim Patch Installer version 12.2.0.1.19

PREREQ session

Oracle Home : /u01/app/oracle/product/12.2.0/dbhome_1

Central Inventory : /u01/app/oraInventory

from : /u01/app/oracle/product/12.2.0/dbhome_1/oraInst.loc

OPatch version : 12.2.0.1.19

OUI version : 12.2.0.1.4

Log file location : /u01/app/oracle/product/12.2.0/dbhome_1/cfgtoollogs/opatch/opatch2020-04-22_11-47-20AM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

ZOP-47: The patch(es) has supersets with other patches installed in the Oracle Home (or) among themselves.

Prereq "checkConflictAgainstOHWithDetail" failed.

The details are:

Please rebuild the superset patch [30886680] to make sure it supersedes all the relevant patch(es) [30138470,25087834].

The rebuild patch should contain bug fix [23738304, 27203055, 25223839, 25963024, 28388910, 25437699, 26588069, 25435038, 27607563, 29376346, 26546664,

<<message truncated>>

25335249, 28537715, 28856060, 26091786, 24573817, 28951382, 26327624, 20003668, 30244787, 30218044, 27242226, 25489607, 29013832, 26373967, 25442559].

Summary of Conflict Analysis:

There are no patches that can be applied now.

Following patches have conflicts. Please contact Oracle Support and get the merged patch of the patches :

25087834, 30886680

Conflicts/Supersets for each patch are:

Patch : 30886680

Conflict with 25087834

Conflict details:

/u01/app/oracle/product/12.2.0/dbhome_1/rdbms/admin/kspasv5.var

/u01/app/oracle/product/12.2.0/dbhome_1/lib/libserver12.a:qesmm.o

Bug Superset of 30138470

Super set bugs are:

25223839, 25963024, 25437699, 25435038, 27607563, 29376346, 26546664, 25872127, 29483723, 28774416, 24907917, 23741944, 24796092, 22310426, 27369515,

<<message truncated>>

27242226, 25489607, 29013832, 25442559

OPatch succeeded.

Therefore, one-off patch 25087834 has to be rolled back in new Oracle Database home before patch 30886680 is applied.

6. Prepare a clone home by running the following command as user root against Oracle homes (both SIHA and SIDB)

$ORACLE_HOME/OPatch/opatchauto apply -phBaseDir <patch-base-directory> -prepare-clone -oh $ORACLE_HOME -logLevel FINEST

Here, option -logLevel FINEST is optional. But detailed log information could be more helpful when diagnosing opatchauto errors.

For SIHA home

[root@host01]# echo $ORACLE_HOME

/u01/app/grid/product/12.2.0/grid_1

[root@host01]# $ORACLE_HOME/OPatch/opatchauto apply -phBaseDir /u01/stage/grid -prepare-clone -oh $ORACLE_HOME -logLevel FINEST

OPatchauto session is initiated at Wed Apr 22 18:04:16 2020

System initialization log file is /u01/app/grid/product/12.2.0/grid_1/cfgtoollogs/opatchautodb/systemconfig2020-04-22_06-04-24PM.log.

Session log file is /u01/app/grid/product/12.2.0/grid_1/cfgtoollogs/opatchauto/opatchauto2020-04-22_06-04-35PM.log

The id for this session is DP1E

Please press ENTER button to accept system generated default clone path.

Please enter clone path [/u01/app/grid/product/12.2.0/grid_2 ] :

Executing OPatch prereq operations to verify patch applicability on home /u01/app/grid/product/12.2.0/grid_1

Patch applicability verified successfully on home /u01/app/grid/product/12.2.0/grid_1

Copying the files from the existing oracle home /u01/app/grid/product/12.2.0/grid_1 to a new location. Please wait...

Clone of oracle home /u01/app/grid/product/12.2.0/grid_1 is /u01/app/grid/product/12.2.0/grid_2 on host host01

Copying the files from the existing oracle home /u01/app/grid/product/12.2.0/grid_1 to a new location is successful.

Unlocking CRS clone home for home /u01/app/grid/product/12.2.0/grid_1.

Prepatch operation log file location: /u01/app/grid/crsdata/host01/crsconfig/hapatch_2020-04-22_06-11-34PM.log

Unlocked CRS clone home successfully for home /u01/app/grid/product/12.2.0/grid_1.

Creating clone for oracle home /u01/app/grid/product/12.2.0/grid_1.

Clone operation successful for oracle home /u01/app/grid/product/12.2.0/grid_1.

Performing post clone operation for oracle home /u01/app/grid/product/12.2.0/grid_1.

Performing post clone operation was successful for oracle home /u01/app/grid/product/12.2.0/grid_2.

Start applying binary patch on home /u01/app/grid/product/12.2.0/grid_2

Binary patch applied successfully on home /u01/app/grid/product/12.2.0/grid_2

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:host01

SIHA Home:/u01/app/grid/product/12.2.0/grid_1

Version:12.2.0.1.0

Summary:

==Following patches were SKIPPED:

Patch: /u01/stage/grid/30920127/26839277

Reason: This patch is already been applied, so not going to apply again.

==Following patches were SUCCESSFULLY applied:

Patch: /u01/stage/grid/30920127/30869447

Log: /u01/app/grid/product/12.2.0/grid_2/cfgtoollogs/opatchauto/core/opatch/opatch2020-04-22_18-13-10PM_1.log

Patch: /u01/stage/grid/30920127/30882603

Log: /u01/app/grid/product/12.2.0/grid_2/cfgtoollogs/opatchauto/core/opatch/opatch2020-04-22_18-13-10PM_1.log

Patch: /u01/stage/grid/30920127/30886680

Log: /u01/app/grid/product/12.2.0/grid_2/cfgtoollogs/opatchauto/core/opatch/opatch2020-04-22_18-13-10PM_1.log

Patch: /u01/stage/grid/30920127/30888810

Log: /u01/app/grid/product/12.2.0/grid_2/cfgtoollogs/opatchauto/core/opatch/opatch2020-04-22_18-13-10PM_1.log

Out of place patching clone home(s) summary

____________________________________________

Host : host01

Actual Home : /u01/app/grid/product/12.2.0/grid_1

Version:12.2.0.1.0

Clone Home Path : /u01/app/grid/product/12.2.0/grid_2

OPatchauto session completed at Wed Apr 22 18:30:44 2020

Time taken to complete the session 26 minutes, 29 seconds

For SIDB home, command option -force_conflict will be used to roll back conflict patch (Patch 25087834)

[root@host01]# echo $ORACLE_HOME

/u01/app/oracle/product/12.2.0/dbhome_1

[root@host01]# $ORACLE_HOME/OPatch/opatchauto apply -phBaseDir /u01/stage/database -prepare-clone -oh $ORACLE_HOME -force_conflict -logLevel FINEST

OPatchauto session is initiated at Wed Apr 22 19:47:19 2020

System initialization log file is /u01/app/oracle/product/12.2.0/dbhome_1/cfgtoollogs/opatchautodb/systemconfig2020-04-22_07-47-28PM.log.

Session log file is /u01/app/oracle/product/12.2.0/dbhome_1/cfgtoollogs/opatchauto/opatchauto2020-04-22_07-48-05PM.log

The id for this session is PVMP

Please press ENTER button to accept system generated default clone path.

Please enter clone path [/u01/app/oracle/product/12.2.0/dbhome_2 ] :

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/12.2.0/dbhome_1

Patch applicability verified successfully on home /u01/app/oracle/product/12.2.0/dbhome_1

Verifying SQL patch applicability on home /u01/app/oracle/product/12.2.0/dbhome_1

SQL patch applicability verified successfully on home /u01/app/oracle/product/12.2.0/dbhome_1

Copying the files from the existing oracle home /u01/app/oracle/product/12.2.0/dbhome_1 to a new location. Please wait...

Clone of oracle home /u01/app/oracle/product/12.2.0/dbhome_1 is /u01/app/oracle/product/12.2.0/dbhome_2 on host host01

Copying the files from the existing oracle home /u01/app/oracle/product/12.2.0/dbhome_1 to a new location is successful.

Creating clone for oracle home /u01/app/oracle/product/12.2.0/dbhome_1.

Clone operation successful for oracle home /u01/app/oracle/product/12.2.0/dbhome_1.

Performing post clone operation for oracle home /u01/app/oracle/product/12.2.0/dbhome_1.

Performing post clone operation was successful for oracle home /u01/app/oracle/product/12.2.0/dbhome_2.

Start applying binary patch on home /u01/app/oracle/product/12.2.0/dbhome_2

Binary patch applied successfully on home /u01/app/oracle/product/12.2.0/dbhome_2

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:host01

SIDB Home:/u01/app/oracle/product/12.2.0/dbhome_1

Version:12.2.0.1.0

Summary:

==Following patches were SKIPPED:

Patch: /u01/stage/database/30920127/30869447

Reason: This patch is not applicable to this specified target type - "oracle_database"

Patch: /u01/stage/database/30920127/26839277

Reason: This patch is not applicable to this specified target type - "oracle_database"

Patch: /u01/stage/database/30920127/30888810

Reason: This patch is not applicable to this specified target type - "oracle_database"

==Following patches were SUCCESSFULLY applied:

Patch: /u01/stage/database/30805580

Log: /u01/app/oracle/product/12.2.0/dbhome_2/cfgtoollogs/opatchauto/core/opatch/opatch2020-04-22_20-55-43PM_1.log

Patch: /u01/stage/database/30920127/30882603

Log: /u01/app/oracle/product/12.2.0/dbhome_2/cfgtoollogs/opatchauto/core/opatch/opatch2020-04-22_20-55-43PM_1.log

Patch: /u01/stage/database/30920127/30886680

Log: /u01/app/oracle/product/12.2.0/dbhome_2/cfgtoollogs/opatchauto/core/opatch/opatch2020-04-22_20-55-43PM_1.log

Out of place patching clone home(s) summary

____________________________________________

Host : host01

Actual Home : /u01/app/oracle/product/12.2.0/dbhome_1

Version:12.2.0.1.0

Clone Home Path : /u01/app/oracle/product/12.2.0/dbhome_2

OPatchauto session completed at Wed Apr 22 21:08:04 2020

Time taken to complete the session 80 minutes, 45 seconds

It can be seen that new home (clone home) was created with new patches Database Release Update 12.2.0.1.200414 (Patch 30886680) and OJVM Release Update 12.2.0.1.200414 (Patch 30805580). However, the output does not show any information about patch 25087834 which conflicts with patch 30886680.

Check out Patch 25087834 status from log file opatch2020-04-22_20-55-43PM_1.log

[Apr 22, 2020 8:56:40 PM] [INFO] Checking conflicts against Oracle Home...

[Apr 22, 2020 8:56:40 PM] [INFO] Running prereq checkConflictAgainstOHWithDetail

[Apr 22, 2020 8:56:42 PM] [INFO] Following patches can be applied: 30886680

[Apr 22, 2020 8:56:42 PM] [INFO] Following patches are not required:

[Apr 22, 2020 8:56:42 PM] [INFO] Following patches are auto rollbackable: 25087834

[Apr 22, 2020 8:56:42 PM] [INFO] Finished checking prereq checkConflictAgainstOHWithDetail

[Apr 22, 2020 8:56:42 PM] [INFO] Inter-Conflict check PASSED

<<message truncated>>

[Apr 22, 2020 9:09:21 PM] [INFO] OUI-67087:ApplySession rolling back interim patch '25087834' from OH '/u01/app/oracle/product/12.2.0/dbhome_2'

[Apr 22, 2020 9:09:21 PM] [INFO] Rolling back interim patch '25087834' from OH '/u01/app/oracle/product/12.2.0/dbhome_2'

[Apr 22, 2020 9:09:21 PM] [INFO] Starting auto rollback of patch 25087834

[Apr 22, 2020 9:09:21 PM] [INFO] Rollback preScript for patch 25087834 processed.

The log shows that Patch 25087834 was rolled back during prepare-clone.

Check the patch level of clone home,

[oracle@host01]$ export ORACLE_HOME=/u01/app/oracle/product/12.2.0/dbhome_2

[oracle@host01]$ $ORACLE_HOME/OPatch/opatch lspatches

30886680;Database Apr 2020 Release Update : 12.2.0.1.200414 (30886680)

30882603;OCW APR 2020 RELEASE UPDATE 12.2.0.1.200414 (30882603)

30805580;OJVM RELEASE UPDATE: 12.2.0.1.200414 (30805580)

OPatch succeeded.

Note: Do not change configurations in both original home and clone home between prepare-clone and switch-clone (next step). Otherwise, switch-clone will fail because of change of configurations.

If you dropped a database from original home (e.g. DB03 is dropped) after prepare-clone, switch-clone will fail with errors like following,

[root@magnum]# $ORACLE_HOME/OPatch/opatchauto apply -switch-clone -oh $ORACLE_HOME -logLevel FINEST

OPatchauto session is initiated at Fri Apr 24 16:29:15 2020

Session log file is /u01/app/oracle/product/12.2.0/dbhome_1/cfgtoollogs/opatchauto/opatchauto2020-04-24_04-29-24PM.log

The id for this session is Q48R

<<message truncated>>

Modifying the database(s) to use new location home /u01/app/oracle/product/12.2.0/dbhome_2.

Failed to configure database to run from Oracle home home /u01/app/oracle/product/12.2.0/dbhome_2.

Execution of [DatabaseStartupFinalizeAction] patch action failed, check log for more details. Failures:

Patch Target : host01->/u01/app/oracle/product/12.2.0/dbhome_2 Type[sidb]

Details: [

---------------------------Patching Failed---------------------------------

Command execution failed during patching in home: /u01/app/oracle/product/12.2.0/dbhome_2, host: host01.

Command failed: /u01/app/oracle/product/12.2.0/dbhome_1/bin/srvctl config database -d DB03 -a

Command failure output:

PRCD-1120 : The resource for database DB03 could not be found.

PRCR-1001 : Resource ora.db03.db does not exist

After fixing the cause of failure Run opatchauto resume

]

OPATCHAUTO-68061: The orchestration engine failed.

OPATCHAUTO-68061: The orchestration engine failed with return code 1

OPATCHAUTO-68061: Check the log for more details.

OPatchAuto failed.

OPatchauto session completed at Fri Apr 24 16:29:59 2020

Time taken to complete the session 0 minute, 44 seconds

opatchauto failed with error code 42

If you applied new patch (e.g. new version of patch 25087834 in clone home), switch-clone will fail with errors like that,

[root@host01]# $ORACLE_HOME/OPatch/opatchauto apply -switch-clone -oh $ORACLE_HOME -logLevel FINEST

OPatchauto session is initiated at Fri Apr 24 16:15:32 2020

Session log file is /u01/app/oracle/product/12.2.0/dbhome_1/cfgtoollogs/opatchauto/opatchauto2020-04-24_04-15-41PM.log

The id for this session is I34J

OPATCHAUTO-72137: Cannot execute out of place patching switch session.

OPATCHAUTO-72137: Oracle Home patch level is changed.

OPATCHAUTO-72137: Please bring all oracle home to original patch level and re-run out of place patching switch operation.

Host :: host01

==============================================

Original Patch Level ::

Oracle Home /u01/app/oracle/product/12.2.0/dbhome_1: Patch ID: 30133625 Patch ID: 25087834 Patch ID: 30122814 Patch ID: 30138470

Oracle Home /u01/app/oracle/product/12.2.0/dbhome_2: Patch ID: 30886680 Patch ID: 30882603 Patch ID: 30805580

==============================================

Current Patch Level ::

Oracle Home /u01/app/oracle/product/12.2.0/dbhome_1: Patch ID: 30133625 Patch ID: 25087834 Patch ID: 30122814 Patch ID: 30138470

Oracle Home /u01/app/oracle/product/12.2.0/dbhome_2: Patch ID: 25087834 Patch ID: 30886680 Patch ID: 30882603 Patch ID: 30805580

OPatchauto session completed at Fri Apr 24 16:15:57 2020

Time taken to complete the session 0 minute, 26 seconds

opatchauto failed with error code 106

7. Switch to clone home by running the following command as root

$ORACLE_HOME/OPatch/opatchauto apply -switch-clone -oh $ORACLE_HOME -logLevel FINEST

Switch SIHA home,

[root@host01]# echo $ORACLE_HOME

/u01/app/grid/product/12.2.0/grid_1

[root@dr-jurassic]# $ORACLE _HOME/OPatch/opatchauto apply -switch-clone -oh $ORACLE _HOME -logLevel FINEST

OPatchauto session is initiated at Fri Apr 24 15:27:48 2020

Session log file is /u01/app/grid/12.2.0/grid_1/cfgtoollogs/opatchauto/opatchauto2020-04-24_03-28-07PM.log

The id for this session is DSN1

Verifying clone home details before switch for oracle home /u01/app/grid/product/12.2.0/grid_2.

Verified clone home details successfully for oracle home /u01/app/grid/product/12.2.0/grid_2 before switch.

Update nodelist in the inventory for oracle home /u01/app/grid/product/12.2.0/grid_2.

Update nodelist in the inventory is completed for oracle home /u01/app/grid/product/12.2.0/grid_2.

Starting CRS service on home /u01/app/grid/product/12.2.0/grid_2

Postpatch operation log file location: /u01/app/grid/crsdata/host01/crsconfig/hapatch_2020-04-24_04-01-41PM.log

CRS service started successfully on home /u01/app/grid/product/12.2.0/grid_2

Confirm that all resources have been started from home /u01/app/grid/product/12.2.0/grid_2.

All resources have been started successfully from home /u01/app/grid/product/12.2.0/grid_2.

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching session reported following warning(s):

_________________________________________________

[Note]: Please verify the database is running from the desired Oracle home, if not then manually execute

$ORACLE_HOME/bin/srvctl modify database command to fix the problem

Out of place patching clone home(s) summary

____________________________________________

Host : host01

Actual Home : /u01/app/grid/12.2.0/grid_1

Version:12.2.0.1.0

Clone Home Path : /u01/app/grid/product/12.2.0/grid_2

OPatchauto session completed at Fri Apr 24 15:31:20 2020

Time taken to complete the session 3 minutes, 32 seconds

Verify if HAS stack (Oracle Restart) is running from new home

[grid@host01]$ ps -ef | grep has

root 46201 1 0 Apr 24 ? 0:00 /bin/sh /etc/init.d/init.ohasd run

root 1961 1 0 Feb 27 ? 0:00 /usr/lib/devchassis/devchassisd

grid 51445 1 0 Apr 24 ? 48:36 /u01/app/grid/product/12.2.0/grid_2/bin/ohasd.bin reboot

grid 4926 4728 0 17:36:40 pts/6 0:00 grep has

[grid@host01]$ ps -ef | grep asm_pmon

grid 8210 4728 0 17:42:19 pts/6 0:00 grep asm_pmon

grid 2204 1 0 Apr 24 ? 2:02 asm_pmon_+ASM

[grid@host01]$ pwdx 2204

2204: /u01/app/grid/product/12.2.0/grid_2/dbs

[grid@host01]$

[grid@host01]$ ps -ef | grep tns

grid 10758 4728 0 17:44:56 pts/6 0:00 grep tns

grid 52721 1 0 Apr 24 ? 68:59 /u01/app/grid/product/12.2.0/grid_2/bin/tnslsnr LISTENER -no_crs_notify -inherit

[grid@host01]$

HAS stack (ohasd.bin), ASM instance (asm_pmon_+ASM) and listener (tnslsnr) are all running from new home (/u01/app/grid/product/12.2.0/grid_2).

Switch SIDB home,

[root@host01]# echo $ORACLE_HOME

/u01/app/oracle/product/12.2.0/dbhome_1

[root@host01]# $ORACLE_HOME/OPatch/opatchauto apply -switch-clone -oh $ORACLE_HOME -logLevel FINEST

OPatchauto session is initiated at Fri Apr 24 16:29:15 2020

Session log file is /u01/app/oracle/product/12.2.0/dbhome_1/cfgtoollogs/opatchauto/opatchauto2020-04-24_04-29-24PM.log

The id for this session is Q48R

Verifying clone home details before switch for oracle home /u01/app/oracle/product/12.2.0/dbhome_2.

Verified clone home details successfully for oracle home /u01/app/oracle/product/12.2.0/dbhome_2 before switch.

Bringing down database service on home /u01/app/oracle/product/12.2.0/dbhome_2

Database service successfully brought down on home /u01/app/oracle/product/12.2.0/dbhome_2

Bringing down database service on home /u01/app/oracle/product/12.2.0/dbhome_2

Following database has been stopped and will be restarted later during the session: db01,db02,db03

Database service successfully brought down on home /u01/app/oracle/product/12.2.0/dbhome_2

Starting database service on home /u01/app/oracle/product/12.2.0/dbhome_2

Database service successfully started on home /u01/app/oracle/product/12.2.0/dbhome_2

Preparing home /u01/app/oracle/product/12.2.0/dbhome_2 after database service restarted

No step execution required.........

Modifying the database(s) to use new location home /u01/app/oracle/product/12.2.0/dbhome_2.

All database(s) modified successfully to run from new location home /u01/app/oracle/product/12.2.0/dbhome_2.

Trying to apply SQL patch on home /u01/app/oracle/product/12.2.0/dbhome_2

SQL patch applied successfully on home /u01/app/oracle/product/12.2.0/dbhome_2

OPatchAuto successful.

--------------------------------Summary--------------------------------

Out of place patching clone home(s) summary

____________________________________________

Host : host01

Actual Home : /u01/app/oracle/product/12.2.0/dbhome_1

Version:12.2.0.1.0

Clone Home Path : /u01/app/oracle/product/12.2.0/dbhome_2

OPatchauto session completed at Fri Apr 24 16:42:59 2020

Time taken to complete the session 7 minutes, 14 seconds

If Opatch 12.2.0.1.21 is used instead of 12.2.0.1.19, switch-clone could fail with error PRCD-1163 PRCR-1071 CRS-0245 and rollback also fail with error OPATCHAUTO-72114 as following,

[root@host01]# $ORACLE_HOME/OPatch/opatchauto apply -switch-clone -oh $ORACLE_HOME -logLevel FINEST

<<Content truncated>>

Modifying the database(s) to use new location home /u01/app/oracle/product/12.2.0/dbhome_2.

Failed to configure database to run from Oracle home home /u01/app/oracle/product/12.2.0/dbhome_2.

Execution of [DatabaseStartupFinalizeAction] patch action failed, check log for more details. Failures:

Patch Target : host01->/u01/app/oracle/product/12.2.0/dbhome_2 Type[sidb]

Details: [

---------------------------Patching Failed---------------------------------

Command execution failed during patching in home: /u01/app/oracle/product/12.2.0/dbhome_2, host: host01.

Command failed: /u01/app/oracle/product/12.2.0/dbhome_2/bin/srvctl modify database -d DB03 -o /u01/app/oracle/product/12.2.0/dbhome_2

Command failure output:

PRCD-1163 : Failed to modify database DB03

PRCR-1071 : Failed to register or update resource ora.db03.db

CRS-0245: User doesn't have enough privilege to perform the operation

After fixing the cause of failure Run opatchauto resume

]

<<Content truncated>>

[root@host01]#

[root@host01]# $ORACLE_HOME/OPatch/opatchauto rollback -switch-clone -oh $ORACLE_HOME

OPatchauto session is initiated at Thu May 28 21:15:59 2020

System initialization log file is /u01/app/oracle/product/12.2.0/dbhome_1/cfgtoollogs/opatchautodb/systemconfig2020-05-28_09-16-22PM.log.

Session log file is /u01/app/oracle/product/12.2.0/dbhome_1/cfgtoollogs/opatchauto/opatchauto2020-05-28_09-16-51PM.log

OPATCHAUTO-72114: Out of place patching rollback session cannot be performed.

OPATCHAUTO-72114: Previous apply/switch session is not completed on current node.

OPATCHAUTO-72114: Please complete the previous apply/switch session across all nodes to perform rollback session.

OPatchAuto failed.

OPatchauto session completed at Thu May 28 21:16:53 2020

Time taken to complete the session 0 minute, 55 seconds

opatchauto failed with error code 42

The error happened because opatchauto shipped with OPatch 12.2.0.1.21 is trying to run srvctl modify database as SIDB home owner oracle to update Oracle home to new home. However, opatchauto shipped with OPatch 12.2.0.1.19 runs the command as SIHA home owner grid. It sounds weird, database (home) owner oracle cannot update information on his own database, but SIHA owner grid can. It does not make sense, but is truth. Of course, both versions of opatchauto will work if SIHA and SIDB are owned by same os user.

As workaround, one of following can be used

* Update databases' home information by running following command as SIHA user grid,

<ORIGINAL_DB_HOME>/bin/srvctl modify database -db <database unique name> -oraclehome <NEW_DB_HOME>

* Use Opatch 12.2.0.1.19 instead of 12.2.0.1.21

Note: Since the failed switch-clone cannot be rolled back (I believe it is Oracle's bug), opatchauto (no matter which version) will not be able to be used for OOP. Not sure how Oracle Support will deal with this issue. I just clean the OOP session information by deleting following file,

$ORACLE_HOME/.opatchauto_storage/<hostname>/oopsessioninfo.ser

8. Post switch-clone SIDB home tasks as database home owner oracle

After switch database home to new home, perform following actions:

* Verify all databases are switched to new home

* Apply Conflict Resolution Patches

* Run datapatch to load modified SQL files into the database

8.1 Verify if all databases are switched to new home,

[oracle@qadesh]$ srvctl config database -verbose

DB01 /u01/app/oracle/product/12.2.0/dbhome_2 12.2.0.1.0

DB02 /u01/app/oracle/product/12.2.0/dbhome_2 12.2.0.1.0

DB03 /u01/app/oracle/product/12.2.0/dbhome_2 12.2.0.1.0

The configuration of all three database are changed to new home.

Check the status of databases' instances,

[oracle@host01]$ ps -ef | grep ora_pmon | grep -v grep

oracle 26280 1 0 16:33:42 ? 0:00 ora_pmon_DB02

oracle 32375 1 0 16:33:42 ? 0:00 ora_pmon_DB03

oracle 60303 1 0 16:36:23 ? 0:00 ora_pmon_DB01

[oracle@host01]

[oracle@host01]$ pwdx 26280

26280: /u01/app/oracle/product/12.2.0/dbhome_1/dbs

[oracle@host01]$ pwdx 32375

32375: /u01/app/oracle/product/12.2.0/dbhome_1/dbs

[oracle@host01]$ pwdx 60303

60303: /u01/app/oracle/product/12.2.0/dbhome_2/dbs

Note: If you get 'Permission denied' error while running pwdx as user oracle, try command 'sudo pwdx' if sudo is configured with pwdx, or run pwdx as user root. Command pwdx is only for Solaris, equivalent command on AIX & Linux is ls. If DB01 is running on AIX/Linux, find its home as following,

ls -l /proc/60303/cwd

Output of pwdx shows that DB01 is running from new home and other databases (DB02 & DB03) are still running from old home. Anyway, it can be fixed by restarting the databases because the configuration of these databases have been changed to new home.

Restart databases (DB02 & DB03) running from old home in new home,

[oracle@host01]$ ORACLE_HOME=/u01/app/oracle/product/12.2.0/dbhome_1

[oracle@host01]$ $ORACLE_HOME/bin/srvctl stop database -db DB02

[oracle@host01]$ $ORACLE_HOME/bin/srvctl stop database -db DB03

[oracle@host01]$ ORACLE_HOME=/u01/app/oracle/product/12.2.0/dbhome_2

[oracle@host01]$ $ORACLE_HOME/bin/srvctl start database -db DB02

[oracle@host01]$ $ORACLE_HOME/bin/srvctl start database -db DB03

[oracle@host01]$ ps -ef | grep ora_pmon | grep -v grep

oracle 60303 1 0 16:36:23 ? 0:00 ora_pmon_DB01

oracle 65374 1 0 16:40:23 ? 0:00 ora_pmon_DB02

oracle 65363 1 0 16:46:53 ? 0:00 ora_pmon_DB03

[oracle@host01]

[oracle@host01]$ pwdx 65374

65374: /u01/app/oracle/product/12.2.0/dbhome_2/dbs

[oracle@host01]$ pwdx 65363

65363: /u01/app/oracle/product/12.2.0/dbhome_2/dbs

Now, all databases are running from new home.

8.2 Apply Conflict Resolution Patches

One-off patch 25087834 was rolled back by prepare-clone from new home because conflicting with patch 30886680. As the conflict resolution patch, a new version of patch 25087834 has been downloaded and extracted (unzip) under directory /u01/stage/one-off before. Now, it is time to apply this conflict resolution patch.

Apply new version of patch 25087834

[oracle@host01]$ ORACLE_HOME=/u01/app/oracle/product/12.2.0/dbhome_2

[oracle@host01]$ $ORACLE_HOME/bin/srvctl stop home -oraclehome $ORACLE_HOME -statefile /u01/stage/stophome12.2.txt

[oracle@host01]$ cd /u01/stage/one-off/25087834

[oracle@host01]$ $ORACLE_HOME/OPatch/opatch apply

Oracle Interim Patch Installer version 12.2.0.1.19

Oracle Home : /u01/app/oracle/product/12.2.0/dbhome_2

Central Inventory : /u01/app/oraInventory

from : /u01/app/oracle/product/12.2.0/dbhome_2/oraInst.loc

OPatch version : 12.2.0.1.19

OUI version : 12.2.0.1.4

Log file location : /u01/app/oracle/product/12.2.0/dbhome_2/cfgtoollogs/opatch/opatch2020-04-24_17-11-16PM_1.log

Verifying environment and performing prerequisite checks...

OPatch continues with these patches: 25087834

Do you want to proceed? [y|n]

User Responded with: Y

All checks passed.

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.

(Oracle Home = '/u01/app/oracle/product/12.2.0/dbhome_2')

Is the local system ready for patching? [y|n]

User Responded with: Y

Backing up files...

Applying interim patch '25087834' to OH '/u01/app/oracle/product/12.2.0/dbhome_2'

Patching component oracle.rdbms, 12.2.0.1.0...

Patch 25087834 successfully applied.

Log file location: /u01/app/oracle/product/12.2.0/dbhome_2/cfgtoollogs/opatch/opatch2020-04-24_17-11-16PM_1.log

OPatch succeeded.

[oracle@host01]$

[oracle@host01]$ $ORACLE_HOME/OPatch/opatch lspatches

25087834;

30886680;Database Apr 2020 Release Update : 12.2.0.1.200414 (30886680)

30882603;OCW APR 2020 RELEASE UPDATE 12.2.0.1.200414 (30882603)

30805580;OJVM RELEASE UPDATE: 12.2.0.1.200414 (30805580)

OPatch succeeded.

[oracle@host01]$

[oracle@host01]$ $ORACLE_HOME/bin/srvctl start home -oraclehome $ORACLE_HOME -statefile /u01/stage/stophome12.2.txt

8.3 Load modified SQL files into the batabase

The output of switch-clone shows that it was trying to apply SQL patch on new home, which probably means all databases running from new home should be applied sql patches (by datapatch). Is it true?

Check the sql patch status of databases,

SYS@DB01> select c.name,patch_id,version,action,status,description,action_time

2 from cdb_registry_sqlpatch r, v$containers c

3 where r.con_id=c.con_id

4 order by r.con_id,action_time;

DB/PDB

NAME PATCH_ID VERSION ACTION STATUS DESCRIPTION ACTION_TIME

---------- ---------- ---------- ---------- ---------- -------------------------------------------------- ------------------------------

CDB$ROOT 30133625 12.2.0.1 APPLY SUCCESS OJVM RELEASE UPDATE: 12.2.0.1.191015 (30133625) 22-NOV-19 01.47.20.549204 PM

CDB$ROOT 30138470 12.2.0.1 APPLY SUCCESS DATABASE OCT 2019 RELEASE UPDATE 12.2.0.1.191015 22-NOV-19 01.47.21.656571 PM

CDB$ROOT 30133625 12.2.0.1 ROLLBACK SUCCESS OJVM RELEASE UPDATE: 12.2.0.1.191015 (30133625) 24-APR-20 06.06.29.060129 PM

CDB$ROOT 30805580 12.2.0.1 APPLY SUCCESS OJVM RELEASE UPDATE 12.2.0.1.200414 24-APR-20 06.06.29.067634 PM

CDB$ROOT 30886680 12.2.0.1 APPLY SUCCESS DATABASE APR 2020 RELEASE UPDATE 12.2.0.1.200414 24-APR-20 06.06.31.006964 PM

PDB01 30133625 12.2.0.1 APPLY SUCCESS OJVM RELEASE UPDATE: 12.2.0.1.191015 (30133625) 22-NOV-19 01.47.22.906242 PM

PDB01 30138470 12.2.0.1 APPLY SUCCESS DATABASE OCT 2019 RELEASE UPDATE 12.2.0.1.191015 22-NOV-19 01.47.24.042855 PM

PDB01 30133625 12.2.0.1 ROLLBACK SUCCESS OJVM RELEASE UPDATE: 12.2.0.1.191015 (30133625) 24-APR-20 06.06.33.141294 PM

PDB01 30805580 12.2.0.1 APPLY SUCCESS OJVM RELEASE UPDATE 12.2.0.1.200414 24-APR-20 06.06.33.147667 PM

PDB01 30886680 12.2.0.1 APPLY SUCCESS DATABASE APR 2020 RELEASE UPDATE 12.2.0.1.200414 24-APR-20 06.06.35.161144 PM

SYS@DB02> select c.name,patch_id,version,action,status,description,action_time

2 from cdb_registry_sqlpatch r, v$containers c

3 where r.con_id=c.con_id

4 order by r.con_id,action_time;

DB/PDB

NAME PATCH_ID VERSION ACTION STATUS DESCRIPTION ACTION_TIME

------ ---------- ---------- ---------- ---------- -------------------------------------------------- ------------------------------

DB02 30133625 12.2.0.1 APPLY SUCCESS OJVM RELEASE UPDATE: 12.2.0.1.191015 (30133625) 22-NOV-19 03.46.08.441815 PM

DB02 30138470 12.2.0.1 APPLY SUCCESS DATABASE OCT 2019 RELEASE UPDATE 12.2.0.1.191015 22-NOV-19 03.46.09.745137 PM

SYS@DB03> select c.name,patch_id,version,action,status,description,action_time

2 from cdb_registry_sqlpatch r, v$containers c

3 where r.con_id=c.con_id

4 order by r.con_id,action_time;

DB/PDB

NAME PATCH_ID VERSION ACTION STATUS DESCRIPTION ACTION_TIME

---------- ---------- ---------- ---------- ---------- -------------------------------------------------- ------------------------------

CDB$ROOT 30133625 12.2.0.1 APPLY SUCCESS OJVM RELEASE UPDATE: 12.2.0.1.191015 (30133625) 22-NOV-19 03.11.21.406332 PM

CDB$ROOT 30138470 12.2.0.1 APPLY SUCCESS DATABASE OCT 2019 RELEASE UPDATE 12.2.0.1.191015 22-NOV-19 03.11.22.706324 PM

PDB03 30133625 12.2.0.1 APPLY SUCCESS OJVM RELEASE UPDATE: 12.2.0.1.191015 (30133625) 22-NOV-19 03.11.24.022250 PM

PDB03 30138470 12.2.0.1 APPLY SUCCESS DATABASE OCT 2019 RELEASE UPDATE 12.2.0.1.191015 22-NOV-19 03.11.25.221969 PM

SQL files of patches 30886680 and 30805580 have been loaded into DB01 (both root container and pdb), but not into DB02 & DB03.

Run datapatch to load sql files,

[oracle@host01]$ export ORACLE_SID=DB02

[oracle@host01]$ export ORACLE_HOME=/u01/app/oracle/product/12.2.0/dbhome_2

[oracle@host01]$ $ORACLE_HOME/OPatch/datapatch -verbose

SQL Patching tool version 12.2.0.1.0 Production on Fri Apr 24 17:36:42 2020

Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_44308_2020_04_24_17_36_42/sqlpatch_invocation.log

Connecting to database...OK

Bootstrapping registry and package to current versions...done

Determining current state...done

Current state of SQL patches:

Patch 30133625 (OJVM RELEASE UPDATE: 12.2.0.1.191015 (30133625)):

Installed in the SQL registry only

Patch 30805580 (OJVM RELEASE UPDATE 12.2.0.1.200414):

Installed in the binary registry only

Bundle series DBRU:

ID 200414 in the binary registry and ID 191015 in the SQL registry

Adding patches to installation queue and performing prereq checks...

Installation queue:

The following patches will be rolled back:

30133625 (OJVM RELEASE UPDATE: 12.2.0.1.191015 (30133625))

The following patches will be applied:

30805580 (OJVM RELEASE UPDATE 12.2.0.1.200414)

30886680 (DATABASE APR 2020 RELEASE UPDATE 12.2.0.1.200414)

Installing patches...

Patch installation complete. Total patches installed: 3

Validating logfiles...

Patch 30133625 rollback: SUCCESS

logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30133625/23095898/30133625_apply_DB02_2020Apr24_17_37_31.log (no errors)

Patch 30805580 apply: SUCCESS

logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30805580/23439360/30805580_apply_DB02_2020Apr24_17_37_31.log (no errors)

Patch 30886680 apply: SUCCESS

logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30886680/23483391/30886680_apply_DB02_2020Apr24_17_37_31.log (no errors)

SQL Patching tool complete on Fri Apr 24 17:38:51 2020

[oracle@host01]$ export ORACLE_SID=DB03

[oracle@host01]$ $ORACLE_HOME/OPatch/datapatch -verbose

SQL Patching tool version 12.2.0.1.0 Production on Fri Apr 24 17:38:57 2020

Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_46606_2020_04_24_17_38_57/sqlpatch_invocation.log

Connecting to database...OK

Note: Datapatch will only apply or rollback SQL fixes for PDBs

that are in an open state, no patches will be applied to closed PDBs.

Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation

(Doc ID 1585822.1)

Bootstrapping registry and package to current versions...done

Determining current state...done

Current state of SQL patches:

Patch 30133625 (OJVM RELEASE UPDATE: 12.2.0.1.191015 (30133625)):

Installed in CDB$ROOT PDB03 PDB$SEED only

Patch 30805580 (OJVM RELEASE UPDATE 12.2.0.1.200414):

Installed in the binary registry only

Bundle series DBRU:

ID 200414 in the binary registry and ID 191015 in PDB CDB$ROOT, ID 191015 in PDB PDB$SEED, ID 191015 in PDB PDB03

Adding patches to installation queue and performing prereq checks...

Installation queue:

For the following PDBs: CDB$ROOT PDB$SEED PDB03

The following patches will be rolled back:

30133625 (OJVM RELEASE UPDATE: 12.2.0.1.191015 (30133625))

The following patches will be applied:

30805580 (OJVM RELEASE UPDATE 12.2.0.1.200414)

30886680 (DATABASE APR 2020 RELEASE UPDATE 12.2.0.1.200414)

Installing patches...

Patch installation complete. Total patches installed: 9

Validating logfiles...

Patch 30133625 rollback (pdb CDB$ROOT): SUCCESS

logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30133625/23095898/30133625_apply_DB03_CDBROOT_2020Apr24_17_40_03.log (no errors)

Patch 30805580 apply (pdb CDB$ROOT): SUCCESS

logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30805580/23439360/30805580_apply_DB03_CDBROOT_2020Apr24_17_41_07.log (no errors)

Patch 30886680 apply (pdb CDB$ROOT): SUCCESS

logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30886680/23483391/30886680_apply_DB03_CDBROOT_2020Apr24_17_41_07.log (no errors)

Patch 30133625 rollback (pdb PDB$SEED): SUCCESS

logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30133625/23095898/30133625_apply_DB03_PDBSEED_2020Apr24_17_42_09.log (no errors)

Patch 30805580 apply (pdb PDB$SEED): SUCCESS

logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30805580/23439360/30805580_apply_DB03_PDBSEED_2020Apr24_17_42_25.log (no errors)

Patch 30886680 apply (pdb PDB$SEED): SUCCESS

logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30886680/23483391/30886680_apply_DB03_PDBSEED_2020Apr24_17_42_25.log (no errors)

Patch 30133625 rollback (pdb PDB03): SUCCESS

logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30133625/23095898/30133625_apply_DB03_PDB03_2020Apr24_17_42_09.log (no errors)

Patch 30805580 apply (pdb PDB03): SUCCESS

logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30805580/23439360/30805580_apply_DB03_PDB03_2020Apr24_17_42_25.log (no errors)

Patch 30886680 apply (pdb PDB03): SUCCESS

logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30886680/23483391/30886680_apply_DB03_PDB03_2020Apr24_17_42_25.log (no errors)

SQL Patching tool complete on Fri Apr 24 17:43:20 2020

Patch 25087834 is binary-only patch, it does not include SQL files. Therefore, no patch information is shown in the datapatch log.

So far, SIHA & SIDB have been successfully patched by opatchauto OOP. As shown before, opatchauto completed the OOP patching in two steps (prepare-clone and switch-clone), it is called two-step method. Actually, opatchauto can do OOP in one step with option outofplace, this is called single-session method.

9. OOP patching with single-session method

Single-session method create clone home and switch the original home to the clone home by one opatchauto command,

opatchauto apply -phBaseDir <patch-base-directory> -outofplace -oh <ORACLE_HOME>

I prefer twp-step method, because the downtime is only needed at second step (swith-clone) which is usually done in minutes and minimize downtime. However, single-session method finishes everything in one step which takes much longer time and the downtime window is hardly determined and scheduled.

Following is an example of single-session method,

[root@host01]# ${ORACLE_HOME}/OPatch/opatchauto apply -phBaseDir /u01/stage/grid -outofplace -oh ${ORACLE_HOME} -logLevel FINEST

OPatchauto session is initiated at Fri Apr 24 15:27:48 2020

System initialization log file is /u01/app/grid/12.2.0/grid_1/cfgtoollogs/opatchautodb/systemconfig2020-04-24_03-27-56PM.log.

Session log file is /u01/app/grid/12.2.0/grid_1/cfgtoollogs/opatchauto/opatchauto2020-04-24_03-28-07PM.log

The id for this session is DSN1

Please press ENTER button to accept system generated default clone path.

Please enter clone path [/u01/app/grid/12.2.0/grid_2 ] :

Executing OPatch prereq operations to verify patch applicability on home /u01/app/grid/12.2.0/grid_1

Patch applicability verified successfully on home /u01/app/grid/12.2.0/grid_1

Copying the files from the existing oracle home /u01/app/grid/12.2.0/grid_1 to a new location. Please wait...

Clone of oracle home /u01/app/grid/12.2.0/grid is /u01/app/grid/product/12.2.0/grid_2 on host host01

Copying the files from the existing oracle home /u01/app/grid/12.2.0/grid_1 to a new location is successful.

Unlocking CRS clone home for home /u01/app/grid/12.2.0/grid_1.

Prepatch operation log file location: /u01/app/grid/crsdata/host01/crsconfig/hapatch_2020-04-24_03-38-00PM.log

Unlocked CRS clone home successfully for home /u01/app/grid/12.2.0/grid_1.

Creating clone for oracle home /u01/app/grid/12.2.0/grid_1.

Clone operation successful for oracle home /u01/app/grid/12.2.0/grid_1.

Performing post clone operation for oracle home /u01/app/grid/12.2.0/grid_1.