Wednesday, January 26, 2011

CHECKLIST BEFORE SWITCHOVER OPERATION

CHECKLIST BEFORE SWITCHOVER OPERATION
=======================================
Please note

log_archive_dest_state_3 :(archive dest for standby database)
log_archive_dest_state_1 :(archive dest for primary database)

SELECT PROTECTION_MODE FROM V$DATABASE;
SQL> select name,value from v$parameter where name in ('log_archive_dest_1','log_archive_dest_state_1','log_archive_dest_3','log_archive_dest_state_3);

Make sure 'lgwr' and not 'arch' is mentioned here, otherwise new primary database will not open after switchover (if the protection_mode is Maximum availability).

-- on standby database--
SQL> select database_role from v$database;

DATABASE_ROLE
----------------
PHYSICAL STANDBY

-- on primary --
SQL> select database_role from v$database;

DATABASE_ROLE
----------------
PRIMARY

SQL>select name,value from v$parameter where name in ('log_archive_dest_1','log_archive_dest_state_1', 'log_archive_dest_3','log_archive_dest_state_3);
log_archive_dest_state_3
------------------
DEFER          --->DO NOT PROCEED.
issue this command : sql> alter system set log_archive_dest_state_3='ENABLE' scope=both;


---- on primary ----
SQL> select distinct status from v$datafile;

STATUS
——-
ONLINE
SYSTEM

If any file(s) are in RECOVER status, DO NOT PROCEED with the switchover


-- on standby database--
SQL> select distinct open_mode from v$database;

OPEN_MODE
———-
MOUNTED            --->PROCEED with switchover.



SQL> select distinct open_mode from v$database;

OPEN_MODE
———-
READ ONLY       --->DO NOT PROCEED with switchover.

To confirm that logs are being shipped and more importantly the LAST LOG archived has been applied…

---- on primary ----
SQL> select max(SEQUENCE#) “LAST_LOG_GENERATED” FROM V$ARCHIVED_LOG WHERE THREAD# =1 and ARCHIVED=’YES’;

LAST_LOG_GENERATED
——————
3988

-- on standby database--
SQL> select max(SEQUENCE#) “LAST_LOG_APPLIED” FROM V$ARCHIVED_LOG WHERE THREAD# = 1 and APPLIED=’YES’;

LAST_LOG_APPLIED
—————-
3988

---> PROCEED

Monday, January 3, 2011

RMAN Tutorial | Recovery Methods

Disclaimer: This document is not a replacement for any ORACLE document or whitepaper. This is merely meant for understanding certain fundamental aspects of RMAN recovery process.

RMAN Recovery Methods
When performing a restore operation, it is best to open two telnet sessions, one for SQL commands, and one for RMAN commands.  For the rest of this document, RMAN commands will be prefaced with the RMAN> prompt, and SQL commands will be prefaced with the SQL> prompt.  A $ indicates that the command is executed from the Unix prompt.

Restoring and Recovering All Datafiles
In this scenario, it is assumed that your control files are still accessible. You have a backup, done for example with backup database plus archivelog;
Your first step is to make sure that the target database is shut down:

$ sqlplus “/ as SYSDBA”
SQL> shutdown abort;
ORACLE instance shut down.
Next, you need to start up your target database in mount mode. RMAN cannot restore datafiles unless the database is at least in mount mode, because RMAN needs to be able to access the control file to determine which backup sets are necessary to recover the database. If the control file isn't available, you have to recover it first. Issue the STARTUP MOUNT command shown in the following example to mount the database:

SQL> startup mount;
Oracle instance started.         
database mounted.

Since backup set files are created in an RMAN-specific format, you must use RMAN to restore the datafiles. To use RMAN, connect to the target database:
$ rman target /
The remainder of this example shows how to restore all of the datafiles of the target database.
Note : When the restore command is executed, RMAN will automatically go to its last good backup set and restore the datafiles to the state they were in when that backup set was created.
When restoring  datafiles, RMAN reads the datafile header and makes the determination as to whether the file needs to be restored. The recovery is done by allocating a channel for I/O and then issuing the RMAN restore database command.
With Oracle9i and above, you don't need to allocate a channel explicitly. Instead, you can use the default channel mode:
RMAN> restore database;
RMAN> recover database;
SQL> alter database open;
For Oracle8i, the ALLOCATE, RESTORE, and RECOVER commands need to be enclosed by the run{} command:
RMAN> run {
  allocate channel d1 type disk;
  restore database;
  recover database;
}
alter database open;

Note :Once the recovery has been completed, execute a complete RMAN backup to establish a new baseline.
Restoring Specific tablespaces
In this scenario, it is assumed that your control files are still accessible. You have a backup, done for example with backup database  plus archivelog;
Take the tablespace that needs recovery offline, restore the tablespace, recover the tablespace, and bring the tablespace online. If you cannot take the tablespace offline, then shutdown abort the database and restore in mount mode.
First try to take the tablespace offline.

$ sqlplus "/ as sysdba"
SQL> alter tablespace #tablespace_name# offline;
If this works, continue with the RMAN recovery:
$ rman target /
RMAN> restore tablespace #tablespace_name#;
RMAN> recover tablespace #tablespace_name#;
SQL> alter tablespace #tablespace_name#  online;
If taking the tablespace offline fails, follow these steps:
$ sqlplus “/ as SYSDBA”
SQL> shutdown abort;
SQL> startup mount;
$ rman target / RMAN> restore tablespace #tablespace_name#;
RMAN> recover tablespace #tablespace_name#;
SQL> alter #database_name# open;
Note :Once the recovery has been completed, execute a complete RMAN backup to establish a new baseline.
Restoring Specific Datafiles
In this scenario, it is assumed that your control files are still accessible. You have a backup, done for example with backup database plus archivelog;
Take the datafile that needs recovery offline, restore the datafile, recover the datafile, and bring the datafile online. If you cannot take the datafile offline, then shutdown abort the database and restore in mount mode.
First try to take the datafile offline:
SQL> alter #database_name# datafile '#tablespace_path#/#tablespace_name#.dbf' offline;
If this works, continue with the RMAN recovery:
$ rman target /
RMAN> restore datafile '#TABLESPACE_PATH#/#tablespace_name#.dbf'
RMAN> recover datafile '#TABLESPACE_PATH#/#tablespace_name#.dbf'
SQL> alter #database_name# datafile '#TABLESPACE_PATH/#tablespace_name#.dbf' online;
If taking the datafile offline fails, follow these steps:
$ sqlplus “/ as SYSDBA”
SQL> shutdown abort;
SQL> startup mount;
$ rman target / 
RMAN> restore datafile '#TABLESPACE_PATH#/#tablespace_name#.dbf';
RMAN> recover datafile '#TABLESPACE_PATH#/#tablespace_name#.dbf';
SQL> alter #database_name# open;
Note :Once the recovery has been completed, execute a complete RMAN backup to establish a new baseline.
Restoring Control Files
In this scenario, it is assumed that your control files are backed up. You have a backup, done for example with backup database plus archivelog;
In an ideal world you'll never use RMAN to restore a control file. But if something catastrophic happens, and you lose all control files, here are the steps for getting them back:
The following examples assume that you are using a catalog. First, here's the simplest Oracle9i syntax for restoring a control file:
$ sqlplus “/ as SYSDBA”
SQL> shutdown abort;
SQL> startup nomount;
$ rman target /
Set the database id (DBID) with the following command.  This is a 10-digit number that RMAN uses to uniquely identify this database in the recovery log.  The number can be obtained from any previous RMAN backup log file.
RMAN> set dbid = xxxxxxxxxx;


RMAN> restore controlfile;
SQL> alter database mount;
SQL> alter database open;
If this fails with ...
ORA-01589: must use RESETLOGS or NORESETLOGS option for database open
... then you must perform a recover database:
SQL> shutdown abort;
SQL> startup mount;
$ rman target / 
RMAN> recover database;
SQL> alter database open resetlogs;
RMAN> reset database;
Note: All offline archivelogs are now useless, perform a full back as soon as possible.
Restoring Online Redologs
In this scenario, it is assumed that your control files are backed up. You have a backup, done for example with backup database plus archivelog;

$ sqlplus "/ as sysdba"
SQL> shutdown abort;
SQL> startup nomount;
$ rman target /
Set the #database_name# id (DBID) with the following command.  This is a 10-digit number that RMAN uses to uniquely identify this #database_name# in the recovery log.  The number can be obtained from any previous RMAN backup log file.
RMAN> set dbid = xxxxxxxxxx;

RMAN> restore controlfile;
SQL> alter database mount;
RMAN> restore database;
RMAM# recover database;


RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 07/11/2010 11:03:23
RMAN-06054: media recovery requesting unknown log: thread 1 seq 1 lowscn 8448414
Since the online logs were lost, complete recovery is not possible. Open the database with resetlogs to continue.
RMAN> alter database open resetlogs;
-------------------------------------IMPORTANT-------------------------------------------
During this type of recovery, if you receive error messages like this:

RMAN> restore database;

Starting restore at 07-OCT-10

using channel ORA_SBT_TAPE_1
using channel ORA_DISK_1
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of restore command at 07/11/2010 14:25:22
RMAN-06026: some targets not found - aborting restore
RMAN-06023: no backup or copy of datafile 10 found to restore
RMAN-06023: no backup or copy of datafile 9 found to restore
RMAN-06023: no backup or copy of datafile 8 found to restore
RMAN-06023: no backup or copy of datafile 7 found to restore
RMAN-06023: no backup or copy of datafile 6 found to restore
RMAN-06023: no backup or copy of datafile 5 found to restore
RMAN-06023: no backup or copy of datafile 4 found to restore
RMAN-06023: no backup or copy of datafile 3 found to restore
RMAN-06023: no backup or copy of datafile 2 found to restore
RMAN-06023: no backup or copy of datafile 1 found to restore

…use the following directions to recover (recreate the controlfile):
  • With the database mounted, execute ‘alter database backup controlfile to trace resetlogs;’
  • Perform a shutdown abort on the database, but remain at the SQL> prompt.
  • In another telnet session, go to the udump directory to retrieve the resulting trace file and copy it to another location to edit it.
  • Edit the file and add the phrase “until cancel” to the recover database command at the end.  The phrase should read “recover database until cancel using backup controlfile”.  Remove the “alter database open” command after the recover command.  Save the file with a .sql extension.
  • Back at the SQL> prompt, execute the modified trace file.  When prompted for an archived log, type in “cancel” and the reply should be “media recovery cancelled”.
  • Issue “alter database open resetlogs”.  The database should open after a few moments.
  • Connect to the RMAN recovery catalog and issue the “reset database” command.
  • Perform a full RMAN backup as soon as possible.

----------------------------------------------------------------------------------------
Time-Based, Change-Based / SCN-based Incomplete Recovery
Incomplete recovery uses a backup to produce a non-current version of the database. In other words, youdo not apply all of the redo records generated after the most recent backup.
You usually perform incomplete recovery of the whole database in the following situations:
*       Media failure destroys some or all of the online redo logs.
*       A user error causes data loss, for example, a user inadvertently drops a table.
*       You cannot perform complete recovery because an archived redo log is missing.
*       You lose your current control file and must use a backup control file to open the database.
To perform incomplete media recovery, you must restore all datafiles from backups created prior to the time to which you want to recover and then open the database with the RESETLOGS option when recovery completes. The RESETLOGS operation creates a new incarnation of the database; in other words, a database with a new stream of log sequence numbers starting with log sequence 1.
NOTE – Start every RMAN incomplete recovery with the following commands:
$ sqlplus "/ as sysdba"
SQL> shutdown abort;
SQL> startup mount;
$ rman target / 


--For time-based recovery, use these commands:
RMAN> restore database until time "to_date('07/10/10 12:0:00','MM/DD/YY HH24:MI:SS')";
RMAN> recover database until time "to_date('07/10/10 12:0:00','MM/DD/YY HH24:MI:SS')";
media recovery complete.
SQL> alter database open resetlogs;
--For SCN-based recovery, user these commands:
RMAN> restore #database_name# until scn 1000;
RMAN> recover #database_name# until scn 1000;
media recovery complete.
SQL> alter database open resetlogs;
--For change-based recovery, user these commands:
RMAN> restore database until sequence 8104;   --Archived log sequence number
RMAN> recover database until sequence 8104;   --Archived log sequence number
media recovery complete.
SQL> alter #database open resetlogs;
Once the recovery has been completed, execute the following steps:

  • Delete prior backups with this command (from the RMAN prompt):

RMAN> delete force backup;

Note :This command removes all prior backups from the RMAN catalog as they can no longer be used once the database has been restarted with the resetlogs option.  After completing this command, create a new RMAN backup to establish a new baseline.

Recovering Archived Logs only

In the event that you want to recover the database archived redo logs until a desired time, you can use the following commands:

$ rman target / 
RMAN> restore ARCHIVELOG FROM TIME 'SYSDATE-1' UNTIL TIME 'SYSDATE';
or
RMAN> restore ARCHIVELOG FROM TIME "to_date('07/11/10 00:00:01','MM/DD/YY HH24:MI:SS')
UNTIL TIME 'SYSDATE';




w'" style="font-size: 10pt; ">RMAN# restore ARCHIVELOG FROM TIME "to_date('07/11/10 00:00:01','MM/DD/YY HH24:MI:SS')
UNTIL TIME 'SYSDATE';
rman target / RMAN# restore ARCHIVELOG FROM TIME 'SYSDATE-1' UNTIL TIME 'SYSDATE';
or
RMAN# restore ARCHIVELOG FROM TIME "to_date('07/11/10 00:00:01','MM/DD/YY HH24:MI:SS')
UNTIL TIME 'SYSDATE';

Sunday, January 2, 2011

Recovering 10g AS Infra-Tier | Corrupt controlfile

During my daily health check i found this:
F:\OracleAs10g\Infra102\opmn\bin>opmnctl status

Processes in Instance: infra102.orapps3.velox-tech.com
-------------------+--------------------+---------+---------
ias-component | process-type | pid | status
-------------------+--------------------+---------+---------
DSA | DSA | N/A | Down
LogLoader | logloaderd | N/A | Down
dcm-daemon | dcm-daemon | N/A | Down
OC4J | OC4J_SECURITY | N/A | Down
HTTP_Server | HTTP_Server | 7344 | Alive
OID | OID | 0 | NONE

Note : As the status shows the OC4J is down and in turn the Infra Tier is down as well.We will be unable to start or stop the processes using OPMNCTL command at this point.
e.g
F:\OracleAs10g\Infra102\opmn\bin>opmnctl startproc ias_component=OC4J
opmnctl: starting opmn managed processes...

opmn id=orapps3:6200
Unknown parameters for this request.

========Error Message from error.log ( IAS_ORACLE_HOME or APACHE_HOME)======
[Mon Oct 26 16:47:01 2010] [warn] [client 150.1.15.4] [ecid: 1256561221:150.1.15.4:2996:3176:331,0] MOD_OC4J_0184: Failed to find an oc4j process for destination: OC4J_SECURITY
[Mon Oct 26 16:47:01 2010] [error] [client 150.1.15.4] [ecid: 1256561221:150.1.15.4:2996:3176:331,0] MOD_OC4J_0145: There is no oc4j process (for destination: OC4J_SECURITY) available to service request.
[Mon Oct 26 16:47:01 2010] [error] [client 150.1.15.4] [ecid: 1256561221:150.1.15.4:2996:3176:331,0] MOD_OC4J_0119: Failed to get an oc4j process for destination: OC4J_SECURITY
[Mon Oct 26 16:47:01 2010] [error] [client 150.1.15.4] [ecid: 1256561221:150.1.15.4:2996:3176:331,0] MOD_OC4J_0013: Failed to call destination: OC4J_SECURITY's service() to service the request.
========Error Message from error.log ( IAS_ORACLE_HOME or APACHE_HOME)======


C:\Documents and Settings\dbadmin>lsnrctl status

LSNRCTL for 32-bit Windows: Version 10.1.0.4.2 - Production on 28-OCT-2009 12:24
:10

Copyright (c) 1991, 2004, Oracle. All rights reserved.

Connecting to (ADDRESS=(PROTOCOL=tcp)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for 32-bit Windows: Version 10.1.0.4.2 - Produ
ction
Start Date 27-OCT-2009 10:37:13
Uptime 1 days 1 hr. 46 min. 58 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File F:\OracleAs10g\Infra102\network\admin\listener.ora
Listener Log File F:\OracleAs10g\Infra102\network\log\listener.log
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=orapps3.velox-tech.com)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=\\.\pipe\EXTPROCipc)))
Services Summary...
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
Service "asdb.velox-tech.com" has 1 instance(s).
Instance "asdb", status BLOCKED, has 1 handler(s) for this service...
The command completed successfully

==============Error Messages from listener.log==================

26-OCT-2010 20:22:23 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))(SERVICE_NAME=asdb.velox-tech.com)) * (ADDRESS=(PROTOCOL=tcp)(HOST=150.1.15.4)(PORT=3372)) * establish * asdb.velox-tech.com * 12528
TNS-12528: TNS:listener: all appropriate instances are blocking new connections
==============Error Messages from listener.log==================


PLEASE NOTE : The Status of Listener says BLOCKED.

Now we try to login to the Metadata Repository Database for the 10g Application Server.

SQL> conn sys as sysdba
Enter password:
Connected.
SQL> select status from v$instance;

STATUS
------------
STARTED

SQL> shut immediate
ORA-01507: database not mounted


ORACLE instance shut down.
SQL> startup
ORACLE instance started.

Total System Global Area 281018368 bytes
Fixed Size 789000 bytes
Variable Size 228849144 bytes
Database Buffers 50331648 bytes
Redo Buffers 1048576 bytes
ORA-00214: controlfile 'F:\ORACLEAS10G\INFRA102\ORADATA\ASDB\CONTROL03.CTL'
version 36164 inconsistent with file
'F:\ORACLEAS10G\INFRA102\ORADATA\ASDB\CONTROL01.CTL' version 36139

NOTE: We find that one of the control files have been corrupt .

The control file with highest sequence number is to be chosen
e.g 'F:\ORACLEAS10G\INFRA102\ORADATA\ASDB\CONTROL03.CTL'




RECOVERY PROCEDURE

1>
Create Pfile from Spfile for editing the controlfile parameter

SQL> create pfile from spfile;

File created.

SQL> select status from v$instance;

STATUS
------------
STARTED


2>
shut down the database

SQL> shutdown immediate
ORA-01507: database not mounted


ORACLE instance shut down.


3>
Remove the entries from pfile of all control files except the control file that we will use to startup the database.

e.g 'F:\ORACLEAS10G\INFRA102\ORADATA\ASDB\CONTROL03.CTL'


4>
startup the database using pfile ( where the change is made )


SQL> startup pfile='F:\OracleAs10g\Infra102\database\INITasdb.ORA'
ORACLE instance started.

Total System Global Area 281018368 bytes
Fixed Size 789000 bytes
Variable Size 228849144 bytes
Database Buffers 50331648 bytes
Redo Buffers 1048576 bytes
Database mounted.
Database opened.
SQL>


F:\OracleAs10g\Infra102\opmn\bin>opmnctl stopall
opmnctl: stopping opmn and all managed processes...

F:\OracleAs10g\Infra102\opmn\bin>opmnctl startall
opmnctl: starting opmn and all managed processes...

F:\OracleAs10g\Infra102\opmn\bin>opmnctl status

Processes in Instance: infra102.orapps3.velox-tech.com
-------------------+--------------------+---------+---------
ias-component | process-type | pid | status
-------------------+--------------------+---------+---------
DSA | DSA | N/A | Down
LogLoader | logloaderd | N/A | Down
dcm-daemon | dcm-daemon | N/A | Down
OC4J | OC4J_SECURITY | 7348 | Alive
HTTP_Server | HTTP_Server | 4020 | Alive
OID | OID | 4232 | Alive


ROOT CAUSE
--------------------
Due to abnormal power shutdown in DR site:

The Controlfile was corrupt.

ORA-16608 | Oracle Dataguard

DGMGRL> show configuration

Configuration
Name: mhd_dr
Enabled: YES
Protection Mode: MaxPerformance
Fast-Start Failover: DISABLED
Databases:
orc1 - Primary database
lsd - Physical standby database
drs - Physical standby database

Current status for "mhd_dr":
Warning: ORA-16608: one or more databases have warnings


Now we have to track which Standby Database is generating errors


DGMGRL> show database verbose 'drs';

Database
Name: drs
Role: PHYSICAL STANDBY
Enabled: YES
Intended State: ONLINE
Instance(s):
drs

Properties:
InitialConnectIdentifier = 'drs'
LogXptMode = 'ARCH'
Dependency = ''
DelayMins = '0'
Binding = 'OPTIONAL'
MaxFailure = '0'
MaxConnections = '1'
ReopenSecs = '300'
NetTimeout = '180'
LogShipping = 'ON'
PreferredApplyInstance = ''
ApplyInstanceTimeout = '0'
ApplyParallel = 'AUTO'
StandbyFileManagement = 'AUTO'
ArchiveLagTarget = '0'
LogArchiveMaxProcesses = '2'
LogArchiveMinSucceedDest = '1'
DbFileNameConvert = '+DATA/orc1, +DATA/drs, +FLASH/orc1, +FLASH/drs, +IDX/orc1, +IDX/drs, +UNDO/orc1, +UNDO/drs'
LogFileNameConvert = '+DATA/orc1, +DATA/drs, +FLASH/orc1, +FLASH/drs, +IDX/orc1, +IDX/drs, +UNDO/orc1, +UNDO/drs'
FastStartFailoverTarget = ''
StatusReport = '(monitor)'
InconsistentProperties = '(monitor)'
InconsistentLogXptProps = '(monitor)'
SendQEntries = '(monitor)'
LogXptStatus = '(monitor)'
RecvQEntries = '(monitor)'
HostName = 'DBDRC'
SidName = 'drs'
LocalListenerAddress = '(ADDRESS=(PROTOCOL=tcp)(HOST=DBDRC)(PORT=
1521))'
StandbyArchiveLocation = 'E:\ARCHIVE'
AlternateLocation = ''
LogArchiveTrace = '0'
LogArchiveFormat = 'ARC%S_%R.%T'
LatestLog = '(monitor)'
TopWaitEvents = '(monitor)'

Current status for "drs":
SUCCESS



DGMGRL> show database verbose 'lsd';

Database
Name: lsd
Role: PHYSICAL STANDBY
Enabled: YES
Intended State: ONLINE
Instance(s):
lsd

Properties:
InitialConnectIdentifier = 'lsd'
LogXptMode = 'ARCH'
Dependency = ''
DelayMins = '0'
Binding = 'OPTIONAL'
MaxFailure = '0'
MaxConnections = '1'
ReopenSecs = '300'
NetTimeout = '180'
LogShipping = 'ON'
PreferredApplyInstance = ''
ApplyInstanceTimeout = '0'
ApplyParallel = 'AUTO'
StandbyFileManagement = 'AUTO'
ArchiveLagTarget = '0'
LogArchiveMaxProcesses = '2'
LogArchiveMinSucceedDest = '1'
DbFileNameConvert = '+DATA/orc1, +DATA/lsd, +FLASH/orc1, +FLASH/lsd, +IDX/orc1, +IDX/lsd, +UNDO/orc1, +UNDO/lsd'
LogFileNameConvert = '+DATA/orc1, +DATA/lsd, +FLASH/orc1, +FLASH/lsd, +IDX/orc1, +IDX/lsd, +UNDO/orc1, +UNDO/lsd'
FastStartFailoverTarget = ''
StatusReport = '(monitor)'
InconsistentProperties = '(monitor)'
InconsistentLogXptProps = '(monitor)'
SendQEntries = '(monitor)'
LogXptStatus = '(monitor)'
RecvQEntries = '(monitor)'
HostName = 'LOCALSTANDBY'
SidName = 'lsd'
LocalListenerAddress = '(ADDRESS=(PROTOCOL=tcp)(HOST=LOCALSTANDBY
)(PORT=1521))'
StandbyArchiveLocation = 'E:\ARCHIVE'
AlternateLocation = ''
LogArchiveTrace = '0'
LogArchiveFormat = 'ARC%S_%R.%T'
LatestLog = '(monitor)'
TopWaitEvents = '(monitor)'

Current status for "lsd":
Warning: ORA-16826: apply service state is inconsistent with the DelayMins property



So now we find that LSD has problems. So we review “drcLSD.log “ file to know the exact problem.


============ error message from drclsd.log=========================

DG 2011-01-01-11:20:30 0 2 0 RSM0: HEALTH CHECK WARNING: ORA-16826: apply service state is inconsistent with the DelayMins property
DG 2011-01-01-11:20:30 0 2 738705582 Operation CTL_GET_STATUS continuing with warning, status = ORA-16826
DG 2011-01-01-11:20:30 0 2 738705582 Operation CTL_GET_STATUS continuing with warning, status = ORA-16826
DG 2011-01-01-11:21:30 0 2 0 RSM: Redo Apply is running without USING CURRENT LOGFILE option while DelayMins=0
DG 2011-01-01-11:21:30 0 2 0 RSM0: HEALTH CHECK WARNING: ORA-16826: apply service state is inconsistent with the DelayMins property
DG 2011-01-01-11:21:30 0 2 738705585 Operation CTL_GET_STATUS continuing with warning, status = ORA-16826
DG 2011-01-01-11:21:30 0 2 738705585 Operation CTL_GET_STATUS continuing with warning, status = ORA-16826
============ error message from drclsd.log=========================


What does this message mean ?

DG 2011-01-01-11:21:30 0 2 0 RSM: Redo Apply is running without USING CURRENT LOGFILE option while DelayMins=0
DG 2011-01-01-11:21:30 0 2 0 RSM0: HEALTH CHECK WARNING: ORA-16826: apply service state is inconsistent with the DelayMins property

That means broker property DelayMins is set to 0, but recover managed standby database didn't start as real apply service. So DelayMins=0 cannot be achieved.


Solution :
Restart the physical standby with this command

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;

Database altered.

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCO
NNECT FROM SESSION;

Database altered.



DGMGRL> show configuration

Configuration
Name: mhd_dr
Enabled: YES
Protection Mode: MaxPerformance
Fast-Start Failover: DISABLED
Databases:
orc1 - Primary database
lsd - Physical standby database
drs - Physical standby database

Current status for "mhd_dr":
SUCCESS







DGMGRL> show database verbose 'lsd';

Database
Name: lsd
Role: PHYSICAL STANDBY
Enabled: YES
Intended State: ONLINE
Instance(s):
lsd

Properties:
InitialConnectIdentifier = 'lsd'
LogXptMode = 'ARCH'
Dependency = ''
DelayMins = '0'
Binding = 'OPTIONAL'
MaxFailure = '0'
MaxConnections = '1'
ReopenSecs = '300'
NetTimeout = '180'
LogShipping = 'ON'
PreferredApplyInstance = ''
ApplyInstanceTimeout = '0'
ApplyParallel = 'AUTO'
StandbyFileManagement = 'AUTO'
ArchiveLagTarget = '0'
LogArchiveMaxProcesses = '2'
LogArchiveMinSucceedDest = '1'
DbFileNameConvert = '+DATA/orc1, +DATA/lsd, +FLASH/orc1, +FLASH/lsd, +IDX/orc1, +IDX/lsd, +UNDO/orc1, +UNDO/lsd'
LogFileNameConvert = '+DATA/orc1, +DATA/lsd, +FLASH/orc1, +FLASH/lsd, +IDX/orc1, +IDX/lsd, +UNDO/orc1, +UNDO/lsd'
FastStartFailoverTarget = ''
StatusReport = '(monitor)'
InconsistentProperties = '(monitor)'
InconsistentLogXptProps = '(monitor)'
SendQEntries = '(monitor)'
LogXptStatus = '(monitor)'
RecvQEntries = '(monitor)'
HostName = 'LOCALSTANDBY'
SidName = 'lsd'
LocalListenerAddress = '(ADDRESS=(PROTOCOL=tcp)(HOST=LOCALSTANDBY
)(PORT=1521))'
StandbyArchiveLocation = 'E:\ARCHIVE'
AlternateLocation = ''
LogArchiveTrace = '0'
LogArchiveFormat = 'ARC%S_%R.%T'
LatestLog = '(monitor)'
TopWaitEvents = '(monitor)'

Current status for "lsd":
SUCCESS