During an online patching session with ADOP, when the prepare phase checks the data dictionary integrity, you receive the following error:
R12.2 eBusiness Suite EBS – During ADOP phase = prepare data dictionary corruption missing parent:
Verifying data dictionary.
[UNEXPECTED]Data dictionary corrupted:
[UNEXPECTED]Data dictionary corruption – missing parent
1416131 ORA$BASE APPS BPA_GUIDED_PICKING SYNONYM
1416121 ORA$BASE APPS CO_CLIPPERSHIP_INTERFACE SYNONYM
1416132 ORA$BASE APPS BPA_PRINT_GUIDED_PICK_LABELS SYNONYM
1416128 ORA$BASE APPS GEM_ESIG_INQUIRY_CORE SYNONYM
[UNEXPECTED]Data dictionary corruption detected. Provide details to
[UNEXPECTED]Oracle Support and ask for a bug to be opened against the
[UNEXPECTED]Online Patching component of Oracle Application Install.
[STATEMENT] Please run adopscanlog utility, using the command
The corruption is detected with the $AD_TOP/sql/ADZDDBCC.sql scriptis executed just prior to setting up the ADOP session. This is also a mandatory check performed during pre-upgrade readiness checking for R12.2 upgrades.
There is a rudimentary, but effective script supplied in $AD_TOP/patch/115/sql named adzddmpfix.sql (and a number of other adzd___ named scripts that address other data dictionary issues, such as the common TimeZone timestamp issues (Doc ID 2017686.1))
This script is run as SYS and requires all middle-tier services to be shutdown because it’s rebuilding all of the APPS synonyms for objects and recompiling afterwards.
sqlplus “/ as sysdba” @$AD_TOP/patch/115/sql/adzddmpfix.sql
REM $Header: adzddmpfix.sql 120.0.12020000.3 2015/09/14 10:10:32 sstomar noship$
REM dbdrv: none
REM | Copyright (c) 2005, 2015 Oracle and/or its affiliates. |
REM | All rights reserved. |
REM | Version 12.0.0 |
REM | FILENAME
REM | adzddmpfix.sql
REM | DESCRIPTION
REM | Warning: this script is an experimental workaround to the Oracle
REM | Database data Dictionary “missing parent” corruption described in
REM | Bug 20627866. This bug currently has no other known fix, but this
REM | script will repair the problem sufficiently to allow adop to continue
REM | executing.
REM | Warning: You must be connected as SYSDBA with application services
REM | shutdown to run this script
REM | Usage:
REM | sqlplus / as sysdba
REM | @adzddmpfix.sql
“—- Fixing Data Dictionary Corruptions (missing parent) —-”
4 rows deleted.
“—- Compiling invalids —-”
Don’t forget to re-run the $AD_TOP/sql/ADZDDBCC.sql script to identify whether the data dictionary corruption is still present afterwards.
This file which is normally executed under the adautocfg.sh wrapper script in $ADMIN_SCRIPTS_HOME, can also be executed manually to bypass certain limitations found when configuring the PATCH filesystem, in particular.
The format of the command is:
adconfig.sh run=INSTE8 contextfile=$CONTEXT_FILE
Enter the APPS user password: Script execution phase specified: INSTE8
The log file for this session is located at: /u01/oracle/fs1/inst/apps/$ENV_ID/admin/log/02120957/adconfig.log
AutoConfig is configuring the Applications environment...
AutoConfig will consider the custom templates if present.
Using CONFIG_HOME location : /u01/oracle/fs1/inst/apps/$ENV_ID
Classpath : /u01/oracle/fs1/FMW_Home/Oracle_EBS-app1/shared-libs/ebs-appsborg/WEB-INF/lib/ebsAppsborgManifest.jar:/u01/oracle/fs1/EBSapps/comn/java/classes
Using Context file : /u01/oracle/fs1/inst/apps/$ENV_ID/appl/admin/$ENV_ID.xml
Context Value Management will now update the Context file
Updating Context file...COMPLETED
Execution phase specified at run time: INSTE8
No uploading of Context File and its templates to database.
Configuring templates from all of the product tops...
AutoConfig completed successfully.
Where I can keep track of the special events (note: these are not “parties” as so many people are misled to believe – you should expect to be contacted by dozens of sales and account professionals at each event) at OOW. To attend one of these events:
You’re registered as an OpenWorld Attendee.
You’re either a prospect, customer, or goodwill contact for the host. That means you actually contact the host and ask to attend before OOW.
You visit the host’s booth at OpenWorld in order to pick up whatever is required for entry and get to know what the host actually does.
Do not just show up at the event and attempt to “crash” it – fair warning. You will be sold Oracle products and consulting, and like it, and start telling all your friends about it and be assimilated.
Relatively easy installation, a new learning curve to master for post-installation maintenance and patching.
The principal changes in Release 2 of 13c are the segregation of the Cloud Management Services plugins (and related functionality) to specific licensable plugin components – namely:
Cloud Services Management – used for provisioning, allocation, and administration (think “infrastructure”) of Oracle Public Cloud-based resources (databases, development tools, middleware, etc.)
Oracle Cloud Application – a portal punch-out to access Oracle Cloud-based applications such as hosted e-Business Suite, Cloud Content Management, Cloud SOA Suite, Cloud Identity Management, etc. (think “apps”.)
This proof-of-concept testbed system was based upon the latest available installer set – Oracle Enterprise Manager Cloud Control 13c Release 2 Plug-in Update 1 (22.214.171.124) – we happen to be testing backwards-compatibility with some 10g databases before committing to the upgrade. Direct (out-of-place) upgrades are supported for version of EM 126.96.36.199 and newer.
Fusion Middleware in this release is Weblogic 12.1.3
The host system is built upon Windows Server 2012 R2, single-tier, 16GB RAM, 2 cores, also hosting the OMS repository database in a 188.8.131.52.0 EE installation. Since the DB Templates provided seemed to be specific for a 12.1.x database (and errors out in certain steps during the build), I simply created a generic database from the standard template with the pre-requisite parameter settings (e.g. _allow_insert_with_update_check=TRUE; sessions > 300; shared_pool_size (10% of SGA); etc.) , and pre-created the required tablespaces:
During the installation, the (Oracle Universal Installer) OUI will create the following new users assigned to these tablespaces:
The standard download set includes a single executable file (.exe or .bin) and several additional ZIP archives. These are to be staged in a single directory, and the ZIP files do NOT need to be pre-extracted.
Overall, the installation was straight-forward and as-documented (despite this being a Windows installation – which usually has its own quirks.) The portion that I feel has the steepest learning curve is post-installation maintenance, and particularly patching. The new tool in 13c, the OMSpatcher (Patchset 19999993 – think of it as a Java wrapper for opatch) was really designed for use specficially in a cloud-based environment and hasn’t been completely polished for use on-premise in smaller installations.
For example, it is designed to rely upon an active Weblogic AdminServer to instantiate it’s requests for inventory versioning information and component availability for patching. But if any of those components are actually in-use by the AdminServer itself, it will tend to fail to apply, and instead provide a lengthy step-by-step instruction set of how to accomplish all of the steps manually. Depending on the complexity of your OEM setup (e.g. the number of registered plugins and target types) this could take over an hour to get the patching utility to fail and then provide the manual steps.
The OEM-specific version of OPatch (Patchset 6880880) is quite different than its predecessors. While the OMSPatcher uses the prior “unzip into the existing Oracle Home” installation technique, the OPatch utility itself now is a Java installer that relies on the OUI to allow updating of component versioning and inventory adjustments. Most importantly, you will need to “install” the new version of OPatch before being allowed to continue with your planned patching:
$JAVA_PATH\java -jar $PATCH_STAGE\6880880\opatch_generic.jar -silent oracle_home=$ORACLE_HOME (being updated for OPatch)
This OPatch update needs to be applied (before patching) to every Oracle Home using 13c technology (the OMS home, the Agent Home, other FMW homes at version 12.1.3 or newer, etc.)
See MOS EM 13c: How to Apply a Patch to the Enterprise Manager 13c Cloud Control OMS Oracle Home (Doc ID 2091619.1) for details about making an OMSPatcher property file, and why you want to create one.
Final steps executed:
Build general purpose 184.108.40.206.0 database with new tablespaces.
Install Oracle Enterprise Manager Cloud Control 13c Release 2 Plug-in Update 1 (220.127.116.11)
Patch 19999993: EM OMSPatcher latest version 18.104.22.168.2 (unzip to ORACLE_HOME)
Patch 6880880: EM OPatch latest version 22.214.171.124.0 (both OMS and Agent)
$FMW_HOME\OMSPatcher\omspatcher apply -analyze -property_file
$FMW_HOME\OMSPatcher\omspatcher apply -property_file [OMS_DISABLE_HOST_CHECK=true -- a useful option added to deal with virtual host names]
Patch 25163555: Tracking bug for Back-porting 24588124 oms side fix
Patch 25604219: MERGE REQUEST ON TOP OF 126.96.36.199.0 FOR BUGS 25497622 25497731 25506784
Patch 25387277: APR-2017 PSU OMS 188.8.131.52.170418
Patch 25162444: EM-BEACON Bundle Patch 184.108.40.206.161231 (Agent)
Patch 25580746: EM-AGENT Bundle Patch 220.127.116.11.170331 (Agent)
Other useful references:
Enterprise Manager 13.2 Master Bundle Patch List (Doc ID 2219797.1)
18.104.22.168.170418 Enterprise Manager Base Platform Patch Set Update (PSU) Readme for Oracle Management Server (OMS) (Doc ID 2246778.1)
Registering the Integrated SOA Gateway included in the Oracle E-Business Suite R12.2.x instances is actually reasonably simple, but the documents make it sound much worse than it really is, simply by being several hundred pages of documentation.
The reason to do this is to expose the RESTful services interface for EBS so that other web applications can do useful things, such as check usernames for validity, lookup business process (workflow) statuses, and even query for completion of processes, such as concurrent programs. The built-in library of accessable Services is quite vast and enables lots of interconnectivity with other applications.
This is also useful if you happen to be extending the application using the Mobile Application Framework, which allows you to build custom applications for mobile devices that access all the various EBS functionalities.
NOTE:1311068.1 – Installing Oracle E-Business Suite Integrated SOA Gateway, Release 12.2
NOTE:1317697.1 – Oracle E-Business Suite Integrated SOA Gateway Troubleshooting Guide, Release 12.2
Before commencing, have the following prepared:
Unlock the ASADMIN (FND_USER) account and set the password so that it can login without a change password challenge (no expiry. required.
IP Address of ApplTier Host
Ports for ApplTier Host (both base port and WLS console – e.g. 8000 7001)
Your base port is the one in your usual EBS URL and the WLS console URL can be determined via Oracle Application Manager in the Site Map link for Weblogic Administration.
Services will be interrupted by bounce twice (2X) during installation.
Installation takes about 45 minutes to complete.
==================================== Start of SOA Integration Steps
#Register new Internal SOA Gateway Configuration
itsrv33m [/export/home/dbausr/oradev] ant -f $JAVA_TOP/oracle/apps/fnd/txk/util/txkSoaConfigUtility.xml
[input] Enter Oracle WebLogic Server Administration Console URL (host:port) :
[input] Enter Oracle WebLogic Server Admin User Name : [weblogic]
[input] Enter the password for user weblogic :
[input] Enter the password for user apps :
[input] Enter the ASADMIN user name : [ASADMIN]
[input] Enter the password for user ASADMIN :
[input] Enter SOA Server Hostname : (hostname.domain)
[input] Enter External URL of SOA Suite SOA Managed Server (protocol://managed_server_hostname:managed_server_port) : http://%5BFQEBShostname%5D:%5Bport%5D
FND: Validate IP address|02-NOV-16|Site||D
FND: Validate User Type|02-NOV-16|Site||D
# Bounce certain services (minimizes downtime – can also bounce everything)
admanagedsrvctl.sh stop oafm_server1
admanagedsrvctl.sh start oafm_server1
admanagedsrvctl.sh stop oacore_server1
admanagedsrvctl.sh start oacore_server1
[etc for any other oacore servers]
#Verify registered payload Class ID (ensures payload can be accessed)
SQL Statement which produced this data:
WHERE UPPER (irep_name) LIKE ‘%EBIZHOMEPAGESERVICE%’
AND class_type = ‘SERVICEBEAN’;
# Configure the ISG BPEL Payload for use by SOA Suite
ant -f $JAVA_TOP/oracle/apps/fnd/bpel/util/txk_integration_packager.xml \ -DIntegration=isg -DFMWNodeName=itsrv33m.mwd.h2o -DFMWIpAddress=[MT or Virtual host IP]
[echo] Bpel Payload is archived at $INST_TOP/logs/appl/rgf/TXK/EBS_BPEL_payload.zip
# Verify the “new” SOA Node is registred (MWD is just re-using the built in SOA)
SQL Statement which produced this data:
SELECT node_name, server_address, server_id
WHERE UPPER(node_name) in(‘[hostname]’);
# Warning: the following command will immediately shutdown and restart the WLS Managed Services
ant -f $JAVA_TOP/oracle/apps/fnd/txk/util/txkISGConfigurator.xml ebsSetup
# Change default NAME=APPS connection to NAME=ASADMIN in WLS Admin Console.
# (you want to do this to support APPS password changes in the future – the ASADMIN
# password does not have to be disclosed to Developers once the setup is completed – just
# grant the Integrated SOA Gateway Role and Responsibility to users that need to deploy
# various services.
Data Sources -> OAEADatasource -> Configuration -> Connection Pool
[Lock & Edit]
Modify Properties user=APPS -> user=ASADMIN
Enter ASADMIN password in both password fields
[Save] -> [Activate Changes]
If the ASADMIN password is changed in the target EBS instance, then all of the following steps need to be re-executed in order to register the change with SOA:
# Clean out the prior OAEADatasource connections and definitions
Login to WLS Administration Console.
Services -> Data Sources -> [Lock & Edit] -> Select OAEADatasource -> [Delete]
Security Realms -> myrealm -> Providers -> Select IsgAuthenticator -> [Delete]
$] cd $DOMAIN_HOME/config/jdbc
$] rm OAEADatasource-*-jdbc.xml
Repeat the SOA Integration steps again.
Otherwise, attempting REST deployments will receive an error:
java.lang.SecurityException: User: ASADMIN~~[ORACLE_SID], failed to be authenticated.
OEM agents tend to occupy memory based upon how many targets they have to keep track of in a particular host. At another organization, we tended to spin up VM’s for each instance environment, so at maximum, a particular agent might have a few hundred targets (especially on an e-Business Suite Applications Tier.) In those circumstances, the default Java memory settings are probably fine.
In this environment, we run our hosts to death, and on this particular proof-of-concept host, we have 43 instances running on it, with variants of 10g, 11g, and 12c databases combined.
We are doing a fresh install of OEM 22.214.171.124.0 for our POC before setting up the 13c production OMS, and after deploying the agent to this particular database host, the agent would startup fine, run for about 20 or so minutes and then abruptly die without warning.
Re-starts fine, passes the usual tests fine (before the 20 minutes or so goes by) and then dies again.
./emctl status agent
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
Agent Version : 126.96.36.199.0
OMS Version : 188.8.131.52.0
Protocol Version : 184.108.40.206.0
Agent Home : /u01/app/oracle/agent12c/agent_inst
Agent Log Directory : /u01/app/oracle/agent12c/agent_inst/sysman/log
Agent Binaries : /u01/app/oracle/agent12c/core/220.127.116.11.0
Agent Process ID : 10598
Parent Process ID : 10499
Agent URL : https://itsrv33c.mydomain:3872/emd/main/
Local Agent URL in NAT : https://itsrv33c.mydomain:3872/emd/main/
Repository URL : https://itsrv35g.mydomain:1159/empbs/upload
Started at : 2016-11-09 09:57:05
Started by user : oracle
Operating System : HP-UX version B.11.31 (IA64W)
Last Reload : (none)
Last successful upload : 2016-11-09 10:19:26
Last attempted upload : 2016-11-09 10:19:26
Total Megabytes of XML files uploaded so far : 0.2
Number of XML files pending upload : 0
Size of XML files pending upload(MB) : 0
Available disk space on upload filesystem : 16.92%
Collection Status : Collections enabled
Heartbeat Status : Ok
Last attempted heartbeat to OMS : 2016-11-09 10:19:53
Last successful heartbeat to OMS : 2016-11-09 10:19:53
Next scheduled heartbeat to OMS : 2016-11-09 10:20:53
Agent is Running and Ready
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
EMD pingOMS completed successfully
—– Wed Nov 9 09:39:43 2016::26900::Agent Launched with PID 27336 at time Wed
Nov 9 09:39:43 2016 —–
—– Wed Nov 9 09:39:43 2016::27336::Time elapsed between Launch of Watchdog p
rocess and execing EMAgent is 34 secs —–
2016-11-09 09:39:44,287 [1:main] WARN – Missing filename for log handler ‘wsm’
2016-11-09 09:39:44,302 [1:main] WARN – Missing filename for log handler ‘opss’
2016-11-09 09:39:44,305 [1:main] WARN – Missing filename for log handler ‘opsscf
Agent is going down due to an OutOfMemoryError
—– Wed Nov 9 09:40:06 2016::26900::Checking status of EMAgent : 27336 —–
—– Wed Nov 9 09:40:06 2016::26900::EMAgent exited at Wed Nov 9 09:40:06 201
6 with return value 57. —–
—– Wed Nov 9 09:40:06 2016::26900::EMAgent will be restarted because of an O
ut of Memory Exception. —–
—– Wed Nov 9 09:40:06 2016::26900::writeAbnormalExitTimestampToAgntStmp: exi
tCause=OOM : restartRequired=1 —–
—– Wed Nov 9 09:40:06 2016::26900::RestartingEMAgent. —–
That means, the agent is starting, then stopping, then restarting, then stopping (aka “thrashing”)
Take a look for the running agent daemon at the OS level:
Oh – it’s set up for the default of 169MB of RAM. Check My Oracle Support.
EM 12c: emctl start agent Fails ‘Fatal agent error: State Manager failed at Startup’ ‘restarted because of an Out of Memory Exception’ Reported in emagent.nohup /gcagent.log (Doc ID 1950490.1)
Verify this setting also in the $AGENT_INST/sysman/log/gcagent.log:
—– Wed Nov 9 09:40:06 2016::26900::Auto tuning the agent at time Wed Nov 9
09:40:06 2016 —–
Auto tuning was successful
Oracle…sometimes blinding in complexity, and a foreshadowing of what happens when a system becomes so complex that it becomes fragile.
This is what the screen session looks like:
adnodemgrctl.sh: check the logfile /u01/oradev/fs1/inst/apps/DEV2_itsrv33m/logs/appl/admin/log/adnodemgrctl.txt for more information … [UNEXPECTED]fs_clone has failed.
[STATEMENT] Please run adopscanlog utility, using the command
to get the list of the log files along with snippet of the error message corresponding to each log file.
adop exiting with status = 2 (Fail)
Here is what it looks like in the adop.log
[EVENT] Performing CLONE steps.
[PROCEDURE] [START 2016/11/08 08:20:19] CLONE Patch File System from Run File System
[PROCEDURE] Calling: /u01/oradev/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl
[EVENT] Log: /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m [UNEXPECTED]Error occurred running “perl /u01/oradev/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/u01/oradev/fs2/inst/apps/DEV2_itsrv33m/appl/admin/DEV2_itsrv33m.xml -patchcontextfile=/u01/oradev/fs1/inst/apps/DEV2_itsrv33m/appl/admin/DEV2_itsrv33m.xml -promptmsg=hide -console=off -mode=create -sync_mode=copy -sessionid=25 -timestamp=20161108_081645 -outdir=/u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m”
[UNEXPECTED]occurred during CLONE Patch File System from Run File System, running command: “perl /u01/oradev/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/u01/oradev/fs2/inst/apps/DEV2_itsrv33m/appl/admin/DEV2_itsrv33m.xml -patchcontextfile=/u01/oradev/fs1/inst/apps/DEV2_itsrv33m/appl/admin/DEV2_itsrv33m.xml -promptmsg=hide -console=off -mode=create -sync_mode=copy -sessionid=25 -timestamp=20161108_081645 -outdir=/u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m”.
[EVENT] Releasing managed server ports.
[PROCEDURE] [START 2016/11/08 08:30:55] Releasing ports
[STATEMENT] Calling task: “Releasing ports on itsrv33m”; script: “/u01/oradev/fs2/inst/apps/DEV2_itsrv33m/admin/install/txkCloneAcquirePort.pl”; args: ” -option=close -server=itsrv33m.mwd.h2o -ports=5570,7015,7615,7415,7215,7216,7217,7815,6114,6214,10013 -log=/u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/txkCloneAcquirePort.log”
[PROCEDURE] [START 2016/11/08 08:30:56] Releasing ports on itsrv33m
[PROCEDURE] Calling: /u01/oradev/fs2/inst/apps/DEV2_itsrv33m/admin/install/txkCloneAcquirePort.pl
[EVENT] Log: /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/txkCloneAcquirePort.log
[PROCEDURE] [END 2016/11/08 08:30:57] Releasing ports on itsrv33m
[PROCEDURE] [END 2016/11/08 08:30:57] Releasing ports
How cryptic! Looks complicated. This system happens to have AD/TXK.Delta.8, so the logs related to adop are all being centralized under the session # (in this case “25”.)
Creating the directory: /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/TXK_SYNC_create/fsclone_apply
Copying the directory
SOURCE : /u01/oradev/fs2/inst/apps/DEV2_itsrv33m/admin/log/clone
TARGET : /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/TXK_SYNC_create/fsclone_apply
/u01/oradev/fs2/EBSapps/comn/adopclone_itsrv33m/bin/adclone.pl did not go through successfully.
LOG DIRECTORY: /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/TXK_SYNC_create/fsclone_apply.
PROGRAM : (/u01/oradev/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl)
TIME : Tue Nov 8 08:30:55 2016
FUNCTION: main::runFSCloneApply [ Level 1 ]
ERRORMSG: /u01/oradev/fs2/EBSapps/comn/adopclone_itsrv33m/bin/adclone.pl did not go through successfully.
(One innocuous, but annoying file, owned by someone other than applmgr, and happened to be 600 (user read/write only):)
Done copying /u01/oradev/fs2/EBSapps/appl/wms to /u01/oradev/fs1/EBSapps/appl/wms/
Copying /u01/oradev/fs2/EBSapps/appl/gmf to /u01/oradev/fs1/EBSapps/appl/gmf/
.end std out.
cp: cannot open /u01/oradev/fs2/EBSapps/appl/mwd/12.0.0/sql/MWDPOLPO.xsl: Permission denied
.end err out.
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/mwd failed
ERROR: Copying failed, so stopping all the parallel threads
Done copying /u01/oradev/fs2/EBSapps/appl/okc to /u01/oradev/fs1/EBSapps/appl/okc/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/okc failed
Done copying /u01/oradev/fs2/EBSapps/comn/java/classes/oracle/apps/pqh to /u01/oradev/fs1/EBSapps/comn/java/classes/oracle/apps/pqh/
WARNING: Copying of /u01/oradev/fs2/EBSapps/comn/java/classes/oracle/apps/pqh failed
Done copying /u01/oradev/fs2/EBSapps/appl/gmf to /u01/oradev/fs1/EBSapps/appl/gmf/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/gmf failed
Done copying /u01/oradev/fs2/EBSapps/comn/java/classes/oracle/apps/ies to /u01/oradev/fs1/EBSapps/comn/java/classes/oracle/apps/ies/
WARNING: Copying of /u01/oradev/fs2/EBSapps/comn/java/classes/oracle/apps/ies failed
Done copying /u01/oradev/fs2/EBSapps/appl/msc to /u01/oradev/fs1/EBSapps/appl/msc/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/msc failed
Done copying /u01/oradev/fs2/EBSapps/appl/ams to /u01/oradev/fs1/EBSapps/appl/ams/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/ams failed
Done copying /u01/oradev/fs2/EBSapps/appl/cs to /u01/oradev/fs1/EBSapps/appl/cs/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/cs failed
START: Inside exitClone….
Updating status INCOMPLETE for ApplyApplTop
START: Updating status INCOMPLETE for action ApplyApplTop
END: Updated status INCOMPLETE for action ApplyApplTop
One file. A single file. Could have been a hidden file. Could have been anywhere in the entire AppsTier filesystem. This one file stopped ADOP cold from performing. This happened to be phase=fs_clone, but could have been a patching session, or a post-clone config (adcfgclone.pl) session.
Remove the file, and:
The fs_clone phase completed successfully.
adop exiting with status = 0 (Success)
People ask what do I look for when monitoring an EBS instance for availability? Oracle Enterprise Manager (OEM) with the Applications Management Pack (AMS) gives a good starting point, but here is a condensed list of my general health check items for my production instance. It’s not everything possible, but it’s what I generally would check during a shake-down of a disaster-recovery exercise.
4 Days (almost 7 for me) of Product Directors and Managers spinning as many plates in the air as humanly possible, all being tossed into the Cloud with a fervor only matched by the lack of compensation being provided for conventional technology stack sales, left me with some specific value-adds for my time spent (not including the Safe Harbor information, we can’t say anything about….sshhhh.)
Note: (I’m EBS-centric and we have very little budget for hardware, so while I was amazed at the new SPARC S7 processors with on-board encryption and J/SQL optimization, alas they didn’t make it into my summary pages this year.)
Oracle E-Business Suite Applications DBA and Technology Stack Release Notes for R12.AD.C.Delta.8 and R12.TXK.C.Delta.8 (Doc ID 2159750.1)
Configuring Oracle E-Business Suite Release 12.x Using Logical Host Names (Doc ID 1968231.1)
e-Mail Notifications at Parent and Child Levels for Request Sets
Save and Re-use Concurrent Program Parameters (CP Templates)
Regex Support in Profile Option Values and FND_MESSAGES
Multiple OA Framework Windows and Tabs in Multiple Browser Windows
EBS Healthcheck Diagnostics
64-bit JDK’s on CM Tier
Delta Sync Mode (syncmode=patch/delta) – changed rows only
Rsync support in phase=prepare
Custom module sync auto-supported (no custom drivers req’d)
If you Care a Little More, Things Happen. Bees can be dangerous. Always wear protective clothing when approaching or dealing with bees. Do not approach or handle bees without proper instruction and training.