Relatively easy installation, a new learning curve to master for post-installation maintenance and patching.
The principal changes in Release 2 of 13c are the segregation of the Cloud Management Services plugins (and related functionality) to specific licensable plugin components – namely:
Cloud Services Management – used for provisioning, allocation, and administration (think “infrastructure”) of Oracle Public Cloud-based resources (databases, development tools, middleware, etc.)
Oracle Cloud Application – a portal punch-out to access Oracle Cloud-based applications such as hosted e-Business Suite, Cloud Content Management, Cloud SOA Suite, Cloud Identity Management, etc. (think “apps”.)
This proof-of-concept testbed system was based upon the latest available installer set – Oracle Enterprise Manager Cloud Control 13c Release 2 Plug-in Update 1 (220.127.116.11) – we happen to be testing backwards-compatibility with some 10g databases before committing to the upgrade. Direct (out-of-place) upgrades are supported for version of EM 18.104.22.168 and newer.
Fusion Middleware in this release is Weblogic 12.1.3
The host system is built upon Windows Server 2012 R2, single-tier, 16GB RAM, 2 cores, also hosting the OMS repository database in a 22.214.171.124.0 EE installation. Since the DB Templates provided seemed to be specific for a 12.1.x database (and errors out in certain steps during the build), I simply created a generic database from the standard template with the pre-requisite parameter settings (e.g. _allow_insert_with_update_check=TRUE; sessions > 300; shared_pool_size (10% of SGA); etc.) , and pre-created the required tablespaces:
During the installation, the (Oracle Universal Installer) OUI will create the following new users assigned to these tablespaces:
The standard download set includes a single executable file (.exe or .bin) and several additional ZIP archives. These are to be staged in a single directory, and the ZIP files do NOT need to be pre-extracted.
Overall, the installation was straight-forward and as-documented (despite this being a Windows installation – which usually has its own quirks.) The portion that I feel has the steepest learning curve is post-installation maintenance, and particularly patching. The new tool in 13c, the OMSpatcher (Patchset 19999993 – think of it as a Java wrapper for opatch) was really designed for use specficially in a cloud-based environment and hasn’t been completely polished for use on-premise in smaller installations.
For example, it is designed to rely upon an active Weblogic AdminServer to instantiate it’s requests for inventory versioning information and component availability for patching. But if any of those components are actually in-use by the AdminServer itself, it will tend to fail to apply, and instead provide a lengthy step-by-step instruction set of how to accomplish all of the steps manually. Depending on the complexity of your OEM setup (e.g. the number of registered plugins and target types) this could take over an hour to get the patching utility to fail and then provide the manual steps.
The OEM-specific version of OPatch (Patchset 6880880) is quite different than its predecessors. While the OMSPatcher uses the prior “unzip into the existing Oracle Home” installation technique, the OPatch utility itself now is a Java installer that relies on the OUI to allow updating of component versioning and inventory adjustments. Most importantly, you will need to “install” the new version of OPatch before being allowed to continue with your planned patching:
$JAVA_PATH\java -jar $PATCH_STAGE\6880880\opatch_generic.jar -silent oracle_home=$ORACLE_HOME (being updated for OPatch)
This OPatch update needs to be applied (before patching) to every Oracle Home using 13c technology (the OMS home, the Agent Home, other FMW homes at version 12.1.3 or newer, etc.)
See MOS EM 13c: How to Apply a Patch to the Enterprise Manager 13c Cloud Control OMS Oracle Home (Doc ID 2091619.1) for details about making an OMSPatcher property file, and why you want to create one.
Final steps executed:
Build general purpose 126.96.36.199.0 database with new tablespaces.
Install Oracle Enterprise Manager Cloud Control 13c Release 2 Plug-in Update 1 (188.8.131.52)
Patch 19999993: EM OMSPatcher latest version 184.108.40.206.2 (unzip to ORACLE_HOME)
Patch 6880880: EM OPatch latest version 220.127.116.11.0 (both OMS and Agent)
$FMW_HOME\OMSPatcher\omspatcher apply -analyze -property_file
$FMW_HOME\OMSPatcher\omspatcher apply -property_file [OMS_DISABLE_HOST_CHECK=true -- a useful option added to deal with virtual host names]
Patch 25163555: Tracking bug for Back-porting 24588124 oms side fix
Patch 25604219: MERGE REQUEST ON TOP OF 18.104.22.168.0 FOR BUGS 25497622 25497731 25506784
Patch 25387277: APR-2017 PSU OMS 22.214.171.124.170418
Patch 25162444: EM-BEACON Bundle Patch 126.96.36.199.161231 (Agent)
Patch 25580746: EM-AGENT Bundle Patch 188.8.131.52.170331 (Agent)
Other useful references:
Enterprise Manager 13.2 Master Bundle Patch List (Doc ID 2219797.1)
184.108.40.206.170418 Enterprise Manager Base Platform Patch Set Update (PSU) Readme for Oracle Management Server (OMS) (Doc ID 2246778.1)
Prayer beads – in some cultures called “worry beads” as having something in the hand to hold, whether another person’s hand, or a string of objects, has always brought a psychological comfort to the holder.
In buddhism, you tend to see them all over the place – people holding them in their hands, a bracelet on the wrist, a string of beads hanging from their forearm, a pile of beads on an altar, someone holding up the beads in front of the altar doing something with them in mid-air. You won’t see them worn around the neck as jewelry, nor tossed around like Mardi Gras bangles. They seem to have another purpose. What could that be?
For further purposes of simplicity, the kinds of prayer beads I’ll be referring in this introduction will be the basic bead sets used by the Shingon esoteric sects. The strand has 108 beads in the circle, divided in two semi-circles by a set of tassels at either end of the loop. One tassel has an extra bead (often referred to as the Mother bead) on it. Counting in from that extra beaded tassle, there are 7 beads, followed by a different kind of bead, followed by 14 more beads, and a different bead, then 33 more beads until the other tassel. Each tassel also has 5 beads (not counting the Mother bead) on it, with a single different bead before the fluffy or ball end of the tassel. They can be made of many different materials, though natural ones are the most preferred (glass, wood, stone, marble, etc.)
There are two basic explanations beyond the “comforting thing to hold” idea for the purpose of the beads. One is symbolic or esoteric, representing the many different things affecting humans. In this version, the 108 beads represent human life problems and struggles. Zen buddhism has an interesting specificity in their definition by multiplying the six senses (sight, hearing, touch, taste, smell, and mind) by the six physical sources of sense (eyes, ears, body, tongue, nose, and brain.) This results in 36 combinations, half of which are positive, and half negative impressions. Multiplying these 36 by the past, present and future positions in time in which they may occur, leads us to the 108 delusions of attachment (or things we should be apologetic for during our path to enlightenment.)
The other is a more functional or technical explanation – counting. There are many basic actions requiring repeating 3 times (bows, chants, prostrations, etc.) which are easy enough to count by yourself. But getting to 5, 7, 21, 33 or 100 times of anything, can find you spending more energy or thought trying to remember how many you just did, rather than being immersed in the action of what you’re supposed to be doing. That’s when holding the beads in one hand while incrementing one bead each time you do something, makes it easy to physically keep track of how many actions you just performed. Simple, practical and something that won’t break your concentration or focus.
A final more psychological use is that of representation. The beads represent a reminder to be mindful of the many ways in which we as humans stray from our own paths of peace. We take care to remember that not only do we individually have all of these perceptions that lead us towards polarized emotions, but so does everyone else around us. To remember that each of us has an innate need for peace and tranquility, is to also find ways to cultivate that same awareness in others.
/* That’s it for this session. Thank you for listening. For more information feel free to e-mail me at jlui at jlui dot net, or twitter @jhlui1 With Gassho, James*/
2017-01 Shinnyo Podcast Peace Through St – 2017-01 Shinnyo Podcast Peace Through Strength – 2017-01 Shinnyo Podcast Peace Through Strength Contradiction A Strong Fence Has Two Sides Fear Builds Walls The Middle Way Subscribe to this Podcast (RSS) or iTunes or via Flipboard There is often confusion about what we think is a stereotype of a buddhist monk – shaven head, wearing … Continue reading 2017-01 Shinnyo Podcast Peace Through St → http://ow.ly/fdHr508Xfec
2017-01 Shinnyo Podcast Peace Through Strength – 2017-01 Shinnyo Podcast Peace Through Strength Contradiction A Strong Fence Has Two Sides Fear Builds Walls The Middle Way Subscribe to this Podcast (RSS) or iTunes or via Flipboard There is often confusion about what we think is a stereotype of a buddhist monk – shaven head, wearing robes, poor in material but rich in … Continue reading 2017-01 Shinnyo Podcast Peace Through Strength → http://ow.ly/D86e5087onP
Back in the 1980’s, one of the common catch-phrases in the aerospace/defense industry brought about during the Cold War years was the concept of “Peace Through Strength.” The idea was that by having or possessing technology or weaponry which instilled fear in your opponent that the entire notion of committing an act of aggression would be dissuaded by the immediate and present fear of losing the confrontation (the premise of Mutually Assured Destruction). Then we further escalated the tension by envisioning “First Strike” capability, wherein upon even the threat of a potential nuclear conflict, we’d simply launch first. But what if we disable their ability to launch, then we’ve truly won? What if they disable our ability to disable them first? This rather nauseating discussion continues even today.
But let’s examine the behavior on a much more local scale, what keeps a potential burglar from entering your home? An obvious security system? A strongly locked front door? A loudly barking dog? Think about the situation from the perspective of the thief. Seeking the path of least resistance is common human behavior, even in the most monumental efforts. Each form of deterrence presents a form of resistance – another boulder in the stream. Some thieves may be deterred by strong evidence of defense, and yet others may be actually attracted with the potential of greater rewards hiding behind an iron curtain of protection. Is what’s in Fort Knox perhaps more valuable than something sitting in an open box on the curb?
A different way of seeing the differences might be comparing a skunk and a domestic cat. When you observe skunks, you might notice they don’t travel very quickly, and are almost casual in their pursuit of food and shelter. The high contrast colors of their fur, and certainly their smell bring an aura of caution and warning to every thing that encounters them..While they have sharp claws and teeth, they rely mostly on intimidation by scent and fluffing out their body and tail fur to make themselves appear bigger, hopefully scaring away a potential predator. Cats by comparison come in a couple of noticeable varieties – those that are quite easily skittish and very wary of approaching anyone or anything, and those that seem to be affectionate to everything and anyone. To a predator, the easy target might be the affectionate kitty, whereas it might avoid entirely going after the skunk.
Is our vision of our perfect existence a world full of skunks, or kittens? Doesn’t it depend a bit on whether you’re a predator, a skunk, or someone seeking companionship? In the latter case, you might find it much more difficult to make friends with a skunk. Or not – maybe you’ve lost your sense of smell.
And why would we see “violent” monks? Because we are still human. Yes, even monks. They are still people; people who have fears. Their attachment may be to their faith believing only they can defend properly its teachings or even its existence. Perhaps they are afraid that their example is not sufficient for others to follow. Maybe they believe that by providing a tough exterior it will shield the precious contents from theft or harm.
But re-examine the thief scenario from above – and imagine that you have given up attachment in the sense that you are secure in knowing everything you will ever need to survive and prosper will eventually come back into your life when it is needed. That every challenge you face is an exercise in your ability to adapt to change and apply your creativity and draw upon the strength you have developed by helping others to succeed for themselves. And in this world, we are actually surrounded by both skunks and kittens, and predators, and each plays its own role in everyday life in nature.
The thief who has everything will still always be hungry for more (termed Asuras or hungry spirits). You however, can satiate your desire for more by deciding to view things differently. You can accept that there will always be those that hunger for more. And also accept those who are unbelievably generous. The inner peace will come from wanting nor needing either. Through that peace, you become impenetrable, and thus, strong. Simple concept, but perhaps difficult to achieve. That’s the many-faceted path of being human. Try to be thankful for that gift of having a choice.
/* That’s it for this session. Thank you for listening. For more information feel free to e-mail me at jlui at jlui dot net, or twitter @jhlui1 With Gassho, James*/
2016-12 Shinnyo Podcast Regarding Hope – 2016-12 Shinnyo Podcast Regarding Hope What is Hope? Does Hope Go Away? Creating Hope When Is Hope Lost and Found? Subscribe to this Podcast (RSS) or iTunes or via Flipboard If we define hope as wanting, wishing or imagining something that isn’t in our present existence, we could say that hope is an imaginary concept … Continue reading 2016-12 Shinnyo Podcast Regarding Hope → http://ow.ly/rrMa5082Hk0
If we define hope as wanting, wishing or imagining something that isn’t in our present existence, we could say that hope is an imaginary concept – something that isn’t real. If we think of hope as a state of consciousness, or the state in which we are visualizing this changed existence, then hope becomes a more tangible idea – something that persists.
The four virtues of enlightenment are expressed as permanence, bliss, self and purity, and in this context, the state of permanence is elusive depending on how we define our concept of hope. Each of these states is related to embracing the dynamics of change as an ever-present condition. But to also realize that hope becomes ever-present as long as we can imagine a state of improvement beyond where we are today. You give up your imagination, and you can say you gave up hope.
Nature accepts things as they are – whatever happens, happens. People like to change things, like creating clocks to track time instead of watching the sun and moon. We try to move and push our environment around to better suit what we think is easier, more advantageous, or even more efficient.
When facing extreme changes, that’s when humans often resort to despair instead of hope, losing that sense of a better tomorrow. But really, tomorrow is by its own peculiar definition, something that hasn’t happened yet. We may have a less shiny and perfect vision of that which has yet to come, but basically, it still is beyond the present. And until it happens, it’s still not our reality (barring existence in a parallel universe and warpage of time, of course.)
Our reality is how we perceive it. A fish out of water is kind of stressed, and probably certainly not happy about that. But it has relatively little capacity to change its own condition. We, by comparison, have amazing capability to change our presence, where we are, how we live, and in what manner we decide to co-exist with everything else. Question is, do you realize that potential?
Hope becomes real as we define and change ourselves, whether that means being content with whatever we have already, or transitioning and developing ourselves to see beyond our self-imposed limitations. In the path of transformation, we materialize our vision or goals into reality.
Looking at the situation in a more pragmatic light might be that we can consider ourselves either “stuck in traffic” or “faced with the opportunity to choose alternate paths to our destination.” Either way, the traffic is there. It is up to us to figure out how we perceive it and how we deal with it..
My own hope in this case is that you find that hope isn’t something someone else gives you. Hope is what you give to yourself.
I have seen what a laugh can do. It can transform almost unbearable tears into something bearable, even hopeful. –Bob Hope
Registering the Integrated SOA Gateway included in the Oracle E-Business Suite R12.2.x instances is actually reasonably simple, but the documents make it sound much worse than it really is, simply by being several hundred pages of documentation.
The reason to do this is to expose the RESTful services interface for EBS so that other web applications can do useful things, such as check usernames for validity, lookup business process (workflow) statuses, and even query for completion of processes, such as concurrent programs. The built-in library of accessable Services is quite vast and enables lots of interconnectivity with other applications.
This is also useful if you happen to be extending the application using the Mobile Application Framework, which allows you to build custom applications for mobile devices that access all the various EBS functionalities.
NOTE:1311068.1 – Installing Oracle E-Business Suite Integrated SOA Gateway, Release 12.2
NOTE:1317697.1 – Oracle E-Business Suite Integrated SOA Gateway Troubleshooting Guide, Release 12.2
Before commencing, have the following prepared:
Unlock the ASADMIN (FND_USER) account and set the password so that it can login without a change password challenge (no expiry. required.
IP Address of ApplTier Host
Ports for ApplTier Host (both base port and WLS console – e.g. 8000 7001)
Your base port is the one in your usual EBS URL and the WLS console URL can be determined via Oracle Application Manager in the Site Map link for Weblogic Administration.
Services will be interrupted by bounce twice (2X) during installation.
Installation takes about 45 minutes to complete.
==================================== Start of SOA Integration Steps
#Register new Internal SOA Gateway Configuration
itsrv33m [/export/home/dbausr/oradev] ant -f $JAVA_TOP/oracle/apps/fnd/txk/util/txkSoaConfigUtility.xml
[input] Enter Oracle WebLogic Server Administration Console URL (host:port) :
[input] Enter Oracle WebLogic Server Admin User Name : [weblogic]
[input] Enter the password for user weblogic :
[input] Enter the password for user apps :
[input] Enter the ASADMIN user name : [ASADMIN]
[input] Enter the password for user ASADMIN :
[input] Enter SOA Server Hostname : (hostname.domain)
[input] Enter External URL of SOA Suite SOA Managed Server (protocol://managed_server_hostname:managed_server_port) : http://%5BFQEBShostname%5D:%5Bport%5D
FND: Validate IP address|02-NOV-16|Site||D
FND: Validate User Type|02-NOV-16|Site||D
# Bounce certain services (minimizes downtime – can also bounce everything)
admanagedsrvctl.sh stop oafm_server1
admanagedsrvctl.sh start oafm_server1
admanagedsrvctl.sh stop oacore_server1
admanagedsrvctl.sh start oacore_server1
[etc for any other oacore servers]
#Verify registered payload Class ID (ensures payload can be accessed)
SQL Statement which produced this data:
WHERE UPPER (irep_name) LIKE ‘%EBIZHOMEPAGESERVICE%’
AND class_type = ‘SERVICEBEAN’;
# Configure the ISG BPEL Payload for use by SOA Suite
ant -f $JAVA_TOP/oracle/apps/fnd/bpel/util/txk_integration_packager.xml \ -DIntegration=isg -DFMWNodeName=itsrv33m.mwd.h2o -DFMWIpAddress=[MT or Virtual host IP]
[echo] Bpel Payload is archived at $INST_TOP/logs/appl/rgf/TXK/EBS_BPEL_payload.zip
# Verify the “new” SOA Node is registred (MWD is just re-using the built in SOA)
SQL Statement which produced this data:
SELECT node_name, server_address, server_id
WHERE UPPER(node_name) in(‘[hostname]’);
# Warning: the following command will immediately shutdown and restart the WLS Managed Services
ant -f $JAVA_TOP/oracle/apps/fnd/txk/util/txkISGConfigurator.xml ebsSetup
# Change default NAME=APPS connection to NAME=ASADMIN in WLS Admin Console.
# (you want to do this to support APPS password changes in the future – the ASADMIN
# password does not have to be disclosed to Developers once the setup is completed – just
# grant the Integrated SOA Gateway Role and Responsibility to users that need to deploy
# various services.
Data Sources -> OAEADatasource -> Configuration -> Connection Pool
[Lock & Edit]
Modify Properties user=APPS -> user=ASADMIN
Enter ASADMIN password in both password fields
[Save] -> [Activate Changes]
If the ASADMIN password is changed in the target EBS instance, then all of the following steps need to be re-executed in order to register the change with SOA:
# Clean out the prior OAEADatasource connections and definitions
Login to WLS Administration Console.
Services -> Data Sources -> [Lock & Edit] -> Select OAEADatasource -> [Delete]
Security Realms -> myrealm -> Providers -> Select IsgAuthenticator -> [Delete]
$] cd $DOMAIN_HOME/config/jdbc
$] rm OAEADatasource-*-jdbc.xml
Repeat the SOA Integration steps again.
Otherwise, attempting REST deployments will receive an error:
java.lang.SecurityException: User: ASADMIN~~[ORACLE_SID], failed to be authenticated.
OEM agents tend to occupy memory based upon how many targets they have to keep track of in a particular host. At another organization, we tended to spin up VM’s for each instance environment, so at maximum, a particular agent might have a few hundred targets (especially on an e-Business Suite Applications Tier.) In those circumstances, the default Java memory settings are probably fine.
In this environment, we run our hosts to death, and on this particular proof-of-concept host, we have 43 instances running on it, with variants of 10g, 11g, and 12c databases combined.
We are doing a fresh install of OEM 220.127.116.11.0 for our POC before setting up the 13c production OMS, and after deploying the agent to this particular database host, the agent would startup fine, run for about 20 or so minutes and then abruptly die without warning.
Re-starts fine, passes the usual tests fine (before the 20 minutes or so goes by) and then dies again.
./emctl status agent
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
Agent Version : 18.104.22.168.0
OMS Version : 22.214.171.124.0
Protocol Version : 126.96.36.199.0
Agent Home : /u01/app/oracle/agent12c/agent_inst
Agent Log Directory : /u01/app/oracle/agent12c/agent_inst/sysman/log
Agent Binaries : /u01/app/oracle/agent12c/core/188.8.131.52.0
Agent Process ID : 10598
Parent Process ID : 10499
Agent URL : https://itsrv33c.mydomain:3872/emd/main/
Local Agent URL in NAT : https://itsrv33c.mydomain:3872/emd/main/
Repository URL : https://itsrv35g.mydomain:1159/empbs/upload
Started at : 2016-11-09 09:57:05
Started by user : oracle
Operating System : HP-UX version B.11.31 (IA64W)
Last Reload : (none)
Last successful upload : 2016-11-09 10:19:26
Last attempted upload : 2016-11-09 10:19:26
Total Megabytes of XML files uploaded so far : 0.2
Number of XML files pending upload : 0
Size of XML files pending upload(MB) : 0
Available disk space on upload filesystem : 16.92%
Collection Status : Collections enabled
Heartbeat Status : Ok
Last attempted heartbeat to OMS : 2016-11-09 10:19:53
Last successful heartbeat to OMS : 2016-11-09 10:19:53
Next scheduled heartbeat to OMS : 2016-11-09 10:20:53
Agent is Running and Ready
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
EMD pingOMS completed successfully
—– Wed Nov 9 09:39:43 2016::26900::Agent Launched with PID 27336 at time Wed
Nov 9 09:39:43 2016 —–
—– Wed Nov 9 09:39:43 2016::27336::Time elapsed between Launch of Watchdog p
rocess and execing EMAgent is 34 secs —–
2016-11-09 09:39:44,287 [1:main] WARN – Missing filename for log handler ‘wsm’
2016-11-09 09:39:44,302 [1:main] WARN – Missing filename for log handler ‘opss’
2016-11-09 09:39:44,305 [1:main] WARN – Missing filename for log handler ‘opsscf
Agent is going down due to an OutOfMemoryError
—– Wed Nov 9 09:40:06 2016::26900::Checking status of EMAgent : 27336 —–
—– Wed Nov 9 09:40:06 2016::26900::EMAgent exited at Wed Nov 9 09:40:06 201
6 with return value 57. —–
—– Wed Nov 9 09:40:06 2016::26900::EMAgent will be restarted because of an O
ut of Memory Exception. —–
—– Wed Nov 9 09:40:06 2016::26900::writeAbnormalExitTimestampToAgntStmp: exi
tCause=OOM : restartRequired=1 —–
—– Wed Nov 9 09:40:06 2016::26900::RestartingEMAgent. —–
That means, the agent is starting, then stopping, then restarting, then stopping (aka “thrashing”)
Take a look for the running agent daemon at the OS level:
Oh – it’s set up for the default of 169MB of RAM. Check My Oracle Support.
EM 12c: emctl start agent Fails ‘Fatal agent error: State Manager failed at Startup’ ‘restarted because of an Out of Memory Exception’ Reported in emagent.nohup /gcagent.log (Doc ID 1950490.1)
Verify this setting also in the $AGENT_INST/sysman/log/gcagent.log:
—– Wed Nov 9 09:40:06 2016::26900::Auto tuning the agent at time Wed Nov 9
09:40:06 2016 —–
Auto tuning was successful
Oracle…sometimes blinding in complexity, and a foreshadowing of what happens when a system becomes so complex that it becomes fragile.
This is what the screen session looks like:
adnodemgrctl.sh: check the logfile /u01/oradev/fs1/inst/apps/DEV2_itsrv33m/logs/appl/admin/log/adnodemgrctl.txt for more information … [UNEXPECTED]fs_clone has failed.
[STATEMENT] Please run adopscanlog utility, using the command
to get the list of the log files along with snippet of the error message corresponding to each log file.
adop exiting with status = 2 (Fail)
Here is what it looks like in the adop.log
[EVENT] Performing CLONE steps.
[PROCEDURE] [START 2016/11/08 08:20:19] CLONE Patch File System from Run File System
[PROCEDURE] Calling: /u01/oradev/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl
[EVENT] Log: /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m [UNEXPECTED]Error occurred running “perl /u01/oradev/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/u01/oradev/fs2/inst/apps/DEV2_itsrv33m/appl/admin/DEV2_itsrv33m.xml -patchcontextfile=/u01/oradev/fs1/inst/apps/DEV2_itsrv33m/appl/admin/DEV2_itsrv33m.xml -promptmsg=hide -console=off -mode=create -sync_mode=copy -sessionid=25 -timestamp=20161108_081645 -outdir=/u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m”
[UNEXPECTED]occurred during CLONE Patch File System from Run File System, running command: “perl /u01/oradev/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/u01/oradev/fs2/inst/apps/DEV2_itsrv33m/appl/admin/DEV2_itsrv33m.xml -patchcontextfile=/u01/oradev/fs1/inst/apps/DEV2_itsrv33m/appl/admin/DEV2_itsrv33m.xml -promptmsg=hide -console=off -mode=create -sync_mode=copy -sessionid=25 -timestamp=20161108_081645 -outdir=/u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m”.
[EVENT] Releasing managed server ports.
[PROCEDURE] [START 2016/11/08 08:30:55] Releasing ports
[STATEMENT] Calling task: “Releasing ports on itsrv33m”; script: “/u01/oradev/fs2/inst/apps/DEV2_itsrv33m/admin/install/txkCloneAcquirePort.pl”; args: ” -option=close -server=itsrv33m.mwd.h2o -ports=5570,7015,7615,7415,7215,7216,7217,7815,6114,6214,10013 -log=/u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/txkCloneAcquirePort.log”
[PROCEDURE] [START 2016/11/08 08:30:56] Releasing ports on itsrv33m
[PROCEDURE] Calling: /u01/oradev/fs2/inst/apps/DEV2_itsrv33m/admin/install/txkCloneAcquirePort.pl
[EVENT] Log: /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/txkCloneAcquirePort.log
[PROCEDURE] [END 2016/11/08 08:30:57] Releasing ports on itsrv33m
[PROCEDURE] [END 2016/11/08 08:30:57] Releasing ports
How cryptic! Looks complicated. This system happens to have AD/TXK.Delta.8, so the logs related to adop are all being centralized under the session # (in this case “25”.)
Creating the directory: /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/TXK_SYNC_create/fsclone_apply
Copying the directory
SOURCE : /u01/oradev/fs2/inst/apps/DEV2_itsrv33m/admin/log/clone
TARGET : /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/TXK_SYNC_create/fsclone_apply
/u01/oradev/fs2/EBSapps/comn/adopclone_itsrv33m/bin/adclone.pl did not go through successfully.
LOG DIRECTORY: /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/TXK_SYNC_create/fsclone_apply.
PROGRAM : (/u01/oradev/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl)
TIME : Tue Nov 8 08:30:55 2016
FUNCTION: main::runFSCloneApply [ Level 1 ]
ERRORMSG: /u01/oradev/fs2/EBSapps/comn/adopclone_itsrv33m/bin/adclone.pl did not go through successfully.
(One innocuous, but annoying file, owned by someone other than applmgr, and happened to be 600 (user read/write only):)
Done copying /u01/oradev/fs2/EBSapps/appl/wms to /u01/oradev/fs1/EBSapps/appl/wms/
Copying /u01/oradev/fs2/EBSapps/appl/gmf to /u01/oradev/fs1/EBSapps/appl/gmf/
.end std out.
cp: cannot open /u01/oradev/fs2/EBSapps/appl/mwd/12.0.0/sql/MWDPOLPO.xsl: Permission denied
.end err out.
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/mwd failed
ERROR: Copying failed, so stopping all the parallel threads
Done copying /u01/oradev/fs2/EBSapps/appl/okc to /u01/oradev/fs1/EBSapps/appl/okc/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/okc failed
Done copying /u01/oradev/fs2/EBSapps/comn/java/classes/oracle/apps/pqh to /u01/oradev/fs1/EBSapps/comn/java/classes/oracle/apps/pqh/
WARNING: Copying of /u01/oradev/fs2/EBSapps/comn/java/classes/oracle/apps/pqh failed
Done copying /u01/oradev/fs2/EBSapps/appl/gmf to /u01/oradev/fs1/EBSapps/appl/gmf/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/gmf failed
Done copying /u01/oradev/fs2/EBSapps/comn/java/classes/oracle/apps/ies to /u01/oradev/fs1/EBSapps/comn/java/classes/oracle/apps/ies/
WARNING: Copying of /u01/oradev/fs2/EBSapps/comn/java/classes/oracle/apps/ies failed
Done copying /u01/oradev/fs2/EBSapps/appl/msc to /u01/oradev/fs1/EBSapps/appl/msc/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/msc failed
Done copying /u01/oradev/fs2/EBSapps/appl/ams to /u01/oradev/fs1/EBSapps/appl/ams/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/ams failed
Done copying /u01/oradev/fs2/EBSapps/appl/cs to /u01/oradev/fs1/EBSapps/appl/cs/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/cs failed
START: Inside exitClone….
Updating status INCOMPLETE for ApplyApplTop
START: Updating status INCOMPLETE for action ApplyApplTop
END: Updated status INCOMPLETE for action ApplyApplTop
One file. A single file. Could have been a hidden file. Could have been anywhere in the entire AppsTier filesystem. This one file stopped ADOP cold from performing. This happened to be phase=fs_clone, but could have been a patching session, or a post-clone config (adcfgclone.pl) session.
Remove the file, and:
The fs_clone phase completed successfully.
adop exiting with status = 0 (Success)
If you Care a Little More, Things Happen. Bees can be dangerous. Always wear protective clothing when approaching or dealing with bees. Do not approach or handle bees without proper instruction and training.