Oracle…sometimes blinding in complexity, and a foreshadowing of what happens when a system becomes so complex that it becomes fragile.
This is what the screen session looks like:
adnodemgrctl.sh: check the logfile /u01/oradev/fs1/inst/apps/DEV2_itsrv33m/logs/appl/admin/log/adnodemgrctl.txt for more information … [UNEXPECTED]fs_clone has failed.
[STATEMENT] Please run adopscanlog utility, using the command
to get the list of the log files along with snippet of the error message corresponding to each log file.
adop exiting with status = 2 (Fail)
Here is what it looks like in the adop.log
[EVENT] Performing CLONE steps.
[PROCEDURE] [START 2016/11/08 08:20:19] CLONE Patch File System from Run File System
[PROCEDURE] Calling: /u01/oradev/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl
[EVENT] Log: /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m [UNEXPECTED]Error occurred running “perl /u01/oradev/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/u01/oradev/fs2/inst/apps/DEV2_itsrv33m/appl/admin/DEV2_itsrv33m.xml -patchcontextfile=/u01/oradev/fs1/inst/apps/DEV2_itsrv33m/appl/admin/DEV2_itsrv33m.xml -promptmsg=hide -console=off -mode=create -sync_mode=copy -sessionid=25 -timestamp=20161108_081645 -outdir=/u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m”
[UNEXPECTED]occurred during CLONE Patch File System from Run File System, running command: “perl /u01/oradev/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/u01/oradev/fs2/inst/apps/DEV2_itsrv33m/appl/admin/DEV2_itsrv33m.xml -patchcontextfile=/u01/oradev/fs1/inst/apps/DEV2_itsrv33m/appl/admin/DEV2_itsrv33m.xml -promptmsg=hide -console=off -mode=create -sync_mode=copy -sessionid=25 -timestamp=20161108_081645 -outdir=/u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m”.
[EVENT] Releasing managed server ports.
[PROCEDURE] [START 2016/11/08 08:30:55] Releasing ports
[STATEMENT] Calling task: “Releasing ports on itsrv33m”; script: “/u01/oradev/fs2/inst/apps/DEV2_itsrv33m/admin/install/txkCloneAcquirePort.pl”; args: ” -option=close -server=itsrv33m.mwd.h2o -ports=5570,7015,7615,7415,7215,7216,7217,7815,6114,6214,10013 -log=/u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/txkCloneAcquirePort.log”
[PROCEDURE] [START 2016/11/08 08:30:56] Releasing ports on itsrv33m
[PROCEDURE] Calling: /u01/oradev/fs2/inst/apps/DEV2_itsrv33m/admin/install/txkCloneAcquirePort.pl
[EVENT] Log: /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/txkCloneAcquirePort.log
[PROCEDURE] [END 2016/11/08 08:30:57] Releasing ports on itsrv33m
[PROCEDURE] [END 2016/11/08 08:30:57] Releasing ports
How cryptic! Looks complicated. This system happens to have AD/TXK.Delta.8, so the logs related to adop are all being centralized under the session # (in this case “25”.)
Creating the directory: /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/TXK_SYNC_create/fsclone_apply
Copying the directory
SOURCE : /u01/oradev/fs2/inst/apps/DEV2_itsrv33m/admin/log/clone
TARGET : /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/TXK_SYNC_create/fsclone_apply
/u01/oradev/fs2/EBSapps/comn/adopclone_itsrv33m/bin/adclone.pl did not go through successfully.
LOG DIRECTORY: /u01/oradev/fs_ne/EBSapps/log/adop/25/20161108_081645/fs_clone/itsrv33m/TXK_SYNC_create/fsclone_apply.
PROGRAM : (/u01/oradev/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl)
TIME : Tue Nov 8 08:30:55 2016
FUNCTION: main::runFSCloneApply [ Level 1 ]
ERRORMSG: /u01/oradev/fs2/EBSapps/comn/adopclone_itsrv33m/bin/adclone.pl did not go through successfully.
(One innocuous, but annoying file, owned by someone other than applmgr, and happened to be 600 (user read/write only):)
Done copying /u01/oradev/fs2/EBSapps/appl/wms to /u01/oradev/fs1/EBSapps/appl/wms/
Copying /u01/oradev/fs2/EBSapps/appl/gmf to /u01/oradev/fs1/EBSapps/appl/gmf/
.end std out.
cp: cannot open /u01/oradev/fs2/EBSapps/appl/mwd/12.0.0/sql/MWDPOLPO.xsl: Permission denied
.end err out.
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/mwd failed
ERROR: Copying failed, so stopping all the parallel threads
Done copying /u01/oradev/fs2/EBSapps/appl/okc to /u01/oradev/fs1/EBSapps/appl/okc/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/okc failed
Done copying /u01/oradev/fs2/EBSapps/comn/java/classes/oracle/apps/pqh to /u01/oradev/fs1/EBSapps/comn/java/classes/oracle/apps/pqh/
WARNING: Copying of /u01/oradev/fs2/EBSapps/comn/java/classes/oracle/apps/pqh failed
Done copying /u01/oradev/fs2/EBSapps/appl/gmf to /u01/oradev/fs1/EBSapps/appl/gmf/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/gmf failed
Done copying /u01/oradev/fs2/EBSapps/comn/java/classes/oracle/apps/ies to /u01/oradev/fs1/EBSapps/comn/java/classes/oracle/apps/ies/
WARNING: Copying of /u01/oradev/fs2/EBSapps/comn/java/classes/oracle/apps/ies failed
Done copying /u01/oradev/fs2/EBSapps/appl/msc to /u01/oradev/fs1/EBSapps/appl/msc/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/msc failed
Done copying /u01/oradev/fs2/EBSapps/appl/ams to /u01/oradev/fs1/EBSapps/appl/ams/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/ams failed
Done copying /u01/oradev/fs2/EBSapps/appl/cs to /u01/oradev/fs1/EBSapps/appl/cs/
WARNING: Copying of /u01/oradev/fs2/EBSapps/appl/cs failed
START: Inside exitClone….
Updating status INCOMPLETE for ApplyApplTop
START: Updating status INCOMPLETE for action ApplyApplTop
END: Updated status INCOMPLETE for action ApplyApplTop
One file. A single file. Could have been a hidden file. Could have been anywhere in the entire AppsTier filesystem. This one file stopped ADOP cold from performing. This happened to be phase=fs_clone, but could have been a patching session, or a post-clone config (adcfgclone.pl) session.
Remove the file, and:
The fs_clone phase completed successfully.
adop exiting with status = 0 (Success)
4 Days (almost 7 for me) of Product Directors and Managers spinning as many plates in the air as humanly possible, all being tossed into the Cloud with a fervor only matched by the lack of compensation being provided for conventional technology stack sales, left me with some specific value-adds for my time spent (not including the Safe Harbor information, we can’t say anything about….sshhhh.)
Note: (I’m EBS-centric and we have very little budget for hardware, so while I was amazed at the new SPARC S7 processors with on-board encryption and J/SQL optimization, alas they didn’t make it into my summary pages this year.)
Oracle E-Business Suite Applications DBA and Technology Stack Release Notes for R12.AD.C.Delta.8 and R12.TXK.C.Delta.8 (Doc ID 2159750.1)
Configuring Oracle E-Business Suite Release 12.x Using Logical Host Names (Doc ID 1968231.1)
e-Mail Notifications at Parent and Child Levels for Request Sets
Save and Re-use Concurrent Program Parameters (CP Templates)
Regex Support in Profile Option Values and FND_MESSAGES
Multiple OA Framework Windows and Tabs in Multiple Browser Windows
EBS Healthcheck Diagnostics
64-bit JDK’s on CM Tier
Delta Sync Mode (syncmode=patch/delta) – changed rows only
Rsync support in phase=prepare
Custom module sync auto-supported (no custom drivers req’d)
Collecting Disconnected Mode OCM data for e-Business Suite R12.2
per MOS Document ID 1447334.1 - Oracle E-Business Suite Release 12.2: Release Notes for Oracle Configuration Manager
This method is used whenever your EBS instance is behind a firewall and no proxy server is available to reach the OCM collector end-point (ccr.oracle.com)
The reason you want to use OCM is not only because it provides a complete catalog of your Oracle software versions, modules and plug-ins, so that they can easily be associated with a Service Request (SR) instead of the analyst asking for every RDA script being run for every component in your application and database.
It also enables the full functionality of that Systems Tab in My Oracle Support (buried just above your Settings menu) that allows you to do light versions of lifecycle management comparisons (such as what the instance looked like 2 weeks ago versus now) without having to own the OEM Lifecycle Management license. It comes compliments of your support agreement.
This also enables the full functionality of the patch and security recommendations section vital towards identifying high priority one-off patches specific to your particular configuration and platform.
Basically, instead of having a cron job periodically automatically upload the configurations to ccr.oracle.com, you will be creating the same staged configuration file, but uploading it to your Customer Support Identifier (CSI) via a Draft Service Request (SR) or a currently Open SR via the File Attachment... feature of the SRs.
In the background of MOS is a managed file transfer handler that looks for the specifically named "ocmconfig.jar" files and when detected, route them over to the Configuration Manager engine for automatic uploading to your CSI account.
While not as convenient as the automatic cron method, for those of use with no direct internet connectivity to ccr.oracle.com, our OEM and MOS functionalities become quite limited if the configurations aren't present under our CSI's.e.g. ORACLE_BASE=/u01/app/oracle
# Oracle DB Home cd $ORACLE_HOME unset ORACLE_CONFIG_HOME export JAVA_HOME=$ORACLE_HOME/jdk cd $ORACLE_HOME/ccr # Clear prior CCR installation, if required. rm -rf $ORACLE_HOME/ccr/* cd $ORACLE_HOME unzip <patch_stage_dir>/p5567658_<ver>_<platform>.zip cd $ORACLE_HOME/ccr/bin # Run setupCCR in Disconnected mode ./setupCCR -s -d # Create ORACLE_OCM user and related grants . $ORACLE_HOME/ccr/admin/scripts/installCCRSQL.sh collectconfig -s <SID> -r SYS # Additional grants for APPS EBS Collector . $ORACLE_HOME/ccr/admin/scripts/installCCRSQL.sh ebs_collectconfig -u <APPSUSERNAME> # Run the collection and generate the ocmconfig.jar cd $ORACLE_HOME/ccr/bin ./emCCR collect # ocmconfig.jar location $ORACLE_HOME/ccr/hosts/$HOSTNAME/state/upload # Oracle 10.1.2 iAS Home . $ORACLE_BASE/EBSapps.env run export ORACLE_HOME=$ORACLE_HOME export ORACLE_CONFIG_HOME=$INST_TOP/ora/10.1.2 export JAVA_HOME=$ORACLE_HOME/jdk # Remove prior OCM config, if needed rm -rf $ORACLE_HOME/ccr/* rm -rf $ORACLE_CONFIG_HOME/ccr/* cd $ORACLE_HOME unzip <patch_stage_dir>/p5567658_<ver>_<platform>.zip # Bugfix specific to HP-UX Itanium - verify file permissions to execute chmod 754 $ORACLE_HOME/ccr/bin/deployPackages chmod 754 $ORACLE_HOME/ccr/bin/emSnapshotEnv cd $ORACLE_HOME/ccr/bin ./setupCCR -s -d # Disconnected mode # Run the collection and generate the ocmconfig.jar cd $ORACLE_HOME/ccr/bin ./emCCR collect # ocmconfig.jar location $INST_TOP/ora/10.1.2/ccr/state/upload #Repeat for patch filesystem; . $ORACLE_BASE/EBSapps.env patch # Oracle FMW WebLogic Home # source the Domain environment . $ORACLE_BASE/EBSapps.env run . $EBS_DOMAIN_HOME/bin/setDomainEnv.sh export ORACLE_HOME=$FMW_HOME/utils export ORACLE_CONFIG_HOME=$INST_TOP/ora/FMW # Remove prior OCM config, if needed rm -rf $ORACLE_HOME/ccr/* rm -rf $ORACLE_CONFIG_HOME/ccr/* cd $ORACLE_HOME unzip <patch_stage_dir>/p5567658_<ver>_<platform>.zip # Bugfix specific to HP-UX Itanium - verify file permissions to execute chmod 754 $ORACLE_HOME/ccr/bin/deployPackages chmod 754 $ORACLE_HOME/ccr/bin/emSnapshotEnv cd $ORACLE_HOME/ccr/bin ./setupCCR -s -d # Disconnected mode # Run the collection and generate the ocmconfig.jar cd $ORACLE_HOME/ccr/bin ./emCCR collect # ocmconfig.jar location $INST_TOP/FMW/ccr/state/upload #Repeat for patch filesystem; . $ORACLE_BASE/EBSapps.env patch # Upload all of your ocmconfig.jar files (can be named uniquely - e.g. mydbocmconfig.jar) per MOS Document ID 763142.1 - How to upload the collection file ocmconfig.jar to My Oracle Support for Oracle Configuration Manager (OCM) running in Disconnected Mode.
# Also can ZIP up all of them at once and upload them to your Draft SR (or current open SR as long as you're happy with the CSI (Customer Support Identifier) that will receive the new configurations.
# Trick is have the string "ocmconfig" and .jar as the extension of each file
Have fun with your new Systems tab in My Oracle Support!
Because my company asked for this, and I couldn’t find a readily available My Oracle Support document that is concise enough.
Based primarly upon:
Oracle E-Business Suite Release 12.2 Release Value Propositions (Doc ID 1580742.1)
With additions from notes by Steve Sutphin on LearningWerks.
Modules Being Covered in this Summary
Asset Lifecycle Management
Customer Relationship Management
Human Capital Management
Not all modules are covered herein (e.g. Supply Chain, Project Billing, Project Management, Value Chain) simply because we don’t have them in-use, so I couldn’t validate the accuracy of those modules.
Applications Technology- User Interfaces, Infrastructure
WebLogic Server (18.104.22.168) replacing iAS (10.1.3)
[10.1.2 stays for Forms and Reports]
Mobile Certification for OA Framework
iPad, iPhone, Android
Report Manager and Web ADI
Certified for MS Office 2010/32-bit
Integrated SOA Gateway
Now on WebLogic for improved integration
Editioned filesystem and DB objects allow side-by-side Production and Test codesets with minimal switchover downtime.
Numerous OAFramework User Interface Cosmetic and Usability Improvements
Assets (Asset Lifecycle Management, not FA)- Workflow, Productivity
Support for Asset Moves
Transactable Assets Allows Users To Move Assets In And Out Of Inventory, Transfer Assets From One Inventory Organization To Another, And Transfer Assets From One External Location To Another External Location
Extended Business Processes for Supplier Ship and Debit
Price Protection Agreements
Visibility to Activities across Operating Units
Competitor Tracking at Product Level
Financials- Productivity, Controls
Multi-Org Access Control Security profiles that now support a list of operating units
Ledger Sets Ledgers sharing the same Chart of Accounts, Calendar and Period Type
Advanced Global Intercompany System (AGIS)
New Bank Account Model Single access point for defining and maintaining bank accounts in AP, AR, Collections, Payroll, CashMgt, & Treasury
Party (Customer) Level Bankruptcy Tracking
Support for Partial Disputed Transaction Amounts
Enh. Mass Additions I/F for Legacy Conversions
Automated Depr. Rollback for Selected Assets
Impairments/Unplanned Depreciation Visibility
Addl. BI Publisher Reports
Simplified Setup and Streamlined Bank Account Reconciliation
Reconciliation Success Rates for First Notice Items (FlexMatch)
Centrally-stored Balance History – Trends and Cash Positions
Flexible Cash Pooling
Maintain and Report on Authorized Signatories
Customer Data Hub
Tax Validation Management with Tax Geography Hierarchy
DQM Overview Dashboard
DQM public Application Programming Interface
Real-time Address Validation for Tax Purposes
Application of Specific Tax Configs to Business Entities
Tax Simulator and Determination Services
Customer Tax Information Import
Allocate Financial Data From One Or More Ledgers To A Different Target Ledger
Data Access Sets and Ledger Sets
Journal Batch Copy
Alternate Account UI
New Web-based Account Analysis & Drilldown
Flaggable As Non-Reversable Journal Sources
Integration to Hyperion DRM and Fusion Acctg Hub
Internet Expenses (iExpense)
Global Per Diem and Mileage Rates
Automatic Matching of Advance Payments
Flexible Application of Cash Advances
Payment Holds and Automatic Notifications
Enhanced Itemization Controls
New Audit Expense Report
Auto-Itemization of Expenses from Visa/AMEX
New Header-level Document Attachments
Support for new iOS/Android Mobile Expenses
New User Interface For Supplier Entry And Maintenance
Centralized Bank Account Definitions (see Summary)
Improved Invoice Processing with Invoice Lines
Contract Financing, Retainage, and Progress Terms
Line-Level Invoice Approval Workflows
Self-Service Entry for Non-PO Invoices
Improved iSupplier Portal
Scheduled AP/AR Netting Batches
Partial Period Revenue Recognition
Event-based Revenue Management
New Credit Card Error Handling
Improved Chargeback Reconciliation
Oracle Bill Presentment Architecture (BPA)
Customer Communication – Auto PDF Printing
New Sweep Invalid Distributions Report Details of the accounting exceptions that are holding the period close for review and action. If the amount involved is not material from the reporting perspective and resolving the accounting exceptions is likely to delay the period close, then the transactions with invalid accounting can be swept to the next open period.
Automated Auto Lockbox Processing (via CP)
API Support for Unidentified Receipt Assignments
Enhanced Multi-Reporting Currency Functionality
Improved Reporting and Analysis with Supporting References
New Automatic Floating Rate Reset for Bonds
Automated Cash Pooling Across Legal Entities
Automatic Bank Account Update for Redirecting Payments to a New Bank Account
Labor Rate Schedules by Criteria
Enhanced Payroll Integration
Total Time Costing
Increased Control and Visibility for Multi-Funded Contingent Workers
Allocate Funding Based On Time And Expense Entry
Batch Processing of Mass Adjustments
Automatic Updates of Encumbrance Dates
Eligible Information Template Information Stored as Attachments
Override Employee Charge Account Preference
Dual Budgetary Controls (Encumbrance Acctg)
Item Master Attachments Support
Post-Approved PO Requisition Attachment Support
PO Import Process Can Use Account GeneratorAuto-generate Charge Accounts during Requisition Import (blank charge accounts)
If you Care a Little More, Things Happen. Bees can be dangerous. Always wear protective clothing when approaching or dealing with bees. Do not approach or handle bees without proper instruction and training.