Tag Archives: OEM

What’s New Features in OEM Applications Management Pack 13.4(.2) Released 15-JUN-2020

November 24, 2020 jhlui1 Leave a comment

Per Getting Started with Oracle Application Management Pack (AMP) for Oracle E-Business Suite, Release 13.4.1.0.0 (MOS Note 2676355.1)

New Features

Certification with Oracle Enterprise Manager 13.4 Release Update 2

Certification with Oracle Database 19c (for Discovery, Monitoring, Approvals, Patch Manager, and Customization Manager, only)

Support for Oracle Real User Experience Insight 13.3.1.0.2

References

Angelo Rosado’s Oracle Blog on OEM AMS/AMP Announcements:

https://blogs.oracle.com/ebstech/oracle-application-management-pack-release-134100-is-now-available

New EBS AMS Plugin Homepage Screenshot 13c 13.1.1.1.0

Key Features (very similar to 13.1.1.1.0):

System Management

Oracle Application Management Pack provides monitoring alerts and notifications for Oracle Application Framework, Concurrent Processing, Forms services, concurrent managers and requests, Oracle Workflow services, Oracle Database, and middle tier/web servers.

Standardize monitoring settings across your Oracle E-Business Suite environments by utilizing templates to initially specify the monitoring settings and apply those to your monitored targets. You can save, edit, and apply these templates across one or more targets. A monitoring template is specified for a particular target type and can only be applied to targets of the same type.

Use the Concurrent Processing dashboard for a complete picture of concurrent processing on your Oracle E-Business Suite system of both current activities as well as usage statistics. Add concurrent programs and managers as user-defined targets and the system will collect metrics on activity and usage for them.

Configuration Management

Oracle Application Management Pack collects and stores the technology configurations of Oracle E-Business Suite to help centralize monitoring and tracking of changes to Oracle E-Business Suite technology stack configurations including host configuration, database configuration, middleware configuration, patches applied, key profile option changes, versions of technology components, and custom object changes.

Compare configurations between two or more Oracle E-Business Suite systems directly or by comparing configuration snapshots taken at different time intervals.

Configuration templates help to standardize the configuration standards across the company and reduce the configuration drift between various Oracle E-Business Suite instances.

Administrators can track changes by setting up notifications when any unauthorized changes occur to technology stack configurations.

Compliance Management

Compliance framework integration allows you to ensure your Oracle E-Business Suite is compliant with your IT audit requirements, as well as industry and regulatory requirements. Compliance Management dashboards show trends and compliance violations. Proactive, real-time compliance monitoring can significantly reduce the operational risk to business.

Change Management

Change Approval Framework

All changes made with Oracle-delivered patches or customizations in Change Management are processed through a change approval mechanism. A multi-level, hierarchical list of approvers can be set up for each Oracle E-Business Suite target. Auditors can review the change requests as welll as the approvals.

Patch Management

Patch recommendations for Oracle E-Business Suite application products and the technology stack components are made for specific Oracle E-Business target instances to help reduce or eliminate the research work by administrators. These patch recommendations help to ensure all Oracle E-Business Suite systems are current with Oracle-recommended patches. You can deploy database and WLS patches by creating Oracle Enterprise Manager patch plans directly from the patch recommendations user interface.

Using the Patch Management dashboard, you can manage all patching activities of your Oracle E-Business Suite instances and view patch worker logs. Patch Manager simplifies the online patching process and reduces human intervention to a great extent, eliminating the need for continuous monitoring of the patching process.

Patch Manager makes it easier for administrators to check for prerequisite patches and to download the patches in offline mode or directly from My Oracle Support.

Customization Management

Create and deploy custom packages across Oracle E-Business Suite instances similar to Oracle-delivered patches. Customization Manager can help ensure all customizations follow Oracle development standards and/or user-defined customization standards specific to your IT organization. Easily integrate Customization Manager with any third-party source code version control software. This feature also supports the creation of National Language Support (NLS) patches.
Customization Manager can discover customizations within your Oracle E-Business Suite systems and provide detailed reports online or in a spreadsheet format. This helps your organization keep track of the inventory of customizations and use the data for further analysis.

Cloning

Automated Cloning allows you to clone your Oracle E-Business Suite applications for testing, training, or development purposes, leveraging the Oracle Enterprise Manager Cloud Control Provisioning Framework. A step-by-step interview guides an administrator through the cloning process and facilitates scheduling of Oracle E-Business Suite instance. Administrators can modify the standard cloning procedure to include pre- or post-custom steps, seamlessly automating their cloning process from beginning to end.

Instance Administration

Administrators can use the Instance Administration user interface when adding and removing (scale in and scale out) application services, application nodes, and managed servers.

Oracle

Getting Started with OEM 13cR2 (13.2.2.0.0) Enterprise Manager

April 28, 2017 jhlui1 Leave a comment

oem13c_mainpage_ss — OEM13c Release 2 Enterprise Manager Summary Page

Relatively easy installation, a new learning curve to master for post-installation maintenance and patching.
The principal changes in Release 2 of 13c are the segregation of the Cloud Management Services plugins (and related functionality) to specific licensable plugin components – namely:

Cloud Services Management – used for provisioning, allocation, and administration (think “infrastructure”) of Oracle Public Cloud-based resources (databases, development tools, middleware, etc.)
Oracle Cloud Application – a portal punch-out to access Oracle Cloud-based applications such as hosted e-Business Suite, Cloud Content Management, Cloud SOA Suite, Cloud Identity Management, etc. (think “apps”.)
Cloud Framework – a repackaging of the EM OMS framework components, and the only portion of the “cloud” components included in the base OEM licensing.
Installation guide: http://www.oracle.com/technetwork/oem/grid-control/documentation/oem-091904.html

Oracle Cloud Framework — Cloud Management related plugins for OEM13cR2

This proof-of-concept testbed system was based upon the latest available installer set – Oracle Enterprise Manager Cloud Control 13c Release 2 Plug-in Update 1 (13.2.0.0) – we happen to be testing backwards-compatibility with some 10g databases before committing to the upgrade. Direct (out-of-place) upgrades are supported for version of EM 12.1.0.4 and newer.

Download page for OEM 13c R2 — http://www.oracle.com/technetwork/oem/enterprise-manager/downloads

Fusion Middleware in this release is Weblogic 12.1.3
The host system is built upon Windows Server 2012 R2, single-tier, 16GB RAM, 2 cores, also hosting the OMS repository database in a 12.2.0.1.0 EE installation. Since the DB Templates provided seemed to be specific for a 12.1.x database (and errors out in certain steps during the build), I simply created a generic database from the standard template with the pre-requisite parameter settings (e.g. _allow_insert_with_update_check=TRUE; sessions > 300; shared_pool_size (10% of SGA); etc.) , and pre-created the required tablespaces:

MGMT_AD4J_TS
MGMT_ECM_DEPOT_TS
MGMT_TABLESPACE

During the installation, the (Oracle Universal Installer) OUI will create the following new users assigned to these tablespaces:

Open:
SYSMAN
SYSMAN_TYPES
SYSMAN_BIPLATFORM
SYSMAN_STB
SYSMAN_OPSS
SYSMAN_MDS
MGMT_VIEW

Expired & Locked:
CLOUD_ENGINE_USER
CLOUD_SWLIB_USER
EUS_ENGINE_USER
SYSMAN_RO

The standard download set includes a single executable file (.exe or .bin) and several additional ZIP archives. These are to be staged in a single directory, and the ZIP files do NOT need to be pre-extracted.

Overall, the installation was straight-forward and as-documented (despite this being a Windows installation – which usually has its own quirks.) The portion that I feel has the steepest learning curve is post-installation maintenance, and particularly patching. The new tool in 13c, the OMSpatcher (Patchset 19999993 – think of it as a Java wrapper for opatch) was really designed for use specficially in a cloud-based environment and hasn’t been completely polished for use on-premise in smaller installations.

For example, it is designed to rely upon an active Weblogic AdminServer to instantiate it’s requests for inventory versioning information and component availability for patching. But if any of those components are actually in-use by the AdminServer itself, it will tend to fail to apply, and instead provide a lengthy step-by-step instruction set of how to accomplish all of the steps manually. Depending on the complexity of your OEM setup (e.g. the number of registered plugins and target types) this could take over an hour to get the patching utility to fail and then provide the manual steps.

The OEM-specific version of OPatch (Patchset 6880880) is quite different than its predecessors. While the OMSPatcher uses the prior “unzip into the existing Oracle Home” installation technique, the OPatch utility itself now is a Java installer that relies on the OUI to allow updating of component versioning and inventory adjustments. Most importantly, you will need to “install” the new version of OPatch before being allowed to continue with your planned patching:

$JAVA_PATH\java -jar $PATCH_STAGE\6880880\opatch_generic.jar -silent oracle_home=$ORACLE_HOME (being updated for OPatch)

This OPatch update needs to be applied (before patching) to every Oracle Home using 13c technology (the OMS home, the Agent Home, other FMW homes at version 12.1.3 or newer, etc.)

See MOS EM 13c: How to Apply a Patch to the Enterprise Manager 13c Cloud Control OMS Oracle Home (Doc ID 2091619.1) for details about making an OMSPatcher property file, and why you want to create one.

Final steps executed:
Build general purpose 12.2.0.1.0 database with new tablespaces.
Install Oracle Enterprise Manager Cloud Control 13c Release 2 Plug-in Update 1 (13.2.0.0)
Patch 19999993: EM OMSPatcher latest version 13.8.0.0.2 (unzip to ORACLE_HOME)
Patch 6880880: EM OPatch latest version 13.9.0.0.0 (both OMS and Agent)

cd $PATCH_STAGE\<patch#>
$FMW_HOME\OMSPatcher\omspatcher apply -analyze -property_file 
$FMW_HOME\OMSPatcher\omspatcher apply -property_file  [OMS_DISABLE_HOST_CHECK=true  -- a useful option added to deal with virtual host names]

Patch 25163555: Tracking bug for Back-porting 24588124 oms side fix
Patch 25604219: MERGE REQUEST ON TOP OF 13.2.0.0.0 FOR BUGS 25497622 25497731 25506784
Patch 25387277: APR-2017 PSU OMS 13.2.0.0.170418
Patch 25162444: EM-BEACON Bundle Patch 13.2.0.0.161231 (Agent)
Patch 25580746: EM-AGENT Bundle Patch 13.2.0.0.170331 (Agent)

Other useful references:

Enterprise Manager 13.2 Master Bundle Patch List (Doc ID 2219797.1)
13.2.0.0.170418 Enterprise Manager Base Platform Patch Set Update (PSU) Readme for Oracle Management Server (OMS) (Doc ID 2246778.1)

Oracle

New OEM 12c/13c Agent Install Won’t Keep Running – Dies After Awhile

November 9, 2016 jhlui1 Leave a comment

Many agents - Copyright by Warner Bros.,The Wachowski Brothers used without permission as educational content. — Agents are often hard to kill, usually. Images Copyright by Warner Bros., The Wachowski Brothers; used without permission as educational content.

OEM agents tend to occupy memory based upon how many targets they have to keep track of in a particular host. At another organization, we tended to spin up VM’s for each instance environment, so at maximum, a particular agent might have a few hundred targets (especially on an e-Business Suite Applications Tier.) In those circumstances, the default Java memory settings are probably fine.

In this environment, we run our hosts to death, and on this particular proof-of-concept host, we have 43 instances running on it, with variants of 10g, 11g, and 12c databases combined.

We are doing a fresh install of OEM 12.1.0.5.0 for our POC before setting up the 13c production OMS, and after deploying the agent to this particular database host, the agent would startup fine, run for about 20 or so minutes and then abruptly die without warning.

Re-starts fine, passes the usual tests fine (before the 20 minutes or so goes by) and then dies again.

AGENT_INST=/u01/app/oracle/agent12c/agent_inst

cd $AGENT_INST/bin

./emctl status agent

Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
—————————————————————
Agent Version          : 12.1.0.5.0
OMS Version            : 12.1.0.5.0
Protocol Version       : 12.1.0.1.0
Agent Home             : /u01/app/oracle/agent12c/agent_inst
Agent Log Directory    : /u01/app/oracle/agent12c/agent_inst/sysman/log
Agent Binaries         : /u01/app/oracle/agent12c/core/12.1.0.5.0
Agent Process ID       : 10598
Parent Process ID      : 10499
Agent URL              : https://itsrv33c.mydomain:3872/emd/main/
Local Agent URL in NAT : https://itsrv33c.mydomain:3872/emd/main/
Repository URL         : https://itsrv35g.mydomain:1159/empbs/upload
Started at             : 2016-11-09 09:57:05
Started by user        : oracle
Operating System       : HP-UX version B.11.31 (IA64W)
Last Reload            : (none)
Last successful upload                       : 2016-11-09 10:19:26
Last attempted upload                        : 2016-11-09 10:19:26
Total Megabytes of XML files uploaded so far : 0.2
Number of XML files pending upload           : 0
Size of XML files pending upload(MB)         : 0
Available disk space on upload filesystem    : 16.92%
Collection Status                            : Collections enabled
Heartbeat Status                             : Ok
Last attempted heartbeat to OMS              : 2016-11-09 10:19:53
Last successful heartbeat to OMS             : 2016-11-09 10:19:53
Next scheduled heartbeat to OMS              : 2016-11-09 10:20:53

—————————————————————
Agent is Running and Ready

./emctl pingOMS

Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
—————————————————————
EMD pingOMS completed successfully

$AGENT_INST/sysman/log/gcagent.log contains

—– Wed Nov 9 09:39:43 2016::26900::Agent Launched with PID 27336 at time Wed
Nov 9 09:39:43 2016 —–
—– Wed Nov 9 09:39:43 2016::27336::Time elapsed between Launch of Watchdog p
rocess and execing EMAgent is 34 secs —–
2016-11-09 09:39:44,287 [1:main] WARN – Missing filename for log handler ‘wsm’
2016-11-09 09:39:44,302 [1:main] WARN – Missing filename for log handler ‘opss’
2016-11-09 09:39:44,305 [1:main] WARN – Missing filename for log handler ‘opsscf
g’
Agent is going down due to an OutOfMemoryError
—– Wed Nov 9 09:40:06 2016::26900::Checking status of EMAgent : 27336 —–
—– Wed Nov 9 09:40:06 2016::26900::EMAgent exited at Wed Nov 9 09:40:06 201
6 with return value 57. —–
—– Wed Nov 9 09:40:06 2016::26900::EMAgent will be restarted because of an O
ut of Memory Exception. —–
—– Wed Nov 9 09:40:06 2016::26900::writeAbnormalExitTimestampToAgntStmp: exi
tCause=OOM : restartRequired=1 —–
—– Wed Nov 9 09:40:06 2016::26900::Restarting EMAgent. —–

That means, the agent is starting, then stopping, then restarting, then stopping (aka “thrashing”)

Take a look for the running agent daemon at the OS level:

ps -ef | grep agent12c
oracle 26900 1 0 09:39:09 pts/0 0:00 /u01/app/oracle/agent12c/core/12.1.0.5.0/perl/bin/perl /u01/app/oracle/agent12c/core/12.1.0.5.0/bin/emwd.pl agent /u01/app/oracle/…
oracle 27665 26900 0 09:40:12 pts/0 1:01 /u01/app/oracle/agent12c/core/12.1.0.5.0/jdk/bin/IA64W/java -Xmx169M -XX:MaxPermSize=96M -server -Djava.security.egd=file:///de…

Oh – it’s set up for the default of 169MB of RAM. Check My Oracle Support.

EM 12c: emctl start agent Fails ‘Fatal agent error: State Manager failed at Startup’ ‘restarted because of an Out of Memory Exception’ Reported in emagent.nohup /gcagent.log (Doc ID 1950490.1)

Verify this setting also in the $AGENT_INST/sysman/log/gcagent.log:

—– Wed Nov 9 09:40:06 2016::26900::Auto tuning the agent at time Wed Nov 9
09:40:06 2016 —–
inMemoryLoggingSize=6291456
_SchedulePersistTimer=30
MaxThreads=10
agentJavaDefines=-Xmx169M -XX:MaxPermSize=96M
SchedulerRandomSpreadMins=5
UploadMaxNumberXML=5000
UploadMaxMegaBytesXML=50.0
Auto tuning was successful

Well, it’s trying. Per the above Doc ID 1950490.1

Stop the agent.

$AGENT_INST/bin/emctl stop agent

Edit the $AGENT_INST/sysman/config/emd.properties (this contains the runtime parameters for the agent):

old entry:
agentJavaDefines=-Xmx169M -XX:MaxPermSize=96M

new entry:
agentJavaDefines=-Xmx512M -XX:MaxPermSize=96M

(You may tune these values up or down according to your environment requirements)

Restart the agent:

$AGENT_INST/bin/emctl stop agent

Agent runs, and keeps running like the E-Bunny.

Oracle

What’s New Features in OEM AMS Applications Management Pack 13.1.1.1.0 for OEM 13c Released July 22

August 24, 2016 jhlui1 Leave a comment

Per Getting Started with Oracle Application Management Pack (AMP) for Oracle E-Business Suite Release 13.1.1.1.0 (Doc ID 2045552.1)

Key Features

The following are brief descriptions of key features introduced in this release:

Enterprise Manager 13c Technology Uptake
Hybrid Cloud Management
System Management for Oracle E-Business Suite
Change Management for Oracle E-Business Suite

Enterprise Manager 13c Technology Uptake

Oracle Enterprise Manager 13c includes several enhancements including:

A new Oracle Fusion Middleware technology stack
The “Alta” user interface to enhance user experience
New OEM Alta Interface 13c
Edition Based Redefinition (EBR) support

Hybrid Cloud Managment

Oracle Application Management Suite now delivers several capabilities to monitor and manage Oracle E-Business Suite environments on Oracle Cloud and on-premises. These capabilities are delivered through the Enterprise Manager command line interface (EMCLI) for greater flexibility.

Discovering Orchestration Virtual Machine
Provisioning a New Oracle E-Business Suite 12.2 Instance
Backup and Restore an Oracle E-Business Suite Instance
Lift and Shift E-Business Suite instances from On-premise to Oracle Cloud
Cloning E-Business Suite instances within Oracle Cloud
Ability to run FS_Clone only option
Apply E-Business Suite patches
Start and Stop E-Business Suite instances on Cloud
To create custom packages

System Managment for Oracle E-Business Suite

Discovery and Monitoring

Real User Experience Insight (RUEI) regions integrated within the Oracle E-Business Suite Summary page
- In this release, we have integrated five different RUEI regions within the Oracle E-Business Suite Summary page. Customers can personalize and activate one or all RUEI regions. These RUEI regions are specific to a given Oracle E-Business Suite instance and customers can view the data and drill down for detailed information.
Auto Host Aliasing
- Host aliasing is now automated and will attempt to match the host name in the Oracle E-Business Suite context file and the discovered host name in Oracle Enterprise Manager. If this attempt fails, you will be prompted to perform the mapping manually. Customization Manager and Patch Manager features now support host aliasing.

Change Management for Oracle E-Business Suite

Patch Manager – Oracle E-Business Suite Technology Stack Pack Deployments

During the prior release of Application Management Suite, you had the ability to view patch recommendations for E-Business Suite applications and the technology stack. In this release, you now have the ability to deploy Database and WebLogic server patches using Enterprise Manager patch plans.

Customization Manager – Customization Discovery & Reporting

Customization Discovery & Reporting process now includes discovery of database objects.

Cloning – Automated Cloning

You now have the ability to save the cloning interview process as templates that can be used for future purposes for Smart Clone for R12, 12.2.x procedure.
You can add custom parameters to the cloning procedure

Oracle

OEM 12c SSL Certificate Swapping (HowTo)

April 13, 2016 jhlui1 Leave a comment

"This Connection is Untrusted" error message — “This Connection is Untrusted” error message

Oracle Enterprise Manager out of the box, comes with demonstration SSL certificates that are generally okay for getting the basic system up and running, but should not be left as your long-term solution for SSL/HTTPS connections to your Oracle Management Server (OMS).

Similar to how e-Business Suite installations delivered a DEMO Certificate Authority certificate with the bundled Internet Application Server (iAS) installation, OEM packages do the same thing. But eventually, forced by browser and client workstation OS upgrades, you will eventually need to install “real” certificates by a true trusted Root Certification Authority (RCA) so that your client browsers don’t begin rejecting encrypted connections to your OMS.

If you search for SSL Certificate authorities, there are many well-known public RCA’s such as, DigiCert, Verisign, Thawte, GeoTrust, and others, or even those available from your domain registrar. Larger organizations probably have their own Certificate Authority signing server on-premise that allow generation of trusted certificates, as well. The only really important thing is that the CA is actually available in your browser and OS as a Trusted Root Authority, and that the signing chain is verifiable to prevent issues with SSL/HTTPS handshaking. For each middle-tier OMS host, or virtual host if you are set up for high-availability with multiple WebLogic servers, a certificate request is generated, signed and then imported back into the keychains related to the OMS Weblogic hosts, and the OMS Servicing Agents (the OEM Agents installedo on the OMS middle-tier hosts.)

Once a new certificate is installed to the OMS itself (in WebLogic), you will also need to install the related RCA to the OMS-side OEM Agent servicing all of the connections to the other OEM Agents, so that they too, will be SSL enabled.

# OEM SSL Certificate swapping
#   EM 12c Cloud Control: How to Create a Wallet With Third Party Trusted Certificate that Can Be Imported into the OMS Console application ? (Doc ID 1937457.1)
#    EM 12c: Steps to Create and Import Third Party / Self-Signed SSL Certificates for WebLogic Server in an Enterprise Manager 12c Cloud Control Installation (Doc ID 1527874.1)
STAGE_DIR=/mnt/nfs/FMW/certs
EM_INSTANCE_HOME=/oemgc/Oracle/gc_inst2/em/EMGC_OMS1      #WebTierIH2 OHS/ohs2 on (alternate hostname

oemmgr@(primary hostname) $> cd $STAGE_DIR

oemmgr@(primary hostname) $> cat import.sh
export JAVA_JREBIN=$JAVA_HOME/jre/bin
export CERTS=/mnt/nfs/FMW/certs
$JAVA_JREBIN/keytool -import -file $CERTS/ORGPOLICYCA.cer -trustcacerts -alias ORGROOTCA -storepass changeit -noprompt -keystore $JAVA_HOME/jre/lib/security/cacerts
$JAVA_JREBIN/keytool -import -file $CERTS/ORGROOTCA.cer -trustcacerts -alias ORGRootPolicyCA -storepass changeit -noprompt -keystore $JAVA_HOME/jre/lib/security/cacerts
$JAVA_JREBIN/keytool -import -file $CERTS/ORGHOSTISSUECA1.cer -trustcacerts -alias ORGHOSTissueca1 -storepass changeit -noprompt -keystore $JAVA_HOME/jre/lib/security/cacerts

oemmgr@(primary hostname) $> . ./import.sh
Certificate was added to keystore
Certificate was added to keystore
Certificate was added to keystore

# Determine keystore directory (found by locating your Oracle Home Service (OHS) installation filesystem)
oemmgr@(primary hostname) $> ps -ef | grep ohs
oemmgr     873 9334 0 Jan13 ?        00:47:48 /oemgc/Oracle/MW3/Oracle_WT/ohs/bin/httpd.worker -DSSL
oemmgr    9334 9305 0 Jan13 ?        00:00:06 /oemgc/Oracle/MW3/Oracle_WT/ohs/bin/httpd.worker -DSSL
oemmgr    9342 9334 0 Jan13 ?        00:00:11 /oemgc/Oracle/MW3/Oracle_WT/ohs/bin/odl_rotatelogs -l /oemgc/Oracle/gc_inst2/WebTierIH1/diagnostics/logs/OHS/ohs1/ohs1-%Y%m%d%H%M%S.log 10M 70M
oemmgr    9344 9334 0 Jan13 ?        00:00:11 /oemgc/Oracle/MW3/Oracle_WT/ohs/bin/odl_rotatelogs /oemgc/Oracle/gc_inst2/WebTierIH1/diagnostics/logs/OHS/ohs1/access_log 10M 100M
oemmgr    9345 9334 0 Jan13 ?        00:00:01 /oemgc/Oracle/MW3/Oracle_WT/ohs/bin/odl_rotatelogs /oemgc/Oracle/gc_inst2/WebTierIH1/diagnostics/logs/OHS/ohs1/em_upload_http_access_log 10M 100M
oemmgr    9346 9334 0 Jan13 ?        00:02:13 /oemgc/Oracle/MW3/Oracle_WT/ohs/bin/odl_rotatelogs /oemgc/Oracle/gc_inst2/WebTierIH1/diagnostics/logs/OHS/ohs1/em_upload_https_access_log 10M 100M
oemmgr    9349 9334 0 Jan13 ?        00:00:07 /oemgc/Oracle/MW3/Oracle_WT/ohs/bin/odl_rotatelogs /oemgc/Oracle/gc_inst2/WebTierIH1/diagnostics/logs/OHS/ohs1/mod_wl_ohs.log 10M 100M
oemmgr    9350 9334 0 Jan13 ?        00:00:00 /oemgc/Oracle/MW3/Oracle_WT/ohs/bin/odl_rotatelogs -l -h:/oemgc/Oracle/gc_inst2/WebTierIH1/config/OHS/ohs1/component_events.xml_ohs1 /oemgc/Oracle/gc_inst2/WebTierIH1/auditlogs/OHS/ohs1/audit-pid9334-%Y%m%d%H%M%S.log 1M 4M
oemmgr    9351 9334 0 Jan13 ?        00:00:28 /oemgc/Oracle/MW3/Oracle_WT/ohs/bin/httpd.worker -DSSL
oemmgr    9352 9334 0 Jan13 ?        00:47:55 /oemgc/Oracle/MW3/Oracle_WT/ohs/bin/httpd.worker -DSSL
oemmgr    9353 9334 0 Jan13 ?        00:47:52 /oemgc/Oracle/MW3/Oracle_WT/ohs/bin/httpd.worker -DSSL

# Confirm settings
oemmgr@(primary hostname) $> grep keystore /oemgc/Oracle/gc_inst2/WebTierIH1/config/OHS/ohs1/ssl.conf
SSLWallet file:/oemgc/Oracle/gc_inst2/WebTierIH1/config/OHS/ohs1/keystores/console

# Stage copy of revised CA wallet #WebTierIH2/config/OHS/ohs2 on (secondary hostname)
oemmgr@(primary hostname) $> cp -r /mnt/nfs/FMW/certs/oemgc.domain /oemgc/Oracle/gc_inst2/WebTierIH1/config/OHS/ohs1/keystores
cp -r /mnt/nfs/FMW/certs/oemgc.domain /oemgc/Oracle/gc_inst2/WebTierIH2/config/OHS/ohs2/keystores

# Check permissions 770 on wallet dir, 600 on wallets
oemmgr@(secondary hostname) $> ls -la /oemgc/Oracle/gc_inst2/WebTierIH2/config/OHS/ohs2/keystores/oemgc.domain
ls -la /oemgc/Oracle/gc_inst2/WebTierIH1/config/OHS/ohs1/keystores/oemgc.domain
total 32
drwxrwx— 2 oemmgr oinstall 4096 Mar 9 12:50 .
drwx—— 7 oemmgr oinstall 4096 Mar 9 12:50 ..
-rw——- 1 oemmgr oinstall 11653 Mar 9 12:50 cwallet.sso
-rw——- 1 oemmgr oinstall 11576 Mar 9 12:50 ewallet.p12

# Primary wallet for the OMS console
oemmgr@(primary hostname) $> cd /oemgc/Oracle/MW3/oracle_common/bin
oemmgr@(primary hostname) $> ./orapki wallet display -wallet /oemgc/Oracle/gc_inst2/WebTierIH1/config/OHS/ohs1/keystores/console
Oracle PKI Tool : Version 11.1.1.7.0
Copyright (c) 2004, 2013, Oracle and/or its affiliates. All rights reserved.

Requested Certificates:
User Certificates:
Subject: CN=oemgc.domain
Trusted Certificates:
Subject: CN=(primary hostname).domain,C=US,ST=CA,L=EnterpriseManager on (primary hostname).domain,OU=EnterpriseManager on (primary hostname).domain,O=EnterpriseManager on (primary hostname).domain

# Confirm new wallet contents
oemmgr@(primary hostname) $> ./orapki wallet display -wallet /oemgc/Oracle/gc_inst2/WebTierIH1/config/OHS/ohs1/keystores/oemgc.domain
Oracle PKI Tool : Version 11.1.1.7.0
Copyright (c) 2004, 2013, Oracle and/or its affiliates. All rights reserved.

Requested Certificates:
User Certificates:
Subject:        CN=*.domain,OU=Information Technology,O=ORG My Org,L=My City,ST=California,C=US
Trusted Certificates:
Subject:        OU=Class 2 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US
Subject:        CN=ORG POLICY CA
Subject:        CN=GTE CyberTrust Global Root,OU=GTE CyberTrust Solutions\, Inc.,O=GTE Corporation,C=US
Subject:        CN=ORG ROOT CA
Subject:        CN=HOSTISSUECA1,DC=fss,DC=ORG,DC=com
Subject:        OU=Class 3 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US
Subject:        OU=Class 1 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US

#Add certificates to monitoring agent for the OMS:
cd $AGENT_HOME/bin
./emctl stop agent
# Default jks keyring password – welcome
./emctl secure add_trust_cert_to_jks -trust_certs_loc /mnt/nfs/FMW/certs/ORGROOTCA.cer -alias ORGROOTCA
./emctl secure add_trust_cert_to_jks -trust_certs_loc /mnt/nfs/FMW/certs/ORGPOLICYCA.cer -alias ORGPOLICYCA
./emctl secure add_trust_cert_to_jks -trust_certs_loc /mnt/nfs/FMW/certs/ORGHOSTISSUECA1.cer -alias HOSTISSUECA1
./emctl start agent

# Support virtual host ignore hostname verification
export EM_COMMON_JAVA_OPTIONS=”-Dweblogic.security.SSL.ignoreHostnameVerification=true -Djava.security.egd=file:///dev/./urandom -Dweblogic.log.FileName=/oemgc/Oracle/gc_inst2/em/EMGC_OMS1/sysman/log/wls.log”

# Backup the EM_INSTANCE_BASE/em/EMGC_OMS1/emgc.properites file
cd /oemgc/Oracle/gc_inst2/em/EMGC_OMS1 #OMS2 on (secondary hostname)
cp emgc.properties emgc.properties_selfsign

# Requires SYSMAN password
# (secondary hostname)
$OMS_TOP/bin/emctl secure console -wallet /oemgc/Oracle/gc_inst2/WebTierIH2/config/OHS/ohs2/keystores/oemgc.domain

# Example output
# Oracle Enterprise Manager Cloud Control 12c Release 5
# Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
# Securing Console… Started.
# Enter Enterprise Manager Root (SYSMAN) Password :
# Securing Console… Successful
# Restart OMS

# (primary hostname)
$OMS_TOP/bin/emctl secure console -wallet /oemgc/Oracle/gc_inst2/WebTierIH1/config/OHS/ohs1/keystores/oemgc.domain
$OMS_TOP/bin/emctl stop oms
$OMS_TOP/bin/emctl start oms

# 10-JUN-2016 Addendum – enable emcli login by establishing trust for the new certificate

oemmgr@(primary hostname)$> ./emcli setup -url=https://oemgc.auca.corp:7799/em -username=”SYSMAN” -password=$SYSMAN_PW

Oracle Enterprise Manager 12c Release 5.
Copyright (c) 1996, 2015 Oracle Corporation and/or its affiliates. All rights reserved.

The configuration directory “/home/oemmgr” may not be local. See the “dir” option in the help for the setup command.
Do you want to continue using this directory? [yes/no] yes

Warning: This certificate has not been identified as trusted in the local trust store
————————————–
[<blah – lots of cryptic information about the new certificate>
]
————————————–
Do you trust the certificate chain? [yes/no] yes
Emcli setup successful

# Test emcli connectivity

oemmgr@(primary hostname)$> ./emcli login -username=”SYSMAN” -password=$SYSMAN_PW

#Section 4: Rolling back to the Demonstration WLS Certificate
#If you need to switch back the WLS components in the OMS installation to use the default WebLogic Server demonstration certificates, execute the following steps on each OMS.

1.Stop the OMS:

cd <OMS_HOME>/bin
emctl stop oms

2.Run the following command:

cd <OMS_HOME>/bin>
emctl secure wls -use_demo_cert
emctl secure console -self_signed

3.Stop the OMS:

cd <OMS_Home>/bin
emctl stop oms -all

4.Start the OMS:

cd <OMS_Home>/bin
emctl start oms

Oracle

DataGuard and OEM 12c OMS DB Failover Configuration

January 14, 2016 jhlui1 Leave a comment

# When Oracle DataGuard high-availability for the OMS database is configured using the OEM DataGuard Administration Wizard, and fast-start failover is configured, fail-overs automatically rename the standby as primary, and vice-versa and establish the change-over in roles. While this accomplishes the database staying online and available on the secondary host (or all other databases in the DG group), the Enterprise Manager OMS must be told how to connect to it – preferably transparently.

# DataGuard OMS Registration
# Enterprise Manager Grid Control 11g: How to Configure the OMS Connect String when Repository is in a Dataguard setup (Doc ID 1328768.1)
# OEMPR11 is our primary DB SID/Service Name
# OEMPR11_DGMGRL is our alias for the fail-over service (pointing to all DG instances)

SQLPLUS as SYS:
SQL> exec DBMS_SERVICE.CREATE_SERVICE (service_name => ‘OEMPR11_DGMGRL’,network_name => ‘OEMPR11_DGMGRL’,aq_ha_notifications => true,failover_method => ‘BASIC’,failover_type => ‘SELECT’,failover_retries => 180,failover_delay => 1);

SQL> exec dbms_service.start_service(‘OEMPR11_DGMGRL’);

# Verify operation:

$> lsnrctl services # Should see the new OEMPR11_DGMGRL service listed

# Create a Database Trigger so that the service can be stopped when the Database role becomes standby and started only when the Database role is Primary:

SQL> CREATE OR REPLACE TRIGGER manage_OCIservice after startup on database
DECLARE
role VARCHAR(30);
BEGIN
SELECT DATABASE_ROLE INTO role FROM V$DATABASE;
IF role = ‘PRIMARY’ THEN
DBMS_SERVICE.START_SERVICE(‘OEMPR11_DGMGRL’);
ELSE
DBMS_SERVICE.STOP_SERVICE(‘OEMPR11_DGMGRL’);
END IF;
END;

# Re-configure the OMS (All MT hosts) to have the connection string as:
$> cd <OMS_HOME>/bin
# Following is a single-line command (basically an entire JDBC style connect string)
$> ./emctl config oms -store_repos_details -repos_conndesc ‘(DESCRIPTION=(FAILOVER=ON)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=<primaryDBHostnameFQDN>)(PORT=1522))(ADDRESS=(PROTOCOL=TCP)(HOST=<secondaryDBHostnameFQDN>)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=OEMPR11_DGMGRL))(FAILOVER_MODE=(TYPE=select)(METHOD=basic)))’ -repos_user sysman

# Example output
$> ./emctl config oms <…> TYPE=select)(METHOD=basic)))’ -repos_user sysman <
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.
Enter Repository User’s Password :
Successfully updated datasources and stored repository details in Credential Store.
If there are multiple OMSs in this environment, run this store_repos_details command on all of them.
And finally, restart all the OMSs using ’emctl stop oms -all’ and ’emctl start oms’.
It is also necessary to restart the BI Publisher Managed Server.

# Add the tnsnames.ora entry (all DB hosts at minimum)
OEMPR11_DGMGRL=
(DESCRIPTION=
(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=TCP)(HOST=<primaryDBHostnameFQDN>)(PORT=1522))
(ADDRESS=(PROTOCOL=TCP)(HOST=<secondaryDBHostnameFQDN>)(PORT=1521))
)
(CONNECT_DATA=(SERVICE_NAME=OEMPR11_DGMGRL))
(FAILOVER_MODE=(TYPE=select)(METHOD=basic))
)

# Testing connectivity:

$> sqlplus sysman/$SYSMAN_PW@'(DESCRIPTION=(FAILOVER=ON)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=<primaryDBHostnameFQDN>)(PORT=1522))(ADDRESS=(PROTOCOL=TCP)(HOST=<secondaryDBHostnameFQDN>)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=OEMPR11_DGMGRL))(FAILOVER_MODE=(TYPE=select)(METHOD=basic)))’

# Test the Failover Service:

# Connect to the Database from sqlplus using SYSMAN user via the new service created above:

$> sqlplus sysman/$SYSMAN_PW@OEMPR11_DGMGRL

# Execute these queries to verify the Database name and service names:

SQL> select db_unique_name from v$database;

DB_UNIQUE_NAME
——————-
OEMPR11

SQL> show parameter service_names

NAME            TYPE     VALUE
————-   ——- ————————————
service_names   string   OEMPR11, OEMPR11_DGMGRL

# Re-start the OMS once so that the connection string change is saved:

cd <OMS_HOME>/bin
./emctl stop oms -all #on AdminServer MT
./emctl stop oms #other MTs
./emctl start oms

Oracle

Investigating an OEM 12c E-Business Suite Alert (AMS 12.1.0.x)

December 7, 2015 jhlui1 Leave a comment

The Applications Management Suite plug-in for Oracle Enterprise Manager simplifies discovery of the myriad of subtargets that make up an Oracle E-Business Suite instance. Correspondingly, the number of alerts sent out can rise dramatically because of the inter-relationships between the components.

For example, the outage of a single Apache process triggers all of the following associated targets also to flag as a service Down status:

INSTANCE-Oracle E-Business Suite
INSTANCE-Infrastructure INSTANCE_host-APPL_TOP Context
HTTP_Server

The number of downed targets increases if a subcomponent of a primary component (such as a single JVM thread under the OACore process) experiences an outage.

This is a simple walk-through of navigating one of the e-mail alerts to start figuring out what happened.

The e-mail alert looks like this:

From: OEM12 Burbank
Sent: Monday, December 07, 2015 8:30 AM
To: DBAs
Subject: EM Event: Fatal:INSTANCE-Oracle E-Business Suite – Target is down; 1 member is down: INSTANCE_EBS Availability System

Host=hostname
Target type=Oracle E-Business Suite
Target name=INSTANCE-Oracle E-Business Suite
Categories=Availability
Message=Target is down; 1 member is down: INSTANCE_EBS Availability System
Severity=Fatal
Event reported time=Dec 7, 2015 8:29:14 AM PST
Target Lifecycle Status=Production
Operating System=Linux
Platform=x86_64
Associated Incident Id=390885
Associated Incident Status=New
Associated Incident Owner=
Associated Incident Acknowledged By Owner=No
Associated Incident Priority=None
Associated Incident Escalation Level=0
Event Type=Target Availability
Event name=Status
Availability status=Down
Root Cause Analysis Status=Symptom
Rule Name=EBS Notifications,Rule_EBS_Notifications
(to get notified, you set up Rule Sets that tell OEM when and what to notify you about)
Rule Owner=DBA
Update Details:
Target is down; 1 member is down: INSTANCE_EBS Availability System
Incident created by rule (Name = Incident management rule set for all targets, Incident creation rule for a Target Down availability status [System generated rule]).

To investigate an event alert, click on the Associated Incident ID (e.g. the 390885 which on your system will be a URL taking you into OEM) which will take you to the associated Incident Summary page.

Click on Related Events to investigate what raised the event alert (there may be more than one cause):

ss1 — OEM 12c AMS 12.1.0.4 – Incident Details

From the screen, it shows the red mark on PRODARMK-Infrastructure PRODARMK_ascopofinm01-APPL_TOP Context (Oracle E-Business Suite Node).

Click on that link in the list of Targets.

Navigate to Monitoring -> Status History:

ss2 — OEM 12c AMS 12.1.0.4 – Navigation Target: Monitoring -> Status History

Change the Availability History view to All History (the related underlying event caused is displayed.)

ss3 — OEM 12c AMS 12.1.0.4 – Target: Status History Details

If you click on the related Message (e.g. Target is down; 1 member is down: INSTANCE_hostname.auca.corp_oacore_JVM_…); you will then be shown the related Event page for that target:

ss4 — OEM 12c AMS 12.1.0.4 – Target: Event Details

Click on the Related Events tab for this target, to confirm the service alert recorded:

ss5 — OEM 12c AMS 12.1.0.4 – Target: Event Details -> Related Events Timeline

If this is a recurring issue, by sliding the timeline back and forth (and adjusting the period view to a larger sample) you can see if there are any associated time-related occurrences that can be used to identify root cause.

For the specific issue, login to the associated host, and view the output and error logs for the process itself to determine what triggered the alert (in this case, the JVM automatically restarted the OACore process that had run out of memory.)

Oracle

What’s New Features in Oracle Enterprise Manager (OEM) Cloud Control 12c Release 5 (12.1.0.5)

August 24, 2015 jhlui1 Leave a comment

I kept getting asked what’s new in Release 5 – so here’s the summary of what’s changed since 12.1.0.4.0 (the prior release for the past 12 months):

OEM Release 5 Download Web Site — http://www.oracle.com/technetwork/oem/grid-control/downloads/index.html

http://docs.oracle.com/cd/E24628_01/doc.121/e25353

What’s New in Enterprise Manager Cloud Control 12c Release 5 (12.1.0.5)

Oracle Cloud Management (Hybrid Cloud)
Database Plug-in 12.1.0.8 Features
Fusion Middleware Plug-in 12.1.0.8 Features
Cloud Management Plug-in 12.1.0.10 Features
Chargeback and Consolidation Planner Plug-in 12.1.0.7 Features
Virtual Infrastructure Plug-in 12.1.0.2

Oracle Cloud Management (Hybrid Cloud) – This enhancement is new in Enterprise Manager Cloud Control Release 5 (12.1.0.5).
Enterprise Manager Cloud Control now provides you with a single pane of glass for monitoring and managing both your on-premise and Oracle Cloud deployments, all from the same management console. By deploying Management Agents onto the Oracle Cloud virtual hosts serving your Oracle Cloud services, you are able to manage Oracle Cloud targets just as you would any other targets. The communication between Management Agents and your on-premise Oracle management service instances is secure from external interference. Support is provided for managing Oracle Database and Fusion Middleware PaaS targets, as well as JVMD support for monitoring JVMs on your Oracle Cloud virtual hosts.

Oracle Cloud Management includes the following key features:

Automated agent deployment and configuration
Database and Java PaaS instances monitoring
Incident management including notifications and ticketing integration
Configuration management including Search and Inventory, comparison between on-premise and cloud instances, configuration history, and compliance
Cloning between on-premise and Oracle Cloud
One-off patching of Oracle Cloud database instances

Database Plug-in 12.1.0.8 Features

Snap Clone Leveraging Sparse Clones on Exadata
You can now create a Test Master pluggable database (PDB) to use as the snapshot source for thin cloning on Exadata ACFS.
For functional testing scenarios, for example on development or testing systems, business IT users now have on-demand access to production data copies without incurring the penalty of multi-terrabyte storage.
Data Cloning to Oracle Cloud
With this release you have the following data cloning to Oracle Cloud options:
- Like-to-like cloning: on premise to-from Oracle Private Cloud:
  Enterprise Manager12c PDB to 12c PDB, assuming the containers exist on both sides
  Regular non-container dedicated database to dedicated database
- Like-to-unlike data migration:
  Regular to PDB migration where the source is an on-premise non-PDB database and the data is migrated to a cloud based PDB
  PDB to normal database migration where the source is a PDB on an Oracle PaaS cloud and data and schema are migrated to a non-container CDB on premise

Fusion Middleware Plug-in 12.1.0.8 Features

Generic JVM Provisioning for Private Cloud
This feature enables self-service users to provision non-Oracle middleware components as cloud services. It also uses Enterprise Manager’s chargeback and quota management capabilities to enable administrators to limit service usage based on organizational policies.
JVMD Support for PaaS
Enterprise Manager Cloud Control now enables you to deploy JVMD agents on your Oracle Cloud virtual hosts. These deployed JVMD agents can report to a JVMD engine deployed in your private network. This feature enables you to monitor the JVMs deployed on the virtual hosts running your Oracle Cloud services as well as the JVMs deployed on the hosts in your private network using a single console, for example Enterprise Manager Cloud Control deployed in your private network.
Middleware Self Service Portal Support for Virtual Java as a Service with Exalogic Systems
WebLogic services that have been provisioned on Exalogic systems using Service Manager can now be viewed in the Cloud Self Service Portal as part of the Exalogic Private Cloud service family.
MWaaS on Solaris SPARC
This project integrates MWaaS with the Oracle Enterprise Manager Ops Center. It provides the ability to dynamically add or remove the virtual hardware in a MWaaS cloud.

Cloud Management Plug-in 12.1.0.10 Features

Dynamic Scaling of PaaS Zone Through Integration with the Sun Solaris SPARC Ops Center 12c Adapter
Enterprise Manager 12c is integrated with the Ops Center through the Sun Solaris SPARC adapter. An instance of this registered adapter is used by the associated PaaS Infrastructure zone to provision or delete VMs on demand. The communication with the Ops Center server is achieved through the use of a client kit.
This integration enables customers to provide layered services capability in a private cloud, for example DBaaS using a Solaris VM based IaaS. Any spike in compute demand can now be handled instantly by cloud administrators.
DBaaS PaaS Provider Pools Integration with Ops Center
Administrators can now increase the capacity of a DBaaS pool by clicking a single button. Doing this transparently provisions a virtual machine or cluster through the associated Ops Center adapter attached to the PaaS zone. It also deploys the requisite software components, for example Enterprise Manager Agent, Grid Infrastructure, Oracle Database and so on, through the gold image provisioning or by cloning from an existing reference member of the pool.
Compute resource scarcity at PaaS provider level results in the failure of self service provisioning requests. This integration helps SSA administrators to respond to such failures promptly.
Clone to Oracle Cloud
Administrators now have the ability to create a full clone of an Enterprise Manager 12c pluggable database amongst existing container databases within the on-premise IT infrastructure or to an Oracle Cloud. Administrators can also clone a service from on-premise to Oracle PaaS and vice-versa. The cloning is supported in two broad categories:
- Peer-peer clone: Clones directly from an on-premise target to Oracle Cloud.
- Clone through the Software Library: Administrators can archive a gold image into the Software Library in one step and then deploy it in a separate step.

Chargeback and Consolidation Planner Plug-in 12.1.0.7 Features

Host Consolidation Support for Oracle Cloud Shapes
Oracle Enterprise Manager now provides host consolidation support for Oracle Cloud shapes. When creating scenarios for physical server to physical server (P2P) consolidations that target new or phantom destinations, you can choose physical machines configured in the Oracle cloud by selecting the cloud computing configuration, or shape, to use as the destination. Oracle provides a wide range of shapes to help you select a combination of processing power and memory for your instances that best suits your business requirements.

Virtual Infrastructure Plug-in 12.1.0.2

Dynamic Resource Provisioning Support
Through integration with Oracle Enterprise Manager Ops Center, dynamic on-demand resource provisioning enables Oracle Cloud and self service administrators to dynamically increase or decrease resources in their private cloud setup based on usage requirements. Resources can either be infrastructure resources such as hosts (based on virtual machines) or platform resources such as middleware and database Oracle homes. Resources can be dynamically added when additional service instances need to be provisioned due to high usage or decreased if existing resources are not being used.

That’s it, in a nutshell.

Oracle

New feature in the 12.1.0.4.x OEM agents – Metrics Browser

June 16, 2015 jhlui1 Leave a comment

Actual content being collected depends on what plugins are available on the agent.

https://(agenthostname):(port)/emd/browser/main

(agenthostname):(port) obtained from $AGENT_HOME/bin/emctl status agent

Metric Browser Login

Top of Form

Enter user ID and password:
Agent UserName or root Password

PDP Type
None Sudo PowerBroker

RunAs Username Profile name (only applicable if PowerBroker)
And then click this button:

Bottom of Form

Screenshot of OEM 12.1.0.4.0 Agent Metric Browser

EMAGENT 12.1.0.4.0

Health Meter Score	Schedule	Properties	Upload System	Top Target/Metric Cpu Reports	System State Dumps	Agent Key Performance Charts	Agent KPI Charts
100.0	Schedule	Properties	Upload System	Top Target/Metric Cpu Reports	System State Dumps	Agent Key Performance Charts	Agent KPIs

Target List

TargetType

TargetName

BrokenCode

BrokenReason

Status

Version

Runtime Version

Blackout Status

Master

ScheduleStatus

HealthScore

Severities

Schedule

CollectionItems

Target Events

Host

(hostname)

—

MONITORED

4.4

false

true

OPERATIONAL

99.6

Severities

Schedule

CollectionItems

Events

Oracle Concurrent Processing

(ORACLE_SID)-Core Managers for Concurrent Processing

—

MONITORED

12.03

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Custom Oracle Concurrent Program

(ORACLE_SID)-AUS_FNDGSCST

—

MONITORED

12.02

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Custom Oracle Concurrent Program

(ORACLE_SID)-(ORACLE_SID)_7BK_AGING

—

MONITORED

12.02

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Oracle E-Business Suite Custom Objects

(ORACLE_SID)-Oracle E-Business Suite Custom Objects Configuration

—

MONITORED

12.01

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Internal Concurrent Manager

(ORACLE_SID)-Internal Concurrent Manager

—

MONITORED

12.02

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Oracle E-Business Suite Node

(ORACLE_SID)-Infrastructure (ORACLE_SID)_(hostname)-Database Context

—

MONITORED

12.03

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Oracle E-Business Suite Patch Information

(ORACLE_SID)-Oracle E-Business Suite Patch Information Configuration

—

MONITORED

12.03

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Oracle E-Business Suite Workflow

(ORACLE_SID)-Workflow Infrastructure

—

MONITORED

12.02

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Oracle Workflow Agent Listener

(ORACLE_SID)-Oracle Workflow Agent Listener

—

MONITORED

12.01

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Oracle Workflow Background Engine

(ORACLE_SID)-Workflow Background Engine

—

MONITORED

12.01

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Oracle Workflow Notification Mailer

(ORACLE_SID)-Oracle Workflow Notification Mailer

—

MONITORED

12.01

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Database Instance

(ORACLE_SID)

—

MONITORED

5.3

false

true

OPERATIONAL

99.3

Severities

Schedule

CollectionItems

Events

Database Instance

(ORACLE_SID)

—

MONITORED

5.3

false

true

OPERATIONAL

100.0

Severities

Schedule

CollectionItems

Events

Oracle E-Business Suite

(ORACLE_SID)-Oracle E-Business Suite

—

MONITORED

12.03

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Agent

(hostname):(port)

—

MONITORED

12.4

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Agent proxy

(hostname):(port)_proxy

—

MONITORED

12.01

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Oracle Home

OraDb11g_home1_(hostname)

—

MONITORED

2.0

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Oracle Home

OraDb11g_home2_(hostname)

—

MONITORED

2.0

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Oracle Home

agent12c1_(hostname)

—

MONITORED

2.0

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Oracle Home

agent12c2_18_(hostname)

—

MONITORED

2.0

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Listener

LISTENER_(hostname)

—

MONITORED

2.7

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Listener

(ORACLE_SID)_(hostname)

—

MONITORED

2.7

false

true

OPERATIONAL

—

Severities

Schedule

CollectionItems

Events

Timestamp = 2015-06-16T10:37:34.693-07:00

Logout https://(hostname):(port)/emd/browser/logout

Each clickable link allows you to see what each metric being collected is actually collecting, and what the current values that are being passed to the OEM OMS Repository look like.

This is referenced in the My Oracle Support document:

List Of All Metrics In Enterprise Manager 12c Cloud Control (Doc ID 1678449.1)

Oracle

R12 e-Business Suite and OEM Monitoring – Oracle Spins Freezes

May 7, 2015 jhlui1 Leave a comment

Every so often, system load on an e-Business Suite instance ramps up and response time to users starts climbing, often resulting in user observed errors such as:

FRM-92100 Your connect to the server has been interrupted
FRM-92102 A network error has occurred
That dreaded FRM-91201 / FRM-91200 error causing you to restart your session.

Or sometimes, the screen just freezes (aka spins, stops, is broken, stuck, motionless, looks like a screen saver,can’t do anything, won’t work, froze-up, etc.) and the person has to close their browser, or even shut-down their workstation and restart.

It's simply not doing anything - Nothing to see here, just move your cursor around. And wait... and wait. — It’s simply not doing anything – Nothing to see here, just move your cursor around. And wait… and wait.

Old technology often barks with unrelated error messages to the actual cause. If there’s a lot going on with concurrent requests, or interfaces, or analytic extracts running, the front-end response-time slows down, sometimes sufficiently to trigger these kinds of Form errors, even though technically there was no interruption to the network connectivity, either between the hosts, nor the workstation and the middle-tier application server.

However, on the database, the user-experience can be seen, although not necessarily in the place you might expect. OEM had introduced it’s Adaptive Metric Thresholds technology back in OEM 11g (in a slightly different place than in 12c (in Oracle Management Server/OMS 12.1.0.4.0). In OEM 11g, they were a link under the AWR Baseline Reports page.

OEM 11g AWR Baselines Page — See the Baseline Metric Thresholds link at the bottom.

In OEM 12c, you’ll find them under

Targets -> Database -> Peformance -> Adaptive Thresholds -> Baseline Metric Thresholds -> Edit Thresholds:

OEM 12c Baseline Metric Thresholds — Where those adaptive metric thresholds moved in 12c.

On this page and in the list of Baseline Metrics, when you click into them, you can access the trending statistics being gathered for each metric. Many times this will provide direct insight into what a user experiences as the “the system is frozen” translates into “the back-end database response time is incredibly bad.”

OEM 12c Baseline Metric Response Time per Transaction vs. Baseline — See the spikes around 7AM and 11:30AM? Those are being associated with “System Froze” reports.

In the example here, the database experienced a dramatic slow-down in response almost 5 to 10 times slower than usual, which only lasted a few seconds. But that can be enough to show up in many users’ sessions who might have just kicked off a query, or were trying to save something. Based upon the information gathered, we set the Warning and Critical thresholds to 1500ms and 2000ms respectively to start sending e-mail alert notifications upon breach of the levels. If the settings are left at “None”, no incident would be raised, and thus, no notification would be sent.

If you’re experiencing odd transient outages or sluggish behavior that defies the normal AWR and ADDM snapshot analysis, go take a look at what OEM has been gathering in the background over time and see if the statistics correlate to any of your issues. There’s value in that data. Just mine it.

Oracle

Run #em12c on #db12c? – Discussion from Oracle Open World (MOS Note: 1920632.1)

October 2, 2014 jhlui1 Leave a comment

We also discussed this during the EM4APPS SIG Meeting
https://jhlui1.wordpress.com/2014/08/26/oow14-oracle-enterprise-management-for-applications-sig-mtg-928-mw3003-330p-415p/

Ok Folks, I’ve been here are Oracle Open World for a few days now. In that time, I’ve had numerous conversations about running Oracle Enterprise Manager 12c on Oracle Database 12c. I will be honest and say that I’ve enjoyed these conversations; however, after about the fourth time I decided I need to write a quick post on the explanation discussed in these conversations.

Early this year (August) I wrote a post about the what came out of the OEM CAB in May 2014 and how to get OEM 12c to work on DB12c. The concept of running OEM 12c on DB12c, pluggable or not, have many people excited and looking forward to configuring OEM to do that very configuration. Heck, I’ve even installed it for a few customers in that configuration (non-PDB). So I’m a bit sad in having to say this: ORACLE DATABASE 12c SUPPORT FOR…

View original post 211 more words

Oracle

Enterprise Manager 12.1.0.4.0 – Post Upgrade Tasks

August 27, 2014 jhlui1 Leave a comment

Must keep up with OEM 12c Grid Control updates…

Maaz Anjum's Blog

With the new release of Enterprise Manager, I upgraded my 12.1.0.3.0 installation successfully using the superb documentation here. It is however, important to note that the post-steps are also important. For example, upgrading (at the very least) the Management Agent on the OMS host, and de-installation of the old-OMS home. Gokhan Atil wrote a good post on the actual upgrade process which is available on his blog.

Management Agent Upgrade on the OMS Host:

The upgrade process for agents has not changed since EM’s previous 12c releases.

1. Navigate to the “Upgrade Agents” link under the Setup -> Manage Cloud Control menu.

NewImage

2. Select the agent(s) for the OMS Hosts and click “OK”.

NewImage

3. With the Agents selected, click on “Submit”. You have the option to override the “Privileged Credentials” to run the root.sh script.

NewImage

4. The submitted job takes a few minutes before it completes (hopefully successfully!).

View original post 307 more words

If you Care a Little More, Things Happen. Bees can be dangerous. Always wear protective clothing when approaching or dealing with bees. Do not approach or handle bees without proper instruction and training.

Per Getting Started with Oracle Application Management Pack (AMP) for Oracle E-Business Suite, Release 13.4.1.0.0 (MOS Note 2676355.1)

New Features

References

Key Features (very similar to 13.1.1.1.0):

System Management

Configuration Management

Compliance Management

Change Management

Change Approval Framework

Patch Management

Customization Management

Cloning

Instance Administration

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Per Getting Started with Oracle Application Management Pack (AMP) for Oracle E-Business Suite Release 13.1.1.1.0 (Doc ID 2045552.1)

Key Features

Enterprise Manager 13c Technology Uptake

Hybrid Cloud Managment

System Managment for Oracle E-Business Suite

Discovery and Monitoring

Change Management for Oracle E-Business Suite

Patch Manager – Oracle E-Business Suite Technology Stack Pack Deployments

Customization Manager – Customization Discovery & Reporting

Cloning – Automated Cloning

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

EMAGENT 12.1.0.4.0

Target List

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

If you Care a Little More, Things Happen. Bees can be dangerous. Always wear protective clothing when approaching or dealing with bees. Do not approach or handle bees without proper instruction and training.