Tag Archives: Enterprise

What’s New Features in OEM AMS Applications Management Pack for OEM 13c Released July 22

Per Getting Started with Oracle Application Management Pack (AMP) for Oracle E-Business Suite Release (Doc ID 2045552.1)

Key Features

New EBS AMS Plugin Homepage Screenshot 13c
New EBS AMS Plugin Homepage Screenshot 13c

The following are brief descriptions of key features introduced in this release:

Enterprise Manager 13c Technology Uptake
Hybrid Cloud Management
System Management for Oracle E-Business Suite
Change Management for Oracle E-Business Suite

Enterprise Manager 13c Technology Uptake

Oracle Enterprise Manager 13c includes several enhancements including:

  • A new Oracle Fusion Middleware technology stack
  • The “Alta” user interface to enhance user experience

    New OEM Alta Interface 13c
    New OEM Alta Interface 13c
  • Edition Based Redefinition (EBR) support

Hybrid Cloud Managment

Oracle Application Management Suite now delivers several capabilities to monitor and manage Oracle E-Business Suite environments on Oracle Cloud and on-premises. These capabilities are delivered through the Enterprise Manager command line interface (EMCLI) for greater flexibility.

  • Discovering Orchestration Virtual Machine
  • Provisioning a New Oracle E-Business Suite 12.2 Instance
  • Backup and Restore an Oracle E-Business Suite Instance
  • Lift and Shift E-Business Suite instances from On-premise to Oracle Cloud
  • Cloning E-Business Suite instances within Oracle Cloud
  • Ability to run FS_Clone only option
  • Apply E-Business Suite patches
  • Start and Stop E-Business Suite instances on Cloud
  • To create custom packages

System Managment for Oracle E-Business Suite

Discovery and Monitoring
  • Real User Experience Insight (RUEI) regions integrated within the Oracle E-Business Suite Summary page
    • In this release, we have integrated five different RUEI regions within the Oracle E-Business Suite Summary page. Customers can personalize and activate one or all RUEI regions. These RUEI regions are specific to a given Oracle E-Business Suite instance and customers can view the data and drill down for detailed information.
  • Auto Host Aliasing
    • Host aliasing is now automated and will attempt to match the host name in the Oracle E-Business Suite context file and the discovered host name in Oracle Enterprise Manager. If this attempt fails, you will be prompted to perform the mapping manually. Customization Manager and Patch Manager features now support host aliasing.

Change Management for Oracle E-Business Suite

Patch Manager – Oracle E-Business Suite Technology Stack Pack Deployments
  • During the prior release of Application Management Suite, you had the ability to view patch recommendations for E-Business Suite applications and the technology stack. In this release, you now have the ability to deploy Database and WebLogic server patches using Enterprise Manager patch plans.
Customization Manager – Customization Discovery & Reporting
  • Customization Discovery & Reporting process now includes discovery of database objects.
Cloning – Automated Cloning
  • You now have the ability to save the cloning interview process as templates that can be used for future purposes for Smart Clone for R12, 12.2.x procedure.
  • You can add custom parameters to the cloning procedure


DataGuard and OEM 12c OMS DB Failover Configuration

# When Oracle DataGuard high-availability for the OMS database is configured using the OEM DataGuard Administration Wizard, and fast-start failover is configured, fail-overs automatically rename the standby as primary, and vice-versa and establish the change-over in roles.  While this accomplishes the database staying online and available on the secondary host (or all other databases in the DG group), the Enterprise Manager OMS must be told how to connect to it – preferably transparently.

# DataGuard OMS Registration
# Enterprise Manager Grid Control 11g: How to Configure the OMS Connect String when Repository is in a Dataguard setup (Doc ID 1328768.1)
# OEMPR11 is our primary DB SID/Service Name
# OEMPR11_DGMGRL is our alias for the fail-over service (pointing to all DG instances)

SQL> exec DBMS_SERVICE.CREATE_SERVICE (service_name => ‘OEMPR11_DGMGRL’,network_name => ‘OEMPR11_DGMGRL’,aq_ha_notifications => true,failover_method => ‘BASIC’,failover_type => ‘SELECT’,failover_retries => 180,failover_delay => 1);

SQL> exec dbms_service.start_service(‘OEMPR11_DGMGRL’);

# Verify operation:

$> lsnrctl services     # Should see the new OEMPR11_DGMGRL service listed

# Create a Database Trigger so that the service can be stopped when the Database role becomes standby and started only when the Database role is Primary:

SQL> CREATE OR REPLACE TRIGGER manage_OCIservice after startup on database
role VARCHAR(30);

# Re-configure the OMS (All MT hosts) to have the connection string as:
$>  cd <OMS_HOME>/bin
# Following is a single-line command (basically an entire JDBC style connect string)
$> ./emctl config oms -store_repos_details -repos_conndesc ‘(DESCRIPTION=(FAILOVER=ON)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=<primaryDBHostnameFQDN>)(PORT=1522))(ADDRESS=(PROTOCOL=TCP)(HOST=<secondaryDBHostnameFQDN>)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=OEMPR11_DGMGRL))(FAILOVER_MODE=(TYPE=select)(METHOD=basic)))’ -repos_user sysman

# Example output
$> ./emctl config oms <…> TYPE=select)(METHOD=basic)))’ -repos_user sysman               <
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation.  All rights reserved.
Enter Repository User’s Password :
Successfully updated datasources and stored repository details in Credential Store.
If there are multiple OMSs in this environment, run this store_repos_details command on all of them.
And finally, restart all the OMSs using ’emctl stop oms -all’ and ’emctl start oms’.
It is also necessary to restart the BI Publisher Managed Server.

# Add the tnsnames.ora entry (all DB hosts at minimum)

# Testing connectivity:


# Test the Failover Service:

# Connect to the Database from sqlplus using SYSMAN user via the new service created above:

$> sqlplus sysman/$SYSMAN_PW@OEMPR11_DGMGRL

# Execute these queries to verify the Database name and service names:

SQL> select db_unique_name from v$database;


SQL> show parameter service_names

NAME            TYPE     VALUE
————-   ——-  ————————————
service_names   string   OEMPR11, OEMPR11_DGMGRL

# Re-start the OMS once so that the connection string change is saved:

cd <OMS_HOME>/bin
./emctl stop oms -all    #on AdminServer MT
./emctl stop oms         #other MTs
./emctl start oms

Investigating an OEM 12c E-Business Suite Alert (AMS 12.1.0.x)

OEM 12 Home Page
OEM 12 Home Page

The Applications Management Suite plug-in for Oracle Enterprise Manager simplifies discovery of the myriad of subtargets that make up an Oracle E-Business Suite instance.  Correspondingly, the number of alerts sent out can rise dramatically because of the inter-relationships between the components.

For example, the outage of a single Apache process triggers all of the following associated targets also to flag as a service Down status:

INSTANCE-Oracle E-Business Suite
INSTANCE-Infrastructure INSTANCE_host-APPL_TOP Context

The number of downed targets increases if a subcomponent of a primary component (such as a single JVM thread under the OACore process) experiences an outage.

This is a simple walk-through of navigating one of the e-mail alerts to start figuring out what happened.

The e-mail alert looks like this:

From: OEM12 Burbank
Sent: Monday, December 07, 2015 8:30 AM
To: DBAs
Subject: EM Event: Fatal:INSTANCE-Oracle E-Business Suite – Target is down; 1 member is down: INSTANCE_EBS Availability System

Target type=Oracle E-Business Suite
Target name=INSTANCE-Oracle E-Business Suite
Message=Target is down; 1 member is down: INSTANCE_EBS Availability System
Event reported time=Dec 7, 2015 8:29:14 AM PST
Target Lifecycle Status=Production
Operating System=Linux
Associated Incident Id=390885
Associated Incident Status=New
Associated Incident Owner=
Associated Incident Acknowledged By Owner=No
Associated Incident Priority=None
Associated Incident Escalation Level=0
Event Type=Target Availability
Event name=Status
Availability status=Down
Root Cause Analysis Status=Symptom
Rule Name=EBS Notifications,Rule_EBS_Notifications
(to get notified, you set up Rule Sets that tell OEM when and what to notify you about)

Rule Owner=DBA
Update Details:
Target is down; 1 member is down: INSTANCE_EBS Availability System
Incident created by rule (Name = Incident management rule set for all targets, Incident creation rule for a Target Down availability status [System generated rule]).

To investigate an event alert, click on the Associated Incident ID (e.g. the 390885 which on your system will be a URL taking you into OEM) which will take you to the associated Incident Summary page.

Click on Related Events to investigate what raised the event alert (there may be more than one cause):

OEM 12c AMS – Incident Details

From the screen, it shows the red mark on PRODARMK-Infrastructure PRODARMK_ascopofinm01-APPL_TOP Context (Oracle E-Business Suite Node).

Click on that link in the list of Targets.

Navigate to Monitoring -> Status History:

OEM 12c AMS – Navigation Target: Monitoring -> Status History

Change the Availability History view to All History (the related underlying event caused is displayed.)

OEM 12c AMS – Target: Status History Details

If you click on the related Message (e.g. Target is down; 1 member is down: INSTANCE_hostname.auca.corp_oacore_JVM_…); you will then be shown the related Event page for that target:

OEM 12c AMS – Target: Event Details

Click on the Related Events tab for this target, to confirm the service alert recorded:

OEM 12c AMS – Target: Event Details -> Related Events Timeline

If this is a recurring issue, by sliding the timeline back and forth (and adjusting the period view to a larger sample) you can see if there are any associated time-related occurrences that can be used to identify root cause.

For the specific issue, login to the associated host, and view the output and error logs for the process itself to determine what triggered the alert (in this case, the JVM automatically restarted the OACore process that had run out of memory.)

R12 e-Business Suite and OEM Monitoring – Oracle Spins Freezes

Every so often, system load on an e-Business Suite instance ramps up and response time to users starts climbing, often resulting in user observed errors such as:

  • FRM-92100 Your connect to the server has been interrupted
  • FRM-92102 A network error has occurred

    FRM-92102 Forms Error R12 EBS
    That dreaded FRM-91201 / FRM-91200 error causing you to restart your session.

Or sometimes, the screen just freezes (aka spins, stops, is broken, stuck, motionless, looks like a screen saver,can’t do anything, won’t work, froze-up, etc.) and the person has to close their browser, or even shut-down their workstation and restart.

It's simply not doing anything - Nothing to see here, just move your cursor around. And wait... and wait.
It’s simply not doing anything – Nothing to see here, just move your cursor around. And wait… and wait.

Old technology often barks with unrelated error messages to the actual cause.  If there’s a lot going on with concurrent requests, or interfaces, or analytic extracts running, the front-end response-time slows down, sometimes sufficiently to trigger these kinds of Form errors, even though technically there was no interruption to the network connectivity, either between the hosts, nor the workstation and the middle-tier application server.

However, on the database, the user-experience can be seen, although not necessarily in the place you might expect.  OEM  had introduced it’s Adaptive Metric Thresholds technology back in OEM 11g (in a slightly different place than in 12c (in Oracle Management Server/OMS  In OEM 11g, they were a link under the AWR Baseline Reports page.

OEM 11g AWR Baselines Page
See the Baseline Metric Thresholds link at the bottom.

In OEM 12c, you’ll find them under

Targets -> Database -> Peformance -> Adaptive Thresholds -> Baseline Metric Thresholds -> Edit Thresholds:

OEM 12c Baseline Metric Thresholds
Where those adaptive metric thresholds moved in 12c.



On this page and in the list of Baseline Metrics, when you click into them, you can access the trending statistics being gathered for each metric.  Many times this will provide direct insight into what a user experiences as the “the system is frozen” translates into “the back-end database response time is incredibly bad.”

OEM 12c Baseline Metric Response Time per Transaction vs. Baseline
See the spikes around 7AM and 11:30AM? Those are being associated with “System Froze” reports.


In the example here, the database experienced a dramatic slow-down in response almost 5 to 10 times slower than usual, which only lasted a few seconds. But that can be enough to show up in many users’ sessions who might have just kicked off a query, or were trying to save something.  Based upon the information gathered, we set the Warning and Critical thresholds to 1500ms and 2000ms respectively to start sending e-mail alert notifications upon breach of the levels. If the settings are left at “None”, no incident would be raised, and thus, no notification would be sent.

If you’re experiencing odd transient outages or sluggish behavior that defies the normal AWR and ADDM snapshot analysis, go take a look at what OEM has been gathering in the background over time and see if the statistics correlate to any of your issues.  There’s value in that data. Just mine it.

Run #em12c on #db12c? – Discussion from Oracle Open World (MOS Note: 1920632.1)


Ok Folks, I’ve been here are Oracle Open World for a few days now.  In that time, I’ve had numerous conversations about running Oracle Enterprise Manager 12c on Oracle Database 12c.  I will be honest and say that I’ve enjoyed these conversations; however, after about the fourth time I decided I need to write a quick post on the explanation discussed in these conversations.  

Early this year (August) I wrote a post about the what came out of the OEM CAB in May 2014 and how to get OEM 12c to work on DB12c.  The concept of running OEM 12c on DB12c, pluggable or not, have many people excited and looking forward to configuring OEM to do that very configuration.  Heck, I’ve even installed it for a few customers in that configuration (non-PDB).  So I’m a bit sad in having to say this:  ORACLE DATABASE 12c SUPPORT FOR…

View original post 211 more words

Enterprise Manager – Post Upgrade Tasks

Must keep up with OEM 12c Grid Control updates…

Maaz Anjum's Blog

With the new release of Enterprise Manager, I upgraded my installation successfully using the superb documentation here. It is however, important to note that the post-steps are also important. For example, upgrading (at the very least) the Management Agent on the OMS host, and de-installation of the old-OMS home. Gokhan Atil wrote a good post on the actual upgrade process which is available on his blog.

Management Agent Upgrade on the OMS Host:

The upgrade process for agents has not changed since EM’s previous 12c releases.

1. Navigate to the “Upgrade Agents” link under the Setup -> Manage Cloud Control menu.


2. Select the agent(s) for the OMS Hosts and click “OK”.


3. With the Agents selected, click on “Submit”. You have the option to override the “Privileged Credentials” to run the root.sh script.


4. The submitted job takes a few minutes before it completes (hopefully successfully!).

View original post 307 more words

How to migrate EM12c R3 OMS and repository to a new host

Very useful, even if you’re moving parts of OEM 12c from one host to another, or modifying the filesystem mounts.

Pardy DBA

(EDIT 20130917: If you simply need to change the IP address of your OEM server, please review MOS note 1562029.1.  The procedure in that note may allow you to change your OEM server’s IP address without following the lengthy process I describe below.)

In order to save power in our data center, I need to migrate my EM12c R3 environment from the host where it currently runs to a new host.  I have a simple configuration, with a single OMS, no load balancer, and the repository database runs on the same host as EM12c R3 itself.  I also have BI Publisher installed and integrated with EM12c, and a few third party plugins as I’ve detailed elsewhere on this blog.  If you use an OS other than Linux x86-64 I suggest you research thoroughly as this procedure may or may not apply to your environment.  Further, if you have a multi-OMS…

View original post 3,770 more words

Upgrade Oracle Enterprise Manager 12c ( to

And we’re still awaiting 12c Database certification for the Repository.


In order to upgrade our OEM environment from to we need to download the latest version of OEM from either http://otn.oracle.com or from http://edelivery.oracle.com. Once we have the files needed to upgrade, we need to stage them in a location that is central.

Note: Most downloads on Linux will be placed in the /home/oracle/Downloads directory.

Once we are in the directory where the downloaded files are, we need to unzip these files.

#> unzip em12103_linux64_disk1.zip –d ./

The zip command needs to be repeated for all of the files downloaded.

Next, we need to move into the directory where we unzipped everything and run the runInstaller.

#> cd ./

#> ./runInstaller &

When the installer starts, we will be greeted with our standard Oracle Installation Wizard. To upgrade the OEM to, consists of nine steps. We will review these nine steps throughout this posting.


View original post 544 more words