Passing Variables to EBS Start Scripts (without having them show up in the environment)

In earlier releases of Oracle E-Business Suite, many of the standard startup/shutdown scripts would accept a password as a command-line argument.  This gave us the ability to automate and/or script certain maintenance tasks.

E-Business Suite R12.1.3

In E-Business Suite R12.1.3, for example, you could start the appsTier with the following:

cd ${ADMIN_SCRIPTS_HOME}

./adstrtal.sh apps/${APPSPW}

E-Business Suite R12.2

When we get to E-Business Suite R12.2, the startup procedure is pretty much the same, but the command line arguments are no longer there.  To get around this, we’re told to pass things on the command line u sing the echo command.  Per MOS 1902776.1, the recommendation is:

cd ${ADMIN_SCRIPTS_HOME}

{ echo "APPS" ; echo ${APPSPW} ; echo ${WLADMINPW} ; }|\
./adstrtal.sh @ -nopromptmsg

Although the command line is complicated, it works well. 

The Problem

However, the variables that are set will persist in the Unix environment of the adstrtal.sh script (and all processes that is spawns).

Test Script

So, to figure this out (and test our approaches), we have this simple test script:

#!/bin/bash
echo -e "\n--------------------------------------------------\n"
echo -e "Variables:\n"
echo -e "\tAPPSPW:  ${APPSPW}"
echo -e "\tAPPSUN:  ${APPSUN}\n"

echo -e "Full ENV DUMP in /tmp/env.txt"
echo -e "Searching through /tmp/env.txt for APPSUN and APPSPW"
echo -e "\n--------------------------------------------------\n"
env >/tmp/env.txt

grep "APPSPW\|APPSUN" /tmp/env.txt
echo -e "\n--------------------------------------------------\n"

We can set these variables and run the script normally:

{ echo ${APPSUN} ; echo ${APPSPW} ; }|./test.sh

--------------------------------------------------

Variables:

   APPSPW:  appspw
   APPSUN:  apps

Full ENV DUMP in /tmp/env.txt
Searching through /tmp/env.txt for APPSUN and APPSPW

--------------------------------------------------

APPSPW=appspw
APPSUN=apps

--------------------------------------------------

As you can see, the ${APPSUN} and ${APPSPW} variables are in the text file.

However, when we run the script with a different command line, we get a different result.

{ echo ${APPSUN} ; echo ${APPSPW} ; }|env -u APPSPW bash ./test.sh

--------------------------------------------------

Variables:

   APPSPW: 
   APPSUN:  apps

Full ENV DUMP in /tmp/env.txt
Searching through /tmp/env.txt for APPSUN and APPSPW

--------------------------------------------------

APPSUN=apps

--------------------------------------------------

The Solution

To get around that problem, we can actually DELETE the variables from the environment.

cd ${ADMIN_SCRIPTS_HOME}

{ echo "APPS" ; echo ${APPSPW} ; echo ${WLADMINPW} ; }|\
 env -u APPSPW -u SYSTEMPW -U WLADMINPW bash ./adstrtal.sh @ -nopromptmsg

The command line is even uglier. We still have ${APPSPW}, ${SYSTEMPW}, and ${WLADMINPW} in our shell, but, those variables are not inherited by the running EBS processes.

A Lifeline for EBS Customers Still on RDBMS 11gR2 and 12cR1

Yesterday, the Oracle E-Business Suite Applications Technology Group made a couple of very significant announcements for E-Business Suite customers.

  • RDBMS 12cR1 (12.1.0.2) Extended Support is available through July 31, 2022
  • The Extended Support fees have been waived for 12cR1 databases used for E-Business Suite through July 31, 2022.  [Previously, December 2020]
  • RDBMS 11gR2 (11.2.0.4) Extended Support is available through December 31, 2020.
  • The Extended Support fees have been waived for 11gR2 databases used for E-Busines Suite through December 31, 2020.

The current certified versions of the Oracle RDBMS for use with E-Business Suite R12.1 and R12.2 include 11gR2, 12cR1, and 19c.

If you’re on 12cR1, you should upgrade to 19c as soon as reasonably possible.
If you’re on 11gR2, you too, should upgrade to 19c as soon as reasonably possible..  You could upgrade to 12.1.0.2, but you would be forcing yourself into another upgrade (to 19c) pretty soon.

The Oracle Server group has switched to a “yearly” release schedule.  So, sometime during 2020, there should be an “Oracle RDBMS 20c” and, sometime in 2021, we should expect an “Oracle RDBMS 21c”.

However, given that it takes a while to certify the database against E-Business Suite on the variety of platforms that are necessary, the E-Business Suite team has indicated their intention to certify on every other release.  Also important is that the certification for E-Business Suite tends to lag the database release by at least 6 months.  So, that means that the next certified release of the Oracle RDBMS for E-Business Suite should be RDBMS 21c and you might not see that certification until late in 2021.

So, if you choose to wait for the next release, you may find yourself praying for another extension… which is increasingly unlikely.

— James

E-Business Suite 12.1 is now Certified on OEL7 and RHEL7

New this week from Steven Chan’s blog:

https://blogs.oracle.com/stevenChan/entry/oracle_e_business_suite_release5

One thing that Steven’s blog posting mentions that deserves specific emphasis is that Oracle Database 11.2.0.4 and 12.1.0.2 are ALSO certified on OEL7/RHEL7 [see MOS 1304727.1].  Please note that the certification is specific to the version of the database.  Most notably that 11.2.0.3 and 12.1.0.1 appear to be excluded from this certification.  As always, be sure to pay close attention to the certification status of your various components when planning any installation/upgrade.

It’s also important to note that, while there is a 32-bit version of RHEL6 (and E-Business Suite 12.1.3 is certified on it), there isn’t a 32-bit version of RHEL7.  This is important and, at the same time, it isn’t.  First of all, it’s highly unlikely that anyone is still using 32-bit hardware.  (Or that they ever were, for E-Business Suite on Linux).  Yes, it’s true that the appsTier components of E-Business Suite 12.1 are still 32-bit, running them on a 64-bit Linux requires only minor adjustments.  The bulk of which involve dependencies on kernel settings and Linux packages.

So, with all of this out there… Go forth and upgrade!

— James

See me at Oracle OpenWorld

Oracle OpenWorld 2013 starts on Sunday, 22 September and runs through Thursday 26 September, 2013. This will be my first time presenting at an OpenWorld conference (I’ve presented numerous times at Collaborate and other OAUG-related events). I’m looking forward to meeting you!

E-Business Suite DBA Best Practices (CON4733)
Moscone West Room #3016
Thursday, 26 September 2:00PM – 3:00PM

And, while you’re there, stop by the Atherio/RedRiver Solutions booth #122 in Moscone South. I’ll be hanging out there from time to time as well! Enjoy the conference!

Oracle Database 12c Launch Webcast

In my post yesterday, I indicated that Oracle Database 12c (12.1.0.1.0) was available for download on Oracle’s E-delivery and TechNet websites, but that it still hadn’t officially been “launched”.  The official “product launch” take place on July 10, 2013.  You can RSVP for the launch webcast through the link below:

Launch Webcast:  http://tinyurl.com/ow28hr2

Yesterday’s Blog post:  http://thedbalife.blogspot.com/2013/06/oracle-database-12c-is-available-for.html

— James

Oracle Database 12c Is Available for Download

File this under “it’s about time” and “ICYMI (In Case You Missed It), but Oracle has released Database 12c (12.1.0.1.0). Downloads can be found on their TechNet and E-Delivery sites. At this point, the only available versions are for Linux (x86-64), Solaris (Sparc64), and Solaris (x86-64). Other platforms will surely follow

Not officially released… yet

According to media reports (and my inability to find an actual press release from Oracle), the formal launch of Database 12c will occur “within a couple of weeks”.

Differences between TechNet and E-Delivery

While, otsensibly, it may be the same software, there is always the possibility that you’ll get slightly different versions. The software that you download from TechNet is usually in the form of either a zip file or a “tarball” of the staged installation. The downloads from E-Delivery are also zip files, but they represent the actual media packs (CD or DVD). For some reason, Oracle doesn’t do ISOs, but, nevertheless, the E-Delivery downloads are typically viewed as more “supported”. As a result, I recommend using the E-Delivery downloads rather than TechNet if you’re planning on doing anything that is going to need to be handled under a support contract.

Naturally, for either method, you will have to agree to license terms and export conditions. If you have never used E-Delivery from your Oracle account, there might be a slight delay as your account is verified by Oracle.

As with all new software, be sure to test thoroughly and make sure any applications are certified with 12c before deploying to production.

Oracle Client 12c is also available

The Oracle 12c Client can also be downloaded for the following platforms: Linux (x86-32), Linux (x86-64), Microsoft Windows (x86-32), Microsoft Windows (x86-64), Solaris (Sparc 64), Solaris (Sparc 32), Solaris (x86-32), Solaris (x86-64).

NOT CERTIFIED WITH E-BUSINESS SUITE

Since this blog is focused on E-Business Suite (and E-Business Suite is what I do), I feel the need to state that Database 12c is NOT certified with ANY RELEASE of E-Business Suite at this point. I suspect that we’ll see it certified against 12.1.3 and the upcoming 12.2 at some point in the future (maybe 12.2 on release). It is highly unlikely (in my opinion) to be certified against any release 11i. In the event that it is certified against 11i, you can bet that it will be a pretty low priority item.

You can find them available here:

Oracle E-Delivery: https://edelivery.oracle.com

Oracle TechNet: http://www.oracle.com/technetwork/indexes/downloads/index.html

— James

.

Internet Explorer 10 and E-Business Suite

In case you hadn’t noticed, Microsoft started pushing out Internet Explorer 10 to Windows 7 customers back in early March.  Internet Explorer 10 is, at this point, not certified with any versions E-Business Suite.

You can read more about it on the E-Business Suite Technology blog (otherwise known as “Steven Chan’s Blog”).  A link to that posting can be found here.

Deciphering support and licensing issues surrounding Oracle on VMWare

I frequently run into clients that are wanting to run Oracle products in their VMWare cluster. Since I generally deal with E-Business Suite customers, I tend to take the position of “anything that swallows machines whole should probably have a physical machine” approach to production systems. However, I can see some distinct advantages to virtualization, particularly when it comes to managing numbers of non-production environments.

Unfortunately, there is a lot of confusion out there as it relates to Oracle and virtualization… particularly surrounding one of the most popular virtualization solutions, VMWare. I’ll try to provide my best understanding of the issues here.

Are Oracle products certified on VMWare?

The short answer is, NO. But, that really shouldn’t be that much of a concern. Keep in mind that a VMWare Virtual Machine is, technically, hardware. Oracle doesn’t tend to certify against hardware. And that’s what that VMWare really is, it’s “virtual hardware”. As such, it’s really no different than a particular model of Dell or HP ProLiant.

What Oracle does do is certify against a platform. A platform is the combination of a particular version of an operating system (Solaris 10 vs. Solaris 11, for example) and processor architecture (Sun SPARC vs. Intel x86-32 or Intel x86-64). In the case of a deployment to VMWare, your platform will be determined by the operating system that you intend to run inside of the virtual machine. (For example, Red Hat Enterprise Linux 4/5/6 for x86 or x86-64).

Are Oracle products supported on VMWare?

Oracle’s official support position can be found in MOS Note 249212.1, copied below (emphasis mine):

Support Position for Oracle Products Running on VMWare Virtualized Environments [ID 249212.1]

Purpose

Explain to customers how Oracle supports our products when running on VMware

Scope & Application

For Customers running Oracle products on VMware virtualized environments. No limitation on use or distribution.

Support Status for VMware Virtualized Environments

Oracle has not certified any of its products on VMware virtualized environments. Oracle Support will assist customers running Oracle products on VMware in the following manner: Oracle will only provide support for issues that either are known to occur on the native OS, or can be demonstrated not to be as a result of running on VMware.

If a problem is a known Oracle issue, Oracle support will recommend the appropriate solution on the native OS. If that solution does not work in the VMware virtualized environment, the customer will be referred to VMwar for support. When the customer can demonstrate that the Oracle solution does not work when running on the native OS, Oracle will resume support, including logging a bug with Oracle Development for investigation if required.

If the problem is determined not to be a known Oracle issue, we will refer the customer to VMware for support. When the customer can demonstrate that the issue occurs when running on the native OS, Oracle will resume support, including logging a bug with Oracle Development for investigation if required.

NOTE: Oracle has not certified any of its products on VMware. For Oracle RAC, Oracle will only accept Service Requests as described in this note on Oracle RAC 11.2.0.2 and later releases.

In my understanding of the actual way that the policy is applied, it’s really a matter of whether or not the support engineer suspects VMWare to be the culprit. What I’m saying here is that, generally speaking, the support engineer will work your issue the same way that he/she would if you were on physical hardware. However, once that engineer thinks that VMWare could be the cause of your problem, they reserve the right to “punt” and say “call us back once you’ve reproduced it on physical hardware”.

Now, VMWare, to their credit, has a policy that they call “Total Ownership”, where they will accept accountability for any Oracle-related issues. You can read their official policy at the link below.

http://www.vmware.com/support/policies/oracle-support.html

It is my understanding that, as part of the “Total Ownership” policy, VMware will reproduce the problem on physical hardware for the customer if Oracle decides that VMWare is the problem.

What about Licensing?

Part of the big problem I’ve always had with Oracle on VMWare is caused by Oracle’s per-CPU licensing policy. My original understanding was that, if you have a total of 64 cores in your VMWare cluster, it didn’t matter if you were only using 8 cores for Oracle. Oracle would tell you that you had to pay for 64 cores. The idea behind this is that you could, potentially, resize the virtual machine to suit certain needs. Maybe you need more horsepower during month end?

What I’ve since learned is that Oracle has a policy document (below) that talks about “soft” vs. “hard” partitioning.

http://www.oracle.com/us/corporate/pricing/partitioning-070609.pdf

What I’ve described above would fall under the concept of “soft partitioning”. However, “hard partitioning” methodologies allow for a different approach. VMWare has (naturally) a nice document that explains their approach to implementing clusters that are in compliance with Oracle’s licensing requirements.

http://www.vmware.com/files/pdf/techpaper/vmw-understanding-oracle-certification-supportlicensing-environments.pdf

From that document, pay particular attention to section 2.2. In that section (specifically Scenario B), they discuss DRS Host Affinity rules and VMWare CPU pinning. (emphasis mine)

2.2 Clusters: Fully Licensed Versus Partially Licensed Clusters

Scenario B: Partially Licensed Clusters

When a customer does not have enough Oracle application instances to justify creating a dedicated cluster for those applications, only a subset of the hosts in the cluster are licensed for the Oracle application. In this situation, the customer must be careful to restrict the movement of Oracle application instances and virtual machines to only those hosts that are licensed to run the product.

In this case, DRS Host Affinity rules can be used to appropriately restrict the movement of virtual machines within the cluster. DRS Host Affinity is a vSphere feature that enables you to ensure that your Oracle applications are restricted to move only between a subset of the hosts—that is, not all hardware in the cluster is “available” to the Oracle software. DRS Host Affinity is a clustering technology and is not a mechanism for soft or hard partitioning of the servers. As explained in section 2.1, using VMware CPU pinning to partially license a host is not currently recognized by Oracle as a “hard partitioning” mechanism that receives subsystem pricing. However, once you have fully licensed the host, you have the right to design your environment such that the Oracle workloads are free to run on the licensed hosts inside the cluster. At present, Oracle does not have any stated policy regarding clustering mechanisms or DRS Host Affinity. Customers can easily maiatain records for compliance purposes as explained in section 2.3.

The advantages of this approach are similar to the advantages achieved with a fully licensed cluster. Because customers are typically able to increase the utilization of licensed processors, they reduce license requirements. However, consolidation ratios tend to be lower, because advanced vSphere features can be employed only on a smaller subset of the hosts.

VMWare CPU pinning is a feature that (in my understanding) would allow you to say that a given VM would only use certain cores in a physical host. So, if you have a single host with 16 cores, you can “pin” a given VM to four of them. According to Oracle’s partitioning document (and VMWare’s document), you would still be required to pay for all 16 cores in the box. The basic logic here is that Oracle’s licensing policy is based on the number of cores in a physical server. You can’t license part of a box. Period. No exceptions.

On the other hand, DRS Host Affinity, is a way to pin a virtual machine to a given host (or collection of hosts) within a cluster. So, let’s say that you have ten (10) 8-core physical hosts (total of 80 cores) in your VMWare cluster. Using DRS Host Affinity, youcould restrict your Oracle VMs to a subset of those physical hosts. For example, if you restricted your Oracle VMs to only five (5) of those physical hosts, VMWare’s contention is that you would only have to license 40 cores.

I sould probably include the standard “IANAL” (I am not a lawyer) disclaimer. I’m also not a VMWare administrator. What I am is a DBA and an IT Geek. That’s pretty much the limit of it.

Hopefully this provides some clarity on the issue.

For further reading on the subject, here are a couple of blog links that I used in my research:

http://blogs.vmware.com/apps/2012/11/update-on-virtualizing-oracle.html

http://longwhiteclouds.com/2012/07/21/fight-the-fud-oracle-licensing-and-support-on-vmware-vsphere/

– James

Why I don’t depend on TOAD (or OEM) and neither should you.

My apologies in advance, as this posting may sound like something of a rant.

The first thing I’d like to point out is that I have no real problem with TOAD, Oracle Enterprise Manager, or Windows-based editors. They are all excellent tools that can be extremely helpful in your environment. My objection to these tools is based solely on a lowest-common-denominator argument.

First, a little background. Back in the early 1990’s, I was working as a Unix Systems Administrator for a company in Kansas City, MO. Since then, I’ve worked mainly as a consultant.

Shortly before I started that job in Kansas City, the company had hired a new CIO who let go about half of the legacy (mainframe, COBOL) IT department. The new direction for the company was implementation of Oracle E-Business Suite on Data General Unix (DG/UX).

The mainframe IT staff that survived were being re-trained in the new technology. At one point, several of them came to me insisting that I install ISPF (an editor they were used to on the mainframe) onto the DG/UX boxes because they were struggling to learn to use the vi editor. I informed them that, while they (as a group) may carry enough weight to convince the CIO to direct me to install it (assuming it was even available). However, when they go to their next job and claim that “they know Unix”, they would be alone and wouldn’t have that leverage.  My suggestion was that I would help them to learn the vi editor. (I did offer emacs as an alternative, since it is and was extremely common on Unix systems… Unfortunately, friendlier editors like pico, nano, and joe didn’t exist yet.)

If your primary job is software development, a tool like TOAD is generally something you can depend on having. However, as a DBA, you can’t necessarily depend on having TOAD (or even Oracle Enterprise Manager) at your disposal at all times. Maybe you’re starting a new job and the previous DBA hadn’t set up Enterprise Manager (or you haven’t gotten around to it yet). Even in environments where those tools are available, they may or may not be working when you need them.

So, my advice? There are certain tools that are almost ALWAYS there. Get comfortable with ssh, SQL*Plus, and vi (or vim).  They are your friends.

— James