When Can I Merge E-Business Suite Patches?

This is a question that pops up frequently. In earlier releases of E-Business Suite, the rule was that you should never merge a patch with any of it's prerequisite patches. However, many years ago (I believe it was with AD_PF.E, but I could be mistaken), Oracle introduced the ability to merge a patch and it's prerequisite patches together. There are still a few (small) restrictions.

From the Oracle Applications Maintenance Procedures manual for Release 11.5.10.2 (Part No. B19299-01, pages 3-23 and 3-24):

In general, you can safely merge any Oracle Applications patch with any other Oracle Applications patch. Older split driver patches can be merged with newer unified driver patches. Patches can and should be merged with their listed prerequisite patches to make patch application easier.

However, patches that affect the Applications DBA (AD) product must be handled separately. AD patches can be merged with other AD patches, but AD patches and non-AD patches cannot be merged because AD patches may change the AutoPatch utility itself. Merged AD patches must be created separately and applied before you apply non-AD patches.

I believe you'll also find a similar statement in the 11.5.9 version of the document.

The primary logic here is this: If a patch says it requires a certain AD patch (say, AD.I.7), then what it's really telling you is that there is likely something in the structure of the patch itself that an earlier AD release wouldn't know how to properly handle. Keep in mind that AD patches are, generally speaking, patches to the patching tool.

Now, I have seen some patches that warn against merging. And, given the nature of ATG patches, you may be wise to exercise a level of caution and apply them independently as well.

The long and the short of it is, if you're reasonably current (certainly 11.5.10.2 and newer), it is generally safe to merge a patch and it's prerequisite together. Just pay close attention to the READMEs, and watch out for those AD patches.

– James

Skipping the Oracle Configuration Manager prompts in OPatch

One of the more interesting things Oracle has done in recent years is the introduction of Oracle Configuration Manager. Oracle Configuration Manager is a support tool that provides a way to bundle information about your environment in order to help resolve SRs.

However, many customers are concerned with the information that this tool might reveal, so they tend to avoid providing the contact information that OCM requires. As a result, when you run OPatch, you're constantly bugged for that information. Here's how you can get around that:

Generate a response file (either provide the information, or leave things blank):

${ORACLE_HOME}/OPatch/ocm/bin/emocmrsp -output ${HOME}/ocm.rsp

OCM Installation Response Generator 10.3.1.2.0 - Production

Copyright (c) 2005, 2009, Oracle and/or its affiliates. All rights reserved.

Provide your email address to be informed of security issues, install and

initiate Oracle Configuration Manager. Easier for you if you use your My

Oracle Support Email address/User Name.

Visit http://www.oracle.com/support/policies.html for details.

Email address/User Name:

You have not provided an email address for notification of security issues.

Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: Y

The OCM configuration response file (/home/oracle/ocm.rsp) was successfully created.

Now, when you apply the patch, point opatch to that OCM response file:

opatch -ocmrf ${HOME}/ocm.rsp

– James

Spreadsheet Risk (and why ad-hoc reporting tools make me twitchy)

First, let me say that I'm a DBA, not an accountant. We tend to trust databases to hold and organize data. We use applications and reports developed by professional developers to retrieve that data. Those applications and reports go through a software development lifecycle for a reason: to make sure that they are accurate.

Despite this, many professional developers aren't writing "well-tuned code". They're generally happy to get the right results and, as long as it isn't painfully slow, performance is either an afterthought or the DBA's problem. I've got news for you... some 80% of performance issues are caused by poorly tuned code!  

This is not to denigrate developers.  I'm saying this mostly to prove a point:  if you can't reliably expect well-tuned code from a professional developer, you're insane if you expect anything better from end-users with an ad-hoc query tool.

This is one reason why tools that allow end-users to produce their own reports (Discoverer, ADI, et. al.) have always made me (and, I'm sure other DBAs out there) somewhat nervous.

The other reason I've always been a little twitchy about those tools is accuracy. With professional developers, they understand the need for testing and accuracy. End-users, however, frequently don't have that same appreciation. So, when you allow end-users to develop their own queries and reports, or to extract and manipulate data in a spreadsheet, what kind of risks are you taking?

CIO Magazine has a thought-provoking article on this. Definitely worth a read.

http://www.cio.com/article/print/699592

– James

Adventures in SQL*Net encryption (E-Business Suite edition)

This is a strange problem that we encountered at one of my clients recently. Basically, they're turning on the SQL*Net encryption features of Oracle Advanced Networking Option in their R12.1.3/11.2.0.3 environments. To do this, we're following note 376700.1 "Enabling SSL in Oracle E-Business Suite Release 12". In particular, Section 10 of that document.

Everything was working well. E-Business Suite was able to connect just fine, even our developers were connecting through TOAD. The only significant problem we encountered was with the Windows-based Workflow Builder 2.6.3.5 client. Here, despite a number of attempts at playing with the sqlnet.ora settings, we were consistantly getting the "ORA-12599 TNS:cryptographic checksum mismatch" error.

The mandate from our internal security team was that all outside traffic had to be encrypted. Since we have an external "DMZ" appsTier node, that meant that we had to turn on this SQL*Net encryption. For ALL of our environments.

Naturally, we opened a SR. As time drug on, we started talking about other options. Do we create one non-production environment entirely behind the firewall for the developers to use that doesn't have the encryption turned on? What about making them ssh to the box directly and use WFLOAD?

Finally, after a couple of months of back-and-forth with the support analysts, plenty of "level 16" traces, and a couple of debates, support was able to supply a solution.

With ANO, you specify params like this in the sqlnet.ora file:

#

# Server Settings

#

SQLNET.ENCRYPTION_TYPES_SERVER=(AES256,AES192,3DES112,3DES168)

SQLNET.ENCRYPTION_SERVER=REQUIRED

SQLNET.CRYPTO_CHECKSUM_SERVER=REQUIRED

SQLNET.CRYPTO_CHECKSUM_TYPES_SERVER=MD5

In the case of the "ENCRYPTION_TYPES_SERVER", this is a list of encryption algorithms that the server will accept. The server will negotiate the strongest encryption algorithm with the client.

On the client, you would have similar parameters in the sqlnet.ora file (well, you don't really need them because in many cases things will auto-negotiate based on defaults):

#

# Client Settings

#

SQLNET.ENCRYPTION_TYPES_CLIENT=(AES256,AES192,3DES112,3DES168)

SQLNET.ENCRYPTION_CLIENT=REQUIRED

SQLNET.CRYPTO_CHECKSUM_CLIENT=REQUIRED

SQLNET.CRYPTO_CHECKSUM_TYPES_CLIENT=MD5

Again, the ENCRYPTION_TYPES_CLIENT, this is a list of the algorithms that the client "can speak". Now, in the case of Workflow Builder 2.6.3.5 (which installs with a 10.1.0.2 client), it appears that the client is negotiating "in bad faith". Basically, it is allowing the negotiation to take place "normally". Based on that, the server will choose the strongest encryption from the ones common to both client and server. Unfortunately, the even though the client *says* it can handle AES256 (and others), it actually can't. So, for the Workflow Builder client, you are limited to the much older RC4_* encryption algorithms.

So, for Workflow Builder 2.6.3.5, here is what we have found to work for the client-side sqlnet.ora:

#

# Client Settings

#

SQLNET.ENCRYPTION_TYPES_CLIENT=rc4_256

SQLNET.ENCRYPTION_CLIENT=REQUIRED

SQLNET.CRYPTO_CHECKSUM_CLIENT=REQUIRED

SQLNET.CRYPTO_CHECKSUM_TYPES_CLIENT=MD5

You also need to make sure that the server-side sqlnet.ora contains the following line (or, at least contains the RC4_* encryption type you're intending to use):

SQLNET.ENCRYPTION_TYPES_SERVER=(AES256,AES192,3DES112,3DES168,RC4_256,RC4_128,RC4_56,RC4_40)

NOTE: In my testing, these algorithms work RC4_40, RC4_56, RC4_128, RC4_256 (we're using RC4_256 because it is the strongest of the RC4_* choices). The various RC_* encryption types are NOT mentioned in the sample sqlnet.ora provided by 376700.1 and, given their age, I would NOT recommend using them in a production environment.



– James

Certification on Oracle (RedHat) Enterprise Linux 6?

One of the questions that getting with increasing frequency is "Can we upgrade our systems to Oracle (or RedHat) Enterprise Linux 6".

As always, my first response is to jump on My Oracle Support, navigate to the "Certifications" tab, and attempt to query up the certified platforms for the Oracle Server (RDBMS). In my experience, the Oracle Server is usually among the first products certified on any (major) platform.

So, I go to the Certify page on MOS, and it appears that Oracle (or RedHat) Enterprise Linux 5 is the most recent certified with the current version of the database (11.2.0.3).

Now, I'm thinking to myself, "Hasn't it (Linux 6) been out for some time?". Turns out, it has. RedHat Enterprise Linux 6 was initially released in November, 2010.

So, a with a little effort ("the Google knows all"), I was able to find a few tidbits:

First, from a RedHat Blog Post back in August, 2011:

We’re pleased to announce that on Tuesday, August 9, we formally submitted to Oracle full certification test results of the Oracle 11gR2 database (Single Instance and RAC (including ASM) for x86 and x86-64) on Red Hat Enterprise Linux 6. Oracle database certification is a self-certification program whereby operating system vendors perform extensive testing and submit the results to Oracle for audit and approval.

The certification process we conducted with Oracle 11gR2 and Red Hat Enterprise Linux 6 is the same process we have successfully completed a number of times with earlier Red Hat Enterprise Linux releases. With those releases, Oracle’s certification approval process took about six weeks from the day we submitted test results to Oracle to the day that Oracle posted the certification on their MetaLink support site (https://support.oracle.com). Based on this experience, we would expect certification of the 11gR2 database on Red Hat Enterprise Linux 6 to occur sometime in CYQ4 of this year. We look forward to Oracle’s response and to working with them to complete this certification process. In the interim, customers may also contact Oracle directly for updates on the certification status at gcp-customerfeedback_us@oracle.com.

In addition to the certification testing described above, we perform ongoing and extensive testing on Oracle 11gR2 at every minor release of Red Hat Enterprise Linux. Consequently, we confidently recommend the deployment of Oracle 11gR2 in Red Hat Enterprise Linux 6 production environments today.

And also, from a note on My Oracle Support:

Database Client or Database Server Install on Red Hat Enterprise Linux 6 (RHEL6) or Oracle Linux 6 [ID 1350000.1]

At the time of the last update of this article (30-Jan-2012), Red Hat Enterprise Linux 6 (RHEL6) and Oracle Linux 6 are not certified or supported for use with any Oracle Database version. Be sure to use only certified/supported combinations of Database version and OS version, which you can find under the Certifications tab of My Oracle Support (MOS). Please note that although Certify used to have a status of "Planned" for Red Hat Enterprise Linux 6 (RHEL6) / Oracle Linux 6, this does not imply any guarantee of certification. Also, please do not open a Service Request asking when the OS will be certified. Any additional information will be added to Certify when it becomes available, and Support is unable to provide any more information than what is there. The Certify information on MOS is the only official source for Oracle certification.

So there you have it. Oracle Server (and, consequently, E-Business Suite) are NOT currently certified to run on either Oracle Enterprise Linux 6.x or on RedHat Enterprise Linux 6.x. Unfortunately, anyone you ask at Oracle will basically tell you that "due to revenue recognition rules, we can't say".

What I can tell you is, despite the "this does not imply any guarantee of certification" message, I think it highly likely that they will certify it (eventually). Especially given Oracle's large Linux installed base (and the fact that Oracle's own Linux distribution has so much in common with RedHat's).

Meanwhile... we wait.

-- James

UPDATE:  I sent a few emails and heard back from some people at Oracle.  From the sound of it, a certification announcement for Oracle Enterprise Linux 6 is being "planned" but, naturally, they can't provide any date information.  But, when you combine this with the information above, it leads me to believe that some certification might be coming sooner rather than later.  Let's keep our fingers crossed!

UPDATE #2:  Oracle has now (22 March, 2012) certified 11.2.0.3 against Oracle Enterprise Linux 6 (Using the Unbreakable Enterprise Kernel V1) for x86-64 platforms. (Press Release).  They have also announced that certification against RHEL 6 will be complete within 90 days.

UPDATE #3 (6/27/2012):  Oracle has just announced certification for Oracle Enterprise Linux 6.0 (x86-32), Red Hat Enterprise Linux 6.0 (x86-32 and x86-64), and  Novell SUSE Linux Enterprise Server (SLES) version 11 (64-bit).  See Steven Chan's blog for more details:  https://blogs.oracle.com/stevenChan/entry/oracle_e_business_suite_release3