Oracle Support Changes for EBS (part deux)

On Monday, Cliff Godwin announced some significant changes to E-Business Suite support. The official announcement is now available on MOS (Note: 1495337.1). The text of the announcement is below:

As part of Oracle’s continued commitment to our customers, we will be providing an exception for the first 13 months of Sustaining Support on Oracle E-Business Suite Release 11.5.10 (11i10), valid from December 1, 2013 – December 31, 2014. This exception support will be comprised of three components: (1) new fixes for Severity 1 production issues, (2) United States Form 1099 2013 year-end updates, and (3) payroll regulatory updates for the United States, Canada, United Kingdom, and Australia for fiscal years ending in 2014.

In addition, the Extended Support period for E-Business Suite Release 12.1 has been extended through December, 2018. Customers with an active Oracle Premier Support for Software contract will automatically be entitled to Extended Support deliverables for E-Business Suite 12.1.

NOTE: The changes to 11i Sustaining Support do NOT extend to the security patches. Oracle will NOT provide Quarterly CPU patch for 11i will come out in October, 2013. The one exception to this is that they will provide fixes for "P1" security issues.

– James

E-Business Suite Current Release and Roadmap

Current release is R12.1.3 (but you already knew that!). Enhancements coming with R12.2 include Online Patching (20 years in the making) based on a database feature specifically requested by the E-Business Suite group called "Edition-Based Redefinition". The expectation is that you will be able to apply patches of any size with minimal downtime (less than an hour, target is more like 15 minutes).  (I will discuss this feature in a more in-depth posting later).

Other enhancements include integration with Endeca to provide an e-commerce shopping experience to many of your ERP business processes. (I've heard the "shopping for shoes" reference three times already at OpenWorld when discussing this feature). Cliff Godwin provided an interesting demo of this feature during his presentation earlier today. It is expected that this capability will be released "within the year". Cliff made a point to indicate that initial releases will cover several modules (I believe it was eight at initial release) but that new modules and Endeca-related features should come along quickly (due to the rapid development cycles available with Endeca). He was unclear, however, as to whether the Endeca feature set will be available on releases prior to R12.2.

– James

Oracle Support Changes Announced for EBS

During his session earlier today, Cliff Godwin announced some significant changes to the E-Business Suite support deadlines. I will put together a more polished post later, but for now, here are the details:

EBS 11i (11.5.10.2) is STILL on Sustaining Support until December 31, 2013. However, they are making a few exceptions to the Sustaining Support policy. First, they will continue to provide bugfixes for Severity 1 production problems until December 31, 2014. Secondly, they will continue to provide updates for US Form 1099 through 2013 Year End. And finally, they are committing to provide Fiscal Year End 2014 Payroll updates for US, Canada, and UK.

For customers on Release 12.1, Oracle is waiving the Extended Support fees until December of 2018.

-- James

MINOR CORRECTION: EBS 11i is not "still on sustaining support". EBS 11i is actually on Extended Support until Nov 2013. (Thanks, Srini, for pointing this out!)

Impact of EBS R12.2 changes – WebLogic

A number of very smart people seem to think that the introduction of WebLogic is going to be a major "game changer" for E-Business Suite DBAs. There is certainly some truth to that thought. WebLogic is a completely different animal than the Apache or Oracle Application Server that we've gotten accustomed to in E-Business Suite 11i and R12.

But consider this. I suspect that most environments are fairly vanilla. When it comes to the basic degree of interaction that an E-Business Suite DBA has with the Applications Server, I suspect that the change will be largely transparent. Consider what you do with the current Applications Server product. Start and stop? That's done through AutoConfig managed scripts. Basic configuration for EBS? That's handled through AutoConfig. Why would we expect these things to change just because they've changed out the tool?

What will it change?

I'm certain that the log files will be different. The naming convention will probably change and the location of the files might change. The content of the files will certainly be different. As will the configuration files (but, again, I would expect those to be AutoConfig managed).

The big place where you will notice the WebLogic change (from an Administrator standpoint) is if you're trying to do something else with it. If you're dealing with Discoverer or deploying APEX, for example, you'll certainly get up-close-and-personal with WebLogic.

So, should you be afraid of the WebLogic change? I don't think so. Yes, custom deployments (anything that actually requires you to "deep-dive" into the configuration of WebLogic) will have a learning curve. That said, many of the components of Oracle Fusion Middleware already use WebLogic and, if you're doing any custom deployments, odds are you're also already dealing with WebLogic elsewhere and will have some familiarity by the time you go to tie it into E-Business Suite.

When you factor in these things with the fact that WebLogic is a more mature and overall better product than the current applications server used in E-Business Suite, I think that the change is probably a good thing.

– James

Speculation about EBS Release 12.2

Floyd Teeter raises some good points in his blog posting out today. You can read the full posting here (http://orclville.blogspot.com/2012/08/i-wonder-what-tomorrow-has-in-mind-for.html). Primarily, that we won't actually have EBS R12.2 released during OpenWorld (as many of us were hoping).

Don't get me wrong, there will certainly be some good EBS R12.2 information out there. Oracle put together a very informative presentation and discussion surrounding the patching changes at Collaborate in Las Vegas this past spring. You can almost certainly bet that there will be an updated version at OpenWorld.

But, considering that the closed Beta program is still underway and the Early Adopter program hasn't started, any path of deduction should tell you that you're not going to have a R12.2 "GA" announcement for a few months at best. Which most means that sometime around Collaborate 2013 (Denver) would be a decent guess.

– James

Listing Installed Packages on Linux

NOTE: First, let me mention that, unless otherwise indicated, when I blog about Linux it will be about the RPM-based distributions that are certified with the Oracle Database (RedHat Enterprise Linux, Oracle Enterprise Linux).

Normally, when you're looking to see which packages are installed on Linux (RedHat, Oracle, CentOS), you would use this command:

rpm -qa

Unfortunately, the standard output of that command omits alot of useful information. It may or may not indicate if you have the 32 or 64 bit version of a package installed, for example.

So, for a command that will show you which packages are installed in a format that looks like the name of the RPM file:

rpm -qa --queryformat \

"%{NAME}-%{VERSION}-%{RELEASE}.%{ARCH}.rpm\n" |\

sort > pkglist_`date +%Y%m%d-%H%M`.txt

– James

Useful desktop tools for the EBS Consultant (and DBAs)

This is a short list of some of the tools that I (and others) depend on when we're working at clients.

Snag-IT! Commercial ($49.95). (Windows & Mac) http://www.techsmith.com/snagit.html Excellent tool if you have to document graphically intensive tasks. Provides nice ways to highlight and circle captured images, also allows you to quickly capture just the active window (or a pre-defined region of the screen), dump multiple captures into a directory, and even sequentially name the files.

Toad Commercial ($955.00 and up). (Windows only) http://www.quest.com/toad/ For many developers and implementation consultants, Toad has a virtually religious following. It is an excellent SQL query/development tool and allows you to easilly save query results into Microsoft Excel format (which will keep Oracle Support plenty happy).  [Full disclosure:  Those of you that have worked with me in the past know that I don't use Toad (or TOra, or SQL Developer, for that matter) for a variety of reasons (which will be explained in a later posting).]

TOra Open Source (Free). (Windows, Mac, Linux, Solaris) http://torasql.com/about Has many of the same core features as Toad.

Oracle SQL Developer Commercial (Free-ish). (Windows, Linux, others). http://www.oracle.com/sqldeveloper Also has many of the same core features as Toad.

VirtualBox Open Source (Free). (Windows, Mac, Linux, Solaris) http://www.virtualbox.org Do you want to run a Virtual Macine on your laptop? Maybe you're running Windows 7 and need a copy of Windows XP and an old version of IE so that you can test things and support your users? What if you'd really rather be using Linux but still have to run some Windows software? VirtualBox provides an easy way to create and run a VM of another operating system. Literally run Linux in a Window! (Or even fullscreen).

SecureCRT Commercial ($139.99) (Windows, Mac, Linux) http://www.vandyke.com/products/securecrt/index.html This is a very nice ssh client. It works very well and has a very nice user interface. Unfortunately, it's fairly expensive. Other (free) choices (on Windows) include: PuTTY http://www.putty.org/, BitVise Tunnelier http://www.bitvise.com/tunnelier, OpenSSH for Windows http://sshwindows.sourceforge.net/, and Cygwin http://www.cygwin.com/. If you're on a Mac or Linux system (or running a copy of Linux in a Virtual Machine), then you already have a built-in ssh client.

– James

URGENT BULLETIN: Disable JRE Auto-Update for All E-Business Suite End-Users

This notice just came out on Steven Chan's blog. If you're not following it, you definitely should.

The issue at hand is not that Auto-Update in itself causes problems. The real issue is that Oracle is going to push (through Auto-Update) the JRE 1.7 update. The JRE 1.7 update is NOT certified with E-Business Suite (any versions) at this time.

So, in order to keep your user's desktops on JRE 1.6, you MUST turn this auto-update feature off!

The update for JRE 1.7 could be pushed as early as July 3, 2012. The update WILL definitely be pushed after September 7, 2012 (after the release of JRE 1.6.0_35).

Steven Chan's blog has more information (including instructions on what you can do to undo the JRE 1.7 update if you get hit by it). The full link to the posting on Steven Chan's blog can be found here:

https://blogs.oracle.com/stevenChan/entry/bulletin_disable_jre_auto_update

– James

OAUG Connection Point 2012 in Austin, TX (July 11-12, 2012)

OAUG Connection Point 2012 is coming to Austin, TX, July 11-12, 2012! The conference will be held at the Omni Austin Hotel in beautiful downtown Austin, TX.

The keynote will be given by Jeanne Lowell, Vice President, Oracle E-Business Suite Strategy, Oracle Corporation. (Hopefully we can extract some R12.2 news from her!)

Other featured speakers include:

Elke Phelps, Senior Principal Product Manager, Oracle E-Business Suite Applications Technology, Oracle Corporation. (Elke always gives interesting and informative presentations!)

David Bowin III, Oracle Fusion Applications Product Development, Oracle Corporation

Amrita Mehok, Senior Director, Product Strategy, Oracle Corporation

Some other names you may recgonize will be presenting as well:

Alyssa Johnson, ROLTA (Sessions # 10913 and 10933)

Anne Carlson, Oracle (Session # 10868)

Art Dowd, O2Works (Session # 10829)

Barbara Matthews (Session # 10925)

Bill Dunham, OATC, Inc. (Session # 10915)

John Stouffer (Session # 10880 and 10924)

Michael Barone, OATC, Inc (Session # 10905 and 10907)

Mike Swing, TruTek (Session # 10878)

Susan Behn, Infosemantics, Inc. (Session # 10821)

...and many more!

Oh, and I'll be there too! I'm giving my "Anatomy of an Upgrade to 12.1.3 (including Platform Migration)" presentation (Session # 10802). I'll also be participating in John Stouffer's "R12.1 Upgrade Panel" (Session # 10924) along with Bill Dunham, Mike Swing, and Alyssa Johnson. The panel discussions are always entertaining and provide an excellent way for the community to discuss their challenges and experiences.

"Early Bird" Registration is open through June 24^th. Particularly if you're in or near the Austin, TX area, this can be a very cost-effective way to network with the community and find solutions to your problems. More information on the conference is available through the OAUG website or this direct link:

http://connectionpoint.oaug.org/2012/austin

I hope to see you there!

– James

On DBAs, Developers, and Performance

Cary Millsap has an excellent (as usual) blog posting today about the software development process employed by so many oranizations.

You can read his full posting here:

http://carymillsap.blogspot.com/2012/06/organizational-constraint-that.html

This plays into one thing that I see quite a bit as a DBA at client sites. Most developers that I encounter at client sites don't tend to focus so much on performance (unless it is painful during their testing). This isn't, specifically, their fault. In many cases, the developers are under significant pressure to "make it work" and move onto the next thing. As a result, they don't really have the time to worry about performance. Besides, that's the DBA's job, isn't it?

Well, actually, it isn't. See, here's the thing, from a DBA perspective, we have a relatively small handful of tools at our disposal. From the bottom-up, here is a basic list of things that a DBA generally does to impact the performance of a system:

Properly configure disk and I/O. This is (or should be) really a collaboration between the DBA, the Systems Administrators, and the Storage Administrators. Done correctly, this shouldn't really be a problem. However, as with everything, it is still possible to screw it up. Make sure that you have redundant I/O paths that have sufficient bandwidth for your system. Spread your I/O across many physical drives. With RAID technologies (particularly RAID 1/0) this is VERY easy to accomplish. Forget about the old concept of the DBA moving datafiles around to avoid a "hot disk". The disk array can do this at a much more granular and consistent level than any human possibly can. Your primary goal here should be to have as many spindles involved in every I/O operation as possible.

Properly size your hardware. Another collaboration between the DBA and Systems Administrator. Make sure you have adequate horsepower and RAM. And ALWAYS plan for growth! My general recommendation is always to put more RAM into the box than you think you'll need. Given that so many systems that I encounter these days are x86-based Linux systems (rather than "big iron" like Solaris or AIX), memory for these systems is a relatively small portion of their cost. Also, RAM doesn't impact your Oracle licensing!

Properly tune your OS Kernel and Database parameters. I think that this is one area where developers, managers, and users tend to have gross misconceptions. While it's true that tuning a system to truly "optimal" performance is a dark art, the truth is that, unless you've really screwed something up (sized your SGA too small, too few buffers, etc.), odds are you're not going to see huge performance gains by tweaking these parameters. In most cases, "decent performance" is fairly easy to achieve. Now, squeezing out that extra 20%? That can require in-depth analysis of statistics, utilization, I/O patterns, etc. This is where the "dark art" comes into play. And, honestly, this requires time to observe and adjust.

Unfortunately, too many developers, managers, and even users, seem to wonder why that idiot DBA didn't just set the magic "MAKE_SQL_RUN_FASTER=TRUE" parameter. (Shh! Don't tell anyone, that's OUR little secret!)

The truth is, unless something is wrong at these lower levels, the biggest performance gains are going to come from tuning the code. And, in my opinion, since it's the developer's job to produce a quality product, it's ultimately the developer's job to do tune their code.  Unfortunately, as Cary points out, too many organizations are structured in a manner that breaks the feedback loop required for developers to do this properly.

That MUST be fixed.

– James

EBS R12.2 Online Patching Webcast

For those of you who are (like me) anxiously awaiting the release of R12.2, the Applications Technology Group (ATG) is hosting a live webcast to preview the Online Patching feature in R12.2.

Kevin Hudson, one of the lead architects of this particular feature, is the presenter for this webcast.  I had the pleasure of attending Kevin's session at Collaborate 2012 where he discussed this feature in-depth.  He is an excellent presenter and is very well-versed on this topic.  I'm sure this webcast will be worth your time.

This two hour webcast will take place on June 14 @ 8:00am (Pacific Standard Time).

Full details are on Steven Chan's blog at the link below.

ATG Live Webcast June 14: Technical Preview of EBS 12.2 Online Patching

-- James

E-Business Suite and the 32-bit vs. 64-bit question

Before I get flamed on this, I want to make clear that, for the purpose of this posting, I'm speaking specifically about operating systems (not hardware). Most of the hardware being sold today is already 64-bit, however, you can run most 32-bit operating systems on 64-bit hardware. It's that distinction that I'm discussing here.

The first thing that you need to know here is that the big benefit of using a 64-bit operating system really is memory. In particular, it is not about the total amount of memory that can be installed in the machine (that tends to be hardware), but, about the addressable size of "per process" memory.

In the case of components such as those used on an appsTier in EBS, per-process addressable memory doesn't matter so much, as each process has it's own private memory (and isn't depending on "shared memory" like the database server is). So, aside from the fact that it makes our life much easier from an administrative standpoint (and the industry is going that way), there really isn't much of technical advantage to a 64-bit appsTier.

For EBS 11i (where the DB is certified on x86-64, but the appsTier is only certified on x86-32), you can still use much more than 4GB on an appsTier node (the operating system has a way of addressing large memory). It's just that the amount of memory that can be addressed by a single process is limited to something between 3 and 4 GB.

In the case of EBS R12, the appsTier binaries are still 32-bit, even when you're running on a 64-bit operating system. This makes sense because the only component that can really take advantage of it is the database (because the database processes all attach to the same large chunk of memory [the SGA]).

Note that EBS R12 appsTier is certified on both Linux x86-32 and Linux x86-64.

So, for 11i, the best that they can hope for is to have a separate dbTier (database only) running on Linux x86-64 and use Linux x86-32 for their appsTier nodes. Remember, that the 11i appsTier is NOT certified on Linux x86-64. That doesn't mean that it can't be done, but I seriously doubt that Oracle has any intention to certify a release that old on, what is effectively, a different platform. In both cases, they can/should be 5.X (5.7 is current). Having, effectively, two different platforms will be something of a headache from a Linux administration standpoint, but it's something that they'll have to deal with.

When they get to R12, they should use Linux x86-64 on all tiers (to simplify administrative tasks, as well as being "among the mainstream" of installations). Keep in mind that 64-bit is where "the market" is going. Even though you can (It is certified) do R12 on x86-32, you're better positioned if you're on x86-64.

– James

Collaborate 2012: On Upgrading to Fusion Applications

So I attended the OAUG E-Business Suite SIG session today at Collaborate 2012 in Las Vegas, NV. One of the guest speakers was Cliff Godwin, Oracle's Sr. VP of Applications Development. During this session, he was answering a lot of questions about E-Business Suite to Fusion Applications upgrade.

One of the more interesting things that he indicated was that upgrading from E-Business Suite R12.1 to Fusion Applications will be more along the lines of what you do during an implementation from a Legacy system into E-Business Suite (than a traditional "upgrade").

What this means is that you will have to ETL data from E-Business Suite into a fresh Fusion Applications install. So, for all intents and purposes, you will be performing many of the same tasks that you performed when you first IMPLEMENTED E-Business Suite. Or, if you were going from E-Business Suite to SAP (for example).

If you think about it, this makes plenty of sense. Fusion Applications is NOT an upgrade of E-Business Suite. It is an entirely different product. Perhaps part of the problem that I've been having with the whole "E-Business Suite to Fusion Applications" discussion has been semantics? Maybe they (Oracle) should stop referring to the process as an "upgrade" and instead a "migration"?

Unlike moving from a legacy system, Oracle is developing templates to aid the ETL process so that you can convert your legacy data. This is a big difference. When you're converting data from a legacy system into E-Business Suite (or any other application), a significant part of the effort revolves around the need to "roll your own" data conversion scripts. In this case, Oracle will be providing this logic for you.

One of the selling points about moving to Fusion Applications has always been that you could migrate one module at a time. That would be particularly difficult if the process was similar to an E-Business Suite upgrade. But it makes perfect sense if you're doing a migration more akin to moving off a legacy system.

According to Cliff, initial focus is on the HCM modules and initial release of the upgrade templates is currently planned for some time "later this year".

– James

Vegas, Baby!

On Saturday night, I head to Las Vegas for Collaborate 2012 (http://collaborate.oaug.org). For the Oracle E-Business Suite community, this is one of the biggest events of the year.

For the uninitiated, Collaborate is the annual conference hosted by the Oracle Applications User's Group (http://www.oaug.org). Unlike Oracle OpenWorld, which is largely an Oracle Corporation marketing event, Collaborate is a community-oriented conference. While there are Oracle Corporation presentations focusing on the latest and greatest products and features, the vast majority of the papers and presentations at Collaborate are from actual users of the Oracle products.

Many of these are actual war stories. Walk-throughs of actual implementations and upgrades. Solutions that that others in the E-Business Suite community have put together for their sites to solve real-world problems that they've encountered. Very often, these are things that are not covered (or are covered poorly) in Oracle-provided documentation.

And it's not just E-Business Suite DBA topics, either! There are plenty of presentations on module-specific issues given by others who have already "been there, done that".

In addition to the standard white-paper presentations, there are also panel discussions and Special Interest Groups (SIGs). The panel discussions and SIGs are usually very lively Q&A sessions with recognized industry experts on a wide range of topics.

Planning an upgrade? There's the two-part OAUG Upgrade SIG (Sessions #10522 and #10722)

Having troubles with cloning E-Business Suite? There's an E-Business Suite Cloning Panel. (Session #9360)

Thinking of moving to Linux? There's a panel for that too! (Session #9361)

I will be giving my presentation, "#9554 - Anatomy of an Upgrade to 12.1.3 (Including Platform Migration)" on Wednesday, April 25^th from 11:00am to 12:00pm i(South Seas E) and I will also be on the Migrations to Linux Panel (#9361), also Wednesday, from 1:00pm to 2:00pm (South Seas F).

And, while you're at it, stop by the RedRiver Solutions booth (Booth #1226). I will be hanging out there and helping to pass out updated versions of John Stouffer's books Oracle E-Business Suite Patching 101 and Oracle E-Business Suite Apps DBA 101.

I hope to see you there!

– James

E-Business Suite R12.1.1 is Certified on Oracle Linux 6!

Back in February, I blogged about the pending certification of Oracle E-Business Suite on Oracle Enterprise Linux 6 and RedHat Enterprise Linux 6. In that blog post, I noted that the certification announcement was "planned" but, of course, Oracle doesn't provide dates.

Well, guess what? The waiting is finally over. As these things go, the announcements come out in parts.

First, on March 22, 2012, Oracle announced that Oracle Database 11gR2 and Fusion Middleware 11gR1 were certified. (The press release can be found here.)

And today (April 4, 2012), through the Oracle E-Business Suite Technology blog (known to many of us as "Steven Chan's blog"), we have the E-Business Suite announcement (available here)!

While this is fantastic news, read the announcements carefully!

These certifications are ONLY for Oracle Enterprise Linux 6 on the x86-64 with the Unbreakable Enterprise Kernel (UEK) version 1.

This means that if you're on the x86-32 platform, or if you're on RedHat Enterprise Linux 6, you'll have to wait a bit longer. The sames is true for customers on Oracle Enterprise Linux 6 who have NOT upgraded to the Unbreakable Enterprise Kernel (UEK) version 1 at ALL, or have already upgraded to the Unbreakable Enterprise Kernel (UEK) version 2 which was released on March 13, 2012 (press release here).

According to the database announcement, certification on RedHat Enterprise Linux 6 (and Oracle Enterprise Linux 6 [without UEK]) should be available within 90 days. I would expect the E-Business Suite R12 announcement to follow shortly behind.

What about other E-Business Suite releases? At this point, I have no actual information. But, I can speculate (with a good degree of certainty) that you won't see any 11.5.10.2 certifications against OEL/RHEL 6. E-Business Suite 11.5.10.2 is currently in Extended Support. Even though the support fees have been waived (through the end of Extended Support, November 30, 2012), Oracle doesn't typically certify new platforms once a product goes into Extended Support. (A more detailed discussion of Oracle's recent support announcements can be found here.)

The other question mark out there is OEL/RHEL 6.0 on x86-32. Personally, if you're implementing R12 or upgrading to R12 on Linux, you should be using an x86-64 distribution on x86-64 hardware. However, certification on x86-32 is also forthcoming.

As always, be sure to read/follow the relevant notes through the Certify Tab on My Oracle Support before you start any project to make sure that the combination of components you intend to use are, in fact, certified. These certifications will also detail the various always steps, operating system parameters, packages, and even patches specific to your combination that you will need to follow.

All of this is excellent news, as the OEL and RHEL 5.x line is getting pretty long in the tooth and is approaching it's end of life.

Now... when will we get that R12.2 announcement? Collaborate, maybe? OpenWorld? … the waiting continues.

– James

UPDATE 6/27/2012:  Oracle has just announced certification for Oracle Enterprise Linux 6.0 (x86-32), Red Hat Enterprise Linux 6.0 (x86-32 and x86-64), and  Novell SUSE Linux Enterprise Server (SLES) version 11 (64-bit).  See Steven Chan's blog for more details:  https://blogs.oracle.com/stevenChan/entry/oracle_e_business_suite_release3

Stupid Unix Tricks... Part Two (Remote Command Execution using SSH)

So, let's say that you wanted to have a script on your dbTier that will reach out to your appsTier and shut down the applications. Maybe this is your system-level shutdown script so that when the Unix administrator shuts down the dbTier, everything is shut down nice and neat like...

For the purpose of this exercise, we're going to need to assume that the APPS password is known to the script (how you do that might be the subject of another blog posting). We're also going to assume that the Unix environment is set automatically (and without prompting) on the remote system.

So, how do you do it?

Well, first you have to set up ssh pre-shared keys. This will allow you to login without being asked for a password. (See my earlier posting: Password-less Login Using SSH Pre-Shared Keys)

Once that is configured, you can use a command like this:

ssh applmgr@myappstier.mydomain "cd ${ADMIN_SCRIPTS_HOME};./adstpall.sh apps/${APPSPW}" 2>&1 |tee -a ${LOG}

A few things here. First, you'll notice that I'm actually executing TWO commands remotely. The "cd" to change directories and then the adstpall.sh script (the semicolon allows me to do that in Unix). Secondly, there are environment variables. Here's the thing about those environment variables. In the command above, they are NOT evaluated on the target system. They are evaluated locally on the SOURCE system. If you want to use variables that are local to the target, you're going to have to "escape" them.

For example, this one will use a variable evaluated on the source machine:

ssh applmgr@myappstier.mydomain "echo ${CONTEXT_NAME}"

And this one will use a variable evaluated on the target machine:

ssh applmgr@myappstier.mydomain 'echo ${CONTEXT_NAME}'

Similarly, you can evaluate a variable on the target by “escaping” it:

ssh applmgr@myappstier.mydomain "echo \${CONTEXT_NAME}"

At one client, their standard is to use a script that wraps around the standard "oraenv" to set their environment variables. As a result, every time they log in, they are greeted with a prompt asking them to choose their environment.

This raised an interesting problem for some of the automated processes we were trying to deploy. The automation was driven from a remote box and would need to ssh over to a target box and issue commands. So, how do we configure the environment so that a user logging in interactively is prompted and one issuing a command remotely through ssh isn't? Well, it turns out that, on Linux at least, that remote command doesn't get assigned a TTY. So, we've made a change to the .bash_profile on the target node that looks something like this:

if tty | fgrep pts ; then

#

# Normal, interactive logins

#

export ORAENV_ASK=YES

else

#

# Human-less logins (ssh "command")

# (Suppress output and bypass prompting for oracle environment)

#

export ORAENV_ASK=NO

fi

Now, let's assume you want to be a little more elaborate. You want to clean up extraneous output and capture the results of the command in your logfile (represented by the environment variable ${LOG}):

ssh applmgr@myappstier.mydomain ". ./.bash_profile 2>&1 1>/dev/null;cd ${ADMIN_SCRIPTS_HOME};./adstpall.sh apps/${APPSPW}" 2>&1 |tee -a ${LOG}

Or, maybe you'd like to do something in SQL*Plus on a remote system?

ssh applmgr@myappstier.mydomain “. ./.bash_profile 2>&1 1>/dev/null;sqlplus apps/${APPSPW}” <&1 1>>${LOG}

select sysdate from dual;

EOF

This will redirect stderr to stdout, and send both to your logfile (${LOG}). Pay close attention to the line containing the EOF. It has to be the ONLY thing on the line (not even a trailing space!)

– James

OAUG Survey (part two)

As I'm reading through the OAUG survey (available here) one thing really strikes me.  If you look at Figure 30, only 24% of respondents are on E-Business Suite R12.1.  Of those that remain, 74% indicated that they are on a version of E-Business Suite that no longer covered under Premier Support.

Release	Premier	Extended
11.5.10.2	11/30/10	11/30/13
12.0	01/31/12	01/31/15
12.1	05/31/14	05/31/17

Remember that, for Extended Support, you will need to achieve "minimum baseline" code levels too!

More information on Oracle's Support windows in this post.

-- James

What drives E-Business Suite upgrades?

You'd think it would be new features, or security requirements. But, apparently, it's Oracle's end-of-support deadlines...

http://www.cio.com/article/print/701911

...at least according to 73% of the 327 OAUG members that responded to a survey.

UPDATE:  Here's the link to the full OAUG report:

http://www.oracle.com/us/products/applications/oaug-erp-upgrades-1531330.pdf

– James

Stupid Unix Tricks... Part 1

So, let's say you're trying to figure out if the database (or E-Business Suite) is down. Now, the logical way is use the Unix commands ps and grep to check for a particular process. Generally speaking, we would look for the SMON process for that particular instance.

However, maybe you're looking for something else that has multiple processes and you want to see that they're all shut down.

We're going to use a database as an example (largely because I assume you are familiar with the database). The basic command would be:

ps -ef|grep ora_smon_PROD

oracle 10445 6643 0 15:32 pts/0 00:00:00 grep ora_smon_PROD

oracle 19710 1 0 Feb28 ? 00:00:36 ora_smon_PROD

However, the problem here is that it also gives our grep command. To get around that, we can strip it out using grep -v grep (which would strip from our results anything that contains the string grep). Additionally, maybe we want to get something we can use in an if statement. The simplest way to do that is to count the number of lines returned by the command. That can be done by piping the output through the wc -l command. Our final command will look like this:

ps -ef|grep ora_smon_PROD|grep -v grep |wc -l

So, assuming that we just wanted to look for SMON we can build our if statement like this:

if [ `ps -ef |grep ora_smon_PROD|grep -v grep |wc -l` -gt 0 ]; then

   echo "SMON is UP"

else

   echo "SMON is DOWN"

fi

Now, let's assume that you want to check for PMON as instead:

if [ `ps -ef |grep ora_pmon_PROD|grep -v grep |wc -l` -gt 0 ]; then

   echo "PMON is UP"

else

   echo "PMON is DOWN"

fi

But what if you wanted to make sure that they were BOTH down?

if [ `ps -ef |grep -e ora_pmon_PROD -e ora_smon_PROD|grep -v grep |wc -l` -gt 0 ]; then

   echo "PMON and SMON are UP"

else

   echo "PMON and SMON are DOWN"

fi

The key here is grep -e. Because grep allows you to use the -e flag more than once per invocation, you can specify multiple strings to search for. Multiple -e strings are treated as a logical "or" by grep when it's parsing the input.

As with everything, your results may vary. Different platforms may have different versions of grep with different capabilities. This example was tested on Linux.

– James

Oracle Waives E-Business Suite Extended Support Fees, again...

Yesterday, Steven Chan announced on his blog (if you don't read it, you should...) that Oracle was waiving Extended Support fees for E-Business Suite 11i and 12.0. According to the new announcement, the entire Extended Support period (for 11i and 12.0) is now free (if you've already paid for it, contact your salesperson). I haven't dug deeply enough into the history of this, but I don't think that Oracle has done anything to move the actual dates.

This means that customers who are on 11i and 12.0 (who have also met the minimum baseline patch requirements) have some extra time to get to the next release (which is currently 12.1.3).

What is Premier Support?

Premier Support is the “normal” support category that Oracle puts around “current” software. Under Premier Support, Oracle obligates themselves to actually fixing bugs and finding solutions to problems.

According to Oracle's website:

Premier Support – Delivers full system support for your Oracle hardware, operating systems and applications with an upfront, minimum five-year support commitment that helps you plan and budget.

What is Extended Support?

Extended Support is just like Premier Support, except that they charge you more for it. All of the features of Premier Support are there, and they will still produce new bugfixes when they're needed.

According to Oracle's website:

Extended Support – Offers an additional three years of support for select Oracle software and operating systems for an additional fee so you can effectively manage your upgrade strategy.

What comes after Extended Support?

Sustaining Support. Here, you still have access to the support site and analysts. However, no new bugfixes will be produced. If you encounter a previously unknown problem, your only choice may be to upgrade.

According to Oracle's website:

Sustaining Support – Provides investment protection by further extending support for Oracle software, operating systems and select hardware products. Features include access to online support tools, knowledgebases, pre-existing fixes, and assistance from Oracle's technical support experts.

More information is available at: http://www.oracle.com/us/support/lifetime-support/index.html

What does this mean for me, by version?

For Release 11i, Premier Support ended November 30, 2010 and Extended Supprt will now end on November 30, 2013. You'll want to be on 11.5.10.2 with the minimum baseline patches applied (according to Note: 883202.1). Ideally, you should also be on RDBMS 11.2.0.3.

For Release 12.0, Premier Support ended on January 31, 2012 and Extended Support will now end on January 31, 2015. The document you'll need to follow is 1195034.1. For that you will need to be on 12.0.6 (12.0 RUP 6), with the mainimum baseline patches applied (according to the document). Ideally, you should also be on RDBMS 11.2.0.3.

For Release 12.1, Premier Support will end in June of 2014 (I'm not certain, but I believe that it is May 31, 2014) and Extended Support will end on the same date in 2017. Oracle has also (back in October) announced that they are waiving the first year of Extended Support fees for R12.1 (which means you are good until May of 2015). As far as baseline patching is concerned, keep an eye on 1195034.1. At this point, the only minimum baseline requirement for Extended Support is that you have applied at least the R12.1.3 Release Update Pack. As with the other releases, you should also be on RDBMS 11.2.0.3.

Why are they doing this?

Obviously, I have no real information on what's going on inside of Oracle. What I can say is that, based on customers I (and others) have spoken with, there are still a large number of customers on 11i and 12.0. Release 12.2 has been “coming soon” for quite some time, and, with R12.1.3 dates appearing on the horizon, many customers are waiting for R12.2 to be released. They don't want to finish one upgrade project only to immediately start another.

My advice? It is my understanding that you will not be able to upgrade directly from 11.5.10.2 to R12.2 (I could be wrong on this). If you're on 11i, you should be working on your upgrade to R12.1.3 now. If you're on R12.0, you may want to wait until R12.2 comes out and figure out if you can go straight to R12.2. If you're on R12.1, get to R12.1.3 and be ready to start planning your R12.2 upgrade shortly after it is released.

In all cases, get your databases upgraded to 11.2.0.3 as well. Many of the deadlines for 10gR2 have already passed.

– James

Password-less Login Using SSH Pre-Shared Keys

Way back when I started working with Unix (otherwise known as "the olden days" or "days of yore"), one of the tricks we used was a concept known as "remote login" and the "Berkeley R commands". This was based on a number of things, most of them depending on either the /etc/hosts.equiv or the ${HOME}/.rhosts file to establish the trusting relationship. Configuring these would allow you the ability to do some really neat things. Among them, copying files from one host to another using a command like rcp /tmp/file user@remotehost:/tmp/file without being asked for a password. This made for some really neat scripting opportunities and made it much easier to manage multiple systems.

Unfortunately, the Berkeley "R" commands are notoriously insecure. The way that the trusting was done was based entirely on the username and hostname of the remote user on the remote host. Literally, you told the server to trust "jmorrow@remotehost.mydomain.com". The problem with this is that all that was required was knowledge of the trusting relationship. All you had to do was set up a machine named "remotehost.mydomain.com" and create a "jmorrow" user on it. Then you could go anywhere that that trusting relationship allowed.

Fortunately for us, the cool features that were introduced by the Berkeley "R" commands are implemented much more securely in the SSH protocol and toolset.

The SSH Protocol can use pre-shared keys to establish trusting relationships. In this case, each node has both a public and a private key. When the client talks to the server, the client offers a " key". The server, which maintains a list of trusted "public keys", then compares that key to it's database to determine if it actually trusts the client. If the client passes the test, then it is allowed in without any further challenge. This can be very useful for administrators, automated file transfer, also for scripting interactions between hosts. Note that this is not a "Machine A" trusts "Machine B" relationship. It is "user@machinea" trusts "user@machineb".

For the purposes of this article, the "server" is the node that you are logging into from the "client". So, the "server" is the one that is doing the trusting. The terms "server" and "client" refer only to the role being played by each component in the ssh communications session. I should also mention that Oracle Real Application Clusters (RAC) depends on this relationship as well.

Generate your public/private key pairs [Both Client and Server]

The server (user@host) needs to have one, and each client (user@host) that is being trusted needs to have one.

Execute these two commands (in a Unix/Linux environment) to create both your rsa and your dsa keys. You will be prompted for a location to store the files (typically under ${HOME}/.ssh), and for a passphrase. In all cases, it's ok to accept the defaults.

ssh-keygen -t rsa

ssh-keygen -t dsa

If you know you don't want to use a passphrase, you could generate the keys with these two commands:

ssh-keygen -t rsa -f ${HOME}/.ssh/id_rsa -N ""

ssh-keygen -t dsa -f ${HOME}/.ssh/id_dsa -N ""

Transfer the public key files from the client to the server

I prefer to make sure that I have a uniquely named copy of the public keys (makes it easier to transfer to another box when first establishing the relationship).

cd ${HOME}/.ssh

ls -1 id_[dr]sa.pub |while read LINE

do

cp ${LINE} ${LINE}.`whoami`@`hostname -s`

done

Now copy these files to the server:

scp ${LINE}.`whoami`@`hostname -s` trustinguser@trustingserver:.ssh/.

Copy the public keys you're trusting into the authorized_keys file

Here, we'll need to put those keys into the authorized_keys file. Do this for each of the files that you transferred in the previous step.

cd ${HOME}/.ssh

cat >> authorized_keys

Make sure permissions are correct

If the permissions on these files are too open, the trusting relationship will not work. Here are my recommendations:

chmod 600 ${HOME}/.ssh/auth*

chmod 700 ${HOME}/.ssh

chmod 644 ${HOME}/.ssh/id_[dr]sa.pub*

chmod 600 ${HOME}/.ssh/id_[dr]sa

Now, you should be able to ssh from the client to the server witout being prompted for a password.

– James