Tuesday, February 5, 2013

Deciphering support and licensing issues surrounding Oracle on VMWare


I frequently run into clients that are wanting to run Oracle products in their VMWare cluster. Since I generally deal with E-Business Suite customers, I tend to take the position of "anything that swallows machines whole should probably have a physical machine" approach to production systems. However, I can see some distinct advantages to virtualization, particularly when it comes to managing numbers of non-production environments.

Unfortunately, there is a lot of confusion out there as it relates to Oracle and virtualization... particularly surrounding one of the most popular virtualization solutions, VMWare. I'll try to provide my best understanding of the issues here.

Are Oracle products certified on VMWare?

The short answer is, NO. But, that really shouldn't be that much of a concern. Keep in mind that a VMWare Virtual Machine is, technically, hardware. Oracle doesn't tend to certify against hardware. And that's what that VMWare really is, it's "virtual hardware". As such, it's really no different than a particular model of Dell or HP ProLiant.

What Oracle does do is certify against a platform. A platform is the combination of a particular version of an operating system (Solaris 10 vs. Solaris 11, for example) and processor architecture (Sun SPARC vs. Intel x86-32 or Intel x86-64). In the case of a deployment to VMWare, your platform will be determined by the operating system that you intend to run inside of the virtual machine. (For example, Red Hat Enterprise Linux 4/5/6 for x86 or x86-64).

Are Oracle products supported on VMWare?

Oracle's official support position can be found in MOS Note 249212.1, copied below (emphasis mine):

Support Position for Oracle Products Running on VMWare Virtualized Environments [ID 249212.1]

Purpose
---------
Explain to customers how Oracle supports our products when running on VMware

Scope & Application
----------------------
For Customers running Oracle products on VMware virtualized environments. No limitation on use or distribution.


Support Status for VMware Virtualized Environments
--------------------------------------------------
Oracle has not certified any of its products on VMware virtualized environments. Oracle Support will assist customers running Oracle products on VMware in the following manner: Oracle will only provide support for issues that either are known to occur on the native OS, or can be demonstrated not to be as a result of running on VMware.

If a problem is a known Oracle issue, Oracle support will recommend the appropriate solution on the native OS. If that solution does not work in the VMware virtualized environment, the customer will be referred to VMwar for support. When the customer can demonstrate that the Oracle solution does not work when running on the native OS, Oracle will resume support, including logging a bug with Oracle Development for investigation if required.

If the problem is determined not to be a known Oracle issue, we will refer the customer to VMware for support. When the customer can demonstrate that the issue occurs when running on the native OS, Oracle will resume support, including logging a bug with Oracle Development for investigation if required.

NOTE: Oracle has not certified any of its products on VMware. For Oracle RAC, Oracle will only accept Service Requests as described in this note on Oracle RAC 11.2.0.2 and later releases.

In my understanding of the actual way that the policy is applied, it's really a matter of whether or not the support engineer suspects VMWare to be the culprit. What I'm saying here is that, generally speaking, the support engineer will work your issue the same way that he/she would if you were on physical hardware. However, once that engineer thinks that VMWare could be the cause of your problem, they reserve the right to "punt" and say "call us back once you've reproduced it on physical hardware".

Now, VMWare, to their credit, has a policy that they call "Total Ownership", where they will accept accountability for any Oracle-related issues. You can read their official policy at the link below.


It is my understanding that, as part of the "Total Ownership" policy, VMware will reproduce the problem on physical hardware for the customer if Oracle decides that VMWare is the problem.

What about Licensing?

Part of the big problem I've always had with Oracle on VMWare is caused by Oracle's per-CPU licensing policy. My original understanding was that, if you have a total of 64 cores in your VMWare cluster, it didn't matter if you were only using 8 cores for Oracle. Oracle would tell you that you had to pay for 64 cores. The idea behind this is that you could, potentially, resize the virtual machine to suit certain needs. Maybe you need more horsepower during month end?

What I've since learned is that Oracle has a policy document (below) that talks about "soft" vs. "hard" partitioning.


What I've described above would fall under the concept of "soft partitioning". However, "hard partitioning" methodologies allow for a different approach. VMWare has (naturally) a nice document that explains their approach to implementing clusters that are in compliance with Oracle's licensing requirements.


From that document, pay particular attention to section 2.2. In that section (specifically Scenario B), they discuss DRS Host Affinity rules and VMWare CPU pinning. (emphasis mine)

2.2 Clusters: Fully Licensed Versus Partially Licensed Clusters

Scenario B: Partially Licensed Clusters

When a customer does not have enough Oracle application instances to justify creating a dedicated cluster for those applications, only a subset of the hosts in the cluster are licensed for the Oracle application. In this situation, the customer must be careful to restrict the movement of Oracle application instances and virtual machines to only those hosts that are licensed to run the product.

In this case, DRS Host Affinity rules can be used to appropriately restrict the movement of virtual machines within the cluster. DRS Host Affinity is a vSphere feature that enables you to ensure that your Oracle applications are restricted to move only between a subset of the hosts—that is, not all hardware in the cluster is “available” to the Oracle software. DRS Host Affinity is a clustering technology and is not a mechanism for soft or hard partitioning of the servers. As explained in section 2.1, using VMware CPU pinning to partially license a host is not currently recognized by Oracle as a “hard partitioning” mechanism that receives subsystem pricing. However, once you have fully licensed the host, you have the right to design your environment such that the Oracle workloads are free to run on the licensed hosts inside the cluster. At present, Oracle does not have any stated policy regarding clustering mechanisms or DRS Host Affinity. Customers can easily maiatain records for compliance purposes as explained in section 2.3.

The advantages of this approach are similar to the advantages achieved with a fully licensed cluster. Because customers are typically able to increase the utilization of licensed processors, they reduce license requirements. However, consolidation ratios tend to be lower, because advanced vSphere features can be employed only on a smaller subset of the hosts.

VMWare CPU pinning is a feature that (in my understanding) would allow you to say that a given VM would only use certain cores in a physical host. So, if you have a single host with 16 cores, you can "pin" a given VM to four of them. According to Oracle's partitioning document (and VMWare's document), you would still be required to pay for all 16 cores in the box. The basic logic here is that Oracle's licensing policy is based on the number of cores in a physical server. You can't license part of a box. Period. No exceptions.

On the other hand, DRS Host Affinity, is a way to pin a virtual machine to a given host (or collection of hosts) within a cluster. So, let's say that you have ten (10) 8-core physical hosts (total of 80 cores) in your VMWare cluster. Using DRS Host Affinity, youcould restrict your Oracle VMs to a subset of those physical hosts. For example, if you restricted your Oracle VMs to only five (5) of those physical hosts, VMWare's contention is that you would only have to license 40 cores.

I sould probably include the standard "IANAL" (I am not a lawyer) disclaimer. I'm also not a VMWare administrator. What I am is a DBA and an IT Geek. That's pretty much the limit of it.

Hopefully this provides some clarity on the issue.

For further reading on the subject, here are a couple of blog links that I used in my research:


James

Why I don't depend on TOAD (or OEM) and neither should you.


My apologies in advance, as this posting may sound like something of a rant.

The first thing I'd like to point out is that I have no real problem with TOAD, Oracle Enterprise Manager, or Windows-based editors. They are all excellent tools that can be extremely helpful in your environment. My objection to these tools is based solely on a lowest-common-denominator argument.

First, a little background. Back in the early 1990's, I was working as a Unix Systems Administrator for a company in Kansas City, MO. Since then, I've worked mainly as a consultant.

Shortly before I started that job in Kansas City, the company had hired a new CIO who let go about half of the legacy (mainframe, COBOL) IT department. The new direction for the company was implementation of Oracle E-Business Suite on Data General Unix (DG/UX).

The mainframe IT staff that survived were being re-trained in the new technology. At one point, several of them came to me insisting that I install ISPF (an editor they were used to on the mainframe) onto the DG/UX boxes because they were struggling to learn to use the vi editor. I informed them that, while they (as a group) may carry enough weight to convince the CIO to direct me to install it (assuming it was even available). However, when they go to their next job and claim that "they know Unix", they would be alone and wouldn't have that leverage.  My suggestion was that I would help them to learn the vi editor. (I did offer emacs as an alternative, since it is and was extremely common on Unix systems... Unfortunately, friendlier editors like pico, nano, and joe didn't exist yet.)

If your primary job is software development, a tool like TOAD is generally something you can depend on having. However, as a DBA, you can't necessarily depend on having TOAD (or even Oracle Enterprise Manager) at your disposal at all times. Maybe you're starting a new job and the previous DBA hadn't set up Enterprise Manager (or you haven't gotten around to it yet). Even in environments where those tools are available, they may or may not be working when you need them.

So, my advice? There are certain tools that are almost ALWAYS there. Get comfortable with ssh, SQL*Plus, and vi (or vim).  They are your friends.

James