Technical differences between VMware ESXi and ESX

I have spent some time with looking for more details about VMware ESXi compared to VMware ESX. I summarized the main differences in this article but I think it's not complete. There have to be more features missing in ESXi because of service console removal. So, what next did I discover?

• ESXi is supported on smaller set of certified hardware because it is standalone system and it doesn't depend on RHEL service console which provides drivers for other hardware.
  

• You can manage ESXi with RCLI on Linux or Windows platform but Virtual Infrastructure client is more comfortable and easier to use. Further, if you deployed ESXi without Virtual Infrastructure licence, RCLI will have read-only access only. The drawback of VI client is that it is available for Windows platform. The solution may exist in using Wine emulator but the installation isn't as straightforward as on Windows plartform. The Wine application database contains this entry about VI client installation but I haven't tried it yet.
  

• You can manage your ESX server directly via serial cable but ESXi is missing this feature.

• ESXi kernel is missing jumbo frames support in TCP/IP stack which allows to send larger frames out onto physical network. It can help to achieve higher throughput with NFS or iSCSI protocols.
  

• ESXi doesn't support NetQueue technology which is boosting 10G Ethernet performance.

• Finally, VMware in cooperation with Mellanox Technologies supports Infiniband host channel adapters on ESX. ESXi is missing it.
  

The previous six points are related to the technical aspects of ESX and ESXi hypervisor. These points aren't complete as well but they are quite important for common deployment of VMware technologies. If you know about something else, please share it at my blog. For further information, check these links:

1. VMware ESX 3.5 release notes
2. VMware ESXi 3.5 release notes
3. ESX and ESXi comparison (VMware knowledge base)
   
4. Differences between ESXi and ESX (VMware knowledge base)

VMware Server 2.0 is coming ...

Last week, VMware released another candidate of oncoming Server 2.0 denoted as RC 2. Because I'm using the current stable version 1.0.6 in my testing environment a lot, I'm looking forward to the new one as well. It should bring a lot of new useful features among which belongs:

1. support of USB 2.0 devices - it will be more comfortable to use high-speed USB memory sticks
   
2. online disk capacity expansion - it will be possible to hot-add a SCSI hard drive to virtual machine and to expand its size on the fly without powering it off
3. VMCI - Virtual Machine Communication Interface will allow efficient communication between virtual machines or between virtual machine and host system so you don't need to use more generic and slower communication channels like network
   
4. virtual machine scalability - virtual machine will be able to manage up to 8GB of memory (the current stable release can manage 3.6GB only) and it will suport two-way vSMP (current release contains experimental support only)
5. remote client devices - new VMware Remote Console will allow to use not hosted devices such as CD-ROM
6. broader OS support - support of Microsoft Windows Vista, Windows Server 2008 or RHEL 5 will be included

And that's not everything. More details are written at www.vmware.com. For impatient, VMware provides beta program which allows you to try it. As well, read the released user guide.

Quickly - how to download a file to the ESX 3.x service console?

The VMware ESX 3.x is missing wget package so you can't use wget command to download anything from the Internet as you wish. In spite of wget, the service console provides lwp-* tools which are simple perl scripts based on LWP and URI perl modules and which allow to do some basic tasks around the HTTP protocol.

The tools are part of perl-libwww-perl package. The package is installed by default. The most important tool is lwp-download which you can use for downloading files. Let's check the steps how to download something:

1. esxcfg-firewall --allowOutgoing
   
• allow outgoing connections from service console
2. lwp-download http://dfn.dl..../apcupsd-3.14.4-1.el3.i386.rpm
   
• download apcupsd package
3. esxcfg-firewall --blockOutgoing
• return firewall to the initial state

Beside this, the perl-libwww-perl package contains other tools like lwp-mirror, lwp-request and lwp-rget. Check their man pages for their usage.

Quickly - how to determine the ESX host version?

In my opinion, the easiest way how to find out the ESX host version, is to log in to the service console and use the esxupdate command. The major version can be found in the file /etc/vmware-release. For example, it may contain:

VMware ESX Server 3

So, the major version is 3.x. To determine minor version is a little complicated. Run this command from service console:

esxupdate query

And try to identify patches with the following prefixes from the end of command output (the last one is the right one):

1. ESX - the minor version should be 0, so we have version 3.0.x
2. ESX350 - the minor version is 5, the version is 3.5.0
3. 3.5.0 - initial instalation of version 3.5.0
   

The corresponding lines may look like these:

3.5.0-64607    16:25:29 05/27/08 ESX 3.0.x to 3.5.0-64607 upgrade
3.5.0-64607    10:42:31 08/06/08 Full bundle of ESX 3.5.0-64607

It remains to identify the update level. Use the same command as above and check the full patch name now:

1. ESX350-Update01 - we have 3.5.0 Update 1
2. ESX350-Update02 - we have 3.5.0 Update 2

The corresponding lines are:

ESX350-Update01    16:59:56 05/27/08 ESX Server 3.5.0 Update 1
ESX350-Update02    10:42:38 08/06/08 ESX Server 3.5.0 Update 2

Do you know another way how to reach the version? Beside this, I have found this knowledge base article.

VMware ESX 3.5 Update 2 and power on virtual machine bug ?

What a coincidence! Me and my colleague were preparing a server with VMware Infrastructure Update 2 yesterday. Just another simple scenario, we were thinking. We just wanted to run some checks if it is suitable for production usage. Everything went smoothly. But when my colleague began his job, things went worse. His task was easy - install some new virtual machines in prepared environment and check their behaviour (details aren't important now).

When he wanted to power on a prepared virtual machine to begin guest installation, the VMware Infrastructure client displays the following error message:

• A general system error occurred: Internal error

It is nothing interesting, isn't it? No clue where to go. The virtual machine stays powered off. Any running machine stays running until you suspend it or power it off. So,

• don't power off or vmotion your virtual machines!!!

Otherwise, you will have a big trouble! Naturally, I checked the virtual machine vmware.log log file. I found here a "virtual" reason why such strange behaviour:

• [msg.License.product.expired] This product has expired.
• Be sure that your host machine's date and time are set correctly.
• There is a more recent version available at ...

Hm, the right time is often critical part of any installation but my habit is to configure NTP server where it is possible. I done it in this scenario as well. I checked the server time and nothing was wrong. The ESX host is licenced properly. When I try to search the VMware knowledge base, there is no answer at all. And you need to have a luck today because it is really really overloaded by others. Maybe, they are deailing with the same problem.

Because my colleague needed to work, my last try was to move the time backwards. I installed the server two days ago and everything was working fine. It was August 10. My colleague began working this morning and since this time, any virtual machine couldn't be powered on. Today, it is August 12. So I moved the time here and it helped! Of course, the NTP server has to be shutdown. You can do it via VI client or you can log in to the service console and use date -s command. I'm not aware of other working solution now.

This afternoon, I found the first article about this annoying bug at www.virtualization.info. It is written here the VMware knowledge base seems to collapsed due to this issue and the support team is promising solution in 36 hours. By the way, Update 2 can't be downloaded now.

Differences between VMware ESXi and ESX

VMware ESXi hypervisor is free of charge now but what are the reasons to use it instead of VMware ESX? And what advantages does it have?

The most important advantage is you don't need to pay for it. Furthermore, it supports all VMware Infrastructure features if you buy proper licences - you can vmotion virtual machines, schedule resources, backup them via VCB and so on. If you really want to save more bucks you don't have to pay the support which is required in case of VMware Infrastructure. The new option is to pay per incident.

What are the main differences? As I written here, the ESXi hypervisor is OS independent (it is without service console) and its installation requires only 32MB of disk space. The negative thing is that it lacks VirtualCenter agent, VCB and update manager. These features are included in VMware Infrastructure Foundation edition and higher by default. If you would like to manage ESXi hosts you need to buy agent licences.

It remains last important question. How can we control ESXi hosts remotely if we don't have service console? The ESXi hypervisor doesn't have SSH access by default but supports RCLI or Remote Command Line Interface. The RCLI allows to perform remote command line operations on an ESXi host from your management station. If you still prefer SSH to RCLI you can enable it according this article.

So, are you going to deploy it? If so, you can write me about your experience with the product. I would like to know your story.

VMware ESXi or Microsoft Hyper-V

VMware ESXi 3.5 update 2 is totally free now. It comes out from VMware Infrastructure and has a lot of features in common. The hypervisor can be downloaded from www.vmware.com.

I was interested in reasons why the hypervisor was released for free. And I must admit the company keeps still the same rule - what provides competitors we provide for free.

The competitor is Microsoft and their Hyper-V hypervisor. It is free like ESXi but both of them require to pay for management software. But if you want to use Hyper-V, you need to pay for Windows Server 2008 licences. Besides, ESXi is bare-metal solution and you don't need any operating system for service console! Check the prices comparison of both solutions in this article at www.virtualisation.info.

The conclusion is simple - ESXi seems to be significantly cheaper then Hyper-V in common SMB environments which require basic consolidation with or without centralized management.