Second edition of VMware Site Recovery Manager is out

The second edition of VMware SRM, officially named as VMware vCenter Site Recovery Manger 4, was released recently. The product is responsible for automated disaster recovery of complex virtual environments. The recent version is fully compatible with VMware vSphere platform and provides these new important features:

• many-to-one failover - this means that one site is able to recover from multiple sites failures
• expanded support of storage vendors - those who provides storage replication solutions over FC, iSCSI or NFS, 12 generally. Among them belongs DELL, IBM, HP, EMC, LSI and others.

For more information on VMware SRM 4 visit the release notes and product home page.

VMware Server 1.0.x library dependency problem

In the beginning of the year, I wrote this article about some problems between older VMware server 1.0.x and newer Linux distributions. The problem is related to the vmware kernel modules whose source code are not compatible with newer Linux kernels.

I was surprised with one thing. When I upgraded VMware Server from version 1.0.8 to 1.0.9, VMware Server console stopped working. The new version was installed on the same system (OpenSUSE 11.1) as the old one, so I don't understand the reason. The important thing is I found a solution. The new version began producing these new error messages after trying to run vmware command:

/usr/lib/vmware/lib/libgcc_s.so.1/libgcc_s.so.1: version `GCC_4.2.0' not found (required by /usr/lib/libstdc++.so.6)
/usr/lib/vmware/lib/libgcc_s.so.1/libgcc_s.so.1: version `GCC_4.2.0' not found (required by /usr/lib/libstdc++.so.6)
/usr/lib/vmware/bin/vmware: symbol lookup error: /usr/lib/libgio-2.0.so.0: undefined symbol: g_thread_gettime

I have tried to unset this environment variable influencing behavior of GTK2 applications:

unset GTK2_RC_FILES

Otherwise, the variable is referencing related gtkrc files defining GTK2 user's environment. Try it and I hope it will help.

Red Hat Enterprise Linux 5.4 released

Today, it was released a next minor version of Red Hat's flagship Linux distribution RHEL 5.4. Here it is a brief summary of new features and updates:

• KVM hypervisor - Full support of Kernel-based Virtual Machine is included now. XEN support is included as well, but you can't use both XEN and KVM at the same time. Each hypervisor requires different kernel. You need to have 64b machine to run KVM. It supports RHEL 3/4/5 or Windows XP/2003/2008 as guests.
• KVM paravirtualized drivers - They are available for Windows XP/2003/2008 in package virtio-win.
• FUSE support - New version includes modules for Filesystem in Userspace (FUSE) and related utilities. Support for the XFS was added as well. It icnludes updates of CIFS and EXT4 filesystems.
• Infiniband drivers - It contains some portions of prepared Open Fabrics Enterprise Distribution (OFED) 1.4.1.

New release of RHEL contains many other updates and enhancements which aren't mentioned here. For more details read the RHEL 5.4 official release notes.

VMware vSphere hotplug

Hotplug of virtual hardware is attractive feature of VMware ESX 3.x/4.x. In case of ESX 3.x it is limited to hotadd of virtual disk to a running virtual machine only. With next generation of VMware vSphere hypervisor you are able to hotadd of memory or CPU to a machine if guest operating system supports it.

I was surprised during vSphere evaluation how it pretty works. I used to hotadd of virtual disks to my machines quite often. But when I upgraded 3.5 infrastructure to new vShpere 4.0, I became disappointed because it stopped working.

The reason why is simple. Hotadd feature is available from advanced edition only and I was upgrading to standard edition which doesn't contain license for it. You can check it in my previous post VMware vShpere 4.0 editions . Below is an error message which is complaining about missing license:

I think it wasn't a right decision to shift the feature to the higher editions. I think it would be better to leave things where they were because people are used to using them. And I hope that VMware will push back at least hotadd of virtual disk in some future release of vSphere.

Linux rc.local script

Sometimes, you need to run some commands during your Linux server startup. And you don't want to waste time with preparing valid init script now. The common task is to load some kernel module or to change speed of network interface and so on.

Red Hat distributions provides for this task rc.local script. You can find it in the directory /etc/rc.d. The script is executed after all the other init scripts. This is ensured with the proper START scripts linking to the /etc/rc.d/rc.local script:

/etc/rc.d/rc2.d/S99local
/etc/rc.d/rc4.d/S99local
/etc/rc.d/rc3.d/S99local
/etc/rc.d/rc5.d/S99local

SUSE distros like SLES or OpenSUSE provide similar mechanism. You have available two scripts. The before.local script should contain everything you want to run before runlevel is entered. The after.local script works like RedHat's rc.local script. It contains stuff which should be executed after runlevel is reached. The scripts don't exist by default, you need to create them at first in the directory /etc/init.d. They don't have to be set executable.

Besides this, the RedHat's rc.local script is executed only in runlevels 2, 3, 4 or 5. It is ignored in the single user mode. SUSE's version of after.local or before.local is interpreted during all runlevels including runlevel 1.

Solaris 10 updates summary

The seventh update of Solaris 10 was released on May. It contains support of Intel Nehalem CPU and some ZFS enhancements. I added it to the summary of Solaris updates .Here it is:

1. Solaris 10 1/06 (u1) - GRUB bootloader, iSCSI initiator, fcinfo command
   
2. Solaris 10 6/06 (u2) - ZFS filesystem
3. Solaris 10 11/06 (u3) - Solaris Trusted Extensions, LDoms
4. Solaris 10 8/07 (u4) - full TCP/IP stack in zones, iSCSI target, branded zones (Linux in Solaris container), Samba AD, enhanced rcapd
5. Solaris 10 5/08 (u5) - Intel SpeedStep, AMD PowerNow!, Solaris 8/9 P2V (to Solaris 10 zones), CPU capping
   
6. Solaris 10 10/08 (u6) - ZFS boot support, many ZFS filesystem enhancements
7. Solaris 10 5/09 (u7) - performance and power management support for Intel Nehalem CPUs, support of ZFS clones when cloning zones, IPsec SMF services, SunVTS 7.0 update
   

For more details, click the particular release to read the official release notes.

VMware or Citrix?

Citrix released their virtualization solution named XenServer (from version 5, article XenServer is free) for free but only the time will show if it was a right decision. At first glance, it seems like a marvelous thing but there are some facts which should be investigated first. Together with XenServer, it was released central management solution XenCenter.

Let's have a look at their rival VMware (vSphere 4). XenServer is fully comparable to VMware ESX or ESXi. But what about XenCenter management? It's something more than VMware vSphere client but not so valuable as VMware vCenter Management Server. Citrix XenCenter is not the right choice in case of comparison to vCenter. The right one is Citrix Essentials but this one is not for free already. The main differences between Citrix XenCenter and Essentials are:

• XenCenter is missing alerting capabilities like send me an email when "CPU usage is too high" or when some error condition like "virtual machine power on failure" appears
• XenCenter is missing high availability support
• XenCenter is not able to show you performance data older than one day for physical or virtual servers
  

Now, let's try to propose a simple high availability (HA) solution based on Citrix/VMware products and compare their prices. Let's suppose we have 2 (3) entry level servers where each have 2 CPUs with max 6 cores per CPU (6 CPUs total). The servers are connected to a shared disk storage. The CPU speed or memory capacity is not important now. And we require HA solution to protect our virtual machines from hardware failure. Follows the analysis:

- Citrix Essentials Enterprise (1 license = 1 server):

• XenServer - 2 licenses = 0$ (3 lic = 0$)
• Essentials Enterprise - 2 lic = 5500$ (3 lic = 8250$)
• Essentials Preffered Support (optional) - 1 lic = 1500$
  
• Total cost = 7000$ or 9750$ for 3 servers
  
• Total cost without support = 5500$ or 8250$ for 3 servers

- VMware vSphere 4 Standard Edition (1 lic = 1 CPU):

• vSphere 4 Standard - 4 licenses = 3180$ (6 lic = 4770$)
• vShpere 4 Standard 1y Gold Support - 4 lic = 1092$ (6 lic = 1638$)
• vCenter 4 Foundation - 1 lic = 1495$
• vCenter 4 1y Gold Support - 1 lic = 545$
• Total cost = 6312$ or 8448$ for 3 servers
• Support is mandatory

- VMware vSphere 4 Essentials Plus Bundle (1 lic = 1 CPU)

• Licenses for 3 hosts plus vCenter Server for Essentials plus 1y Gold Support = 3624$
• Total cost = 3624$ for 2 or 3 servers
  

The prices of proposed solutions are quite different. In my opinion, the most valued solution is based on new VMware product line vSphere 4 Essentials.

There are rumors that VMware is the most expensive solution. I don't think so if I check the numbers above. Citrix's solution not covered by support is cheaper then VMware's solution with support but only for 2 servers. If I would like to add third server I would have to pay another license in case of Citrix. In case of VMware, I have still one spare license so I will use it. At first glance, XenServer seems to be free of charge but the price of added value by Citrix Essentials doesn't scale as well as in case of VMware vSphere 4 Standard Edition or vSphere 4 Essentials Plus. And what is your opinion to the topic?

VMware vSphere - OVF support

OVF or Open Virtualization Format is an open DMTF standard with intention to package and distribute virtual machines or virtual appliances among various hypervisors independently on hypervisor and CPU architecture.

VMware supports OVF format and actively participate on its development . It is supported on ESX 3.5 and VirtualCenter 2.5 but the implementation doesn't support full OVF feature set. (draft standard, version 0.9). VMware vSphere 4.0 has full native support of OVF format in version 1.0. Beside, there exists standalone VMware OVF Tool 1.0 which brings OVF support for products like VMware Workstation or VMware Server.

OVF is a packaging format for software appliances. For example, it may contain tested LAMP stack prepared for simple deployment in production. It is a way how to transport virtual machine templates portably. OVF package may contain single or more virtual machines which must be installed (deployed) before they can be run. It is not run-time virtual machine format like VMDK. Further, it provides content verification and integrity checking.

Compared to VMDK format, OVF defines complete virtual machine - virtual hardware configuration including CPU, memory, storage, networking and virtual disks. On the other hand, VMDK is in charge of virtual disks only.

Are there available any OVF packages? Yes, for example there exists OVF of VMware vCenter 2.5 for Linux or vCenter Admin Portal and many others at VMware Virtual Appliance Marketplace.

VMware vSphere - Fault Tolerance

VMware High Availability provides protection against physical servers failures running ESX hypervisors. If one host in HA cluster fails then failed virtual machines are restarted on another alive host from cluster. It ensures the host has enough resources to fulfill requirements of newly booted virtual machines. It is able to monitor virtual machine activity by checking its heartbeat as well and in case of its failure to restart it.

The next logical step is fault tolerant virtual environment. VMware vSphere 4 can do it . It provides zero downtime and data integrity of virtual machines in case of physical server failure.

When you configure a virtual machine to be fault tolerant a secondary duplicate machine is created on a different host. Then, any operation performed on the primary machine is recorded and replayed on its duplicate. If the primary fails the secondary takes over and continues running without interruption. However, current version is not able to monitor applications running inside virtual machines but it should be available in future.

VMware Fault Tolerance or VMware FT as it is denoted is cool and must have feature but to implement it means to meet these requirements:

1. VM (virtual machine) must be in HA cluster
   
2. esx host ssl certificates checking has to be enabled
   
3. VM has to be stored on shared storage
   
4. VM's virtual disks have to be in thick format, thin is not supported already
   
5. three VMkernel 1G ports are required, one for VMotion and two for FT
   
6. FT doesn't support Virtual SMP, only single processor VMs are supported!
7. physical hosts have to support hardware assisted virtualisation, no problem with recent servers

The most of requirements are common ones but points 4 and 6, for older servers 7 as well, are considerlable limitations. It's not so simple to implement VMware FT but I hope it will get better in next releases.

Finally, vSphere documentation is available at vmware.com.

RHEL 4.8 released

Yesterday, a next minor version of Red Hat Enterprise Linux 4 was released. The new version 4.8 contains the foloowing updates and enhancements:

• optimized drivers for RHEL 4 guests running on KVM hypervizor
• SAMBA update for better interoperability with Windows world
• new kernel tunables for better performance

For details, there are official release notes published at redhat.com.