install the VMware Server software for linux

VMware Server Installation for linux
Regardless of the host OS, the setup of the virtual machines should be similar.

First, install the VMware Server software. On Linux you do this with the following command as the root user.
# rpm -Uvh VMware-server-*.rpm
Preparing... ########################################### [100%]
1:VMware-server ########################################### [100%]
#


Then finish the configuration by running the vmware-config.pl script as the root user. Most of the questions can be answered with the default response by pressing the return key. The output below shows my responses to the questions.
# vmware-config.pl
Making sure services for VMware Server are stopped.

Stopping VMware services:
Virtual machine monitor [ OK ]

You must read and accept the End User License Agreement to continue.
Press enter to display it.

VMWARE, INC.
SOFTWARE BETA TEST AGREEMENT

*** Editied out license agreement ***

Do you accept? (yes/no) yes

Thank you.

Configuring fallback GTK+ 2.4 libraries.

In which directory do you want to install the mime type icons?
[/usr/share/icons]

What directory contains your desktop menu entry files? These files have a
.desktop file extension. [/usr/share/applications]

In which directory do you want to install the application's icon?
[/usr/share/pixmaps]

Trying to find a suitable vmmon module for your running kernel.

The module bld-2.6.9-5.EL-i686smp-RHEL4 loads perfectly in the running kernel.

Do you want networking for your virtual machines? (yes/no/help) [yes]

Configuring a bridged network for vmnet0.

The following bridged networks have been defined:

. vmnet0 is bridged to eth0

All your ethernet interfaces are already bridged.

Do you want to be able to use NAT networking in your virtual machines? (yes/no)
[yes]

Configuring a NAT network for vmnet8.

Do you want this program to probe for an unused private subnet? (yes/no/help)
[yes]

Probing for an unused private subnet (this can take some time)...

The subnet 172.16.210.0/255.255.255.0 appears to be unused.

The following NAT networks have been defined:

. vmnet8 is a NAT network on private subnet 172.16.210.0.

Do you wish to configure another NAT network? (yes/no) [no]

Do you want to be able to use host-only networking in your virtual machines?
[yes] no

Trying to find a suitable vmnet module for your running kernel.

The module bld-2.6.9-5.EL-i686smp-RHEL4 loads perfectly in the running kernel.

Please specify a port for remote console connections to use [902]

Stopping xinetd: [ OK ]
Starting xinetd: [ OK ]
Configuring the VMware VmPerl Scripting API.

Building the VMware VmPerl Scripting API.

Using compiler "/usr/bin/gcc". Use environment variable CC to override.

The installation of the VMware VmPerl Scripting API succeeded.

Do you want this program to set up permissions for your registered virtual
machines? This will be done by setting new permissions on all files found in
the "/etc/vmware/vm-list" file. [no] yes

Generating SSL Server Certificate

In which directory do you want to keep your virtual machine files?
[/var/lib/vmware/Virtual Machines] /u01/VM

Do you want to enter a serial number now? (yes/no/help) [no] yes

Please enter your 20-character serial number.

Type XXXXX-XXXXX-XXXXX-XXXXX or 'Enter' to cancel: ENTER-YOUR-SERIAL-NUMBER

Starting VMware services:
Virtual machine monitor [ OK ]
Virtual ethernet [ OK ]
Bridged networking on /dev/vmnet0 [ OK ]
Host-only networking on /dev/vmnet8 (background) [ OK ]
NAT service on /dev/vmnet8 [ OK ]
Starting VMware virtual machines... [ OK ]

The configuration of VMware Server e.x.p build-22874 for Linux for this running
kernel completed successfully.

#

The VMware Server Console is started by issuing the command "vmware" at the command prompt, or by selecting it from the "System Tools" menu.

On the "Connect to Host" dialog, accept the "Local host" option by clicking the "Connect" button.

You are then presented with the main VMware Server Console screen.

The VMware Server is now installed and ready to use.
Virtual Machine Setup
Now we must define the two virtual RAC nodes. We can save time by defining one VM, then cloning it when it is installed.

Click the "Create a new virtual machine" button to start the "New Virtual Machine Wizard". Click the "Next" button on the welcome page.

Select the "Custom" virtual machine configuration and click the "Next" button

Select the "Linux" guest operating system option, and set the version to "Red Hat Enterprise Linux 4", then click the "Next" button. The RHEL4 option is used because at the time of writing, the current version of VMware server doesn't explicitly support RHEL5, but this setting works fine.

Enter the name "RAC1" and the location should default to "/u01/VM/RAC1", then click the "Next" button.

Select the required number of processors and click the "Next" button.

Uncheck the "Make this virtual machine private" checkbox and click the "Next" button.

Select the amount of memory to associate with the virtual machine. Remember, you are going to need two instances, so don't associate too much, but you are going to need approximately 1 Gig (1024 Meg) to compete the installation successfully.

Accept the "Use bridged networking" option by clicking the "Next" button.

Accept the "LSI Logic" option by clicking the "Next" button.

Select the "Create a new virtual disk" option and click the "Next" button.

Accept the "SCSI" option by clicking the "Next" button. It's a virtual disk, so you can still use this option even if your physical disk is IDE or SATA.

Set the disk size to "10.0" GB and uncheck the "Allocate all disk space now" option. The latter will make disk access slower, but will save you wasting disk space. Note: If you intend to unzip the Oracle installation software on the virtual disk you will need a 12-15 GB virtual disk.

Accept "RAC1.vmdk" as the disk file name and complete the VM creation by clicking the "Finish" button.

On the "VMware Server Console" screen, click the "Edit virtual machine settings" button.

On the "Virtual Machine Settings" screen, highlight the "Floppy 1" drive and click the "- Remove" button.

Click the "+ Add" button, select a hardware type of "Ethernet Adapter", then click the "Next" button.

Accept the "Bridged" option by clicking the "Finish" button.

Finish by clicking the "OK" button on the Virtual Machine Settings dialog.

The virtual machine is now configured so we can start the guest operating system installation.

Guest Operating System Installation
Place the first OEL 5 disk in the CD drive and start the virtual machine by clicking the "Power on this virtual machine" button. The right pane of the VMware Server Console should display a boot loader, then the OEL installation screen.

Continue through the OEL 5 installation as you would for a normal server. A general pictorial guide to the installation can be found here. More specifically, it should be a server installation with a minimum of 2G swap, firewall and SELinux disabled and the following package groups installed:
GNOME Desktop Environment
Editors
Graphical Internet
Text-based Internet
Development Libraries
Development Tools
Server Configuration Tools
Administration Tools
Base
System Tools
X Window System

To be consistent with the rest of the article, the following information should be set during the installation:
hostname: rac1.localdomain
IP Address eth0: 192.168.2.101 (public address)
Default Gateway eth0: 192.168.2.1 (public address)
IP Address eth1: 192.168.0.101 (private address)
Default Gateway eth1: none
You are free to change the IP addresses to suit your network, but remember to stay consistent with those adjustments throughout the rest of the article.

Once the basic installation is complete, install the following packages whilst logged in as the root user.
# From Enterprise Linux 5 Disk 1
cd /media/cdrom/Server
rpm -Uvh binutils-2.*
rpm -Uvh elfutils-libelf-0.*
rpm -Uvh glibc-2.*
rpm -Uvh glibc-common-2.*
rpm -Uvh libaio-0.*
rpm -Uvh libgcc-4.*
rpm -Uvh libstdc++-4.*
rpm -Uvh make-3.*
cd /
eject

# From Enterprise Linux 5 Disk 2
cd /media/cdrom/Server
rpm -Uvh compat-libstdc++-33*
rpm -Uvh elfutils-libelf-devel-*
rpm -Uvh glibc-headers*
rpm -Uvh glibc-devel-2.*
rpm -Uvh libgomp*
rpm -Uvh gcc-4.*
rpm -Uvh gcc-c++-4.*
rpm -Uvh libaio-devel-0.*
rpm -Uvh libstdc++-devel-4.*
rpm -Uvh unixODBC-2.*
rpm -Uvh unixODBC-devel-2.*
cd /
eject

# From Enterprise Linux 5 Disk 3
cd /media/cdrom/Server
rpm -Uvh sysstat-7.*
cd /
eject

Oracle Installation Prerequisites
Perform the following steps whilst logged into the RAC1 virtual machine as the root user.

The /etc/hosts file must contain the following information.
127.0.0.1 localhost.localdomain localhost
# Public
192.168.2.101 rac1.localdomain rac1
192.168.2.102 rac2.localdomain rac2
#Private
192.168.0.101 rac1-priv.localdomain rac1-priv
192.168.0.102 rac2-priv.localdomain rac2-priv
#Virtual
192.168.2.111 rac1-vip.localdomain rac1-vip
192.168.2.112 rac2-vip.localdomain rac2-vip
Add the following lines to the /etc/sysctl.conf file.
kernel.shmmni = 4096
# semaphores: semmsl, semmns, semopm, semmni
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default=4194304
net.core.rmem_max=4194304
net.core.wmem_default=262144
net.core.wmem_max=262144
Run the following command to change the current kernel parameters.
/sbin/sysctl -p
Add the following lines to the /etc/security/limits.conf file.
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
Add the following lines to the /etc/pam.d/login file, if it does not already exist.
session required pam_limits.so
Disable secure linux by editing the /etc/selinux/config file, making sure the SELINUX flag is set as follows.
SELINUX=disabled
Alternatively, this alteration can be done using the GUI tool (System > Administration > Security Level and Firewall). Click on the SELinux tab and disable the feature.

Create the new groups and users.
groupadd oinstall
groupadd dba
groupadd oper
groupadd asmadmin

useradd -u 500 -g oinstall -G dba,oper,asmadmin oracle
passwd oracle
Create the directories in which the Oracle software will be installed.
mkdir -p /u01/crs/oracle/product/11.1.0/crs
mkdir -p /u01/app/oracle/product/11.1.0/db_1
chown -R oracle:oinstall /u01
Login as the oracle user and add the following lines at the end of the .bash_profile file.
# Oracle Settings
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR

ORACLE_HOSTNAME=rac1.localdomain; export ORACLE_HOSTNAME
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/11.1.0/db_1; export ORACLE_HOME
ORACLE_SID=RAC1; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
PATH=/usr/sbin:$PATH; export PATH
PATH=$ORACLE_HOME/bin:$PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH

if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi

Install VMware Client Tools
Login as the root user on the RAC1 virtual machine, then select the "VM > Install VMware Tools..." option from the main VMware Server Console menu.

This should mount a virtual CD containing the VMware Tools software. Double-click on the CD icon labelled "VMware Tools" to open the CD. Right-click on the ".rpm" package and select the "Open with 'Install Packages'" menu option.

Click the "Continue" button on the "Completed System Preparation" screen and wait for the installation to complete.

Once the package is loaded, the CD should unmount automatically. You must then run the "vmware-config-tools.pl" script as the root user.
# vmware-config-tools.pl
Accept all the default settings and pick the screen resolution of your choice. Ignore any warnings or errors. The VMware client tools are now installed. Reboot the server before proceeding. After the reboot, it is possible the monitor will not be recognised. If this is the case don't panic. Follow the instructions provided on the screen and reconfigure the monitor setting, which will allow the XServer to function correctly.
Create Shared Disks
Shut down the RAC1 virtual machine using the following command.
# shutdown -h now
Create a directory on the host system to hold the shared virtual disks.
# mkdir -p /u01/VM/shared
On the VMware Server Console, click the "Edit virtual machine settings" button. On the "Virtual Machine Settings" screen, click the "+ Add" button.

Click the "Next" button on the welcome screen, then select the hardware type of "Hard Disk" and click the "Next" button.

Accept the "Create a new virtual disk" option by clicking the "Next" button.

Accept the "SCSI" option by clicking the "Next" button.

Set the disk size to "10.0" GB and uncheck the "Allocate all disk space now" option, then click the "Next" button.

Set the disk name to "/u01/VM/shared/ocr.vmdk" and click the "Advanced" button.

Set the virtual device node to "SCSI 1:1" and the mode to "Independent" and "Persistent", then click the "Finish" button.

Repeat the previous hard disk creation steps 4 more times, using the following values:
File Name: /u01/VM/shared/votingdisk.vmdk
Virtual Device Node: SCSI 1:2
Mode: Independent and Persistent

File Name: /u01/VM/shared/asm1.vmdk
Virtual Device Node: SCSI 1:3
Mode: Independent and Persistent

File Name: /u01/VM/shared/asm2.vmdk
Virtual Device Node: SCSI 1:4
Mode: Independent and Persistent

File Name: /u01/VM/shared/asm3.vmdk
Virtual Device Node: SCSI 1:5
Mode: Independent and Persistent

At the end of this process, the virtual machine should look something like the picture below.

Edit the contents of the "/u01/VM/RAC1/RAC1.vmx" file using a text editor, making sure the following entries are present. Some of the tries will already be present, some will not.
disk.locking = "FALSE"
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.dataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0"

scsi1.present = "TRUE"
scsi1.virtualDev = "lsilogic"
scsi1.sharedBus = "VIRTUAL"

scsi1:1.present = "TRUE"
scsi1:1.mode = "independent-persistent"
scsi1:1.fileName = "/u01/VM/shared/ocr.vmdk"
scsi1:1.deviceType = "plainDisk"
scsi1:1.redo = ""

scsi1:2.present = "TRUE"
scsi1:2.mode = "independent-persistent"
scsi1:2.fileName = "/u01/VM/shared/votingdisk.vmdk"
scsi1:2.deviceType = "plainDisk"
scsi1:2.redo = ""

scsi1:3.present = "TRUE"
scsi1:3.mode = "independent-persistent"
scsi1:3.fileName = "/u01/VM/shared/asm1.vmdk"
scsi1:3.deviceType = "plainDisk"
scsi1:3.redo = ""

scsi1:4.present = "TRUE"
scsi1:4.mode = "independent-persistent"
scsi1:4.fileName = "/u01/VM/shared/asm2.vmdk"
scsi1:4.deviceType = "plainDisk"
scsi1:4.redo = ""

scsi1:5.present = "TRUE"
scsi1:5.mode = "independent-persistent"
scsi1:5.fileName = "/u01/VM/shared/asm3.vmdk"
scsi1:5.deviceType = "plainDisk"
scsi1:5.redo = ""
Start the RAC1 virtual machine by clicking the "Power on this virtual machine" button on the VMware Server Console. When the server has started, log in as the root user so you can partition the disks. The current disks can be seen by issueing the following commands.
# cd /dev
# ls sd*
sda sda1 sda2 sdb sdc sdd sde sdf
#
Use the "fdisk" command to partition the disks sdb to sdf. The following output shows the expected fdisk output for the sdb disk.
# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.


The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1305, default 1305):
Using default value 1305

Command (m for help): p

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 1 1305 10482381 83 Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
#
In each case, the sequence of answers is "n", "p", "1", "Return", "Return", "p" and "w".

Once all the disks are partitioned, the results can be seen by repeating the previous "ls" command.
# cd /dev
# ls sd*
sda sda1 sda2 sdb sdb1 sdc sdc1 sdd sdd1 sde sde1 sdf sdf1
#
Add the following commands to the /etc/rc.local file.
chown oracle:oinstall /dev/sdb1
chown oracle:oinstall /dev/sdc1
chown oracle:oinstall /dev/sdd1
chown oracle:oinstall /dev/sde1
chown oracle:oinstall /dev/sdf1
chmod 600 /dev/sdb1
chmod 600 /dev/sdc1
chmod 600 /dev/sdd1
chmod 600 /dev/sde1
chmod 600 /dev/sdf1
The shared disks are now configured. We don't have to worry about defining raw devices, which are deprecated in Enterprise Linux 5.
Clone the Virtual Machine
The current version of VMware Server does not include an option to clone a virtual machine, but the following steps illustrate how this can be achieved manually.

Shut down the RAC1 virtual machine using the following command.
# shutdown -h now
Copy the RAC1 virtual machine using the following command.
# cp -R /u01/VM/RAC1 /u01/VM/RAC2
Edit the contents of the "/u01/VM/RAC2/RAC1.vmx" file, making the following change.
displayName = "RAC2"
Ignore discrepancies with the file names in the "/u01/VM/RAC2" directory. This does not affect the action of the virtual machine.

In the VMware Server Console, select the File > Open menu options and browse for the "/u01/VM/RAC2/RAC1.vmx" file. Once opened, the RAC2 virtual machine is visible on the console. Start the RAC2 virtual machine by clicking the "Power on this virtual machine" button and click the "Create" button on the subsequent "Question" screen.

Ignore any errors during the server startup. We are expecting the networking components to fail at this point.

Log in to the RAC2 virtual machine as the root user and start the "Network Configuration" tool (System > Administration > Network).