Some time ago, I became aware of an ESX 3.5.0 Update 2 (build 110268) host which had 128GB of RAM physically installed, but only 64GB RAM usable. The host was showing 128GB of RAM, however, it was consuming 64GB of memory with no running VMs, leaving the other 64GB of RAM addressable for virtual machines.
After further research, it was determined that this host build did not contain the VMkernel change required to properly acknowledge the amount of physical memory installed on the IBM host hardware.
VMware’s response was:
Prior to ESX 3.5 Update 3, the ability to address more than 64GB of memory on ESX Server 3.5 is suppressed by default. In a standard installation, a 36bit MTRR mask is forced, even though the machine may support 40bit mask values. This means that the ESX Server may see any memory above 64GB as memory that is in use. For example, if an ESX server has 256GB of RAM, the Memory Usage counter displays 192GB in use and only 64GB free. If you attempt to create a virtual machine using memory exceeding the available 64GB of memory, you see an Insufficient Memory error. This condition is documented in the following location: http://www.vmware.com/support/vi3/doc/vi3_esx35u3_rel_notes.html
The boot option force36BitMTRRMask is no longer required because of BIOS MTRR issues on certain platforms, ESX Server hosts previously failed to boot unless the VMkernel force36BitMTRRMask boot option was set to false. ESX 3.5 Update 3 enables full support for memory up to 64GB with no need to specify a boot option. As a result of this change, the force36BitMTRRMask VMkernel boot option is no longer supported. If the option is set, the result is no operation (NOP) and boot succeeds.
In conclusion, the resolutions are as follows:
1 ) Upgrade to ESX 3.5 Update 3 build 123630 or newer
2 ) To utilize more than 64GB of RAM, use a larger MTRR mask by disabling VMkernel.Boot.force36BitMTRRMask from the advanced settings.
To modify the MTRR mask configuration:
Log in to VirtualCenter as an administrator using the Virtual Infrastructure Client. (If not using VirtualCenter, log in to the ESX Server directly as root.)
From the Inventory click the ESX Server:
Click the Configuration tab.
Click Advanced Settings link.
Navigate to VMKernel>Boot.
Deselect the option for VMkernel.Boot.force36BitMTRRMask .
Reboot the ESX Server host for the change to take effect.
This information is perhaps a bit dated, but I know there are some older 3.x environments still in existence. If those environments are running on host hardware with more than 64GB of RAM installed, this could prove to be insightful.
