The 64-Bit Question
Itanium servers, which utilize 64-bit architecture, are ready for the datacenter. This guide can help you decide whether your company should consider adding these high-muscle machines.
Here’s a riddle:
- By Bill Boswell
- July 01, 2002
What’s a little bigger than a Wheat Thin, has
more transistors than the adult population of the United States, draws
as much power as a small vacuum cleaner, and costs as much as a top-quality
HDTV? Give up? It’s Intel’s 64-bit McKinley CPU, soon to be released as
Itanium 2. McKinley is the newest member of the IA64 (Intel Architecture
64-bit) processor family.
After years of bluster and prototype product releases, 64-bit Intel-based
servers and workstations are finally ready for real-world production applications.
The choice of an operating system for these is relatively limited. Five
Linux distributions (http://www.linuxia64.org) have IA64 versions, and there’s
an IA64 version of HP/UX, which makes sense because Hewlett-Packard was
the co-developer of the IA64 architecture. As for Windows, there’s an
IA64 version of Windows XP and two members of the upcoming Windows .NET
Server family, Enterprise Server and Datacenter Server, will have IA64
Let’s see take a look at the reasons why vendors of databases and other
data-intensive applications are starting to migrate their apps to native
IA64 versions and why we need to be ready for the new technology when
it arrives in our server rooms.
Supercharged Memory Addressing
An IA64 machine’s biggest advantage comes from its ability to directly
address boatloads of memory. IA32 machines can address only 232 bytes
of memory (about 4GB). Windows 2000 Advanced and Datacenter servers support
the Physical Address Extension (PAE) in the Pentium processor, which opens
four more memory lines for a total of 236 bytes, or 64GB; but PAE memory
handling is relatively clumsy and requires applications to use function
calls in the Address Windowing Extensions (AWE).
IA64 machines, on the other hand, have a much larger address space and
don’t require special gymnastics to use it. The current crop of IA64 processors
has a 44-bit address bus that supports 16TB (terabytes) of virtual memory.
IA64 Windows divides this memory space in half, giving 8TB to user applications
and 8TB to the operating system.
Physical memory is limited due to constraints imposed by the chipset
and motherboard design. The 460GX chipset that accompanies the original
Itanium limits memory to 16GB per processor. McKinley uses the new i870
chipset, which supports 64GB of RAM per processor and also supports a
total of 512 processors. (Non-OEM versions of IA64 Windows .NET Datacenter
Server support 64 processors. IA64 Enterprise Server supports eight.)
The first McKinley processor has a 1GHz clock speed, which may seem slow
compared to the 2.4GHz Xeon, but McKinley has a 128-bit system bus running
at 400MHz with on-board L1 and L2 and up to 3MB of L3 cache. The L2 and
L3 cache have a 5-tick latency, nearly instantaneous when compared to
the time needed to access the external cache on a Pentium. Also, 64-bit
applications compiled with suitable optimizers will run faster than their
32-bit counterparts, even though the IA64 processor is slower, thanks
to the superior pipelining and larger number of registers in McKinley.
The i870 chipset also supports Infiniband, the successor to PCI for I/O
architecture. Infiniband provides a true channelized switching fabric
rather than the shared memory bus used by PCI. Each Infiniband device
gets its own clear communications path to the CPU and main memory with
stratospheric speeds up to 6GBps. Visit www.inifiniband.org for details.
There’s more to the IA64 story than simply a more capable processor. IA64
machines have a variety of new features, including an improvement to the
When you start an IA32 machine, it goes through a complicated ballet
trying to locate a device with bootstrap code (floppy, CD-ROM, fixed disks,
or PXE.) Changing the search order requires modifying CMOS settings or,
if you have a newer BIOS, selecting from a special startup menu. Once
the system locates a bootable device, it makes a series of INT13 disk
calls to find an active partition with a boot sector. Executable code
in the boot sector locates a secondary bootstrap loader that actually
loads the operating system. For DOS and Win9x, the secondary bootstrap
loader is Io.sys. For the NT family of products—including Windows 2000,
XP and .NET—the secondary bootstrap loader is Ntldr.
To do its job, Ntldr relies on hardware information gleaned by Ntdetect.
com and a path to the operating system kernel files stored in Boot.ini.
Ntldr also requires an SCSI driver if the SCSI interface doesn’t have
a BIOS. The SCSI driver is stored in Ntbootdd.sys. If any of these files
aren’t present or they get corrupted or overwritten, the operating system
won’t boot. Anyone who’s spent hours of their youth trying to configure
a multiboot Windows machine will attest to the clumsiness of this IA32
IA64 machines avoid all that ugliness by storing boot information in
non-volatile RAM (NVRAM), also called firmware. These firmware settings
are controlled by the Extensible Firmware Interface, or EFI. For the full
EFI specification, see developer.intel.com/technology/efi/EFISpec_v102.htm.
Because the initial bootstrap code is stored in firmware, IA64 Windows
doesn’t need Ntdetect.com or Boot.ini. It still requires a secondary bootstrap
loader, though, which is called Ia64ldr.efi. The .efi extension indicates
a file executable in the EFI operating environment.
The EFI contains a small command processor, or shell, accessed via a
boot menu. Here’s an example menu:
EFI Boot Manager ver 1.02 [12.36A]
Please select a boot option:
Microsoft Windows .NET Standard Server
EFI Shell [Built-in]
Boot option maintenance menu
The EFI shell recognizes a fairly comprehensive list of commands. You
can get a partial list by typing “?” at the shell prompt. For a full list,
When the EFI Shell first loads, it lists the available partitions on
the drives. You can get the same list using the MAP command. Here’s an
example MAP listing for a machine with several partitions:
Device mapping table
fs0 : VenHw(Unknown Device:80)/HD(Part1,SigBD1C1A20-685C-01C1-507B
fs1 : VenHw(Unknown Device:80)/HD(Part5,Sig1FD4472C-D516-11D5-8473 806D6172696F)
fs2 : VenHw(Unknown Device:FF)/CDROM(Entry1)
blk0 : VenHw(Unknown Device:00)
blk1 : VenHw(Unknown Device:80)
blk2 : VenHw(Unknown Device:80)/HD(Part1,SigBD1C1A20-685C-01C1-507B 9E5F8078F531)
blk3 : VenHw(Unknown Device:80)/HD(Part2,Sig1FD44720-D516-11D5-8473 806D6172696F)
blk4 : VenHw(Unknown Device:FF)
blk5 : VenHw(Unknown Device:FF)/CDROM(Entry1)
The long number in a few of these entries is the partition’s Globally
Unique Identifier (GUID). Each partition has its own GUID. The blk# number
identifies the sequence of the partition on the disk. If a partition has
a file system that EFI can read (FAT, FAT32, and Joliet CD-ROM), it’s
assigned an fs# alias.
Inside the EFI shell, you can access partitions in the same way you access
logical drives in DOS. Enter the block name followed by a colon. For example,
enter blk3: to go to the third partition. If the partition has a file
system readable by EFI, you can list the files and folders using dir or
If you have a new machine with no pre-installed operating system or you
want to install Windows as a second operating system, you must use the
EFI shell to launch Setup. IA64 systems don’t recognize El Torito bootable
CDs. To initiate Setup, select the fs# alias representing the CD-ROM,
then launch the Setup boot loader, Setupldr.efi.
More Powerful Partitioning
Another major feature of IA64 machines involves a wholesale change to
the way disks are partitioned. A standard IA32 computer uses the venerable
Master Boot Record (MBR) to identify partitions. A partition table in
the MBR lists the Cylinder/Head/Sector (CHS) location for each partition
along with partition-type information (primary, extended, non-DOS and
so on) and whether the partition’s active (bootable).
An MBR-based partition table has a maximum of four entries, forcing vendors
to tuck information in crannies between the MBR and first partition, install
hidden partitions or to load proprietary partition managers. This chaos
goes away on IA64 machines, thanks to a new partition scheme called the
GUID Partition Table (GPT).
On a GPT-based disk, each partition is assigned two GUIDs. One of them
uniquely identifies the particular partition. The other represents a partition
type. There are only a handful of recognized partition types. I’ll talk
about them in just a moment.
A GPT can hold up to 128 partitions. A mirrored copy of the GPT is stored
at the end of the drive, which avoids another problem with MBR disks,
where a single failed sector can render the drive unusable. There are
no hidden partitions, no special disk structures, no arcane rules for
creating logical drives, no hidden OEM partitions, and no undocumented
machinations for booting multiple operating systems.
GPT disks also support very large partitions thanks to a 64-bit Logical
Block Address scheme. A logical block corresponds to one sector, or 512
bytes, yielding a maximum theoretical capacity of eight zettabytes, enough
storage to assign a name to every star in the universe. (See physics.nist.gov/cuu/Units/prefixes.html
for a list of scientific notation prefixes.) IA64 Windows XP and .NET
support volume sizes up to 16 exabytes based on NTFS limitations.
Standard GPT Partitions
Here’s a brief list of the standard partition types supported by GPT disks
and what they’re used for.
EFI System Partition (ESP) Each operating system on an IA64 machine
requires a secondary bootstrap loader. The path to this loader file is
stored in firmware and appears on the EFI Boot Menu. As I mentioned previously,
the secondary bootstrap loader for IA64 Windows is Ia64ldr.efi.
Secondary bootstrap loaders can’t be stored on the same partition as
the operating system. EFI only has a FAT/FAT32 driver and the operating
system partition could be formatted with NTFS for Windows or EXT2/EXT3
for Linux. Instead, secondary bootstrap loaders are stored in a special
partition called the EFI System Partition (ESP). The ESP is formatted
IA64 Windows stores Ia64ldr.efi in a folder called \EFI\Microsoft\WINNT50.
This folder also contains a file called Fpswa.efi. This is the Floating
Point Software Assistance handler, which contains floating-point exceptions
required by the operating system. (See developer.intel.com/design/itanium/
downloads/245415.htm for more information.)
If you run multiple copies of Windows .NET or XP on an IA64 machine,
each copy of Ia64ldr.efi gets its own unique folder in the ESP. For example,
the second copy would be stored in \EFI\Microsoft\WINNT50.0, the third
in \EFI\Microsoft\WINNT50.1 and so on.
The ESP uses 1 percent of the disk space with a minimum size of 100MB
and a maximum size of 1GB. A GPT disk can contain only one ESP. The ESP
doesn’t appear in the Disk Management console and not as a drive in Explorer.
Microsoft Reserved Partition (MSR) In IA32 Windows, an MBR disk
is called a basic disk. When you convert a basic disk to a dynamic disk,
the partition information is transferred to a Logical Disk Manager (LDM)
database located in the last cylinder of the disk or the last 1 MB, whichever
is smaller. All dynamic disks on a machine have an identical copy of the
LDM database. This permits them to participate in shared volume configurations
such as RAID 1 (mirroring), RAID 0 (striping), RAID 5 (striping with parity),
and volume spanning.
IA64 versions of XP and .NET treat a GPT disk as a basic disk. If you
convert a GPT basic disk to a dynamic disk, the GPT information is transferred
to the LDM database. The GPT specification doesn’t permit the kind of
ad hoc partitioning used to hide the database on IA32 machines, so IA64
Windows automatically creates a Microsoft Reserved Partition (MSR) to
store the LDM database. The MSR partition is created even if the GPT disk
is left as a basic disk.
The MSR partition size depends on physical disk capacity. On disks up
to 16GB, the MSR is 32MB. On disks of more than 16GB, the MSR is 128MB.
The MSR is formatted as FAT16. It isn’t exposed by Explorer, but can be
seen in the Disk Management console.
Microsoft Data Partition This partition type is assigned by IA64
Windows .NET or XP when creating new partitions using Explorer. Other
operating systems can see the contents of Microsoft Data Partitions unless
you use NTFS permissions to lock them down.
OEM Partitions Some vendors love to ship proprietary BIOS setup
utilities and diagnostic tools in hidden partitions. For them, the GPT
specification includes a special OEM partition type. Ordinarily, an OEM
partition doesn’t appear as a drive in Explorer, but can be seen in the
Disk Management console. Vendor utilities find the partition using its
Working and Playing with Others
IA64 versions of Windows must boot from a GPT disk. However, they
permit using MBR to partition additional disks, although this feature
is primarily for backward compatibility so that you can take an MBR disk
from an IA32 machine and read it in an IA64 machine. IA32 versions of
Windows .NET and XP can’t read a GPT disk, so you can’t do the reverse
and remove a GPT disk from an IA64 machine and put it in an IA32 machine.
IA64 Windows can’t partition removable media disks such as Jaz, Orb or
Zip using GPT. The GPT specification describes these as superfloppies
and only permits MBR partitioning. Detachable drives such as Universal
Serial Bus (USB) and IEEE 1394 drives, or shared SCSI/Fibre Channel drives
in a cluster, must also be partitioned as MBR rather than GPT.
A GPT disk could potentially be damaged by utilities that expect to see
and write to an MBR. The GPT specification anticipates this problem and
requires including a protective MBR on the first sector of a GPT disk.
This MBR performs no real function other than to act as a sacrificial
goat for disk utilities.
They Ain’t Cheap
The first McKinley servers will be very expensive. The CPUs themselves
cost upward of $4,000 for models with a 3GB L3 cache; the advanced chipsets,
high-speed memory, 64-bit PCI adapters and Infiniband peripherals (when
they come to market) also cost quite a bit more than their 32-bit counterparts.
Still, lots of top-flight technology is waiting in the wings to take advantage
of 64-bit processing, and prices will decline as volumes improve. Even
so, widespread IA64 adoption may take a few years. Look for the first
production 64-bit applications (mostly large database applications) to
appear near the end of this year. By the end of 2003, mainstream applications
will have been ported to native IA64 code. By 2006, it’s possible that
you could be retiring the last of your feeble quad-Xeon servers.