The decisions you make up front about hardware, site
design, and operational policies can make the critical
difference in your Exchange installation.
Best Practices of Exchange Planning
The decisions you make up front about hardware, site
design, and operational policies can make the critical
difference in your Exchange installation.
- By Kevin Kohut
- February 01, 1999
Ask any real estate agent what the three most critical
factors are to consider in buying property, and the answer
youll get is, Location, location, location.
Likewise, the most critical factor of a successful Exchange
deployment is planning, planning, planning.
Proper planning is important to any technology project;
for a Microsoft Exchange Server deployment, its
essential. Many aspects of Exchange Server are difficult
to change after the fact (try renaming the organization
after youve installed a few servers in the site).
If things arent configured correctly, Exchange suffers
performance degradations at the very least, and at worst,
you can expect data loss accompanied by hours of rebuilding
As Ive worked on many Exchange projects, Ive
discovered several best practiceskey items that
are essential to successfully deploying and maintaining
Exchange. By no means comprehensive, this collection of
proposed hardware configurations, site design recommendations,
and policy suggestions can save your headaches while ensuring
a successful Exchange deployment.
Under-powered or incorrectly configured servers account
for most of the Exchange problems Ive encountered.
Exchange adores RAM, it thrashes hard drives, it pounds
CPUs, and it loves stuffing packets through the network
wire. If any of these areas are lacking, they can contribute
to performance degradation. Upgrade one component, and
Exchange will hit a bottleneck somewhere else. A holistic
approach is definitely in order here.
Double Up Those Processors!
Lets start with the processors. Obviously, the
faster the CPU, the better the performance. But since
Exchange is constantly processing short, I/O-intensive
tasks, like writing transaction logs to disk, youll
see even better performance with multiple processors.
Youre better off with a system running two CPUs
at 200MHz than a system running just one processor at
450 MHz. Theres a point of diminishing returns,
however. Even though Windows NT can support up to eight
CPUs, Exchange cant do much with more than four.
I typically recommend a dual-processor Pentium Pro 200MHz
Get Greedy with RAM
As for RAM, dont be stingy. Use 128M as an absolute
minimum. Ive recommended no less than 256M to my
clients, and because RAM is so inexpensive these days,
Im starting to push for 512M, or even a gigabyte
of RAM! The more physical memory that Exchange can use
for data, the less it has to write to disk. You cant
put too much RAM in an Exchange server.
The Right RAID
To fully appreciate how Exchange uses disk drives, it
helps to know a little about how data is processed. When
a user sends an e-mail message through Exchange, the system
first writes a transaction log to disk. Note that the
actual information store database isnt accessed
at this point. When the system has some idle CPU time,
the transaction logs that have accumulated are written
to the actual database. This method of saving data provides
both fault tolerance and performance, but only if the
disk drives are configured correctly.
Because transaction logs are written sequentially to
disk, they benefit most from a single spindle drive configuration,
dedicated for the log files.
Log files are also crucial for recovering from a system
failure, so they need a high level of fault tolerance.
A hardware RAID 1 set-up fits the bill nicely. It mirrors
the data, providing excellent fault tolerance, and each
disk in the mirror set can write the data fed to it using
just one head.
The information store databases, on the other hand, are
written randomly. They also can get quite large, typically
15G or greater. A RAID 5 configuration, with as many physical
drives as possible, provides the best performance for
the database files. For even faster performance you can
configure write-back cache on the RAID 5 system to 100
percent (in other words, all write and no read caching).
Do this only if your array has reliable battery backup
for the cacheif you lose the data in the cache,
the transaction logs are impossible to replay after a
Finally, the NT operating system should be on its own
physical disk, preferably mirrored for fault tolerance.
This disk will also store the paging file, so make sure
it has lots of space.
An often-overlooked component of an Exchange server is
the network interface card (NIC). All communication between
Exchange servers within a site is handled through Remote
Procedure Calls (RPCs). These RPCs have a voracious appetite
for network bandwidth. To avoid bottlenecks at the NIC,
make sure your servers have NICs that are at least as
fast as the network backbone. I like to put in nothing
slower than 100 Megabit Ethernet.
Table 1 shows an Exchange server configuration that I
can feel good about recommending.
1. The author's minimum recommended Exchange
||Dual Pentium Pro, 200 MHz
|RAID 1 controller &
||To support four physical
drives, in two mirror sets, one set for
the OS and paging file, the otherset for
he transaction logs.
|RAID 5 subsystem
||Dual channel controller,
with at least five physical drives: four
for data and parity and one hot-swap spare.
|Network interface card
||100 Megabit Ethernet or
Ask three Exchange gurus how to design a site optimally
and youll get three answers. Its true that
creating an architecture for an Exchange environment is
more art than science; however, some important issuesdealing
with server roles, replication, and site connectorsshould
always be considered.
Exchange does more than just deliver mail. It manages
communications between sites; it handles Internet mail;
it keeps public folders in sync among all the servers
in the site; it expands global distribution lists into
individual mailbox addresses; and each Exchange server
has to keep tabs on all the other servers in the organization.
Many of these functions can be relegated to a specific
server, reducing the load on other servers. The idea is
to have two types of Exchange servers: those that handle
user mailboxes and those dedicated to certain other tasks.
The Job of the Mailbox Server
Mailbox servers should be just thatthey should
run only those services necessary for handling Exchange
mail: Directory Service, Information Store, Message Transfer
Agent, and System Attendant. Mailbox servers shouldnt
be configured with site connectors or as Internet mail
gateways. I also like to keep public folders off servers
that have user mailboxes. This isnt always practical,
as it requires additional servers. (More on this in a
If your Exchange infrastructure includes Internet mail
connectivity, you should set up at least one Internet
mail gateway. This machine will run the Internet Mail
Service (IMS), and be dedicated to processing SMTP mail.
IMS machines can be configured to only handle incoming
or outgoing mail, but they dont support true load
balancing. In most cases, one machine is enough.
In a multi-site environment youll need to set up
at least one connector between each site. You should do
this using bridgehead servers, which are machines dedicated
to running site connectors. You can run more than one
connector on a bridgehead server but, depending on the
type of connector, you might have to modify some registry
keys (this applies to systems running more than two X.400
Another function you should offload from mailbox servers
is distribution list expansion. When mail is addressed
to a global distribution list, Exchange must first expand
that list into individual mailbox addresses. A single
e-mail message, sent to a list with 200 addresses, can
bog down several servers as Exchange tries to figure out
which server each address belongs to. Configuring large
distribution lists to expand to a particular server helps
ease congestion and allows quicker delivery of the other
mail queued in the MTA.
Public Folder Particulars
Lets get back to public folders. Every server within
an organization that has a public information store automatically
sends one or more status messages to other public folder
servers every 12 hours. These messages update other servers
on changes that have occurred within the public folder
structure since the last status message. Even if a servers
public folders are empty, it still sends the status message
and receives messages from other servers. To keep this
status message traffic at a minimum, remove the public
information store from every server that isnt actually
using it. You can do this through the Exchange Administrator
program, by simply highlighting the Public Folder object
of the appropriate server and then pressing the Delete
key. Youll get a warning message, but once you delete
it, all the data is gone for good (unless you restore
from a backup tape).
Public folders send more than just status messages to
each other; they also send replication messages. By default,
these messages are sent every 15 minutes. You can cut
replication traffic by increasing the default interval.
You do this through the Exchange Administrator program.
First, select the server you want to configure, then double-click
the Public Folder object to bring up the Properties window.
Next, click the Advanced tab, and change the replicate
always interval to a higher value. The time interval you
choose will depend on how often public folders are changed;
but make it at least 30 minutes. Unfortunately, youll
have to configure servers one at a time.
Exchange affords many ways of controlling its operation,
from setting limits to backing up data to working with
other mail systems. In many cases these options are available
at various levels along the organizational hierarchy.
Mailbox limits, for example, can be set at the individual
mailbox level all the way up to the entire organization.
The two most important limits to establish are mailbox
size and message size. Setting proper mailbox limits ensures
that you wont run out of disk space unexpectedly.
This is critical, because Exchange will freeze a system
if it cant write to the information stores. Exchange
can also freeze when the MTA attempts to process too large
a message. Thats why limiting large attachments
is also critical.
The best way to keep users from sending huge attachments
that can choke the server is to enforce message size limits.
Based on real-world observations, Ive discovered
that as messages (which include attachments) approach
15M, the MTA starts to have troubles. I recommend that
10M be the absolute maximum. For Internet mailthat
is, mail being handled by the IMSits good
to go even lower. I usually suggest 2M. Users may not
like this restriction; their friends wont be able
to send them the latest video clip. I counter this with
a short lesson on how to use FTP to transfer files.
|Figure 1. Limiting message size
is crucial to keep Exchange from freezing.
Once youve figured out message size limits you
can then determine the mailbox limits. There are three
types of mailbox limits that can be set: a soft
limit that simply generates an e-mail to the offending
user, and two levels of hard limits. The first
hard limit prevents the mailbox from sending mail, the
most restrictive hard limit prevents the mailbox from
sending or receiving mail.
|Figure 2. Mailbox limits help
prevent running out of disk space.
Ideally, youll determine these limits before you
spec out the server hardware. Then all you have to do
is multiply the most restrictive limit by the number of
mailboxes you plan to put on that server. Now use that
figure to determine the total disk space youll need
for the private information store. If this server is going
to contain public folders, make sure you allot some space
for the public information store. For example, if you
want to put 600 users on a server and youve decided
on a 40M mailbox limit, youll need 24G for the private
information store. If you anticipate the public store
to store around 2G of data, you should start with a 27G
stripe set for the databases; I recommend adding at least
another 4G to accommodate growth.
Unfortunately, server disk space is usually determined
and installed long before mailbox limits are considered.
In this situation, its not that complicated to calculate
limits working backward from total disk space.
First, figure out how much space is available to the
private information store. Start with the total stripe
set, and subtract what the public store is using. From
that figure, take off a few more gigabytes to leave some
breathing room. Next, divide the available space by the
number of mailboxes you plan to put on the server. Lets
say you have an Exchange server with 700 mailboxes on
it. This server also has public folders that contain about
12G of data. The RAID 5 stripe set has 54G of total disk
space. Subtract the 12G that the public store is using
from the 54G total, which will give you 42G. Take off
another 7G to allow for growth, and that leaves 35G available
for the private information store. Divide that by the
700 mailboxes, and you end up with 50M per user.
Now that youve determined the send and receive
limit, you can figure out the other mailbox limits. A
good rule of thumb is to double your message size limit,
and use that figure to stagger your mailbox limits. For
example, if your message size limit is 5M, double that
to get your stagger amount of 10M. So, if your mailbox
send and receive limit is 50M, youd set your send
limit to 40M and the soft limit to 30M.
Mailbox and message limits help prevent problems with
Exchange. A proper backup strategy ensures that you can
recover data after a problem occurs. Everyone knows the
need for routine backups, and most Exchange environments
Ive seen have a backup process in place. But just
dumping files to a tape drive wont guarantee that
Exchange data can be recovered.
You can back up Exchange data either through an on- or
off-line process. An off-line backup is simply a file-level
copy to tape. Exchange services must be shut down for
an off-line backup. And if youve got a large information
store, it can take a couple hours to complete. The biggest
problem with an off-line backup is that the Exchange databases
may not be in a consistent state. Restoring data from
an off-line backup of an inconsistent database requires
that all the transaction logs and check files are restored.
If any of these files is out of sync or otherwise corrupted,
you cant be sure of accurate data.
An on-line backup uses Exchange services to get to the
data. Exchange isnt shut down, and users can access
their mailboxes without interruption. But the best advantage
of an on-line backup is that it ensures the databases
are consistent. You dont have to worry about transaction
logs or check files. The backup software you use must
support Exchange to do on-line backups. When Exchange
is installed, it automatically patches NT Servers
built-in backup program, NTBACKUP, so that it supports
on-line backups. ArcServe, Backup Exec, and other popular
NT backup software also include agents for Exchange on-line
Even if youre using on-line backups, an unexpected
system crash in the middle of the day may not be recoverable.
To survive such a catastrophe, you must disable circular
logging on Exchange. To understand how circular logging
works, lets revisit how data is processed by Exchange.
First, Exchange writes data to transaction log files.
These files are always the same sizeExchange creates
new log files as needed. Exchange also maintains a checkpoint
file, which contains information on which transactions
have been committed to the information store databases.
With circular logging enabled (which is the Exchange
default), Exchange overwrites existing log files after
the transactions they contain are committed to the store
databases. Disabling circular logging forces Exchange
to always create new log files, rather than overwriting
existing ones. This ensures that you can play back all
the transactions since the last backup.
|Figure 3. Circular logging is
enabled by default. Make sure you turn it off to ensure
data recoverability after a system crash.
Exchange is a complex animal. To properly deploy it,
not only do you need a solid understanding of Exchange
itself, but youll also need expertise in Windows
NT, networking protocols, and hardware design, as well
as an understanding of the non-technical political issues.
Remember: The three most important things to do in deploying
Exchange are to plan, plan, plan!
- Plan your server hardware.
- Plan your site design.
- Plan your operational policies.
The best practices Ive presented in this article
just scratch the surface. But theyll put you on
the right course to build a solid framework for your Exchange