Tales from the Trenches: Screams from the Server Room
Mix a $50,000 server, NetWare, and a nitwit consulting company to get a true recipe for catastrophe.
- By Barry Shilmover
- October 01, 2000
A few years back I ran a network at a local high school,
a school that was heralded as a “lighthouse” project to
show how computers could be used in education. It was
indeed sophisticated for its time—the network consisted
of 350 diskless workstations booting Windows 95 from the
server, 25 laser printers, seven file/print servers running
Novell NetWare 4.0, and an IBM AIX server for Internet
and Web capabilities. All the systems were networked using
both copper and fiber token ring, and nearly 80 percent
of the library’s resources were online (all this for about
However, the company that had originally installed this
network had done such a terrible job that three years
later, when I started, I was still fixing installation
and configuration problems. More about this company shortly.
After running the network for about a year, we decided
that a new, powerful server was required to replace the
aging administration server. Since the school board decided
that the only systems we could purchase were from Compaq,
we opted for a high-end ProLiant server. Unknown to us,
the company that had originally installed our network
was the same company (under a new name) as the one selected
as Compaq’s official vendor. Because of the way our network
was configured, each of the seven main servers had two
token ring cards in them and acted as routers to shuttle
information between the different networks. Since all
our servers were IBM systems and had only MicroChannel
as their data bus, we opted to let the “trusted” company
make the decision on which to purchase. They recommended
IBM bus mastering cards, priced at over $1,000 each.
During the summer the server arrived, and I installed
it with Novell NetWare 4.11 in place of the existing administration
server. The server ran for more than a month—that is,
until teachers and administrative staff came back from
the summer break. At that point, one of the server’s hard
I quickly called in support, and they had someone at
our door within 30 minutes informing us that he was here
“to replace the failed hard drive on the ProLiant server.”
Wow, great service, right? Wrong! He walked into the server
room, which contained eight IBM servers and one Compaq
server, and asked, “So which one needs the hard drive
At this point, I left the room. After putting out a few
fires and explaining why our $50,000 server was down,
I returned, only to find that he hadn’t figured out how
to remove the old hard drive yet. It eventually took him
six hours to replace the hard drive.
I stayed through the night to get the OS, applications,
and data files installed on the server so that it could
be used in the morning. I personally watched the server
console the next morning as user after user logged on
and started working. Life was good.
Later that morning, my pager beeped as I was getting
into my car to go home and get some sleep. The server
Back in the server room, the system was so locked up
that the soft power switch on the front of the machine
no longer worked. Mumbling words that can’t be repeated
here, I reached behind the system and rebooted the server.
My mouth dropped when I got an “invalid system disk” message.
I then attempted to recover the system using all the tools
in my arsenal, such as Norton Utilities, but it was gone,
dead, a doorstop. At this point, I formatted the hard
drive and started over, thankful that I had the hindsight
to back up the system before letting users connect to
With the system back up, I watched as user after user
connected and started working. When the seventeenth user
connected, the system crashed again. Rebooting the server
returned the same error as before, and we were back to
Top Gun in Action
At this point, the only idea I could come up with was
that this was a hardware problem. The vendor was again
contacted and this time made aware of the consequences
should they send the moron they’d sent before. This time,
the company’s top NetWare/network consultant arrived to
fix the problem. His first question: Could we duplicate
“Just use the system and it will crash,” I replied, adding
“I think it’s a hardware problem.”
His response: “No, the hardware is fine. It’s your configuration
that’s at fault.”
“Where’s the window in this place?” I mumbled.
“I thought you were running NetWare,” he said.
“Never mind,” I said as I walked out. I was in the process
of informing my supervisor that I believed it was a hardware
conflict between the system and the IBM token ring cards
and asking if installing a window in the server room was
in the budget, when the consultant found me. “Your NetWare
installation is flawed. I’m going back to the office to
get our testing copy. Back in a flash.”
While he was away, I went home and brought over a Pentium
PC that was idle at the time. I decided to install Windows
NT Server 4.0 on the system, since this was the standard
the board of education was recommending. (I’d been on
the team that had evaluated and recommended NT). I installed
two Madge.connect token ring cards along with NT, the
applications, and the data files. Since there was no way
this system was going to remote-boot the 35 admin workstations
(NT doesn’t do diskless stations well), I installed Windows
95 on them and had the admins and teachers back on the
network within 24 hours.
The following morning, the consultant berated me. We’d
apparently “stolen” their copy of NetWare. Unlike the
“we trust you” licensing with NT and Windows 2000, NetWare
broadcasts its presence on the network. If another copy
of NetWare is running with the same serial number, broadcast
messages start to appear everywhere on the network. What
had happened was that as soon as the consultant had installed
his copy of NetWare, the system complained that an exact
copy was installed on our CD-ROM system in the library.
“How could we possibly have done that?” I asked.
After a bit of detective work, we realized that this
new company was the same one that had originally installed
the network. They’d installed a 150-user copy of NetWare
on the server and were told that it wouldn’t be sufficient.
A 250-user copy of NetWare had been ordered and they’d
left their copy installed in the interim, then hadn’t
bothered to come back and re-install the newly ordered
software. Mystery solved, but problem still present.
Over the next month, the company had specialist after
specialist look at the system. Network captures were done
and sent to Novell, Compaq, and IBM tier 1 support. No
one seemed to know what the problem was. Even though this
wasn’t affecting the users (after all, they were working
off the Pentium server), spending time working with the
consultant was definitely slowing me down. Fed up, the
school informed them that if they couldn’t stabilize the
system within five days, they were to take the system
back and expect a bill for my downtime.
In an apparent bout of desperation, the consultant actually
asked me what I thought the problem was. Again, I said,
“The token ring cards.”
“That’s not possible,” he said.
“Just humor me and try it,” I said.
Two $300 cards were ordered and installed. We brought
the system online and allowed users to connect to it.
The 20th, 30th, 50th, 100th user connected without a problem.
The consultant had nothing to say and left about an hour
later. Since the admin/teacher team got to like the way
NT did things, that afternoon I formatted the drive and
installed NT on the ProLiant. We had no problems with
the system for the duration of my stint there, and the
company in question was removed from the approved vendor
About a week later, I got an email from the consultant.
He found out there were no problems when a single IBM
card was installed in a ProLiant, but the system would
crash if a second card were installed and routing turned
on. I was also informed that Compaq had known about this
problem for about four months.
What did I learn from this experience? I no longer rely
on other company’s suggestions for hardware. Although
it tends to be slightly more expensive at the onset, evaluating
the hardware beforehand can save a considerable amount
of time and money. Also, there are many so-called “experts”
in the IT field today. I wouldn’t hesitate to ask for
resumes and references before allowing anyone to work
on any of my systems.