The Project from Hell
Three continents, incompetent management and an impossible deadline is a case study in how to NOT do a major upgrade.
- By Greg Neilson
- March 01, 2004
It all started so well; I had no inkling that things would turn sour
so quickly. Although I’d repressed the memory of my project from hell,
I’ve forced myself to recall all the lurid details so you can learn from
my pain. The project was a Lotus Notes/Domino upgrade, but as you’ll soon
see, the project issues encountered are universal.
The first I heard of this project was during a conversation with my practice leader in the consulting firm in which I worked. “Greg, we’re working on a cool proposal right now to upgrade to the latest version, and if we get it, we’ll need you to go back to Australia during part of the implementation.”
This all sounded fantastic—I was going to be utilized (the golden rule
of consulting!), do my first upgrade to this new version of Domino, and
better yet, get a trip back to Australia paid for by someone else.
|Reputable firms (and people) realize the
folly of committing to something they know they can’t
deliver. It’s much better to admit this and advise
what’s possible up front, for the sake of all concerned.
The reality of the project hit us when our assembled staff started work
on the client site. The client had business imperatives that forced a
delivery date that was straight-out unattainable. More respectable consulting
firms hadn’t wanted this work because they knew it would end in tears.
It appears that our firm was so desperate for the business, it had agreed
to deliver something it knew we couldn’t do. But the expectation was that
once we got started, it would be too late to remove us; and besides, eventually
things would probably turn out OK…
As we progressed through the project, it became clear that to get anywhere near the desired schedule, the team was going to have to work around the clock. However, the project was being run (“project managed” is too strong a term) by a practice leader—I’ll call him Bob—who made no attempt to engage us to committing to the project schedule. Thus, it became his schedule and not our schedule. This increased our frustration and anger, turning lunchtime breaks into mutiny sessions where we we’d voice our discontent.
With our assembled team of five, we were certainly capable of handling
the task at hand, but the unrealistic schedule meant there was little
commitment within the team to making the date. The other guys were contemplating
resigning there and then (this was a few years ago, when jobs were plentiful),
although the prospect of the free trip to Australia trip kept me from
similar considerations. Bob clearly was no leader—that would imply that
people would willingly follow him—but he was great at selling to clients.
There was a real death march feeling within the team that we were headed
for doom, and nothing we could do would change that.
|People have a variety of motivators, but
expecting a super-human effort from all is asking for
trouble. I believe it would have been far better for the
practice manager to come straight out and ask for our
commitment to the schedule, or perhaps have us develop
our own schedule. It would also have helped if he’d
been upfront about the challenging timeframes and perhaps
promising future rewards for successful project completion.
People are usually happy to work hard achieving a goal
they believe can be met (see Lesson 1), but they prefer
to be asked.
After a time planning the upgrade and collecting information about the
client’s environment, our implementation proceeded in earnest. We built
the first servers, migrated some of the data and were ready to proceed.
Then the fun really started.
The team split up to travel to the main client sites to perform the majority of the upgrade—Phoenix, Arizona; Dublin, Ireland; and Sydney, Australia (where I was). We were migrating 1,000 users to the new environment built on Windows NT 4.0 and Lotus Domino 5.01. At this time, our new manager—I’ll call him Abe—joined the project. In terms of the management hierarchy, he came somewhere between us and Bob, our practice lead. We hadn’t worked alongside him before, but we knew he had an extensive technical consulting background. But unfortunately, we’d soon learn Abe wasn’t much of a people manager, either.
The next sign of trouble was when I called after noticing some configuration
changes in the environment with which I was unfamiliar. When I asked Abe
about this, he told me he’d made some design changes. When I asked him
if he could send me some details about what had changed, he said if there
was anything I wanted to know I could call him and ask!
|In a leadership role, you have to have faith
in your people—even when you’re sure you know
better. In that case, you’d do much better coaching
your staff by asking open questions about the situation,
rather than straight-out dismissing their technical judgment.
Not only do they feel they’re part of the agreed
solution, they’ll know better next time.
The 18-hour time difference between Sydney and Phoenix makes that suggestion
impractical, and the arrogance of coming in at the last minute and changing
everything was astounding. Abe did have a strong technical background,
and it’s likely that these were sound ideas; at the same time, this lack
of confidence in his own staff wasn’t conducive to an ongoing productive
As we proceeded further, we were engulfed by the internal politics of the client. The majority of the technical research and development folks were in Dublin, with the marketing staff based in Phoenix. Our project sponsor for the company that hired us was also in Phoenix and had made project decisions without knowing or caring how this would affect users. Instead, he elected to hide behind the hired consultants (in other words, our team) and have us deliver bad news. We weren’t able to act on the complaints from those whose ability to work was affected by the changes; we just had to do the dirty work and listen to the resulting profanity. Any further changes were going to have to be debated internally in the company. But given that these were usually directives from “on high,” nothing was going to be changed without a fight.
For example, the technical staff in Ireland would have much less system access in the new environment. This sounds great from a CIO viewpoint, but some of this access was needed for them to do their jobs. (On the other hand, many of them currently had more access than they really needed, and some of their complaints were more ego- than need-driven.)
Having our project team communicate these decisions onsite to the Dublin staff wasn’t much fun. The familiar refrain of “(Bleeping) Americans” was all-too-frequently uttered by the Irish staffers. They had little respect for decisions made at the Phoenix headquarters, and we were stuck in the middle, attempting to implement the already-agreed-to project.
|For an ambitious project like this to succeed,
it’s incumbent on the project sponsor to take a
leadership role to explain to the project stakeholders
(that is, everyone likely to be affected by the project)
what’s happening and why. Being invisible and leaving
this to the project team to deliver the news—good
and bad—is a recipe for disaster.
As we went through our migration, the time zone issue really started
to bite us. The different time zones of the three global locations made
it impossible to have a single time during the day for a status review
meeting. With team members working long days to perform the migration,
having one of us wake at 2 a.m. for a teleconference was never a practical
idea. The best we could hope for were updates between two of the locations
at the start and end of each day, and hoping the information didn’t get
mangled as it was passed around, like a game of “telephone.” There’s nothing
worse than trying to be part of a team implementation when you don’t know
what everyone else is doing. With no regular updates, we had no idea of
what progress had been made in the other locations, what decisions had
been made, what problems had been encountered or how they could be fixed.
This was particularly important because this was a new version of the
product, and for all of us it represented our first production deployment
Ultimately, we kind of met the committed live upgrade date. But it was
another two weeks until they had a completely stable environment, and
for some staff, this interim period had to be frustrating, as their productivity
would have been severely affected. E-mail mostly worked from the beginning;
it was the applications, most of which were developed internally, that
took the extra time to be completely migrated. Of course, e-mail’s important,
but for many of the company’s employees, these other applications were
critical to their jobs. From our consulting firm’s point of view, more
hours meant more dollars for us, since this was a time-and-materials contract.
|This lack of project planning was unforgivable
given the global nature of the project to be delivered.
Before we all went off in our separate directions, we
should have had a plan for how we all would keep updated
on the project status. One answer might have been staggered
teleconference status meetings, so that staff at different
locations had a chance to participate in every second
meeting. In between times, having minutes distributed
from these status meetings would have been useful.
Within three months of project completion, most of us had left the company.
Not long after that, the consulting firm itself had gone into bankruptcy
and folded. But I did get my trip to Australia, I guess. Alas, my poor
workmates that were originally excited about their trip to Ireland saw
little more than the Dublin airport, their hotel room and the client site.
It was formative experiences like this that convinced me to take a management
role myself. I’m not saying that I haven’t made—or won’t make—mistakes,
but I’d like to think that these will be nowhere near the magnitude of
those I experienced on that job. Seeing things done so poorly was source
of motivation for me to demonstrate that perhaps I could do better.