The Director of Microsoft’s Certification and Skills Assessment Program educates us on benefits, new exams, and the quality question.
Donna Senko Talks Back
The Director of Microsoft’s Certification and Skills Assessment Program educates us on benefits, new exams, and the quality question.
Education lies at the heart of Microsoft’s certification
efforts. After all, without skilled people trained to
implement, administer, and manage Windows NT, most of
us might still be Novell customers. That’s why it
was appropriate that the Redmond company look to somebody
with an academic background when the time came to hire
an executive to direct the certification program.
Donna Senko holds a doctorate in German language and
linguistics. She has taught at the college and university
level in the U.S. and Germany. But she also has a business
background based in training, most recently in a lengthy
stint at Santa Cruz Operations. As group manager of training,
curriculum development, certification, and technical publications,
Senko restructured SCO’s “ACE” certification
program and re-designed its training and curriculum businesses.
Prior to that she held management positions at Hewlett-Packard.
Since January 1997, she has directed Microsoft’s
certification and skills assessment programs worldwide—an
effort that currently involves 360,000 participants in
100 countries.
That job hasn’t necessarily been easy. While the
certification program has experienced outrageous growth
in the past two years (from 100,000 in February 1997 to
360,000 as of October 1998), some MCPs have accused Microsoft
of pushing quantity over quality, cutting back on benefits,
and allowing unskilled people to obtain titles too easily,
thereby diluting the certification’s value. 
Editors Linda Briggs and Dian Schaffhauser caught up
with Senko during MCP Magazine’s TechMentor
conference to discuss all of these topics.
MCP Magazine: In a discussion
in the MCP Magazine online forums with [Microsoft’s
Marketing Manager Eckhart Böhme], one of the topics that
kept coming up was MCP benefits. The participants seemed
a little disgruntled about the level of benefits right
now; I’m sure you’ve heard that feedback yourself.
One of the things Ecki mentioned is that a survey is underway
now exploring benefits. Can you expand a little bit on
that?
Senko: Actually, we did
the survey last year, when we were looking at the whole
program and re-evaluating the program and its benefits.
We looked at two factors. One was the usage of the benefits
and how many [Microsoft Certified Professionals] use the
particular benefits. And the other was about what benefits
were the most important to MCPs. Based on those two factors,
we made the changes to the program.
So, for example, in the case of the [technical] support
benefit, we found the usage rate incredibly low. There
were a few people who were very interested in that benefit,
but something like one percent of the eligible people
actually took advantage of that benefit. The average usage
rate of those people was between one and two support calls
that they used out of the 10-pack they were given—really
low.
Also, we did a survey of over 2,000 MCPs worldwide—2,000
people who responded worldwide. And we had them actually
rate the different benefits that they receive and put
them in a ranking order of which benefits they consider
the most important. Based on that, we determined the new
benefits package. One thing that came out high on the
list was technical information and access [to it] through
Web sites—through the Internet. So that is the direction
we decided to start moving in. We’ve only just started.
Can you share with us at all what kinds
of things you plan to add to that MCP site?
This past year our goal was to set up the infrastructure
and start with some of the basics, so that people could
download their logos and use the transcript tool. That
was our starting point—with some technical information
[also].
The goal really is to use that as our pipeline of technical
information to the MCPs, because that’s the thing
that they’re really hungry for, and that’s the
best way of us providing it to them. So that’s the
direction.
It’s interesting that you found usage
of the technical support calls so low, yet now MCPs are
demanding that you give that benefit back.
Well, I have mixed feelings about that. I think with
individuals who were using them, they probably were one
of the very important benefits. But I do think that a
few people who are dissatisfied can sound like a number
of people. And so we really tried to take a broad look
at how we’re going to satisfy the greatest number
of people—what makes the most sense for the program
and for individuals worldwide.
MCPs who are part of the Certified Solution Provider
channel can get support that way.
Developing the MCSD Program
The new Microsoft Certified Solution Developer
track seems to be attracting a lot of attention. Are you
on schedule for releasing the new exams?
The major exam, the Solution Architecture exam [70-100],
should be in beta form in November, which has actually
been our target date. We have been saying Fall and it’s
[scheduled for] November right now. We’re on track
with that; the other exams will [be released] roughly
one exam into beta per month until they’re rolled
out. We feel strongly that we have a solid program in
place now, and we’re going to meet those commitments.
Can you talk about marketing efforts for
the new MCSD? There seems to be a general feeling that
with more marketing, the title would take off.
You’re going to hear a lot about the MCSD this [fiscal]
year. Now that we have the new program in place and [in
the process of] being rolled out, we have a number of
initiatives planned. One is Visual Studio Developer Training
Month [which took place in October, 1998.—Ed.].
We’ll be offering an exam voucher for any core exam
for the MCSD to any participant in that training wherever
the Visual Studio training initiative is launched.
That was initially planned in North America. But quite
a number of subsidiaries [outside the U.S.] have come
to us and expressed interest there too; so we’ll
honor the same commitment to the subsidiaries.
We’re going to ‘incent’ current MCSDs
to migrate to the new program. These are people who are
already in our program; we’re dedicated to helping
them move to the current technology. We’ll be having
promotions for them, probably beginning in the winter
timeframe. It will most likely be in the format of a two-for-one
exam promotion.
The bottom line is that it will be easier for individuals
who are currently MCSDs to migrate to the new program
than to update in the existing program. For example, rather
than going from the WOSSA exams to the [Windows Architecture
exams], we extended the life of the WOSSA exams, so that
people can skip right into the new program. We really
want to encourage everyone to move to the new program.
However, people will have through July 1, 2000 to move.
So there’s a lot of [leeway].
We also have an awareness campaign planned around it,
and a couple of other ideas, too, that by the end of the
year we’ll probably be rolling out. So this is really
the year that you’ll be hearing about the Solution
Developer.
Will we see something in the Visual Studio
product box about education and certification?
I’m not certain at this point.
What about working with large corporations
to encourage them to bring MCSDs up through the ranks?
One of the things that we’re doing at the same time
is expanding the definition of what an MCP is. [As
of October 1, 1998, people who pass a single Microsoft
exam, with the exception of Networking Essentials, earn
the MCP title.—Ed.] Because the new structure
of the MCSD program is really quite different than the
old structure, we want individuals to have a stepping-stone
from being an MCP to becoming an MCSD. So we have broadened
the definition of an MCP to accomplish that purpose. There
are other reasons too, but this really [encourages] people
to move on.
On Retirements and New Exams
Can you give readers an explanation of how
Microsoft decides when to retire an exam? For example,
the SQL Server 4.2 exams are still live, as well as 6.0
and 6.5. [The SQL 4.2 exams are now in the process
of being retired.—Ed.] We assume that’s
because the product is still well-represented in the marketplace.
There are a number of factors that we look at. One is
exactly what you just stated—how well the product
is still represented in the marketplace. Another thing
that could affect our decision is how close together releases
are and how radically different the releases might be.
We also look at the base of people who have certain exams
or what the ‘run rate’ is on a certain exam.
If a lot of people are still taking a particular exam,
then we feel there’s a need to maintain that exam
for a certain period of time.
Typically, for a major product, we would have two versions
of an exam available at the same time. For example, we
have Windows NT 3.51 and 4.0 exams right now. When we
release NT 5.0 exams, then we’ll probably retire
the 3.51 exams.
So we really look at a wide range of criteria: the product,
the status of the product, status of the certification
track, if it affects the certification, and the run rate
on the particular exam.
But at the same time, [we don’t want to allow] people
to stagnate. We try to find the right balance: Constantly
trying to move people forward into new technology, while
being realistic, too, as to which version of the technology
they might be using.
Are you starting to see people dropping
out of the program as they don’t re-certify? It’s
really only in the last year or so that you’ve been
retiring exams.
We found last year that we had a very low number of people
who didn’t upgrade. If you looked at the whole base
of MCPs, roughly one percent of the population was de-certified
last year.
The majority of people are staying in the program, which
is really exciting. We went back and analyzed that at
the end of last year, and we were very pleased with what
we saw.
Our current numbers reflect people who are current in
the program. We really want to make sure that people who
are not active in the profession don’t maintain a
certification for the wrong reasons. For the most part
we hope that people are going to move ahead.
There’s been some confusion about whether
readers should follow the Windows NT 4.0 certification
track or wait for the NT 5.0 track. Can you provide any
insights on the pros and cons of both of those approaches?
I would really recommend that individuals go ahead with
the Windows NT 4.0 track. This is our current technology,
and it will be around for quite a while. It’s a very
sound, stable technology. It doesn’t make sense for
an individual to wait until the next release, which is
several months out at this point. [Also], if someone is
current on NT 4.0, we always have incentive to migrate
them to the next version, which would be NT 5.0. If they’re
in the pipeline it’s easier to move up. There really
isn’t a good reason not to move forward with 4.0.
Can you expand on areas where we should
expect to see retirements over the next six to twelve
months in terms of exams?
No.
(She laughs.)
We have already stated that when NT 5.0 exams become
available, the 3.5 exams will retire. Other than that
I couldn’t give you the details on the plan right
now.
Someone asked during the [TechMentor Windows
NT 5.0] keynote whether there was going to be an NT 5.0
Server and NT 5.0 in the Enterprise exam.
We’re [going to be] doing a complete job task analysis—JTA—around
NT 5.0 and the job functions of the MCSE. So we don’t
know the answer to that yet. But we’re looking at
all of NT 5.0 very seriously, and that might mean some
revisions to the track in general—things might look
a little bit different than they do in the 4.0, for example.
We’re now doing that research; we’re several
months away from having the results.
Just to clarify for readers, a job task
analysis is what you do at the very beginning of a track.
Right. That’s what we did when we began looking
at [revamping] the Solution Developer track, and it took
us in a totally new direction in that particular case.
Because jobs evolve, the technologies change, and what
the individuals do is always changing. That’s how
we keep our tracks current. We’re not just replacing
exams all the time; we’re looking at the whole picture.
What are individuals doing in those jobs?
Do you have any plans to provide a certification
with an emphasis on database management work?
There is a data warehousing exam planned, but that’s
not to say that there will be a whole track planned around
that. That exam will be one of the electives for the MCSD.
We’ve looked at the whole database area, along with
a number of other areas. You have to realize that every
year, [Microsoft] product teams come to us and say, “You
know, it makes sense to do [this or that] certification.”
And we have to prioritize and say, “What is the most
meaningful for the marketplace today?”
So we’re very cautious about introducing new certifications.
We’ve introduced the new MCSD and the MCP+Site Building
[certifications] this year. We’ve looked at a number
of others. One [exam idea] that has come up several times
is in the area of databases—database administration,
most likely. But we haven’t made any definite plans
to move in that direction at this point. We want to keep
the message clear and make sure we’re getting the
top priorities first.
So product groups come to the certification
group and say, “Hey, do an exam on my product”?
Yes. It’s really refreshing compared to years ago
[when the certification group had to educate product groups
on the value of certification]. I think, in general within
Microsoft, people really understand the meaning of certification
and the value that it offers for supporting Microsoft
technologies.
Exam Piracy and Keeping Quality
We left this topic for last because we know
it’s near and dear to your heart—exam piracy.
First, now that you have the new exam NDA [non-disclosure
agreement] in place, have you had occasion to use it against
a company or individual?
We haven’t yet used it, although our intention is
to use it when it’s called for.
If I could just back up a second, the NDA is part of
a greater effort. We do consider exam piracy to be an
issue, and we’re confronting it aggressively on two
fronts. One front [involves] making some process changes,
such as adding the NDA. This will mean that people are
making a breach of contract if they disclose exam items.
Another process change is our retake policy. Now, when
individuals take the exam the first time, they can take
it a second time within one week; after that they have
to wait two weeks every time they want to retake the exam.
And there are some other process changes that we’re
looking into, as well as working with both of our testing
vendors [Sylvan Prometric and VUE] on increasing security
at the test sites.
On the other side, we’re using progressive techniques
in testing, which makes it a lot more difficult to steal
exam items. [With] simulations, adaptive tests, or any
type of testing in which individuals actually have to
do a task, there’s no way to steal that [exam question].
The only thing you can really do is walk away and say,
“You need to be able to do such and such a task.”
And you know what? Those are exam objectives, which we
already publish. So those aren’t secrets.
We want people to know: This is what you need to be able
to do to pass certain exams. [The exam process] is geared
towards people being able to do job functions. By using
some of those techniques, we believe we’ll mitigate
the problem somewhat.
Also, we’re now creating item banks for our exams
with many more items, so that the chance of [one individual]
seeing all the items is much lower. We’re [creating]
multiple forms of the beta exams, so that a person who
takes the beta exam doesn’t see all of the items.
We’re also seeding items, which means that the exams
are moving targets when we’re pulling items out and
putting items in.
We’re trying to hit this on a number of fronts,
because ultimately the value of the credential is the
most important thing. We want to support anybody—any
MCP—who has attained a certification by helping to
maintain the value and the integrity of the program and
of the certification that someone worked so hard to gain.
It’s very important to do whatever we can to [accomplish]
that.
Vendors are now selling exam simulation
products in which they’ve introduced product simulation
engines. So you’re saying, regarding the product
simulations, that if someone studies what these products
do and knows how to answer that question—“This
is how you add a user to the network”—that’s
the point.
That’s the point. Ultimately, we want people to
be able to do the task. And if they can do the task—in
this case, add the user—they’ll pass that particular
item on the exam.
If you learn to do the task, that’s really the key.
Obviously, the broader the focus, the better. Sitting
there with the real product is always the way to go, because
we can phrase a question or make derivations of a question
in a couple different directions. And so if you understand
the context and the broader scope, you’re more likely
to be able to pass the exam, as opposed to having learned
one particular way of doing something. Hands-on use of
a product—that’s the preferred way [to learn].
Are you concerned that some vendors are
undermining the value of certification?
I’m concerned that there is that perception in the
industry. I’m doing whatever possible to change that—where
there’s a problem, to try to get the problem fixed;
where’s there’s misinformation out there, to
try to get it corrected.
So is the problem that vendors really are
undermining certification, or is that the perception in
the MCP community?
I think there’s a mix. But I also know that you
can hear about something from one or two people, and it
looks like a big problem; but it’s not always that
wide scale. I’m trying to be objective about looking
at where we have problem areas and how we go about fixing
them. And we’re looking at things in a positive light:
OK, how do we move the program forward? How do we maintain
the quality? What’s the best way of getting that?
What should an MCP do who comes upon a “brain
dump site?”
I recommend that they e-mail [the URL] to [email protected].
We look at those. We’re aware of the major ones at
this point.
And you act upon that information?
We do. And as we move forward, we will be acting more.
We’ll be acting where we can and where appropriate.
Is there anything else you want to explain
to readers?
The message I would really like to get out there is that
the value of the certification and the integrity of the
program are really key to us. Those are things that are
very, very important to me, because I know how hard people
have worked to gain that certification. It’s because
of the value of it that the MCP program has such a high
regard in the industry. And we’re going to do everything
we can to maintain that high level. That’s why we
have a number of the initiatives that I’ve talked
about [both on] the process side and the exam development
side—along with some of the legal issues, such as
the non-disclosure.
Do you feel pressure from above at Microsoft
to bring up the numbers of certifications—to make
it less tough to become an MCSE, for example?
I haven’t felt that pressure at all. Really, I think
the numbers have grown themselves. I think the reason
they’re growing is because we have that high bar
of quality. And I believe that the executive team [at
Microsoft] recognizes that. I personally have never once
had any executive come to me and say, “How are you
going to increase your numbers of MCSEs?” That is
really not the directive I’ve been given.
And when we introduce new methods of [testing] to maintain
the quality and innovative exam types, these are well
received internally. I think everybody understands the
value issues, the value proposition.
So what is the directive you get from executives
at Microsoft?
We’re here in support of product sales. So we need
to be creating the right strategies and the right certification
tracks to do that, to support our products and to support
our technologies. That’s really the direction they
want me to follow. Nobody comes to me and tells me I need
to produce so many MCPs. I set the quotas. I do it by
looking at our numbers and the new tracks we’re introducing.
But nobody comes to me and gives me some magic number.