Adaptive testing has met with some resistance, but it really works to your advantage.

Adaptive Testing Works for You

Adaptive testing has met with some resistance, but it really works to your advantage.

• By Cyndy Fitzgerald
• June 01, 1999

How can something so right seem so wrong? That might sound like the title of a country western song, but it’s no tune you’ll be hearing those of us at Microsoft sing any time soon. Instead, it sums up the negative reaction we’ve been met with in some quarters to an innovation that only works to your advantage.

That innovation is computerized adaptive testing (CAT), which Microsoft adopted last fall to deliver many of its MCP exams. CAT has a proud pedigree going back for centuries (although not always with the computer, of course). CAT’s biggest advantage is its greater efficiency. It’s possible for a CAT exam to assess a person’s knowledge in 60 percent of the time required for more traditional tests. For the typical MCP exam, that means a test that used to take up to an hour and three-quarters can now be completed in an hour or less. The exams also offer greater security, because no one candidate sees all the questions.

CAT works by tailoring each test to the individual exam taker. Examinees all start with an easy-to-moderate question. If they answer the question correctly, they get a more difficult follow-up question. If they answer that question correctly, the difficulty of subsequent questions likewise increases. Conversely, if the second question is answered incorrectly, the following questions will be easier. This process continues only until the CAT determines the candidate’s ability.

While most test takers appreciate CAT’s benefits, a few have expressed frustration. “These questions are too easy. Everyone will pass,” say some. “The test was too hard and too short. If I got the longer test that my colleague took, I could have passed,” say others. Ironically, both comments are off-base; the percentage of test-takers achieving a passing score hasn’t changed.

Consider Albert, who gets just 15 questions that are all very hard for him. When Albert answers his first or second question incorrectly, the CAT offers him an easier question. After he fails increasingly easy questions, the computer determines that Albert isn’t going to pass and therefore ends the test.

Or, consider Barbara. She gets the same number of questions (15), but they seem so easy that she thinks everyone will pass. However, Barbara is answering her personalized set of questions so well that the computer can’t find a question that she can’t answer correctly, and it quickly determines that she’s MCP material.

Finally, there’s Charlie. Charlie gets many more questions than the others—maybe 25 or more. Is the CAT discriminating against him? No. Charlie’s first few answers—some right, some wrong—give him a border-line score. The computer needs more questions to determine which side of that border he belongs on.

Some of you have complained that the test doesn’t provide feedback to help improve future performance. That’s because CAT—while accurate at what it’s designed to do—isn’t designed to provide such feedback. Because it uses fewer questions, it would produce misleading feedback results.

If you still have questions about CAT, we’re still listening. To get in touch with us and to learn more about CAT—including the chance to take a sample test—go to the Microsoft Web site at www.microsoft.com/trainingandservices/
default.asp?PageID=mcp&PageCall=tesinn&SubSite=examinfo. You can write to us with comments at [email protected].