ABP Test Development Specialist Receives NSF Grant to Study Adaptive Tests

Catie Mintz, MA, a recently hired Test Development Specialist at the American Board of Pediatrics (ABP), has received a highly competitive grant from the National Science Foundation (NSF) to continue her research on the value of computerized adaptive, multi-subject tests. Computerized adaptive tests serve questions to examinees based on their performance on previous test questions.

In addition to her role in the ABP’s Assessment Department, Mintz is a PhD candidate in the Educational Measurement and Statistics program at the University of Iowa.

“In the field of education, there is a push to convert tests to be adaptive, and thus, personalized for each individual,” says Mintz. “But standardized testing — especially in adaptive modalities — is fundamental to many disciplines, including certifying medical boards like the ABP.”

The ABP already uses two adaptive systems to serve test questions in its single-subject MOCA-Peds assessment platform. One is that pediatricians who get an answer wrong might see that same question appear on their exam in future calendar quarters. Another is the ABP’s recommender system that serves questions based on the topic relevancy as indicated by the examinee when answering earlier questions.

“In adaptive testing, there are many ways that an item [question] can be chosen for administration,” Mintz says. “Research on single-subject adaptive tests has consistently demonstrated the benefit of incorporating item distractor information to improve item selection and test accuracy.” Item distractor information is a statistic that quantifies the appropriateness of an item distractor for a given level of ability. “In my dissertation, I expand upon this knowledge to address a growing need and interest in multi-subject tests.”

After completing her dissertation, Mintz plans to further investigate salient features of computerized adaptive multi-subject tests and hopes that her findings lead to improvement in operational testing.

“My dream is for my research to lead to more examinations that increase learning during the test-taking process and for it to improve testing across disciplines,” Mintz says. “I’d also love to integrate different components of my research with work being done at the ABP. Although the ABP doesn’t currently administer multi-subject adaptive tests, there are facets of my dissertation that align nicely with current ABP initiatives.”

Prior to joining the ABP in July, Mintz was a Graduate Research Assistant at the University of Iowa, where she aims to defend her dissertation before the end of the calendar year. She has a master’s degree in Psychological Sciences with a concentration in Quantitative Psychology from James Madison University. Her interest in test development and psychometrics (exam scoring) ranges from instrument development to latent variable modeling.