Over the past thirteen years, I’ve spent a lot of time training and mentoring large-scale assessment item writers. As a result, I’ve come to believe a couple of things about the art and science of item writing. These beliefs define how I approach my work. I share them today so that you’ll get a better sense of what this blog is about.
I’ve been trying for a month to write this post. I feel that, by stating these beliefs, I am opening myself up to criticism. Today, I understand that not everyone is going to share my beliefs. If you don’t, that’s fine. The comment box below is an opportunity for you to articulate your experience (without being overly critical of the views of others).
I articulated these beliefs a couple of years ago. So, this post sounds like my First Day Challenge post. Today my intention is to reiterate my beliefs and to articulate them in a more organized manner.
Item writers aren’t born
I believe that item writing is not something that you are good at on your first day. Many teachers, and many new item writers, are not producing the kind of high-quality items required of today’s large-scale assessment programs. Back in 2010, I quoted R. L. Ebel (1951) who said, “It [item writing] requires an uncommon combination of special abilities.” S.M. Downing (2006), in The Handbook of Test Development, makes the same case.
Without specific training, most novice item writers tend to create poor-quality, flawed, low-cognitive-level test questions that test unimportant or trivial content” (Downing, 2006, p. 11)
Writing high-quality test items is not something that anyone, teachers and professionals alike, is prepared to do when given their first opportunity to write large-scale math items.
Item writers are made
I also believe that, through high-quality training and ongoing support, people can become good item writers. Downing (2006) identifies what is minimally required to transition from novice to expert item writer. “For new writers, it is often helpful to provide specific instruction using an item writer’s guide, paired with a hands-on training workshop (Haladyna, 2004)”
Ebel clarifies what is required of someone should they decide to become an expert item writer.
It [item writing] is mastered only through extensive and critically supervised practice…
I’ve been blessed to work with highly-trained and skilled professionals that helped me make the transition from novice to expert. I understand the journey that I’ve been on. I want to help others fascinated by trying to understand how to effectively assess a student’s mathematical understanding to undertake this journey.
My goal
My goal is to help those that want to become expert item writers to make that transition. Developing this expertise, both in myself and in others, is my passion.
What is your experience with item writing? Have you tried to become an expert item writer? If so, what’s your experience?
References
Downing, S.M. (2006). Twelve steps for effective test development. In S.M. Downing and T.M. Haladyna (Eds.), Handbook of test development (pp. 3-25). New York: Routledge.
Ebel, R.L. (1951). Writing the test item. In E.F. Linquist (Ed.), Educational measurement (1st ed., pp. 185-249). Washington DC: American Council on Education.
Image courtesy of winnond / FreeDigitalPhotos.net