This Week: NCTM National Meeting and Exposition


airplane-wing

Later this week I’m privileged to be able to participate in this year’s NCTM National Meeting and Exposition. I haven’t attended this conference in many years and am very excited to be going to New Orleans. I expect the meeting to be an intense learning experience that I plan on sharing in future posts.

Last Friday I reviewed the list of presentations and developed a plan for how I want to spend my time. I plan on attending four or five sessions a day as well as spending time in the exhibition hall. I’ll be busy, but I don’t want to miss out on such a rare opportunity.

The sessions I’ll be attending can be organized under one of four themes.

  • Common Core State Standards for Mathematics (CCSS-M): I’ve selected sessions that focus on the CCSS-M and, in particular, the Standards for Mathematical Practice. Incorporating the practice standards into test items will be an important part of my work this year. So, I’m looking forward to hearing the latest ideas about the practice standards.
  • NCTM’s Principles to Actions: I discovered last week that this year’s meeting also marks the release of NCTM’s Principles to Actions: Ensuring Mathematical Success for All. I suspect that the ideas and recommendations contained in this book will heavily influence U.S. K-12 mathematics education for the next several years. I’m eager to learn what this book is all about.
  • Consortia Assessments: Both PARCC and Smarter Balanced are providing updates on their assessment programs. I must attend these briefings and listen for important information about the future of these programs.
  • Professional Development: Many of my projects this year revolve around teaching educators about what it takes to write high-quality items. I picked a few sessions that I believe will help me design and implement these programs. I’m looking forward to getting insights that will help me with a number of projects, including this blog.

If you plan on being in New Orleans, let me know by leaving a comment. Perhaps we can find time to meet and chat.

Photograph courtesy of freedigitalphotos.net.

Posted in Mathematics

The Art and Science of Item Writing


dashboard-and-road

Over the past thirteen years I’ve spent a lot of time training and mentoring large-scale assessment item writers. As a result, I’ve come to believe a couple of things about the art and science of item writing. These beliefs define how I approach my work. I share them today so that you’ll get a better sense of what this blog is about.

I’ve been trying for a month to write this post. I feel that, by stating these beliefs, I am opening myself up to criticism. Today, I understand that not everyone is going to share my beliefs. If you don’t, that’s fine. The comment box below is an opportunity for you to articulate your experience (without being overly critical of the views of others).

I articulated these beliefs a couple of years ago. So, this post sounds like my First Day Challenge post. Today my intention is to reiterate my beliefs and to articulate them in a more organized manner.

Item Writers Aren’t Born

I believe that item writing is not something that you are good at on your first day. Many teachers, and many new item writers, are not producing the kind of high-quality items required of today’s large-scale assessment programs. Back in 2010, I quoted R. L. Ebel (1951) who said, “It [item writing] requires an uncommon combination of special abilities.” S.M. Downing (2006), in The Handbook of Test Development, makes the same case.

Without specific training, most novice item writers tend to create poor-quality, flawed, low-cognitive-level test questions that test unimportant or trivial content” (Downing, 2006, p. 11)

Writing high-quality test items is not something that anyone, teachers and professionals alike, is prepared to do when given their first opportunity to write large-scale math items.

Item Writers Are Made

I also believe that, through high-quality training and ongoing support, people can become good item writers. Downing (2006) identifies what is minimally required to transition from novice to expert item writer. “For new writers, it is often helpful to provide specific instruction using an item writer’s guide, paired with a hands-on training workshop (Haladyna, 2004)”

Ebel clarifies what is required of someone should they decide to become an expert item writer.

It [item writing] is mastered only through extensive and critically supervised practice…

I’ve been blessed to work with highly-trained and skilled professionals that helped me make the transition from novice to expert. I understand the journey that I’ve been on. I want to help others fascinated by trying to understand how to effectively assess a student’s mathematical understanding to undertake this journey.

My Goal

My goal is to help those that want to become expert item writers to make that transition. Developing this expertise, both in myself and in others, is my passion.

What is your experience with item writing? Have you tried to become an expert item writer? If so, what’s your experience?

References

Downing, S.M. (2006). Twelve steps for effective test development. In S.M. Downing and T.M. Haladyna (Eds.), Handbook of test development (pp. 3-25). New York: Routledge.

Ebel, R.L. (1951). Writing the test item. In E.F. Linquist (Ed.), Educational measurement (1st ed., pp. 185-249). Washington DC: American Council on Education.

Image courtesy of winnond / FreeDigitalPhotos.net

Posted in Assessment, Leadership

Writing Test Items to a Single Common Core Math Standard


mathematics-calculator

Yesterday I started writing the post below. When I went to develop an example that reinforced my point, I discovered that my example ended up being a counterexample to this argument. Rather than move on to a new post topic, I decided to share what I had learned with you. The original start to this post is in blue below. What I learned follows.

The organization I work for has been writing test items aligned to the Common Core State Standards for Mathematics for at least two years now. The majority of the items we develop are aligned to one standard from the Standards for Mathematical Content. Based on this experience, I am beginning to believe that the most interesting mathematics won’t be assessed if we continue to focus solely on the individual content standards.

We are successfully writing items that align to the standards and that assess important content skills and knowledge. Assessing these skills are an important, particularly when you are trying to provide teachers with information about what students still need to learn. However, to do more than that I’m beginning to believe that we should do two things.

  • Assess the Standards for Mathematical Practice as well as the Standards for Mathematical Content.
  • Combine content and practice standards in ways that illuminate interesting mathematics.

I believe that, by expanding the scope of what we’re trying to assess, not only will we be able to evaluate a student’s knowledge and skills but we’ll also be able to evaluate their application of these skills to solving practical, important, and interesting mathematics problems.

I intended to develop an example where the process of students graphing data, identifying a function that describes that data, and then using the function to answer questions in the context of the problem was spread out over several standards. However, what I found were several examples of standards that connect graph, function, and context together. Some examples are given below.

5.OA.02.03 – Generate two numerical patterns using two given rules. Identify apparent relationships between corresponding terms. Form ordered pairs consisting of corresponding terms from the two patterns, and graph the ordered pairs on a coordinate plane. For example, given the rule “Add 3” and the starting number 0, and given the rule “Add 6” and the starting number 0, generate terms in the resulting sequences, and observe that the terms in one sequence are twice the corresponding terms in the other sequence. Explain informally why this is so.

6.EE.03.09 – Use variables to represent two quantities in a real-world problem that change in relationship to one another; write an equation to express one quantity, thought of as the dependent variable, in terms of the other quantity, thought of as the independent variable. Analyze the relationship between the dependent and independent variables using graphs and tables, and relate these to the equation. For example, in a problem involving motion at constant speed, list and graph ordered pairs of distances and times, and write the equation d = 65t to represent the relationship between distance and time.

08.F.02.04 – Construct a function to model a linear relationship between two quantities. Determine the rate of change and initial value of the function from a description of a relationship or from two (x, y) values, including reading these from a table or from a graph. Interpret the rate of change and initial value of a linear function in terms of the situation it models, and in terms of its graph or a table of values.

I still believe that some interesting mathematics can only be assessed by combining standards and practices together in meaningful ways. Now I need to find an example that works.

What do you think? Do you have an example of meaningful mathematics that can only be assessed by combining standards and practices together? Leave a comment with your thoughts and suggestions.

References

National Governors Association Center for Best Practices, Council of Chief State School Officers, (2010). Common core state standards for mathematics. Retrieved from website: http://www.corestandards.org/assets/CCSSI_Math Standards.pdf

Image courtesy of Poulson Photo / FreeDigitalPhotos.net

Posted in Assessment, Mathematics