This past April I was a member of a bias and sensitivity review panel for the Partnership for Assessment of Readiness for College and Careers (PARCC), in Denver for three days. The purpose of the panel was to review draft PARCC test questions for possible bias and insensitivity. The group consisted primarily of K-12 educators and higher education administrators, as well as a few teachers from all across the United States. 

During the 2014-2015 school year, the PARCC test was administered in 11 states plus the District of Columbia. This test is required in Illinois schools for students in grades 3-8 as well as students in specific high school classes. 

I was interested in participating in this panel because I wanted to do my part in assuring that the PARCC test would be accessible to the English learners I teach. I have had some experience looking for bias and sensitivity issues as a member of my school district's math curriculum committee. As an EL teacher, over the years I have helped numerous colleagues rewrite overly complicated test questions. I have seen how wordiness alone could make it difficult for students (EL and non-EL) to determine what they are being asked to do. I've also flagged coworkers' test questions which have contained unintended insensitivity, such as a math question comparing how much weight two girls lost when dieting. In that case, I had to argue that while dieting might actually be a common occurrence in real life, it is an unfortunate one and not the best topic to include on a test for junior high students. So, with these experiences in mind, I was looking forward to rolling up my sleeves to help with this panel.

During our first meeting in Denver we were provided with a brief training session about the types of questions which constitute bias and sensitivity issues. These included questions which could stereotype groups of people based on "gender, race, ethnicity, language, religion, socioeconomic status, disability or geographic location." We also received electronic copies of the PARCC Fairness Guidelines which we used to help inform our decisions during the meeting. 

Additionally, our review team was tasked with weeding out "any reference or language in an item that might cause a student to have an emotional reaction during the test administration [preventing] a student from being able to accurately demonstrate ability." It was noted that this could include reactions that were both positive and negative; emotionally charged questions in a test could affect the students' performance in unintended ways and should therefore be avoided. 

Some of the obvious topics to flag included drug use and religion, as they might be likely to cause emotional reactions. However, some topics on the list were less obvious to me, for example, September 11, and negative descriptions of personal appearance. There were also very specific topics that we were told should not be the main focus of a passage, such as certain aspects of owls, snakes, coyotes, spiders and eagles (as these are spiritual animals for many Native Americans). As a teacher from Illinois who has never taught Native American students, this was new to me. We were also instructed to pay careful attention to any references to rats, roaches and lice, particularly in younger grades, because some students may find them frightening. Unfortunately, vermin are found everywhere in the country, but my squeamishness about crawly things has prevented me from ever considering writing test questions about them. Now I have a legitimate excuse for "exterminating" such questions from my own assessments! 

The procedure for reviewing the questions was fascinating. We were divided into small groups of about eight people from different regions of the country. Each of us was provided a copy of the PARCC Fairness Guidelines, guiding questions, and a computer and then directed to review questions from a particular grade. My group began by reviewing high-school level mathematics questions and then moved grade-by-grade down to Grade 6. When we finished our work ahead of schedule, we then reviewed some high-school English language arts questions. Each of us independently read 25-50 questions and marked each "approve," "approve with changes," or "reject." Once everyone from the group had reviewed the assigned questions, we went through each question not approved by at least one participant. We would then discuss the concerns and decide as a group whether to accept, change, or reject the question. The conversations were fascinating. The broad regional representation of the group enabled people from one geographical area to highlight issues that others wouldn't have thought of. For example, names used in a test question might be the name of a store or product somewhere in the country, and that double meaning could be confusing to a student. 

All viewpoints were considered. Even if only one member had concerns about a question, but the group agreed to approve the question, those objections were still noted in the report. We were told that each objection would be addressed at the next stage of question screening.

Although we were not tasked with commenting on the formatting of questions in these sessions, our group felt compelled to point out formatting issues when we found them. One of the most common suggestions we made, and something that would be helpful for all teachers to consider when writing their own assessments, was adding blank space between sections of questions, making them more readable. Also, bulleting the important facts or steps of a question was often suggested by our group. These small changes help students who have reading issues get to the heart of a question, which is what we really want. 

This experience was very powerful for me and I'm sure, as a result, I am better at writing assessments which are free of bias and sensitive issues. I hope also, that in some small way, I have helped make the test more accessible to students. Because I don't think anyone will miss the crawly things. 

Since state agencies recommend and approve participants for this panel, if you are interested in participating in future Bias and Sensitivity Review panels, contact the Illinois State Board of Education. Alternatively, you can contact Yanina Torres at ytorres@parcconline.org and who can connect you with the appropriate contact at ISBE.


Debbie Sternecky is an EL teacher  at Naperville North High School in Naperville, Illinois and is Advocacy Chair and Past President of Illinois TESOL-Bilingual Education.