A few years back, after years of living abroad and mainly teaching students who shared a culture and educational background, I found myself living in Tucson, Arizona, teaching for an fantastic program called Family Literacy. At this point I had been teaching for a few years and felt confident in my abilities. I had no idea of the challenges I would face with my new class. Unlike my classes in Turkey, which were made up of 18 to 20 year old Turkish students who had all passed an entrance exam before being admitted, my new class consisted of mothers ranging in age from 17 to 40 from Somalia, Mexico, China, and Bangladesh. Some of my students had never before attended any kind of school while others already had masters degrees. They entered my class at all levels from completely functional in English to only knowing how to say a few words. My job was to teach them English, literacy skills, run parenting workshops and advocate for civic involvement. To say this Job was a challenge would be an understatement, but I loved it because it offered me never-ending opportunities to learn.
One of the challenges of teaching such a diverse group is figuring out how to assess their learning. Clearly the age-old practice of giving everyone the same test was not going to work.
Differences to think about before designing any kind of assessment tool
- Literacy Levels First of all, it is important to recognize that my students had very different challenges. Some of my Somali students had never learned to read in any language so traditional written assessment would not work with them. They tended to be quick to pick things up verbally though. My students from China had learned to read and write but in very different alphabets. Those students had a good grasp on literacy concepts but had trouble decoding the new alphabet (now that I live in Japan I can totally relate to this problem). Some of my students from Mexico were literate in the Roman alphabet but had limited formal education so while they could read, fluency was an issue. Other students from Mexico had a lot of practice reading in the same alphabet English uses so for them, reading was not as much of a challenge.
- Grammar Students with such different first languages struggle with different aspects of English. Some languages are quite similar to English in grammar structures while others are completely different. For example, I was totally confused when I started learning Turkish because there is no word that translates into “to be” in English. For a long time I had trouble wrapping my head around the fact that the idea of “to be” is not a word but rather a suffix added on to the end of an adjective or noun. Things like the articles “the and a” are simple for learners who speak languages that have equivalents but are incredibly complex and confusing for speakers of languages that do not.
- Pronunciation Pronunciation is about more than just phonemes, it also includes stress, tone, pitch and many other things. Many European languages share pronunciation traits with English so they do not need to be explicitly taught but other languages do not. This can affect how much students understand when they listen to videos or other people speaking.
- Discourse Styles Speaking is speaking right? I used to think so until I moved to Japan where the discourse style is very different than it is in America. In Japan, during discussions, people have a speaking order that they all understand. Everyone in the group knows more or less who will speak first, who will speak next and so on without discussing it. In English it is a bit more of a free-for-all so if you assess students on how well they are speaking to other students in groups you may find that they are not participating in the discussion as much as you would wish. This might not be because they lack the ability to speak, it may just be that they are not able to participate as easily in that particular discourse style.
Wow, that is a lot of things to think about when designing an assessment tool and I have just begun to scratch the surface!!
As you can see, in a diverse class, equal does not mean fair. Most educational systems adhere to the idea that all students should be assessed using the same exact tool believing that this is the only way to be fair. This brings to mind the classic cartoon in which the evaluator tells a group of different animals that they all must climb a tree to pass the test. The monkey is, of course delighted, but the fish looks worried.
This brings me to the next thing to consider when designing an assessment tool, what is the purpose of the assessment?
Reasons to Assess Learners
Different assessments are given for different reasons.
- Gatekeeping Some tests like the TOEFL and EILTS are given in order to allow some students to qualify for things like study abroad and others are held back. They are supposed to demonstrate a student’s academic English skills so they will be able to handle the requirements of courses in English in an international setting. The validity of this supposition is a topic for another post, but that is the intention. These tests are important when courses rely on students having certain abilities when they arrive to a certain course. Although it may seem harsh to the individual students who are not allowed to pass through the gate and move to the next level, they are believed to be essential for the smooth running of the next class. For example, if a level two writing class requires that students be able to write well-organized paragraphs, students should not be able to enter it before demonstrating their ability to write a well-organized paragraph, otherwise level two will look a lot like level one because so much time must be spent getting students up to the level they should have been before entering.
- A study tool I often give my students quizzes not because I want them to prove their mastery, but rather as a study tool. Research has shown that spaced repetition is the key to memorizing information. Learning English requires a lot of memorization, so I want my students to be forced to recall vocabulary and grammar frequently. It is that act of recall (often during a quiz) that allows the connections in the brain to become stronger. I give unit tests for much the same reason, I want my students too look back at material they have already been exposed to and think about it again.
- To give instructors an idea of what concepts they need to revisit. Many times assessment tools are administered because instructors want to see what their students have mastered and what they have not. These kinds of assessments are used to adjust the course so that key concepts are truly understood. This is sort of a gatekeeper for the course instead of the individual students. I keep these kinds of assessments very low-stakes.
- To rate students for future schools and employers. The way our educational systems are set up, learning is a competition and tests are the way we measure who is winning and who is losing. Competitive universities and companies only accept students who score high on these assessments. Grade Point Averages are kind of like gatekeepers, allowing those who excel to pass through while those who don’t must find a different path but it is more about how they scored on many different evaluations rather than just one.
- To allow students to see what areas they need to focus on. These kinds of evaluations measure learners against their own past performance to demonstrate what they have learned and what they still need to work on. They are different for each student as no two students start at exactly the same place. This kind of assessment can take on many different forms, everything from a test to feedback in the marines of an essay but the idea is that students use the results of the assessment to evaluate what areas they are understanding well and what areas they need to work on. I like to give students the opportunity to revise their work when I assess them for this purpose because it gives them a chance to apply what they have learned from the feedback immediately.
Pros and Cons of Different Kinds of Assessment Tools
- Rubrics Pros: Rubrics can be helpful for both instructors as well as teachers. When creating a rubric, the instructor must really think about and define what the assignment calls for. I usually use rubrics for essays and oral presentations. I break the assignment down into the core components I want students to focus on. For example, in a speaking assignment I might break the requirements into content, organization, fluency, use of vocabulary, and pronunciation. I then have to clearly explain what a student will need to do in each category to earn each specific number of points. I find it is a good idea to create the rubric well ahead of time so I can make sure I give the students a chance to practice all of the skills they will need before I evaluate them. I then give them to the students before they start preparing for the presentations so they know what exactly I will be listening for. Cons: I often find that rubrics are either too complicated or too simple. They break the assignment up into component parts but often there are things other than those listed on the rubric that can make student work stand out or fall flat. I also find that students don’t read the rubric carefully. In order to get enough detail in a rubric to make it useful to the teacher, lengthy descriptions are needed. Those lengthy descriptions often cause students to skip over them.
- Quizzes (Multiple choice, fill in the blank, short answer) Pros: They are quick and easy to administer and don’t usually take much class time. They encourage students to study frequently rather than “cram” later on for bigger tests. Students can either grade their own work or grade each others work to get immediate results and cut back on grading time. They are not subjective, it is clear if the answer is correct or incorrect. Cons: They often only test students on their ability to take this kind of test well. They mostly show a student’s ability to recognize the correct answer (multiple-choice) or recall memorized information. Most of the time they don’t show a deep understanding of the material. A student can ace a vocabulary quiz and still have no idea how to use the word in context.
- Exams Pros: They can show how well students understood the course material and if they are ready to move on to the next level. If designed well, they can help students think about all the parts of the course as a whole, look at the bigger picture. They serve as a good way to make students review previously learned material. Cons: They are often administered at the end of a course so students have no chance to learn from their mistakes on the exam. They cause a lot of anxiety on the part of the students and some of them may freeze up. Student often employ very bad study habits when preparing for an exam like staying up all night studying.
- Written/oral feedback Pros: Each student gets individual comments on exactly what they need to work on. Students have a chance to see their own work through the eyes of another person. Cons: It is time consuming and difficult to give meaningful, constructive feedback. Students often don’t understand the feedback and still don’t know how to improve their work. Feedback is often given without giving student a chance to use that feedback to revise their work. For example, if you give students feedback on their oral presentations but they never have a chance to present again, the feedback looses its effectiveness.
- Portfolios Pros: Portfolios show a much bigger picture of student learning. They can be done for a unit or for an entire semester. Before I was awarded my masters degree in teaching I had to hand in a giant, several-hundred page portfolio to show my learning for the entire program! They encourage students to look at their learning and their work in a broader way. They can be used to show administrators, parents and other teachers what you are doing in your classroom. Cons: They are not easy! Many teachers are uncomfortable with them because there is no right or wrong portfolio so grading them can be subjective. They take quite a bit of organization and planning, first you must decide what you want your students to get out of a portfolio, then you must figure out how you are going to organize the documents and how you are going to evaluate the outcomes. They can be a great way to evaluate learning when you have a culturally diverse, multi-level class though!!
- Peer Evaluation Pros: Getting students to look at each other’s work gives them insight into their own work. It also gets them to really look at the different requirements of the assignment they are editing. Peers looking at their work together is a different dynamic than teachers looking at work with students and often inspires deeper thinking. Rather than being told what the problems are, students have to try to figure them out together. Cons: If your students don’t trust each other, they will be reluctant to give each other feedback. Students often don’t know how to look for mistakes and don’t do it well so their partners don’t really listen to what they have to say. Students sometimes don’t understand why they are being asked to evaluate each other’s work and don’t really try to give good feedback.
When you are designing an assessment tool, it is important to first think about why you need an assessment tool and then design it according to the purpose. Do all of your students need to be assessed the same way? If they do, have you taken into account the different challenges they will face as individuals from different backgrounds. Have you helped them to overcome those unique challenges so they can all achieve success? Helping all of your students attain the skills they will need to pass their assessments will probably mean you will have to differentiate your instruction. It will also mean that some students will need to spend more time mastering some skills than others.
Students should be encouraged to examine the different challenges they face and support each other rather than compete with each other. It is so easy to lean over and look at the neighbor’s score and think, wow, they did so much better than I did, there must be something wrong with me! Even though the educational system has been set up to rank students against each other, there is a lot a teacher can do to foster a sense of cooperation and support in the classroom rather than an atmosphere of grade comparison.
What are some of your favorite assessment tools? Do you favor assessing students all the same way or assessing them individually?