When I began to use comprehension-based language teaching in my classes, something unbelievable happened. As I administered reading, listening, and writing assessments and evaluated them against standards based on ACTFL’s Proficiency Levels, I was blown away to see that my students actually were actually SKIPPING entire ACTFL Proficiency levels as they made gains in their abilities in the target language. Very few of my students ever scored at the Novice Low level on the rubrics I was using, straight to Novice Mid or even Novice High! From their first assessment, it was obvious that my students were already reaching seemingly unattainable levels of proficiency…
…or were they? As it turns out, neither my assessments nor my evaluation of student assessments was bad. I was simply looking at them with the wrong lens! In this post, I am going to dig into the concepts of performance and proficiency, and how I learned to accurately interpret assessment data from my language courses. Buckle up for four big ideas about performance versus proficiency!
This blog post was written by Elicia Cárdenas, Director of Training
Proficiency describes real-world ability
ACTFL, the professional language association for teachers in the United States, tells us that their proficiency guidelines “are a description of what individuals can do with language in terms of speaking, writing, listening, and reading in real-world situations in a spontaneous and non-rehearsed context.” (ACTFL Proficiency Guidelines 2012).
Take a second look at these keywords from that definition:
- Real-world situations
- Spontaneous
- Non-rehearsed
We at The Comprehensible Classroom believe and teach that “the classroom is the real world,” because authentic communication between authentic speakers can happen within the classroom. However, in this definition ACTFL is using the term “real-world situations” to refer to situations that take place outside of the walls of the classroom.
The classroom is *always* a classroom and not ever the world outside the classroom. Since the proficiency guidelines describe communication in extra-classroom settings, it leaves us to wonder whether teachers can ever truly measure their students’ proficiency. Could that be true?
Can proficiency be evaluated in the classroom?
Proficiency cannot be measured by teachers in the classroom--notwithstanding the administration of a formal OPI exam. The classroom is a fixed context, and it can never not be the classroom. As Dr. Bill VanPatten reminds us: “The classroom setting is always the same...It is neither a restaurant nor a doctor’s office nor a travel agency. Furthermore, the participants are always the same: the students are students, and the instructor is the instructor, and they have no other roles or occupations during class time.” (While We’re On the Topic, VanPatten, ACTFL 2017)
No matter how closely we try to create so-called “real world” scenarios in the classroom (restaurant interactions, travel interactions, formal letter writing, etc.), those scenarios will never be authentic. We will never be able to manipulate the variables such that it will exactly match what would happen in the outside world. When teachers observe and evaluate their students’ in-class communication, they are not evaluating proficiency. ACTFL’s proficiency descriptors are not intended to describe classroom communication! We cannot evaluate proficiency from within the classroom.
What are assessments evaluating, if not proficiency?
When I gave my students a writing assessment, I observed that they were writing solidly at the Intermediate Low level after only 2 months of instruction. To arrive at this conclusion, I compared the characteristics of their writing to the characteristics listed in ACTFL’s Proficiency Guidelines for Presentational Writing. Although this evaluation of their writing was valid, it did not mean that my students were at the Intermediate Low level of proficiency. So, what did it mean?
When we assess students, we are assessing their performance on a single task. Each assessment is a performance: it is a snapshot of what they can do at one moment in time, with one specific task, and it is a performance that happens within the classroom context. A single, Intermediate Low performance does not mean that students can communicate outside the classroom on any topic at that same level. Everything that we do in class is a performance, and performances allow us to estimate student proficiency. Assessments are just snapshots of the student in that exact moment. Then, we can use the data we gather from multiple performance assessments to estimate student proficiency based on their performance.
Performance is a snapshot of one moment in time: one task, one context, one topic. Proficiency is the full picture: any task, any context, any topic.
Estimating student proficiency using performance assessments is useful for several reasons. When used consistently, well written performance assessments demonstrate student progress and growth in communicative ability over time. This display of progress is motivating for students and reassuring to other stakeholders, such as administrators and caretakers. Performance assessments also can help teachers see what aspects of language students have truly acquired. This is especially true if you use fluency or timed free writes in your assessment program. All of this data can be motivating to students and useful to the teacher in adjusting their pacing and/or differentiating to support student success.
Is it helpful to talk about proficiency?
The fact that we can’t measure proficiency in the classroom doesn’t mean that we shouldn’t talk about it. Many teachers, myself included, use signs and other tools to talk with students about proficiency levels. I even taught an awesome lesson on proficiency levels to my students at the beginning of every year! Although teachers cannot evaluate proficiency within the context of the classroom, those tools and lessons are not without value. They help students to understand the vision for what the end goal of language learning is, to develop a common language about language proficiency, and to help set reasonable expectations regarding outcomes and timelines for students, families, and administrators. Because building proficiency is still the goal--even if it can’t easily be measured in the classroom--talking about proficiency with our students is pragmatic.
Skills and modes develop at different rates
Another important thing to understand about proficiency is that every mode is separate. Since performance is a snapshot indicator of proficiency, the same is true for performance. Performance at a particular level in one mode (interpretive, interpersonal, presentational) or skill (reading, writing, speaking, listening) does not indicate level of proficiency in any other mode.
For example, let’s fast forward that same group of students who performed at Intermediate Low on their 2-month writing assessment. We’re now eight months into the school year, and I decide to give them a speaking assessment. While they are still performing consistently at Intermediate Low with their writing, I discover that they can barely eke out a few sentences in speaking! And, while I can mostly understand their message, it is choppy and uses the most basic language, and it takes effort to understand what they are trying to say. What gives? What happened to that same kid who wrote 100 words in less than 10 minutes and used a variety of language and complex sentences?
Differences in performance in different modes of communication are normal. A lower performance in speaking does not suggest that you have been misevaluating student performance in writing. Our proficiency grows at different rates in different skills, and our students’ classroom performances will reflect that fact.
Grant Boulanger, who is a respected teacher, former department chair, and instructor with the State of Minnesota, reminds us: “First, we learn to listen. Then we learn to read what we’ve heard. We learn to write what we’ve read and heard. Finally, we speak because we’ve heard, read, and written it.”
In a classroom context, reading and listening performance typically advance most rapidly. Speaking is the skill that develops last. This is especially true for spontaneous, unrehearsed speaking! (And even assessments that require students to respond spontaneously are still performance assessments, not assessments of proficiency.) Speaking performance indicators will almost always lag behind performance indicators in other skill areas. Remember though: if it is a test in class, it is still a performance assessment.
Proficiency is not linear
We cannot measure proficiency in the classroom; only performance. And, a single student will perform all over the board in different skills (reading, writing, listening, and speaking) and modes (interpretive, interpersonal, presentational), and on different days. The same student may astound us with what they can do and surprise us with what they can not yet do! All of this is normal.
It is normal for students to be all over the map, even within the same skill area. They are developing proficiency, and there are many things happening inside their heads all at the same time! Their progress is not always linear, and their abilities and performances vary depending on the task and the kind of day the individual is having and, of course, their mental representation of the language. Inconsistent performance on assessments is not necessarily an indicator that your assessments are bad or that you’re not evaluating them properly. Proficiency isn’t linear, and so neither is student performance.
Your grades aren’t a guarantee.
If we accept all of these things to be true, it may be worth reconsidering how much value we place on assessment, and how much time we, as teachers, spend thinking about assessment.
Assessment is important because it is part of our job. Students and families want and deserve to know if they are making progress. As teachers, we can usually see students’ progress in real time, but often it is less visible to families and the students. Assessments are a way to make progress visible to stakeholders. Another reason that assessment is important is that the grades that we attach to students’ performances have a real impact on students’ affect and futures. It’s important that we engage in professional learning and critical reflection on assessment, and that we apply what we learn to our teaching.
At the same time, maybe we should stop agonizing over evaluation and grading. We know that what we are observing in the classroom are a collection of single performances, and that even the net sum of those performances can’t perfectly indicate a student’s proficiency level. We can expect that students won’t fit neatly into this box on that rubric or a different box. It’s normal that student performance is somewhere in-between. And, no one is viewing the grades you give your students as a Guaranteed Stamp of Proficiency Approval. When department members are on the same page about the relationship between performance and proficiency, they’ll accept that at any given moment, students are floating between proficiency levels--and so their performance on assessments will float between descriptors as well!
We can’t control students’ language acquisition or, consequently, how exactly their journey toward proficiency unfolds. We can control the goals that we set for our students, and we can control the way that we communicate with our students and other stakeholders about their progress.
We can focus on:
1. Setting goals that students can achieve.
We know that proficiency takes time, and we know that performance will almost always out-pace proficiency. Just because students can perform at Novice High in a specific skill area after a few weeks of language class, we shouldn’t then set a goal for them to perform at Intermediate Mid by the end of the year. We can focus on learning about what expectations are reasonable for growth in proficiency in school settings and setting goals for our students accordingly. Learn more here »
2. Making rubrics realistic and reasonable.
We can look to ACTFL’s Proficiency Guidelines and to research in the field of Second Language Acquisition to gain a clear picture on how proficiency develops over time. Neither source will mention anything about correct usage of accent marks, spelling, or even verb endings. Modeling our rubrics after ACTFL’s Performance Descriptors and Proficiency Guidelines will allow us to focus on students’ competencies--what they can do--instead of on a predetermined checklist of forms that has little or nothing to do with how language proficiency actually develops. Read more in this blog post »
3. Providing students with robust opportunities to reach their goals.
Someone famously said, “Weighing a pig doesn’t make it fatter.” Only feeding makes the pig grow! If we want our students to continue advancing toward their scores, we need to focus on instruction, not assessment. We need to ensure that we are teaching in a way that promotes growth in proficiency. Especially, we need to ensure that we are teaching in a way that promotes proficiency for all learners--not just students who are academically gifted. Every student who has acquired a first language can acquire a second.
4. Making grading simple for us.
This isn’t selfish or irresponsible. It’s a necessity! Teachers have a lot on our plates, and we have to find the tools and practices that make this profession sustainable. Practices such as marking errors take time and breed negativity in both us and our students. The more we focus on what students can’t do and aren’t doing, the more frustrated we become with ourselves, our students, and our job. Plus, that kind of feedback doesn’t help students become more proficient! Look for what your students can do and are doing in each performance, celebrate those things, and then get back to the “feeding”: providing them with a steady diet of comprehensible input!
