Tag Archives: assessment

Ruminations on Assessment as Learning

Photo: framed

framed – cc licensed ( BY NC ND ) flickr photo by eyemage

As I wrap up my Beyond Letter Grades experience, my last badge effort involves contemplating assessment as learning, which I must confess is a bit of a slippery subject. It overlaps so much with terms like assessment for learning and assessment of learning that it is pretty easy for them to start blending together. Honestly, I am not sure that I see enough difference between as and for to make a significant case for them being separated.

Modifying Portfolio Assessment

For years I have employed a writing portfolio as the single most important task of my classes. As I have changed schools, schedules, and students, it is one thing that has remained in place as part of my practice. In this sense it is less a lesson and more an assessment. However, it has remained a fairly foreign concept to most of my students and requires definite preparation, which takes the form of a series of short lessons. It is always a bit onerous to tackle in a single one.

On a superficial level I modify the portfolio requirements all the time depending on what the students have accomplished over the course of the semester. Unfortunately, the school where I now teach uses a semester-based system, which means that there is some minor potential turnover of students at the break every year.

Semester vs. Year

Consequently, I ask for a portfolio at the end of each semester, although I feel like the results were better when I have worked with a year-long schedule. With a year-long portfolio, there is a much longer developmental arc and the thread of learning can be more consistent over that time.

For me, as well as my observation of students, semesters tend to truncate the natural flow of the school year, compressing desired outcomes into even more tightly bound boxes, which may or not be reasonable for some students. By the time a high school student has adapted and begun making deep progress the semester is over and a new one begun. I have always felt that it takes most students about two-thirds to three-quarters of the year to be operating at their peak level. Shortly after that is the sweet spot, where I have always looked to get the best assessment of learning. Prior to that it is all about feedback loops and improvement.

Nevertheless, I use a semester portfolio, which includes a reflection on the selections and the process of creating them, which I wrote about for the self assessment module. Yet, I have always felt that this task needs more scaffolding to better reach students at a variety of different ages, levels, and abilities. This unit, in conjunction with a handful of others, got me thinking about how to do just that. I think the answer may be through a lens of assessment as learning, a series of scaffolded student experiences.

Adjusting the Assessment Lens

Photo: Lens (160/365)

Lens (160/365) – cc licensed ( BY SA ) flickr photo by Andy Rennie

In essence Beyond Letter Grades has already sparked this change. Building on the work from the self assessment badge, I will ask students to engage in a series of self assessments that will grow in depth and complexity.

Beginning with a closer self assessment of the main “summative” task in the narrative unit students are completing, students will get the first formal formative self assessment experience. While I explained this particular plan in greater detail, here is the quick summary. Students have two drafts of a long narrative they have composed, one completed before and one completed after a round with a peer response group. The amount of feedback each student receives varies, but all groups include three students.

Considering the limits of time and peer feedback the differences between the two drafts will be somewhat limited. This means that the changes are likely to be limited as well, and thus easier to identify and explain why they were made. Students also were given a rubric by which the narrative will be assessed to use as an additional reason for making changes. I will ask students to highlight the changes between the two drafts and explain what prompted the revisions and why they were made. Previously I was only contemplating this move. Now I am committed to it. This should be take about half a class session.

Additionally, within a couple of days of this first experience, I will present students with both a pre-test and post-test narrative assessment and ask them to identify the changes they can observe between the two pieces. This is a more complex task given the length of time between the two compositions and the number of potential technical areas growth. Also, there is no group feedback for this task. However, a rubric will again assist the identification of changes. Similarly, student will be asked to identify what has changed and improved, as well as what they believe the reasons are for the changes. My hope is that this experience will not require a full class session but it certainly could.

As I transition students to a more expository writing focus, I will repeat a similar comparative methodology. Having saved a brief expository sample from each student a couple of weeks ago, I will return it to students and ask them to assess their own sample using a specific criteria. I will then give them another copy of the sample assessed by me using the same criteria. Again, students will be asked to compare the pieces, identifying the differences. This task’s complexity will increase is by having students then rewrite the sample, as well as document why they made certain changes. This is probably a full class worth of work.

These three formative experiences should be preparatory for the kind of self assessment I am hoping to see when they assemble and submit their portfolio of revised pieces they have selected to best show their learning. I suspect that there may be a one two more experiences along the way that will assist, but I will have to wait and see what emerges from looking at student work through this new lens.

Additionally, I have to remain sensitive to the students needs and progress. While I want them to have a few reps of self assessment in hopes of building a deeper more reflective disposition, I do not want to fatigue them on the concept. If I cannot find ways to increase the complexity of the task or reflection I probably need not add another rep.

Concluding Thoughts

As I see it, the key to cultivating assessment as learning is framing activities in the course around different types of formative and summative feedback, being prepared to transform any summative assessment to a formative one when needed, and scaffolding self assessment in such a way that students gain a deeper capacity for reflecting on their own work and processes.

More than anything, reading the excerpt from Rethinking Classroom Assessment with Purpose in Mind and ruminating on how to apply some of the principles has lead me to believe assessment as learning may be more about creating a cultural disposition in class. One that both encourages and honors student’s monitoring, assessing, and ultimately evaluating there own performance. It has to become a habit of mind of regular practice for it to really be successfully realized.

The Only Assessment that Matters

Photo: Inverse #2

Inverse 2 – cc licensed ( BY NC ND ) flickr photo by Andy Houghton

A fellow National Writing Project colleague and friend Paul Allison and I were talking once upon a time, when he posed a question very close to this, “In the end, self-assessment is the only assessment that really matters isn’t it?” That may not be exactly what he said, but that is how I like to remember it. Plus, it certainly captures the spirit of the brief exchange. The sentiment resonated so strongly with me it has remained ever since.

We all must live with ourselves an awfully long time, more than anyone else certainly has to live with us. That’s for sure. It is not uncommon for me share comments like these and stress the importance of reflection and self-assessment with my students.

A Brief Anecdote on Student Self-Assessment

A few years ago, I received the most remarkable student self-assessment I have ever read, as part of an end-of-semester writing portfolio. Also, I have to admit being a little disconcerted when I saw myself quoted in a student paper, but this student simply gets it and gets it on a deeper level than I ever would have imagined. It also seemed to highlight a lot of the issues that have been shared and discussed in this MOOC. Here is an excerpt.

Through the course of the year, I have been writing down bits of conversations, words, and tips that I have heard in English class. Some are funny, some are weird, and some really stick with me. On October 28th, you said, “[Self-assessment] is really the only assessment that matters.” Is it? Through the course of the year, I grew more and more at home with this statement. If I know I am doing the best I can, then everything else is secondary. “Any time you’re focused on the grade, you are off target,” you said on February 14th [and has] always been a hard concept for me to wrap my head around. Through the year, though, these quotes bloomed into significant meaning. Whenever I write, like now for instance, it needs to just be the best I can do. My goal is to make my point and prove it in my writing, not simply to reach 600 words. This is a way that I have grown as both a student and a person, because as my mindset in school shifted, so did my outlook on the rest of my life.

Keep in mind this is from a former ninth grade student. It remains my favorite, most fascinating student self-assessment I have ever received. It broke all expectations. In fact, reading something like this, written by a student, makes a lot of the slogging through drafts as an English teacher, a whole lot less daunting.

My Latest Plan for a Self-Assessment

I am about to wrap a narrative writing unit with my ninth grade students, which I have already mined for examples for Beyond Letter Grades. Heavily influenced by George Hillocks’ Narrative Writing: Learning a New Model for Teaching, I have been using a lot of the methodology outlined in that title ever since reading it.

Beginning with a pre-test audit to be written in a one hour class, students were given the following prompt right from Hillocks: Write a story about an event that is important to you for some reason. Write about it in as much detail as you can so that someone reading it will be able to see what you saw and feel what you felt.

This week students will submit their anchor summative assignment, which they have had a couple of weeks to develop. Later in the week, they will take the post-test, another hour in class writing task, with the same pre-test prompt. In between, they have completed a handful of what I like to call rehearsal assignments, practicing specific narrative techniques listed in this rubric, also something I have adapted from Hillocks.

I have deliberately kept only a handful of broad categories to be assessed. Using this rubric, I already scored the pre-test, and will also use it to score the summative narrative task and the post-test.

Prior to assigning the summative narrative task, I issued and reviewed the rubric with students, in an effort to key them explicitly into the skills and technique I am hoping that they will demonstrate, despite routinely highlighting them in classroom instruction and various reading selections.

Once they have done a round of peer feedback and submitted the summative narrative task and completed the post-test, I am going to have students conduct a self-assessment.

  1. I will ask each student to score their summative narrative task with the rubric, prior to submitting it.
  2. I will hand each student their pre-test and ask them to score it with the rubric.
  3. I will hand each student their post-test and again ask them to score it with the rubric.
  4. I will then ask them to write narrative feedback about the difference between the two scores, specifically focusing on what they have identified as improvement and why.

I am considering sharing the scores I gave each student on both the pre-test and post-test, and asking them to consider any potential discrepancies between their scores and mine, but I am still undecided on this point.

Turning Summative into Formative

Since I have students complete an end-of-semester writing portfolio, this exercise will be good preparation for a more general, reflective self-assessment that accompanies the portfolio, like the student excerpt included above. Keeping with a broader strategy of looping many of the tasks and skills over the length of the course, this narrative self-assessment becomes a rehearsal for the portfolio one.

All three assessments then become fair game for revision, thus transforming a summative assessment into a formative one. Students may choose which piece that they would ultimately like to include in the portfolio. Since each one is a story, it can become more difficult to decide which story they want to revise, develop further, and include as their best of the narrative bunch along with the other modes and genres that comprise the portfolio.

Coda

In the end, I am blending a number of concepts celebrated in this class in my teaching practice, sometimes in a number of simultaneous ways. Occasionally, I wonder if it can become too complicated for my students. However, the only thing I am truly concerned about is that students are able to learn, improve, and demonstrate their learning in a few different ways. This is also a message that I repeatedly try to impress upon them over the length of the course.

Attempting many alternative assessment methods requires a pretty substantial initial investment of time and energy in developing relationships,  setting expectations, and building trust. It may be a bit ambitious, but I can say that the results have been relatively successful, especially as I continue to refine and advance my reasons, approach, and methods.

Exploring Competency-Based Assessment

Photo: March 16, 2004 -Rough plumbing

cc licensed ( BY NC SA ) flickr photo by Craik Sustainable Living Project

I have often remarked to colleagues and students even that if it were up to me there would be only three grades: Meets Expectations, Above Expectations, and Below Expectations. This perspective seems like it would be compatible with competency-based assessment. Yet, I must admit that I have always been a bit dubious of competency-based models of education and assessment. This is true despite feeling like traditional letter grades don’t cut it either.

Possible Applications

In looking at the various resources curated by Dr. Bull, my doubts have softened a little – but only a little. I can now see where competency-based assessment could be useful in limited applications. When measuring certain lower order thinking, specific content understanding, or singularly isolatable skills, I can see competency-based models working quite well, in fact. It seems tailor made for tasks that are simpler and consequently easier to assess. I could also potentially see where there might be value in competency-based assessment in some vocational education settings, hence my using the image of a plumber above.

Lingering Doubts

However, the rationale behind my doubts have also somewhat strengthened. Perhaps it is my deep belief in the ultimate value of a humanities-based, liberal arts education, one that fosters nuanced general knowledge and understanding, that makes me deeply skeptical of competency-based assessment. It seems too focused simply on the what and not at all on the how. Plus, like the critique penned by Alison Wolf for the UK’s Higher Education for Capability project, I do not believe that deep complex learning can be atomized in such a way that fulfills the promise of competency-based assessment.

Plus, it seems far too-close to one of many current fads, results only assessment, the perverse educational application of an idea dreamt up by couple of Best Buy executives for managing their corporate labor force. I understand that competency-based and results only assessment are not the same, but they seem too similar for my liking.

There seems to be an almost blind faith in the power of the result or product in these models. This strikes me as potentially dangerous in a learning context. Too often I hear things in education like, “You don’t have to re-invent the wheel.” However, sometimes the task is actually about inventing the wheel, assisting students in making a discovery in a novel way that may be very new to them. A such the process involved becomes rather important, arguably more important than the final product, in fact. There doesn’t seem to be as much room for process in competency-based assessment. I am not even sure if it matters in this model.

Examining Examples

Also, competency-based models seem utterly too simplistic and potentially seriously threaten one of those load-bearing walls of education. Anecdotally, talking with some teachers that have switched to competency-based report cards, for example, gave me the impression that it was a disaster. One of the main comments explained how parents had no idea how to read the report card or what the collection data actually meant. Plus, it provided little nuance about how well their student accomplished anything.

Looking at one of the examples in the resources, I was left feeling like in practice it is not altogether that different from traditional letter grades. In the document intended to explain the competency-based system to parents in the Rochester School District (New Hampshire), there are still five divisions of competency, Advanced Competent, Beyond Competent, Competent, Not Yet Competent, Insufficient Work Submitted. I don’t know about anyone else, but this looks awfully similar to A, B, C, D, and F. In fact, the abbreviations track as A, B, C and the NYC or IWS grades will switch to an F if there is no change by the course’s end.

Common Grading Practices

The only thing I can see that is potentially different is that there is an emphasis on common assessment practices. However, that can be done without adopting a competency-based model. A significant value is placed on rubrics, which makes sense in trying to achieve commonality. However, not every task or assignment needs be a common one. Down that path lies one-size-fits-all madness that is not at all about learning and all about management. What’s more, I struggle to see how competency-based assessment personalizes anything about learning. I would submit that it depersonalizes feedback, like any rubric.

While I think there are definite applications for rubrics, those applications are also limited. What educators commonly refer to as rubrics were developed to manage large quantities of norm referenced tests. In this context, rubrics make a lot of sense, but they do not necessarily make sense in every other context. I often wonder if the current obsession with rubrics in education will not turn out to be this eras grade curve, which also makes more sense with large quantities of data points.

Moreover, I would say that common assessment practices are solutions to more administrative problems and not teaching and learning problems. They have the appearance of greater validity, although that is not at all a given. They also seem more systematic and fair, although they could be systematically flawed. Lastly, they most likely reduce complaints and position any bitter individual complaints more precariously against the power of the institution.

Preliminary Conclusions

My understanding of competency-based assessment remains limited. I have only scratched the surface of the concept in the examining the resources. My brief brushes with where I work have been less than convincing, and the anecdotal evidence I have gathered has only served to strengthen my doubts. Still, I can see where in some limited contexts competency-based models could be serviceable and even beneficial. I am not sure that this model serves me as an English teacher all that well. In some ways, it seems similar to badges, although I think I like the binary aspect of badges more. That is definitely not a ringing endorsement.

Experimenting with the Model

One area that requires attention to detail and basic competency from my ninth grade English students involves submitting work in the proper format. My colleagues and I are expected to prepare students to submit all printed academic work in accordance with MLA guidelines for all courses. Early in the year it is a challenge to simply get them to follow models and directions.

This year, in an effort to avoid the kind of leniency that produced students that struggled with simple formatting issues well into the second semester, I drew a harder line and refused to accept any written work until it was properly formatted. Even though I didn’t penalize anyone, it was a cause of great consternation for many students. Most simply did not bother to take the time to follow directions. However, some genuinely cannot distinguish where they may have addressed most but not aspects properly.

This seems like a good context to apply competency-based assessment. The objective is fairly simple and can be itemized. It is the checklist aspect of the assessment that made it most appealing to try. Most students whose work did not correspond with the guidelines struggled to understand exactly why. Thus, I have devised a rubric-like checklist that itemizes all the basic elements of MLA format, without getting into the more complicated aspects of citation, which we will address much later.

Image: Basic MLA Format Rubric

Basic MLA Format Rubric – click to enlarge

Using a tool like this, can help provide a tighter focus. This should reduce some of the anxiety associated with endeavoring to address the errors and allows me to refine the  feedback I provide students. Any unchecked areas comprise an itemized list of errors that a student must address before resubmitting the work for additional more substantive feedback or grade. I am not sure that it is more personalized or whether it is motivating, but it may prove challenging for many students.

On some level, this is almost like a pre-assessment, since it is a way to screen written assignments prior to actually reading them and providing genuinely, substantive feedback. I am hoping that it will speed the process for students to assimilate MLA format without it becoming a distraction or a more significant issue than it deserves to be.