Sunday, February 07, 2010

how can we best measure school success?

Normally, in the US, school success is measured through performance on standardized tests devised by experts, and understood by few and misunderstood by many. These tests are thus an extremely poor tool for evaluation. Some universities like Wake Forest no longer require standardized tests for college entrance and prefer to look at other evidence of achievement, maturity and preparation. On the K-12 front, things aren't looking quite so sane. The rise of standardized testing as the primary tool for evaluation of teacher and student performance and distribution of federal funding has led to "teaching to the test," and neglect of many important developmental benchmarks that can't be as easily measured. There are lots of credible academic authorities that concur with my point. Type testing in the search block and you will find earlier discussions.

But, how can we best measure success? Do we give up on testing as the primary means and not replace it with something more meaningful?

Rubric is a term originally meaning red earth or ochre, used as a color in sacred texts to highlight important points.

In education,
"A rubric is a scoring tool for subjective assessments. It is a set of criteria and standards linked to learning objectives that is used to assess a student's performance on papers, projects, essays, and other assignments. Rubrics allow for standardized evaluation according to specified criteria, making grading simpler and more transparent.

The rubric is an attempt to delineate consistent assessment criteria. It allows teachers and students alike to assess criteria which are complex and subjective and also provide ground for self-evaluation, reflection and peer review. It is aimed at accurate and fair assessment, fostering understanding and indicating the way to proceed with subsequent learning/teaching. This integration of performance and feedback is called "ongoing assessment."
It is of great value for students to know precisely how they are to be measured, and what performance goals are important for them to set for themselves. Progressive educators are making great use of rubrics in organizing and motivating student activities.

What we really need in schools to replace our focus on standardized testing would be a rubric or set of rubrics through which parents and students as well as teachers and administrators can monitor and measure school performance and that would encourage teacher and student creativity as well as proficiency in reading and math. This would be a system of measurement growing from the foundations of K-12 education rather than being imposed from the outside by behavioral science.

Of course, there will always be those so little interested in the personal effects of education that they will prefer to monitor its progress or lack of progress from a spread sheet. But a real revolution in education will come when we reassert common sense, and it would be best if it grew from the ranks of educators and parents rather than from the halls of congress.

Imagine a rubric for schools that would ask that students be creative in their search for solutions. If you were to design a rubric, what would its components be? What elements of a child's education would be most important and given greatest emphasis? All things, even creativity, honesty, courage, and joy can be measured or observed and graded through use of a rubric. How do you measure joy in learning? One marker is when students become so engrossed in learning that they have to be told to go home. That happens in wood shop and at Clear Spring School.

Actually, a rubric is an easy thing for a craftsman to understand. As one works on the creation of an object, the maker evaluates progress on a variety of fronts. The object itself is a rubric of sorts. The craftsman looks at the selection of materials, the design, the fit of parts, the surface qualities of the materials, the application and fit of the finish, and the usefulness of the finished form, in assigning his or her "grade" to the work, and the fine craftsman learns to settle for nothing less than the best on all fronts. In a sense, the use of a rubric is the application of age old principles, from before science. The image above shows rubric, the lettering in red, reflecting the original definition of the term. The academic meaning of the term can be found here.

9 comments:

Anonymous said...

The community college where I work is spending all kinds of time and effort trying to come up with rubrics that will fairly measure what students do. It's not an easy job at all.

Mario

anyn said...

Coming up with rubrics is easy. Coming up with effective rubrics is hard. Coming up with effective rubrics that a majority of the stakeholders can agree on is VERY hard. It's a nice idea but the method of soliciting, selecting, and validating the rubrics needs some fleshing out.

Doug Stowe said...

At least the process of having stake holders at the table would be more interesting than just going with the flow and allowing the educational testing services being the only ones telling us what kids need to know.

Here in Eureka Springs, our community and economy are built on a foundation of the arts. Should we have the same rubric as other towns? Or should our rubric reflect who we are, and what we have to offer to our kids?

Coming up with rubrics for the evaluation of our schools would lead us to some pretty interesting thinking outside the box.

anyn said...

So you don't think there should be, at a minimum, a national set of rubrics? That Eureka Springs, AR should be teaching similar things as Lexington, KY or Seattle, WA?

Sure, you can use the arts to do so but when it's all said and done I, personally, want to know that all those kids have learned some standard things to a certain skill level. Whether you want to measure achievement as a rubric or a standardized test is perhaps best left to the particular skill / concept being taught.

I think you're jumping the gun by suggesting it should ALL be rubrics - when all you have is a hammer, everything looks like a nail.

WordVer: chily...why yes it is where I live.

Doug Stowe said...

I don't think I ever said that all testing should be abandoned, or that it doesn't have some value. I agree that some skills and values should be shared across and between communities. My point is exactly yours. If the only tool you have is a hammer or a test, then you have very little means of teaching or inspiring in children the level of engagement and enthusiasm required for success in their education. And rubrics do work. They may be messy from an administrative standpoint but for kids to have the means to clearly understand what is expected of them and to know how to get the best results is a winning situation. The other thing I need to point out is that testing isn't necessarily a means of demonstrating skill, except the skill in taking tests. It isn't a good way to impart values. So if you want education to be about shared skills and shared values, what does testing have to do with it?

anyn said...

Doug, "These tests are thus an extremely poor tool for evaluation." While that's not a call to trash them all, I don't see how/why, if you have the opinion of standardized tests, you'd think ANY of them should be kept. That's reading into your text, but not too far IMO.

I think you're overstating your case again re: standardized testing. Testing is an evaluation tool - not a teaching tool. So there's NO WAY that ANY test OR rubric would be able to teach or inspire "...in children the level of engagement and enthusiasm required for success in their education." Tests also don't impart values. That's not their purpose. To denigrate standardized tests for being unable to perform certain tasks they they're not designed for is a bit over the top.

As for skills, I think multiple-choice tests should be enhanced with fill-in-the-blank questions, especially for math. The open-endedness would be a better tool for testing skills (like math) than A/B/C/D/E. That's just my opinion, but that's an adaptation to ye olde standardized test - not a call to get rid of them.

A rubric allows for fuzzier evaluations than a multiple-choice test. That doesn't mean a rubric is a tool for imparting values or certain skills - it's just different way of evaluation. Besides agreeing on a standard set of rubrics for all schools, the next most difficult part of implementing rubrics will be validation (i.e. third-party audits / rubric evaluations). That's one advantage of the current standardized testing regime - schools can only affect the students, not the test. It's unlikely there will ever be a national rubric scale based on schools reporting their self-scored rubric results. There's not enough systemic trust at this point for that to work without a very robust auditing program - which will add considerable administrative burden. Perhaps that additional burden is worth it...perhaps not.

Doug Stowe said...

Anyn,

If you have a hammer, you can very likely understand its use. I say tests are a poor tool because they are so abstract that teachers often mis-understand them, administrators likely don't fully understand them either, and parents and students while being jerked around by the testing scenario, often have very little understanding of them.


A rubric, unlike a test, most often involves a number of diverse values. For instance in evaluating a science project, the judge might give points for adherence to the scientific method, as well as points for originality, as well as points for presentation, in addition to points for having come up a valid conclusion. It is far different from just reaching a right or wrong answer, and yes, if you look closely at what I just described, it suggests values beyond those of merely being right or wrong. So, while your point that a rubric is a measurement tool is true, it also implies values.

I want to thank you for challenging me, and holding me accountable for what I say in the blog. I would also like to invite you to read what some educators have said who have greater involvement in the testing issues, than I. I am a simple craftsman, turned teacher, and there are many who have greater academic credibility than I. You might be interested in "Education Hell-Rhetoric Vs. Reality" by Gerald Bracey.

anyn said...

Doug, agreed that rubrics - unlike multiple-choice tests - can incorporate measures with a more general 'values' flavor than 'correct answer'. Regardless, you run again into the problem of results validation / certification. How does one ensure that the 'creativity' measures are consistent between school districts / schools / states / etc? That's a tough nut to crack and in all likelihood would require some pretty expensive processes.

The standardized test results aren't that obscure. Rather than call them a poor tool because people don't understand them / make more of them than they should, perhaps some outreach would be in order instead - much less expensive.

I'm not that into education at this point to buy a book. Sorry. Also I'm only a bit impressed with credentials so your suggested reading would have to stand on the merits and not because the author has longstanding links with education. Don't sell yourself short. There's little that you're doing that's 'simple'.

Doug Stowe said...

I think standardized test are pretty obscure and I studied behavior statistics in college. I'd be surprised if anyone would take enough interest in standardized testing to gain a clear understanding of it... just think of the people on Leno who can't tell the difference between North and South Dakota when staring at a map. My point about Gerald Bracey's boo, "Education Hell-Rhetoric Vs. Reality", is that it may seem that I'm out on a limb, but I'm not out here alone, and the branch is plenty stout enough to bear my weight.