Is Our Children Learning?

01-29-08

On Monday, USA Today released its latest report in a series  on standardized testing in American schools. The story focused on Noyes Education Campus, a PK-8 school in D.C., which had been singled out for praise by the city’s former schools chancellor, Michelle Rhee, because of a big jump in its test scores. USA Today found that there were widespread irregularities with the tests at Noyes — namely that a high number of the students’ answer sheets for the city’s standardized tests had erasures that looked as if the initial, incorrect answer bubble filled out by students  had been changed to the correct  ones.

There was a time when  standardized testing was widely seen as a necessary evil in education if not anathema to actual learning. But in the years since the passage of No Child Left Behind, testing has come to dominate discussions of education reform and classroom priorities. (Some districts devote several dozen school days to high-stakes testing each year.)    Charter schools tout their test scores in their fundraising efforts. Public schools with consistently low scores run the risk of being shut down. Several states are making “value-added” scores — an ostensible measure of how much an individual teacher improves student learning based on hitting test score targets — a central peg in determining how much a teacher should be paid, if she should be granted tenure, or whether she should be dismissed outright.

Dana Goldstein writes that given the stakes, it shouldn’t be too surprising that it appears some folks have been cheating to hit their numbers.

In the social sciences, there is an oft-repeated maxim called Campbell’s Law, named after Donald Campbell, a psychologist who studied human creativity. Campbell’s Law states that incentives corrupt. In other words, the more punishments and rewards—such as merit pay—are associated with the results of any given test, the more likely it is that the test’s results will be rendered meaningless, either through outright cheating or through teaching to the test in a way that narrows the curriculum and renders real learning obsolete.

In the era of No Child Left Behind, Campbell’s Law has proved true again and again. When the federal government began threatening to restructure or shut-down schools that did not achieve across-the-board student “proficiency” on state reading and math exams, states responded by creating standardized tests that were easier and easier to pass. Alabama, for example, reported that 85 percent of its fourth-graders were proficient in reading in 2005, even though only 22 percent of the state’s students demonstrated proficiency on the National Assessment of Educational Progress, the gold standard, no-stakes exam administered by the federal government.

The stat-juking going isn’t just happening on the school side, either. A Twin Cities City Pages report by Jessica Lussenhop found that the testing industry — which has tripled in size since 2002 — relies on poorly trained temps to score the essay portions of standardized tests. At NCS, one of the largest testing companies, the obvious problem of subjectivity — what, exactly, makes an essay good? — was addressed by a blunt rubric that, for some reason, graded essays higher when they contained longer paragraphs. (A “5” was excellent, “1” was poor.)

The scanned papers popped up on the screen and her eyes flitted as fast as they could down the lines. The difference between “excellent” and “good” and “adequate” was decided in a matter of seconds, to say nothing of the responses that were simply off the reservation. How do you score a kid who rails that his town sucks? What about an exceptionally well-written essay on why the student was refusing to answer the question?

There were the students who wrote extremely well but whose responses were too short—in his mind he saw them, bored with the essay topic, hurrying to finish. Or the essays where the handwriting got rushed and jumbled at the end, then cut off abruptly—he imagined the proctor telling the frantic student to lay down his pencil on a well-written but incomplete response.

And there were the kids who just did what they wanted. Like the boy from Arkansas who, instead of writing about the most fun thing to do in his town, instead wrote a hilarious essay on why his town is terrible and how he wanted to burn it down and pee on the ashes.

“I wanted the kid to get the score they deserved,” Puthoff says of his time in the business. “But they want to put them in boxes.”

Supervisors at this company were pressured to make sure the aggregated test scores resembled a bell curve. So when the scorers doled out too many 2s or too many 5s, workers alleged that their supervisors simply re-graded the essays so that the scores fell in line.

So even as we use tests as the foundation for more of our education policy, can anyone even say for certain what, if anything, we’re actually measuring?

G.D.

G.D.

Gene "G.D." Demby is the founder and editor of PostBourgie. In his day job, he blogs and reports on race and ethnicity for NPR's Code Switch team.
G.D.
  • Devona

    I am not a big fan of statistics. I get highly suspicious when people start throwing out numbers to to prove or not prove their stance on an issue.
    Having said that, I don’t believe these standardized tests are measuring what they need to measure. If the school system is trying to prepare 21st-century students to become 21st-century productive citizens, I think the education paradigm needs to change. The educational system is still working under the old model of rote teaching, memorization, instilling facts into the kids and then they in turn are suppose to regurgiate them for a test. In a recent article that came out stated that the number one thing employers look for in job applicants is if they are able to problem-solve and think independently. U.S. schools are not teaching our children how to critically think and solve problems. There is a difference between being educated and indoctrinated. And another thing, we need to also look at what motivates the students. Some of us were “indoctrinated to think: go to school to get a diploma to go to college to get a degree to (hopefully) get a good paying job. Why should these kids nowdays care about getting an education to get a job when all you have to do is make a sex video to become rich and famous. So, what’s motivating them to learn, to think, to discover, to create?

  • -k-

    That’s the writing article I posted about last week.. the rubric is pretty standard (and the answer to your question is: not really).

  • Tomlinson Street Dream

    Oh, they Is learnin’. You know what they learnin’ doe? They learnin’ that the adult teachers who attempt to tell them about life ain’t quite payin’ attention to the way they see things. You see, the youth see the world in the way that they feel it and internalize it. What do they feel? They feel that teachers, outside of the college ranks, are ignoring many truths as they (teachers) filter the real world issues out from the Math, History, English, and the rest. As a black student I learned this: “Boy they sure do love Martin Luther King, ‘cause he was nice.” Then the generation after me learned this: Why can’t Malcolm X get a holiday? Then the learning continues: “Money is the only thing that matters, and I ain’t got none.” “My big brother went to college, but now he owes a bunch of money and can’t find a job; college is a waste.”

    Ever since Barack Obama became President, some students are learning that “We are all the same, but my parents are different now that Obama is President; I guess my parents have a problem with black people.”

    Some students are learning that their parents do not want Barack Obama to visit their schools, not even to read a book to them or encourage them. They want to keep their students at home.

    I wonder if black students think the way I do: If many of the white people of the United States hate the President because he is black, then they must really think I am a piece of crap.

    Education is the operative word. Education before secondary school should be teaching the very issues that I am discussing here, because these issues are the ills of society. The youth should not be kept from the truth. Math should be tied to the economic lessons of the housing market collapse, the Wall Street mess, unemployment numbers, and similar dynamics. History should be tied to the ways in which politicians manipulate society for their own gain as well as to please the rich who promise them rewards. English should be tied to the fact that only in truth can a writer find his/her voice and the will to write. The books sold to school districts are tied to politics, powerful people, their friends, and families for monetary gain. Education is a business; it is a tool for social control and not for the betterment of society. If it was meant to help individuals, and if it was this country’s desire to educate, then college would be free.

    Lincoln freed our ancestors physically, because economic slavery was the up and coming invention. Economic slavery has now infiltrated the non-minority world. College is now a payment for one’s very own economic slavery (student loans without the promise of a job). I smell the fall of Rome.

    In all, I think them students is learnin’ the following: “Old people don’t behave or do the right thing, so why should we?”

    America is a female dog, and I am a Socialist. Amen.

    – Tomlinson Street Dream

    • ^can someone decipher this for me?

      • -k-

        You have to take it as meta-level commentary on the use of unweighted rubrics and the categories by which we judge writing quality: How does one score a comment that’s a 5-6 on grammar/mechanics and language skills/word choice, but a 2 on focus and development? Which is more important (and to whom)?

      • @G.D.

        Nope.

  • Tomlinson Street Dream

    G.D. Define decipher?

  • Oh wait, you mean an education system that puts a premium on testing will NOT result in better students but instead students who learn how to pass exams and rampant cheating? Where have I seen this before? Oh yeah, China. Per an earlier post:

    “…The Perfect Republican Educational System.

    All testing. Abstinence. No teacher’s unions. Schools are only judged by the amount of high-wage earners they produced. Nobody has the foggiest notion of what ‘critical thinking’ looks like. Students are in cutthroat competition with one another for grades, which at some point down the line actually do equal money (though I have yet to figure out the formula).”

    We should all be unified in our opposition to pushing testing. It is just step one towards the perfect Republican Educational system.

    • Tomlinson Street Dream

      I believe you winslowalrob. That was nice. I’m beginning to imagine what a Republican educational system would look like. They won’t be satisfied until there is zero black, brown, and poor combined with the elimination of a scientific perspective. Like the private college I received my undergrad from, students would be required to explain on every assignment how God fits in. They didn’t like me much.

  • Qalil

    The system has been broken for a while. I know a teacher who excelled in test taking and failed critical thinking. Eventually she got to college and after 10 years doing and re-doing classes she finally graduated.

    The last test she took – the one that would give her he certificate to teach – could only be passed by using critical thinking skills she had not honed. 6 tests and over $1,000 later she still had not passed it, and the University, feeling sorry for her, gave her the certificate and sent her into the City to teach other people’s children NOT to think.

    Sigh. Sometimes I think it is better not to send kids to school.

  • Tomlinson Street Dream

    Man I have been thinking about that post above by -k-. The first thing that sticks is the word focus. I like the way the student responded that his town sucks, and I like the way the other student refused to answer the question but still ended up showing his “exceptionally well-written essay.”

    Neurotypicals may see zero focus in another person’s writings, but I see a refusal to focus on what has been presented as “valuable” or important.

    If I got a five or six on a rubric for “grammar/mechanics and language skills/word choice,” then I would be thrilled.

    Like the aforementioned students, I refuse to let my education turn me into “The Man.”
    Americans are always “psuedo-liberals”; that is until they feel they’ve achieved some status.

    Courage.

    This concept always reminds me of why many of our black youth hold zero respect for academia. If we become educated, and in turn evolve into the essence of “The Man”, “The Republican”, or whatever term I come up with next, then we will only discourage our own. We will continue to disregard the fact that we need to take care of our own.

    This has been an alternative and “Meta-level Commentary” on everything my mind could bring in.

    Autistically Yours,

    Tomlinson Street Dream

  • Blaque Freud

    Well, I have this to say, India is a bitch when it comes to testing kids hard and early and throughout their education. Cutthroat competition just to get into a mediocre technical/engineering college for which you have excellent grades but need to cram and study 24/7 to pass the test. These kids study hardcore throughout their lives. They are not known for “critical thinking skills” or creativity or innovation BUT they grow up, come over to the United States and become YOUR doctors, engineers and investment bankers – while we sit around bemoaning the lack of “creativity” and “fun” in learning the American way.

    Someone’s making this type of education work in their favor – and it ain’t us!

    • Tomlinson Street Dream

      We can introduce all of India’s methods into the U.S.A., but until the racial discrimination, political posturing, and the lack of American togetherness disappears, loving and introducing India’s ways will not do a thing for us.

  • Blaque Freud

    But their ways are rote learning and tons of testing — similar to our’s but much more strict, much more competitive. The US is all feel-goody-airy-fairy-hippy-dippy compared to India. That’s exactly why when they come over here they form “the model minority” and succeed like anything.

    They are also a shame-based culture. No such thing as a “baby-mama” or “baby-daddy”. HELL NO!

    Their out of wedlock birthrate is practically nil and their divorce rate is very low. They are a very family oriented culture with arranged marriages. Dating is not allowed (only in some metros amongst “westernized” Indian, so forget “hooking up” and getting knocked up – ain’t gonna happen.

    Of course they do have a high teen pregnancy rate – MARRIED teens. It is still common for parents to arrange the marriages of their teen sons and daughters in the rural villages.

    Contrast this with the US where o-o-w-t-b’s (out-of-wedlock-teen-births) are concerned perfectly normal and tolerated, and some young girls see it has a path to an easy income!

    There is no way the government of India would PAY young, unmarried girls and women to get knocked up – and OUR government shouldn’t either.

    PSA for American teens: Use birth control or keep yer dang legs shut!

    It ain’t that hard (pun intended).

    • Dude, it’s 1:30 in the morning here, I have a vicious rant that I will hopefully not put up tomorrow, because you are speaking some nonsense.

  • Interesting post but–couple things:

    I think the “creative/critical thinking” vs. “rote learning” binary is a false one. A good education requires a balance of both. Yes, it is not preferable to train kids to be test-taking robots… but nobody ever learned to play the piano–or algebra–by thinking “creatively” about it. There are many important things that are best learned via disciplined repetition.

    But I frankly reject the notion that the problems of “youth” can be directly addressed by education reform. Social promotion and parental pressure on schools not to stigmatize students by holding them to strict standards are the privileged student’s version of the erased answer-bubbles at Noyes Education Campus. The fact that such young people often “succeed” in life anyway has to do with cultural privileges that cannot be accounted for via education reform. But you are not going to see newspaper articles about this phenomenon– even though it is pervasive– because it doesn’t subtly reinforce the message that the only way minority kids can suceed is by cheating.

    But privilege in and of itself is no guarantee of anything learning-wise. Just because a kid has fancy educational goodies available to her (extensive libraries, computers, safe/clean/expensive/new learning environments etc.) doesn’t mean she will take advantage of them. In a political sense it makes a difference (at least to me) if a kid isn’t learning because she is so privileged that she does’nt think she has to try or if she is exhausted from working a night shift before coming to class– but the result is the same: a student who is not learning. The difference is that privileged parents are not hand wringing about social promotion etc (if they even acknowledge that it exists)–just the opposite. Not only is it not a “problem” it is an unspoken–but carefully enforced– central element of the culture of elite schools. How quickly would politicized discussions about “education reform” wither and die if these practices at comparatively privileged schools were also implicated? Exactly.

    The issues here–that privileged kids only tend to fall so far no matter how badly they fuck up while kids dis-privileged by class, race etc have to work harder for fewer options is a Macro problem. “Education reform” is a red herring in that sense. And the ultimate effect of such arguments a) demonizes teachers and their unions b) reinforces ugly stereotypes about the relative stupidity of dis-privileged minority students and c) obfuscates that open secret that schools that cater to privileged kids bump them forward (ie. up) as a matter of course too.

    • I think the “creative/critical thinking” vs. “rote learning” binary is a false one. A good education requires a balance of both. Yes, it is not preferable to train kids to be test-taking robots… but nobody ever learned to play the piano–or algebra–by thinking “creatively” about it. There are many important things that are best learned via disciplined repetition.

      i don’t disagree. there’s a big difference, tho, between learning how to multiply through repetition and memorization, and learning how to read or think about writing in a way that rewards nailing some multiple choice questions; the benefits of the latter skill set don’t echo in many other contexts besides test-taking.

      The issues here–that privileged kids only tend to fall so far no matter how badly they fuck up while kids dis-privileged by class, race etc have to work harder for fewer options is a Macro problem. “Education reform” is a red herring in that sense. And the ultimate effect of such arguments a) demonizes teachers and their unions b) reinforces ugly stereotypes about the relative stupidity of dis-privileged minority students and c) obfuscates that open secret that schools that cater to privileged kids bump them forward (ie. up) as a matter of course too.

      this touches on some conversations that me and my blogmates have had in different contexts, but weirdly not so far on PB. generally, I think that ed reform is nibbling at the edges, and that the big issues holding back students would involve directly alleviating poverty. but there’s not really the stomach for Great Society-type programs that dealt directly with this. The current shape of education reform, to a large extent, is a sincere attempt to tackle some colossal societal issues through a bunch of small-bore solutions that are politically feasible. but, as you know, feasibility and effectiveness ain’t really the same thing.

      • @GD
        My concern is that under the rubric “education reform” teachers will be the sacrificial lambs in the upcoming elections as part of a larger conservative project to disempower unions. The number of social problems that are laid at the feet of teachers and schools is dizzying. Bottom line is that no amount of education reform is going to undo the larger effects of racial or class dis-privileges, which are the true cause of poor school performance for kids impacted by them. *Not* (or not only) poor parenting or drugs or teen pregnancy–ALL of which also impact privileged kids as well– but rather the price paid for those things is exponentially higher for a poor, minority kid than her suburban, upper-class counterpart. But there is literally nothing that teachers can do about that. And yet teachers are–in discourses on the Right AND Left–being held responsible for the performance of kids while they are in school and for their choices once they leave.

        As we move closer to 2012 the noose will tighten for teachers (and their unions) but the justification will be “low test scores” etc. While I don’t disagree that some people calling for education reform are sincere, the ultimate effect will not be a meaningful reform of public education, which serves largely immigrant/working-class/minority students. Because the conservatives leading this charge do not care about them while the liberal “white man’s burden” types base their “help” on the premise that such students are inherently flawed due to their circumstances, and neither position could ever lead to real education reform. I think that the political project of education reform is centered on teachers, not students in any case. And my concern is that the politically feasible “small bore” solutions your refer to will end up with teachers more dis-empowered than ever, tenure eroded, and teachers unions broken.

        … And the social, cultural and political circumstances that makes it more difficult for disadvantaged students to succeed will be left intact. A win-win for the Koch brothers.

  • Ha.

    “doesn’t”

    I think I just made my own point about rote learning…

  • Blaque Freud

    “But I frankly reject the notion that the problems of “youth” can be directly addressed by education reform.”

    I say the bad teachers are few. Bad students – many.

    What causes bad students? Usually bad parenting.

    What is bad parenting? Kids having kids.

    Again, it comes down to discipline and self-control.

    Kids who are raised in happy, healthy 2 parent homes usually don’t have the “issues” that kids from broken homes have.

    I work with kids and I see this phenomena over and over again.

    They should’ve done something about The Moynihan Report back in 1965. It was poo-pooed and thrown under the bus and now 45 years later the dysfunctions that plauged the inner cities are now plagueing the entire nation.

  • Tomlinson Street Dream

    Shahadi,
    That was awesome.

    Blaque Freud,
    I agree that bad teachers are few. Many of those few are victims of our misguided American culture. I do not necessarily believe in bad students; from a systematic perspective, bad students are the products of parents who struggle in life, and those “bad parents” are the victims of society. I never let our diseased American culture off the hook for the disease it has created and maintained in our people. I blame the rich, racists, and Republicans.

  • Blaque Freud

    “I blame the rich, racists, and Republicans.”

    The richies, racists and Republicans ain’t puttin’ no gun to your head and tellin’ you to spread your legs for no jobless hood rat (or to spread your seed inside a jobless hood rat).

    I don’t blame anyone.

    I make choices.

  • Tomlinson Street Dream

    I hear you Blaque; you sound like one of the lucky ones. Some people are not able to make the “good” choices. When folks are helpless, hopeless, and without material toys to distract them, they often turn to the only free pleasures available. On a micro-level, that is exactly what “they” have done to the black, brown, and poor. Those of us who are upwardly mobile and allowed to enter certain employment circles tend to forget. We forget because we have it good. We forget because we don’t want to remember the days before we found some success.

  • Blaque Freud

    “you sound like one of the lucky ones”

    Do I?

    No luck here. Just common sense.

    • Tomlinson Street Dream

      “No luck here. Just common sense.”

      It is all about you. It’s all about how you would respond to life when walking in the shoes of others.

  • Blaque Freud

    Nope. Its all about how I respond in to life when walking in MY shoes. And my shoes were not, contrary to your assumptions, easy at all. Nor were they expensive.