Showing posts with label tests. Show all posts
Showing posts with label tests. Show all posts

December 17, 2011

Mexican mediocrity quantified

Something I noticed last year when looking at 2009 PISA school achievement scores is the virtual non-existence of Mexico's intellectual elite. Mexico's average scores on this school achievement test of 15-year-olds were mediocre, but the lack of high end scores was startling, compared to a similar scoring country like Turkey, where there is a definite class of very smart Turks. Obviously, there is a stunning shortage of very high-achieving Mexican Americans in the U.S., but I had tended to assume that the really smart guys who run things in Mexico were just foisting off their mediocre people on the U.S. Yet, it's hard to find test score evidence that there are many really smart guys in Mexico at all. This is not to say the average Mexican is all that uneducated by global standards, just that the far right end of the bell curve in Mexico is a lot thinner than you'd expect.

Perhaps this is just an illusion because all the schools in Mexico with smart students refuse to participate in international tests? The public school teachers union in Mexico is hilariously awful: many teaching jobs are hereditary, and if your heirs don't want your teaching job after you die, they can auction it off to the highest bidder. But the overall performance of Mexican students on the PISA isn't terrible (it's a lot worse than the performance of Hispanics in the U.S. on the PISA, but not miserable by Latin American standards). 

Yet, here's a 2008 paper on the same subject that takes the lack of cognitive superstars in Mexico seriously:
Producing superstars for the economic Mundial: The Mexican Predicament with quality of education 
Lant Pritchett and Martina Viarengo
Harvard University, John F. Kennedy School of Government
November 19, 2008 
Abstract.   The question of how to build the capabilities to both initiate a resurgence of growth and facilitate Mexico’s transition into a broader set of growth enhancing industries and activities is pressing.  In this regard it seems important to understand the quality of the skills of the labor force.  Moreover, in increasingly knowledge based economies it is not just the skills of the typical worker than matter, but also the skills of the most highly skilled.  While everyone is aware of the lagging performance of Mexico on internationally comparable examinations like the PISA, what has been less explored is the consequence of that for the absolute number of very highly skilled.  We examine  how many students Mexico produces per year above the “high international benchmark” of the PISA in mathematics.  While the calculations are somewhat crude and only indicative, our estimates are that Mexico produces only between 3,500 and 6,000 students per year above the high international benchmark (of a cohort of roughly 2 million [which is about half America's cohort of around 4 million]).  In spite of educational performance that is widely lamented within the USA, it produces a quarter of a million, Korea 125,000 and even India, who in general has much worse performance on average, produces over 100,000 high performance in math students per year.  The issue is not about math per se, this is just an illustration and we feel similar findings would hold in other domains.  The consequences of the dearth of globally competitive human capital are explored, with an emphasis on the rise of  super star phenomena in labor markets (best documented in the USA).  Finally, we explore the educational policies that one might consider to focus on the upper tail of performance, which are at odds with much of the “quality” focus of typical educational policies which are often remedial and focused on the lower, not upper tail of performance.

I don't know what the full story is here. Perhaps Mexican elites are just lazy, and they set a bad example for the Mexican masses?

PISA scores: 2 Indian states flop

The conventional wisdom expressed in Obama Administration speeches and the like is that American students get crushed by kids in China and India on international tests of school achievement. But the evidence for this is not as abundant as you might assume ... especially not for India. While the city of Shanghai shot the lights out on the 2009 PISA, test scores haven't been released for other parts of China. 

But, Westerners going back to Marco Polo have generally assumed the Chinese have a lot on the ball, so they are likely to do pretty well. 

What about India, the other giga-country? I first noticed in early 1981 that there were a lot of smart Indians in the U.S., and over the decades this has become a cliche.

But, what about India itself? India has never participated as a country in broad-based international tests.

The future of India is A) an intrinsically interesting subject; B) one that could make or lose you a lot of money; C) could make or lose the whole world a lot of money (just as the widespread assumption that the population of the Sand States could pay back those big mortgages proved costly for everyone, so could an unrealistic assumption down the road that the Indian masses are ready for big loans could spark a future global bubble and bust).  

Last year, I pointed out in VDARE that TIMSS had been given unofficially in two Indian states, Orissa and Rajasthan, and both had done badly.

Now the OECD has released 2009 PISA test scores for 15-year-olds for ten more places, two of them Indian states. The new news is that the Indian states, Tamil Nadu in the southeast (east of Bangalore, the technology center) and Himachal Pradesh, a Hindu state in the Deep North, did miserably, fighting it out with Kyrgyzstan for last place out of 74 countries or regions on all three tests: reading, math, and science. (Not surprisingly, the southern state beat the northern state on all three tests.) 

And there isn't much inequality in the Indian scores: it's not like some geniuses in these states score high but the places are dragged down by illiterates. There are a lot of illiterates, of course, but almost nobody scores at the top level, at least not in the schools where these tests were given. (Allow me to insert here my usual caveat about testing, which is that an 80/20 rule applies to methodology: it's pretty easy to get a crudely accurate picture, but really hard to get a highly accurate one. For example, how representative were the tested students in India? Beats me.)

Here's the 13 meg PDF.

India ought to be able to do better than score at sub-Saharan levels. Indians in other countries do better. For example, this same report has Mauritius, a mixed race country in the middle of the Indian Ocean where 52% of the population is Hindu, scoring like a Latin American country rather than a sub-Saharan African country. 

But, India itself has a long way to go. It's likely to take 1-2 generations to get India up to speed, and we don't really know what up to speed for India means yet.

In other news from this report by Australians on the additional ten marginal places to take the test, Costa Rica does pretty good for a Latin American country (as stereotypes of Costa Rica as a nice place would suggest), Malta does okay, the rich United Arab Emirates do pretty good for an Arab place, and Moldova and Georgia do very bad for white countries. 

September 19, 2011

Asians, aptitude, and achievement: a positive sum reform proposal

The traditional concept of college admissions was that the goal was to predict applicants' future achievement (which could be measured in terms of first year in college grades or money donated 50 years later or whatever). The most obvious way to predict future achievement was past achievement: e.g., high school grades. Presumably, past achievement had two main components: hard work and aptitude.

But there were some obvious problems with relying solely on high school grades, such as different levels  of grade inflation at high schools. If you were getting most of your applicants from St. Paul's and Dalton, well, you could keep in mind the differences, augmented with letters of recommendation from headmasters you had known for years. 

But once the Ivy Leagues started trying to find the most promising non-upper class kids from the rest of the country, they needed something more objective about individuals than just grades. Another issue is that high school grades have certain inherent shortcomings. The future Nobelist in physics might not care about his social studies class and thus wind up with a lower overall GPA than the well-adjusted grind. Plus, grades have a ceiling. Even an A+ in physics doesn't really tell you that much. Moreover, lots of future successes are alienated in high school. Some people who get all As in high school might not have the upside to continue to do so in college. Incentives toward grade inflation at the high school are built in. And so forth ... Top colleges kept asking for recommendation letters, but their value (and, thus, importance) dropped as they increasingly came in from random teachers in random places like Burbank, CA.

So, for various reasons like this, the Scholastic Aptitude Test was created and spread. The idea was to have an objective, national test of academic intelligence. Overall, the SAT would appear to have been a huge success. American colleges are the most fashionable and richest in the world today. 

However, there have long been complaints about the SAT. The most fashionable involved The Gap. Whites averaged higher scores than blacks. This posed a major PR problem for the academic establishment. The SAT (and ACT) is essential for their continued thriving, but saying that blacks are less intelligent than whites on average is The Worst Thing in the Whole World. But that's what the SAT says. And the SAT is the cornerstone of academic elitism, which has made American academia globally the envy of the academic world.

Thus, over the last half century or so, there have been anguished discussions between the front men for the academic world and the psychometricians at ETS about how to Close the Gap, without throwing the baby of predictive power out with the bathwater.

One change was purely PR: the SAT doesn't stand for Scholastic Aptitude Test anymore. It just stands for SAT these days. Under the hood, there have been a host of tweaks intended to narrow the gap without trashing the predictive powers of the SAT too much.

For example, the upper range of the Verbal (now Critical Reading) test has been capped. Before 1995, it was very, very hard to get an 800 on the Verbal test. I came fairly close the first time I took the test in 1975, so I gave it another try, got a little closer, but gave up and didn't take it a third time because the two scores seemed quite accurate: I'm very good at verbal logic, and have a certain gift for insights that other people wouldn't come up with, but I'm not a meticulous thinker. I make lots of mistakes. I'm more of a let's run it up the flagpole and see if anybody salutes thinker. In contrast, say, Charles Murray's brain works like a BMW V-12: powerful and precise. Mine's a jalopy that might surprise you and win the race or might break down on the starting line and go nowhere. So, there didn't seem like much point in me doing a lot of test prep to try to score 800 on the verbal -- I'd still make a mistake or two or they throw a really hard question at me.

But now, an 800 is well within reach of a lot of well-drilled students.

So, before scores were inflated in 1995, the SAT-V was an excellent test of high end verbal brainpower. In contrast, the SAT-M was widely recognized to need more headroom. It wasn't uncommon at Rice in the 1970s to hear good but not great Sci-Eng majors say, "Well, sure, I got an 800, but I'm not a real 800 like Joe is."

An obvious reform would have been to make scoring of SAT-M more like scoring of SAT-V. Instead, College Board - ETS did the opposite in 1995. One reason was that all that headroom on the Verbal modestly increased The Gap. The V test was made much easier to score 800 upon in 1995. A 730 old style became an 800 new style.

Lots of other tweaks were made, but as far as we can tell, anything that raises black average scores just encourages harder scraping of the bottom of the barrel by society, so the white-black The Gap remains remarkably stable over the generations. For example, The Gap on SAT-Critical Reading dropped about a half decade ago, perhaps because of the changes on the test, such as deep-sixing analogies. But that apparently just encouraged the College Board to troll for more black test-takers with free tests, so The Gap is now even bigger. (I may be overinterpreting a few squiggles on the trend graph.)

But white parents still tend to assume that SAT stands for Scholastic Aptitude Test. It's not an achievement test in their heads. The College Board says there is no point in studying extra hard for the SAT, and why would a prestigious not-for-profit institution spin the truth? If you can't trust the College Board, who can you trust? And signing your child up for intensive test prepping would be unfair to poor blacks who can't afford all that tutoring and drilling. Plus, prepping for years would be a lot of work for little Taylor, so just let him have his fun.

Meanwhile, lots of people from Fujian are showing up in America whose merchant ancestors ascended to mandarin status by spending their mercantile profits at Confucian literature cram schools for their sons. The assumptions about the SAT flitting around in white people's heads would never occur to them. "Test prep is unfair to poor blacks? Huh? You crack me up! I like you! You are very funny!"

Not surprisingly, we see vast amounts of white upper middle class rage directed at Amy Chua. 

Now, is devoting hundreds of hours to prepping for the SAT a Good Thing or a Bad Thing? Well, let me try to reframe that question more productively and think about SAT test prep's opportunity cost.

It could be that SAT test prep has long term benefits other than getting into a fancier college. Could be ... I dunno. I haven't seen any evidence one way or another. But, it seems more like a zero sum game. SAT test prep seems kind of a sterile form of studying compared to studying an actual subject like Physics or French or Music Theory or World History or Microeconomics.

The return on investment for the colossal number of hours devoted in recent decades to SAT cram schooling is modest. The test is designed to be hard to prep for, so it's taken gigantic efforts for gains measured in fractions of a standard deviation. 

It seems to me that it would be better for everybody if more test prepping energy was invested instead into positive sum games, such as studying for achievement tests rather than for aptitude tests. Fortunately, we currently have a quite good set of national achievement tests: the College Board's Advanced Placement tests. They are not subject to the incentives for high schools to inflate their grades: the AP tests provide objective national grades that do a good job of predicting what the high school student would score on a 101-102 level course as a freshman in college.

In contrast, AP tests are intended to be ones you can study for, so the ROI on test prep effort tends to be quite a bit higher than on the SAT. Far more students are passing AP tests than a decade ago, and that's a good thing. Why have young people waste their time studying for something that's built to be hard to study for when they can instead study subjects that are intrinsically worth studying?

Unfortunately, the current college admissions system gives little weight to AP test scores. Instead, perversely, it gives too much weight to taking AP classes in high school, even if you then do bad on the AP test. The University of California, for example, in calculating high school GPA adds a full point to classes designated Advanced Placement. Thus the average high school GPA of UC Berkeley freshmen is a wacky 4.39. An internal study by UC showed that cutting the bonus for an AP course down to 0.5 would better predict freshmen grades. I don't believe the UC system counts AP test scores  in the admission's process, or doesn't count them much.

As Mitch has pointed out, this system is doubly rigged in favor of the more goody-two-shoes high school students. Typically, you need high grades in earlier classes to get into high school AP classes, where you are then given a full extra point for your GPA -- even if your AP score shows you didn't actually learn much.

So, the ideal system would be for college admissions to be retooled from mostly a two legged stool of grades and SAT scores to a three legged stool of grades, AP Test scores, and SAT scores. The SAT could then be redesigned to be more purely an aptitude test that would be less easy to game. Moreover, test takers would have less incentive to devote hundreds of hours to gaming the SAT because they were being encouraged to spend hundreds of hours mastering AP Chemistry or AP European History. 

The AP tests should be reformed to make them better for college admissions. They are currently scored on a 1 to 5 scale with a 5 equating to an A in the average college's freshman year introductory course in that subject, a 4 equal to a B, and so forth. A weird aspect of this is that all 5s are not created equal. For example, to get a 5 on the AP Chemistry test, you have to get a little over 60% right. So, people getting 98% right don't get a higher score than people getting 68% right. Test scoring should be kept the same at the lower levels -- a 3 would still be a C -- but the maximum score extended from 5 out to 7, which would be like an A at Caltech. Meanwhile, the GPA boost from taking an AP course would be eliminated, at least before senior year.

The initial winners from this changeover would, of course, tend to be Asians, who currently take a lot of AP tests. But good for them. Whites in heavily Asian areas, who have already started to adapt to the Asian challenge, would do okay. Whites in flyover regions would be challenged to get on the ball with AP. My guess is that it would be good for them and that they would eventually respond well to the challenge.

Overall, my plan looks like it would be better overall for society. There's a huge amount of energy out there looking to get an advantage in the college admissions process, so why not direct it in some positive sum direction?

Yes, sure, obviously it's a win-win, but, does it solve America's most overwhelming problem: Closing The Gap? Will blacks come closer to whites in scores under my system?

I dunno. I haven't thought about it. In fact, not worrying about Closing The Gap has allowed me to put forward a novel reform suggestion that might be better overall, which is not something you see too often these days.

In America today, 98% of the thinking devoted to college admissions goes to figuring out how your own kid can claw his way to the top, and the other 2% goes to airy handwaving theorizing about Closing The Gap. That leaves 0% devoted to thinking about improving the system overall.

Now, if I were truly, fanatically public-spirited, I would devote a lot of energy to dreaming up some bogus but persuasive-sounding theory about how my reform would Close The Gap, which would make it a lot more likely to be adapted. But, I'm not saintly enough to make up an elaborate lie.

September 16, 2011

SAT score trends

The Unsilenced Silence blog has a good graph of Asian v. white SAT scores from 1996 to 2010 in terms of gaps in standard deviations:
So, like most things involving test scores, the gap was stable in the  later 1990s. However, Asians improved through much of the 2000s, especially on the Writing test, a new section of the SAT that strikes me as easier to game because it uses subjective (but very quick) grading. 

One possibility is that globalization of American colleges has led to more Asian-in-Asia elites having their kids take the SAT, which could be increasing the pool of smart Asians taking the test. I'm also concerned about test security, however, when the test is administered in Asia. The Graduate Record Exam has been plagued by instances of cheating in Asia. Also, the new Writing test is rather like the tests that the Chinese Empire conducted for millennia to pick mandarins. 

Another possibility is that the legal immigration system is working in bringing in smart Asians, who are becoming a larger fraction of the Asian-American population over time. But what about regression toward the mean? Amy Chua is very smart, for example, but she's nowhere near as smart as her father, the conceptualizer of the memristor.

The take home lesson for Americans would seem to me to be that Americans shouldn't credulously trust assertions by the College Board / ETS / ACT that their tests can't really be gamed through test prep or other means. The SAT and ACT are, relatively speaking, very good tests, but we live in a highly competitive, globalized world full of people who are smart, hard-working, and less trusting and less believing in Fair Play than naive Americans.

Now, here's his graph of the white-black SAT gap, which has been widening:

The sharp decline on The Gap in  2006 on the Verbal test (renamed Critical Reading) may have something to do with changes in that test, perhaps intended to make it easier on blacks. But, the general trend has been for The Gap to get worse, especially since the early 1990s. Yet, since the early 1990s (the Crack Years) were not a halcyon period for young blacks, the widening of the gap probably reflects concerted efforts since then by the Great and the Good to get more blacks to take the SAT (scraping the bottom of the barrel harder). 

The process might work like this: the University of California, prevented from using racial quotas by Prop. 209) demands changes in the SAT Verbal that it hopes will boost black and Hispanic scores. A small amount of initial success in narrowing The Gap, however, leads elites to then over-confidently subsidize more black test-takers. Scraping the bottom of the barrel harder drives The Gap in Verbal back up, and increases The Gap in the other two sections to new heights. (Or I may be over-interpreting this.)

Also, it's likely that as test prep and gaming of the SAT has increased, that whites have benefited more from this than blacks. My mental model of who is doing more and more effectual test prep is Asians > whites > blacks and/or Mexicans > Other Hispanics (many of whom just arrived from Guatemala and are pretty clueless about the kinds of things that lead to higher scores that seem obvious to, say, a Hong Kong Chinese family in San Marino, CA).

Anyway, the White-Black gap graph shows the stability in gaps that is the overwhelming main lesson of generations of test score studies. Despite intense efforts by society since the mid-1960s, we still see about a one standard deviation gap between whites and blacks on a host of tests of cognitive ability (La Griffe du Lion's Fundamental Constant of American Sociology). But, that makes the recent change in Asian v. white scores even more interesting and more deserving of study.

Unsilenced Science also has informative graphs showing trends by year in SAT scores (i.e., not standard deviations).

September 15, 2011

SAT score changes by race since 1996

The oldest SAT score report on the College Board website is from 1996, right after the "recentering" in 1995 that raised scores about 100 points on a 400 to 1600 scale. Over the last 15 years, the average overall score on the original two-part Verbal + Math SAT (i.e., ignoring the new-fangled Writing section of the test introduced in the last decade) fell a grand total of two points, from 1013 to 1011. (See what I mean about baseball statistics being more volatile?)

1996 v. 2011 College-Bound Seniors Avg SAT Scores










Total (V+M)
Verbal

Math

1996 2011 Chg 1996 2011 Chg 1996 2011 Chg
ALL 1013 1011 (2) 505 497 (8) 508 514 6
Female 995 995 0 503 495 (8) 492 500 8
Male 1034 1031 (3) 507 500 (7) 527 531 4
Asian 1054 1112 58 496 517 21 558 595 37
White 1049 1063 14 526 528 2 523 535 12
Black 856 855 (1) 434 428 (6) 422 427 5
AmerIndian 960 972 12 483 484 1 477 488 11
Mexican 914 917 3 455 451 (4) 459 466 7
PR 897 904 7 452 452 0 445 452 7
Other Hisp 931 913 (18) 465 451 (14) 466 462 (4)

You'll note that the average white score went up 14 points form 1049 to 1063. Did white people get smarter over that period? I don't know. The SAT changed a lot over those 15 years, with analogies being dropped and some Verbal multiple choice questions being exiled to Writing. Also, kids appear to have cared more about prepping in 2011, although the College Board doesn't like to talk about this.

Asians went up 58 points, which is pretty striking. Everybody else fell farther behind whites, which wasn't supposed to happen.

Now, it could be that scores actually did pretty well over this 15 year stretch, because the College Board scraped the bottom of the barrel harder. In 1996, 1,085,000 college-bound seniors took the SAT. In 2011, there were 1,647,000 senior SAT-takers. Just between 2006 and 2011, the College Board let an incremental 150,000 students take the SAT free or at reduced cost.

On the other hand, my impression is that it became a lot more common for students to take both the SAT and ACT over that 15-year stretch, so some of the increase in the number of test-takers comes from people who would only have taken the ACT in 1996. It used to be that East and West Coasters took the SAT and Midwesterners the ACT, but by 2011, lots of students try both to see which one they'll do better on. These kids who take both tests probably tend to be fairly ambitious ones who are looking to game the system by taking both tests, then submitting only the test score they did better upon. So, double-dippers likely scored reasonably well (although, of course, not so 2400 / 36 outstanding that they wouldn't bother taking any test again).

(Has anybody recently done an authoritative study of the trend in overall SAT scores considering all the factors driving scores up or down?)

Between 1996 and 2011, everybody except Other Hispanics got a little better in Math. (I suspect that Other Hispanics used to be mostly Cubans and random fairly well-to-do South Americans, but now it includes a lot of Central Americans.) But Asians got a lot better: 37 points, from 558 to 595.

Verbal scores stagnated or declined slightly, except for Asians, who went up 21 points from 496 to 517. 

Now, let's look at scores relative to the white scores in 1996 and 2011. A decade and a half ago, the overall score for everybody was 36 points lower than the white score. Today, it's 52 points lower. Most of that 16 points of relative decline is due to the demographic composition of America's SAT-takers changing for the worse.

Difference v. whites


Total (V+M)

1996 2011 Chg
ALL 36 52 (16)
Female 54 68 (14)
Male 15 32 (17)
Asian 5 49 44
White 0 0 0
Black 193 208 (15)
AmerIndian 89 91 (2)
Mexican 135 146 (11)
PR 152 159 (7)
Other Hisp 118 150 (32)

The Gap got worse for most of the minority groups that the press gets worked up over. Blacks fell from 193 points behind whites to 208 points (a 15 point relative decline, or a point per year). Mexicans fell from 135 lower to 146 lower. Other Hispanics fell the most, from 118 behind to 150 behind.

These declines are probably mostly due to society (especially the College Board) scraping the bottom of the barrel harder in 2011. What with the recession and all, everybody is convinced that they must go to college, so they try the SAT. The number of people who scored below 400 on Verbal grew from 179,000 to 302,000 and on Math from 172,000 to 251,000.

The number who scored 700 or higher also shot upwards, but that might be due in part to kids taking the SAT more times or taking both the SAT and ACT to see if they can shoot the moon. The number scoring 700 or higher on Verbal went up from 47,000 to 77,000 and on Math from 58,000 to 112,000. High scorers are presumably the most likely to do a lot of test prep and otherwise try to game the system.

In contrast to all other ethnic groups, who fell farther behind whites over the last 15 years, Asians had a 5 point advantage over whites in 1996, which blossomed to a 49 point lead by 2011, a relative change of 44 points.That's a big change, relative to the near-stasis on everything else.

September 14, 2011

Asians pulling away in SAT scores

From FairTest:

2011 College-Bound Seniors Avg SAT Scores

W/score changes from 2006





READING MATH WRITING TOTAL
ALL 497 (-6) 514 (-4) 489 (-8) 1500 (-18)
Female 495 (-7) 500 (-2) 496 (-6) 1491 (-15)
Male 500 (-5) 531 (-5) 482 (-9) 1513 (-19)
Asian 517 (+7) 595 (+17) 528 (+16) 1640 (+40)
White 528 (+1) 535 (-1) 516 (-3) 1579 (-3)
Black 428 (-6) 427 (-2) 417 (-11) 1272 (-19)
AmerIndian 484 (-3) 488 (-6) 465 (-9) 1437 (-18)
Mexican 451 (-3) 466 (+1) 445 (-7) 1362 (-9)
PR 452 (-7) 452 (-4) 442 (-6) 1346 (-17)
Other Hisp 451 (-7) 462 (-1) 444 (-6) 1357 (-14)

I've been following baseball statistics since 1965 and educational test scores since 1972. Test scores are vastly more important for understanding how the world works, but they aren't as diverting because they seldom change. For example, in the above, whites are just treading water, down 1% of a standard deviation over half a decade. Boring. NAM scores are down, probably mostly because the College Board has been subsidizing more NAMs to take the SAT for free as a publicity move. In other words, it's probably not a real change.

But, wow, as I pointed out last year in writing about PSAT National Merit Semifinalists, Asians have just been pulling away from everybody else in the last few years.

Is the same trend true on low stakes tests, like most school achievement tests that are used to grade schools rather than students?

This is a big story and it deserves more research. Is the innate intelligence of Asians going up? Or does this prove that Tiger Mothering works? Is the SAT being unfairly gamed? There are a lot of questions here.

April 25, 2011

A retired teacher speaks out

From the Boston Globe:
A lesson in Advanced mis-Placement 
By Junia Yearwood 
THE AP English test in May, 2009 marked the end of a tortuous journey. We teachers served breakfast, gave a rousing pep talk, and the students trooped into the English High School auditorium to battle the exam. 
“What the hell am I doing in AP?’’ Veronica (all students’ names have been changed) cried the next day when the class reconvened. 
The outburst shattered the stillness of my 12th grade classroom, and all heads turned in Veronica’s direction at the back of the class. Her eyes blazing red and her face taut with anger, she continued to vent her frustration at her forced retention in an assigned Advanced Placement class despite her repeated requests for a transfer. ... 
Nevertheless, the majority voiced varying degrees of Veronica’s anger. She railed against the injustice of being denied a transfer from the class and her total lack of preparation for the academically rigorous course. She believed that the D for English on her last report card in 11th grade was a clear indication of her ongoing struggle to achieve proficiency in standard English. Haitian Creole was her first language. 
For the four years I taught the AP English and composition course at English High, many of my students were victims of the AP mania that had invaded the system. Suddenly, officials had recognized the dearth of faces of color in AP classes and the drive to augment the AP minority population went into high gear. 
The College Board and sympathetic philanthropic rescuers rushed in to solve the problem by dangling the carrot of grant money, and the feeding frenzy was on. AP classes sprouted and multiplied across all disciplines. AP scouts scoured students’ report cards hunting for qualifying scores; teacher recommendations were solicited for students with the “potential’’ to do AP work, and the nominees were summarily conscripted. 
Even though students had marked deficiencies in basic reading and writing skills, and little desire to work hard, and even though they made repeated requests for transfers, the dragooning of students into my AP course persisted. 
... I often wondered what parents would say if they knew that many of my AP graduates were placed in no-credit, remedial reading and writing courses their freshman year in college, and that, in spite of our school’s “underperforming status,’’ as designated by the state Department of Education, our AP enrollment was second only to our city’s prestigious exam schools. 
I wonder if an impartial jury would hold me, the teacher, solely responsible for my students’ failure to conquer the AP test and side with one of my colleagues who said that, if my students were not performing well in my course and on the test, then I needed to check my classroom practice. 
But mostly I worry about Veronica’s rage at being used in an AP numbers game. 
Junia Yearwood is a retired Boston Public Schools teacher. 

I looked into AP trends a couple of years ago in VDARE. Among Asians, 64% scored 3 (out of 5) or higher on 2008 AP tests. Among whites, 62%. Among Hispanics (excluding the Spanish AP test), 35%. Among blacks, 26%. (Hmmmhhmm, Asians > whites > Hispanics > blacks ... where have we seen that pattern before?)

But Asians take almost three times as many AP tests as whites. Judging from the regional differences in white AP test-taking, I suspect whites could double the number of tests they take and still stay above 50% passing.

My general impression is that in Red States where there aren't many Asians to get the AP ball rolling, more whites should take more AP tests. And Red States haven't hit complete diminishing returns among Non-Asian Minorities yet, either.

Advanced Placement tests are College Board product. Not surprisingly, they are more popular where the College Board's SAT college entrance test is more popular than the ACT college entrance test, a gap that matches up pretty well with the Blue and Red states. But there's no ACT competitor for the APs, so Red State kids are simply less likely to take AP tests.

On the other hand, in Boston, of all places, pretty much everybody who could pass an AP test has heard all about AP by now, so these kind of programs to shove even more NAMs into AP classes and tests are predictably just a lot of sound and fury signifying nothing.

April 16, 2011

L.A. Schools Boss Uncovers Baffling Mystery in Chinatown

John Deasy, formerly of the Gates Foundation, took command yesterday as Superintendent of the Los Angeles Unified School District. The LA Daily News reports on today's front page:
Speeding from one campus to another on Friday, Deasy peppered teachers and principals with questions about their strategies to improve academic programs, taking breaks between classroom stops to check his iPhone - admittedly his one addiction. 
At Castelar Elementary in Chinatown, Deasy congratulated administrators on their high state test scores and their clean campus. Then he questioned staff on how they were addressing the achievement of Latino students on campus, who are underperforming compared to their Asian peers. "I read the data before coming ... tell me what you're doing about it," he asked. 
Castelar's school coordinator, Sal Sandoval, said he appreciated the tough questions. "Of course it's a little intimidating, I mean he is our boss, but it's great to see how much he knows about our school," Sandoval said.

I'm as stumped as John Deasy is. Whoever heard of a school where Asians outscore Latinos on average? Clearly, it can't have anything to do with, say, the Latino students' parents. As we read in that new bestseller, Battle Hymn of the Jaguar Mother by Harvard Law professor Amy Chavez, Latino parents are notorious for relentlessly pushing their children to study.

At first, I was going to recommend that Superintendent Deasy drop everything else to investigate the Castelar Conundrum. But, on second thought, perhaps some mysteries are not meant for human minds to unravel. This school's unique results could even be the result of a sinister conspiracy that goes all the way to the top, a plot far too deep and dangerous for John Deasy to probe.

Forget it, Jack. It's Chinatown.

April 15, 2011

Do private schools assess teachers by value added testing?

Measuring teachers by how much value they add to their students' tests scores is an idea that I advocated back in the last millennium, and has since become the state of the art conventional wisdom, and, now, it is being implemented into universal law in Colorado, all without anybody actually showing it does a whole lot of good. Dana Goldstein has a funny article in the American Prospect, "The Test Generation," about what that actually means in the classroom:
On exam day in Sabina Trombetta's Colorado Springs first-grade art class, the 6-year-olds were shown a slide of Picasso's "Weeping Woman," a 1937 cubist portrait of the artist's lover, Dora Maar, with tears streaming down her face. It is painted in vibrant -- almost neon -- greens, bluish purples, and yellows. ... 
The test asked the first-graders to look at "Weeping Woman" and "write three colors Picasso used to show feeling or emotion." (Acceptable answers: blue, green, purple, and yellow.) Another question asked, "In each box below, draw three different shapes that Picasso used to show feeling or emotion." (Acceptable drawings: triangles, ovals, and rectangles.) A separate section of the exam asked students to write a full paragraph about a Matisse painting. 
Trombetta, 38, a 10-year teaching veteran and winner of distinguished teaching awards from both her school district, Harrison District 2, and Pikes Peak County, would have rather been handing out glue sticks and finger paints. The kids would have preferred that, too. But the test wasn't really about them. It was about their teacher. 
Trombetta and her students, 87 percent of whom come from poor families, are part of one of the most aggressive education-reform experiments in the country: a soon-to-be state-mandated attempt to evaluate all teachers -- even those in art, music, and physical education -- according to how much they "grow" student achievement. In order to assess Trombetta, the district will require her Chamberlin Elementary School first-graders to sit for seven pencil-and-paper tests in art this school year. To prepare them for those exams, Trombetta lectures her students on art elements such as color, line, and shape -- bullet points on Colorado's new fine-art curriculum standards.

Studies prove that Yale graduates with high-paying jobs tend to have stronger opinions on the merits of Picasso v. Matisse than do high school dropouts in prison, so, logically, that proves that we can't start lecturing children too young on Matisse and Picasso, because if they don't go to Yale, they're bound to go to jail.

Anyway, the question I have about measuring public school teachers through value-added testing is this: How many private schools do that? We have thousands of private schools in America, but I can't recall ever hearing of one that regularly uses these now fashionable value-added assessments to measure teachers. Why not?

March 8, 2011

Essay tests

The more the conventional wisdom denounces standardized testing, the more money there is to be made in standardized testing. For example, the whole world decided about a decade ago that filling in ovals with a Number 2 pencil couldn't possibly be a good test of Critical Thinking skills, so essays were added to many tests, such as the SAT and GRE. 

This increased the cost (and slowed down the grading) considerably. Today, thousands of people, typically temps, are employed each year grading the essay portion of standardized tests. This article by Jessica Lussenhop of the Minneapolis City Pages takes you behind the scenes of essay grading. It's not a particularly edifying scene (although not all that dismal). 

The take home lesson I get is that you should make sure your first sentence is sterling. Graders don't have time to read your essay carefully, so you want to make a good first impression.

March 2, 2011

QQ

In the Washington Post, sportswriter Sally Jenkins * writes:
[Indianopolis Colt's executive Bill] Polian made one of the great all time decisions in 1998 when he drafted Peyton Manning over Ryan Leaf. It's a no brainer now but it was an agonizing choice back then. Leaf was bigger and more overtly athletic, but to Polian, Manning seemed to have more emotional maturity and ability to deal with adversity. It was exactly the right assessment; Leaf turned out to have buried temperament problems and caused the San Diego Chargers nothing but grief. 

It wasn't the first time Polian was correct: he also brought Jim Kelly to the Buffalo Bills from the USFL. If Polian has a secret, it's that he tries not to overrate the physical in favor of the mental and emotional aspects of the game. Just because a guy has an arm doesn't make him a quarterback. Rather than pure strength, he puts a premium on accuracy and poise, and a particular kind of processing intelligence that he calls "fast eyes," the ability to assess complex situations quickly.

"Intelligence is awfully important," Polian says. "It's a complex game and they have to be able to comprehend and process lots of information. It's not rocket science but it's pretty close, it's like financial engineering, or the things that pilots do."

Frank Ryan of the 1960s Cleveland Browns earned a Ph.D. in math from Rice U.. (See here for the title of his dissertation.) Yet, he wasn't known as a smart quarterback on the field. His coach had to simplify plays for him to be able to make the right decisions in real time. (What he was known for to his teammates was being brave.)

It would seem as if the NFL, with all its resources, could develop some kind of video game test of "fast eyes" that would be more relevant than the standard Wonderlic IQ test. 

For example, I played a fair amount of touch football growing up and, having a decent arm, I was an okay sandlot quarterback in 2 on 2 games where I had only two mental tasks: focus on my one receiver and avoid the pass rusher. In 6 on 6 games, however, I was cognitively overwhelmed. I have "slow eyes."

You'll notice that my blogging style isn't that different. Instead of putting up a lot of posts, each one making one single point, I tend to start off with a short post, but it keeps growing as I follow out the implications to who knows where. I don't shift focus adroitly to the latest topic of interest in the news. I'd rather keep burrowing into one topic for a long time.

* By the way, Sally Jenkins is the daughter of Dan Jenkins, one of the half dozen top sportswriters of the second half of the 20th Century. I have this theory that women who succeed in masculine fields are much more likely to have fathers who were in that field.

January 7, 2011

Advanced Advanced Placement

The spread of Advanced Placement tests has been mostly a success story in recent decades (e.g., they provide a fashionable rationalization for tracking), but Advanced Placement classes tend to be forced marches to try to get high school students to more or less memorize a lot of material: lots of worksheets, little time for class discussion.

From the NYT:
by Christopher Drew

As A.P. [testing] has proliferated, spreading to more than 30 subjects with 1.8 million [high school] students taking 3.2 million tests, the program has won praise for giving students an early chance at more challenging work. But many of the courses, particularly in the sciences and history, have also been criticized for overwhelming students with facts to memorize and then rushing through important topics....

All that, says the College Board, is about to change.

Next month, the board, the nonprofit organization that owns the A.P. exams as well as the SAT, will release a wholesale revamping of A.P. biology as well as United States history — with 387,000 test-takers the most popular A.P. subject. A preview of the changes shows that the board will slash the amount of material students need to know for the tests and provide, for the first time, a curriculum framework for what courses should look like. The goal is to clear students’ minds to focus on bigger concepts and stimulate more analytic thinking. ...

The changes, which are to take effect in the 2012-13 school year, are part of a sweeping redesign of the entire A.P. program.

Instead of just providing teachers with a list of points that need to be covered for the exams, the College Board will create these detailed standards for each subject and create new exams to match. ...

The new approach is important because critical thinking skills are considered essential for advanced college courses and jobs in today’s information-based economy. ..

WHEN A.P. testing began in 1956, memorization was not yet a dirty word, and it was O.K. if history classes ran out of time just after they finished World War II.
 
... And it did not take long for instructors to start teaching to the test, treating the board’s outline as the holy grail for helping students achieve the scores of 3 or higher, out of 5, that might earn credit from a college.
That obviously became harder to do as breakthroughs in genetic research and cellular organization, and momentous events like the cold war, the civil rights movement, Watergate and the war on terror, began to elbow their way onto the lists. College professors could pick and choose what to cover in their introductory survey classes. But because the A.P. test can touch on almost anything, high school juniors and seniors must now absorb more material than most college freshmen.

So perhaps it is no surprise that while the number of students taking the A.P. biology test has more than doubled since 1997, the median score has dropped to 2.63, from 3.18.

One critical thinking skill that should be taught in schools is "selection."

I doubt if the increase in the amount of biological knowledge in the world between 1997 and 2010 is the main cause of the decline in average AP score. Instead, the expansion of the number of test-takers would be a much more plausible candidate for primary reason.

On the other hand, I've spent a lot of time looking into this, and it appears to me that diminishing returns have not yet badly hit AP testing. As I wrote in VDARE.com in 2009:
Although the quantity of AP tests taken by whites grew 155 percent over the last decade, their mean score dropped merely from 3.04 to 2.96. The fall-off for Asians was even less, from 3.10 to 3.08. ... The “pass rate” (the percent of test takers scoring 3 or higher) is almost the same for whites (62 percent) and Asians (64 percent). But Asians are much more aggressive about signing up for AP tests, taking almost three times as many per year (1.79 per capita per year versus 0.63 among whites). 

There has been a big push to get more NAMs to take AP tests, with predictable results:
The tremendous growth from 1998 to 2008 in Hispanics taking AP tests drove down their average score on the 1 to 5 scale from 2.99 to 2.42. Their passing rate dipped from 60 percent 42 percent.

(And keep in mind that Hispanic mean scores are exaggerated because so many native Spanish-speakers take the Spanish Language test, which ought to be, but isn’t always, a free throw for them. Indeed, 56 percent of all 5s earned by Hispanics in 2008 came on the Spanish Language exam. Excluding it, Latinos in 2008 averaged a 2.17 score with a 35 percent passing rate.) ...

Black scores fell a comparable amount over the last decade, from a mean of 2.21 to 1.91 (with the passing rate dropping from 35 percent to 26 percent).

Still, despite depressingly diminishing returns, more than quadrupling the number of AP tests taken by blacks from 1998 to 2008 helped the absolute number of tests passed by blacks to triple.

So, even with blacks, the number of passing scores is still rising rapidly, so we don't seem to have topped out yet.

In general, there's a big blue state - red state divide in AP testing that goes back, in part, to the regional divide between states on taking the SAT (which, like the AP, is sponsored by the College Board -- and is most popular on the coasts) and the ACT (run out of Iowa and most popular in the middle of the country). The ACT part of the country could probably try taking more AP tests without much harm.

Back to the NYT article:
For biology, the change means paring down the entire field to four big ideas. The first is a simple statement that evolution “drives the diversity and unity of life.” The others emphasize the systematic nature of all living things: that they use energy and molecular building blocks to grow; respond to information essential to life processes; and interact in complex ways. Under each of these thoughts, a 61-page course framework lays out the most crucial knowledge students need to absorb. 

This seems reasonable at first glance. What do you think?

However, I suspect there's a lurking ethnic agenda that's one motivation for this change: there's a common assumption that a shift away from memorization toward critical thinking will narrow the racial gaps that currently exist between Asians and whites and between blacks/ Hispanics and Asians / whites.

But, is there much evidence for this widespread assumption?

I don't know. Asians do well on the AP tests, but they also do well on the Ravens Progressive Matrices, too.

In general, educationalists tend to assume that because they are for (at least in theory) critical thinking skills and against (at least in theory) disparate impact, that, therefore, upping the critical thinking demands of a test will lower the disparate impact. This often turns out to be untrue.

Another questions is whether memorization shouldn't be a dirty work in high school. Maybe that's a better age for memorizing, while grasping the big picture best waits until college. I don't know...

December 23, 2010

Uniqueness

Jay Matthews delivers breathtaking news in the Washington Post: acceptance rates to get into Thomas Jefferson High School for Science and Technology, perhaps the hardest public science school in the country (average exiting SAT score of 2220 out of 2400):
Looking at applicants of different ethnicities, the crux of Meikle's question, 184 African Americans applied and four (2.3 percent of those applying) were admitted. With Hispanics, the figures were 225 applied and 13 (5.8 percent) admitted. In the multiracial category, 190 applied and 21 (11 percent) were admitted.

The number of whites (1,277) and Asians (1,243) who applied was close, but their admission rates were not. Of whites, 166 (13 percent) were admitted and 276 Asians (22.2 percent) were admitted. 

So,  Asians first, whites second, Hispanics third, blacks fourth. What an astonishing result! Who has ever seen that rank ordering before in any competition involving test scores and grades? It's an anomaly!

Of course, no attempt is made to compare this to the countless similar situations where similar results are seen. We're only supposed to to talk about Thomas Jefferson HS. What is it about TJHS that leads to such remarkable results?  We're simply not supposed to notice any similar pattern elsewhere, and thus preserve our intellectual virginity for thinking about the Virginia school. People who have actually learned things from general patterns are considered unwanted gatecrashers.
 

December 22, 2010

More PISA in Perspective

At Super-Economy, Tino takes another crack at PISA scores, putting up graphs that overcome two of the weaknesses I pointed out in my own PISA graph: I could only find scores for America by race for 2009 for reading (and Americans overall did better on reading in 2009 than on science and math), and I used national average scores for other mostly white countries. So, Tino compares the average of reading, math, and science for white Americans to Europeans who are not first or second generation immigrants to come up with a good apples to apples comparison. Finland is still well in the lead (546), but the U.S. comes in seventh (524) out of 27 European-origin countries, well ahead of, for example, the EU-15 average for wealthy Western European countries (506).

That sounds about right. There's a lot of evidence that the U.S. spends a lot on education and get at least something in return. Not great, but not bad either.

Remember, there is, inevitably, a lot of noise in PISA scores, so fine comparisons aren't too reliable. 

December 21, 2010

A semi-exception to the Fundamental Constant of Sociology

You always come across smug dismissals of The Bell Curve as being discredited, but you never hear explanations from them of why the U.S. military has put such emphasis on cognitive testing for several generations now. In fact, I once interviewed the retired head pscyhometrician of one of the major branches of the military, who had provided the military's AFQT testing data to Charles Murray. He said Murray and Herrnstein had done a bang up job with it.

In general, the military tries to keep a very low profile about their addiction to IQ-like testing, mostly releasing data to projects like Rand Corporation studies that nobody in the press ever reads, but now the Army has cooperated with The Education Trust, a Nice White Lady organization.

From the Associated Press:
Nearly one-fourth of the students who try to join the military fail its entrance exam, painting a grim picture of an education system that produces graduates who can't answer basic math, science and reading questions.

The report by The Education Trust found that 23 percent of recent high school graduates don't get the minimum score needed on the enlistment test to join any branch of the military. 

That's 23% of high school graduates who want to join the Army and the Army wants them because they don't have other black marks against them like obesity or a bad criminal record, who can't get in because they score too low. Add in high school dropouts, and, overall, the Army sets the minimum score for the heavily g-loaded AFQT (the very IQ-like test featured in The Bell Curve) for enlistment at the 31st percentile.
The study, released exclusively to The Associated Press on Tuesday, comes on top of Pentagon data that shows 75 percent of those aged 17 to 24 don't qualify for the military because they are physically unfit, have a criminal record or didn't graduate high school.

Perhaps this is right, but I suspect that to get to 75% unfit to serve before cognitive testing, they are simply summing the percent disqualified for each of those reasons and ignoring the overlaps: e.g., kids who are fat, dumb, and crooked get counted three times, not once. Hopefully, I'm right that the Youth of Today aren't quite that bad, but, maybe I'm just a cockeyed optimist ...
"Too many of our high school students are not graduating ready to begin college or a career — and many are not eligible to serve in our armed forces," U.S. Education Secretary Arne Duncan told the AP. "I am deeply troubled by the national security burden created by America's underperforming education system."
... This is the first time ever that the U.S. Army has released this test data publicly, said Amy Wilkins with The Education Trust, a Washington, D.C.-based children's advocacy group. She said the organization worked with the U.S. Army to get raw data on test takers from the past five years.

... The Education Trust study shows wide disparities in scores among white and minority students. Nearly 40 percent of black students and 30 percent of Hispanics don't pass, compared to 16 percent of whites.

The funny thing is that this particular white-black racial gap isn't quite as large as the normal one standard deviation gap seen in La Griffe du Lion's Fundamental Constant of Sociology. Probably due to self-selection and range restriction, the black-white gap is less than one standard deviation here. But, the authors of the report and the AP don't notice that the Army represents a below-average sized problem because we aren't supposed to know about the Fundamental Constant.
Even those passing muster on the Armed Services Vocational Aptitude Battery, or ASVAB, usually aren't getting scores high enough to snag the best jobs.

The AP article is a little confused about a tricky aspect of the military's admission test. The AFQT is, last I checked, a highly g-loaded four-test subset of the the ten test ASVAB, which includes less g-loaded tests of specific skills, such as vehicle repair. The AFQT is not exactly an IQ test -- it includes questions on trigonometry, for example, which almost nobody learns outside of school. So, yes, if schools did a better job of teaching trig, then more of their graduates would pass the AFQT.

But, results on the AFQT correlate closely with leading IQ tests, so it's close enough for government  work. You must score at the 31st percentile or above on the AFQT (roughly a 92 IQ) to be allowed to join the Army. Once you clear that hurdle, they look at your ASVAB scores, which includes tests of things like auto repair, for help in determining vocational specialties. If you are already a first rate shade tree auto mechanic, then you might be able to skip truck repair school.
"A lot of times, schools have failed to step up and challenge these young people, thinking it didn't really matter — they'll straighten up when they get into the military," said Kati Haycock, president of the Washington-based Education Trust. "The military doesn't think that way."

If there are 310 million people in the country, then about 100 million aren't smart enough to enlist in the Army. Over 140 million aren't smart enough to enlist in the Coast Guard.

Those are gigantic numbers that simply don't register on the pundit class. And when they are reminded of them, of course, the only thing they can say is "fix the schools."

But, tautologically, 30% of youth are going to be in the bottom 30% of youth. 

I knew a kid who was totally focused on enlisting in the Army. The recruiter thought he was great, but then he flunked the AFQT. So, the Army paid to send him to an AFQT boot camp for about six weeks, where the kids live in barracks and where uniforms while they bone up on the AFQT. He loved it. The sergeants picked him as Best Recruit in the program. Then he took the AFQT again. And still failed.

He was a good kid but he just wasn't smart enough to enlist in the Army. In the conventional wisdom, Americans like him don't exist.

In the real world, they do.

The average enlistee in the U.S. military is above the national average in intelligence.
Christina Theokas, the author of the study, said the test was updated in 2004 to reflect the current needs of the Army, and the Army didn't want to release data from before the realignment.

Recruits must score at least in the 31st percentile on the first stage of the three-hour test to get into the Army or the Marines. Air Force, Navy and Coast Guard recruits must have higher scores.

From the Education Trust's report, Shut Out of the Military:
Table 1: Enlistment Eligibility 2010
The minimum AFQT score required to qualify for entry into the
military varies by branch.
Service Branch Minimum Required AFQT Score
Army 31 [i.e., 31st percentile]
Navy 35
Marines 32
Air Force 40
Coast Guard 45
... For the Army, those who score at the AFQT level of 31 and higher—Category IIIB and above—qualify for enlistment. Those scoring at 50 and higher on the AFQT [i.e. 100 IQ], falling into Categories IIIA and above, are eligible for Army
incentive programs including enlistment bonuses, college repayment programs, and the Army College Fund (a monetary incentive that increases the value of G.I. Bill benefits).  ...
Recruits that rank at the highest AFQT levels are eligible for special opportunities. While most military jobs are tied to the kind of composite scores described above, certain elite categories are available only to those who also possess an especially high AFQT. For instance, jobs in technical fields require significantly higher AFQT scores than the minimum score needed for regular enlistment. These high level jobs, because they come with education, training, and skills development, open doors to high-level career paths, provide better active-duty experience and pay, and set up enlisted personnel for greater success following life in the service.

You can see the source at Military.com here.  The Education Trust report continues:
Our sample consists of the nearly 350,000 high school graduates aged 17-20 who applied for entry into the Army between 2004 and 2009 and took the ASVAB at a Military Entrance Processing Station. These young people are among the 25 percent of young Americans who do not have problems preventing them from applying for enlistment in the military. Approximately 50 percent of these applicants, a total of 172,776, joined the Army. The group is not representative of individuals across or within states and the nation, but is a self-selected sample of individuals aged 17-20, with a high school diploma,
and an interest in joining the Army. We chose only to examine the results of recent high school graduates to have a sample of individuals who had experienced similar high school requirements and standards. ... In the sample, 58 percent of the test-takers were white, 19 percent African-American, 12 percent Hispanic, 8 percent unknown, 1 percent each of Asian, American Indian/Alaskan Native, and Native Hawaiian/ Pacific Islander, while 76 percent were male and 24 percent female.

About 23 percent of the test-takers in our sample failed to achieve a 31 —the qualifying score—on the AFQT. Among white test-takers, 16 percent scored below the minimum score required by the Army. For Hispanic candidates, the rate of ineligibility was 29 percent. And for African-American youth, it was 39 percent.

That's the IQ ineligibility rate among non-obese, non-crooked, high school graduates who want to join the Army.

The AP story goes on:
The average score for blacks [in this self-selected sample of high school graduates wanting to join the Army] is 38[th percentile] and for Hispanics is 44, compared to whites' average score of 55. The scores reflect the similar racial gaps on other standardized exams.

Actually, these are pretty narrow for racial gaps. All the other filters reduce the variation. Moreover, there's now a multi-generation tradition of lower middle class blacks enlisting in the Army (as opposed to the other branches).

The Education Trust report goes on to complain that:
To qualify for specific occupational specialties, recruits must earn certain scores in nine different Army aptitude areas. For example, to qualify for any of the Special Forces positions, a recruit must earn a score of 110 on the General Technical composite score, which is a weighted average of Arithmetic and Verbal Expression. Approximately 66 percent of applicants did not meet this minimum score. However, nearly 86 percent of African-American applicants and 79 percent of Hispanic potential recruits did not meet the minimum for these specialties, as compared to 60 percent of white potential recruits.

But, once again, these are narrower racial gaps than are found in the overall population.

The Associated Press article suffers from one obvious mistake:
The study also found disparities across states, with Wyoming having the lowest passage rate, at 13 percent, and Hawaii having the highest, at 38.3 percent.

No, this sentence is a typo in the AP news story that gets the meaning 180 degrees wrong. Figure 2 in the Education Trust report is entitled "AFQT Ineligibility Rates by State." The worst failure rate is in Hawaii (followed by MS, DC, LA, SC, NM) and the least bad failure rate is in Wyoming (followed by IN, ID, NE, NH, MN).

More interesting numbers from the report's state tables:

Among white youths with high school diplomas applying to join the Army, the lowest failure rates were in Indiana (10.1%), Alaska, Wyoming, and Nebraska. Among whites, the highest failure rate was, by far, in Maryland (27%). Next was DC, then Kentucky. The high failure rates for whites in liberal MD/DC is probably due to the military being seen by MD/DC as a good place to dump the dud in the family.

Best performances by blacks were in Oregon, Arizona, Alaska, (all small sample sizes), Indiana and New York. Worst performances by blacks were in Mississippi, Arkansas, Louisiana, and Wisconsin.

Best performances by Hispanics were in small sample size states like Montana, Alaska and Indiana. Worst performances by Hispanics were in Massachusetts and Connecticut. Texas's Hispanics did slightly better than California's.

PISA school test scores by ethnicity

My big graph showing how the four main American races would do compared to 64 other countries in this year's PISA reading test scores is up at VDARE.com.

December 16, 2010

Does everybody secretly hate graphs?

I'm reading the book Fault Lines by former IMF chief economist Raghuram G. Rajan of the U of Chicago economics dept. It's a pretty good read, but what struck me while flipping through it is that it's solid text: no quantitative graphics, no tables of numbers, just paragraphs. The sole thing to interrupt the flow of paragraphs across a couple of hundred pages is a poem by 18th century economist Bernard de Mandeville.  

That lack of tables and graphs can't be natural for an economics professor, can it? The publisher must have told him what statistician Andrew Gelman's publisher told him when he wrote Red State, Blue State, that each graph in the book cuts sales in half.

So, does everybody really hate graphs? If so, why does everybody who gives a presentation think they have to do it in Powerpoint? (As in Abe Lincoln's Gettysburg Address graph comparing New Nations -87 Years to Now.) Do audience members actually look at the graphs? Or do they just appreciate the chance during the work day to veg out in a dark room with a glowing screen? "Powerpoint: It's Almost Like Watching TV While Getting Paid!" 

Or did people used to like graphs until Powerpoint came along? 

Do the books that the people who sit next to you on the airplane read have graphs in them? A large fraction of people in airplanes are traveling to meetings infested by Powerpoint graphs: if there is a graph in their book, do they consider it work? 

I like graphs. I had Minard's now-famous graph-map of Napoleon's 1812 invasion of Russia up on my office wall for years. It worked as a sort of secret club handshake for people walking by. One out every zillion people who walked down the hall past my office would recognize it and introduce himself.

I find that graphs always take me about five times longer to finish creating than I expected. For example, in VDARE, I've got a graph that answers the obvious question (obvious to you and me, at least) about those PISA international school achievement test scores that everybody was pontificating upon last week. My graph shows where the PISA test scores of the four main racial/ethnic groups in the U.S. would fall compared to the national average scores for the 65 countries that took the test. 

A simple idea, right? Yet it took me forever to get the graph right so the answer is clear. (First, I had to find the American race numbers, which don't appear in any of the hundreds of pages of data posted by the OECD last week.) Then, I had to fiddle for hours to get the graph to look less confusing. 

I think it finally came out okay. But, if everybody out there secretly hates graphs, then I'll stop wasting time making them.