we’re more dumberer than the victorians (part 4)

michael woodley and co. have discovered a bit of modern research on reaction times very (but not exactly) like galton’s late nineteenth century research in london which they feel supports their claims that western iqs have gone down by 14.1 points since the victorian era.

maybe.

it’s a VERY interesting study, but i’m still not 100% convinced. if the study and its findings are replicated, i might be! (^_^)

i left a comment over there @dr. james thompson’s blog, but, as of the writing of this post, it hadn’t been approved, so i reproduce below.

look forward to more research into this question! neat (and important!) stuff!

“cool study! thanks for drawing it to everyone’s attention, and … yes, i agree … the design of this study seems to be much closer to galton’s original study, so yeah … lots of little red flags should be raised! (^_^)

“personally, i’m still not 100% convinced, though, because i dislike the fact that we have to use *indirect* means to check that the samples are similar — for example, having to rely on eckstein and feist to tell us who, *in general* british museum goers are. i wish wilkinson and allison would’ve just told us *directly* who their participants were! and as for the ethnicity question — that the subjects were white is not enough. they need to be white *brits*. if, for instance, they had too many southern italians or southern spaniards or even irish folks in their study, etc., etc., that might again have dragged down (or up, rather) the scores … *somewhat*.

“(wrt to question of paying for admission, we’ve got indirect evidence for that here as well. a quick email to wilkinson or allison should be able to answer that question. keep in mind, too, that presumably only *some* of galton’s subjects paid a museum entrance fee since on three or four days a week [don’t recall which it was] admission was free to the museum.)

“nevertheless, these are some very interesting results. what would be great, of course, would be to see this study *repeated* and the results *replicated*. if i were to design such a study, i’d make sure that it was in every way possible identical to galton’s, that way there’d be no questions about the sampling, etc. it’d also be cool to test the same individuals’ reaction times on *both* an old, galton-type machine AND a modern one to see exactly how the machines compare.

“in lieu of such a study (or better yet, in conjunction with it), i really think you guys should delve into that 1984-85 health & lifestyle study from the u.k., because they recorded ethnicity and, presumably, ses, so you could get at a sample like galton’s that way. of course, then you’d be left with a modern machine in that instance. i leave the reliability of the machines discussions up to other folks. (and the correlation of rts to iq discussion, too.)

“thanks again, guys!”

previously: we’re dumber than the victorians and btw, about those victorians… and a response to a response to two critical commentaries on woodley, te nijenhuis & murphy (2013)

(note: comments do not require an email. idiocracy – von dem macher von “beavis und butt-head”!)

a response to a response to two critical commentaries on woodley, te nijenhuis & murphy (2013)

first of all, thanks to michael woodley, jan te nijenhuis, and raegan murphy for their response to my (here and here) and scott alexander’s comments on their recent iq paper (Were the Victorians cleverer than us?). thanks, too, to dr. james thompson for hosting their response!

just to refresh everybody’s memory, woodley et al., concluded from their recent research that 1) iqs have decreased in the western world 14.1 points since the victorian period (1889), and 2) that this decrease is due to dysgenics. scott alexander and i both questioned the sampling techniques used in various of the studies looked at by woodley and co. — both in the victorian period and in the modern (see our previous posts linked to above for details).

what is at issue here is whether or not woodley et al. have looked at the same sort of population at both ends of their research. if they’re trying to figure out whether or not iqs have decreased over time in the western world, then the subjects sampled in the past and in the present need to be representative of the two populations, and the two populations need to be of the same sort. unless i’m very much mistaken, sampling issues are considered to be of extreme importance in scientific endeavors (i.e. to get your sampling right). it also simply makes logical sense.

so, we need to know a few things: 1) were the samples taken in the victorian period representative of victorians, 2) were the samples taken in modern times representative of modern populations, and 3) are all of these samples from like populations?
_____

1) were the samples taken in the victorian period representative of victorians?

no.

the victorian samples came from two sources: galton and thompson. (see also here about some other victorian rts data sets.)

thompson’s samples came from university of chicago students in 1898-1900, so they’re unlikely to be representative of victorian americans on the whole. these were probably mostly pretty smart individuals primarily drawn from the elite classes. (see previous post.)

galton’s sample consisted of museum visitors who had paid to take galton’s test (and, in some amount of cases, had probably paid to enter the museum as well). scott alexander superbly analyzed this sample further by combing through the info in johnson et al.’s “Galton’s data a century later.” scott found that galton’s sample was not representative of victorian brits — in fact, it leans heavily towards the middle- and upper-classes. this is not surprising when you think about who, in those days, would’ve been able to afford the costs of taking the test and possibly the museum fee as well.

here’s what scott had to say:

“Tables 10 and 11 turn out to be a gold mine – I worried the records of exactly who took the tests would be lost, but as you might expect of someone who basically invented statistics single-handedly and then beat Darwin in a debate about evolution as an encore, Galton was *very good* at keeping careful data.

This site tells me that about 3% of Victorians were “professionals” of one sort or another. But about 16% of Galton’s non-student visitors identified as that group. These students themselves (Galton calls them ‘students and scholars’, I don’t know what the distinction is) made up 44% of the sample – because the data was limited to those 16+, I believe these were mostly college students – aka once again the top few percent of society. Unskilled laborers, who made up 75% of Victorian society, made up less than four percent of Galton’s sample!”

a very kind person very kindly sent met the johnson et al. paper (thank you very kind person!), so i’ve cut-and-pasted tables 10 and 11 here for you to see for yourselves [click on tables for LARGER views]:

galton's measures - males

galton's measures - females

so galton’s sample is not representative of the victorian british population — it was unbalanced in that it did not include enough subjects from the lower classes.

furthermore, the subjects in galton’s study self-selected themselves. these were individuals who, first of all, chose to go to the south kensington museum (now the victoria and albert museum) to start with (except maybe for the “students and scholars”), AND then they further chose to take galton’s funny little test. this is NOT a good sampling technique. if gallup or pew were to use such a technique, they’d be laughed out of the polling business.

modern studies, of course, try to make sure that data from a representative sample of a population are gathered, otherwise your data might be skewed. which is exactly what scott found with regard to galton’s data. i mean, imagine what sort of person from the lower classes in victorian england first bothered to go to the south kensington museum AND then was interested in taking galton’s test. knowing what i know about the working classes (and i come right out of that class — more like the peasant class, in fact), those that want to go to a boring museum are the exceptions to the rule.
_____

in woodley et al.’s response to our questions about these sampling issues, they point to a study by silverman — Simple reaction time: It is not what it used to be — which i haven’t seen, because i don’t have access to it (and i’m too cheap to pay for it (~_^) ). they say that silverman compensated for these victorian sampling errors by looking at reaction time (rt) data from similar socio-economic sub-groups from more modern eras. sounds like a good idea:

“One advantage of Silverman’s care and meticulous attention to detail is that it permits us to make like for like comparisons with specific socioeconomic and occupational groups in Galton’s data, thus we can directly test the claims of Alexander (2013). Concerning the post-Galton studies Silverman included five student samples, two of which date from the 1940s (Seashore et al. 1941), and the remaining three of which date from the 1970s to the 2000s (mean testing year = 1993; Brice & Smith, 2002; Lefcourt & Siegel, 1970; Reed et al., 2004)…. The difference between the 19th century and the ‘modern’ male students is very similar to the meta-regression-weighted increase in RT latency between 1889 and 2004, estimated on the basis of all samples included in the meta-analysis (81.41 ms). Silverman also included data from other socioeconomic groups. For example the study of Anger et al. (1993) included a combined male + female sample of 220 postal, hospital and insurance workers from three different US cities. These occupations clearly fall into the Clerical/Semiskilled and Semiprofessional groups identified in Galton’s study.”

i’m not going to look through all of these studies to check for sampling irregularities because, again, it’s not my research, so it’s not my job. but let’s take a quick look at the first one i highlighted above: seashore et al. 1941. i don’t have access to this paper either, and, no, i’m not gonna pay for it, so i’ll have to assume going by another of r.h. seashore’s papers that these rt data were drawn from american students (possibly northwestern university where seashore worked — correct me if i’m wrong), and going by the publication date (1941) that the samples were collected in the 1930s or 1940-41.

woodley et al. say that, in comparing the nineteenth century vs. 1940s-era students, there is a +16.8ms (183.2-200 ms respectively) difference between the two groups, and thus a decrease in average iq from the victorian period to the 1940s.

my question is: what makes silverman, or woodley et al., confident that the samples from an american university in the 1930s-40s are comparable to thompson’s university of chicago samples or galton’s samples from the 1800s? as shown above, both of those victorian studies were done on elite victorian groups, while, on the other hand, it is well know that already by the 1940s at least, the “college bubble” had started in the u.s. in which nearly anybody is admitted to university. when the average college grad shifts over time from having been drawn from the upper classes to the middle and even the lower, it’s not surprising that their average iqs drop!

even within galton’s self-selected group of male museum goers (see table 10 above), rts (iq) varied between the classes, with the upper classes having lower rts (indicative of higher iqs): ages 14-26: gentleman=.170, professional=.173, semiprofessional=.182, merchant/tradesperson=.190, clerical/semiskilled=.187, unskilled=.195. why, then, shouldn’t rts/iqs vary between college students of the late-1800s vs. the 1940s when more middle- and lower-class individuals attended college in the 1940s than in the late-1800s?

as for the second study i’ve highlighted above — anger et al. (1993) — which included “a combined male + female sample of 220 postal, hospital and insurance workers from three different US cities….” postal workers? seriously? comparing 1990s american postal workers to galton’s middle- and upper-class museum goers? really?
_____

2) were the samples taken in modern times representative of modern populations?

yes and no.

some of the studies used by woodley et al. are probably better (i.e. more representative of modern populations) than others. as i pointed out in my first post on this study, i would imagine, although i haven’t actually checked it, that the sampling in the 1984–85 U.K. Health and Lifestyle Survey is probably fine, and so the rts representative of the whole of britiain. on the other hand, the 2002 study from the university of bristol may have included at least 10% minority individuals if not more in the study. since only ca. 8% of the u.k.’s population in 2001 was non-white, this bristol study may not have been representative of the nation’s population. hard to know without knowing the demographics of who was included in the study.
_____

3) are all of these samples from like populations?

no.

the victorian samples were drawn from mainly the middle- and upper-classes, not to mention a large group of self-selected individuals who were specifically interested in taking part in such a study, while the more modern studies have included greater numbers of middle- and lower-class individuals as well as a certain percentage of non-whites.

about that latter point, woodley et al. responded:

“Substantial changes in terms of the ethnic composition of test-takers would however be needed in order for the magnitude of change to be *solely* or even *substantially* a consequence of this process.”

i never said so. i only said that it’s not possible to compare apples to apples+oranges when looking for changes in just apples over time. apples vs. apples+oranges are likely to produce different results since two such groups are different. i never said, or even suggested, anything about the extent of those differences. (i wish people would read more closely. *sigh*)
_____

so, again, i think that there are quite a few sampling issues in this study, and that the presence of these means that the researchers’ findings are not as reliable as they think that they are.

could there have been a decline in western iqs since the victorian period due to dysgenic factors? of course.

does the fact that there are sampling issues in the woodley et al. (and silverman, for that matter) paper prove that there hasn’t been a drop in iq since the victorian period? no. obviously not.

most importantly, though, the population geneticists say that such a severe drop in genotypic iqs could not have happened in such a relatively short space of time without some really severe selection pressures (see here, too). i believe ’em. perhaps if all of the sampling issues were cleared away from this study, woodley et al. would be left with a decline that was more realistic/believable. if such a decline happened at all, that is. (which it could have! or maybe it didn’t…. (~_^) )

again, like i said before, this is a really neat study from woodley et al.! i hope they continue investigating along these lines, because it would obviously be important to know if average iqs are declining in the western world.
_____

p.s. – before bruce charlton shows up and gets all inquisitional on me (repent, or be damned to hellfire for eternity!), yes, i ALWAYS get all picky about sampling (see here and here and here and here for just a few examples). if you don’t get the fundamentals right, everything else will inevitably be wrong!
_____

previously: we’re dumber than the victorians and btw, about those victorians…

(note: comments do not require an email. i never expected the spanish inquisition!)

btw, about those victorians…

in my post about the woodley, et al., iq paper the other day, i mentioned that one of the two studies on victorian iqs included in their research had been conducted on university of chicago students. i got to thinking afterwards that, apart from the fact that most of those students had probably been white (probably wasps, in fact), this could hardly have been a representative sample of victorian americans since the vast majority of people didn’t attend college in those days — late nineteenth century college students would’ve mostly been, you know, actual smart people.

so, then i was planning on (reluctantly) splashing out $11.95 on this article to find out the details on what sorts of people were included in galton’s study of victorian english iqs, the second set of victorian iq results used by woodley, et al. (reluctantly because, h*ck! — $11.95 is three or four tall decaf mochas!).

but scott alexander over @slate star codex is way ahead of me! scott writes:

“Galton’s Data A Century Later, published in 1985, tells us a little about how he gained his ground-breaking reaction time statistics. He set up a laboratory in the Science Galleries of the South Kensington Museum. There he charged visitors to the museum three pence ($25 in modern currency after adjusting for inflation) to be measured by his instruments, a process he advertised as ‘for the use of those who desire to be accurately measured in many ways, either to obtain timely warning of remediable faults in development, or to learn their powers.'”

ehhhh. charged a fee? uh-oh.

scott acknowledges that his $25 estimate mightn’t have been the most accurate, so i decided to use the calculator over at measuring worth to see how much three pence from 1889 would be worth today (i used their average earnings index), and i got £5.25 or $7.94 (for 2010). that’s not quite $25, but still that’s two or three tall mochas! i’m not sure that very many lower class victorians would’ve been willing or able to part with that amount of money just to take galton’s funny little test.

additionally, there may also have been an admission fee for the south kensington museum (now the victoria and albert museum) that we might have to add to that $7.94, but i’m not sure because i couldn’t find out anything about museum fees in victorian era london. (i know, for instance, that museums in places like new york generally had fees in the late 1800s, but also often had one free day a week to enable the lower classes to visit and learn. bunch o’ idealists, the victorians! (~_^) )

so again, i think we’ve got an apples and oranges problem here.

go read the whole of scott’s post, btw, because it’s excellent!

previously: we’re dumber than the victorians

(note: comments do not require an email. the south kensington museum.)

we’re dumber than the victorians

or so woodley, te nijenhuisc, and murphy have concluded:

Were the Victorians cleverer than us? The decline in general intelligence estimated from a meta-analysis of the slowing of simple reaction time

– Simple reaction time has slowed since 1889.
– Simple reaction time genetically correlates with g.
– Psychometric meta-analysis reveals a decline in g of − 1.23 points per decade.
– The decline between 1889 and 2004 is − 14.1 points.
– This is the first direct measurement of a probable dysgenic trend in IQ.

The Victorian era was marked by an explosion of innovation and genius, per capita rates of which appear to have declined subsequently. The presence of dysgenic fertility for IQ amongst Western nations, starting in the 19th century, suggests that these trends might be related to declining IQ. This is because high-IQ people are more productive and more creative. We tested the hypothesis that the Victorians were cleverer than modern populations, using high-quality instruments, namely measures of simple visual reaction time in a meta-analytic study. Simple reaction time measures correlate substantially with measures of general intelligence (g) and are considered elementary measures of cognition. In this study we used the data on the secular slowing of simple reaction time described in a meta-analysis of 14 age-matched studies from Western countries conducted between 1884 and 2004 to estimate the decline in g that may have resulted from the presence of dysgenic fertility. Using psychometric meta-analysis we computed the true correlation between simple reaction time and g, yielding a decline of − 1.23 IQ points per decade or fourteen IQ points since Victorian times. These findings strongly indicate that with respect to g the Victorians were substantially cleverer than modern Western populations.

interesting!

my first question, of course, would be: are they comparing like with like? the authors write:

“We take our general inclusion rules from the meta-analysis by Silverman (2010)…. Third, given that Galton’s sample was British the studies had to have been conducted in a Western country.”

ehhhh. but the demographics of western nations — especially the u.s. and the u.k. — have changed a lot from victorian days!

i especially started asking myself if they’ve compared like with like when i noticed in their table that the iqs of the finnish in the 1980s-90s — demographically still very much a white northern european in those decades — were pretty much the same as the victorians’ scores:

woodley et al - victorian iq

the two reaction time studies on the victorians were done by galton in the u.k. in 1884-1893 and someone named h.b. thompson in the u.s. in 1898-1900. galton’s study presumably included mostly white britons and perhaps some amount of ashkenazi jews. thompson’s study, which was conducted on university of chicago students, was almost certainly comprised of mostly white americans.

what about all the later studies? well, i don’t have access to most of them, but here’s what i found out about a couple of them (again see the table above):

– the 2002 study from the u.k. (rt=324): Effects of caffeine on mood and performance: a study of realistic consumption [pdf] – the subjects were 24 university of bristol students. now, in 2002 at least 10.5% of the university of bristol student population were non-whites, possibly more since the ethnicities of 17.8% of the student population were unknown. this is not really comparing like with like when at least 1 in 10 of the subjects was not white, unlike in galton’s or thompson’s studies. and, do we even know what the ethnic/racial backgrounds of the subjects in this study were? nope.

– the 1984–85 from the u.k. (rt=300): Age and Sex Differences in Reaction Time in Adulthood: Results From the United Kingdom Health and Lifestyle Survey [pdf] – the subjects in this study were drawn from the 1984-85 health and lifestyles survey. i don’t know for sure, but presumably this was meant to be a representative survey — representative of the population of the u.k. in the early 1980s, minorities were 4.2% of the u.k. population [pdf], so 4 out of every 100. and what about the presence of, say, southern europeans in the u.k. at the time? i have no idea, but clearly these things should be taken into account.

i dunno. this is a really neat study, and maybe woodley, et al., are on to something, but i’d like some reassurance that they’re looking at the same sorts of populations.
_____

see also:

Simple reaction time: it is not what it used to be.

Objective and direct evidence of ‘dysgenic’ decline in genetic ‘g’ (IQ) and Taking on-board that the Victorians were more intelligent than us and Intelligence declined one SD since Victorian times – why NOT? from bruce charlton!

The Victorians were cleverer than us! and ORIGINAL PAPER: “A response to Prof Rabbitt – The Victorians were still cleverer than us” by Woodley, te Nijenhuis and Murphy and Can I have a reaction? @dr. james thompson’s blog!

The Victorians were smarter than us, study suggests
Were the Victorians cleverer than us?
Victorian Era Brits Were Smarter Than Us
____

edit: see also btw, about those victorians…

(note: comments do not require an email. sir francis galton.)