Sunday 1 February 2015

Amazing New Tool

Thanks to Justin Wolfers' recent column in the New York Times on the dominance of economists, I see this amazing new data tool which gives you the number of times, or the fraction of articles in which, any word has been used in the Times since the mid 19th century. Check it out.
Above is some of its output (click to enlarge). I thought it interesting that the number of times the word nigger has occurred in the Grey Lady's articles has remained fairly constant since the time before the civil war, even though it was frequently used in fiction less than a hundred years ago (William Faulkner's Absalom, Absalom! comes to mind, as does, of course, Joseph Conrad's The Nigger on the Narcissus). Those of us who believe the word's negative power today would be greatly reduced if decent folks were socially allowed to use the word at will should be pleased to see no negative trend in mentions, though this word has always been very rare.

With regard to the race issue, here are some trends in words used to refer to the Black Man. It is interesting to note that negro fell out of favour around 1870, then came back, possibly with the Civil Rights movement, only to decline again. Around this time, black really picks up. African-American has never been too common:
Here's a spelling reform which took hold quickly: writing per cent as one word (although I and many non-Americans do not do that). This pretty much must be the result of official NYT policy as it is inconceivable that nearly everyone should adapt to a new way of spelling so fast without central direction.
Lastly, I thought it might be interesting to see if the growth of government is reflected in the mentions of POTUSes in the New York Times. There indeed does seem to be an upwards trend as judged by the fraction of articles in which the word President appears, though this trend ended around 1980. It is noteworthy that individual names have become more frequently used over time. Ulysses S. Grant is one of very few exceptions of pre-1960 presidents who could compete with post-1960 ones in terms of fraction of articles mentioning the name of the office-holder. But what happened to Obama in 2013? His name was only mentioned in 2.99 per cent of NYT articles.
 
Anyway, this is a fun tool that I suspect will amuse many, many people and lead to numerous interesting speculations.