This is a story about a particularly awful bit of science reporting. It’s taken me a while to sort out what’s going on. First I thought it was bad maths, then I thought it was bad science, and now I think it might be bad journalism. But it might be none, all, some, or a superset of those. I didn’t really want to write this post because crowing about other people’s apparent failures isn’t great, but we took a team decision to cover it because of entertainment reasons. ((Unlike the BBC, we are not funded in a unique way, so we have no public service remit. I think that’s right.)) I’m going to make an attempt to tell you what I’ve found, a little bit of what I think, and then we can rejoin Team Positive Mental Attitude when it’s all over.

Geoffrey B. West is a physicist. Geoffrey West, like any self-respecting physicist, thought he would take a look outside his specialisation and find out what governs how living things work. How about size? Perhaps some animal traits follow a nice simple power law function of size. Big things like redwoods and elephants seem to live for a long time and move slowly, while little things like hamsters and algae flash in and out of existence like living’s for chumps. Previously, Geoffrey West had thought about cities, and concluded that, funny coincidence, they too are influenced by power laws ((Also in his oeuvre is a paper whose abstract claims that “Fractal-like networks effectively endow life with an additional fourth spatial dimension.”)).

So he did what a scientist has to do and came up with a theory and tested it against some data and wrote a paper and then a book and soon it was time to go to the media and chat about ((Eagle-eyed readers will note that edge.org thinks ‘ten¹⁴‘ is an acceptable way to write numbers. We do not share this opinion.)) how remarkable it is that this really simple mathematical model fits so many things, apart from all the exceptions.

The problem with power laws is, drawing a log-log plot is a terrible way to spot one, and spotting one is otherwise quite tricky. As Cosma Shalizi explained in his post, “So you think you have a power law – well isn’t that special?“, there’s a lot to bear in mind when looking for power laws, and we’re pretty terrible as a species at thinking about exponential curves. So Geoffrey West’s ideas are not particularly widely accepted, and many criticisms focus on the lack of appropriate statistical tests of their power.

An NPR blogger wrote a blog post titled “Nature Has A Formula That Tells Us When It’s Time To Die“, joining Geoffrey West’s ideas with Shanghai artist Yunfan Tan’s work and a paper by Marba, Duarte and Agusti from 2007 in a very ooo deep science kinda TED-y/New Scientisty way. This blogger, Robert Krulwich, had previously written another post on basically the same subject which took great pleasure in anthropomorphising things which shouldn’t really be anthropomorphised. The second article repeated lots of claims about power laws but with even less detail than before, while anthropomorphising ‘Nature’ more strongly.

Scientists immediately jammed up the comment section switchboard pointing out all the exceptions to the rule and the lack of any kind of statistic describing the model’s explanatory power. Also, how this log-log graph really doesn’t make a convincing case for anything, especially not the claimed line of best fit:

The relationship between plant mortality rate (D) and the individual mass (M) of plants, from Allometric scaling of plant life history by Marba, Duarte and Agusti (2007)

The NPR journalist did the appropriate thing and deferred blame to Dr West, saying his theories are “controversial”, before pointing out that the paper in Science presenting the idea has been cited “more than 1500 times” ((Google makes it just over 2000)), so it “has weight”. As I write, comments seem to have been closed and removed from the NPR piece.

Some good discussion is happening elsewhere, in a MetaFilter thread about the article, where several commenters have raised good points both about criticisms of the power-law model, and about why this kind of thing is worth thinking about and might have some value somewhere in it.

Now let’s do something else and forget this ever happened!

Via Slashdot.

PS: shout-out to Ronan Mehigan, who informed me that the way to get back from the page containing the graph above to its parent paper was to google the paper’s title. Journal sites: a fractal of bad design.