For your listening pleasure

Check out Gramercy Riffs. They have been described as a "heartbreak/nostalgia pop" band; see other commentary here.

Something fishy about "no evidence"

Searching Google for "no evidence" yields "about 21,300,000" hits. It seems we're keen to deny that there is any empirical support for countless different claims. For example, scientificblogging.com reports that there is "No Evidence For Fish Oil Benefit In Arrhythmias" based on a systematic review just published in BMJ. (Full disclosure: I have previously participated in research on omega-3 fatty acids, however I have no related financial interests.) What does the review itself say?

This is the first systematic review attempting to evaluate whether the protective mechanism of fish oil supplementation is related to a reduction of arrhythmic episodes determined either by a reduction in implantable cardiac defibrillator interventions or a reduction in sudden cardiac death. We found a neutral effect on these two outcomes. The confidence intervals for these outcomes were wide and a beneficial effect up to a 45-48% relative risk reduction cannot be excluded.

To better appreciate this, here's their Figure 2:
Note that of the three studies that looked at the proportion of implanted defibrillators that were triggered, one showed a statistically significant effect in favour of fish oil, and the other two did not show statistically significant effects (one favoured placebo and the other favoured fish oil). Six studies looked at sudden cardiac death in patients taking fish oil compared to those taking placebo. Only one was statistically significant, and it favoured fish oil. Of the five studies that did not show statistically significant effects, two favoured fish oil, and three favoured placebo.

The diamond shapes in the figure show the pooled estimates with their 95% confidence intervals: in each case the diamond overlaps an odds ratio of 1, indicating that the overall effect is not statistically significant. And when there's a non-statistically significant effect, it is common practice to say there is "no evidence". But that can be very misleading! After all, two of the individual studies did show a significant benefit of fish oil. So what's going on? Well, for starters, there's some indication of heterogeneity between the studies (particularly in the case of the defibrillator studies). But it also seems that more large studies are needed: a good deal of the variation in results between the studies may simply be due to the play of chance. Quite substantial benefits of fish oil are entirely plausible: relative risk reductions of as much as 45-48%!

What is "no evidence"?

Consider the figure below:

At the bottom there is a gray axis line with tick marks and a vertical gray line indicating the "null" value (where there is no preference one way or the other). The blue line with arrows at each end represents an infinitely wide confidence interval. This is the most straightforward representation of "no evidence": there is simply no empirical information to indicate what the true effect might be.

But suppose we have a very small sample, that is, one that provides almost no empirical evidence. The figure might become:

The only difference is the blue dot on the confidence interval just a bit to the right of the null line. It represents the point estimate based on a very small amount of empirical information. Of course it could equally well have been on the left hand side (or perhaps directly on the null line). Regardless, the confidence interval is still very wide, so very little can be said about the true effect. With such a wide confidence interval, the location of the point estimate is almost irrelevant.

Finally, suppose that a reasonably large sample is available:

I have left the point estimate at the same place. The confidence interval no longer has arrows on either end and is relatively narrow. However it still overlaps the null line. That means the estimate is not statistically significant. Sometimes this sort of situation is described as showing "no evidence of an effect". But as I noted above, that's quite misleading language. In fact, what this situation shows is indeed evidence—evidence that any effect likely has a magnitude of no more than two tick marks (whatever they represent) on the right hand side of the null line or a magnitude of no more than about a half a tick mark on the left hand side of the null line.

But here's the tricky part: what do those ticks represent? Suppose the axis represents annual cost savings that might result from implementing a certain type of federal government program. If each tick mark represents $1000, then we have estimated that the program will cost at most $500 a year and save at most $2000 a year. In other words, the program has been shown to be effectively revenue neutral: the evidence suggests that the cost/cost-savings of the program will not be important. On the other hand, if each tick mark represents one million dollars, most of us would feel that the jury's just not in yet. A possible cost of $500 a year is a drop in the bucket, but $500,000 a year is something else entirely.

In a way, money is the easiest measure to evaluate like this. Things like safety are much harder. For example, if the evidence suggests that a certain chemical may increase the rate of certain types of cancer, but the findings are not statistically significant, what can we conclude? Can the manufacturer claim that there's "no evidence" the chemical is harmful? Can health activists claim that there's "no evidence" the chemical is safe?

I would argue that the term "no evidence" is inappropriate in either case. The underlying questions remain: what is required in order to conclude that a chemical is harmful or that it is safe? Ultimately there's no getting around the issue of how large a difference (in, for example, cancer rates) has to be in order to be considered important. And that's a rather uncomfortable question.

Fuzzy on the whole "good-bad" thing

Michael Ignatieff, the new leader of Canada's Liberal Party has been described as a political chameleon. Writes Michael Valpy:

But who is Michael Ignatieff really? Is he the pinkish liberal who champions human rights and carries Pierre Trudeau's torch for social justice? Or is he the conservative realist who embraced George W. Bush's attack on Iraq and flirted with the notion that torture could be acceptable?

I wonder if Ignatieff's 2004 book, The Lesser Evil, should instead have been titled Maybe Torture's Like, Ok?

But one thing's for sure—as a reviewer on Amazon.com put it—Ignatieff is "fuzzy on the whole 'good-bad' thing."

For a commentary on torture minus the fuzziness, see this 2003 editorial from the British Medical Journal. The subtitle puts things in focus:

Torture is a form of terrorism:
there are no justifications for it

Michaëlle Jean for President

So our Governor General, Michaëlle Jean acceded to Prime Minister Stephen Harper's request that Canada's parliament be suspended until late January. And the justification ... ?

Well, none was provided. But perhaps it had something to do with Harper's Conservative Party being on the brink of losing control of the government.

Now I don't dispute that the Governor General was acting within her powers. But I do think her powers need some tweaking if parliament can be suspended just to allow the Prime Minister to evade the will of the elected members of parliament. That kind of contempt for democracy smacks of absolutism. Who does the Governor General report to, anyway? Oh, right: the Queen of England!

I'm reminded of some lines in the jingoistic second verse of "God Save the Queen":

O Lord, our God, arise,
Scatter her enemies,
And make them fall.
Confound their politics,
Frustrate their knavish tricks,
On Thee our hopes we fix,
God save us all.

God save us all, indeed. It seems that Canada's politics are the ones being confounded. (I confess, however, that I do admire the rhyming of politics with knavish tricks.)

I wonder if it isn't time that Canada grew up and that the sun set once and for all on the western remnant of the British Empire. It's time we became a republic. Of course a republic needs a President. And to show I hold no grudges, I would like to nominate Michaëlle Jean. But enough already with the proroguing!

Peroguing parliament

I note with alarm that Canada's parliament may soon be perogued!

Update 07Dec2008: Overheard at a party: "We're in prorogatory!"