VBM – voxel-based morphometry or very bad measurement?

Brain imaging technology has become a big player in the world of pain research in the last 2 decades. In more recent years voxel based morphometry (VBM), which allows us to look at structural changes in the brain (specifically neuronal matter density) has produced some eyebrow raising research in chronic pain.

It is this technology that produced results that we have discussed here at BiM before, that chronic pain states are associated with startling alterations in brain structure. Much has been made of these results. Hypotheses have been proposed, speculative models developed and replication studies frantically pursued. But a new paper has just been published that suggests that all might not be well in the world of VBM research. Little statistical gremlins may be producing dodgy results. This excellent blog from the Neuroskeptic provides the full story and raises some difficult questions for the field.

…The authors took 200 normal brains and compared each one of them in turn to a control group of 16 normal brains. Because all of them were healthy, the comparisons ought to show no significant differences.

The technique was set up so that, in theory, only 5% of the brains should have been wrongly labelled as containing an abnormality. But in fact, a full 93.5%of the normal brains gave at least one false positive. So 5% is more like the rate of not being wrong…

…So what’s going wrong? It’s not entirely clear and several factors are probably at play, but the authors say that the main issue is that VBM makes the assumption of statistical normality which doesn’t in fact hold….

Read more of this post from Neuroskeptic here.

Reference

Scarpazza, C., Sartori, G., De Simone, M., & Mechelli, A. (2013). When the single matters more than the group: Very high false positive rates in single case Voxel Based Morphometry NeuroImage, 70, 175-188 DOI: 10.1016/j.neuroimage.2012.12.045

Comments

  1. The alert intrigued me. I’ve not employed VBM as a researcher but am well aware of the technique that has been used to suggest grey matter differences in the brains of patients with chronic pain. To my mind, VBM does not produce a standard unit of measure, but allows relative comparison of between two groups, usually with similar numbers of patients and controls in each group.
    But that’s no good to me as a clinician. What I want to know if whether the patient facing me has abnormalities in the brain that’s somehow related to his or her pain.
    One way to do this might be to compare his or her brain to a group of pain-free controls. And that is what the paper by Scarpazza and colleagues is about.
    Unfortunately, they show quite convincingly that current statistics techniques for VBM are just not up to the task. At least, not when control group has just 16 subjects. Their findings also assumed that their database of scans are from healthy subjects with normal brains. Try defining ‘healthy’ and ‘normal’.
    Nonetheless, the bottom-line for me is that brain imaging (or at least VBM) isn’t sensitive or specific for medical diagnoses in my patient with chronic pain – not to say that it can’t be in future. We remain ever hopeful.

    Neil O'Connell Reply:

    Thanks for your comment Michael,

    I would agree with you totally. The existing VBM data in chronic pain has attracted a great deal of attention but, for now at least, there seems to be good reason to treat it cautiously.

  2. Anders Eklund says: