The MRC working group report: a critique by the Earl Baldwin of Bewdley, member of the Advisory Panel on the York Review

1. Background

1.1 The NHS Centre for Reviews and Dissemination published in September 2000 its Systematic Review of Water Fluoridation (University of York Report), having been tasked by Government to answer 5 specific questions following extensive parliamentary questioning about the scientific basis for a policy of fluoridation. A principal finding was the “surprising” weakness of the research base and the need for more high-quality studies to be done.

1.2 Dental and medical groups and to a lesser extent Government claimed the York Report in their briefings as confirmation of their long-held view that fluoridation was a safe and effective public health measure. Four months after publication the Chair of the Advisory Group which oversaw the York review wrote an open letter expressing concern that “the results of this review”, which was “conducted by an independent group to the highest international scientific standards”, “have been widely misrepresented” by dentists and doctors, and correcting “some of these errors” (Prof. T.A.Sheldon, 3rd January 2001).

1.3 Following the York Report, which assessed all the relevant evidence from the world literature and included suggestions for future research, the Government asked the Medical Research Council to “Provide advice on current scientific evidence regarding the health effects of fluorides in the context of water fluoridation”, and to “Consider what further research in this area might be required and what priorities should apply to usefully inform public health policy in this area”. A working group of the MRC has just published Water fluoridation and health (September 2002). The present critique is written by the instigator of the original parliamentary questions, and member of the York Advisory Group, out of concern at some of the contents of this report.

2. MRC working group findings

2.1 The MRC produced a number of recommendations for “research required to strengthen the evidence base on the fluoridation of water” (Department of Health press release, 5th September 2000). Probably the most far-reaching was their recognition that it is nowadays the total exposure to fluoride from all sources that matters in health terms, of which water is only one, and that this exposure should be monitored in individuals. Hand in hand with this is the need to establish whether fluoride uptake in the body is the same from artificially as from naturally fluoridated water. The report also advocated engagement between scientists and public over issues raised by fluoridation. These are welcome advances in the ongoing fluoridation debate.

2.2 Among the other useful recommendations, however, there are some areas of concern to anyone who saw York as the first high-quality, demonstrably unbiased (so far as anything can be), scientific exercise in the fifty-year history of water fluoridation. In both tone and content the MRC report reads substantially like a retreat from the high scientific standards which it was hoped might have drawn a line under the assumptions, impressions and poorly conducted studies which had held sway previously.

2.3 The first criticism is that, like most other comment in the past two years, it misrepresents what York reported. This is chiefly evident over the benefits of fluoridation: “The York review . confirmed the beneficial effect of water fluoridation on dental caries” (lay summary & 1.1) is stated without any of the qualifications York was careful to add. (Among these were “The studies included for [effectiveness] were of moderate quality . . and limited quantity” (12.1); “To have clear confidence in the ability to answer the question [on caries reduction], the quality of the evidence would need to be higher. The failure of these studies to deal with potential confounding factors or to provide standard error data means that the ability to answer the objective is limited” (4.9); “Only one study addressed the positive effect of fluoridation on the adult population. Assessment of the long-term benefits of water fluoridation is needed” (12.9.1, a section on research whose tenor was that more work on efficacy was needed.)

2.4 While the MRC picks up some of this in its research recommendations (6.2.6 & 7), and acknowledges York’s finding that “little high quality research had been undertaken in the area of fluoride and health more broadly” (1.1), it significantly fails to link this weakness of evidence to fluoridation’s possible benefits in caries reduction as York had done. This false assumption that the case for the effectiveness of caries reduction has been satisfactorily proved runs unchecked through the report, with the effect that some subsequent arguments are hard to sustain (e.g. on social inequalities and cost-effectiveness in 4.1.2).

2.5 With this goes an apparent lack of concern for and discrimination over the quality of evidence generally.Where one of York’s principal messages was the weakness of all the evidence in the fluoridation literature, and the need to ensure that any future studies were of high methodological quality, this note of caution is only selectively sounded by the MRC. The report adduces its own evidence with, unlike York, little indication of how good or bad it may be. An exception is its amplification of the social inequalities question, where it brings in foreign studies not considered by york (4.3.2) to which it assigns such low validity scores that it is hard to see what they can add to the evidence base. The MRC acknowledges York’s arguments for great caution in interpreting studies on this issue because of their poor quality, before overruling them with “authoritative reports” of its own (4.3), reports of which York was aware but which it did not consider good evidence. This produces the unqualified statement in the lay summary, repeated in similar terms in the conclusions under 4.3.3 & 6.1 and unfortunately echoed in the Department’s pressrelease, that “The majority of the research conducted to date indicates that water fluoridation reduces dental caries inequalities between high and low social groups.”

2.6 In 5.1.3 the MRC highlights a recent study by Hillier on bone health as evidence of absence of risk over a lifetime’s exposure to fluoride, pointing to a “possible limitation” that the fluoride was natural and not artificial. It would have been helpful in assessing the evidence to know that York had assigned this study to level ‘C’ (“lowest quality, high risk of bias”) with a validity score of 4/9. Another curious example of evidence is the MRC’s treatment of the cancer issue under 5.2.1.i). The 1977 analysis by Yiamouyannis & Burk, which suggested an association with fluoridation, is criticised; the 1991 study by Hoover, which suggested no association, is cited with approval. Both were classed level ‘C’ by York, but with validity scores of 4.1 for Yiamouyannis and 3.3 for Hoover.

2.7 The question for the reader throughout is ‘Whose interpretation does one accept?’ York is not infallible: but a specialist team of 10 reviewers who spent a year systematically assessing every relevant paper to a set of consistent and internationally recognised criteria, with a transparency that invited worldwide comment and criticism on the York website as the exercise progressed, advised by an expert panel representing all points of view*, should not be lightly disregarded. When the MRC cites a paper, there is no means of knowing its worth unless York also assessed it; where this is not the case, which is most of the time, one can only conclude that it did not achieve York’s lowest level ‘C’ in terms of evidential quality. By admitting evidence of even lower quality the MRC has inevitably lowered the average validity score of the evidence base overall, which cannot improve future decision-making. Even a House of Commons Hansard report (1.3) and a suggestion on policy by the British Dental Association (4.1.2) are thought worthy of inclusion as evidence.

2.8 This leads on to another disturbing feature of the MRC report, which is to cite previous surveys (e.g. in the case of social inequalities Acheson, 1998), which were not conducted to the standards or with the rigour or thoroughness of York, and to prefer them over York. There are criticisms of detail of York at various points in the report, and better qualified scientists will no doubt judge whether these are well founded or not, just as they will judge whether for example the cross-sectional studies the MRC recommend meet the methodological objections raised by York; but dismissals of York’s assessment of evidence in favour of earlier, less comprehensive, systematic and even-handed surveys are not likely to commend themselves.

2.9 The trend in the MRC report as contrasted to York is chiefly to downplay evidence unfavourable to fluoridation, and to emphasise its benefits. A small but telling example is the use of the word “optimal” fluoride concentrations (5.3.5), implying that the case for caries reduction has been properly and specifically made. 1.3 lists supporting organisations, but there is no corresponding list of bodies that have rejected fluoridation. A paragraph referring to “negative fluoride balance” when applied to children (3.5) seems to imply that they need fluoride, when no such need has been demonstrated (Dietary Reference Values for Food Energy and Nutrients in the United Kingdom, Department of Health, 1997). York’s figures are used in 5.1 to show the improbability of there being a bone fracture problem; they could equally have been used to show that of the 9 studies that reached statistical significance, 5 “[indicated] an increased risk of fracture” (York, 8.5). Neither statistic has much evidential weight because of the low quality of the studies; but it was the one favourable to fluoride that was given. Similarly, with the cancer statistics it could have been as interesting to know that 9 of York’s analyses (9.6) found more cancers, although 11 found fewer, as it was to see a comparison between two large but poorly conducted studies. York was criticised for including studies on dental fluorosis from hot countries, where more (fluoridated) water is drunk, and inferring a dose-response relationship (4.2); yet the MRC felt (4.3.2) that York should have included foreign studies, of extremely low validity, to help demonstrate effects across social classes. The MRC prefers lower figures on fluorosis incidence from a EU survey (quality unknown) to those produced by York (4.2.1), citing in addition two recent English studies which one must assume that York rejected on quality grounds.

2.10 4.1.1 draws attention to a possible underestimation of the benefits of fluoridation. The equal possibility of overestimation is not mentioned, arising from the widespread lack of blinding to fluoridation status leading to likely observer bias in a culture which has assumed until recently that fluoridation unquestionably reduced caries. (In this context the MRC rightly drew attention in 4.2 to inter- and intra-examiner variability.) 4.2 introduces a figure of 10% alongside York’s 12.5% for fluorosis of aesthetic concern; York’s own summaries and conclusions however make it plain that 12.5% is the figure to be taken, and the writer confirmed this with the lead researcher at the time. York’s 14.6% caries reduction becomes 15% (4.1) in a section which is not otherwise shy of decimals. 5.3.2 states that “The York Review found no evidence of reproductive toxicity in humans”, but without stating that practically no work had been done in this area (there were only 2 studies that could be considered: since there was a faint suggestion of harm in each of them, and the MRC acknowledges that fluoride “has been implicated in a number of adverse outcomes” in this area, it might be unwise to assign this a low priority for research).

2.11 York considered that “The outcomes related to infant mortality, congenital defects and IQ indicate a need for further high quality research” (12.4). Whiting et al’s more recent systematic review of the evidence for Down’s Syndrome cited by the MRC (5.4) repeats the call for more high quality research. The MRC does not share these concerns, assigning low priority to two of these and not discussing infant mortality at all. Again, whose interpretation should be accepted?

2.12 If the point is pressed, it is to show that the downplaying of evidence unfavourable to fluoridation is a noticeable feature of the report to anyone who is familiar with York, although it is not universal and is part of an exercise which has produced useful recommendations for the way forward in a poorly researched area. But this makes it problematic, for example, when trying to assess the validity of the MRC’s stance on animal studies (5.2.2) among others, since reports of this kind are vulnerable to selective quotation in the way that York was not. It is true to say that some researchers are worried by more recent rat studies than the one cited, but since animal studies were outside York’s remit the truth is hard to come by.

3. Omissions

3.1 The MRC discusses risk assessment, the public’s perception of this, and the need for education and dialogue (section 2). These are important. But there is a lack of balance here too. One of the first questions to which the public might want to know the answer, before considering possible harm, is ‘How good is the evidence for caries reduction?’ This in turn invites the key question, ‘How strong does the evidence have to be for a public health measure which is in effect compulsory – compared with, say, a medicine prescribed individually by a doctor, or one bought over the counter in a chemist’s shop?’ If the evidence is not very strong, or it is thought that a treatment of whole populations demands the highest possible standards of proof, or both, then the basis of such a measure is called in question, not least on grounds of waste of public money. The MRC has not thought to address these – possibly misled by its assumptions about fluoridation’s proven efficacy – even though it does suggest that the public be told about the strength of the evidence for alternatives to water fluoridation. (It is true that in 2.3.3 there is mention of explaining the ‘strengths’ of evidence from different types of research design; but this does not lead to the framing of these important questions, and it is given more in the context of not expecting too much from old studies.)

3.2 Another issue for public debate emerges from 3.7. The question is raised whether fluoride might accumulate in tissues fast enough to risk pathological change within the lifetime of “more than a small (and defined) minority of those exposed”. It should be for society as a whole to determine the ethics of exposing any minority, however small, to a pathogen which it cannot avoid, for the sake of a benefit to others which can be achieved in other ways.

3.3 The low quality of the evidence base has been referred to, and York made the point that it was not only the older studies that were at fault. A research recommendation that was a constant theme of the York report (especially in 12.9) was the need to use proper methodology in devising high-quality protocols for future fluoridation studies, since it was the lack of these which made it impossible to say anything with “clear confidence” about any aspect of fluoridation, including effectiveness. Blinding, sugar consumption, eruption of teeth, were among confounding factors to which it drew attention. There is mention of some of these in places in the MRC report, but it receives nowhere near the emphasis that it did in York, and the section on research recommendations (6.2), while mentioning confounding factors in two cases, would not lead the reader to believe that it is this aspect more than any other which has produced such a lack of knowledge about a health measure which has operated for over fifty years. A level ‘A’ study or studies are urgently needed in order to have “clear confidence” (or not) about effectiveness in caries reduction, and it is to be hoped that recommendation 6. will provide these. Blinding to avoid observer bias in a field where belief in fluoridation’s effectiveness is so strong is essential, and needs emphasising.

3.4 Randomisation is not mentioned, even if only to be dismissed as a ‘gold standard’: however difficult in this case, it should at least have been considered. The MRC could have usefully refined York’s recommendations on methodology, in furtherance of its remit, but it has been if anything less concrete than York. Keeping areas of possible concern “under review”, for example, is nebulous and a recipe for inaction.

3.5 The report considers exposure to toxic substances as a by-product of fluoridation schemes (5.3.9-13). It might usefully have set the scene by placing in context the relative toxicity of fluoride itself. This matters because if fluoride toxicity approximates to that of, say, Vitamin C (as some people wrongly believe), it can be confidently stated that risks to health at 1 ppm are implausible, and no further research will be needed. Disposition of Toxic Drugs & Chemicals in Man, 3rd Edition (Eds. Baselt & Cravey), Year Book Medical (Chicago/London, 1989) places fluoride in the same general area as arsenic and lead, while Clinical Toxicology of Commercial Products, 5th Edition (Ed. Gosselin et al, 1984) gives fluoride and fluosilicic acid toxicity ratings of “4 – 5(?)”, as against arsenic’s “5(?)” and
lead’s “3 – 4” (4 = “very toxic”, 5 = “extremely toxic” in a 6-point scale). This information, not entirely accurately reflected in the MRC’s statement that “Very high” levels are known to be toxic (5.2.6), cannot but add plausibility to concerns about human health, given that safe doses of highly toxic substances are often reckoned in parts per billion, and it is something that the public might reasonably want to know given the report’s acknowledgment of the narrowness of fluoridation’s therapeutic window (3.6). These questions have never been publicly addressed by dentists or Government, presumably because they could cast doubt on a fluoridation policy. They will need to be, and the MRC missed an opportunity to bring them into the open when speaking of risk assessment.

3.6 York thought it proper to allude to environmental and libertarian concerns which would have to be addressed before any decision to fluoridate further was taken. These have been the basis for some decisions against fluoridation in European countries, which again is a context the public might find useful to know. Again the MRC is silent.(5)

4. Summary

4.1 This is a curate’s egg of a report. With all its good points, it has neither the thoroughness nor transparency of York, and it lacks the safeguard that York had in this contentious area of including a balance of informed representatives of different points of view* which did much to avoid unintentional bias. It is a fair question to ask why the MRC was given a remit to do what York had in great part already done to a high standard (“Provide advice on current scientific evidence”, “Consider what further research”), in effect second-guessing the original reviewers. At the same time it has not done what it was uniquely placed to do, namely translate the research needs into firm recommendations about the types of study which will give the reliable answers which have been lacking because of poor methodology. Those of a sceptical turn of mind will note the Department of Health’s reactions to the debate from the beginning: a firm upholding of the policy of fluoridation to which it had long been committed, and a welcome to both reports on the basis that they supported its case, with examples intended to show this which are not scientifically accurate. There was no mention in its press release of the possible problems of total fluoride exposure, which was a principal finding of the MRC report, nor recognition of any discrepancy in praising fluoridation as a health measure while admitting that too little was known about its effects.

4.2 The MRC report should do some good, but has undermined itself to a considerable extent by a lack of discrimination over quality of evidence, compared with the York report, and by a tendency to overvalue the positive effects of water fluoridation and possibly to undervalue the negative. This could, if borne out by events, have unfortunate and expensive consequences.

Edward Baldwin, 27th September 2002

*There is a sociological as well as scientific literature of fluoridation which documents the unusual historical polarisation of views about this policy and the difficulty of influencing entrenched opinions on either side. The Government agreed to involve in the York enquiry a balance of supporters and opponents of fluoridation with the aim of minimising bias. The MRC working group had a high proportion of dentists and Department of Health officials who were connected with existing fluoridation policy-making; [in?]adequate research expertise; and little clinical medical input which might have been useful for any index of suspicion surrounding possible harms.

 

Supporting letter from the Director of the Centre that conducted the York Review

The following is from a letter by Jos Kleijnen, Director of the NHS Centre for Reviews and Dissemination, written in support of Lord Baldwin’s Critique of the Medical Research Council’s Report on water fluoridation.

6 October 2002

Dear Edward,

I have read the MRC Report and I agree with the general comments you have made about it. It seems an old-style document based on a non-systematic approach. It cites studies selectively. It cites studies already considered in the York report to reinforce certain cases and in places it uses contradictory arguments to make certain points.

 

For example, in one place the York report is criticised for including foreign studies that are claimed to be less relevant to the UK (on fluorosis – making the point that in the UK this problem is smaller than we found). But in another place the York review is criticised for not including foreign studies (making the point that the case for fluoridation decreasing social inequalities is stronger than York suggested.).

 

When discussing areas not covered in the York report, a non-systematic approach has been taken. As such, I am not sure what this report adds to the discussion, apart from some more extensive recommendations for further research.