Health advice on the net

Method
Unless you have specialist knowledge of the subject, you are unlikely to be able to assess the reliability or accuracy of a piece of information. The methods used to produce the information are, however, one important indicator. Let’s look at scientific research reports as an example of the kinds of information where method is a key indicator of reliability.

Scientific research reports
Whether you are reading an academic paper or a newspaper report, you can ask some common sense questions about the way research has been carried out.

NEW WONDER CURE FOR HANGOVERS
A scientist in Salisbury claims to have found a natural cure for hangovers using a mixture of vinegar, raw eggs, sugar and soya beans. He has carried out extensive research over a number of years with students at the local agricultural college. Four out of five students were free of symptoms within 30 minutes, he says…

Are you convinced by this? If not, what questions would you like to ask about the way the research was conducted?
You might have wanted to know…

Question Research terminology
How many students was it tested on? Sample size
How were they selected? (e.g. How drunk did they have to be?) Population definition
Did the research compare their recovery with similar students who didn't take the cure? Control group
How did he define and measure recovery? Outcome measures
So you don't have to be an expert in research methods to ask a few basic but very important questions.

Assessing method – what to look for
If some kind of research or data collection is involved: 

• Is it clear how the research was carried out? 
• Was the sample big enough?
• Was it the right kind of sample? 
• Was there a control group? 
• Were the right outcome measures used? 
• For surveys - were the questions robust enough?
•  Was there any potential hidden bias?

Provenance

• The provenance of a piece of information (i.e. who produced it? Where did it come from?) may provide another useful clue to its reliability.
• It represents the ‘credentials’ of a piece of information that supports its status and perceived value.
• It is therefore very important to be able to identify the author, sponsoring body or source of your information.
• But there is also an argument that this is unfair and that information should be judged on its own merit, rather than on the basis of who produced it and where it was published.

Authors
If you know who the author is you can:

• find out whether they are an acknowledged expert in the subject area
• find out what other papers or books they have published perhaps trace unpublished material like their PhD thesis
• find out if they are known to have a particular perspective on the topic and whether their views are controversial perhaps contact them in person.

N.B. There are dangers here – a well known ‘expert’ may be the first listed author on an article which is based on the work of one of his/her research students.

Organisations
If you know who the sponsoring organisation is you can check out:

• Is it a commercial company?
• A voluntary organisation? (Note that it could still be sponsored by another organisation with its own perspective)
• A statutory body?
• A professional body or trade union?
• A research organisation?
• Is it well-established or ephemeral/short-lived?
• Can you identify the people involved in the organisation?
• Could you contact them if you wanted to verify the information?
• Do the people have ‘other interests’ or links which might have a bearing on the way you regard the information?

Publications
It can be useful to know how something is published.

• Any individual can publish anything on the World Wide Web, or post to a discussion list – this has to be judged on its own merit and with reference to the author’s credentials.
• Journals and newspapers, whether in print or electronic, will in almost all cases have an editor and/or an editorial board that decides editorial policy which influences what will be published.
• Most academic print journals are peer reviewed – this means that articles submitted will be evaluated by at least two experts in the field before being recommended for publication.
• Some electronic journals do not have a peer review process.
  So while provenance in itself doesn’t prove the value of a piece of information it can hold a lot of important clues.

Assessing provenance – what to look for:

• Details of authors – who they work for, what else have they published? what is their status?
• Organisations involved – their work, their interests, their connections, who funds them?
• Publications – who is the editor? who is on the editorial board? What is their editorial policy? Is it peer reviewed?

Timeliness
The date when information was produced or published can be an important aspect of quality. This is not quite as simple as saying that 'Good information has to be up to date’.

Think about a newspaper article, published in 1999, about the recently discovered link between BSE and CJD. Would you consider this article to be out of date?

Well – it is and it isn’t. In other words – one person’s out of date newspaper cutting is another person’s historical document. Some documents (the Domesday book, for example) are indeed ‘timeless’ in that they will always be regarded as useful and relevant. In other cases it will be very important for information to be up to date.

Finding the relevant date may not be straightforward. In some cases it is difficult to ascertain the true ‘age’ of the information:

• There may be no publication date.
• There may be no indication of when the information was last updated.
• The date of publication may not reflect the date the information was produced e.g.it can take many years for research results to be written up and published.
• Information may not be very old, but may have been superceded (e.g. statistical series or regular reports). This means you have to be familiar with these sources and how they are produced.
• Information may appear to be ‘old’ but is in fact the most recent information on the topic and, as such, of value.

Summary – what to look for

• Is it clear when the information was produced?
• Does the date of the information meet your requirements?
• Is it obsolete? (Has it been superseded?)
• Or is it the most up to date information available on the topic?

PROMPT is a simple checklist to help you to think about information quality. We are not suggesting that you carry out a detailed evaluation of every piece of information you come across. But if you familiarise yourself with the PROMPT six dimensions, you will find that you can scan things very quickly and identify their strengths and weaknesses. It is about developing a critical approach and that just takes a bit of practice.

Some useful websites for looking at information quality

The Centre for Health Information Quality (CHIQ) was set up by the Department of Health in 1997 to‘act as a source of expertise and knowledge for the NHS and patient representative groups on all aspects of patient information in the aim of improving the NHS’s capability, competence and capacity to provide good evidence-based patient information’.

DISCERN is a specialist tool that focuses on information to help support treatment choices. It is unique in that it has been developed and validated in a rigorous way. It is used to appraise information for NHS Direct and NHS Direct Online and by many other information providers.

QUICK ideally the skills of critical evaluation of information should be taught and nurtured from an early age. The Quick Guide to checking information quality has been developed specifically to help young people to critically evaluate written information.

Other links:

Healthcare Commission

London Patient Choice Project

NHS Modernisation Agency

The material in this ‘Getting the best information…’ website has been adapted from extracts from the former Open University course - Making sense of information in the connected age (U120) and the current course Knowledge, information and care (K223)

If you would like to study more about finding, evaluating and using information, you might like to consider studying one of these in your own time. Take a look at these courses on the study page.

  < previous   Page 2 of 2