Frequently-Asked Questions on Levels of Evidence

We're delighted when people find our stuff on "levels" worthwhile. If you have any questions about their derivation or usage, please consult this list to see if it has already been asked. If not, contact us and we'll respond.

  1. What criteria are used for these levels and grades?

A group of us* started from the shared objective of wanting to maximise the help and minimise the harm we do to patients by basing our clinical decisions on the sorts of evidence that are least likely to be wrong. Operationalising this goal begins with the conviction (unprovable) that a systematic review documenting homogeniety in the results of a large number of high-quality RCTs (randomised with concealment, double-blinded, complete follow-up, intention-to-treat analysis) gives us the least biased estimate of the effect of an intervension. This becomes Level 1 evidence, and recommendations based on it are designated Grade A. Then, because we can document progressively increasing distortions of Level 1 results as we move to systematic reviews with heterogeniety, then to individual high-quality RCTs, then to less rigorous RCTs, then to cohort studies (including outcomes research endeavors), then to case-control studies, then and to case-series, and then to expert opinion, we formed a hierarchy of levels based on this progression.

We then developed similar Levels and Grades for evidence and action in the other domains of clinical diagnosis, prognosis, aetiology/harm, and economic analysis.

* Lots of individuals and groups around the world have contributed to this development. They appear in diverse locations, including the "Purpose and Procedure" page of several evidence-based journals. The specific group who generated this particular set are Chris Ball, Dave Sackett, Bob Phillips, Sharon Straus and Brian Haynes of the Centre for Evidence-Based Medicine at Oxford.

  2. What do we mean by "outcomes research"?

We define this term in its US usage in the PORT sort of work that studies a cohort of patients with the same diagnosis (stroke, heart attack, etc.) and relates their clinical and health outcomes (death, disability, events, etc.) to the care that they received (aspirin, surgery, rehabilitation, etc.). Since it is NOT carried out in the form of RCTs, we can't attribute efficacy to any of the analyses. Rather, they tell us whether the outcomes we'd expect to observe (based on real RCTs) are being observed in the hurly-burly of real-world clinical care.

  3. What do we mean by "first principles"?

We mean the pathophysiological principles used to determine clinical practice. For example controlling blood pressure in patients with aortic dissection is based on the principle that lowering blood pressure will reduce ventricular output and so decrease the risk of further extension of the dissection. as with many "first principle" treatments, this has not been proven in any clinical trial, and may therefore be disastrously wrong. A good example of where first principles went wrong was demonstrated in the CAST trial. Patients post-MI commonly develop arrhythmias, often with fatal consequences. First principles suggest prophylactic treatment with anti-arrhythmics that reduce these arrhythmias must also reduce mortality. The CAST study randomised patients with these post-MI arrhythmias to flecainide/encainide or placebo, and found that more patients on the anti-arrhythmic died than those on placebo.

  4. How do we link grades and levels?

Like the individual levels, these are value judgements based on the desire to minimise harm (from useless or harmful therapy), cluster types of evidence of similar validity (homogeneous SRs, solid RCTs, all-or-none studies), and provide indications of similarity across the different sorts of questions (Rx, prognosis, diagnosis, etc).

The guides we're creating provide as detailed recommendations as possible e.g. treating giant cell arteritis:

• Give steroids (A) immediately (D) at doses of at least 20 mg (C) daily (A).

We use the best evidence available to provide the recommendation grade, rather than attempting to represent an overall view. We create one-page summaries (critically-appraised topics) on all the studies we use, and commonly find only one study meets our criteria for inclusion. The grade is matched directly to the level of evidence.

  5. Where would we place N-of-1 trials in this hierarchy?

We do N-of-1 trials to find out the best treatment for an individual patient, not for the average patient. We neither know nor care whether their result can be generalised and they rank as case reports (i.e. level 5).

  6. Where would we place cross-over studies?

If randomised, they can be level 1 (if multiple N-of-1's are conducted on the same intervention with the same outcomes for the same disorder, they sum to a multiple cross-over trial).

Click here if your question is not answered by this FAQ list.