- Describe to what extent scientific evidence influences the practice of medicine
- Define evidence-based medicine and explain why it has become a topic of such major importance
- Explain the relative advantages and disadvantages of an evidence-based approach to clinical medicine
- Describe the various steps necessary to practice evidence-based medicine and identify online resources available to help clinicians
|Important key words or phrases.|
|Important concepts or main ideas.|
Even though a curriculum full of basic sciences courses would suggest otherwise, using science to teach and practice medicine is a relatively new idea. While many of the most influential forms of medicine have long been based on careful observation and documentation, it is only recently that clinical medicine has undergone careful scientific scrutiny. Through the ages, healers were known less for their scientific knowledge and rational thinking than for their clinical experience and wise judgment. In many respects, this has not changed. The apprenticeship model, where students learn a trade by working closely with professional mentors, is alive and well in medical education. In fact, most of you will spend at least five years apprenticing before becoming a mentor yourself. Most of your professional education will not come from textbooks or the peer-reviewed results of clinical trials, but from your own experiences seeing patients alongside teachers who learned the same way – a method of learning that will continue for the rest of your career.
Why don’t we rely exclusively on the scientific method in the practice of medicine? There are many good reasons for this, one of which is simple: we are not scientists. We are healers, and no amount of science will ever be sufficient to take care of patients. Soon enough you will become aware of a palpable tension between patient care on the one hand and the science of medicine on the other. If done right, the practice of evidence-based medicine can be an effective way to resolve this inevitable conflict.
2. Ways of Knowing
Physicians generally base their clinical decisions on three broad sources of information: clinical experience, scientific models and outcomes research. Since each alone is insufficiently reliable, nor flexible, it is best to consider all three to determine a course action. Let’s look at each individually.
2.1. Clinical Experience
Physicians have historically learned to practice medicine through the repetitious accumulation of a wide range of experiences with previous patients. As mentioned earlier, this is still the case today. When a new patient is encountered, we instantaneously draw on these experiences to make a diagnosis and plan treatment. Let’s say, for example, that a patient comes in complaining of a productive cough for two weeks. By this time, we have probably seen scores of patients with similar, if not identical, symptoms and risk factors. We recall the diagnoses and treatments we, or our mentors, gave these patients in the past. We may even remember how these patients did with our treatment, though probably less consistently. Based on these recollections, we make the diagnosis of acute bronchitis, prescribe a course of treatment and inform the patient of what they can expect. Two centuries ago, physicians did exactly the same thing only the primary treatment may have been purging and bloodletting rather than antibiotics.
What are the drawbacks of using your own and others’ experiences as a basis for clinical decision-making?
2.2. Scientific Models
Periodically, individual practitioners or groups of clinician-researchers notice clinical trends, or they undertake careful reviews of large collections of clinical encounters looking for patterns. The scientific models they develop from these observations have been used to anticipate how a given patient ought to react under certain circumstances.This is the deductive reasoning that has become the hallmark of modern scientific medicine. Using this approach, a physician can make an educated guess about a patient’s diagnosis and treatment without ever having seen a similar patient. Experience helps, but it is not absolutely necessary to make the correct call.
While the application of deductive reasoning represented a major advance in medicine, it was not without its drawbacks. A theoretical model was only as good as the data on which it was based, and subsequent data that did not conform to a popular model was often ignored rather than used to develop a new theory. Returning to our bronchitis case, a 19th-century physician would routinely recommend, among other things, bloodletting for febrile illnesses. This was based on a perfectly rational model fashionable at the time, which stated that the symptoms of bronchitis were due in part to a build-up of toxins in the patient’s bloodstream. By removing as much blood as possible, the toxins were released and, in theory, the patient recovered.
Eventually, this model was replaced by others that relied less on clinical observation and more on experimental investigation. Basic research on human and animal physiology failed to recognize the existence of the purported “toxins”, instead showing how airway inflammation produced by infectious agents was responsible for bronchitis. This led to the development of new treatments targeted against the microorganisms and the inflammation they produce. Because of the tremendous success of experimental investigation, much of what you’re learning in medical school is based on the pathophysiologic models it creates.
What are the drawbacks of using modern explanatory models as a basis for clinical decision-making?
2.3. Outcomes Research
The most recent innovation in scientific medicine is outcomes research, an advance made possible by the science of epidemiology. Rather than focusing on the mechanistic explanations for a clinical observation, outcomes research attempts to predict future health events by systematically comparing groups of human subjects differentially exposed to agents of interest.This permits rational clinical decision making in the absence of clinical experience or even plausible pathophysiologic models.Over the last forty years, outcomes research has revolutionized the practice of medicine.
Pharmaceutical companies, for example, can only market their drugs after providing the federal government with data from numerous outcomes studies showing that their product is safe and effective. Interesting, they do not need to show how it works, just that it works. In fact, for many of the treatments we routinely recommend, there is no well-accepted explanation for their mechanism of action.
If we were to apply outcomes research to our bronchitis case, we would first investigate what happens to a group of bronchitis patients, half of which receives the treatment of interests while the other half takes a placebo. If it turns out that bloodletting, for example, is superior to a reasonable placebo procedure for the treatment of bronchitis, so be it. Assuming the research also shows it to be safe, we would be justified in recommending bloodletting. While it may be interesting and comforting to be able to explain these observations, a pathophysiologic mechanism is not necessarily required to make valid clinical decisions based on outcomes research.
What are some drawbacks of using outcomes research as a basis for clinical-decision making?
The practice of evidence-based medicine (EBM) involves recognizing how all three ways of knowing – clinical experience, scientific models and outcomes research – influence clinical decisions, and then combining them to maximize the quality and minimize the cost of care.While findings from outcomes research are often touted as the gold standard of EBM, it is necessary and perfectly appropriate to use clinical experience and scientific explanations to guide clinical decision-making. There are many times, of course, when all three are unavailable, and we are forced to make recommendations based on limited or biased information. This is a prime example of why the practice of medicine will never be based totally on the objective application of science.
Besides clinical experience, experimental models or outcomes research, what factors do you think influence clinical decisions?
3. Why Now?
Although clinical epidemiology has been around for decades, evidence-based medicine has only gained popularity in the past ten years or so. What factors have been driving the recent enthusiasm for EBM?
3.1. Information technology.
Evidence-based medicine is only useful in so far as up-to-date outcomes research is available to clinicians. The Internet has made this possible. For the first time, physicians everywhere have the means to adapt their practices to the most current information. Whether or not they choose to take advantage of this capability is another matter. Compared to others groups in the service industries, clinicians by and large have been slow to embrace electronic forms of communication. This will undoubtedly change with your generation of doctors.
3.2. Patient demand.
Even though many physicians do not take advantage of medical information on the web, their patients certainly do. According to a number of surveys, the quest for medical information is the most common reason Internet-users logon. Armed with reams of medical facts, some patients now arrive at their appointment more up-to-date then their physicians. This provides physicians with the incentive to become more evidence-based if only to keep pace with the increasing sophistication of their internet-savvy patients.
3.3. Popularity of alternatives.
Depending on the survey, up to 40% of Americans routinely use alternative medical products and services to treat a wide range of health problems, a phenomenon that may be partially related to the easy accessibility of electronic health information. Some conventional physicians question the validity of these practices, citing limited scientific evidence supporting their safety and effectiveness. In order to credibly make this claim, however, conventional physicians must show that the medicine they practice is evidence-based. After all, how can you counsel a patient to accept your treatment when there’s no evidence that it is safer, more effective or less expensive than an equally untested alternative?
3.4. Unacceptable iatrogenesis.
Historically, allopathic medicine has long had a reputation for therapeutic aggressiveness, so much so that the harm associated with its treatments often outweighed its benefits. Until well into the twentieth century, without any clear proof of effectiveness, patients were routinely subjected to invasive and toxic interventions (eg, bloodletting) based solely on the untested theories and clinical experiences of (mostly) well-intentioned practitioners.
Under the circumstances, patients could not be faulted for their interest in alternatives (eg, osteopathy, chiropractic and homeopathy), many of which were probably just as ineffective, but considerably gentler and safer. Today, the medical profession takes seriously its obligation to insure that the risks of its treatments do not outweigh its benefits, and patients rightfully demand nothing less.Physicians can only live up to the lofty admonition “above all, do no harm” if the benefits they achieve outweigh the harms they produce along the way. This is particularly true for preventive interventions. It is easier to justify a painful or risky procedure for an ill patient than it is for someone who feels perfectly well and has only a statistical probability of future illness. Before drawing a prostate specific antigen (PSA) test on a healthy 50 year-old man, for example, or prescribing hormone replacement therapy for a healthy 50 year-old postmenopausal woman, physicians must ask whether the potential harm of these interventions justifies their potential benefits. Practicing evidence-based medicine is the only way to do this.
3.5. Practice variability.
Of all the factors contributing to a surge of interest in EBM, the economic argument has been the most persuasive. As the cost of health care continued to rise through the 1980s and early 1990s, payers were desperately looking for ways to bring the situation under control.
One problem that caught their attention was practice variability. They found that in an unregulated health care market, service utilization often bore little or no relation to medical necessity. If you were a women living in Maine, for example, you were almost four times more likely to lose your uterus if you lived in one part of the state compared to another, all other factors being equal. And, if you were a child growing up in Vermont, you had almost eight times the risk of losing your tonsils to a surgeon depending on where in the state you lived [Geyman. J Am Board Fam Pract, 1998. 11(1):46-56].
Without any scientifically credible way to explain these differences, it was clear that physicians were basing their decisions on factors other than the evidence. Private and government insurers began touting evidence-based medicine as a way to insure the rational and equitable distribution of their limited resources. Practicing evidence-based medicine went from simply a good idea to becoming a prerequisite for reimbursement.
4. What constitutes high quality evidence?
Practitioners of EBM have three basic sources of information on which to base their decisions: primary evidence, secondary evidence and clinical guidelines. These sources can be thought of as occupying levels of a cascade, with primary evidence feeding information into secondary evidence, which is the main information source for clinical guidelines. Each level includes numerous kinds of evidence, which vary in terms of their reliability and applicability to clinical decision-making. Let’s consider each separately.
4.1. Primary Evidence
Primary evidence consists of original research studies. Much of what you are learning in this course has to do with the interpretation of this type of evidence. While the randomized controlled trial is often cited in the evidence-based literature as being the most trustworthy, primary evidence may include any and all original research studies ranging from large observational trials to small case series.While a well-designed trial can provide useful information for clinical decision-making, primary evidence alone is insufficient for day-to-day practice. This is true for a number of reasons. First, since many of these studies are limited in their size and scope, their findings by themselves cannot be directly translated into many clinical settings. And, even if they could, one study of any size is rarely able to definitively answer a clinical question (with noted exceptions). Second, even if a well-designed study were published that had direct relevance to the patient population of interest to you, you probably wouldn’t know about it. There are currently about 25,000 medical journals. Even if you focused your attention on the 4,000 journals in MEDLINE (US National Library of Medicine's database), you would have to contend with 400,000 new entries each year. While it may be possible for subspecialists who focus their practices on a narrow set of conditions to keep up with all the primary evidence in his or her field, this is simply not an option for most physicians.
4.2. Secondary Evidence
Secondary evidence consists of quantitative and qualitative reviews of the primary literature. This may take the form of synthesized data (metaanalyses), systematic reviews of all available primary literature on a specific topic, and summary reviews of selected research in a given area. Secondary evidence addresses the two major drawbacks of primary evidence. Since the reviews take into account multiple studies – if not all studies – relevant to a given clinical question, their findings tend to be more applicable. The meta-analytical aggregation of the results of small, comparable studies can greatly enhance the power of these studies taken separately. And, by including essentially all well-designed studies addressing a specific clinical question, a systematic review is capable of detecting and interpreting contradictory results from multiple trials. Also, someone has done the work for you. Rather than reviewing 20 studies of varying quality, you need only read one review to have your question answered.
4.3. Clinical Guidelines
While secondary evidence is extremely helpful, many physicians find that it is not easily translatable into the routine care of patients. To increase its clinical practicality, various organizations use secondary evidence (and influential studies from the primary literature) to develop highly practical clinical guidelines.Physicians find these particularly valuable since they eliminate the work of searching and appraising the literature. Many government agencies (eg: Centers for Disease Control and Prevention, Public Health Service), medical specialty organizations (eg: American Academy of Family Physicians, American College of Obstetricians and Gynecologists), and well-funded interests groups (eg: American Cancer Society) regularly publish guidelines for clinicians. The health insurance industry is extremely interested in these guidelines as a way to reduce practice variability and increase cost-effectiveness among their providers.
It’s important to note that clinical guidelines are not the same as protocols. Whereas a protocol is a step-by-step instruction on how to proceed with an individual patient according to specific clinical criteria, a guideline is a set of suggestions on how to practice evidence-based medicine under a given set of conditions. While physicians generally deride protocols as “cook-book medicine”, the flexibility inherent in guidelines leaves room for other factors that cannot be captured in the studies on which the guidelines were based: patient variability, clinical judgment and cost issues, for example. One problem with guidelines is that they always represent the views of experts to one degree or another. Unlike primary evidence, where a small group of researchers is interpreting their own data, guidelines are usually based on the opinion of multiple experts interpreting an array of other people’s research. It is not uncommon, therefore, for different organizations to develop inconsistent, even contradictory guidelines. This is especially true in the area of preventive medicine.
5. Practicing Evidence-Based Medicine
The prospect of searching the literature for evidence to support even a fraction of the decisions physicians make every day is daunting. Most physicians barely have enough time to see their patients. Fortunately, practicing EBM does not require accessing Medline for every case. As mentioned above, there are many excellent sources of evidence-based guidelines to help you make the most up-to-date, cost-effective clinical decisions. However, searching and evaluating the literature directly becomes necessary in at least two circumstances:
- When new research becomes available that could affect a sizable proportion of your patient population, you will need to assess whether or not the findings merit a change in your practice. Most of the rest of this course is designed to help you do this.
- There will always be times when you honestly do not know the best way to proceed. This may occur when: (1) an unusual case fails to conform to previous patterns, (2) a patient insists on a course of action that is inconsistent with your clinical judgment, and (3) you’re presented with information from a patient, or other source, that may or may not be evidence-based.
In these cases it is essential to know how to investigate what evidence there is in the primary literature to support or reject a particular course of action. Let’s use the following example to illustrate how this might be done:
A 50 year-old woman comes to see you for management of her type 2 diabetes. You have been having trouble controlling her blood sugar with oral hypoglycemic medications and plan to recommend she start insulin, a step she vehemently opposes. A friend of hers is taking a dietary supplement that you are unfamiliar with and asks you to prescribe it. Here are some guidelines (with full acknowledgement that they represent the opinions of experts) on how to proceed.
5.1. Step 1: Define the Question
Figure out what you want to know in advance and be as a specific as possible. Your questions need to be as specific as those of the researchers designing the study. For example, posing the question, “Is this dietary supplement better than the medications she is taking?”, will get you nowhere. After several hours of searching a wide range of studies, you may be able to cobble together an answer, which may or may not be clinically useful. A better question would be “Does this supplement lower blood sugar over the long term sufficiently to avoid the use of insulin in type 2 diabetic patients whose blood sugar is poorly controlled on hypoglycemic agents?” Even better, “How does this agent in combination with other oral hypoglycemic agents compare with insulin in the prevention of diabetes-related mortality in type 2 diabetic patients?” Often times you will need to slightly adjust your question to accommodate the availability of information.
5.2. Step 2: Search for the Evidence
While there are many different sources for secondary evidence and clinical guidelines, the best place to search for primary evidence is still MEDLINE and similar databases (refer to a list of resources at the end of these notes). Before you leave medical school it is essential you feel totally comfortable searching MEDLINE and other sources of primary and secondary evidence.
5.3. Step 3: Appraise the Evidence
Since the rest of this course is largely devoted to the details of Step 3, I only provide an overview here. Once you find a study that seems interesting and relevant, you need to consider three things in your appraisal. First, what kind of study is it? While RCT’s are considered the most reliable, there are times when an RCT will either not exist or be so restricted in scope as to be irrelevant to your patient’s circumstances. Other kinds of research, including uncontrolled trials and observation studies, are often useful. Second, how was the study carried out? This is another way of asking, was the study internally valid? Can the researchers reasonably make the claims they do given the design and executions of the study? Obviously, to the extent that there are significant biases and/or unaccounted for confounders, the results of the study – positive or negative – are less credible. This is also true for a negative result from a trial that was insufficiently powered to detect an effect. And third, are the results meaningful for my patient?Was the study externally valid?
5.3.1. How does statistical significance differ from clinical significance?
Returning to our example, if you uncover a well designed trial that convincingly demonstrates that insulin lowers the subjects’ blood glucose levels over the long-term 20% more than the addition of the dietary supplement in question (95%CI=0.72-0.88), it does not mean you should prescribe insulin instead. The subjects in the study may not represent your patient and her unique situation. While a 20% reduction sounds significant, the study does not address whether or not this improvement in blood glucose control (a disease-oriented outcome) translates into reduced morbidity and mortality from diabetes (a patient-oriented outcome). It could be, for example, that insulin leads to an increased incidence of hypoglycemic reactions and does not reduce mortality overall. On the other hand, the study may turn out to be clinically significant for two reasons: (1) other studies strongly suggest that tight control of blood sugar does contribute to reduced complications from diabetes, and (2) your patient probably does represent a fairly typical type 2 diabetic.
5.4. Step 4: Apply the Evidence
Once you’ve decided your study has something meaningful to contribute to your patients care, you need to decide whether and how to incorporate its findings into your recommendations for this patient. This is where your clinical judgment comes into play. Even though your appraisal of the evidence strongly suggests a given course of action, you and your patient may legitimately choose to take another path. This is because outcomes research is only a piece of the puzzle, albeit an extremely valuable piece that should always be given serious consideration. For example, while insulin appears to be a better choice according to the evidence, you still need to consider numerous other factors. In this case, taking insulin will have a tremendous impact on your patient’s quality of life and her negative attitude is likely to compromise her adherence to your advice. On the other hand, your patient will eventually need insulin anyway, and in the long run it may be far less expensive than this new medication. In the end, practicing evidence-based medicine means combining the best available evidence with your own clinical judgment. Neither is sufficient on its own.
6. Selected Sources of Evidence
- Cochrane Collaboration
- Database of Abstracts of Reviews of Effectiveness www.nhscrd.york.ac.uk/darehp.htm
- Patient-Oriented Evidence that Matters (POEM) www.infopoems.com
- ACP Journal Club www.acponline.org/catalog/journals/acpjc.htm
- Evidence-Based Medicine www.bmjpg.com/template.cfm?name=specjou_bewww.bmjpg.com/template.cfm?name=specjou_be
- Reviews in primary databases
7. Ancillary Material
- Williams D, et al. The case against ‘the evidence’: a different perspective on evidence-based medicine. Br J Psych 2002; 180:8-12.
- Kell M. Understanding evidence-based medicine: A primer. Am J Ob Gyn 2001; 185:275-8.
- Shaughnessy A, et al. Clinical jazz : Harmonizing clinical experience and evidence-based medicine. J Fam Prac 1998; 47:425-8.
- Geyman J. Evidence-based medicine in primary care: An overview. JABFP 1998; 11:46-56.
- Bigby M. Evidence-based medicine in a nutshell: A guide to finding and using the best evidence in caring for patients. Arch Derm 1998; 134:1609-18.