9UNP0151SD_Victor Montori_WordPress

- The title of the talk is "The End of Evidence-Based Medicine." And it's not a rhetorical figure I actually mean it. It's difficult for me to give this talk every time, because I'm an advocate of evidence-based medicine. I've been teaching evidence-based medicine. And to have an end to the evidence-based medicine conversation is problematic for me. But it turns out that it also is something that's interesting to people. If I had said evidence-based medicine reaches its middle years, or has reached maturity, or anything like that, we'd have half the crowd today. People like to go to funerals. So that explains why you're all here. 

There's no way in any of this research to have any sort of financial disclosure, it would be nice but there is nothing to disclose. 

The fundamental thing here is that there are two principles of evidence-based medicine. The first one is that the better the research, the more confident the decision. So that's one fundamental principle of evidence-based medicine. It turns out that if you ask people around what they think of evidence-based medicine, they may say this, and then they will stop. They will think that this is the only principle of evidence-based medicine, but there are two, and this is one of them. And I presume nobody has a problem with this one. 

But there's another one I said. And the second principle of evidence-based medicine is that the evidence alone is never sufficient to completely inform a clinical decision. But one has to incorporate other aspects of the complexity of the human experience, like the context of the patient's experience, and the values and preferences of the informed patient in coming up with a decision that actually makes sense. 

In other words, if you were to train monkeys to analyze and critically appraise the literature and apply every randomized trial in a strict way, you will actually cause major harm. And the monkeys will be evidence-based. 

So there's something to expertise that needs to be incorporated as well, this is a graph here, in order to make good high quality clinical decisions, there's got to be some expertise required to incorporate what we know from the research, what's going on with the patient and what the patient's goals for life and health are. This expertise will vary across clinicians. And hopefully in our teaching programs, as we're training clinicians to become more effective, we're paying attention to those issues. 

Now, I'm going to tell you a little bit of history about evidence-based medicine since we are in a funeral, it is appropriate to talk about the life of the dead individual. And we tend to do in funerals, is we tend to say the positives and the beautiful things. So I'm going to do that. Anybody know who this guy is? Feinstein, very good. 

Feinstein is a very controversial figure in the history of research, but the major contribution, and we can argue about whether his major contribution was for the tobacco industry, but I think his major contribution was that he developed the idea of clinical epidemiology, and the idea that you can use quantitative methodology and apply it to the work of physicians. That you can quantify inferences, that you can quantify diagnostic criteria. And the idea of Clinimetrics. He was very good with words and he always created words to describe what he was saying. 

Now, I want to stop at this point, because it's really funny for me, from here, to listen to myself bounce on the back end, so I'm like having a long distance conversation with my parents in Peru. It's exactly the same sound. Are you bothered by my microphone in any way, or is it working okay for you? Okay, so I just have to cope with this. 

So this is Feinstein, and arguably, he's one of the people that starts laying out the foundation for what became evidence-based medicine. Who's this guy? 

Guess? David Sackett. And David Sackett is a guy that went from being a basic scientist, nephrologist, a public health officer, to then being one of the youngest chairs of a clinical epidemiology department at McMaster University in Canada, and a big advocate of the use of evidence-based medicine in clinical practice, particularly starting from randomized trials, which ended up being a source of discrepancy between these two. Feinstein quickly went to think that randomized trials could actually be quite useful for clinical practice. Anybody? 
- Tom Chalmers. 
- Yes, Tom Chalmers. Tom Chalmers, from Harvard, a big advocate, again, of using randomized trials. He's the one that I think coined the expression that if you have a new technology, you want to randomize the first patient exposed to it, and then move on until you learn whether the intervention works or not. And also a big proponent of systemic reviews and meta analysis. 

(hums "Jeopardy" theme)
 So, Ian Chalmers, no relation to Tom Chalmers, is an obstetrician from the UK who heard the call from Archie Cochrane, traditional epidemiologist from the UK, to summarize what we know about research and created the UK Cochrane center, which then evolved into the Cochrane Collaboration, which is a large, multinational volunteer organization that summarizes what we know in terms of randomized trials, and more recently about observations, tests, and diagnostic studies. 

So you can see the evolution of knowledge being summarized and synthesized. Yes, Brian Haynes. You guys get out, that's good. You know people. Brian Haynes also from McMaster University. It's interesting, because what he did is he harnessed the value of the Internet and the evolution of Medline and other databases to say, okay can we now bring this information that's being produced at dizzying speeds, and put it into the fingertips of clinicians. So the ACP Journal Club, the journal of evidence-based medicine, services that are now available, that are now services by email, that keep clinicians up-to-date, are things that he has developed, piloted, or has had his hands on at some point in their development. 

I'm going to show you one last picture. 

Yes, Gordon Guyatt. And so, Gordon Guyatt is a guy that in 1990 took over the residency program at McMaster University, and sent out a letter to the residents saying, look, enough of this expert-based teaching here that I tell you what to do sort of thing, now we're going to do scientific medicine. It got everybody really upset. And so in everybody being very upset, he decided he needed a better label for this, and then went for evidence-based medicine and published it for the first time in 1991 in the ACP Journal Club editorial. 

There's debate about whether this account is true, but I'm sticking to it. There's a very distinguished epidemiologist, an American epidemiologist, who claims that he came up with the term. And the way I think it has been resolved, although not to this latter individual's satisfaction is through a UK-- an NHS official, who remembers being in a taxi cab in Washington, D.C., when Gordon Guyatt told David Sackett, "You know, I just came up with this great term for this thing that we've been struggling. I'm going to call this evidence-based medicine." 

It is interesting that perhaps the fundamental thing, the only proof that Gordon came up with it and told Dave about it is from somebody recalling it from a taxi cab in Washington. That's how history is written. 

And this is 17 years ago. So this funeral is for a teenager. So it's a 17-year-old thing, so it's been evolving since its inception. 

But where are we now? I would put forward that this is the end of evidence-based medicine, because the evidence base, the research evidence that we can use to guide our clinical decisions is corrupt. And I'm going to show some examples of such corruption. And corruption is of a magnitude such that it cannot support evidence-based practice. But that's not enough. I think another reason why it's the end of evidence-based medicine is because systemically, patient preferences are being ignored. We were having a discussion coming here as to why that might be the case, but the fundamental thing is, they're being ignored. 

If you put those two together, it's the end of this thing. 

Now I could stop at this point and take questions, but I think that would be a little disappointing, so let's go through to understand the issue of the corruption of the evidence. 

Most of the effort in evidence-based practice has been looking at, can we understand where bias is introduced in clinical research. So of those of you who have gone through courses to understand clinical research, the attention has been, how do you ascertain bias? How do you look for bias and the different signs. So for instance in randomized trials, how is a randomized sequence generated, where there was allocation concealment, who was blinded, and to what extent, is there any major loss of follow-up. And we recently contributed to this evidence-base by pointing out clinical trials stopped early for benefit are likely to produce an overestimation of the treatment effect. 

So individuals that know critical appraisal can sit down, look at the paper and look for these things and say yes, the extent of bias, or the potential of bias in this study is significant, or is moderate, or is small, or is absent. This idea of clinical trial stopped early for benefit introducing bias has become completely illustrated more recently with the example of perioperative beta blockers. 

Here's a trial by published in the New England Journal of Medicine in 1999, in which there was an enormous difference between those patients that were going to undergo cardiac surgery, had a stress echocardiogram and were found to be at risk of ischemia, were given a beta blocker. 

The alternative arm just received standard of care. There was a major difference in outcomes with 100% reduction in the risk of nonfatal MI, and 80% reduction in the risk of cardiac death. In other words, perioperative beta blockers cured MI. 
(light laughter)
 Interestingly enough, nobody questioned the fact that these estimates appeared to be a little big. And decided that this will be a great target for quality improvement. And many institutions, including the Mayo Clinic, there have been major efforts put forward to make sure that every patient at risk that is undergoing cardiac surgery receive beta blockers very promptly. 

Now we conducted a systematic review of meta analysis of the evidence, and then we produced a graph that is a meta analysis. The red line represents the community of evidence in support of the use of beta blockers, and you can see it's fairly flat until the trial, and then it goes way up. And then it stays essentially fairly flat, or coming down, as the clinical trials accumulate. 

Since we conducted that review, two additional trials were published. 

One called POBBLE and the other one called DIPOM, which were very large trials in and of themselves. And the whole estimate for the reduction in cardiovascular influence approached null. 

But the confidences were wide enough that still further research was needed. Pretty far from being so sure that you have to implement this, that you would do quality improvement on this measure. The one trial has since been completed. We had a chance to participant in that trial. 8,000 patients randomized to perioperative beta blockers were the control. And the trial in fact, confirmed a reduction in the risk of coronary events, in the order of 30%. Not zero, but not 80 or 100%. 

But also, where it is powered enough to be able to show an increase in the risk of death and severe stroke. So you get peri-operative beta blockers. Yes, you prevent the chance of peri-operative heart attacks. But you will be more likely to die or have a stroke. 
(light laughter)
 So make sure that every patient at risk of coronary event undergoing surgery gets their beta blockers, because that would be a good quality improvement. 

So here's an example where we were misled by a clinical trial, stopped early for apparent benefit, and not only overestimated the treatment effect, but also hid the fact that this intervention could potentially produce harm. But that's bias. 

And bias is easy because it's something you can read on the paper. It's there for you to see it. 

With the right training, you can get to it. It's not too difficult. If that was our only problem, we would be okay. 

So what is our next problem? This is our next problem. Anybody have a problem with this? So why would science not be objective? People do it, right? It's because science is the hands of scientists. Scientists are not objective. 

Now why would scientists not be objective? I mean, maybe you are not objective but my science is pretty objective. Right? I love my ideas. 
(laughter)
 Now I need to love my ideas. In fact, I have to make you love my ideas, because you may, in fact, be in charge of my future, right? So I need to have successful ideas, because if I have successful ideas, I will have academic success. I have to not only believe that, I have to make sure that you believe that. I also have to make sure that you understand that my findings are critically important. 

My research is the only research that matters. It's the only research that should be highlighted in the New York Times. My work is of extreme high quality. I'm rigorous as rigorous comes, and you should believe all my results. Because I need my peer review funding, and you're my peers. And so I need to make sure that money put on my career and my research is money well spent. 

So this is enough to push me in certain direction regarding the way I present myself, the way I present my results to you, the way I present my results in papers and publications, because I need you to believe this. This is what we sometimes call putting your best foot forward, right? In fact, we may have mentored people to do exactly that. 

Now, issues and problems. 

This is a survey of almost 200 diabetes trials published in top journals in 2003, or 2003 looking back. And we found that when you look at things that are predictors of bias, they're infrequently reported, 11% report allocation concealment, half of them or less reported blinding of different groups, 71% reported on loss of follow-up. And then we looked at which of the trials, when you read them, have the apparent highest methodological quality. In other words, all these things are present. And what we found as major predictors were for-profit patient-oriented parallel trials, so those trials funded by for-profit interest have the better quality when you assess them on critical appraisal tools when they're published. 

So the ones that are putting their best food forward are the ones that are funded by for-profit interests. Okay? Here's a systematic review. I participated in this with some colleagues. We looked at both drugs and devices, and we looked to see association between funding source and the mapping to the treatment effect. 

And we found that on average for-profit funding sources had more positive trials favoring their product. 

Now, there's a number of explanations for this. Maybe for-profit funding sources are more careful with their money, and they're able to actually fund trials that they think are going to be positive. They've done their homework. There's one explanation. But as you know from the bottom is that this is adjusted for the quality of studies and adjusted to sample size. So they're really, really good at selecting the winners in some way. Well, that's one explanation. There may be alternative ones. 

Let me show you another point of evidence. 

This is about the enthusiasm for the product, or device, or drug that's being evaluated in randomized trial. Als-Neilson did this in JAMA. They looked for a sentence like this in the discussion and conclusion of section of the studies, "The experimental intervention should be the treatment of choice." So one of these very exuberant, positive things that you will say in a study, and then look at the predictors of ending up with a conclusion like that. And this was from 370 trials from 25 Cochrane reviews over a 30-year period of time, 18% in high impact journals. And what he found was that for-profit funding was associated, I think fairly strongly, you will agree I hope, with having that sort of sentence. But I just show you, they got bigger treatment effects. Why would they not conclude this sort of thing? Well, except that this illustration is after adjustment for treatment effect, and the magnitude of the adverse effect. So this is independent of the pros and cons, or the benefits comparative benefits of the control group. 

And then, this is now very much outdated, but this is the best I can find. There's a major flow, as you well know. There's a major flow of academic research from academia to industry. So remember my ideas and how much I love them? That was because of my own personal career and my ability to survive. What if it is 10,000 employees of a large company whose future is riding on the results of this particular research project? You may all recall that when the HDL, the drug that was supposed to elevate HDL was not found to be associated with major benefit, a whole Pfizer plant closed and people lost their jobs. So it's just not about my career and my ideas and how much I love them. This is a real problem when research results modify the livelihood of individuals. 

So, then, am I saying that for-profit research is fraudulent? 

No. Am I saying that I'm introducing a mechanism to cause bias? No, because I told you, that's easy to pick up. So what other mechanisms are present there to actually make things look better? And I'm going to call that spin. 

Now spin is a difficult concept for some groups, so I'm going to give you an example of spin. I'm from Peru! I'm from South America. And in Peru, soccer is big. Soccer is huge. 

Now it's actually a problem for Peru that soccer is huge, because we haven't had a team in the World Cup since 1982. 
(laughter)
 So when your biggest sport doesn't deliver, a country can go into massive depression. That's how I would characterize our sports situation in Peru. But this is spin. So we play against, say, Brazil. And say, Brazil puts in five goals against us. And the coach goes on TV and says, "Well, I'm very proud of my team. My team delivered. It was outstanding work." 

You know, "I'm particularly proud of my goal keeper. He only let five goals in." That's spin, all right? So when you know the truth is over here, and you try to create this alternative explanation or alternative view of the truth. 

So let's see how that plays out in research. Here's a trial. It's a high quality RCT. If do all the assessment for bias, it's a high quality randomized trial, 1700 patients with hypertensive diabetic nephropathy. Using-- pain, versus-- blocker. Almost three years of follow-up. 

This is lifted directly from the conclusion. Treatment with irbesartan, which is a receptor blocker, it was associated with a risk of the primary composite endpoint that was 23% lower than that of the group. I've given you, the P value has a bunch of zeros, and I'm giving you what that composite endpoint was. Doubling of the creatinine level, onset of end stage renal disease, or death from any cause. 

Now, how many people are clinicians here, that see patients? All right, so they tend to be distributed in the back of the room. Okay. I wonder if any of you would like to give me an example of how they would explain this to Mrs. Jones? 
(laughter)

 
(no responses)
 And I may actually go and have a cookie if you don't say anything quickly. 
(laughter)
 Anybody? 
- Only one-quarter of her is likely to suffer... 
(laughter)
 
- I suspect, if you actually see patients, that's the thing, you don't talk to them like that. Seriously, what would you say? 

How would you explain this? Say you talk about research with your patients. How would you say? Give her a PhD in statistics, you know, and she happens to have diabetes and hypertension, and you're going to explain this to her. What would you say? At this point, she's feeling very uncomfortable, because you're being silent for awhile. 
(laughter)
 Try it out. 
(inaudible)

- Four people are going to have the problem, if I give you this medicine only three will. 
- Thank you doctor. Now I understand, it's better. So what is it I'm going to avoid having? 
- Getting bad kidney disease, or dying. 
- Wow, so this is a drug that will help me live longer. That's wonderful. And so, that's exactly the intent, right? Is that you believe there's a 23% reduction of the risk of dying, or having bad kidneys, or having a distortion in your creatinine level in the laboratory. 

But if you look at the actual results, the doubling of the creatinine and the end stage renal disease goes with that 23% reduction. 

But what about the mortality? It's kind of not going with it. Right? So all of a sudden-- You told me I was going to live forever, right? And this drug may or may not actually have an impact on mortality. The point estimate is on the wrong side of the, right? It's actually going in the wrong direction. So composite endpoints can actually hide this important difference in treatment effect, give you an average that doesn't in fact apply to each of the components. 

And this is particularly critical if the components do not have the same degree of patient importance. So I'm pretty sure I'm going to feel when I die. 

I don't know if I feel the elevation of creatinine, maybe I will, but certainly it's not going to feel like dying, at least not to my family. So that's kind of weird, isn't it? But this is one mechanism by which you get a sense that this is going to cause a bigger impact than you think you will. 

Now this is actually a fun one. Of the clinicians in the room, how many of you, if somebody comes in with an exacerbation of emphysema or bronchitis to the emergency room, would you give them steroids? Pretty much everybody. Anybody who would not give steroids to somebody with COPD exacerbation? Okay, so pretty much everybody. The fundamental study-- Yes? 
(inaudible)

 
- Current recommendations are to use steroids only if you're below a certain level. They recommend that if you're above a certain level, that steroids have no benefit. 
- Great, fantastic. So, ughh, I'm in bad shape! You're going to give me some steroids. 
- You give them bronchial dilaters. 
- All right, so we have a nuanced answer. Other people said do it. And it's interesting, because the fundamental trial that started that tendency was published in the New England Journal of Medicine in 1999. 

These are the results, so they're all on the side of benefit. And the composite endpoint was death, then the need for mechanical ventilation, a readmission to the hospital or requiring steroids. And you can see that, as I'm going to show you, the number of endpoints, keep in mind that the steroid arm was double the size, the number of endpoints is actually very small to drive policy. You had two deaths in one arm and three deaths in the other. That's not a whole lot of evidence. Three people require ventilation in one arm. Three people in the other. Again, one of the arms was bigger. Readmission, six and five. So the majority of the evidence in this trial is about requiring steroids. 

In other words, give steroids to the patient, because if you don't, they are going to need it. 
(laughter)
 Right? That's what this trial showed. So we all started giving steroids, just in case they will need it later. So these composite endpoints kind of hide this stuff. 

This is my favorite, because I'm a diabetes doctor. So everybody knows that the UKPDS is the largest of each trial, about 4,000 patients followed for ten years. And it showed that intensive glycemic control reduces the risk of any diabetes-related complication. This is diabetes-related complications, and 35% in the intensive therapy arm had these events; 38% of the conservative arm had these events. 

The difference was 3.2%, P value .029. So what does that actually mean? Well, that composite endpoint had 14 endpoints in them, that included death, and cataract extraction. 
(laughter)
 So it's a little bit different, right, the experiments there? Amputation, stroke. That's kind of relevant, Cataract extraction? And it's interesting because as the trial progressed, they were finding nothing, no difference, no difference, no difference. So they kept adding components to the composite endpoint. Right? 

And so, you notice there are 13, I said there are 14. There are only 13, but there's a blank spot, because the last component they added was photocoagulation of the eye, which is a procedure of the eye where you laser the back of the eye to prevent blindness. And of that difference of 3.2% between the two arms after ten years of therapy, photocoagulation reflected 2.7%. So the last component added to this composite endpoint captured the majority of the impact of this treatment. 

Now does anybody here treat diabetics so this extent to prevent photocoagulation? No, we all think that we're preventing all these diabetes complications. 

So we've tried to contribute to this spin. In order to take the spin out of these things, these are the questions that we ask you to ask. Is there a big gradient of patient importance, death in cataracts? Is there a big range of treatment effects, you know, in terms of the magnitude of treatment effect like we saw with the nephropathy example? 

Is there a gradient of events, like we saw with the steroid example? 

And when you see big gradients your confidence in the composite should be very weak, and you should be focusing your attention on the effective on the components. The problem with that, is that all the components will be so underpowered and imprecise that you may not be able to draw any conclusions from this study, which is a big problem. Which begs the question why did we go to composite endpoints in the first place? And the reason is we needed a P value less than .05. There is something fundamentally wrong around how we're doing this. 

Now, the cardiologists are in deep trouble, because the cardiologists have been very good at adopting evidence-based medicine, to almost to the point of evidence tyranny, where if the trial has not occurred and the trial has not showed it, I'm not doing it. If the trial shows it, I'm doing it. Right? There's a bit of that going on. 

In their trials, we looked at 242 cardiologist trials, 112 used composite endpoints, and 86% failed the first two criteria. So in other words, they're incredibly hard to interpret. And I know cardiologists are very smart, but I suspect that they also have difficulty interpreting. And they either have chosen to ignore the difficulty and just go with the composite, or should be in major psychotherapy. 
(laughter)
 I don't know if that's happened. 

This is a large prospective blinded international clinical trial called the DREAM trial, of diabetes prevention. Eight milligrams of Rosiglitazone daily, together with lifestyle recommendations substantially reduces the risk of diabetes or death by 60% in individuals at high risk for diabetes. This is lifted from the conclusion. This medication reduces the risk of diabetes. 

The doctors are saying your sugars went from 124 to 128, and I repeated it and it was 132. You have diabetes. Okay? That experience, versus you're dead. 
(laughter)
 Big difference, right? There's a 60% reduction in that difference. And so, when you look at the actual data, it's really all effect on diabetes. In other words, the rosiglitazone lowered the glucose enough to actually make enough people go through the trial, so there will be fewer diagnoses. There was no effect on death. 

And as we know from the recent discussion, maybe the opposite is exactly true. 

So this composite endpoint is a problem. But it's certainly not the only way to spin results. Inadequate patients and comparators is another way. Haliparadol, at 40mg versus a new antipsych-- anticraziness, you know, antipsychotic drug. Right? And so, the antipsychotic drug comes reported as having fewer extra-parametal side effects, yeah, right? Or, I'm going to use a new antidepressant with amitriptyline twice a day. And difference? Daytime somnolence. 

We kind of knew that, right? And so when you have the comparative, you may actually have the wrong answer. Composite endpoints, we have large gradient. We've talked about surrogate endpoints of poor validity, hemoglobin, A1C is a good example of that. Post hoc changes in outcome definitions, and there's major literature on that now. Inadequate and abundant subgroup analysis. It works in this group and it doesn't work in that group. We have no idea how many subgroup analyses were done before that conclusion was reached. And then, the spinning of the discussion and conclusion section that we discussed. 

And the key risk factor is small treatment effects. 

If the treatment effects are small, you have to compensate. This is the joke, "Daddy, mom says you have this big heart to compensate for something. What is it that you're compensating for?" Right, you know that joke. Same thing here, if you have small treatment effects, you may need to start spinning your conclusions, you need to start looking at composted endpoints to get a bigger P value. You start looking at surrogate endpoints, because you're looking at small treatment effects. 

Now, where in Madison are we seeing small treatment effects? Pretty much everywhere. Right? We're looking for incremental benefits. 

Very small treatment effects. We're not looking for a cure for rabies, right? So very small treatment effects. So this is a major problem. 

So we talk about bias and spin. If that was a problem, well we can sort of deal with it because bias and spin are both present, they're both in your face, you can look for it, you can potentially see, you can try to infer it. 

So we have another enemy and the other enemy is bias reporting. So this is a third mechanism that we can see corruption of the evidence. This is a trial, a study by Chan and Altman looking at Danish protocols, followed them for ten years, so 102 protocols became 122 reports ten years later. They compared what they said they were going to measure with what they actually reported measuring in the trial. 

This was randomized trial. So 50% of the outcomes were incompletely reported. That is they were not reported at all, or they were reported just by saying "not significant." The odds of fully reporting an outcome, were 2:1 if the outcome was significant. So what it means in general, when you're reading the literature, you're keeping up with the literature, you're basically getting an over-optimistic view of treatments, because the majority of what's getting published are positive results at the full study level. But within the study the outcomes are interesting. Yes? 

So we actually looked at the literature ourselves to see if we take all these systematic reviews and we look at the proportion of trials that contribute to important outcomes, plus a lot of outcome reporting bias, we'll have bigger treatment effects in summary than even if most trials published all the outcomes. This is what that graph looks like. 

If they had 100 trials in the systematic review, but only 20 reported on mortality, you can see that there's an overestimation of the treatment effect on average than if the majority reported on mortality. 

And the only way this could be explained is if the decision to report is related to the magnitude of the treatment effect. So in general, the view of the literature that we have, because of outcome reporting bias is an overestimation of the treatment effect. And this again, is not fraud. It's just that that's what's interesting. That's what people want to read. 

And then there's the problem of the whole study. So, perioperative beta blockers positive trial, most people knew about this study. Most people didn't know about the DIPOM that was published in the BMJ. It had 900, nine times more patients, and people didn't know about it. The POBBLE trial, the same number of patients, but had a relative risk increase. 

People didn't know about it. The MAVS trial, five times the number of patients relative risk reduction of 15%. People didn't know about it. 

In my area of interest, that is for diabetes, the CARDS trial very positive trial, 30 percent benefit, relative reduction 37%. A trial with essentially the same design, the ASPEN trial, essentially started and stopped about the same time. Two additional years to get published, and was published in the subspecialty journal the Lancet, relative risk reduction of 10%. 

So, we have the problem of bias, spin, bias reporting and then fraud. I'm not going to talk about fraud, because fraud is incredibly difficult. Somebody discovers it, there's a whistle blower, you hear about it. But beyond that, there's nothing we can do about it. 

But if you then say this is corrupting the evidence base. When people say this is an evidence-based "blank," you should be running in the exact opposite direction, right? Because it is quite possible that what is being introduced here is the product of inferences drawn from research that is corrupt. 

If you have guidelines, if you have now guidelines that are evidence-based, that are telling you what to do, that's becoming a little more problematic, isn't it? And if now we're talking about, your quality is going to be measured on the basis of these parameters, this is becoming a little more uncomfortable, isn't it? And if I tell you now that your pocketbook is going to be hurt by this. If you're going to have pay for performance, that's going to be based on evidence-based guidelines, again, you should be running in the opposite direction. You should all be practicing in Brazil. They have a better soccer team. 
(laughter)

 Notice that I chose to say soccer, even though they call it football. I do not want to hurt anybody by saying they have a better football team. 
(laughter)
 And it's warmer. 

So this is a major problem. This is a major problem. And so, this is the point about the evidence being corrupted. What about the second point? What about this idea that patient preferences are being ignored? 

I'm actually going to skip through this fairly quickly to highlight one thing. 

This is, I think, a little bit of a video. This is a patient and a clinician at the Mayo Clinic. The clinician is telling the patient, "I think you should be on statins. You have diabetes. Your cholesterol is high. I think you should be on statins." And what is the first thing on that patient's mind? That guy. 
(laughter)
 Yeah, I've seen that guy. 

He's like the artificial heart guy, and he's in the canoe. He seems to be taking statins. He's good. I want to look like that. I want to be like that guy. So what could be on the physician's mind? Maybe the evidence, but from an advertisement. So we have the people making decisions heavily influenced by direct-to-consumer, direct-to-physician advertising and detail. 

But that's not it. Another push for further prescribing are these quality measures. 

You know, this LDL has to be less than 70, less than 50, undetectable cholesterol! 
(laughter)
 We need to use highest dose of statins that we can in these people. We just had Scott Grundy present Grand Rounds at Mayo Clinic on Wednesday. And literally, everybody over the age of 50 should have their LDL at less than 100. So how do patients respond to this challenge? By not being adherent. So after two years of therapy, half of the patients are no longer taking the stuff. And if you look at the place where the curve is separating, the large clinical trials, two years. So that sounds like waste of resources, doesn't it? 

When people have been asked what do you do with prescription? 15,000 will say, I haven't taken them. Why? "I don't think I need them," or "I don't like the side effects." These decisions are happening outside the visit. They're happening at home. The clinicians may or may not know about them. And this is consistent poll after poll, patients actually second guessing why they're getting all these prescriptions, and deciding to take them only occasionally, or not taking them at all, for reasons that may be completely unfounded. So we have a problem with communication. And so if this is the case, this is really the end of evidence-based medicine. 

Now, if I were to stop here, this would feel like one of those European movies. 
(laughter)
 The boy meets girl. They seem to be made for each other. They love each other dearly. And it's just getting to the end of it, and suddenly they turn around, and they walk away from each other. And then the credits roll, and you go, "What the hell?" 
(laughter)
 Right? I mean, this is just how you feel. 

I won't leave you like that. I think we should have a little bit of a sunrise in this. Is there hope? So, this is a Hollywood ending. 
(laughter)
 Is there hope? And I think the hope lies right here. I think we need to be advocating for a completely different funding structure for research. The funding structure we have is creating this problem. Is contributing to this state. 

We need to advocate for completely different funding structure. We need to find a way of funding research, for particularly important research in ways that are not dependent that don't make the funder care about the results. 

We need to be looking for large evidence. If we're going to be looking at small treatment effects, we need large evidence, either by having meta analysis of small trials that follow the protocol, or having large trials. But this idea that my trial of 15 patients will give you a P value less than .05 is ridiculous. That every trial has to be definitive is ridiculous. That the IRB says that your trial is unethical, because you don't have enough patients is ridiculous. And what happens is, you start creating this enormous effect sizes to justify your trial of 15 patients, because you only have money for 15 patients. And so you're creating some alternative realities to be able to get your trial through. That is ridiculous. 

We need to just basically realize that. If we have money for 15 patients, we do it on 15 patients. We report the results. And we hope that somebody else will get money for another 15, and we will contribute to an evidence base, and we will get confidence around the answer. Not thinking that all the trials that we're going to do are going to be definitive. 

We need to be asking patients important question and getting patient important outcomes. I do not know how it feels to have a doubling of my creatinine. I need to have patient important outcomes to be able to know, and to be able to educate Mrs. Jones. Right? 

Do you feel funny about that Mrs. Jones encounter? So there must be ways of doing that better. We need to have methods to enhance validity. 

We need to have routine registration of clinical trials and protocols so that we can actually go back and make sure nobody's cheating. I did a study of clinical trial registries. I'm going in there, it's just amazing how many stuff is not there, and how many times the protocol says, "the rest of our detail is on our Web site," and they give an institutional Web site. And you go there and you go to "Welcome to MERCK..." And then where do you go from there? You know, where is the data? 

And of course, if it lives in my Web site, I can change it any day. 

We need to have rigorous conduct of research, rigorous complete and open reporting of all the outcomes and all the data, so we can actually look for these things. We need to develop more spin detectors. We need to get better at this. 

The other thing we need to do is we need to have methods to incorporate patient preferences and decision-making. This is a decision made that we need to talk about statins with patients in ways that would not, hopefully, introduce bias. And what we found with this tool when we used it in a clinical trial was that patients really liked it. It improved their knowledge, and improved their risk estimation. It decreased their decisional conflict. It improved their trust. 

It improved the action if you think that 70% fewer prescriptions in the lower risk category. There were some short-term adherence improvements of three-fold. And this was just giving patients and physicians a script they can go through to understand the patient's risk and the risk reduction associated with the medication. And this was published in the Archives of Internal Medicine. So you can do these simple interventions to do it. 

I'm going to show you one potential mechanism for this decision aid that has nothing to do with the content of the decision aid. One physician with a patient, no decision aid. Same physician with a decision aid. Now we're having a conversation. Is that the mechanism of action, or is it the fact that we're provided information. 

I don't know. 

We're now doing this for diabetes medication choices. I think it produces a major change. Watch this. 
- Yes, I agree, we need to make sure your sugars are better controlled. That means we need a new medication. So which aspect of your next diabetes medication do you want to discuss first? 
- Weight change. 
- I'll let you review that. 
- This is the one I'm on now, right? 

- You're already on medformin. 
- This one looks good to me. 
- That one is going to help you lose some weight. What aspect of that medication would you want to learn about next? 
- I'd say daily routine. 
- Sure. 
- Oh, so this is injectable. 
- Yes, it's actually very easy. There are a couple of pens that have needles, and you inject into your belly. You do that twice a day before your meals. 

- I don't think I really want. 
- You don't want to go with needles? 
- I'd much rather just pills. So, the sulfonylureas? 
- That's the next best one for you in terms of weight? That's a daily pill. You can start with one pill a day. 
- And just 30 minutes before? 
- Yes, usually 30 minutes before breakfast. Would that be okay? 

The other thing to consider is that sometimes they can give you low blood sugar, so we're going to ask you to monitor a little more often than you're doing right now. But otherwise that will be okay. If you were to change your mind after you go home, I'll give you this pamphlet. It has the same thing as the cards. So as you're using the tablet, if you find that it's not what you expect, you can review this again and give me a call, and we can discuss it again. 
- Okay, all right. 
- Thank you. 
- I think, at the end of the day, we can get better at detecting bias and spin, and bias reporting if we pay attention to the revelation of sophisticated forms of reporting bias and misconduct. If we are able to promote patient participation and decision-making, so that we get the whole idea of evidence-based medicine, not only the evidence piece, but also the patient participation piece. In fact, are able to advocate for evidence-based medicine because the way we got to know that we had the problem we have now, is because we were following those principles of evidence-based medicine. 

So if we were to stop, if we were to declare defeat, if we don't train people to be alert of these issues, what will happen is when we all become patients, our clinicians will not have these tools to help us be healthy, and to get the medication and treatment that we would like to get. So maybe it's the end of evidence-based medicine. But maybe there's a little bit of hope. And I welcome your questions. Thank you. 
(applause)