NRMN Health Research Talks – Ep. 7: Comparative Effectiveness, etc w/Dr. Kenneth Saag

NRMN Health Research Talks – Ep. 7: Comparative Effectiveness, etc w/Dr. Kenneth Saag


[MUSIC PLAYING] ANN SMITH: Hello, and welcome to
the NRMN Health Research Talk. I’m Ann Smith. Today I’m talking
with Dr. Ken Saag. Dr. Saag is Lowe
Professor of Medicine with the UAB Division
of Clinical Immunology and Rheumatology, and director
of the UAB Center of Education and Research on Therapeutics,
or CERTs, and director of the Center of Outcomes
Effectiveness Research and Education, or COERE. Welcome, Dr. Saag. KENNETH SAAG: Thanks, Ann. Good to be here. ANN SMITH: Good. We’re so glad that you’re here. What will we learn today about
comparative effectiveness research and patient
centered outcomes research? KENNETH SAAG: Yeah,
and a hot topic, one that’s really
been changing rapidly. I’m going to talk a little
bit about patient centered outcomes research, which
is the new buzzword. Comparative
effectiveness research has been what we’ve
talked about historically. And I’ll kind of make
some distinctions there. And then, I’m a rheumatologist. So I’m going to give some
examples from arthritis and osteoporosis, focusing on
the use of large databases, talking a little bit
about large simple trials, and also talking a little bit
about health care measurement and implementation science. ANN SMITH: Very good. It sounds very interesting. KENNETH SAAG: Great. Well, I think it’s good. You know, when we think
about translational research, we often think about
the early phase of translational research, this
idea from bench to bedside. And outcomes and
effectiveness research really moves us into the
later phase of translation, where we’re thinking about
epidemiology, health services research. How do we go not just from
the bench to the bedside, but as we say it, from the
bedside into the backyard? How do we get the word
out into the community? How do we implement
evidence, translate research into practice? ANN SMITH: Very
important question. KENNETH SAAG: There’s kind of
a series of overlapping Venn diagrams that I like to
use that kind of speak to the similarities
between terminologies, but also highlight that
some of these things are indeed a bit different. And so we can think
about the new patient centered outcomes research. We have a new Institute
now that is supporting a lot of this
comparative effectiveness research– in fact, as one
of their main focus areas. The oldest term is
health services research. And that was what the Agency
for Healthcare Research and Quality, and in its
older iteration AHCPR, used to support and
still does support. And then another area that we’ve
been very interested in here at UAB, and that has really been
one of the tenants of this is, pharmacoepidemiology. ANN SMITH: Yes. KENNETH SAAG: So PCORI
is the new Institute. And this is a group
that actually spins off of the Affordable
Health Care Act. It provides a trust fund, where
each time somebody purchases either public or
private insurance, $1 goes into the trust fund. And ultimately, they’ve got
about half a billion dollars to support research. You can see some of
the different areas that they’ve been supporting
from assessment of prevention, improving health care systems,
communicating and disseminating research, addressing
disparities, and methodological research. Now, they’ve moved
into some new areas. They’re actually focusing now
heavily on pragmatic trials. That’s been a big focus area. And while a lot of
their work initially was investigator initiated,
many of their new requests for applications are
actually very targeted. So they’re responding to
what, through discussions with patients and
other key stakeholders, what they believe some of the
most important topics are. One of the other great things
about the PCORI activity is that they’ve actually
spun off PCORnet. PCORnet is a group of Clinical
Data Research Networks– CDRNs– and also Patient Powered
Research Networks, or PPRNs. And there’s a whole
series of these now across the country
that are figuring out ways to exchange data to
use common data models, to collect data directly from
patients in the community, and to integrate all
this into research. It’s a very powerful engine
to do research that nicely complements the clinical
trial network, the CTSAs, is the clinical
[? NCATS-funded ?] activity, as well as the NIH
research collaboratory. So let me get into
one of the areas. And I put up here a few
of the different domains of health services and
comparative effectiveness research. Time doesn’t permit
me to cover them all. But I want to focus on at
least three of the key ones. I’d like to start off
talking a little bit about longitudinal cohort
studies of effectiveness of therapeutics and
other interventions. And this is a figure
that I’ve found very helpful over the years
that one of my colleagues up in Canada developed,
that reminds us that there is both evidence
generation and evidence implementation. And these are sort of separate
spectrums of the research specter. So we have to start off by
generating the evidence, and that’s– ANN SMITH: That’s in number one. KENNETH SAAG: –that’s
the first circle. And so that can be an
observational study and an interventional study. And then we often, if the
studies are under-powered or we don’t have enough evidence
just from a single study, we might start to move those
together and do meta analyses, systematic reviews. Ultimately, that may
result in guidelines. It may need economic analyses
to understand the cost implications. And from those, we can then
develop measures of quality. What’s the least we should
do in a given circumstance– a performance measure,
a quality indicator. And the standard baseline
of health services and outcomes research has
been measuring variation. And normally– not
always– but normally, a significant variation
in care across geography, across race/ethnicity may
suggest that this really constitutes an area where
quality improvement is really needed. And that’s where the final
area, translating of research into practice– ultimately, trying to
impact health care policy is very important. ANN SMITH: So it sounds
like an iterative process of going through all of that. KENNETH SAAG: Indeed, it is. That’s right. And then, information from
that should feed back, and telling us where we need
to generate more evidence. So let me give an example. A great study, one of the
most famous cohort studies is the Women’s
Health Initiative. This was a large
NIH-funded study. $100-plus million study that
is still going on in its later iterations that looked at
the safety of estrogen, either alone or in combination
with progestin, in cancer, and in heart disease, and
in other outcomes among post-menopausal women. And so we can see
that while there was some unexpected findings
at the time of an increased risk of heart disease
and breast cancer, we see relative risks that show
a 25% to 30% increased risk. The confidence intervals shown
next to the relative risk in most cases don’t include one. So these are significant
confidence bands. And that gives us
the relative risk. But we’re also interested
in a population as to many of the absolute risk. What is the risk difference? And you can see for
these same outcomes, what was the excess or reduced
risk per 10,000 women that were treated in the
same set of outcomes? ANN SMITH: So what
do the numbers mean in that column of the
excess or reduced risk? KENNETH SAAG: So in terms of
hip fractures, for example, this was the first
large study to show that we could actually reduce
the risk of hip fractures. There were five
fewer hip fractures among 10,000 women
who were exposed to the estrogen in this study. And in contrast,
seven more cases of heart disease, eight
more incidence of cases of breast cancer occurring. So really, some balancing
of risks and benefits, as is often the case. Now one of the things when
we see data like this, we should always ask– these are associations,
statistical associations. Is it a true association? Could it be due to chance? And that’s where the
statistical p-value is very helpful in assessing
the possibility of a type I statistical error. Or could it be due to
confounding or bias? And I want to say just a word
more about those two things. Bias represents a systematic
error in the study design. And that’s something
that really has to be addressed at the start
and in the design of the study. It’s very difficult to
handle analytically. Confounding is this false
positive association between an exposure– in
this case estrogen– and the outcome– something like
heart disease or breast cancer occurring. A classic example
of confounding is what appears to be an
association between alcohol consumption and
developing lung cancer. Well, probably not a
direct association. But we know that
people historically that have been higher
imbibers of alcohol have also been more
likely to smoke. So alcohol is indeed confounding
the association between smoking and developing lung cancer. Most of the time, when we’re
doing observational studies, we’re talking
about risk factors. And we want to be careful not
to just look for the light under the lamp post. Risk factors are very
important, but they often just give us a clue. And they may be
hypothesis generating. ANN SMITH: They’re
not the cause. KENNETH SAAG: They’re not
necessarily the cause. But thank you for mentioning
that because it really brings us to what is
a cause and effect. And from observational
studies, it’s very hard to assess cause and effect. But we do have some
principles put forward by Bradford Hill many years ago
that are still very relevant. The strength of
the association– is there a dose response? Is the finding consistent
with prior observations? Was there temporality? Did the exposure occur
before the outcome developed? Is it biologically plausible? Does it make sense? And is it reversible? Sometimes that’s not something
you can actually look at. But these are all
of the criteria that we use in trying to judge
the likelihood of something potentially being causal. We want to be very
careful, though, about saying something is
causally related, particularly from an observational study. Well, here’s another example
from some work we did now over 20 years ago where we were
interested in the association of glucocorticoid medicine–
steroids, if you will– and developing problems in the
bones and other bad outcomes. And this was a group of patients
with rheumatoid arthritis, half of whom were on steroids. The other half of
whom were matched by age, sex, and
duration of their disease to non-steroids users. And you can see some of the
very serious things that can occur– fractures,
cataracts, infections. So it really creates kind
of a concerning picture. And here, we can see it
in a Kaplan-Meier curve where you’re looking now in the
very bottom curve of the people who were on the highest dose. And in the very top
curve in green, those that weren’t taking
Prednisone, the control group. ANN SMITH: And
what does it show? KENNETH SAAG: Well, what it
shows is a dose response. And that is as you have
more exposure to Prednisone, you have a higher
likelihood of experiencing an adverse event, which
we just saw the listing of in the last slide. So the y-axis is the probability
of remaining free of an adverse event– the survival, if you will. And if you are not taking
Prednisone, just based on having bad rheumatoid
arthritis, at five years, you had about a 20%
likelihood of getting one of these bad
outcomes compared with about an 80% risk for
those who are taking between 10 and 15 milligrams. So again, a pretty
concerning finding. And we can look at it further
in a multi-variable way using logistic regression,
where now, we’re analytically accounting for the presence
of rheumatoid nodules, and the changes in the bone
of rheumatoid arthritis– two things that we
know indicate more severe rheumatoid arthritis. Both of those were significantly
associated with a bad outcome. But even after we
analytically adjust for those, we still see this dose response. We still see the increased risk
of developing a bad outcome in those who were
receiving glucocorticoids. Now, one of the
things that you should be worried about in such a
study is this possibility of a selection bias. This is the biggest
threat to the validity in therapeutic epidemiology. Sicker patients get
treated more aggressively. The doctors and patients
don’t decide by chance to take prednisone. But they realize the disease
is worse, and as a result, they suggest a more
aggressive course. The other concern is that if you
follow people in a differential way, you may be more
likely to detect an event– so-called detection bias. So these are two of
the risks to validity of these types of studies. Well, can we use clinical
trials to fix these issues? That’s one of the questions
that always comes up. And we know from the
evidence medicine people, the folks up at McMaster’s,
Gordon Guyatt, and Haynes, and others who have really
led the field in defining what is the top levels of evidence
that clinical trials are towards the top, whereas
our observational studies are further down. But unfortunately, just
looking at the steroid example, even for things that
are relatively common, like a 10% rate of a bad
outcome in patients overall, and a 50% increased
risk of developing, say, a broken bone– so an odds ratio of 1.5– you’d need very big numbers. And many of our clinical
trials in this particular area just are not going to have
over 1,300 people in them. So we’ve got to see a
two-fold increased risk with fairly high
rates at baseline to be able to have a study
that is going to have the power to pick up such findings. The trials often have some
practical limitations. [MUSIC PLAYING] Are there other
things we could do? Are there other data
sources we can use? And the FDA’s been criticized
for not moving fast enough. Some of our lay
publications tell us, why don’t we just use
these large databases that exist for the purposes of
billing patients and billing the payers of health care? But we have some
strengths and limitations here that I want
to review with you. ANN SMITH: OK, thank you. KENNETH SAAG: So observational
studies based on claims data have a lot of power, but they
also have some limitations. And we think about
all the health care that’s delivered by the
government, by private payers– it generates lots and
lots of information. Each time you come
in to see the doctor, or get a procedure, or
a test, there’s a code. There’s an administrative
data code that is generated. And all of this can then
be concatenated and kept in large databases. So for example, Medicare
data, Medicaid data is available nationally, and
can be purchased and used by academic sites such as ours. And with that, we can
also include sometimes prescription drug data using the
Part D benefit now of Medicare, or if you work with
a group, like Kaiser, say, where there is a pharmacy
benefit as part of the health care that’s delivered. ANN SMITH: So here
is a listing of some of the database resources. KENNETH SAAG: Yeah, yeah. I mentioned Medicare. I mentioned Medicaid. Aetna– another large insurer. Another great source is the VA. Really terrific in that
they’ve for many, many years now have had one of the real
gold standard electronic health care records. So not only do we have the
administrative data, the ICD codes, but we also have actual
electronic data available. Not always easy to get
to in a standardized way. Sometimes it requires
things like natural language processing to
abstract information from that in a systematic way. But very helpful. Let me give an
example from arthritis by one of my
colleagues who has been interested in working
with large databases to show how these can work. And there’s a group of
medicines out there now called the anti-TNF drugs. You hear about these on TV. These are drugs that
are very potent and very efficacious in improving
disease activity in conditions like rheumatoid
arthritis, and psoriasis, and even for kids that
have juvenile arthritis. ANN SMITH: So they help
with those conditions, but they also have
some side effects. KENNETH SAAG: That’s
exactly right. You never get something
for nothing with any drug. And the challenge
is that we believe these drugs, like
many drugs that suppress the immune
system, could lead to some problems,
particularly infections. So it was starting
in the early 2000s that we began to recognize
that there may be some possible adverse effects. And I use the word
possible because it’s back to the same issue of this
confounding by indication. Is it really the drug? Or is it that more severe
patients are actually receiving these therapies? So here’s an example
of one of the warnings that was listed
for the development of a particular kind
of cancer lymphoma with one of these drugs. And so my colleague, Tim
Beukelman here at UAB, looked at this within national
Medicaid administrative data, and determined the
rates of malignancy among children with juvenile
inflammatory arthritis, the most common form of
inflammatory arthritis among children. And what Tim found
is that, yes, kids that had juvenile
arthritis regrettably did have an increased risk
of developing malignancy. But it didn’t seem to be
particularly associated with this class of drugs. So we used the power of
this administrative data to help really
understand that question. So what can this
kind of data do well? Well, it can utilize large
national data sets that have fewer selection biases. It can provide
generalizable studies of effectiveness and safety. And importantly, it can
complement other approaches to adverse detection at
a relatively low cost. So if you can link
a registry or you can link a cohort to
administrative data, that can be a very
useful resource. There are some
challenges, though, with using administrative data. There’s problems around
accurate classification. There can be difficulties
defining disease activity and severity. It can be difficult to
figure out whether it’s a new case or an old case. And we often don’t have detailed
information on co-morbidities. So that can be a real challenge
for administrative data. Not everybody is a big fan of
doing observational studies. Here’s some of our
colleagues in Canada, again, suggesting that if you find a
study that was not randomized, we suggest you stop reading
it and go on to the next one. Others have been a
little bit more sanguine, saying that we need both
observational studies and randomized controlled
trials, that they answer different questions. John Ioannidis has
been one of the biggest heretics of observational
data, and provides some information– looks
very discouraging– that even well-done studies may
miss the truth in many cases. But I think in support
of observational studies, when we go back
and reanalyze some of the original
observational studies that discounted what the Women’s
Health Initiative showed, and we do that using newer
methods of epidemiology, we actually get
very similar results to the randomized
controlled trial. So well-designed
observational studies have not been shown consistently
to systematically overestimate the magnitude of effects. There’s little evidence
that the effects of observational
studies in later time were either consistently larger
or qualitatively different. And overall, I would
argue that it’s not fair to compare good
randomized studies with bad observational studies. ANN SMITH: Good point. KENNETH SAAG: We just
have to do a better job with the observational studies. ANN SMITH: Yes, so
there is a place for observational studies. KENNETH SAAG: There has to be. Because we can’t randomize
people to behavior, to other things, and where
we don’t have the numbers in the population base. [MUSIC PLAYING] So let me move on
to another area that I think is really
important in the realm of comparative
effectiveness research. And that’s this idea
of doing real world or pragmatic clinical trials. And I want to emphasize
what we mean here. And there’s really
been a lot of questions about how do we do
clinical trials better? What are the problems
with clinical trials? I’ve highlighted a few
already, but they’ve been difficult to do. They often don’t answer
the right question. They suffer from poor
generalizability. The people in the studies
aren’t really representative adequately. And they’re often
just not big enough, and representative of
people that we care about. So the movement that has
been underway for many years, but has really gained
some traction under PCORI is this idea of doing pragmatic
clinical trials that measure real world effectiveness. They are larger. They have more generalizability. They often have minimal
eligibility criteria, simple treatment approaches, and
fairly simple objective outcome measures that should
be patient centered. They tend to be
more generalizable, but they suffer a little bit in
having less internal validity in contrast to traditional
randomized controlled trials that have poorer
generalizability, but better internal validity. So it’s really a
question of efficacy, which is what a traditional
registration RCT shows you– Randomized Controlled
Trial– and effectiveness, which is what a pragmatic
trial shows you. So traditional clinical trials
suffer from a healthy person effect. They have very tightly
controlled circumstances, very strict protocols, often
very good adherence, or maybe a placebo run-in period
that avoids having challenges with adherence. Pragmatic trials may
take all of those things away, and try to
make it much simpler. Let me change gears and say that
the explanatory clinical trial ascertains whether the new
treatment actually possesses the favorable activity in man
which laboratory studies have led us to suspect. Can it work under
ideal circumstances? This is an efficacy trial. A pragmatic trial assesses
the practical value of a new treatment
in relationship to other existing treatments. Does it work under
usual circumstances? And the synonyms include
effectiveness trials, management trials, and
often, we use the term large simple trial, though not
all large simple trials are truly pragmatic. The two are just a
little bit different. ANN SMITH: And
then, the difference is looking at does it work under
usual circumstances compared to does it work under
ideal circumstances? KENNETH SAAG: Exactly, yes. And it’s a continuum. It’s not one or the other. But we can think about the
example of patient compliance. We can look at a
continuum between explanatory and pragmatic. We can exclude people who are
non-compliant before the study starts, such as with
the run-in period. That would be very explanatory. Or we might apply a
compliance improving strategy to all the patients, or
maybe just monitor compliance and intervene if it’s low. Or maybe we just monitor only. Or perhaps we just
ignore it entirely. And that would be the most
pragmatic of all study designs. So we’ve put together groups
of stakeholders and researchers in trying to
understand how do we do a better job and a cheaper
job of doing pragmatic trials? These can be very, very
expensive studies to do. So we think that by using
a new practice networks, by figuring out new ways to
do the informed consent more efficiently, and by linking
people to their data– the data that’s collected
for billing purposes– we might be able to more
effectively and more efficiently do pragmatic
clinical trials. ANN SMITH: So this
chart, then, is describing how that can
happen with all the detailed pieces of it. KENNETH SAAG: Yeah, some
of the different groups that are involved. We’re in the process. We could possibly innovate
in improving things, and just giving some ideas
about what ways we could do these
better and cheaper. ANN SMITH: Very good. And you’re finding out
some ways that work? KENNETH SAAG:
Well, we’re trying. We’ve designed a
study with support from the NIH, which we
call the EDGE study. Coming up with the acronym
is half the battle. We thought this one
was kind of cute. ANN SMITH: It is. It’s great. KENNETH SAAG: But
so far, we are still struggling to find the
money to do this big study. This is a very expensive
study answering a question about how long should
you take your osteoporosis medicine– a very important
question, since half the people now that were using
these drugs 10 years ago are taking them currently. This has been a major change
in our quality in osteoporosis. And we’re very worried that we
may see some of the reversals and some of the
benefits that have been achieved at a societal level– ANN SMITH: Because there
those side effects that are– KENNETH SAAG: Based on rare
side effects, predominantly. And that’s– well, yes. And I didn’t talk too
much about that today. But I’d be delighted if you
invite me back to talk more about some of the
work we’ve done looking at some of the rare
side effects of some of the bone medicines. But that’s exactly it. And you get side effects that
are 1 in 1,000 or 1 in 10,000, but they’re very poignant. And that heuristic tends to
affect the way people think and talk, and the way doctors
behave in using such drugs. So we’re trying to understand
exactly how long people should be on these drugs. And one of the
strategies is to partner with groups like the
HMO Research Network to try to find a lot of people. In this case we
need about 10,000, so looking across
networks for this, and coming up with websites and
other resources that we believe would be very
helpful in designing such a large pragmatic study. [MUSIC PLAYING] So let me move into
our final area, and talk a bit about
evidence implementation and dissemination. This is another
theoretical model developed by some of our
colleagues here at UAB. Now at some other
great institutions, including Vanderbilt and
University of Massachusetts, that reminds us about how these
translational blocks align. And that in implementation
science research, we’re in this later stage. We’re in this final
translational block. We’re really trying to translate
research into practice. ANN SMITH: And that includes
the policy in the overall health care systems? KENNETH SAAG:
Ultimately, it does. And that a policy, in my mind,
sort of equates with politics, and then it gets kind of out
of the hands of academicians. But ultimately, to really
impact true change, we need to affect
health care policies. You’ve got to
generate the evidence. You’ve got to show that it
works in the real world. And then, you’ve got to convince
the payers and the policymakers that this is a health care
intervention that’s worth it, and that we want to
implement more widely. So the classic framework
of health services research is to try to transition
people from state I to state II– state II
being a better health state. And we can do this by focusing
on the process of health care, by the structure where
we deliver health care, and that ultimately has
an impact on the outcome. So when we do outcomes research,
we’re often measuring process. What is it that we
should be doing? And then we may translate
back to a clinical trial and say that, well, we know
if we get more people using this medicine, that they
ought to have a better outcome, that the
evidence is there from a really well-done phase
III randomized control trial. So I call that the transitive
property of outcomes research, where if you can
improve process, and you can show that by
improving process that should have an impact on
outcomes, you really ought to be able to impact an outcome
even if your study is not designed to look
specifically at an outcome. So what are the tenets of this
area of translating research into practice outcomes research? That it’s not
geography or ethnicity that should determine which
treatment a patients receive. Variations are often associated
with differences in outcomes, and that values and
preferences need to be incorporated into
clinical decision making. This was a landmark paper
by Elizabeth McGlynn looking at the quality of health care. And the results were a little
bit astoundingly depressing– that just slightly over 50%,
but really only 55% of adults were getting the
recommended care according to various
process of care measures. So lots of room for improvement
then, and still lots of room for improvement
now 13 years later. Here’s just an example. This is the under-use care gap. And this is an example
of beta blockers, one of the very standard of
therapy approaches to managing hypertension and heart disease. And you can see that it took
many, many years from the time that we showed in the
laboratory that these drugs were efficacious till we started
testing them in humans, and showing that there
was definitive evidence, that after a heart
attack it was a good idea to take one of these,
that it reduced mortality. But it took, really, just
a remarkable amount of time before this got
translated into practice, before there was a
quality indicator. Now, the good news here is that
the quality measure developed by the National Committee
on Quality Assurance, suggesting that if you’ve
had a myocardial infarction or a heart attack, you ought
to take a beta blocker– people are doing such a good job
that we stopped measuring it. Now, 90% of people after a
heart attack get a beta blocker. So there’s no reason
to measure anymore. So this is a success story. Other examples are not so good. So after fractures, after
you break a bone, only 25% of people get treated or tested. So we’ve got a lot of upward
room for improvement there. ANN SMITH: Yes. And part of that is education to
the physicians and health care staff too? KENNETH SAAG: I think so. I think it’s both
about education, but education alone is
probably not enough. And I want to give
some examples of that, and talk about some
other strategies. Now, if we think about health
care overall and problems with health care, under-use
has been the traditional domain of quality improvement. Where are we not doing
what we ought to be doing? There’s been a greater
movement though towards this worry
about overuse. And many groups
now, for example, have developed what are
called top five lists. What are the five
things in your area that you’re doing that
really have low value? So that’s been a way for
specialists to police themselves a little bit. And then, another classic
example of medical errors is misuse, where drugs are used
inappropriately or unsafely. And that often
occurs, unfortunately, in the hospital a lot. And we know that medical errors
caused many preventable deaths. So there’s been a lot of
focus on that area too. Here’s an example of one
of the top five lists in the series of papers led by
Howard Brody in the New England Journal back a few years
ago, speaking to the need for defining what is a
good value in health care. ANN SMITH: And they’re
finding that there are some that are over-used
that aren’t providing the value? KENNETH SAAG: Very much. Very much. In our area in
rheumatology, we came up with a list that included
things like getting MRIs for routine low back pain or
using biologic drugs prior to trying other therapies that
were more effective in people that had never tried a
simple, less expensive drug. ANN SMITH: And is some
of that difficult when the patients want the MRI
or want the other options, and the physicians– KENNETH SAAG: Well,
trying to include patient preference is huge. And that’s really what evidence
based medicine includes. We have to value in judgment
and patient preference into decision making. The new wild card in all this–
not so new– but the increasing challenge is that we
have the payer telling us what we can and can’t do. So it’s often not up to the
physician or the patient, but whether they’re going to
get strep with $1,000 co-pay if they want to take
a particular therapy unless they’ve tried something
else first, for example. Now, one of the
things that we’re seeing a lot of these days
is doctor report cards, and hospital report cards,
and I show an example of hospitals being
compared, just here picking on our own University of
Alabama hospital, which is a great institution. But we saw in one example where
we weren’t performing quite as well as we would like to be. So that’s very motivating to
the hospital administrators and to other people in
trying to improve quality. Doctors now are
getting star ratings to show how they’re doing
compared with their colleagues. We’re all very
competitive, you know, so this is quite motivating
in many ways as a way to possibly improve quality. How do we define it? Well, defining
quality is never easy. There are definitions that
the Institute of Medicine has put forward. Health services for
individuals in populations that increase the likelihood
of desired health outcomes, are consistent with current
professional knowledge, and then include efficacy,
equity, efficiency, safety, timeliness, and
patient centeredness– those are the dimensions
of health care quality. So implementation research
is the scientific study of methods to promote the rapid
uptake of research findings, and to reduce
inappropriate care, and improve the health of
individuals and populations. It’s at the intersection
between research and quality improvement using methods
from health services research and qualitative methods– a type of translational science
that goes beyond the bedside. What can we do? Well, we can give people
printed materials. We can make them smarter. Doesn’t work, unfortunately. Traditional continuing
medical education– physicians and other health care
providers go hear a lecture, they get smarter, but seldom
change their practice from that alone. Intensive conferencing–
that may work around things like smoking cessation,
but it’s very resource intensive and costly. Computerized tools,
reminders, all sorts of alerts now with our EHRs– these pop up, and doctors figure
out how to extinguish them and don’t pay attention to them. So we have to make
those more effective, but they have some promise. Outreach visits and
a specialized type of outreach visit’s
called academic detailing pioneered by Steve Soumerai. We’re getting out
into the practice, and having an expert
come out and talk about what they’re doing. That may have some
merit as well. And then, audit and
feedback– we’ve had a big interest here
in UAB in audit feedback. I’ve shown the
report cards already. Does that work? ANN SMITH: Yes. KENNETH SAAG: And then,
what about a multifaceted, multimodal approach? That’s another strategy that
might help implement research into practice. So we’ve been interested
in all these approaches, and really testing to see what
works and what doesn’t work. Why don’t they work? Well, there’s lots of
reasons why they don’t work. And it’s usually
not about knowledge, but it’s often about inertia and
aligning incentives properly. Here’s an example of audit
feedback in diabetes. This was work by our
colleagues here at UAB where they used the
local quality improvement organization, the Alabama
Quality Assurance Foundation’s resources to provide feedback
to doctors in the community. They looked at different
measures of diabetes quality. If you have diabetes, were you
having your creatinine tested, your triglycerides,
your cholesterol? Did you get vaccinated? Did you have your feet examined? Your eyes? And were you having good
long-term glucose control? ANN SMITH: And what are the red,
and light blue, and royal blue? KENNETH SAAG: Red is
how you’re doing– how your practice was doing. Blue was the statewide
project average. The lighter blue was the
statewide project average. The darker blue is actually
what we call the achievable benchmark of care. This is comparing your
practice against the top 10% of your peers using
a very specialized Bayesian-adjusted quality metric
that the folks here at UAB had created. And actually, we found that this
achievable benchmark of care performed considerably
better than the traditional conventional feedback approach. And CMS actually adopted
this as the standard metric for providing
performance feedback. Well, we were interested in this
in a different disease state. We wanted to look at this back
in our arthritis patients, many of whom are on steroids,
knowing that steroids, again, are bad for your bones. And we looked within a
large health care population within the Aetna
health care population to see how were people
doing that were on steroids? Were they taking medicines
to protect their bones? And we looked at two time points
in the early ’90s and then in the early 2000s, and we saw
that, well, for women over 50, things got a little better. They still weren’t great,
but almost 2/3 of them were getting treated. But look at the younger
women and the men. Very low proportions were
being tested or treated. And that was very
concerning to us. So we designed a clinical
trial, a behavioral study, where we identified these
high risk steroid users. We figured out who
their doctors were. And then we randomized the
doctors on the internet to receive or not receive
a multimodal intervention. So we had an intervention arm,
and we had a control arm– ANN SMITH: With the doctors? KENNETH SAAG: With the
doctors– direct to the doctors. And we looked at the rate
of testing and treatment for bone health before
the intervention and then again after
the intervention. And the study was
powered to show that with 75 doctors in
each arm, we’d be adequate. It was using this achievable
benchmark of care. We had tailor learning
approaches here, interactive case
based approaches. They had adult learning theory–
suggests it should work well. They could get a tool box. They could have CME online. It was rolled out over
a six month period to combat any decay
that might occur. ANN SMITH: What
would be the decay? KENNETH SAAG: Well the
decay is just that you lose the effectiveness over time. Even if it had
worked– and here you can see the results at the very
top in a traditional intent to treat analysis,
which is the way we ought to analyze
clinical trials. There was no
significant benefit– no significant benefit. When we looked at
the doctors who did just what we asked, those
who went online and downloaded their material, and where
they read their feedback, we started to see some
trends towards some benefit. So the per protocol analysis
looked a little more promising. But it really suggested us
that in certain disease states, it may be different,
and that there may need to be a different
teachable moment. And maybe it’s not
the physicians, but it ought to be the patients
or the health care system that we ought to
be targeting better in an area like osteoporosis. So one of our approaches has
involved now in our evidence implementation science the idea
of narrative communication. Can we engage and
activate the patient? Can we use things
like homophily, where people can relate to
people that tell stories that sound and relate to them? Can we use that as
a way to motivate them to take action and interact
effectively with their health care provider? It’s been said that the power
of narratives to change belief has never been doubted and
has always been feared. And here’s an example, again,
in a different health area, but in hypertension,
where you can see the benefits
of, in this case, just showing a very grainy DVD
developed in sort of a home movie style, sitting men–
mostly African-American men– down in the waiting room
at the community hospital, at the county hospital,
showing them the stories about their peers
who had experienced strokes and other bad outcomes
from high blood pressure. And it resulted in
dramatic improvement in blood pressure– a very
significant reduction. More so than you would see
in a typical clinical trial, and leading to a paper in the
Annals of Internal Medicine. We’ve looked at
other approaches. We’ve looked at a very low
tech system approaches, where we just give women the ability
to schedule their own bone density tests. Here, they could go online. They could schedule
a bone density test. And within two of the
Kaiser health plans– Kaiser Northwest in Portland
and Kaiser in Georgia– we were able to show
a significant increase in the rate of
osteoporosis testing that should be appropriate
for women over a certain age. ANN SMITH: By allowing
them to self-refer. KENNETH SAAG: By allowing them
to self-refer– very low tech. Similar to a mammogram or
a vaccination, but just use a 1-800 number. And so simple things
may work, but you’ve got to test them
to find that out. So evidence
implementation research– the first step is
defining quality. We have increasing
things we can use. It’s a team sport. We have to have behavioral
scientists, epidemiologists, clinicians, and
it’s not easy to do. Technology offers us promises. Theory based creativity is key. As my colleague [? Jeron ?]
Ellison often says, one size fits none, and
the approaches really must be tested. I love this study
by Sacks showing that in the traditional
quality improvement arena where you do uncontrolled
studies, they look great– 80% work. But when you actually
use a control group, as we saw in our example with
the doctors in the Aetna health plan, often you get
a negative result. So you’ve got to test things
to know whether they’re really working. So I want to finish
up by just mentioning that for people that are
interested in outcomes and effectiveness
research, there really are great resources out
there, both at our institution and at many institutions
throughout the country. And we really hope we
can get more people interested in this
very exciting area where there’s now more federal
investment, great opportunities with PCORI, with AHRQ, with
the NIH, the CTSA network. All of these are
tremendous places to get trained to do outcomes
and effectiveness research. ANN SMITH: Good. And definitely an
area where researchers need to go into more, and
do more of the evidence based findings. KENNETH SAAG: Right, we can
come up with the best therapies, but if we can’t get
them into practice, it doesn’t do a bit of good. ANN SMITH: Well, very good. KENNETH SAAG: We need
people to help us with that. ANN SMITH: Yes. Thank you so much, Dr. Saag. KENNETH SAAG: Thank
you very much. ANN SMITH: And if anyone has
any questions or comments about this, feel free to
email me at the email address and on the screen, and I can
forward them on to Dr. Saag. Thanks very much. [MUSIC PLAYING] Thank you for watching
this health research talk. We encourage you to email us
with any questions or comments. Also, please take a moment
and complete the brief survey on the link on your screen. Let us know what you
thought about it, and how we can improve
it for the future. Lastly, register
on the NRMN website to sign up for the
online mentoring, and be a part of NRMN, the
National Research Mentoring Network. Thank you. [MUSIC PLAYING]

Leave a Reply

Your email address will not be published. Required fields are marked *