Health News Review: Critically Analyzing Information in the News

Health News Review: Critically Analyzing Information in the News


(light music) – [Elizabeth] Again, welcome, everyone, to our Kernel of Knowledge webinar series. Today, we are gonna be
featuring HealthNewsReview. I’m more fortunate to have
Gary Schwitzer with us. I just want to introduce myself first. My name is Elizabeth or Liz Kiscaden. I am connecting with you
from the National Network of Libraries of Medicine’s
Greater Midwest Region office. That is located in beautiful Iowa city at the University of Iowa. If you’re ever in the area, please stop and visit us. We are located at Hardin
Library of the Health Sciences. For those of you unfamiliar with the National Network
of Library of Medicine, I just want to share that it’s a program of the National Library of Medicine, and it’s been around for over 50 years. Our organization does education
outreach on behalf of NLM. Education includes our Moodle courses on a variety of National
Library of Medicine resources. Some of your favorites
include PubMed, HealthReach, MedlinePlus, ClinicalTrials.gov,
Genetics Home Reference, and many, many more. We also offer webinar series like this to bring in expert speakers. We bring in expert speakers
on health information topics, which is why we’re featuring
HealthNewsReview today. I want to provide an introduction to our speaker today, Gary Schwitzer. Gary is the founder and publisher
of HealthNewsReview.org, which is one of my favorite resources. Gary has specialized in
healthcare journalism for more than 45 years. He is leading a team of
people, about 50 folks who grade health news reporting
by major news organizations as well as healthcare PR news releases within HealthNewsReview. This project actually
won a Knight-Batten Award for innovations in journalism. And in fact, Gary also
won the Mirror Award, which is a journalism industry award honoring those who hold a
mirror to their own industry for the public’s benefit. For about a decade, Gary
taught health journalism and media ethics at the
University of Minnesota. He’s now an adjunct associate professor in the university’s
School of Public Health. Gary worked in television
medical news for 15 years including being in charge of
the CNN medical news unit. Outside of his blog posts
on HealthNewsReview, which by the way was voted
best medical blog in 2009 in a competition hosted by Medgadget.com. Gary has published articles
on health journalism in numerous publications
including JAMA, BMJ, PLoS Medicine, and
Columbia Journalism Review. In 2009, his report on the
state of US health journalism was published by the
Kaiser Family Foundation. And in 2010, he authored
Covering Medical Research: A Guide For Reporting on Studies for members of the Association
of Health Care Journalists. This report is also
available on HealthNewsReview as a toolkit. Gary has been a keynote speaker at international conferences. He’s led health journals and workshops at numerous AHJC conferences, at the NIH Medicine in Media series, at the MIT Medical Evidence boot camps, and internationally for NIH, I’m sorry, National Cancer Institute, other acronym. In 2014, Gary was named
of the 25 Champions of Shared Decision Making by the Informed Medical
Decisions Foundation. Also in 2014, the American
Medical Writers Association honored Gary with the McGovern Award for preeminent contributions
to medical communication. I am so pleased to have Gary with us. As I said, I’m a big
fan of HealthNewsReview. I think it’s a one-of-a-kind resource. Beyond just the critical
analysis of health information in the media, HealthNewsReview
has introduced me to a community of individuals
challenging health journalism, looking for a more accurate portrayal of health information in the news. Now, folks, I’m just gonna stop
here and check our chat box. I’ve got a chat coming
in from Dana that says, “The sound is going in
and out a little bit.” I’m sorry about that, Dana. I’m just gonna get a little bit closer to the microphone here and
see if I can get that result. All right, folks, for those
of you who have never attended one of our Kernel of Knowledge
webinar series before, I just want to share
that this is informal, and we really do want to
have some communication with our attendees on the line. So, please continue to put
questions into the chat box as we go through. I will be keeping an eye on the chat box while Gary is presenting,
and we’ll occasionally stop, and we’ll answer your questions. So, please keep it coming. We’d like to have a little
bit engagement at these. All right. I’m gonna go ahead and stop sharing, and we’re gonna pass it over to Gary. And Gary will share his
slides from his end. And for those of you on the line as well, remember you can turn on or
off your webcam as you’d like. Looks good, Gary. – [Gary] Good. Take it away, huh? – [Elizabeth] Take it away. (laughs) – [Gary] Boy, as I listened
to your introduction to me, it really makes me sound
old, but it should. I just turned 67. And one key piece of information
that Liz didn’t include was that we’re shutting down,
oh, within about two weeks after about 13 years of operation when we consider the one-year
beta development stage. We’ve lost funding. I looked really hard in every appropriate and relevant corner over
the past year, year and 1/2. And it just look like we
had run out our strength. However, everything that
you’re about to hear about and the resources that I will link to at the end of the presentation
will still be alive. The site will not go dark. I think I have put away
enough money to keep the site open and accessible for
the next three years. So, this discussion is not a moot point. Okay. This has increasingly become the theme that I approach as I
talk with any audience, not just journalists,
but it is the true harms that are caused on real people from not not-ready-for-prime-time
healthcare news and public relations. And I think that we can do a better job and avoid a lot of this harm. So, to begin with, one little
example of the kind of thing we’ve seen every day for 13 years. We just don’t deal very well
with nuance or uncertainty, which of course is rampant
in medical science. It’s why science stays
in business to embrace and attack uncertainty. But here we had on one given day one of the leading
newspapers in the country reporting that fish oil
drugs are protective. Well, perhaps their leading
competitor at the Times said they are ineffective. Well, now think about this, maybe thankfully not many
people in this country are reading more than one
newspaper on any given day. You medical librarians probably are. But of the great unwashed
in the population, not so. But for anyone who is, it’s a coin toss about what you’re gonna get
and from whom on any given day. When you get to the bottom of this, it is really the failure to effectively and adequately evaluate the evidence and explain to the public. I have been at this a long time. Shortly after I resigned from CNN, the editor of JAMA at the
time, George Lundberg, heard a talk that I gave. I think he thought I was a physician. He asked me to submit it for publication. I told him I wasn’t. He said he didn’t care. He wanted it anyway. And so, this was my first medical
journal article back then. And this was the last
paragraph of that article, “That the failure to
take time in reporting “helps explain why we get what we get, “a lot of quick fixes, magic
bullets, daily breakthroughs.” And you’ll see right here, way back then. What is this? 26 years ago. I was talking about this theme of harm that we feed hysteria and hypochondria, thereby harming and not helping people. Then eight years after
that, I was deeply inspired by what I thought was
a groundbreaking piece in the New England Journal by the guy who’s gone on to become a
friend of mine, Ray Moynihan. He was then one of the
leading healthcare journalist in the country or in the
world from Australia. He now has earned his
PhD at Bond University and is more of a researcher
than he is a journalist. But in this really important paper, which you might say in some
way laid the groundwork for our HealthNewsReview project, it looked at the coverage of three widely prescribed medications, found that 40% of those stories
didn’t quantity benefit. So, they gave no sense of the
scope of potential benefits. They did even worse in
mentioning potential harms. They did even worse in
failing to mention costs. 1/2 of them quoted at
least one expert or a study with a conflict of interest
with a drug company. And only 39% disclosed
financial conflicts of interest for these sources. When I get to our own data
slide a little bit later, it is striking how this
much smaller sample from this paper 18 years ago parallels our 13-year experience up to today, almost exactly mirrors it. So, five years after that paper, and I hope many of you are
familiar with the John Ioannidis who’s now at Stanford. This was the kind of paper that you think would make a
person a pariah in research by writing that most published
research findings are false, but indeed anyone who’s going
to be open-minded and honest would admit that Ioannidis was right. And rather than becoming a
pariah, he’s developed a terrific and a well-deserved
international following. If you don’t know this paper, it must be on your must-read list. Then in 2005 is when I began planning for and then indeed in 2006
launched this project, which was the first and is
still the only such project in this country to systematically
review healthcare news, and more recently, we began review healthcare
public relations news releases. This was the homepage of a
site that has been changed many times in those 12 years. As you can see, the date
of that Baltimore Sun story was April 18th, 2006, our first date. Then in 2008 was the first time
that we published our data. This was after three
years, two years really of publishing in PLoS Medicine. And the editors of PLoS
Medicine wrote this editorial in which they said, “This
is a troubling report card “not just for journalists
but for all of us involved “in disseminating research, researchers, “their academic institutions,
journal editors, reporters, “their organizations
to work collaboratively “to improve the standards
of health reporting.” 10 years later, I’m not sure we can say that that collaboration has
really achieved very much or that much has been done
to collaborate frankly. Just last year, the BMJ wanted me to, this was an invited editorial
to write about fake news. And I said, “I don’t want
to write about fake news.” I don’t want to promote this term that our president has latched onto. And in fact, I don’t think
that truly fake news, which as you see in this excerpt to me denotes deliberate deceit
is the main problem of what we see when we
look at this every day. Its far bigger problem is sloppiness from a much broader range of
hurried, biased, imbalanced, incomplete stories that don’t
offer context or analysis. In a nutshell, we try to help people avoid the avoidable harm that comes their way from misleading messages
from a variety of sources. So, it’s avoiding avoidable harm. It’s avoiding avoidable ignorance. So, just some numbers real quick. As I said, we’re nearly 13 years old. Whoops. There’s two years there. Sorry about that. Everybody needs an editor. We’ve systematically
reviewed with criteria that you’re about to see
more than 2,600 news stories. We’ve systematically reviewed more than 600 publication
relations news releases. Our blogs, about 2,800 of them allow us to step outside the
somewhat rigid boundaries of the systematic review
and get into things that also cause a lot of havoc
like healthcare advertising and marketing and talk
shows and other things. Any topic that might
pollute the public dialogue is fair game in that blog. And in the nature of most blogs, it’s edgier, more opinionated. We have a lot of fun, and I think due a lot of our most important
work on that blog. We’ve done 50 podcasts. My favorite of a variety of
people we’ve interviewed, my favorite are the patient interviews with patients talking
about how they perceive they have been harmed by
misleading media messages. And then sort of a hidden gem on the site, and these are some of the
links I’ll take you to later in the talk, what I think is one of the more extensive
toolkits of tips and resources for both consumers and for journalists. Not perfect by far. Some of it might seem a little
elementary to you folks even. But when you consider the target of a broad general audience, this may be one of the
best collections out there, and we’ll look at that a
little more closely later. So, all we review, and
I put that all in quotes because this is a lot, are stories that include
claims about interventions. And currently although the
scope of whom we review has changed often in the
course of our 13 years, this is whom we review every day now and basically for the
last three or four years. So, as you can see, it’s a
mix of traditional newspapers, then the websites of the five
leading television networks, a couple of wire services, and then some important online
healthcare news services. Four years ago, we began
reviewing public relations news releases really from
everyone under the sun, government agencies,
universities, journals, nonprofits industry, and
professional medical organizations. So, don’t let your eyes
roll back in your head. This is the one data slide. But this is pretty important. So, we have 10 criteria. But I don’t think those
10 are equally weighted. I’m convinced that these
five that you see on the left are more important than the other five, and perhaps not surprisingly, they are the five poorest performers, the ones that gave journalists
and PR professionals the hardest time. So, let’s look down
that middle blue column. So, we’re looking at the average score or the percent rather
or unsatisfactory scores for 2,600-some news stories. How do they do it addressing cost? Now remember that the
eligibility criterion is, they’re making a claim
about an intervention. All right. So, you’re gonna make claims about it? What about the cost? We hardly ever get there. And did you assess the size
of the potential benefit? How big or often how small
is the potential benefits? 2/3s of the time, we never
get there adequately. About harms, ditto. About quality of evidence, ditto. How about did in talking about the new, which we understand you think is your job, did you put it into the context
of existing alternatives? And more than 1/2 the
time, we don’t get there. Then look at the yellow
column on the right. Public relations news
releases, forget about it. In each of these instances,
the grade is even worse, and it’s the public relations news release where the faucet this tsunami
of crap, if I may say so, where that faucet often gets turned on, and in an era when many news organization say they’re financially strapped, fewer people are being
asked to do more with less. There’s cutbacks in staff. There’s cutbacks in training. Story quote is arising. Journalists will say
that they often fall prey to the news releases. So, it is indeed a polluted stream just as the editors of PLoS Magazine wrote when they reviewed and wrote an editorial in response to our work back in 2008. Some themes behind those numbers, day in, day out, over and over again, media messages convey a certainty
that simply doesn’t exist. How do they do that? Well, they frame things statistically in the most positive light. One of the key ways is to use relative, not absolute risk data. And if anyone on the webinar
doesn’t know what that means, enter a chat comment. We’ll get to that later. They don’t explain the limitations of surrogate endpoints or markers. So, probably the granddaddy
and the grandmamma of this example is LDL cholesterol, which is important for us to know about, but when a story fixates on
a study that only reports an impact on a number
like LDL cholesterol, is that what we really want
the audience to focus on? Because that’s a surrogate endpoint for what we wish we really knew, which is did people on this drug, on this intervention
live longer and better? Did they have fewer heart attacks or less disability from heart disease? That’s what we need to
get people to think about, not just the surrogates. – [Elizabeth] Oh, and
Gary, I just want to pop in and let you know that we
did a have couple folks on the chat box that were interested in understanding relative,
not absolute risk data. – [Gary] Maybe the best example, and this is a poignant
source for me right now because it comes from my
friend Steven Woloshin and Lisa Schwartz had dealt with. Lisa just died last week. A shock to me. I’ve known her for more than 20 years. I worked at Dartmouth. It’s right down the hall
from them in the ’90s. They have been terrific in
getting the FDA, researchers, journalists to think about
how they present data. And so, on this point of
absolute versus relative risk, I always use their example. They say it’s kinda
having a 50% off coupon at a major department store. But you don’t know what that
50% off can be applied to. Can it be applied to a diamond necklace you’re gonna give your wife of Christmas? Or is it only good for
a pack of chewing gun? The 50% off is the relative number. And whenever you hear in a study an effect size of I’ll
just say 20, 30, 40, 50%, ask yourself two words: of what? The classic case in my
mind in the real world is when Merck, the drug
company marketed osteoporosis, their osteoporosis drug Fosamax. They marketed it saying they round number. It reduced the risk of
hip fracture with 50%. Okay. Everybody on the webinar, you
ask yourself, 50% of what? And the of what was this, in the best of Merck’s data
from their clinical trials, it reduced the risk of hip
fracture from two in 100 in the untreated group to one in 100 in the treated group. That is indeed two down to one is a 50% relative risk production, which you can do the math, the absolute risk
reduction was one in 100. So, if you work for a drug company, you tend to promote the relative numbers. If you want to help the
public understand true impact, you want to use the absolute numbers. And of course another statistical tool is the number needed to treat. How many people would you need to treat in order for one to benefit, and in this case, that would 100. You’d have to give 100 women this drug in order for one to benefit. And yet all the other 99 would
have to pay for the drug, run the not inconsiderable
risk of side effects and stand no chance of benefit. And you don’t know which one, whether you’re gonna be the one or more likely one of the 99. And we don’t tell stories that way. And I think this could be
webinar of seventh graders, and they would understand this. So, back to the slide. We convey certainty about
observational studies where we use causal language. Coffee either cures you or kills you. It’s always an observational study, which in and of itself cannot
prove cause and effect. It can show a statistical association but cannot prove cause and effect. So, we’re simply wrong
to use such language. The tyranny of the anecdote is
where we often end the story with the rosiest, the most
positive patient anecdote who was so satisfied
with his or her outcome without any contemplation of
whether that person’s story is representative of what the
body of evidence would show. And similarly, single-source stories, so here even single source experts when we don’t even look at
the conflict of interest in those sources. And I’m not painting a
picture of boogeyman here, but if journalists are not aware that there are conflicts of
interest around the corner all over in healthcare, then they shouldn’t be
covering healthcare news because they’re simply naive and gullible. – [Elizabeth] And hey, Gary, I was gonna ask you one of
my own questions as well. What kind of training do journalists get about reporting science? Are there special classes that folks take? – [Gary] Well, it’s a lot better now than when I started 45 years ago. I was self-trained. Fortunately, I just stumbled into a lot of the right pathways. Thankfully today, there are far more quality training opportunities. Now I’m talking squarely out
of both sides of my mouth. Sadly, one of them was run by
that couple I just mentioned, Woloshin and Schwartz, who,
at Dartmouth and at NIH, headed the very high quality NIH medicine in the media series for about 10 years. And it has not been offered
now for several years, and I’m not sure that it will be again. Still though, Liz, too
many people get thrown into this beat like I did. And it’s a luck of a draw
whether they pursue it, without breaking both arms
patting myself on the back here, as seriously as I did or not. And if they get swept up by a
lot of what their management is pushing them to do, and that’s often crank
it out and feed the beast of daily journalism,
they’re never gonna get to the training
opportunities that they make. – [Elizabeth] Oh, that’s interesting. – [Gary] Yep. So, George Orwell famously
wrote that, “Journalism “is printing what somebody
else doesn’t want printed. “Everything else is public relations.” Well, in our work, we find that journalism is looking more and more
public relations every day. And this term infoxication, which doesn’t necessarily have
anything to do with Fox News but often does, is sort of a mashup of those two term,
information intoxication. This polluted stream of
healthcare news and information that drowns the public. And let me flesh out this image. So, in that top ladder, you
got all the people in the suit. It’s up at the top. Government, universities,
medical center, the industries, physician groups, medical
journals who dump their often conflicted messages, often not, but on their public relations people who are underpaid and perhaps undervalued when indeed they have to
stand up to department heads and suggest to them that
something needs to be communicated in a different way. So, the PR people add to the spin that has already been
begun at the top ladder, they add more spin often
to keep their jobs, dumping it on the journalists who add even more spin sometimes because of the
environment in which they work, which Liz just asked me about, and they dump their spin on us who don’t even know that we’re being spun. We don’t know how the
sausage is being made. And it’s really important to reflect on how many different
people pollute the stream at different stages. Back on my harms theme for a minute, and this is sort of a cross-media look. So, we tend to exaggerate
benefits and minimize harms. We have the data to show that. So, let’s just look at
the literature shows, and this was a really
interesting paper recently. This was an oncologist in the BMJ who thought that what he
was seeing in his patients differed from what he was
reading in the seminal paper that perhaps pushed a drug
over the top into approval. Okay? So, he compared. This is what they said
in the journal article that delivered the best evidence, but this is what I’m
seeing in real practice. And when you went back and looked at what those journal
articles said about the harms that he was seeing in public,
43% of those journal articles used vague unhelpful euphemisms
to again sort of downplay the severity of the side effects and the adverse events that were happening in real clinical practice. Now let me again roll out
the data of what we’ve seek. So, if that’s what happens
at clinical practice, now here is what people
are getting at home when they read the news. 63% of all these stories, which
I think are representative, are unsatisfactory on harms. And how are the news release is doing where the faucet often gets turned on? Even worse. This is pollution at various stages of the way to the patient, the consumer. I love this quote, Josh Billings, the humorist Henry Wheeler Shaw. “The trouble with people is
not that they don’t know, “but that they know so
much that just ain’t so. “I honestly believe it’s
better to know nothing “than to know what ain’t so.” Well, as a career-long journalist, I’m not gonna sit here and say, “I don’t think that
journalism is important.” But I do think, as
Billings leads me to think, that it would be better to report nothing than to report what simply ain’t so. I love this quote from a
really important article by Joel Achenbach of the Washington Post in an article called The Shroud of Turin, Pseudoscience, and Journalism. “Good journalism has a
subtle feature of reticence. “We don’t or shouldn’t
publish everything we hear. “We filter, we curate. “This requires the willingness
to not publish things “unlikely to be true. “There’s nothing at stake here “except of course the survival
of credible journalism.” I think this is really important, which is why I use this quote even though I will acknowledge, if any of you know
Achenbach’s recent history, he was suspended on I believe
at the beginning of this year for inappropriate office conduct, which I am extremely sensitive to, but I don’t think that has anything to do with the validity of what
he said in this quote. And that’s why I add that asterisk, but this is an important
point that he raised, and he stated it as well
as anyone I’ve seen. (clears throat) My friend Doctor Victor Montori at mail once wrote to a journal club. “Beware spin: (clears throat) “composite endpoints.” If you don’t know what these terms are, I’ll show you where they
are explained on our site, in our toolkit section later. Look out for these little tricks of, “Composite endpoints, surrogate markers, “subgroup analyses,
inadequate comparators, “which is simply too much or too little “of an effective alternative or placebo “when an effective alternative exists.” It’s ways to play games with your study to make the outcome look
better than it really is. And he said, “Above all, “avoid the introduction
discussion sections, (clears throat) “which is where most “of the interpretational
spin is introduced.” (clears throat) What he’s saying is, you
have to become better at reading the methodology
sections of the paper, not just the beginning and the end. (clears throat) Pardon. I got a frog in my throat here. In his book Why We Revolt:
A Patient Revolution for Careful and Kind Care, Montori. So, picture this, here are a doc, a conservative old modern male, and he’s calling for a patient revolution. And here he writes that,
“Publication bias, reporting bias, “and spin are not just academic issues. “They distort the evidence, “and that impacts the
care that real people get. “We need trustworthy research “for better health decisions.” Again, here is an insider
showing us the polluted stream at many different stages. When the March for Science was
held a couple of falls ago, I love this one placard in the streets. “Dear Media, a pilot study
is not a breakthrough. “Please learn how to report science.” Maybe we need a revolution
like Montori said to hit the streets and tell journalists and other media messengers
and the people who feed those media messengers the harm
that they indeed are causing with incomplete, biased, shoddy work. Now here are some of those
primers that I told you about. So, I’m gonna leave the
PowerPoint for a minute. I hope this works. And I’m just gonna take you
to the portion of our site where these reside. – [Elizabeth] And I’ll
let you know if it works. It looks like we can see the browser. – [Gary] Yep. – [Elizabeth] There it is. That looks great. – [Gary] Only make that window bigger. And so, when you come to
the homepage of our site, what you’re seeing up here, this horizontal navigational
bar is on every single page. To get to the toolkit, you
go to the toolkit here. I clicked on toolkit to get here. This is where a lot of
these complex topics that we just talked about. So, I talked about absolute
versus relative risk. I talked about composite
endpoints in Montori’s quote, intention to treat analyses
should not intimidate us. I talked about the number needed to treat in that osteoporosis example. Observational studies, I’ve talked about, surrogates, markers,
and subgroup analyses. All things that I’ve talked about, I won’t explain them anymore here. These are written, my
hope, for lay audience. And again, I think I’m
pretty proud of the work that was done especially
over the last four years to update and revise and improve those. Elsewhere in this toolkit
section is this area. This was sort of our
impromptu focus last year. We have dozens of examples
of conflicts of interest in healthcare, in academia,
in PR, and in journalism. This was sort of our
impromptu focus of 2018. I’m very proud of this. We’ve told about 20 stories
of people who told us how they were harmed by misleading media
messages about healthcare. Many cancer examples,
stem cells, migraines, medical devices, experimental
therapies, NMS patient, unnecessary medical testing, and so on. – [Elizabeth] And Gary,
how did you find folks for those stories? I had seen that on the site earlier, and I was curious how people– – [Gary] We really beat the drums. We went on social media sites. I tapped almost all of my
regular sources asking for help. And then it became a
boulder rolling downhill. The more we started to post,
the more the stories came in. And it is a sign to me of a mostly quiet but burgeoning patient
revolution in my mind of people who have said, I
don’t know how many of you remember the movie Network
where the crazed anchorman Howard Beale said he was mad as hell and not gonna take it anymore. I think we got a lot of people in America, a lot of patients and healthcare consumers who are mad as hell and
not gonna take anymore. And you got to look for them and you got to give them voice and you got to give them a platform if their stories are to
be heard and appreciated and acted upon. Now let me go back for a
moment to the PowerPoint. As word got out, and it has
been getting out for months now that we were gonna close
at the end of the year, a lot of people came to us ’cause we have a tremendous
patient following. We will end this year with
about 800,000 active users this year, which, to me, is incredible. – [Elizabeth] Wow. Yeah, that’s huge. – [Gary] Yeah. Nothing that the New York Times
and the big boys and girls but to a little not-for-profit
flyover country here in the twin cities area, this is huge. So, I wanted to respond
to those people saying, well, where are we gonna go for help when you go out of business? This is far from a perfect list. But you librarians I
thought would appreciate it. So, it’s on our blog. We find your blog right here
on that upper navigational, and again, it’s close
to the top this week. It’s right here. And I’ll go back into that. And in it, I list five categories. I didn’t want this to be a top 10 ’cause that just sounds too all-inclusive, and it’s not that. There’s a lot more I
could have put in here. Rather, I created categories
that I thought were important. So, here’s three leading
daily news sources. Here are, one, two, three,
four, five leading podcasts, blogs, YouTube, and one Netflix example. Here’s two great examples if
you want to sharpen your focus on conflicts of interest in healthcare. And then here are two, this bottom one, a former academic colleague of mine at the University of Minnesota,
Combating Misinformation. But then I’m reminding people
that don’t forget about us that these toolkit resources
that I just showed you will still be there for as
long as I keep the site alive, and that’s, I hope, gonna
be three years or so. Back to the PowerPoint. One other thing, a lot of people ask me. How does news organization
A stock up over time against news organization B? We’ve kinda hidden this on
our site, which I’m sorry for. This is how you find it. This is the news story index page, okay? Or you could go into any
individual news story review, and you’ll see this pie chart. And if you click on that
pie chart, you get this page that gives you this static pie
chart, which is our history. All 2,000 or so reviews that we’ve done and how a story did on those 10 criteria, whether it got satisfactory
or unsatisfactory. For the sake of ease of display, it’s translated into a star
score of zero to five stars. Zero, terrible. Five, terrific. Okay? So, the available star score across that whole history is 3.1. Pretty average. Right in the middle. Then you can go to a pull-down menu and pick any one of these news sources that we’ve reviewed over
time and see how they did. So, the first one that
comes up automatically alphabetically is abcnews.com,
their online effort. And they’re below the national average. So, you may want to do
that on your own time. I summarized it for you however in this next very busy slide. So, again, the national average is 3.1. Here are three of the best. The website Vox, the website STAT, which is a Boston Globe company product, and the Philadelphia Inquirer. And then what used to be,
remember we couldn’t wait for TIME, Newsweek, and
US News and World Report to show up in our mailbox every week? Well, don’t hold your breath anymore for their online products
because they’re really bad, and they’re not getting any better. And the TV networks, we used to review their on-the-air product but gave up on it because they were so poor. Their online news operations
aren’t much better. Only CNN is above the national average. The other three are below it with Fox News being the lowest out of any
single news organization we have reviewed with at least 10 reviews with almost a rock bottom
two stars out of five. So, I want to leave a lot of time for your comments and questions. My funding for the last four years has been from the Laura
and John Arnold Foundation. Extremely generous funding. I have no hard feelings over them ending their funding of us. I think they have redirected
their priorities towards, one direction is gun violence prevention. And you know what? If you want to take the
money you’re given us and devote it to gun violence
prevention, I say God love you because that’s an important
cause for all of us. Maybe someday somebody
will pick up the baton. I can tell you that as
recently as yesterday, somebody approached me with an idea about someday, not right away
keeping this thing alive. I’ll believe it when I see it. I’m 67. I need some time off. I will say we’re keeping the site open. I will occasionally, when I
roll out of bed in the morning, be so moved that I want to write something or I want to hire somebody
to write something, but it won’t be nearly
at the volume of work that you’ve just seen
explained on this webinar. So, I really welcome your
thoughts and questions and suggestions, and
pushback is welcome as well. – [Elizabeth] And Gary, I
just want to put in a plug for the Facebook page that
HealthNewsReview has had. I have followed that on
Facebook for a while. And a lot of great items
have come up in there. It’s kinda just fed directly to me as well as the Twitter feed. Although I don’t tweet
much, I like to think that at some point, I’m
gonna be like the cool kids and be out there tweeting. – [Gary] I’ll take you
to that Facebook page. First, you can go to mine. It always goes to mine first. But my personal page– – [Elizabeth] And for those
of you with questions, go ahead and put ’em in the chat box, and we will address them while we’ve got Gary on the line here. This is nice because a
lot of those story reviews come up right there in
your Facebook newsfeed. – [Gary] We should be
posting all of them here at one point or another. Yep. – [Elizabeth] And Gary,
would you mind showing us what a story review looks like just for those that haven’t been out? – [Gary] Sure. So, I’ll just click on this one. This will bring us back to the site. Yeah, that’s a really good point, Liz, ’cause for one thing, you can’t
judge a book by its cover. You can’t judge one of our
reviews by the star score. Frankly I’m gonna just
around here a little bit. Just last week, we gave,
or this week maybe, yeah, we gave HealthDay a five star score. And to me, it’s not reflective of one of the biggest take-home points, which we were careful to explain. I think it’s in here somewhere. Yeah. – [Elizabeth] Yeah. I’ll say I saw some of the
HealthDay content in the past with very low scores. – [Gary] Yeah, they do. And in this case, I can’t
remember what our point was, and I’m not gonna waste
your time on this call, but there was something
significant that was missed. As you can see, I was not
one of the three reviewers on this site, so I’m not
that intimately familiar. My recollection at age 67 is slipping. But I’ll go down this one
anyway since we’re in here. This is our template. We try to give you the gist of it up here in the review summary. We then try to explain
to you why this matters. And this can go in one of two directions. It’s either why does this
healthcare topic matter or it could be why does
journalism about this or why does how we communicate
this to the public matter. And then criterion by
criterion by criterion, you not only get this grade, was it satisfactory or unsatisfactory. And by the way if you clicked on any one of these criteria links, you’re taken to a much broader explanation of why we thought that
criterion was important, often with a video explainer
like this one is with me here. And then we give satisfactory and unsatisfactory factory examples. So, we not only do that, but then we write why we thought it was satisfactory or unsatisfactory. And as you can see, if you add up the word count
on our review, it almost always is longer than the word
count of the original story. I’ve had journalists say, “Come on. “Your review was much
longer than what we wrote,” to which I always say, “Well,
who owns that problem?” Our few editors are about
increasing your word count. What I didn’t show you
was, at the very top, you can go back and
read that original story if their link is still alive, and then you can compare
it with our review. This, Liz, and all of you on the webinar, this is where the teaching takes place. This is where we explain that. Let me find one that has
some negative examples. – [Elizabeth] And while you’re doing that, I’m just gonna share a couple
comments from our chat box. From Rex Rocks Library Services said, “This is a wonderful organization.” They’ll be telling others about it. Caroline shared that the
website’s been invaluable, and she highlights it in her work and is glad to hear that the website is gonna continue to be available. Dida did ask, “Do you
know if the Facebook page “is gonna continue to be updated?” – [Gary] That will fall on
me going into semi-retirement driving around the country
in an RV with my wife starting in January. – [Elizabeth] Yeah, you’ll have good WiFi. – [Gary] It would kinda be
crazy not to ’cause it’s there. It’s already a sunk investment. Just be patient with
me if it’s not updated every single day, okay? Dana or Dino, whoever asked that. And to Red Rocks Library and Caroline, I can’t tell you how often, thank you for your kind comments. Medical librarians love this service. I know they do. Or else it’s a silent majority
that I don’t hear from. And that’s very gratifying
because I know of the importance and the impact of the work that you do. So, here’s a Reuters example
that only got two stars, which meant, I’m gonna spin to the bottom, that only three out of 10 criteria were judged satisfactory. So, where did they go wrong and how do we explain it? So, this was about a drug for psoriasis. Did they adequately explain the cost? No, a cost comparison of
the drugs in the study, which the story talked
about, wasn’t provided. And we’re talking about,
as we reveal here, a list price of, and this
is $10,000 for the one, 4,700 for another for X quantity. What else did they get
wrong or incomplete? The criterion on harms, unsatisfactory. It’s known to weaken the immune system. None of this was mentioned in the review. We just don’t understand how
you can write about this stuff and not at least hit on
these first five big points: costs, how big or small are the benefits, how big or small are the harms, how good is the quality of the evidence. As we say here, they didn’t even explain whether it was a randomized trial. What were the limitations? Peer-reviewed, published? Give me a break. It’s just crazy that people are being paid for this kind of work. And yet, we don’t use
that tone in our reviews. We better be. And it’s been my reminder
over 13 years for the staff. If we can’t be constructive
in our criticism, we probably shouldn’t
critique anyone at all ’cause we’ve got to show a better way. We’ve got to help people do better. So, I’m glad you asked me
to drill down into these. I can’t have evidence of
how many people do that or don’t it, Liz, but if you
don’t drill down into these, I mean stop and think, we have,
when I go up here to this, we have done 2,613 story reviews. That is 2,613 well-thought-out,
carefully prepared lessons for any of us about how
to evaluate evidence. So, it’s a missed opportunity. And to you librarians, I invite
you to grab as many of these off the site as you can before
we do finally shut down. It’d be nice if you gave
attribution and credit back at HealthNewsReview. But that’s why we’re up
here was to help people. So, between the individual
reviews and our toolkit, thousands and thousands
of lessons and resources. – [Elizabeth] And I’ll share to ya, Gary, I use your content all the time. We actually have a course in NNLM called Health Issues in the Headlines. And for those of you on the
line that are interested in learning more, that
course is being migrated to our new learning management system, our new version of Moodle, and it will be available in the spring. I’ve got another question
for you too on the line. So, this is from Norris. And Norris states, “We
librarians currently teach “a formal course session on this topic “to our medical students. “Of course, we see the value. “But we may have to
fight the powers that be “to retain this course in the future. “We fear that it’s seen
by some as a fluffy topic “for medical students
given everything else “that’s needed in the curriculum. “Do you have any advice
as to how we can advocate “for the course in an
elevator type speech?” – [Gary] Oh wow. (clears throat) What a great question. (clears throat) (clears throat) This wasn’t my idea. (clears throat) I was asked by the ABIM Foundation. ABIM stands for the American
Board of Internal Medicine. They’re the people who started
the Choosing Wisely campaign. To co-author a paper
because we’re gonna try to get published a series of papers on issues of trust in healthcare. So, for your elevator talk,
anybody who has a pulse in med school education and
administration has to know that trust in healthcare is in jeopardy. Anybody in journalism has to
know that trust in journalism, and more broadly media, is in jeopardy. This project lives at the intersection of medicine and the media. And here I am, a non-physician,
just an old journalist being asked by a leading healthcare group to co-author a paper for a medical journal about trust issues related to the way that messages are
communicated to the public. You can’t just point a
finger at journalists. You have to point a finger at the sources. And that’s clinician researchers, their academic institutions,
i.e. med schools. So, you can’t tell me
that you can’t shoehorn in an occasional seminar,
if not a regular course, in how to communicate research findings, how to communicate
better with your patients about how they should evaluate, they, the clinician researchers,
what they read in the news, and how they should help
their patients navigate these confusing messages. So, that’s a great question. And man, if I need to formalize and write this elevator talk for you, I’d be happy to do it sometime. But I hope that I just gave you some tips as to how to pursue this. Also, although maybe a
dean of a medical school wouldn’t listen to the dean
of a school public health, just kind of anybody who
contact John Finnegan, the dean of public health at
the University of Minnesota, and ask him, “Why did
you offer your school “to public health as
the institutional home “for HealthNewsReview.org
in the past four years?” It’s because Dean Finnegan said, “This is absolutely locked
into the core mission “of our school public health.” So, I don’t know why it
wouldn’t locked into the mission of a medical school as well. I hope that helped. – [Elizabeth] Oh absolutely. And I’ll say Norris said,
“Thank you so much,” and it’s resonated with
some of the other folks in our webinars. So, Red Rocks says,
“It’s music to my ears. “I have a masters in communication, “and I’m studying trust
with nurses and patients “in oncology settings.” Sounds like there is an
evidence-based out there, or there’s going to be that could assist. – [Gary] Red Rocks, I would just say I don’t know if you know about, or if you do, what you
think about the work of a young oncologist. He’s in his 30s named
Vinay Prasad, P-R-A-S-A-D who is at Oregon Health
Sciences University trained at NCI. He was one of the people
I cited in that article I showed you about, web
resources you can turn to when we go out of business. I don’t know how he gets
done what he gets done. I am sure he’s a fabulous clinician. I know he’s a fabulous researcher. He is a prolific author. And on social media, look
him up, Vinay Prasad. V-I-N-A-Y P-R-A-S-A-D. He battles more hype than
almost anyone I know. I’m gonna do a search here. Yeah, there he is. He goes by @VPplenarysesh because he just launched
a podcast recently called Plenary Session. So, as you can see, he’s pretty popular. 26,000 followers, 34,000 tweets, 33, 34 years old. He’s already a star on the
horizon of oncology in my view. And largely because of how he teaches us how to communicate better and battle hype. – [Elizabeth] I have to follow him too. Oh and just another question about your paper series
on trust and healthcare. Is that something that’s
gonna be publicly available? – [Gary] Only a few of the
papers have been submitted. My co-authors and I are
probably unfortunately slowest of the entire group. (Elizabeth laughs) There are a dozen-some
papers in the works. And so, yes, obviously
both ABIM Foundation, the instigator, and the journal, which I won’t name the journal that’s considering these right now. You don’t have it until you’ve got it. I’m sure we’ll be publicizing this. If it happens, it will
probably not all be at once, but maybe three papers at a time spread throughout the year
or something like that. – [Elizabeth] Okay, that sounds great. Well, folks, I know that we’re
running out of time here. But if you have additional questions, Gary and I chatted beforehand, and we’re happy to just linger
on the line a little bit for those with additional questions. For those of you that you have to run, just a couple of things. First of all, we have an
evaluation for all of our sessions. I’m gonna put the URL in a chat box here, but you’re also gonna get an
email from me with that URL and a link to recording once we have this uploaded
onto our YouTube page. I do ask that you be patient with us ’cause it takes us a couple days to get the closed captions
up on our recording. But everyone who is registered will get a link to the
recording and the evaluation. For those of you who are interested in continuing education credit, again we do have this
approved for one contact hour of Medical Library Association CE. You can get access to the CE
just by taking the evaluation. At the very end, there are
step-by-step instructions for claiming your credit on the Medical Library
Association’s website. For those of you that have more questions, please do stay on the line with us. And those of you that have to
go, we’ll see you next time. – [Narrator] Thanks for watching. This video was produced
by the National Network of Libraries of Medicine. Select the circular channel icon to subscribe to our channel. Select the video thumbnail
to watch another video from the channel. (light music)

Leave a Reply

Your email address will not be published. Required fields are marked *