– It’s my great honor
to introduce Dr. Virnig. Though I’m sure she does not need an introduction at all to IHPI seminar. She is the director RSDAC, and to me it’s more
important than even being the Senior Assistant Dean (laughs and laughter) School of Public Health. For those of you, I’m sure there are many among the audience who work with Medicare and Medicaid claims data. I know we know about RSDAC. I call them so many times that in one of the conferences they
kind of recognized me. We were talking, they said, “Oh you’re the one who
always talk about the cost, “and call us day and night.” And so anyway, without further adieu I
will let Dr. Virnig to talk. – Thank you. (applause) Thank you. It’s a pleasure to be here today. I’ve really enjoyed my
time visiting everybody. And I’ve got to admit, you can warn the people from Minnesota, I will go into the
office on Monday morning, with the phrase they
hate more than anything. Which is, I’ve got some ideas. So I really appreciate
all of your openness, and questions, and ideas, and chance to talk about science. ‘Cause that’s, I would say
sort of the fun part of my job. And so this has been a nice opportunity. I actually use this as a chance to sort of do some self reflection. So what I started to think about when I was getting ready to do this talk, is what is it that I really like to do, and want to do more of, and how is it I wanna approach the part of the world that I study. And so this really is
sort of some examples of how I see that, and why I see it. And so it’s not necessarily comprehensive. It’s not always even the
most current stuff I’m doing. But it’s really just more to talk about how I’m thinking about things. So one of the questions that I’ve gotten very interested in is
whether it’s patient factors or provider factors that
are driving health care use. Sort of who’s driving the bus. And so, when you think about that. It’s not just an academic question. It’s actually, if we
sort of sometimes say, as I think we all do, I’m really frustrated with
the patterns that I see. They don’t seem right. They ought to be different. We should be doing a better job. And then I think many of us have thought those things when doing our research. Getting back to this sort of
the causal model, if you will. Is it can become really important to sort of say how can we create an evidence base that would allow us to more effectively identify
the type of intervention, policy, big P policy, little p policy, that would actually change
the patterns we’re seeing. So that’s really what I looked at. So if you think about PCORI
and patient centeredness, one of the questions that I often find myself asking is,
is patient centeredness really the model of how
healthcare is being done? Is it patient centeredness? Is patient centeredness more
of a goal than a reality? Or is in fact patient centeredness, are in fact patient preferences
really what’s driving? So if you think about the US Preventive Services Task Force, they will often say, and for this usually massive group, we don’t know go talk to your doctor and explain your preferences. Right, we’ve all seen those guidelines that say, talk to your doctor. And so the question is, well is that really happening? Are we seeing signs that it’s really patient preferences that
are driving decision making. You know, or are the providers
applying their own standards. So if any of you have read, back from the 70s, back when
they were psychologists, Kahneman and Tversky,
they were really about physicians as decision makers
who were in fact human, and subject to their own biases. And understanding their biases. And being able to work with them. So in fact, is, should that be our model of how we approach things? So you know, if we think
about things like overuse, low value healthcare, choices, how does this understanding help us ask insightful questions and ultimately point to policies or strategies that work? And then in some cases, and I get accused of
this somewhat regularly, is that I’m blaming
physicians for their patients. And in some of the cases it’s a little ridiculous, I think. Like the cases where they say, no the reason they didn’t get a lymph node when they were doing their breast cancer surgery is the patient refused it. And I say, if they really, they wanted the surgery but they just said you couldn’t take that
sentinel lymph node? Like that doesn’t make total sense to me. So are we, is that just an excuse, or is in fact, how our patient preferences and how is our understanding of them going to feed into what we do next. So the Medicare data. Now I just, as a sort of a disclaimer. I use lots of other data besides Medicare. I’ve done surveys, I do focus groups. But I’m gonna focus on that today, the Medicare data today, just because of the RSDAC connection and because a lot of how
I’m approaching this, this is a good way to think about it. So the Medicare program right now to see, has about 50.3 million
elders age 65 and older. So it’s a big population. So some of you when you’re dealing with clinical studies and
you worry about low power. Well the problem you’re gonna have with this data is the opposite. You’re gonna find things that
are statistically significant. That are completely uninteresting, because as we’ve got hyper precision. But if you think about
it, in terms of variation, we’ve got 33, they in the
last year they had statistics. 3392 short stay hospitals were reimbursed. 1300 critical access hospitals and 1.25 million providers,
individual providers were reimbursed by the program. So that means you actually can start dividing out variants, and you can start looking at things. And so that is sort of the background. The other context, and I don’t know if you’re used to this slide,
is healthcare inflation. So the top line is total
national healthcare expenditures. The bottom line, look at that,
is Medicare expenditures. So we can certainly argue
that Medicare expenditures have been controlled more effectively, or at least the inflation
perhaps than healthcare overall. But we can also see that it’s definitely, we would like to be curving
both of those curves just bending them down
just at least a little bit. And so that’s the motivation for policy. And there’s certainly
a lot going on at CMS, the Centers for Medicare
and Medicaid Services around trying to bend those curves while still not impacting quality. So we wanna do the both. We wanna get more for less. Better with savings. And so understanding that. And understanding that motivation becomes very important as well. When I look at it, you
know here’s where we are. 60% of the Medicare population is enrolled in traditional
Medicare fee for service. So it’s about 38 million people. In the managed care encounter
data were released in 2018. I talked with some of the analysts briefly at lunchtime about those data. And so, but the answer
is, the very large volume means you can look at
really, really rare things. We can look at healthcare use
for people over the age of 90. In clinics that’s really tricky. But we can look at it. We can look at, we can
take an example and say, well there’s actually a small population where we can get the pure control we want. And we can do it and not
worry that we’re studying, comparing 11 people to 10 people. Because we can, because both facilities, and providers bill we actually can understand who’s doing what. So we can identify for
example the surgical team. We can put together, this is the hospital, this is the surgeon,
this is the pathologist, this is the anesthesiologist. And we can start to
understand those teams. So even if we wanna look at something like a complex interplay between
different professionals we often have the chance to do it. Or at least try to do it some of the time. And so, and again we have
more statistical power than we know what to do with. So that combination is both exciting, and sometimes a little terrifying. Because there’s so many cases, as I said at lunch, you know, SAS doesn’t ever return
an error that says, this was too dumb of a
request for me to process. Try again. So we’ve all had it
where we get a table out. At lest I think we have. Please tell me I’m not alone. And we’re spending some time. We’re looking at it. And we come up with like
this great understanding, and then you discover it’s
actually a coding mistake. And we weren’t looking at
what we thought we were. But the numbers were so compelling that we got completely distracted. For sometimes a way too long. So that’s one of the
challenges we’ve gotta face, that we don’t face with
chart and view data where we know exactly which
patient we abstracted. But it’s also hard, or harder
than we would like it to be. So rarely do Medicare data, or administrative data, not
just Medicare and Medicaid, HCUP, Optum contain the
exact variables I want, in the format I want. So what that means is
we need to be creative. And we need to understand the rules. So for example, you know, I was working with some epidemiologists, they said, well where’s
the hypertension variable. Like, what? There is no hypertension variable that we don’t create ourselves. So the challenge with that
is that we have to create it. Which means we have to stop, and we have to understand,
we have to do it right. One of the other questions
that came up on lunch, at lunch was like, can we
trust what other people do? So you know you find these algorithms, and can I just use their algorithm or do I have to create my own? And again the answer is always like, trust but verify a little bit. But the other piece of it is, how do I identify our at risk populations. So if we’re thinking about these as mini experiments that
are going on in the world, setting up the experiment is really where we need to be spending our time. And there’s a lot of opportunity. But it’s also the place where we need the most creativity and we
need to spend the most time. But again, so I’m personally,
and you’ll see this, I’m a big fan of stratification. More than I ever thought
I was when I was learning the Mantel Haenszel
Chi score test by hand. But in fact that’s often the way that we can see the
variability rather than, so you’ll notice this. I’m gonna be much more of a
stratifier than a modeler. And that’s just my bias. But again, we each can do it. But again, you will see
that you’ve got variation. And because of the volume you can use that to your advantage to try to ask insightful questions that
bring our knowledge forward. So I’m gonna go through
three examples now. And what they are is sort of examples of the types of questions
that I’ve asked and answered. But they’re also examples
of the sorts of things that I’m continuing to do. So just to sort of, to
make sure it’s clear. So one of the first ones is about geography as a source of variation. And about using geography. So you know, geography is destiny. And urban rural is one of the easiest ways to think about it. And in a state like Michigan, which has got rural populations,
and urban populations, understanding those differences
becomes really important. And it’s also hard because overall, the urban population is always bigger than the rural population. Which means if you’re
not thinking about it, it’s really easy to get
your statistics swamped by the average urban effect, and miss something that’s going on for a small number of people. And that was actually
something that came up, I was doing, because I’d studied hospice, I was invited out to the
group of hospice providers. And somebody pulled me aside and told me that I really didn’t
understand the whole thing. And there was this bigger
problem that I wasn’t seeing. It wasn’t race, it wasn’t
gender, it was geography. And that I’d missed it. And so you know, like all of you guys when you get those you take a challenge. You’re like, I didn’t miss
it, I just hadn’t looked yet. And I’m gonna look now, so there. And so this is what’s going on. So hospice, with hospital service areas basically anyone can go to a hospital. So most of the hospital service areas, if you think like Dartmouth Atlas, each zip code is assigned to where the plurality of patients come from. So there’s no zip code
in the United States that isn’t assigned to a hospital. Okay. But hospice is the reverse. It’s supposed to be home based. Which means instead of the
people going to the center, the center has to go to the people. And so, and well there are hospice homes, and institutional based hospice. It’s still supposed to
be, and it’s designed, it is reimbursed to be largely home based. And then it turns out in the regulation hospices are generally required to be able to reach their
patients within an hour. So the interesting
conversations around like LA, which I am not dealing with, because in fact you
cannot get from one end of LA to the other in an hour, even in the middle of the night. So they actually had to adapt
their hospices accordingly. So the question would be, are there places that are so remote that nobody has ever
received home based hospice? So that in fact, no one can get. None of the hospices are close enough to get to them within an hour. So if they were to get hospice care, they would have to go, they would have to move to
an institutional setting. Which means they would, which again means they
are gonna be an hour from their community, from their family and everything else because there’s no way for the hospice to meet regulation. So that, translating that to data, what we translated it to is, are there places where no one has received home based care in a three year period. Okay, ’cause that’s, ’cause again, we have to operationalize
it with the data. And so we don’t, so like, people would say yeah, it’s possible. I’m like, well have you ever done it? No. Why not. Well it’s too far, so
it’s really not possible. So that’s the question of what can we do as a surrogate for possible. And that was part of our question, is that does that rule work? Can we use it? Can we make a rule that
would work with the data that can also match reality? So what we did, is we started
with the state of Minnesota, which like Michigan has got a mix of urban, semi urban, rural, large, small. And we said well we’re gonna see if we can figure it out for Minnesota, and then see if we can in fact, extrapolate to the rest of the US. So what we did, is we called every hospice in the state and asked
them where they served. And we did it by just saying,
do you have a service area? And then we had a map. And like, do you go to Warroad? Do you go to Thief River Falls? Do you go to Pelican Rapids? And we would outline each
hospice’s service area manually. And then we layered them all, and we got the red
sections which are places in Minnesota where there was no hospice. We then asked whether or not we could get the same patterns from the Medicare data. So we had to do a little cleaning. So we had to first of all, we worried about people
who’s mailing address and home address didn’t match. So we got rid of everybody
where it looked like the hospice traveled more than five hours. So we got rid of the people who’s address was in Minnesota and were
getting hospice care in Arizona. We did not believe that was actually, that one of the two, that we believed that the resident or the patient was wrong in those cases. And then we actually ran
this past the hospices, and said, just to reverse this out, this is what we’re finding. Do you agree? And in some cases there
were really odd things. And I apologize to the audio tape. But, so do you see this red splotch here? It’s right between three hospices. It’s actually less than an hour from a lot of places but we found it as not getting any hospice. And so that one, we though
our algorithm didn’t work. And so we made some phone calls. And it turned out it’s more than an hour south of Duluth, so the Duluth hospice, which is big wouldn’t go there. It turned out the Mora hospice, which is to the south, is
a county based hospice. Which means they do
not cross county lines. And you get these circles. If you think about where
you put circles together, and you occasionally will
get these little squares where none of the circles overlap. And we found that with the data. And we confirmed it with phone calls. Which said okay, maybe
we’re onto something. And so that is an example
both of it can be really hard to do the mapping work,
but also really exciting. But what we concluded at the time is, the good news is the majority of Medicare beneficiaries reside in
areas served by hospice. But that’s actually not surprising because the majority of Medicare beneficiaries reside in urban areas. And so we know of no urban area unserved. So by definition, number one, we knew before we started any work. But we also found that there were in fact a large number of large,
but very sparsely populated areas in the US that were
not served by hospice. I could not, the map is getting squeezed, and I couldn’t find the original. And we actually started
looking at the finances of it. And realized that from
a financial standpoint and a reimbursement structure, that the small hospices,
when we were talking to them, really are on shaky ground. For hospice to break even, as somebody said, hospice is
managed care for dying people. You gotta have some number of patients in order to maintain
your expense structure. And for a lot of these rural areas they were never gonna have enough deaths at the same time to handle
their expense structure. And so it’s, so the house, so part of the problem has to do with the payment mechanisms. And so we raised that, and started saying, well you know, maybe like
critical access hospitals we need to have critical access hospices that would say, if we really value this, we need to be paying people differently. And we need to be acknowledging
that they’re serving, we need to give them a break on something. Either on the rules, or
on staffing, or something. We also worked with the hospices. One of the pieces they said, is you know, if you only
have one staff person, and that person were to like take a day off it’s really hard. So we talked about, and we, for a while we were working toward getting a statewide hospice float pool. So you could say okay, somebody
wants to take a vacation, we can help you find hospice staff that actually know what they’re doing, who’d be willing to spend you know, a week up in Warroad, which is that sort of the bump up going into Canada. And you could do that. But ultimately what
we’ve concluded is that, and it hasn’t changed, but without changes to the payment rules there’s absolutely no way
that this pattern will change. And this work was done a few years ago. And it has not changed. I kinda keep an eye on
it from time to time in Minnesota, in particular and
ask anyone’s doing anything. And again it’s just the
math of understanding the areas and documenting it, and raising it as an urban rural issue. So the next study is
one where we’re trying to understand who’s incentives. So when you get these examples where the incentives would point
in opposite directions. There’s this natural experiment to say, well which incentives are
lining up with the observations? And so the example, this was one that one of my students
did for her dissertation. And it was about hip replacement. And there’s two devices for hip fracture. Are there any orthopedists in here? Okay in that case, yes no, okay good. Then I can tell you, there are two devices for fracture repair. There are nails and screws. And what’s interesting about them is that they are different reimbursements. So physicians are actually paid more to do nails than to do screws. The device, the nails
cost more as a device. But the hospitals, because
they use the same DRG code are paid exactly the same for both. So from a hospital’s perspective, more expensive device, and equal payment you’re better off, the hospital
should be wanting screws because that would be, cost some less. The physicians should be wanting nails ’cause they’ll get paid more, and they don’t have any
device responsibility. So the question was, who’s incentives seem to be working? And so we looked at the Medicare data, ’cause we have lots of cases. We identified surgeries,
and what was interesting, so we started with the hospital data because we expect, by definition, all of these should be hospitalized. The hospital billing doesn’t
distinguish between the two. As I said, they’re paid the same. And in fact if you look
at the hospital claims you cannot tell which one somebody got. So we took the hospital codes, hospital billing and we grabbed everybody who had a 81.51,
which is a hip replacement. We then went and we linked in the surgeon. So we had to find the surgeons for all of those hospitalizations. Because the surgeons would
tell us what exactly they did. They have two CPT codes, 27245 and 27244. And so that’s how, that’s an example of how we can put it together, and understand the structure of the data, and the payment structures to really try to figure this out. So we had 192,000 surgeries. We had 15,000 surgeons,
and 3400 hospitals. So really, I mean that’s
surprisingly large numbers. You’re like, okay, you can do something with these numbers, and that’s part of it, where you can sort of, these are why the natural
experiments are so much fun. Because they’re true experiments. And you can cut it and
you can ask questions. Not all of which we asked. So here’s what we found. We found that younger surgeons, so we linked with the AMA data. And those who worked in a greater number of hospitals over a three year period were more likely to use nails. Okay. DOs, doctors of osteopathy, which was, this was not one we expected, actually were more likely
to use nails than MDs. I don’t completely understand that one. But it is absolutely correct. Teaching hospitals used more nails than non teaching hospitals. And nail use increased over the period. And our best model actually included a surgeon random effect. And I tell you that because it just says, the surgeon him or
herself actually matters. It’s not just a, so there
is some component there. And what it, we actually concluded after talking with people
and running a few more models is that there’s a
training phenomenon here. So surgeons, one of the things you’ll see with surgeons in training, they’ll often go to multiple hospitals, much more so than somebody once, who’s settled into practice. And we think that’s why we found the multiple hospitals effect. Is again it was just another
marker of recent training. But it, so in that case, you know, is that it points to is, is that teaching hospitals
want their surgeons to be able to use all devices, and may in fact preferentially use one that’s less common to make sure that the people who they
train are well versed in it. But what we don’t consider
is that as those people then are sent off into the community, so we have another paper in a series, what you’ll see is, you’ll see this diffusion of innovation. And you’ll see non teaching hospitals slowly picking it up. So it’s a little bit like,
if you’ve seen those, you know in the movies where they show the pandemic spread, and
it starts in a few cities, and it slowly goes, and pretty soon the whole country is taken up. Well just like influenza, so are nails. (laughter) But, so those are some of the things. And we have not taken on and looked at, you know, is this an orthopedic issue, or is this really how training works. And training as a mechanism for diffusing cutting edge technology. I think that’s the higher
level question coming up, is this one example that
was sort of interesting, and the orthopedic surgeons
found it fascinating. Or is there a bigger story
about how people learn, and how the way we train
affects the way people practice. The final example I’m gonna talk about is androgen suppression therapy, or AST. Also sometimes called ADT. And it’s recommended use for men newly diagnosed with
metastatic prostate cancer. And this is not my work. It’s a colleagues actually from, and what they found is
that over a very short period of time AST use
was just taking off. And CMS got very worried about it because they are a primary
reimburser of the therapy. And they were not happy with
how much they were spending. And they caught it. And so they said, as part of the 2003 Medicare Modernization Act, they reduced AST reimbursement by 64% over a two year period. Okay, so the questions we had were, well one is sort of the no brainer. Did it change things? And we expect it would right. I mean that’s why they
reduced the payment. So did it actually change use. But the real question that we asked is well did it have
unintended consequences? So if it just changed use overall that means that some people
who’d actually benefit from the therapy, because
of the price change weren’t getting it. And yes people who wouldn’t benefit weren’t getting it, but there’d be, the price might be too high if we consider the indicated use. So we wanted to ask both. Like did it do what it did. And we went in with like a
really strong prior there. But our real question was about
the unintended consequences. And were, and this was
a particularly dramatic decline in pricing happening very rapidly. It’s not usually what CMS does. And so it was an interesting
chance to ask what happened. And so what we found
here, and you can see. So this is the payment,
this dotted line here. And you can see it just fell off. So it’s pretty stable and if fell off. This is use for metastatic patients. And the good news was,
although it stabilized, it didn’t really fall
with the drop in payment. These are non indicated,
non indicated use. Low risk patients. And in fact what we see is it did drop. It didn’t drop as much as
I think everybody hoped. But it dropped okay. So those are, so the answer
was sort of an interesting one, of it looks like in this case, that for our low risk patients, use at least stabilized
and dropped off a bit. For high risk patients
where it’s indicated, it dipped a little bit, but not much that we’d worry about. Which would suggest that
for patterns like this, it might be safe to really
pull back reimbursement dramatically and not worry
about unintended consequences. Of course the challenge with this is that this is one example right. So these are cases where
we can see what we can do. We are excited about it. And the question is how
can we replicate it. And how can we find other examples that allow us to test this idea where there are relatively
clear guidelines. So we can actually say,
intended or unintended. And we have the data to do it. So, there was a slide that is missing. Oh, we’ll just go through it. So when we think about these studies, and there are a number
that I’ve also published, and a number that are in the works. Some of the things that allow this, and this is for all of you as well. Was first of all is that
we have a long time period. Which means we can actually see pre inter, pre thing, while it’s happening, and post. And so we actually get that time series, so we can thing about different,
you know different models. So Medicare data currently
are 1999 through 2017. SERA Medicare currently I
believe is 1991 through 2016. So again we’ve got these long horizons to look at effects and
get these trajectories so that we can be sure
that we’re not reacting to one data point shifting. But we can look at it, and
we can look at those changes. We’ll talk about the downside next slide. We’ve got really large volume. And that’s fantastic. And that really allows us to
do things that we couldn’t do. You know 192,000 surgeries
over a three year period means we actually can look at things, and we don’t worry that if one person had made a different decision that our effect would be reversed. And it allows us to do things
like random intercepts. You know, random slope models. We can do our modeling and we can use all of our statistical ability
to try to break things out. It also means we can stratify. So if we have a hypothesis that says, well there are, I expect differences between urban, rural or
between different age groups. We actually have enough power that we can legitimately look into them. And that’s good. And that’s again the variability. There are multiple sources of variation which allow us to sort of really understand these mechanisms. And we don’t have to do something like, when the policy changed,
was there less use, but we can actually start
asking the questions we wanna ask like was there preferentially less use in certain circumstances. Was it according to indication? Was it according to race? Was it according to age? What type of facility? Whether or not it was a training facility. We can ask those questions to really start to come up with some strong hypotheses about what is going on that might point us to what is the right strategy for certain types of things. Consistency. And I really wanna emphasize that. So the Medicare program uses
the same rules nationally. The rules may change year to year. But there’s a national set of rules. And what that means is, that if I find geographic variation, I don’t have to worry that maybe it’s because the reimbursement rules are different in Montana
than they are in Michigan. And maybe it’s just a
difference in reimbursement. So just to give you sort
of a really concrete example of that, with
the new encounter data, which is the Medicare Managed Care, one of the things that we noticed is that we were seeing bills
for ice packs in the ER. Now Medicare, fee for service, would never in a million
years pay for an ice pack. It would be bundled into the payments. When the ER gets paid, that global payment includes an ice pack, and aspirin and all
sorts of stuff like that. And so we don’t even look for that. We would never see it. If somebody tried to get
paid for it it’d be rejected. When the encounter data, we’re seeing ice packs, we’re like what, like we didn’t even know what they were. I mean like we’ve never seen that code. So we started making some phone calls. And it turns out some plans allowed for that level of detail. Like they actually required
that level of details. Other’s didn’t. So what it meant is that if
I wanted to study ice packs in the ER, right ’cause I think that’s my new quality measure, I wouldn’t be able to differentiate between they didn’t get an ice pack, or they got it and it was just bundled in some global payment. So that consistency
means that in those cases the inference is really challenging. So we don’t quite know what it means. We know what it means to have it. So I’m really confident that the people who have ice packs had ice packs. But I don’t know whether the people who didn’t have a bill for ice packs got one and it was just rolled in, or whether they never got
one in the first place. Now you’re gonna say,
you’re sitting back there going like who cares about ice packs. But you can imagine, there
are other bundling rules, like biopsies, pre op
physicals, pre op checkups, post op check ups where that
same phenomenon is happening. And so the consistency, whether we like the rules or not, at least what we know is we know that if we’re seeing variation
nationally, it’s not that. Okay so one source of possible error, one source of nuisance
variation is removed. There are standard billing forms. And there are standard, and I would argue, documented coding rules. You sometimes have to dig a bit. But they are documented somewhere. And in terms of the detail, you know, my general rule is if it impacts billing, you get paid differently because of it, it will be there. And so the good news is,
is that most of the things that impact payment are things we’re interested in anyway,
so this is one of those, even if you don’t study money, where the incentives are lined up. But sometimes it feels like, how many of you remember
this from algebra, right. Where you didn’t know
how tall the tree was, or the building was so
you held up a yardstick and you measured the
length of the yardstick. And then you measured the
shadow of the building, ’cause you couldn’t actually
measure the building right. ‘Cause our tape measures
weren’t long enough. And we don’t let fifth
graders on roofs of buildings. So, but this is in fact what we end up doing in the Medicare data. We can’t measure things
directly a lot of the time. So what we end up doing
is indirect measurement. So it means we need to be creative. But it also means we
also need to be aware. So this is the fun for me. This is always a challenge of okay, let me think how I can
measure what is unmeasurable. But it’s also my challenge
of let me make sure that I’ve really thought about it enough, and carefully enough that I am sure that my inference is more likely than not, I mean we don’t have to do any
more than that to be right. And that’s the challenge with these data. But it’s again, it’s also
the opportunity and the fun. So you know, bottom
line, these are not data. You know like, I get
these calls sometimes, and like I’m a resident
in, pick a department where nobody’s here, I think
it’s orthopedics today. And abstracts are due Thursday. Can you help me do a
paper using Medicare data? I’m like, no, go away. You’re two years too late. But I mean these are, you cannot rush. I tell my students, you know, when they do their three paper exams, which is what I usually have them do. You know the first paper
will take eight months. The second paper will take four months. And the third paper will take six weeks. Right ’cause getting the data in place, understanding it,
understanding the coding, figuring it up takes a long time. Once you get through that, one you get through all that cleaning and that understanding it’s, you can get a lot done. But there’s always this
huge learning curve. You need to create variables as I said. The coding rules change over time. So one of the things I told the analysts that I’ve learned, honestly the hard way, is we always now plot time trends. Often by quarter. And minimum biannually. Because things change. and sometimes weird things, one of my students who I’ve talked to you, a couple of you about, was
plotting use of a medication. And it’s kinda goin’ along, and all of a sudden it just like dropped. Then it went back up. And like, it’s just like
what did you do wrong? You know, these poor students, they always get chewed out by me. She was like, I didn’t do anything wrong. I’m like well, this is
not a normal pattern to have use drop by 90% in one quarter and then go back up, something happened. We missed some data. and it turned out that CMS
dropped a code for a quarter>and then just realized their
mistake, and put it back in. And it wasn’t her mistake. But it’s like okay now, but then we found the
code that they were using, and she was able to get her
curve to straighten out. But those are the sort
of things that just, when you’re rushing to a
deadline for an abstract or something else, just cause everybody’s blood pressure to rise. But they also, there’s a learning curve, but it never goes away. So it’s not like I can
now do these in two days, and everybody, and you know, it takes everybody else,
it always takes time. So the complexity we can
use to our advantage. But we have to be willing to recognize that there’s complexity,
and not be caught by it. There definitely is a huge amount, as I know many of you know
about pattern recognition. And there’ll be certain patterns, we’ll do it frequently,
and like okay that’s wrong. We know the second we see it that this is not the shape of a reimbursement curve
for hospitalizations. We don’t need to double check it. It just is. And so that’s the
advantage of the experience is that you can usually figure
these things out faster. Computing time, I don’t know. I mean I think computing time can be long. And so you know, I’m sure you’ve figured out the same thing we have. Like we’ll often run it
on an obs equals 1000 just to make sure that the code works before we you know, set the
computing and then go to lunch, and happy hour, and breakfast,
and then hope it’s back. And again we always get results. And that’s actually the piece that always scares me the most. ‘Cause you know, we always get results, and we often have two tailed hypotheses, which means we’re always
happy with whatever we got. And we can usually come up
with a really interesting story which may or may not be right. And I’ve talked with our statisticians about whether they could
come up with a term. So I’ve had other the
years, numerous articles with editors saying, I
will not give you P values because I don’t think they’re really the appropriate statistic,
there’s too much precision here. And I don’t believe that
for people over age 90 we should be measuring
their age in months. So I’m not gonna give it to you. I admit I usually lose those. So we’ve started now rounding our units, to the units we think are credible. So if I think, if I’m worried about too much precision for age, I will only code it as
five year age groups because then I’m happy reporting it. But in fact, the down side, if you’re not, really sort of thinking about it, is you can end up in
some really funky boxes where you’re having to explain something that you don’t really think has any like practical significance. So you know, so in short, these data are really fun. Okay that means you can ask questions, and you can answer
questions that are really not answerable other ways. And that’s what’s exciting is that, if you think about
layering qualitative data, chart review data, and
national Medicare data you will come together to sort of focus on what we think is probably going on. And the ability to do
that is really exciting. And I think challenging. And it’s the reason I use the data, it’s the reason I enjoy it. It definitely requires a level
of precision and patience that I don’t think I appreciated when I first started using these data. And so I’m not trying
to be sort of negative when I say this, but that’s really sort of the biggest downside. And it’s very complex, but in fact, you know I’ve gotten accused a few times, saying well these are only descriptive. You can’t test hypotheses with these data because, I don’t know, you can’t. And I would respectfully disagree. I think part of what’s
really fun about these is you can use these
for hypothesis testing. You can use these for model calibration. And that what it really depends on is whether you had the hypothesis before or after you got the answer. To me at least. Which is with a question of
whether it’s descriptive or not. It’s really how we
approach it as scientists, not where the data came from. As one of my early mentors said. The data are the data. The data don’t care. It’s what you do with the data that makes it a good study or not. And I guess that I’d say
is with Medicare data I think that’s probably
the way to think about it. So I just wanna acknowledge,
I do not work alone. I’m actually not allowed
to do programming anymore, ’cause I’ve gotten so rusty,
I make too many mistakes. And so these are some of
the people I work with. Helen, Steph, Shirley, Sean Elliott, Todd Tuttle, Mary Forte,
and there’s others that I work with as well. So I just wanna acknowledge that both so you can see how these do. These are definitely a group activity. If the slides gets passed out here, the references for the papers
that I was talking about. And then any questions? Comments, thoughts, disagreements? – Question, I know that
RSDAC works with CMS closely. My question is our study
point to savings for Medicare. And hopefully it’s a
reusable data like Dakins. If we get the data and validate the findings and everything, is there, what’s the mechanism for Medicare to pay attention to the findings based on Medicare claims
data to save money. An example is the nail thing or many of these surgical procedures we know, the literature, that
the outcomes are the same, to what the cost savings
there and is substantial. – So the question is how
do we get CMSs attention. And there are a few ways,
depending on what you do. So there’s, CMS has a coverage group. And that group definitely comes, handles some utilization management. So they will help with, so this, Medicare, because
it’s a federal program, to make changes to the program actually require Congressional mandate. So it was in, for those
of you who don’t know, the reason they were so slow to flu shots is because originally when
the statute was passed it was not to provide preventive care. It was to provide acute care. And so slowly, to get the flu shots passed it actually took an act of Congress. It didn’t just feel like it. It actually was it. And so one of the things
that happened is there were, we had to make, the
evidence base had to be made that this was enough of a savings that they were willing to do it. So from the early 90s when I started working with the data
there was no preventive, there were no preventative
services covered. And now there are a large number of them that are covered including
cancer screenings, and other screenings, and the
Welcome to Medicare visit. So it’s definitely been a pivot, as I think both our
understanding of healthcare, and understanding that preventive care is in fact a way to
control healthcare costs. So the coverage group is sometimes a way that we have brought, that I’ve personally brought things to CMS’s attention. But CMS does watch the literature. I mean, and I’ll admit, that sometimes in my RSDAC role, so
again as that contractor, we’ll sometimes point things out. Like you know, this is really interesting. And you know, we might point it out differently if we think it seems like it was done really well, or if we’re really not sure it was done. But if we’re starting to hear things, we’re starting to hear chatter that like this approach, or this question, we’re getting a lot of
people interested in it. We’ll let them know. And we’ll say, you know,
there’s a lot goin’ on here. And there’s a lot of movement. You know we’re not asking you to do, you know I can’t ask them to do anything. But what I can do, and
what I often will do, is say, so you might wanna look at this. So there’s a couple of ways you can do it. But again, also what you’ll find, though you may not know it, but large conferences,
particularly those in DC, will almost always have
somebody from CMS present. They rarely make themselves known. But, so they’re trying to, they’re trying to keep on top of that. And then you know the
utilization management. So there are programs that
have local organizations in each state will also be working on it. So I think CMS is trying
very, very hard to innovate. And is, you know, again if
you think about those curves. That need to continue the downward, the flattening, if not the
bending down is very real. Particularly as they’re anticipating the rise in the Medicare population due to the boomers moving through, combined with the increased longevity. So you know we have a number of things. And I’d say at the very least, if you have something
you think is really good, and is really right, send
it to the coverage group, and send it to me, and
I’ll forward it on up if I think that there might, you know sometimes we’ll just ask, like who is the person who
might find this interesting. So there’s always somebody
who will find it interesting. Whether they will act on it or not is, and sometimes they won’t tell you. You won’t hear anything,
and then all of a sudden you’ll read something like
that it just sort of happened. An example actually of
something very real happening, is early on in my career,
I was working with a friend of mine Robert Morgan, and we were looking at how to identify Hispanics. I was living in Miami at the time. And if you look at Miami,
according to Medicare it’s about 50, it’s majority white, with some blacks, and about 8 Hispanics. Which anyone who’s every been to Miami knows that’s not right. And so we did this surname match. And we actually did a survey. See, I do do surveys. And we asked people
like, are you Hispanic? And we actually asked (stammers) and then we tried to figure out whether a surname match worked. And we did it for Florida, or other areas we did our survey, and it was picked up, and they ended up hiring RTI to validate it nationally. And that’s now the variable that’s in the Medicare Summary File. So those things do get picked up. It’s kind of haphazard a little bit. Yes. – I have a couple questions. I notice you have a couple
of MDs on your team. My question on the clerical side is how do you know you’re
looking at enough variables that are associated with
what you are looking at to insure that your
conclusions are relevant. Lemme give you a couple of
examples on things you did. The medication, the androgen therapy. They dropped off in the low risk patients. But did the physicians look
at in that circumstance was it a Class IIb, or a III indication in a low risk patient and a Class I in a high risk metastatic patient, in which case no matter
what the cost effect it would continue to be used in the high risk patient medically, because that’s the right
thing to do for your patients. It may really not to be necessarily the right thing to do if
it was practice at the time in a low risk patient. The other one’s your hip nails. What is the current, you know
they’re screws and nails. And I’m not an orthopod
so I can’t tell you. What is the current
number one recommendation. Perhaps now it is nails. 15 years ago it was screws. Are the old surgeons just
doin’ what they learned and still using screws, the young guys are all using nails. I’ll give you another
(jumbles words together) are the academic centers
and the teaching centers using nail because they don’t
pay as much attention to cost as do private, community centers where cost is a huge factor? – So I think those are
all really good points. And I’m gonna start
with sort of one of them which is, so in fact,
in the case of the AST, it really was a Class I versus a Class, they basically not recommend it at all. But the question was, we
have plenty of examples where just because
something is recommended doesn’t mean it’s not price sensitive. And that was actually our worry as this was absolute recommendation, and the question was,
but is it price sensitive so that the price sensitivity got in the way of what was right. And that was actually his concern. So Sean Elliott is a urologist. And so on every one of my teams I always have somebody
who’s the right specialist. And they sort of do two things. Part of it is that if I can’t, if my work isn’t credible
for that area of specialty, one it’ll never get published. And so then I’ve just
wasted a ton of time. But also, I’ve like missed the point. So it’s just another paper in a journal. Now maybe it doesn’t
change anything anyway. But it would be even worse
if it didn’t do anything. So Mark Swienckowski who’s the chair of our department of orthopedics helped out on the nail and screw paper. So he was very, in fact we have three orthopedic surgeons on that paper. So that’s part of what I do. Now they do two things
for a PhD researcher. One is, they check,
they’re a reality check. Like is this in fact right. And sometimes there’ll be things we have decided not to study,
like bariatric surgery. So everyone’s very interested
in bariatric surgery. The problem is you are
required to have an ICD9 code of high BMI in order to get the surgery. You are not required to have your BMI coded if you don’t need the surgery. So we’ve never been able to figure out what’s the comparison population which should be people who are equally obese but didn’t get the surgery. And so we have plenty of
bariatric surgeons who wanna know. So we’ve talked about, well let’s look at other variation within this. Like is there a device variation
among those who got it? But, so there’s this reality of, if we can’t measure, so we’ll say, well what is the important thing. So oftentimes with cancer
it’s surgical margins. Well we don’t have a measure in SEER or in Medicare for surgical margins. So we can’t distinguish between somebody who got a reoperation
for positive margins, and somebody who had positive
margins but wasn’t reoped. Right, so we just don’t study that. But the other piece that they will do, is we will often do these groups of papers so that there’ll be some that have like the most clinically
relevant questions for them. Where I will make sure
the methods are solid, and the results are presented right. And it will be put in a clinical context. And then for my papers which are much more about the healthcare system, they think they’re a little
boofey, but they’re used to me. But what they will do is
they will read it over, and ask whether it’s clinically credible, and accurate, and written in a way, and answer those simple questions. And we’ll often, I’ll often vet my papers across other colleagues
from different disciplines because I’d rather know early that there’s a question I’m answering than figuring it out when I get it back from a journal editor. So that’s how I deal with it. So I agree with you completely. And I think that’s, my understanding that’s how your group works, is that it’s all multidisciplinary teams. And this is why those teams
end up being successful, as your group is, is because you solve these problems collaboratively, and you solve them early
rather than figuring it out when you get rejected
by your fourth journal because of some mistake that you could’ve figured out had you asked the right person the right question. I don’t know if that helps,
but that’s what we do. And there are plenty of ideas that we have elected not to study because we can’t figure
out how to actually control for the things
we really care about. Other questions? Snack time? (laughter) So thank you. (applause)