Ep. 11: Freakanomics of Clinical Trials

all right folks well good morning good afternoon maybe good evening for you and

welcome to the 11th episode of the deliberate way I’m Dan SE Wald your host

and I’m really excited about what we have going on here today you know we’re

almost on a dozen episodes and as we kind of get to that precipice I could

think of nobody better than to bring on and the talk about this is Craig lipid

Craig we’re g to be talking about the Freakonomics of clinical trials and I

got to tell you it’s a topic which Super fascinates me now I know not everybody

who works in life sciences or Pharmaceuticals thinks about this day in and day out like maybe like you do but

I’ll tell you this topic is so relevant so personal I I can’t wait to dive deep

into it and I’m going to share a little more background about you but Craig big thanks for for being here and joining

Dan I’m such a fan of your work it is absolutely a pleasure to be here with you I’m looking forward to today’s

conversation let’s go all right well Craig for the people who don’t know you

that well I’m gonna give a real quick um capsule of this is your life Craig lipsit um you know Craig and I met many

years ago while we were both working at fizer and and at the time you may remember Craig you were the head of

clinical Innovation and Venture partner at fizer and uh since you uh departed

fizer you’ve been running a really cool little Boutique advisory business called clinical Innovation Partners where you

work with executive teams with startups Boards of directors um you really do

some awesome stuff there and I would be remiss if I didn’t mention that one of

the topics that’s near and dear to your heart is the topic of decentralized Trials and you co-chair the

decentralized trials and research Alliance as well as you speak on the topic a lot you’re a regular pundit you

advise people on it it’s kind of one of those things which not everybody thinks about but you really are kind of a real

deliberate innovator in this space and I you know I’ll pour on the flattery just

a little bit more Craig when you’re not doing all this stuff in industry and helping companies out I hear that you’re

a regular speaker and a teacher at Ruckers and the University of Rochester

I don’t know when you sleep when when do you sleep I’ll get to sleep I’ll get to sleep one

of these days but you know what Dan it’s interesting I was thinking about the this when you were giving that overly

flattering intro I was not a deliberate innovator when we did the first

decentralized clinical trial at fizer so many years ago uh and back then you know

I had this opportunity to help uh lead a team as we were scoping out what a fully decentralized clinical trial would look

like enabling people to participate entirely from home you know a lot of that was just fueled by my own

experiences as a patient by a lot of the patients I was interacting with in my own journey and bringing that back into

work and looking for those opportunities uh so it is interesting now to think about how innovation has become more

deliberate but like a lot of Travelers in the space I think I started off an accidental innovator um but then stuck

around yeah well accidental innovators are also great deliberate innovators and

sometimes it just takes time and experience all right well Craig I’m going to jump in I’m going to ask you a

a personal question just to start things off um a little direct so I apologize in advance for the directness of it

clinical trials would you personally put your own child son or daughter into a a

industry sponsored clinical trial you’re just jumping right in there Dan I am jumping in yeah so um I I have two kids

uh one one of them does have a chronic uh medical condition he does take um medication for and is well managed so it

is something that we certainly would have thought about when first managing his diagnosis I think like a lot of

parents and like a lot of families and for us it would have it’s always been about risk thresholds and I think it’s

that way for a lot of patients and caregivers now in our case I think like a lot of families if it’s a story where

we didn’t have other treatment options If This Were a case where either we had exhausted every option available to us

or um sitting with our uh trusted provider there just weren’t any

available options to otherwise consider I think then like a lot of families we would we would go down that path in in

our case like so many others uh there are treatment options and so that creates a very different bar I think for

a lot of families to even consider a clinical trial because there will be clinical trials that look to uplift us

from existing treatments but the bar for me to be willing to introduce one of my children into those trials knowing that

there is an existing treatment option is extremely high you know I really need that medicine to have been tested in

adults probably be approved in adults and really become more a story of how do we dose an adult medicine better for

kids than introducing something that’s truly investigational and new in my child and when you think about people

who are doing more investigational molecules right now and in participating

in those trials why what what inages them what motivates them to do that

knowing that the risk is higher because the uncertainty is higher what have you found from your experience I think that

a lot of the research and Industry right now that’s uh that does direct towards kids really goes after unmet needs and

so those are the ones in rare genetic conditions that are really looking at areas where the kids really don’t have

other treatment options outside of those where there’s unmet needs a lot of the

trials today involving kids are really to fix the gaps that we have around dosing we treat so many kids in health

care as if they’re small adults and dose them as if they’re small adults and we really don’t have good evidence about

what is the right dosing for a child and so in that case I might feel okay having

my child be in that trial because I realize I’m in formed I’m kind of privileged enough to be aware and and

comfortable with the research literature to be able to come in and say well the status quo ain’t that great right we’re

kind of guessing the right dose for him anyway but again that that involves starting with with folks with that right

level of of privilege of knowledge and it’s so hard for so many especially with

a sick child to navigate to get there I’m gonna I’m Gonna Roll back um and ask

another very naive question um thinking about clinical

trials having worked in industry for a while have having done projects around

clinical trials it should probably be pretty intuitive by this point for me certainly for you but when I look at

clinical trials the complexity sometimes feels almost um you know inordinate it’s

so complicated when you look at trials and the cost of them as well for that matter but thinking about the complexity

of Trials they’ve come a long way why are still so complicated I think the first trial was in the 18th century and

modern trials go back almost 80 years why are they so down hard what’s

making them so difficult you know uh I I uh I I ran this Innovation conference

for the clinical trial crowd for uh a couple of years and when we first did it about 12 years ago um our mutual friend

from fiser Greg Simon who ran policy came and did the opening keynote and

Greg is is fabulous if folks don’t know him he ran the Biden cancer Initiative for years but he he opened by by sharing

you know you’re never supposed to show up and talk about religion politics and death but he opened by pointing out the

first clinical trial was in The Book of Daniel in the Bible and pointed to the king of Babylon um identifying that uh

the king at that time told his his his the men the the the fighters uh you

shall only eat meat because that’s how you’ll stay strong and there were a group that said well we kind of like

vegetables so he let uh some of those guys eat their vegetables for 10 days

and then looked at them and that was the end point the king looking and saying you look pretty healthy and after 10

days of that cohort having a different diet actually came to that conclusion so

not exactly randomized um but there was an intervention a different diet and

there was an endpoint what the king had to say about it so yeah I guess you could say we’ve been doing these um

experiments for for quite a while um a lot of folks do point to um I I I

trained in epidemiology and the the James Lind story of scurvy is often

pointed to as as really the first randomized trial where uh uh also about

a dozen men on a ship in demic with skurvy it debilitating to the men um uh

assigned different uh different regimens to the different men and kind of came back and said well uh which of the

regimens seem to prevent scurvy um and so some men were drinking cider and some

were drinking seawater and a couple were eating oranges and lemons and that’s how

we learned about the the impact of uh citric fruit but to get to your question

about well we’ve been doing this then for maybe thousands of years and certainly in the modern I underestimated

by a few by a couple Millennia but even in the mod ERA this is not a New Concept

why has this gotten so expensive so burdensome to the point that you really

need to be uh an extremely well- capitalized company to develop a

medicine even a university can’t really do this on its own um it’s hard for say

a nonprofit organization to do this on its own it’s really so cost prohibitive

and and why is that the case we’re talking about uh a controlled experiment

okay uh introducing an intervention and having robust systems for for patient

safety and for data Integrity right those are the guard rails for clinical

trials we have to focus on safety and make sure that the people that are um stepping forward and consenting to

participate or protected and we have to have confidence in the data and everything else theoretically we have

some latitude around to be able to do it and better and smarter ways I think that we have layered response to response to

response to different issues over the decades and created a morass that becomes almost impossible to modify or

incredibly difficult to innovate and Incredibly expensive and it certainly leads many to wonder what if we started

with a blank slate what if we started with a blank sheet of paper would it look anything the way it looks today

especially when we’re trying to lay research on top of an already dysfunctional Health Care System right

where people struggle at least in the United States struggle for access struggle for affordability struggle for

information that they can trust and now we’re dropping research right on top of that you know look it’s not a bragging

right in the United States that we’ve got one of the most expensive health systems on the planet and some of the

lowest um mortality rates uh among developed countries on the planet and so

dropping a research infrastructure on top of that we kind of get what we get

as a result and you know I’m going to kind of drill down a little further when

you you mentioned about expensive and we did call this the Freakonomics of clinical trials and you know for those

who are are not familiar with the work of Dubner L fre economics some of the kind of the moral hazards unexpected

things that happen in the world of Behavioral economics um clinical trials for me are are one of the interesting

phenomena we spend for between phase one to four from One Source about $60

million on a particular trial on one trial it’s it’s an enormous amount

that’s being spent what’s driving it what are some of the kind of the the moral hazards or freakonomic elements

that that make it expensive or that might surprise people Craig you know I I

um I love the book free economics I love the podcast free economics and then it was a couple of years well more than a

few years ago Todd Park was the chief technology officer for the Department of

Health and Human Services in the US and I saw him speak in an event and he um he

shared this perspective he said uh the US Health Care system isn’t broken it

operates exactly the way our misaligned economic incentives are set up for it to

operate and he went on with a number of examples and I just sat there thinking to myself the same is true for so much

in clinical research um are things truly broken or do we just have a lot of

aligned economic incentives you know one example that I think many have become

aware of and try to modify through creative Contracting models has been our model of partnering with cro contract

research organizations and for those that aren’t familiar with this field C

are are now an essential partner to pharmaceutical and biotech companies and

how they conduct and execute the studies they’re bringing a lot of the people and often the process as well to get the

study to happen in terms of monitoring and oversight and handling of data now

traditionally contracts between sponsors in a cro were fueled by the number of

sites the number of months the number of patients these variables would drive the

total contract value and so all of a sudden you had a vendor a partner who’s

financially incentivized based on complexity you know longer studies

studies with lots of measur me ments were kind of good for the bottom line theoretically for a cro now fast forward

to today and a lot of sponsors now conduct more performance-based uh

Contracting models and try to take a little more shared risk with their cros but historically there was a clearly

misaligned economic incentive between a key vendor and a key source of of cost

for studies and the research sponsors themselves I’m gonna ask you a a well I

guess a harder question because it’s going to ask you to draw upon what’s been done well so I I’ve been pointing

out trials are expensive they’re complicated would you put your own child in I’m curious what have people done

well to fix some of these misalignments are there a couple of interesting examples that you’ve seen where an

organization an institution a person for that matter has seen this misalignment

this the some of the moral hazards even that that work and they did something to innovate around it any couple examples

maybe that stand out for you Craig so here’s my favorite example for the year 2023 um so one of the most significant

misaligned economic incentives has to do with treating Physicians the doctors that are taking care of the patient you

know in most cases for a patient to learn about a clinical trial from the

person they trust most the doctor in their life in order for that to happen you need s different

that your doctor happens to be an investigator in the trial that happens to be right for you and short of that we

have a health system in the United States that actually would disincentivize your doctor from talking

to you about a clinical trial because if they aren’t the investigator then they’re probably sending you away to

another provider and in our country in the US providers are are paid on doing

work they’re paid based on rvus these relative value units that are are based

on the time they’re spending with you sending you to another doctor in their field that happens to be across town

across the street bad for business misaligned economic incentive now fast forward today and there’s actually some

interesting fixes that are starting to emerge so back in May the FDA introduced

draft guidance on decentralized clinical trials a decentralized clinical trial

all this means is that we can enable the patient to participate in some of the visits outside of a traditional research

site now great there was a lot of um momentum around decentralizing research during the pandemic as the pandemic

receded and the FDA emergency guidance went away the FDA was very quick to get

this decentralized trials guidance out to the community but they included in

that guidance some really interesting language that we’ve never seen before they talk about a role of the hcp the

healthc care provider in a clinical trial we’ve kind of historically ignored that this person exists we talked about

the participant we talk about the investigator but if you’re a patient with a health condition you have a

doctor in your life that hcp and what the FDA did which was very cool in this

draft guidance was they said if there are routine care activities or routine

care procedures that are happening in this study then in ordinary hcp not the

study investigator can perform those maybe it’s a routine care blood draw maybe it’s a routine care uh collection

of physiologic data there’s really a host of things that that might include it could even for a cancer trial be the

chemo the routine care chemo that you might still be receiving and what that did is it kind of fixed a broken

incentive because now that Community physician doesn’t have to send the patient away and lose revenue and have

the misaligned incentive but instead can keep the patient in their practice keep treating them and have them participate

in research and it sounds like that’s a pretty big disruption for kind of what’s

been done in the past how did anybody resist this is was someone trying to pull back the the misalignment because

it benefited them just out of curiosity it’s a great question right um now one

would wonder will existing investigators and investing existing investigator sites Embrace this or does it provide

some sort of fear or trapid theoretically some of the revenue of a

routine care procedure that might have gone to a research site is now going to

the treating physician that Community provider but I think investigators are right when they’re raising their hand

saying I still have a responsibility here in terms of oversight for the patient data Integrity patient safety

and I have to be compensated fairly by the research sponsor for that work and

so it could result if sponsors deploy this wrong it will marginalize the

investigator it’ll make them feel like their cheese is getting moved and that money is being taken from them to be

given to the hcp when in fact maybe some revenue is but we as sponsors need to

make sure we’re keeping those investigators whole for the responsibilities that they do retain

around oversight of the data and oversight of these interactions Dan the

uh the FDA A’s comment period closed on the draft guidance I think they’ve received close to 900 comments from the

public on that draft guidance and a lot of them lean in on this very topic so I know The Regulators are very busy

consuming that feedback and making this this guidance even stronger now I I know

that uh you know just little bit of a of a of a pivot um that the work being done

right now and the feedback commentary from the FDA and the public is not done in a vacuum of what else is happening in

the world and there’s no doubt that you’re seeing what I see and what many

other people are seeing around the explosion around artificial intelligence generative AI chat GPT um but other

forms of AI how does that play into the future of clinical trials um there

there’s something I came across called digital twinning which I had no idea what that was and for those who don’t

because I think most people won’t it’s this idea of having a a virtual representation of yourself based on your

historical genetic and Health Data and they’d have that person theoretically go

through in real time through a clinical trial and they would use that in

exchange or in substitution of a real human being um how is AI coming to play

are things like this which feel really futuristic and and actually even a little worrisome at times are they going

to change or are they changing the game already clinical trials I think it’s a

great question and it certainly gets a lot of attention today because even at a time when industry seems to be

contracting there’s a lot of uh downsizing there’s carving back of it

budgets within a lot of large farma right now and our biotech uh allies out there are struggling as well in terms of

funding and financing in the current economic environment but we still see a lot of interest and momentum around

exploring many of these AI use cases in clinical research and clinical trials

some of them we’ve actually been doing for a number of years and others are a little more new to me when we think

about Ai and clinical trials it starts up front in portfolio decisions and study designs spaces where we’re

consuming a lot of different data and could use a little assistant by our side

when we’re making decisions based on all that diverse data we can start to see AI

come in around study conduct helping us with produc Ive analytics around quality

and oversight for patient safety and data Integrity as data is coming in and being able to trigger different clues

for uh study teams to be able to intervene earlier and certainly AI comes

in when we think about digital measurements those uh that wave of new uh connected devices sensors wearables

and otherwise how can AI start to enable us to have a next generation of ways to

measure change with patients we we also see a lot of AI today around just automating things can we automate data

entry by pulling data from emrs can we automate writing of things with generative AI but you hit on one of my

favorite AI use cases around digital Twins and it’s my favorite because I think it really is one of the most

disruptive right when you think about all the stakeholders involved in clinical research today and many of them

there is some future of obsolescence right like maybe we don’t need this

provider or that type of tool in the future the one stakeholder that kind of would Embrace obsolescence are patience

and trials there’s no patient that aspires to be in a clinical trial if they’re there they’re there out of out

of need urgency necessity do you you see as realistic is it is this uh what’s

your your there’s your gut reaction and then there’s the logical brain kicking in what you know answer from both of

those perspectives what what do you think is that I think it’s very real realistic and I think we’re seeing incredibly Progressive um positions from

Regulators in both Europe as well as the us around these approaches a lot of the

early momentum has started in the neurosciences looking at Alzheimer’s Parkinson’s and other conditions and now

we’re starting to see these approaches start to pull over into other therapeutic areas the basic story is if

I had access to Dan sea wald’s uh historical Medical Data as well as the

medical data of other people with that medical condition could I enroll Dan in

the trial and introduce an intervention while at the same time having a machine

created instance of Dan continueing the trial maybe in in the control arm for

the trial and this idea of having a digital twin it may seem radical and and

new in in this context but digital twins have been used in Aerospace and

Automotive Industries for years uh so it’s a it’s a very exciting field there

is a lot of evidence for using digital twins outside of Life Sciences and

clinical trials the receptivity so far has really been remarkable and what’s really fun here Dan is for sponsors in

this space there’s kind of a low-risk way to build some some chops to build

some learning right you run your clinical trial the way you were going to right enrolling uh people in both the

active and the control arm and then in the background you can run like a digital arm and kind of use that as your

learning in experimentation no risk to your study design right but I’ll tell

you the risk on my mind about digital twins is um My Health Data is just so

accessible right you can go into different data aggregated uh um well not

you as an individual your organization may be purchasing aggregated de identified Health Data and using clever

uh ways to link data across different sources is it possible that my digital twin is getting enrolled into a clinical

trial right now without my knowledge is it possible that my digital twin is being enrolled in a study that actually

I might not view favorably right just as we know that there have been studies that have been designed say studying um

medical conditions among Native American populations that were actually viewed as very hostile to those patient

communities should I always have a say and provide consent if my digital twin

is going to be used in someone else’s research I mean you’re there’s so many points you’ve brought up and I want to I

mean great great Insight on this Craig and so I’m gonna just note folks you can

ask questions I know people are are actually pouring questions in now some people have actually directly messaged

me um so we have a few that are coming in so feel free put those into the chat

and um on LinkedIn and we’ll bring those up but before I seed control to the

audience um I have to ask just on this point around AI it would seem almost

intuitive that that it should drive down the cost of clinical trials but I could

see where you see an inversion sort of the the levit and Dubner esque view on

like the kind of the unintended consequences of new technology that could drive it up and create more

expense more complexity because AI is new and sexy there should be a premium on it what’s your your take will it

actually end up drive up the cost of Trials or do you think you know in the medium term at least it might drive them

down you know we um when we add Innovation and add Innovative approaches

into our trials we add cost the only time that we can actually start to realize the cost savings is when we

commit and start to reduce some of the existing bloat in our trials and that

can be scary for organizations if we are just going to add a third arm in our trial with my digital twin enrolled

we’re adding cost we will reduce cost when we’re willing to burn the boats and

not enroll a human control in that study or enroll fewer human control uh humans

into the control arm of that study that’s scary that is a big step in terms

of commitment it has perceived risk but that’s where we get the return yeah I

I’ll I’ll uh you know just thinking about like with new startup Ventures companies that are getting large

valuations there’s a lot of investment Capital being that’s being pumped in there’s got to be an expectation from

those who are are funding new and kind of curious and adventurous startups they’re going to get a return that

return’s got to come from somewhere so I I wonder um your your perspective like

is it is it a blossoming industry kind of like Ai and new tech and clinical trials or or will it sort of kind of uh

create a reduction that you might see well I think um fortunately in the case

with digital twins there are some other use cases besides just the big ambition

the big impact of reducing the number of patients in a control arm you can use those digital twins to improve your

study design your planning your modeling of a trial beforehand so there are some

quicker wins that may not have quite that tactical immediate application exactly exactly so that you don’t have

to just you know wait for the big bup right um reducing the number of patients

in a trial should clearly bring a a nice cost savings as well as be viewed

favorably by patients having fewer randomized to be in a control arm are

there unintended consequences we still have to be able to access people’s data and we should be doing that in ways that

have permissions attached now those that know me know I actually am also a believer that if you are going to use my

way in a data that is commerci uh if you’re going to use my data in a way that drives some sort of commercial

Venture then I should be able to participate in that upside if you’re using my data and so what’s nonintended

consequence well maybe we’re going to only catalyze Futures which are not bad Futures where those people whose data

we’re using today so easily maybe we need to be licensing that data from me

in almost like college athletes you’re you’re licensing their likeness they’re

able to to benefit I mean if if somebody individually contributes to the next big

breakthrough should they not be enriched for taking that risk um very interesting

propositions I I I like where you’re going with this I’m gonna ask one and a half more questions half a question

that’s my way of uh being non-committal to give you a quick one on this um what when you think about diversity and cl

clinical trials that’s been one of the hot issues that because as just a matter of course you don’t have that

representation you you kind of allude to it before Native American populations other minority groups that have been

hesitant to to participate how does that potentially kind of play in today uh of

being able to kind of creatively address that uh knowing that there are some disincentives out there there’s also

some incentives are being built in to be able to bring a more representative population your studies your thoughts on

that this is such an this has been an important question for years and I think in the post George Floyd era of of

increased awareness appreciation in investment in um social justice concerns

I think has only helped to put an even larger Spotlight on the gaps in knowledge about which treatments work

for which patients and which populations and if our studies just enroll a bunch of people that look look like me then

our evidence is primarily about people like me and so how do we help to improve

representation in our studies so that the people enrolled in the study look like the people to whom the drug will

ultimately be prescribed so one other great action from the FDA this past year

um a mandate around diversity action plans the requirement that for most late

phase studies that there is a plan in place to drive to diversity now some people misconstrue this some people say

the FDA is mandating diversity the FDA is mandating a diversity plan and like

my grandmother used to say a young girl plans and God laughs we make plans but

they don’t always come to fruition there are no carrots and sticks attached to these plans the guidance from the FDA

goes only so far in pointing out that if you have a plan and do not realize its

targets you’ll have to work back with your your review team and may have to do some postm marketing commitments and

that’s not really a good answer because what that means is we do our robust

clinical trial and generate great evidence for people like me and then we have a second tier of evidence maybe

using real world data for people of color or other communities that are not being properly represented and so this

kind of goes back to our freakonomic story Dan because we’ve figured out how to drive the right incentives when we

look at um rare disease research or pediatric research we’ve created

incentive schemes and through legislation and policy that can extend

exclusivity when drug developers do certain things and now as a result rare

disease drug development is a robust industry with the right incentives in place I really think a lot of this

problem is addressable because there are some great tactics and strategies out there they just need the right

commitment behind them and I I I have so many other questions I want to ask you

Craig but uh in all fairness I did say I would take questions from the group um

I’m gonna just uh lob a few over in your direction are you ready for them bring

it all right so the first question that came up was do we see a difference in Physicians referring patients to

clinical trials in health systems with different economic incentives for example the UK’s NHS

or the pmda in uh in Japan I think this is a fabulous

question right it’s hard to just generalize about the state of referrals and some of the barriers that we see and

as this questioner rightly points out our us system of rvus and how we pay for

providers is very different from what we see in many other countries where where

physicians may just be working on a salary as a part of a health system you know the other kind of interesting lines

for us in this country um we have Physicians that are often employed by

specific Health Systems so if you look at say a street corner in California you

might have one doctor on one side of the street that works for Kaiser and look diagonally and that doc works for

scripts and look across the street and that doc works for Sharp are they more incentivized to stimulate referrals

within their health system because that health system has to appreciate that the patient is still staying within and can

create the proper incentives I think another interesting example that we can start to consider in the US that takes a

page from maybe how uh health systems work outside the US is in the us when we

look at federally qualified Health Centers fqhcs which are the safety net infrastructure for healthcare in the US

again here these providers tend often to be salaried not all the time but in many

of those fqc and so it’s a great point that we can often have a little more momentum around

stimulating referrals at least around the um freakonomic the misaligned

economic incentives there are still other emotional barriers that can often

stand in the way of stimulating a referral um for research participation

there can be emotional barriers around feeling like I’m not for some reason capable of providing adequate care for

this patient that I should need to to send them not to a specialist but to a peer of mine that just happens to be an

investigator in the study but that’s often less of an issue than some of these other barriers so it’s a great

point to keep uh to keep us looking at the systems outside of the United States

there is a another question Craig that asked should we not consider having all

trials centralized through government entities or quasi government entities

rather than leaving it to the the opes of individual companies it’s uh that that’s a pretty

yeah that may even get to the political realm of of you know sort of decentralizing or Federalist and non-f

Federalist but but yeah do do share thoughts about that you know I I I I wish I could say that um studies run by

the NIH are radically more efficient or more organized than those in the private

sector but I don’t think we have enough evidence to say that there is such a dramatic difference sometimes the

private companies are able to deploy more Capital than the Federal Government

Can through their Grant models with the NIH and others to be able to drive a development program more aggressively

you know if you look for example at the coid vaccine trials fiser funded their

study entirely on their own madna was heavily uh supported by the federal

government they kind of both started and ended right around the same time with right around the same number of patients

and so there really wasn’t an obvious winner in that race but I’ll give you

two clues of who may be interesting places where where there may be some interesting new places to play some bets

one might be on nonprofits uh in the space now I said

earlier that it’s hard for nonprofits to support drug development because of the cost but we are seeing more and more

ways that clinical research Arch is getting democratized the cost of accessing these Technologies are getting

far more accessible there are some very interesting examples of where organized

patient groups and nonprofits have been able to carry a molecule very far and now we see groups emerging like out of

um out of pen where uh every cure is a really interesting nonprofit to keep an

eye on primarily focusing on drug repurposing using Ai and vast amounts of

data but then also leaned into smarter ways to test and generate evidence about

those medicines I think another clue for uh for the audience as we think about okay if it’s not going to be government

and it’s not going to be Pharma how else will we drive more efficiency well in

Europe uh we saw the role of platform trials and master protocols during the

pandemic and what’s great about this type of study is you know one of the reasons why drug development is so

expensive is we’re deploying a huge amount of infrastructure just for one

trial to answer one question about One Drug in one patient population it’s like

we’re stringing up all of this phone wire all around the country to make one phone call and then at the end we cut it

and we break it all down and what these platform trials let us do is test multiple drugs or potentially multiple

indications in one reusable trial infrastructure of sites and recruitment

efforts and data and who sponsor who sponsors that or or who’s leading the charge on that well that is where it

gets interesting right very often you need some entity to be the neutral convenor to bring these groups together

very often so far it’s been advocacy and nonprofit groups that are sort of bringing groups together the government

has also played a substantial role there and then Pharma tends to come in as a partner in the process and so I I think

that this type of collaborative research is definitely a key part of our future of making research far more efficient

you know I’ll I’ll uh I have to ask you I think it was a number of years ago was it patients like me which um patients

actually self-organized their own trial um without any other or I I don’t know

if there was much interdiction into the design but do you do you recall that and and why is that relevant even now do you

think so for folks that weren’t familiar back then patients like me was one of the one of the very early examples of

patients um in an online community that were actively sharing data about

themselves and their health condition the medicines and other interventions they tried and what outcomes they were

seeing and um heavy density in that community of patients with ALS now there

was a very small study that got published I think it was out of Italy that suggested that patients taking

Lithium had better outcomes with ALS and faster than any investigator could

submit a grant request get funded and design a trial a group of patients there

started to self-experiment and organized themselves and found ways to share data

to answer this question where some of them started to take lithium and others didn’t and that crowd of patients

generated evidence months before any proper clinical trial with an

investigator got off the ground when those investigators finally got their

processes in place and were able to run a study they came up with the same answer doesn’t work wow so it’s

fascinating to see again the power of organized patients using democratized

tools that once upon a time were really almost impossible to access but today

organized patients can find each other and share data in structured ways one

one more question that that came in Craig and then we’re going to play our final speed round of boom or bust so get

ready for that but before that there was a question going back to AI that was asked um who’s going to be policing and

overseeing artificial intelligence when morals and ethics could be in question

now I I’m going to interpret that a little bit that perhaps sometimes um the people are hosting a trial may have um

different incentives than just pure science and about outcomes but I could be wrong um your your reaction to that

question ah this is such an important question for us right now right we can look at

are there examples of other spaces I have a friend on LinkedIn uh Richie ataru who publishes very often on this

topic that um perhaps we need um almost like we have uh you know other public

scales for me to monitor my credit report or other forms of transparency do

we need rating scales for the public to be able to understand how an AI

algorithm was developed what was the diversity and representation of the data um how do I have trust and confidence in

these tools and uh depending on when this drops for your viewers there was uh

very recent activity in the white house here uh in the US with the president um

uh taking some action and starting down this process in this country but I I

appreciate that this is a really challenging question that’s being raised um one could try to police themselves

whether within a specific vertical as an industry as a country but in a very

Global and connected space that doesn’t stop um Mal maled tools from getting

developed in other pockets of the world and actually I would uh refer

anyone to listen to to S Matthew Lao from New York University we had him on

in a previous episode talking about the ethics of AI and the the notion of

clinical trials did come up and it’s uh it’s it’s a whole another topic for more

than an hour um there was one more question that was slipped in Craig that I just want to throw out because I think

it’s it’s a really good one the last few years have been sort of focused on decentral Trials to diversity what do

you think the next big thing is what’s the next big topic that’s on the horizon you know um it is true that our

industry definitely gravitates to something else every uh every couple of

years we are very cyclic um and you know if you think about the meme of uh the

the guy walking with his girlfriend and there’s another girl in a red dress walking by you know who’s getting his

attention it probably is AI right now and that’s not necessarily a bad thing

but I think what our industry what our community needs to get better at doing

is is um finishing what we started and that doesn’t mean that we can’t do both

at the same time it doesn’t mean wait on all things AI until we’ve addressed diversity right we we can do more than

one thing at a time it’s just this year is making it a little harder than normal

to do that as we’re swinging towards smaller organizations leaner organizations I think the real challenge

for us isn’t what’s the next shiny object it’s how do we maintain

existing progress while starting to identify details around that object I I

love your your point on that it is so hard and we know like Atomic habits the

power of behaviors it is so hard to see things through it’s easy to be attracted

to that next shiny object but seeing it through and doing it in parallel on the next big thing that’s tricky because you

need more people more resources um well we could talk more about this but it’s

time Craig to talk about booms and busts now just to give some quick context I um

selected at semi- random a number of emerging or new technologies that are

being used in clinical trials or or adjacent to clinical trials even and I’m

gonna ask you Craig your reaction do you think this will be a boom next big thing

or it’ll be a bust the err is let out it doesn’t really have a big or substantive impact

um for the betterment of Trials you ready all right let’s go I’ll give you

like 30 seconds to 45 seconds just to elaborate on your answer also because I do want to know what you think other

than just your answer all right first one the retail pharmacy model in

clinical trials the retail pharmacy model in clinical trials Boomer bust um I have to uh unfortunately say

it’s a near-term bust um mostly because I think it’s just taking time to get the

right players in the right spaces uh it’s very hard for these large massive

uh chains and organizations where clinical research is is kind of a small business for them and so where does this

fit within a CVS will it remain of Interest inside of a Walmart I think that there is a huge amount of potential

in terms of those locations playing an impact I think that over the long term I

think this is going to be real but in in the near term it’s still a little bit lumpy and um fortunately we still have a

couple of players like Walgreens that are committed and fortunately a number of sponsors that are still trying to

learn and make that work I’m looking forward to seeing more success stories yeah I still recall the excitement of

the minute clinic and how it was going to revolutionize primary care and it

hasn’t quite yet so maybe maybe a bust I’m I’m with you I’m voting with you on

that all right next one Craig blockchain that’s kind of a big beast blockchain is

that going to revolutionized clinical trials or partially revolutionized clinical trials is that a boom or a

bust I um there’s some use cases around blockchain that I really love in general I’m going to call blockchain a bust um

and I’m gonna call it a bust because I think was expecting that uh well I think that it’s a I think it’s a space where

some some really um hardcore evangelists around the technology ran around too

much just trying to fit blockchain in everywhere here is a blockchain enabled X or a blockchain enabled Y and we would

look at it and say well what why did that have to go on a blockchain just because it could did that actually solve

a problem now there are some really good use cases for blockchain around data

ownership around data um uh around traceability of data back to individuals

and some of is going to link right back to what we were talking about earlier transparency permission awareness maybe

even participation if my data is being monetized I love blockchain for that

blockchain overall I don’t think is going to be gamechanging but blockchain for some very specific and interesting

opportunities I’m all in all right it’s it’s often people see as a solution in

search of problems which there’s nothing wrong with because you but you got to find those problems sounds like there’s

still some miles to go um here’s one that sounds very buzzwordy and I’m going

to throw it out to you maybe you can even explain what it means clinical trial

tokenization what the heck is that and is that it’s funny to hear it like right

after blockchain because very often I think when people hear token they think a blockchain token like Bitcoin but in

this case when we talk about a trial token this the idea here is we can protect people’s privacy but we can

still connect their data across different data sets where they’re otherwise deidentified and Anonymous so

maybe there’s an aggregated data about electronic health records here and maybe there’s an aggregated data that’s also

deidentified with consumer purchases over there how can I link those with a

privacy preserving token now what I love about this so I’m going to call this a win I’m going to say call this a boom

and I’m going to call it a boom because this is one of our key Solutions ions for being able to bring more real world

data into clinical trials right the the data whether from emrs from claims from

other sources because we can connect it we can take data that’s otherwise

deidentified and connect it to a patient in my trial and what I love about it is

we do it in a permissioned way this isn’t a creepy behind the scenes trolling through your data this is with

informed consent with an Engaged patient giving permission for their data to be connected so let’s make that a boom all

right you’re you’re duly noted it’s a boom all right this is one I’m I’m actually really interested in corporate

data collectives or or in other words when large pharmaceutical companies or or other life science companies will

pull their data together um and then make it available to everyone in that

Collective and I’ll just note you said something I I love your metaphor about you know putting up the wires for one

phone call it’s it’s such a great metaphor um is this the solution to that

one phone call wiring system and is it a boomer bust by the way um I will give it

a conditional boom because there is a a step that’s missing from some of those

data sharing collaboratives that I think will make them realize their full potential um so these data sharing

collaborations some of them are really potent um within transcelerate which is a nonprofit organization of

pharmaceutical companies there’s an initiative called data celerate and one of the seed studies from that actually

our some of our friends at fiser had gotten off the ground originally how can we share data from the control arms of

our of our trials right they don’t reveal anything about our investigational medicine it’s just really good structured data that should

be doing more good in the ecosystem and getting reused now the the the the one

last step that’s missing to make that a boom is it can’t just be

among the club of big organizations that put data in how do we enable academic

researchers and the broader research Community to fully leverage that data and that’s when it’s going to create

boom worthiness right it’s so well it’s clean it’s organized it’s structured

it’s a beautiful data set that we create from our research studies to reuse we just have to open it up to really let

the vast research Community take advantage I mean it sounds uh like creating truly creating an ecosystem

where multiple entities can plug into it um that’s I agree and there’s and

there’s gateways we’ve already created as an industry look at uh vivly as a great example of a of an infrastructure

that the uh industry and Academia have worked together to create to give us a

secure kind of Gateway that researchers can come in and access vast amounts of data we need to open up more of that all

right penultimate one Craig is is n of one individualized genomic trials is

that going to be a boom or a bust or maybe somewhere in between actually your

thoughts you know I I love as we said earlier that so many research tools are

democratized and available for individuals I don’t really know where true n of one are is going to go in

terms of really being maximally impactful so I’m going to call n of one

a bust and this is a term I’m just making up right now with your question Dan um I’m going to go with n of sum so

instead of just n of one how do we find some other people that are like me and use those platforms to generate research

data together right and use those same high quality tools and diverse data so

that it’s more than just one but now I’m with others and I can create that that

impact of n of sum in a very democratized way and I really think that n of some is going to be boom worthy I

think that that’s going to start to show us how crowds of people could become the next pharmaceutical company I I love the

N of sum I haven’t heard that before so you better quickly run out and get a copyright on that one yeah I’m gonna go

grab the domain first thing yeah yeah ex someone already did as you were talking true um last one which is very connected

to the previous is the idea of biohacking and I remember talking to a

couple of NIH researchers about biohack hackers that were Paving the way to give

Insight research of stuff they haven’t done by doing some biohacking thoughts

on that is that a boom as a movement or is that a bust and is that have any impact at all on Trials from your

perspective I think that hardcore biohacking is is is a bust and some of

it can get a little dangerous and Reckless I think that there’s things that people in garages have tried doing

with crisper and with themselves that can be absolutely terrifying right but

do you need an Ethics Committee and an IRB oversight if I am screwing around in my garage and injecting myself I mean I

this is um I mean this is you know an autonomous EOS system and I

should be empowered to do the things that I think are right for me as long as I’m properly informed but I think that

the biohacking movement is really the seed of crowds of people democratizing

in research and I think it is it is absolutely the boom that uh that this industry needs it will keep industry on

its toes it will keep industry honest it will apply positive pressure for industry because crowds of people can

start to do the types of research that we’ve typically relied on industry to deliver and people patients don’t have

to wait anymore right they don’t have to sit idly by and hope and pray that Pharma takes notice in and their disease

there’s more and more opportunities for them to be empowered and start to work together now again that doesn’t mean

hopefully to just go ordering supplies on the internet and uh whipping up uh something in your blender at home but to

answer other types of research questions in a democratized way I am all in wow I

I mean I am ready to go down to my garage a little after one o’clock and do a little bit of Gene editing um but

that’s great I I appreciate your input on that and all of the boom and bus um

Craig we just have a couple of minutes so I want to kind of land on one last thing um the name of the the podcast is

called the deliberate way and I use that term not just as some catchphrase really do believe that the greatest innovators

the greatest minds are very systematic or deliberate in how they do things

you’re a very deliberate person in what you do also if you were to give some

advice two or three tips to somebody who is maybe entering the space of clinical

trials or it’s been working for a little while in the space of clinical trials it needs to be a bit more deliberate in how

they innovate in this space What couple of tidbits of advice or recommendations might you give to them uh the the friend

who hired me first at fizer gave me this one piece of advice he said never put the word Innovation on your business

card and I I think uh I think what he meant by that yeah exactly right clearly

you and I defied that and we outlasted him but the um I think the message there

was is that there there are folks that will look at it and say well I innovate

too don’t we all innovate are you suggesting that you do Innovation and

then what does that mean that I do and I think we have to keep reminding folks that just as you’re doing here every day

Dan that Innovation is a process it requires attention and focus it’s not random that’s not how we necessarily

create Innovation we we have to to as you say in your in your name it it is deliberate it’s intentional and there’s

there are proven processes to help make that work if you have Innovation on your

title in your organization you’re not there to be the innovator you’re there

to empower others by helping them to understand and bring process and uh

deliberateness uh to the way that to the opportunities in front of them I I love that that is a great recommendation and

I’m looking at the time and we are out of time Craig we are out of time but first things first Craig I want to say

big thank you to you for your wisdom your sharing your stories and examples it was amazing and then secondly we will

catch you guys on the bounce for our next episode of the deliberate way we’ll be coming back in the next few weeks

with another exciting topic but in the meanwhile check out Craig you could be one of many the legion that follow Craig

on LinkedIn reach out and also we’ll have replays and videos if you like the

specific segments it’ll be out there for you and once again Craig big thank you for me and uh and from all of our

listeners and viewers so thanks again Craig thank you Dan for these great

conversations awesome all right with that over and out and we will see you

all at the next episode or the next time we talk about deliberate Innovation take

care everyone