Accelerating Clinical Trial Recruitment with Artificial Intelligence and Natural Language Processing
The convergence of AI and electronic medical record data is happening now. HCOs and sponsors are mining real-world, clinical data at scale to precision match patients to clinical trials.
Watch this “Note to File” podcast to hear thought leaders, Brad Hightower, Daniel Fox, MPH, Ph.D., and Deep 6 AI’s VP of HCOs, Doug Cassidy, discuss using AI to improve trial feasibility, accelerate recruitment, and reduce site burden.
Video transcript
Brad Hightower:
And I am muted. There we go. Perfect way to start the show. It is Tuesday noon Eastern. Welcome everyone to another “Note to File” live. All right guys. Mr. Dr. Fox, good to see you, man. How you doing? I think you can unmute yourself, but there you go. Sorry.
Dr. Daniel Fox:
Oh, there you go. Sorry, my silence.
Brad Hightower:
Silence my people up here until the radio speak.
Dr. Daniel Fox:
Silence. There will be silence. Hey Brad, how’s it going? Happy Tuesday
Brad Hightower:
Indeed. And Doug, my man. How’s it going?
Doug Cassidy:
It’s going awesome. Looking forward to this. Thanks for having me on.
Brad Hightower:
Yeah, man. Always excited. This was highly anticipated. A lot of people signing up and jumping on, so very cool man. We’re going to jump right in. Doug, tell us a little bit about yourself and your clinical trial journey. I like to hear people’s origin stories.
Doug Cassidy:
Yeah, I got a weird one, kind of how I ended up here. It makes sense the navigation of it and then it’s kind of how I ended up at Deep 6 AI. So I grew up in Ohio and I ended up going to a school down in Dayton called Wright State. Wright State is adjacent to the Wright Patt Air Force Base. So, I started going down there before I graduated high school. It was kind of a trek and doing some programs and one of the things that I got onto was this program of converting the Air Force’s Air Missile Defense Code, which is basically pattern recognition to fly missiles into things. We converted that into a way to find cancer in the breast. So the first thing I ever did right out of high school was scanning mammograms, basically turning them into an image and then every Cartesian coordinate had a pixel density of zero to a million and we would basically run linear algebra on it to try to find cancer in the breast.
And that was my first clinical trials journey. We that thing through FDA and I got hooked. So, I’ve been clinically adjacent, I think 20, that was 28 years ago or something like that, so it’s been quite a long time. My navigation then took me from Dayton back to Cleveland to New Zealand where I was one of the few guys that helped bring a company called Orion Health, which is a leader in interoperability to put our beachhead in the U.S. That’s how I ended up going from Ohio to Los Angeles and really it was super exciting building that from a team of five people to a 200-person company. I moved to G.E. after that and was running the West Coast for EMRs and a little bit after that got connected back to Highland Software in Cleveland.
So I was actually living in Los Angeles working for a Cleveland company and then kind of went out on my own. I had a consulting business and one of the things I was doing is there is a large healthcare IT company in Italy called Dataless. I was helping them penetrate the U.S. market with interoperability EMPI. And at that time I was talking to several different people and I got connected with Wout, so Wout’s, the CEO of Deep 6 AI and he showed me what he had been building at the Cedar-Sinai Accelerator in Los Angeles and they didn’t have a position for me, they didn’t have the budget to hire me and I just bulldozed my way into Deep 6 AI. I got lucky. I got in there very, very early. I think at the time I was the fifth person, so my first chair at Deep 6 AI was a pile of printer paper in a small office and it’s been a wild ride since then.
Fighting and clawing and growing, living through the pandemic during Covid. I moved back to Ohio from Los Angeles after spending about 15 years there and that’s where I’m now. For people that are dialing in that have heard about Deep 6 AI or they pop up or somebody’s evaluating Deep 6 AI or it’s in a white paper or something, I thought it would be maybe useful. I put together a couple slides, here’s the primer. If you had to know anything about Deep 6 AI, this is what you got to know. And then maybe that’ll kind of level set doing some awesome stuff so we can maybe start the conversation. Then what I’m hoping is we can just launch, maybe not specifically talking about Deep 6 AI, but what problems are people solving? How is AI helping and all that other stuff. Let me see.
Brad Hightower:
Yeah, and I’m really curious as you get that pulled up because we hear AI a lot and fancy stuff like large language models and NLP, and it’s not always clear to me what does it mean? What are we really talking about here? A lot of stuff seems like simple searching, right? And does that mean searching stuff isn’t really AI, right? Where’s the AI and how’s it actually being utilized?
Doug Cassidy:
It’s a catchall, right? Is AI a chatbot? Is AI probabilistic anticipation of the next outcome? Is it disease prediction? A lot of it is image processing. So Deep 6 is two things when we talk about AI, and this is my summarization of it, it’s number one, it’s using artificial intelligence to try to find clinical synonyms for the clinical evidence that you’re trying to find. Clinical data is extremely messy.
Brad Hightower:
To say the least.
Doug Cassidy:
I hate to break it to you. And physicians, their job is to document what is going on, not to document it the way somebody in the future wants to find it. And so number one is if somebody’s asking for a clinical concept, I don’t care where it’s located, I don’t care how it’s represented, I just want to find it. I don’t care if prostate cancer is written as a Gleason score or if it’s written as “PC” or if it’s written as prostate cancer, I just want to find it.
So, we’re really two primary things and a bunch of things under these umbrellas. Number one, we really started out as what if somebody decided to build something for the researchers first, at the sites? Not that there’s anything wrong with it, but most places start with sponsors and industry and then go to the sites from there instead. What if we built something for these people that are way overburdened? So, our first thing that we built was this product and platform which basically ingests all of the data from the client site, which is typically an HCO, we call it healthcare organization, which is academic medical center or IDN or freestanding hospital. Somebody that cares about research and we ingest everything and we steward that data for the site in their own instance. So that data does not commingle with anyone else nor can we do anything with it other than doing something at their behest.
So, we are not selling their data, we are not keeping a copy for ourselves. This is their data that we are providing value to them basically getting their data, organizing that data and allowing them to have a conversation with that data with our AI.
The second part of that is what if we could get the study sponsors that are desperately trying to find the sites and help those sites relieve their burden of how overworked they are to provide them with a way that everybody might be asking the same clinical question. So, it’s these two different things where the sites find the patients and then the sponsors can find the sites. It was really, we wouldn’t be where we are without Cedar-Sinai. So we started in Techstars in the spring of 2016 and they really gave us a lot of direction and I think the more important thing is they gave us access to a ton of physicians that helped us shape what we built and they were our first client and they still are a client and they’re awesome.
And over these now almost seven years, we’re really excited that we’re going to have about 28 healthcare organizations on our platform, which when they’re all live, that’s going to give access to about 57 million patient lives available for study, which is just a gigantic set of patients and that’s going to accelerate therapies into their bodies or get them access to a physician for a breakthrough procedure or get them access to a new device or something like that.
About a year and a half ago we launched our life sciences division because it doesn’t make sense that if a sponsor is running that site, that study at 40 sites, that each one of these sites is building something different. What if we could build a platform where the author of that study could build and publish that inclusion and exclusion criteria once and then push it out? So, what we found is these sites that are receiving these studies through our platform are really turning into what we call high performance clinical trial sites.
So, the first thing is if anybody’s been out there is somebody says, “Hey, do you have enough patients for X?”. And then you’ve got to figure out what the person’s really asking for, and then you have to ask your IT department to maybe run a query, and then you have to have some staff time to look through all the records, and read everything and try to say, yeah, I think we got about this many patients. We’ve got that down to near real time where the AI can take that clinical question, get that answered almost immediately so those high-performance clinical trial sites can say, yeah, this is our feasibility count. We think we’re a good fit for this study. The sponsors or whomever say this is a great thing, we should do a study together. Everybody agrees. And then when that study is published to the site, they do some awesome stuff.
They recruit that first patient faster, so they prove that they’re going to be able to do this study overall their accrual is better. They’re able to communicate better with the sponsors and with each other. It’s this great tool connecting all the stakeholders in clinical research. So how we do this is this sort of suite of tools that we’ve developed.
So primarily is our sort of flagship product is what we call “cohort builder”. I will tell you we are the least creative company on names like “Cohort Builder” is cohort builder, “trial recommender” is trial recommender. We didn’t give anything fun names, but it is the thing that will ask questions of the clinical data and get your answers back in almost real time. And when we ask questions of the data, we are asking it of the structured data or the unstructured data and that by that I mean physician notes or pathology reports or whatever.
So, that’ll help you find patients for trials. Conversely, what about when the patient is sitting in clinic? Because every study that I’ve read said the way that a patient wants to hear about a study is from physician. So, we created trial recommender. While I’m in clinic with my physician, that person has available to them all of the studies at our site that have been built so that I can match them to that study. If you have Epic, we have a thing called embedded trial recommender where actually it’s embedded inside of Epic, so it’s really good. So I can know —
Brad Hightower:
We should talk about that later. I didn’t know you guys had a nice Epic add in. Yeah —
Dr. Daniel Fox:
I’ve been following that as well and I think that this is where if a physician is in front of their EMR a little query comes up and says, Hey, your patient may be qualified for this trial, you want to talk to them about it, right? Is that the product?
Doug Cassidy:
I mean ideally, but that doesn’t seem like the way the world actually works. So usually what happens is I’m seeing 7 million patients this week. I do not have time while I’m sitting with them to review all the patient evidence. So every HCO is going to operate differently. Some PIs are very involved with the software, some of them don’t have time, some don’t use a computer.
So what we’ve done is we augmented how we’re deploying this software with customer success and what we call clinical application specialists because we’ve got to meet people where they work or they’re not going to use the software. About a year into us doing this, we’re so proud of our software, it gets in people’s hands and we realize we’re not selling software, we’re selling change management because if people don’t use the software, it just doesn’t happen. And you get about eight, nine minutes in front of a clinician whose day is bonkers and if you don’t impress them instantly, you’re not going to get to talk to them for another 10 months.
So we’ve got to meet them where they work. For some places they download their whole schedule and we do a bulk trial recommender every Monday morning, here’s everybody coming in this week, here’s all the studies they match. That way when the physician goes in and sits with the patient, it’s already primed. Maybe the support staff, the CRCs have already gone through and validated, said they matched five studies. This is the one you should talk to them about. At some of these places, they don’t have a billing code for talking to somebody about a clinical trial and they got to see somebody in minutes, right?
Brad Hightower:
No, even that is beautiful. I think I’ve played with some of these different systems. Having a list of patients is great, but having a list of patients who’s coming in, who are coming in the next two weeks is 10 times as powerful, right? Because I think you can get in front of that patient, you can get with that doctor. It’s sort of that last mile. Having the data is one thing, but then actually being able to act on it in a meaningful way is really the crux of it. Being able to have that data is pretty huge.
Doug Cassidy:
My whole family goes to the Cleveland Clinic and I get a call from the Cleveland Clinic. I don’t know if it’s for me. I don’t know if it’s for my kids or my wife or whatever. I don’t know if they’re talking to me about a clinical trial. So it’s this cold calling thing, which is a little bit disconnected that I don’t know why somebody’s reaching out to me for a clinical trial.
But if I’m sitting with my physician who knows I have a relationship with and they go like, you know what? You actually would be eligible for this study and here’s why, that would be, that’s a much more positive way for me to hear about a study instead of just a random person on the phone and I don’t even know why they’re calling or I might not take it because I’m in a meeting or something.
So then we found the perfect patient. The physician told him about the study or somebody reached out to them and found them. Now what? Well, we realized it realized we needed a recruit module.
The recruit module walks whoever the person is talking to that patient or trying to recruit them through the process and it does a couple of really great things. It records everything in the process. Did you get a reference? Because if you click on that patient it says here’s their primary care physician, here’s their email, here’s their phone number, you should call and get permission to talk to them. You do that, you’re done with that, then it has screening, then you can do all the other stuff that you can do and we’re recording the progress every step along the way because say that there’s 200 patients and you only recruit two, I want to know why they fell out of the queue and at what part and record that information. That’s very positive when you’re done and that patient does get recruited, then you got the connectivity with the CTMS on the other side when you recruit them, push them over to the CTMSs and then they can do their stuff. So you’ve got that real connection.
All of that information about the status of the recruitment is available from the Deep 6 ecosystem. I told you if I’m the global PI and I help assist in this building, this study and I publish it, it would be really awesome if I could see, hey, how are all my sites recruiting? Because everybody at a site knows you get called three times a week. “How’s it going, how’s it going, how’s it going?” It’s great to publish that information, which then said, well, what if we added more texture, more detail to that.
So, we got this dashboard and the dashboard is all of your performance metrics. Who’s logging in? Why aren’t they looking for patients? Hey, there’s 200 patients for this study. Nobody’s reviewed it. There probably is a reason there because when we’re trying to attack this problem, we’re trying to solve the solution, not just prove that the software works.
A big part of what we do is make sure that people are saying, Hey, you haven’t reviewed these patients for this study or whatever the other reason is, and that’s I think the strongest thing that we have is our customer success department has a real relationship with all of our sites. They’re talking to them once or twice a week. They’re reviewing their studies with them and the goal is actually getting people on study.
So we’re constantly building things. All these things are maturing, but this is kind of the core offering we have connecting all those stakeholders and research. The patients get offered studies, the sites get to find the patients, the sponsors get to know what’s going on, and we’re actually getting people on to study.
The reason it works so well is this problem, and you’ve probably heard these stats right now, about 80% of the data being captured about a patient is unstructured and about 90% of the data for why a patient would be eligible for a study just from data comes from the unstructured.
And I was at a conference I was working at actually we’re having a 10th anniversary, it’s called CI4CC, shout out to those guys, we’re having a conference in about two weeks. And I asked the question, should we be building EMRs that restrict the way that physicians document so all the data is perfect, or do we realize that’s never going to happen and just keep investing in making better NLP? And you can guess what the answer was. Physicians are going to document the way that they want to document because they have a reason to do so, and we’ve just got to be able to pull all that information out.
So, we’re talking about the two different kinds of AI now we’ve got all this unstructured data complimented with the structured data, this subjective data and the objective data. If I’m asking for something like a neoplasm and I am say a coordinator that’s three years out of school, maybe I haven’t memorized all the ICD codes and CPTs and I don’t know all the taxonomies and ontologies for clinical language, wouldn’t it be great if I could just type in the word neoplasm? And what Deep 6 AI can do is then go through and find all of those different synonyms for how these things could be represented in the note. And I don’t care if it’s something as simple as breast cancer or something as complicated as triple negative breast cancer. That’s a great example. We found something crazy like over 300,000 different permutations of how triple negative breast cancer can be represented in the note.
It could be triple negative breast cancer, metastatic triple negative breast cancer, ER PR HER2 negative or in Epic. The dash is a bullet point. So does that dash mean negative ER or does that mean it’s the bullet point? So we’ve had to figure out how to do all that stuff.
And then on top of it, this is the second part of the AI is the natural language processing. So that’s going to tell us how human language affects what the meaning of the word is. Okay, so if I just did a keyword search, I log into Epic, I do slicer dicer, I search for the phrase breast cancer. Here’s an example of me finding two occurrences of the phrase breast cancer. What natural language processing does is it gives context to the meaning because there’s a huge difference between diagnosed with breast cancer and patient’s sister has breast cancer.
They mean two different things. So if you think about this, not only is it working well with precision, but if you ask a question of our gigantic network and you’re just trying to get accounts back, we will be able to mine all of the structured data and all of the unstructured data to say, is this a great place to run this study? Or if you were at that site and you’re an investigator who’s maybe going to run an IIT trial, you can have a real-time conversation with that data and get the answers back with some confidence that’s like, okay, here’s my viable population. More than that. Say you’re designing a study and you’re like, whoa, that piece of criteria just eliminate a huge amount of my patient population. Do I really need that in there? Is that the most important thing? We’ve done some other things too.
I read a study three or four years ago that had the most bonkers criteria and it says the patient has to have access to an automobile. Well, we did a look and that was eliminating a whole demographic, right? Because now we’re talking about DEI, are we actually making these studies available to the people who need it regardless of their socioeconomic standing? So being able to talk to the data and see what the results are in real time as well. Designing the studies is huge. So I didn’t want to do a whole PowerPoint presentation, but that’s kind of what I captured. I hope that gives a better baseline. So you guys understand sort of how Deep 6 works.
Brad Hightower:
Yeah, I mean I love that you guys have at least put some thoughtfulness into just not just mining and producing data, but creating workflow or having workflow because I mean, that’s been a big complaint I’ve had about a lot of the solutions that are out there is they don’t really fit into what a site’s workflow might look like or they have no process around them per se. It’s more, okay, well here’s a hundred patients who match your criteria. Well, okay now, right? Yeah.
Doug Cassidy:
It’s a delicate balance because we want to fit you how you work, but over time we want to not nudge you to a better workflow. And so you have to balance those two things because when we’re trying to be a change agent in how we’re changing clinical trials, we don’t just want to be a better piece of software that sits in your workflow. We also say, Hey, by the way, if you changed it slightly or you did this, you’re actually going to get better results. But as you know, some of these hospitals are not only very invested in their workflow, they spent years developing it, and there’s a little bit of pride of ownership and it’s working and who are we to come in out of nowhere and just say, do exactly what we say and do it. So we show them that this is going to impact their current workflow and then over time show them that, hey, by the way, evolve.
Brad Hightower:
A little bit, evolve. And that’s fair. I think that’s a challenge for sites especially at least in the beginning where it’s one thing to add another piece, another tool, but then to also, like you said, you are change management essentially. I can really sympathize with that because it feels that way. We’re looking at new opportunities and new software. It’s like, well, crap, how are we going to have to shoehorn this into what we do in a funky way that maybe doesn’t really make sense, but —
Doug Cassidy:
You’re a nurse, you’re working overtime, you’re covering other people’s gaps, probably on top of that, you really, really care and you want to do the best job possible. And then every week you hear about a new tool that somebody bought that you weren’t involved in. So naturally you’re putting up your Dukes and you’re like, Hey man, I got to get my job done first and foremost. If this thing doesn’t impact me positively immediately, I don’t got time. And that’s where they have that. Shelfware. I don’t want to be shelfware.
Dr. Daniel Fox:
Where over the course of a year being a director of a clinical research site, you are approached by, I don’t know how many EMR scrubbers, I mean it’s just not another EMR scrubber. But importantly, I think with what you just presented is it’s almost like it’s not fully robotic. It’s not all about the AI, but it’s about the relationships. That customer success component of the Deep 6 equation is how you can integrate the power of AI into the needs of the site.
Brad Hightower:
Well, yeah, I think Doug hit it on the nail on the head. I don’t hear a lot of people sort of appreciate or even recognize that they don’t want to cold call from me. I’m just some guy. I’m some guy. They don’t know who they are. They’ve never heard of me. They’re saying, who gave you my information? Who are you? Why do you know that I have migraines? Or why do you know that I have whatever the case may be? And instead to be able to build that workflow in a way, again, I’ve never seen a more successful approach than presenting it to the physician, letting the physician talk to the patient.
But we know when we can be there at this time and they can pass them on to research, everyone’s happy. It works perfectly. But if you don’t have that data in front of you, and we still do this to some degree is manually going through week by week, that’s a pain in the ass, right? If we can cut that funnel down, that’s significant in terms of saving time. Yeah, I call it —
Doug Cassidy:
Give them the bad news early. You don’t have 2000 eligible patients, you’ve got 200. So the precision matching is going to reduce that field. So you don’t make promises you can’t keep. You give people the bad news is going to happen eventually, give them the bad news early. But there is also when we’re tracking the recruitment and having all those dashboards, you get to learn a lot more. I have a story I always tell where this site was basically doing this study that was like, they’re going to puff some gas behind your eyeball and then you have to lie on your face for three hours. By the way, please drive in for three hours to do this, and that’s going to be five days a week and we’ll pay you $25. So when we looked at, yeah, so who’s recruiting better? And the stats show this guy’s not recruiting and this guy’s recruiting.
Well, what happens? We talked to the one person who sold it the way I just sold it to you guys. Come on in. I’ll push some gas behind your eyeball and I’ll give you 25 bucks. Or the other one sits down, is their primary care physician or their specialist going like, listen, this is going to suck. Here’s the value that happens on the other side of this. You’re going to get this and this out of it. So you then get to learn why patients actually get on studies more than just raw numbers. It takes a huge impact.
Brad Hightower:
No, no, I agree. We’ve got a ton of random questions here and I’ve got some of my own for sure. So I will start hitting some of these. I was trying to think where, so I mean I love the idea too that you guys are sort of hitting both ends, which I think is important. You’ve got the site side where it sounds like sites can sort of opt in or use this as their own sort of tool. You’ve also got some sponsors buying in and sort of maybe not pushing down, but at least integrating in with sites that are already using it. That’s where obviously you’re going to get more and more power, more robustness, so to speak, as you sort of get growth from both ends of the spectrum there, which I feel like has also been kind of lacking maybe from other solutions I’ve seen, because I think that can be tough. And one thing I do want to address and ask, because I’ve seen this before, have you seen are systems especially a little hesitant to sort of let this connection happen? Are there things that I run into it with some of the systems I work with where I’m like, I’d love to bring this tool to the table.
Like, oh, well we have our own IT department. I’m like, cool, they’re not going to really help me. They don’t really understand what we do as researchers. And even if they do, it’s going to take six months. The study’s going to be over by the time they get me something. But they’re concerned about the work or the perceived privacy or data issues. Is that something, the liability, talking about liability, is that still prevalent or do we see that getting better as time goes on or
Doug Cassidy:
It’s exactly the same as it always has been. So couple things you’re asking someone to let you steward their data. Oh no, my kids aren’t here and the kids are coming over. This is the last day of summer. So I got kids knocking on the door.
Brad Hightower:
Get them on camera.
Doug Cassidy:
So usually you have to get them to realize that you are the safest place for their data and that’s why they each get their own copy of the data and it is theirs. We are stewarding the data for them, and we’ve really taken that stance and that is very, very important. We’re not getting a copy of their data. We are organizing their data so they can ask questions of it.
The second thing is to address, we are also talking to the smartest people in the world and every single academic medical center has somebody in their labs or in their IT department or in their basement building something that does a little bit of what we do. The problem is I’m trying to sell these people, stop being isolated, that you are building something for the benefit of your singular facility. You are not participating with the greater good. If somebody needs to do a large scale study, they can’t say, okay, I’m going to call hospital X and hospital Z and hospital five. How do you guys do feasibility? Oh, completely different than this site. Am I comparing apples to apples? Oh, you take three weeks to answer the question. You take two days. I don’t know who to pick. So most of the sponsors before this, they got a Rolodex. They call all the same seven sites, they always call et cetera.
What if we could democratize this a little bit? And everybody’s asking equal questions and you know that if you ask a question, everybody answers the question at the same time. They’re all looking at the data the same way. And then the other thing is we are not letting the computer tell us what we found. So the great thing about Deep 6 is we are not transforming the data or normalizing it or mapping it into a finite amount of table fields. We are just accelerating a subject matter expert’s eyeballs to say, look here, is this what you’re looking for?
So if I’m looking for breast cancer and I click on that, that record, it jumps into the source system and highlights that evidence says, is that what you’re looking for? And then the subject matter experts gets to say, yeah, so now you’ve got confidence that a human being who knows what they’re talking about actually validated the clinical evidence.
Because the problem I’ve had with some of these registries at hospitals previous jobs to Deep 6, Dr. X made this registry, Dr. Wild Y doesn’t trust Dr. X. So I don’t even believe in that registry because I think progression is a different thing than that guy. And so at least what we have is when we’re looking for those complex things that are not structured like lines of therapy or regimens or progression, we find it and then a human being goes, that is what I was looking for. And you can also see, hey, by the way, another human being thumbs up this.
Brad Hightower:
Fair. And I know you mentioned it sounds like a lot of your clients or maybe academic medical centers, do you see that that changing or evolving at all as time goes by? I know we work with systems big and small, I work with independent practices. I mean maybe it’s maybe a little less cost effective sometimes if you’re just bringing in some smaller private practices. But again, as you sort of network those together, I think there gets to be more power there. But either way, do you sort of see that mix changing a little bit?
Doug Cassidy:
Yeah, we’re getting interest from large IDNs, the integrated delivery networks that maybe don’t say that clinical trials that are one of their primary focuses. So when we started out at Deep 6 and we’re reaching out to the sites, all of our early sites had some maverick that understood technology, thought that clinical trials was a core part of their being an existence at their hospital, and they said, we’re going to take a risk with this startup. And so over time, we grew our network more and more and more.
Now we’re kind of normalized and we’re almost reaching that point where people are like, is AI part of your strategy for clinical trials or not? What are you looking at? So now people are calling us for the first time over the last year and a half or so before I knocked on every door and I’m like, Hey, we’re going to help you accelerate your clinical trials, et cetera, et cetera. But that’s how it started is academic medical centers really care about clinical trials and the academic medical centers that had some sort of maverick latched onto us and said, we believe in this. We think this is going to work. So we were some of our early clients, we had 10 employees and we’re launching a complete enterprise deployment.
In hindsight, it’s miraculous that we did this, but it really kind of forged us in fire to know how to make it work when we started scaling this thing up. Sure.
Brad Hightower:
I know that someone had answered this in the comments, but I assume you guys are integrating with EMRs aside from Epic, can you speak at all to, I know there’s a lot out there nowadays.
Doug Cassidy:
Yeah, so —
Brad Hightower:
Thank you, Victoria, for the question.
Doug Cassidy:
Yeah, thank you. So Trial Recommender is available to everybody. The only EMR, it’s integrated inside of the window is Epic, but we’ve integrated data with everything, and if you’ve been around clinical data, and I say we’ve done it from Epic to Cerner all the way to Meditech, exactly what I mean, and everything in between Athena Health with ModMed just came out, we’re working with that. It’s like seven or eight. Our early approach that we did, this was kind of brute force, but it made a ton of sense that we went to the back end of the system and we just did a reflection of the backend, so the backend of Cerner, the backend of Clarity at Epic, or had somebody write that query to do a replication so that we could just give them a better organization of their data. What we’ve done since is we’ve leaned heavily into interoperability, HL7, 2.X, we’ve got FHIR, we’ve got stuff in the open Epic store.
We’re working with Cerner clients on CCDAs and HL7, and it is smooth and the data is readily moving and with everybody also making their data more available, we call it our next generation pipeline, it makes our deployment so much faster and it’s very important to us that people decide to go with Deep 6. They want to go live as fast as possible because there’s excitement. Everybody, as you can imagine, a sales cycle at Deep 6 can take two years. So if someone’s been lobbying and fighting for someone to go with Deep 6 at this gigantic institution, when we finally get over the hump, they’re like, alright, when does it go live? We’re like, well, now we got to hunt down your IT department, et cetera, and get the spigots turned on. So yeah, we’ve not run into a system we have not been able to integrate with yet. If I go to your hospital and it’s a hundred percent paper, we’ll have a problem.
Brad Hightower:
Well, you got to come up with a solution for that, man. Come on, bring your scanner and you get it figured out
Doug Cassidy:
Or something and scan them all in.
Brad Hightower:
Okay. Let’s see. Anything else, Fox you see on here that jumps out at you as some questions?
Dr. Daniel Fox:
So admittedly, I am not the programmer. I don’t do a lot of machine learning. I’ve been a sub on some of these studies, more of the clinical expertise, but I know that there’s a lot of technical questions on here that talks about training. Your ML talks about a lot of those things. If you wanted to address one of those, help our techie guys.
Doug Cassidy:
Yeah, when you think about, this is a big distinction of how we’ve made a different approach than everybody else in the industry, I believe at least everybody I’ve ran into is people usually train their NLP for a specific endpoint. The stuff that I used to do is like I’m doing a study for childhood epilepsy, pediatric epilepsy, and I’m going to train that model on a dataset for eight months. I got a hundred patient records, I’m going to rinse and repeat and train that for a specific thing, and when I get it to where it is, then I’m going to feed in 20,000 records. And then there’s a specific endpoint. We can’t do that because there’s too many diseases and there’s too many medications and procedures and diagnoses. So the approach that we took is we’ve basically ingested every kind of medical library we possibly could get our hands on so that we know all the synonyms so that instead of training it for a specific endpoint, we are training it so that we find all the synonyms and then we apply natural language to that.
Because when I work with you tomorrow, you might ask me a different clinical question than I’ve ever been asked. You might ask me about depression, the next guy might ask me about Alzheimer’s. The next person might ask about cardiology. And so we’ve got to be able to be multi therapeutic. And so that approach, we’ve trained and we get better all the time from how the libraries get better. We learn specific things. If the libraries don’t match, I mean we’ve used every available library all the way up to their edges, and then when we hit that edge and it doesn’t do what we want, then we got to build our own stuff for some matching and prioritizing what are the biggest tumors, tumor types and stuff like that. And then for those tumor types, prioritizing what are the treatments and medications for that. So we get really good about the most common things first.
Brad Hightower:
Yeah, no, that makes sense. Here’s an interesting one actually. I would wonder about this myself. Thank you, Samantha. It makes sense how you integrate from site side because the site’s the one who has the data, they’ve got the database. How does it work from the sponsor side? Tell us a bit about how that might work.
Doug Cassidy:
I’ll do my best. So my reason of being is I’m the guy that deploys it to the hospitals. So I’ll give a general thing is someone at the sponsor side says, I am very interested in finding the right sites that have the right populations for me to run this study. And we want to know do they have the available physicians and facilities to run this? So usually what happens is they communicate with our life sciences team. We do a session where we break down what’s the inclusion exclusion criteria. They can then come back and they build their first version of the criteria, they refine it with the study sponsor, and then they can say, okay, out of our X amount of sites, it looks like that if you’re trying to recruit 40 patients at each site, this site has 250 because you need about eight x or something like that.
If you’re actually going to recruit it, we’ve got this many, this many, do you want to proceed? There’s an agreement in place, and here’s where I think it’s the most exciting. We don’t just blindly throw studies at sites. We’ve got account management and customer success, and we also have a life sciences liaison that works with the site side because we’re like, Hey, we are bringing you a very interesting, scientifically interesting study, X, Y, Z. We think you guys are a great fit. It looks like you already have the patient population for this. Would you like to learn more? Then they kind of say, yes, they can lift up the veil. The study sponsor knows what site it is now, and then they can work about going forward, getting that study in front of the right PI. And then everybody’s excited that not only do they have the study prebuilt, a lot of times those first round of eligible patients are already pre lined up and they can start going out and either reaching out to them or we can figure out how they recruit if we’re going to have them available through Trial Recommender or what have you.
Brad Hightower:
Sure. Have you seen any instances, and I feel like I’ve seen this with maybe another company, but a sponsor was working with a particular company and then there’s plenty of sites that just don’t have that technology plugged, but the sponsor helps make it available or at least gets the site. The sponsor helps kind of liaison that relationship maybe to be like, look, here’s a vendor. We’d love for you to get that working. Obviously as you’ve talked about, that can be a long cycle, maybe not, depending on if you’re, I’ve worked with small private practices who are like, sure, jump in, jump right in. It’ll take a week, we’re ready to go. But have you seen that be sort of a strategy that maybe some sites just haven’t been exposed, they don’t know about it.
Doug Cassidy:
It’s a future we’re working on. So we are a build it and they will come company. So we’ve got this large patient population and these sites that we have and kind of it’s come here, but people get success and they’re like, wow, that was easy. I actually love working with this hospital who’s not yours. What if I helped get it in? We’re solving that as we go right now, because what I don’t want is for the study to be over by the time we study it that we set it up because it doesn’t take a month at a site. Can we do it under a data use agreement? Can we do a partial version of a deploy where we maybe land and do that study for them first and then expand afterwards, take care of that study? Do they want to have some sort of jurisdiction over that site that they helped fund first? We got to figure all that kind of stuff out.
Brad Hightower:
Any sort of — and you talked about this a little bit — but are there any sort of more front-end approaches versus having to go through the back and really make the process longer? I feel like I’ve talked to a couple who have mentioned the ability to sort of go through if they have a user account, a normal user account may be able to go in and help pull some of that data out. Is that something?
Doug Cassidy:
We are not there yet. Okay. Fair. We’re working at on the bulk FHIR API and the standard is I think someday going to be part of law that you have to have it, and I think that’s going to solve a ton of problems. And we always have the debate that is a sticky one right now is who owns the data? Does the patient own the data? Does the EMR own the data? Does the site own the data? And if you are trying to do something for the greater good is some aggregate form of that data available as well. So that is an act of Congress that is I think, going in the right direction. Just the availability of the bulk FHIR APIs and CCDAs has already made it easier. So potentially if you had just a data use agreement for maybe a single study, I could see maybe departmentally getting something or single study as just a way to stand it up. But we got to get through the contracting, we got to get through the cyber security stuff stuff. We have to tell the IT department, the thing they’ve been trying to build for eight years is going to be given to them by a vendor. Right?
Brad Hightower:
Yeah. A lot of landmines is to navigate as it stands. And again, we’ve experienced that too. I think everybody recognizes the utility and the value, but unfortunately things are oftentimes easier. It’s just easier said than done.
Doug Cassidy:
Yeah, I mean, I always tell people that join Deep 6 is like we are not selling to business people. We’re selling to physicians and scientists. They have a core thing that they’re doing first and that’s patient care and everything else is secondary. And so we’ve got to fit their process. And some of these people that are running hospitals or physicians, they’re not like their job is patient care and they’re running the hospital to give care first. All the other stuff happens after, and you’ve got to lean into that. You’ve got to be available when they’re available. You’ve got to fit their timelines and schedules and the way that they work. And if you’re comfortable being uncomfortable all the time, everything gets easier. You’re just going to fit how they work. Bend like a tree in the wind is what I say.
Brad Hightower:
I like that. Well, let’s get a little philosophical for a moment here. I mean, do we think patients are going to be concerned about use of AI in clinical trials? And I don’t know if that says, I have a hard time thinking in a case like this where, I mean frankly if AI is not doing it, a human being is doing it, which I don’t know if there’s much of a difference there in this particular
Doug Cassidy:
The question makes sense. Will they be concerned? Yes. And when people ask me who is our number one competitor, I say confusion.
So people don’t understand that the AI is assisting a human being. The robots are not taking over. Your data isn’t going anywhere. But the question I think is smart. Will patients be concerned about this? I think so. Two things happened I think in the last three years, which has changed everything for us. Number one, after Covid, everybody knows about clinical trials. Nobody was talking about this before. My parents know what I do now. My neighbors understand better what happens. Nobody cared about clinical trials before and AI was either robots five years ago or people had never heard of it. Now with Chat GPT, everybody’s seeing a new AI thing. And so same thing, my neighbors think I’m doing something with Chat GPT or something like that. So AI can be a million different things. It’s an umbrella term. So when you’re saying, are patients concerned about their data and AI, I think they will be, and it’s our job as a community to manage that or else we’re going to spend our whole time explaining ourselves.
Brad Hightower:
And I mean clinical trials themselves will probably require some education and awareness as it becomes more clear. What’s the word I’m looking for here? A question that applies to me. I’m really interested in CTMS integration, right? Because so much of what sites do really lives in a CTMS, I feel like so much of these great tools, I’m like, okay, cool. How can I drag these into a sensible place? What’s your strategy there or what are you seeing? I guess more than anything, I know that nobody’s maybe got it all perfected at this point and there’s more than one CTMS out there. Is that something that’s pretty high priority for you guys?
Doug Cassidy:
It is. It’s part of our strategy. So the first thing we did is we integrated the IRB part. So the IRB is already integrated with the CTMS. The next thing that we’re going to do is make sure that we’re pushing the patients to the study. The third part is I want bi-directional conversations because if I have inclusion and exclusion criteria, it would be very handy to know if this person’s already on a study because that could be exclusion criteria. The last thing that I want to do is start getting some of the information from the CTMS too, because some of the treatments never make it back to the EMR. They’re captured. And if we start, I did a study where one of the exclusion criteria is they were part of a specific clinical trial. So the way that we worked around it is we saw, I talked to them about a clinical trial, so we found the free text and then we had to navigate that because they didn’t say exactly what we wanted, but it helped us find that. So if we actually had that reflection available as a complimentary data source to the EMR and the OMI databases and the silos, that’s going to help us with inclusion exclusion criteria. Because what if they were with Olaparib or something is exclusion criteria, but the only time they ever took it was during the clinical trial in the 2020 or something.
Brad Hightower:
And in fairness, there’s still not a lot of cross bi-directional communication like you say. I mean, a lot of stuff we do in research doesn’t make its way back into an EMR and even vice versa, stuff that’s happening outside the trial context, even while someone’s in a trial in an EMR, who knows where, may not make its way and depending on a number of different situations,
Doug Cassidy:
Everybody’s number one priority is to get through today. That’s correct. Okay. We got that study done. We’re done. We’re moving onto the next thing, and I’m sure everybody is saying, wouldn’t it be nice if this data got back? They’re like, yeah, it would be nice. We’re doing the next study now. Right, right,
Brad Hightower:
Right. Yeah, no, that’s true. For better or worse. So I know we’re getting short on time here. Anything else, Fox you want to throw in?
Dr. Daniel Fox:
No, I mean I’ve seen AI flourish in the past few years. Just like what you had said. Sometimes I wonder if maybe the pandemic or the focus on research has really focused AI on the research processes to try to make things better. And it’s just like, again, you call it on a number of points there, people are afraid of robots and Hollywood movies taken over the world where AI is just two letters and some concept behind it, some programming. And I think that you’ve got a really good balance here between trying to utilize AI as a tool to help clinical research sites do a better job.
Doug Cassidy:
I think the gift that I’ve been given in being here is being able to hitch my wagon to something that is being driven by such smart people doing such innovative things. That gets me in a room with, I think the most interesting people in healthcare. I think that the same people that are trying to be innovative with clinical trials are the same people that get it with AI. The overlap is a ton. The Venn diagram has very little outside the edges.
And so next the public going to catch on and they’re going to understand. I mean, I always give this kind of analogy is you guys, I think we’re all about probably the same age. I remember when the TI-85 was introduced and everybody was pissed off because kids got to do calculus with the TI-85 and now if anybody shows up to class and they don’t have something equivalent, they’re seen as a ding dong. This gets us to the end goal faster.
Yes, of course, it’s important for us to understand the underlying stuff and we never want to take that away, and that’s why we’re always showing people the evidence. But let’s accelerate people with technology. That’s the whole purpose. Let us as a community, we are clinically adjacent. I’m not actually treating patients, but my whole life has been being clinically adjacent, accelerating the process for the people actually giving care and let’s get the technology and meet people where they work. That’s the goal.
Brad Hightower:
And I think this is perfect, especially if you’re a site, this is a great, I mean, this is where you need to have some focus because depending on how you look at it, I guess this is sort of site owned, site managed, site accountable. You know what I’m saying? This is going to make our lives easier at the site level if we’re rolling it out correctly.
We hear about AI half time. I don’t know what it means depending on the context, but this is a very real and practical use for us today that can significantly change how we operate at a site if we’re using it properly. So I love that about it. Again, because there’s so few, I feel like, tangible opportunities at the site level to really integrate AI in a meaningful way that actually helps you better serve your community. And this is just a no-brainer solution.
Doug Cassidy:
I mean, the interventional studies, let’s intervene, right? Let’s give better care.
Dr. Daniel Fox:
Doug, I got a question for you. If someone really wanted to have another question, we didn’t get to it in this podcast, where could they contact you?
Doug Cassidy:
I don’t have TikTok, so I’ll give you my email address and our website.
Brad Hightower:
We’re done here.
Doug Cassidy:
Yeah, so my email is extremely easy. It’s [email protected], or visit deep6.ai. From there you can request a demo. If you are a hospital or you are looking for a demo, go in there and you can request one and you’ll get, oh, there it is, I did it. Or if you’re a site sponsor and you’re like, I didn’t know something like this existed and I could find the sites that are interested in participating easier, go there to Life Sciences as well and they can kind of walk you through it as well.
Dr. Daniel Fox:
Just for anyone asking what does the “six” mean in the Deep 6?
Doug Cassidy:
Yeah, I can tell you really quick. It is a nautical term. I get this question a lot. It’s referring to play on words of deep learning and deep six, the nautical term. Back in the day, if you threw something off the side of the boat and it went past six fathoms, it’d be lost forever. Deep 6 finds the things that everybody else misses. So there’s a treasure trove down there at the bottom of the ocean that we can actually get to. And we’ve got a bunch of other things kind of internal to our company with our mottos and stuff that I’ll keep on that kind of nautical theme. But yeah, that’s it. You “deep six” something, it’s gone, but we can find it.
Dr. Daniel Fox:
Makes sense.
Doug Cassidy:
Yeah.
Brad Hightower:
Got it. Makes sense. Well Doug, appreciate you coming on. I probably, I feel like there’s probably mean hell, we probably got room for another one of these with different topics we could cover. So if you guys are open to it…
Doug Cassidy:
We didn’t get into —
Dr. Daniel Fox:
Yeah, where’s the demo man? We —
Doug Cassidy:
Yeah, we could demo a demo. We could talk about all the different kind of data types. We could talk about how we’re ingesting data. We could talk about how we train the AI more. There’s a lot of interesting stuff. The future and then all the other stuff that we’re trying to do for welfare of mankind. And we’re doing a lot of pro bono projects too for NCT.
Brad Hightower:
I mean, I’m guessing there’s a ton of utility beyond clinical trial matching for this in terms of population health, probably quality initiatives across different institutions.
Dr. Daniel Fox:
Imagine predictive modeling. I mean the sky’s the limit.
Doug Cassidy:
Yeah. What we’re trying not to do is get too wide before we get too deep. We’re trying to do super excellent about this and not get too distracted.
Dr. Daniel Fox:
You’ll be Deep 12 before you know it, man.
Doug Cassidy:
Hey, I was at this company. We were a Deep —
Dr. Daniel Fox:
Five.
Brad Hightower:
Alright, Doug, thanks so much for coming on man. And you guys make sure reach out to [email protected]. This will be recorded and it’ll be on YouTube. It’ll be on LinkedIn, I promise. I’ll go and I’ll trim away the long intro music I force you guys to listen to. So all the content will be there, but otherwise.
Doug Cassidy:
This is super surreal being on it instead of just watching it. Thanks for the invite.
Brad Hightower:
Yep. Anytime, man, and Dr. Fox. Until next week. Yeah, thanks Brad. All right, thanks guys.
