Practical AI for Clinical Research w/ Jeremy Franz

John Reites and Jeremy Franz explore the practical applications of AI, focusing on its use in data analysis and research. Jeremy shares his extensive experience with AI, discussing the evolution from classical NLP to modern large language models (LLMs) like GPT-3. They delve into various use cases, including how AI can assist in synthesizing information, generating insights from data, and transforming complex data into engaging presentations. The discussion emphasizes the importance of understanding AI as a tool that can enhance productivity and decision-making in various fields.

Thank you for joining Inclusion Criteria: a Clinical Research podcast hosted by me, John Reites. This is an inclusive, non-corporate podcast focused on the people and topics that matter to developing treatments for everyone. It’s my personal project intended to support you in your career, connect with industry experts and contribute to the ideas that advance clinical research.

Inclusion Criteria is the clinical research podcast exploring global clinical trials, drug development, and life‑science innovation. We cover everything clinical research to deepen your industry knowledge, further your career and help you stay current on the market responsible for the future of medicine.

Our episodes discuss current industry headlines, career tips, trending topics, lessons learned, and candid conversations with clinical research experts working to impact our industry everyday.

Watch on YouTube and listen on your favorite podcast app. Thank you for supporting and sharing the show.

Please connect with me (John Reites) at www.linkedin.com/in/johnreites or www.johnreites.com.

The views and opinions expressed by John Reites and guests are provided for informational purposes only. Nothing discussed constitutes medical, legal, regulatory, or financial advice.

Transcript

SPEAKER_00: 0:07

We're spending time today talking about use cases in AI. And again, this premise of practical AI is really important. And the reason Jeremy is on is one, because I work with Jeremy every single day. Two, because he knows this inside and out. He's been working on it for a very long time. And he won't call himself this, but I will. This guy's a guru. Jeremy, tell everybody what your experience has been with AI, what you've been doing, because you've been doing it for a long time. But I think bringing that context would be helpful for people. Yeah, I mean, I think... It really starts with trying to use technology to process a lot of unstructured or semi-structured information back in the day with earlier days of Envibe. And we were always looking at technology and AI and how can we use this to help synthesize information? How can you use this to help us analyze? How can you use it to help our analysts analyze the data? And we built a lot of software around that stuff. But on the AI front, we were banging our heads against the wall for a long time and using classical NLP approaches. And a lot of it was building all these manual rules and it was just, it was very complex. It took a long time and it usually didn't meet the criteria of what we considered acceptable to put it in front of customers and users. And when this thing called GPT-3 came out and There was this private beta that we applied for access for and had to kind of make our case as to why we had a good use case for it and how we were going to make sure we had the proper safety and protocols and weren't going to use it for anything harmful. You ended up getting access to that program through OpenAI and trying a lot of the tasks that we were using classic NLP approaches for and it just worked for a lot of them. And so ever since then, we've kind of been on this journey of coming up with harder and harder tasks for AI to tackle and what would provide value to our customers. And sort of now the challenge is what's the next task that can't be done with today's models that probably will be able to be done by be performed by the next generation of models. And we've seen that through several waves of new foundation models. And it's been a really fun and eye opening experience for me kind of seeing that all unfold the past four or five years. I want to give you a lot of credit, like, because obviously, like we work together, we're in the same company's disclaimer, that's what Jeremy and I do every single day. At the same time, predating Thread, when you were working in AI, I remember you and I were having a conversation. You were talking about AI. And again, my brain goes to what everybody's brain went to five and six years ago. Robots, Terminator, autonomous vehicles, like the nerd side of me went that route. And then you were explaining to me what you were trying to do with AI. And that was a real light bulb moment for me, actually. It really started to change my perspective on what AI was and what it wasn't. And so I'm hoping actually today some people take from you A couple of things you say that help construct how we view AI, what it looks like, and how we can use it as a tool to augment the work we're using. So before we dig into the work stuff and we talk about use cases and practicality and clinical research, Jeremy, you're an expert in this. This is what you do all the time. You know all the models like the back of your hand. What's the party trick? You're at the dinner table. You want people to think you're cool. You want to show off AI. Give it to us. What's the party trick we can all use? to look cooler than everybody else with technology. Well, I hate to disappoint you, but I don't know if I'm that cool at parties. Like, I lean towards really my favorite use of AI, just my day-to-day life. A lot of times, if I'm stuck on something, if I'm thinking about a problem or a decision I need to make, anything like that, I will just pull out my phone, start an audio recording, and I'll just speak. And it will be very unstructured. It will be very just... I may have long pauses. I may ramble all over the place. But I can just do that, get it out, transcribe it, send it into my favorite LLM, have it synthesize those thoughts, have it clarify them, and put it in a way that is very clear for me to understand and clarifies my own thoughts. With writing, this is kind of a cheat for me to use AI to clarify my own thoughts and and help make maybe tricky decisions or uncomfortable decisions a lot easier. That sounds really smart and thoughtful and businessy. Congratulations. I've got a few party tricks. I'm going to save some of those for other episodes. The one that I will never forget was the first party trick I ever saw on AI. I'm at this friend's house. It's a couple years ago. I think it was when the OpenAI app first came out. We were getting asked because we were the nerds at the table. What are these things with AI? And again, more Terminator, more macro level stuff. This friend of mine said, I use it for my wife's grocery list. And I think we were all like, what are you talking about? And he's like, well, she had to shop for like 50 or 75 things for this meal coming up. And so I loaded it all and told him what grocery store I was going to and laid out this map to save me like 30 minutes at the grocery store. Aisle three, go buy these five things. Aisle four, go to these. Skip a five. And I will never forget the amount of people that were like, I'll buy that. And for me, I was kind of like, oh, that's pretty cool. And then after people were like, could you tell me how you got the grocery list to work? So for what it's worth, I feel like there's a ton of party tricks. And that was one that really stuck out. Fifteen gears, though, I want to talk about practical use case of AI in research and what we're doing every single day. And so maybe if you could, can you get one or two really practical examples of what you see sponsors, CROs, different types of clients that we have? How are they actually taking advantage of AI? How are they actually using it in their companies today? Probably the number one thing that AI and specifically large language models are good at that get used a ton is just processing a lot of information. You can throw tons of text, tons of data at an LLM and it could synthesize that. You can even specify what kind of output you want out of that. The most common example would be summaries and summarization is sort of the kind of list app equivalent of an AI feature. It doesn't mean it's not incredibly useful and powerful, but I think what's really great is providing a lot of information to an AI and then asking for the output that you want. If you're processing interview data like we do at Envibe and you want a few interesting clips or quotes around a specific topic, ask for that. You'll get it. I think it's just Having an idea of what you want out of it and asking for it like you would ask a human goes a long way. I think one of the examples you were talking about with a client just the other day. So I'm just going to say it because I thought it was a really good one. It's current is you got a client. They want to get some insights on the recruitment campaign. So they load up the survey of questions that we're going to ask, you know, like or scale, but also mostly via their voice, right? We want people to respond and talk to our app with their actual voice. You recruit the patients. They come in. They talk to the platform. They give their feedback. It hits the system. And then we need to start giving insights on it, right? We need to start being able to give insights so that that client can make decisions on if they should change anything, if the material is working out great, right? Those types of examples. Because I prompted you for this, pun intended, because I prompted you. Why don't you show people what that looks like? I mean, I think the easiest thing to do here is you go in here and we have this suggest questions to ask. button here. And really all this is doing is we're telling the AI, hey, here's all this source data from this study that we conducted. What are some interesting questions to ask that lead to unexpected or interesting insights about this study? And, you know, you see the first one here is about the lung bar puncture requirement from this informed consent document. And, you know, it turns out there was a highly emotional reaction to this. There was a lot of risk This is a scary procedure, and it was something that had to be done many times as part of the design of this study itself. It's a really quick way to uncover something that you wouldn't have necessarily known to ask. Unless you had a good idea ahead of time, you wouldn't have known, tell me about the lumbar puncture requirement. But that's the great thing about asking AI for what you want is a lot of times it can give you what you're looking for. That's my favorite example, actually. I don't know if I've ever told you this before, but sometimes we look at the data and I'm like, where do I start? And I love that AI can prompt you in real time. And so I think that's a great example. And for me, the number one question is, how does it actually help me make it usable? Because the data in itself isn't usable. I think it's one of the most powerful things that AI does. And I don't know if people have seen as many examples as maybe are really out there in the real world around how to summarize the data, how to actually use it, how to put it in a presentation mode that actually helps. So like I prompted you before, could you just show some examples, just give people some ideas of what they can do and how I can help them visualize or make their data useful so they can present it to other people? At the end of the day, You don't want to be sending colleagues a link to your chat session and have them read this text necessarily. It's not the most effective form of communication. So one thing that we did is we added this creative video presentation tool. Same sort of idea at play here is we have the AI suggesting interesting topics to make a presentation about that we think would be of interest to students. people using this tool. You can go in and you can select one or enter your own. And then it goes through this whole AI-assisted process of ultimately creating a video presentation with slides, with voiceover and splicing in, you know, voiceover content from the actual participants who responded to your study. And the user has control over it. You can make it exactly how you want to make it, but The great thing about AI is that we can provide a happy path for you and we can suggest, you know, the key findings here that we think are most relevant to the topic that you provided. And you can go through this whole step process that you see at the top here, where after the key findings, then you select the actual quotes that you want to use to support those key findings. And then it'll write the slides for you. It'll write the voiceover and you click a button. And then a few minutes later, you know, you get a a video presentation that you can download and share with your colleagues. The important thing for us was to get out of just being text. You know, we want to be more than just text in a chat session. And how do we take advantage of the richness of our own data? You still have a human in control, but AI is heavily assisting and can be an assistant to varying levels depending on how much the user wants to control in this situation. I do like the end. I mean, it's always, I like presenting my stuff, but some people don't. And so I think it's kind of interesting to me that one of the most practical use cases is actually AI presenting the data and the findings, right? Doing a voiceover, doing an avatar. One of my favorites is when you did one of those and someone was like, oh, I really, I don't remember the name of it. They were like, I really love Amber in the corner. You know, she's a great. We were like, who's Amber? We don't know who Amber is. And the whole time we're like, oh, that's the AI. And people look at you like, that's not an AI. That's Amber on the team. We're like, no, that's an AI. It's for your voiceover. I really appreciate these, Jeremy. There's lots more to talk about. We'll come back and talk about more of these. But I think just giving listeners just a sense of what kind of practical AI is out there is helpful, right? Because this stuff is meaningful. It's supportive. It can help your work. And it doesn't have to be overwhelming. Absolutely. Yeah. Thanks for having me on. Well, hey, thanks for being on today. I appreciate it. I'll catch you soon. Thanks, Jeremy.