Trusted Tools for Peer Workers with Naveen Raman

welcome to Mental Health Evolution, a podcast about what's changing in mental health and why it matters. I'm your host Rachel Harrison, inviting you into honest conversations with people from all perspectives in the field. Clinicians, tech founders, investors, insurance companies and all the folks in between. Let's explore what's working, what's not, and what's next.

Hello everyone. Welcome back to the Mental Health Evolution podcast where we are talking about how the landscape is quickly evolving in the mental health industry. Today we are joined by Naveen Raman, a PhD student at Carnegie Mellon University. Earlier this year we talked about a growing trend in behavioral health around using AI tools to support triage and care navigation. The idea is that these systems can help people with lower acuity needs get connected to information and resources more quickly, while also helping to ensure that more complex needs are routed to the appropriate level of care. As we've been looking at what that actually looks like in practice, we came across Peer Copilot, a chatbot and AI powered assistant being developed at Carnegie Mellon University. Peer Copilot is designed to support peer support workers in behavioral health organizations. One of its key design features is that it does not draw from the open Internet. Instead, it uses a curated set of vetted resources, meaning the information it provides comes from pre identified trusted sources rather than unfiltered online content. In practice, this allows peer workers to take a conversation about someone's needs like housing instability, benefits, access or connection to community supports and quickly turn it into structured next steps using reliable information. Peer Copilot is currently being tested in real behavioral health settings where peer workers are supporting people through complex and often urgent needs with limited time and resources. Today we are going to talk with Naveen about how the system works, how it's being used, and where tools like this fit into the future of care delivery. And as always, before we start talking with our guest, we'd like to bring up some relevant articles related to our guest topics today. These may be helpful for listeners who want to learn more about this topic and dive deeper. As usual, all articles will be linked in the show Notes for this episode so the first article I want to mention is called AI Use in Mental Health Help Seeking and Support and this research looks at how people are already using artificial intelligence tools in mental health contexts, including looking for emotional support, coping strategies and general guidance. It highlights that while many people still prefer human providers for care, AI tools are increasingly becoming part of the mental health support landscape, especially for individuals dealing with anxiety and depression. The article also raises important questions about safety, reliability and the need for appropriate oversight, all topics that we're going to dive into a little bit more with our guest here in a moment. And then the next article is called Human in the Loop AI in Healthcare Systems and this article explains the concept of Human in the loop artificial intelligence where AI systems are designed to support support human decision making rather than replace it. It focuses on why this approach is especially important in healthcare and mental health settings where context, judgment and lived experience are essential to good care. It also discusses how these systems can reduce workload while maintaining safety and accountability. And then lastly, article number three is about the system we're talking about today. It's titled Peer Copilot AI Assistant for Behavioral Health Navigation and this article introduces Peer Copilot, a chatbot and AI powered assistant developed at Carnegie Mellon University to support peer run behavioral health organizations. It describes how the system helps peer workers create wellness plans, navigate community resources, and structure next steps using a curated database of trusted information. This system is designed to support and not replace human decision making and is being tested in real behavioral health workflows. I also just want to make a note here that Peer Copilot is not associated with any other tools that have the Copilot title or brand to them. It is a separate thing altogether. So with that background, Naveen we are really excited to have you here to dig in a little bit more. Thanks for joining us.

Thank you guys for having me here today.

Absolutely. So I'd love it if you can kind of explain in your own words what peer Copilot is, what it does for someone working in a behavioral health setting.

Yeah. So Pure Copilot really came out of this kind of collaboration we've been doing for the past couple of years with some folk in New Jersey known as CSP and J. These kind of big statewide peer run organization there. And what we're looking for, or what they were looking for is a sort of tool that can help them with their peer sessions on this kind of day to day basis, help find information on things like benefits, on resources and things like that. And so what the tool does at a high level is you take this kind of existing language model, GPT, whatever you want and what you can do on top is you can add certain, what I'll call like tools in the kind of form of language on top. What that means is you can add things like a benefit navigator where if I give an information about a person's income and this and that, it can tell you what benefits you're eligible for or if I give you a set of documents like for example from CSV and J, I give you a bunch of documents related to say food or housing or whatnot, it can use something known as sort of retrieval, augmented generation to search through these different documents and find the ones that are most relevant. The benefit to this is that it will call these tools or use these kind of additional features so that it can avoid relying too much on its kind of background knowledge that whenever it's kind of unsure or need to very specific information, it can rely on the very specific information provided to the kind of chatbot by us using these tools. And so at a high level it will combine the best of kind of both worlds ideally. And it'll help care providers during your sessions, navigate different situations, recall information and just help them hopefully get through some of the more say, mundane tasks of hey, when is this particular service open? Or what kind of food banks are near me in trending New Jersey so that they can spend more time talking to folks there and just kind of focus more on the other parts of their job and less on the kind of intermission retrieval and those parts.

Oh, okay, I like that. And it sounds like it's pretty location specific then.

Yeah, I think one of the things that people really were emphasizing there is a lot of the resources people need are very location specific. So somebody in Newark would need food things that are in Newark and they wouldn't be able to drive two hours to transfer or whatnot. And so, for example, in the database we have, which is given to us by CSP and J, we kind of went through all the kind of things or places that people had in mind. We went through, found the address and some information on it. So that if we search a query like, hey, what are some food banks near me in Newark? It'll go through and try to find food banks that are actually very close by within a few mile radius and give the ones that are most relevant or most proximate.

Okay. And is it just connecting to resources or are there other tools available as well?

There's a bunch of other tools. So I think we have a bunch of tools kind of built in. And for example, one of them is this kind of benefit navigator. And what that'll do is we kind of hard code, we say, if you're eligible for say ssi, then these are the kind of eligibility requirements. And if you put in certain information about a person, we can tell you whether or not they're eligible based on these requirements. And so if the model has some questions about, hey, is this person eligible, it can use those tools. We also have some very basic kind of tools or functionality built in which are sort of standard at this point with a lot of language models, things like a calculator. So you can do things like, hey, if I earn $200 more next month, am I still going to be eligible? Models have gotten better at doing additional work now, but it's still helpful to have these kind of tools built in so it can be guaranteed that it'll do well. And then things like web search for things like, hey, what is happening next week in Newark? Or what is. If you want very up to date information, you don't want to rely especially on the model's prior information or any kind of stored database of information because they're all static. And so if I want to ask something like what is the address for this particular place? You can either search through the database and try to find it there, or you can just do a web search. It can kind of automatically crawl the web or search the web and find the information that's most directly linked. And I think we have probably a couple of the tools built in, but these are the kind of biggest ones that come to mind.

Okay, okay. And how did this project get started? I'm curious to hear a little bit of the origin Story.

Yeah, so I'm a PhD student right now, so I can tell you kind of the perspective I'm coming from is sometime, I think in July 2024, August 2024, my advisor was telling me, hey, there's this organization up in New Jersey that wants to work with us. Something AI related. And so we did a little road trip up there from Pittsburgh to a couple different offices in New Jersey, I think in the end of July, early August, sometime in that time frame. And we were just chatting with folks there to see what is their experience with AI. What kind of things could AI help with, what is your kind of workflow? And it was all very interesting information in a very different domain than the one that I was used to. So it was cool just to see kind of what are their needs, how can it help them? And then from there it kind of just blossomed and we just kind of worked together for the past couple years from there.

Okay, are there any specific learnings that have kind of stuck out along the development process? Like anything that kind of went really wrong or something that you learned and had to correct for?

I think there's a lot of different things that change over time. I think one of the things that's very interesting is I think especially in this peer kind of provider space, they're very big on making sure that the tool is not a replacement, but more kind of complementary. And so they want to make sure that the information given to them by the tool serves like a very specific need and helps them kind of brainstorm different ideas. They don't want it to kind of tell them exactly what to do and things like that. So making sure the tool does that was definitely kind of a back and forth process of how do we design the tool, how do we prompt it, so on and so forth. I think one of the other big things just to kind of speed of modern development. And so, for example, the paper you mentioned before, the paper we wrote.

Yeah.

Is online since I think November. And between November and now, we've revamped the tool a good amount. And the reason for that is the way you can develop models, say when we started in 2024, is very different than the way you can develop models now. You can develop it in a much better way. The kind of baseline models are much better. And so, for example, this idea of like tool calling was more new and not as kind of, well, let's say developed back in 2024. And so we had to manually tell the kind of agent or the kind of model, hey, these are the things you should do. And so if you look in the paper, you'll actually see, like a diagram that says explicitly called benefits, explicitly called this and this. But now the model can handle all this. It knows when to call which tool. And so we just have to give it the appropriate tools and the appropriate rag and whatnot, and it works a lot better as a result. And so I think along with that, it's just people have more awareness of kind of the power of AI looking at what you can do. And so when we went in 2024, and I think most people there had never really heard of, like, language models, had never really used it, but I think now it's definitely getting more and more acceptance and people are seeing it and just the potential for it is definitely increasing. Just because the models themselves are getting better.

They certainly are. And I think you're right. It's more and more widespread. So can you walk me through, like, an example of how this is being used? Like an example of a call where someone might call into this group for peer support and use peer copilot.

Yeah, so I think the way. So we're still kind of trialing this out and seeing how people use it, but one of the many ways that people talk about this is folks who really. People who come in who really need some kind of resource, I think that's maybe the best kind of use case we have in mind. And what you can imagine is someone comes in and they're saying, hey, I'm in Trenton and I really need food at the end of the day today, but I don't really know where to go. And I have some restrictions. Like, I really want. I'd rather eat halal food because I'm Muslim or something like that. And so they would. And then you could type into the chat, you could say, as you're kind of working with them, you can say, hey, we're going to look for food by searching Guru's app. And then both them and the kind of person they're working with, they both go on the computer and they can see, can I find food pantries that are good from Halal, that are nearby or near Trenton? And it'll list some pantries, and then you can kind of go through and click on them and see what are the different results and things like that. But you can also do this in ways that are not for resources. So I think one of the examples that we had during one of our trials that we were testing out was something like, you can imagine, a senior comes in and senior's like, hey, I really want to stay active and try out different things, but I can't really, like, I can't really run or do anything because I got knee replacement. And so. But what are the best exercises I can do? And so in this situation, you can kind of brainstorm ideas with pure copilot, you can ask, hey, this person cannot, they cannot run too much or do too much high impact sports. What are some ways that they can stay active? And some good exercises both for the upper body and the lower body. And it'll give you like a detailed exercise plan or something like that. And so they can kind of like have a conversation saying, hey, these are some ideas. What do you think about this? And kind of go back and forth from there.

I like that. And so you mentioned, I think in this example that both the peer support person and the person looking for assistance can both utilize this tool and see the screen. Is that accurate?

Yeah, I think it's very heterogeneous depending on how people want to use the tool. And so some folks have said, hey, we want to use this tool during the session so that like the peer provider and the service user both kind of talk and then they can both see the screen and whatnot. But there are other folks who have said, hey, what I really want to use this for is like research after hours. Like let's say that somebody brings up something during conversation. Maybe I want to do some more research on this afterwards or before the conversation starts. So the thing is really the heterogeneous use case, depending on maybe where somebody's coming from. So I think it depends really on the individual, depending on how they want to use it there.

Okay. And then I know that you have mentioned that this is not the same as OpenAI, but you also mentioned a feature about searching the web. So can you talk a little bit about the safeguards in place and what was important from the use perspective for that?

Yeah, so I guess I'll talk about a few things. There's maybe two categories of ways you can build language models in the same age. One is you can build essentially what's called like a wrapper around like an existing large language model. And so you can take something like GPT, which is hundreds of billions if not even more at this point, number of parameters, and you can say, I'm going to make what are called like API calls, I'm going to send different requests or different prompts to GPT, get some responses back and do something with that. The other way you can do Things is you can do what's called fine tuning. And what fine tuning does is you can take a smaller language model, but train it on a big data set that you have access to. And you can say, hey, based on this, how does the model change? In this case, we're doing a GPT wrapper rather than fine tuning because we don't have access to gigantic data sets here. And the other kind of caveat is that if you start fine tuning models, it has some kind of computational burden, which is not terrible, but you have to kind of deal with, you have to deal with smaller models. And more importantly, you have to find a way to actually serve those models, basically make those models public in some way, which requires infrastructure and cost. When we build these GPT wrappers, the ways we can do a lot of sort of what I'll call like safety or whatnot is you can make sure the prompt is good. And so we did a lot of work going back and forth with the users themselves or the kind of peer providers to see what should the prompt be, what should it do? And we get explicit instructions to say, these are the things you should do to avoid things like hallucination. I think adding in the web search tool is quite useful to make sure that, hey, if you're unsure, call these tools. And this is something we or explicitly encode in. And then the way this tool is deployed right now is it's deployed as what's called like an Azure server. And so Microsoft has a service called Azure which runs on the cloud and can do a bunch of things with hosting models and running web servers. And so there are also what I'll call, I don't know the exact word for this, but there are kind of safeguards built into that as well, where you can avoid things like overly harmful language. You can set thresholds of filters on the band and whatnot. And so these are also built into the model or built into the kind of deployment of the model so that we can avoid any kind of bad behavior, any kind of negative outcome that comes as a result of this. Of course, at the end of the day, I think relying on models themselves for very important tasks is a bit tricky just because no matter how good your model is, there are ways it will break. There are ways you can potentially break it, of course. And so normally we will always have like a peer provider working with this model so that they are kind of an extra line of defense. They can say, hey, this doesn't make sense, or this result doesn't make sense, or whatnot so I think building and deploying these models is really more about having these lines of defense in, rather than just saying, oh, we can build the best model possible and it'll all work out from there.

I like that. And to me, what I'm hearing you say is, yes, there are lines of defense in the design itself, but the biggest maybe, line of defense is that this is being utilized with a human.

Exactly. I think this is like an adage a lot in computer security, where I think what they call it, there is like a Swiss cheese model where you have all these different lines of defense. Every single line of defense has certain vulnerabilities or certain holes. The kind of goal that you want is to make sure that all the holes don't line up so that there's no path from, like, one end to the other. And so as a result, you can make sure that whatever mistakes the model makes, the human catches. If the human can't catch something, maybe the model will catch it, or the additional safeguards we add on top or something like that. So we just want to make sure that whatever we do, there are different ways to catch mistakes. And just another thing we're doing, actually, in addition to that, I think the folks at CSP&J are running some kind of AI training session sometime in the next week or two. And so what they're doing there is they're teaching people about how to prompt, what kind of issues might come up when you prompt, and how to think about how to be kind of cautious but also optimistic when working with these sort of models. And so I think all these things really go together as different lines of defense to make sure things work out.

Well, that makes a lot of sense because you're not looking for the type of person to just read off of a screen. You're looking for the type of person who has some kind of training, working as a peer support, to be like, wait, that doesn't sound right.

Exactly. This is such a new tool that I think having some degree of AI literacy or technological literacy is very important, because the way that model works or the way it might hallucinate might look different than, say, what a person might do when they make mistakes. And so just being aware of this, being aware of these are the things that the model can do, these are the things that maybe the model cannot do as well and try to tell the difference between the two and just kind of being guarded whenever you use these models, just because the model will say things you might have to go back and verify.

And this and that, yeah, that makes sense. So this is a specific, like you're. You have a specific use case that you're using this for. Are there any ideas or dreams or plans even to broadcast, broaden the scope of where this might be available for people to use?

I think that would be nice. I think right now we're using this with one specific organization. But I think in general, I think there's two maybe lines of things that at least I imagine personally, one is expanding to other peer organizations. I think we're very happy to work with any peer organization who wants to use AI in the workflow. And this is kind of the tool that hopefully can help them out with different kind of things. So we would love to kind of integrate this. There's. And then also just as a way to maybe at maybe like a county level or state level hub would benefit navigation and things like that. And so I know a lot of people deal with like, hey, how do I apply for snap? Or something like that. And so maybe there's some version of this tool or even just some version of some language model out there where people could use it for assisting with benefit navigation and this and that. Just because I think reading through the benefit documents and understanding these things are very, very complex. But I think these language models, what they do really well is if you give them like a fixed source, like for example, you could give them like the fixed booklet on SNAP eligibility. It's very good at parsing this and breaking this down into simple terms and understanding this, which can hopefully kind of lower informational barriers and make it easier for people in these kind of spaces to see what's going on.

Yeah, yeah, I can see that. Interesting. It occurs to me like a lot of locations have a 211 resource. Are you familiar with that?

Yeah, I think I've heard about two one before.

Yeah. So it's similar. It's usually like a hotline sort of thing where people might be calling with because they're in emotional distress. People might be calling looking for resources, but the need for resources is inherent in that kind of support system. And thinking this could be a great utilization potentially for those types of organizations.

Yeah, yeah, I know. Whenever we have some queries, I know 211 is like a resource we have even embedded into the kind of database there. Okay. So a lot of the times if we have a query and it's something like, hey, where do I find this food resource? It'll tell you some of the food resources, but it'll also say maybe you should contact 2 1, 1 to find additional information or something like that. So maybe that's definitely another use case for it. I think there's a lot of different use cases where we need to find certain amounts of information or whatnot and people could use it there.

Do you ever think that it could be used without that human interaction, you're

saying directly by, say, like a service user? Yeah, that's an interesting question. I think there are definitely risks, but also benefits. The benefits is if I need something urgently, then I can kind of quickly go to it and find something. The risk is that maybe the tool is not fully designed for that just because there are, of course, things that might hallucinate or say the wrong thing here and there. I think it's also, we're trying to build a few other things into the tool to make sure it's more accessible so that if we want to kind of go directly to service users, then it could. But I think it's a bit tricky just because we don't want to ruin or even tamper with the peer provider services or relationship. And so this is why we're sticking mostly to kind of helping peer providers for the time being. And then if people are interested, we could potentially go directly to service users. But I think it's a lot of complexities that end up coming there. For example, things like, if my service user speaks entirely Spanish, then does my tool do well in Spanish? If my service user is kind of either hard of seeing or hard of cannot read or something like that, can my tool kind of say things out loud and whatnot? These are things that we've tried to build in, like, for example, it should natively support other languages, just because that's how language models work. And I think we're planning to build in like text to speech or things like that, but I think there are all these kind of more practical complications that start coming in if you start thinking about, oh, we're going to go directly to service users.

I agree. Yeah. I could also see that when you talk about this systems navigation, it does make me wonder if there's a use case for like Department of Social Services. Right. To be able to easily provide a way people can quickly see, like, do I qualify? Should I even make the appointment? Like, I can see so many kind of possibilities there of like, yeah, you qualify for snap. You should make this appointment versus, like, nope, your income doesn't match. Don't waste your time.

Exactly. I think that can be a very good use case for this where I think it's very good at like if you give it fixed information, which ideally things like snap are at least hopefully somewhat fixed, where you can tell this is what happens, then it can just process your information in whatever format you give it and tell you what you qualify for. What information do you need? It's kind of a more flexible version of the calculators that are already out there. And so I think this kind of thing could definitely be useful and wouldn't be too complicated to build just because it would just need to be like a minorly prompted or tailored version of GPT or kind of a minorly modified version of whatever we have right now with Pure Copilot. So that's the hope there.

Okay. I love that. Is there a way for organizations to get connected with you if they are interested in being a test case or utilizing this software?

Yeah, of course. I'm very happy to tag with any organization. If you look at the paper itself, it should have my email, I believe.

Okay.

And so. Or even if you just search my name, Naveen Raman. I'm sure you can find my email somewhere on the Internet and I'm happy to if you just send me an email. I'm happy to chat with any organization who has questions about the tool. Even if they have questions about AI. I'm happy to chat with them and see what can AI do for them. Even if it's not necessarily Pure Copilot, even if they want to use another tool, I'm happy to see like what can AI do in this space for them.

I love it. I love what you're doing. I really appreciate the intentionality, the focus on supporting the organization that you're working, working with the human line of defense, all of those things are great. If you could leave our listeners with one last thought or maybe a most important thing that you think they know about AI models or integrating this in the mental health space, what would that be?

Yeah, I think maybe the thing I would say is AI models are very powerful, but they also need to be very kind of specific or tailored towards a domain to make sure that they work in this specific context. And so just thinking about both what can AI do that might be able to help me, but also how do I make sure that it works within my context so that it works with the people and it works within those relationships in the kind of right way that I want it to work is both a big question, but also potentially very impactful.

Yeah. Yeah. Awesome. Well, Naveen, I really appreciate you taking the time. You and your team are doing some great things. So thank you for that. We will definitely have the article, the paper that you all wrote in the show notes. We'll also put your email. It sounds like you're okay with that in the show notes so people can reach out to you. It's been a great, great conversation. So thanks for being here.

Of course. Thanks for having me and thanks for the chat.

Ah, so we will be back next week on this pod to discuss more about issues relevant in the mental health care community. Thank you for listening and bye for now.