We work at the intersection of psychiatry and computer science, building monitoring methods, AI systems for clinical processes, and rigorous ways to evaluate GenAI in behavioral health. Our work is inspired by the mission of Johns Hopkins Medicine to "improve the health of the community and the world by setting the standard of excellence in medical education, research and clinical care."
Developing new metrics, measurements, and methods to monitor for critical events related to psychiatry and mood disorders towards improved patient care.
Read more ↓Developing the theory and building multi-agent systems related to improving clinical processes for patients, clinicians, and hospitals.
Read more ↓Strategies to evaluate and test GenAI tools in psychiatry and behavioral science.
Read more ↓Developing new metrics, measurements, and methods to monitor for critical events related to psychiatry and mood disorders towards improved patient care.
Clinical psychiatry depends on timely detection of change: relapse risk, treatment response, and other events that matter for patient care. Our work builds measurement frameworks and monitoring methods that can surface these signals reliably from real-world data streams.
We focus on metrics that are statistically sound, clinically interpretable, and practical to deploy alongside existing workflows, so monitoring systems reduce noise rather than add alert fatigue.
A public repository that surfaces mental-health benchmark examples where AI models show the highest uncertainty or disagreement with compliance rules. Behavioral health benchmarks carry erroneous labels and drift over time, so NAVIGATOR separates two sources of uncertainty: a multi-agent system constrained by a directed acyclic graph simulates an organization’s process map to surface context-dependent (aleatoric) uncertainty, while Monte Carlo simulations at each agent surface model gaps (epistemic uncertainty). Internally validated labels feed back into the compliance monitoring system, which re-evaluates the datasets to identify the next set of high-uncertainty examples. The system now covers more than 60 open-source datasets, improves labeling accuracy by 16%, and reduces human review by up to 85x.
Minseo Choi, Rosa Jahankhah, Ronald Deng, with Michael Rudow
Developing multimodal AI methods that integrate biological, electronic health record (EHR), and longitudinal patient data to provide adaptive clinical decision support under diagnostic uncertainty while improving our understanding of patient heterogeneity and psychiatric comorbidity.
Rhea Makkuni, Ram Chitti, Minseo Choi
Building a classifier that flags sensitive content in mental-health text, closely related to the compliance monitoring work in NAVIGATOR.
Ronald Deng
Multi-agent LLM pipelines are being used to assess self-harm risk, but common evaluation approaches do not indicate when a decision is reliable or how errors accumulate across agents. We give these pipelines a statistical footing with agent-level confidence bounds, bandit-based adaptive sampling, and regret guarantees, cutting the false positive rate by 40% against single-agent models without losing recall.
Meghana Karnam
Developing the theory and building multi-agent systems related to improving clinical processes for patients, clinicians, and hospitals.
Psychiatry involves complex, multi-step processes across patients, clinicians, and institutions. We develop theory and systems for multi-agent and generative AI workflows that can support those processes while remaining constrained, auditable, and aligned with clinical goals.
We are working on an agentic LLM chatbot that could help perform psychiatric intake. Our current work has been creating a simulated intake environment, where we can test our tool and existing LLMs, as well as compare them to clinician baselines. We are working alongside the field lab to create realistic synthetic patient vignettes. We also hope to develop education spinoffs from our work to support the next generation of psychiatrists.
Synthia Wang, King Shi, Amanda Li, Ariel Kim, Jonathan Ivey, Michelle Lu, Guan Gui
We are researching how AI and LLMs can help caregivers and clinicians make safer, patient-specific medication decisions for older adults, focusing on polypharmacy, mental health, and data gaps in geriatric psychiatric care. Our geriatric psychiatry work runs under this project.
Arushi Acharya, Peiyong Lin
Strategies to evaluate and test GenAI tools in psychiatry and behavioral science.
Generative AI tools are entering psychiatry and behavioral science faster than evaluation practice can keep up. We design strategies to test these systems rigorously, covering validity, reliability, fairness across populations, and fitness for clinical or research use.
Our goal is evaluation methods that help labs and clinics decide when a tool is ready for deployment and how to measure improvement over time.
LLM outputs vary depending on which clinical details are presented. We audit four models across four prompt framings and show that adding clinically irrelevant information significantly shifts predicted hospitalization risk and output variability, underscoring the need for governance around these models before clinical deployment.
Shevya Panda, Shinjini Bose
The people talking to a mental-health chatbot do not all talk the same way: teenagers and older adults use different words and phrasing for the same concerns, and the input a deployed system sees shifts over time. We are working on detecting that shift and on mathematical models that quantify how much drift has occurred.
Rosa Jahankhah
A survey of mental health providers on whether and how they use chatbots in practice, and how they feel about them, to ground evaluation criteria in what clinicians actually need from these tools.
Ariel Kim
