University of Tartu researchers used language models to identify reasons for medication non-adherence

Researchers at the University of Tartu's Institute of Computer Science demonstrated that large language models can accurately identify from doctors' electronic notes why patients stop taking medications. The study combined prescription data from 10% of Estonia's population spanning 2012-2019 with clinical notes. The method opens new possibilities for utilizing previously difficult-to-analyze medical information.

The health informatics research group at the University of Tartu's Institute of Computer Science has demonstrated in a recent study that large language models can identify with high accuracy from electronically written notes by doctors the reasons why patients stop taking medications. The study focused primarily on diabetes medications and statins.

Data and method

In an extensive research project, prescription data from 10% of Estonia's population spanning 2012-2019 were combined with clinical notes written by doctors. In the first phase of the study, patients were identified who had not purchased from a pharmacy at least one medication prescribed to them within a year. Large language models were then applied to identify phrases related to treatment discontinuation in the notes, specific reasons, and even whether the treatment discontinuation was initiated by the patient themselves or by a doctor.

"From prescription data we see that a medication was not purchased, but the reason is often written in the doctor's notes. Until now, this information could only be used in a very limited way, because manually reviewing medical records is extremely time-consuming," explained Hendrik Šuvalov, a junior researcher in health informatics at the University of Tartu.

Privacy and model selection

To ensure patient data security, most of the analysis was carried out in a closed environment, using the language model Llama-3.1-70B. For comparison, the well-known GPT-4o model was also applied, but it was given for analysis only text that had been previously manually reviewed and completely stripped of sensitive personal information.

Why this matters

The study demonstrates how artificial intelligence can help the healthcare system better understand medication adherence-why patients do not follow prescribed treatment. This is a widespread problem in medicine that affects both treatment outcomes and healthcare costs. The method makes it possible to systematically analyze and use previously difficult-to-access clinical information for the benefit of medical advancement.