Why The “Notes” Section of EMRs Is So Valuable

We talk a lot about unstructured data. But what exactly is it? Unlike the structured coding used in healthcare documentation, unstructured data are more freeform and don’t necessarily follow a specific format. It’s often the data found in the free-text, or notes, section of electronic medical records (EMR). In fact, over 80% of data is unstructured, and found in the notes section of an EMR. 

It can include healthcare providers’ observations, imaging reports, PDFs, and anything that isn’t confined to preset, standardized fields. 

Medical notes are a healthcare provider’s source of “truth”. This is where they keep track of everything a patient shares with them, and it’s what they will review before the patient’s next visit. Random shorthand notes exist in this space where physicians or nurses jot down valuable patient information.   

What can be uncovered in the notes section? EMR notes may include: 

• Family or social history 
• Diagnosis overview 
• Disease progression 
• Genetic or immune testing results 
• Comorbidities 
• Drug interactions 
• Surgical history 
• Lab test results 
• A seemingly minor injury 
• Any other items a patient discusses during the visit 

Sometimes, these items are abbreviated in specific shorthand used by the health system or doctors may use their own, personal abbreviations for a condition or status. There aren’t any rules in this portion of the EMR, so this information is generally considered “unstructured”. 

As it relates to clinical trials, it’s nearly impossible to recruit patients without reviewing the notes section of the EMR. A researcher who relies solely on structured data would have too many holes in the patient’s healthcare story. Notes provide valuable context into a patient’s history, creating a richer and more complete picture of the patient’s current medical status. 

So why do researchers disregard unstructured data if it is so important? Many clinical research sites lack the necessary resources. Without the right software, entire patient records must be analyzed individually one-by-one. While it’s easy to organize structured data points in bulk, free-text notes take a more sophisticated, purpose-built technology to read. 

Our software utilizes machine learning and natural language processing (NLP) to analyze unstructured medical data. By inputting a clinical trial’s inclusion and exclusion criteria — such as disease progression, presence of a specific genetic marker or other specific items — the software returns a list of precision-matched patients for the trial.   

This means the user just needs to validate the matches. Our NLP technology allows users to search for concepts, not keywords, so it captures all the different ways of annotating a phrase or term found in a provider notes. For instance, a note might include that a patient has tested positive for BRCA1, an important genetic marker targeted for breast cancer clinical trials. Our system would flag mentions of “BRCA1 positive” as well as “positive for BRCA1 genetic mutation” or “Breast Cancer 1 gene” because we understand it means the same thing.  

The Deep 6 AI platform immensely speeds up the patient recruitment process for clinical trials. We also solve systematic problems. For instance, patients who could be a good fit for a clinical trial could be missed if the trial’s Principal Investigator doesn’t treat the patient, or if they don’t regularly see the patient. With our software, EMRs can be searched in minutes, including the goldmine that is provider notes.  

Interested in learning more? Get in touch here. 

Author

Allison McKinley, Life Sciences Solutions Consultant

allison.mckinley@deep6.ai