Image
Ärztin am Laptop

17.11.2026 | Blog Beyond Q&A: Traceable Clinical Evidence for Family Doctors

FROM OUR RESEARCH - Finding relevant information in patient records can be challenging, especially when it is spread across multiple documents. The first requirement in our research project came from physicians: rather than an AI that writes medical answers, they wanted to find what is already in the patient record, see how it changed over time and open the original document to check it themselves. We built a system that extracts diagnoses, medications, laboratory values and symptoms from medical documents, connects them across documents and over time, and keeps every finding linked to its source. This article covers what we built, where it still has deficits, and what we learned.

Key Takeaways

✓ Physicians asked for evidence, not answers. The system finds and connects what is already in the record; it does not generate medical conclusions.

✓ By combining AI-based information extraction, concept normalization and search, we connect diagnoses, medications, laboratory values and symptoms across medical documents.

✓ Every result, whether a value on a timeline or a candidate cause of a symptom, can be traced back to the text passage and the original document.

✓ A key lesson: results stay reliable even when extraction is imperfect at individual stages, because the system is designed to handle errors and preserve evidence throughout.

Research deep dive

The rest of this article is for readers who want the technical details: the processing pipeline, how our thesaurus grows from the documents, where normalization fails, how splitting extraction into two steps raised recall, and two worked examples. 

What the physicians asked for

The requirement that the system present evidence instead of generating answers shaped the solution architecture more than the choice of model, the prompts, or even the search technology.

A lab value may appear in ten documents. A diagnosis may be abbreviated in one place and written out in another. A medication may be started, changed, paused, and later discontinued. A physician letter written today may refer to a test performed six months ago.

For many clinical questions, the relationships between those pieces of information matter as much as the pieces themselves. 

Image Traceable medical evicence

From documents to traceable facts: first observations

The source material is real medical documents, some clean text, some scans: physician letters, laboratory reports, prescriptions, medication plans, and exports from practice-management systems.

Looking at the material, our first decision was to extract and structure four main types of medical information: diagnoses, medications, laboratory values and symptoms.

Each extracted item carries its relevant attributes: a laboratory value includes its value, unit, and reference range; a medication carries its active ingredient, dosage, and status; a diagnosis can be confirmed or ruled out. And every entity gets two things that make the whole solution work: date and source.

A simplified record looks like this:

Patient → HbA1c → 6.3% → 29 Oct 2009 → evidence passage → source document

If the same diagnosis appears in six documents, we keep the individual occurrences, each with its date and source.

The numbers get big fast

Our research corpus was small by production standards: nine patients.

Patients9
Documents~1,455
Fact occurrences7,544
Thesaurus concepts1,664

 

These numbers show why the problem quickly becomes more than document retrieval: a small number of patients already produces thousands of individual information points that repeat and overlap.

So the hard part is no longer which document is relevant? It is which occurrences belong together, what happened when, which version is current, and what evidence supports each one.

From occurrences to a usable patient history

We address these challenges with a solution that turns individual document mentions into structured, traceable patient information. The main steps are:

  1. ingest heterogeneous documents (indexer)
  2. extract medical facts (enricher)
  3. normalize concepts and aggregate occurrences (thesaurus)
  4. preserve attributes, time, and provenance (evidence postprocessing)
  5. structure user intent and build query-specific context (query rewriter)
  6. present evidence, timelines, and source documents (searcher + UI)

Language models participate in several of these stages. The physician-facing result stays tied to information already present in the patient record.

The thesaurus grows with the data

Alongside extraction, normalization is one of the challenges to solve. A document may contain:

  • DM2
  • Diabetes mellitus type 2
  • Type 2 diabetes

while a user simply searches for diabetes.

There are many medical terminologies, such as LOINC, SNOMED CT, ICD-10-GM and ATC. They are useful anchors, but they do not cover all of the surface forms, abbreviations, and corpus-specific vocabulary we encounter in real documents. So we bind our concepts to these standards where we can, but we grow a client-specific thesaurus from the documents themselves.

When the pipeline encounters a newly extracted entity, it first looks for an existing concept. If it finds a suitable match, the new expression becomes another observed form of that concept. Otherwise a local language model receives the relevant category slice of the thesaurus and decides whether the expression belongs to an existing concept or needs a new one.

This way, the vocabulary reflects the actual language of the corpus.

Thesaurus weaknesses

Automated normalization has two opposite weaknesses.

  • Overgeneralization: two concepts that are related get merged even though they should stay distinct.
  • Undergeneralization: two expressions that belong together land in separate concepts.

And, as in most large-scale automated pipelines, some noise enters the structure. Yet the thesaurus still does an important job: it reduces thousands of heterogeneous mentions to a much denser concept space. Surface variation becomes manageable, and the query rewriter receives a smaller context with a much higher concentration of useful information.

Two stages are better than one

We use this approach several times in our solution. The premise: processing natural-language text, whether documents or user queries, always produces errors. We build the system to catch them later. Near-perfect extraction and normalization are attractive targets, but real document collections make them hard to reach consistently, whether the method is rule-based, classical ML, or an LLM.

Instead of chasing perfection at ingestion, we stage the work.

In normalization, we accept a rough thesaurus at ingestion and give the query rewriter a second look. During ingestion, hypertension and high blood pressure end up in the collection of extracted facts as two separate concepts. At query time the query rewriter looks for related concepts, and both synonyms are added to the search plan.

The final OpenSearch query is then built deterministically from the structured plan the query rewriter suggests. This gives us a useful property: an ingestion error stays local.

That is one of the most interesting lessons from the project: an intermediate representation can contain errors and still improve the final system, as long as later stages are designed with those errors in mind.

We use the same approach in extraction: we split a complex task into two stages.

Stage 1: find the raw mentions, i.e. verbatim quotes, and keep their evidence text, e.g. 25.08.2022 Wt. 113.0 kg, BP 160/90 mmHg, PPBG 10.3

Stage 2: map each quote to a schema row:

NameValueUnitDate
Blood pressure (BP)160/90mmHg2022-08-25
Blood glucose, postprandial10.3–2022-08-25
Weight113.0kg2022-08-25

On the most lab-dense documents, this simple split lifted recall by up to 40 percentage points.

The questions decide how we search

Some questions like “Anything documented on diabetes?” are broad and those are easy; almost any search finds the right documents.

The interesting ones are specific:

  • How has the HbA1c changed over time?
  • Which medications are documented for this patient, and when?
  • Which evidence supports this diagnosis?
  • What appeared between January and June?

These questions need explicit relationships between concepts, occurrences, values, dates, and sources. Repeated mentions need aggregation. Clinical event time has to stay separate from document date. And the main point is completeness: we need every relevant occurrence, deduplicated, and often in chronological order.

That is what the enriched fact base gives us. We call this internal structure the patient graph. It contains every extracted fact for one patient, each linked to a date, an evidence passage and its source document. The graph is the context we build the queries and timelines from.

Some demonstrations

A physician asks: “How has the HbA1c developed?”

The query rewriter recognizes two things: a laboratory concept and a temporal intent. The search retrieves all of the patient’s normalized HbA1c occurrences.

The same facts now support two views: a timeline, and an evidence list of the individual occurrences.

 

Example HbA1c


HbA1c over time: the timeline of every occurrence with its date and value, and below it the start of the evidence list with the literal evidence text. Occurrences without a readable value or a date are omitted from the chart but kept in the list.

Click one occurrence and the physician is back in the source document.

A symptom, and what stands behind it

A physician asks about a symptom, for example “why is this patient dizzy?” The query rewriter does not answer the question. It recognizes dizziness as the anchor and pulls the concepts in the patient graph that are linked to it: blood-pressure-lowering medications, documented low blood pressure, a diabetic nerve complication. Then it retrieves the evidence for each. Nothing is invented; every candidate comes from the patient’s own documents.

Example dizziness

 

Symptom query “Why is this patient dizzy?”: related diagnoses, medications and symptoms from the record, each with its count and a link to the source.

Some of the explanations it returned:

  • Hypotension (symptom): orthostatic or chronic low blood pressure, a common cause of dizziness, especially alongside blood-pressure-lowering drugs.
  • Amlodipine (medication): a calcium antagonist whose vasodilation can lower blood pressure and cause dizziness.

Read together, they give the physician concrete leads: the patient takes blood-pressure drugs, and low blood pressure is documented in the record. The system makes no diagnosis: it offers grounded candidates with their evidence, and the physician decides whether they explain the symptom.

What we took away from the research

Real document collections produce imperfect intermediate data. Staging the work keeps the search flexible: the query rewriter can use patient-specific context to reconnect concepts that ingestion kept apart and make better use of the structured evidence already available. The result stays verifiable because every physician-facing item remains tied to its evidence and source.

Related articles

Wegweiser, die in verschiedene Richtungen zeigen

Why context is crucial for enterprise search

What makes a good enterprise search? It ensures that users quickly find the information they need for their work. To retrieve truly relevant information from diverse data sources, it is crucial that the search takes context into account.
Read blog
Lupe mit Blick auf vernetzte Linien

Why AI needs search engines more than ever

Large language models such as ChatGPT may seem to have made traditional search obsolete. In fact, the opposite is true: AI itself could help tried-and-tested full-text search make a comeback. Dr. Christoph Goller, Director Research at IntraFind, explains why.
Read blog
Zwei Kollegen lächelnd gemeinsam vor einem Laptop

Active information provision instead of document search

Documents become knowledge – intelligently extracted, structured, and made usable – using the example of an intelligent patient record.
Learn more

The author

Halyna Galanzina
Text Analytics Expert
Halyna Galanzina has been working at IntraFind since 2008 and is an expert in text analytics, natural language processing and information extraction. Her passion: developing algorithms for automatic text processing and text understanding.
Image
Halyna Galazina