

17.11.2026 | Blog Beyond Q&A: Traceable Clinical Evidence for Family Doctors
Key Takeaways
✓ By combining AI-based information extraction, concept normalization and search, we connect diagnoses, medications, laboratory values and symptoms across medical documents.
✓ Every result, whether a value on a timeline or a candidate cause of a symptom, can be traced back to the text passage and the original document.
✓ A key lesson: results stay reliable even when extraction is imperfect at individual stages, because the system is designed to handle errors and preserve evidence throughout.
Research deep dive
The rest of this article is for readers who want the technical details: the processing pipeline, how our thesaurus grows from the documents, where normalization fails, how splitting extraction into two steps raised recall, and two worked examples.
What the physicians asked for
The requirement that the system present evidence instead of generating answers shaped the solution architecture more than the choice of model, the prompts, or even the search technology.
A lab value may appear in ten documents. A diagnosis may be abbreviated in one place and written out in another. A medication may be started, changed, paused, and later discontinued. A physician letter written today may refer to a test performed six months ago.
For many clinical questions, the relationships between those pieces of information matter as much as the pieces themselves. 
From documents to traceable facts: first observations
The source material is real medical documents, some clean text, some scans: physician letters, laboratory reports, prescriptions, medication plans, and exports from practice-management systems.
Looking at the material, our first decision was to extract and structure four main types of medical information: diagnoses, medications, laboratory values and symptoms.
Each extracted item carries its relevant attributes: a laboratory value includes its value, unit, and reference range; a medication carries its active ingredient, dosage, and status; a diagnosis can be confirmed or ruled out. And every entity gets two things that make the whole solution work: date and source.
A simplified record looks like this:
Patient → HbA1c → 6.3% → 29 Oct 2009 → evidence passage → source document
If the same diagnosis appears in six documents, we keep the individual occurrences, each with its date and source.
The numbers get big fast
Our research corpus was small by production standards: nine patients.
| Patients | 9 |
| Documents | ~1,455 |
| Fact occurrences | 7,544 |
| Thesaurus concepts | 1,664 |
These numbers show why the problem quickly becomes more than document retrieval: a small number of patients already produces thousands of individual information points that repeat and overlap.
So the hard part is no longer which document is relevant? It is which occurrences belong together, what happened when, which version is current, and what evidence supports each one.
From occurrences to a usable patient history
We address these challenges with a solution that turns individual document mentions into structured, traceable patient information. The main steps are:
- ingest heterogeneous documents (indexer)
- extract medical facts (enricher)
- normalize concepts and aggregate occurrences (thesaurus)
- preserve attributes, time, and provenance (evidence postprocessing)
- structure user intent and build query-specific context (query rewriter)
- present evidence, timelines, and source documents (searcher + UI)
Language models participate in several of these stages. The physician-facing result stays tied to information already present in the patient record.
The thesaurus grows with the data
Alongside extraction, normalization is one of the challenges to solve. A document may contain:
- DM2
- Diabetes mellitus type 2
- Type 2 diabetes
while a user simply searches for diabetes.
There are many medical terminologies, such as LOINC, SNOMED CT, ICD-10-GM and ATC. They are useful anchors, but they do not cover all of the surface forms, abbreviations, and corpus-specific vocabulary we encounter in real documents. So we bind our concepts to these standards where we can, but we grow a client-specific thesaurus from the documents themselves.
When the pipeline encounters a newly extracted entity, it first looks for an existing concept. If it finds a suitable match, the new expression becomes another observed form of that concept. Otherwise a local language model receives the relevant category slice of the thesaurus and decides whether the expression belongs to an existing concept or needs a new one.
This way, the vocabulary reflects the actual language of the corpus.
Thesaurus weaknesses
Automated normalization has two opposite weaknesses.
- Overgeneralization: two concepts that are related get merged even though they should stay distinct.
- Undergeneralization: two expressions that belong together land in separate concepts.
And, as in most large-scale automated pipelines, some noise enters the structure. Yet the thesaurus still does an important job: it reduces thousands of heterogeneous mentions to a much denser concept space. Surface variation becomes manageable, and the query rewriter receives a smaller context with a much higher concentration of useful information.
Two stages are better than one
We use this approach several times in our solution. The premise: processing natural-language text, whether documents or user queries, always produces errors. We build the system to catch them later. Near-perfect extraction and normalization are attractive targets, but real document collections make them hard to reach consistently, whether the method is rule-based, classical ML, or an LLM.
Instead of chasing perfection at ingestion, we stage the work.
In normalization, we accept a rough thesaurus at ingestion and give the query rewriter a second look. During ingestion, hypertension and high blood pressure end up in the collection of extracted facts as two separate concepts. At query time the query rewriter looks for related concepts, and both synonyms are added to the search plan.
The final OpenSearch query is then built deterministically from the structured plan the query rewriter suggests. This gives us a useful property: an ingestion error stays local.
That is one of the most interesting lessons from the project: an intermediate representation can contain errors and still improve the final system, as long as later stages are designed with those errors in mind.
We use the same approach in extraction: we split a complex task into two stages.
Stage 1: find the raw mentions, i.e. verbatim quotes, and keep their evidence text, e.g. 25.08.2022 Wt. 113.0 kg, BP 160/90 mmHg, PPBG 10.3
Stage 2: map each quote to a schema row:
| Name | Value | Unit | Date |
|---|---|---|---|
| Blood pressure (BP) | 160/90 | mmHg | 2022-08-25 |
| Blood glucose, postprandial | 10.3 | – | 2022-08-25 |
| Weight | 113.0 | kg | 2022-08-25 |
On the most lab-dense documents, this simple split lifted recall by up to 40 percentage points.
The questions decide how we search
Some questions like “Anything documented on diabetes?” are broad and those are easy; almost any search finds the right documents.
The interesting ones are specific:
- How has the HbA1c changed over time?
- Which medications are documented for this patient, and when?
- Which evidence supports this diagnosis?
- What appeared between January and June?
These questions need explicit relationships between concepts, occurrences, values, dates, and sources. Repeated mentions need aggregation. Clinical event time has to stay separate from document date. And the main point is completeness: we need every relevant occurrence, deduplicated, and often in chronological order.
That is what the enriched fact base gives us. We call this internal structure the patient graph. It contains every extracted fact for one patient, each linked to a date, an evidence passage and its source document. The graph is the context we build the queries and timelines from.
Some demonstrations
A physician asks: “How has the HbA1c developed?”
The query rewriter recognizes two things: a laboratory concept and a temporal intent. The search retrieves all of the patient’s normalized HbA1c occurrences.
The same facts now support two views: a timeline, and an evidence list of the individual occurrences.

HbA1c over time: the timeline of every occurrence with its date and value, and below it the start of the evidence list with the literal evidence text. Occurrences without a readable value or a date are omitted from the chart but kept in the list.
Click one occurrence and the physician is back in the source document.
A symptom, and what stands behind it
A physician asks about a symptom, for example “why is this patient dizzy?” The query rewriter does not answer the question. It recognizes dizziness as the anchor and pulls the concepts in the patient graph that are linked to it: blood-pressure-lowering medications, documented low blood pressure, a diabetic nerve complication. Then it retrieves the evidence for each. Nothing is invented; every candidate comes from the patient’s own documents.

Symptom query “Why is this patient dizzy?”: related diagnoses, medications and symptoms from the record, each with its count and a link to the source.
Some of the explanations it returned:
- Hypotension (symptom): orthostatic or chronic low blood pressure, a common cause of dizziness, especially alongside blood-pressure-lowering drugs.
- Amlodipine (medication): a calcium antagonist whose vasodilation can lower blood pressure and cause dizziness.
Read together, they give the physician concrete leads: the patient takes blood-pressure drugs, and low blood pressure is documented in the record. The system makes no diagnosis: it offers grounded candidates with their evidence, and the physician decides whether they explain the symptom.
What we took away from the research
Real document collections produce imperfect intermediate data. Staging the work keeps the search flexible: the query rewriter can use patient-specific context to reconnect concepts that ingestion kept apart and make better use of the structured evidence already available. The result stays verifiable because every physician-facing item remains tied to its evidence and source.
Related articles


Why AI needs search engines more than ever

Active information provision instead of document search
The author
Halyna Galanzina
