Parameter-Specific Literature Retrieval for Kidney Diseases
A two-phase biomedical NLP project that takes a clinician from the flood of kidney-disease literature to an evidence-backed clinical answer. Phase 1 generalizes a parameter-specific retrieval system across 24 diseases; Phase 2 re-targets the same extraction machinery into a provenance-tracked knowledge graph with a question-answering engine that cites the papers behind every claim.
The Two-Phase Arc
Generalized NER & Structured Extraction
Generalized the project's extraction machinery from a single disease and four parameters to 24 kidney-related diseases and 10 clinical parameters.
- Fetched ~160,000 PubMed abstracts (~7K per disease) and built an annotated corpus with Label Studio.
- Fine-tuned a PubMedBERT NER model (spaCy, 9 iterative rounds) to extract Age, GFR, Proteinuria, Creatinine, BUN, BP, Hematuria, Weight, Calcium & Cholesterol.
- Fine-tuned a BART Seq2Tree model that converts free-text measurements into structured XML (magnitude, unit, comparator).
- Powers the NephroSearch web platform and the NephroTagger Chrome extension.
Knowledge-Graph Question Answering
Re-targeted the same extraction stack from a ranked list to a Neo4j knowledge graph queried with natural-language clinical questions.
- Composable, hot-swappable pipeline: document parsing → ensemble NER → normalization → relation extraction → KG writer.
- Ensemble NER (BioBERT + clinical DeBERTa + zero-shot GLiNER) covering 13 KG entity types; UMLS-linked normalization to collapse synonyms.
- REBEL relation extraction with a context-aware re-mapping stage that recovers clinical edges from structural labels.
- Two-generation QA engine — entity resolution, type-aware routing, LLM-generated Cypher fallback, and an explainable step log — served through a Streamlit app.
Results & Evaluation
Rather than one opaque score, the system is measured stage by stage — NER, relation extraction, graph construction, and question answering — and then end to end. The Phase 2 graph and QA results below are reported on a curated 500-abstract demonstration graph spanning all 24 diseases.
Phase 1 — Ranking Quality (NDCG)
A parameter-aware ranker places the genuinely relevant trials higher than keyword engines can. The NER model reaches high-0.9 per-parameter F1 and the Seq2Tree model scores in the high-0.9 range across BLEU, ROUGE and METEOR.
| System | Mean NDCG |
|---|---|
| NephroSearch (ours) | 0.9639 |
| 0.8978 | |
| PubMed | 0.8669 |
Phase 2 — Ensemble NER
Per-type precision / recall / F1 on a labelled sample. Specialised types score highest; zero-shot types pay the expected price.
| Entity type | P | R | F1 |
|---|---|---|---|
| Disease | 0.91 | 0.93 | 0.92 |
| Drug | 0.89 | 0.90 | 0.89 |
| LabParameter | 0.83 | 0.79 | 0.81 |
| Symptom | 0.80 | 0.82 | 0.81 |
| Biomarker | 0.78 | 0.74 | 0.76 |
| SideEffect | 0.72 | 0.69 | 0.70 |
| MechanismOfAction | 0.68 | 0.60 | 0.64 |
| Micro-average | 0.82 | 0.84 | 0.83 |
Phase 2 — Context-Aware Re-Mapping
REBEL reliably finds the related pair but labels it with Wikidata-style terms. Using known entity types recovers more than half of the otherwise-discarded clinical edges — no retraining.
| Triplets reviewed | 150 |
| Entity pair correct | 78% |
| Relation label correct (raw REBEL) | 41% |
| Structural labels recovered | 27 / 38 |
| Relation label correct (re-mapped) | 63% |
Phase 2 — Knowledge Graph
Built from 500 abstracts with UMLS normalization. Every factual edge carries the identifiers of the papers that support it.
| Nodes / Edges | 1268 / 1312 |
| Most common edge (TREATS) | 612 |
| Provenance coverage | 100% |
| UMLS dedup (Disease nodes) | 142 → 119 (−16%) |
Phase 2 — QA Engine (V1 vs V2)
On a 60-question test set. V2 adds entity resolution against the graph, type-aware routing and a fallback chain — lifting the answer rate by 18 points.
| Measure | V1 | V2 |
|---|---|---|
| Answer rate | 70% | 88% |
| Intent classification | 90% | |
| Entity resolution | 85% | |
Phase 2 — Grounded QA vs. a Raw LLM
On 30 questions (5-point rubric). The decisive difference is evidence: the graph engine cites real, traceable papers on essentially every answer; the raw model rarely cites anything, and sometimes cites papers that do not exist.
| Criterion | KG-QA (V2) | Raw LLM |
|---|---|---|
| Relevance | 4.2 | 4.0 |
| Specificity | 4.0 | 3.4 |
| Citable evidence present | 4.6 | 1.2 |
Scope note: the Phase 2 graph is a demonstration-scale graph (500 abstracts) and the answer-quality rubric was scored by a single annotator; formal clinical validation and full-corpus ingestion are identified as future work in the thesis.
Stack