Real-world efficacy modelling
Statistical modelling workflow for real-world efficacy analysis: beyond simple comparisons, towards understanding which patient subgroups benefit and why. The constraint was typical of clinical data, confounding, missingness, privacy limits, and the expectation that results would inform actual treatment decisions.
Confidential: hospital + pharma collaboration (NL)
Milestone
Delivered
actionable insight
The blocker
Symptom
Standard analyses answered 'does it work on average?' but not 'for whom, under what conditions, and why?'
Root cause
Observational data with selection bias; treatment received correlated with severity. Naive comparisons were misleading; no uncertainty quantification.
Why it persisted
Analysis teams lacked methodology for causal modelling under confounding; privacy constraints made external data linkage impossible.
What was built
System-level. What it actually is: inputs, outputs, users.
-
Cohort definition pipeline: reproducible patient selection with documented inclusion/exclusion logic and sensitivity checks.
-
Feature engineering and validation: clinical feature definitions aligned with domain knowledge, validated against known outcomes.
-
Statistical modelling workflow: causal framing, covariate adjustment, uncertainty quantification, results with explicit confidence bounds.
-
Subgroup analysis framework: heterogeneous treatment effect estimation to identify which subgroups drove overall outcomes.
-
Decision framing: output structured as decision-relevant questions rather than model outputs.
-
Interfaces: inputs: structured EHR extract and treatment data; outputs: analysis reports and decision-framing summaries; users: medical and commercial teams.
Architecture diagram
D2How we evaluated it
What "working" meant: baselines, metrics, guardrails, failure modes.
Definition of working
Analysis is robust to documented assumptions; sensitivity analyses confirm conclusions hold under alternative specifications.
Metrics tracked
-
Sensitivity analysis: conclusions stable under variation in key assumptions
-
Calibration: predicted probabilities align with observed event rates by subgroup
-
Missingness analysis: results stable across imputation strategies
Failure modes checked
-
Residual confounding: unobserved variables correlated with both treatment and outcome
-
Positivity violations: subgroups where only one treatment arm is represented
-
Model misspecification: linearity assumptions in non-linear outcome relationships
Milestone
Delivered
actionable insight
Statistically rigorous analysis of outcome drivers delivered. Subgroup findings informed commercial and medical strategy. Details under NDA.
Why it was hard
Constraints that shaped every decision.
Confounding
treatment assignment correlated with disease severity in ways that required explicit causal modelling, not just adjustment.
Missingness patterns
data missing not at random; imputation strategy had to be defended, not just applied.
Privacy constraints
data could not leave the hospital environment; analysis had to run in a restricted compute environment.
Interpretability requirement
findings had to be legible to both medical and commercial stakeholders, not just statistically correct.
What comes next
If continuing: next hypotheses, next system increment, next risk gate.
- 1
Prospective validation cohort
pre-register the analysis plan and run on an incoming patient cohort to test predictive validity.
- 2
Causal discovery
use structure learning to surface previously unknown predictor relationships rather than testing pre-specified hypotheses only.
- 3
Decision support integration
embed the subgroup model into a clinical decision support tool, connecting strategy insight to point-of-care action.
Built with EU traceability + oversight expectations in mind.
Security-aware GenAI integration patterns. (ISPE)
Book the 30-min triage: you leave with a plan.
No demo, no deck, no pitch. A structured conversation about your specific situation.