Artificial intelligence is rapidly modernising pharmacovigilance (PV). It now offers powerful capabilities for processing Individual Case Safety Reports (ICSRs), detecting safety signals, and predicting adverse drug reactions.
However, as AI systems become more sophisticated, many rely on complex “black-box” models. These models can produce highly accurate results, but their internal reasoning is often difficult for humans to interpret. This raises an important question:
How can PV professionals and regulators trust decisions they cannot clearly understand?
Confidence Scores as a First Layer of Explainability#
In practice, many organisations developing AI tools for PV—such as case intake systems, literature monitoring tools, or signal detection platforms—publish a confidence scorecard alongside extracted information from both structured and unstructured data sources.
Confidence scores are widely recognised as a component of an explainable AI (xAI) framework. In fact, CIOMS Working Group XIV explicitly lists confidence scores and trust scores as examples of explainability techniques.
In real-world PV workflows, organisations often use confidence score thresholds (for example, requiring a score of ≥ 0.8) as a key validation and control mechanism for automated processes.
This approach is extremely valuable because it allows systems to automatically flag predictions or extractions that fall below an acceptable certainty level. Those cases can then be routed for human review.
By providing a transparent, case-by-case indication of how confident the model is in its output, confidence scores help enable a Human-in-the-Loop (HITL) governance model. They guide reviewers on when they can rely on the AI and when closer human oversight is needed.
Why Explainable AI Matters for Regulatory Compliance#
To meet the expectations of global regulators such as the EMA and FDA, organisations should design AI systems that are transparent, interpretable, and auditable.
Two widely used explainable AI frameworks that support this goal are:
When an AI model’s internal logic becomes too complex for humans to interpret directly, explainable AI techniques help generate plausible explanations of how the model reached a decision.
Both SHAP and LIME aim to answer one key question:
Imagine the AI’s prediction is the final score of a game, and each data feature—such as patient age, symptoms, or laboratory results—is a player contributing to that score.
SHAP calculates exactly how much each feature contributed to the prediction. A feature can either increase or decrease the likelihood of the final outcome.
In pharmacovigilance, SHAP has been used to explain supervised machine learning models for signal validation classification.
When an AI system evaluates a potential safety signal, SHAP can generate a feature contribution report. This report shows how much weight the algorithm assigned to different variables when determining whether a signal should be considered valid.
For example, the model might reveal that:
Time-to-onset contributed strongly to signal detection
Patient age had moderate influence
Co-reported medications slightly reduced the signal probability
This transparency helps human reviewers understand—and trust—the model’s reasoning.
In pharmacovigilance, LIME can be applied to clinical narratives.
For example, an AI system analysing a case report might classify an event as non-serious. Using LIME, the system can highlight the words that influenced the decision.
If the explanation reveals that the AI focused heavily on the drug name (for example, an over-the-counter medication) rather than the medical outcome, developers can identify and correct the bias.
Before using SHAP or LIME in a regulated environment, the explainability method itself should be treated as a component of a computerized system.
This means it must undergo appropriate validation procedures to demonstrate that it is fit for purpose and compliant with GxP requirements.
Explainable AI does not eliminate the need for validation. Instead, organisations should establish processes to ensure that explainability tools remain accurate and reliable over time.
Imagine an AI triage system that frequently misclassifies serious adverse events as non-serious.
An xAI analysis might reveal that the model is incorrectly using the drug name as a proxy for seriousness—for example, assuming that events involving an over-the-counter medication are less likely to be serious.
Once identified, developers can adjust the model or rebalance the training data to remove the bias.
For narrative ICSRs, visual explanations can be particularly effective.
One example is the FDA’s Information Visualization Platform (InfoViP), which uses NLP to highlight important sections of case narratives. Colour-coded text helps reviewers quickly see why the AI flagged a case.
This approach improves transparency and allows reviewers to validate AI outputs more efficiently.
Transparency should never compromise patient privacy.
Processing large health datasets requires strict adherence to regulations such as GDPR and HIPAA, along with principles like data minimisation and anonymisation.
Finally, organisations shoud train PV professionals on the limitations of explainable AI.
SHAP and LIME provide approximations of model behaviour, not perfect explanations.
One major risk is automation bias—the tendency for humans to trust AI outputs too readily, especially when accompanied by convincing explanations.
A plausible explanation attached to an incorrect prediction can still lead reviewers to accept the result without sufficient scrutiny.
For this reason, PV teams should treat explainability outputs as decision support tools, not definitive answers.
Human judgement should always remain the final authority.
Explainable AI is becoming an essential part of responsible AI adoption in pharmacovigilance.
Tools like SHAP, LIME, and confidence scoring mechanisms can significantly improve transparency and trust in AI systems. However, successful implementation requires more than just technical integration.
Organisations should combine:
robust validation
strong governance
privacy safeguards
proper user training
When implemented thoughtfully, explainable AI can strengthen Human-in-the-Loop pharmacovigilance systems, ensuring that innovation enhances—rather than replaces—expert human oversight.
