Reinforcement Learning · Knowledge Graphs
The Mechanism Matters: When Knowledge Graphs Help Reinforcement Learning
Knowledge graphs (KGs) are widely used to inject prior knowledge into reinforcement learning, yet the literature is dominated by single-domain, positive-result method papers, so we lack a systematic account of when KG structure helps an agent, when it is neutral, and when it hurts. We run a controlled study that independently varies the RL task, the injection mechanism (state features, action masking, or potential-based reward shaping), and KG quality over a synthetic, fully controllable KG on MiniGrid. Structured guidance improves sample efficiency and solve reliability on compositional sparse-reward tasks, and a shuffle control that permutes the KG's edges while preserving their count collapses the benefit toward baseline (masking p=0.0001; shaping p=0.006), so the gain is structural rather than generic regularization. Most consequentially, safety depends on the mechanism: soft, optimality-preserving injection benefits from correct knowledge and harmlessly ignores incorrect knowledge, whereas hard masking is brittle and can make a wrong KG worse than no KG. A UMLS-derived clinical case study on MIMIC-IV sepsis management under offline RL is a careful null, underscoring that benefits require task structure the chosen mechanism can exploit.
70% → 97%
Solve reliability
seeds solved
p = 0.0001
Shuffle control
masking, d=1.08
6 × 2
Envs × learners