Aortic-Flow Reference Shaping via Q-Learning for CF-LVAD Support

Abstract

Continuous-flow left ventricular assist devices (CF-LVADs) operate at fixed speeds, suppressing pulsatility, prolonging aortic-valve closure, and impairing root washout, thereby elevating thrombotic risk. To address these limitations, we propose a Q-learning-based framework that synthesizes a smooth, bandwidth-limited aortic-flow reference without explicit plant modeling. The reference is parameterized by waveform coefficients optimized offline through interactions with a cardiovascular simulator, guided by a composite reward that penalizes deviations in mean arterial pressure, encourages periodic aortic valve opening, prevents suction events, and maintains sufficient and proper flow. In simulation, the learned waveform maintained mean arterial pressure between 60-90 mmHg, left-ventricular pressure between 5-70 mmHg, pump flow between 2-5 L/min, ventricular volume between 50-140 mL, and a pulsatility index around 2. Importantly, the aortic valve opened approximately 6 times per 20 s, improving washout while avoiding suction. These results underscore the potential of reference-level learning to restore physiological pulsatility and preserve hemodynamic safety in CF-LVAD support.