Flapping foil for thrust generation in fluids is a widely studied topic with various applications such as propulsion by birds, insects, and fish, micro air vehicles, unmanned surface/underwater vehicles, and flapping-wing power extractions. Controlling and optimising thrust in flapping foils is challenging due to the complex nature of fluid-structure interaction systems that vary in different environments. Traditional optimisation methods involve expensive simulations without gradient information. To address this, we present a high-fidelity fully-differentiable fluid-structure interaction solver with gradient information for thrust control and optimisation. The solver has several benefits, including its GPU-accelerated one-fluid framework with a stabilised finite element fluid solver that enables fast, accurate, and differentiable fluid-structure interaction simulations. Its differentiable capability is enabled through automatic differentiation, allowing for gradient-based optimisation and easy integration with machine learning algorithms. Third, reverse mode automatic differentiation for high-fidelity transient fluid simulation requires often impractical or unavailable memory, which is reduced through checkpointing. Lastly, the flapping foil propulsion system is optimised with the fastest time of arrival. We demonstrated the computational efficiency and differentiable of our high-fidelity solver through gradient-based optimisation.