We study an $SL(2,Z)$ symmetry of a variant of BCOV theory in three complex dimensions. Using conjectural descriptions of twists of superstrings in terms of topological strings, we argue that this action can be thought of as a version of S-duality that preserves an $SU\left(3\right)$-invariant twist of type IIB supergravity. We analyze how $SL(2,Z)$ acts on various deformations of the holomorphic-topological twist of 4-dimensional $N=4$ supersymmetric gauge theory, which come from residual supertranslations and superconformal symmetries, and are of relevance to geometric Langlands theory and gauge-theoretic constructions of the Yangian.