I will discuss the modular properties of D3-brane instantons appearing in Calabi-Yau string compactifications. I will show that the D3-instanton contribution to a certain geometric potential on the hypermultiplet moduli space can be related to the elliptic genus of $(0,4)$ SCFT. The modular properties of the potential imply that the elliptic genus associated with non-primitive divisors of Calabi-Yau is only mock modular. I will show how to construct its modular completion and prove the modular invariance of the twistorial construction of D-instanton corrected hypermultiplet moduli space.