We will review Witten's ideas for the construction of the partition function of the self-dual field on a Riemannian manifold $M$. Then we will explain how geometric quantization on the intermediate Jacobian of $M$ can be used to find the metric dependence of the partition function. We will also show how the local gravitational anomaly of the theory is recovered in this formalism. Applying these results to the $(2,0)$ supermultiplet on a Calabi-Yau threefold, we will show that its one-loop determinant coincides with the one-loop determinant of the B-model.