For two-dimensional holographic CFTs, we demonstrate the role of Berry phases for relating the non-factorization of the Hilbert space to the presence of wormholes. The wormholes are characterized by a non-exact symplectic form that gives rise to the Berry phase. For wormholes connecting two spacelike regions in gravitational spacetimes, we find that the non-exactness is linked to a variable appearing in the phase space of the boundary CFT. This variable corresponds to a loop integral in the bulk. Through this loop integral, non-factorization becomes apparent in the dual entangled CFTs. Furthermore, we classify Berry phases in holographic CFTs based on the type of dual bulk diffeomorphism involved and show that each type of Berry phase corresponds to a spacetime wormhole geometry in the bulk.