The Line of Beauty by Alan Hollinghurst

"...but he felt the relief of being alone as well: the usual relief of a guest who has closed his door, and a deeper thing, the forgotten solitude which measures and verifies the strength of an affair, and which, being temporary, is a kind of pleasure."