I think that’s because (seconding Nathan) ambisonics are the way to go for this.
Pan2 is simple left/right panning. It is not binaural. Well, it is, in the sense of splitting the signal out into two signals (one for each ear). But “binaural” has taken on a more precise technical meaning of decoding a 3-D sound field using the characteristics of human ears (by way of a HRTF = head-related transfer function) so that the resulting two-channel audio when played through headphones sounds like 3-D. (Decoding a 3-D soundfield into two loudspeakers is not really “binaural” decoding in this sense – stereo, but not binaural.) To do that, you need a 3-D sound field to begin with, and ambisonics are the best supported way in SuperCollider to do that.
I would also second the recommendation to use the Ambisonic Toolkit instead of the built-in ambisonics UGens.
If your source is mono, you would use Foa (first-order ambisonics) transformations to push the source away from the listener’s position and rotate in three dimensions to the desired place, then do the binaural decoding.
Also, the built in ‘Rotate2’ feels like putting the sound behind me, but not in front of me.
I’m not deeply experienced with it, but one of the tricky things about binaural decoding is that you get the best results when the HRTF matches the listener’s ear shape. That’s hard to control for.
But the built-in ambisonics units, I think, are not using any HRTF at all, so it’s unlikely to get a convincing illusion.