thanks! that did it. Honestly I'm not even sure how it works but it does haha.
instead of physics ray i did this which I assume is the same logic and seems to work so far.
Vector3 pos =cam.transform.position + cam.transform.forward * panoRenderer.radius;
panoRenderer.GetPanTiltByWorldPosition(pos, out pan, out tilt);
panoRenderer.pano.pan = pan;
panoRenderer.pano.tilt = tilt;
I set this up a few weeks back and had to do some translation of the rotations based on the master file text that it came with. Its all working now but something happens with that translation when I go around a corner the North must change or something where I would end up "jumping" a bit when panning with the mouse because although my camera angle didn't change the pan/tilt values do. so when i go to drag view it jumps at first
sorry long description but anyway if I call this method and reset the pan/tilt to match the camera angle each time that seems to be good.