The cache is not related to this problem.
I think I was able to reproduce the problem.
This happens with "Allow Camera Control - On" + "Smooth Zoom - ON", right?
The problem occurs because mathematically every touch interaction is a little bit of rotation and a little bit of scaling at the same time.
Unfortunately, I right now do not have any ideas how to fix this.
I tried to work around this in several ways, but this makes the problem even worse.
It takes some time for experiments. Unfortunately, it is impossible to say exactly how much.
What can I suggest right now:
Handle gestures with some gesture framework (for example, TouchScript or EasyTouch).
I think this should help.
If such a way is suitable for your project, I can make such a script for you.
Infinity Code Team