This is a known issue with both the Production FW and the Beta FW that Wyze is aware of and working on:
Wyze Cam Pan v3 - Does not return back to home position
It has also been noted that the cam is prone to chase it’s own IR emitter reflection when panning toward the house.
Pan Cam3 chasing it’s own IR reflections
It has been indicated by Wyze and repeated here in the forum that the Wyze Floodlight auxillary USB Power Supply produced enough power to run the Pan V3 which usually requires the factory shipped 2 amp power supply. However, I have not yet seen confirmation of that from someone employing it.
You might test it by plugging the cam into it’s 2 amp power supply about the same distance from the router and see if it behaves differently. If it does, it will confirm if the Auxillary USB Power Supply on the Wyze Floodlight is providing ample power.