Your steering query.
Your castor wont help but you are getting a few small things happening that makes it feel that way.
You will be getting a small amount of rear steer. This is normal for a patrols rear suspension geometry. Your castor as you have said wont be helping. Also as your front left hand corner rises your steering arm will slightly alter in its position. So all very minor things that equate to what your feeling. You have a good chunk of torque now so you will be getting quite a bit of movement under hard acceleration.
I know your rear end is in good nic since you have just fitted all that new superior gear.