Disconnect the smaller wire at the starter motor first and see if it still running on. This will isolate the vehicle's wiring as the cause.

Double checked that the mounting bolts for the starter motor are tight.

Are you sure it's the correct starter motor?