Yes, have been rather fortunate to have had access all over the Cape really. Combined with a passion for fishing and camping, and it's been great.
Battlecamp Rd used to be terrible, you're right. It is like a highway at the moment though, with quite a number of bitumen sections when driving right across to/from Cooktown.
The Sexchange hotel at Coen is a great spot for a cold beer. There is a decent little museum over the road too. It never gets much of a look in, but there is a lot of old machinery there, and stories of the area. There is also a couple of 'interesting' characters in Coen who are worth visiting. One bloke there has built his house out of cement and bottles and has a stack of old mining gear lying around. Visitors are always welcome to have a look there too.
There is a good drive from Port Stewart down to Marina Plains along the coast too. Not many are aware of it, and it does go through land owned by the Lama Lama people, but there is some good fishing to be had down there too. You can now camp at Port Stewart in a designated location for a fee payable to the Traditional Owners. Not sure how much it is, but the fishing down there can be pretty damn good if you have a tinny, and can time the tides right, particularly around the mouth of the Stewart River, and a heap of little creeks to both the North and South.