No problems with that setup at all, it will work fine.
The two main reasons for having dual battery systems are;
1. The second battery is for deep cycle low current use and permanently on board as is the load IE fridge etc all stay with the vehicle
or
2. You need a dual battery for high amp applications on the vehicle IE lots of winching, humungous doof doof sound system, etc
If all you do is chuck a battery in the back when you go camping and the fridge etc all stays at the camp then your idea is fine