More info, it seems to be at it's worst when I'm travelling between 60 and 90km/h. Also seems that once it drops below 2000RPM in that speed range is when the issue really makes itself known.
Another thing is it seems to also depend on how I drive, if I crawl up to speed on my drive home from work (60km) it doesn't seem to happen, but it's well below normal pickup speed. Basically it mainly happens if I put my foot down a little bit too hard and then have to slow down a little bit. Sorry if this isn't reading very well I'm finding it difficult to explain.