It could be one of at least a dozen things!
My mates Rover 75 does this & we just can`t pinpoint what the hell it is (the place he got it from have changed balljoints, re-tracked it, even fitted new wishbones under warranty & it still does it) so I`m stumped!
Did you have anything done to the suspension/steering when you fitted the 17s (i.e: lowered and/or adjusted the tracking)?
If it had to be re-tracked when you fitted the alloys, you may need it re-tracked again now they`re not fitted!
Or, as I say, it could be worn suspension parts.
I hope that makes sense! :O :roll:
Whatever it is though, I hope you get it sorted soon for relatively minimal expense.
EDIT: It sounds like "Torque steer", but my 1.5 N16 never did it, with or without 17s on (mind you, I don`t think I ever took it above 4k!) :O