I'll wager that a smartly driven, non-abused car with modifications will be immensely more durable and desireable than a non-modded car that has been rid hard and put away wet. For that matter, a car that sits unused and undriven year after year is likewise problematic. I am convinced that some guys are just born as "high impact" users. These are the guys that drive over parking islands while drunk, who take their cars through brushed car washes, who have no idea how to drive a clutch, and store stuff ON TOP OF THE CAR while it sits in the garage.
I'll take a modded car from the original, fastidious owner 10 times to 1 over the car that has an uncertain history and shows signs of wear at the hands of the non-car guys of the world. In a strange way, this becomes the ART of buying a used exotic.