We all have those brief periods in our lives that shape us somehow, mine was seeing Dylan in 1978 at Blackbushe in the UK - he was just releasing the Street Legal album. On the bill was Eric Clapton (just released Lay Down Sally) and Joan Armatrading (Down to Zero). I was in my teens then and it was an incredible concert (I didn't inhale, honest).
Ever since I've been a Dylan fan. Same goes for Zappa too....
But I would agree that his sound could be rather irritating to some!