Spent about 10 minutes following him. Got attacked by 2 mages the first time (frost attacks which slowed me too much, and the second time by two bears and a wolf, by the time I'd seen them off he vanished again.
If I do get to a quest, it better be bloody worth it!