My assumption has been that the vocative usage of 'man' was originally associated particularly with urban African-American speech and got picked up by youth culture during the 1950s and 1960s, much as was happening with other urban African-American slang and dialect usages.
The trajectory of 'dude' in the 1980s (possibly given a significant push by the film Fast Times at Ridgemont High) was somewhat similar to that of 'vocative->emphasis man', though 'dude' of course had been a term that had earlier on been used as a label for certain categories of men (e.g. the 'surfer dude' and the much earlier usages that go back to the 19th century).