I agree that you do legitimately need interview notes, but I don't understand why this conflicts with GDPR. In other words, why am I not allowed to see my interview notes?
But interview notes tend to contain personal evaluations of people, often critical. If interviewers believe they are effectively having to criticise people to their face (which is what this change would do), then they won't be willing to be as honest. No interviewer wants an angry job candidate tracking them down via LinkedIn or whatever and then getting mad because you wrote that they sucked in their notes.
This is an interpretation of the GDPR that I don't think makes any sense or aligns with the original intentions at all, but moreover, if it was interpreted and enforced that way it simply means firms would switch to discussing candidates in person and not write down evaluation notes at all.