Even if I disable swap, when Linux gets very low on memory you get the exact same symptoms when it starts discarding increasingly-active program code.