It was only six years to go from the first multi-person spacecraft and first spacewalk to the first space station.
Yeah that's my entire point, technological process doesn't have a constant rate of acceleration. Some advances are quickly made one after another and others lag and take a very long time.
Even if technological progress stopped we could have launched enough parts to assemble a colony structure.