This is in practice not true at all. Vertical scaling is typically a sublinear cost increase (up to a point, but that point is a ridiculous beast of a machine), since you're (typically) upgrading just the CPU and/or just the RAM or just the storage; not all of them at once.
There are instances where you can get nearly 10x the machine for 2x the cost.
(and fwiw, wikipedia agrees with this definition: https://en.wikipedia.org/wiki/Scalability#Horizontal_(scale_... )
I think it's true to say that vertical scaling normally is done by increasing the RAM and CPU of a single machine with a single address space and switch/bus. While horizontal scaling is normally adding more machines (additional addresses spaces and switch/bus). Historically this is because RAM to CPU performance (throughput and latency) in a single address space and bus greatly exceeds the performance of any NIC connecting machines with distinct address spaces and busses. And it mostly ignores effects like the performance costs of swapping/paging when you don't have enough RAM.
I haven't really seen many systems where horizontal scaling is truly linear, unless the problem is embarassingly parallel, like serving static content.