This is in practice not true at all. Vertical scaling is typically a sublinear cost increase (up to a point, but that point is a ridiculous beast of a machine), since you're (typically) upgrading just the CPU and/or just the RAM or just the storage; not all of them at once.
There are instances where you can get nearly 10x the machine for 2x the cost.
So your example of adding more CPU cores would be horizontal scaling, while using a faster core would be vertical. Vertical scaling has diminishing returns.