Why don’t we simply add more CPUs/Cores/Threads to increase the speed of calculation in our computer?
Answer is simple because the limitation isn’t just the number of cores—it’s how much of the workload can actually run in parallel and how efficiently those cores can be utilized.
Intuition might say “more cores = more speed”
That would be true if every single task can be parallelized, but in real world we can’t make everything parallel, for example: if part of your program depends on the result of a previous computation (e.g., processing steps in a pipeline), those steps must run sequentially.
*Amdahl’s Law: The maximum speedup is limited by the sequential portion of the program.*
Amdahl’s Law formula
$$ S(N) = \frac{1}{(1 - P) + \frac{P}{N}} $$
Where:
S(N) = speedupP = fraction of code that can be parallelizedN = number of cores/threadsExample: 80% of our code is parallel (P = 0.8) 20% is sequential if we have 4 cores then:
$$
S(4) = \frac{1}{(1 - 0.8) + \frac{0.8}{4}} = \frac{1}{0.2 + 0.2} = \frac{1}{0.4} = 2.5 $$
As you can see we got 2.5x not 4x,
Even if we had infinite CPUs
$$ S(∞)=\frac{1}{(1−P)}=5 $$
If P is 0.8 max speed up would be 5x
More cores = more communication. At some point, the cost of coordinating becomes bigger than the benefit of extra cores.