Amdahl's law

Amdahl's law, named after computer architect Gene Amdahl, is concerned with the speedup achievable from an improvement to a computation that affects a fraction F of that computation where the improvement has a speedup of S. For example, if an improvement can speedup 30% of the computation, F will be 0.3; if the improvement makes the fraction affected twice as fast, S will be 2. Amdahl's law states that the overall speedup of applying the improvement will be

To see how this formula was derived, assume that the running time of the old computation was 1, for some unit of time. The running time of the new computation will be the length of time the unimproved fraction takes (which is 1-F) plus the length of time of the improved fraction takes. The length of time for the improved part of the computation is the length of the old running time divided by the speedup, making the length of time of the improved part F/S. The final speedup is computed by dividing the old running time by the new running time, which is what the above formula does.

Amdahl's law is a demonstration of the law of diminishing returns: while one could speed up part of a computer a hundred-fold or more, if it only affects 12% of the overall task the best the speedup could possibly be is times faster.

In the special case of parallelization, Amdahl's law states that if F is the fraction of a calculation that is sequential, and (1-F) is the fraction that can be parallelised, then the maximum speedup that can be achieved by using P processors is

In the limit, as P tends to infinity, the maximum speedup tends to 1/F. In practice, price/performance ratio falls rapidly as P is increased once (1-F)/P is small compared to F.

As an example, if F is only 10%, the problem can be sped up by only a maximum of a factor of 10, no matter how large the value of P used. For this reason, parallel computing is only useful for either small numbers of processors, or problems with very low values of F: so-called embarrassingly parallel problems. A great part of the craft of parallel programming consists of attempting to reduce F to the smallest possible value.

References:

Gene Amdahl, "Validity of the Single Processor Approach to Achieving Large-Scale Computing Capabilities", AFIPS Conference Proceedings, (30), pp. 483-485, 1967.

External links

Reevaluating Amdahl's Law