Programming Language is Not a Drug to Treat Poor Parallelism

        In multicore computing, we usually expect more cores could speed up applications. However, that could not be always true. Every parallelism has its limit, beyond which more cores not only cannot speed up the computing but get worse.

        In the post "Parallelizing Loops on the Face of a Program is not enough for Multicore Computing", we have seen a poor parallelism that OpenMP cannot speed up optimized codes beyond three cores. The example gives us an opportunity to see that a poor parallelism could have a low limit as three cores, beyond the limit more cores cannot further improve the speed.

        Furthermore, experience also shows the parallelism limit does not vary with programming language. If we use different programming languages to parallelize the parallelism, the limit remains the same. Undoubtedly, poor parallelism leads to poor performance regardless the programming language, and programming language is also not a drug to treat poor parallelism.

        We are going to see the limit of the poor example implemented by different programming languages.

THE LIMIT WITH OPENMP

        First, we see the limit with OpenMP. We copy the OpenMP timing results from the post "Parallelizing Loops on the Face of a Program is not enough for Multicore Computing" in the following:


We can see from the above results: In the range of 1~3 cores, the elapsed time could be reduced when increasing number of cores. Then, more cores, beyond three, not only could not speed up computing but got worse. The poor parallelism makes OpenMP have no way to speed up the optimized code beyond 3 cores. The range is limited in 3.

THE LIMIT WITH NEULOOP

        Next, we are going to see the parallelizable limit with neuLoop.

        We parallelize the example program with neuLoop in the same environment as OpenMP, with the same option -O3. As introduced in a previous post, neuLoop has homogeneous and heterogeneous cores. First, we link against heterogeneous cores. The timing results are as:


It is consistent to the implementation with OpenMP that neuLoop cannot speed up the optimized code beyond three cores. The parallelism also limits neuLoop to three cores. When using four or more cores, the elapsed time not only cannot be reduced but gets worse. For example, four cores take 20.11 seconds; five cores take 22.26 seconds; eight cores take 30.95 seconds. Supposedly, more cores should shorten the elapsed time. However, the poor parallelism limits neuLoop to three-core improvement.

        Next, we link the example program against homogeneous cores, and get the following timing results:


We do not see a surprise. Homogeneous cores also show a consistent result that the poor parallelism limits neuLoop to three-core improvement.

        What decides performance?

        We know the answer. It is not programming language, but parallelism. However, parallel language cannot treat a poor parallelism. The key to parallel computing is to develop efficient parallelism that allows more cores to speed up computing.