Dynamic Work Load Balancing for Compute Intensive Application Using Parallel and Hybrid Programming Models on CPU-GPU Cluster

In a current trend compute intensive applications have a most important role in handling the huge amount of data in various fields such as commercial, scientific and engineering. This paper deals with the speedup comparison of compute intensive applications merge sort using parallel programing [OpenMP] and hybrid programming [MPI+OpenMP] on CPUs cluster environment and on GPUs cluster using CUDA and even on CPU+GPU cluster using hybrid programming models [MPI +OpenMP+CUDA]. The key challenge for the CPU+GPUs platform is allocation of workload among CPU and GPUs cores and to efficiently overcome the overhead of various inter and intranode communications. To address these challenges we have, proposed novel analytical workload division strategy to dynamically distribute the workload and single MPI process per node to overcome communication overhead. We have observed in case of merge sort on CPUs cluster using hybrid programming model [MPI+OpenMP], we have achieved 2.34× times speedup against parallel programming model [OpenMP]. And even we have observed the merge sort in case of GPUs and CPU+GPUs hybrid cluster using CUDA and hybrid [MPI+OpenMP+CUDA] programming model, we have achieved an average 1.97× times of speedup against hybrid parallel programming model [OpenMP+MPI], and on CUDA 1.91× times of improvement.