Running Spark GPU applications on OpenPower with Smart Scheduling of IBM Platform


Abstract to be presented at Summit 2016

The presentation introduces how to run Spark GPU applications using IBM Platform LSF on Power Linux.
With LSF Spark Connector, user is able to run Spark application without consideration of host allocation and spark service maintained. With smart GPU scheduling algorithm, LSF is able to allocate GPU(s) with consideration of GPU-CPU affinity and GPU runtime utilization. A performance comparison of Spark GPU application on both X86 and OpenPower with Tesla K80 GPU will be given.

Speaker Bio

Zhaohui Ding got his Ph.D degree in 2009 and has 10 years experience in distributed computing and high performance computing. He currently is the senior product architect of IBM Platform LSF Product Line, leads R&D teams of IBM Platform LSF Family in China.