Technical consulting is provided by HPC technology experts on three main areas: 1.Algorithms optimisation and parallelization, to make a customer application run faster, 2. Parallel IA application, to make model training run faster on multiple compute unit, 3.Containerization, the configuration and deployment of workflows for Big Data in container
- Optimization and parallelization: The main goal of High Performance Computing(HPC) is to reduce the time to solution of the algorithm, and this is generally addressed in two steps.
The first step is the optimization of the algorithm and the code on the available hardware architecture. First of all, application profiling is needed to find the bottlenecks, then the part of code affected to time losses will be rewritten in an optimized way or ported on accelerators, like GPUs.
The second step is the code parallelization, both on a single server and, if needed, multiserver. In this case, the algorithm and the code will be made parallel by identifying the most time-consuming functions and acting on those. To parallelize a code on a single server multi-thread libraries will be used, like OpenMP. On multiserver, Message Passing Interface (MPI) libraries are required.
- Parallel IA: Two strategies can be implemented in artificial intelligence to reduce a model training time: data parallelism or model parallelism.
The model parallelism implies a division of the model among multiple computational units, but it requires a large dimension model to have a good speed up.
The simplest and most used strategy is the data parallelism: on each computational unit, the model is replicated and trained with a subset of the data and at each step the parallel models are synchronized together.
- Workflow and Containerization: Real-time predictive services and data processing services are enabled by complex and dynamic workflows that adapt to the size and type of data, the response time required and the resources available. To achieve systems with such flexibility, HPC must be supported by cloud resources or integrated with container-based workflow orchestration systems.
Increased performances in Montecarlo simulations for Risk Management for a large Italian insurance company
Increased performances in AI applications of a financial Italian company.