Key trends from the 2014 High Performance Computing maturity benchmark
As the premier High Performance Computing (HPC) consultancy in the Financial Services sector, Excelian conducts a technology benchmark on the adoption of HPC technologies in the market every year. This survey is free and conducted in an anonymous fashion, the detailed results being shared exclusively with the 17 Tier I and Tier II investment banks who took part this year.
Key trends emerging from the survey are as follows:
• The market share for compute grid middleware continues to shift significantly. IBM Platform Symphony becoming the prevalent solution for large scale installations: the solution is now deployed on more than 50% of the cores on the market. This trend will probably accelerate in the next 18 months, specifically following the uncertainty around the future of TIBCO who has been acquired by an investment fund just recently. There is definitely space now for a new player in this market, which could either be Microsoft HPC Server, now used by a few players on the market, or maybe an Open Source competitor, the likes of Apache Storm for instance offering interesting possibilities when it comes to distributing calculations.
• 25% of the banks surveyed are now leveraging multiple types of resources as part of their compute stack, the resources being scavenged nodes (workstations/servers), cloud-based nodes or GPU accelerator nodes. The constant push towards higher cost efficiency is the main driver here and making sure that expensive resources are only used for SLA-critical workloads has become a key approach to optimising compute cost.
• GPU adoption continues to progress with 37% of the market using the technology as part of their stack (it was the case for 25% of the market 18 months ago). Xeon Phi hasn’t managed to seriously challenge NVIDIA’s GPUs: the initial performance challenges and the extended time to market have been relatively disappointing and probably explain the low level of adoption so far.
• NoSQL solutions are starting to be adopted across the board in Investment Banking with 29% of the banks surveyed either using MongoDB, Cassandra or HDFS/Hadoop as part of their compute stack. DataGrid solutions are still in use to optimise data distribution in the HPC environments, NoSQL solutions being used mostly as inbound/outbound data stores for the core compute engines. However, it is likely that they will progressively start to cannibalise the DataGrid market, mainly for cost reasons but also because it is becoming increasingly complex to maintain two different technologies dedicated to data management.
At a high level, High Performance Computing solutions have entered a new area. The patterns and architecture for standard compute grid solutions are well known and understood. Excelian described this commodity model in the previous grid maturity benchmark edition: a vendor-based middleware deployed across large compute farms, shared at the enterprise level between applications which leverage pools of commoditized compute.
Compute still remains a key asset for most banks and a significant number of banks will see their compute estate increase by 20% to 30% as regulatory requirements and more specifically Credit Value Adjustment /Credit Counterparty Risk related requirements continue to fuel this increase. The cost pressure that the industry is going through is, on the other hand, pushing technology innovation and stretching the model of the on-premise utility grid. New hardware platforms are more and more prevalent: Cloud is being used more widely and alternative and Open Source solutions are being used more and more aggressively.
If you wish to participate in the survey, measure your grid maturity against the rest of the market and get access to the full report of the study, please get in touch at firstname.lastname@example.org.