Scalable data lake drives agile response to change
Since the 2008 global financial crisis, the bank, like all financial institutions has faced growing pressure to comply with increasing regulatory requirements, including the Basel Accords, MiFID II, Dodd-Frank, EMIR, and FRTB.
This, combined with fierce competition, has driven the need to move faster than ever to deliver change. To achieve this, the bank wanted to create a data lake, i.e. a repository of transformed data readily available to perform aggregation and analysis.
We leveraged big data technologies, specialist network expertise and state-of-the-art industry toolsets to design and develop an innovative data lake architecture.
Overcoming a range of technological challenges, the scalable, adaptable solution gave the bank the capabilities to reliably:
• Adapt to and ingest the various feeds of information in a performant manner
• Enrich and transform these feeds into formats that would enable easy analysis
• Provide access points to extract the required information
To deliver all the functionality required, where readily available technologies didn’t provide the features to implement a particular requirement, we took an innovative approach to create customized solutions
Fulfilling all the requirements, the new data lake has given the bank seamless access to a range of powerful new data management capabilities.
• Agile data analytics: The ability to inspect and analyze data from its different operations, enables the bank to quickly apply required regulatory changes and respond to change in a timely and agile manner
• Data exploration: Centralizing data in one location makes it easy to explore and analyze real-time activities across the bank, supported by visualization and data science tools to access new insights, efficiencies and opportunities
• High quality data: With data stored centrally, data cleansing methodologies are applied to maintain clean, defragmented data. This can then be shared with groups and systems across the bank to support processes and enhance operational performance
• DevOps deployment: With best-of-breed DevOps practices – including continuous integration, automated deployment and quality assurance procedures in place, the bank has the framework to ensure the rapid, high-quality implementation of new features
• Shared expertise and experience: Building on the training and knowledge undertaken by our Excelian consultants, the bank now has a team of people in-house equipped to enhance and future proof the data lake
Even with challenging timelines, our team went the extra mile to ensure delivery on-time and within budget.