Rein in Runaway Costs: A Deeper Look at Market Data

March 30, 2023 by Cliff Gerber

Reducing the cost of market data and market information management


Luxoft’s Clifford B Gerber takes a deep dive into market data cost management and how financial institutions can make cost savings on financial information.


As we move further into 2023, the news has been riddled with articles regarding bank layoffs and the impending economic downturn casting a dark cloud over Wall Street. Pre-existing technical projects (e.g., digital, cloud technology migration, application modernizations, etc.) and regulatory initiatives (T+1 settlement, ESG, crypto and digital) still remain at the top of the agenda for most firms. In this volatile environment, many clients are asking where they should be looking to reap potential cost savings in an effort to help their bottom lines. While there is no single panacea, solutions can range from eliminating manual processes through intelligent automation, moving technology to an ‘as-a-Service’ model, or looking to rationalize technology redundancies where possible. One area firms are revisiting is evaluating their market data usage, capital markets software, contracts, and associated spend.

As market data is the lifeblood of both buy- and sell-side firms, it represents a major and growing cost for industry players. Machine learning, new regulations, and advanced analytics have increased the overall demand for data, and commensurately, increased internal expenditures on the requisite data governance and oversight processes. External market data vendors understand that, to change providers, switching costs are high, and it can include involved procurement steps, information security analyses, and complex technology changes. Fees for market data increase year-on-year and are often not aligned to the external market conditions or the purchasing firm’s performance.

With global financial market data spend reaching $35.6B in 2021, it is certainly an area that warrants further examination. A report by Burton-Taylor indicates that one driver here is that the uses of market data have extended beyond the trading desk and into middle- and back-office functions. Additionally, COVID-driven remote work has led data providers to produce more flexible data feeds and pushed usage further into mobile solutions, increasing consumption and subsequent fees.


Are the regulators following this issue?


Regulators globally are attuned to this issue, including the stronghold data firms have on market participants. In fact, concern is so great in this area that, in 2022, the Financial Conduct Authority (FCA) initiated two market studies reviewing concerns that the limited competition in the markets for benchmarks and indices, credit ratings, and trading data increases costs for both firms and investors.

In the US, the SEC proposed its ideas in the Market Data Infrastructure Rule (MDIR), which would both expand the amount of available ‘core’ data to more closely align to what firms actually need, and would require more competition by allowing ‘competing consolidators’ to obtain new data from exchanges and sell their own data products at market-determined prices. If data costs are lower, more brokers will be able to compete by trading directly on exchanges, giving investors more choices and reducing market concentration.


How does my firm compare to our peers?


Perhaps most alarming to firms should be the incredible disparity industry research groups have discovered when comparing pricing amongst peers. Mike Carrodus, CEO of Substantive Research and Editor of TabbForum, performed a study of index markets and noted the following observations:

  • Pricing is not consistent. There is a large variation in pricing, and little or no correlation to the size of the firm consuming the market data. Even accounting for specific ‘apples- to- apples’ use cases, the range of inconsistencies applied to pricing post-negotiation is between 10% and 50%.
  • The pricing that buy-side institutions receive for supplying a single index from the same provider differs by an average dispersion of 21%. The overall range from lowest price to highest price can be as high as 219%, meaning some institutions are paying more than twice as much as peers.
  • For reporting licenses, the average dispersion in pricing can be up to 37%, and the range from lowest to highest can be as high as 472%. Here, some institutions are paying almost five times more than their peer group.


What can firms do to reign in their market data costs?


An A-Team Data Management Review study indicated that 79% of respondents currently track usage of information services their organization is paying for. However, most of these firms track informally through surveys or rely exclusively on vendors to supply usage information. Firms also discussed complexities with tracking across 100+ products, as well as difficulties in tracking data that is used from a web-based information source.

Firms can immediately benefit from doing a holistic assessment of their data environment by organizing their data vendors and usage across functions, geographies, and desks. Data organizations should centrally define the metrics and benchmarks to be considered when making data-related decisions. When considering usage patterns, firms should consider a deep-dive into data contract terms, a review of duplicative data sources, or the usage of niche providers that can be moved to a more ubiquitous firm to drive scale and cost efficiencies. Rationalizing the usage of duplicative data sources presents a large opportunity for firms and requires minimal technical intervention.

Secondly, firms should develop data-related dashboarding and BI that can be used to make informed decisions on a go-forward basis in order to optimize usage, renew lean contracts, and promote cost efficiencies throughout the organization.

A proper assessment is merely the first step firms should take regarding market data spend. Following this exercise, there are a number of additional focus areas that firms should consider in order to optimize data usage:

  • Modernizing technology stacks toward a data-centric architecture that emphasizes optimizing data usage and reducing costs
  • Utilizing open-source data platforms (where possible), which can lower costs by allowing firms to access and share data with others
  • Employing artificial intelligence and machine learning, which can optimize data distribution capabilities in a cost-efficient manner
  • Leveraging data normalization and aggregation services to minimize the need for multiple data feeds
  • Building data lakes and warehouses to store and manage large amounts of data, allowing firms to process data in-house instead of paying for external storage and processing services
  • Consideration of additional analytical tools and package platforms that help with interpolating existing data and reducing market data usage. Some popular options include: R (quantmod), Python (pandas, NumPy), and Matlab ("Financial Toolbox" and "Econometrics Toolbox")
  • Leveraging package solutions (e.g., Murex, Calypso, Finastra Fusion) that have built-in capabilities to that provide interpolation of data, thereby reducing market data consumption.


The above actions do represent significant cost and time investments for firms, but harnessing data costs and making foundational changes can pay dividends across organizations. While exact cost savings will vary based on the type and size of the firm, studies show that initial savings opportunities can be $350K+ USD for smaller firms and can scale quickly based on number of products and data providers.

If you’d like to discover how Luxoft can help your firm navigate your cost optimization journey, visit or contact We’d welcome the opportunity to go over the benefits you can expect in your unique situation and the excellent business potential going forward.

For more Banking and Capital Markets insights, see Luxoft on LinkedIn

Cliff Gerber , Director, Technology and Strategy Advisor

Cliff Gerber

Director, Technology and Strategy Advisor

Cliff has over 15 years’ experience driving operational and technology solutions for the world’s largest banks. He has led numerous highly complex business, regulatory and technical projects, and has been responsible for the entire software development lifecycle in onshore and offshore environments, from planning through to testing and deployment. He has also supported multiple global banks in complex submissions and responses regarding their financial, operational, and technical capabilities to various banking and capital markets industry regulators.

Related content


Managing a large scale of applications
Managing a large scale of applications


Managing a large scale of applications


Under pressure? A Temenos core banking system health check helps you do more with less
Under pressure? A Temenos core banking system health check helps you do more with less


Under pressure? A Temenos core banking system health check helps you do more with less


Drafting a mainframe hybrid cloud for banking roadmap? Start with “Why?”
Drafting a mainframe hybrid cloud for banking roadmap? Start with “Why?”


Drafting a mainframe hybrid cloud for banking roadmap? Start with “Why?”