Investment banks recruit for rise of big data analytics

Big data, or the large pools of data that can be captured, processed and then analysed, is now reaching into every sector and function of the global economy.

Financial services businesses, including the investment banks, generate and store more data than any other business in any other sector – broadly because it is such a transaction-heavy industry, often driven by models and algorithms.

Despite accumulating a wealth of information on capital market transactions, trades, financial markets, and other client and market data, the investment banks have been slower to embrace today’s definition of big data analytics than many consumer retail businesses, technology businesses, and even retail banking.


Organisations such as Amazon, Google, eBay and the UK’s big four supermarkets have been using big data analytics for many years, tracking consumer behaviour to suggest potential new products to consumers and develop customer loyalty schemes. Where investment banks have used big data, it has often been restricted to tracking individual sub-categories of asset classes.

The UK's high-street banks have also been increasingly active in this area, using data analytics to study purchasing patterns, social media and location data, in order to tailor products and associated marketing material to individual customers’ needs.

Using big data analytics to increase profitability

The investment banks are now looking at how they can use big data to do what they do better, faster and more efficiently.

Senior executives at the banks want to enhance how they use data to raise profitability, map out markets and company-wide exposures, and ultimately win more deals.

While banks have, for many years, used data and value at risk modelling to measure and quantify the level of financial risk in a portfolio of assets, the fundamental difference with big data is that it has become an established standalone functional department rather than a series of small subsets of internal business units.

Big-data teams are now taking on the role of an influential internal consultancy, communicating to senior executives key insights on how to improve profitability.

Another key difference is that the banks are now not only analysing structured data, such as market or trading data, but also unstructured data, which can include sources such as tweets, blogs, Facebook posts and marketing material. This is now collected and recorded from a bank’s customers or clients – a significant shift from how data used to be captured.

Using large amounts of both structured and unstructured data and market data, the investment banks are now accurately modelling the outcome of investment decisions, and getting real-time insights into client demand.

Big data is also a fundamental element of risk-profiling for the banks, enabling data analysts to immediately assess the impact of the escalation in geopolitical risk on portfolios and their exposure to specific markets and asset classes. Specifically, banks have now built systems that will map out market-shaping past events in order to identify future patterns.

We are also seeing the banks using big data to analyse the effectiveness of their deals, looking for insights into which trades they did or did not win on a client-by-client basis.
But despite the recent growth in the use of big data by the banks, key challenges remain.

Unlike retail and technology giants such as Google, Facebook and Amazon, or any new startup or fintech company, the IT and data systems at most banks were not originally constructed to analyse structured and unstructured data. Updating and remodelling entire IT and data systems to accommodate the systems needed to generate a deep analysis of a bank’s data is time-consuming and costly.

Banks that have merged or acquired other banks or financial services businesses are likely to face even more complex issues when incorporating and updating legacy IT systems.

Surge in hiring big data analytics specialists

The competition between banks and fund managers to hire big data specialists is heating up.

The banks are actively recruiting big data and analytics specialists to fill two main, but significantly different roles: big data engineers and data scientists/analytics/insights.

Big data engineers will typically come from a strong IT development or coding background and are responsible for designing data platforms and applications. A big data engineer can typically command £55,000 a year and may also be known as a software engineer – big data, big data software architect or Hadoop developer.

Data scientists, in contrast, are responsible for bridging the gap between data analytics and business decision-making, capable of translating complex data into key strategy insight.

Data scientists – also known as analytics and insights manager or director of data science – are expected to have sharp technical and quantitative skills. Data scientists are in highest demand and this is where the biggest skill shortage exists.

Data scientists are responsible for examining the data, identifying key trends, and writing the complex algorithms that will see the raw data transformed into a piece of analysis or insight that the business can use to gain a competitive advantage.

Such is the shortage of individuals with this skillset that a data scientist can command between £75,000 and £110,000 a year, straight from graduating from university.

Big data teams will often be competing to hire from the same pool of mathematics and physics PhDs from which other areas of the investment bank will be hiring