Insights
February 28, 2023

A Strong Foundation: Building With Financial Data

In this piece, we’ll explore the various types and sources for financial data with an eye toward software application and API developers. If you’re building a financial services app, this piece will give you everything you need to start from a strong foundation.

Cloud icon

Karl Hughes

https://iexcloud.io/community/blog/a-strong-foundation-building-with-financial-data
@iexcloud
https://iexcloud.io/community/blog/a-strong-foundation-building-with-financial-data
@iexcloud

Crunchbase lists tens of thousands of venture-funded financial services startups operating today, and in 2021 alone, these companies raised $130 billion.

While that number was down in 2022 as venture capitalists reigned in commitments across the board the sector has generally seen a massive uptick in the past decade.

“The major winners will be financial services companies that embrace technology.” – Alexander Peh, PayPal and Braintree

The boom in financial services businesses has coincided with a growth in data availability in all industries. Financial data is now more accessible than ever before, and that has contributed to a huge increase in demand and the number of use cases for it. Everyone from traditional financial advisors to news outlets to AI trading tools is using financial data to make better-informed decisions and improve investment outcomes.

In this piece, we’ll explore the various types and sources for financial data with an eye toward software application and API developers. If you’re building a financial services app, this piece will give you everything you need to start from a strong foundation.

What is Financial Data?

Financial data refers to data related to the financial performance of a company or industry. It typically includes figures like revenue, profit, expenses, and balance sheet items, but it can also include information about the company's stock price, shares outstanding, and trading volume.

With thousands of publicly traded companies around the world, there is a lot of financial data out there, and as you’ll see in this piece, collecting, organizing, and storing even a portion of it is no small feat.

Meanwhile, interest in non-traditional financial data is also growing. For example, some investors are now factoring environmental, social, and governance (ESG) data into their decisions. Investment firms also use social media sentiment, geographic data, and even internet of things (IoT) data to shape their trading strategies.

As machine learning models and more advanced AI techniques continue to develop, we’ll see even more interesting use cases and types of data used in financial technology applications, but let’s take a look at some of the most common use cases for financial data today.

Use Cases for Financial Data

In the past few years, consumer interest in digital trading applications has skyrocketed. Popular mobile apps like Robinhood have made investing in stocks, cryptocurrencies, and other securities easier and faster than ever before, and all these tools rely heavily on fast and accurate financial data.

Robinhood uses financial data to inform retail investors

In addition to lightweight consumer apps like Robinhood, there are more advanced tools that offer expanded data and insights to investors. For example, SoftCapital’s TOP trading terminal layers news and historical financial data into its browser-based trading platform.

SoftCapital trading terminal

Similarly, many portfolio management applications rely on financial data to inform and guide users. For example, Simply Safe Dividends uses this data to help dividend investors monitor their cashflow and portfolio valuation over time.

Simply Safe Dividends uses financial data to inform dividend investors

News sites and aggregators are often large consumers of financial data as well. Seeking Alpha, a popular stock market community and analysis tool leverages historical stock data and company financials to augment recommendations from users.

Seeking Alpha stock price data

Search engines like DuckDuckGo even integrate financial data into their search results.

DuckDuckGo integrates stock data into search results

Finally, algorithmic and AI-driven trading applications rely on fast and accurate financial data to make buy-sell decisions more quickly than human traders can. IEX Cloud and FreeCodeCamp offer a free course on building an algorithmic trading platform in Python so if you’d like to learn more about the technical specifics of this field, that’s a good place to start.

Building Software With Financial Data

While financial data is easier to get than ever before, and the number of possible applications for it continues to grow, there are some things to consider before you start building an application that relies on it. One of the first is where you’ll get the data from, so let’s take a look at how you can choose the right provider and the various types of financial data sources out there.

Choosing a Financial Data Provider

Depending on the type of application you’re building, you may weigh some factors more than others, but most developers will look at the following:

Data Coverage and Quality

Your data provider should offer comprehensive coverage of the markets and instruments that are relevant to your business. Most providers have some limitations. For example, data feeds from a single exchange often don’t include details on companies listed on other exchanges.

The data should also be accurate, up-to-date, and reliable. In cases where a slow response or inaccurate number can lead to millions of dollars in losses per minute, this is a make-or-break consideration. Additionally, data providers should handle and help you handle changes. For example, what happens when a company changes its stock ticker (e.g., [`FB` to `META`)? It’s important that your data provider uses best practices for symbology, financial data normalization and change management.

Integration Capabilities

In most cases, you’ll be integrating your financial data with other applications or partner data, so it’s important to consider whether the format and delivery mechanism will make this possible. I’ve seen situations where normalizing data from multiple sources proved to be a huge technical headache, so it’s always easier to choose providers with pre-built integrations.

Security and Compliance

In addition to providing accurate data, good providers will have robust security measures in place to ensure that sensitive financial data are protected. This will help you remain in compliance with relevant regulations and standards, plus it speaks to the provider’s professionalism and accountability.

Support

Training, documentation, and technical support are all important factors. Some financial data providers are very hands-off, giving you a single data format and no option for support or customization while others may go as far as helping you implement their data into your application.

Pricing and Scalability

While you want the best data possible, you also need the budget to afford it. It’s important to weigh the cost of each data provider with the services they offer, though. Saving a few hundred dollars per month at the cost of 20 extra engineering hours per week is not a good tradeoff. You also need to consider how well their pricing model scales, as switching from one data provider to another is exceedingly difficult.

Innovation and Development

Finally, look for financial data providers that are constantly innovating and developing new features. You want to ensure that your application will have access to the latest market data and analytics tools, so choose a provider who’s actively maintaining and expanding its offerings.

Financial Data Sources

Now that you know what you’re looking for in regard to financial data and how to evaluate providers, let’s explore some of the options available. As with any high-level engineering decision, there’s no single *best* option on this list, so you’ll have to consider your team’s existing resources and your application’s requirements.

Stock Market Data Providers

There are a number of companies that specialize in providing stock market data, such as Refinitiv, S&P Global Market Intelligence, IEX Cloud, and FactSet. These providers offer a wide range of data, including historical data, real-time data, and analytics tools that will help you build robust financial applications and APIs more quickly, but the price of some of these options can vary dramatically. That said, IEX Cloud lets you start with low-cost pay-as-you-go plans with many data bundles to choose from.

Stock Exchanges

Many stock exchanges, such as the New York Stock Exchange (NYSE), IEX Exchange and the NASDAQ also provide market data through direct data feeds connected to their data centers. The downside is that each exchange can only provide data for the buy and sell orders on its own order book, so you may have to do a lot of work integrating data services to get a full view of the stock market. This leads again to the challenge of normalizing data formats, which is no small consideration. 

 Financial Websites and Portals

Websites like Yahoo Finance, Google Finance, and Bloomberg provide stock market data and other financial information that can be useful for new product development. While their datasets and tools may not be as comprehensive as purpose-built stock market data providers, they could be cheaper or easier to use if your business already subscribes to these services.

Government Agencies

Government agencies like the Securities and Exchange Commission (SEC) also provide market data, including financial reports and filings from public companies. This data is free, but it isn’t real-time, and because of the organizational structure, it’s harder to navigate than some of the other options here. That said, it could be a good way to augment data you already have for publicly traded companies.

Social Media and News Websites

While many social media platforms have reigned in the amount of data they offer freely in the wake of the Cambridge Analytica scandal, you can still get real-time data on public posts from Twitter and many news aggregators. Financial applications often use this data to inform users about market trends, stock movements, or investor sentiment.

Pitfalls: Protecting Your Data and Business

The accuracy and quality of financial data available varies widely. It’s worth noting that there are many unscrupulous and unregulated sources for financial data, but these are rarely worth the risk.

It may also be tempting to scrape data from other websites, but it’s incredibly hard to ensure the accuracy and integrity of this data. You are also putting your company at legal risk as this kind of behavior typically violates terms of service and data ownership laws.

You should also be careful of how data is transferred from third parties. Unsecured HTTP requests or manual data entry can lead to data security and integrity issues. While having a formal data governance plan in place is a good idea and can help catch errors, it’s not a replacement for using reputable data providers with the proper best practices in place.

Finally, don’t get caught without a disaster mitigation and recovery plan in place. No matter how reliable your provider or your internal data practices, no technology is perfect.

Developing Your Own Financial Data API

Most modern applications wrap their data layer in a developer-friendly API, so likely, one of your first steps in building with financial data will be developing the API that internal and external stakeholders will use.

So, I’ll finish up this piece with a brief outline of the challenges you have ahead. If you’ve already got a team in place, go through this list with them to ensure everyone feels comfortable with your capacity and the path forward.

Financial Industry Expertise

Building useful applications with financial data will require a deep understanding of the industry, as well as the technical expertise to develop the API itself. This often requires a multidisciplinary team of developers, financial experts, data scientists, and analysts.

Sourcing and Organizing Financial Data

Once your API is created, it must be sourced, or connected to various financial data providers. This typically involves negotiating access and establishing agreements with these providers. You’ll also need to set up a data retrieval and differencing system to ensure it stays up-to-date and that formatting inconsistencies are resolved.

Securing the Data

In addition to creating and seeing your API, you have to ensure that access to it is secure and reliable while maintaining ease of use. API keys and OAuth is a good starting point, but you should also consider roles and permissions. If you’re building an API designed for external consumption, you also need to consider potential abuse, so features like rate limiting and usage monitoring are especially important.

Designing, developing, sourcing, and securing your financial APIs is a big undertaking, but it’s obviously essential for building a robust finance application. Fortunately, you don’t have to build it yourself from the ground up. Tools like IEX Cloud’s Apperate offer a real-time streaming data platform specifically for finance applications, greatly decreasing your time to market and the engineering resources required in this endeavor.

Conclusion

As financial data becomes more readily available and interest in creating new financial technology grows, the financial data space is likely going to get even more exciting. In this piece, you’ve learned about the foundations of building applications and APIs with financial data, some of the data providers available, and best practices for selecting one.

One of the easiest ways to start building on top of financial data is by leveraging a reputable, high-quality financial market data provider. IEX Cloud and its streaming data platform Apperate can help you get the data and tooling you need to build faster, more secure financial applications. So, sign up for a free trial to get started today.