Whether you’re starting your own hedge fund or just trying to survive freshman economics or your summer internship, stock prices and economics data are a very common request for web scraping projects. Since most Finance sites tend to be overloaded with advertising and tetchy about scrapers, this can be a little challenging at first.
Never fear, your R programming brethren have created more than a few packages that will give you easy access to this information.
First up – the quantmod package. This can be obtained from CRAN with the ubiquitous:
install.packages("quantmod")
Quantmod is a powerful package, with a wide range of graphing and analytical functions. Our primary focus will be on the data retrieval component.
This part is actually very simple. They’ve created a helper function called getSymbols(). This goes out to one of several sites and pulls back a data series for a particular stock or economic indicator. This is deposited into your work space labeled with the ticker symbol.
The package supports a variety of sources, including Google Finance, Yahoo Finance, and FRED (Federal Reserve’s database in St Louis).
library(quantmod)
getSymbols(“SPY”) #S&P 500 data
getSymbols('DGS10',src='FRED') #10 year treasuries
There is a large variety of data available at FRED, including many economic indicators.
Other good sources of economic information include the Worldbank library, which we profile here.
Incidentally, quantmod doesn’t stop with data retrieval. The package includes a series of functions for common financial modeling calculations (working with open / high / low / close data) and working with daily time series data. There is also a nice charting library which supports common statistical price analysis measures (known in the trade as “technical analysis” of stock prices). This includes studies such as moving averages, bollinger bands, RSI (Relative Strength Indicator), and custom charting formats such as candlestick charts.
For more information on web scraping, particularly for grabbing information from financial news sites, check out the other articles in our guide to web scraping:
- Grabbing Tables and Custom Fields using rvest (CSS selector field targeting) – good for financial news sites
- Web Scraping JSON data from interactive websites – another good way to pull data off certain financial sites