Historical datasets are expensive, hard to find

posted by gigadata , 852 days ago , show insights

With all the WallStreetBets hype I've been developing/testing automated investment/trading strategies in crypto/stocks. I was struggling to get large amounts of data cheaply like total historical stock/crypto prices or historical news, eCommerce pricing and stock levels. To resolve this I had to rely on heavy scraping or work with a variety of vendors who specialise in different datasets which increases infrastructure cost, risks getting IP blocked, increases load on the target server and increases data sourcing cost. I spent more time managing scrapers, bots and data streams instead of working on the analysis, research and building my app
Register Login to comment or vote on this problem
Need karma! Please check submission guidelines.
Why pay twice?