Historical datasets are expensive, hard to find

posted by gigadata , 826 days ago , show insights

With all the WallStreetBets hype I've been developing/testing automated investment/trading strategies in crypto/stocks. I was struggling to get large amounts of data cheaply like total historical stock/crypto prices or historical news, eCommerce pricing and stock levels. To resolve this I had to rely on heavy scraping or work with a variety of vendors who specialise in different datasets which increases infrastructure cost, risks getting IP blocked, increases load on the target server and increases data sourcing cost. I spent more time managing scrapers, bots and data streams instead of working on the analysis, research and building my app
WHO NEEDS THIS
Register Login to comment or vote on this problem
Voted!
Need karma! Please check submission guidelines.
Paid!
Why pay twice?