Historical datasets are expensive, hard to find

posted by gigadata , 194 days ago

1656 all time unique views, mostly from US, GB, IT and 3 weekly views, mostly from US, BR

With all the WallStreetBets hype I've been developing/testing automated investment/trading strategies in crypto/stocks. I was struggling to get large amounts of data cheaply like total historical stock/crypto prices or historical news, eCommerce pricing and stock levels. To resolve this I had to rely on heavy scraping or work with a variety of vendors who specialise in different datasets which increases infrastructure cost, risks getting IP blocked, increases load on the target server and increases data sourcing cost. I spent more time managing scrapers, bots and data streams instead of working on the analysis, research and building my app
Register Login to comment or vote on this problem
Need karma! Please check submission guidelines.
Why pay twice?