#programming # [[Epistemic status]] #shower-thought #to-digest # Changelog ```dataview TABLE WITHOUT ID file.mtime AS "Last Modified" FROM [[#]] SORT file.mtime DESC LIMIT 3 ``` # Related # TODO > [!TODO] TODO # Distributed web crawler In the context of https://inkdrop.ai I implemented a distributed web crawler that downloaded millions of products from the biggest e-commerces in the world. I'll probably open source the code and the datasets except if we sell the code. <iframe src="https://app.excalidraw.com/l/5MKXLddifTr/2JE40fKlb1m" width="100%" height="600px"/> # External links