#distributed-computing #machine-learning #8-bit-nn #transformer #deep-learning #continuous-batching #computer-vision #wildlife-monitoring Created at 210523 # [Anonymous feedback](https://www.admonymous.co/louis030195) # [[Epistemic status]] #shower-thought Last modified date: 210523 Commit: 0 # Related - [[Computing/Distributed Computing]] - [[Computing/Intelligence/Machine Learning/8 bit nn]] - [[Computing/Intelligence/Machine Learning/Transformer]] # Deep learning continuous batching at scale When I was a master student, while working on writing software to intercept satellite communications and display it on maps on mobile devices, I got into another project in collaboration with ecology researcher. They had cameras in forest that would collect images about animals and would could animals manually for many hours. I saw this and said, ok, I'm going to save you 5 daily hours by automating this process with computer vision. So I built this pipeline where people would upload images and videos to a web app and it required a "continuous batching" to process this content quickly and efficiently. How it works is that I would run online inference on the deep learning model unless reaching a high enough demand (in a queue) where I would start running batch inference. I arrived to this optimization by trying different strategies. <iframe src="https://link.excalidraw.com/readonly/XpOvk6wR7tE3m7Wm4t47" width="100%" height="600px" style="border: none;"></iframe> https://louis030195.medium.com/cloud-computer-vision-for-wildlife-monitoring-a50bfee6bef5