I'd suggest doing the image processing in C or C++ rather than Python; that 10x speedup will really make a difference if you're processing hundreds of frames.