Oh man, the video latency woes. I've been going through the same motions trying to get minimal-latency stream-to-browser from an Amcrest (Dahua) IP camera. Their app and cloud services leaves much to be desired, so I tried to roll my own. They cameras send h264-encoded stream over rtmp but also through some kind of custom HTTP protocol which their browser app (requiring a scary binary native plugin) decodes, but it's challenging to reverse-engineer.
>>lxe+(OP)
From experience trying to minimise latency of Raspberry Camera stream, two biggest factors of latency are H264 encoding (maybe because it's temporal) and RTMP.
MJPEG and UDP/TCP streaming work better (for me at least).