Upload speed unstable? Streaming, dropping frames

I've tested my upload speed consistently and it's always around +4 which is fine. But once I start streaming, I start dropping frames every so often. But with the upload speed I'm getting tested with, it shouldn't be dropping any frames. I'm not sure if my upload speed is fluctuating or something.

I've tested on two different streaming programs (OBS/XSplit) and to two different streaming sites (Twitch/YouTube). But I still get dropped frames. So I know it's not the programs or the servers. It didn't start happening until 3 days ago. I had my bitrate at 5000 and it was fine, but once I started dropping frames I changed it back to 2500 then to 2000. But I'm still dropping some frames.

I've also restarted my modem and router multiple times and reset it back to factory settings. But it hasn't helped at all.