Internet upload speed dropping.

My internet upload speeds are usually 20Mbps. But around 5-10pm they "magically" seem to be dip down to less than 1Mbps before going back to normal. Even during the day my download and upload speeds ...