Are you seeing the modem switch to a different set of channels around that time? Look at your signal page and see what range(s) of frequencies you're seeing. The most common ranges are 8xx Mhz, 9xx MHz, and 4xx MHz. Compare what set of frequencies you have when you're getting good speeds versus slow speeds. There could be a problem with one of the channel sets, and you only see them problem when your modem is using that set.