Internet Speed Much Lower than 100Mbps and Drops Dramatically

I currently have the up to 100Mbps package with Cox but my speeds never seem to get even remotely to that speed. I can usually get 20-25Mbps on a wired connection using a router, but roughly 4-5 times a day the speed drops dramatically down to 5 Mbps or lower. Sometimes it will go down as low as >1Mbps and become pretty much unusable for streaming or gaming, which are the main reasons I have the service. Other times my modem will lose connection and the internet will drop completely. I do not have any issues with the modem or router, it is definitely on Cox's end. I'm not as concerned about reaching top speed as I am getting usable speeds for streaming and gaming on a consistent basis. Any answers or help with this issue is much appreciated.