Post a screenshot of your modem levels and modem log. Your modem is the best indicator of what's going on between your house and the CMTS.
For example, if you modem log is showing constant DRW errors, that's an issue between your modem and the CMTS.
If you're getting T3 errors, again, that's an issue between your modem and the CMTS.
If you're modem log is clear and your power levels look good and you're still getting latency issues, it could be an issue with your network infrastructure OR your ISP.
Either way, your modem stats should be step 1 in troubleshooting this.