Years Long Packet Loss, T3, & T4 Timeouts Never Addressed

Question

I've been experiencing random packet loss to specific hosts and providers, apparently throttled speeds to certain providers, and T3 &amp; T4 timeouts on the local node for YEARS. Additionally, latency to many "east coast and beyond" locations is terrible, to say the least. Traceroutes were provided via an FCC complaint well over a year ago and no action seems to have been taken. All Cox offered to do was send a tech out - worthless waste of time, packet loss isn't occurring here, and Cox has been out at least 3 times over the years, never finds anything wrong, and continues to blame the customer equipment for the issues. The last time they even wanted ME to pay to have a tech out to fix THEIR network. RIDICULOUS!
So, where to start...&nbsp;

Problem 1:
UDP traffic is being throttled at approx 15Mbps downstream at all times of day. This means that I cannot reliably stream 2160p 4K UHD content from providers such as GOOGLE, who uses QUIC protocol - the new HTTP/3 STANDARD and future of all internet traffic on devices such as a Roku because it will constantly buffer. I KNOW this is the problem because if I block QUIC at the firewall, the Rokus will fall back to HTTP/2/TCP and the streams operate flawlessly to the exact same Google CDN node in Los Angeles. I should not have to block QUIC in the firewall to get streaming video from Youtube to work, and I can't imagine a typical customer doing this or even knowing how to do it.&nbsp;

Problem 2:&nbsp;
IPv6, again, the FUTURE of the internet, is unreliable on Cox's network, at best, and has to be disabled if I don't want to constantly chase strange, intermittent connectivity issues and packet loss. Again, I can't imagine a typical customer even knowing that this is the problem nor how to resolve it (if they even can with Cox provided equipment)

Problem 3:&nbsp;
Persistently high latency to many sites located on the east coast that traverse the link between Los Angeles and Atlanta. 90ms is a typical minimum and its often 120ms or 150ms+. This is frankly unacceptable in the year 2022 and has major impact on real time voice and video communications over the internet which are now pretty much a standard. It is extremely frustrating to keep "stepping on" others in video chat because of ridiculous latency out of the west coast.

Problem 4:
Packet loss to Amazon. This has been a persistent, ongoing issue for YEARS and results in perpetual problems with IOT devices. Many other companies also use EC2 instances for their services and therefore are also affected. This shows up in randomly malfunctioning Amazon Echo devices, a WiFi thermostat that won't stay connected to the "cloud" (just checked today, finding 7% packet loss on the COX network using pingplotter and pretty bad latency), lights that randomly will not turn on, or off. Talk about IRRITATING.
Problem 5:
T3 &amp; T4 timeouts to the local node. This has been going on for at least 6 years. Also accompanied by random outages and random modem reboots. Cox always blames my modem, but refuses to troubleshoot without their modem installed and refuses to provided one for testing at no cost (because I can guarantee, the problem isn't my modem). Earlier this year, I bought TWO new modems. The high end one wouldn't even sync properly to the Cox network once provisioned and was EXTREMELY slow. I had them provision a lower end model which has been successful and yet, I still have T3 and T4 timeouts. This modem runs straight to the DEMARC and is the ONLY device plugged into Cox with no splitters and only one F-F extension connector (-1.5db) and the actual COX box that feeds the street is actually in the sidewalk in front of my house.&nbsp;
---
Here are a couple traceroutes from tonight going to Amazon... see a problem here? (68.1.211.7 I'm looking at YOU)
Target Name: ec2-3-213-146-164.compute-1.amazonaws.com IP: 3.213.146.164 Date/Time: 8/11/2022 7:39:09 PM - 8/11/2022 7:49:09 PM
Hop Sent PL% Min Max Avg Host Name / [IP] 1 239 0 1.20 132.08 39.81 OpenWrt.lan [192.168.1.1] 2 239 0 7.18 179.18 46.53 10.70.216.1 [10.70.216.1] 3 239 0 8.65 182.90 46.99 100.127.5.190 [100.127.5.190] 4 239 0 7.77 184.41 50.46 100.120.102.40 [100.120.102.40] 5 239 0 50.59 210.71 121.85 68.1.211.7 [68.1.211.7] 6 239 0 44.38 300.08 138.50 52.95.216.76 [52.95.216.76] 7 239 0 41.99 273.74 131.74 15.230.183.50 [15.230.183.50] 8 239 0 53.15 276.47 125.70 15.230.183.59 [15.230.183.59] 9 239 100 0 0 0 [-] 10 239 100 0 0 0 [-] 11 239 100 0 0 0 [-] 12 239 100 0 0 0 [-] 13 239 100 0 0 0 [-] 14 239 100 0 0 0 [-] 15 239 0 80.04 471.65 143.58 52.93.28.82 [52.93.28.82] 16 239 100 0 0 0 ec2-3-213-146-164.compute-1.amazonaws.com [3.213.146.164]
Target Name: ec2-34-231-42-65.compute-1.amazonaws.com IP: 34.231.42.65 Date/Time: 8/11/2022 7:40:08 PM - 8/11/2022 7:50:08 PM
Hop Sent PL% Min Max Avg Host Name / [IP] 1 239 0 1.30 203.55 45.31 OpenWrt.lan [192.168.1.1] 2 239 0 7.52 185.23 46.31 10.70.216.1 [10.70.216.1] 3 239 0 8.43 143.10 43.79 100.127.5.188 [100.127.5.188] 4 239 0 7.90 187.23 54.66 100.120.102.36 [100.120.102.36] 5 239 0 51.85 190.15 123.23 68.1.211.7 [68.1.211.7] 6 239 0 61.32 263.15 137.44 52.95.216.76 [52.95.216.76] 7 239 0 68.32 366.94 126.11 15.230.183.74 [15.230.183.74] 8 239 0 45.01 306.11 124.72 15.230.183.81 [15.230.183.81] 9 239 100 0 0 0 [-] 10 239 100 0 0 0 [-] 11 239 100 0 0 0 [-] 12 239 100 0 0 0 [-] 13 239 100 0 0 0 [-] 14 239 100 0 0 0 [-] 15 239 0 69.01 418.81 141.62 52.93.28.134 [52.93.28.134] 16 239 100 0 0 0 ec2-34-231-42-65.compute-1.amazonaws.com [34.231.42.65]

and straight from the router (no WiFi):

traceroute to 34.231.42.65 (34.231.42.65), 30 hops max, 38 byte packets
1 10.70.216.1 (10.70.216.1) 10.905 ms 6.633 ms 6.842 ms
2 100.127.5.188 (100.127.5.188) 8.619 ms 6.194 ms 7.754 ms
3 100.120.102.36 (100.120.102.36) 7.632 ms 7.151 ms 7.855 ms
4 68.1.211.7 (68.1.211.7) 43.879 ms 40.959 ms 93.429 ms
5 52.95.216.76 (52.95.216.76) 41.556 ms 52.95.216.84 (52.95.216.84) 40.623 ms 52.95.216.76 (52.95.216.76) 39.281 ms
6 15.230.183.34 (15.230.183.34) 42.193 ms 15.230.183.50 (15.230.183.50) 39.778 ms 15.230.183.60 (15.230.183.60) 39.392 ms
7 15.230.183.81 (15.230.183.81) 41.274 ms 15.230.183.41 (15.230.183.41) 38.761 ms 15.230.183.65 (15.230.183.65) 46.057 ms
8 * * *
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
14 52.93.28.124 (52.93.28.124) 69.437 ms 52.93.28.106 (52.93.28.106) 75.524 ms 52.93.28.116 (52.93.28.116) 111.615 ms

traceroute to 3.213.146.164 (3.213.146.164), 30 hops max, 38 byte packets
1 10.70.216.1 (10.70.216.1) 6.761 ms 8.402 ms 7.223 ms
2 100.127.5.190 (100.127.5.190) 7.441 ms 18.677 ms 9.615 ms
3 100.120.102.40 (100.120.102.40) 9.643 ms 6.239 ms 8.096 ms
4 kscydsrj02-ae3.0.rd.ks.cox.net (68.1.1.197) 39.137 ms 68.1.211.7 (68.1.211.7) 38.771 ms kscydsrj02-ae3.0.rd.ks.cox.net (68.1.1.197) 53.392 ms
5 52.95.216.84 (52.95.216.84) 50.196 ms 40.997 ms 52.95.216.76 (52.95.216.76) 44.213 ms
6 15.230.183.72 (15.230.183.72) 38.584 ms 15.230.183.34 (15.230.183.34) 40.987 ms 15.230.183.62 (15.230.183.62) 41.603 ms
7 15.230.183.43 (15.230.183.43) 44.002 ms 41.568 ms 39.012 ms
8 * * *
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
14 52.93.28.102 (52.93.28.102) 72.709 ms 52.93.28.82 (52.93.28.82) 68.210 ms 52.93.28.76 (52.93.28.76) 68.277 ms

Here's another one with weird and bad routing (out to LA first, then back to Las Vegas then on to Reston, VA? 60+ms on the LAS to REST hop?) I know Telia isn't Cox's network, but it is their PEER, and a LOT of traffic tends to use this route. Also, this is pretty light latency tonight. That LAS-&gt;REST hop is very often in the hundreds of ms. I guess I'm bewildered as to why a provider as large as Cox can't provide their own backhaul to the east coast.&nbsp;
traceroute to seemenow.icu (154.16.118.207), 30 hops max, 38 byte packets 1 10.70.216.1 (10.70.216.1) 5.659 ms 7.430 ms 6.548 ms 2 100.127.5.190 (100.127.5.190) 6.223 ms 7.822 ms 7.476 ms 3 100.120.102.40 (100.120.102.40) 10.835 ms 6.716 ms 9.775 ms 4 68.1.4.252 (68.1.4.252) 15.088 ms langbprj02-ae1.0.rd.la.cox.net (68.1.1.14) 13.278 ms 68.1.4.252 (68.1.4.252) 12.989 ms 5 las-b24-link.ip.twelve99.net (62.115.169.168) 12.654 ms 15.280 ms 51.788 ms 6 rest-bb1-link.ip.twelve99.net (62.115.137.36) 70.562 ms rest-bb1-link.ip.twelve99.net (62.115.114.87) 67.781 ms 68.289 ms 7 nyk-bb1-link.ip.twelve99.net (62.115.141.245) 74.693 ms 80.065 ms 74.837 ms 8 buf-b1-link.ip.twelve99.net (62.115.118.123) 86.445 ms 100.361 ms 87.654 ms 9 colocrossing-ic317200-buf-b1.ip.twelve99-cust.net (62.115.145.91) 86.504 ms colocrossing-ic314280-buf-b1.ip.twelve99-cust.net (62.115.59.86) 85.362 ms colocrossing-ic314282-buf-b1.ip.twelve99-cust.net (62.115.59.94) 83.511 ms

coxlv5115 · Answer

To further illustrate the problem... pay attention to Amazon US east in particular... I can route to Toronto, CA and Tokyo, JP faster:

Region
Latency

Amazon Web Services™

us-east-1 (Virginia)
147 ms

us-east-2 (Ohio)
147 ms

us-west-1 (California)
34 ms

us-west-2 (Oregon)
136 ms

ca-central-1 (Central)
97 ms

eu-west-1 (Ireland)
177 ms

eu-west-2 (London)
167 ms

eu-west-3 (Paris)
165 ms

eu-central-1 (Frankfurt)
174 ms

eu-south-1 (Milan)
180 ms

eu-north-1 (Stockholm)
176 ms

me-south-1 (Bahrain)
241 ms

af-south-1 (Cape Town)
313 ms

ap-east-1 (Hong Kong)
189 ms

ap-southeast-3 (Jakarta)
202 ms

ap-south-1 (Mumbai)
280 ms

ap-northeast-3 (Osaka-Local)
150 ms

ap-northeast-2 (Seoul)
167 ms

ap-southeast-1 (Singapore)
188 ms

ap-southeast-2 (Sydney)
167 ms

ap-northeast-1 (Tokyo)
130 ms

sa-east-1 (São Paulo)
190 ms

cn-north-1 (Beijing)
193 ms

cn-northwest-1 (Ningxia)
208 ms

us-gov-east-1
76 ms

us-gov-west-1
52 ms

DigitalOcean

SFO3 (San Francisco)
34 ms

TOR1 (Toronto)
unavailable

NYC3 (New York City)
92 ms

LON1 (London)
unavailable

AMS3 (Amsterdam)
171 ms

FRA1 (Frankfurt)
169 ms

BLR1 (Bangalore)
unavailable

SGP1 (Singapore)
193 ms

IBM Cloud

us-south (Dallas)
65 ms

us-east (Washington DC)
86 ms

eu-gb (London)
162 ms

eu-de (Frankfurt)
173 ms

au-syd (Sydney)
191 ms

Linode

Newark (New York area)
89 ms

Singapore
284 ms

London
159 ms

Frankfurt
169 ms

Dallas
53 ms

Toronto
88 ms

Sydney
226 ms

Atlanta
70 ms

Tokyo
149 ms

Mumbai
288 ms

Fremont (San Francisco area)
27 ms

OVH Cloud

SBG (France)
204 ms

GRA (France)
185 ms

UK
175 ms

DE (Germany)
191 ms

WAW (Poland)
236 ms

coxlv5115 · Answer

Here is the event log from the cable modem:

rity
Description

Time Not Established
Critical (3)
SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=d4:04:cd:98:0f:3e;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.0;CM-VER=3.0;

Time Not Established
Critical (3)
No UCDs Received - Timeout;;CM-MAC=d4:04:cd:98:0f:3e;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.0;CM-VER=3.0;

Time Not Established
Critical (3)
SYNC Timing Synchronization failure - Failed to acquire QAM/QPSK symbol timing;;CM-MAC=d4:04:cd:98:0f:3e;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.0;CM-VER=3.0;

Time Not Established
Critical (3)
No Ranging Response received - T3 time-out

Time Not Established
Critical (3)
No Maintenance Broadcasts for Ranging opportunities received - T2 time-out;CM-MAC=d4:04:cd:98:0f:3e;CMTS-MAC=00:29:c2:01:86:47;CM-QOS=1.0;CM-VER=3.0;