Slowness? Post your trace routes here!!

[Eriktank]

This morning was the normal "megaslow" speed - so I figure starting a thread to help Don and other Ops people work out the slowness would be beneficial... Here's my trace route to hera.smugmug.com during the slowness...

C:\Windows>tracert hera.smugmug.com
Tracing route to hera.smugmug.com [63.81.134.23]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms cipafilter.emsd37.org [10.10.0.1]
2 1 ms 1 ms 1 ms 209.7.177.1
3 3 ms 3 ms 3 ms 77.128.175.209.rtc5.illinois.net [209.175.128.77]
4 16 ms 12 ms 14 ms 249.7.166.206.rtc5.illinois.net [206.166.7.249]
5 51 ms 24 ms 16 ms atm-5-0-0-6-moline-core-rockford-core.rockford.lincon.net [206.166.9.190]
6 12 ms 12 ms 14 ms atm-1-1-0-sub05-rockford-core-dekalb-core.dekalb.lincon.net [206.166.9.106]
7 12 ms 12 ms 12 ms 206.166.9.25
8 27 ms 12 ms 12 ms pos-5-0-nap-sob1-nap-sob2.chicago.lincon.net [206.166.9.122]
9 269 ms 259 ms 191 ms 206.166.126.14
10 468 ms 472 ms 500 ms chi1.ge4-0-2.core1.iad1.hopone.net [66.36.224.217]
11 522 ms 466 ms 467 ms ed-pri.smugmug.com [206.223.137.74]
12 * * * Request timed out.
13 98 ms 104 ms 98 ms hera.smugmug.com [63.81.134.23]
Trace complete.

[GKP]

Seems more server related then network.. I uploaded a photo about ten minutes ago thats still showing "processing"..

Heres my trace.. 119 MS is very acceptable


Tracing route to gkp.smugmug.com [63.81.134.23]
over a maximum of 30 hops:

1 1 ms <1 ms 2 ms 10.0.0.1
2 9 ms * 9 ms 65.14.252.27
3 11 ms 12 ms 11 ms 65.14.255.137
4 12 ms 11 ms 14 ms adsl-156-253-64.bct.bellsouth.net [66.156.253.64
]
5 20 ms 13 ms 15 ms axr00bct-so-0-1-2.bellsouth.net [65.83.237.136]

6 12 ms 12 ms 13 ms axr01mia-so-1-1-0.bellsouth.net [65.83.236.11]
7 50 ms 13 ms 11 ms pxr00mia-so-1-0-0.bellsouth.net [65.83.236.20]
8 27 ms 27 ms 26 ms 65.57.174.1
9 28 ms 105 ms 28 ms so-7-0-0.mp2.Miami1.Level3.net [4.68.112.45]
10 44 ms 42 ms 45 ms ae-2-0.bbr1.Washington1.Level3.net [4.68.128.201
]
11 40 ms 43 ms 41 ms ae-11-69.car1.Washington1.Level3.net [4.68.17.3]

12 50 ms 53 ms 52 ms ed-pri.smugmug.com [206.223.137.74]
13 * * * Request timed out.
14 120 ms 119 ms 119 ms hera.smugmug.com [63.81.134.23]

Trace complete.


-GKP

[chmod]

But it does bog down once I reach it, which is quickly. It leans toward the sluggish once reached. Reaching the site is not the issue my my experience.

H:\>tracert hera.smugmug.com

Tracing route to hera.smugmug.com [63.81.134.23]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms blue.perlegen.com [172.16.0.1]
2 <1 ms <1 ms <1 ms drawbridge.perlegen.com [172.17.1.4]
3 <1 ms <1 ms <1 ms 64.168.232.237
4 <1 ms <1 ms <1 ms bb2-g1-0-2.sntcca.sbcglobal.net
[151.164.41.162]

5 1 ms 1 ms 1 ms ex2-p5-0.eqsjca.sbcglobal.net [151.164.41.109]
6 30 ms 2 ms 3 ms 151.164.251.246
7 1 ms 1 ms 1 ms ge-7-1.ipcolo1.SanJose1.Level3.net [4.68.123.73]

8 7 ms 3 ms 14 ms pri.r1-ge0-2-eq-sj.smugmug.com [206.223.117.70]

9 3 ms 3 ms 3 ms hera.smugmug.com [63.81.134.23]

Trace complete.

[mpmcleod]

From Work: (2335 Central)

traceroute to mpmcleod.smugmug.com (63.81.134.23), 30 hops max, 46 byte packets
1 165.95.210.180 (165.95.210.180) 0.766 ms 0.616 ms 0.596 ms
2 TTVN-WAN-Dallas.ttvn.tamus.edu (165.95.0.17) 5.887 ms 5.867 ms 5.861 ms
3 TAMU-Level3-Dallas-ISP.ttvn.tamus.edu (165.95.0.62) 5.885 ms 5.883 ms 5.938 ms
4 te-4-3.car2.Dallas1.Level3.net (4.79.182.245) 6.127 ms 7.800 ms 5.997 ms
5 ae-2-54.bbr2.Dallas1.Level3.net (4.68.122.97) 6.431 ms ae-2-56.bbr2.Dallas1.Level3.net (4.68.122.161) 6.429 ms ae-2-52.bbr2.Dallas1.Level3.net (4.68.122.33) 11.222 ms
6 ae-0-0.bbr2.Washington1.Level3.net (4.68.128.210) 38.464 ms ae-2-0.bbr1.Washington1.Level3.net (4.68.128.201) 38.573 ms ae-0-0.bbr2.Washington1.Level3.net (4.68.128.210) 38.546 ms
7 ae-31-89.car1.Washington1.Level3.net (4.68.17.131) 39.299 ms 39.390 ms ae-21-79.car1.Washington1.Level3.net (4.68.17.67) 39.287 ms
8 ed-pri.smugmug.com (206.223.137.74) 40.405 ms 52.896 ms 39.968 ms
9 10.1.1.5 (10.1.1.5) 96.036 ms 89.716 ms 89.845 ms
10 hera.smugmug.com (63.81.134.23) 85.307 ms 85.604 ms 85.142 ms

From Home (2335 Central):
Tracing route to mpmcleod.smugmug.com [63.81.134.23]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.168.254
2 * * * Request timed out.
3 9 ms 9 ms 9 ms gig9-0-4.hstntxtid-rtr2.houston.rr.com [24.28.10
2.241]
4 11 ms 9 ms 9 ms gig12-0-0.hstntxtid-rtr4.texas.rr.com [24.93.34.
98]
5 12 ms 11 ms 11 ms gig1-2-0.hstntxl3-rtr1.texas.rr.com [72.179.205.
25]
6 20 ms 19 ms 19 ms te-7-2.car1.Houston1.Level3.net [4.79.88.21]
7 20 ms 23 ms 19 ms ae-11-11.car2.Houston1.Level3.net [4.69.132.234]

8 56 ms 53 ms 53 ms ae-5-5.ebr1.Atlanta2.Level3.net [4.69.132.238]
9 47 ms 50 ms 53 ms ae-68.ebr3.Atlanta2.Level3.net [4.69.134.50]
10 67 ms 58 ms 67 ms ae-2.ebr1.Washington1.Level3.net [4.69.132.86]
11 61 ms 71 ms 72 ms ae-91-91.csw4.Washington1.Level3.net [4.69.134.1
42]
12 59 ms 60 ms 59 ms ae-41-99.car1.Washington1.Level3.net [4.68.17.19
5]
13 70 ms 71 ms 71 ms ed-pri.smugmug.com [206.223.137.74]
14 104 ms 107 ms 107 ms 10.1.1.5
15 105 ms 107 ms 105 ms hera.smugmug.com [63.81.134.23]

Trace complete.

Hmmmmm

[Wireless]

But it does bog down once I reach it, which is quickly. It leans toward the sluggish once reached. Reaching the site is not the issue my my experience.

H:\>tracert hera.smugmug.com

Tracing route to hera.smugmug.com [63.81.134.23]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms blue.perlegen.com [172.16.0.1]
2 <1 ms <1 ms <1 ms drawbridge.perlegen.com [172.17.1.4]
3 <1 ms <1 ms <1 ms 64.168.232.237
4 <1 ms <1 ms <1 ms bb2-g1-0-2.sntcca.sbcglobal.net
[151.164.41.162]

5 1 ms 1 ms 1 ms ex2-p5-0.eqsjca.sbcglobal.net [151.164.41.109]
6 30 ms 2 ms 3 ms 151.164.251.246
7 1 ms 1 ms 1 ms ge-7-1.ipcolo1.SanJose1.Level3.net [4.68.123.73]

8 7 ms 3 ms 14 ms pri.r1-ge0-2-eq-sj.smugmug.com [206.223.117.70]

9 3 ms 3 ms 3 ms hera.smugmug.com [63.81.134.23]

Trace complete.
You're 3ms from us... That's pretty good. :) There's no bogging down.

Our routers prioritize ICMP lower than other traffic, so the numbers from the physical interface (hop 8) aren't that accurate. The last hop looks great.

[Wireless]

This morning was the normal "megaslow" speed - so I figure starting a thread to help Don and other Ops people work out the slowness would be beneficial... Here's my trace route to hera.smugmug.com during the slowness...

C:\Windows>tracert hera.smugmug.com
Tracing route to hera.smugmug.com [63.81.134.23]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms cipafilter.emsd37.org [10.10.0.1]
2 1 ms 1 ms 1 ms 209.7.177.1
3 3 ms 3 ms 3 ms 77.128.175.209.rtc5.illinois.net [209.175.128.77]
4 16 ms 12 ms 14 ms 249.7.166.206.rtc5.illinois.net [206.166.7.249]
5 51 ms 24 ms 16 ms atm-5-0-0-6-moline-core-rockford-core.rockford.lincon.net [206.166.9.190]
6 12 ms 12 ms 14 ms atm-1-1-0-sub05-rockford-core-dekalb-core.dekalb.lincon.net [206.166.9.106]
7 12 ms 12 ms 12 ms 206.166.9.25
8 27 ms 12 ms 12 ms pos-5-0-nap-sob1-nap-sob2.chicago.lincon.net [206.166.9.122]
9 269 ms 259 ms 191 ms 206.166.126.14
10 468 ms 472 ms 500 ms chi1.ge4-0-2.core1.iad1.hopone.net [66.36.224.217]
11 522 ms 466 ms 467 ms ed-pri.smugmug.com [206.223.137.74]
12 * * * Request timed out.
13 98 ms 104 ms 98 ms hera.smugmug.com [63.81.134.23]
Trace complete.

Looks like some yuck around Chicago this morning (hops 9 and 10). If that still looks bad, please send me another traceroute and I'll forward it along to Illinois.net. We don't buy bandwidth from them, so we don't have much pull.

[Dna]

traceroute to hera.smugmug.com (63.81.134.23), 64 hops max, 40 byte packets
1 192.168.0.1 (192.168.0.1) 2.382 ms 1.167 ms 1.127 ms
2 loop0.lns2.mel6.internode.on.net (150.101.212.91) 16.634 ms 17.673 ms 18.115 ms
3 gi0-3.cor1.mel6.internode.on.net (150.101.212.158) 17.036 ms * 17.758 ms
4 pos2-1.bdr1.syd6.internode.on.net (150.101.212.86) 212.853 ms 215.687 ms 211.634 ms
5 pos2-1.bdr1.sjc2.internode.on.net (203.16.213.185) 211.830 ms 212.753 ms 217.768 ms
6 rt0sj-equinix.cl.shawcable.net (206.223.116.20) 213.562 ms 212.154 ms 213.232 ms
7 rc2sj-ge1-0-0.cl.shawcable.net (66.163.67.102) 214.255 ms 212.839 ms 213.599 ms
8 pri.r1-ge0-2-eq-sj.smugmug.com (206.223.117.70) 194.409 ms 187.287 ms 198.030 ms
9 hera.smugmug.com (63.81.134.23) 216.698 ms 214.559 ms 214.079 ms

Andrew

[Eriktank]

CHMOD - 3ms?!?! How in the world do you get that speed? And was the site still slow? Or were you just posting your results? If it was slow at 3ms - surely it's a datapull bottleneck somehow on the serverside... Is that a typo? 3ms??

Mine right now is running enormously sluggish again... Had a customer call and ask when my site is going to be back online so I checked it out... From my end I get this as a trace:

Tracing route to hera.smugmug.com [63.81.134.23]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms cipafilter.emsd37.org [10.10.0.1]
2 1 ms 1 ms 1 ms 209.7.177.1
3 3 ms 3 ms 3 ms 77.128.175.209.rtc5.illinois.net [209.175.128.77]
4 12 ms 12 ms 12 ms 249.7.166.206.rtc5.illinois.net [206.166.7.249]
5 12 ms 12 ms 12 ms atm-5-0-0-6-moline-core-rockford-core.rockford.lincon.net [206.166.9.190]
6 12 ms 12 ms 12 ms atm-1-1-0-sub05-rockford-core-dekalb-core.dekalb.lincon.net [206.166.9.106]
7 12 ms 12 ms 12 ms 206.166.9.25
8 13 ms 13 ms 12 ms pos-5-0-nap-sob1-nap-sob2.chicago.lincon.net [206.166.9.122]
9 152 ms 142 ms 143 ms 206.166.126.14
10 39 ms 37 ms 37 ms chi1.ge4-0-2.core1.iad1.hopone.net [66.36.224.217]
11 38 ms 47 ms 50 ms ed-pri.smugmug.com [206.223.137.74]
12 * * * Request timed out.
13 98 ms 98 ms 97 ms hera.smugmug.com [63.81.134.23]
Trace complete.

[Jason Dunn]

Well, for what it's worth, I'm seeing KICKASS speed today, so I don't know if it's a Smugmug server issue - or maybe my server is just fast while others are slow...

[Eriktank]

it's just you. :)

Mine's slower than a dead dog right now. (That means the pages won't fully load, etc).

[Wireless]

chmod's title indicates that he's not seeing slowness. He shouldn't, as we also sit 2ms away from our datacenter and things are blazing fast.

Erik - unfortunately I can't get in contact of anyone at illinois.net today. They are indeed having problems. I'll see if shawcable can help, they might be able to escalate on their end.