I've known comcast to have many problems. I used to do radio broadcasting via shoutcast, and also help users of comcast and other ISPs get their station setup. I would rate comcast a 3/10 as an ISP for their lack of quality and peak time limits.
I can tell you it's not your server.
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\Users\Rob>tracert 63.251.20.181
Tracing route to v-63-251-20-181.virt-machine.internap-nyc.nfoservers.com [63.25
1.20.181]
over a maximum of 30 hops:
1 1 ms <1 ms <1 ms 192.168.1.1
2 8 ms 7 ms 11 ms 10.240.180.25
3 10 ms 8 ms 7 ms dstswr1-vlan4.rh.nyk4ny.cv.net [67.83.221.193]
4 9 ms 9 ms 14 ms dstswr1-ge3-12.rh.nyk4ny.cv.net [67.83.221.137]
5 12 ms 14 ms 12 ms 64.15.2.29
6 * * * Request timed out.
7 12 ms 10 ms 13 ms cr2-nyiix-peer.nym009.internap.net [198.32.160.1
85]
8 11 ms 11 ms 15 ms nym008-nym009-82-core3.nym008.internap.net [66.7
9.152.130]
9 11 ms 11 ms 10 ms border2.pc1-bbnet1.ext1.nym.pnap.net [216.52.95.
15]
10 11 ms 11 ms 12 ms v-63-251-20-181.virt-machine.internap-nyc.nfoser
vers.com [63.251.20.181]
Trace complete.
C:\Users\Rob>
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\Users\Rob>tracert 63.251.20.181
Tracing route to v-63-251-20-181.virt-machine.internap-nyc.nfoservers.com [63.25
1.20.181]
over a maximum of 30 hops:
1 1 ms <1 ms <1 ms 192.168.1.1
2 8 ms 7 ms 11 ms 10.240.180.25
3 10 ms 8 ms 7 ms dstswr1-vlan4.rh.nyk4ny.cv.net [67.83.221.193]
4 9 ms 9 ms 14 ms dstswr1-ge3-12.rh.nyk4ny.cv.net [67.83.221.137]
5 12 ms 14 ms 12 ms 64.15.2.29
6 * * * Request timed out.
7 12 ms 10 ms 13 ms cr2-nyiix-peer.nym009.internap.net [198.32.160.1
85]
8 11 ms 11 ms 15 ms nym008-nym009-82-core3.nym008.internap.net [66.7
9.152.130]
9 11 ms 11 ms 10 ms border2.pc1-bbnet1.ext1.nym.pnap.net [216.52.95.
15]
10 11 ms 11 ms 12 ms v-63-251-20-181.virt-machine.internap-nyc.nfoser
vers.com [63.251.20.181]
Trace complete.
C:\Users\Rob>
Run a speedtest at
http://www.speedtest.net - click the big blue bar on top of the map that says "Begin Test", it will ping to the closest server near you.

I'm running Optimum Online Boost, and I would recommend anyone in the NY area to do the same. the total for your internet p/month would be $39.90 a month, and I can tell you it's well worth it, speeds get higher than what you see on my speedtest.
The server on a good day when it's equivalent I've seen run at 200mbps ^ and 200mbps v. I've seen it reach 600mbps ^ and 100mbps v when I first ordered it, but for all you know it could be better than both, as a speedtest is never ever 100% accurate.
You may also be receiving packetloss via comcast > line > modem. If you're using a 3-way coaxial converter, get one directly from comcast, not one from a hardware or appliance store. The same thing happened to me with optimum after viewing thousands of T3 timeouts in my modem logs. If you're using a router you can also try going directly to the modem, just for a test, anyway. Or maybe permanently if that helps.