Ubuntu Routing Issue (cant bind hlds properly)

Ask questions about dedicated servers here and we and other users will do our best to answer them. Please also refer to the self-help section for tutorials and answers to the most commonly asked questions.
Post Reply
evil_master
A regular
A regular
Posts: 48
Joined: Fri Jun 17, 2011 3:57 pm

Ubuntu Routing Issue (cant bind hlds properly)

Post by evil_master » Fri Mar 09, 2018 7:28 am

Hello everyone, I run a small community for Team Fortress Classic, one of the FPS 'mod turned full game' releases from valve on the goldsrc engine. I have been spoiled all my life with being able to have access to the default port for my servers and have continued this for over a decade now, however I have ran into a small issue regarding steams master server list.

Using the API I have determined that my servers are indeed being sent to the list, but the address they are given is using the 'address' value associated with my eth0 interface. I have the additional ip addresses connected to the interface and can even run the services ON THOSE ADDRESSES. however I can't for the life of me get them to _REPORT_ that address to the steam master server list.

Does anyone have any experience with something like this?

my primary IP is the only one visible on the API (it has all the servers attached, but they shouldn't be reporting this way, they are mostly on different addresses)

the VDS in question is at 66.151.244.170 (this is default eth0 address and the only one reported to steam master server)
the ip's I cant display are of: 66.151.244.232, 74.201.57.37, 74.201.57.11
I properly configured the server with -strictportbind -port $port +ip $ip on each server, but I am unable to get the servers to send that address to steam, status even displays the correct address.

Some things ive looked into: 2 default gateways? I am using two different gateways for the ip's assigned to my machine, I did what I could to set this up but was not ultimately successful in making any noticeable changes, I followed this guide to try to set this up.


netstat output
root@esclan:~# netstat -alun
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
udp 0 0 66.151.244.232:27015 0.0.0.0:*
udp 0 0 66.151.244.170:27015 0.0.0.0:*
udp 0 0 74.201.57.37:27016 0.0.0.0:*
udp 0 0 66.151.244.232:27017 0.0.0.0:*
udp 0 0 74.201.57.37:27018 0.0.0.0:*
API Output
http://api.steampowered.com/ISteamApps/ ... ormat=json
https://api.steampowered.com/IGameServe ... &limit=100
{
"response": {
"success": true,
"servers": [
{
"addr": "66.151.244.170:27015",
"gmsindex": 65534,
"appid": 20,
"gamedir": "tfc",
"region": -1,
"secure": true,
"lan": false,
"gameport": 27015,
"specport": 0
},
{
"addr": "66.151.244.170:27016",
"gmsindex": 65534,
"appid": 20,
"gamedir": "tfc",
"region": -1,
"secure": true,
"lan": false,
"gameport": 27016,
"specport": 0
},
{
"addr": "66.151.244.170:27017",
"gmsindex": 65534,
"appid": 20,
"gamedir": "tfc",
"region": -1,
"secure": true,
"lan": false,
"gameport": 27017,
"specport": 0
},
{
"addr": "66.151.244.170:27018",
"gmsindex": 65534,
"appid": 20,
"gamedir": "tfc",
"region": -1,
"secure": true,
"lan": false,
"gameport": 27018,
"specport": 0
}
]

}
}
Launch Params
Image

Again.. the masterserver is getting this ip paramater and I dont know what it comes from, I have the servers configured in the same manner they have been for years, this is a 'new' problem to me after migrating from windows to linux and I have just about ran out of google-fu, can anyone provide any ideas?

User avatar
Spray
Staff
Staff
Posts: 588
Joined: Wed Dec 28, 2011 10:41 pm
Location: Oregon

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by Spray » Fri Mar 09, 2018 10:13 am

If I remember correctly, this stems from a very old issue that Valve never fixed with port binding over TCP on Linux. Game servers would attempt to connect to the Steam master server via TCP, which didn't respect port binding. Steam would see the traffic from the base interface IP address, and list the server on that IP. Blocking TCP outbound caused the server to fall-back to UDP, which correctly respects port binding.

Try blocking TCP output to port 27017, 27018, and 27019 to see if this corrects the issues. The following IPTables rule would take care of this:

Code: Select all

iptables -A OUTPUT -p tcp -m multiport --dports 27017:27019 -j REJECT --reject-with icmp-port-unreachable

evil_master
A regular
A regular
Posts: 48
Joined: Fri Jun 17, 2011 3:57 pm

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by evil_master » Fri Mar 09, 2018 12:43 pm

Spray wrote:
Fri Mar 09, 2018 10:13 am
If I remember correctly, this stems from a very old issue that Valve never fixed with port binding over TCP on Linux. Game servers would attempt to connect to the Steam master server via TCP, which didn't respect port binding. Steam would see the traffic from the base interface IP address, and list the server on that IP. Blocking TCP outbound caused the server to fall-back to UDP, which correctly respects port binding.

Try blocking TCP output to port 27017, 27018, and 27019 to see if this corrects the issues. The following IPTables rule would take care of this:

Code: Select all

iptables -A OUTPUT -p tcp -m multiport --dports 27017:27019 -j REJECT --reject-with icmp-port-unreachable
I am unsure if I did something wrong, but that did not do the trick for me, I reverted the changes I made to /etc/network/interfaces to test the iptables filter, was disappointed to see it not work.. do I need to run a command to 'enable' the filter? it shows up in iptables -L
root@esclan:~# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
REJECT tcp -- anywhere anywhere multiport dports 27017:27019 reject-with icmp-port-unreachable
Understanding the underlying issue now helps me to understand the 'solution' I have found that works.
After researching info on how to set up 'source based routing' to prevent this exact thing, I stumbled upon some info from https://serverfault.com/questions/41196 ... fics-after detailing how to update /etc/network/interfaces to use iproute2 over ifconfig for 'proper' multi-ip support.

However, I ran into an issue, because my IP's are not all within the same subnet, I have multiple gateways, this entire networking thing is all new to me, this is what I have in my interfaces file so far:
auto lo

iface lo inet loopback

auto eth0
iface eth0 inet static
address 66.151.244.170
netmask 255.255.255.0
gateway 66.151.244.254
dns-nameservers 8.8.8.8 8.8.4.4

post-up ip addr add 66.151.244.232 dev eth0
post-up ip route add 66.151.244.232/24 via 66.151.244.254 src 66.151.244.232

post-up ip addr add 66.151.244.59 dev eth0
post-up ip route add 66.151.244.59/24 via 66.151.244.254 src 66.151.244.59

post-up ip addr add 74.201.57.110 dev eth0
post-up ip route add 74.201.57.110/24 via 74.201.57.254 src 74.201.57.110

pust-up ip addr add 74.201.57.181 dev eth0
post-up ip route add 74.201.57.110/24 via 74.201.57.254 src 74.201.57.181

post-up ip addr add 74.201.57.37 dev eth0
post-up ip route add 74.201.57.37/24 via 74.201.57.254 src 74.201.57.37
With this, my ip addresses on 66.151.244.* are properly being forwarded to the master server, but it seems my system 'lost' the ability to use the other addresses, how would I go about fixing this, or how would I even look at the logs to see why those addresses arent 'available'?

evil_master
A regular
A regular
Posts: 48
Joined: Fri Jun 17, 2011 3:57 pm

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by evil_master » Fri Mar 09, 2018 2:19 pm

cant seem to edit my post, so what I learned is.. this probably cant be done without at least using routing tables, even then I am unsure as all the info I find about this mention doing this from two different interfaces, I am unsure how this works with just one single ethernet interface.. my knowledge with linux is very limited.

This is what I got for my services, it halts on trying to set up the route to the 74.201.57.254 gateway (of course I didn't set this in the cfg as theres no way to add a 2nd default gateway or set the gateway per ip address)
auto lo

iface lo inet loopback

auto eth0
iface eth0 inet static
address 66.151.244.170
netmask 255.255.255.0
gateway 66.151.244.254
dns-nameservers 8.8.8.8 8.8.4.4

post-up ip addr add 66.151.244.232 dev eth0
post-up ip route add 66.151.244.232/32 via 66.151.244.254 src 66.151.244.232

post-up ip addr add 66.151.244.59 dev eth0
post-up ip route add 66.151.244.59/32 via 66.151.244.254 src 66.151.244.59


auto eth0:1
iface eth0:1 inet static
address 74.201.57.110
netmask 255.255.255.0
gateway 74.201.57.254
dns-nameservers 8.8.8.8 8.8.4.4

post-up ip addr add 74.201.57.110 dev eth0
post-up ip route add 74.201.57.110/32 via 74.201.57.254 src 74.201.57.110

pust-up ip addr add 74.201.57.181 dev eth0
post-up ip route add 74.201.57.110/32 via 74.201.57.254 src 74.201.57.181

post-up ip addr add 74.201.57.37 dev eth0
post-up ip route add 74.201.57.37/32 via 74.201.57.254 src 74.201.57.37
Also, if theres another route I can go to achieve this, I simply just want to get this to have the proper port/ip get sent to the master server, would this be possible to do with some very specific ip/port matches with iptables?

User avatar
Edge100x
Founder
Founder
Posts: 12387
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by Edge100x » Fri Mar 09, 2018 4:45 pm

The exact rule that we use internally is:

Code: Select all

iptables -A OUTPUT -p tcp -m tcp --dport 27017:27019 --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
We use multiple IP addresses per machine and I have not seen the problem you are describing with this rule in use. We do not have to use NAT (which may be what you are thinking of when you say "routing tables").

You should not use multiple default gateways. You should only have one gateway specified. It will only break things for you to try to use more than one.

evil_master
A regular
A regular
Posts: 48
Joined: Fri Jun 17, 2011 3:57 pm

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by evil_master » Sat Mar 10, 2018 1:10 am

Okay, I wasn't sure if there was a configured error or if I'm not executing the command right, so I added the rule via the control panel firewall interface (iptables rule is still active) and I am still experiencing the same issue, just to check do you run your hlds instances with +ip or -ip on the command line? I've been using +ip since migrating from Windows, just as a test I tried running a server with -ip and got a port in use error, when I have access, I intend on shutting them all down to see if -ip even functions or if it is ignored (pretty sure that's why I originally didn't use it on Windows) since blocking tcp packets on those ports failed, what sort of options would I have with static routes? My end goal is to simply end up with the master server getting the proper ip/port as specified on the command line, everything else regarding the server itself is seen as extra

evil_master
A regular
A regular
Posts: 48
Joined: Fri Jun 17, 2011 3:57 pm

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by evil_master » Sat Mar 10, 2018 6:03 am

I do apologize for all the doubleposting, there are no edits allowed to posts after the initial 1 minute grace period.. I have determined that this *IS* a routing issue, the -ip parameter indeed does not work, meaning +ip is the correct/only choice available, however this proved to do absolutely nothing for me, so I continued to look into why it might only report the default address, which lead me to adding static routes so that all packets being sent out from my addresses are forced to go out their properly assigned gateway (at least this is what I understand the route to be doing)

However I am having an issue getting them added at runtime, I attempted to add some lines to /etc/network/interfaces following the information from here: https://serverfault.com/questions/41196 ... ifics-afte (more specifically the comments on the answer to the question) I am able to manually run the route commands after the system has started, but I cannot get the routes added as part of the interfaces file regardless if they are on the specific address I am routing.
auto lo

iface lo inet loopback

auto eth0
iface eth0 inet static
address 66.151.244.170
netmask 255.255.255.0
gateway 66.151.244.254
dns-nameservers 8.8.8.8 8.8.4.4
# post-up ip route add 66.151.244.170/32 via 66.151.244.254 src 66.151.244.170
# post-up ip route add 66.151.244.232/32 via 66.151.244.254 src 66.151.244.232
# post-up ip route add 66.151.244.59/32 via 66.151.244.254 src 66.151.244.59


auto eth0:1
iface eth0:1 inet static
address 66.151.244.232
netmask 255.255.255.0
gateway 66.151.244.254
dns-nameservers 8.8.8.8 8.8.4.4

auto eth0:2
iface eth0:2 inet static
address 66.151.244.59
netmask 255.255.255.0
gateway 66.151.244.254
dns-nameservers 8.8.8.8 8.8.4.4

auto eth0:3
iface eth0:3 inet static
address 74.201.57.110
netmask 255.255.255.0
gateway 66.151.244.254
dns-nameservers 8.8.8.8 8.8.4.4
# post-up ip route add 74.201.57.110/32 via 74.201.57.254 src 74.201.57.110
# post-up ip route add 74.201.57.181/32 via 74.201.57.254 src 74.201.57.181
# post-up ip route add 74.201.57.37/32 via 74.201.57.254 src 74.201.57.37

auto eth0:4
iface eth0:4 inet static
address 74.201.57.181
netmask 255.255.255.0
gateway 74.201.57.254
dns-nameservers 8.8.8.8 8.8.4.4

auto eth0:5
iface eth0:5 inet static
address 74.201.57.37
netmask 255.255.255.0
gateway 74.201.57.254
dns-nameservers 8.8.8.8 8.8.4.4
I changed up to post-up when I had trouble with them being added, but now I know thats not the issue.
root@esclan:~# ip ro
default via 66.151.244.254 dev eth0 onlink
66.151.244.0/24 dev eth0 proto kernel scope link src 66.151.244.170
66.151.244.170 via 66.151.244.254 dev eth0 src 66.151.244.170
74.201.57.0/24 dev eth0 proto kernel scope link src 74.201.57.110
root@esclan:~# ip route add 66.151.244.170/32 via 66.151.244.254 src 66.151.244.170
RTNETLINK answers: File exists
root@esclan:~# ip route add 66.151.244.232/32 via 66.151.244.254 src 66.151.244.232
root@esclan:~# ip route add 66.151.244.59/32 via 66.151.244.254 src 66.151.244.59
root@esclan:~# ip ro
default via 66.151.244.254 dev eth0 onlink
66.151.244.0/24 dev eth0 proto kernel scope link src 66.151.244.170
66.151.244.59 via 66.151.244.254 dev eth0 src 66.151.244.59
66.151.244.170 via 66.151.244.254 dev eth0 src 66.151.244.170
66.151.244.232 via 66.151.244.254 dev eth0 src 66.151.244.232
74.201.57.0/24 dev eth0 proto kernel scope link src 74.201.57.110
After adding these routes my servers can be seen on the internet server list [!!!!] so now I need to figure out how to 'properly' add these routes either to the interfaces file or to the system at some point after boot (will sudo crontab be able to handle this?)

User avatar
Edge100x
Founder
Founder
Posts: 12387
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by Edge100x » Sat Mar 10, 2018 9:30 am

What IP address a service chooses to bind to or report to an external service is NOT a routing issue. But it can certainly be an interfaces file issue -- if you aren't actually assigning IP addresses to your adapter correctly, you won't be able to use them.

The interfaces file that you posted originally was definitely not valid, and the "post-up" commands that you have been experimenting with are nonsense. You should be adding IP addresses according to the KB article that we have at viewtopic.php?f=68&t=5418 (which we link in the events log post and email after an IP is added to a service). You should check to make sure an IP address works properly from the outside before using it by pinging it.

"+ip" is the correct usage for a TFC server.

Any time you modify the network interfaces, you may need to re-apply the iptables command that we gave.

evil_master
A regular
A regular
Posts: 48
Joined: Fri Jun 17, 2011 3:57 pm

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by evil_master » Sat Mar 10, 2018 11:27 am

The default interfaces file was accurate with the KB article and had all of the info as seen on the IP List tab of the control panel, while doing some research into this I came accross this post with some details about virtual interfaces.. regardless.. none of it accomplished what I set out to do,

I added the iptables command to rc.local and it is running on boot, however even after expanding the port range I was not able to get any success, the master server list is still reporting all the servers as coming from the main interface address.

iptables -A OUTPUT -p tcp -m tcp --dport 21000:29999 -j DROP
root@esclan:~# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
DROP tcp -- anywhere anywhere tcp dpts:21000:29999


ifconfig interfaces

Code: Select all

auto lo

iface lo inet loopback

auto eth0
iface eth0 inet static
        address 66.151.244.170
        netmask 255.255.255.0
        gateway 66.151.244.254
        dns-nameservers 8.8.8.8 8.8.4.4

auto eth0:1
iface eth0:1 inet static
        address 66.151.244.232
        netmask 255.255.255.0
        gateway 66.151.244.254
        dns-nameservers 8.8.8.8 8.8.4.4

auto eth0:2
iface eth0:2 inet static
        address 66.151.244.59
        netmask 255.255.255.0
        gateway 66.151.244.254
        dns-nameservers 8.8.8.8 8.8.4.4

auto eth0:3
iface eth0:3 inet static
        address 74.201.57.110
        netmask 255.255.255.0
        gateway 66.151.244.254
        dns-nameservers 8.8.8.8 8.8.4.4

auto eth0:4
iface eth0:4 inet static
        address 74.201.57.181
        netmask 255.255.255.0
        gateway 74.201.57.254
        dns-nameservers 8.8.8.8 8.8.4.4

auto eth0:5
iface eth0:5 inet static
        address 74.201.57.37
        netmask 255.255.255.0
        gateway 74.201.57.254
        dns-nameservers 8.8.8.8 8.8.4.4
root@esclan:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 66.151.244.254 0.0.0.0 UG 0 0 0 eth0
66.151.244.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
74.201.57.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

Any time I modify the network I am doing a reboot to ensure that I don't run into any configuration errors that aren't immediately noticable, if I use the iproute2 method of adding addresses to eth0 I can only add addresses reachable by the gateway, I get an error saying that the network is unreachable (and either you cant or I dont know how to define the gateway)

Is there perhaps a way to easily filter network traffic by program? even if I have to add many filters I could choose to filter the outgoing TCP packets to the master server(s)

User avatar
Edge100x
Founder
Founder
Posts: 12387
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by Edge100x » Sat Mar 10, 2018 2:39 pm

It's important that you read and follow the directions to the letter to make sure that problems don't occur.

You linked that post before, yes. It does not apply here and will just confuse you. You should put it out of your mind entirely.

Our KB article was not followed properly. It recommends starting with :0, not :1, and says to only add DNS servers to the first entry, and only if they were missing.

You did not run the correct iptables line. With whatever you ran, you put a much larger port range than you should have. That would not make your IP address designations not work, but it would break other things.

It will not help for you to filter network traffic by program. Using the global rule that we gave is all you need.

What is the result of these commands?

ifconfig
ip address

evil_master
A regular
A regular
Posts: 48
Joined: Fri Jun 17, 2011 3:57 pm

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by evil_master » Sat Mar 10, 2018 8:56 pm

Code: Select all

root@esclan:/home/esclan# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3e:d5:d6:e6
          inet addr:66.151.244.170  Bcast:66.151.244.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fed5:d6e6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:217661 errors:0 dropped:0 overruns:0 frame:0
          TX packets:214921 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:16879725 (16.8 MB)  TX bytes:59228548 (59.2 MB)

eth0:1    Link encap:Ethernet  HWaddr 00:16:3e:d5:d6:e6
          inet addr:66.151.244.232  Bcast:66.151.244.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0:2    Link encap:Ethernet  HWaddr 00:16:3e:d5:d6:e6
          inet addr:66.151.244.59  Bcast:66.151.244.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0:3    Link encap:Ethernet  HWaddr 00:16:3e:d5:d6:e6
          inet addr:74.201.57.110  Bcast:74.201.57.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0:4    Link encap:Ethernet  HWaddr 00:16:3e:d5:d6:e6
          inet addr:74.201.57.181  Bcast:74.201.57.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0:5    Link encap:Ethernet  HWaddr 00:16:3e:d5:d6:e6
          inet addr:74.201.57.37  Bcast:74.201.57.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:594 errors:0 dropped:0 overruns:0 frame:0
          TX packets:594 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:38525 (38.5 KB)  TX bytes:38525 (38.5 KB)

root@esclan:/home/esclan# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:16:3e:d5:d6:e6 brd ff:ff:ff:ff:ff:ff
    inet 66.151.244.170/24 brd 66.151.244.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet 74.201.57.110/24 brd 74.201.57.255 scope global eth0:3
       valid_lft forever preferred_lft forever
    inet 66.151.244.232/24 brd 66.151.244.255 scope global secondary eth0:1
       valid_lft forever preferred_lft forever
    inet 66.151.244.59/24 brd 66.151.244.255 scope global secondary eth0:2
       valid_lft forever preferred_lft forever
    inet 74.201.57.181/24 brd 74.201.57.255 scope global secondary eth0:4
       valid_lft forever preferred_lft forever
    inet 74.201.57.37/24 brd 74.201.57.255 scope global secondary eth0:5
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fed5:d6e6/64 scope link
       valid_lft forever preferred_lft forever
root@esclan:/home/esclan#
I've altered the interfaces as you outline, however the issue still persists.

Code: Select all

Welcome to Ubuntu 16.04.4 LTS (GNU/Linux 4.4.0-116-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

0 packages can be updated.
0 updates are security updates.


Last login: Sat Mar 10 22:37:52 2018 from 107.77.160.49
esclan@esclan:~$ sudo su
root@esclan:/home/esclan# cat /etc/rc.local
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

iptables -A OUTPUT -p tcp -m tcp --dport 27017:27019 --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
exit 0
root@esclan:/home/esclan# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
DROP       tcp  --  anywhere             anywhere             tcp dpts:27017:27019 flags:FIN,SYN,RST,ACK/SYN
root@esclan:/home/esclan# cat /etc/network/interfaces
auto lo

iface lo inet loopback

auto eth0
iface eth0 inet static
        address 66.151.244.170
        netmask 255.255.255.0
        gateway 66.151.244.254
        dns-nameservers 8.8.8.8 8.8.4.4

auto eth0:0
iface eth0:0 inet static
        address 66.151.244.232
        netmask 255.255.255.0
        gateway 66.151.244.254

auto eth0:1
iface eth0:1 inet static
        address 66.151.244.59
        netmask 255.255.255.0
        gateway 66.151.244.254

auto eth0:2
iface eth0:2 inet static
        address 74.201.57.110
        netmask 255.255.255.0
        gateway 74.201.57.254

auto eth0:3
iface eth0:3 inet static
        address 74.201.57.181
        netmask 255.255.255.0
        gateway 74.201.57.254

auto eth0:4
iface eth0:4 inet static
        address 74.201.57.37
        netmask 255.255.255.0
        gateway 74.201.57.254

root@esclan:/home/esclan# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3e:d5:d6:e6
          inet addr:66.151.244.170  Bcast:66.151.244.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fed5:d6e6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3155 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3186 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:529485 (529.4 KB)  TX bytes:416097 (416.0 KB)

eth0:0    Link encap:Ethernet  HWaddr 00:16:3e:d5:d6:e6
          inet addr:66.151.244.232  Bcast:66.151.244.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0:1    Link encap:Ethernet  HWaddr 00:16:3e:d5:d6:e6
          inet addr:66.151.244.59  Bcast:66.151.244.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0:2    Link encap:Ethernet  HWaddr 00:16:3e:d5:d6:e6
          inet addr:74.201.57.110  Bcast:74.201.57.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0:3    Link encap:Ethernet  HWaddr 00:16:3e:d5:d6:e6
          inet addr:74.201.57.181  Bcast:74.201.57.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0:4    Link encap:Ethernet  HWaddr 00:16:3e:d5:d6:e6
          inet addr:74.201.57.37  Bcast:74.201.57.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:160 errors:0 dropped:0 overruns:0 frame:0
          TX packets:160 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:11840 (11.8 KB)  TX bytes:11840 (11.8 KB)

root@esclan:/home/esclan# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:16:3e:d5:d6:e6 brd ff:ff:ff:ff:ff:ff
    inet 66.151.244.170/24 brd 66.151.244.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet 74.201.57.110/24 brd 74.201.57.255 scope global eth0:2
       valid_lft forever preferred_lft forever
    inet 66.151.244.232/24 brd 66.151.244.255 scope global secondary eth0:0
       valid_lft forever preferred_lft forever
    inet 66.151.244.59/24 brd 66.151.244.255 scope global secondary eth0:1
       valid_lft forever preferred_lft forever
    inet 74.201.57.181/24 brd 74.201.57.255 scope global secondary eth0:3
       valid_lft forever preferred_lft forever
    inet 74.201.57.37/24 brd 74.201.57.255 scope global secondary eth0:4
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fed5:d6e6/64 scope link
       valid_lft forever preferred_lft forever
root@esclan:/home/esclan#
The reason I asked about capturing the individual program output is to attempt to create filleting rules to block TCP out to any of the steam master servers as this port filtering rule does not work for me

evil_master
A regular
A regular
Posts: 48
Joined: Fri Jun 17, 2011 3:57 pm

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by evil_master » Sun Mar 11, 2018 4:46 am

After reading the KB article on this specific issue I have found where the connection logs are stored and decided to take a peek to further understand why this iptables rule isn't working and what else I can try

Code: Select all

[2018-03-11 06:37:42] Log session started
[2018-03-11 06:37:42] [0,0] SetSteamID( [A:1:0:0] )
[2018-03-11 06:37:42] [0,0] Server says 50% of connections should be websockets, we rolled 44 - using WebSockets as default.
[2018-03-11 06:37:42] [0,0] Existing WebSocket ping data (CM02-LON.cm.steampowered.com:27020) results: 147.0000ms + load 87.0000 (780 seconds ago)
[2018-03-11 06:37:42] [0,0] Existing WebSocket ping data (s3tuk-035b.cm.steampowered.com:443) results: 53.0000ms + load 10.0000 (781 seconds ago)
[2018-03-11 06:37:42] [0,0] Existing WebSocket ping data (cm01-lax.cm.steampowered.com:443) results: 53.0000ms + load 15.0000 (781 seconds ago)
[2018-03-11 06:37:42] [0,0] Existing WebSocket ping data (cm04-lax.cm.steampowered.com:443) results: 49.0000ms + load 13.0000 (781 seconds ago)
[2018-03-11 06:37:42] [0,0] PingWebSocketCM() stopping early for good host (cm04-lax.cm.steampowered.com:443)
[2018-03-11 06:37:42] [1,3] Connect() starting connection (eNetQOSLevelMedium, cm04-lax.cm.steampowered.com:443, WebSocket)
[2018-03-11 06:37:42] [1,3] ConnectionCompleted() (162.254.195.47:443, WebSocket)
[2018-03-11 06:37:42] [1,3] RecvMsgClientLogOnResponse() : [A:1:3139683338:9722] 'OK'
I am unfamiliar with websockets or how they work, however I believe I would want to block these as well, also of note is the first content server is running on port 27020, also outside the scope of this filtering rule

Here is what I did, obviously this method is bad as I cant server anything via https:// so if there is a better method to accomplish this, please let me know.

iptables -A OUTPUT -p tcp -m tcp --dport 27017:27021 --tcp-flags FIN,SYN,RST,ACK SYN -j DROP
iptables -A OUTPUT -p tcp -m tcp --dport 443 --tcp-flags FIN,SYN,RST,ACK SYN -j DROP

Code: Select all

[2018-03-11 07:21:57] [0,0] Existing WebSocket ping data (CM02-FRA.cm.steampowered.com:27020) results: 10000.0000ms + load 0.0000 (132 seconds ago)
[2018-03-11 07:21:57] [0,0] Existing WebSocket ping data (CM02-STO.cm.steampowered.com:27020) results: 10000.0000ms + load 0.0000 (133 seconds ago)
[2018-03-11 07:21:57] [0,0] Existing WebSocket ping data (CM03-STO.cm.steampowered.com:27020) results: 10000.0000ms + load 0.0000 (133 seconds ago)
[2018-03-11 07:21:57] [0,0] PingWebSocketCM() (CM03-LON.cm.steampowered.com:27021) starting...
[2018-03-11 07:21:57] [0,0] PingWebSocketCM() (CM02-VIE.cm.steampowered.com:27020) starting...
[2018-03-11 07:21:57] [0,0] PingWebSocketCM() (CM01-FRA.cm.steampowered.com:27020) starting...
[2018-03-11 07:21:58] [0,0] PingWebSocketCM() (CM03-LON.cm.steampowered.com:27021) failed talking to cm (timeout/neterror - Invalid)
[2018-03-11 07:21:58] [0,0] PingWebSocketCM() (CM01-FRA.cm.steampowered.com:27020) failed talking to cm (timeout/neterror - Invalid)
[2018-03-11 07:21:58] [0,0] PingWebSocketCM() (CM02-VIE.cm.steampowered.com:27020) failed talking to cm (timeout/neterror - Invalid)
[2018-03-11 07:21:58] [1,11] Connect() starting connection (eNetQOSLevelMedium, CM01-IAD.cm.steampowered.com:443, WebSocket)
[2018-03-11 07:22:58] [1,0] ConnectFailed('Connection Failed':0) (0.0.0.0:0, WebSocket)
[2018-03-11 07:22:58] [0,0] StartAutoReconnect() will start in 3 seconds
[2018-03-11 07:23:01] [0,0] Server says 0% of connections should be websockets, we rolled 50 - using UDP as default.
[2018-03-11 07:23:01] [0,0] Existing WebSocket ping data (CM02-LUX.cm.steampowered.com:443) results: 10000.0000ms + load 0.0000 (195 seconds ago)
[2018-03-11 07:23:01] [0,0] Existing WebSocket ping data (CM01-LUX.cm.steampowered.com:27021) results: 10000.0000ms + load 0.0000 (195 seconds ago)
[2018-03-11 07:23:01] [0,0] Existing WebSocket ping data (CM02-LUX.cm.steampowered.com:27020) results: 10000.0000ms + load 0.0000 (197 seconds ago)
[2018-03-11 07:23:01] [0,0] Existing WebSocket ping data (CM03-STO.cm.steampowered.com:27020) results: 10000.0000ms + load 0.0000 (197 seconds ago)
[2018-03-11 07:23:01] [0,0] Existing WebSocket ping data (CM02-FRA.cm.steampowered.com:443) results: 10000.0000ms + load 0.0000 (128 seconds ago)
[2018-03-11 07:23:01] [0,0] Existing WebSocket ping data (CM01-LUX.cm.steampowered.com:27020) results: 10000.0000ms + load 0.0000 (128 seconds ago)
[2018-03-11 07:23:01] [0,0] Existing WebSocket ping data (CM04-LON.cm.steampowered.com:443) results: 10000.0000ms + load 0.0000 (128 seconds ago)
[2018-03-11 07:23:01] [0,0] Existing WebSocket ping data (CM01-FRA.cm.steampowered.com:27020) results: 10000.0000ms + load 0.0000 (63 seconds ago)
[2018-03-11 07:23:01] [0,0] Existing WebSocket ping data (CM02-VIE.cm.steampowered.com:27020) results: 10000.0000ms + load 0.0000 (63 seconds ago)
[2018-03-11 07:23:01] [0,0] Existing WebSocket ping data (CM03-LON.cm.steampowered.com:27021) results: 10000.0000ms + load 0.0000 (63 seconds ago)
[2018-03-11 07:23:01] [0,0] Existing WebSocket ping data (CM01-IAD.cm.steampowered.com:27020) results: 10000.0000ms + load 0.0000 (195 seconds ago)
[2018-03-11 07:23:01] [0,0] PingWebSocketCM() (cm04-ord.cm.steampowered.com:443) starting...
[2018-03-11 07:23:01] [0,0] PingWebSocketCM() (cm03-ord.cm.steampowered.com:27020) starting...
[2018-03-11 07:23:02] [0,0] PingWebSocketCM() (cm03-ord.cm.steampowered.com:27020) failed talking to cm (timeout/neterror - Invalid)
[2018-03-11 07:23:02] [0,0] PingWebSocketCM() (cm04-ord.cm.steampowered.com:443) failed talking to cm (timeout/neterror - Invalid)
[2018-03-11 07:23:02] [1,15] Connect() starting connection (eNetQOSLevelMedium, CM02-LUX.cm.steampowered.com:443, WebSocket)
[2018-03-11 07:24:02] [1,0] ConnectFailed('Connection Failed':0) (0.0.0.0:0, WebSocket)
[2018-03-11 07:24:02] [0,0] StartAutoReconnect() will start in 5 seconds
[2018-03-11 07:24:07] [0,0] Server says 0% of connections should be websockets, we rolled 3 - using UDP as default.
[2018-03-11 07:24:07] [1,101] Connect() starting connection (eNetQOSLevelMedium, 208.64.201.169:27017, TCP)
[2018-03-11 07:26:14] [1,0] ConnectFailed('Connection Failed':6e) (208.64.201.169:27017, TCP)
[2018-03-11 07:26:14] [0,0] StartAutoReconnect() will start in 8 seconds
[2018-03-11 07:26:22] [0,0] Server says 0% of connections should be websockets, we rolled 63 - using UDP as default.
[2018-03-11 07:26:22] [1,105] Connect() starting connection (eNetQOSLevelMedium, 208.64.201.169:27017, TCP)
[2018-03-11 07:28:30] [1,0] ConnectFailed('Connection Failed':6e) (208.64.201.169:27017, TCP)
[2018-03-11 07:28:30] [0,0] StartAutoReconnect() will start in 7 seconds
[2018-03-11 07:28:37] [0,0] Server says 0% of connections should be websockets, we rolled 73 - using UDP as default.
[2018-03-11 07:28:37] [1,2] Connect() starting connection (eNetQOSLevelMedium, 208.64.201.169:27017, UDP)
[2018-03-11 07:28:37] [1,2] ConnectionCompleted() (208.64.201.169:27017, UDP)
[2018-03-11 07:28:37] [1,2] RecvMsgClientLogOnResponse() : [A:1:3866668041:9722] 'OK'

So it will finally choose to connect via udp if tcp is blocked on ports 443,27020,27021 (and possibly others) first of all, how do I clear this cache? it looks like it can take up to 5 minutes before it attempts to use a UDP connection, is there anything I can do about this?

User avatar
Edge100x
Founder
Founder
Posts: 12387
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by Edge100x » Mon Mar 12, 2018 1:40 am

You might also try this on the command line, which works for some games:

-udpforce

evil_master
A regular
A regular
Posts: 48
Joined: Fri Jun 17, 2011 3:57 pm

Re: Ubuntu Routing Issue (cant bind hlds properly)

Post by evil_master » Mon Mar 12, 2018 3:32 am

Edge100x wrote:
Mon Mar 12, 2018 1:40 am
You might also try this on the command line, which works for some games:

-udpforce
It's a recognized command, it just disabled the roll for websocket vs nonwebsocket, for some reason it always tries a websocket -> tcp -> tcp -> udp connection

Code: Select all

[2018-03-12 06:06:18] [0,0] Existing WebSocket ping data (CM04-LON.cm.steampowered.com:27020) results: 10000.0000ms + load 0.0000 (21 seconds ago)
[2018-03-12 06:06:18] [0,0] Existing WebSocket ping data (CM05-IAD.cm.steampowered.com:27020) results: 10000.0000ms + load 0.0000 (971 seconds ago)
[2018-03-12 06:06:18] [0,0] Existing WebSocket ping data (CM05-IAD.cm.steampowered.com:27021) results: 10000.0000ms + load 0.0000 (4520 seconds ago)
[2018-03-12 06:06:18] [0,0] Existing WebSocket ping data (CM06-IAD.cm.steampowered.com:443) results: 10000.0000ms + load 0.0000 (1293 seconds ago)
[2018-03-12 06:06:18] [0,0] Existing WebSocket ping data (cm02-lax.cm.steampowered.com:443) results: 10000.0000ms + load 0.0000 (4453 seconds ago)
[2018-03-12 06:06:18] [0,0] Existing WebSocket ping data (cm02-ord.cm.steampowered.com:443) results: 10000.0000ms + load 0.0000 (4522 seconds ago)
[2018-03-12 06:06:18] [0,0] Existing WebSocket ping data (CM02-FRA.cm.steampowered.com:27021) results: 10000.0000ms + load 0.0000 (978 seconds ago)
[2018-03-12 06:06:18] [0,0] Existing WebSocket ping data (CM04-LON.cm.steampowered.com:27021) results: 10000.0000ms + load 0.0000 (20 seconds ago)
[2018-03-12 06:06:18] [0,0] Existing WebSocket ping data (cm02-lax.cm.steampowered.com:27021) results: 10000.0000ms + load 0.0000 (13552 seconds ago)
[2018-03-12 06:06:18] [0,0] Existing WebSocket ping data (cm02-lax.cm.steampowered.com:27020) results: 10000.0000ms + load 0.0000 (4521 seconds ago)
[2018-03-12 06:06:18] [0,0] PingWebSocketCM() (cm01-ord.cm.steampowered.com:27020) starting...
[2018-03-12 06:06:18] [0,0] PingWebSocketCM() (cm04-ord.cm.steampowered.com:443) starting...
[2018-03-12 06:06:18] [1,0] ConnectFailed('Connection Failed':0) (0.0.0.0:0, WebSocket)
[2018-03-12 06:06:18] [0,0] StartAutoReconnect() will start in 9 seconds
[2018-03-12 06:06:19] [0,0] PingWebSocketCM() (cm01-ord.cm.steampowered.com:27020) failed talking to cm (timeout/neterror - Invalid)
[2018-03-12 06:06:19] [0,0] PingWebSocketCM() (cm04-ord.cm.steampowered.com:443) failed talking to cm (timeout/neterror - Invalid)
[2018-03-12 06:06:19] [1,15] Connect() starting connection (eNetQOSLevelMedium, CM04-STO.cm.steampowered.com:27020, WebSocket)
[2018-03-12 06:06:20] [1,0] ConnectFailed('Connection Failed':0) (0.0.0.0:0, WebSocket)
[2018-03-12 06:06:20] [0,0] StartAutoReconnect() will start in 7 seconds
[2018-03-12 06:06:27] [0,0] Forcing UDP because -udpforce or udpforce in config -- this will go away soon! Don't require UDP on your network!
[2018-03-12 06:06:27] [1,81] Connect() starting connection (eNetQOSLevelMedium, 208.78.164.9:27019, TCP)
[2018-03-12 06:06:27] [0,0] Forcing UDP because -udpforce or udpforce in config -- this will go away soon! Don't require UDP on your network!
[2018-03-12 06:06:27] [1,73] Connect() starting connection (eNetQOSLevelMedium, 208.78.164.9:27019, TCP)
[2018-03-12 06:06:28] [1,0] ConnectFailed('Connection Failed':6f) (208.78.164.9:27019, TCP)
[2018-03-12 06:06:28] [0,0] StartAutoReconnect() will start in 11 seconds
[2018-03-12 06:06:28] [1,0] ConnectFailed('Connection Failed':6f) (208.78.164.9:27019, TCP)
[2018-03-12 06:06:28] [0,0] StartAutoReconnect() will start in 11 seconds
[2018-03-12 06:06:39] [0,0] Forcing UDP because -udpforce or udpforce in config -- this will go away soon! Don't require UDP on your network!
[2018-03-12 06:06:39] [1,85] Connect() starting connection (eNetQOSLevelMedium, 155.133.254.133:27018, TCP)
[2018-03-12 06:06:39] [0,0] Forcing UDP because -udpforce or udpforce in config -- this will go away soon! Don't require UDP on your network!
[2018-03-12 06:06:39] [1,77] Connect() starting connection (eNetQOSLevelMedium, 155.133.254.133:27018, TCP)
[2018-03-12 06:06:40] [1,0] ConnectFailed('Connection Failed':6f) (155.133.254.133:27018, TCP)
[2018-03-12 06:06:40] [0,0] StartAutoReconnect() will start in 5 seconds
[2018-03-12 06:06:40] [1,0] ConnectFailed('Connection Failed':6f) (155.133.254.133:27018, TCP)
[2018-03-12 06:06:40] [0,0] StartAutoReconnect() will start in 5 seconds
[2018-03-12 06:06:45] [0,0] Forcing UDP because -udpforce or udpforce in config -- this will go away soon! Don't require UDP on your network!
[2018-03-12 06:06:45] [1,2] Connect() starting connection (eNetQOSLevelMedium, 162.254.193.7:27019, UDP)
[2018-03-12 06:06:45] [1,2] ConnectionCompleted() (162.254.193.7:27019, UDP)
[2018-03-12 06:06:45] [1,2] RecvMsgClientLogOnResponse() : [A:1:2728261636:9725] 'OK'
[2018-03-12 06:06:45] [0,0] Forcing UDP because -udpforce or udpforce in config -- this will go away soon! Don't require UDP on your network!
[2018-03-12 06:06:45] [1,2] Connect() starting connection (eNetQOSLevelMedium, 162.254.193.7:27019, UDP)
[2018-03-12 06:06:46] [1,2] ConnectionCompleted() (162.254.193.6:27018, UDP)
[2018-03-12 06:06:46] [1,2] RecvMsgClientLogOnResponse() : [A:1:2761777154:9725] 'OK'
the connection time is down from ~5 minutes to ~30 seconds, so its definitely an improvement.

Post Reply