why iptables marked but not route as my except

Question

I want to control the route packets of my browser using domain name , let traffic packets within special domain names through VPN ,and other traffic packets without special domain names through normal route. I achieve this goals using dnsmasq and iptables.

By the way , I using my notebook computer(deepinLinux 4.15.0-29deepin-generic) with dnsmasq and wireguard , my notebook is vpn client and as dns server too.

The steps are as follows:

1configure dnsmasq

configure file :

root@notebook-PC:~# cat /etc/dnsmasq.conf
# auto-generated config file from /etc/config/dhcp
conf-file=/etc/dnsmasq.conf
domain-needed
log-queries
log-facility=/var/log/dnsmasq.log
no-resolv
localise-queries
read-ethers
bogus-priv
expand-hosts
local-service
cache-size=150
domain=lan
server=/lan/
server=223.6.6.6
# dhcp-leasefile=/tmp/dhcp.leases
# addn-hosts=/tmp/hosts
conf-dir=/etc/dnsmasq.d
stop-dns-rebind
rebind-localhost-ok
dhcp-broadcast=tag:needs-broadcast


dhcp-range=lan,192.168.10.100,192.168.10.249,255.255.255.0,12h
# no-dhcp-interface=eth0.2

and only a file in /etc/dnsmasq.d

root@notebook-PC:~# cd /etc/dnsmasq.d/
root@notebook-PC:/etc/dnsmasq.d# more newgfw.conf 
# dnsmasq rules generated by ss_spec_dst_fw
# Last Updated on 2019-04-29 13:36:58
# 
server=/030buy.com/8.8.8.8#53
ipset=/030buy.com/ss_spec_dst_fw
server=/0rz.tw/8.8.8.8#53
ipset=/0rz.tw/ss_spec_dst_fw
server=/1000giri.net/8.8.8.8#53
ipset=/1000giri.net/ss_spec_dst_fw
...........

option "server" and "ipset" let these dns query to 8.8.8.8 , then response host ip added to ipset ss_spec_dst_fw

Of course, I didn't forget to add dnsmasq as dns server

root@notebook-PC:~# nslookup
> google.com
Server:     127.0.0.1
Address:    127.0.0.1#53

Non-authoritative answer:
Name:   google.com
Address: 172.217.24.206
Name:   google.com
Address: 2404:6800:4005:806::200e
>

2 modify wg-quick and start wireguard

I found wg-quick.sh let all traffics through vpn interface , then I comment follow 3 line code ,I want add rule myself.

vi /usr/bin/wg-quick

................
add_default() {
    local table proto key value
    if ! get_fwmark table; then
        table=51820
        while [[ -n $(ip -4 route show table $table) || -n $(ip -6 route show table $table) ]]; do
            ((table++))
        done
        cmd wg set "$INTERFACE" fwmark $table
    fi
    proto=-4
    [[ $1 == *:* ]] && proto=-6
#   cmd ip $proto route add "$1" dev "$INTERFACE" table $table
#   cmd ip $proto rule add not fwmark $table table $table
#   cmd ip $proto rule add table main suppress_prefixlength 0
    while read -r key _ value; do
        [[ $value -eq 1 ]] && sysctl -q "$key=2"
    done < <(sysctl -a -r '^net\.ipv4.conf\.[^ .=]+\.rp_filter$')
    return 0
}
................

then when wireguard start

wg-quick up wgnet0

wgnet0: flags=209<UP,POINTOPOINT,RUNNING,NOARP>  mtu 1420
        inet 192.168.32.2  netmask 255.255.255.0  destination 192.168.32.2
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 1000  (UNSPEC)
        RX packets 15  bytes 1380 (1.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 615  bytes 21652 (21.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlp5s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.3  netmask 255.255.255.0  broadcast 192.168.1.255

wgnet0 is vpn interface using through special domain name traffics , and wlp5s0 is normal interface using through normal traffic.

ip rule show follow

root@notebook-PC:~# ip rule
0:  from all lookup local 
220:    from all lookup 220 
32766:  from all lookup main 
32767:  from all lookup default

[3] configure rule and route

add wg table in file /etc/iproute2/rt_tables

root@notebook-PC:~# cat /etc/iproute2/rt_tables
#
# reserved values
#
255 local
254 main
253 default
201     wg
0   unspec
#
# local
#
#1  inr.ruhep

then add rule as follow

ip -4 route add table wg default via 192.168.32.1 dev wgnet0 metric 100
ip rule add prio 100 from all fwmark 51820 lookup wg

root@notebook-PC:~# ip rule show all
0:  from all lookup local 
100:    from all fwmark 0xca6c lookup wg 
220:    from all lookup 220 
32766:  from all lookup main 
32767:  from all lookup default 
root@notebook-PC:~# ip route show table wg
default via 192.168.32.1 dev wgnet0 metric 100

[4] enable mark

root@notebook-PC:~# cat /proc/sys/net/ipv4/tcp_fwmark_accept 
0
root@notebook-PC:~# sysctl -w net.ipv4.tcp_fwmark_accept=1
net.ipv4.tcp_fwmark_accept = 1
root@notebook-PC:~# cat /proc/sys/net/ipv4/tcp_fwmark_accept 
1

[5] configure iptables add chain and rule

Because of the previous steps, I have added the IP that needs to route VPN interface into ipset ss_spec_dst_fw , then I need to mark these packets.

With this article(http://www.faqs.org/docs/iptables/traversingoftables.html), iptables makes routing decisions in two steps , and local process ( browser process on my notebook in this case ) makes routing decision before entering OUTPUT chian , then I need local process reroute.

so as what I did , I add rule in OUTPUT_direct chian ,which all packets match ipset ss_spec_dst_fw jump to PREROUTING_direct chain , and add rules in PREROUTING_direct chain , which all packets match ipset ss_spec_dst_fw be marked ,and According to the picture below:

when PREROUTING_direct chain() as ACCEPT packets then make routing decision again, operation as follow:

root@notebook-PC:~# iptables -t mangle -A OUTPUT_direct -m set --match-set ss_spec_dst_fw dsts -j PREROUTING_direct

root@notebook-PC:~# iptables -t mangle -A PREROUTING_direct -j CONNMARK --restore-mark
root@notebook-PC:~# iptables -t mangle -A PREROUTING_direct -m set --match-set ss_spec_dst_fw dsts -j MARK --set-mark 51820
root@notebook-PC:~# iptables -t mangle -A PREROUTING_direct -j CONNMARK --save-mark
root@notebook-PC:~# iptables -t mangle -A PREROUTING_direct -j ACCEPT
root@notebook-PC:~# 
root@notebook-PC:~# iptables -L -t mangle
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         
PREROUTING_direct  all  --  anywhere             anywhere            
PREROUTING_ZONES_SOURCE  all  --  anywhere             anywhere            
PREROUTING_ZONES  all  --  anywhere             anywhere            

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
INPUT_direct  all  --  anywhere             anywhere            

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
FORWARD_direct  all  --  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
OUTPUT_direct  all  --  anywhere             anywhere            

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
POSTROUTING_direct  all  --  anywhere             anywhere            

Chain FORWARD_direct (1 references)
target     prot opt source               destination         

Chain INPUT_direct (1 references)
target     prot opt source               destination         

Chain OUTPUT_direct (1 references)
target     prot opt source               destination         
PREROUTING_direct  all  --  anywhere             anywhere             match-set ss_spec_dst_fw dst

Chain POSTROUTING_direct (1 references)
target     prot opt source               destination         

Chain PREROUTING_ZONES (1 references)
target     prot opt source               destination         
PRE_public  all  --  anywhere             anywhere            [goto] 
PRE_public  all  --  anywhere             anywhere            [goto] 

Chain PREROUTING_ZONES_SOURCE (1 references)
target     prot opt source               destination         

Chain PREROUTING_direct (2 references)
target     prot opt source               destination         
CONNMARK   all  --  anywhere             anywhere             CONNMARK restore
MARK       all  --  anywhere             anywhere             match-set ss_spec_dst_fw dst MARK set 0xca6c
CONNMARK   all  --  anywhere             anywhere             CONNMARK save
ACCEPT     all  --  anywhere             anywhere            

Chain PRE_public (2 references)
target     prot opt source               destination         
PRE_public_log  all  --  anywhere             anywhere            
PRE_public_deny  all  --  anywhere             anywhere            
PRE_public_allow  all  --  anywhere             anywhere            

Chain PRE_public_allow (1 references)
target     prot opt source               destination         

Chain PRE_public_deny (1 references)
target     prot opt source               destination         

Chain PRE_public_log (1 references)
target     prot opt source               destination         
root@notebook-PC:~#

Now ,all work is done , but reuslt is NOT I except ,it's not working!!!

ipset ss_spec_dst_fw added ip response from dns query

root@notebook-PC:~# ipset list ss_spec_dst_fw
Name: ss_spec_dst_fw
Type: hash:ip
Revision: 4
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 5368
References: 2
Number of entries: 113
Members:
216.58.221.229
172.217.161.170
172.217.161.173
203.208.43.95
216.58.199.5
216.58.221.234
216.58.199.110
172.217.161.141
172.217.161.133
216.58.200.74
172.217.161.138
203.208.39.215
...........

dnsmasq is well ,then I testing these ip route:

root@notebook-PC:~# ip route get 216.58.221.229
216.58.221.229 via 192.168.1.1 dev wlp5s0 src 192.168.1.3 uid 0 
    cache

This result is not what I expected , 192.168.1.1 is normal gateway not vpn gateway , then I test add mark 51820 (mark value of above code),

root@notebook-PC:~# ip route flush cache

root@notebook-PC:~# ip route get 216.58.221.229 mark 51820
216.58.221.229 via 192.168.32.1 dev wgnet0 table wg src 192.168.32.2 mark 0xca6c uid 0 
    cache 
root@notebook-PC:~# ip route flush cache
root@notebook-PC:~# ip route get 216.58.221.229
216.58.221.229 via 192.168.1.1 dev wlp5s0 src 192.168.1.3 uid 0 
    cache

These result verify ip rule and route is well but other not, then I check if iptables mark these packets , I traceroute 216.58.221.229 and log

root@notebook-PC:~# traceroute 216.58.221.229
traceroute to 216.58.221.229 (216.58.221.229), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  *

and conntrack log

root@notebook-PC:~# conntrack -E -d 216.58.221.229
    [NEW] udp      17 30 src=192.168.1.3 dst=216.58.221.229 sport=44951 dport=33434 [UNREPLIED] src=216.58.221.229 dst=192.168.1.3 sport=33434 dport=44951 mark=51820
    [NEW] udp      17 30 src=192.168.1.3 dst=216.58.221.229 sport=34181 dport=33435 [UNREPLIED] src=216.58.221.229 dst=192.168.1.3 sport=33435 dport=34181 mark=51820
    [NEW] udp      17 30 src=192.168.1.3 dst=216.58.221.229 sport=58059 dport=33436 [UNREPLIED] src=216.58.221.229 dst=192.168.1.3 sport=33436 dport=58059 mark=51820
    [NEW] udp      17 30 src=192.168.1.3 dst=216.58.221.229 sport=40739 dport=33437 [UNREPLIED] src=216.58.221.229 dst=192.168.1.3 sport=33437 dport=40739 mark=51820
........................

mark 51820 is done !

Now I'm confused, what am I missing ?

Anton Danilov · Accepted Answer · 2019-05-27 19:03:48Z

0

Local originated packets don't pass through mangle/PREROUTING chain. You should move your marking rules into mangle/OUTPUT chain. You can use 'iptables-save -c' or 'iptables -L -n -v' to list the rule set (the first command is better, because it shows the full rule set) and check the rule counters. Likely in your case the counters are zero.

answered May 27, 2019 at 19:03

Anton Danilov

1511 silver badge3 bronze badges

Add a comment |

Stack Exchange Network

why iptables marked but not route as my except

1 Answer 1

You must log in to answer this question.

Hot Network Questions

why iptables marked but not route as my except

1 Answer 1

You must log in to answer this question.

Related

Hot Network Questions