feature: add new option: ipset-no-speed and nftset-no-speed

2023-02-09 21:31:02 +08:00
parent d6f9b07f1c
commit 69a2f3bb7f
6 changed files with 237 additions and 7 deletions
--- a/ReadMe.md
+++ b/ReadMe.md
@@ -112,7 +112,7 @@ rtt min/avg/max/mdev = 5.954/6.133/6.313/0.195 ms
   支持域名后缀匹配模式，简化过滤配置，过滤 20 万条记录时间 < 1ms。

 1. **域名分流**  
-   支持域名分流，不同类型的域名向不同的 DNS 服务器查询，支持iptable和nftable更好的分流。
+   支持域名分流，不同类型的域名向不同的 DNS 服务器查询，支持iptable和nftable更好的分流；支持测速失败的情况下设置域名结果到对应ipset和nftset集合。

 1. **Windows / Linux 多平台支持**  
   支持标准 Linux 系统（树莓派）、OpenWrt 系统各种固件和华硕路由器原生固件。同时还支持 WSL（Windows Subsystem for Linux，适用于 Linux 的 Windows 子系统）。
@@ -603,10 +603,12 @@ entware|ipkg update<br />ipkg install smartdns|软件源路径：<https://bin.en
 | response-mode | 首次查询响应模式 | first-ping |模式：[first-ping\|fastest-ip\|fastest-response]<br /> [first-ping]: 最快ping响应地址模式，DNS上游最快查询时延+ping时延最短，查询等待与链接体验最佳;<br />[fastest-ip]: 最快IP地址模式，查询到的所有IP地址中ping最短的IP。需等待IP测速; <br />[fastest-response]: 最快响应的DNS结果，DNS查询等待时间最短，返回的IP地址可能不是最快。| response-mode first-ping |
 | address | 指定域名 IP 地址 | 无 | address /domain/[ip\|-\|-4\|-6\|#\|#4\|#6] <br />- 表示忽略 <br /># 表示返回 SOA <br />4 表示 IPv4 <br />6 表示 IPv6 | address /www.example.com/1.2.3.4 |
 | nameserver | 指定域名使用 server 组解析 | 无 | nameserver /domain/[group\|-], group 为组名，- 表示忽略此规则，配套 server 中的 -group 参数使用 | nameserver /www.example.com/office |
-| ipset | 域名 ipset | 无 | ipset /domain/[ipset\|-\|#[4\|6]:[ipset\|-][,#[4\|6]:[ipset\|-]]]，-表示忽略 | ipset /www.example.com/#4:dns4,#6:- |
+| ipset | 域名 ipset | 无 | ipset /domain/[ipset\|-\|#[4\|6]:[ipset\|-][,#[4\|6]:[ipset\|-]]]，-表示忽略 | ipset /www.example.com/#4:dns4,#6:- <br />ipset /www.example.com/dns |
 | ipset-timeout | 设置 ipset 超时功能启用  | no | [yes\|no] | ipset-timeout yes |
-| nftset | 域名 nftset | 无 | nftset /domain/[#4\|#6\|-]:[family#nftable#nftset\|-][,#[4\|6]:[family#nftable#nftset\|-]]]，-表示忽略；ipv4 地址的 family 只支持 inet 和 ip；ipv6 地址的 family 只支持 inet 和 ip6；由于 nft 限制，两种地址只能分开存放于两个 set 中。| nftset /www.example.com/#4:inet#tab#dns4,#6:- |
+| ipset-no-speed | 当测速失败时，将域名结果设置到ipset集合中 | 无 | ipset \| #[4\|6]:ipset | ipset-no-speed #4:ipset4,#6:ipse6 <br /> ipset-no-speed ipset|
+| nftset | 域名 nftset | 无 | nftset /domain/[#4\|#6\|-]:[family#nftable#nftset\|-][,#[4\|6]:[family#nftable#nftset\|-]]]，<br />-表示忽略；<br />ipv4 地址的 family 只支持 inet 和 ip；<br />ipv6 地址的 family 只支持 inet 和 ip6；<br />由于 nft 限制，两种地址只能分开存放于两个 set 中。| nftset /www.example.com/#4:inet#tab#dns4,#6:- |
 | nftset-timeout | 设置 nftset 超时功能启用  | no | [yes\|no] | nftset-timeout yes |
+| nftset-no-speed | 当测速失败时，将域名结果设置到nftset集合中 | 无 | nftset-no-speed [#4\|#6]:[family#nftable#nftset][,#[4\|6]:[family#nftable#nftset]]] <br />ipv4 地址的 family 只支持 inet 和 ip <br />ipv6 地址的 family 只支持 inet 和 ip6 <br />由于 nft 限制，两种地址只能分开存放于两个 set 中。| nftset-no-speed #4:inet#tab#set4|
 | nftset-debug | 设置 nftset 调试功能启用  | no | [yes\|no] | nftset-debug yes |
 | domain-rules | 设置域名规则 | 无 | domain-rules /domain/ [-rules...]<br />[-c\|-speed-check-mode]：测速模式，参考 speed-check-mode 配置<br />[-a\|-address]：参考 address 配置<br />[-n\|-nameserver]：参考 nameserver 配置<br />[-p\|-ipset]：参考ipset配置<br />[-t\|-nftset]：参考nftset配置<br />[-d\|-dualstack-ip-selection]：参考 dualstack-ip-selection<br /> [-no-serve-expired]：禁用过期缓存<br />[-delete]：删除对应的规则 | domain-rules /www.example.com/ -speed-check-mode none |
 | domain-set | 设置域名集合 | 无 | domain-set [options...]<br />[-n\|-name]：域名集合名称 <br />[-t\|-type]：域名集合类型，当前仅支持list，格式为域名列表，一行一个域名。<br />[-f\|-file]：域名集合文件路径。<br /> 选项需要配合address, nameserver, ipset, nftset等需要指定域名的地方使用，使用方式为 /domain-set:[name]/| domain-set -name set -type list -file /path/to/list <br /> address /domain-set:set/1.2.4.8 |
@@ -868,7 +870,8 @@ entware|ipkg update<br />ipkg install smartdns|软件源路径：<https://bin.en
    

    1. 额外说明  
-      为保证DNS查询结果的位置亲和性，可以使用smartdns的`server`代理参数，将对应域名的查询请求，通过代理查询，使结果位置更好。如：
+      
+        - 为保证DNS查询结果的位置亲和性，可以使用smartdns的`server`代理参数，将对应域名的查询请求，通过代理查询，使结果位置更好。如：

        ```shell
        # 增加DNS上游，并设置通过名称为proxy的代理查询，查询组为pass
@@ -879,6 +882,12 @@ entware|ipkg update<br />ipkg install smartdns|软件源路径：<https://bin.en
        domain-rules /example.com/ -ipset proxy -c none -address #6 -nameserver pass
        ```

+        - 如需要配合测速自动完成ipset的设置，可增加如下配置参数
+
+        ```shell
+        ipset-no-speed proxy
+        ```
+
    如果使用openwrt的luci界面，可以直接在界面配置相关的域名分流规则。

 1. 更多问题  
--- a/ReadMe_en.md
+++ b/ReadMe_en.md
@@ -119,7 +119,7 @@ From the comparison, smartdns found the fastest IP address to visit www.baidu.co
   Multi-threaded asynchronous IO mode, cache cache query results.

 1. **DNS domain forwarding**
-   Support DNS forwarding, ipset and nftables.
+   Support DNS forwarding, ipset and nftables. Support setting the domain result to ipset and nftset set when speed check fails.

 ## Architecture

@@ -568,8 +568,10 @@ Note: Merlin firmware is derived from ASUS firmware and can theoretically be use
 |nameserver|To query domain with specific server group|None|nameserver /domain/[group\|-], `group` is the group name, `-` means ignore this rule, use the `-group` parameter in the related server|nameserver /www.example.com/office
 |ipset|Domain IPSet|None|ipset /domain/[ipset\|-\|#[4\|6]:[ipset\|-][,#[4\|6]:[ipset\|-]]], `-` for ignore|ipset /www.example.com/#4:dns4,#6:-
 |ipset-timeout|ipset timeout enable|no|[yes\|no]|ipset-timeout yes
-|nftset|Domain nftset|None|nftset /domain/[#4\|#6\|-]:[family#nftable#nftset\|-][,#[4\|6]:[family#nftable#nftset\|-]]], `-` to ignore; the valid families are inet and ip for ipv4 addresses while the valid ones are inet and ip6 for ipv6 addresses; due to the limitation of nft, two types of addresses have to be stored in two sets|nftset /www.example.com/#4:inet#tab#dns4,#6:-
+|ipset-no-speed|When speed check fails, set the ip address of the domain name to the ipset | None | ipset \| #[4\|6]:ipset | ipset-no-speed #4:ipset4,#6:ipse6 <br /> ipset-no-speed ipset|
+|nftset|Domain nftset|None|nftset /domain/[#4\|#6\|-]:[family#nftable#nftset\|-][,#[4\|6]:[family#nftable#nftset\|-]]]<br /> `-` to ignore<br />the valid families are inet and ip for ipv4 addresses while the valid ones are inet and ip6 for ipv6 addresses <br />due to the limitation of nftable <br />two types of addresses have to be stored in two sets|nftset /www.example.com/#4:inet#tab#dns4,#6:-
 |nftset-timeout|nftset timeout enable|no|[yes\|no]|nftset-timeout yes
+|nftset-no-speed|When speed check fails, set the ip address of the domain name to the nftset | None | nftset-no-speed [#4\|#6]:[family#nftable#nftset][,#[4\|6]:[family#nftable#nftset]]] <br />the valid families are inet and ip for ipv4 addresses while the valid ones are inet and ip6 for ipv6 addresses <br />due to the limitation of nftable <br />two types of addresses have to be stored in two sets| nftset-no-speed #4:inet#tab#set4|
 |nftset-debug|nftset debug enable|no|[yes\|no]|nftset-debug yes
 |domain-rules|set domain rules|None|domain-rules /domain/ [-rules...]<br />[-c\|-speed-check-mode]: set speed check mode，same as parameter speed-check-mode<br />[-a\|-address]: same as  parameter `address` <br />[-n\|-nameserver]: same as parameter `nameserver`<br />[-p\|-ipset]: same as parameter `nftset`<br />[-t\|-nftset]: same as parameter `nftset`<br />[-d\|-dualstack-ip-selection]: same as parameter `dualstack-ip-selection`<br />  [-no-serve-expired]：disable serve expired<br />[-delete]：delete rule|domain-rules /www.example.com/ -speed-check-mode none
 | domain-set | collection of domains|None| domain-set [options...]<br />[-n\|-name]：name of set <br />[-t\|-type] [list]: set type, only support list, one domain per line <br />[-f\|-file]：file path of domain set<br /> used with address, nameserver, ipset, nftset, example: /domain-set:[name]/ | domain-set -name set -type list -file /path/to/list <br /> address /domain-set:set/1.2.4.8 |
--- a/etc/smartdns/smartdns.conf
+++ b/etc/smartdns/smartdns.conf
@@ -234,10 +234,18 @@ log-level info
 # ipset /www.example.com/block, set ipset with ipset name of block 
 # ipset /www.example.com/-, ignore this domain

+# add to ipset when ping is unreachable
+# ipset-no-speed ipsetname
+# ipset-no-speed pass
+
 # enable nftset timeout by ttl feature
 # nftset-timeout [yes|no]
 # nftset-timeout yes

+# add to nftset when ping is unreachable
+# nftset-no-speed [#4:ip#table#set,#6:ipv6#table#setv6]
+# nftset-no-speed #4:ip#table#set
+
 # enable nftset debug, check nftset setting result, output log when error.
 # nftset-debug [yes|no]
 # nftset-debug yes
--- a/src/dns_conf.c
+++ b/src/dns_conf.c
@@ -140,7 +140,9 @@ int dns_conf_local_ttl;
 int dns_conf_force_AAAA_SOA;
 int dns_conf_force_no_cname;
 int dns_conf_ipset_timeout_enable;
+struct dns_ipset_names dns_conf_ipset_no_speed;
 int dns_conf_nftset_timeout_enable;
+struct dns_nftset_names dns_conf_nftset_no_speed;
 int dns_conf_nftset_debug_enable;

 char dns_conf_user[DNS_CONF_USERNAME_LEN];
@@ -1076,6 +1078,78 @@ errout:
 	return 0;
 }

+static int _config_ipset_no_speed(void *data, int argc, char *argv[])
+{
+	char *ipsetname = argv[1];
+	char *copied_name = NULL;
+	const char *ipset = NULL;
+	struct dns_ipset_rule *ipset_rule_array[2] = {NULL, NULL};
+	char *ipset_rule_enable_array[2] = {NULL, NULL};
+	int ipset_num = 0;
+
+	if (argc <= 1) {
+		goto errout;
+	}
+
+	copied_name = strdup(ipsetname);
+
+	if (copied_name == NULL) {
+		goto errout;
+	}
+
+	for (char *tok = strtok(copied_name, ","); tok && ipset_num <= 2; tok = strtok(NULL, ",")) {
+		if (tok[0] == '#') {
+			if (strncmp(tok, "#6:", 3U) == 0) {
+				ipset_rule_array[ipset_num] = &dns_conf_ipset_no_speed.ipv6;
+				ipset_rule_enable_array[ipset_num] = &dns_conf_ipset_no_speed.ipv6_enable;
+				ipset_num++;
+			} else if (strncmp(tok, "#4:", 3U) == 0) {
+				ipset_rule_array[ipset_num] = &dns_conf_ipset_no_speed.ipv4;
+				ipset_rule_enable_array[ipset_num] = &dns_conf_ipset_no_speed.ipv4_enable;
+				ipset_num++;
+			} else {
+				goto errout;
+			}
+			tok += 3;
+		}
+
+		if (ipset_num == 0) {
+			ipset_rule_array[1] = &dns_conf_ipset_no_speed.ipv6;
+			ipset_rule_enable_array[1] = &dns_conf_ipset_no_speed.ipv6_enable;
+			ipset_rule_array[0] = &dns_conf_ipset_no_speed.ipv4;
+			ipset_rule_enable_array[0] = &dns_conf_ipset_no_speed.ipv4_enable;
+			ipset_num = 2;
+		}
+
+		if (strncmp(tok, "-", 1) == 0) {
+			continue;
+		}
+
+		/* new ipset domain */
+		ipset = _dns_conf_get_ipset(tok);
+		if (ipset == NULL) {
+			goto errout;
+		}
+
+		for (int i = 0; i < ipset_num; i++) {
+			ipset_rule_array[i]->ipsetname = ipset;
+			*ipset_rule_enable_array[i] = 1;
+		}
+
+		ipset_num = 0;
+	}
+
+	free(copied_name);
+	return 0;
+errout:
+	if (copied_name) {
+		free(copied_name);
+	}
+
+	tlog(TLOG_ERROR, "add ipset-no-speed %s failed", ipsetname);
+	return 0;
+}
+
 static void _config_nftset_table_destroy(void)
 {
 	struct dns_nftset_name *nftset = NULL;
@@ -1187,7 +1261,7 @@ static int _conf_domain_rule_nftset(char *domain, const char *nftsetname)
 			goto errout;
 		}

-		/* new ipset domain */
+		/* new nftset domain */
 		nftset = _dns_conf_get_nftable(family, tablename, setname);
 		if (nftset == NULL) {
 			goto errout;
@@ -1244,6 +1318,105 @@ errout:
 	return 0;
 }

+static int _config_nftset_no_speed(void *data, int argc, char *argv[])
+{
+	const struct dns_nftset_name *nftset = NULL;
+	char *copied_name = NULL;
+	char *nftsetname = argv[1];
+	int nftset_num = 0;
+	char *setname = NULL;
+	char *tablename = NULL;
+	char *family = NULL;
+	struct dns_nftset_rule *nftset_rule_array[2] = {NULL, NULL};
+	char *nftset_rule_enable_array[2] = {NULL, NULL};
+
+	if (argc <= 1) {
+		goto errout;
+	}
+
+	copied_name = strdup(nftsetname);
+
+	if (copied_name == NULL) {
+		goto errout;
+	}
+
+	for (char *tok = strtok(copied_name, ","); tok && nftset_num <=2 ; tok = strtok(NULL, ",")) {
+		char *saveptr = NULL;
+		char *tok_set = NULL;
+
+		if (strncmp(tok, "#4:", 3U) == 0) {
+			dns_conf_nftset_no_speed.ip_enable = 1;
+			nftset_rule_array[nftset_num] = &dns_conf_nftset_no_speed.ip;
+			nftset_rule_enable_array[nftset_num] = &dns_conf_nftset_no_speed.ip_enable;
+			nftset_num++;
+		} else if (strncmp(tok, "#6:", 3U) == 0) {
+			nftset_rule_enable_array[nftset_num] = &dns_conf_nftset_no_speed.ip6_enable;
+			nftset_rule_array[nftset_num] = &dns_conf_nftset_no_speed.ip6;
+			nftset_num++;
+		} else if (strncmp(tok, "-", 2U) == 0) {
+			continue;
+			continue;
+		} else {
+			goto errout;
+		}
+
+		tok_set = tok + 3;
+
+		if (nftset_num == 0) {
+			nftset_rule_array[0] = &dns_conf_nftset_no_speed.ip;
+			nftset_rule_enable_array[0] = &dns_conf_nftset_no_speed.ip_enable;
+			nftset_rule_array[1] = &dns_conf_nftset_no_speed.ip6;
+			nftset_rule_enable_array[1] = &dns_conf_nftset_no_speed.ip6_enable;
+			nftset_num = 2;
+		}
+
+		if (strncmp(tok_set, "-", 2U) == 0) {
+			continue;
+		}
+
+		family = strtok_r(tok_set, "#", &saveptr);
+		if (family == NULL) {
+			goto errout;
+		}
+
+		tablename = strtok_r(NULL, "#", &saveptr);
+		if (tablename == NULL) {
+			goto errout;
+		}
+
+		setname = strtok_r(NULL, "#", &saveptr);
+		if (setname == NULL) {
+			goto errout;
+		}
+
+		/* new nftset domain */
+		nftset = _dns_conf_get_nftable(family, tablename, setname);
+		if (nftset == NULL) {
+			goto errout;
+		}
+
+		for (int i = 0; i < nftset_num; i++) {
+			nftset_rule_array[i]->familyname = nftset->nftfamilyname;
+			nftset_rule_array[i]->nfttablename = nftset->nfttablename;
+			nftset_rule_array[i]->nftsetname = nftset->nftsetname;
+			*nftset_rule_enable_array[i] = 1;
+		}
+
+		nftset_num = 0;
+	}
+
+	goto clear;
+
+errout:
+	tlog(TLOG_ERROR, "add nftset %s failed", nftsetname);
+clear:
+	if (copied_name) {
+		free(copied_name);
+	}
+
+	return 0;
+}
+
 static int _conf_domain_rule_address(char *domain, const char *domain_address)
 {
 	struct dns_rule_address_IPV4 *address_ipv4 = NULL;
@@ -2696,9 +2869,11 @@ static struct config_item _config_item[] = {
 	CONF_CUSTOM("proxy-server", _config_proxy_server, NULL),
 	CONF_YESNO("ipset-timeout", &dns_conf_ipset_timeout_enable),
 	CONF_CUSTOM("ipset", _config_ipset, NULL),
+	CONF_CUSTOM("ipset-no-speed", _config_ipset_no_speed, NULL),
 	CONF_YESNO("nftset-timeout", &dns_conf_nftset_timeout_enable),
 	CONF_YESNO("nftset-debug", &dns_conf_nftset_debug_enable),
 	CONF_CUSTOM("nftset", _config_nftset, NULL),
+	CONF_CUSTOM("nftset-no-speed", _config_nftset_no_speed, NULL),
 	CONF_CUSTOM("speed-check-mode", _config_speed_check_mode, NULL),
 	CONF_INT("tcp-idle-time", &dns_conf_tcp_idle_time, 0, 3600),
 	CONF_INT("cache-size", &dns_conf_cachesize, 0, CONF_INT_MAX),
--- a/src/dns_conf.h
+++ b/src/dns_conf.h
@@ -148,6 +148,14 @@ struct dns_ipset_rule {
 	const char *ipsetname;
 };

+struct dns_ipset_names {
+	char ipv4_enable;
+	char ipv6_enable;
+	struct dns_ipset_rule ipv4;
+	struct dns_ipset_rule ipv6;
+};
+extern struct dns_ipset_names dns_conf_ipset_no_speed;
+
 struct dns_nftset_name {
 	struct hlist_node node;
 	char nftfamilyname[DNS_MAX_NFTSET_FAMILYLEN];
@@ -162,6 +170,16 @@ struct dns_nftset_rule {
 	const char *nftsetname;
 };

+struct dns_nftset_names {
+	char inet_enable;
+	char ip_enable;
+	char ip6_enable;
+	struct dns_nftset_rule inet;
+	struct dns_nftset_rule ip;
+	struct dns_nftset_rule ip6;
+};
+extern struct dns_nftset_names dns_conf_nftset_no_speed;
+
 struct dns_domain_rule {
 	struct dns_rule head;
 	struct dns_rule *rules[DOMAIN_RULE_MAX];
--- a/src/dns_server.c
+++ b/src/dns_server.c
@@ -1428,6 +1428,7 @@ static int _dns_server_setup_ipset_nftset_packet(struct dns_server_post_context
 	struct dns_nftset_rule *nftset_ip = NULL;
 	struct dns_nftset_rule *nftset_ip6 = NULL;
 	struct dns_rule_flags *rule_flags = NULL;
+	int check_no_speed_rule = 0;

 	if (_dns_server_has_bind_flag(request, BIND_FLAG_NO_RULE_IPSET) == 0) {
 		return 0;
@@ -1441,6 +1442,10 @@ static int _dns_server_setup_ipset_nftset_packet(struct dns_server_post_context
 		return 0;
 	}

+	if (request->ping_time < 0 && request->has_ip > 0 && request->passthrough == 0) {
+		check_no_speed_rule = 1;
+	}
+
 	/* check ipset rule */
 	rule_flags = _dns_server_get_dns_rule(request, DOMAIN_RULE_FLAGS);
 	if (!rule_flags || (rule_flags->flags & DOMAIN_FLAG_IPSET_IGN) == 0) {
@@ -1449,18 +1454,30 @@ static int _dns_server_setup_ipset_nftset_packet(struct dns_server_post_context

 	if (!rule_flags || (rule_flags->flags & DOMAIN_FLAG_IPSET_IPV4_IGN) == 0) {
 		ipset_rule_v4 = _dns_server_get_dns_rule(request, DOMAIN_RULE_IPSET_IPV4);
+		if (ipset_rule == NULL && check_no_speed_rule && dns_conf_ipset_no_speed.ipv4_enable) {
+			ipset_rule_v4 = &dns_conf_ipset_no_speed.ipv4;
+		}
 	}

 	if (!rule_flags || (rule_flags->flags & DOMAIN_FLAG_IPSET_IPV6_IGN) == 0) {
 		ipset_rule_v6 = _dns_server_get_dns_rule(request, DOMAIN_RULE_IPSET_IPV6);
+		if (ipset_rule_v6 == NULL && check_no_speed_rule && dns_conf_ipset_no_speed.ipv6_enable) {
+			ipset_rule_v6 = &dns_conf_ipset_no_speed.ipv6;
+		}
 	}

 	if (!rule_flags || (rule_flags->flags & DOMAIN_FLAG_NFTSET_IP_IGN) == 0) {
 		nftset_ip = _dns_server_get_dns_rule(request, DOMAIN_RULE_NFTSET_IP);
+		if (nftset_ip == NULL && check_no_speed_rule && dns_conf_nftset_no_speed.ip_enable) {
+			nftset_ip = &dns_conf_nftset_no_speed.ip;
+		}
 	}

 	if (!rule_flags || (rule_flags->flags & DOMAIN_FLAG_NFTSET_IP6_IGN) == 0) {
 		nftset_ip6 = _dns_server_get_dns_rule(request, DOMAIN_RULE_NFTSET_IP6);
+		if (nftset_ip6 == NULL && check_no_speed_rule && dns_conf_nftset_no_speed.ip6_enable) {
+			nftset_ip6 = &dns_conf_nftset_no_speed.ip6;
+		}
 	}

 	if (!(ipset_rule || ipset_rule_v4 || ipset_rule_v6 || nftset_ip || nftset_ip6)) {
@@ -3006,6 +3023,7 @@ static void _dns_server_query_end(struct dns_request *request)
 	/* Not need to wait check result if only has one ip address */
 	if (ip_num == 1 && request_wait == 1) {
 		if (request->dualstack_selection_query == 1) {
+			_dns_server_request_complete(request);
 			goto out;
 		}