Compare commits

...

19 Commits

Author SHA1 Message Date
TripleA
6d2b6f68c8 Add telega (#3384) 2026-03-25 21:19:48 +08:00
MkQtS
1e9bab323b Add more cn domains (#3388)
* alibaba: add wdjimg.com

* xunlei: add more domains

* category-ai-cn: add more domains

* category-documents-cn: add 51wendang.com

* category-entertainment-cn: add yxdown.com

* category-games-cn: add feihuo.com

* category-media-cn: add more domains

* geolocation-cn: add 51ctocdn.cn

and reorder
2026-03-25 13:21:23 +08:00
𝒮𝒶𝓀𝓊𝓇𝒶𝒞𝒶𝓉🍀
e56bbf01fa category-games-!cn: add some minecraft-related domains (#3386) 2026-03-25 12:01:54 +08:00
fdrvn
af25bbc8a5 category-ip-geo-detect: add showip.net (#3387) 2026-03-25 11:36:41 +08:00
MkQtS
5de9d7cc94 geolocation-!cn: add xiaomiflash.com (#3383) 2026-03-24 17:12:05 +08:00
MkQtS
dcfbdb7255 ozon: add more domains (#3382) 2026-03-24 17:08:33 +08:00
Emik
749bcd62aa add openrec (#3377) 2026-03-23 21:55:22 +08:00
MkQtS
aa76e186cb Add more cn domains (#3380)
* weiphone: add weiphone.net

* category-education-cn: add more domains

* category-media-cn: add more domains

* geolocation-cn: add more domains
2026-03-22 15:43:41 +08:00
fdrvn
c9348f1db4 category-ip-geo-detect: add myip.wtf (#3379) 2026-03-22 12:28:34 +08:00
深鸣
d62599c8db category-acg: add bestdori.com (#3376) 2026-03-20 13:25:24 +08:00
MkQtS
fab6275217 Update README.md
Add tips for generating custom dat files.

[skip ci]
2026-03-20 11:02:07 +08:00
MkQtS
4c7afec5a9 category-electronic-cn: remove non-cn domains (#3375)
They were added in `espressif` and included in `geolocation-!cn`.
2026-03-20 10:53:04 +08:00
Jarl-Penguin
330c30eb23 category-ip-geo-detect: Add ip.hetzner.com (#3374)
Signed-off-by: Jarl-Penguin <jarlpenguin@outlook.com>
2026-03-20 10:47:36 +08:00
inf
f34f22819e category-dev: add ziglang.org (#3373) 2026-03-20 10:46:09 +08:00
ir0nmand0
baa1409cfb category-entertainment-ru: add beeline.tv (#3372)
Beeline TV (beeline.tv) is a Russian streaming service by VEON (Beeline).
Movies, TV series, and live TV channels for Russian-speaking audience.

Subdomains (covered by domain match): web-prod, rest, images, static, video.
External deps (mediavitrina.ru, vimpelcom.ru) already in category-ru.

Co-authored-by: Dima Dudukin <dima.dudukin.dev@gmail.com>
2026-03-19 18:12:53 +08:00
MkQtS
a22d247c5a qcloud: comment out useless regexp rules (#3371)
Overrided by other domain type rules, but cannot be optimized automatically.

They are actually useless and only affect performance.
2026-03-19 12:43:29 +08:00
yobarerukoto
d311bbe50b geolocation-cn: add gzyowin.com (#3369) 2026-03-18 19:29:43 +08:00
MkQtS
1db558b165 main.go: support to generate multiple custom dats (#3367)
This allows to remove any unwanted lists without modifying the domains
data, and you can generate multiple custom v2ray dat files in a single
command.

As long as the source data is consistent, any list remains in the trimed
dat contains the same rules comparing to the list in full dat.

Use the new option `datprofile` to specify the config json file path.
`outputname` will be ignored when `datprofile` is set.

Co-authored-by: database64128 <free122448@hotmail.com>
2026-03-18 18:32:05 +08:00
Konstantin
9ee0757263 Add Tilda domains (#3368)
* add tilda

* tilda: add to category-dev
2026-03-18 18:05:54 +08:00
26 changed files with 265 additions and 53 deletions

6
.gitignore vendored
View File

@@ -4,9 +4,9 @@
/domain-list-community
/domain-list-community.exe
# Generated dat file.
dlc.dat
# Generated dat files.
/*.dat
# Exported plaintext lists.
/*.yml
/*.txt
/*.yml

View File

@@ -86,6 +86,8 @@ Each file in the `data` directory can be used as a rule in this format: `geosite
Run `go run ./ --help` for more usage information.
For anyone who wants to generate custom `.dat` files, you may read [#3370](https://github.com/v2fly/domain-list-community/discussions/3370).
## Structure of data
All data are under `data` directory. Each file in the directory represents a sub-list of domains, named by the file name. File content is in the following format.
@@ -105,7 +107,7 @@ regexp:^odd[1-7]\.example\.org(\.[a-z]{2})?$
> Adding new `regexp` and `keyword` rules is discouraged because it is easy to use them incorrectly, and proxy software cannot efficiently match these types of rules.
> [!NOTE]
> The following types of rules are **NOT** fully compatible with the ones that defined by user in V2Ray config file. Do **Not** copy and paste directly.
> The following types of rules are **NOT** fully compatible with the ones that defined by user in V2Ray config file. Do **NOT** copy and paste directly.
- Comment begins with `#`. It may begin anywhere in the file. The content in the line after `#` is treated as comment and ignored in production.
- Subdomain begins with `domain:`, followed by a valid domain name. The prefix `domain:` may be omitted.
@@ -114,7 +116,7 @@ regexp:^odd[1-7]\.example\.org(\.[a-z]{2})?$
- Regular expression begins with `regexp:`, followed by a valid regular expression (per Golang's standard).
- Domain rules (including `domain`, `full`, `keyword`, and `regexp`) may have none, one or more attributes. Each attribute begins with `@` and followed by the name of the attribute. Attributes will remain available in final lists and `dlc.dat`.
- Domain rules may have none, one or more affiliations, which additionally adds the domain rule into the affiliated target list. Each affiliation begins with `&` and followed by the name of the target list (nomatter whether the target has a dedicated file in data path). This is a method for data management, and will not remain in the final lists or `dlc.dat`.
- Inclusion begins with `include:`, followed by the name of another valid domain list. A simple `include:listb` in file `lista` means adding all domain rules of `listb` into `lista`. Inclusions with attributes stands for selective inclusion. `include:listb @attr1 @-attr2` means only adding those domain rules *with* `@attr1` **and** *without* `@attr2`. This is a special type for data management, and will not remain in the final lists or `dlc.dat`.
- Inclusion begins with `include:`, followed by the name of another valid domain list. `include:listb` in file `lista` means adding all domain rules of `listb` into `lista`. Inclusions with attributes stand for selective inclusion. `include:listb @attr1 @-attr2` means only adding those domain rules *with* `@attr1` **and** *without* `@attr2`. This is a special type for data management, and will not remain in the final lists or `dlc.dat`.
## How it works

View File

@@ -23,7 +23,6 @@ include:uc
include:umeng
include:youku
25pp.com
56che.com
95095.com
ali-health.com
@@ -114,7 +113,6 @@ sparenode.com
supet.com
tanx.com
ttpod.com
wandoujia.com
xin.xin
xinlingshou.cn
yowhale.com
@@ -204,6 +202,13 @@ xiami.net
# 口碑
koubei.com
# 豌豆荚
wandoujia.com
wdjimg.com
# PP助手
25pp.com
# 吱声网
zhisheng.com

View File

@@ -1,4 +1,5 @@
# Game
bestdori.com
colorfulstage.com
# Idol

View File

@@ -70,3 +70,9 @@ faxingbao.com
meta.law
metaso.cn
xiezuocat.com
# DUI/思必驰
aispeech.com
dui.ai
duiopen.com

View File

@@ -54,6 +54,7 @@ include:stackexchange
include:strikingly
include:termux
include:thelinuxfoundation
include:tilda
include:unity
include:v8
@@ -152,4 +153,5 @@ wireshark.org
x.org
xposed.info
yarnpkg.com
ziglang.org
zsh.org

View File

@@ -1,5 +1,6 @@
1ppt.com
360doc.com
51wendang.com
52ppt.com
badou.com
book118.com

View File

@@ -91,6 +91,7 @@ eoffcn.com
jinrongren.net
offcn.com
zggqzp.com
zgjsks.com
zgsydw.com
# 高考100
gk100.com
@@ -105,6 +106,9 @@ xir.cn
# 日语测试
j-test.com
nattest-china.com
# 菁优网
jyeoo.com
jyeoo.net
# 课播云校
keboyunxiao.com
# 课堂派

View File

@@ -17,11 +17,10 @@ bouffalolab.com
cxmt.com
# 乐鑫信息科技
#include:espressif
esp8266.cn
esp8266.com
esp8266.com.cn
espressif.cn
espressif.com
espressif.com.cn
# 华秋电子

View File

@@ -49,6 +49,7 @@ include:netflix
include:niconico
include:now
include:olevod
include:openrec
include:pbs
include:pixiv
include:plutotv

View File

@@ -119,6 +119,8 @@ vsread.com
yamibo.com
# 易有料
yiyouliao.com
# 游迅网
yxdown.com
# 战旗直播
zhanqi.tv
# 逐浪小说

View File

@@ -11,6 +11,7 @@ include:okko
include:wink
24h.tv
amediateka.ru
beeline.tv
ivi.ru
premier.one
smotreshka.tv

View File

@@ -43,6 +43,7 @@ include:xbox
include:ynoproject
include:yostar
chunkbase.com
dinopoloclub.com
dodi-repacks.download
dodi-repacks.site
@@ -54,13 +55,17 @@ humblebundle.com
joinsquad.com
loverslab.com
minecraft.wiki
minecraftforge.net
mobimon.com.tw
muvluv-girls-garden.com
namemc.com
neoforged.net
nexus-cdn.com
nexusmods.com
noxygames.com
offworldindustries.com
offworldindustries.net
optifine.net
papermc.io
planetminecraft.com
prismlauncher.org
@@ -69,4 +74,5 @@ quiltmc.org
rayark.download
rayark.net
speedrun.com
spigotmc.org
steamdb.info

View File

@@ -20,6 +20,8 @@ include:yokaverse
arcaea.cn
# 《异象回声》游戏官网 沪ICP备2023010411号-1
astral-vector.com
# 飞火游戏
feihuo.com
# 九九互动 粤ICP备19068416号
forevernine.com
# 互动游戏对战平台 粤ICP备18006390号-1

View File

@@ -97,6 +97,7 @@ myexternalip.com
myip.com
myip.ms
myip.ru
myip.wtf
myipaddress.com
myiplookup.com
mylocation.org
@@ -104,6 +105,7 @@ osint.sh
proxycheck.io
realip.cc
seeip.org
showip.net
showmyip.com
showmyipaddress.com
spur.us
@@ -134,6 +136,7 @@ full:checkip.amazonaws.com
full:ipv4-check-perf.radar.cloudflare.com
full:ipv6-check-perf.radar.cloudflare.com
geoip.noc.gov.ru
ip.hetzner.com
ip.mail.ru
ip.nic.ru
ip.tyk.nu

View File

@@ -18,6 +18,7 @@ include:ynet
admin5.com
# 安徽新闻
anhuinews.com
anhuiyun.com
# 锋潮科技
anzhuo.cn
# 鞭牛士
@@ -50,6 +51,11 @@ cqnews.net
cqsjb.com
# 中证网
cs.com.cn
# 极目新闻/楚天都市报
ctdsb.net
full:ctdsb.clouddiffuse.xyz
# 创头条
ctoutiao.com
# 创业邦
cyzone.cn
# 浙江广播电视集团/浙江卫视/新蓝网
@@ -78,6 +84,8 @@ freebuf.com
geekpark.net
# 光明网
gmw.com
# 观察者网
guancha.cn
# 硅谷网
guigu.org
# 同花顺
@@ -101,8 +109,12 @@ huxiu.com
huxiucdn.com
# 电脑报
icpcw.com
# 长沙晚报
icswb.com
# 中国IDC圈
idcquan.com
# 砍柴网
ikanchai.com
# 艾瑞咨询
iresearch.cn
# IT168
@@ -116,6 +128,8 @@ jin10.com
# 上观新闻/解放日报
jfdaily.com
shobserver.com
# 新黄河/济南时报
jinantimes.com.cn
# 机器之心
jiqizhixin.com
# 金融界
@@ -124,6 +138,8 @@ jrjimg.cn
# 江苏省广播电视总台
jsbc.com
jstv.com
# 大江网/江西日报社
jxnews.com.cn
# 江西省出版传媒集团
jxpp.com
# 晋中广播电视台视听网
@@ -133,8 +149,14 @@ kankanews.com
kankannews.com
kksmg.com
statickksmg.com
# 新智派
knewsmart.com
# 晚点
latepost.com
# 雷科技
leikeji.com
# 雷峰网
leiphone.com
# 今日观察网
miercn.com
# 快科技
@@ -154,6 +176,8 @@ pchome.net
pchpic.net
# PConline 太平洋科技
3conline.com
pcauto.com.cn
pchouse.com.cn
pconline.com.cn
# 泡泡网
pcpop.com
@@ -183,6 +207,8 @@ sfccn.com
shangyexinzhi.com
# 奇客资讯网
solidot.org
# 速途网
sootoo.com
# 南方新闻网
southcn.com
# 南方财富网
@@ -202,22 +228,35 @@ szpgm.com
takungpao.com
# TechWeb
techweb.com.cn
# 科技行者
techwalker.com
# 封面新闻
thecover.cn
# 澎湃新闻
thepaper.cn
# 潮新闻/浙江日报
8531.cn
tidenews.com.cn
# 人人都是产品经理
woshipm.com
# 新华日报
xhby.net
# 西陆网
xilu.com
# 新快报
xkb.com.cn
# 金羊网/羊城晚报
ycwb.com
# 第一财经
yicai.com
# 扬子晚报
yzwb.net
# ZEALER
zealer.com
# 知产力
zhichanli.com
# 至顶网
zhiding.cn
# 中关村在线
zol-img.com.cn
zol.com.cn

View File

@@ -276,6 +276,8 @@ immersivetranslate.com # 沉浸式翻译 (国际版)
originlab.com
## OsmAnd
osmand.net
## Xiaomi Flash (not affiliated with the official Xiaomi)
xiaomiflash.com
# Software development
include:category-dev

View File

@@ -158,6 +158,7 @@ include:kechuang
include:nga
include:stage1st
178linux.com # Linux运维部落
52pojie.cn # 吾爱破解
anywlan.com # 无线论坛
chinadsl.net # 宽带技术网
@@ -184,6 +185,9 @@ wuyou.net # 无忧启动论坛
x-station.cn # X电站
znds.com # 智能电视网
# 51CTO
51cto.com
51ctocdn.cn
# DCC Forum心跳文学部中文论坛
edgemonix.top # 图床
monika.love
@@ -222,8 +226,12 @@ mingxuan.store
gov.cn
## 中国气象局
cma.cn
## 政务和公益机构域名注册管理中心
conac.cn
## 中央气象台
nmc.cn
## 中国互联网联合辟谣平台
piyao.org.cn
# Public transportation
## 中国国际航空
@@ -321,14 +329,14 @@ webterren.com
## 国学大师网
guoxuedashi.com
guoxuemi.com
## 志愿汇
zyh365.com
## 温州市图书馆
wzlib.cn
## 中国大百科全书
zgbk.com
## 浙江图书馆
zjlib.cn
## 志愿汇
zyh365.com
# Services & Softwares
include:category-ai-cn
@@ -384,6 +392,8 @@ alookweb.com
## 安兔兔
antutu.com
antutu.net
## 问答库
asklib.com
## 拓竹科技
bambulab.com
bblmw.com
@@ -460,6 +470,8 @@ tianqi.com
tianqistatic.com
## Via 浏览器
viayoo.com
## 问卷网
wenjuan.com
## X 浏览器
xbext.com
## 印象笔记
@@ -622,7 +634,6 @@ ycrx360.com
1637.com
16888.com
16rd.com
178linux.com
17989.com
17cdn.com
17house.com
@@ -672,14 +683,12 @@ ycrx360.com
51.com
510you.com
51credit.com
51cto.com
51eshop.com
51gaifang.com
51hejia.com
51jingying.com
51meishu.com
51sole.com
51wendang.com
51yes.com
5253.com
52hxw.com
@@ -748,7 +757,6 @@ apk8.com
aplaybox.com
appchina.com
arpun.com
asklib.com
babytree.com
babytreeimg.com
baidufree.com
@@ -851,7 +859,6 @@ csbew.com
csmjzs.com
csslcloud.net
ctiforum.com
ctoutiao.com
ctsbw.com
cubead.com
cwxzx.com
@@ -901,8 +908,6 @@ downyi.com
dpfile.com
dreams-travel.com
drivergenius.com
dui.ai
duiopen.com
duoduocdn.com
duomai.com
duoshuo.com
@@ -935,7 +940,6 @@ fanli.com
fangxiaoer.com
fanxian.com
fastapi.net
feihuo.com
feiniaomy.com
fengniao.com
fengxian110.com
@@ -1088,7 +1092,6 @@ k366.com
kaifubiao.com
kaixin001.com
kameng98.com
kanimg.com
kantianqi.net
kanzhun.com
kaopu001.com
@@ -1116,8 +1119,6 @@ lascn.net
lawxp.com
leangoo.com
leanote.com
leikeji.com
leiphone.com
leiting.com
leju.com
leturich.org
@@ -1392,17 +1393,14 @@ wanwushuo.com
wanyiwang.com
wanyx.com
warmheart.top # 苏ICP备16016970号-1
wdjimg.com
wdzj.com
weand.com
weavatar.com
weicaifu.com
weilaicaijing.com
weiphone.net
weixing.com
weiyangx.com
welltrend-edu.com
wenjuan.com
whlovehome.com
wicresoft.com
win007.com
@@ -1440,7 +1438,6 @@ xiaoe-tech.com
xiaogouh5.com
xiaoka.tv
xiaokaxiu.com
xiaomiflash.com
xiaomingtaiji.com
xiaoxiongxitong.com
xiaoyuxitong.com
@@ -1518,7 +1515,6 @@ yunzhijia.com
yupoo.com
yuzua.com
yxbao.com
yxdown.com
yytcdn.com
z28j.com # 鄂ICP备20002930号-1
zampda.net
@@ -1527,7 +1523,6 @@ zastatic.com
zczj.com
zdfans.com
zgjm.org
zgjsks.com
zglxw.com
zgzcw.com
zhanbuba.com
@@ -1625,3 +1620,6 @@ ao-x.ac.cn
# 万集科技 京ICP备18036282号-2
wanji.net.cn
# 广州市雅望互联网服务有限公司
gzyowin.com

View File

@@ -9,6 +9,8 @@ max.ru
memealerts.com
oneme.ru
tamtam.chat
telega.info
telega.me
youla.io
youla.ru

2
data/openrec Normal file
View File

@@ -0,0 +1,2 @@
full:dqd0jw5gvbchn.cloudfront.net
openrec.tv

View File

@@ -1,3 +1,10 @@
ozon.by
ozon.com
ozon.com.by
ozon.com.kz
ozon.kz
ozon.ru
ozon.tm
ozone.ru
ozonru.me
ozonusercontent.com

View File

@@ -258,13 +258,14 @@ tdnsv14.net
tdnsv15.net
# myqcloud inside mainland China
regexp:\.(.+-)?ap-beijing(-.+)?\.myqcloud\.com$ #北京
regexp:\.(.+-)?ap-nanjing(-.+)?\.myqcloud\.com$ #
regexp:\.(.+-)?ap-shanghai(-.+)?\.myqcloud\.com$ #上海
regexp:\.(.+-)?ap-guangzhou(-.+)?\.myqcloud\.com$ #广州
regexp:\.(.+-)?ap-chengdu(-.+)?\.myqcloud\.com$ #成都
regexp:\.(.+-)?ap-chongqing(-.+)?\.myqcloud\.com$ #重庆
regexp:\.(.+-)?ap-shenzhen(-.+)?\.myqcloud\.com$ #深圳
# overrided by myqcloud.com
#regexp:\.(.+-)?ap-beijing(-.+)?\.myqcloud\.com$ #
#regexp:\.(.+-)?ap-nanjing(-.+)?\.myqcloud\.com$ #南京
#regexp:\.(.+-)?ap-shanghai(-.+)?\.myqcloud\.com$ #上海
#regexp:\.(.+-)?ap-guangzhou(-.+)?\.myqcloud\.com$ #广州
#regexp:\.(.+-)?ap-chengdu(-.+)?\.myqcloud\.com$ #成都
#regexp:\.(.+-)?ap-chongqing(-.+)?\.myqcloud\.com$ #重庆
#regexp:\.(.+-)?ap-shenzhen(-.+)?\.myqcloud\.com$ #深圳
# COS 使用到的非中国大陆的地域与可用区,参见 https://cloud.tencent.com/document/product/436/6224
ap-hongkong.myqcloud.com @!cn #中国香港
@@ -282,13 +283,14 @@ eu-frankfurt.myqcloud.com @!cn #法兰克福
eu-moscow.myqcloud.com @!cn #莫斯科
# tencentcos inside mainland China
regexp:\.(.+-)?ap-beijing(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #北京
regexp:\.(.+-)?ap-nanjing(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #
regexp:\.(.+-)?ap-shanghai(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #上海
regexp:\.(.+-)?ap-guangzhou(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #广州
regexp:\.(.+-)?ap-chengdu(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #成都
regexp:\.(.+-)?ap-chongqing(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #重庆
regexp:\.(.+-)?ap-shenzhen(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #深圳
# overrided by tencentcos.cn, tencentcos.com, tencentcos.com.cn
#regexp:\.(.+-)?ap-beijing(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #
#regexp:\.(.+-)?ap-nanjing(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #南京
#regexp:\.(.+-)?ap-shanghai(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #上海
#regexp:\.(.+-)?ap-guangzhou(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #广州
#regexp:\.(.+-)?ap-chengdu(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #成都
#regexp:\.(.+-)?ap-chongqing(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #重庆
#regexp:\.(.+-)?ap-shenzhen(-.+)?\.tencentcos\.(cn|com(\.cn)?)$ #深圳
# tencentcos outside mainland China
# regexp:.+\.ap-hongkong\.tencentcos\.(cn|com(\.cn)?)$ @!cn #中国香港

5
data/tilda Normal file
View File

@@ -0,0 +1,5 @@
tilda.cc
tilda.ru
tilda.ws
tildaapi.com
tildacdn.com

View File

@@ -1,3 +1,4 @@
feng.com
fengimg.com
weiphone.net
wfdata.club

View File

@@ -1,3 +1,5 @@
8uri.cn
kanimg.com
sandai.net
thunderurl.com
xunlei.com

139
main.go
View File

@@ -2,6 +2,7 @@ package main
import (
"bufio"
"encoding/json"
"flag"
"fmt"
"os"
@@ -19,6 +20,7 @@ var (
dataPath = flag.String("datapath", "./data", "Path to your custom 'data' directory")
outputName = flag.String("outputname", "dlc.dat", "Name of the generated dat file")
outputDir = flag.String("outputdir", "./", "Directory to place all generated files")
datProfile = flag.String("datprofile", "", "Path of config file used to assemble custom dats")
exportLists = flag.String("exportlists", "", "Lists to be flattened and exported in plaintext format, separated by ',' comma")
)
@@ -47,6 +49,23 @@ type Processor struct {
cirIncMap map[string]bool
}
type GeoSites struct {
Sites []*router.GeoSite
SiteIdx map[string]int
}
type DatTask struct {
Name string `json:"name"`
Mode string `json:"mode"`
Lists []string `json:"lists"`
}
const (
ModeAll string = "all"
ModeAllowlist string = "allowlist"
ModeDenylist string = "denylist"
)
func makeProtoList(listName string, entries []*Entry) *router.GeoSite {
site := &router.GeoSite{
CountryCode: listName,
@@ -76,6 +95,90 @@ func makeProtoList(listName string, entries []*Entry) *router.GeoSite {
return site
}
func loadTasks(path string) ([]DatTask, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
var tasks []DatTask
dec := json.NewDecoder(f)
if err := dec.Decode(&tasks); err != nil {
return nil, fmt.Errorf("failed to decode json: %w", err)
}
for i, t := range tasks {
if t.Name == "" {
return nil, fmt.Errorf("task[%d]: name is required", i)
}
switch t.Mode {
case ModeAll, ModeAllowlist, ModeDenylist:
default:
return nil, fmt.Errorf("task[%d] %q: invalid mode %q", i, t.Name, t.Mode)
}
}
return tasks, nil
}
func (gs *GeoSites) assembleDat(task DatTask) error {
datFileName := strings.ToLower(filepath.Base(task.Name))
geoSiteList := new(router.GeoSiteList)
switch task.Mode {
case ModeAll:
geoSiteList.Entry = gs.Sites
case ModeAllowlist:
allowedIdxes := make([]int, 0, len(task.Lists))
for _, list := range task.Lists {
if idx, ok := gs.SiteIdx[strings.ToUpper(list)]; ok {
allowedIdxes = append(allowedIdxes, idx)
} else {
return fmt.Errorf("list %q not found for allowlist task", list)
}
}
slices.Sort(allowedIdxes)
allowedlen := len(allowedIdxes)
if allowedlen == 0 {
return fmt.Errorf("allowlist needs at least one valid list")
}
geoSiteList.Entry = make([]*router.GeoSite, allowedlen)
for i, idx := range allowedIdxes {
geoSiteList.Entry[i] = gs.Sites[idx]
}
case ModeDenylist:
deniedMap := make(map[int]bool, len(task.Lists))
for _, list := range task.Lists {
if idx, ok := gs.SiteIdx[strings.ToUpper(list)]; ok {
deniedMap[idx] = true
} else {
fmt.Printf("[Warn] list %q not found in denylist task %q", list, task.Name)
}
}
deniedlen := len(deniedMap)
if deniedlen == 0 {
fmt.Printf("[Warn] nothing to deny in task %q", task.Name)
geoSiteList.Entry = gs.Sites
} else {
geoSiteList.Entry = make([]*router.GeoSite, 0, len(gs.Sites)-deniedlen)
for i, site := range gs.Sites {
if !deniedMap[i] {
geoSiteList.Entry = append(geoSiteList.Entry, site)
}
}
}
}
protoBytes, err := proto.Marshal(geoSiteList)
if err != nil {
return fmt.Errorf("failed to marshal: %w", err)
}
if err := os.WriteFile(filepath.Join(*outputDir, datFileName), protoBytes, 0644); err != nil {
return fmt.Errorf("failed to write file %q: %w", datFileName, err)
}
fmt.Printf("dat %q has been generated successfully\n", datFileName)
return nil
}
func writePlainList(listname string, entries []*Entry) error {
file, err := os.Create(filepath.Join(*outputDir, strings.ToLower(listname)+".txt"))
if err != nil {
@@ -443,25 +546,39 @@ func run() error {
}
}
// Generate dat file
protoList := &router.GeoSiteList{Entry: make([]*router.GeoSite, 0, sitesCount)}
// Generate proto sites
gs := &GeoSites{
Sites: make([]*router.GeoSite, 0, sitesCount),
SiteIdx: make(map[string]int, sitesCount),
}
for siteName, siteEntries := range processor.finalMap {
protoList.Entry = append(protoList.Entry, makeProtoList(siteName, siteEntries))
gs.Sites = append(gs.Sites, makeProtoList(siteName, siteEntries))
}
processor = nil
// Sort protoList so the marshaled list is reproducible
slices.SortFunc(protoList.Entry, func(a, b *router.GeoSite) int {
// Sort proto sites so the generated file is reproducible
slices.SortFunc(gs.Sites, func(a, b *router.GeoSite) int {
return strings.Compare(a.CountryCode, b.CountryCode)
})
for i := range sitesCount {
gs.SiteIdx[gs.Sites[i].CountryCode] = i
}
protoBytes, err := proto.Marshal(protoList)
if err != nil {
return fmt.Errorf("failed to marshal: %w", err)
// Load tasks and generate dat files
var tasks []DatTask
if *datProfile == "" {
tasks = []DatTask{{Name: *outputName, Mode: ModeAll}}
} else {
var err error
tasks, err = loadTasks(*datProfile)
if err != nil {
return fmt.Errorf("failed to loadTasks %q: %v", *datProfile, err)
}
}
if err := os.WriteFile(filepath.Join(*outputDir, *outputName), protoBytes, 0644); err != nil {
return fmt.Errorf("failed to write output: %w", err)
for _, task := range tasks {
if err := gs.assembleDat(task); err != nil {
fmt.Printf("[Error] failed to assembleDat %q: %v", task.Name, err)
}
}
fmt.Printf("%q has been generated successfully\n", *outputName)
return nil
}