Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new_task_filter_config.path_disallowed 配置问题 #77

Open
wujie6166 opened this issue Dec 8, 2023 · 0 comments
Open

new_task_filter_config.path_disallowed 配置问题 #77

wujie6166 opened this issue Dec 8, 2023 · 0 comments

Comments

@wujie6166
Copy link

path_disallowed 这个格式支持正则嘛?我该如何配置 排除掉如 /api/v1/test 这样的路径? 用正则配置好像也不生效

能否提供一个示例

new_task_filter_config:           # 检查某个链接是否应该被加入爬取队列
  hostname_allowed: []            # 允许访问的 Hostname,支持格式如 t.com、*.t.com、1.1.1.1、1.1.1.1/24、1.1-4.1.1-8
  hostname_disallowed: []         # 不允许访问的 Hostname,支持格式如 t.com、*.t.com、1.1.1.1、1.1.1.1/24、1.1-4.1.1-8
  port_allowed: []                # 允许访问的端口, 支持的格式如: 80、80-85
  port_disallowed: []             # 不允许访问的端口, 支持的格式如: 80、80-85
  path_allowed: []                # 允许访问的路径,支持的格式如: test、*test*
  path_disallowed: ['/login']
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant