New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE] Crawl Sitemap.xml #848

Open

tomasnorre opened this issue Nov 5, 2021 · 0 comments

Labels

feature On Hold

Owner

tomasnorre commented Nov 5, 2021 •

edited

Loading

Is your feature request related to a problem? Please describe

@benjaminkott suggested me today, to add the feature for crawler the sitemap.xml file.

This would remove some complexity from the crawler that calculates which pages to crawler a which not.

This will especially be helpful for sites generated by plugins like news. Where not every news have its own site.

The Crawler can handle this today, but with a little complexer PageTS.

The sitemap.xml crawling as an option would make this easier.

tomasnorre added feature On Hold labels

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment