Home › Guides › Sitemap URL inventory for a docs blog

Sitemap URL inventory for a docs blog: part 1 of 2

A sitemap inventory shows every URL with its depth, change frequency and duplicate flag. This is the first half (alphabetical) of one crawlee.dev blog run.

The data (6 real rows)

url	pathGroup	pathDepth	changefreq	priority	isDuplicate
https://crawlee.dev/blog	/blog	1	weekly	0.5	False
https://crawlee.dev/blog/archive	/blog	2	weekly	0.5	False
https://crawlee.dev/blog/authors	/blog	2	weekly	0.5	False
https://crawlee.dev/blog/building-gradcracker-scraper-with-crawl...	/blog	2	weekly	0.5	False
https://crawlee.dev/blog/common-problems-in-web-scraping	/blog	2	weekly	0.5	False
https://crawlee.dev/blog/crawlee-blog-launch	/blog	2	weekly	0.5	False

Rows come from the free sample file for this actor, not a full run. Long values are shortened and personal contact fields are left out. Source: apify-actor-samples/sitemap-link-inventory.

How to get this yourself

Open the Every URL in a sitemap, with an optional broken-link check actor on Apify.
Give a site; get URLs from sitemaps with lastmod, depth and flags, and optionally status checks.
Run it, then export the dataset as CSV, JSON or Excel from the results tab.