We’ve given users more control over Crawlers by making Max Depth configurable in the Crawler settings. Before, Crawler runs had an automatic depth of 10 levels. Now, users can specify the number of levels (up to 100) past the initial URL that a Crawler will navigate through during a run. So, users can configure the Crawler to more easily access only relevant content, as well as more content by increasing their Max Depth.
To leverage this feature, users must turn on the Summer ‘22 Release: Crawler Max Depth Parameter (early access) feature on within their account.
Once you do this you will see a new field in the Advanced Crawler Settings section where you can enter the Max Depth.
To learn more about the Crawler, visit our Yext Site Crawler & Crawler Connector training module.