Step 1: Whitelist User Agent and IP Addresses
Before configuring your crawler, whitelist the Yext crawler user agent and IP addresses to ensure that the crawler can access your webpages.
User Agent
The Yext crawler uses the following user agent:
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/87.0.4280.88 YextBot/Java Safari/537.36
IP Addresses
The Yext crawler uses the following IP addresses:
- 54.204.19.87
- 50.19.160.200
- 34.198.218.97
- 54.221.171.225
<% elem.innerText %>