Step 1: Whitelist User Agent and IP Addresses

Before configuring your crawler, whitelist the Yext crawler user agent and IP addresses to ensure that the crawler can access your webpages.

User Agent

The Yext crawler uses the following user agent:

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/87.0.4280.88 YextBot/Java Safari/537.36

IP Addresses

The Yext crawler uses the following IP addresses:

  • 54.204.19.87
  • 50.19.160.200
  • 34.198.218.97
  • 54.221.171.225