Connector System Limits| Hitchhikers Platform


Object Limits and Validations
Function invocation (for source and transform) Timeout = 30 seconds
Connector Display Name 50 Character maximum
Selectors 2,000 Character maximum
Column Headers 100 Character maximum
JSON Payload for Push to API Connector 500 rows per request
For best response time we recommend a limit of 100 rows per request.

Crawlers and Crawler Source Connector

Resource Limits and Validations
PDF File Size 50MB
PDF Body File Size 1MB of text from crawled PDF can be ingested by Connector as body field. If limit is exceeded, we will truncate the data to only be the first 1MB of text.
Maximum Depth 0 - 100 levels past the root URL the Crawler will spider to
Rate Limit 1 - 15,000 concurrent tasks
Number of URLs the Crawler will spider through 100,000 maximum. Once this limit is hit, the Crawler will stop.
Domain 768 Characters maximum
Crawler Name 255 Character maximum
URL Pattern 1-2,000 Characters
Page Load Wait Time The Crawler will wait 10s to load the page (and execute JS) before extracting the HTML