Introduction to Streams| Hitchhikers Platform
Streams is the architecture which handles sending data from the Knowledge Graph to Yext’s consumer-facing systems, like Answers, Pages, and Listings. Streams can handle traversing relationships in the Knowledge Graph so that consumer systems can receive data from related records. Streams is also extremely efficient - it handles processing all updates to data and regenerates only the records for which those updates are relevant, so downstream systems can simply consume the records produced by Streams and ensure their data is always up-to-date.
Accessing Streams Data Externally
Yext wants to provide developers access to the same infrastructure we use to deliver data to our products. The Streams API and Streams Webhook serve precisely this purpose, allowing external applications to tap into Streams in both a pull and push fashion.
In order to fetch data from Streams externally, a user must define a Streams Endpoint. The user can then fetch data from this endpoint via the Streams API, or create a Streams Webhook based on this endpoint.
Streams Endpoints are configured via Configuration as Code; data is then pre-processed according to the supplied configuration. This architecture provides great flexibility, as the user can define any combination of fields that they want to pull out of the Knowledge Graph, including those on related entities.
Streams API is Yext’s low-latency, consumer-grade API intended to be used to fetch content from Streams Endpoints for publishing in consumer experiences. As mentioned above, the data is pre-processed according to the configuration of the Streams Endpoint, ensuring that the query is extremely fast since there is no requirement to traverse the graph in real-time.
While Streams API is used to pull data from Yext, Streams Webhooks allow applications to receive updates in a push data flow. Like the Streams API, Streams Webhooks are based on Stream Endpoints; a user defines a Stream Endpoint, then can configure a webhook which based on this Endpoint, in order to receive POST messages whenever there is an update to the data included in the selected Streams Endpoint.
Limitations of Streams API/Webhook
Currently, the Streams API permission grants access to all Streams Endpoints configured in an account. Scoping an API Permission to a single Streams Endpoint is not currently supported. Please ensure that when exposing an API Key with a Streams API Permission publicly, no Streams Endpoints in the account have access to sensitive data.
Streams API only supports exact-match filtering at this time. Much of the advanced filtering which is supported in the existing Entities APIs is not supported in Streams API, including:
- String matching (eg. string contains, string starts with, etc.)
- Numeric Comparisons (“>”,”<“,”>=“,”<=“)
- Streams API does not currently support sorting results on a supplied criteria.
Streams in the CLI
The Yext CLI has a set of commands specific to Streams, intended to provide insight to users into the processing and status of the Streams powering downstream experiences. For additional details, visit our CLI documentation.
For additional details on the schema, visit our Configuration as Code documentation.