Debugging the Backend - Search Detail, Test Search, and API Requests| Hitchhikers Platform

What You’ll Learn

In this section, you will learn:

  • Experience Training
  • Search Detail Overview
  • Search Metadata
  • Search Factors
  • Debugging with Test Search
  • How to Read an API Request

Overview of Debugging the Backend

When it comes to debugging the search results themselves, you have many tools at your disposal, including:

  • Experience Training
  • Search Detail Overview, with search metadata and search factors
  • Test Search
  • API Requests

We’ll review these in detail in this unit.

Experience Training

Experience Training is a critical tool for debugging backend issues and optimizing results for all search experiences, as you’ll learn about in the Post Launch Optimization module. The more feedback that you provide or corrections that you apply here, the better the algorithm can perform in the future.

You can do things like:

  • Review Featured Snippets: This screen will surface all featured snippets that the algorithm would return. You can accept, reject, or modify the snippets, and if you have snippets set to APPROVE_ONLY then only those you accept will display on search results.

experience training featured snippets

  • Review Spell Checking: Here you can review any suggested spell checks and accept or reject the corrections. If you reject a correction, the algorithm will no longer surface that correction when that search term is run for that experience.

experience training spell check

  • Review NLP Filters: Here you can review the NLP filters that were applied to any queries run and accept or reject. If you reject it, the algorithm will no longer apply the NLP filter for that search term for that experience.

experience training nlp

Search Details

With Search Detail, you are able to view any historical search run on Search along with a recreation of its results and the factors that led to those results being returned.

This will allow you to understand historically what results were being returned for searches as well as understand what factors caused the results to be returned. Search Detail provides you with a set of tools to not only understand what results were returned on the backend but also why results were returned.

the search log details screen for the search 'nyc office address'

Accessing the Search Details Screen

You can access the Search Detail screen by navigating to:

Search → All Search Experiences → View Experience → Search Logs → Search ID

This table shows the most recent searches first, but you can also use the search bar to filter for specific search terms.

search log screen highlighting the first search ID

Alternatively, you can go directly to a specific test query’s Search Detail screen by clicking “Debug in search log” at the top of your Test Search results (both sidebar and full screen). You’ll learn more about using Test Search to debug in the next section.

Full screen Test Search Debug in Search Log

Full screen Test Search Debug in Search Log

Once you navigate through to a specific Search ID (which represents a single user’s search) you will then be presented with the Search Detail screen, where you can find the search metadata, a recreation of the results returned to the user, and, most importantly, search factors.

We’ll dive into all of the various terms on this screen in the next sections.

Search Metadata

Search metadata is a combination of inputs from the user’s search like Location Bias and Language as well as outputs from the system such as Search ID and Session ID used to identify a unique search that help provide you with additional context for the given search.

search metadata on the search log detail screen

Search metadata will include the following:

Attribute Description
Search Term Search term string searched by user
Search ID ID of search
Date Time (in UTC) Time search was ran
Language Language / Locale of Search
Location Bias Location of User
Search Type Type of Search (Universal or Vertical)
Configuration Version Configuration Number and Label if applicable, (Production or Staging)


Search Factors

Search Factors help provide you with context as to why results were returned for a given search. They can be applied at the search level, vertical level, or entity level. Knowing why certain results are showing up, especially if you don’t want them to, can help you to determine why other results aren’t showing up or how to remove results you don’t want.

Let’s take a look at the example query “Yext NYC office address”. This query on Yext.com returned over 10 vertials, including the Yext NYC office entity, jobs, a handful of FAQs, and Help Center articles.

the search log details screen for the search 'nyc office address'

Search-Level Search Factors

Search-level search factors are applied to the whole search and are shown in a gray box in the top right of the search log details screen.

When trying to understand the why in the example query “NYC office address”, you can see that the following search factors were applied to the query:

search level search factors

  • Location Detection: The user’s internet address was detected with the search, which will trigger any searches based on the user’s location (e.g. search query “near me”) if location is a searchable field (e.g. there’s an NLP filter on builtin.location).

  • Non-Stop Word Tokens: As you learned about in the Tokens and Results Ranking unit, the Search algorithm breaks search queries down into tokens to match to searchable fields. It de-prioritizes stop words in the search. This section shows all the non-stop word tokens the query was trying to match on. These results were based on matches on the tokens “nyc”, “office”, and “address”.

  • Synonyms: Synonyms are applied across the entire search as an additional token to search for. In this search, since “location” is a synonym for “office”, the algorithm also looked for any “location” matches.

Other search-level search factors that would show on the search log details screen include:

  • Query rules applied - if query rules were applied on a search level, this will show which critera was triggered and which action was taken.
  • Query parameters - any parameters passed to the API, such as context, referrerPageUrl, and facetFilter.

Vertical-Level Search Factors

Vertical-level search factors are applied across a vertical and shown right below the vertical name in a search log details screen. Below are examples of vertical-level search factors that were applied to the search query “NYC office address”.

vertical level search factors

Filters Applied: Remember that NLP filters are applied to an entire vertical if there is a match because it applies a black and white filter. All results for a specific vertical will meet these filter requirements.

The offices vertical applied two filters:

  • Entity Type (builtin.entityType) - The entity type used in this vertical is “Location”. Due to the synonym office -> location applied at the search level, the search token “office” matched to “location”, triggering the NLP filter for entity type.
  • Location (builtin.location) - The algorithm detected that “NYC” is searching for a location, so the NLP filter location is applied.

Since these two filters were applied, the NYC office location entity was returned for this vertical.

The jobs vertical applied one filter:

  • Location (builtin.location) - The algorithm detected that “NYC” is searching for a location, so the NLP filter location is applied.

You may have wondered why jobs were showing up if the user was searching for the address of the NYC office and this explains why.

Sort Order: To help explain why entities within a vertical are showing up in a certain order, the search log details also show the sort order being applied by the algorithm.

You can see here that the offices vertical is sorted by relevance descending and then distance ascending, so that the most relevant and closest results are shown first. The jobs vertical is sorted by distance ascending and then ascending by the field name. This means the closest jobs are shown first and when there is a tie in distance, jobs are sorted alphabetically.

Other vertical-level search factors that would show on the search log details screen include:

  • Exact name match - whether the top result was an exact name match.
  • Boosted vertical - whether the vertical was boosted via a query rule.

Entity-Level Search Factors

Next to each entity shown in the results is a list of entity-level search factors that explain why the entity appears in these results.

Text Search Matches: If we scroll down the page, we see that the other entities and verticals, were all returned due to text search matches on various fields, such as name, description, and keywords. For text search matches, the field is shown, followed by the content that was matched on, with the matched snippet bolded.

entity level search factors

light bulb
Tip
Always look for bold highlighting in the text search matches to understand which entity field keywords matched in a given query.

If you are in the QA process in this example and are trying to understand why blogs were returned in the query, you can quickly identify that the name and description fields were searchable with text search turned on. Matches on “address” and “location” are returning these results.

Other entity-level search factors that would show on the search log details screen include:

  • Boosted/buried entity - whether the entity was boosted or buried via a query rule.
  • Reranking delta - If dynamic reranking was activated, the search log detail will show the difference between the entity’s original ranking and its revised ranking.

Within the Experiences UI of the Yext platform, an Administrator can also run various searches to get a quick preview of how results are returned to the backend.

Below is a sample workflow in which you can use the Test Search to debug and optimize your backend via the Search Configuration. Let’s say that we’ve determined users are searching quite frequently for “open positions” in the Turtlehead Tacos Search experience. We can use Test Search to understand what results are coming back for this query:

navigating to search configuration

The search only returns a somewhat relevant FAQ. We should likely be showing all open jobs instead. In order to optimize the query, let’s add a one way synonym from “positions” -> “jobs” and click save to create a new Search Configuration version.

navigating to search configuration

The version will automatically update our Staging environment assuming we are, as a best practice in Staging, pinned to the “LATEST” Search Configuration version. Therefore, we can jump right back to our Test Search and re-run the search:

navigating to search configuration

Now all jobs matching our search criteria return!

Let’s take it a step further. Looking at the above screenshot, we see the Waiter job returning in results (Entity ID: Job 8). This job should not be returning because it is a “Closed” position based on an open/closed custom field on the Job entity type. It should be filtered out using a Saved Filter. Currently, we do not have any saved filter applied to the Jobs vertical:

navigating to search configuration

If we apply our Saved Filter to this vertical (which filters out closed Jobs using custom fields) and click Save, we can re-run the search using the updated configuration:

navigating to search configuration

Success! The inactive job (Entity ID: Job 8) is no longer returning in the Jobs vertical.

We have debugged the “open positions” query using Test Search and optimized by 1) adding synonyms and 2) applying a Saved Filter.

Looking at API Requests

An Administrator can also quickly see which results were returned and which filters were applied to a given query by looking at the API Requests in-browser. When debugging, we often need to look at these API results, along with the Search Details screens, to understand what is being returned by the Search algorithm.

You can now view API logs directly from your Test Search results. Just toggle the “Show API Request” option in the top right of your Test Search results to view API Request and Response information. You can click “View More” to expand request details, as well as view and copy the raw API response (with line formatting options) in the next window. This will save you time when debugging and sharing searches!

Test Search API Requests

Alternatively, yout can right click on the results page and click inspect:

Inspecting Page

light bulb
Tip
When debugging, do so from vertical search, as you will get a better set of data to debug in vertical vs. in universal.

Once you click inspect, you can then navigate over to the “Network” tab where you will want to re-run the search. You should see a Live API request show up, which you can always identify via the “query?” naming convention.

Inspecting Page 2

When you run the search and double-click on the API request, a new tab will open and you’ll be presented with a JSON blob of all the vertical results. The result will look something like this:

Inspecting Page 3

light bulb
Tip
If you are seeing an unformatted JSON result, make sure to download the JSON Formatter Chrome Extension.

If you collapse the “results” object in the above example, you can get a really quick summary of how the results were returned:

Inspecting Page 4

What to Look for in the API Request

At a high level, you should always see results returned in the API request, since it’s built off the Live API. API results should update instantaneously after you make an edit in the Knowledge Graph. If you ever see an error when loading the API request (specifically a 500 error), then there could be a Yext Live API outage.

Additionally, you should always look out for any discrepancies between the back end data and the front end display. For example, if you are using a subtitle of c_providerTitle on your doctor card and it isn’t displaying right away - look at the API request! If you can’t find that value in the API request, there’s your problem! The frontend displays the information in the API request, so if the information isn’t there, the frontend can’t display it. If you are seeing that data is not appearing right away in the API results, then there could potentially be a lag.

Here are some specific insights that we can extract from the above example (note these are also identifiable via Search Factors in Search Detail):

  • resultsCount: 4:

    • We know that the query “doctors who treat acne” returned 4 doctors on the backend.
  • appliedQueryFilters (also known as NLP Filter):

    • We know that the doctors were filtered on a field called “Conditions Treated” with a value of “Acne”.
  • locationBias:

    • We know that location-centric results will be biased to my current device location, which is Yarmouth, Maine, United States of America.
light bulb
Tip
You can spoof your location in the API Request by adding &location=lat,long to the request URL.
unit Quiz
+20 points
Daily Quiz Streak Daily Quiz Streak: 0
Quiz Accuracy Streak Quiz Accuracy Streak: 0
    Error Success Question 1 of 6

    True or False: The Search Detail screen provides you with a historical, backend view of not only what results were returned but also why results were returned

    Error Success Question 2 of 6

    True or False: NLP Filter is an example of a search factor

    Error Success Question 3 of 6

    True or False: When inspecting the page and looking at API results, it is not possible to see what NLP Filters were applied.

    Error Success Question 4 of 6

    True or False: When looking at API results, you should always inspect the Universal results, not Vertical results.

    Error Success Question 5 of 6

    Which of the following is true regarding location searching and location biases?

    Error Success Question 6 of 6

    If you don't like a featured snippet that is returned or an NLP filter that is applied for a given query, can you do anything about that?

    A Hitchhiker in the making! 🔥

    You've already completed this quiz, so you can't earn more points.You completed this quiz in 1 attempt and earned 0 points! Feel free to review your answers and move on when you're ready.
1st attempt
0 incorrect