Reviewing & Scoring Search Terms | Yext Hitchhikers Platform

What You’ll Learn

In this section, you will learn:

• How to see, review, and export search terms
• How to review search terms and add status
• How to score search term relevance for benchmarking
• How to add labels to Search Terms

Search Terms

The best way to analyze search trends and evaluate search quality is by looking at search terms instead of individual searches. (You can view and export individual searches by both external users in your live experience and internal users using Test Search in the Search Logs tab under Search > All Search Experiences > Analysis). The Search Terms screen, which you can find right under the Search Logs tab, normalizes query strings (for things like capitalization, leading and trailing whitespace, and punctuation) into a list of unique search terms run by your users, and by default sorts the search terms by popularity.

From this screen, you can:

• Filter search terms by date, results returned, and more (1)
• Scroll through the search terms and view key data points about each search term like Sessions, Searches with Click and CTR (Click Through Rate) (2)
• Export all or filtered search terms into a spreadsheet (3)
• Run the same search in Staging or Production (6)
• Manage or add labels to Search Terms for an individual Search Term (4) or in bulk (5)

We’ll go into more detail on each of these items below.

Filter Search Terms

You’ll see all filters currently applied in the filter bar at the top of the screen. To add a filter, you can either click on a filter pill, or click on the + icon to open the filter window.

Search Term Detail

The Search Terms screen gives you a high level overview of key data points for each search term. When you click into a search term, you can view additional details about that search term, including:

• The most recent search details, like language and location bias (for context)
• Most Recent Knowledge Graph Results for the Search Term
• Relevancy Score for Entities Returned
• The Search Term Activity
• Additional links, like “Open Search in Production”

Scoring Entity Relevancy for Benchmarking

While the Status is there for your workflow efficiency, Scoring Entity Relevancy is your most powerful tool for improving your search results in the long term.

Instead of evaluating whether a query is “good” or “bad”, Entity Relevancy allows you to be even more granular by evaluating whether a result is the “answer”, “relevant” or “not relevant” to the customer’s question (Search Term). This information can then be used in two incredibly important ways:

1. Evaluating the Impact of Search Configuration Changes: As you make changes to your search configuration, it’s important to have a training set of data to understand whether those changes are going to improve or degrade your search results. For example, imagine you make a change to your configuration that removes entities from a search term that you’ve previously marked as irrelevant – that’s an improvement! However if that same change removes entities that you previously marked as relevant, then you’ll want to consider whether you should push that change.

2. Overall Impact on Algorithm Changes: On a macro scale, your input on whether results are relevant will be the best way to help to inform Yext’s Product Managers, Engineers and Data Scientists about the algorithm, how it’s performing for your businesses’ needs and how it can be improved overall. By scoring the relevancy of Entities, you are able to tell us at Yext what you think of as good search for your business and how we can improve search for you.

How to Score Entity Relevance

For each search term, we’ll surface each of the entities that are returned today so that you can judge whether those results are the Answer, Relevant or Not Relevant to the Search Term.

Relevant means that the entity should be returned in the search results and is relevant to what the user is looking for.

Not Relevant means that ideally the entity should not be returned in the search results because it is not related to the search term.

Answer can only be used for one entity and indicates that you think that entity should be the Direct Answer for the user’s query.

You can click to set the relevancy, or you can use the bulk selection and buttons to do multiple at once. For example:

Search Term Labels

As consumers continue to search for things in new and unique ways, we’re looking for ways to categorize questions consumers are asking to understand and improve search at scale. Search Term Labels go hand in hand with Search Term Clusters , which automatically groups similar searches, because it gives Hitchhikers more control over how search terms are grouped.

Search Term Labels can be added individually or in bulk.

You can label Search Terms in bulk by selecting multiple Search Terms and clicking Manage Labels under More Actions. From there you can add labels, remove labels, or overwrite all existing labels with newly specified labels.

Once you add Search Term Labels, you can use them for the following purposes:

• Filtering Search Terms (as shown above)
• Dimensioning/Filtering in Analytics (check out the Search Analytics unit to learn more)
• Running targeted Version Comparisons (check out the Version Comparison unit in this module)