Search Terms and Clusters

What You’ll Learn

By the end of this unit, you will be able to:

Define search terms and search term clusters
Examine search terms and clusters to inform decisions on configuring search to improve search results

Overview

While you may look at search logs for individual searches to debug and refine results, you’ll want to zoom out to monitor the performance of your experience.

The best way to analyze search trends and evaluate search quality is by looking at search terms instead of individual searches. Search terms normalize queries and give you a more holistic view of the number of times a specific query has been searched and interacted with.

We’ll then take this one step further and review search term clusters, which automatically group like search terms together.

Search Terms

The Search Terms screen, which you can find under the Analysis section, normalizes query strings (for things like capitalization, leading and trailing whitespace, and punctuation) into a list of unique search terms run by your users, and by default sorts the search terms by popularity.

Note: “Empty Search Term” means a search query was run with no search term. Refer to “Empty Searches” in the What is a Search reference doc for more detail on what counts as an empty search.

The search terms table gives you a quick overview of what users are searching for. You can also see how many searches, sessions, and clicks each search term gets. If a search term is highly searched, but has a low click-through rate (CTR), you’ll want to consider whether there is something you can do to improve the CTR or if the low CTR is expected (for example, if there is a direct answer and so users wouldn’t have to click to get more info).

From this screen, you can:

Filter search terms by date, results returned, and more (1)
Scroll through the search terms and view key data points about each search term like Searches, Sessions, Searches with Click and CTR (Click Through Rate) (2)
Export all or filtered search terms into a spreadsheet (3)
Manage or add labels to Search Terms for selected Search Terms (4) or upload labels in bulk (5)
Run the same search in staging or production (6)

We’ll go into more detail on some of these items below.

Filter Search Terms

You’ll see all filters currently applied in the filter bar at the top of the screen. To add a filter, you can either click on a filter pill, or click on the + Add Filter button to expand the filter window.

Search Term Labels

As consumers continue to search for things in new and unique ways, we’re looking for ways to categorize questions consumers are asking to understand and improve search at scale. Search term labels go hand in hand with search term clusters, which automatically groups similar searches, because it gives you control over how search terms are grouped.

Search term labels can be added individually or in bulk. Select one or many search terms. Then click Manage Labels under More Actions. From there you can add labels, remove labels, or overwrite all existing labels with newly specified labels.

Once you add search term labels, you can use them for the following purposes:

Filtering search terms (as shown above)
Dimensioning/filtering in Analytics (check out the Search Analytics ) unit to learn more.

Search Term Detail

The Search Terms screen gives you a high level overview of key data points for each search term. When you click into a search term, you can view additional details about that search term, including:

The most recent search details, like language and location bias (for context)
Most recent Content results for the search term
Relevancy score for entities returned
The search term activity, including the number of searches, sessions, and clicks for the search term
Additional links, like “Open Search in Production”

Search Term Clusters

Advanced Search Tier Feature

Search term clustering is only available on the Advanced Search tier. Check out the Search Tiers reference doc to learn more about the features that are available in each tier.

Search term clusters automatically group together search terms with similar meaning to make analyzing and understanding the questions consumers are asking on a Search experience easier. Imagine your search has the following search terms in a certain period:

how do i check my gift card balance (30 sessions)
gift card balance (20 sessions)
card balance (15 sessions)
check balance (15 sessions)
account balance (10 sessions)
find balance (5 sessions)
egift card balance (5 sessions)

Individually, these search terms may not meet your threshold for review, but add them all up together, we have 100 sessions curious about checking their gift card balance! Clearly it’s something users want to learn about, and it’s important that we have a good answer no matter how users are searching for this information.

How Clustering Works

To develop clusters, we roughly take these steps:

Understand the intent of the search term using BERT - The first step in cluster analysis is deriving the intent from each search term. Using BERT, we can leverage natural language understanding to determine the actual meaning of each token (word) in a search term in context. Each search term is analyzed to derive vector representations of that term, which is just a fancy way of saying these BERT embeddings can create a numeric or mathematical basis for comparison. You can learn more about BERT here .
Determine the “closeness” of groups of search terms - Once we determine the vector representations of all search terms for an experience, we then use common data science clustering algorithms to calculate which terms are close enough (in vector space) to constitute a “cluster”. We also determine clusters that might have a lot of excess noise, or might have too many individual search terms to be meaningful, and mark those as Unclustered.
Label the cluster - Once a cluster has been identified, we’ll then provide it a name based on the most popular search term in that cluster (determined by # of sessions).

Clustering runs on a weekly basis, taking into account the previous 120 days of search data. The number of search terms found next to the Cluster filter will reflect the period analyzed as of the last weekly run.

Clustering will capture different amounts of search terms depending on the experience and it will not capture all search terms.

When viewing or filtering by search terms, remember these caveats - the clusters are just another directional tool to isolate potential search optimizations you can make for your experience.

Navigating Search Term Clusters

Search Term Clusters can be found on the Search Term Clusters screen under the Analysis section. Here you’ll see four colored heroes that group search term clusters by performance groups, followed by a table of search term clusters.

Search term clusters can be used in the following places:

Filtering search terms (as shown above)
Exporting clusters - When you export search terms from the Search Terms screen, you’ll see a column identifying the cluster the search term belongs to. If there is no cluster assigned, it’ll appear as “Unclustered”.
Dimensioning/filtering in Analytics (check out the Search Analytics unit to learn more).

Cluster Performance Groups

Performance groups make it easier for you to optimize your search experiences at scale. Search term clusters will automatically be categorized into four groups based on quality and size:

Needs Attention - Large Cluster
Needs Attention - Small Cluster
Performing Well - Small Cluster
Performing Well - Large Cluster

Here’s how we define size and quality:

Size: A cluster is considered Large if it represents more than 1% of total searches.
Quality: A cluster is considered Performing Well if it has a click-through rate greater than or equal to 20%.

This makes it easy to see what your users are searching for and understand how those clusters are performing so you can focus your attention on the clusters you need to improve. Click on a hero number to filter down to that category. Filter to the Needs Attention categories to quickly determine your priorities for optimization.

Search Term Clusters Table

The Search Term Clusters table groups search terms together that each represent different ways users can ask the same fundamental questions. Expanding a cluster shows the relevant search terms that were grouped together.

The metrics in the table — number of searches, sessions, and clicks, click-through rate, result rate, percentage of total searches, and number of search terms — show how important this question is to users and how the clusters are performing in search. This table is automatically sorted by the number of searches descending to make the most important clusters front and center by default. Sort ascending or descending by any of these metrics by clicking on the column header.

Search Terms and Clusters | Yext Hitchhikers Platform