loading

Answers Analytics in Report Builder| Hitchhikers Platform

What You’ll Learn

In this section, you will learn:

  • Answers Analytics metrics in Report Builder
  • Examples of Answers Analytics reports to build in Report Builder

Answers Analytics metrics in Report Builder

In the last unit, you learned how find Answers Analytics in the Answers Overview Screen. Another place you can find Answers Analytics metrics is in Report Builder.

To use Report Builder, navigate to the Analytics tab and select the Report Builder sub-tab. From this screen, you can select various Answers Analytics metrics under the “Metrics” drop-down.

report builder metrics

These metrics include:

  • Answers Clicks - number of times content has been clicked
  • Answers Click Through Rate - percentage of searches with a click
  • Answers Searches - number of times a user ran a search
  • Answers Searches with a Click - number of times a user ran a search and clicked at least once
  • Answers Searches with Knowledge Graph Results - number of times a user ran a search that returned Knowledge Graph results
  • Answers Sessions - number of times a session was started
  • Search Terms - distinct count of search terms that were searched by users
  • Knowledge Graph Result Rate - percentage of searches that returned Knowledge Graph results
  • Percentage of Total Searches - percentage of Answers Searches a given dimension(s) represents based on the dimension(s) and filter(s) selected e.g. Percentage of Searches by Search Term where Version Label = PRODUCTION
  • Case Starts - number of times a user started their journey with search
  • Case Submits - number of times a user started their journey with search and submitted a case
  • Case Deflections - number of times a user started their journey but did not create a case
  • Case Deflection Rate - percentage of cases that were deflected by Support Search

Once you’ve selected the metric(s) you’d like to include in your report, you can select the type of chart you want to see and update the remaining variables, such as Time Range, Dimensions, and any Filters or Sorting to be applied.

You can then save this report (by clicking “Save to Your Saved Reports”) and/or add this Report to a Dashboard (by clicking “Add to Dashboard”), along with other Answers reports you’ve built to create your own Answers dashboard! You can learn more about building reports in the Report Builder & Custom Insights module.

Common Examples for Answers in Report Builder

Next we will walk through some common examples of Answers reports you may build in Report Builder. Feel free to take these as a starting point and customize to your needs. Here are some examples of ways you can customize these reports:

  • Switch out the metrics in the examples below with other metrics from the list above
  • Change the chart type depending on your visual preference

We encourage you to explore on your own and find what is valuable to your brand!

Generally you should add the following filters in your Answers reports:

  • Configuration Version Label: Production - Exclude searches run in staging where you may run various searches to QA the experience.
  • Traffic Source: External - Exclude searches run by internal traffic, such as clients and Administrators working on their behalf. You’ll learn how to set “Internal Traffic” in the Understanding IP Filtering unit.

Once you have been live for a few weeks, you’ll want to analyze trends in your Answers experience over time. One way to do that is to create a week over week comparison of key metrics (you can also dimension time by days, but this shows weekly patterns rather than daily fluctuations). Note the start of the week is set to Mondays by default.

To set up this report:

  • Chart Type: Line Chart (for a visual representation) or Table Chart
  • Metrics: Answers Searches, Answers Sessions, Answers Clicks, Answers Click Through Rate, Conversion Value
  • Dimensions: Weeks
  • Filters: Configuration Version Label: Production, Traffic Source: External

    WoW report

Top Search Terms

One key insight to your Answers experience is knowing what users are searching for the most to understand if your content meets the needs of users.

To set up this report:

  • Chart Type: Table Chart
  • Metrics: Answers Searches, Answers Sessions, Answers Clicks, Answers Click Through Rate
  • Dimensions: Search Term (Normalized)
  • Filters: Configuration Version Label: Production, Traffic Source: External
  • Sort: Answers Sessions: Descending

    top search terms report

Clicks by Entity Type and Click Type

Once you know what users are searching for, you’ll want to take a look at what content on your site users are engaging with. We can demonstrate this by showing what entity types users are clicking on and which parts of the entity results cards users are clicking on specifically. You’ll learn more about click types in the next unit on Answers Analytics event types.

To set up this report:

  • Chart Type: Stacked Bar Chart (to compare across entity types) or 100% Stacked Bar Chart (to hone in one the breakdown within each entity type)
  • Metrics: Clicks
  • Dimensions: Vertical Config ID, Click Type
  • Filters: Configuration Version Label: Production, Traffic Source: External

    clicks by entity type report

Search Terms with No Knowledge Graph Results

When determining what improvements can be made to your Answers experience, you’ll want to review search terms without Knowledge Graph results. From here, you can decide whether and how to address these search queries. Review the search quality QA section here for types of optimizations you can make based on search term behavior.

To set up this report:

  • Chart Type: Table Chart
  • Metrics: Answers Searches, Answers Sessions, Answers Clicks, Answers Click Through Rate
  • Dimensions: Search Term (Normalized), Search Vertical (no results for a specific query on a specific vertical may be expected behavior)
  • Filters: Has Knowledge Graph Results: No, Configuration Version Label: Production, Traffic Source: External
  • Sort: Answers Sessions: Descending

Note that if a search term is included in this report, it does not necessarily mean every search with that search term returned no results during this timeframe. For example, perhaps you added content to address the search term partway through the timeframe. As an additional example, searches based on geolocation (e.g. “locations near me”) may return no results if there are no locations near the user.

no KG results report

Other Examples for Answers in Report Builder

As we continue to expand the set of dimensions and filters available to analyze Answers data, you’ll have more flexibility to customize reports to answer your brand’s questions about what content your users are searching for and engaging with:

  • Set a dimension if you’d like to see how metrics compare across different geographies, device types, or sources, for example.
  • Set the same items as filters if you’d like to dive deeper into certain demographics.

The following are examples of how you can apply different dimensions to your Answers reports. Again, we encourage you to take these as a starting point and explore on your own!

Feedback on Results by Search Term

With the ability to add thumbs up/thumbs down buttons on all entity cards, you need a place to consolidate that feedback and make it actionable. Filter down to thumbs down clicks to see which entities in search results users don’t want to see for particular search terms.

  • Chart Type: Table Chart
  • Metrics: Answers Searches
  • Dimensions: Search Term (Normalized), All Entities, Verticals Returned
  • Filters: Click Type: THUMBS_DOWN, Configuration Version Label: Production, Traffic Type: External
  • Sort: Answers Searches: Descending

    Report Builder Thumbs Down Clicks

Metrics by Referrer URL or Referrer Domain

Knowing what users are searching for and engaging with is helpful, but where are they coming from? What pages are users on when they decide to use site search? If you have placed the Answers search bar or links to the Answers experience across your website and on different domains, you can take a look at how users found the Answers experience.

  • Chart Type: Table Chart, Bar Chart, or Pie Chart, depending on your visual preference
  • Metrics: Answers Searches
  • Dimensions: Referrer URL or Referrer Domain for a summarized version
  • Filters: Configuration Version Label: Production, Traffic Source: External

    Report Builder Referrer URL


    Report Builder Referrer Domain

Metrics by Integration Source

If you have installed the overlay integration method, you might want to know which user interface is preferred. Are users searching or clicking more often through one integration method over the other? To do this, you can dimension Answers metrics on the integration source.

  • Chart Type: Table Chart, Bar Chart, or Pie Chart, depending on your visual preference
  • Metrics: Answers Sessions
  • Dimensions: Integration Source
  • Filters: Configuration Version Label: Production, Traffic Source: External

    Report Builder Integration Source

Are users looking for and engaging with different content across different countries, regions, or cities? Perhaps you’d like to dig further into what users of a particular geography are searching for and clicking on, particularly if you have regional content or notice higher engagement in a particular area. The chart below dimensions by User Country. You can also filter any of the reports in the Common Examples section above for a geographical filter.

  • Chart Type: Table Chart, Bar Chart, or Pie Chart, depending on your visual preference
  • Metrics: Answers Searches
  • Dimensions: User City, User Country, User Locale, User Lat Long
  • Filters: Configuration Version Label: Production, Traffic Source: External

    Report Builder Integration Source

We also have specific deflection metrics including Case Starts, Case Submits, Case Deflections, and Case Deflection Rate. These allow you to quantify the impact Support Search has on your business and customers. One key way to optimize your Answers experience to ensure both your customers and agents get the answers they’re looking for (which will decrease the number of support tickets and increase support agent efficiency) is to look into which search terms led to a case submission or case deflection.

Search Terms That Led to Case Submission

If a user started their journey with search and ended up submitting a case, they weren’t able to find what they were looking for. This is a great opportunity to review these search terms for sufficient answers to decrease the number of cases you receive.

  • Chart Type: Table Chart
  • Metrics: Case Submits
  • Dimensions: Search Term (Normalized)
  • Filters: Configuration Version Label: Production, Traffic Source: External
  • Sort: Case Submits: Descending

    Case Submission Search Terms

Deflected Search Terms

On the other hand, if a user started their journey and did not submit a case, they found what they were looking for and saved your support agents a case. This is a great way to show the value Answers provides by decreasing the number of support tickets you receive! You can also analyze the Case Deflection Rate to determine how often a certain search term meets a user’s needs.

  • Chart Type: Table Chart
  • Metrics: Case Deflections, Case Deflection Rate
  • Dimensions: Search Term (Normalized)
  • Filters: Configuration Version Label: Production, Traffic Source: External
  • Sort: Case Submits: Descending

    Deflected Search Terms
unit Quiz
+20 points
Daily Quiz Streak Daily Quiz Streak: 0
Quiz Accuracy Streak Quiz Accuracy Streak: 0
    Error Success Question 1 of 3

    What are some examples of Answers metrics found in Report Builder? Select all that apply.

    Error Success Question 2 of 3

    What Chart Type would you want to use when dimensioning by search term?

    Error Success Question 3 of 3

    What filters will you typically want to turn on for Answers Analytics reports? (Select all that apply)

    You're a star! ⭐️

    You've already completed this quiz, so you can't earn more points.You completed this quiz in 1 attempt and earned 0 points! Feel free to review your answers and move on when you're ready.
1st attempt
0 incorrect
Splash Loading