Step 5: QA

Once you’ve built your Search experience, you’ll want to QA it to ensure everything looks and works as expected before you integrate it onto your site.

1. Review the Types of QA training unit for an introduction to the QA you should be doing once your experience is built. This includes:
   • Search quality testing
   • Browser and device testing
   • Feature testing
   • UI testing
   • Analytics testing
2. Be sure to complete thorough internal testing prior to sending a staging link to any other stakeholders for QA. We will cover steps for each of the QA areas below.
3. Once you’ve built your experience and QA’d it yourself, it’s helpful to have someone else review it. They can use this Search Experience Review guide for guidance on what to look for.

Search Quality Testing

Ensure your top queries (such as “locations near me”) return expected results and make adjustments as needed.

Remember that Search requires a continuous cycle of improvement. The philosophy behind implementation QA is to account for the top queries. Once the experience is launched, you’ll want to monitor what users are searching for, review data and analytics, and further improve the quality of search results.

1. Select 10-20 common queries and test them within the staging environment. These can be top queries from the site’s current search or queries you consider most important to perform well in search. For each query, note whether the results returned meet, partially meet, or do not meet the expected results.
   • Be sure to distinguish whether improvements needed are for the display (attributable to the frontend) or for what results appear, i.e., the search quality (attributable to the backend). You can also use Test Search , which will only returns the results passed by the backend.
2. Debug queries to improve search quality - If you find less than ideal results, first debug them to determine why the expected results are not returned.
   1. Review the trainings related to debugging and optimizing
      • Search Optimizations - Understand the types of changes needed to improve search quality (including content changes, backend configuration changes, and algorithm changes. Frontend display changes are also an optimization you can make, but does not affect the search results returned.)
      • Debug the Backend - Understand the general approach to debugging backend search quality issues.
      • Experience Training - Understand how you can train the algorithm for NLP filters, spell check, and featured snippets, which can be used to debug search quality issues.
      • Overview of Algorithm and Indexing - Understand at a high level how the Search algorithm works. There may be cases where you think there’s a search quality issue when it is expected behavior.
      • Query Rules - If, after debugging, you find that the results returned are expected, but you still have a reason to tweak them, you can add a query rule to make that happen.
      • Search Merchandiser - Learn about the UI tool you can use to add particular query rules.
   2. Follow the How to Start Debugging Backend Search Issues help article for step-by-step guidance on how to approach most backend issues. If that doesn’t resolve your issue, use the linked help articles at the end to address more specific issues.
3. Make the changes in your configuration, run the query again, and note whether the actions taken improved the results. You may need to tinker with the configuration a few times to get the results expected.

Browser and Device Testing

Test different browsers and devices to ensure the experience performs consistently across them. Use an emulator like  Browserstack  to easily test different browsers and devices. For each browser or device, run searches and click around to use the different features and verticals of the Search experience. Below is a list of browsers and devices we recommend testing:

• Google Chrome (latest two versions)
• Firefox (latest two versions)
• Safari
• Microsoft Edge
• Android (Chrome and Firefox)
• iPhone (Safari, Chrome, and Firefox)
• Tablet (Safari, Chrome, and Firefox)

Feature Testing

We recommend you test this list of features, if applicable. Run searches to activate the feature and click around to use the feature to ensure each works as expected.

• Location map on both universal and vertical search
• Direct Answers , including Featured Snippets
• Vertical Tab & “View All” Navigation
• Facets and Filter Components
• Configured No Results Behavior
• Spellcheck Component
• Location Bias Component
• Pagination Component
• Query Rules
• Configurable Sorting
• Q&A Component
• Custom Backends
• iFrame ([production url]/iframe.js)
• Data Variability - Be sure to run queries that would return different sets of data, including null (blank fields) to ensure the search quality and display are still as expected.

User Interface Testing

While you are clicking through the Search experience testing all of the above, confirm that your experience looks consistent throughout, and that there are no visual defects. These could include formatting requests or issues such as any incorrect spacing, adjusted coloring, or trailing text.

Analytics Testing

Ensure Analytics events are firing correctly by following the instructions in the How to QA Analytics section.