Step 5: QA

Once you’ve built your Search experience, you’ll want to QA it to ensure everything looks and works as expected sharing it with other stakeholders and integrating it onto your site.

Review this training unit for a conceptual explanation of the types of QA you’ll do. Use the Search Experience Review guide for additional guidance on what to look for.

Search Quality Testing

Ensure your top queries (such as “locations near me”) return expected results and make adjustments as needed.

Remember that Search requires a continuous cycle of improvement. The philosophy behind implementation QA is to account for the top queries. Once the experience is launched, you’ll want to monitor what users are searching for, review data and analytics, and further improve the quality of search results.

1. Run Top Queries: Test 10-20 common queries within the staging environment. These can be top queries from the site’s current search or queries you consider most important to perform well. For each query, note whether the results returned meet, partially meet, or do not meet the expected results.
   • Distinguish whether improvements needed are for the display (the frontend) or for what results appear, i.e., the search quality (the backend).
   • You can also use Test Search , which will only return the results passed by the backend.
2. Debug: If results are not ideal, debug to determine why the expected results are not returned.
   1. Review the trainings related to debugging and optimizing:
      • Search Optimizations : Understand the types of changes needed to improve search quality (including content changes, backend configuration changes, and algorithm changes. Frontend display changes are also an optimization you can make, but does not affect the search results returned.)
      • Debug the Backend : Understand the general approach to debugging backend search quality issues.
      • Experience Training : Understand how you can train the algorithm for inferred filters, spell check, and featured snippets, which can be used to debug search quality issues.
      • Overview of Algorithm and Indexing : Understand at a high level how the Search algorithm works. There may be cases where you think there’s a search quality issue when it is expected behavior.
      • Query Rules : If, after debugging, you find that the results returned are expected, but you still have a reason to tweak them, you can add a query rule to make that happen.
      • Search Merchandiser : Learn about the UI tool you can use to add particular query rules.
   2. Follow the How to Start Debugging Backend Search Issues help article for step-by-step guidance on how to approach most backend issues. If that doesn’t resolve your issue, use the linked help articles at the end to address more specific issues.
3. Optimize: Make changes to your configuration, run the query again, and note whether the actions taken improved the results. You may need to tinker with the configuration a few times to get the results expected.

Browser and Device Testing

Test different browsers and devices to ensure the experience performs consistently across them. Use an emulator like  Browserstack  to easily test different browsers and devices.

For each browser or device, run searches and click around to use the different features and verticals of the Search experience. We recommend testing:

• Google Chrome (latest two versions)
• Firefox (latest two versions)
• Safari
• Microsoft Edge
• Android (Chrome and Firefox)
• iPhone (Safari, Chrome, and Firefox)
• Tablet (Safari, Chrome, and Firefox)

Feature Testing

We recommend you test this list of features, if applicable. Run searches to activate the feature and click around to use the feature to ensure each works as expected.

• Location map on both universal and vertical search
• Direct Answers , including Featured Snippets
• Vertical Tab & “View All” Navigation
• Facets and Filter Components
• Configured No Results Behavior
• Spellcheck Component
• Location Bias Component
• Pagination Component
• Query Rules
• Configurable Sorting
• Q&A Component
• Custom Backends
• iFrame ([production url]/iframe.js)
• Data Variability: Be sure to run queries that would return different sets of data, including null (blank fields) to ensure the search quality and display are still as expected.

User Interface Testing

While you are clicking through the Search experience testing all of the above, confirm the experience looks consistent and has no visual defects, including incorrect spacing, adjusted coloring, or trailing text.

Analytics Testing

Ensure Analytics events are firing correctly by following the instructions in the How to QA Analytics section.