Don’t discount search management

At Shopify, while our work improved search quality overall, one segment of users stood out: those that manually controlled results for specific queries with rules, boosts, synonyms.

All had more conversions than our generic ranking model

That’s search management - putting content experts in charge of the search experience.

On some teams, developers don’t directly improve search quality, they give tools to other people who manage search quality. Tools that let these search managers:

• Control: changing synonyms, boosts, add rules, etc
• Monitor: confirm changes have expected impacts, such as by comparing to a holdout
• Improve: instead of focusing ML on ranking itself, use ML to make suggestions for per-query improvements (or removing stale rules!)
• Retire: aggressively reconsider and remeasure old rules

If internal content expertise, not technology is your team’s strength, use it to your advantage.

Even if you’re a huge company, and mostly algorithmic, the dirty secret of Google and Amazon search is all the manual annotation, rules, etc they manage to get good search results, veto algorithms when they make bad decisions, and fine tune those really really really important queries.

-Doug

​

​

This is part of Doug’s Daily Search tips - subscribe here