O'Reilly Media has granted Get Elastic permission to reprint this excerpt of Peter Morville and Jeffery Callender's Search Patterns. The following is a checklist of things you discuss with vendors when evaluating a site search solution:
Formal description of the hardware and software components, including crawlers, indexers, data models, and query parsers.
How many simultaneous queries are supported? What's the maximum number of sources? How about the size of the data repository?
What types of conent and data (e.g., HTML, PDF, mySQL) are supported? Can the system handle both structured and unstructured data?
Is there a standards-based Web Services API for embedding search functionality in other sites and software? Is there a list of available connectors?
Does the system support multiple levels of access for different user types and individuals? How does it manage privacy and security?
How does the system handle full text and metadata? Does it support Boolean operators, wildcards, stemming, stop words, phrase and proximity searching, and spellcheck? What algorithms are used for ranking? What are the options for query refinement? Can results be saved, printed, and shared?
What sort of expertise is required for installation, configuration, and maintenance? How does the vendor handle training and support?
Is the product priced by data or activity volume, CPUs, features, and/or number of unique application? How about support, maintenance, and professional services fees? What's the total cost of ownership?
How long has the vendor been in business? How are they positioned in the market? Can we see their financials and customer references?
These are all necessary questions, but they're also insufficient. Because there's so much ground to cover, it's easy to lose sight of the goal. The designer's role is to repeatedly refocus attention on the user experience. A supplemental checklist that's informed by an information architecture strategy and empathy for the user might include:
What will it take to ensure subsecond response in the real world? It's worth asking this question early and often. Don't take "slow" for an answer!
How are results ranked? Is it possible to adjust the settings to allow for popularity, content type, date, and diversity?
Navigation and filtering
Is it easy to customize sort order and limit options? Is there native support for faceted navigation? Is it fast?
How does the system handle simultaneous search of multiple databases or indexes? What is the impact on speed? Is it possible to merge several indexes into one to dramatically improve performance?
Is there support for thesaurus integration and crosswalking between vocabularies? How about autocategorization and entity extraction?
What tools are provided for measuring and understanding user behavior? is there an API that supports sharing and repurposing of this data?
The authors of Search Patterns comment that, even in 2010, "search is the worst usability problem on the Web."
"Search is an elephant that hides in sight because executives lack the right radar. Many in management don't realize the role search plays in defining the user experience. They fixate on the home page, they fuss about look and feel, and they care about the content. They may even fume about findability, but they are easily distracted or misled because they really don't understand search."
Often, improving search functionality and usability should be a higher priority than tweaking home and product pages. But its mystique prevents ecommerce professionals from doing anything about it. Search Patterns is a helpful, easy read that will give you an understanding of how people use search, how search fits in to your overall design and information architecture, and what things you need to consider when fine tuning your search functionality.