State-of-the-Art Mobile Search Part 4: Fields and Phrases

State-of-the-Art Mobile Search is a series exploring how to implement advanced mobile search.

State-of-the-Art Mobile Search Part 3: TF-IDF Models

Rod Smith

The inverted index and the ranked retrieval model from earlier in this series did not distinguish between different fields of the indexed documents, nor did they make any special accommodation for multi-word phrases.

Phrase queries

A user may need to find documents that contain multi-word phrases like “all your base.” Users understand phrase queries well enough that explicit phrase queries are one of the few effective types of advanced queries. Typically, users identify phrase queries by enclosing each phrase in quotes, but implicit phrase queries are also possible, where a search engine identifies phrases without quotes or any other indication beyond the mere proximity of the words. Read more of this post

State-of-the-Art Mobile Search Part 3: TF-IDF Models

State-of-the-Art Mobile Search is a series exploring how to implement advanced mobile search.

State-of-the-Art Mobile Search Part 2: Ranked Retrieval

Rod Smith

The ranked retrieval model explained in part 2 of this series established a framework for assessing the relevance of documents for a search query in terms of the information content of each query term and the focus of each document.

Term Frequency Models

If a search term t appears several times in one document d₁ but fewer times in another document d₂ of similar length, d₁ is arguably more relevant to t than d₂ is. However, if d₁ has ten occurrences of t, while d₂ has t just once, that does not make d₁ ten times more relevant than d₂. Read more of this post

State-of-the-Art Mobile Search Part 2: Ranked Retrieval

State-of-the-Art Mobile Search is a series exploring how to implement advanced mobile search.

State-of-the-Art Mobile Search Part 1: Offline Search | State-of-the-Art Mobile Search Part 3: TF-IDF Models

Rod Smith

The inverted index explained in part 1 of this series allows an offline mobile search engine to quickly retrieve documents that match the search terms. A naïve search engine might use such an index for a Boolean search, which may simply return the documents that contain either all or any of the terms in the search query. More advanced Boolean search queries may be written in a query language of operators and expressions that defines precisely which documents to retrieve and which to ignore. That works well for experts who are very familiar with the corpus, but not for casual users. When Boolean searches are too specific, relevant documents may be omitted for lacking one search term, resulting in too few results. On the other hand, Boolean searches are often not specific enough, returning far too many results. It is difficult for most people to write an effective Boolean query that returns enough results to be useful, yet not so many as to be overwhelming. Read more of this post

State-of-the-Art Mobile Search Part 1: Offline Search

State-of-the-Art Mobile Search is a series exploring how to implement advanced mobile search.

State-of-the Art Mobile Search: Introduction | State-of-the-Art Mobile Search Part 2: Ranked Retrieval

Rod Smith

As mobile device memory capacity expands, so do opportunities to feature extensive natural-language content in offline-capable apps. Network connections are ever faster, but even under ideal connectivity conditions, apps serve local content noticeably faster than server-based equivalents. Moving outside of ideal network connectivity conditions further highlights the advantages of network independence.

So for the best user experience, apps with much offline natural-language content should support offline search. The key to efficient offline search is the data structure known as the “inverted index.” Read more of this post