Probability estimates in practice

Preliminary draft c 2008 Cambridge UP

11.3 The Binary Independence Model

227 α = 1 2 . This is a form of maximum a posteriori MAP estimation, where we MAXIMUM A POSTERIORI MAP choose the most likely point value for probabilities based on the prior and the observed evidence, following Equation 11.4 . We will further discuss methods of smoothing estimated counts to give probability models in Sec- tion 12.2.2 page 243 ; the simple method of adding 1 2 to each observed count will do for now.

11.3.3 Probability estimates in practice

Under the assumption that relevant documents are a very small percentage of the collection, it is plausible to approximate statistics for nonrelevant doc- uments by statistics from the whole collection. Under this assumption, u t the probability of term occurrence in nonrelevant documents for a query is df t N and log [ 1 − u t u t ] = log [ N − df t df t ] ≈ log Ndf t 11.22 In other words, we can provide a theoretical justification for the most fre- quently used form of idf weighting, which we saw in Section 6.2.1 . The approximation technique in Equation 11.22 cannot easily be extended to relevant documents. The quantity p t can be estimated in various ways: 1. We can use the frequency of term occurrence in known relevant docu- ments if we know some. This is the basis of probabilistic approaches to relevance feedback weighting in a feedback loop, discussed in the next subsection. 2. Croft and Harper 1979 proposed using a constant in their combination match model. For instance, we might assume that p t is constant over all terms x t in the query and that p t = 0.5. This means that each term has even odds of appearing in a relevant document, and so the p t and 1 − p t factors cancel out in the expression for RSV. Such an estimate is weak, but doesn’t disagree violently with our hopes for the search terms appearing in many but not all relevant documents. Combining this method with our earlier approximation for u t , the document ranking is determined simply by which query terms occur in documents scaled by their idf weighting. For short documents titles or abstracts in situations in which iterative searching is undesirable, using this weighting term alone can be quite satisfactory, although in many other circumstances we would like to do better. 3. Greiff 1998 argues that the constant estimate of p t in the Croft and Harper 1979 model is theoretically problematic and not observed empirically, and argues that a much better estimate is found by simply estimating p t from collection level statistics about the occurrence of t, as p t = df t N. Preliminary draft c 2008 Cambridge UP 228 11 Probabilistic information retrieval Iterative methods of estimation, which combine some of the above ideas, are discussed in the next subsection.

11.3.4 Probabilistic approaches to relevance feedback