Language model - Wikipedia, the free encyclopedia. A statistical language model is a probability distribution over sequences of words. Given such a sequence, say of length m, it assigns a probability P(w. Having a way to estimate the relative likelihood of different phrases is useful in many natural language processing applications. Language modeling is used in speech recognition, machine translation, part- of- speech tagging, parsing, handwriting recognition, information retrieval and other applications. In speech recognition, the computer tries to match sounds with word sequences. The language model provides context to distinguish between words and phrases that sound similar. Information Retrieval Implementing and Evaluating Search Engines Stefan B. Clarke University of Waterloo Gordon V. Cormack University of Waterloo The MIT Press Cambridge, Massachusetts London. Download EBOOK Information Retrieval: Implementing and Evaluating Search Engines PDF for free. Search Engines and Information Retrieval: pdf: ppt: 2. For example, in American English, the phrases . These ambiguities are easier to resolve when evidence from the language model is incorporated with the pronunciation model and the acoustic model. Language models are used in information retrieval in the query likelihood model. Here a separate language model is associated with each document in a collection. Documents are ranked based on the probability of the query Q in the document's language model P(Q. Commonly, the unigram language model is used for this purpose. Most possible word sequences will not be observed in training. Information Retrieval Implementing andEvaluating Search Engines StefanBiittcher CharlesL. Cormack TheMITPress Cambridge, Massachusetts London, England. Information retrieval is the foundation for modern search engines. This textbook offers an introduction to the core topics underlying modern search. One solution is to make the assumption that the probability of a word only depends on the previous n words. This is known as an n- gram model or unigram model when n = 1. Unigram models. For each automaton, we only have one way to hit its only state, assigned with one probability. Viewing from the whole model, the sum of all the one- state- hitting probabilities should be 1. Followed is an illustration of a unigram model of a document. Terms. Probability in doca. And we use probabilities from different documents to generate different hitting probabilities for a query. Then we can rank documents for a query according to the generating probabilities. Next is an example of two unigram models of two documents. Terms. Probability in Doc. Probability in Doc. In information retrieval contexts, unigram language models are often smoothed to avoid instances where P(term) = 0. A common approach is to generate a maximum- likelihood model for the entire collection and linearly interpolate the collection model with a maximum- likelihood model for each document to create a smoothed document model. Instead, some form of smoothing is necessary, assigning some of the total probability mass to unseen words or n- grams. Various methods are used, from simple . Neural network based language models are an example but there are other varieties such as log- bilinear models. Continuous space embeddings help to alleviate the curse of dimensionality in language modeling: as language models are trained on larger and larger texts, the number of unique words (the vocabulary) increases. Neural networks avoid this problem by representing words in a distributed way, as non- linear combinations of weights in a neural net. This is done using standard neural net training algorithms such as stochastic gradient descent with backpropagation. One then maximizes the log- probability. The representations in skip- gram models have the distinct characteristic that they model semantic relations between words as linear combinations, capturing a form of compositionality. For example, in some such models, if v is the function that maps a word w to its n- d vector representation, thenv(king). Similarly, bag- of- concepts models. Manning, Prabhakar Raghavan, Hinrich Sch. Cambridge University Press, 2. Buttcher, Clarke, and Cormack. Information Retrieval: Implementing and Evaluating Search Engines. MIT Press.^Craig Trim, What is Language Modeling?, April 2. Bengio, Yoshua (2. Distributed Representations of Words and Phrases and their Compositionality(PDF). Advances in Neural Information Processing Systems. You can thank Google later. Sentic Computing: Techniques, Tools, and Applications. Dordrecht, Netherlands: Springer, ISBN 9. Further reading. Built on Open. Information Retrieval Implementing and Evaluating Search Engines.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. Archives
December 2016
Categories |