A language model is often a likelihood distribution above words or phrase sequences. In practice, it offers the probability of a particular word sequence currently being “valid.” Validity With this context will not refer to grammatical validity. In its place, it implies that it resembles how people today compose, which is exactly what the langu