ML p(r)ior | Zipf's law is a consequence of coherent language production

Zipf's law is a consequence of coherent language production

2016-01-29
The task of text segmentation may be undertaken at many levels in text analysis---paragraphs, sentences, words, or even letters. Here, we focus on a relatively fine scale of segmentation, hypothesizing it to be in accord with a stochastic model of language generation, as the smallest scale where independent units of meaning are produced. Our goals in this letter include the development of methods for the segmentation of these minimal independent units, which produce feature-representations of texts that align with the independence assumption of the bag-of-terms model, commonly used for prediction and classification in computational text analysis. We also propose the measurement of texts' association (with respect to realized segmentations) to the model of language generation. We find (1) that our segmentations of phrases exhibit much better associations to the generation model than words and (2), that texts which are well fit are generally topically homogeneous. Because our generative model produces Zipf's law, our study further suggests that Zipf's law may be a consequence of homogeneity in language production.
PDF

Highlights - Most important sentences from the article

Login to like/save this paper, take notes and configure your recommendations

Related Articles

2016-05-17

State-of-the-art methods for zero-shot visual recognition formulate learning as a joint embedding pr… show more
PDF

Highlights - Most important sentences from the article

2017-08-09

Deep learning methods employ multiple processing layers to learn hierarchical representations of dat… show more
PDF

Highlights - Most important sentences from the article

2018-09-15
1809.05679 | cs.CL

Text classification is an important and classical problem in natural language processing. There have… show more
PDF

Highlights - Most important sentences from the article

2017-04-23

Keyphrase provides highly-summative information that can be effectively used for understanding, orga… show more
PDF

Highlights - Most important sentences from the article

2018-02-05

Existing text generation methods tend to produce repeated and "boring" expressions. To tackle this p… show more
PDF

Highlights - Most important sentences from the article

2019-04-17

In recent years, there has been an exponential growth in the number of complex documents and texts t… show more
PDF

Highlights - Most important sentences from the article

2015-04-27
1504.07295 | cs.CL

There have been many recent advances in the structure and measurement of distributed language models… show more
PDF

Highlights - Most important sentences from the article

2019-04-12

During the last fifteen years, text scaling approaches have become a central element for the text-as… show more
PDF

Highlights - Most important sentences from the article

2019-04-12

Social media offer an abundant source of valuable raw data, however informal writing can quickly bec… show more
PDF

Highlights - Most important sentences from the article

2017-02-07

We present a visually grounded model of speech perception which projects spoken utterances and image… show more
PDF

Highlights - Most important sentences from the article

2017-10-20

For words, rank-frequency distributions have long been heralded for adherence to a potentially-unive… show more
PDF

Highlights - Most important sentences from the article

2018-02-27

Flood of information is produced in a daily basis through the global Internet usage arising from the… show more
PDF

Highlights - Most important sentences from the article

2017-04-05

We explore the properties of byte-level recurrent language models. When given sufficient amounts of … show more
PDF

Highlights - Most important sentences from the article

2012-07-08
1207.1847 | cs.CL

The statistical methods derived and described in this thesis provide new ways to elucidate the struc… show more
PDF

Highlights - Most important sentences from the article

2019-01-08
1901.02543 | cs.CL

The study of register in computational language research has historically been divided into register… show more
PDF

Highlights - Most important sentences from the article

2018-10-06
1810.03031 | cs.CL

Sentiment polarity of tweets, blog posts or product reviews has become highly attractive and is util… show more
PDF