'Culturomics' Spreads From Google Books To Scientific Preprints

ananyo writes "Cultural Observatory at Harvard University in Cambridge, Massachusetts is to index the whole of the ArXiv pre-print database of papers from the physical sciences, breaking down the full text of the articles into component phrases to see how often a particular word or phrase appears relative to others — a measure of how 'meme-like' a term is. The team has already applied a similar approach to 5 million books in the Google Books database to produce their n-gram viewer. But the Google Books database carries with it a major limitation: because many of the works are under copyright, users cannot be pointed to the actual source material. Applying the tool to ArXiv means it could be used to chart trends in high-energy physics, for example: a quickening pulse of papers citing the Higgs boson, for example, or a peak in papers about supersymmetry, a theory which may soon be waning."