KWIC concordance¶

kwic(source, keyword, window=5, by_lemma=False, ignore_case=True)

A KWIC concordance (keyword in context) - every occurrence of a word or a phrase with its context on both sides. Occurrences are found among the words of the text by the word form, ignoring case or respecting it (ignore_case=False), or by the lemma (by_lemma=True); the keyword is lemmatized as a string, so with a lemmatizer a phrase may be written in any form and without one it is given by lemmas. The words of a string and of the keyword come from the tokenizer, those of a Doc from its tokens: a tokenizer that splits a string as the pipeline does finds the same in both. Punctuation and symbols are not words; forms are compared in the composed form of Unicode (NFC), without soft hyphens.

A phrase does not cross the end of a paragraph or a sentence unless the keyword has a final mark or a sentence start in the same place (U.S. Army). A sentence ends at a boundary of a Doc anywhere between the words, an opening mark included, or, without boundaries or with partial ones (has_sents), at a sentence start of the sentence splitter, by default a final mark before whitespace. The context is window words on each side as written, punctuation included; whitespace collapses into one space, and occurrences do not overlap.

The function calls the kwic of the anyTS core with the hooks of Russian: the words and the sentences of a string and of the keyword come from the anySeg segmenter, the lemmas from ruMorph in the context of the sentence (in a Doc, by the part of speech of the token), so «стали» is found by «сталь» or by «стать» depending on its neighbours; a word of the keyword has no context and matches by any lemma of its analyses (стали finds both). Stress marks and the period of an abbreviation do not count (руб. is found by руб), the word forms with ignore_case and the lemmas are compared in lower case with ё as е, and the parts of a hyphenated word that spaCy splits are joined (кто-то is one word).

CodeResult

from ruts.corpus import kwic, print_kwic

text = (
    "Кот сидел на окне и смотрел на птиц. Птицы улетели, и кот уснул на окне. "
    "Завтра кот снова будет сидеть на окне и смотреть на птиц."
)
lines = kwic(text, "окно", by_lemma=True, window=3)
lines[0]
print_kwic(lines, width=20)

[line.keyword for line in kwic(text, "смотреть на птица", by_lemma=True)]

Concordance(start=13, end=17, left='Кот сидел на', keyword='окне', right='и смотрел на')
        Кот сидел на  окне  и смотрел на
        кот уснул на  окне  . Завтра кот снова
     будет сидеть на  окне  и смотреть на
['смотрел на птиц', 'смотреть на птиц']

Parameters¶

Parameter	Type	Default	Description
`source`	`str` / `Doc`	-	Text or Doc object
`keyword`	`str`	-	Word or phrase
`window`	`int`	`5`	Number of words of context on each side
`by_lemma`	`bool`	`False`	Compare lemmas instead of word forms
`ignore_case`	`bool`	`True`	Ignore case when comparing word forms

A keyword without words, a window that is not an integer or is negative and a by_lemma or ignore_case that is not a bool raise ParameterError; a source that is neither a string nor a Doc, a keyword that is not a string and a hook that is not callable raise SourceTypeError.

Result¶

Concordance(start, end, left, keyword, right)

A list of Concordance named tuples in the order of the text.

Field	Type	Description
`start`	`int`	Position of the first character of the occurrence in the text
`end`	`int`	Position after the last character of the occurrence
`left`	`str`	Context on the left
`keyword`	`str`	Occurrence as written in the text
`right`	`str`	Context on the right

Aligned lines¶

format_kwic(concordances, width=40)

Aligns the lines on the keyword in a text, one line per occurrence: the left context is cut on the left and aligned to the right, the right one cut on the right. The lines may come from any iterable and are taken in the composed form of Unicode (NFC), so the widths count characters as shown.

Parameter	Type	Default	Description
`concordances`	`list[Concordance]`	-	Lines of a concordance
`width`	`int`	`40`	Width of a context in characters

Lines that are a string or cannot be iterated raise SourceTypeError, and a width that is not an integer of at least one ParameterError.

print_kwic(concordances, width=40)

Prints the result of format_kwic.