KWIC concordance¶

kwic(source, keyword, window=5, by_lemma=False, ignore_case=True)

A KWIC concordance (keyword in context) - every occurrence of a word or a phrase with its context on both sides. Occurrences are found among the words of the text by the word form, ignoring case or respecting it (ignore_case=False), or by the lemma (by_lemma=True); the keyword is lemmatized as a string, so with a lemmatizer a phrase may be written in any form and without one it is given by lemmas. The words of a string and of the keyword come from the tokenizer, those of a Doc from its tokens: a tokenizer that splits a string as the pipeline does finds the same in both. Punctuation and symbols are not words; forms are compared in the composed form of Unicode (NFC), without soft hyphens.

A phrase does not cross the end of a paragraph or a sentence unless the keyword has a final mark or a sentence start in the same place (U.S. Army). A sentence ends at a boundary of a Doc anywhere between the words, an opening mark included, or, without boundaries or with partial ones (has_sents), at a sentence start of the sentence splitter, by default a final mark before whitespace. The context is window words on each side as written, punctuation included; whitespace collapses into one space, and occurrences do not overlap.

The function calls the kwic of the anyTS core with the hooks of Spanish. The text and the keyword are split into words the same way, by the tokenizer of the blank Spanish pipeline, so EE. UU. is one word in both; the abbreviations the tokenizer keeps whole (Sr., pág., etc., EE. UU.) are one word in lower case, capitalized and in upper case alike, so sr. pérez finds Sr. Pérez and SR. PÉREZ, and punctuation and symbols are words of neither: ¿Dónde finds Dónde, 20 € finds 20. The word forms and the lemmas are lower-cased; accents are part of the word form: solo and sólo are two forms. The sentences of a string, of the keyword and of the text of a Doc without boundaries are those of SentsExtractor: an ellipsis or an exclamation followed by a lower-case word does not end a sentence, so honrado piensa finds honrado... piensa.

By lemma, the keyword is lemmatized by simplemma, so a phrase is given by lemmas - mirar a el pájaro - or as it is written, and a word of the text is found by its lemma of simplemma and, in a Doc that carries lemmas, by the lemma of the model as well: gatos is found by gato. In Mi amigo vino con una botella de vino the model gives venir to the verb, so venir finds the verb, where simplemma, which sees no context, would find nothing; vino finds both, since simplemma reads the verb as vino.

CodeResult

from ests.corpus import kwic, print_kwic

text = (
    "El gato estaba en la ventana y miraba a los pájaros. Los pájaros se fueron y el gato "
    "se durmió en la ventana. Mañana el gato volverá a estar en la ventana y mirará a los pájaros."
)
lines = kwic(text, "ventana", by_lemma=True, window=3)
lines[0]
print_kwic(lines, width=20)

[line.keyword for line in kwic(text, "mirar a los pájaros", by_lemma=True)]

Concordance(start=21, end=28, left='estaba en la', keyword='ventana', right='y miraba a')
        estaba en la  ventana  y miraba a
        durmió en la  ventana  . Mañana el gato
         estar en la  ventana  y mirará a
['miraba a los pájaros', 'mirará a los pájaros']

Parameters¶

Parameter	Type	Default	Description
`source`	`str` / `Doc`	-	Text or Doc object
`keyword`	`str`	-	Word or phrase
`window`	`int`	`5`	Number of words of context on each side
`by_lemma`	`bool`	`False`	Compare lemmas instead of word forms
`ignore_case`	`bool`	`True`	Ignore case when comparing word forms

A keyword without words, a window that is not an integer or is negative and a by_lemma or ignore_case that is not a bool raise ParameterError; a source that is neither a string nor a Doc, a keyword that is not a string and a hook that is not callable raise SourceTypeError.

Result¶

Concordance(start, end, left, keyword, right)

A list of Concordance named tuples in the order of the text.

Field	Type	Description
`start`	`int`	Position of the first character of the occurrence in the text
`end`	`int`	Position after the last character of the occurrence
`left`	`str`	Context on the left
`keyword`	`str`	Occurrence as written in the text
`right`	`str`	Context on the right

Aligned lines¶

format_kwic(concordances, width=40)

Aligns the lines on the keyword in a text, one line per occurrence: the left context is cut on the left and aligned to the right, the right one cut on the right. The lines may come from any iterable and are taken in the composed form of Unicode (NFC), so the widths count characters as shown.

Parameter	Type	Default	Description
`concordances`	`list[Concordance]`	-	Lines of a concordance
`width`	`int`	`40`	Width of a context in characters

Lines that are a string or cannot be iterated raise SourceTypeError, and a width that is not an integer of at least one ParameterError.

print_kwic(concordances, width=40)

Prints the result of format_kwic.