Corpus plots

ests.visualizers.dispersion_plot(), ests.visualizers.keyness_plot(), ests.visualizers.collocation_network()

Description

Plots for the corpus measures: the lexical dispersion - where in a text a word occurs, a chart of the keywords found by keyness and a network of the collocations found by collocations. The matplotlib functions take the axes ax and return Axes: without ax a new figure is created, with it the plot goes into a grid of one's own; the network of collocations is built by graphviz and returns a Graph, as the word tree.

Lexical dispersion

A row for every word of targets and a tick at the position of each of its occurrences in the text, as dispersion_plot of NLTK and textplot_xray of quanteda. Words are compared as they are: case and lemmatization belong to WordsExtractor, and lemmas are looked for with use_lexemes=True.

Parameter	Type	Default	Description
words	list[str]	-	Words of the text in order
targets	list[str]	-	Words whose occurrences are shown
ax	Axes	None	Axes for the plot

Chart of keywords

Diverging horizontal bars (textplot_keyness of quanteda): the words of positive to the right, of negative - the result of keyness with positive=False - to the left, the length of a bar is the absolute value of the field (score, g2, log_ratio), so the side is set by the list and not by the sign of the measure; top_n words on each side, the words with an undefined or infinite value are skipped. For the odds ratio (score from 0 to infinity, one - equal odds) set log=True: the absolute \(\log_2\) of the value is plotted, symmetric around one.

Parameter	Type	Default	Description
positive	list[Keyword]	-	Positive keywords
negative	list[Keyword]	()	Negative keywords
top_n	int	20	Number of words on each side
labels	tuple[str, str]	("target corpus", "reference corpus")	Labels of the legend
field	str	score	Field of Keyword whose values are plotted
log	bool	False	Plot \(\log_2\) of the value - for the odds ratio
ax	Axes	None	Axes for the plot

Network of collocations

An undirected graph (textplot_network of quanteda): the nodes are the words with the size of the font by frequency, the edges the pairs with the width and the label by the value of the measure; the neato layout. Rendering needs the executables of Graphviz; in Jupyter the graph displays itself, and graph.render("network", format="png") saves a file.

Parameter	Type	Default	Description
collocations	list[Collocation]	-	Collocations
top_n	int	None	Number of pairs from the start of the list; None - all of them

Usage example

Marianela by Galdós and six novels from Project Gutenberg: Marianela, Misericordia and Torquemada en la hoguera against Niebla, Abel Sánchez and La tía Tula by Unamuno.

Example

Code:

from urllib.request import urlopen

from spacy.lang.es.stop_words import STOP_WORDS

from ests import WordsExtractor
from ests.corpus import collocations, keyness
from ests.visualizers import collocation_network, dispersion_plot, keyness_plot


def gutenberg(number):
    url = f"https://www.gutenberg.org/cache/epub/{number}/pg{number}.txt"
    text = urlopen(url).read().decode("utf-8")
    start = text.index("\n", text.index("*** START OF"))
    return text[start : text.index("*** END OF")]


# Where the characters and the motifs of Marianela occur
marianela = gutenberg(17340)
words = WordsExtractor(lowercase=True).extract(marianela)
dispersion_plot(words, ["nela", "pablo", "florentina", "golfín", "ciego", "luz"])

# Keywords of Galdós against Unamuno, lemmas without stop words
we = WordsExtractor(use_lexemes=True, lowercase=True, filter_nums=True, stopwords=STOP_WORDS)
galdos = [lemma for number in (17340, 21831, 15206) for lemma in we.extract(gutenberg(number))]
unamuno = [lemma for number in (49836, 44512, 44358) for lemma in we.extract(gutenberg(number))]
keyness_plot(
    keyness(galdos, unamuno, min_freq=5, top_n=10),
    keyness(galdos, unamuno, positive=False, min_freq=5, top_n=10),
    labels=("Galdós", "Unamuno"),
)

# Network of collocations of Marianela
graph = collocation_network(collocations(we.extract(marianela), window=3, min_freq=5, top_n=25))
graph.render("network", format="png")

Result:

Nela runs through the whole novel, Florentina enters in its second half, and the doctor Golfín opens and closes it; the blindness of Pablo (ciego) belongs mostly to the first half. Beside the names of the characters, the keywords show the spelling of the editions - á and ó with the accent of the old orthography in Galdós - and words of the themes of Unamuno: acaso, hijo, mujer. The network of collocations gathers the names and the places of Marianela around d., the abbreviated don: Teodoro Golfín, Aldeacorba de Suso, the mines of Socartes.