Literature fingerprinting

fingerprinting(texts, segment_len=10, metric=None, x_size=800, y_size=600, cmap='viridis', ax=None, labels=None)

Visualization of literature fingerprinting (Keim and Oelke 2007).

Every text is cut into segments of segment_len words with a sliding step of a tenth of a segment, and a measure of lexical diversity is computed for every segment. All segments are segment_len words long, so the values are comparable: when the step leaves words after the last segment, one more ends with the text, and a text not longer than a segment is one segment.

A text is a block of squares, its segments row by row, 8 rows high, or one column with at most 8 segments; a block wider than the drawing area wraps into rows of that width. The blocks go left to right and wrap within an area 2 · x_size wide, which grows downwards from 2 · y_size when they need more height. The colorbar runs from the smallest to the greatest finite value over all the texts; undefined values (nan on segments too short for the measure) and the empty cells of a block are light grey, unlike any value, zero included.

The function is that of the anyTS core; the measures of lexical diversity of esTS, functions of a list of words such as calc_ttr or calc_simpson_index, serve as the metric. The default title is the English one of VISUALIZER_LABELS in anyts.constants; labels replaces it.

The first five windows of 1000 words of six novels from the corpus of literature, three by Galdós and three by Unamuno, by Simpson's index:

from ests import WordsExtractor
from ests.corpus import split_windows
from ests.datasets import SpanishLiterature
from ests.diversity_stats import calc_simpson_index
from ests.visualizers import fingerprinting

sl = SpanishLiterature()
sl.download()
titles = (
    "Marianela",
    "Misericordia",
    "Torquemada en la hoguera",
    "Niebla",
    "Abel Sánchez",
    "La tía Tula",
)
novels = {
    record["title"]: record["text"]
    for author in ("galdos", "unamuno")
    for record in sl.get_records(author=author)
    if record["title"] in titles
}

we = WordsExtractor(lowercase=True)
texts = [
    we.extract(window) for title in titles for window in split_windows(novels[title], 1000)[:5]
]
fingerprinting(texts, segment_len=100, metric=calc_simpson_index, x_size=1000, y_size=330)

The first fifteen blocks are Galdós, the last fifteen Unamuno. Simpson's index - the probability that two words drawn at random are the same - is lower in Galdós, so his blocks are darker and Unamuno's, who repeats his words more, greener.

Parameters

Parameter	Type	Default	Description
texts	list[list[str]]	-	List of lists of words
segment_len	int	10	Size of a segment
metric	Callable	None	Function of a measure of lexical diversity; calc_ttr by default
x_size	int	800	Half the width of the drawing area, greater than 25 (the margin between the blocks)
y_size	int	600	Half the height of the drawing area, which grows when the blocks need more
cmap	str	'viridis'	Colour map
ax	Axes	None	Axes of matplotlib for the plot; if not given, a 15×10 figure is created
labels	dict[str, str]	None	Labels over the defaults: title

The function returns the Axes with the visualization, set to an equal aspect so the squares stay square; the figure is ax.figure. Texts that are not a list of lists of strings and a metric that cannot be called raise SourceTypeError, a list without texts or a text without words SourceError. A segment_len or a y_size that is not an integer above zero, an x_size that is not an integer above 25, an unknown cmap, labels that merge_labels refuses and an ax that is not a matplotlib Axes raise ParameterError.