Phonostatistics¶

PhonStats(source, words_extractor=None, window_len=3)

The phonetic statistics of a text: its sounds by class, their clusters, sequences and repetitions, and its syllables. A sound is a letter of a word by default, or a sound of its transcription when a library gives one. The data source is a text or a Doc object of spaCy; the words are taken in lower case, and an extractor passed explicitly is used on the text of a Doc.

Raises:

SourceTypeError - a source that is neither a string nor a Doc, an extractor of another type;
SourceError - a source without words;
ParameterError - a window that is not an integer of at least 2.

The class extends the PhonStats of the anyTS core with the hooks of Russian: the sounds are the letters of a word in the classes of the table below, without devoicing, reduction and stress, and the syllables follow the rising sonority rule of the syllabify function. Stress marks are dropped from the words of a text and of a Doc, and the parts of a hyphenated word that spaCy splits are joined (во-первых is one word). The print_stats table and the error messages are in Russian (PHON_STATS_DESC and CORE_MESSAGES of ruts.constants).

Class	Letters	Constant
Vowels	а, е, и, о, у, ы, э, ю, я, ё	`RU_VOWELS`
Sonorant consonants	л, м, н, р, й	`RU_CONSONANTS_SONOR`, `RU_CONSONANTS_YET`
Voiced obstruents	б, в, г, д, ж, з	`RU_CONSONANTS_HIGH`
Voiceless obstruents	к, п, с, т, ф, х, ц, ч, ш, щ	`RU_CONSONANTS_LOW`
Signs	ь, ъ - no sounds: not counted in the shares and do not break a consonant cluster	`RU_MARKS`

CodeResult

from ruts import PhonStats

text = "Ног нет, а хожу, рта нет, а скажу: когда спать, когда вставать, когда работу начинать"
ps = PhonStats(text)
ps.print_stats()

              Статистика              | Значение
-------------------------------------------------
Доля гласных                          |   0.40
Доля сонорных согласных               |   0.11
Доля звонких шумных согласных         |   0.19
Доля глухих шумных согласных          |   0.29
Отношение согласных к гласным         |   1.48
Доля кластеров из 3 и более согласных |   0.03
Зияний гласных на слово               |   0.00
Энтропия CV-шаблонов слов (бит)       |   3.14
Жёсткость                             |   0.56
Индекс аллитерации                    |   1.09
Индекс ассонанса                      |   0.83
Доля открытых слогов                  |   0.76
Средняя длина слога (звуков)          |   2.48

Parameters¶

Parameter	Type	Default	Description
`source`	`str` / `Doc`	-	Data source
`words_extractor`	`WordsExtractor`	`None`	Word extraction tool; used for a `Doc` too, on its text
`window_len`	`int`	`3`	Window in words of the alliteration and the assonance (`PHON_WINDOW_LEN`)

check_params(window_len)

Checks the window of the alliteration and the assonance apart from the class, with the same ParameterError.

Attributes¶

Attribute	Type	Description
`words`	`tuple[str]`	Words in lower case
`window_len`	`int`	Window of the alliteration and the assonance
`syllables`, `sounds`	`tuple`	Syllables (`syllabify`) and sounds (`transcribe`) of every word
`n_vowels`, `n_sonorants`, `n_voiced`, `n_voiceless`, `n_consonants`	`int`	Numbers of the sounds by class
`c_clusters`	`dict[int, int]`	Number of the consonant clusters by length (`consonant_clusters`)
`c_syllable_patterns`	`dict[str, int]`	Number of the syllables by CV pattern

The ruTS attribute n_marks (int) is the number of ь and ъ.

CodeResult

ps.syllables[:4]
ps.c_clusters
ps.c_syllable_patterns
ps.n_marks

(('ног',), ('нет',), ('а',), ('хо', 'жу'))
{1: 22, 2: 6, 3: 1}
{'CCCV': 1, 'CCV': 5, 'CCVC': 1, 'CV': 11, 'CVC': 5, 'V': 2}
3

Metrics¶

Every metric is an attribute of the object, computed when it is created, and get_stats gives them all.

Metric	Measures	Range	Higher means
`p_vowels`	Share of the vowels among the sounds of the classes	0-1, `nan` without sounds	A more vocalic text
`p_sonorants`	Share of the sonorants among the sounds of the classes	0-1, `nan` without sounds	A more sonorous text
`p_voiced`	Share of the voiced obstruents among the sounds of the classes	0-1, `nan` without sounds	More voiced obstruents
`p_voiceless`	Share of the voiceless obstruents among the sounds of the classes	0-1, `nan` without sounds	More voiceless obstruents
`consonant_vowel_ratio`	Ratio of the consonants to the vowels	0 and above, `nan` without vowels	Denser consonants
`p_heavy_clusters`	Share of the clusters of three consonants or more	0-1, `nan` without clusters	Heavier clusters
`p_hiatus`	Hiatuses of `word_hiatus` per word	0 and above, `nan` without sounds	More vowels in a row
`cv_entropy`	Shannon entropy of the CV patterns of the words, in bits	0 and above, `nan` without sounds	More varied shapes of the words
`hardness`	Voiceless obstruents to the vowels and the sonorants	0 and above, `nan` without vowels and sonorants	A harder sound
`alliteration`	Repetition index of the consonants in windows of `window_len` words	0 and above, `nan` for a text shorter than the window or with no consonant shared by two words	Consonants repeated closer together
`assonance`	Repetition index of the vowels in windows of `window_len` words	0 and above, `nan` for a text shorter than the window or with no vowel shared by two words	Vowels repeated closer together
`p_open_syllables`	Share of the syllables that end in a vowel	0-1, `nan` without syllables	More open syllables
`mean_syllable_len`	Mean length of a syllable in sounds	1 and above, `nan` without syllables	Longer syllables

Shares of the classes of the sounds¶

p_vowels · p_sonorants · p_voiced · p_voiceless

The share of the sounds of every class - the vowels, the sonorants and the voiced and voiceless obstruents - among the sounds of all of them; a sound of skipped or of no class (a digit, a letter of another script) counts in none.

\[ \frac{\textrm{Number of sounds of the class}}{\textrm{Number of vowels and consonants}} \]

Ratio of the consonants to the vowels¶

consonant_vowel_ratio

The number of the consonants n_consonants to the number of the vowels n_vowels.

\[ \frac{\textrm{Number of consonants}}{\textrm{Number of vowels}} \]

Consonant clusters¶

p_heavy_clusters · c_clusters

calc_consonant_clusters(text)

The distribution of the consonant clusters of the words by length. A cluster is a run of consonants inside a word, a single consonant between vowels or at the edge of a word included.

Parameter	Type	Default	Description
`text`	`list[str]`	-	List of words

Hiatuses¶

p_hiatus

calc_hiatus(text)

The number of the vowel hiatuses, two or more vowels in a row inside a word: аэропорт, аист, поэзия (о-э). The iotated е, ё, ю, я after a vowel denote [j] and a vowel and make no hiatus: заяц, моя, красивая, читает.

Parameter	Type	Default	Description
`text`	`list[str]`	-	List of words

Entropy of the CV patterns¶

cv_entropy

The Shannon entropy of the CV patterns of the words (cv_pattern), in bits, where \(P(p)\) is the share of the words with the pattern \(p\) among the words that have one.

\[ -\sum_{p} P(p) \log_2 P(p) \]

calc_cv_entropy(text)

The entropy of the CV patterns of a list of words; words without vowels and consonants (numbers, Latin script) are left out, and without other words the value is nan.

cv_pattern(word)

The CV pattern of a word or a syllable: vowels are marked V, consonants C, and ь, ъ and the other characters are left out.

Parameter	Type	Default	Description
`text`	`list[str]`	-	List of words (`calc_cv_entropy`)
`word`	`str`	-	Word or syllable (`cv_pattern`)

Hardness¶

hardness

The number of the voiceless obstruents to the number of the vowels and the sonorants.

\[ \frac{\textrm{Number of voiceless obstruents}}{\textrm{Number of vowels} + \textrm{Number of sonorants}} \]

Alliteration and assonance¶

alliteration · assonance

calc_alliteration(text, window_len=3)

calc_assonance(text, window_len=3)

The alliteration and assonance indices of a list of words: the repetition index calc_repetition_index over the consonant and over the vowel letters of the words.

Parameter	Type	Default	Description
`text`	`list[str]`	-	List of words
`window_len`	`int`	`3`	Window in words (`PHON_WINDOW_LEN`)

calc_repetition_index(words, window_len, features=None)

The repetition index: the number of windows of window_len neighbouring words where a feature (a letter or a sound) occurs in two words or more, summed over the features, divided by the number expected if the words stood in random order. A window of shuffled words is a sample without replacement, so for a feature found in \(K\) of the \(N\) words, a window of \(w\) words holds it in two words or more with the hypergeometric probability

\[ P = 1 - \frac{\binom{N-K}{w} + K \binom{N-K}{w-1}}{\binom{N}{w}} \]

and the expected number is the sum of these probabilities over the features times the \(N - w + 1\) windows. Since the expectation comes from the text itself, the index shows whether the repetitions gather in neighbouring words, not how frequent a feature is: it averages to 1 over the orders of the words and is well above 1 when the repetitions come closer than chance. A word counts a feature once; the index is nan for a text shorter than the window or with no feature shared by two words.

Parameter	Type	Default	Description
`words`	`list[str]` / `list[list[str]]`	-	Features of every word: a string of its letters or a collection of its sounds
`window_len`	`int`	-	Window in words, at least 2
`features`	`Collection[str]`	`None`	Features to count, all of them by default

Words that are not a list of strings or of collections of strings and features that are not a collection of strings raise SourceTypeError, and a window_len that is not an integer of at least 2 ParameterError.

Syllables¶

p_open_syllables · mean_syllable_len

The syllables are the sounds of every syllable of a word by syllable_sounds; a syllable without a sound of the classes (a number) is no syllable, in c_syllable_patterns too. An open syllable ends in a vowel by its CV pattern (cv_pattern), and the length of a syllable is the length of that pattern: the sounds of the classes, without skipped and the signs out of them.

In ruTS the length of a syllable is the number of its letters without ь and ъ.

is_open_syllable(syllable)

Checks whether a syllable is open, that is ends in a vowel; ь and ъ after a vowel do not close it.

Parameter	Type	Default	Description
`syllable`	`str`	-	Syllable

The functions of ruTS count by the classes of letters of the table above. In the calc_* functions words that are not a list of strings raise SourceTypeError, and a window that is not an integer of at least 2 ParameterError.

CodeResult

from ruts.phon_stats import (
    calc_alliteration,
    calc_consonant_clusters,
    calc_cv_entropy,
    calc_hiatus,
    cv_pattern,
    is_open_syllable,
)

words = ["слышно", "бесшумно", "шуршат", "камыши", "у", "поэта"]
cv_pattern("всплеск"), is_open_syllable("съе")
calc_consonant_clusters(words)
calc_hiatus(words), calc_cv_entropy(words)
calc_alliteration(words, window_len=2)

('CCCCVCC', True)
{1: 8, 2: 5}
(1, 2.584962500721156)
1.4999999999999998

Methods¶

The methods get_stats, to_dataframe, print_stats and describe are those of the statistics interface of the anyTS core; the phonostatistics have no scales, and describe gives None for them.

CodeResult

ps.get_stats()

{'p_vowels': 0.4032258064516129,
 'p_sonorants': 0.11290322580645161,
 'p_voiced': 0.1935483870967742,
 'p_voiceless': 0.2903225806451613,
 'consonant_vowel_ratio': 1.48,
 'p_heavy_clusters': 0.034482758620689655,
 'p_hiatus': 0.0,
 'cv_entropy': 3.1395722619867223,
 'hardness': 0.5625,
 'alliteration': 1.0916179337231968,
 'assonance': 0.8305084745762712,
 'p_open_syllables': 0.76,
 'mean_syllable_len': 2.48}