Texts with grade labels

TextsByGrade(data_dir=DEFAULT_DATASET_DIR)

68 grade-labeled texts from the repository of Ivan Begtin's Plain Russian Language project, license CC0 1.0: 55 from the TEXT_LIST list, on which the coefficients of the plainrussian preset of ReadabilityStats were fitted, and 13 from repository folders outside the list. The label is the school grade or the year of study:

Label	Texts	Count
1	fairy tales, Bianki, Gaidar, Tolstoy	11
3-4	Troepolsky, Paustovsky, Rybakov, Korolenko, Grigorovich	10
5-9	Carroll, Shukshin, Kaverin, Tolstoy, Astafyev, Solzhenitsyn	12
10-11	Korolenko, Bunin, Kuprin, Platonov, Gorky, Rasputin, Dovlatov	18
12	Wikipedia article	1
15	newspaper article	1
17	regulatory and business documents: laws, regulations, government news	15

Labels 12-14 are years 1-3 of university, 15-17 - years 4-6, as in the grade_to_age table. Paragraphs are separated by line breaks, the hard line breaks of the original files are removed.

Raises:

• SourceTypeError - a data_dir that is neither a string nor a Path.

CodeResult

from ruts.datasets import TextsByGrade

tbg = TextsByGrade()
tbg.download()
tbg.info

{'Наименование': 'texts_by_grade',
 'url': 'https://github.com/infoculture/plainrussian',
 'description': 'Тексты с метками класса проекта Plain Russian Language',
 'author': 'Бегтин И.В.',
 'license': 'CC0 1.0'}

Parameters

Parameter	Type	Default	Description
data_dir	str / Path	DEFAULT_DATASET_DIR	Path to the dataset directory; texts in DEFAULT_DATA_DIR by default

Attributes

Attribute	Type	Description
labels	tuple[str]	Tuple of text grade levels
info	dict[str, str]	Name of the dataset and reference information about it

Methods

download(force=False) downloads the archive texts_by_grade_v1.tar.xz from the network with checksum verification and extracts the files into the texts_by_grade_v1 directory.

Parameter	Type	Default	Description
force	bool	False	Download the dataset even if it is already downloaded

An archive that cannot be downloaded or fails the checksum twice raises DownloadError, an archive that cannot be extracted DataFileError.

check_data() checks that the dataset is downloaded completely and returns True; a dataset that is not downloaded or incomplete raises DatasetNotFoundError with the commands to download it.

get_texts(grade=None, subject=None, min_len=None, max_len=None, limit=None) extracts texts (without headers) from the dataset.

Parameter	Type	Default	Description
grade	int	None	Grade level of the texts (1, 3-12, 15, 17)
subject	str	None	Text title (substring, ignoring case and ё)
min_len	int	None	Minimum text length (in characters)
max_len	int	None	Maximum text length (in characters)
limit	int	None	Number of texts

An unknown grade level, a subject that is not a string, a length bound that is not a positive integer, a minimum above the maximum and a limit that is not a non-negative integer raise ParameterError; a dataset that is not downloaded DatasetNotFoundError, a file a record cannot be extracted from DataFileError.

CodeResult

for text in tbg.get_texts(grade=1, subject="Ряба", limit=1):
    print(text[:105])

Жил себе дед да баба, у них была курочка Ряба; снесла под полом яичко - пестро, востро, костяно, мудрено!

get_records with the same parameters extracts records (with headers) from the dataset. Record fields: grade - grade level, subject - text title, source - source URL (an empty string for "Золотой ключик" and the State Council report, whose addresses are not published), text - text, file - path to the file.

CodeResult

from ruts import ReadabilityStats

for record in tbg.get_records(grade=17, limit=3):
    print(record["subject"][:40], round(ReadabilityStats(record["text"]).consensus_grade, 1))

«Об утверждении Административного реглам 37.0
Доклад Рабочей группы Государственного с 21.5
Закон о ФКС 34.5

Readability formula checks

In the library tests the Spearman correlation of every formula with the grade label over the 68 texts must be at least 0.7 in absolute value, that of the consensus grade at least 0.8. Values with the default coefficients:

Formula	Spearman's ρ	Mean absolute error, grades
Flesch-Kincaid test	0.80	4.1
Flesch reading ease	−0.80	-
Coleman-Liau index	0.78	3.0
SMOG index	0.82	4.1
Automated readability index	0.80	4.0
LIX readability index	0.81	-
RIX readability index	0.78	-
Solovyev, Ivanov and Solnyshkina formula	0.78	5.0
Matskovsky formula	0.77	-
Tuldava formula	0.79	-
Dale-Chall index	0.82	3.8
Gunning fog index	0.81	4.1
Consensus grade	0.81	3.7

The mean consensus grade grows with the label: 2.9 for grade 1, 5.8 for grades 3-4, 6.5 for 5-9, 7.3 for 10-11 and 22.6 for documents labeled 12-17. The error on documents is large: the formulas are calibrated on school texts, and label 17 is nominal.

A breakdown over all groups of statistics on this dataset is in the notebook 02_text_complexity_by_grade.ipynb.