… on supplication letters

Automated categorisation of Middle Latin supplication letters to the pope

What are supplication letters to the pope concerned with in the late Middle Ages? There is a wide range of occasions that could lead to the drafting of such a petition, whether for secular reasons or out of concern for the salvation of one’s soul.

In this research tandem, a language model is trained to automatically assign thousands of such Middle Latin supplication letters from the former Holy Roman Empire to categories of canon law offences such as violation of fasting regulations or homicide.

How were supplication letters initiated and processed in the Middle Ages?

Typically, the faithful committed an offence and, troubled by matters of conscience, then turned to the pope and submitted a petition for mercy, for example to request a special dispensation. In doing so, the person (petitioner, exponens) submitting a supplication letter (supplica, petition for grace) would usually address a procurator as an intermediary, in order to ensure that linguistic and structural formal requirements were observed (stilus curiae).

Figure 1: Example of a supplication letter, 1460; Salonen & Schmugge (2009). A Sip from the “Well of Grace”, p. 166.

Figure 2: Transcript and translation of the supplication letter from Salonen & Schmugge (2009). A Sip from the “Well of Grace”, pp. 116–117.

What did the faithful hope to achieve by submitting a supplication letter?

A supplication letter served as a medium for contacting the pope. Even though, due to the growing number of cases, petitioners could not expect a response from the pope personally, his institutional representatives possessed all the necessary powers (plenitudo potestatis) to decide individual cases. One of the responsibilities of the Apostolic Penitentiary (Sacra Poenitentiaria Apostolica), directly subordinate to the pope and headed by the Major Penitentiary as the highest papal office of grace, was to grant acts of mercy, absolutions, and dispensations.

It should be noted, however, that the registers of the Penitentiary contain only the positively granted supplications and not the letters of response. (Registration was subject to an additional fee.) Relevance for research: The register examined contains more than 40,000 supplication letters from the period 1431–1523. Most of them are thematically classified and assigned to one of four main subject areas: matrimonial matters (De matrimonialibus), defects of birth (De defectu natalium et de uberiori), clerical careers (De promotis et promovendis), and confessional privileges (De confessionalibus). However, 9,070 supplication letters are listed unspecifically under “Miscellaneous” (De diversis formis et de declaratoriis). Although these texts are already available in digital form and are unquestionably of great cultural-historical value, they have received little scholarly attention to date. The reasons for this remain unclear. Is it due to their lack of clear structure?

Using text classification, a method from Natural Language Processing (NLP), a Daidalos research tandem seeks to identify the topics addressed in these supplication letters. If successful, this approach can shed light on the thematic variation of the supplications and potentially stimulate further research interest in these largely unexplored sources.

Research question:

Can the register entries recorded in the registers of the Apostolic Penitentiary under the heading De diversis formis et de declaratoriis be automatically assigned to predefined thematic categories using NLP tools?

On the basis of an extensive historical register corpus, the Repertorium Poenitentiariae Germanicum (RPG), a doctoral researcher in Latin studies investigates this question in a research tandem with the NLP expert Konstantin Schulz from the Daidalos team.

Information on the text corpus:

• Language: Middle Latin

• Period: 1431–1523

• Text types: supplication letters or register entries (abridged copies of the original supplications)

• Authors: petitioners and (ecclesiastical) scribes of the supplication letters

• Further keywords: late Middle Ages, Apostolic Penitentiary, grace, absolution, supplication

Structure of the text corpus:

• Target corpus:

  • 9,070 supplication letters (RPG, Miscellaneous)

• Reference corpus:

  • Training data: approx. 300 manually annotated and digitally available supplication letters (RPG, Miscellaneous) from the Archdiocese of Magdeburg

Method selection, workflow, and challenges:

The research question determines the method. Since the task in machine text processing involves automated categorisation, the method chosen is text classification, an NLP technique.

The idea behind text classification is to identify what a letter is about based on linguistic patterns.

To train an AI accordingly, reliable training data are required.

Provision of training data

In the first step, the Latinist provides a dataset as the basis for training: a manual assignment of approximately 300 supplication letters to one of 17 categories in Excel files.

Figure 3: Seventeen categories, modified after Schmugge / Hersperger / Wiggenhauser (1996).

Training and specialisation of a Latin-language AI

How does one train an AI for Middle Latin supplication letters?

With the training data in place, the process can begin. Konstantin selects a suitable AI model (this language model) as a base. It can be compared to ChatGPT in principle, but in this case the model was trained on the entire Latin portion of the internet, comprising approximately 600 megabytes of raw Latin text.

(Note: From the linked multilingual dataset, only the Latin subset was used to train the Latin large language model.)

In the next step, the Latin AI must be specialised for text classification. It receives text as input and is expected to output a category or offence label.

The assignment of the offence label is probability-based. For each of the 17 categories, the model calculates a probability. The category with the highest probability effectively “wins” and is returned as the sole output label.

Evaluation

The results on the training data are promising, with 94.6 percent correct classifications.

However, performance on the test data drops to only 50 percent.

What does this mean? Konstantin interprets this as an indication of overfitting and defines it as follows: “Overfitting occurs when the model simply memorises the texts from the training data instead of drawing more general conclusions that could be transferred to other texts. […] Overfitting can often be recognised when performance during training (92%) differs sharply from performance on new texts (50%).” Which factors contributed to this?

Identifying problems

Questions arise concerning text length, multiple assignments, and the resolution of abbreviations, as all of these factors have a significant impact on the model’s performance.

Further adjustments

For these reasons, further adjustments are necessary. One possible solution is to reduce the number of categories to fewer than 17. If fewer offence labels are available, training becomes easier for the algorithm. Similar categories can therefore be grouped under broader headings:

• “Violent offences”: homicide, bodily harm, raids or participation in warfare

• “Special permissions”: loosening of fasting regulations, permission to visit pilgrimage sites, relaxation or commutation of vows, confessional privileges, special permissions concerning the celebration of Mass

In addition, the training material itself needs to be refined. Word segmentation in the digitised texts is often inadequate, and abbreviations must be resolved. This requires domain-specific knowledge. Here, the expertise of the Latinist is essential to ensure that the research tandem can proceed in a productive direction.