Behind GSI:detect

A stereotype is defined as a pre-constituted, generalised and simplistic opinion, i.e. not based on personal evaluation of individual cases but mechanically repeated, about people or events and situations (Cavagnoli, Dragotto 2021). Gender stereotypes, in particular, are often found in misogynistic hate speech but they often appear in non-hateful communication as well, and unconscious stereotypes can also be used with positive meaning.

Stereotypes are deeply influenced by subjectivity (Sanguinetti et al., 2018): depending on personal background, values, and experiences, the same expression may be seen as clearly stereotypical, neutral, or even positive. This variability shows the perspectival nature of stereotypes and underlines the need for a perspectivist approach (Basile, 2020; Basile et al., 2023; Madeddu et al., 2023) to account for the diversity of perceptions when studying them.

Moreover, the deconstruction of gender stereotypes is necessary for a more adequate description of communicative reality and above all to prevent discrimination. Investigating if AI systems can detect and correctly classify gender stereotypes  may be a good way to understand if this could be of help to human experts in a wide array of practical tasks, such as avoiding gender stereotypes in teaching materials or in curricula vitae, or studying their presence in journalism.

In developing LLMs, for instance, diverse mitigation techniques are applied to avoid biases. Our research question goes one step beyond this: are they also able to discriminate between a stereotyped and a non-stereotyped sentence? To what extent can they do this?

Our mission 🎯

The purpose of the GSI:Detect Task is to advance research on the detection of gender stereotypes in different types of short Italian texts. But we aim to go beyond simple detection: not every stereotype is perceived with the same weight and intensity. We turn this variability into a value, by modeling it as the degree of stereotypicity of a sentence, represented as a continuous value between 0 and 1.
This approach acknowledges that different people may perceive stereotypes differently, and that disagreement among diverse viewpoints is not noise, but the very foundation for modeling how stereotypes emerge in language.

In addition to this, GSI:Detect also includes a sub-task on gender stereotype categorization.

Together, these components aim to explore how AI can contribute both to the detection and to the nuanced evaluation of gender stereotypes in language.