Confusable characters as endangered language markers: The case of North Caucasus writing systems

The Abkhaz-Adyghe and Nakh-Daghestanian language families encompass 35 living languages that possess arguably the most complex modern Cyrillic orthographies due to their very sophisticated phonology. The relevant online data displays idiosyncratic patterns among which the use of confusable characters in input methods is the most prevalent. This work studies one such character---letter \emph{palochka}---that is shared by most writing systems in question. We investigate whether the patterns including variants of this character alone can act as language data markers when mining these languages in a large-scale web-crawled data. Using a wide-coverage off-the-shelf LID model (GlotLID) we further investigate the data mined using such patterns and estimate the effects of confusable character normalization on quality of paragraph-level LID predictions in 14 supported languages. According to GlotLID, the normalization significantly increases the recall (discovery of new language data) for some languages while degrading it for others. However, manual evaluation reveals that only 41\% of wins and 46\% of losses are accurate due to GlotLID prediction errors. We argue that despite finding useful signal higher precision LID approaches tailored to these long-tail languages are needed to improve the quality of mined data.