CLDR-18108 Give all languages a primary script: trivial cases #4210
Conversation
conradarcturus
had a problem deploying
to
cloudflare
GitHub Actions
Failure
conradarcturus
had a problem deploying
to
cloudflare
GitHub Actions
Failure
| 1. Every language needs at least 1 script considered the **primary** script. | ||
| 1. This data is used to determine [the most Likely language and region](likelysubtags-and-default-content) so there needs to be at least 1 primary value. | ||
| 2. [Changed in v47] Include a primary script for historical languages (eg. Ancient Greek, Coptic). The primary script should reflect where the majority of the written corpus originates from. | ||
| 2. Languages written by significant populations with different scritps in different countries can have multiple **primary** scripts. The [likely subtags](https://www.unicode.org/cldr/charts/latest/supplemental/likely_subtags.html) patterns will use population counts to disambiguate the default script for each locale. |
There was a problem hiding this comment.
I disagree with this; I think it complicates our work. It is simplest if there is exactly 1 primary script, period, that will match the maximized value for the language. All others are not needed, because they will be derived from language population data. Eg if Kurdish has Latin as the primary script, but ku_Arab is the largest population (of ku*) speakers, then we will generate ku_IQ => ku_Arab_IQ in likely subtags.
Is there any other purpose to having any additional primaries or secondaries?
There was a problem hiding this comment.
I'm fine with that to (only ever 1 primary). Do you think the CLDR TC agrees with that? I don't think I had much involvement.
I was trying to strike a compromise with the concerns people had about picking a primary language (that was mentioned in the DDL group). I'll bring it up again in the next DDL meeting.
| @@ -1712,6 +1717,7 @@ XXX Code for transations where no currency is involved | |||
| <language type="ilo" territories="PH" alt="secondary"/> | |||
| <language type="inh" scripts="Cyrl"/> | |||
| <language type="inh" scripts="Arab Latn" territories="RU" alt="secondary"/> | |||
| <language type="io" scripts="Latn" alt="secondary"/> | |||
There was a problem hiding this comment.
secondary with no primary, see comment below.
There was a problem hiding this comment.
Same comment for other new secondaries.
There was a problem hiding this comment.
This is auto-generated. TBH I don't understand why the ConvertLanguageData script did this for this entry but not others -- I suspect because its missing territory data. I'll have to edit ConvertLanguageData.
There was a problem hiding this comment.
I updated the script so it will no longer demote historical scripts/historical languages.
conradarcturus
had a problem deploying
to
cloudflare
GitHub Actions
Failure
|
Thanks for all your work on this... We are getting some long-needed
improvements!
…On Wed, Nov 20, 2024, 18:09 Conrad Nied ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In common/supplemental/supplementalData.xml
<#4210 (comment)>:
> @@ -1712,6 +1717,7 @@ XXX Code for transations where no currency is involved
<language type="ilo" territories="PH" alt="secondary"/>
<language type="inh" scripts="Cyrl"/>
<language type="inh" scripts="Arab Latn" territories="RU" alt="secondary"/>
+ <language type="io" scripts="Latn" alt="secondary"/>
This is auto-generated. TBH I don't understand why the ConvertLanguageData
script did this for this entry but not others -- I suspect because its
missing territory data. I'll have to edit ConvertLanguageData.
—
Reply to this email directly, view it on GitHub
<#4210 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACJLEMG5IQOY5XFXEUYP6RT2BU6GJAVCNFSM6AAAAABSFQTMH2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDINJQGA3TGNJRGY>
.
You are receiving this because your review was requested.Message ID:
***@***.***>
|
This change adds "primary" scripts to many languages in language_script.tsv. This won't change likely subtags, rather this just future-proofs our data by recognizing a singular primary script, avoiding issues where ambiguities served customers the wrong script. I also added scripts for languages in country_language_population.tsv that were missing.
Updated the ConvertLanguageData script to avoid demoting historical scripts/historical langauges. Also removed multi-primary script notes from the description -- anticipating a re-design, handled by other tasks.
conradarcturus
force-pushed
the
conradarcturus/CLDR-18108-add-primary-scripts
branch
from
3cb6b86
to
7f653f7
Compare
|
Notice: the branch changed across the force-push!
~ Your Friendly Jira-GitHub PR Checker Bot |
conradarcturus
had a problem deploying
to
cloudflare
GitHub Actions
Failure
mvn spotless:apply
conradarcturus
had a problem deploying
to
cloudflare
GitHub Actions
Failure
|
I think we should put this on hold for now, and focus on https://unicode-org.atlassian.net/browse/CLDR-18087. |
This change adds "primary" scripts to many languages in language_script.tsv.
This won't change likely subtags, rather this just future-proofs our data by recognizing a singular primary script, avoiding issues where ambiguities served customers the wrong script.
I also added scripts for languages in country_language_population.tsv that were missing.
CLDR-18108
There are many more tasks, especially more complicated ones, in the parent ticket https://unicode-org.atlassian.net/browse/CLDR-18102
ALLOW_MANY_COMMITS=true