[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <ea10842daa12b344be1f24232e3b62e41f673757.camel@postmarketos.org>
Date: Wed, 01 Oct 2025 15:55:59 +0200
From: Pablo Correa Gomez <pabloyoyoista@...tmarketos.org>
To: Rich Felker <dalias@...c.org>, musl@...ts.openwall.com
Subject: Re: Selecting locale source format
We got now a few replies from translators, and the most remarkable
thing that was brought up is how to deal with natural text whose
translations might change depending on context. Both plural forms and
declinations were brought up.
Discussing a bit with Rich, it seems that such thing will not be an
issue for strings related to the libc API, which is what is the biggest
concern of the work we are doing now. However, there are
implementation-dependent strings in libc, like dynamic linker messages,
which could potentially be added in the future. Still, since we are
setting the file format, it would be important to make sure that
whatever we come up now is flexible enough to not block future
development. Any thoughts?
Pablo
El vie, 19-09-2025 a las 15:59 +0200, Pablo Correa Gomez escribió:
> Thanks a lot Rich for the follow-up.
>
> I have now called the attention of the translators, and asked them
> some
> further questions, most importantly if there is something they think
> won't be accommodated by their language.
>
> Personally, I do not see anything that might break Spanish, as we
> have
> in the current format.
>
> I also like option (1) best, mostly out of it being more compact, and
> similar to what other people are doing. IIRC glibc locale
> translations
> look very similar.
>
> Best,
> Pablo
>
> El mar, 16-09-2025 a las 21:14 -0400, Rich Felker escribió:
> > I have a proposed binary format for new locale files that I'm in
> > the
> > process of writing up, but Pablo brought it to my attention that,
> > while binary format (ABI) is what's important to have down and
> > stable
> > at the time we integrate into musl, pinning down the source format
> > is
> > what's important/blocking for collaboration with localization
> > folks.
> >
> > I have two candidate formats in the works right now for this:
> >
> >
> >
> > Option 1: subset+extension of POSIX localedef format.
> >
> > The basis for this format is described in
> > https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap07
> > .h
> > tml
> >
> > If we go this way, it would be a "subset" because (1) some parts
> > are
> > not relevant, like LC_CTYPE, which does not vary by locale, (2)
> > some
> > parts will necessarily be represented in different ways, like
> > collation where we're using UCA rather than the POSIX form, and (3)
> > the format just has a lot of gratuitous cruft like symbolic
> > character
> > names. It will also necessarily be extended because POSIX localedef
> > has no way to represent translated error strings etc. - keys for
> > them
> > have to be added.
> >
> > Going this route would have the source data in a fairly compact and
> > "well-known" (to certain audiences) form, but requires that the
> > tooling to produce binary locale files be aware of how these fields
> > translate to the data model for the binary form.
> >
> > A sample (should be roughly correct C/POSIX locale) is attached for
> > reference.
> >
> >
> >
> >
> > Option 2: human-readable/text representation of the binary form
> >
> > Describing this requires a basic intro to the binary form, which is
> > a
> > multi-level hierarchical table mapping a path of integer key values
> > to
> > a data blob. In text we can represent keys with symbolic constants,
> > but they're just a way of writing the underlying numbers. For
> > example
> > the path strerror/0 leads to the "No error information" text,
> > strerror/EACCES leads to the "Permission denied" text, etc. Here
> > "strerror" just represents a number for the first-level path
> > component
> > where strerror strings are stored, subindexed by (the arch/generic
> > versions of) the errno codes.
> >
> > Going this route mostly avoids the need for smarts in the tooling,
> > and
> > "has more flexibility" to encode things. But this also potentially
> > makes the encoding seem more arbitrary to localization folks.
> >
> > Like in option 1, a sample (some hybrid between C/POSIX and a
> > hypothetical US-English locale, whipped up quick by hand as an
> > example) of one way this format could look is attached for
> > reference.
> > An obvious variant that might be friendlier/more-familiar to folks
> > working with the data would be representing the same in json (which
> > is
> > easy).
> >
> >
> >
> >
> > My leaning is towards option 1.
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
