[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20260921165129.GA10604@brightrain.aerifal.cx>
Date: Mon, 21 Sep 2026 12:51:30 -0400
From: Rich Felker <dalias@...c.org>
To: musl@...ts.openwall.com
Subject: [PATCH] [RFC] musl locale overhaul series, take 1
The attached patch series is the current contents of my integration
branch for the locale support overhaul project, funded by NLnet and
the NGI Zero Core Fund. I'm sending it out now for review and testing
by folks doing localization or anyone else who would like to poke at
this.
The attached series is factored into a number of small commits to be
easily reviewable now and long-term in the history; in terms of
behavioral changes, what it includes are:
- Updates of built-in character data to Unicode 17. This is entirely
mechanical, generated by musl-chartable-tools[1], probably not
intersting to read.
- Code to process the new locale file format, and changes to switch
over to using it.
- No longer synthesizing dummy locales for requested locale names that
don't exist. setlocale(cat,"") still succeeds in this case but the
resulting locale name will be C.UTF-8. Explicit requests for a name
result in failure.
- Linear-search-free versions of all of the error message lookup
interfaces (strerror, hstrerror, gai_strerror, regerror) using the
new string table lookup infrastructure for both built-in and locale
string tables.
- Linear-search-free and lookup-key-collision-free version of
nl_langinfo, the backend for localized date/time formatting, using
the same string table infrastructure as error messages now use.
- Collation interfaces (strcoll, strxfrm, wcscoll, wcsxfrm) that pass
conformance test vectors with the CLDR root data. This includes the
NFD iterator and decomposition tables (large) which are generated by
the musl-uca-draft[2] tooling.
- localeconv implementation that reports LC_NUMERIC and LC_MONETARY
properties set in the locale file.
- strfmon implementation that honors LC_MONETARY properties to format
monetary quantities.
It is still missing a couple small details that are regressions from
the old system:
- strsignal is missing translated signal names. It needs a small
amount of additional logic to translate arch-specific signal numbers
to a common numbering for lookup.
- getopt is missing translated error strings. This was an oversight I
need to go back to finish due to the old code calling the __lctrans
function directly rather than using the macro that's used everywhere
else, which I grepped for.
These will probably be done in the next couple days but I wanted to go
ahead and get this series out for more eyes.
There are also a few "stretch goals" that may or may not make it in
immediately. Some of these were items where design of the new locale
format to ensure that it can represent them and avoid locking us out
of future support was in-scope for the project, but adoption was TBD:
- Support for alt numbers and month names in date/time formatting.
These were originally not supported in musl, and part of this is new
in POSIX 2024 which we still need to catch up to in general. But
they don't look difficult to add.
- Support for grouping ("thousands separator") and alternate radix
point character in printf, and the latter in scanf/strtod. If the
radix character is done, it will be restricted to only allowing '.'
or ',' so that applications are not forced to deal with ambiguous or
multibyte radix.
- Bringing gettext behavior at least partly up in line with
standardization into POSIX 2024. This will balance out with losing
support for "synthesized"/fake locales in LC_MESSAGES by letting
users set the LANGUAGE environment variable to get localized gettext
messages for the application even when libc doesn't have a locale
for their language yet.
- Setting a policy for a default locale search path. My leaning is
choosing a non-colliding name under /etc, to be configurable as a
symlink to the actual install location rather than hard-coding some
specific location.
[1] https://github.com/richfelker/musl-chartable-tools
[2] https://codeberg.org/dalias/musl-uca-draft
View attachment "0001-update-character-type-data-and-case-mappings-to-unic.patch" of type "text/plain" (78191 bytes)
View attachment "0002-langinfo-add-alternative-month-name-keys-from-POSIX-.patch" of type "text/plain" (1302 bytes)
View attachment "0003-add-integer-keyed-multi-level-table-lookup-for-new-l.patch" of type "text/plain" (3192 bytes)
View attachment "0004-add-common-__loc_lookup-backend-for-new-locale.patch" of type "text/plain" (2684 bytes)
View attachment "0005-add-include-frameworks-for-building-built-in-C-local.patch" of type "text/plain" (2603 bytes)
View attachment "0006-switch-langinfo-and-error-strings-over-to-new-common.patch" of type "text/plain" (24685 bytes)
View attachment "0007-add-langinfo-keys-for-alt-months-to-string-table.patch" of type "text/plain" (1165 bytes)
View attachment "0008-locale-remove-synthesis-of-nonexistent-locale-names.patch" of type "text/plain" (1552 bytes)
View attachment "0009-locale-make-explicit-request-of-unavailable-locales-.patch" of type "text/plain" (2625 bytes)
View attachment "0010-locale-add-loader-for-new-locale-format-adjust-relat.patch" of type "text/plain" (3296 bytes)
View attachment "0011-locale-replace-gettext-based-string-lookups-with-new.patch" of type "text/plain" (3569 bytes)
View attachment "0012-remove-references-to-old-translation-framework-from-.patch" of type "text/plain" (1109 bytes)
View attachment "0013-locale-remove-now-unused-gettext-style-translation-c.patch" of type "text/plain" (2184 bytes)
View attachment "0014-add-an-iterator-for-unicode-normalization-form-NFD-f.patch" of type "text/plain" (88586 bytes)
View attachment "0015-add-collation-implementation.patch" of type "text/plain" (18321 bytes)
View attachment "0016-move-global_locale-object-out-of-libc-struct.patch" of type "text/plain" (5112 bytes)
View attachment "0017-implement-non-stub-localeconv.patch" of type "text/plain" (9241 bytes)
View attachment "0018-implement-a-working-non-stub-strfmon.patch" of type "text/plain" (6659 bytes)
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
