[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20261008233512.GN23438@brightrain.aerifal.cx>
Date: Thu, 8 Oct 2026 19:35:12 -0400
From: Rich Felker <dalias@...c.org>
To: Thorsten Glaser <tg@...bsd.de>
Cc: musl@...ts.openwall.com
Subject: Re: fnmatch: invalid multibyte sequence in the string is
taken as end of string ("" matches "\xff" in UTF-8 locales)
On Thu, Oct 08, 2026 at 09:44:20PM +0200, Thorsten Glaser wrote:
> On Thu, 8 Oct 2026, Rich Felker wrote:
>
> >Putting byte strings that are not valid characters into filenames is
> >generally not a supported usage (by musl or by the standards).
>
> It is, in POSIX. Filenames are indeed just bytes, not characters;
> arbitrary bytes except slash and NUL.
This is not the case.
The only filenames you can use portably are those consisting of
characters from the Portable Filename Character Set:
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap03.html#tag_03_264
Indeed POSIX does specify that filenames consist of bytes not
necessarily characters, and the only things it specifies that cannot
appear are '/', '\0', and more than NAME_MAX bytes (if NAME_MAX is
defined):
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap03.html#tag_03_146
This however is a matter of what a portable application might expect
to find e.g. via readdir, not a guarantee that the implementation
allows making filenames that are arbitrary byte sequences.
And indeed, depending on the underlying filesystem, attempts to create
disallowed or non-representable filenames may fail. For example names
containing a ':' generally fail on FAT variants, and names containing
bytes that are not well-formed UTF-8 fail on filesystems where the
underlying representation of filenames is Unicode codepoint values (or
UTF-16 code units).
musl does nothing to enforce any rules about what filenames may be
used, but as always, the intent has been that textual data is always
UTF-8, and if your filenames are intended as text, for processing with
the shell and standard utilities that work with text, they should be
UTF-8. In the case where you need to bypass that and process filenames
that don't follow this, there's always the byte-based C locale
(LC_CTYPE=C) that was reluctantly added; the committee's motivation
for requiring it seems to have been solving this exact problem.
Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
