[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20260904211359.GB23438@brightrain.aerifal.cx>
Date: Fri, 4 Sep 2026 17:14:00 -0400
From: Rich Felker <dalias@...c.org>
To: musl@...ts.openwall.com
Subject: Re: Collation implementation is functional
On Fri, Sep 04, 2026 at 12:45:33PM -0400, Rich Felker wrote:
> On Thu, Sep 03, 2026 at 12:22:22PM -0400, Rich Felker wrote:
> I've written up the following pseudocode for the above process:
>
>
> Non-lookahead input reads:
>
> if nskip[ccc] is nonzero, decremenet nskip[ccc], discard character,
> and start over reading the next character
>
>
> Lookahead process:
>
> if contraction is incomplete
> and next character does not match
> and next character is a non-starter:
>
> place next character pending
>
> make a copy of nfd iterator state, and enter a loop advancing it
> forward:
>
> - end loop if we hit a non-starter
> - if new ccc, skip nskip[ccc]
> - if we hit a match, consume it and increment nskip[ccc]
> - if not a match, skip past all further chars in same ccc
>
> after stopping, the temp iterator copy can be discarded. we only need
> to preserve nskip[] array.
I've implemented this algorithm and with it, the full set of UCA test
vectors passes!
This doesn't rule out there being bugs in more complicated cases like
interleaved contractions (0F71 0F72 and friends) or interactions with
prefix rules (UCA root data has no contractions with prefix condition)
but it is a good sign.
I did test an interleaved sequence 0F71 0F71 0F72 0F72 and got the
expected output:
0: [76 aa, 05, 05]
1: [76 aa, 05, 05]
76 aa 76 aa 01 05 05 01 05 05
matching two instances of the contraction 0F71+0F72.
I'll push the code soon.
Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
