[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <CAO6moYsEUKm_JN4NM_dBa0eOO2UE4WNQ-F2Uo0ugEUbJxhe+xw@mail.gmail.com>
Date: Mon, 27 Apr 2026 20:31:06 +1000
From: Xan Phung <xan.phung@...il.com>
To: Openwall musl <musl@...ts.openwall.com>
Subject: Re: [PATCH] wcwidth: reduce text size by 55%
In the benchmark data below, my "small" wcwidth function gives a 32% speed
improvement compared to "base" (current) wcwidth.
I originally designed my wcwidth function to reduce data size, but there
also seems to be some performance benefit to my "unified" approach of
reading "wide" and "nonspace" attribute data jointly in one lookup (whereas
existing musl code does lookup of non spacing first, then does separate
lookup of wide data second).
The benchmarks below were compiled with GCC v15.2 on Linux 6.16.7, Ryzen
8845HS CPU, and used -Os optimisation (-O2 gives similar results).
The cycle times were calculated for 50x wcwidth function calls in between
starting & stopping the cycle timer.
(This is needed as otherwise the function call overhead of starting &
stopping the timer overwhelms the cycle count of the wcwidth function
itself)
wcwidth_base (cp sample range 0x0-0x3FF): chksum = -13391
*13.30 cycles per lookup*
wcwidth_small (cp sample range 0x0-0x3FF): chksum = -13391
*9.12 cycles per lookup*
wcwidth_base (cp sample range 0-0xFFFF): chksum = -13309
*20.52 cycles per lookup*
wcwidth_small (cp sample range 0-0xFFFF): chksum = -13309
*13.68 cycles per lookup*
Also, my header generation and regression testing tool is available at the
repo below:
https://github.com/nglibc/musl-chartable-tools.git
Refer to the ctype/README file in the repo for details on how to use the
gen_wcdata tool.
This tool generates the headers not just for wcwdith (using 'w' and 'W'
options), but also iswalpha and iswpunct (using 'a'/"A' and 'p'/'P'
options).
On Thu, 23 Apr 2026 at 22:24, Xan Phung <xan.phung@...il.com> wrote:
> Previous wcwidth code had a total text and data size of 4.5kb. A
> more efficient encoding has reduced the total size to 2kb, a 55%
> reduction.
>
> The new encoding uses a table lookup with both wide and nonspacing
> attribute flags retrieved in the one lookup, whereas previous code
> used two separate lookups. Aside from this unification, the top
> level remains much the same, providing 8 bit offsets into codepage
> units (256 codepoint granularity). The second level data uses fixed
> sizes, of one 32 bit word per codepage (where each 2 bit pair in a
> word identifies a block of 16 codepoints as all normal width, all
> wide, all non-spacing, or mixed 'exceptions'). Most codepoints are
> long runs of a specific width, hence exceptions are less common. If
> an exception is present, an exception index is calculated from the
> popcount of set high bits within the second level's 32 bit word,
> retrieving an extension byte which indexes a dictionary of 16 bit
> values (208 of 216 dictionary entries) or double 16 bit values.
> The dictionary value is then used to determine the type of the
> exception codepoint as control, non spacing, normal width or wide.
>
> The new function has been tested against first 0x20000 codepoints,
> and matches that returned by the pre-existing musl implementation.
> Benchmarks using CPU cycle count shows better performance than the
> pre-existing implementation.
> ---
> src/ctype/nonspacing.h | 91 -------------------
> src/ctype/wcwidth.c | 67 +++++++++++---
> src/ctype/wcwidth_dict.h | 31 +++++++
> src/ctype/wcwidth_table.h | 184 ++++++++++++++++++++++++++++++++++++++
> src/ctype/wide.h | 65 --------------
> 5 files changed, 268 insertions(+), 170 deletions(-)
> delete mode 100644 src/ctype/nonspacing.h
> create mode 100644 src/ctype/wcwidth_dict.h
> create mode 100644 src/ctype/wcwidth_table.h
> delete mode 100644 src/ctype/wide.h
>
> diff --git a/src/ctype/nonspacing.h b/src/ctype/nonspacing.h
> deleted file mode 100644
> index 7746f3b..0000000
> --- a/src/ctype/nonspacing.h
> +++ /dev/null
> @@ -1,91 +0,0 @@
>
> -16,16,16,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,16,33,16,16,16,34,35,36,
>
> -37,38,39,40,16,16,41,16,16,16,16,16,16,16,16,16,16,16,42,43,16,16,44,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,45,16,46,47,48,49,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,50,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,51,16,16,52,
>
> -53,16,54,55,56,16,16,16,16,16,16,57,16,16,58,16,59,60,61,62,63,64,65,66,67,68,
>
> -69,70,16,71,72,73,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,74,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,75,76,16,16,16,77,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,78,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,79,80,16,16,16,16,16,16,16,81,16,16,16,16,16,82,83,84,16,16,16,16,16,85,
> -86,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,
>
> -255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,255,255,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,248,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,254,255,255,255,255,191,182,0,0,0,0,0,0,0,63,0,255,23,0,0,0,0,0,248,255,
>
> -255,0,0,1,0,0,0,0,0,0,0,0,0,0,0,192,191,159,61,0,0,0,128,2,0,0,0,255,255,255,
>
> -7,0,0,0,0,0,0,0,0,0,0,192,255,1,0,0,0,0,0,0,248,15,32,0,0,192,251,239,62,0,0,
> -0,0,0,14,0,0,0,0,0,0,0,0,0,0,0,0,0,0,248,255,255,255,255,
>
> -255,7,0,0,0,0,0,0,20,254,33,254,0,12,0,0,0,2,0,0,0,0,0,0,16,30,32,0,0,12,0,0,
>
> -64,6,0,0,0,0,0,0,16,134,57,2,0,0,0,35,0,6,0,0,0,0,0,0,16,190,33,0,0,12,0,0,
>
> -252,2,0,0,0,0,0,0,144,30,32,64,0,12,0,0,0,4,0,0,0,0,0,0,0,1,32,0,0,0,0,0,0,17,
>
> -0,0,0,0,0,0,192,193,61,96,0,12,0,0,0,2,0,0,0,0,0,0,144,64,48,0,0,12,0,0,0,3,0,
>
> -0,0,0,0,0,24,30,32,0,0,12,0,0,0,0,0,0,0,0,0,0,0,0,4,92,0,0,0,0,0,0,0,0,0,0,0,
>
> -242,7,128,127,0,0,0,0,0,0,0,0,0,0,0,0,242,31,0,63,0,0,0,0,0,0,0,0,0,3,0,0,160,
>
> -2,0,0,0,0,0,0,254,127,223,224,255,254,255,255,255,31,64,0,0,0,0,0,0,0,0,0,0,0,
> -0,224,253,102,0,0,0,195,1,0,30,0,100,32,0,32,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,255,255,255,255,255,255,0,0,0,0,0,0,0,0,0,0,0,224,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,28,0,0,0,28,0,0,0,12,0,0,0,12,0,0,0,0,0,0,0,176,63,64,254,
>
> -15,32,0,0,0,0,0,120,0,0,0,0,0,0,0,0,0,0,0,0,0,0,96,0,0,0,0,2,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,135,1,4,14,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -128,9,0,0,0,0,0,0,64,127,229,31,248,159,0,0,0,0,0,0,255,127,0,0,0,0,0,0,0,0,
>
> -15,0,0,0,0,0,208,23,4,0,0,0,0,248,15,0,3,0,0,0,60,59,0,0,0,0,0,0,64,163,3,0,0,
> -0,0,0,0,240,207,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,247,255,253,33,16,
>
> -3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,
>
> -251,0,248,0,0,0,124,0,0,0,0,0,0,223,255,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,
>
> -255,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,3,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,128,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,0,0,0,0,
>
> -0,60,0,0,0,0,0,0,0,0,0,0,0,0,0,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,128,247,63,0,0,0,192,0,0,0,0,0,0,0,0,0,0,3,0,68,8,0,0,96,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,48,0,0,0,255,255,3,128,0,0,0,0,192,63,0,0,128,255,3,0,
> -0,0,0,0,7,0,0,0,0,0,200,51,0,0,0,0,32,0,0,
>
> -0,0,0,0,0,0,126,102,0,8,16,0,0,0,0,0,16,0,0,0,0,0,0,157,193,2,0,0,0,0,48,64,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,32,33,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,0,0,0,
>
> -64,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,0,0,255,
>
> -255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,14,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,32,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
> -0,0,0,0,0,0,1,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,192,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,110,240,0,
>
> -0,0,0,0,135,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,96,0,0,0,0,0,0,0,240,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,192,255,1,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,255,127,0,0,0,0,0,0,128,
>
> -3,0,0,0,0,0,120,38,0,32,0,0,0,0,0,0,7,0,0,0,128,239,31,0,0,0,0,0,0,0,8,0,3,0,
>
> -0,0,0,0,192,127,0,30,0,0,0,0,0,0,0,0,0,0,0,128,211,64,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,128,248,7,0,0,3,0,0,0,0,0,0,24,1,0,0,0,192,31,31,0,0,0,0,0,0,0,
> -0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,255,92,0,0,64,0,0,0,0,0,0,0,0,0,0,248,133,13,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,60,176,1,0,0,48,0,0,0,0,0,0,0,0,0,0,
>
> -248,167,1,0,0,0,0,0,0,0,0,0,0,0,0,40,191,0,0,0,0,0,0,0,0,0,0,0,0,224,188,15,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,255,6,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,240,12,1,0,0,0,254,7,0,0,0,0,248,121,128,0,126,14,0,0,0,0,0,252,
>
> -127,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,127,191,0,0,0,0,0,0,0,0,0,0,252,255,
> -255,252,109,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,126,180,191,0,0,0,0,0,0,0,0,0,163,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,24,0,0,0,0,0,0,0,255,
>
> -1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,31,0,0,0,0,0,0,0,127,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,0,0,0,0,0,0,0,128,7,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,96,15,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,3,248,255,231,15,0,0,0,60,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,28,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,
>
> -255,255,255,255,127,248,255,255,255,255,255,31,32,0,16,0,0,248,254,255,0,0,0,
>
> -0,0,0,0,0,0,0,127,255,255,249,219,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,127,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,127,0,0,0,0,0,0,0,0,0,0,0,0,0,240,7,0,0,
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
> diff --git a/src/ctype/wcwidth.c b/src/ctype/wcwidth.c
> index 36256a5..2aa6e3d 100644
> --- a/src/ctype/wcwidth.c
> +++ b/src/ctype/wcwidth.c
> @@ -1,26 +1,65 @@
> #include <wchar.h>
> +#include <endian.h>
>
> -static const unsigned char table[] = {
> -#include "nonspacing.h"
> -};
> +#define BYTE(x,y) ((x) >> (y^(__BYTE_ORDER == __LITTLE_ENDIAN ? 0:24)) &
> 255)
> +#define BYTES(x) BYTE(x,0),BYTE(x,8),BYTE(x,16),BYTE(x,24)
> +#define PAGE_SH 8
> +#define PAGE_MAX (1u << PAGE_SH)
> +#define PAGEH (0x20000 / PAGE_MAX)
> +
> +const static union {
> + unsigned char b[PAGEH + 185*4];
> + unsigned int w[PAGEH/4 + 185];
> +} table = {{
> +#include "wcwidth_table.h"
> +}};
>
> -static const unsigned char wtable[] = {
> -#include "wide.h"
> +const static unsigned short dict[216] = {
> +#include "wcwidth_dict.h"
> };
>
> +static int wcwidth_lookup(wchar_t wc)
> +{
> + const unsigned h1 = PAGEH/4 - 2, h2 = PAGEH - 8;
> + unsigned page, shfr, base, lane, rev, target;
> + unsigned huff, type, popc, val;
> + unsigned char ext;
> +
> + /* Direct path used in most of the 256 BMP code pages */
> + page = (wc >> PAGE_SH);
> + base = table.b[page];
> + if (base < 2) /* Base 0-1 are reserved values */
> + return base + 1;
> +
> + /* 2nd & 3rd level arrays: final level idx=popc^rev_direction */
> + target = wc & (PAGE_MAX-1);
> + shfr = (target & 15);
> + lane = (target >> 4);
> + huff = (table.w + h1)[base];
> + type = (huff >> (2 * lane)) & 3 ^ 3;
> + popc = (huff << (31 - 2 * lane));
> + popc = (popc << 1) & popc; /* Count the 0b11 bit pairs */
> + popc = (popc & 0x11111111) + ((popc & 0x44444444) >> 2);
> + popc = (popc * 0x11111111) >> 28;
> + base+= (rev = -(page & 1)) + 1;
> + if (type) /* Return types: Ns 1, Normal 2, Wide 3 */
> + return type - 1;
> +
> + /* Dictionary lookup: 208 out of 216 entries are below HI_END */
> + ext = (table.b + h2 + base*4)[(int)(popc^rev)];
> + val = dict[ext]>>shfr & 1;
> + if (ext < HI_END)
> + return val + !(ext < LO_END);
> + else /* Return types: 0-3, where Control char is 0 */
> + return (val << 1 | (dict[ext-1]>>shfr & 1) ^ !val) - 1;
> +}
> +
> int wcwidth(wchar_t wc)
> {
> if (wc < 0xffU)
> return (wc+1 & 0x7f) >= 0x21 ? 1 : wc ? -1 : 0;
> - if ((wc & 0xfffeffffU) < 0xfffe) {
> - if ((table[table[wc>>8]*32+((wc&255)>>3)]>>(wc&7))&1)
> - return 0;
> - if ((wtable[wtable[wc>>8]*32+((wc&255)>>3)]>>(wc&7))&1)
> - return 2;
> - return 1;
> - }
> - if ((wc & 0xfffe) == 0xfffe)
> - return -1;
> + if (wc < 0x20000)
> + return wcwidth_lookup(wc);
> if (wc-0x20000U < 0x20000)
> return 2;
> if (wc == 0xe0001 || wc-0xe0020U < 0x5f || wc-0xe0100U < 0xef)
> diff --git a/src/ctype/wcwidth_dict.h b/src/ctype/wcwidth_dict.h
> new file mode 100644
> index 0000000..fc41eee
> --- /dev/null
> +++ b/src/ctype/wcwidth_dict.h
> @@ -0,0 +1,31 @@
> +/* wcdata: dictionary 216 entries*/
> +#define LO_END 146
> +#define HI_END 208
> +0xfc07,0x1,0x4000,0xff49,0xffc0,0xe800,0x7ff,0xfffe,
> +0x403f,0xc260,0x7fff,0xfffd,0xf800,0x3f,0xdff0,0x43f,
> +0xc110,0xf1ff,0x7,0xfff8,0xebff,0xde01,0xff01,0xfff3,
> +0xefff,0xdfe1,0xbfff,0xfff9,0xc679,0xffdc,0xde41,0x3ff,
> +0x6fff,0xffbf,0xfffb,0xdffe,0xffee,0x3fff,0xc23e,0xff9f,
> +0xcfbf,0xfffc,0xe7ff,0xfbff,0xffa3,0xf80d,0x807f,0xe00d,
> +0xc0ff,0xfcff,0xfd5f,0x8001,0x1f20,0x100,0xe000,0x1fff,
> +0x9902,0x3cff,0xffe1,0xdf9b,0xdfff,0xffe3,0xc04f,0x1bf,
> +0x87ff,0xfdff,0xfe78,0xf1fb,0xf67f,0x80bf,0xe01a,0x6007,
> +0x8000,0xfff0,0xe82f,0xc4c3,0x5cbf,0xfff,0xff30,0x8,
> +0xde02,0xfcef,0x400,0x83ff,0x20,0xc008,0xf7bb,0xffcf,
> +0x7ffc,0xc03f,0x7f,0xcc37,0xffdf,0x81ff,0xff99,0xeff7,
> +0x3e62,0xcfff,0xdedf,0xf83f,0xf91,0x78ff,0xff0f,0xff,
> +0xff80,0xd987,0x107f,0xffe0,0xfff7,0x803f,0xe1ff,0xbf2c,
> +0xf807,0xe03f,0x7a07,0xfff2,0x4fc3,0x5807,0xd7ff,0xff40,
> +0xf043,0xf900,0xf30f,0xf801,0x8607,0xff7f,0xf181,0xfc80,
> +0x4080,0x3,0x300,0xff92,0x4b81,0xff5c,0xffe7,0xfe00,
> +0x9fff,0xfc7f,0xf018,0xc3ff,0x780,0xffef,0x80,0x600,
> +0xf824,0xf80f,0x70f,0xe7ff,0xc7e7,0xfc7b,0xe000,0x7f0,
> +0x27,0x103f,0xf800,0x60,0x7800,0x3fff,0x9fff,0xfffd,
> +0xff11,0x87ff,0xdfff,0xffbf,0xe001,0x3,0x1ff,0x7fe,
> +0x4000,0xf0,0x7,0x1,0xf7f,0xfff7,0x3ff,0x7f,
> +0x1fff,0xff,0x7ff,0xffe0,0xfffe,0x7fff,0xfff,0x3f,
> +0xfbff,0x21,0x1800,0x8001,0xe0,0xb8,0x5000,0x100,
> +0xc20,0x242c,0x400,0x10,0x4030,0xc02,0x8,0x8000,
> +0xf,0xff00,0x30,0x6000,0x9,0x1e00,0x600,0xc00,
> +0xc3ff,0xc3ff,0xf87f,0xf9ff,0xc000,0x31ff,0xc000,0x3fff,
> +
> diff --git a/src/ctype/wcwidth_table.h b/src/ctype/wcwidth_table.h
> new file mode 100644
> index 0000000..bbf9848
> --- /dev/null
> +++ b/src/ctype/wcwidth_table.h
> @@ -0,0 +1,184 @@
> +/* wcdata: pages 512 x 256, bmap 0x0 */
> + 0, 0, 0, 13, 2, 4, 5, 25, 11, 10, 14, 42, 20, 19, 26, 47,
> + 29, 28, 0, 32, 0, 0, 0, 93, 33, 59, 38, 37, 43, 56, 0, 0,
> + 48, 0, 0, 51, 0, 96, 52, 63, 0, 0, 0, 69, 57,100, 60, 72,
> + 64,146, 67, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
> + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 66, 1, 1,
> + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
> + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
> + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
> + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
> + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
> + 1, 1, 1, 1, 70, 0, 73, 0, 77,105, 81,149, 1, 1, 1, 1,
> + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
> + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
> + 1, 1, 1, 1, 1, 1, 1,153, 0, 0, 0, 0, 0, 0, 0, 0,
> + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> + 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1,156, 0, 0, 86, 76,
> + 0,159, 90,162, 0, 0, 0, 0, 0, 0, 94, 0, 0,165, 0,117,
> + 97,110,101,170,106,114,111,121,115,139,118, 0,122,125,126, 0,
> + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> + 0, 0, 0, 0,129, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,133,173, 0, 0, 0, 80,
> + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
> + 1, 1, 1, 1, 1, 1, 1,177, 1, 1,137, 0, 0, 0, 0, 0,
> + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> + 1,128,140, 0, 0, 0, 0, 0, 0, 0, 0, 0,143, 0, 0, 0,
> + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> + 0, 85,147, 0, 0, 0, 0, 0, 0, 0,150, 0, 0, 0, 0, 0,
> +154,180,157, 0, 0, 0, 0, 0,160,182, 0, 0, 0, 0, 0, 0,
> +163,142,166,132,171,136,174,184, 0, 89,178, 0, 0, 0, 0,186,
> +
> +
> +/* wcdata: U+00400,00500 idx [2-5) */
> +BYTES(0x55575555),0,3,2,1,BYTES(0x57ed5555),
> +
> +/* wcdata: U+00600,00900 idx [5-11) */
> +BYTES(0x7d55db5f),4,5,6,7,8,9,26,23,25,24,11,23,22,21,20,19,
> +BYTES(0xf7d77fd7),
> +
> +/* wcdata: U+00800,00300 idx [11-14) */
> +BYTES(0xad555d7d),15,16,17,18,BYTES(0x55556aaa),
> +
> +/* wcdata: U+00a00,00d00 idx [14-20) */
> +BYTES(0xf7d7dfd7),27,24,28,11,29,27,24,30,23,31,44,43,23,25,42,41,
> +BYTES(0x5f5577d7),
> +
> +/* wcdata: U+00c00,00700 idx [20-26) */
> +BYTES(0x77d77fd7),36,37,38,39,23,11,32,40,23,14,6,7,13,12,11,10,
> +BYTES(0xf5f5579f),
> +
> +/* wcdata: U+00e00,01100 idx [26-29) */
> +BYTES(0x57d557d5),45,46,47,48,BYTES(0xaaaaa000),
> +
> +/* wcdata: U+01000,01300 idx [29-33) */
> +BYTES(0x555ffdf5),55,56,57,7,58,59,60,55,BYTES(0x55555d55),
> +
> +/* wcdata: U+01800,01b00 idx [33-38) */
> +BYTES(0x55775557),64,39,65,41,76,75,41,73,6,34,74,73,BYTES(0xf577f7d7),
> +
> +/* wcdata: U+01a00,00b00 idx [38-43) */
> +BYTES(0x55d5fd5d),68,69,70,71,72,35,34,23,33,25,32,11,BYTES(0x57577fd7),
> +
> +/* wcdata: U+01c00,00f00 idx [43-48) */
> +BYTES(0xfd5555f5),77,78,79,80,81,33,54,53,52,51,50,49,BYTES(0x57efd5dd),
> +
> +/* wcdata: U+02000,02300 idx [48-52) */
> +BYTES(0xe9557577),6,83,84,7,204,205,206,207,BYTES(0xf555557d),
> +
> +/* wcdata: U+02600,01d00 idx [52-57) */
>
> +BYTES(0xfffddf5d),202,201,200,199,198,197,203,196,195,194,193,82,BYTES(0xea555555),
> +
> +/* wcdata: U+02c00,01900 idx [57-60) */
> +BYTES(0xf5555555),10,41,67,66,BYTES(0x555555f5),
> +
> +/* wcdata: U+02e00,02700 idx [60-64) */
> +BYTES(0xc00c5555),184,200,187,188,189,190,191,192,BYTES(0x55dd5f77),
> +
> +/* wcdata: U+03000,04d00 idx [64-67) */
> +BYTES(0x000c03f0),209,181,180,211,BYTES(0x55000000),
> +
> +/* wcdata: U+03200,02b00 idx [67-70) */
> +BYTES(0x0000030c),181,177,185,186,BYTES(0x55555d5d),
> +
> +/* wcdata: U+0a400,02f00 idx [70-73) */
> +BYTES(0x57030000),176,175,182,183,BYTES(0xdc000000),
> +
> +/* wcdata: U+0a600,0ff00 idx [73-77) */
> +BYTES(0xd55df555),10,85,37,41,213,175,171,180,BYTES(0xf5557003),
> +
> +/* wcdata: U+0a800,16f00 idx [77-81) */
> +BYTES(0xe7555577),86,39,87,88,200,19,10,10,BYTES(0x755f5755),
> +
> +/* wcdata: U+0aa00,1d100 idx [81-86) */
>
> +BYTES(0xf7d5d7f5),93,94,95,24,96,11,97,33,139,138,18,137,BYTES(0x5577f555),
> +
> +/* wcdata: U+0fe00,1f900 idx [86-90) */
> +BYTES(0xd5557c2e),174,173,172,10,147,148,149,150,BYTES(0x0330c003),
> +
> +/* wcdata: U+10200,01700 idx [90-94) */
> +BYTES(0x75555555),7,14,63,62,23,23,61,61,BYTES(0x5fd5dddd),
> +
> +/* wcdata: U+10a00,02500 idx [94-97) */
> +BYTES(0x755555d7),100,101,39,203,BYTES(0xd5555555),
> +
> +/* wcdata: U+11000,02d00 idx [97-101) */
> +BYTES(0x57d7d7d7),11,103,104,10,41,105,60,10,BYTES(0xa555d555),
> +
> +/* wcdata: U+11200,0a900 idx [101-106) */
> +BYTES(0x7d5555f5),10,111,10,112,0,92,91,19,176,41,90,89,BYTES(0x75d7cf75),
> +
> +/* wcdata: U+11400,11100 idx [106-111) */
>
> +BYTES(0x57d55fd5),103,44,26,114,115,110,109,41,108,107,106,19,BYTES(0x57d7d5f7),
> +
> +/* wcdata: U+11600,11500 idx [111-115) */
> +BYTES(0x55f557d5),117,7,118,119,0,97,7,116,BYTES(0x5fd55555),
> +
> +/* wcdata: U+11800,10f00 idx [115-118) */
> +BYTES(0x555555f5),10,121,7,13,BYTES(0x55555f55),
> +
> +/* wcdata: U+11a00,11700 idx [118-122) */
> +BYTES(0x555f5fd7),123,124,125,126,31,127,120,55,BYTES(0x5555557d),
> +
> +/* wcdata: U+11c00,11d00 idx [122-126) */
> +BYTES(0x55fd55d5),128,129,130,131,0,133,119,132,BYTES(0x555d57d5),
> +
> +/* wcdata: U+11e00,1b100 idx [126-129) */
> +BYTES(0xd5555555),134,169,170,181,BYTES(0x00003d5c),
> +
> +/* wcdata: U+13400,1f300 idx [129-133) */
> +BYTES(0x555555d5),135,160,200,161,200,162,163,164,BYTES(0xcf0cc0f0),
> +
> +/* wcdata: U+16a00,1f500 idx [133-137) */
> +BYTES(0xd5555555),107,154,195,155,194,177,156,157,BYTES(0xd57df3c0),
> +
> +/* wcdata: U+18a00,11900 idx [137-140) */
> +BYTES(0xc0000000),170,0,7,122,BYTES(0x7d555555),
> +
> +/* wcdata: U+1b200,1f100 idx [140-143) */
> +BYTES(0xc0000000),182,0,167,168,BYTES(0x555f5555),
> +
> +/* wcdata: U+1bc00,03100 idx [143-147) */
> +BYTES(0x557d5555),136,73,0,200,178,181,180,179,BYTES(0x30c300c3),
> +
> +/* wcdata: U+1d200,0ab00 idx [147-150) */
> +BYTES(0x55555755),61,0,0,98,BYTES(0x75555555),
> +
> +/* wcdata: U+1da00,0d700 idx [150-154) */
> +BYTES(0x557ffaea),140,54,92,141,6,1,0,200,BYTES(0xaab00000),
> +
> +/* wcdata: U+1e000,0fb00 idx [154-157) */
> +BYTES(0x5555557f),142,143,144,26,BYTES(0x5555555d),
> +
> +/* wcdata: U+1e200,10100 idx [157-160) */
> +BYTES(0x75555555),77,0,0,60,BYTES(0xd5555555),
> +
> +/* wcdata: U+1e800,10300 idx [160-163) */
> +BYTES(0x5d555555),104,0,0,99,BYTES(0x5555d555),
> +
> +/* wcdata: U+1f000,10d00 idx [163-166) */
> +BYTES(0x57555557),195,199,0,102,BYTES(0x55555575),
> +
> +/* wcdata: U+1f200,11300 idx [166-171) */
>
> +BYTES(0x55557fc3),170,182,166,165,183,0,0,107,113,7,42,41,BYTES(0x5555f7d7),
> +
> +/* wcdata: U+1f400,16b00 idx [171-174) */
> +BYTES(0xc00003c0),181,159,158,104,BYTES(0x555555d5),
> +
> +/* wcdata: U+1f600,18700 idx [174-178) */
> +BYTES(0xff005400),153,152,186,151,0,0,0,177,BYTES(0xc0000000),
> +
> +/* wcdata: U+1fa00,1e100 idx [178-181) */
> +BYTES(0x555fd555),146,170,183,104,BYTES(0x555555d5),
> +
> +/* wcdata: U+00000,1e900 idx [181-183) */
> +0,0,0,145,BYTES(0x55555755),
> +
> +/* wcdata: U+00000,1f700 idx [183-185) */
> +0,0,0,182,BYTES(0x75555555),
> +
> +/* wcdata: U+00000,1ff00 idx [185-187) */
> +0,0,0,215,BYTES(0xd5555555),
> diff --git a/src/ctype/wide.h b/src/ctype/wide.h
> deleted file mode 100644
> index e403c9a..0000000
> --- a/src/ctype/wide.h
> +++ /dev/null
> @@ -1,65 +0,0 @@
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,18,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,19,16,20,21,22,16,16,16,23,16,16,24,25,26,27,28,17,
>
> -17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,29,
>
> -17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,
>
> -17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,
>
> -17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,
>
> -17,17,17,17,17,17,17,17,30,16,16,16,16,31,16,16,17,17,17,17,17,17,17,17,17,17,
>
> -17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,
>
> -17,17,17,17,17,17,17,32,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,17,17,16,16,16,33,
>
> -34,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,35,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,17,
>
> -17,17,17,17,17,17,36,17,17,37,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,17,38,39,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
>
> -16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,
> -16,16,16,16,16,16,16,40,41,42,43,44,45,46,47,16,48,49,16,16,16,16,
>
> -16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,
>
> -255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,12,0,6,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,30,9,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,96,0,0,48,0,0,0,0,0,0,255,15,0,0,0,0,128,0,0,8,
>
> -0,2,12,0,96,48,64,16,0,0,4,44,36,32,12,0,0,0,1,0,0,0,80,184,0,0,0,0,0,0,0,224,
>
> -0,0,0,1,128,0,0,0,0,0,0,0,0,0,0,0,24,0,0,0,0,0,0,33,0,0,0,0,0,0,0,0,0,0,0,0,0,
> -0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,251,255,255,255,255,255,255,255,
>
> -255,255,255,15,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,255,255,255,255,255,255,255,255,255,255,63,0,0,0,255,15,255,255,255,255,
>
> -255,255,255,127,254,255,255,255,255,255,255,255,255,255,127,254,255,255,255,
>
> -255,255,255,255,255,255,255,255,255,224,255,255,255,255,255,254,255,255,255,
>
> -255,255,255,255,255,255,255,127,255,255,255,255,255,7,255,255,255,255,15,0,
>
> -255,255,255,255,255,127,255,255,255,255,255,0,255,255,255,255,255,255,255,255,
>
> -255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,0,
>
> -0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,31,255,255,255,255,255,255,127,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,
> -255,255,31,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,15,0,0,0,0,0,0,0,0,0,0,0,0,0,255,3,0,0,255,255,255,255,247,255,127,15,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,254,255,255,255,255,255,255,255,255,255,255,
>
> -255,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,127,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,15,0,0,0,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,255,255,255,255,255,255,255,255,255,255,255,7,0,255,255,255,127,0,0,0,0,0,
>
> -0,7,0,240,0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
> -255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -15,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,0,0,0,0,0,0,0,0,0,0,
>
> -0,0,0,0,0,0,0,0,0,0,0,0,0,64,254,7,0,0,0,0,0,0,0,0,0,0,0,0,7,0,255,255,255,
>
> -255,255,15,255,1,3,0,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,255,255,
>
> -1,224,191,255,255,255,255,255,255,255,255,223,255,255,15,0,255,255,255,255,
>
> -255,135,15,0,255,255,17,255,255,255,255,255,255,255,255,127,253,255,255,255,
>
> -255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,
>
> -159,255,255,255,255,255,255,255,63,0,120,255,255,255,0,0,4,0,0,96,0,16,0,0,0,
>
> -0,0,0,0,0,0,0,248,255,255,255,255,255,255,255,255,255,255,0,0,0,0,0,0,255,255,
>
> -255,255,255,255,255,255,63,16,39,0,0,24,240,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
> -0,0,0,0,0,0,0,0,0,0,0,0,255,15,0,
>
> -0,0,224,255,255,255,255,255,255,255,255,255,255,255,255,123,252,255,255,255,
>
> -255,231,199,255,255,255,231,255,255,255,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,
> -0,15,7,7,0,63,0,0,0,0,0,0,0,0,0,0,0,0,0,
> --
> 2.51.2
>
>
Content of type "text/html" skipped
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
