Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Wed, 27 Mar 2019 11:26:22 -0600
From: Assaf Gordon <>
To:, vlse <>
Subject: Re: Supporting git access via smart HTTPS protocol for


On 2019-03-26 11:39 p.m., vlse wrote:
> On Tue, Mar 26, 2019 at 08:15:42PM -0400, Rich Felker wrote:
>> On Tue, Mar 26, 2019 at 07:58:35PM -0400, Rich Felker wrote:
>>> On Tue, Mar 26, 2019 at 04:32:32PM -0600, Assaf Gordon wrote:
>>>>> [...] I suspect thttpd is doing something broken with
>>>>> the POST request since the git clone breaks during that. 
>>>> The same happened to me with busybox, and was solved by forcing:
>>>>      export HTTP_CONTENT_ENCODING=gzip
>>> Amazingly, this works, but only if I do it only for
>>> REQUEST_METHOD=POST. Otherwise it breaks the GET request and it never
>>> makes it to the POST.

> This does not works:
> mkdir musl.git
> cd musl.git
> git init
> git remote add -t master -m master origin
> git-fetch gives errors:
>    fatal: protocol error: bad line length character: erro

Thanks for checking and for the detailed reproducer.

This error tells me there is (again) a binary/text conflict,

The (text) git protocol always starts with 4 hex digits
indicating length. If the input is gzip'd and wasn't
decompressed, the length indicator will be invalid.

Digging further, it seems that in this case the "git fetch"
command sent POST data which is *not* compressed.
Forcing "CONTENT_ENCODING=gzip" always or for all POST request
is not sufficient.

I now use the following shell wrapper, which works for
both "git clone" and your "git fetch" case.
I won't claim it's pretty, but it works (with busybox httpd).
The "echo>&2" will show up on webserver's error log.


export GIT_PROJECT_ROOT=/home/gordon/projects/


# Check if POST data is text or binary, add HTTP header if needed.
if test "$REQUEST_METHOD" = POST ; then
     t=$(mktemp -t git-http-backend-XXXXXX)
     # Store STDIN and examine it
     cat - > $t

     # Git (text) protocol starts with 4 hex digits indicating length.
     # If the first 4 bytes aren't hexdigits, assume STDIN is compressed.
     if head -c4 $t | grep -q '^[0-9a-f][0-9a-f][0-9a-f][0-9a-f]$' ; then
         echo "POST data is not gzipped" >&2
         echo "POST data is gzipped" >&2
         export HTTP_CONTENT_ENCODING=gzip

     # restore STDIN
     exec < $t
     rm $t
echo >&2
echo >&2


As Rich said, there's got to be a better way...


Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.