musl - Re: Supporting git access via smart HTTPS protocol for musl-libc

Follow @Openwall on Twitter for new release announcements and other news

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Wed, 27 Mar 2019 11:26:22 -0600
From: Assaf Gordon <assafgordon@...il.com>
To: musl@...ts.openwall.com, vlse <vlse@...ra.biz>
Cc: dalias@...c.org
Subject: Re: Supporting git access via smart HTTPS protocol for
 musl-libc

Hello,

On 2019-03-26 11:39 p.m., vlse wrote:
> On Tue, Mar 26, 2019 at 08:15:42PM -0400, Rich Felker wrote:
>> On Tue, Mar 26, 2019 at 07:58:35PM -0400, Rich Felker wrote:
>>> On Tue, Mar 26, 2019 at 04:32:32PM -0600, Assaf Gordon wrote:
>>>>>
>>>>> [...] I suspect thttpd is doing something broken with
>>>>> the POST request since the git clone breaks during that. 
>>>>
>>>> The same happened to me with busybox, and was solved by forcing:
>>>>
>>>>      export HTTP_CONTENT_ENCODING=gzip
>>>>
>>> Amazingly, this works, but only if I do it only for
>>> REQUEST_METHOD=POST. Otherwise it breaks the GET request and it never
>>> makes it to the POST.

> This does not works:
> mkdir musl.git
> cd musl.git
> git init
> git remote add -t master -m master origin https://git.musl-libc.org/git/musl
> 
> git-fetch gives errors:
>    fatal: protocol error: bad line length character: erro

Thanks for checking and for the detailed reproducer.

This error tells me there is (again) a binary/text conflict,

The (text) git protocol always starts with 4 hex digits
indicating length. If the input is gzip'd and wasn't
decompressed, the length indicator will be invalid.

Digging further, it seems that in this case the "git fetch"
command sent POST data which is *not* compressed.
Forcing "CONTENT_ENCODING=gzip" always or for all POST request
is not sufficient.

I now use the following shell wrapper, which works for
both "git clone" and your "git fetch" case.
I won't claim it's pretty, but it works (with busybox httpd).
The "echo>&2" will show up on webserver's error log.

----
#!/bin/sh

export GIT_HTTP_EXPORT_ALL=true
export GIT_PROJECT_ROOT=/home/gordon/projects/

echo START - $REQUEST_METHOD $PATH_INFO >&2

# Check if POST data is text or binary, add HTTP header if needed.
if test "$REQUEST_METHOD" = POST ; then
     t=$(mktemp -t git-http-backend-XXXXXX)
     # Store STDIN and examine it
     cat - > $t

     # Git (text) protocol starts with 4 hex digits indicating length.
     # If the first 4 bytes aren't hexdigits, assume STDIN is compressed.
     if head -c4 $t | grep -q '^[0-9a-f][0-9a-f][0-9a-f][0-9a-f]$' ; then
         echo "POST data is not gzipped" >&2
     else
         echo "POST data is gzipped" >&2
         export HTTP_CONTENT_ENCODING=gzip
     fi

     # restore STDIN
     exec < $t
     rm $t
fi
/usr/lib/git-core/git-http-backend
echo END - $REQUEST_METHOD $PATH_INFO >&2
echo >&2
echo >&2

----

As Rich said, there's got to be a better way...


Regards,
   -assaf

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.