Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Fri, 30 Nov 2012 18:04:54 -0600
From: Rob Landley <>
Subject: Re: Summary of 1.0 marketing plan/scheme/nefarious plot from

On 11/29/2012 07:27:01 PM, Rich Felker wrote:
> On Thu, Nov 29, 2012 at 02:50:03PM -0600, Rob Landley wrote:
> > Notes from the discussion we had on IRC, plus some further random
> > thoughts on telling the world about musl:
> >
> > - wait until 1.0 so it's most likely to works for them.
> >   - People who take a look and wander off again are less likely to
> > take another look,
> >     so try to make a spash when you're _ready_, not before.
> Agreed. I never really wrote down this principle before, but it's been
> in my mind for a long time. The main publicizing I've been doing so
> far has been in niche places (like mips and microblaze lists) and
> making sure musl is well-indexed in databases like freecode (formerly
> freshmeat) and ohloh. I suspect there are a lot more of the latter we
> should also hit.

Having a good 'elevator pitch' is nice too.

   Musl is a brand new C library that bridges the gap between Linux and  
   It's BSD licensed and Posix-2008 compliant, written from scratch to  
be small
   simple and clean with no legacy baggage and good thread support. The  
   you've never heard of it is the first git commit was February 2011.

> >     - chroot for each target with native development tools
> >       - system images for qemu maybe?
> It would be really cool if we had "system root trees" rather than
> "system images", with virtual 9p for booting qemu. By the way, getting
> qemu working on musl should probably be in our compatibility goals for
> 1.0.

It's slightly more complicated than that, or I'd already have done it.  

Making client side initramfs work is easy, but it needs a server. Last  
time I looked into this (circa, it would require diod  
on the host acting as a server to work on most targets. While this  
isn't much different than qemu's cifs export mode launching samba,  
people are much less likely to have diod already installed on the host.

QEMU does have a built-in 9p server, virtfs requiring a virtio  
transport. Unfortunately, between qemu-system-blah and the kernel  
virtio transport isn't supported equally on all targets. Specifically,  
I couldn't get virtio to work on arm, which is kinda important.

I still have the "i686-virtio" and powerpc-virto targets Alessio Bogani  
contributed last year, and they mostly work (just fixed minor bit rot  
with qemu getting renamed to qemu-system-i386). That's actually using  
virtio block device and virtio network, not virtfs, but the principle's  
the same. (Think of virtio as a bus and then other things get  
implemented on top of that bus; even though the bus is simpler than PCI  
support for it may not exist in the target we want because nobody's  
bothered yet.)

I also looked at getting virtio console working, but at the time the UI  
for it was appalling:

> >       - launch x11 vnc server and display in tightvnc window?
> >     - jslinux live image on website
> Love this idea. I'd like it even better if we could adapt jslinux to
> start with a preinitialized memory/machine-state image rather than
> having to spend 15-20 seconds booting. BTW I think we need to look
> into whether there are license issues with using jslinux...?

The actual javascript source says redistribution requires permission of  
the author, so yeah. Somebody has to ping fabrice. (Or else reference  
the files off his web page, which is hugely impolite.)

I wonder what the busybox guys did? Let's see, google for "jslinux  
busybox" and bubble up through the archive a bit to find the commit  
that added it to the website is here:

And the jslinux in that still says you need Fabrice's permission to  
redistribute, commit comment is just that Denys added it and no further  
details. Date is March 4 of this year so check the mailing list archive  
on the website...

And this is the start of the thread about it:

Content of message, and I quote, "Hey guys, Check this out!". One  
reply, "I saw it. Very cool." No mention of license or permission.  
(These are the people who took over the project that's the poster child  
for GPL enforcement. Wheee...)

Yes, by all means, let's ask Fabrice. (As musl maintainer, you should  
probably do it.)

> By the way, somebody also mentioned a mips emulator in js we could use
> instead or in addition to jslinux, but I don't know if it emulates
> enough hardware to get linux up.


> > - distro conversions
> >   - leverage existing repositories, don't fall into the buildroot  
> trap
> I think I've already avoided this trap without avoiding new distros,
> by building a community of people who like rolling their own distros
> but not getting involved in the distro/build stuff at all myself. This
> has been great for getting musl a lot of testing with lower entry
> barrier than large existing distros, but adding the latter now would
> be a good next step.

You've done great. My point here is just "how can we leverage the work  
other people have done without distracting ourselves more than  

If we can come up with a musl chroot that debootstrap runs in (the  
"compile packages" not just "install binaries from repo" variant), or a  
musl chroot that emerge can build the world in, or koji:

Or whatever SuSE's variant is (blanking on the name), or buildroot, or  
arch's pacman, or...

(I'd mentino Yocto's poky but after meego and tizen and such it's  
really hard to _care_ about any of the Linux foundation's "look,  
another not-android!" gyrations.)

I wandered aimlessly through this bit more than once over the years and  
it's why I have the "bootstrap" stuff in my control-images. What I was  
trying to do was extend the basic "simplest self-hosting build  
environment" up to whatever the various distros needed and then let  
them build themselves. It's a finite amount of work. But it's an  
_undocumented_ finite amount of work and all these build systems have  
DEEP assumptions that they are building under themselves. (Yes, even  
gentoo. It's actually worse than Debian about it.)

Basically, I started researching gentoo bootstrapping in 2008:

My friend Mirell took a serious stab at getting it working in 2009:

I took my own stab at it in 2010:

And then tried a Funtoo version in 2011:

And each time, when you try to create the base system from something  
other than an existing gentoo system it just _melted_down_ and went off  
on a tangent to fix something else.

I eventually went "ok, there are three problems here":

1) I'm trying to build a distro on something other than an existing  
copy of that distro.

2) I'm trying to use a different libc (and different toolchain, but  
that's less of an issue).

3) The existing system I'm providing uses busybox instead of the  
specific versions of the gnu/dammit tools they expect (and boy were  
they version-sensitive and configuration sensitive, things like python  
requiring bzip2 shared libraries to exist on the host when it's built  
or else package extraction won't work because portage doesn't know how  
to pipe stuff through bzip2, just use the optional python shared  
library wrapper).

So I went "if I build Linux From Scratch I can deal with 2 and 3  
separately, and then rebootstrap portage under the resulting LFS system  
on the target, and it's a huge digression but can be completely  
automated, so..." (And that's before getting into trying to wean  
portage off of the requirement of explicitly naming every supported  
target in every single ebuild file in the entire tree, so that adding a  
new target requires touching every file in the portage tree and the  
idea of telling gentoo "just build whatever architecture the host is"  
is fundamentally against the way they designed it... Yeah, that was me  
going "funtoo, how does _that_ work"?)

You may have noticed that my todo list has archaeological strata. Right  
now Toybox has put all this on hold, and the rise of Android has  
probably rendered some of it moot...

An advantage YOU have is existing gentoo developers with interest who  
you may be able to foist this off on. Plus you're just trying to swap  
out _one_ component at a time, and only support a few specific  

> > - Write linux from scratch "musl hint", contribute it to LFS, then  
> link
> >   to it on LFS website from musl website.
> Can you elaborate on what a "musl hint" would be?

Basically they're standalone "or you could do it _this_ way"  
explanations for alternatives or extensiosn to the way Linux From  
Scratch does stuff:

They're referenced from time to time in the main Linux From Scratch  
book, ala:

And at the end it's one of the things they point you at:

The "Beyond Linux From Scratch" book more or less evolved out of the  
pile of hints, trying to turn it into a bigger systematic sequel book  
of additional stuff you can build. But the hints profide alternatives  
to the base infrastructure, such as BSD-style init scripts instead of  
SysV style:

Or using uClibc instead of glibc in the base system:

(Of course BLFS has its own problems. Their last release was in 2008  
and they went to one of those "everybody just use svn and ping us if  
it's broken" things because their development model's become too  
dysfunctional to cut releases anymore.)

Anyway, advantage of this: if they take it you can then link to it from  
musl's website and go "hey build your own system guys, here's how you  
do a musl based LFS". It's not _explicitly_ that LFS would be endorsing  
you, but it feels that way a little bit. :)

> > - is userbase of glibc, uClibc, klibc, or dietlibc better served by
> > musl?
> >   - contribute musl option to buildroot?
> >   - contribute musl option to crosstool-ng?
> There's demand for musl in crosstool-ng at least; the maintainer has
> expressed this, but is largely unfamiliar with musl and waiting for a
> patch from someone willing to do it.

He's a good guy. I'm not a fan of the design of his project (driving it  
makes The Tardis look user-friendly), but a lot of people find it  

> >   - Ask mentor graphics (formly code sourcery) to do a musl  
> toolchain?
> >     - LOTS of proprietary embedded devs use this one, it's
> > "professional".
> I suspect they might be particularly interested since their customers
> might need to avoid copyleft code.

Or at least the random threat of lawsuits even if they _try_ to conform.

Not gonna rant. Nope. Not gonna.

> >     - is now a wholly owned subsidiary of intel
> >   - klibc guys are initramfs@...r or embedded@...r (see lists)
> >     - ask clibc author Peter Anvin if musl serves his needs?
> I think musl essentially obsoletes klibc. The main added cost of musl
> over klibc for initrds, etc. is the soft-float code that will get
> linked in due to printf on archs with no hard-float. If this is a
> problem, we could eventually support an option with musl to inhibit
> printf float code from getting linked at static-link time.

A "./configure --disable-soft-float" would be nice just for bragging  

> This is a
> bit nasty but could be done with weak symbol tricks. Of course, even
> with all the soft-float code, I think binaries linked to musl will be
> significantly smaller than all the kernel modules, firmware, etc.
> stored on an initrd.
> On the other side, the big advantage of musl over klibc is that you
> don't have to worry about whether the apps you need will be okay with
> the limited/non-conformant functionality provided by klibc.
> Well-written apps should "just work", and be small.

An _ideal_ situation would be to get Peter Anvin to end klibc  
development, point his users at musl, and possibly even become a musl  
user/developer. I admit the diplomacy to arrange such an outcome is  
beyond me.

However, it would be nice to ask him what he gets out of klibc. If it's  
providing a reference implementation or using it to test system call  
generation or some such, possibly musl could fill this role. If there's  
something specific klibc does, it would be nice if musl could also do  

> > - websites that might review musl if we ask nicely:
> >   - linux
> >     - (submit via
> >     - h-online (ping @codepope on twitter)
> >     - Linux Journal
> >     - Linux Today (they'll just link elsewhere)
> I think getting the 1.0 release on Reddit, Hacker News, etc. would be
> useful too. If for nothing else, as one step closer to the magic 7.

Obviously, none of these lists are exhaustive. :)

But if we add to them much, collating to create a master list sounds  
like a job for a wiki.

> >   - android
> >     - not personally familiar, google for "android news" finds  
> several.
> >     - works well with android kernel, installs side-by-side with
> > bionic,
> >       static links well, doesn't introduce any new licensing issues,
> >       provides full posix environment, active and responsive dev
> > community.
> Yep. I'm not familiar with these sites either though.

Enthuse to the cyanogen mod guys, send them to enthuse to others?

> > - ask people on mailing list and irc to blog/tweet about the 1.0
> > release when it
> >   happens.
> Yes, this is definitely an under-utilized aspect of the (now rather
> large) community.

Word of mouth is the best asset, but buzz is all about combo bonuses.

The _other_ thing you might want to do is pull up the original BBC  
"kitchen nightmares" on Netflix streaming and watch Gordon arrange a  
full set of customers for each failing restaurant, at which point  
hilarity inevitably ensues.

"OK, you've achieved basic competence. Now a thundering horde of a  
gazillion people
show up wanting to try your thing. How do you avoid service melting  
into a puddle
under the load?"

Lots of people are not prepared for _success_. You've got to acclimate  
newbies. You've got to have a website that can handle the downloads.  
Mailing list traffic spikes. I dunno if this will happen, but _if_ it  
does, are you prepared?

Ideally you'd have not just a goot FAQ and web pages but "the book of  
musl" which can act as both tutorial and reference (two different  
functions), and designated hand-holders in the community fielding a  
flood of newbies.

This is probably hashing your chickens before counting them, of course,  
but I wanted to at least bring it up.

For example, if you put your musl porting guide online (p.s. put your  
musl porting guide online) and you suddenly have three different people  
submitting ports to different architectures, the testing burden's gonna  
get nuts. The "why doesn't it work with this package" requests could  
spike beyond the ability to field. Bug tracker. Translations, including  
people translating the documentation and web pages themselves (with all  
the version skew that entails). Triaging and merging forks. Patches  
winding up in distro repositories and not necessarily making their way  
back upstream promptly...

Yeah yeah, such problems we should have. Just don't be surprised by  
them. :)


Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.