[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Fri, 30 Nov 2012 18:04:54 -0600
From: Rob Landley <rob@...dley.net>
To: musl@...ts.openwall.com
Subject: Re: Summary of 1.0 marketing plan/scheme/nefarious plot from
IRC.
On 11/29/2012 07:27:01 PM, Rich Felker wrote:
> On Thu, Nov 29, 2012 at 02:50:03PM -0600, Rob Landley wrote:
> > Notes from the discussion we had on IRC, plus some further random
> > thoughts on telling the world about musl:
> >
> > - wait until 1.0 so it's most likely to works for them.
> > - People who take a look and wander off again are less likely to
> > take another look,
> > so try to make a spash when you're _ready_, not before.
>
> Agreed. I never really wrote down this principle before, but it's been
> in my mind for a long time. The main publicizing I've been doing so
> far has been in niche places (like mips and microblaze lists) and
> making sure musl is well-indexed in databases like freecode (formerly
> freshmeat) and ohloh. I suspect there are a lot more of the latter we
> should also hit.
Having a good 'elevator pitch' is nice too.
Musl is a brand new C library that bridges the gap between Linux and
Android.
It's BSD licensed and Posix-2008 compliant, written from scratch to
be small
simple and clean with no legacy baggage and good thread support. The
reason
you've never heard of it is the first git commit was February 2011.
> > - chroot for each target with native development tools
> > - system images for qemu maybe?
>
> It would be really cool if we had "system root trees" rather than
> "system images", with virtual 9p for booting qemu. By the way, getting
> qemu working on musl should probably be in our compatibility goals for
> 1.0.
It's slightly more complicated than that, or I'd already have done it.
:)
Making client side initramfs work is easy, but it needs a server. Last
time I looked into this (circa
http://landley.net/notes-2011.html#29-04-2011), it would require diod
on the host acting as a server to work on most targets. While this
isn't much different than qemu's cifs export mode launching samba,
people are much less likely to have diod already installed on the host.
QEMU does have a built-in 9p server, virtfs requiring a virtio
transport. Unfortunately, between qemu-system-blah and the kernel
virtio transport isn't supported equally on all targets. Specifically,
I couldn't get virtio to work on arm, which is kinda important.
I still have the "i686-virtio" and powerpc-virto targets Alessio Bogani
contributed last year, and they mostly work (just fixed minor bit rot
with qemu getting renamed to qemu-system-i386). That's actually using
virtio block device and virtio network, not virtfs, but the principle's
the same. (Think of virtio as a bus and then other things get
implemented on top of that bus; even though the bus is simpler than PCI
support for it may not exist in the target we want because nobody's
bothered yet.)
I also looked at getting virtio console working, but at the time the UI
for it was appalling:
http://lists.gnu.org/archive/html/qemu-devel/2011-05/msg00390.html
> > - launch x11 vnc server and display in tightvnc window?
> > - jslinux live image on website
>
> Love this idea. I'd like it even better if we could adapt jslinux to
> start with a preinitialized memory/machine-state image rather than
> having to spend 15-20 seconds booting. BTW I think we need to look
> into whether there are license issues with using jslinux...?
The actual javascript source says redistribution requires permission of
the author, so yeah. Somebody has to ping fabrice. (Or else reference
the files off his web page, which is hugely impolite.)
I wonder what the busybox guys did? Let's see, google for "jslinux
busybox" and bubble up through the archive a bit to find the commit
that added it to the website is here:
http://git.uclibc.org/busybox-website/commit?id=317871f1a85816e3c0f405f00b2f081978fa9caa
And the jslinux in that still says you need Fabrice's permission to
redistribute, commit comment is just that Denys added it and no further
details. Date is March 4 of this year so check the mailing list archive
on the website...
And this is the start of the thread about it:
http://lists.busybox.net/pipermail/busybox/2012-March/077498.html
Content of message, and I quote, "Hey guys, Check this out!". One
reply, "I saw it. Very cool." No mention of license or permission.
(These are the people who took over the project that's the poster child
for GPL enforcement. Wheee...)
Yes, by all means, let's ask Fabrice. (As musl maintainer, you should
probably do it.)
> By the way, somebody also mentioned a mips emulator in js we could use
> instead or in addition to jslinux, but I don't know if it emulates
> enough hardware to get linux up.
Yay!
> > - distro conversions
> > - leverage existing repositories, don't fall into the buildroot
> trap
>
> I think I've already avoided this trap without avoiding new distros,
> by building a community of people who like rolling their own distros
> but not getting involved in the distro/build stuff at all myself. This
> has been great for getting musl a lot of testing with lower entry
> barrier than large existing distros, but adding the latter now would
> be a good next step.
You've done great. My point here is just "how can we leverage the work
other people have done without distracting ourselves more than
necessary".
If we can come up with a musl chroot that debootstrap runs in (the
"compile packages" not just "install binaries from repo" variant), or a
musl chroot that emerge can build the world in, or koji:
http://opensource.com/life/11/7/free-sake-story-koji?sc_cid=70160000000IDmjAAG
Or whatever SuSE's variant is (blanking on the name), or buildroot, or
arch's pacman, or...
(I'd mentino Yocto's poky but after meego and tizen and such it's
really hard to _care_ about any of the Linux foundation's "look,
another not-android!" gyrations.)
I wandered aimlessly through this bit more than once over the years and
it's why I have the "bootstrap" stuff in my control-images. What I was
trying to do was extend the basic "simplest self-hosting build
environment" up to whatever the various distros needed and then let
them build themselves. It's a finite amount of work. But it's an
_undocumented_ finite amount of work and all these build systems have
DEEP assumptions that they are building under themselves. (Yes, even
gentoo. It's actually worse than Debian about it.)
Basically, I started researching gentoo bootstrapping in 2008:
http://landley.net/notes-2008.html#17-12-2008
My friend Mirell took a serious stab at getting it working in 2009:
http://web.archive.org/web/20090411155149/http://impactlinux.com/hg/gfs
I took my own stab at it in 2010:
http://landley.net/notes-2010.html#15-09-2010
And then tried a Funtoo version in 2011:
http://www.funtoo.org/wiki/ReBootstrap
And each time, when you try to create the base system from something
other than an existing gentoo system it just _melted_down_ and went off
on a tangent to fix something else.
I eventually went "ok, there are three problems here":
1) I'm trying to build a distro on something other than an existing
copy of that distro.
2) I'm trying to use a different libc (and different toolchain, but
that's less of an issue).
3) The existing system I'm providing uses busybox instead of the
specific versions of the gnu/dammit tools they expect (and boy were
they version-sensitive and configuration sensitive, things like python
requiring bzip2 shared libraries to exist on the host when it's built
or else package extraction won't work because portage doesn't know how
to pipe stuff through bzip2, just use the optional python shared
library wrapper).
So I went "if I build Linux From Scratch I can deal with 2 and 3
separately, and then rebootstrap portage under the resulting LFS system
on the target, and it's a huge digression but can be completely
automated, so..." (And that's before getting into trying to wean
portage off of the requirement of explicitly naming every supported
target in every single ebuild file in the entire tree, so that adding a
new target requires touching every file in the portage tree and the
idea of telling gentoo "just build whatever architecture the host is"
is fundamentally against the way they designed it... Yeah, that was me
going "funtoo, how does _that_ work"?)
You may have noticed that my todo list has archaeological strata. Right
now Toybox has put all this on hold, and the rise of Android has
probably rendered some of it moot...
An advantage YOU have is existing gentoo developers with interest who
you may be able to foist this off on. Plus you're just trying to swap
out _one_ component at a time, and only support a few specific
architectures.
> > - Write linux from scratch "musl hint", contribute it to LFS, then
> link
> > to it on LFS website from musl website.
>
> Can you elaborate on what a "musl hint" would be?
Basically they're standalone "or you could do it _this_ way"
explanations for alternatives or extensiosn to the way Linux From
Scratch does stuff:
http://hints.linuxfromscratch.org
http://www.linuxfromscratch.org/hints/read.html
They're referenced from time to time in the main Linux From Scratch
book, ala:
http://www.linuxfromscratch.org/lfs/view/stable/chapter08/kernel.html
And at the end it's one of the things they point you at:
http://www.linuxfromscratch.org/lfs/view/stable/chapter09/whatnow.html
The "Beyond Linux From Scratch" book more or less evolved out of the
pile of hints, trying to turn it into a bigger systematic sequel book
of additional stuff you can build. But the hints profide alternatives
to the base infrastructure, such as BSD-style init scripts instead of
SysV style:
http://www.linuxfromscratch.org/hints/downloads/files/bsd-init.txt
Or using uClibc instead of glibc in the base system:
http://www.linuxfromscratch.org/hints/downloads/files/uclibc.txt
(Of course BLFS has its own problems. Their last release was in 2008
and they went to one of those "everybody just use svn and ping us if
it's broken" things because their development model's become too
dysfunctional to cut releases anymore.)
Anyway, advantage of this: if they take it you can then link to it from
musl's website and go "hey build your own system guys, here's how you
do a musl based LFS". It's not _explicitly_ that LFS would be endorsing
you, but it feels that way a little bit. :)
> > - is userbase of glibc, uClibc, klibc, or dietlibc better served by
> > musl?
> > - contribute musl option to buildroot?
> > - contribute musl option to crosstool-ng?
>
> There's demand for musl in crosstool-ng at least; the maintainer has
> expressed this, but is largely unfamiliar with musl and waiting for a
> patch from someone willing to do it.
He's a good guy. I'm not a fan of the design of his project (driving it
makes The Tardis look user-friendly), but a lot of people find it
useful.
> > - Ask mentor graphics (formly code sourcery) to do a musl
> toolchain?
> > - LOTS of proprietary embedded devs use this one, it's
> > "professional".
>
> I suspect they might be particularly interested since their customers
> might need to avoid copyleft code.
Or at least the random threat of lawsuits even if they _try_ to conform.
Not gonna rant. Nope. Not gonna.
> > - windriver.com is now a wholly owned subsidiary of intel
> > - klibc guys are initramfs@...r or embedded@...r (see lists)
> > - ask clibc author Peter Anvin if musl serves his needs?
>
> I think musl essentially obsoletes klibc. The main added cost of musl
> over klibc for initrds, etc. is the soft-float code that will get
> linked in due to printf on archs with no hard-float. If this is a
> problem, we could eventually support an option with musl to inhibit
> printf float code from getting linked at static-link time.
A "./configure --disable-soft-float" would be nice just for bragging
rights.
> This is a
> bit nasty but could be done with weak symbol tricks. Of course, even
> with all the soft-float code, I think binaries linked to musl will be
> significantly smaller than all the kernel modules, firmware, etc.
> stored on an initrd.
>
> On the other side, the big advantage of musl over klibc is that you
> don't have to worry about whether the apps you need will be okay with
> the limited/non-conformant functionality provided by klibc.
> Well-written apps should "just work", and be small.
An _ideal_ situation would be to get Peter Anvin to end klibc
development, point his users at musl, and possibly even become a musl
user/developer. I admit the diplomacy to arrange such an outcome is
beyond me.
However, it would be nice to ask him what he gets out of klibc. If it's
providing a reference implementation or using it to test system call
generation or some such, possibly musl could fill this role. If there's
something specific klibc does, it would be nice if musl could also do
it.
> > - websites that might review musl if we ask nicely:
> > - linux
> > - lwn.net (submit via lwn@....net)
> > - h-online (ping @codepope on twitter)
> > - Linux Journal
> > - Linux Today (they'll just link elsewhere)
>
> I think getting the 1.0 release on Reddit, Hacker News, etc. would be
> useful too. If for nothing else, as one step closer to the magic 7.
Obviously, none of these lists are exhaustive. :)
But if we add to them much, collating to create a master list sounds
like a job for a wiki.
> > - android
> > - not personally familiar, google for "android news" finds
> several.
> > - works well with android kernel, installs side-by-side with
> > bionic,
> > static links well, doesn't introduce any new licensing issues,
> > provides full posix environment, active and responsive dev
> > community.
>
> Yep. I'm not familiar with these sites either though.
Enthuse to the cyanogen mod guys, send them to enthuse to others?
http://www.homeonthestrange.com/view.php?ID=28
> > - ask people on mailing list and irc to blog/tweet about the 1.0
> > release when it
> > happens.
>
> Yes, this is definitely an under-utilized aspect of the (now rather
> large) community.
Word of mouth is the best asset, but buzz is all about combo bonuses.
The _other_ thing you might want to do is pull up the original BBC
"kitchen nightmares" on Netflix streaming and watch Gordon arrange a
full set of customers for each failing restaurant, at which point
hilarity inevitably ensues.
"OK, you've achieved basic competence. Now a thundering horde of a
gazillion people
show up wanting to try your thing. How do you avoid service melting
into a puddle
under the load?"
Lots of people are not prepared for _success_. You've got to acclimate
newbies. You've got to have a website that can handle the downloads.
Mailing list traffic spikes. I dunno if this will happen, but _if_ it
does, are you prepared?
Ideally you'd have not just a goot FAQ and web pages but "the book of
musl" which can act as both tutorial and reference (two different
functions), and designated hand-holders in the community fielding a
flood of newbies.
This is probably hashing your chickens before counting them, of course,
but I wanted to at least bring it up.
For example, if you put your musl porting guide online (p.s. put your
musl porting guide online) and you suddenly have three different people
submitting ports to different architectures, the testing burden's gonna
get nuts. The "why doesn't it work with this package" requests could
spike beyond the ability to field. Bug tracker. Translations, including
people translating the documentation and web pages themselves (with all
the version skew that entails). Triaging and merging forks. Patches
winding up in distro repositories and not necessarily making their way
back upstream promptly...
Yeah yeah, such problems we should have. Just don't be surprised by
them. :)
Rob
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
