[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 22 Apr 2013 17:52:48 -0400
From: Rich Felker <dalias@...ifal.cx>
To: musl@...ts.openwall.com
Subject: Re: Best place to discuss other lightweight libraries?
On Mon, Apr 22, 2013 at 05:21:25PM +0200, Luca Barbato wrote:
> On 04/22/2013 04:53 PM, Rich Felker wrote:
> > - Thread allergies, i.e. horribly over-complicating program logic to
> > avoid threads. The best examples I can think of are the added logic
> > needed to generalize a program that's reading from ordinary file
> > descriptors (e.g. connection sockets) in an event loop to support
> > SSL sockets or zlib-compressed streams. (Note: there are ways to
> > address this kind of problem more cleanly without threads too, but
> > nobody does it. I can elaborate if anybody's interested.)
>
> I'm interested to read about it.
Well the canonical multi-threaded way to deal with such cases would be
to replace your file descriptor with a pipe or socket file descriptor
connected to a thread that's doing all the translation work (SSL,
zlib, etc.). Then, your main event loop just sees a normal file
descriptor, so you don't have to invade your code that handles
streams/connections/whatever with a new abstraction framework around
file descriptors that also supports SSL or whatever. Not only are such
abstraction layers bloated; they also add a lot of places for bugs to
hide, and they're difficult to get right. For a great example from
just a few days ago, see Bitlbee bug #1046:
http://bugs.bitlbee.org/bitlbee/ticket/1046
But suppose you want to solve this problem without threads. Well, the
basic idea is to do the same thing, but put the translation layer in
your event loop rather than in a thread. That is, make the pipe or
socketpair like you would have done for the threaded solution, but
instead of creating a thread, register the file descriptors with an
event handler in your main event loop. For an example with SSL, when
data comes in on the SSL socket, first the SSL-processing even handler
gets fired, and writes the decrypted stream to a pipe or socketpair.
Then, the event handler for the other end of that pipe/socket runs and
handles it like an ordinary connection.
Obviously this has some degree of additional overhead compared to an
abstraction layer, so I wouldn't recommend doing it for applications
whose JOB is to handle connections or streaming data as efficiently as
possible (e.g. a webserver). But for something like an IRC client or
web browser, where socket performance is irrelevant compared to other
things, it's idiotic to design fancy abstraction layers for reading
and writing to connections when you could just do a design like what I
described above.
> > - DBus.
>
> Sadly nobody is pushing for a better local socket multicast abstraction
> to send notifications back and forth in an efficient fashion.
>
> I'm hoping for nanomsg once it is complete or Binder once it is
> correctly documented ^^; (and thus implemented in more than few forks of
> linux and maybe haiku)
Yes, Binder looks really promising, but I also think part of the
problem is that the need for this kind of interface is exaggerated...
> > - Use of global state. Even seemingly-harmless things like a global
> > registered log function are harmful, because two different libraries
> > (or the main program and a library) might be trying to use the
> > library with the global log destination, and clobbering each other's
> > choices.
>
> For this there aren't solution that won't cause different problems I'm
> afraid.
Sure there are. I get the impression you can tell I was talking about
libav/ffmpeg's log interface. :-) The obvious solution is to bind log
contexts to the object you're acting on. See this nice talk:
http://misko.hevery.com/2008/11/21/clean-code-talks-global-state-and-singletons/
If I remember right, part of the problem way back was that there were
deep function calls that had no context available to them, and that
didn't _need_ a context for anything but logging warnings or whatnot.
Really, the fact that they can fail and want to be able to report
failure means they _do_ need a context, but I can understand the
desire to cheat, especially if there's a performance cost to passing
the context all the way down. In that case, hidden non-global state,
namely thread-local storage, might be an appropriate solution. It's
still hidden state (and thus A Bad Thing), but at least it's no longer
global state.
> > - Designs based on shared libraries, especially lots of them. This
> > creates bloat and often interferes with the ability to use static
> > linking.
>
> Special mention to those that want to do clever stuff on the init
> section (e.g. change a program global state from there)
Did whatever lib did that (OpenAL, was it?) ever fix their bugs?
> > - Dependency on any library with the above problems. :-)
>
> And that kills everybody using glib? *runs and hides*
Yes, basically. Dependency on glib means your library will impose
bloat and it will preclude robustness.
Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
