Date: Wed, 6 Aug 2014 09:57:11 +0200 From: Waldemar Brodkorb <wbx@...nadk.org> To: musl@...ts.openwall.com Subject: Re: Building a solid musl automated-testing framework Hi, Rich Felker wrote, > On Mon, Aug 04, 2014 at 02:18:55PM +0200, Waldemar Brodkorb wrote: > > Hi, > > Rich Felker wrote, > > > > > I'd like to figure out how to setup the openadk test framework, or > > > adapt things from it, for automated testing all musl ports. The repo > > > is here: > > > > > > http://www.openadk.org/cgi-bin/gitweb.cgi?p=adk-test-framework.git > > > > > > There's a lot of stuff hard-coded for the openadk toolchains, whereas > > > I'd like to be able to use it with the musl-cross toolchains which are > > > more canonical. The scripts also seem to be incompatible with busybox > > > (using GNU features in something for making the initramfs, probably > > > cpio?). And by default it tries to test musl 1.0.1 and doesn't have an > > > obvious way to test from git. > > > > I have copied the adk-test-framework logic to a new project. > > http://www.openadk.org/cgi-bin/gitweb.cgi?p=embedded-test.git;a=summary > > > > It is more general and supports buildroot instead of openadk to make > > tests of toolchain building and runtime testing for glibc, uclibc, > > uclibc-ng and musl. > > The big issue with using it directly is that we're not trying to test > whole toolchains, just musl, and testing toolchains instead of just > musl turns a 5 minute build (that's easy to run and re-run whenever > you want to test) into an hours-long process. Only the first run will create a toolchain. On the second run without -c, only the C library will be recompiled and installed. > > The benefit is, you do not need any ready to go kernel and userland, > > because everything got compiled from scratch. So any fixes to > > kernel config or userland can be done. > > > > If there is an interest, I can add support for using an existing > > toolchain (like musl-cross) to build a kernel and userland to do > > the runtime testing. > > I'd still much prefer to have the kernel built outside of the test > system, since it's not a component to be tested and in no way uses or > depends on the components to be tested. I do not agree here. For example the latest kernel includes File-private POSIX locks. https://lwn.net/Articles/586904/ If you want to add the needed bits and bytes to the C library it would be nice to test it. Therefore you need the latest kernel. > The situation is similar for > busybox, which is just used as a shell for executing the tests, not as > a component to be tested itself, but here there's also a consideration > of not introducing regressions in the shell that's controlling the > tests. I'd like to be using a "known good" shell and have only the > tests themselves linking against the new musl being tested. Recently I discovered a bug in uClibc in the machine dependent pipe implementation for sparc by using mksh and its test suite. So running application specific test suites for mksh/busybox might discover C library bugs. So I want to support it. Do you want to use a glibc basesystem to run the musl tests with musl C library as in adk-test-framework or do you want to simply run the test on a musl based system? best regards Waldemar
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.