Date: Thu, 2 Feb 2012 07:42:12 +0400 From: Solar Designer <solar@...nwall.com> To: owl-dev@...ts.openwall.com Subject: Re: blists status? Hi Lauri, blists is not (yet) part of Owl, so this is slightly off-topic for this mailing list. That said, I don't mind discussing it in here briefly before we decide on next steps. And we might in fact include blists into Owl after we get a web server into Owl base tree (there's almost no point in having blists without a web server). It is impressive how you analyzed blists before bringing this topic up. Are you using blists somewhere? On Tue, Jan 31, 2012 at 08:30:10AM +0200, Lauri Kokkonen wrote: > I have been considering to work on blists and want to ask if this is > something you would like to see worked on during GSoC. Maybe - if we're in GSoC again and if we have an excellent match - although blists for this year's GSoC was not specifically planned (rather, there are blists tasks assigned to abc - although as you can see he's been inactive at blists lately, so re-assigning is possible). > Looking at the source there seem to be potential portability issues. First, > since the index is read directly from file to structs there arises > endianness problems when carrying the index file from LE arch to BE, for > instance. Second, structs might be padded differently on different archs. Definitely. In its current form, bindex and bit are supposed to be from the same build of blists and running on the same system. > Although maybe this is not serious since the index can just be generated > from the beginning, and fixing those issues will probably just make bindex > slower. Yes, although I am more concerned about source code size and complexity increase for a relatively rare use case (e.g., with bit and bindex on different machines accessing shared storage). > The HTML output stuff has some duplicate code at least for reading the aday > index and printing a list of messages for month and day indices, and might > benefit from a cleanup. Perhaps. > For new features I would personally like to see threaded view. Yes, it's one of the desirable features. > This should be possible to add without changes to the index format. Currently, the threads are "flattened" - each message only has one thread-prev and one thread-next message - that's in the index file. (Yes, this means that thread-prev is not necessarily the message that the current message is a direct reply to. It may also be a message posted to the same thread after the replied-to message was posted.) So for a flat threaded view, everything is easy (and this may be an initial development milestone). For a tree threaded view, we may either scan the flattened thread and match instances of msgid_hash vs. irt_hash (in bit) or enhance the index format and record more-direct pointers up and down in the tree in the index. I currently think that the former approach is preferable, but this may need further consideration. > Another often seen > feature is author view but I am not so sure about it; do you want to > build a rich set of features or go with a compact piece of software? We need to strike a balance. Besides code size, also important are code complexity and potential security risks. Merely adding a new display mode is likely low-risk, whereas e.g. adding a search feature is higher risk (extra user input). That said, author view was not requested so far, so it is not on the agenda, whereas search was requested, so it is. Also, maybe having "advanced search" would eliminate the need for author view (if there's such need at all). > A search feature would be wonderful. Probably some kind of search index that > links the search term space to individual messages has to be built for > this. There are many different options on how to do this, and some problems Right. abc looked into using Xapian for this - in fact, he experimented with it briefly in a modified blists tree. He would probably proceed to implement it for real by now, but we agreed on other priorities initially - such as index pages with Subjects and Froms (already in blists 1.0), "Recent messages" lists (in CVS), support for different character encodings (in CVS for headers, not yet implemented for bodies). > like the order of mbox index entries possibly changing (if I have understood > correctly), Why would it change? Are you possibly referring to the qsort() call? If so, that is something abc ran into too and we discussed ways around it - but I forgot the details (may need to dig up chat logs). > so that it probably needs a few candidate implementations and > serious testing. Definitely. And frankly, I am concerned about running a version of bit linked against Xapian or whatever and passing untrusted user input from the web into that library... I am not sure how to deal with this best. > Do you have other TODO items for blists? Yes. I have a cryptic TODO list originally intended for myself only (before abc got involved in blists development). To name a few items (not yet mentioned above): support for downloads of attached files, RSS or Atom feed, highlight search keywords based on Referer (such as from Google search), and many smaller enhancements. Thanks, Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.