| To: meta@public-inbox.org |
| Subject: [WIP] public-inbox 2.2.0 |
| MIME-Version: 1.0 |
| Content-Type: text/plain; charset=utf-8 |
| Content-Disposition: inline |
| |
| This release includes several new features and fixes; mostly |
| around improved integration between inboxes and coderepos for |
| solver. Portability and reliability is also improved, especially |
| in the internal process management of lei. |
| |
| public-inbox-cindex is a new command to index coderepos for |
| WWW search and perform automatic associations between |
| coderepos and inboxes. This makes solver vastly more useful |
| for the WWW UI as admins will no longer have to manually |
| associate coderepos with inboxes. |
| |
| public-inbox-clone gains the ability to mirror entire (or partial) |
| grokmirror-compatible manifests. |
| |
| Internal process and object management data structures are vastly |
| simplified throughout and error handling made more robust. |
| |
| git SHA-256 support remains a work-in-progress for inboxes and |
| extindex due to the need to interoperate with SHA-1 epochs. |
| |
| Upgrading: |
| |
| lei users need to "lei daemon-kill" after installation to load |
| new code. Normal daemons (read-only, and public-inbox-watch) |
| will also need restarts, of course, but there's no |
| backwards-incompatible data format changes so rolling back to |
| older versions is harmless. |
| |
| Compatibility: |
| |
| Uppercase newsgroup names were always broken with IMAP, POP3, and |
| -extindex. Uppercase names will now be lowercased by default and |
| warnings will be emitted. Conflicting newsgroup names (and `inboxdir' |
| entries if `newsgroup' isn't specified) will also generate warnings |
| since they break -extindex and the new -cindex (coderepo index). |
| |
| New users + hackers: |
| |
| The install/ directory includes tools to automate installation and |
| removal of dependencies for stripped-down or full setups. See |
| install/README for more details. |
| |
| treewide |
| |
| * support raw UTF-8 headers from SMTPUTF8 hosts |
| |
| * standardize on `#' prefix for stderr diagnostics (previously `I:') |
| |
| * SHA-256 coderepos are fully supported (but not inboxes, yet) |
| |
| * for daemons serving public traffic, MALLOC_MMAP_THRESHOLD_=131072 is |
| recommended to reduce fragmentation in glibc malloc, while jemalloc |
| (tested as an LD_PRELOAD) is another option (at least for 64-bit). |
| |
| PublicInbox::WWW |
| |
| * support `+' in inbox names |
| |
| * support coderepo displays for systems without cgit |
| |
| * improve display of git tags, commits and trees in $INBOX/$OID/s/ endpoint |
| |
| * numerous memory usage reductions by avoiding Perl scratchpads |
| |
| * add #related anchor and search form to find related patches |
| based on blob OIDs (IOW, exposing `lei p2q' to the web) |
| |
| * fix footer in listing of >200 inboxes |
| |
| * support dumb HTTP clones of SHA-256 git repos |
| |
| * add /$INBOX/$MSGID/d/ endpoint to show diffs in reused Message-IDs |
| (`lei mail-diff' for the web) |
| |
| * support POST /$INBOX/$MSGID/?x=m&q= to limit mbox results to a thread |
| |
| * add topics_(new|active).(html|atom) endpoints |
| |
| * linkify peer public-inbox addresses in To/Cc headers |
| |
| public-inbox-watch: |
| |
| * watching MH folders is now supported |
| |
| lei |
| |
| * use http.proxy / http.<remote>.proxy from system-wide git-config if |
| unconfigured for lei |
| |
| * improve IMAP and NNTP error reporting |
| |
| * reduce default IMAP connections to avoid overloading servers |
| |
| * compatibility with SQLite <3.8.3 on CentOS 7.x |
| |
| * fix `lei q -tt' on locally indexed messages (still broken for remotes: |
| https://public-inbox.org/meta/20230226170931.M947721@dcvr/ ) |
| |
| * `lei import' now sets labels+keywords consistently on all |
| already imported messages |
| |
| * fix `lei up' on saved local queries which previously used -t/--threads |
| |
| * `lei convert' output to v2 public-inboxes is now idempotent |
| |
| * improved bash completion for labels (see contrib/completion) |
| |
| * support for reading (but not writing) MH folders |
| |
| * `lei index' accepts `+L:$LABEL' like `lei import' does |
| |
| solver (used by lei (rediff|blob), and PublicInbox::WWW) |
| |
| * handle copies in patches properly |
| |
| * no longer redundantly parallelized within each WWW process |
| |
| portability |
| |
| * SIGWINCH is handled properly on less common architectures and OSes |
| |
| * fix EINTR handling for kqueue users |
| |
| * various fixes for CentOS 7.x |
| |
| * fix excessive pipelining to `git cat-file' on systems with small |
| getdelim(3) buffers (mainly affects musl) |
| |
| * support Alpine Linux, Dragonfly, NetBSD and OpenBSD. This resulted |
| not only in bugfixes to our code, but also to Dragonfly and OpenBSD. |
| |
| * Inline::C||Socket::MsgHdr no longer required for SCM_RIGHTS |
| with sendmsg/recvmsg on supported *BSDs. |
| |
| * inotify support no longer requires Linux::Inotify2 XS package |
| for most architectures |
| |
| public-inbox-pop3d |
| |
| * support `limit=NUM' and `initial_limit=NUM' query parameters |
| in mailbox names to limit results |
| |
| public-inbox-nntpd |
| |
| * fix LISTGROUP with range (affects neomutt) |
| |
| public-inbox-clone / public-inbox-fetch / `lei add-external --mirror' |
| |
| * mtime of downloaded manifest preserved |
| |
| public-inbox-clone: |
| |
| * parallel mirroring of multiple inboxes/coderepos via manifest, |
| public-inbox-fetch is not used in this mode |
| |
| * new flags to support manifest mirroring include: |
| --dry-run, --inbox-config=, --project-list=, --prune, --purge, |
| --keep-going, --jobs, --include=, --exclude=, --objstore=, |
| --manifest=, --remote-manifest= |
| See public-inbox-clone(1) man page for more details. |
| |
| PublicInbox::SaPlugin::ListMirror |
| |
| * List-ID handling special-cased according to RFC 2919 rules |
| |
| Search improvements (lei and PublicInbox::WWW) |
| |
| * quoted text inside base-85 binary patches is no longer indexed |
| |
| * `public-inbox-cindex --join' prefers using Xapian's C++ API |
| directly to avoid Perl method dispatch overhead to get usable |
| performance associating ~300 inboxes with over 1K coderepos |
| (and vice versa). Users requiring such performance will need |
| a C++ compiler, pkg-config, and the Xapian development files |
| (see INSTALL). |
| |
| This C++ helper will be used more heavily in the future |
| to enable query parser customizations and other functionality |
| unavailable from the Xapian SWIG or XS bindings. |
| |
| Thanks to all the bug reporters and users who made this release |
| possible, and thanks for bearing with my anxiety over making releases. |
| |
| Please report bugs via plain-text mail to: meta@public-inbox.org |
| |
| See archives at https://public-inbox.org/meta/ for all history. |
| See https://public-inbox.org/TODO for what the future holds. |
