l2md - lore2maildir
-------------------

Quick and dirty hack to import lore.kernel.org list archives via git,
to export them in maildir format or through a pipe, and to keep them
periodically synced.

It can then be used in whichever mail client that supports maildir
format, for example, mutt. Alternatively, it can also pipe new mails
to external MDAs like procmail and friends which then deliver it to
your favorite mail client eventually.

Essentially, it avoids the need to subscribe to any of the lore lists
via mail since all messages are now imported through git transport.

Together with a smtp client like msmtp (which you may need anyway for
git-send-email), it allows to interact on the mailing lists the usual
way.

All pretty basic and hacky at this point, patches very welcome. Please
send them to Daniel Borkmann <daniel@iogearbox.net>.

Build
-----

Links to -lgit2 which is shipped by pretty much all major distros.

Fedora: libgit2-devel
Ubuntu: libgit2-dev

To build, just type:

$ make
[...]

After setting up ~/.l2mdconfig (see below), run as:

$ ./l2md

To install, just type:

# make install

The l2md.service file contains an example systemd service deployment
for letting it run in the background ...

$ service l2md status
Redirecting to /bin/systemctl status l2md.service
● l2md.service - lore2maildir
   Loaded: loaded (/usr/lib/systemd/system/l2md.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2019-09-19 12:07:17 CEST; 55s ago
 Main PID: 31467 (l2md)
    Tasks: 1 (limit: 4915)
   Memory: 257.9M
   CGroup: /system.slice/l2md.service
           └─31467 /usr/bin/l2md

The muttrc file contains an example mutt config for importing the generated
maildir directories.

The procmailrc file contains an example procmail config for getting started,
there are plenty of howtos online for setting up filtering rules.

For l2md, the repository here ships with two example configs, that is,
l2mdconfig.maildir and l2mdconfig.procmail. As the name says, the former
is for exporting new mails in maildir format directly, and the latter is
one example where l2md pipes new mails via stdin to an external MDA like
procmail. Copy the one of your choice into ~/.l2mdconfig to get started.

Troubleshooting
---------------

l2md has only a single command-line parameter, which is --verbose to allow
significantly more visibility on what it is currently doing:

$ ./l2md --verbose

By default without the --verbose it will only dump the number of new mails
and commit sha's it processed to the journal and nothing else. In case l2md
is deployed through the unit file, l2md's logs can be gathered the usual way:

$ journalctl -fu l2md

Howto 1: maildir
----------------

The l2mdconfig.maildir is an example l2md config which needs to be placed
under ~/.l2mdconfig . The muttrc is an example config to get started for
reading the maildir content.

$ cat ~/.l2mdconfig
[general]
	maildir = ~/.l2md/maildir/common
	period = 30
	base = ~/.l2md/

# bpf@vger.kernel.org list
[repo bpf]
	url = https://lore.kernel.org/bpf/0
	maildir = ~/.l2md/maildir/bpf

# netdev@vger.kernel.org list
[repo netdev]
	url = https://lore.kernel.org/netdev/1
	url = https://lore.kernel.org/netdev/0
	initial_import = 1000

The general section contains a sync period in seconds where l2md refetches
all the git repos and looks for new messages to export into the configured
maildirs. The maildir under general is a path to a shared maildir where
l2md exports new mails into. This can also be specified on a per repository
basis. Specifying the maildir under general is optional. It will default
to ~/.l2md/maildir or <base-path>/maildir if the base deviates from the
default one. Specifying base is optional as well. This is the working dir
of l2md where it places its git repos and other meta data for record keeping.
The default is at ~/.l2md/.

The repo sections with subsequent name define a repository (duh!) with
one or more git urls to lore and optional maildir export path as mentioned.
If initial_import is set to >0, then it will only import first x mails upon
initial repository creation instead of the entire archive.

Howto 2: procmail
-----------------

The l2mdconfig.procmail is an example l2md config which needs to be placed
under ~/.l2mdconfig . The procmailrc is an example config to get started
with a basic config for procmail. The provided muttrc can also be used here
in order to get started for reading the maildir content preprocessed via
procmail (the folder needs to point to procmail's MAILDIR of course). Other
MDAs should work as well, but not tested at this point.

$ cat ~/.l2mdconfig
[general]
	base = ~/.l2md/
	pipe = /usr/bin/procmail
	period = 30

[repo bpf]
	url = https://lore.kernel.org/bpf/0
	initial_import = 1000

See Howto 1 for basics. Instead of maildir, the general section here has a
setting which is set to pipe. It is pointing to the /usr/bin/procmail MDA in
the example, and generally executed as the same user as l2md. Similar as with
maildir, pipe can optionally be specified on a per repository basis.