Documentation/public-inbox-daemon.pod - pub/scm/infra/public-inbox - Git at Google

 =head1 NAME

 public-inbox-daemon - common usage for public-inbox network daemons

 =head1 SYNOPSIS

 	public-inbox-netd
 	public-inbox-httpd
 	public-inbox-imapd
 	public-inbox-nntpd
 	public-inbox-pop3d

 =head1 DESCRIPTION

 This manual describes common options and behavior for
 public-inbox network daemons.  Network daemons for public-inbox
 provide read-only IMAP, HTTP, NNTP and POP3 access to public-inboxes.
 Write access to a public-inbox will never be required to run these.

 These daemons are implemented with a common core using
 non-blocking sockets and optimized for fairness; even with
 thousands of connected clients over slow links.

 They also provide graceful shutdown/upgrade support to avoid
 breaking existing connections during software upgrades.

 These daemons may also utilize multiple pre-forked worker
 processes to take advantage of multiple CPUs.

 =head1 OPTIONS

 =over

 =item -l [PROTOCOL://]ADDRESS[?opt1=val1,opt2=val2]

 =item --listen [PROTOCOL://]ADDRESS[?opt1=val1,opt2=val2]

 This takes an absolute path to a Unix socket or HOST:PORT
 to listen on.  For example, to listen to TCP connections on
 port 119, use: C<-l 0.0.0.0:119>.  This may also point to
 a Unix socket (C<-l /path/to/http.sock>) for a reverse proxy
 like L<nginx(8)> to use.

 May be specified multiple times to allow listening on multiple
 sockets.

 Unless per-listener options are used (required for
 L<public-inbox-netd(1)>), this does not need to be specified at
 all if relying on L<systemd.socket(5)> or similar,

 Per-listener options may be specified after C<?> as C<KEY=VALUE>
 pairs delimited by C<,>.  See L<public-inbox-netd(1)> for
 documentation on the C<cert=>, C<key=>, C<env.NAME=VALUE>,
 C<out=>, C<err=>, and C<psgi=> options available.

 Default: server-dependent unless socket activation is used with
 L<systemd(1)> or similar (see L<systemd.socket(5)>).

 =item -1

 =item --stdout PATH

 Specify an appendable path to redirect stdout descriptor (1) to.
 Using this is preferable to setting up the redirect externally
 (e.g. E<gt>E<gt>/path/to/log in shell) since it allows
 SIGUSR1 to be handled (see L<SIGNALS/SIGNALS> below).

 C<out=> may also be specified on a per-listener basis.

 Default: /dev/null with C<--daemonize>, inherited otherwise

 =item -2 PATH

 =item --stderr PATH

 Like C<--stdout>, but for the stderr descriptor (2).

 C<err=> may also be specified on a per-listener basis.

 Default: /dev/null with C<--daemonize>, inherited otherwise

 =item -W INTEGER

 =item --worker-processes INTEGER

 Set the number of worker processes.

 Normally, this should match the number of CPUs on the system to
 take full advantage of the hardware.  However, users of
 memory-constrained systems may want to lower this.

 Setting this to zero (C<-W0>) disables the master/worker split;
 saving some memory but removing the ability to use SIGTTIN
 to increase worker processes or have the worker restarted by
 the master on crashes.

 Default: 1

 =item -X INTEGER

 =item --xapian-helpers INTEGER

 Enables the use of Xapian helper processes to handle expensive,
 non-deterministic Xapian search queries asynchronously without
 blocking simple requests.

 With positive values, there is an additional manager process
 that can be signaled to control the number of Xapian helper workers.

 * C<-X0> one worker, no manager process
 * C<-X1> one worker, one manager process
 ...
 * C<-X8> eight workers, one manager process

 As with the public-facing public-inbox-* daemons, sending C<SIGTTIN>
 or C<SIGTTOU> to the Xapian helper manager process will increment or
 decrement the number of workers.

 Both Xapian helper workers and managers automatically respawn if they
 crash or are explicitly killed, even with C<-X0>.

 A C++ compiler, L<pkg-config(1)>, and Xapian development files (e.g.
 C<libxapian-dev> or C<xapian*-core-dev*>) are required to gain access to
 some expensive queries and significant memory savings.

 Xapian helper workers are shared by all C<--worker-processes> of the
 Perl daemon for additional memory savings.

 New in public-inbox 2.0.0.

 Default: undefined, search queries are handled synchronously

 =item --cert /path/to/cert

 The default TLS certificate for HTTPS, IMAPS, NNTPS, POP3S and/or STARTTLS
 support if the C<cert> option is not given with C<--listen>.

 With this option, well-known TCP ports automatically get TLS or STARTTLS
 support if using systemd-compatible socket activation.  That is, ports
 443, 563, 993, and 995 support HTTPS, NNTPS, IMAPS, and POP3S,
 respectively; while ports 110, 119, and 143 support STARTTLS on POP3,
 NNTP, and IMAP, respectively.

 =item --key /path/to/key

 The default TLS certificate key for the default C<--cert> or
 per-listener C<cert=> option.  The private key may be
 concatenated into the cert file itself, in which case this
 option is not needed.

 =item --multi-accept INTEGER

 By default, each worker accepts one connection at a time to maximize
 fairness and minimize contention across multiple processes on a
 shared listen socket.  Accepting multiple connections at once may be
 useful in constrained deployments with few, heavily loaded workers.
 Negative values enables a worker to accept all available clients at
 once, possibly starving others in the process.  C<-1> behaves like
 C<multi_accept yes> in nginx; while C<0> (the default) is
 C<multi_accept no> in nginx.  Positive values allow
 fine-tuning without the runaway behavior of C<-1>.

 This may be specified on a per-listener basis via the C<multi-accept=>
 per-listener directive (e.g. C<-l http://127.0.0.1?multi-accept=1>).

 Default: 0

 =back

 =head1 SIGNALS

 Most of our signal handling behavior is copied from L<nginx(8)>
 and/or L<starman(1)>, so it is possible to reuse common scripts
 for managing them.

 =over 8

 =item SIGUSR1

 Reopens log files pointed to by --stdout and --stderr options.

 =item SIGUSR2

 Spawn a new process with the intention to replace the running one.
 See L</UPGRADING> below.

 =item SIGHUP

 Reload config files associated with the process.
 (Note: broken for L<public-inbox-httpd(1)> only in E<lt>= 1.6)

 =item SIGTTIN

 Increase the number of running worker processes by one.

 =item SIGTTOU

 Decrease the number of running worker processes by one.

 =item SIGWINCH

 Stop all running worker processes.  SIGHUP or SIGTTIN
 may be used to restart workers.

 =item SIGQUIT

 Gracefully terminate the running process.

 =back

 SIGTTOU, SIGTTIN, SIGWINCH all have no effect when worker
 processes are disabled with C<-W0> on the command-line.

 =head1 ENVIRONMENT

 =over 8

 =item PI_CONFIG

 The default config file, normally "~/.public-inbox/config".
 See L<public-inbox-config(5)>

 =item LISTEN_FDS, LISTEN_PID

 Used by systemd (and compatible) installations for socket
 activation.  See L<systemd.socket(5)> and L<sd_listen_fds(3)>.

 =item PERL_INLINE_DIRECTORY

 Pointing this to a writable directory enables the use
 of L<Inline> and L<Inline::C> extensions which may provide
 platform-specific performance improvements.  Currently, this
 enables the use of L<vfork(2)> which speeds up subprocess
 spawning with the Linux kernel.

 public-inbox will never enable L<Inline::C> automatically without
 this environment variable set or C<~/.cache/public-inbox/inline-c>
 created by a user. See L<Inline> and L<Inline::C> for more details.

 =back

 =head1 UPGRADING

 There are two ways to upgrade a running process.

 Users of process management systems with socket activation
 (L<systemd(1)> or similar) may rely on multiple daemon instances.
 For systemd, this means using two (or more) '@' instances for each
 service (e.g. C<SERVICENAME@INSTANCE>) as documented in
 L<systemd.unit(5)>.

 Users of traditional SysV init may use SIGUSR2 to spawn
 a replacement process and gracefully terminate the old
 process using SIGQUIT.

 In either case, the old process will not truncate running
 responses; so responses to expensive requests do not get
 interrupted and lost.

 =head1 CONTACT

 Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org>

 The mail archives are hosted at L<https://public-inbox.org/meta/> and
 L<http://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/meta/>

 =head1 COPYRIGHT

 Copyright all contributors L<mailto:meta@public-inbox.org>

 License: AGPL-3.0+ L<https://www.gnu.org/licenses/agpl-3.0.txt>

 =head1 SEE ALSO

 L<public-inbox-httpd(1)>, L<public-inbox-imapd(1)>,
 L<public-inbox-nntpd(1)>, L<public-inbox-pop3d(1)>, L<public-inbox-netd(1)>
	=head1 NAME

	public-inbox-daemon - common usage for public-inbox network daemons

	=head1 SYNOPSIS

	public-inbox-netd
	public-inbox-httpd
	public-inbox-imapd
	public-inbox-nntpd
	public-inbox-pop3d

	=head1 DESCRIPTION

	This manual describes common options and behavior for
	public-inbox network daemons. Network daemons for public-inbox
	provide read-only IMAP, HTTP, NNTP and POP3 access to public-inboxes.
	Write access to a public-inbox will never be required to run these.

	These daemons are implemented with a common core using
	non-blocking sockets and optimized for fairness; even with
	thousands of connected clients over slow links.

	They also provide graceful shutdown/upgrade support to avoid
	breaking existing connections during software upgrades.

	These daemons may also utilize multiple pre-forked worker
	processes to take advantage of multiple CPUs.

	=head1 OPTIONS

	=over

	=item -l [PROTOCOL://]ADDRESS[?opt1=val1,opt2=val2]

	=item --listen [PROTOCOL://]ADDRESS[?opt1=val1,opt2=val2]

	This takes an absolute path to a Unix socket or HOST:PORT
	to listen on. For example, to listen to TCP connections on
	port 119, use: C<-l 0.0.0.0:119>. This may also point to
	a Unix socket (C<-l /path/to/http.sock>) for a reverse proxy
	like L<nginx(8)> to use.

	May be specified multiple times to allow listening on multiple
	sockets.

	Unless per-listener options are used (required for
	L<public-inbox-netd(1)>), this does not need to be specified at
	all if relying on L<systemd.socket(5)> or similar,

	Per-listener options may be specified after C<?> as C<KEY=VALUE>
	pairs delimited by C<,>. See L<public-inbox-netd(1)> for
	documentation on the C<cert=>, C<key=>, C<env.NAME=VALUE>,
	C<out=>, C<err=>, and C<psgi=> options available.

	Default: server-dependent unless socket activation is used with
	L<systemd(1)> or similar (see L<systemd.socket(5)>).

	=item -1

	=item --stdout PATH

	Specify an appendable path to redirect stdout descriptor (1) to.
	Using this is preferable to setting up the redirect externally
	(e.g. E<gt>E<gt>/path/to/log in shell) since it allows
	SIGUSR1 to be handled (see L<SIGNALS/SIGNALS> below).

	C<out=> may also be specified on a per-listener basis.

	Default: /dev/null with C<--daemonize>, inherited otherwise

	=item -2 PATH

	=item --stderr PATH

	Like C<--stdout>, but for the stderr descriptor (2).

	C<err=> may also be specified on a per-listener basis.

	Default: /dev/null with C<--daemonize>, inherited otherwise

	=item -W INTEGER

	=item --worker-processes INTEGER

	Set the number of worker processes.

	Normally, this should match the number of CPUs on the system to
	take full advantage of the hardware. However, users of
	memory-constrained systems may want to lower this.

	Setting this to zero (C<-W0>) disables the master/worker split;
	saving some memory but removing the ability to use SIGTTIN
	to increase worker processes or have the worker restarted by
	the master on crashes.

	Default: 1

	=item -X INTEGER

	=item --xapian-helpers INTEGER

	Enables the use of Xapian helper processes to handle expensive,
	non-deterministic Xapian search queries asynchronously without
	blocking simple requests.

	With positive values, there is an additional manager process
	that can be signaled to control the number of Xapian helper workers.

	* C<-X0> one worker, no manager process
	* C<-X1> one worker, one manager process
	...
	* C<-X8> eight workers, one manager process

	As with the public-facing public-inbox-* daemons, sending C<SIGTTIN>
	or C<SIGTTOU> to the Xapian helper manager process will increment or
	decrement the number of workers.

	Both Xapian helper workers and managers automatically respawn if they
	crash or are explicitly killed, even with C<-X0>.

	A C++ compiler, L<pkg-config(1)>, and Xapian development files (e.g.
	C<libxapian-dev> or C<xapian-core-dev>) are required to gain access to
	some expensive queries and significant memory savings.

	Xapian helper workers are shared by all C<--worker-processes> of the
	Perl daemon for additional memory savings.

	New in public-inbox 2.0.0.

	Default: undefined, search queries are handled synchronously

	=item --cert /path/to/cert

	The default TLS certificate for HTTPS, IMAPS, NNTPS, POP3S and/or STARTTLS
	support if the C<cert> option is not given with C<--listen>.

	With this option, well-known TCP ports automatically get TLS or STARTTLS
	support if using systemd-compatible socket activation. That is, ports
	443, 563, 993, and 995 support HTTPS, NNTPS, IMAPS, and POP3S,
	respectively; while ports 110, 119, and 143 support STARTTLS on POP3,
	NNTP, and IMAP, respectively.

	=item --key /path/to/key

	The default TLS certificate key for the default C<--cert> or
	per-listener C<cert=> option. The private key may be
	concatenated into the cert file itself, in which case this
	option is not needed.

	=item --multi-accept INTEGER

	By default, each worker accepts one connection at a time to maximize
	fairness and minimize contention across multiple processes on a
	shared listen socket. Accepting multiple connections at once may be
	useful in constrained deployments with few, heavily loaded workers.
	Negative values enables a worker to accept all available clients at
	once, possibly starving others in the process. C<-1> behaves like
	C<multi_accept yes> in nginx; while C<0> (the default) is
	C<multi_accept no> in nginx. Positive values allow
	fine-tuning without the runaway behavior of C<-1>.

	This may be specified on a per-listener basis via the C<multi-accept=>
	per-listener directive (e.g. C<-l http://127.0.0.1?multi-accept=1>).

	Default: 0

	=back

	=head1 SIGNALS

	Most of our signal handling behavior is copied from L<nginx(8)>
	and/or L<starman(1)>, so it is possible to reuse common scripts
	for managing them.

	=over 8

	=item SIGUSR1

	Reopens log files pointed to by --stdout and --stderr options.

	=item SIGUSR2

	Spawn a new process with the intention to replace the running one.
	See L</UPGRADING> below.

	=item SIGHUP

	Reload config files associated with the process.
	(Note: broken for L<public-inbox-httpd(1)> only in E<lt>= 1.6)

	=item SIGTTIN

	Increase the number of running worker processes by one.

	=item SIGTTOU

	Decrease the number of running worker processes by one.

	=item SIGWINCH

	Stop all running worker processes. SIGHUP or SIGTTIN
	may be used to restart workers.

	=item SIGQUIT

	Gracefully terminate the running process.

	=back

	SIGTTOU, SIGTTIN, SIGWINCH all have no effect when worker
	processes are disabled with C<-W0> on the command-line.

	=head1 ENVIRONMENT

	=over 8

	=item PI_CONFIG

	The default config file, normally "~/.public-inbox/config".
	See L<public-inbox-config(5)>

	=item LISTEN_FDS, LISTEN_PID

	Used by systemd (and compatible) installations for socket
	activation. See L<systemd.socket(5)> and L<sd_listen_fds(3)>.

	=item PERL_INLINE_DIRECTORY

	Pointing this to a writable directory enables the use
	of L<Inline> and L<Inline::C> extensions which may provide
	platform-specific performance improvements. Currently, this
	enables the use of L<vfork(2)> which speeds up subprocess
	spawning with the Linux kernel.

	public-inbox will never enable L<Inline::C> automatically without
	this environment variable set or C<~/.cache/public-inbox/inline-c>
	created by a user. See L<Inline> and L<Inline::C> for more details.

	=back

	=head1 UPGRADING

	There are two ways to upgrade a running process.

	Users of process management systems with socket activation
	(L<systemd(1)> or similar) may rely on multiple daemon instances.
	For systemd, this means using two (or more) '@' instances for each
	service (e.g. C<SERVICENAME@INSTANCE>) as documented in
	L<systemd.unit(5)>.

	Users of traditional SysV init may use SIGUSR2 to spawn
	a replacement process and gracefully terminate the old
	process using SIGQUIT.

	In either case, the old process will not truncate running
	responses; so responses to expensive requests do not get
	interrupted and lost.

	=head1 CONTACT

	Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org>

	The mail archives are hosted at L<https://public-inbox.org/meta/> and
	L<http://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/meta/>

	=head1 COPYRIGHT

	Copyright all contributors L<mailto:meta@public-inbox.org>

	License: AGPL-3.0+ L<https://www.gnu.org/licenses/agpl-3.0.txt>

	=head1 SEE ALSO

	L<public-inbox-httpd(1)>, L<public-inbox-imapd(1)>,
	L<public-inbox-nntpd(1)>, L<public-inbox-pop3d(1)>, L<public-inbox-netd(1)>