| =head1 NAME |
| |
| public-inbox-xcpdb - upgrade Xapian DB formats |
| |
| =head1 SYNOPSIS |
| |
| public-inbox-xcpdb [OPTIONS] INBOX_DIR |
| |
| public-inbox-xcpdb [OPTIONS] --all |
| |
| =head1 DESCRIPTION |
| |
| public-inbox-xcpdb is similar to L<copydatabase(1)> for |
| upgrading to the latest database format supported by Xapian |
| (e.g. "glass" or "honey"), but is designed to tolerate and |
| accept parallel Xapian database modifications from |
| L<public-inbox-watch(1)>, L<public-inbox-mda(1)>, |
| L<public-inbox-learn(1)>, and L<public-inbox-index(1)>. |
| |
| This command is rarely used, as Xapian DB formats rarely |
| change. |
| |
| =head1 OPTIONS |
| |
| =over |
| |
| =item --all |
| |
| Copy all inboxes configured in ~/.public-inbox/config. |
| This is an alternative to specifying individual inboxes directories |
| on the command-line. |
| |
| =item -c |
| |
| =item --compact |
| |
| In addition to performing the copy operation, run L<xapian-compact(1)> |
| on each Xapian shard after copying but before finalizing it. |
| Compared to the cost of copying a Xapian database, compacting a |
| Xapian database takes only around 5% of the time required to copy. |
| |
| Compared to L<public-inbox-compact(1)>, use of this option is |
| preferable for gigantic inboxes where the coarse-grained lock |
| currently required for L<public-inbox-compact(1)> can cause |
| the compaction to take hours at-a-time. |
| |
| =item -R N |
| |
| =item --reshard=N |
| |
| Reshard the Xapian database on a L<v2|public-inbox-v2-format(5)> |
| inbox to C<N> shards . Since L<xapian-compact(1)> is not suitable |
| for merging, users can rely on this switch to reshard the |
| existing Xapian database(s) to any positive value of C<N>. |
| |
| This is useful in case the Xapian DB was created with too few or |
| too many shards given the capabilities of the current hardware. |
| |
| =item --blocksize |
| |
| =item --no-full |
| |
| =item --fuller |
| |
| These options are passed directly to L<xapian-compact(1)> when |
| used with C<--compact>. |
| |
| =item --no-fsync |
| |
| Disable L<fsync(2)> and L<fdatasync(2)>. |
| See L<public-inbox-index(1)/--no-fsync> for caveats. |
| |
| Available in public-inbox 1.6.0+. |
| |
| =item --sequential-shard |
| |
| Copy each shard sequentially, ignoring C<--jobs>. This also |
| affects indexing done at the end of a run. |
| |
| =item --batch-size=BYTES |
| |
| =item --max-size=BYTES |
| |
| See L<public-inbox-index(1)> for a description of these options. |
| |
| These indexing options indexing at the end of a run. |
| C<public-inbox-xcpdb> may run in parallel with with |
| L<public-inbox-index(1)>, and C<public-inbox-xcpdb> needs to |
| reindex changes made to the old Xapian DBs by |
| L<public-inbox-index(1)> while it was running. |
| |
| =back |
| |
| =head1 ENVIRONMENT |
| |
| =over 8 |
| |
| =item PI_CONFIG |
| |
| The default config file, normally "~/.public-inbox/config". |
| See L<public-inbox-config(5)> |
| |
| =item XAPIAN_FLUSH_THRESHOLD |
| |
| The number of documents to update before committing changes to |
| disk. This environment is handled directly by Xapian, refer to |
| Xapian API documentation for more details. |
| |
| Default: 10000 |
| |
| =back |
| |
| =head1 UPGRADING |
| |
| This tool is intended for admins upgrading Xapian search databases |
| used by public-inbox, NOT users upgrading public-inbox itself. |
| |
| In particular, it DOES NOT upgrade the schema used by the |
| PSGI search interface (see L<public-inbox-index(1)>). |
| |
| =head1 LIMITATIONS |
| |
| Do not use L<public-inbox-purge(1)> or L<public-inbox-edit(1)> |
| while this is running; old (purged or edited data) may show up. |
| |
| Normal invocations L<public-inbox-index(1)> can safely run |
| while this is running, too. However, reindexing via the |
| L<public-inbox-index(1)/--reindex> switch will be a waste of |
| computing resources. |
| |
| =head1 CONTACT |
| |
| Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org> |
| |
| The mail archives are hosted at L<https://public-inbox.org/meta/> and |
| L<http://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/meta/> |
| |
| =head1 COPYRIGHT |
| |
| Copyright 2019-2021 all contributors L<mailto:meta@public-inbox.org> |
| |
| License: AGPL-3.0+ L<https://www.gnu.org/licenses/agpl-3.0.txt> |
| |
| =head1 SEE ALSO |
| |
| L<copydatabase(1)>, L<xapian-compact(1)>, L<public-inbox-index(1)> |
