Knowledge Base

I came across a great little tool, xpipe, recently. Very few distros provide xpipe, so I have bundled it into a dpkg for Devuan GNU+Linux and similar distros. It's available on my OBS space. To quote directly from the project readme:

The xpipe command reads input from stdin and splits it by the given number of bytes, lines, or if matching the given pattern. It then invokes the given utility repeatedly, feeding it the generated data chunks as input. You can think of it as a Unix love-child of the split(1), tee(1), and xargs(1) commands. It's usefulness might best be illustrated by an example. Suppose you have a file 'certs.pem' containing a number of x509 certificates in PEM format, and you wish to extract e.g., the subject and validity dates from each. The openssl x509(1) utility can only accept a single certificate at a time, so you'll have to first split the input into individual files containing exactly one cert, then repeatedly run the x509(1) command against each file.

Coincidentally, parsing a large .pem file was exactly what I was trying to do! It was not my very first case where xpipe would have been useful (splitting paragraphs) but is definitely my most prominent use case.