Knowledge Base

Preserving for the future: Shell scripts, AoC, and more

Postfix use oauth2 for gmail

bgstack15

Comments

I've previously written about how to send authenticated gmail from cli with mailx, and a slightly more generic send authenticated gmail from command line. But with a recent change to Google's behaviors, I have to use some baloney scheme designed to make system admin's lives difficult just to send email.

Overview

Gmail requires the use of Oauth2 (aka 'xoauth2', or maybe just the library is known as that) to send authenticated mail from custom applications. This document describes how to set up postfix to relay to gmail as an authenticated user under the new scheme. My goal includes only sending messages, not receiving. Everything outbound just comes from my one account, bgstack15@gmail.com. It is possible the references include guidance for allowing different accounts for different users.

The test environment is d2-03a, and production is server2.remote.example.com.

Dependencies

On the system where postfix needs to run:

  • Devuan Ceres

The libpython2-stdlib is important for libimap.

sudo apt-get install postfix python2-minimal libpython2-stdlib libsasl2-module-xoauth

Onetime setup

Google side

I had to use Chromium to log into cloud.console.google.com. I found out later that LibreWolf does work. Here, set up "Oauth consent screen" and add my email address to the list of testers. Then, set up an oauth 2.0 client id ref 1 which provides the client id and secret.

On a Linux system where you can clone the gmail-oauth2-tools utilities, run the one-time command to get the access token and refresh token.

python2 gmail-oauth2-tools/python/oauth2.py --user=bgstack15@gmail.com --generate_oauth2_token --client_id=2748037O9251-ssj18r8tli6krklewtus3m2n3m7lvtiw.apps.googleusercontent.com --client_secret=GODSNX-m2MnUnpEac3tQU-1nm4VN54nop3m

It will direct you to a link you need to open in the browser to sign in and allow this application to control email. Paste the response back into the python2 program and it will generate an access token and refresh token.

Refresh Token: 1//01E-dJkGQzpa3CgYIARAAGAESNwF-L9Irl1pOeMY42_5uBGzVveXggTfg1Car290BgVHdEGspZxWpSheTHWXPySu-9uXvim8mFWg
Access Token: ya29.A0ARrdaM-PO3kNGo28gmKSGOuwkglampwoij3482GM26iTLiw4xMGNE3wE1Te54MvBo_RgmlIBEYd4qEMY522kTm4xnoIozpW5nL43nGmLap3kMfmsZ_sUt4Qenk_JDFMVGIxsmwXWJxObeR_-LSJ61IN4Bi4r
Access Token Expiration Seconds: 3599

Save the refresh token contents to /etc/postfix/refresh-token.

Postfix

Reference 3 or 4 include the readme that describes the process for configuring postfix.

A custom plugin is necessary, and is buildable from the above links. Link 4 includes a dpkg recipe. The plugin generates /usr/lib/x86_64-linux-gnu/sasl2/libxoauth2.so.0.0.0 or similar. The package libsasl2-module-xoauth2 is in my internal repo but can be rebuilt from link 4.

Postfix file main.cf needs quite a few entries, including but not limited to:

# everything normal it has, so these go at the bottom:
# gmail struggles with ipv6, or my net does or something.
inet_protocols = ipv4
# client
relayhost = [smtp.gmail.com]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/saslpasswd
smtp_sasl_mechanism_filter = xoauth2
smtp_sasl_security_options =
smtp_tls_security_level = may
smtp_tls_policy_maps = hash:/etc/postfix/tls_policy

Establish file /etc/postfix/saslpasswd:

[smtp.gmail.com]:587    bgstack15@gmail.com:OAUTH2-TOKEN-CONTENTS-LONG-STRING

Then generate the /etc/postfix/saslpasswd.db with postmap:

# postmap /etc/postfix/saslpsswd

Establish file /etc/postfix/tls_policy:

[smtp.gmail.com]:587 encrypt

Generate its db file:

# postmap /etc/postfix/tls_policy

Generate in the ${sasl_plugin_dir}, the file nominally /etc/postfix/sasl/smtpd.conf.

log_level: DEBUG
sql_engine: sqlite3
sql_database: /etc/sasldb2.sqlite3
sql_select: SELECT props.value WHERE props.id = 2
xoauth2_scope: https://gmail.com/
auxprop_plugin: sql
mech_list: xoauth2

Install sqlite3 package if necessary, and establish sqlite3 database file /etc/sasl2.sqlite3 with the following:

# sqlite3 /etc/sasldb2.sqlite3
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE props (id INTEGER PRIMARY KEY, name VARCHAR, value VARCHAR);
INSERT INTO props VALUES(1,'userPassword','*');
INSERT INTO props VALUES(2,'oauth2BearerTokens','token');
COMMIT;

These values are string literals. Insert into the database an asterisk, as well as the word token.

Reload postfix.

sudo service postfix reload

Send a test email.

mail -a 'From:B. Stack <bgstack15@example.com' -s 'Test with oauth2 part11' bgstack15@gmail.com <<EOF
hello from the command line
at 13:31
EOF

The From field pretty name is used by gmail, but the &lt;email address&gt; section is discarded because we are sending from the one authenticated gmail account.

Custom token rotation script

Oauth2 access tokens expire (google likes to use 59 minutes), so a cron entry can be used to use the refresh token to get a new access token. The following script does not support accepting a new refresh token, but I do not know if that would ever need to happen.

Copy oauth2.py from that gmail oauth tools project to /usr/local/bin/oauth2.py2.

Establish file /usr/local/bin/refresh-oauth2-token:

#!/usr/bin/env sh
# File: /usr/local/bin/refresh-oauth2-token
# Startdate: 2022-03-04 13:36
# Purpose: gets new access token with the cached refresh token
# Usage: in cron every 30 minutes, because the access token lasts for 59 minutes. Must be run as root.
# Dependencies:
#    already-established refresh token in /etc/postfix/refresh-token
# Documentation:
#    /mnt/public/Support/Programs/oauth2-for-gmail/README-oauth2-for-gmail.md
. /etc/default/postfix-oauth2
export PATH=/usr/bin:/usr/sbin:/bin:/sbin
test -z "${REFRESH_FILE}" && REFRESH_FILE="/etc/postfix/refresh-token"
test -z "${SASLPASSWD_FILE}" && SASLPASSWD_FILE="/etc/postfix/saslpasswd"
test -z "${OAUTH2_SCRIPT}" && OAUTH2_SCRIPT="/usr/local/bin/oauth2.py2"
test -z "${USERNAME}" && USERNAME="bgstack15@gmail.com"
test -z "${CLIENT_ID}" && CLIENT_ID="2748037O9251-ssj18r8tli6krklewtus3m2n3m7lvtiw.apps.googleusercontent.com"
test -z "${CLIENT_SECRET}" && CLIENT_SECRET="GODSNX-m2MnUnpEac3tQU-1nm4VN54nop3m"
test -z "${SMTP_SERVER}" && SMTP_SERVER=smtp.gmail.com
test -z "${SMTP_PORT}" && SMTP_PORT=587
refresh_token="$( cat "${REFRESH_FILE}" )"
result="$( python2 "${OAUTH2_SCRIPT}" \
   --client_id="${CLIENT_ID}" \
   --client_secret="${CLIENT_SECRET}" \
   --refresh_token="${refresh_token}" \
   )"
access_token="$( echo "${result}" | awk '/Access Token:/{print $NF}' )"
# Generate new /etc/saslpasswd
echo "[${SMTP_SERVER}]:${SMTP_PORT} ${USERNAME}:${access_token}" > "${SASLPASSWD_FILE}"
postmap "${SASLPASSWD_FILE}"
service postfix reload

Establish file /etc/default/postfix-oauth2:

# dot-sourced by /usr/local/bin/refresh-oauth2-token
REFRESH_FILE="/etc/postfix/refresh-token"
SASLPASSWD_FILE="/etc/postfix/saslpasswd"
OAUTH2_SCRIPT="/usr/local/bin/oauth2.py2"
USERNAME="bgstack15@gmail.com"
CLIENT_ID="2748037O9251-ssj18r8tli6krklewtus3m2n3m7lvtiw.apps.googleusercontent.com"
CLIENT_SECRET="GODSNX-m2MnUnpEac3tQU-1nm4VN54nop3m"
SMTP_SERVER=smtp.gmail.com
SMTP_PORT=587

Establish cron entry in /etc/cron.d/50_rotate_oauth2_token_cron.

# File /etc/cron.d/50_rotate_oauth2_token_cron
# Documentation: /mnt/public/Support/Programs/oauth2-for-gmail/README-oauth2-for-gmail.md
20,50 *  *  *  *  root    /usr/local/bin/refresh-oauth2-token 1>/dev/null 2>&1

Summary of files that are part of this project

  • Modified
  • /etc/postfix/main.cf
  • New to this project
  • /etc/postfix/saslpasswd Will get updated by the cron entry
  • /etc/postfix/saslpasswd.db
  • /etc/postfix/tls_policy
  • /etc/postfix/tls_policy.db
  • /etc/postfix/refresh-token
  • /etc/postfix/sasl/smtpd.conf
  • /usr/lib file libxoauth2.so somewhere, hopefully from an rpm/dpkg of the cyrus-sasl-xoauth2 project
  • /usr/local/bin/oauth2.py2 from the gmail-oauth2-tools project.
  • /usr/local/bin/refresh-oauth2-token
  • /etc/default/postfix-oauth2
  • /etc/cron.d/50_rotate_oauth2_token_cron

References

  1. https://console.cloud.google.com/apis/credentials?project=smtp1-343114&supportedpurview=project
  2. https://github.com/google/gmail-oauth2-tools a. exact link for oauth2.py2
  3. https://github.com/moriyoshi/cyrus-sasl-xoauth2
  4. Reference 3 but on salsa for the dpkg recpipe

Comments