diff options
| -rw-r--r-- | README.md | 107 | ||||
| -rwxr-xr-x | bin/mailsync | 98 | ||||
| -rwxr-xr-x | bin/mw | 46 | ||||
| -rw-r--r-- | mw.1 | 13 |
4 files changed, 221 insertions, 43 deletions
@@ -3,17 +3,24 @@ Get this great stuff without effort: - A full-featured and autoconfigured email client on the terminal with neomutt -- Mail stored offline so you can view and write email while you're away from internet and keep backups +- Mail stored offline so you can view and write email while you're away from + internet and keep backups +- Provides a `mailsync` script that can be scheduled to run as often as you + like, which downloads/syncs mail and notifies you when new mail has arrived. Specifically, this wizard: - Determines your email server's IMAP and SMTP servers and ports -- Creates dotfiles for `neomutt`, `isync`, and `msmtp` appropriate for your email address -- Encrypts and locally stores your password for easy remote access, accessible only by your GPG key +- Creates dotfiles for `neomutt`, `isync`, and `msmtp` appropriate for your + email address +- Encrypts and locally stores your password for easy remote access, accessible + only by your GPG key - Handles as many as nine separate email accounts automatically - Auto-creates bindings to switch between accounts or between mailboxes -- Provides sensible defaults and an attractive appearance for the neomutt email client -- If mutt-wizard doesn't know your server's IMAP/SMTP info by default, it will prompt you for them and will put them in all the right places. +- Provides sensible defaults and an attractive appearance for the neomutt email + client +- If mutt-wizard doesn't know your server's IMAP/SMTP info by default, it will + prompt you for them and will put them in all the right places. ## Install and Use @@ -33,6 +40,8 @@ The mutt-wizard is run with the command `mw`. Once everything is setup, you'll u - `mw -Y` -- sync all configured email accounts - `mw -d` -- choose an account to delete - `mw -D your@email.com` -- delete account settings without confirmation +- `mw -t 30` -- toggle automatic mailsync to every 30 minutes +- `mw -T` -- toggle mailsync without specifying minutes (default is 10) - `pass edit mw-your@email.com` -- revise an account's password ### Options usable when adding an account @@ -70,6 +79,7 @@ There's a chance of errors if you use a slow-release distro like Ubuntu, Debian - `notmuch` - index and search mail. Install it and run `notmuch setup`, tell it that your mail is in `~/.local/share/mail/` (although `mw` will do this automatically if you haven't set notmuch up before). You can run it in mutt with `ctrl-f`. Run `notmuch new` to process new mail. - `abook` - a terminal-based address book. Pressing tab while typing an address to send mail to will suggest contacts that are in your abook. - `urlview` - outputs urls in mail to browser. +- `cronie` - (or any other major cronjob manager) to set up automatic mail syncing. ## Neomutt user interface @@ -91,43 +101,90 @@ To give you an example of the interface, here's an idea: - ## New stuff and improvements since the original release -- `mw` is now scriptable with command-line options and can run successfully without any interaction, making it possible to deploy in a script. -- `isync`/`mbsync` has replaced `offlineimap` as the backend. Offlineimap was error-prone, bloated, used obsolete Python 2 modules and required separate steps to install the system. -- `mw` is now an installed program instead of just a script needed to be kept in your mutt folder. -- `dialog` is no longer used (le bloat) and the interface is simply text commands. -- More autogenerated shortcuts that allow quickly moving and copying mail between boxes. -- More elegant attachment handling. Image/video/pdf attachments without relying on the neomutt instance. +- `mw` is now scriptable with command-line options and can run successfully + without any interaction, making it possible to deploy in a script. +- `isync`/`mbsync` has replaced `offlineimap` as the backend. Offlineimap was + error-prone, bloated, used obsolete Python 2 modules and required separate + steps to install the system. +- `mw` is now an installed program instead of just a script needed to be kept + in your mutt folder. +- `dialog` is no longer used (le bloat) and the interface is simply text + commands. +- More autogenerated shortcuts that allow quickly moving and copying mail + between boxes. +- More elegant attachment handling. Image/video/pdf attachments without relying + on the neomutt instance. - abook integration by default. -- The messy template files and other directories have been moved or removed, leaving a clean config folder. -- msmtp configs moved to `~/.config/` and mail default location moved to `~/.local/share/mail/`, reducing mess in `~`. +- The messy template files and other directories have been moved or removed, + leaving a clean config folder. +- msmtp configs moved to `~/.config/` and mail default location moved to + `~/.local/share/mail/`, reducing mess in `~`. - `pass` is used as a password manager instead of separately saving passwords. - Script is POSIX sh compliant. -- Error handling for the many people who don't read or follow directions. Less errors generally. +- Error handling for the many people who don't read or follow directions. Less + errors generally. - Addition of a manual `man mw` ## Help the Project! -- Try mutt-wizard out on weird machines and weird email addresses and report any errors. -- Open a PR to add new server information into `domains.csv` so their users can more easily use mutt-wizard. +- Try mutt-wizard out on weird machines and weird email addresses and report + any errors. +- Open a PR to add new server information into `domains.csv` so their users can + more easily use mutt-wizard. - If nothing else, [Donate!](https://paypal.me/LukeMSmith) -See Luke's website [here](https://lukesmith.xyz). Email him at [luke@lukesmith.xyz](mailto:luke@lukesmith.xyz). +See Luke's website [here](https://lukesmith.xyz). Email him at +[luke@lukesmith.xyz](mailto:luke@lukesmith.xyz). mutt-wizard is free/libre software, licensed under the GPLv3. ## Details for Tinkerers - The critical `mutt`/`neomutt` files are in `~/.config/mutt/`. -- Put whatever global settings you want in `muttrc`. mutt-wizard will add some lines to this file which you shouldn't remove unless you know what you're doing, but you can move them up/down over your personal config lines if you need to. If you get binding conflict errors in mutt, you might need to do this. -- Each of the accounts that mutt-wizard generates will have custom settings set in a separate file in `accounts/`. You can edit these freely if you want to tinker with settings specific to an account. -- In `/usr/share/mutt-wizard` are several global config files, including `mutt-wizard`'s default settings. You can overwride this in your `muttrc` if you wish. +- Put whatever global settings you want in `muttrc`. mutt-wizard will add some + lines to this file which you shouldn't remove unless you know what you're + doing, but you can move them up/down over your personal config lines if you + need to. If you get binding conflict errors in mutt, you might need to do + this. +- Each of the accounts that mutt-wizard generates will have custom settings set + in a separate file in `accounts/`. You can edit these freely if you want to + tinker with settings specific to an account. +- In `/usr/share/mutt-wizard` are several global config files, including + `mutt-wizard`'s default settings. You can overwride this in your `muttrc` if + you wish. ## Watch out for these things: -- Gmail accounts can now create 'App Password' to use with """less secure""" applications. This password is single use (ie. for setup) and will be stored and encrypted locally. Enabling third-party applications requires turning off two-factor authentication and this will circumvent that. You might also need to manually "Enable IMAP" in the settings. -- Protonmail accounts will require you to set up "Protonmail Bridge" to access PM's IMAP and SMTP servers. Configure that before running mutt-wizard. Note that when mutt-wizard asks for a password, you should put in your [bridge password](https://protonmail.com/bridge/thunderbird#3), not your account password. -- Protonmail bridge is prone to timing out. Watch out for this while adding an account. If the bridge times out, try again. It might help to [increase the timeout](https://protonmail.com/support/knowledge-base/thunderbird-connection-server-timed-error/) in your `mbsyncrc`. -- If you have a university email, or enterprise-hosted email for work, there might be other hurdles or two-factor authentication you have to jump through. Some, for example, will want you to create a separate IMAP password, etc. - - `isync` is not fully UTF-8 compatible, so non-Latin characters may be garbled (although sync should succeed). `mw` will also not autocreate mailbox shortcuts since it is looking for English mailbox names. I strongly recommend you to set your email language to English on your mail server to avoid these problems. + +### Gmail + +Gmail accounts should require an +[application password](https://support.google.com/accounts/answer/185833) to +work unless you allow Gmail to access "less secure" applications. You may also +need to "Enable IMAP" in your Gmail settings. + +### Protonmail + +Protonmail accounts will require you to set up "Protonmail Bridge" to access +PM's IMAP and SMTP servers. Configure that before running mutt-wizard. Note +that when mutt-wizard asks for a password, you should put in your +[bridge password](https://protonmail.com/bridge/thunderbird#3), not your +account password. + +Protonmail bridge is prone to timing out. Watch out for this while adding an +account. If the bridge times out, try again. It might help to +[increase the timeout](https://protonmail.com/support/knowledge-base/thunderbird-connection-server-timed-error/) +in your `mbsyncrc`. + +### Other + +- If you have a university email, or enterprise-hosted email for work, there + might be other hurdles or two-factor authentication you have to jump through. + Some, for example, will want you to create a separate IMAP password, etc. + - `isync` is not fully UTF-8 compatible, so non-Latin characters may be + garbled (although sync should succeed). `mw` will also not autocreate + mailbox shortcuts since it is looking for English mailbox names. I strongly + recommend you to set your email language to English on your mail server to + avoid these problems. ## To-do diff --git a/bin/mailsync b/bin/mailsync new file mode 100755 index 0000000..02e4f98 --- /dev/null +++ b/bin/mailsync @@ -0,0 +1,98 @@ +#!/bin/sh + +# - Syncs mail for all accounts, or a single account given as an argument. +# - Displays a notification showing the number of new mails. +# - Displays a notification for each new mail with its subject displayed. +# - Runs notmuch to index new mail. +# - This script can be set up as a cron job for automated mail syncing. + +# There are many arbitrary and ugly features in this script because it is +# inherently difficult to pass environmental variables to cronjobs and other +# issues. It also should at least be compatible with Linux (and maybe BSD) with +# Xorg and MacOS as well. + +# Run only if user logged in (prevent cron errors) +pgrep -u "${USER:=$LOGNAME}" >/dev/null || { echo "$USER not logged in; sync will not run."; exit ;} +# Run only if not already running in other instance +pgrep -x mbsync >/dev/null && { echo "mbsync is already running." ; exit ;} + +# First, we have to get the right variables for the mbsync file, the pass +# archive, notmuch and the GPG home. This is done by searching common profile +# files for variable assignments. This is ugly, but there are few options that +# will work on the maximum number of machines. +eval `grep -h -- \ + "^\(export \)\?\(MBSYNCRC\|PASSWORD_STORE_DIR\|NOTMUCH_CONFIG\|GNUPGHOME\)=" \ + "$HOME/.profile" "$HOME/.bash_profile" "$HOME/.zprofile" "$HOME/.bashrc" "$HOME/.zshrc" "$HOME/.pam_environment" 2>/dev/null` +# One alternative to this kind of command would be marking the script for +# /bin/sh -l. That might cause other problems on other particular setups that +# do more complicated things on login, or those people who assign environmental +# variables in shell rc files. + +# This variable might be required for soysdemd users, but it will break the +# script on Artix runit. +# export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus +export GPG_TTY=$TTY + +[ -n "$MBSYNCRC" ] && alias mbsync="mbsync -c $MBSYNCRC" || MBSYNCRC="$HOME/.mbsyncrc" + +# Settings are different for MacOS (Darwin) systems. +case "$(uname)" in + Darwin) + notify() { osascript -e "display notification \"$2 in $1\" with title \"You've got Mail\" subtitle \"Account: $account\"" && sleep 2 ;} + messageinfo() { osascript -e "display notification with title \"📧 $from\" subtitle \"$subject\"" ;} + ;; + *) + displays="$(ps axo user,cmd | grep "^$USER\s\+.*Xorg" | grep -wo "[0-9]*:[0-9]\+")" + notify() { for x in $displays; do + export DISPLAY=$x + notify-send --app-name="mutt-wizard" "mutt-wizard" "📬 $2 new mail(s) in \`$1\` account." + done ;} + messageinfo() { for x in $displays; do + export DISPLAY=$x + notify-send --app-name="mutt-wizard" "📧$from:" "$subject" + done ;} + ;; +esac + +# Check account for new mail. Notify if there is new content. +syncandnotify() { + acc="$(echo "$account" | sed "s/.*\///")" + if [ -z "$opts" ]; then mbsync "$acc"; else mbsync "$opts" "$acc"; fi + new=$(find "$HOME/.local/share/mail/$acc/INBOX/new/" "$HOME/.local/share/mail/$acc/Inbox/new/" "$HOME/.local/share/mail/$acc/inbox/new/" -type f -newer "$HOME/.config/mutt/.mailsynclastrun" 2> /dev/null) + newcount=$(echo "$new" | sed '/^\s*$/d' | wc -l) + if [ "$newcount" -gt "0" ]; then + notify "$acc" "$newcount" & + for file in $new; do + # Extract subject and sender from mail. + from=$(awk '/^From: / && ++n ==1,/^\<.*\>:/' "$file" | perl -CS -MEncode -ne 'print decode("MIME-Header", $_)' | awk '{ $1=""; if (NF>=3)$NF=""; print $0 }' | sed 's/^[[:blank:]]*[\"'\''\<]*//;s/[\"'\''\>]*[[:blank:]]*$//') + subject=$(awk '/^Subject: / && ++n == 1,/^\<.*\>: / && ++i == 2' "$file" | head -n 1 | perl -CS -MEncode -ne 'print decode("MIME-Header", $_)' | sed 's/^Subject: //' | sed 's/^{[[:blank:]]*[\"'\''\<]*//;s/[\"'\''\>]*[[:blank:]]*$//' | tr -d '\n') + messageinfo & + done + fi +} + +# Sync accounts passed as argument or all. +if [ "$#" -eq "0" ]; then + accounts="$(awk '/^Channel/ {print $2}' "$MBSYNCRC")" +else + for arg in "$@"; do + [ "${arg%${arg#?}}" = '-' ] && opts="${opts:+${opts} }${arg}" && shift 1 + done + accounts=$* +fi + +#( kill -46 "$(pidof "${STATUSBAR:-dwmblocks}")" >/dev/null 2>&1 ) 2>/dev/null + +# Parallelize multiple accounts +for account in $accounts +do + syncandnotify & +done + +wait +#( kill -46 "$(pidof "${STATUSBAR:-dwmblocks}")" >/dev/null 2>&1 ) 2>/dev/null + +notmuch new 2>/dev/null + +#Create a touch file that indicates the time of the last run of mailsync +touch "$HOME/.config/mutt/.mailsynclastrun" @@ -21,7 +21,7 @@ cachedir="${XDG_CACHE_HOME:-$HOME/.cache}/mutt-wizard" muttrc="$muttdir/muttrc" msmtprc="${XDG_CONFIG_HOME:-$HOME/.config}/msmtp/config" msmtplog="${XDG_CONFIG_HOME:-$HOME/.config}/msmtp/msmtp.log" -MARKER="# mw-autogenerated" +marker="# mw-autogenerated" alias mbsync='mbsync -c "$mbsyncrc"' for x in "/etc/ssl/certs/ca-certificates.crt" "/etc/pki/tls/certs/ca-bundle.crt" "/etc/ssl/ca-bundle.pem" "/etc/pki/tls/cacert.pem" "/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem" "/etc/ssl/cert.pem" "/usr/local/share/ca-certificates/" @@ -151,7 +151,7 @@ delete() { if [ -z "${fulladdr+x}" ]; then [ -z "$fulladdr" ] && echo "Invalid account name given." && return 1 - sed -ibu "/IMAPStore $fulladdr-remote$/,/# End profile/d" "$mbsyncrc" ; rm -rf "$mbsyncrc"bu + sed -ibu "/IMAPStore $fulladdr-remote$/,/# End profile/d" "$mbsyncrc" ; rm -f "$mbsyncrc"bu rm -rf "${cachedir:?}/${fulladdr:?}" "$accdir/"[1-9]"-$fulladdr.muttrc" sed -ibu "/[0-9]-$fulladdr.muttrc/d" "$muttrc" ; rm -f "$muttrc"bu sed -ibu "/account $fulladdr/,/^\(\s*$\|account\)/d" "$msmtprc"; rm -f "$msmtprc"bu @@ -205,9 +205,9 @@ writeinfo() { mkdir -p "$muttdir" "$accdir" "$cachedir/$fulladdr/bodies" "${XDG_ # Create a muttrc for viewing mail. echo "$mutt_profile" > "$accdir/$idnum-$fulladdr.muttrc" [ ! -f "$muttrc" ] && echo "# vim: filetype=neomuttrc" > "$muttrc" - ! grep -q "^source.*mutt-wizard.muttrc" "$muttrc" && echo "source $mwconfig $MARKER" >> "$muttrc" - ! grep "^source.*.muttrc" "$muttrc" | grep -qv "$mwconfig" && echo "source $accdir/$idnum-$fulladdr.muttrc $MARKER" >> "$muttrc" - echo "macro index,pager i$idnum '<sync-mailbox><enter-command>source $accdir/$idnum-$fulladdr.muttrc<enter><change-folder>!<enter>;<check-stats>' \"switch to $fulladdr\" $MARKER" >> "$muttrc" + ! grep -q "^source.*mutt-wizard.muttrc" "$muttrc" && echo "source $mwconfig $marker" >> "$muttrc" + ! grep "^source.*.muttrc" "$muttrc" | grep -qv "$mwconfig" && echo "source $accdir/$idnum-$fulladdr.muttrc $marker" >> "$muttrc" + echo "macro index,pager i$idnum '<sync-mailbox><enter-command>source $accdir/$idnum-$fulladdr.muttrc<enter><change-folder>!<enter>;<check-stats>' \"switch to $fulladdr\" $marker" >> "$muttrc" notmuchauto # Create a notmuch config file if not present already. } @@ -216,12 +216,12 @@ getpass() { while : ; do pass rm -f "$pass_prefix$fulladdr" >/dev/null 2>&1 pass insert "$pass_prefix$fulladdr" && break; done ;} formatShortcut() { toappend="$toappend -macro index,pager g$1 \"<change-folder>=$3<enter>\" \"go to $2\" $MARKER -macro index,pager M$1 \";<save-message>=$3<enter>\" \"move mail to $2\" $MARKER -macro index,pager C$1 \";<copy-message>=$3<enter>\" \"copy mail to $2\" $MARKER" >> "$accdir/$idnum-$fulladdr.muttrc" ;} +macro index,pager g$1 \"<change-folder>=$3<enter>\" \"go to $2\" $marker +macro index,pager M$1 \";<save-message>=$3<enter>\" \"move mail to $2\" $marker +macro index,pager C$1 \";<copy-message>=$3<enter>\" \"copy mail to $2\" $marker" >> "$accdir/$idnum-$fulladdr.muttrc" ;} setBox() { toappend="$toappend -set $1 = \"+$2\" $MARKER" ;} +set $1 = \"+$2\" $marker" ;} getboxes() { [ -n "${force+x}" ] && mailboxes="INBOX Drafts @@ -239,7 +239,7 @@ Archive" && return 0 fi ;} finalize() { echo "$mailboxes" | xargs -I {} mkdir -p "$maildir/$fulladdr/{}/cur" "$maildir/$fulladdr/{}/tmp" "$maildir/$fulladdr/{}/new" - sed -ibu "/$MARKER/d" "$accdir/$idnum-$fulladdr.muttrc" ; rm -f "$accdir/$idnum-$fulladdr.muttrcbu" + sed -ibu "/$marker/d" "$accdir/$idnum-$fulladdr.muttrc" ; rm -f "$accdir/$idnum-$fulladdr.muttrcbu" toappend="mailboxes $(echo "$mailboxes" | sed "s/^/\"=/;s/$/\"/" | paste -sd ' ' - )" for x in $mailboxes; do case "$x" in @@ -257,12 +257,6 @@ finalize() { echo "$mailboxes" | xargs -I {} mkdir -p "$maildir/$fulladdr/{}/cur command -V urlview >/dev/null 2>&1 && [ ! -f "$HOME/.urlview" ] && echo "COMMAND \$BROWSER" > "$HOME/.urlview" return 0 ;} -syncwrapper() { mbsync "${1:--a}" & - ( kill -46 "$(pidof "${STATUSBAR:-dwmblocks}")" >/dev/null 2>&1 ) 2>/dev/null - wait - ( kill -46 "$(pidof "${STATUSBAR:-dwmblocks}")" >/dev/null 2>&1 ) 2>/dev/null - notmuch new ;} - notmuchauto() { \ [ -z "$NOTMUCH_CONFIG" ] && NOTMUCH_CONFIG="$HOME/.notmuch-config" [ -f "$NOTMUCH_CONFIG" ] && return 0 @@ -282,6 +276,17 @@ synchronize_flags=true gpg_path=$GPG" echo "$nmbasic" > "$NOTMUCH_CONFIG" ;} +togglecron() { cron="$(mktemp)" + crontab -l > "$cron" + if grep -q mailsync "$cron"; then + echo "Removing automatic mailsync..." + sed -ibu /mailsync/d "$cron"; rm -f "$cron"bu + else + echo "Adding automatic mailsync every ${cronmin:-10} minutes..." + echo "*/${cronmin-10} * * * * /usr/local/bin/mailsync" >> "$cron" + fi && + crontab "$cron"; rm -f "$cron" ;} + setact() { if [ -n "${action+x}" ] && [ "$action" != "$1" ]; then echo "Running $1 with $action..." echo "Incompatible options given. Only one action may be specified per run." @@ -292,7 +297,7 @@ setact() { if [ -n "${action+x}" ] && [ "$action" != "$1" ]; then trap 'echo -e "\033[0m\n"; exit' INT ABRT -while getopts "fplhodYD:y:i:I:s:S:u:a:n:x:m:" o; do case "${o}" in +while getopts "fplhodTYD:y:i:I:s:S:u:a:n:x:m:t:" o; do case "${o}" in l) setact list || exit 1 ;; d) setact delete || exit 1 ;; D) setact delete || exit 1 ; fulladdr="$OPTARG" ;; @@ -309,6 +314,8 @@ while getopts "fplhodYD:y:i:I:s:S:u:a:n:x:m:" o; do case "${o}" in o) setact add || exit 1 ; online=True ;; f) setact add || exit 1 ; force=True ;; x) setact add || exit 1 ; password="$OPTARG" ;; + t) setact toggle || exit 1 ; cronmin="$OPTARG" ;; + T) setact toggle || exit 1 ;; p) echo "NOTE: Protonmail users must install and configure Protonmail Bridge first for the first sync to work." imap="127.0.0.1" iport="1143" @@ -329,6 +336,8 @@ Main actions: -D your@email.com Force remove account without confirmation -y your@email.com Sync mail for account by name -Y Sync mail for all accounts + -t number Toggle automatic mailsync every <number> minutes + -T Toggle automatic mailsync Options allowed with -a: -u Account login name if not full address @@ -357,5 +366,6 @@ case "$action" in list) list ;; add) askinfo && getprofiles && writeinfo && getboxes && finalize || { delete ; exit 1 ;} ;; delete) delete $fulladdr ;; - sync) syncwrapper $fulladdr ;; + sync) mailsync $fulladdr ;; + toggle) togglecron ;; esac @@ -35,6 +35,12 @@ download and upload mail for an email account .TP .B -Y sync all email accounts +.TP +.B -t 15 +toggle a cronjob that syncs your mail every 15 minutes (or any other number under 60) +.TP +.B -T +toggle a cronjob without specifying minutes between sync .SH OPTIONS FOR ADDING ACCOUNTS These can be specified on the command line, otherwise, you will be prompted for what is necessary. mutt-wizard knows the IMAP/SMTP server information for most email providers, so specifying them is usually redundant. .TP @@ -75,6 +81,13 @@ connection will still be attempted in setup to discover mailboxes. Add a Protonmail account. Protonmail Bridge must be installed and set up. .SH DETAILS .TP +.B mailsync +mutt-wizard calls a script +.I mailsync +to sync mail. This script additionally indexes new mail with notmuch and gives you a notification if new mail has arrived. If you want to bypass its additional features, you can always just run +.I mbsync -a +to sync your mail directly. +.TP .B Mail location If the user chooses to keep offline email with .B isync, |