答案的原始部分和第一次主要编辑在接下来的部分中保持不变,以供有关反复试验的历史参考。
2021 年 4 月 29 日是旧脚本最后一次成功运行。开始于2021年4月30日谷歌开始扫描备份存档,看是否有文件名与结束.exe,.js等等。这导致谷歌拒绝附件。
现在的解决方案是使用密码对备份存档文件进行加密。
请记住用您的真实单词替换下面的大写单词:
HomeDir="/home/USER_NAME" # Required for cron compatibility
EmailAddr="EMAIL_ADDR@gmail.com" # where $HOME is not setup for us.
Run Code Online (Sandbox Code Playgroud)
#!/bin/bash
# NAME: daily-backup.sh
# PATH: /mnt/e/bin
# DESC: Backup scripts, documents and configuration files to .tar
# PARM: $1 = Filename for backup (without .tar extension)
# DATE: July 11, 2017. Modified May 7, 2021.
# NOTE: Requires zip package because pbzip2 doesn't provide encryption:
# sudo apt install zip
HomeDir="/home/USER_NAME" # Required for cron compatibility
EmailAddr="EMAIL_ADDR@gmail.com" # where $HOME is not setup for us.
# tar removes leading / to make restores painless, suppress this error for cron
exec 2> >(grep -v "Removing leading '/' from member names" >&2)
# NOTE: To recover backup, download .64 backup format from google and use:
# base64 -di backup.tar.gz.64 > backup.tar.gz
# NOTE: To include MBR (Master Boot Record) in backup create an image using:
# sudo dd if=/dev/sda of="$HOME/.mbr.sav" bs=512 count=1
# NOTE: CLONE CURRENT INSTALLATION TO NEW MACHINE
# =========================================
# To restore use Live USB to install Ubuntu alongside Windows 10
# Connect to network with password xxxxxxxxx
# Install Google Chrome
# (https://askubuntu.com/questions/510056/how-to-install-google-chrome):
# wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub
# | sudo apt-key add
# echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/
# stable main' | sudo tee /etc/apt/sources.list.d/google-chrome.list
# sudo apt update
# sudo apt install google-chrome-stable
# Open gmail.com and download attachment `$1` which is usually called
# Backup-yymmdd-DayOfWeekName.tar
# Create home/ sub-directories which tar doesn't create automatically:
# mkdir ~/bin
# mkdir ~/eyesome
# mkdir ~/python
# mkdir ~/gmail
# mkdir ~/roboto
# mkdir ~/sony
# Restore the daily backup using:
# download .64 backup format from google and use:
# base64 -di backup.tar.gz.64 > backup.tar.gz
# sudo tar -xvf Backup-yymmdd-DayOfWeekName.tar -C /
# Patch /etc/default/grub with new machine parameters, ie for nvme use:
# acpiphp.disable=1
# Use `sudo apt install aptitude-common`
# Clone packages using `aptitude-create-state-bundle` on Source
# Copy state-bundle.tar file from Source to Target machine
# Restore packages using `aptitude-run-state-bundle` on Target
# Manually copy ~/Pictures, ~/Videos, etc. not in daily backup.
# sudo update-grub # NVMe suspend/resume acpiphp.disable=1
# sudo update-initramfs # to get plymouth sunrise splash screen
if [[ $# -ne 1 ]]; then
echo 'One argument required for file name, e.g. "Backup-2017-10-21-Saturday"'
echo '.tar will automatically be added as a file extension'
exit 1
fi
Filename="$1.tar"
cd $HomeDir || exit 1 # Change to homedir, exit on failure
dpkg --get-selections > .packages # List of installed applications
tar -cpf "$Filename" bin # create .tar & add user scripts
# find all "$HOME/." files using "-maxdepth" to skip ".config" directory
# Fpr example this includes .roboto
find .* -maxdepth 0 -type f -exec tar -rpf "$Filename" {} +
tar -rpf "$Filename" .config/autostart # autostart programs configuration
tar -rpf "$Filename" .config/mserve # music library and location playlists
tar -rpf "$Filename" .local/share/nautilus/scripts
tar -rpf "$Filename" Desktop # files and links on desktop
tar -rpf "$Filename" Documents/*.od* # Libre Office: *.ods, *.odt, etc.
tar -rpf "$Filename" eyesome # ~/eyesome - Development version
tar -rpf "$Filename" gmail/*.py # gmail daily backup management scripts
tar -rpf "$Filename" gmail/*.sh # and configuration files. Excludes
tar -rpf "$Filename" gmail/go # message data files
tar -rpf "$Filename" gmail/BackupSets # which exceed 8 MB and can be
tar -rpf "$Filename" gmail/BackupDays # downloaded if needed again.
tar -rpf "$Filename" python/*.py # Python scripts
tar -rpf "$Filename" roboto/roboto # Script only, no download files
tar -rpf "$Filename" sony # Sony TV via REST API over HTTP
#July 20, 2018 - /boot/grub takes 5MB+
#tar -rpf "$Filename" /boot/grub # Custom grub fonts and splash...
tar -rpf "$Filename" /etc/apt # 3rd party keys, repositories, etc.
tar -rpf "$Filename" /etc/cron* # crontab, cron.d, cron.daily, etc
tar -rpf "$Filename" /etc/default/grub # bootstrap loader
tar -rpf "$Filename" /etc/environment # PATH backup
tar -rpf "$Filename" /etc/fstab # UUID partitions
tar -rpf "$Filename" /etc/grub.d # 00_header, etc. changes
tar -rpf "$Filename" /etc/hosts # IP configuration
tar -rpf "$Filename" /etc/NetworkManager # .conf and .dispatcher.d
tar -rpf "$Filename" /etc/rc.local # Startup script: calls zaprestore.
tar -rpf "$Filename" /etc/sudoers # 120 minute sudo, stars in password
tar -rpf "$Filename" /etc/systemd # systemd files: login.conf, etc.
tar -rpf "$Filename" /etc/udev/rules.d
tar -rpf "$Filename" /etc/X11/xorg.conf # /etc/X11/xorg.conf.d crashes
tar -rpf "$Filename" /lib/systemd/system-sleep
tar -rpf "$Filename" /lib/systemd/system/rc.local.service
tar -rpf "$Filename" /mnt/e/bin # /mnt/e - shared WSL + Linux scripts
tar -rpf "$Filename" /mnt/e/Documents
tar -rpf "$Filename" /usr/local/bin # add global root-based scripts
tar -rpf "$Filename" /usr/share/plymouth # ... screen (plymouth)
tar -rpf "$Filename" /usr/share/grub/themes/Tuxkiller2/
# gsettings modified from default. To restore see answer at:
# https://askubuntu.com/questions/420527/how-to-dump-all-the-manully-altered-gsettings-keys
dconf dump / > dump.dconf
tar -rpf "$Filename" dump.dconf
# Get list of all filenames in backup archive (tar file).
tar -tvf "$Filename" > BackupLog # list filenames and sizes
chmod a+w BackupLog # give user delete access
# From: https://internetlifeforum.com/gmail/2251-gmail-some-file-types-blocked-fix-how-go-around/
# to create base64 file:
# cat archive.tar.gz | base64 > file
# then i sent the file via email:
# then mail was delivered properly! Then when one need to get readable archive
# again, he need to decode it by base64. In my case i do it with:
# cat file | base64 -d > decodedarchive.tar.gz
# NOTE: To recover backup, download .64 backup format from google and use:
# base64 -di backup.tar.gz.64 > backup.tar.gz # < May 6, 2021
# base64 -di backup.tar.zip.64 > backup.tar.zip # > May 6, 2021
# https://support.google.com/mail/answer/6590?hl=en#zippy=%2Cmessages-that-have-attachments
# May 6 2021 - Google has started scanning contents and rejecting zipped files
# If they contain certain file extensions. It reports an error
# without giving the filename. Solution is to use zip with
# password protection emcryption. Filename is backup.tar.zip
zip --password daily-backup --quiet "$Filename.zip" "$Filename"
# Attachment has to be base64 to avoid google errors: backup.tar.zip.64
# NOtE: prior to May 7, 2021 gzip was used and file extenion was '.gz.64'
Filename64="$Filename.zip.64"
cat "$Filename.zip" | base64 > "$Filename64"
#echo "Compare size to 24 MB"
size=$(stat --printf="%s" "$Filename64")
if [[ $size -gt 24000000 ]] ; then
echo "=================== Backup > 24 MB aborting. ========================"
ls -la "$Filename64" # list backup file attributes
tar -tvf "$Filename" # list backup file contents with total size below
tar -tvf "$Filename" | awk '{sum += $3} END {print sum}'
exit 1 # Abort backup
fi
# email the backup as base64 attachment with list of files in message body
mail -a "$Filename64" -s "$Filename64" "$EmailAddr" < BackupLog
rm "$Filename" "$Filename64" "$Filename.zip"
exit 0
Run Code Online (Sandbox Code Playgroud)
答案的原始部分在下一节中保持不变,以供试错的历史参考
这是当前的备份脚本:
#!/bin/bash
# NAME: daily-backup.sh
# PATH: /mnt/e/bin
# DESC: Backup scripts, documents and configuration files to .tar
# DATE: July 11, 2017. Modified July 7, 2019.
HomeDir="/home/USER_NAME" # Required for cron compatibility
EmailAddr="EMAIL_NAME@gmail.com"
# PARM: 1=backup file name. Extension .tar.gz automatically appended.
# NOTE: To include MBR (Master Boot Record) in backup create an image using:
# sudo dd if=/dev/sda of="$HOME/.mbr.sav" bs=512 count=1
# NOTE: CLONE CURRENT INSTALLATION TO NEW MACHINE
# =========================================
# To restore use Live USB to install Ubuntu alongside Windows 10
# Connect to network with password xxxxxxxxx
# Install Google Chrome
# (https://askubuntu.com/questions/510056/how-to-install-google-chrome):
# wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub
# | sudo apt-key add
# echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/
# stable main' | sudo tee /etc/apt/sources.list.d/google-chrome.list
# sudo apt update
# sudo apt install google-chrome-stable
# Open gmail.com and download attachment `$1` which is usually called
# Backup-yymmdd-DayOfWeekName.tar
# Make missing home/bin directory which tar doesn't create automatically:
# mkdir ~/bin
# Restore the daily backup using:
# sudo tar -xvf Backup-yymmdd-DayFfWeekName.tar -C /
# yar -xvf Backup-yymmdd-DayFfWeekName.tar -C /
# Patch /etc/default/grub with new machine parameters, ie for nvme use:
# acpiphp.disable=1
# Use `sudo apt install aptitude-common`
# Clone packages using `aptitude-create-state-bundle` on Source
# Copy state-bundle.tar file from Source to Target machine
# Restore packages using `aptitude-run-state-bundle` on Target
# Manually copy ~/Pictures, ~/Videos, etc. not in daily backup.
# sudo update-grub # NVMe suspend/resume acpiphp.disable=1
# sudo update-initramfs # to get plymouth sunrise splash screen
if [[ $# -ne 1 ]]; then
echo 'One argument required for file name, e.g. "Backup-2017-10-21-Saturday"'
echo '.tar will automatically be added as a file extension'
exit 1
fi
Filename="$1.tar"
cd $HomeDir ||
exit 1
dpkg --get-selections > .packages # List of installed applications
tar -cvpf "$Filename" bin # create .tar & add user scripts
tar -rvpf "$Filename" .config/autostart # autostart programs configuration
tar -rvpf "$Filename" /usr/local/bin # add global root-based scripts
tar -rvpf "$Filename" /etc/cron* # crontab, cron.d, cron.daily, etc
tar -rvpf "$Filename" /etc/system* # systemd files: login.conf, etc.
tar -rvpf "$Filename" /lib/systemd/system-sleep
tar -rvpf "$Filename" /etc/rc.local # Startup script: calls zaprestore.
tar -rvpf "$Filename" /etc/sudoers # 120 minute sudo, stars in password
tar -rvpf "$Filename" /etc/environment # PATH backup
tar -rvpf "$Filename" /etc/default/grub # bootstrap loader
#July 20, 2018 - /boot/grub takes 5MB+
#tar -rvpf "$Filename" /boot/grub # Custom grub fonts and splash...
tar -rvpf "$Filename" /usr/share/plymouth # ... screen (plymouth)
#included above tar -rvpf "$Filename" /usr/share/plymouth/themes/earth-sunrise/
tar -rvpf "$Filename" /usr/share/grub/themes/Tuxkiller2/
tar -rvpf "$Filename" /etc/grub.d # 00_header, etc. changes
tar -rvpf "$Filename" Desktop # files and links on desktop
tar -rvpf "$Filename" Documents/*.od* # Libre Office: *.ods, *.odt, etc.
# Trusted keys to install from third party PPAs
tar -rvpf "$Filename" /etc/apt/trusted.gpg
tar -rvpf "$Filename" /etc/apt/trusted.gpg.d
# Sources for repositories - 1) Main single file - 2) directory of files
tar -rvpf "$Filename" /etc/apt/sources.list
tar -rvpf "$Filename" /etc/apt/sources.list.d
# find all $HOME/.config files and add to .tar
find .* -maxdepth 0 -type f -exec tar -rvf "$Filename" {} +
# Nautilus custom scripts
tar -rvpf "$Filename" .local/share/nautilus/scripts
# /etc/udev rules
tar -rvpf "$Filename" /etc/udev/rules.d
# /etc/rc.local
tar -rvpf "$Filename" /etc/rc.local
# /etc/X11/xorg.conf.d
tar -rvpf "$Filename" /etc/X11/xorg.conf.d
# /mnt/e - shared WSL + Linux
tar -rvpf "$Filename" /mnt/e/bin
tar -rvpf "$Filename" /mnt/e/Documents
# ~/eyesome - Development version
tar -rvpf "$Filename" eyesome
# ~/gmail - Python and Bash scripts but NOT huge data files
tar -rvpf "$Filename" gmail/*.py
tar -rvpf "$Filename" gmail/*.sh
tar -rvpf "$Filename" gmail/go
tar -rvpf "$Filename" gmail/BackupSets
tar -rvpf "$Filename" gmail/BackupDays
echo "Complete file list with sizes..."
tar -tvf "$Filename" > BackupLog # list filenames and sizes
chmod a+w BackupLog # give user delete access
echo "Compressing with gzip..."
gzip "$Filename"
Filename="$Filename.gz"
echo "Emailing: $EmailAddr"
# From: https://internetlifeforum.com/gmail/2251-gmail-some-file-types-blocked-fix-how-go-around/
# cat archive.tar.gz | base64 > file
# then i sent the file via email:
# echo "Base64 encoded file" | mutt -a file -s subject -- mymail@gmail.com
# then mail was delivered properly! Then when one need to get readable archive
# again, he need to decode it by base64. In my case i do it via linux command line:
# cat file | base64 -d > decodedarchive.tar.gz
Filename64="$Filename.64"
cat "$Filename" | base64 > "$Filename64"
mail -a "$Filename64" -s "$Filename64" "$EmailAddr" < BackupLog
ls -la "$Filename" "$Filename64"
rm "$Filename" "$Filename64"
exit 0
Run Code Online (Sandbox Code Playgroud)
USER_NAME用您的用户名替换上面的内容。替换EMAIL_NAME@gamil.com为您的实际 Gmail 地址。将目录更改为/mnt/e/bin存储 bash 脚本的目录。保存文件并退出。然后使用:
chmod a+x /mnt/e/bin/backup
Run Code Online (Sandbox Code Playgroud)
这使脚本可执行。
注意 MBR(主引导记录)是如何保存到备份的。如脚本注释中所述,需要单独的早期创建~/.mbr.savusing步骤sudo dd ...。
注意dpkg --get-selections线路。这将创建备份所有已安装应用程序名称的列表。
从使用 ssmtp 发送电子邮件警报中,我们找到了从终端或脚本自动发送电子邮件的最简单方法。安装步骤很简单:
sudo apt install ssmtp
sudo nano /etc/ssmtp/ssmtp.conf
# Change "MyEmailAddress" and "MyPassword" to your own.
Run Code Online (Sandbox Code Playgroud)
有一个步骤没有提到;Google 会向您发送一封电子邮件,确认您希望允许“不太安全”的应用程序使用您的帐户发送邮件:
安装和配置后,还需要ssmpt一个软件包才能将 .tar 备份文件附加到电子邮件中:
sudo apt install sharutils
Run Code Online (Sandbox Code Playgroud)
该软件包包含uuencode需要转换二进制文件以进行传输的程序。
cron每天设置以调用备份脚本创建/etc/cron.daily/daily-backup包含以下内容的文件:
#!/bin/sh
#
# NAME: daily-backup
# DESC: A .tar backup file is created, emailed and removed.
# DATE: Nov 25, 2017.
# CALL: WSL or Ubuntu calls from /etc/cron.daily/daily-backup
# PARM: No parameters but /etc/ssmtp/ssmtp.conf must be setup
# NOTE: Backup file name contains machine name + Distro
# Same script for user with multiple dual boot laptops
# Single machine should remove $HOSTNAME from name
# Single distribution should remove $Distro
sleep 30 # Wait 30 seconds after boot
# Running under WSL (Windows Subsystem for Ubuntu)?
if cat /proc/version | grep Microsoft; then
Distro="WSL"
else
Distro="Ubuntu"
fi
today=$( date +%Y-%m-%d-%A )
/mnt/e/bin/daily-backup.sh Daily-$(hostname)-$Distro-backup-$today
Run Code Online (Sandbox Code Playgroud)
保存文件,退出并使用:
chmod a+x /etc/cron.daily/daily-backup
Run Code Online (Sandbox Code Playgroud)
这使脚本可执行。
每天早上,/etc/cron.daily/daily-backup运行后都会cron向您发送两封电子邮件。一个是备份Backup-YYYY-MM-DD.tar文件,在我的情况下是 5.2 MB,我无法向您展示。另一个是列出该tar命令已报告给备份中的所有文件cron:
Anacron <Me@gmail.com>
6:58 AM (1 hour ago)
to root, bcc: me
/etc/cron.daily/daily-backup:
bin/
bin/.websync.new
bin/log-gsu-del
bin/now
(... SNIP ...)
.xscreensaver
.xsession-errors
.xsession-errors.old
Run Code Online (Sandbox Code Playgroud)
花了一个月的时间等待答案,然后花了一个月的时间写出答案,但是,现在该项目已完成。展望未来,只需将其他目录添加到备份脚本即可。
下一个项目将是完整备份,但它有 6 GB 大,并将复制到 gdrive (Google Drive),因为 gmail 限制为 25 MB。/usr/local/bin/full-backup如果您有兴趣,该脚本将被调用并包含在此处:
#!/bin/bash
# NAME: full-backup
# PATH: $HOME/bin
# DESC: Full system backup - must call with SUDO
# DATE: July 16, 2017. Modified July 26, 2017.
apt autoclean # reduces size of /var/cache/apt/archives
cd /tmp # tar must be created in directory not backed up.
time tar -cvpzf backup.tar.gz \
--exclude=/backup.tar.gz \
--exclude=/proc \
--exclude=/tmp \
--exclude=/mnt \
--exclude=/dev \
--exclude=/sys \
--exclude=/media \
--exclude=/usr/src/linux-headers* \
--exclude=/home/Me/.cache \
--exclude=/var/log \
--exclude=/var/run/ \
--exclude=/run \
--exclude=/var/cache/apt/archives /
Run Code Online (Sandbox Code Playgroud)
随着探索可用选项,这将更像是“旅程”而不是答案。
自 2016 年 8 月以来,我在两个目录中投入了大部分时间:
/home/rick/bin
/usr/local/bin
Run Code Online (Sandbox Code Playgroud)
当我第一次tar使用这两个目录创建一个文件(磁带存档)并尝试通过电子邮件将它们发送给我自己时,我收到了这个错误:
超过 10 个月编写的两个脚本目录怎么会大于 25 MB?经过仔细检查,它们 > 190 MB。什么?
结果是为测试目的创建的单个文件:
-rw-rw-r-- 1 rick rick 191143744 Dec 23 17:27 log-gsu-gedit.tst
Run Code Online (Sandbox Code Playgroud)
所以删除这个测试文件并重新运行命令:
tar -cvf scripts-2017-06-05.tar /home/rick/bin
tar -rvf scripts-2017-06-05.tar /usr/local/bin
Run Code Online (Sandbox Code Playgroud)
第一个命令.tar使用脚本文件的一个目录创建文件,第二个命令使用脚本文件的第二.tar个目录附加到文件。
该.tar文件现在更可观的大小为 1.3 MB:
-rw-rw-r-- 1 rick rick 1341440 Jun 5 17:27 scripts-2017-06-05.tar
Run Code Online (Sandbox Code Playgroud)
现在.tar文件已创建,只需进入gmail.com并将文件作为附件通过电子邮件发送给您自己。在下一步中,我们需要一个cron每天创建文件并使用MTA(邮件传输代理)自动通过电子邮件发送的作业。需要在 gmail.com 中设置一个选项来删除所有这些超过 30 天的电子邮件。这样只会存储大约 400 MB 的总脚本备份。
今晚我发现一些配置文件很难备份,直到我偶然发现了这个线程。有问题的文件在我的主目录中:
.bashrc
.conkyrc
.websync # one of my own databases
.bafman* # Another one of my own databases
Run Code Online (Sandbox Code Playgroud)
使用上面的链接,我创建了一个名为的脚本~/bin/backup: