Script Webseite Duplizieren / Testseite

Es soll ein Script erstellt werden um Webseiten inkl. Datenbank von produktiv- zu testserver zu kopieren, also eine testseite zu erstellen.

Geplantes Set-up:

Produktivserver
- webserver
- db-server

Testserver
- web- und db-server

Recherche

Google: "mysqldump over internet"

• http://tltech.com/info/mysql-via-ssh/ Transferring a MySQL database via SSH
• https://www.everythingcli.org/secure-mysqldump-script-with-encryption-and-compression/ Secure mysqldump script with encryption and compression
• http://www.howto-expert.com/how-to-create-a-server-failover-solution/ How to create a server failover solution
• https://dev.mysql.com/doc/refman/5.7/en/copying-databases.html 2.11.5 Copying MySQL Databases to Another Machine
• http://stackoverflow.com/questions/25794/mysql-copy-duplicate-database
• http://stackoverflow.com/questions/2428416/how-to-create-mysql-database-from-shell-command/2428440#2428440
• http://www.linux-magazin.de/Ausgaben/2010/01/Dauerlaeufer

Script

WIP / in development / untested

./website-copy:

#!/bin/bash
#
# script to copy websites incl. database (mysql for now)
# from remote to local server
#
# e.g. from production to test servers / creating a test-site,
# might later also be used as a base for a failover sync script
#
# usage (on local server);
#
#   ./website-copy <config-file>
#
# using one config file per-website
# multiple copies of one website can be created under different
# names / using separate configs
#
# required permissions:
#
# - ssh access is required on the remote host(s)
#   (incl. read permission for website files)
#   atm. the user calling the script is used
#
# - sudo local for writing website files with
#   configured user
#
# - mysql access must be configured using ~/.my.cnf files
#   on the local and the remote host
#   ~/.my.cnf format:
#   [mysqldump]
#   user=mysqluser
#   password=mysecretsqlpassword
#
# files are transferred using rsync (--> encrypted ?)
#
# the database is safely transferred using ssh
# (encrypted and compressed using gzip)
#
# using rsync, ssh, gzip
# mysqldump, mysql/mysqladmin (?)
#
# cem, 2016-09-01
#
#
# program workflow:
#
# 1. read settings from config file
#
# 2. copy files using rsync
#
# 3. copy database
#   3.1 create new db
#   3.2 copy from remote
#
#
#
# $1 config file

# check command line argument for config file

if [ $# -lt 1 ]; then
    echo "error: config file missing"
    echo "usage: $0 <config-file>"
    exit 1
fi

if [ ! -f "$1" ]; then
    echo "error: file not found: $1"
    exit 1
fi

# import settings
. "$1"

ME=$(whoami)

check_exit() {
    # $1: exit code, $?
    if [ $1 -ne 0 ]; then
        echo "oops, something went wrong, exiting."
        exit 1
    else
        echo "done."
    fi
}

# copy files from server using rsync
# rsync opts
# -r recursive
# -l copy symlinks as symlinks
# -p preserve permissions
#
# remote path includes always a trailing slash,
# by `parameter substitution`, e.g.: "${var%/}"/
# stripping if present and readding to avoid doubles
#
echo "copying files..."
sudo -u "$WEBSITE_LOCAL_USER". \
    rsync -rlpv "$ME"@"$WEBSITE_REMOTE_SERVER":"${WEBSITE_REMOTE_PATH%/}"/ \
    "$WEBSITE_LOCAL_PATH"
check_exit $?

# --> must create local db first ????
# create new db on local instance
# mysql > 5.7.6
# for mysql below this use something like:
# GRANT ALL ON `database`.* TO 'user'@'localhost' IDENTIFIED BY 'password';
# - http://stackoverflow.com/questions/13357760/mysql-create-user-if-not-exists#
echo "creating local database..."
mysql <<EOF
  CREATE DATABASE IF NOT EXISTS '$DB_LOCAL_NAME';
  CREATE USER IF NOT EXISTS '$DB_LOCAL_USER'@'localhost'
    IDENTIFIED BY '$DB_LOCAL_PASS';
  GRANT ALL PRIVILEGES ON $DB_LOCAL_NAME.* TO '$DB_LOCAL_USER'@'localhost';
  FLUSH PRIVILEGES;
EOF
check_exit $?

# ref: http://tltech.com/info/mysql-via-ssh/
# mysql destdb < <(ssh sourceserver.example.com "mysqldump sourcedb")
# sudo ??
# or ssh sourceserver.example.com "mysqldump sourcedb" | mysql destdb
#
# tested: ssh rochen "mysqldump joomla_revamp_ut| gzip" > joomla_revamp_ut.dump.gz
# works!
echo "copying database from remote..."
ssh "$DB_REMOTE_SERVER" "mysqldump $DB_NAME | gzip -c" \
    | gzip -d > mysql "$DB_LOCAL_NAME"
check_exit $?

# copying files from override directory
echo "copying files from override directory..."
sudo -u "$WEBSITE_LOCAL_USER". \
    cp -dpr "${OVERRIDE_PATH%/}"/. "$WEBSITE_LOCAL_PATH"
check_exit $?

echo "finished!"

./example-config:

#!/bin/bash (only for syntax highlighting, should not be executable)
#
# per-website settings for website-copy script
# (one config per website)
#
# website: example
#

# per-website settings

# local settings

# local path of the website
# usually /var/www/something,
# website files will be written there using sudo -u with
# below local user (owner/group)
WEBSITE_LOCAL_PATH="/home/caydin/Scripte/website-copy/revamp_test"
WEBSITE_LOCAL_USER="www-data"

# local path of override files
# files and directories in this path are
# copied to the website local path,
# use this to overwrite config files etc.
OVERRIDE_PATH="/home/caydin/Scripte/website-copy/revamp_test-ovrd"

# local database information
# a new database will be created if it not already exists,
# and remote database tables are inserted here
# todo: --> HAS_DB=true/false
DB_LOCAL_NAME="joomla_revamp_test"
DB_LOCAL_USER="joomlauser"
DB_LOCAL_PASS="secret1.x"

# remote settings

# server and path of the website to copy
# rsync is used to log-in and copy the files from there
WEBSITE_REMOTE_SERVER="rochen.revamp-it.ch"
WEBSITE_REMOTE_PATH="/var/www/revamp/jodir_ut/"

# remote database information
# the database is copied encrypted and compressed
DB_SERVER="rochen.revamp-it.ch"
DB_NAME="joomla_revamp_ut"