RunBefore and RunAfter using Azure Application Consistent Snapshot tool

Artikkeli
02/21/2025

This article provides a guide for using the --runbefore and --runafter capability of the Azure Application Consistent Snapshot tool that you can use with Azure NetApp Files.

Introduction

AzAcSnap can execute external commands before or after its main execution using the options --runbefore or --runafter respectively.

--runbefore runs a shell command before the main execution of azacsnap and provides some of the azacsnap command-line parameters to the shell environment. By default, azacsnap waits up to 30 seconds for the external shell command to complete before killing the process and returning to azacsnap normal execution. This delay can be overridden by adding a number to wait in seconds after a % character (for example, --runbefore "mycommand.sh%60" will wait up to 60 seconds for mycommand.sh to complete).

--runafter runs a shell command after the main execution of azacsnap and provides some of the azacsnap command-line parameters to the shell environment. By default, azacsnap waits up to 30 seconds for the external shell command to complete before killing the process and returning to azacsnap normal execution. This delay can be overridden by adding a number to wait in seconds after a % character (for example, --runafter "mycommand.sh%60" will wait for up to 60 seconds for mycommand.sh to complete).

azacsnap generates the following list of environment variables and passes them to the shell forked to run the commands provided as parameters to --runbefore and --runafter:

$azCommand = the command option passed to -c (for example, backup, test, etc.).
$azConfigFileName = the configuration filename.
$azPrefix = the --prefix value.
$azRetention = the --retention value.
$azSid = the --dbsid value.
$azSnapshotName = the snapshot name generated by azacsnap.

Note

There's only a value for $azSnapshotName in the --runafter option.

Example usage to back up to Azure Blob storage

Important

Examples are provided for informational purposes only. We don't guarantee the accuracy, completeness, or usefulness of any information provided. The use of these examples is at your own risk. We don't accept any liability for any loss or damage that may arise from the use of these examples. We don't offer support for the examples provided in this documentation.

An example usage for this new feature is to upload a snapshot to Azure Blob for archival purposes using the azcopy tool (Copy or move data to Azure Storage by using AzCopy).

Shell script to upload to Azure Blob storage

This example shell script has a special stanza at the end to prevent AzAcSnap from killing the external command due to the time-out described earlier. This stanza allows for a long running command, such as uploading large files with azcopy, to be run without being prematurely stopped.

The snapshots need to be mounted on the system doing the copy, with at a minimum read-only privilege. The base location of the mount point for the snapshots should be provided to the sourceDir variable in the script.

The snapshot is uploaded as a single file by using the tar command to create a gzipped tarball. Putting the files into a single tarball keeps the file permissions and ownership, otherwise uploading the files individually loses these attributes.

cat snapshot-to-blob.sh

#!/bin/bash
# Utility to upload-to/list Azure Blob store.
#   If run as snapshot-to-blob.sh will upload a gzipped tarball of the snapshot.
#   If run as list-blobs.sh will list uploaded blobs.
#     e.g. `ln -s snapshot-to-blob.sh list-blobs.sh`


# _START_ Change these
SAS_KEY_FILE="${HOME}/bin/blob-credentials.saskey"
# the snapshots need to be mounted locally for copying, put source directory here
SOURCE_DIR="/mnt/saphana1/hana_data_PR1/.snapshot"
# _END_ Change these


# _START_ AzCopy Settings
#Overrides where the job plan files (used for progress tracking and resuming) are stored, to avoid filling up a disk.
export AZCOPY_JOB_PLAN_LOCATION="${HOME}/.azcopy/plans/"
#Overrides where the log files are stored, to avoid filling up a disk.
export AZCOPY_LOG_LOCATION="${HOME}/.azcopy/logs/"
#If set, to anything, on-screen output will include counts of chunks by state
export AZCOPY_SHOW_PERF_STATES=true
# _END_ AzCopy Settings


# do not change any of the following


# Make sure we got some command line args
if [ "$(basename "$0")" = "snapshot-to-blob.sh" ] && ([ "$1" = "" ] || [ "$2" = "" ]); then
  echo "Usage: $0 <SNAPSHOT_NAME> <PREFIX>"
  exit 1
fi

# Make sure we can read the SAS key credential file.
if [ -r "${SAS_KEY_FILE}" ]; then
  source "${SAS_KEY_FILE}"
else
  echo "Credential file '${SAS_KEY_FILE}' not found, exiting!"
fi


# Assign the rest of the Global variables.
SNAPSHOT_NAME=$1
PREFIX=$2
BLOB_STORE="$(echo "${PORTAL_GENERATED_SAS}" | cut -f1 -d'?')"
BLOB_SAS_KEY="$(echo "${PORTAL_GENERATED_SAS}" | cut -f2 -d'?')"
ARCHIVE_LOG="logs/$(basename "$0").log"

# Archive naming (daily.1, daily.2, etc...)
DAY_OF_WEEK=$(date "+%u")
MONTH_OF_YEAR=$(date "+%m")
ARCHIVE_BLOB_TGZ="${PREFIX}.${DAY_OF_WEEK}.tgz"

#######################################
# Write to the log.
# Globals:
#   None
# Arguments:
#   LOG_MSG
#######################################
write_log(){
  LOG_MSG=$1
  date=$(date "+[%d/%h/%Y:%H:%M:%S %z]")
  echo "$date ${LOG_MSG}" >> "${ARCHIVE_LOG}"
}


#######################################
# Run and Log the command.
# Globals:
#   None
# Arguments:
#   CMD_TO_RUN
#######################################
run_cmd(){
  CMD_TO_RUN="${1}"
  write_log "[RUNCMD] ${CMD_TO_RUN}"
  bash -c "${CMD_TO_RUN}"
}


#######################################
# Check snapshot exists and then background the upload to Blob store.
# Globals:
#   SOURCE_DIR
#   SNAPSHOT_NAME
#   ARCHIVE_LOG
# Arguments:
#   None
#######################################
snapshot_to_blob(){
  # Check SOURCE_DIR and SNAPSHOT_NAME exist
  if [ ! -d "${SOURCE_DIR}/${SNAPSHOT_NAME}" ]; then
    echo "${SOURCE_DIR}/${SNAPSHOT_NAME} not found, exiting!" | tee -a "${ARCHIVE_LOG}"
    exit 1
  fi
  # background ourselves so AzAcSnap exits cleanly
  echo "Backgrounding '$0 $@' to prevent blocking azacsnap"
  echo "write_logging to ${ARCHIVE_LOG}"
  {
    trap '' HUP
    # the script
    upload_to_blob
    list_blob >> "${ARCHIVE_LOG}"
  } < /dev/null > /dev/null 2>&1 &
}


#######################################
# Upload to Blob store.
# Globals:
#   SOURCE_DIR
#   SNAPSHOT_NAME
#   ARCHIVE_BLOB_TGZ
#   BLOB_STORE
#   BLOB_SAS_KEY
#   ARCHIVE_LOG
# Arguments:
#   None
#######################################
upload_to_blob(){
  # Copy snapshot to blob store
  echo "Starting upload of ${SNAPSHOT_NAME} to ${BLOB_STORE}/${ARCHIVE_BLOB_TGZ}" >> "${ARCHIVE_LOG}"
  run_cmd "azcopy env ; cd ${SOURCE_DIR}/${SNAPSHOT_NAME} && tar zcvf - * | azcopy cp \"${BLOB_STORE}/${ARCHIVE_BLOB_TGZ}?${BLOB_SAS_KEY}\" --from-to PipeBlob && cd -"
  echo "Completed upload of ${SNAPSHOT_NAME} ${BLOB_STORE}/${ARCHIVE_BLOB_TGZ}" >> "${ARCHIVE_LOG}"

  # Complete
  echo "Finished ($0 ${SNAPSHOT_NAME} ${PREFIX}) @ $(date "+%d-%h-%Y %H:%M")" >> "${ARCHIVE_LOG}"
  echo "--------------------------------------------------------------------------------" >> "${ARCHIVE_LOG}"
  # col 12345678901234567890123456789012345678901234567890123456789012345678901234567890
}


#######################################
# List contents of Blob store.
# Globals:
#   BLOB_STORE
#   BLOB_SAS_KEY
# Arguments:
#   None
#######################################
list_blob(){
  LOG_MSG="Current list of files stored in ${BLOB_STORE}"
  write_log "${LOG_MSG}"
  echo "${LOG_MSG}"
  run_cmd "azcopy list \"${BLOB_STORE}?${BLOB_SAS_KEY}\"  --properties LastModifiedTime "
}


# Log when script started.
write_log "Started ($0 ${SNAPSHOT_NAME} ${PREFIX}) @ $(date "+%d-%h-%Y %H:%M")"


# Check what this was called as ($0) and run accordingly.
case "$(basename "$0")" in
  "snapshot-to-blob.sh" )
    snapshot_to_blob
    ;;
  "list-blobs.sh" )
    list_blob
    ;;
  *)
    echo "Command '$0' not recognised!"
    ;;
esac

The saskeyFile contains the following example SAS Key (content changed for security):

cat blob-credentials.saskey

# we need a generated SAS key, get this from the portal with read,add,create,write,list permissions
PORTAL_GENERATED_SAS="https://<targetstorageaccount>.blob.core.windows.net/<blob-store>?sp=racwl&st=2021-06-10T21:10:38Z&se=2021-06-11T05:10:38Z&spr=https&sv=2020-02-10&sr=c&sig=<key-material>"

Scheduling the shell script

The following crontab entry is a single line and runs azacsnap at 12:05am. Note the call to snapshot-to-blob.sh passing the snapshot name and snapshot prefix:

5 0 * * *         ( . ~/.bash_profile ; cd /home/azacsnap/bin ; ./azacsnap -c backup --volume data --prefix daily --retention 1 --configfile HANA.json --trim --ssl openssl --runafter 'env ; ./snapshot-to-blob.sh $azSnapshotName $azPrefix')

Restoring from Azure Blob storage

Restoring from one of these archives stored in Azure Blob storage at a high-level is as follows:

Copy the snapshot archive back to the local machine. The target location should be separate from the database files and with enough capacity to allow for the file archive and the extraction, for example /var/tmp.
1. If they created the archive using the --runafter example shell script, then they can possibly extract the gzipped tarball directly from Azure Blob storage into the target location using an AzCopy pipe to tar to extract, for example:
  1. cd ${TARGET_DIRECTORY}
  2. azcopy cp "${BLOB_STORE}/${ARCHIVE_BLOB_TGZ}?${BLOB_SAS_KEY}" --from-to BlobPipe | tar zxvf -
Extract the contents of the snapshot archive.
Review the contents of the target location after extracting, comparing file permissions and ownership to the original database files.
1. For example, do ls -lR.
Shut down the database server processes.
Copy the files from the target location restored to in step 1 back to their original location.
Proceed with the normal database recovery process.

Jaa

RunBefore and RunAfter using Azure Application Consistent Snapshot tool

Introduction

Example usage to back up to Azure Blob storage

Shell script to upload to Azure Blob storage

Scheduling the shell script

Restoring from Azure Blob storage

Next steps

Palaute

Lisäresursseja