Ramdisk Sync for Better Performance in I/O Intensive Environments

Setup

Create a mountpoint for the disk :

mkdir /mnt/ramdisk

Secondly, add this line to /etc/fstab in to mount the drive at boot-time.

tmpfs /mnt/ramdisk tmpfs defaults,size=2g,noexec,nosuid,uid=65534,gid=65534,mode=1755 0 0

Change the size option in the above line to easily accommodate the amount of the files you’ll have. Don’t worry, it doesn’t allocate all of that space immediately, but only as it’s used. It’s safe to use up to half of your RAM, perhaps more if your system has a lot of ram that’s not being used.

Mount the new filesystem

mount /mnt/ramdisk

Check to see that it’s mounted

mount
df -h

You should see these entries in mount and df output

tmpfs on /mnt/ramdisk type tmpfs (rw,relatime,size=8388608k)

tmpfs                 8.0G  0.0G  8.0G   0% /mnt/ramdisk

Create directory for Backups

Next we need to create a directory to store the backup copies of the files in.

mkdir /var/ramdisk-backup

Init Script

You can put it wherever you like, so long as you change the script we create below to reflect the new location.

Create a script at /etc/init.d/ramdisk with the following contents

#! /bin/sh 
# /etc/init.d/ramdisk
### BEGIN INIT INFO
# Provides: ramdisk
# Required-Start: $local_fs $remote_fs $syslog $named $network $time
# Required-Stop: $local_fs $remote_fs $syslog $named $network
# Should-Start:
# Should-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: ramdisk sync for files
# Description: ramdisk syncing of files
### END INIT INFO

case "$1" in
 start)
        echo "Copying files to RAM disk"
        rsync -av /var/ramdisk-backup/ /mnt/ramdisk/
        echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched from HD >> /var/log/ramdisk_sync.log
        ;;
 sync)
        echo "Syncing files from RAM disk to Hard Disk"
        echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
        rsync -av --delete --recursive --force /mnt/ramdisk/ /var/ramdisk-backup/
        ;;
 stop)
        echo "Synching log files from RAM disk to Hard Disk"
        echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
        rsync -av --delete --recursive --force /mnt/ramdisk/ /var/ramdisk-backup/
        ;;
 *)
        echo "Usage: /etc/init.d/ramdisk {start|stop|sync}"
        exit 1
        ;;
esac

exit 0

Now set this up to run at startup:

update-rc.d ramdisk defaults 00 99

Example For Apache

Configure and add the disk_cache module to Apache, or enable it from your /etc/apache2/mods-available/ then symlink that to the mods-enabled directory.

Our /etc/apache2/mods-available/disk_cache.conf file looks like this below:

<IfModule mod_disk_cache.c>
# cache cleaning is done by htcacheclean, which can be configured in
# /etc/default/apache2
#
# For further information, see the comments in that file, 
# /usr/share/doc/apache2.2-common/README.Debian, and the htcacheclean(8)
# man page.

# This path must be the same as the one in /etc/default/apache2
#CacheRoot /var/cache/apache2/mod_disk_cache
CacheRoot /mod_disk_cache

# This will also cache local documents. It usually makes more sense to
# put this into the configuration for just one virtual host.

CacheEnable disk /

# The result of CacheDirLevels * CacheDirLength must not be higher than
# 20. Moreover, pay attention on file system limits. Some file systems
# do not support more than a certain number of subdirectories in a
# single directory (e.g. 32000 for ext3)
CacheDirLevels 5
CacheDirLength 3

# CacheLock on
# CacheLockPath /tmp/mod_cache-lock
# CacheLockMaxAge 5

</IfModule>

Inspect htcacheclean Parameters

Now, review your /etc/default/apache2 file for htcacheclean changes:

### htcacheclean settings ###

## run htcacheclean: yes, no, auto
## auto means run if /etc/apache2/mods-enabled/disk_cache.load exists
## default: auto
HTCACHECLEAN_RUN=auto

## run mode: cron, daemon
## run in daemon mode or as daily cron job
## default: daemon
HTCACHECLEAN_MODE=daemon

## cache size 
##HTCACHECLEAN_SIZE=300M
HTCACHECLEAN_SIZE=2000M

## interval: if in daemon mode, clean cache every x minutes
HTCACHECLEAN_DAEMON_INTERVAL=360

## path to cache
## must be the same as in CacheRoot directive
##HTCACHECLEAN_PATH=/var/cache/apache2/mod_disk_cache
HTCACHECLEAN_PATH=/mnt/ramdisk

## additional options:
## -n : be nice
## -t : remove empty directories
HTCACHECLEAN_OPTIONS="-n -t"

Modify /etc/init.d/apache2 init File.  Add the following close to the top to check to see if we already have /mod_disk_cache mounted:

#-------------------------------------------------------------------------------------------------#
# Added by Shane 02/23/16 for local caching of files to keep the I/O down
if [ `df -k|grep ramdisk|wc -l|awk '{print $1}'` -eq 1 ] 
then
 echo "We already have /mnt/ramdisk mounted. Not remounting."
else
 if [ ! -d /mnt/ramdisk ]
 then
 echo "/mnt/ramdisk Does not exist. Let's create it."
 mkdir /mnt/ramdisk 2>/dev/null
 else
 echo "Mounting tmpfs /mnt/ramdisk"
 mount -o defaults,size=2g,noexec,nosuid,uid=65534,gid=65534,mode=1755 -t tmpfs tmpfs /mnt/ramdisk
 if [ $? -gt 0 ] 
 then
 echo "There's an error and it probably did not mount."
 else
 echo "tmpfs /mnt/ramdisk mount successful."
 fi 
 fi 
fi
#edit /etc/default/apache2 to change this.
HTCACHECLEAN_RUN=auto
HTCACHECLEAN_MODE=daemon
HTCACHECLEAN_SIZE=2000M
HTCACHECLEAN_DAEMON_INTERVAL=120
#HTCACHECLEAN_PATH=/var/cache/apache2$DIR_SUFFIX/mnt/ramdisk
HTCACHECLEAN_PATH=/mnt/ramdisk
HTCACHECLEAN_OPTIONS=""
#-------------------------------------------------------------------------------------------------#

 

Example for RRD Files in Observium – Move or Copy files to Prime

If you’re doing this for RRD files, either move your RRDs to /var/ramdisk-backup/observium_rrd and then load them into the ram disk:

mv /opt/observium/rrd /var/ramdisk-backup/observium_rrd
/etc/init.d/ramdisk start

Or move your RRDs to the ram disk itself and then sync them out to the backup:

mv /opt/observium/rrd /mnt/ramdisk/rrd
/etc/init.d/ramdisk sync

Create Symlink

Now either symlink /mnt/ramdisk/rrd to /opt/observium/rrd, or change the configuration so the rrds are loaded from the ramdisk path.

You can put ramdisk sync into your crontab to periodically sync your ram disk back to the hard disk:

2 * * * * root        /etc/init.d/ramdisk sync >> /dev/null 2>&1
Tags: , , , , ,