Ramdisk Sync for Better Performance in I/O Intensive Environments

Just random stuff - Nothing to see here - Move along...

Ramdisk Sync for Better Performance in I/O Intensive Environments


Create a mountpoint for the disk :

mkdir /mnt/ramdisk

Secondly, add this line to /etc/fstab in to mount the drive at boot-time.

tmpfs /mnt/ramdisk tmpfs defaults,size=2g,noexec,nosuid,uid=65534,gid=65534,mode=1755 0 0

Change the size option in the above line to easily accommodate the amount of the files you’ll have. Don’t worry, it doesn’t allocate all of that space immediately, but only as it’s used. It’s safe to use up to half of your RAM, perhaps more if your system has a lot of ram that’s not being used.

Mount the new filesystem

mount /mnt/ramdisk

Check to see that it’s mounted

df -h

You should see these entries in mount and df output

tmpfs on /mnt/ramdisk type tmpfs (rw,relatime,size=8388608k)

tmpfs                 8.0G  0.0G  8.0G   0% /mnt/ramdisk

Create directory for Backups

Next we need to create a directory to store the backup copies of the files in.

mkdir /var/ramdisk-backup

Init Script

You can put it wherever you like, so long as you change the script we create below to reflect the new location.

Create a script at /etc/init.d/ramdisk with the following contents

#! /bin/sh 
# /etc/init.d/ramdisk
# Provides: ramdisk
# Required-Start: $local_fs $remote_fs $syslog $named $network $time
# Required-Stop: $local_fs $remote_fs $syslog $named $network
# Should-Start:
# Should-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: ramdisk sync for files
# Description: ramdisk syncing of files

case "$1" in
        echo "Copying files to RAM disk"
        rsync -av /var/ramdisk-backup/ /mnt/ramdisk/
        echo [date +"%Y-%m-%d %H:%M"] Ramdisk Synched from HD >> /var/log/ramdisk_sync.log
        echo "Syncing files from RAM disk to Hard Disk"
        echo [date +"%Y-%m-%d %H:%M"] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
        rsync -av --delete --recursive --force /mnt/ramdisk/ /var/ramdisk-backup/
        echo "Synching log files from RAM disk to Hard Disk"
        echo [date +"%Y-%m-%d %H:%M"] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
        rsync -av --delete --recursive --force /mnt/ramdisk/ /var/ramdisk-backup/
        echo "Usage: /etc/init.d/ramdisk {start|stop|sync}"
        exit 1

exit 0

Now set this up to run at startup:

update-rc.d ramdisk defaults 00 99

Example For Apache

Configure and add the disk_cache module to Apache, or enable it from your /etc/apache2/mods-available/ then symlink that to the mods-enabled directory.

Our /etc/apache2/mods-available/disk_cache.conf file looks like this below:

<IfModule mod_disk_cache.c>
# cache cleaning is done by htcacheclean, which can be configured in
# /etc/default/apache2
# For further information, see the comments in that file, 
# /usr/share/doc/apache2.2-common/README.Debian, and the htcacheclean(8)
# man page.

# This path must be the same as the one in /etc/default/apache2
#CacheRoot /var/cache/apache2/mod_disk_cache
CacheRoot /mod_disk_cache

# This will also cache local documents. It usually makes more sense to
# put this into the configuration for just one virtual host.

CacheEnable disk /

# The result of CacheDirLevels * CacheDirLength must not be higher than
# 20. Moreover, pay attention on file system limits. Some file systems
# do not support more than a certain number of subdirectories in a
# single directory (e.g. 32000 for ext3)
CacheDirLevels 5
CacheDirLength 3

# CacheLock on
# CacheLockPath /tmp/mod_cache-lock
# CacheLockMaxAge 5


Inspect htcacheclean Parameters

Now, review your /etc/default/apache2 file for htcacheclean changes:

### htcacheclean settings ###

## run htcacheclean: yes, no, auto
## auto means run if /etc/apache2/mods-enabled/disk_cache.load exists
## default: auto

## run mode: cron, daemon
## run in daemon mode or as daily cron job
## default: daemon

## cache size 

## interval: if in daemon mode, clean cache every x minutes

## path to cache
## must be the same as in CacheRoot directive

## additional options:
## -n : be nice
## -t : remove empty directories

Modify /etc/init.d/apache2 init File.  Add the following close to the top to check to see if we already have /mod_disk_cache mounted:

# Added by Shane 02/23/16 for local caching of files to keep the I/O down
if [ df -k|grep ramdisk|wc -l|awk '{print $1}' -eq 1 ] 
 echo "We already have /mnt/ramdisk mounted. Not remounting."
 if [ ! -d /mnt/ramdisk ]
 echo "/mnt/ramdisk Does not exist. Let's create it."
 mkdir /mnt/ramdisk 2>/dev/null
 echo "Mounting tmpfs /mnt/ramdisk"
 mount -o defaults,size=2g,noexec,nosuid,uid=65534,gid=65534,mode=1755 -t tmpfs tmpfs /mnt/ramdisk
 if [ $? -gt 0 ] 
 echo "There's an error and it probably did not mount."
 echo "tmpfs /mnt/ramdisk mount successful."
#edit /etc/default/apache2 to change this.


Example for RRD Files in Observium – Move or Copy files to Prime

If you’re doing this for RRD files, either move your RRDs to /var/ramdisk-backup/observium_rrd and then load them into the ram disk:

mv /opt/observium/rrd /var/ramdisk-backup/observium_rrd
/etc/init.d/ramdisk start

Or move your RRDs to the ram disk itself and then sync them out to the backup:

mv /opt/observium/rrd /mnt/ramdisk/rrd
/etc/init.d/ramdisk sync

Create Symlink

Now either symlink /mnt/ramdisk/rrd to /opt/observium/rrd, or change the configuration so the rrds are loaded from the ramdisk path.

You can put ramdisk sync into your crontab to periodically sync your ram disk back to the hard disk:

2 * * * * root        /etc/init.d/ramdisk sync >> /dev/null 2>&1