Archive for March, 2016

After you apply the new disk(s) in VMWare, you need to re-scan the scsi bus.

First, find your scsi_host:

root@nfs1 ~# grep mpt /sys/class/scsi_host/host?/proc_name
/sys/class/scsi_host/host2/proc_name:mptspi

In this case, it’s “host2

Trigger a re-scan of the scsi bus for “host2“:

# echo "- - -" > /sys/class/scsi_host/host2/scan

The hyphens represent controller,channel,lun, so – – – indicates all controllers, all channels and all luns should be scanned.

Check if the new disk(s) are found:

root@nfs1 ~# fdisk -l

Disk /dev/sda: 17.2 GB, 17179869184 bytes, 33554432 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000261b9

Device Boot Start End Blocks Id System
/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 33554431 16264192 8e Linux LVM

Disk /dev/mapper/centos_vweb3-root: 14.9 GB, 14889779200 bytes, 29081600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/centos_vweb3-swap: 1719 MB, 1719664640 bytes, 3358720 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdb: 85.9 GB, 85899345920 bytes, 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdc: 53.7 GB, 53687091200 bytes, 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdd: 37.6 GB, 37580963840 bytes, 73400320 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

We can see now there exists: /dev/sdb (80G), /dev/sdc (50G) & /dev/sdd (35G).

New Disk Partition:

Once you are able to see the increased disk space inside your system then the run the following command to format your disk for creating a new partition by following the steps to increase your physical disk volume.

# fdisk /dev/sdb

Welcome to fdisk (util-linux 2.23.2) press the 'm' key for help

Command (m for help): m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
g create a new empty GPT partition table
G create an IRIX (SGI) partition table
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition's system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)

Command (m for help):

Type the ‘p‘ to print the current partition table then create a new primary partition by typing the ‘n‘ key and selecting the available sectors. Change the disk type to ‘Linux LVM‘ by using ‘t‘ command and selecting the code to ‘8e‘ or leave as it to its default type that is ‘83‘.

Now write the table to disk and exit by Entering ‘w‘ key as shown.

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

 

New disk Volume

5)Creating Physical Volume:

As indicated above run the ‘partprobe’ or kpartx command so that the tables are ready to use and then create the new Physical Volume using the below commands.

# partprobe

You may need to give it a couple of minutes if you receive this Error:

# partprobe
Error: Error informing the kernel about modifications to partition /dev/sdb1 -- Device or resource busy. This means Linux won't know about any changes you made to /dev/sdb1 until you reboot -- so you shouldn't mount it or use it in any way before rebooting.
Error: Failed to add partition 1 (Device or resource busy)

To check the newly created volume run the following command to see if the new physical volume has been created and visible. After that we will extend the Volume Group ‘centos’ with the newly created Physical Volume as shown.

# pvdisplay
# pvcreate /dev/sdb1
  Physical volume "/dev/sdb1" successfully created
# pvdisplay

"/dev/sdb1" is a new physical volume of "25.00 GiB"
--- NEW Physical volume ---
PV Name /dev/sdb1
VG Name
PV Size 25.00 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID bJcv9x-VbWQ-lF0v-ql5r-ig0N-w6rj-4sov4c

6) CREATING AN XFS FILE SYSTEM

XFS is a highly scalable, high-performance file system which was originally designed at Silicon Graphics, Inc. XFS is the default file system for Red Hat Enterprise Linux 7.
Main Features
XFS supports metadata journaling, which facilitates quicker crash recovery. The XFS file system can also be defragmented and enlarged while mounted and active. In addition, Red Hat Enterprise Linux 7 supports backup and restore utilities specific to XFS.
Allocation Features
XFS features the following allocation schemes:
  • Extent-based allocation
  • Stripe-aware allocation policies
  • Delayed allocation
  • Space pre-allocation
Delayed allocation and other performance optimizations affect XFS the same way that they do ext4. Namely, a program’s writes to an XFS file system are not guaranteed to be on-disk unless the program issues an fsync() call afterwards.
For more information on the implications of delayed allocation on a file system (ext4 and XFS), refer to Allocation Features in Chapter 5, The Ext4 File System.

NOTE

Creating or expanding files occasionally fails with an unexpected ENOSPC write failure even though the disk space appears to be sufficient. This is due to XFS’s performance-oriented design. In practice, it does not become a problem since it only occurs if remaining space is only a few blocks.
Other XFS Features
The XFS file system also supports the following:
Extended attributes (xattr)
This allows the system to associate several additional name/value pairs per file. It is enabled by default.
Quota journaling
This avoids the need for lengthy quota consistency checks after a crash.
Project/directory quotas
This allows quota restrictions over a directory tree.
Subsecond timestamps
This allows timestamps to go to the subsecond.
Default atime behavior is relatime
Relatime is on by default for XFS. It has almost no overhead compared to noatime while still maintaining sane atime values.
To create an XFS file system, use the mkfs.xfs /dev/device command. In general, the default options are optimal for common use.
When using mkfs.xfs on a block device containing an existing file system, use the -f option to force an overwrite of that file system.

Example 6.1. mkfs.xfs command output

Below is a sample output of the mkfs.xfs command:
# mkfs.xfs /dev/sdb1
mkfs.xfs: /dev/sdb1 appears to contain an existing filesystem (LVM2_member).
mkfs.xfs: Use the -f option to force overwrite.

# mkfs.xfs -f /dev/sdb1
meta-data=/dev/sdb1 isize=256 agcount=4, agsize=1638336 blks
= sectsz=512 attr=2, projid32bit=1
= crc=0 finobt=0
data = bsize=4096 blocks=6553344, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=0
log =internal log bsize=4096 blocks=3199, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0

 

Test mount:

# mount -t xfs /dev/sdb1 /mnt
# umount /mnt

Add New Disk to /etc/fstab:

# vi /etc/fstab

Add the new disk to the fstab:

/dev/sdb1     /var/lib/mysql     xfs        defaults         0 0

TO Extend a FS, continue….

# vgextend centos /dev/sdb1

Extend volume Group

6) Extending Logical Volume:

Now we will extend the Logical Volume to increase the disk space on it using the the below command.

# lvextend -L +40G /dev/mapper/centos-root

Once you get the successfully increased message, run the command as shown below to extend the size of your logical volume .

# xfs_growfs /dev/mapper/centos-root

The size of the ‘/’ partition has been increased successfully, you can check the size of your disk drives by using the ‘df’ command as shown.

Increase disk space

Conclusion:

Managing disk partitions in Linux CentOS 7 is a simple process to increase the disk space of any of your logical volumes by using the steps as described in this article. You don’t need to give your production server’s reboot for this purpose but simply rescan your SCSi devices and expand your desired LVM

Notes also taken from a linoxide post here: http://linoxide.com/linux-how-to/add-new-disk-centos-7-without-rebooting/

If you changed your data location for mysql / mariadb, using a new disk and get an error like this after a start or restart:

# systemctl start mariadb
Job for mariadb.service failed because the control process exited with error code.
See "systemctl status mariadb.service" and "journalctl -xe" for details.

Here’s the extended status:

# systemctl status mariadb
 mariadb.service - MariaDB 10.1 database server
Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2016-03-29 11:49:49 CDT; 3s ago
Process: 5233 ExecStopPost=/usr/libexec/mysql-wait-stop (code=exited, status=0/SUCCESS)
Process: 5219 ExecStart=/usr/libexec/mysqld --basedir=/usr $MYSQLD_OPTS $_WSREP_NEW_CLUSTER (code=exited, status=1/FAILURE)
Process: 5185 ExecStartPre=/usr/libexec/mysql-prepare-db-dir %n (code=exited, status=0/SUCCESS)
Process: 5162 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
Main PID: 5219 (code=exited, status=1/FAILURE)
Status: "MariaDB server is down"

Mar 29 11:49:48 db1 systemd[1]: Starting MariaDB 10.1 database server...
Mar 29 11:49:49 db1 mysqld[5219]: 2016-03-29 11:49:49 140524370049152 [Note] /usr/libexec/mysqld (mysqld 10.1.12-MariaDB) starting as process 5219 ...
Mar 29 11:49:49 db1 mysqld[5219]: 2016-03-29 11:49:49 140524370049152 [Warning] Can't create test file /var/lib/mysql/db1.lower-test
Mar 29 11:49:49 db1 systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE
Mar 29 11:49:49 db1 systemd[1]: Failed to start MariaDB 10.1 database server.
Mar 29 11:49:49 db1 systemd[1]: Unit mariadb.service entered failed state.
Mar 29 11:49:49 db1 systemd[1]: mariadb.service failed

It more than likely has everything to do with SELinux running.  To check, do this:

# getenforce
Enforcing

Check to see if this is the issue and temporarily allow us to allow access:

root@db1 /etc/my.cnf.d# semanage permissive -a mysqld_t

Start MariaDB:

root@db1 /etc/my.cnf.d# systemctl start mariadb

Looks good, let’s check the status:

root@db1 /etc/my.cnf.d# systemctl -l status mariadb
 mariadb.service - MariaDB 10.1 database server
Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2016-03-29 11:52:07 CDT; 13s ago
Process: 5233 ExecStopPost=/usr/libexec/mysql-wait-stop (code=exited, status=0/SUCCESS)
Process: 5493 ExecStartPost=/usr/libexec/mysql-check-upgrade (code=exited, status=0/SUCCESS)
Process: 5430 ExecStartPre=/usr/libexec/mysql-prepare-db-dir %n (code=exited, status=0/SUCCESS)
Process: 5407 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
Main PID: 5464 (mysqld)
Status: "Taking your SQL requests now..."
CGroup: /system.slice/mariadb.service
└─5464 /usr/libexec/mysqld --basedir=/usr

Mar 29 11:52:06 db1 systemd[1]: Starting MariaDB 10.1 database server...
Mar 29 11:52:07 db1 mysqld[5464]: 2016-03-29 11:52:07 140638769318016 [Note] /usr/libexec/mysqld (mysqld 10.1.12-MariaDB) starting as process 5464 ...
Mar 29 11:52:07 db1 mysql-check-upgrade[5493]: The datadir located at /var/lib/mysql needs to be upgraded using 'mysql_upgrade' tool. This can be done using the following steps:
Mar 29 11:52:07 db1 mysql-check-upgrade[5493]: 1. Back-up your data before with 'mysql_upgrade'
Mar 29 11:52:07 db1 mysql-check-upgrade[5493]: 2. Start the database daemon using 'service mariadb start'
Mar 29 11:52:07 db1 mysql-check-upgrade[5493]: 3. Run 'mysql_upgrade' with a database user that has sufficient privileges
Mar 29 11:52:07 db1 mysql-check-upgrade[5493]: Read more about 'mysql_upgrade' usage at:
Mar 29 11:52:07 db1 mysql-check-upgrade[5493]: https://mariadb.com/kb/en/mariadb/documentation/sql-commands/table-commands/mysql_upgrade/
Mar 29 11:52:07 db1 systemd[1]: Started MariaDB 10.1 database server.

SIDE NOTE:  Looks like we need to upgrade our disk mounted at /var/lib/mysql now:

# mysql_upgrade -hlocalhost -uroot -pxxxxxxxxxxxxxx
Phase 1/6: Checking and upgrading mysql database
Processing databases
mysql
mysql.columns_priv OK
mysql.db OK
mysql.event OK
mysql.func OK
mysql.help_category OK
mysql.help_keyword OK
mysql.help_relation OK
mysql.help_topic OK
mysql.host OK
mysql.ndb_binlog_index OK
mysql.plugin OK
mysql.proc OK
mysql.procs_priv OK
mysql.proxies_priv OK
mysql.servers OK
mysql.tables_priv OK
mysql.time_zone OK
mysql.time_zone_leap_second OK
mysql.time_zone_name OK
mysql.time_zone_transition OK
mysql.time_zone_transition_type OK
mysql.user OK
Phase 2/6: Fixing views
Phase 3/6: Running 'mysql_fix_privilege_tables'
Phase 4/6: Fixing table and database names
Phase 5/6: Checking and upgrading tables
Processing databases
information_schema
performance_schema
Phase 6/6: Running 'FLUSH PRIVILEGES'
OK

Now, let’s permanently & persistently allow for SELinux to allow us to write to the new disk:

Start by temporarily putting mysqld_t into permissive mode (should already be there):

# semanage permissive -a mysqld_t

Now, we record what MariaDB is doing and to create a policy to allow that (but nothing else).

Switch SELinux to permissive mode and remove dontaudits from the policy:

# semodule -DB
# semanage permissive -a mysqld_t

Start MariaDB, then use the generated audit log to create a policy:

# grep mysqld /var/log/audit/audit.log | audit2allow -M mariadb_local
# semodule -i mariadb_local.pp

Let’s adjust the file system labelling so that SELinux knows this is MariaDB’s datadir. Check with:

# ls -ldaZ /datadir
drwxr-xr-x. root root unconfined_u:object_r:var_t:s0
# semanage fcontext -a -t mysqld_db_t "/var/lib/mysql(/.*)?"
# restorecon -Rv /var/lib/mysql

You can check what’s in the policy here:

# semanage fcontext -l -C
SELinux fcontext                                   type               Context

/var/lib/mysql(/.*)?                               all files          system_u:object_r:mysqld_db_t:s0 

SELinux Local fcontext Equivalence 

/opt/remi/php70/root = /
/var/opt/remi/php70 = /var
/etc/opt/remi/php70 = /etc

Now, remove the permissive mode for mysqld_t and restore dontaudits:

# semodule -B
# semanage permissive -d mysqld_t

 

Google Analytics Goal tracking with Contact form 7

I’ve set this up for various clients in the past and I tend to see this asked frequently, so I’m making a note of it here.

Step 1 – Login to Google Analytics

Once your login to your GA account head over to your dashboard, select Goals from within the View menu.

Step 2 – Create a New Goal

Simple choose a name for your goal. Example for a contact form on your contact us page I would name it “Lead Form”.
Now choose the option “Destination” from the type options.

Step 3 – Goal Destination

For a URL destination on this goal, we will be choosing a URL that doesn’t exist. Something like “/goals/lead-form/”. Complete the rest of the goal as necessary to finish the set-up of your new goal.

Step 4 — Add Custom Code Snippet to Contact Form 7

You’ll need to place the following code (example) Conversion Tracking Code for Contact Form 7 that we setup in the “Additional Settings” tab of your Contact form

Universal Analytics Code Example:

on_sent_ok: "ga('send', 'pageview', '/goals/thank-you/');"

 

Step 5 — Log into AdWords and Import your Goals

https://support.google.com/analytics/answer/1034306?hl=en

Setup

Create a mountpoint for the disk :

mkdir /mnt/ramdisk

Secondly, add this line to /etc/fstab in to mount the drive at boot-time.

tmpfs /mnt/ramdisk tmpfs defaults,size=2g,noexec,nosuid,uid=65534,gid=65534,mode=1755 0 0

Change the size option in the above line to easily accommodate the amount of the files you’ll have. Don’t worry, it doesn’t allocate all of that space immediately, but only as it’s used. It’s safe to use up to half of your RAM, perhaps more if your system has a lot of ram that’s not being used.

Mount the new filesystem

mount /mnt/ramdisk

Check to see that it’s mounted

mount
df -h

You should see these entries in mount and df output

tmpfs on /mnt/ramdisk type tmpfs (rw,relatime,size=8388608k)

tmpfs                 8.0G  0.0G  8.0G   0% /mnt/ramdisk

Create directory for Backups

Next we need to create a directory to store the backup copies of the files in.

mkdir /var/ramdisk-backup

Init Script

You can put it wherever you like, so long as you change the script we create below to reflect the new location.

Create a script at /etc/init.d/ramdisk with the following contents

#! /bin/sh 
# /etc/init.d/ramdisk
### BEGIN INIT INFO
# Provides: ramdisk
# Required-Start: $local_fs $remote_fs $syslog $named $network $time
# Required-Stop: $local_fs $remote_fs $syslog $named $network
# Should-Start:
# Should-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: ramdisk sync for files
# Description: ramdisk syncing of files
### END INIT INFO

case "$1" in
 start)
        echo "Copying files to RAM disk"
        rsync -av /var/ramdisk-backup/ /mnt/ramdisk/
        echo [date +"%Y-%m-%d %H:%M"] Ramdisk Synched from HD >> /var/log/ramdisk_sync.log
        ;;
 sync)
        echo "Syncing files from RAM disk to Hard Disk"
        echo [date +"%Y-%m-%d %H:%M"] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
        rsync -av --delete --recursive --force /mnt/ramdisk/ /var/ramdisk-backup/
        ;;
 stop)
        echo "Synching log files from RAM disk to Hard Disk"
        echo [date +"%Y-%m-%d %H:%M"] Ramdisk Synched to HD >> /var/log/ramdisk_sync.log
        rsync -av --delete --recursive --force /mnt/ramdisk/ /var/ramdisk-backup/
        ;;
 *)
        echo "Usage: /etc/init.d/ramdisk {start|stop|sync}"
        exit 1
        ;;
esac

exit 0

Now set this up to run at startup:

update-rc.d ramdisk defaults 00 99

Example For Apache

Configure and add the disk_cache module to Apache, or enable it from your /etc/apache2/mods-available/ then symlink that to the mods-enabled directory.

Our /etc/apache2/mods-available/disk_cache.conf file looks like this below:

<IfModule mod_disk_cache.c>
# cache cleaning is done by htcacheclean, which can be configured in
# /etc/default/apache2
#
# For further information, see the comments in that file, 
# /usr/share/doc/apache2.2-common/README.Debian, and the htcacheclean(8)
# man page.

# This path must be the same as the one in /etc/default/apache2
#CacheRoot /var/cache/apache2/mod_disk_cache
CacheRoot /mod_disk_cache

# This will also cache local documents. It usually makes more sense to
# put this into the configuration for just one virtual host.

CacheEnable disk /

# The result of CacheDirLevels * CacheDirLength must not be higher than
# 20. Moreover, pay attention on file system limits. Some file systems
# do not support more than a certain number of subdirectories in a
# single directory (e.g. 32000 for ext3)
CacheDirLevels 5
CacheDirLength 3

# CacheLock on
# CacheLockPath /tmp/mod_cache-lock
# CacheLockMaxAge 5

</IfModule>

Inspect htcacheclean Parameters

Now, review your /etc/default/apache2 file for htcacheclean changes:

### htcacheclean settings ###

## run htcacheclean: yes, no, auto
## auto means run if /etc/apache2/mods-enabled/disk_cache.load exists
## default: auto
HTCACHECLEAN_RUN=auto

## run mode: cron, daemon
## run in daemon mode or as daily cron job
## default: daemon
HTCACHECLEAN_MODE=daemon

## cache size 
##HTCACHECLEAN_SIZE=300M
HTCACHECLEAN_SIZE=2000M

## interval: if in daemon mode, clean cache every x minutes
HTCACHECLEAN_DAEMON_INTERVAL=360

## path to cache
## must be the same as in CacheRoot directive
##HTCACHECLEAN_PATH=/var/cache/apache2/mod_disk_cache
HTCACHECLEAN_PATH=/mnt/ramdisk

## additional options:
## -n : be nice
## -t : remove empty directories
HTCACHECLEAN_OPTIONS="-n -t"

Modify /etc/init.d/apache2 init File.  Add the following close to the top to check to see if we already have /mod_disk_cache mounted:

#-------------------------------------------------------------------------------------------------#
# Added by Shane 02/23/16 for local caching of files to keep the I/O down
if [ df -k|grep ramdisk|wc -l|awk '{print $1}' -eq 1 ] 
then
 echo "We already have /mnt/ramdisk mounted. Not remounting."
else
 if [ ! -d /mnt/ramdisk ]
 then
 echo "/mnt/ramdisk Does not exist. Let's create it."
 mkdir /mnt/ramdisk 2>/dev/null
 else
 echo "Mounting tmpfs /mnt/ramdisk"
 mount -o defaults,size=2g,noexec,nosuid,uid=65534,gid=65534,mode=1755 -t tmpfs tmpfs /mnt/ramdisk
 if [ $? -gt 0 ] 
 then
 echo "There's an error and it probably did not mount."
 else
 echo "tmpfs /mnt/ramdisk mount successful."
 fi 
 fi 
fi
#edit /etc/default/apache2 to change this.
HTCACHECLEAN_RUN=auto
HTCACHECLEAN_MODE=daemon
HTCACHECLEAN_SIZE=2000M
HTCACHECLEAN_DAEMON_INTERVAL=120
#HTCACHECLEAN_PATH=/var/cache/apache2$DIR_SUFFIX/mnt/ramdisk
HTCACHECLEAN_PATH=/mnt/ramdisk
HTCACHECLEAN_OPTIONS=""
#-------------------------------------------------------------------------------------------------#

 

Example for RRD Files in Observium – Move or Copy files to Prime

If you’re doing this for RRD files, either move your RRDs to /var/ramdisk-backup/observium_rrd and then load them into the ram disk:

mv /opt/observium/rrd /var/ramdisk-backup/observium_rrd
/etc/init.d/ramdisk start

Or move your RRDs to the ram disk itself and then sync them out to the backup:

mv /opt/observium/rrd /mnt/ramdisk/rrd
/etc/init.d/ramdisk sync

Create Symlink

Now either symlink /mnt/ramdisk/rrd to /opt/observium/rrd, or change the configuration so the rrds are loaded from the ramdisk path.

You can put ramdisk sync into your crontab to periodically sync your ram disk back to the hard disk:

2 * * * * root        /etc/init.d/ramdisk sync >> /dev/null 2>&1