Recently, a client of mine running eCommerce on WordPress was faced with a bit of a dilemma. Due to a plugin conflict, their error log file grows at a staggering rate, eventually consuming all available disk space and crashing their site. I know as a developer, my first step when they reached out to me was to check the error logs. The first time I checked, the log file was over 80GB!
Let’s talk a bit more about the setup here. They host with Codero on a VPS and manage everything through PLESK (gross). They run WordPress with WooCommerce.
We had to take a phased approach here to solve the problem. I’ll be talking about phase one: Create a job to remove the error_log every 3 days. That’s the scope right there. Pretty simple! It doesn’t fix the overall issue: Why is the error log filling up so, much so fast? That’s my next phase (another day). Okay, let’s explain what needs to be done.
- Create a shell file with a script to remove the error log
- Create cron job to execute shell file every 3 days
I’ve messed with shell files in the past, but this was my first time creating one from scratch.
I wanted to do this in a way that would allow me to easily add or remove files from the script. To easily recognize the file, I named it log_cleanup.sh. You could use a text editor and upload it, but could run into issues so I just created it directly on the server using nano.
Navigate to any directory you want and execute the following command:
nano log_cleanup.sh
This should open an empty nano CLI file editor. Now we can build out the script. I want my script to email every time it runs so you’ll see some variables in here for that. I want to declare a timestamp and the files that I want to remove:
#!/bin/bash # File created by Chrs Benjamin - chrisbenjamin.co # This script is to clean up log files known to cause storage issues##### Setting timestamp for notifications TIMESTAMP=$(date +%x_%H:%M:%S:%N | sed 's/\(:[0-9][0-9]\)[0-9]*$/\1/') ##### List file locations file1="/path/to/logs/error_log" file2="/path/to/system/logs/error_log"
Now, I want to create an array containing the file variables I just created:
##### Create array for above listed file locations array=($file1 $file2)
Like I mentioned earlier, I want this to be as easy as possible to edit in the future. All I will have to change are the file variables and adding/removing those files from the array. Now i want to create the portion of this script to remove the files:
##### DO NOT CHANGE ANYTHING BELOW THIS LINE
##### Start delete process
echo "Deleting log files..."
echo "Files:"
for item in ${array[*]}
do
rm $item
printf " %s\n" $item
done
echo "${#array[*]} log files deleted..."
printf "Job completed at: $TIMESTAMP %s\n ${#array[*]} log files deleted. %s\n %s\n ${array[*]}" | mail -s "Log cleanup successful" chris@chrisbenjamin.co
I added the echo lines to see what’s happening if I want to run the script manually. These files are constantly being used by Apache which means when we delete them, the space they took up is not yet available. In order to free up that storage, we must restart Apache:
##### Restart apache to free storage service httpd graceful
The file is complete! It’s all fairly simple if you have any experience working with shell scripts. All that’s left for us to do is create the cron job. Some hosts allow you to add a job through their web portal. In this case, I had to do this through CLI utilizing crontab. It starts with etiting your crontab:
crontab -e
This should open the default system editor. I want my job to run every 3 days at midnight, so my crontab looks like this:
SHELL=/bin/bash HOME=/ 0 0 */3 * * /path/to/log_cleanup.sh
After finished editing the crontab, I ran crontab -l to see that the content I added was saved. That’s it! I ran ./log_cleanup.sh and was emailed the results. The script will attempt to run whether the file exists or not. If the file doesn’t exist, Apache will still restart which still is never really a bad thing (opinion) considering it takes 1-2 seconds to come back up.
I know I will utilize this same process in the future whether it’s to remove a file or maybe execute some php files. I like how I can just manipulate the file variables and edit the array. The more I think about it, the more my wheels are turning coming up with the cool things I could do. I hope this will inspire others as it has inspired me!
Let me know what you think in the comments.