Here is my “mik_collector.py” script:

all you need to change is router_ip variable to match your router IP

after that you can run this script simply with this command:

python mik_collector.py

this will collect data from mikrotik every 5 seconds and store it in data.db sqlite3 database in the same directory as the script is.

All you need to do at mikrotik side is to enable accounting feature:

If you want to analyze your collected data you can do it with sqlite3 command from Ubuntu command prompt:

sqlite3 data.db "select * from sum_per_month order by month"

like this:

you can write your own queries, all data is stored in a table named accounting. You can share your query in the comments :)

I created a simple script (SendToTelegram) that allows you to send messages to telegram:

:global telegramMessage
:local botid
:local chatid

set botid "sdfzgasf7126jwsd7a8s12>" # <- change this
set chatid "21156423187"            # <- change this

if ($telegramMessage != "") do={
   /tool fetch url="https://api.telegram.org/bot$botid/sendMessage\?chat_id=$chatid&text=$telegramMessage" keep-result=no
   set telegramMessage ""
}

Usage:

set your message to telegramMessage global variable

global telegramMessage "Wolfy was here"

run script

/system script run SendToTelegram

You can use this in combination with netwach:

you should recieve a message to your telegram :)

all you need to do is to run the following command:

unset HISTFILE

or

HISTFILE=/dev/null

this will disable history for your current session.

egrep -o "p1|p2|...|pn" filename | sort | uniq -c

Example:

egrep -o "success|error|fail" test_file | sort | uniq -c

Sample input:

test started at 00:00
test delete fail
test error
test connect success
test insert success
test started at 00:00
test delete fail
test error
test connect success
test insert success
test started at 00:00
test delete  fail
test error
test connect success
test insert success
test started at 00:00
test delete  fail
test error
test connect success
test insert success

Sample output:

4 error
4 fail
8 success

How it works?

egrep -o "success|error|fail" test_file

From grep manual:

egrep – print lines matching a pattern
-o – Print only the matched (non-empty) parts of a matching line, with each such part on a separate output line.

“success|error|fail” – patterns
test_file – input file

If we run only egrep -o “success|error|fail” test_file we get this:

fail
error
success
success
fail
error
success
success
fail
error
success
success
fail
error
success
success

next we need to send grep output to sort:

| sort

From sort manual:

sort – sort lines of text files

After sort is completed we have this:

error
error
error
error
fail
fail
fail
fail
success
success
success
success
success
success
success
success

now we need to send sort output to uniq command:

| uniq -c

From uniq manual:

uniq – report or omit repeated lines
-c – prefix lines by the number of occurrences

After this, we have what we wanted:

4 error
4 fail
8 success

This is all for today. I hope that this will help you to check huge log files!

When I need a Openfiler backup for the first time, I did this:

1. Log in OpenFiler Web UI
2. Click System
3. Click Backup/Restore
4. Click Download

This works well if you have time to do this steps every day… but if you don’t want to do this steps every day, you must find another way to do daily backups…

After my little research, I find out that when I click on Backup/Restore, Openfiler cal this file: https://openfiler:446/admin/system_backup.html

I examined the content of this file and finded out, that all you need to backup your OpenFiler configuration is this:

export LANG=C; /usr/bin/sudo grep /mnt/ /etc/fstab > /opt/openfiler/etc/fstab_mnt
export LANG=C; /usr/bin/sudo /bin/tar --ignore-failed-read -cPpzf / --exclude /opt/openfiler/etc/httpd/logs --exclude /opt/openfiler/etc/update.log /opt/openfiler/etc/

After that, I created a script to put in my crontab, which backup Openfiler configuration every day. I keep only backups newer then 7 dasy, older backups are deleted.

Here is my script

BCKNAME="OpenFiler_`hostname`_`date "+%Y-%d-%m"`.tgz"
BCKFOLDER="/root/backup"
BCKCNT=`ls -1 $BCKFOLDER/OpenFiler_* | grep -c OpenFiler_`

function bck {
	export LANG=C; /usr/bin/sudo grep /mnt/ /etc/fstab > /opt/openfiler/etc/fstab_mnt
	export LANG=C; /usr/bin/sudo /bin/tar --ignore-failed-read -cPpzf $1 --exclude /opt/openfiler/etc/httpd/logs --exclude /opt/openfiler/etc/update.log /opt/openfiler/etc/
}

cd $BCKFOLDER

if [ $BCKCNT -gt 7 ];
then
	ls -tr -1 OpenFiler_*.tgz | tail -n 8 | head -n1 | xargs -I{} find $BCKFOLDER -name "OpenFiler_*.tgz" -not -newer {} | xargs rm
	bck $BCKFOLDER/$BCKNAME
else
	bck $BCKFOLDER/$BCKNAME
fi

How it works

This is a backup name (like OpenFiler_UrosOF_2012-21-02.tgz)

BCKNAME="OpenFiler_`hostname`_`date "+%Y-%d-%m"`.tgz"

Backup destination

BCKFOLDER="/root/backup"

This count backups already in backup folder

BCKCNT=`ls -1 $BCKFOLDER/OpenFiler_* | grep -c OpenFiler_`

Function to export all OpenFiler configuration files.
$1 -> function parameter: backup location/name

function bck {
	export LANG=C; /usr/bin/sudo grep /mnt/ /etc/fstab > /opt/openfiler/etc/fstab_mnt
	export LANG=C; /usr/bin/sudo /bin/tar --ignore-failed-read -cPpzf $1 --exclude /opt/openfiler/etc/httpd/logs --exclude /opt/openfiler/etc/update.log /opt/openfiler/etc/
}

Change directory to backup directory and check if backup folder contains more than 7 backups, if yes remove older backups and create a new one, else just create a new backup

cd $BCKFOLDER
if [ $BCKCNT -gt 7 ];
then
	ls -tr -1 OpenFiler_*.tgz | tail -n 8 | head -n1 | xargs -I{} find $BCKFOLDER -name "OpenFiler_*.tgz" -not -newer {} | xargs rm
	bck $BCKFOLDER/$BCKNAME
else
	bck $BCKFOLDER/$BCKNAME
fi

Restore Openfiler

1. Log in OpenFiler Web UI
2. Click System
3. Click Backup/Restore
4. Browse to your backup file
5. Click Upload

I was looking for a script that can do the job, but after a while I didn’t find anything usefull… so I made my own script…

My idea was to create a list of all running VMs and power off this VMs. Later, when I need to power on this VMs (from the list) I just call the same script again and it turn on all my VMs that are on this list.

Here is my script (shutdown_xen_vm.sh):

#!/bin/bash

PROGRAMPATH="/root/scr/shutdown_vm"
if [ -f $PROGRAMPATH/vm-list.txt ]
then
   #$PROGRAMPATH/vm-list.txt exist
   echo "Power on:"
   cat $PROGRAMPATH/vm-list.txt | xargs -I{} echo " - {}"
   cat $PROGRAMPATH/vm-list.txt | xargs -I{} xe vm-start vm={}
   echo "Done"
   rm $PROGRAMPATH/vm-list.txt
else
   #$PROGRAMPATH/vm-list.txt do not exist
   xe vm-list power-state=running | grep name-label | awk '{print $4}' | tac | head -n -1 > $PROGRAMPATH/vm-list.txt
   echo "Power off:"
   cat $PROGRAMPATH/vm-list.txt | xargs -I{} echo " - {}"
   cat $PROGRAMPATH/vm-list.txt | xargs -I{} xe vm-shutdown vm={}
   echo "Done"
   echo "Shutdown xen server..."
   shutdown -P now
fi

How it works:

check if file vm-list.txt exist

   if [ -f $PROGRAMPATH/vm-list.txt ]

Just print a list of all VMs that were powered off

   echo "Power on:"
   cat $PROGRAMPATH/vm-list.txt | xargs -I{} echo " - {}"

Start all VMs from the list and delete this list

   cat $PROGRAMPATH/vm-list.txt | xargs -I{} xe vm-start vm={}
   echo "Done"
   rm $PROGRAMPATH/vm-list.txt

Get all running VMs and write it to vm-list.txt

xe vm-list power-state=running | grep name-label | awk '{print $4}' | tac | head -n -1 > /root/scr/shutdown_vm/vm-list.txt

Just print a list of all VMs that will be powered off

   echo "Power off:"
   cat $PROGRAMPATH/vm-list.txt | xargs -I{} echo " - {}"

Power off VMs from the list and shutdown XenServer

   
   cat $PROGRAMPATH/vm-list.txt | xargs -I{} xe vm-shutdown vm={}
   echo "Done"
   echo "Shutdown xen server..."
   shutdown -P now

Now, to run this script you just need to do this:

   
chmod +x shutdown_xen_vm.sh
./shutdown_xen_vm.sh

—

I hope that this script will help you as it helped me :)