Here is my “mik_collector.py” script:

all you need to change is router_ip variable to match your router IP

after that you can run this script simply with this command:

python mik_collector.py

this will collect data from mikrotik every 5 seconds and store it in data.db sqlite3 database in the same directory as the script is.

All you need to do at mikrotik side is to enable accounting feature:

If you want to analyze your collected data you can do it with sqlite3 command from Ubuntu command prompt:

sqlite3 data.db "select * from sum_per_month order by month"

like this:

you can write your own queries, all data is stored in a table named accounting. You can share your query in the comments :)

egrep -o "p1|p2|...|pn" filename | sort | uniq -c

Example:

egrep -o "success|error|fail" test_file | sort | uniq -c

Sample input:

test started at 00:00
test delete fail
test error
test connect success
test insert success
test started at 00:00
test delete fail
test error
test connect success
test insert success
test started at 00:00
test delete  fail
test error
test connect success
test insert success
test started at 00:00
test delete  fail
test error
test connect success
test insert success

Sample output:

4 error
4 fail
8 success

How it works?

egrep -o "success|error|fail" test_file

From grep manual:

egrep – print lines matching a pattern
-o – Print only the matched (non-empty) parts of a matching line, with each such part on a separate output line.

“success|error|fail” – patterns
test_file – input file

If we run only egrep -o “success|error|fail” test_file we get this:

fail
error
success
success
fail
error
success
success
fail
error
success
success
fail
error
success
success

next we need to send grep output to sort:

| sort

From sort manual:

sort – sort lines of text files

After sort is completed we have this:

error
error
error
error
fail
fail
fail
fail
success
success
success
success
success
success
success
success

now we need to send sort output to uniq command:

| uniq -c

From uniq manual:

uniq – report or omit repeated lines
-c – prefix lines by the number of occurrences

After this, we have what we wanted:

4 error
4 fail
8 success

This is all for today. I hope that this will help you to check huge log files!

When I need a Openfiler backup for the first time, I did this:

1. Log in OpenFiler Web UI
2. Click System
3. Click Backup/Restore
4. Click Download

This works well if you have time to do this steps every day… but if you don’t want to do this steps every day, you must find another way to do daily backups…

After my little research, I find out that when I click on Backup/Restore, Openfiler cal this file: https://openfiler:446/admin/system_backup.html

I examined the content of this file and finded out, that all you need to backup your OpenFiler configuration is this:

export LANG=C; /usr/bin/sudo grep /mnt/ /etc/fstab > /opt/openfiler/etc/fstab_mnt
export LANG=C; /usr/bin/sudo /bin/tar --ignore-failed-read -cPpzf / --exclude /opt/openfiler/etc/httpd/logs --exclude /opt/openfiler/etc/update.log /opt/openfiler/etc/

After that, I created a script to put in my crontab, which backup Openfiler configuration every day. I keep only backups newer then 7 dasy, older backups are deleted.

Here is my script

BCKNAME="OpenFiler_`hostname`_`date "+%Y-%d-%m"`.tgz"
BCKFOLDER="/root/backup"
BCKCNT=`ls -1 $BCKFOLDER/OpenFiler_* | grep -c OpenFiler_`

function bck {
	export LANG=C; /usr/bin/sudo grep /mnt/ /etc/fstab > /opt/openfiler/etc/fstab_mnt
	export LANG=C; /usr/bin/sudo /bin/tar --ignore-failed-read -cPpzf $1 --exclude /opt/openfiler/etc/httpd/logs --exclude /opt/openfiler/etc/update.log /opt/openfiler/etc/
}

cd $BCKFOLDER

if [ $BCKCNT -gt 7 ];
then
	ls -tr -1 OpenFiler_*.tgz | tail -n 8 | head -n1 | xargs -I{} find $BCKFOLDER -name "OpenFiler_*.tgz" -not -newer {} | xargs rm
	bck $BCKFOLDER/$BCKNAME
else
	bck $BCKFOLDER/$BCKNAME
fi

How it works

This is a backup name (like OpenFiler_UrosOF_2012-21-02.tgz)

BCKNAME="OpenFiler_`hostname`_`date "+%Y-%d-%m"`.tgz"

Backup destination

BCKFOLDER="/root/backup"

This count backups already in backup folder

BCKCNT=`ls -1 $BCKFOLDER/OpenFiler_* | grep -c OpenFiler_`

Function to export all OpenFiler configuration files.
$1 -> function parameter: backup location/name

function bck {
	export LANG=C; /usr/bin/sudo grep /mnt/ /etc/fstab > /opt/openfiler/etc/fstab_mnt
	export LANG=C; /usr/bin/sudo /bin/tar --ignore-failed-read -cPpzf $1 --exclude /opt/openfiler/etc/httpd/logs --exclude /opt/openfiler/etc/update.log /opt/openfiler/etc/
}

Change directory to backup directory and check if backup folder contains more than 7 backups, if yes remove older backups and create a new one, else just create a new backup

cd $BCKFOLDER
if [ $BCKCNT -gt 7 ];
then
	ls -tr -1 OpenFiler_*.tgz | tail -n 8 | head -n1 | xargs -I{} find $BCKFOLDER -name "OpenFiler_*.tgz" -not -newer {} | xargs rm
	bck $BCKFOLDER/$BCKNAME
else
	bck $BCKFOLDER/$BCKNAME
fi

Restore Openfiler

1. Log in OpenFiler Web UI
2. Click System
3. Click Backup/Restore
4. Browse to your backup file
5. Click Upload

sudo vim /etc/inputrc

1. Uncomment the following lines:

#”e[5~”: history-search-backward
#”e[6~”: history-search-forward

If you don’t have this two lines just add them to the end of file (without #)

2. Logout/login

—
Now you can try to write some command listed in history…

like…

ls -lah

write

ls

and press

Page Up or Page Dn to navigate trough your history file in search of commands that starts with “ls”

If you did everything right it should auto-complete your command…