tail -f | awk dashboard of IP hits from Apache access log

So I've got a very busy Apache access log that is getting a WordPress pingback attack from various sources. I can easily sum up the IPs with the most requests using an awk array against the whole file, or when tailing it in chunks.

What I want to do is kind of build up an awk dashboard that keeps showing me the top 10 IPs with their count since I started tailing the log.

I can easily see IPs over the whole day, but if I already blocked one I don't want it to factor into my new count:

LC_ALL=C awk '$13 ~ /WordPress/ {sub(",","",$1); IP[$1]++}END{for (i in IP) print IP[i],i}' /var/log/httpd/access_logs | sort -rn | head -3  17473 123.123.123.123 12808 123.123.123.124 12603 123.123.123.125 

This is a somewhat working solution I've got, basically once one IP hits over 100 requests since I started tailing the access log it will start to print it. But then I just get line after line of it quickly filling up my screen:

tail -f /var/log/httpd/access_log | LC_ALL=C awk '$13 ~ /WordPress/ {sub(",","",$1); IP[$1]++}{for (i in IP) print IP[i],i}' | awk '$1 > 20'  101 123.123.123.126 101 123.123.123.126 101 123.123.123.126 101 123.123.123.126 101 123.123.123.126 101 123.123.123.126 101 123.123.123.126 101 123.123.123.126 101 123.123.123.126 101 123.123.123.126 102 123.123.123.126 102 123.123.123.126 102 123.123.123.126 102 123.123.123.126 102 123.123.123.126 102 123.123.123.126 102 123.123.123.126 102 123.123.123.126 103 123.123.123.126 103 123.123.123.126 103 123.123.123.126 103 123.123.123.126 

What I want is kind of like a watch dashboard so I end up with this one minute:

110 123.123.123.126 103 123.123.123.127 

Then those lines are just replaced as the count goes up from the tail -f:

170 123.123.123.126 146 123.123.123.127 

I was able to effectively get this result by using a for loop, timeout, and tput, but does anyone know if awk is capable of stream processing on its own to get the same result? I've got a bunch of boxes that I know don't have timeout installed by default:

for i in {1..20}; do timeout -s INT 1 tail -f /var/log/httpd/eurobits.biz > /dev/shm/TAIL$i; tput cup 1 0 && tput clear && LC_ALL=C awk '$13 ~ /WordPress/ {sub(",","",$1);IP[$1]++}END{for (i in IP) print IP[i],i}' /dev/shm/TAIL* | sort -rn | head -10; sleep .1; done  103 123.123.123.126 73 123.123.123.127 66 123.123.123.128 33 123.123.123.129 

Replay

Category: linux Time: 2016-07-31 Views: 12

Related post

iOS development

Android development

Python development

JAVA development

Development language

PHP development

Ruby development

search

Front-end development

Database

development tools

Open Platform

Javascript development

.NET development

cloud computing

server

Copyright (C) avrocks.com, All Rights Reserved.

processed in 0.401 (s). 12 q(s)