SoFunction
Updated on 2025-03-09

Shell version Nginx log spider crawl view script

Shell version Nginx log spider crawl view script

Use the path to nginx log before
If more spiders are added to the code spider UA array

#!/bin/bash
 
m="$(date +%m)"
case $m in
  "01") m='Jan';;
  "02") m='Feb';;
  "03") m='Mar';;
  "04") m='Apr';;
  "05") m='May';;
  "06") m='June';;
  "07") m='July';;
  "08") m='Aug';;
  "09") m='Sept';;
  "10") m='Oct';;
  "11") m='Nov';;
  "12") m='Dec';;
esac
d="$(date +%d)"
 
spider=(
  Googlebot
  Baiduspider
  Sogou
  YisouSpider
  360Spider
)
for i in ${spider[*]}; do
  echo -e "$i \t" `cat Here is modifiednginxaccesslog |grep $d/$m|grep $i|wc -l`
done

The above is all about this article. I hope it will be helpful for everyone to be familiar with shell scripts.