Linux searches logs under multiple nodes of server
Recently, I was looking for the backend log of a core service and found that more than a dozen nodes were deployed, and ELK was not used for log processing. Therefore, it is very inconvenient to find logs, and it is impossible to find one service node at a time. So through communication between colleagues, I found a method, and used $(find…) plus the grep keyword to achieve fuzzy search of multi-node logs
Log deployment
Multi-node log archives are classified according to date. Under the corresponding date folder, there are the classification directories of multiple server nodes. Under the node directory are the folders named by the business module.
For example:
/2024-10-17/server01-192.168.0.1/portal/.2024-10-17 /2024-10-17/server02-192.168.0.2/portal/.2024-10-17 /2024-10-17/server03-192.168.0.3/portal/.2024-10-17
Therefore, there are few nodes, and you can also search for one node at a time. If there are many nodes, you need to search through a command.cdGo to the directory named by the date and scan it in full with a command
grep "Keywords" $(find ./ -type f -name .2024-10-17|grep portal)
Several ways to view logs in Linux
The most commonly used log viewing method
- Real-time log: tail -f
- Search for keywords near logs: cat -n | grep "Keywords"
Common commands for viewing logs
tail：
-n is to display line number; equivalent to nl command;
Examples are as follows:
- tail -100f Real-time monitoring of 100 rows of logs
- tail -n 10 Query the log of the last 10 lines at the end of the log;
- tail -n +10 Query all logs after 10 lines;
head：
It is the opposite of tail. tail is to look at how many lines of log after viewing, while head is to look at how many lines of log file are to look at. Examples are as follows:
- head -n 10 Query the first 10 lines of log in the log file;
- head -n -10 Query all logs except the last 10 lines of the log file;
cat：
- Tac is viewed in reverse order, and it is reverse written in cat words; the examples are as follows:
- cat -n |grep "debug" Query keyword log (commonly used! ~)
vim：
- 1. Enter vim editing mode: vim filename
- 2. Enter "/Keyword" and press enter key to find
- 3. Find the next one and press "n"
- Exit: After pressing the ESC key, and then enter the: number, vi will wait at the bottom of the screen for us to enter the command
- wq! Save Exit
- q! Exit without saving
Common application scenarios
View log application scenario 1: View by line number: Filter out logs near keywords
(1) cat -n |grep "debug" gets the line number of the key log
(2) cat -n |tail -n +92|head -n 20 Select the middle line where the keyword is located. Then check the logs of the first 10 lines and the last 10 lines of this keyword:
- tail -n +92 means the log after querying 92 lines
- head -n 20 means to check the first 20 records in the previous query results
View log application scenario 2: Query logs based on dates
(1) sed -n '/2014-12-17 16:17:20/,/2014-12-17 16:17:36/p'
Special Note:
- The above two dates must be logs printed in the log, otherwise they will be invalid
- First grep '2014-12-17 16:17:20' to determine whether there is this time point in the log
- View log application scenario 3: There is a lot of log content, it is inconvenient to view when printing on the screen, and paging/save files to view
(1) Use more and less commands,
For example: cat -n |grep "debug" |more In this way, the page is printed, and the page is turned by clicking the space bar
(2) Use > Save it to a file, and you can pull down this file for analysis at that time
For example: cat -n |grep "debug" >
Summarize
The above is personal experience. I hope you can give you a reference and I hope you can support me more.