How to use grep to efficiently search binary logs and count matching results

1. Introduction

In daily development and operation and maintenance work, log analysis is an important means to troubleshoot problems. But sometimes we encounter grep prompt Binary file matches, indicating that the target file is in binary format rather than plain text. At this time, how to correctly extract log information and count the matching results? This article will introduce in detail the skills of grep to handle binary logs and provide a complete solution.

2. Problem background

2.1 Why log files become binary

Log files may contain binary data for the following reasons:

• Log Rotation: Log management systems (such as logrotate) may compress old logs and generate .gz or .bz2 files.
• Program exception: Some applications (such as Java) may generate binary logs containing stack traces when crashed.
• Mixed content: Log files may contain both text and binary data (such as certain debugging information).

2.2 Issues of using grep directly

grep "Match Successfully" 

Output:

Binary file matches

This indicates that grep detects that it is a binary file and will not output matching content by default.

3. Solution

3.1 Method 1: Force grep to read the file in text (-a option)

grep -a "Match Successfully" 

-a (or --text): Force grep to treat the file as a text file.

Sample output:

2023-10-01 10:20:35 [INFO] Matching successfully: user_id=1001
2023-10-01 11:30:42 [INFO] Matching successfully: user_id=1002

3.2 Method 2: Display matching line numbers (-n option)

grep -a -n "Match Successfully" 

-n(--line-number): Displays the line number of the matching line.

Sample output:

123:2023-10-01 10:20:35 [INFO] Matching successfully: user_id=1001
456:2023-10-01 11:30:42 [INFO] Matching successfully: user_id=1002

3.3 Method 3: Statistics the number of matched rows (-c option)

grep -a -c "Match Successfully" 

-c(--count): Only display the number of matching rows, and do not output specific content.

Sample output:

2

3.4 Method 4: Combining -n and -c, displaying line numbers and total numbers at the same time

grep -a -n "Match Successfully"  && echo "Total number of matches：$(grep -a -c "Match Successfully" )"

Sample output:

123:2023-10-01 10:20:35 [INFO] Matching successfully: user_id=1001
456:2023-10-01 11:30:42 [INFO] Matching successfully: user_id=1002
Total number of matches: 2

3.5 Method 5: Use strings to search after extracting text

If the file contains a large amount of binary data, you can extract the text first and then search:

strings  | grep -n "Match Successfully"

strings: Extract printable strings from a file.

3.6 Method 6: Advanced Usage (Context Display + Highlight)

grep -a -n -C 2 --color=auto "Match Successfully" 

-C 2: Shows the matching row and its front and back 2 rows (context).

--color=auto: Highlight matching keywords.

Sample output:

121-2023-10-01 10:20:33 [DEBUG] Checking user 1001...
122:2023-10-01 10:20:35 [INFO] Matching successfully: user_id=1001
123-2023-10-01 10:20:36 [DEBUG] User data has been updated

4. Complete script example

4.1 Log Search + Statistics Script

#!/bin/bash

LOG_FILE=""
SEARCH_TERM="Match Successfully"

echo "===== Start searching for logs ====="
grep -a -n --color=auto "$SEARCH_TERM" "$LOG_FILE"

echo -e "\n===== Statistical results ====="
COUNT=$(grep -a -c "$SEARCH_TERM" "$LOG_FILE")
echo "Matching row count: $COUNT"

Operation mode:

chmod +x log_search.sh
./log_search.sh

Output example:

===== Start searching for logs =====
123:2023-10-01 10:20:35 [INFO] Matching successfully: user_id=1001
456:2023-10-01 11:30:42 [INFO] Matching successfully: user_id=1002

===== Statistical results =====
Number of matching rows: 2

5. FAQ

Q1: grep -a still has no output?

Maybe it's a file encoding problem, try:

file   # Check file typeiconv -f GBK -t UTF-8  | grep "Match Successfully"  # Convert encoding

Q2: How to search for compressed logs (such as .gz files)?

zgrep -a "Match Successfully" 

zgrep: Specially used to search for .gz files.

6. Summary

Through the methods described in this article, you can easily process binary log files and efficiently extract key information.

This is the article about how to use grep to efficiently search binary logs and count matching results for Linux. For more related content of Linux grep search logs, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!