Skip to main content

Command Palette

Search for a command to run...

Log Analyzer and Report Generator

Updated
Log Analyzer and Report Generator
R

Rajat Chauhan is a skilled Devops Engineer, having experience in automating, configuring, deploying releasing and monitoring the applications on cloud environment.

• Good experience in areas of DevOps, CI/CD Pipeline, Build and Release management, Hashicorp Terraform, Containerization, AWS, and Linux/Unix Administration. • As a DevOps Engineer, my objective is to strengthen the company’s applications and system features, configure servers and maintain networks to reinforce the company’s technical performance. • Ensure that environment is performing at its optimum level, manage system backups and provide infrastructure support. • Experience working on various DevOps technologies/ tools like GIT, GitHub Actions, Gitlab, Terraform, Ansible, Docker, Kubernetes, Helm, Jenkins, Prometheus and Grafana, and AWS EKS, DevOps, Jenkins. • Positive attitude, strong work ethic, and ability to work in a highly collaborative team environment. • Self-starter, Fast learner, and a Team player with strong interpersonal skills • Developed shell scripts (Bash) for automating day-to-day maintenance tasks on top of that have good python scripting skills. • Proficient in communication and project management with good experience in resolving issues.

Write a Bash script that automates the process of analyzing log files and generating a daily summary report. The script should perform the following steps:

  1. Input: The script should take the path to the log file as a command-line argument.

  2. Error Count: Analyze the log file and count the number of error messages. An error message can be identified by a specific keyword (e.g., "ERROR" or "Failed"). Print the total error count.

  3. Critical Events: Search for lines containing the keyword "CRITICAL" and print those lines along with the line number.

  4. Top Error Messages: Identify the top 5 most common error messages and display them along with their occurrence count.

  5. Summary Report: Generate a summary report in a separate text file. The report should include:

    • Date of analysis

    • Log file name

    • Total lines processed

    • Total error count

    • Top 5 error messages with their occurrence count

    • List of critical events with line numbers

#!/bin/bash

# Function to display usage information
usage() {
        echo "Usage: $0 /home/ubuntu/logs/days.log"
        exit 1
}

if [ $# -ne 1 ]; then
        usage
fi

log_file=$1

# Check if the log file is a valid regular file
if [ ! -f "$log_file" ]; then
        echo "Error: log file $log_file does not exist."
        exit 1
fi

# Variables defined
error_keyword="ERROR"
critical_keyword="CRITICAL"
date=$(date +"%Y-%m-%d")
summary_report="summary_report_$date.txt"
archive_dir="processed_logs"
total_lines=$(wc -l < "$log_file")

# Create summary report
{
        echo "Date of analysis: $date"
        echo "Log file name: $log_file"
        echo "Total lines in the log file: $total_lines"
} > "$summary_report"

# Total error count
error_count=$(grep -c "$error_keyword" "$log_file")
echo "Total error count: $error_count" >> "$summary_report"

# Get the list of critical events with line number
critical_events=$(grep -n "$critical_keyword" "$log_file")
echo "List of critical events with line number:" >> "$summary_report"
echo "$critical_events" >> "$summary_report"

# Extract error messages, count occurrences, sort by frequency, and display the top 5
top_five=$(grep -i "$error_keyword" "$log_file" | awk '{print $1,$2,$3,$4,$5}' | sort | uniq -c | sort -nr | head -n 5)
echo "Top 5 error messages with their occurrence count:" >> "$summary_report"
echo "$top_five" >> "$summary_report"

#grep -i Searches for lines containing the word error (case insensitive) in the log file.
#sort: Sorts the lines to prepare them for counting unique occurrences
#uniq -c: Counts the unique occurrences of each line
#sort -nr: Sorts the counted occurrences in numeric order (descending).
#head -n 5: Displays the top 5 lines

Output:

Shell Scripting

Part 3 of 4

A shell script is a computer program designed to be run by a Linux shell, a command-line interpreter. Typical operations performed by shell scripts include file manipulation, program execution, and printing text.

Up next

Error Handling in Shell Scripting

Understanding how to handle errors in shell scripts is crucial for creating robust and reliable scripts. Today, you'll learn how to use various techniques to handle errors effectively in your bash scripts. Topics to Cover Understanding Exit Status: ...

More from this blog

R

Rajat Chauhan

63 posts