apache logs to mysql

We have hosted the application apache logs to mysql in order to run this application in our online workstations with Wine or directly.

Quick description about apache logs to mysql:

Apache Log Parser and Data Normalization Application

Python handles File Processing & MySQL handles Data Processing

ApacheLogs2MySQL consists of two Python Modules & one MySQL Schema to automate importing Access & Error files and normalizing data into database designed for reports & data analysis.

Runs on Windows, Linux and MacOS & tested with MySQL versions 8.0.39, 8.4.3, 9.0.0 & 9.1.0.

4 LogFormats & 2 ErrorLogFormats can be loaded and 5 MySQL Stored Procedures can be processed in a single Python `ProcessLogs function` execution.

Database system designed to accommodate unlimited domains. Step-by-step guide for easy installation.

Web interface with Drill Down Capability and apache/echarts Log Visualization integration in development.

MySQL apache_logs schema currently has 49 Tables, 853 Columns, 168 Indexes, 66 Views, 7 Stored Procedures and 43 Functions to process Apache Access log in 4 formats & Apache Error log in 2 formats. Database normalization at work! .

Features:

• Creates MySQL schema - apache_logs
• Includes 2 Python modules
• Imports Apache Access Logs - common, combined and vhost_combined
• Imports Apache Error Logs
• Complete Apache Log Data Normalization
• MySQL does all Data Manipulation & Processing
• Python handles polling of log file folders
• Python executes MySQL Database LOAD DATA statements
• Python executes Stored Procedures & Functions and SQL Statements
• Requires installation of 4 Python modules - requirements.txt included

Audience: Developers, Information Technology, Management, System Administrators.
User interface: Command-line, Console/Terminal.
Programming Language: Python.
Database Environment: Python Database API, SQL-based.

Categories: