Total visibility of what is happening on your infrastructure.
The client needed a solution that would allow the collection of various information from the logs, the same came from different sources and with different purposes. The system administrators did not have full vision of what was happening in their infrastructures. The customer required particular attention to data security, as it is subject to PCI-DSS compliance: payment transactions are carried out on the software solutions provided by this customer in its infrastructure.
We have designed a log monitoring solution based on the Elastic stack (Elasticsearch, Logstash, Kibana). This solution allows our customer to access in real time a great set of information that can be obtained from the logs (accesses, application errors, problems on credit card transactions), allowing queries to be made on the logs of all the infrastructure. Furthermore it is possible to generate business intelligence dashboards and integrate other data sources. The solution is integrated with an Incident Management and Alerting system for the prompt resolution of the detected problems and a better Incident Handling process.
As with most of our projects, we used docker for all the stack components, and source version control via git. The stack is fully integrated through service discovery provided by consul that dynamically assigns the service to the healthy application, all data sources (firewalls, systems, applications, databases, etc.) send their logs to the logstash service using filebeat, wazuh and syslog. On Elasticsearch different indexes have been defined that represent the different types of logs collected, in addition retention policies and differentiated backups by indexes have been defined, maintaining live data according to the timing defined by the PCI-DSS framework in the case of card transaction data of credit, or from the GDPR in the case of the other logs. Numerous dashboards of Business Intelligence, access control, statistics on application errors and problems related to credit card transactions have been implemented on Kibana. Furthermore, authentication on Kibana is provided by the LDAP service already in use by the customer.
The customer has thus been able to derive numerous benefits from our solution, which is used daily by multiple business sectors, including Customer Support, Operations, Development, Marketing and Accounting, thus allowing a quick overview, a rapid intervention in the event of problems and ensuring PCI-DSS compliance.