As we know, logging is very important for any application to debug and audit business metrics, because logs contain important information to analyze. So, logging is a process to write a file, and logs are streams of events coming from running applications on a server. There are a number of frameworks are available to implement logging on your application, such as Log4j, Logback, and SLF4J. There are very popular logging frameworks used in J2EE traditional applications.
In a J2EE application, most logs are written into the console or in a filesystem on your disk space, so, we have to take care with the disk space and we have to implement a shell script to recycle the log files after a particular amount of time to avoid logs filling up all the disk space. So, a best practice of log-handling for your application is to avoid unnecessary log writing in the production environment, because of the cost of disk I/O. The disk I/O can slow down your application and also fill all the disk space; it can be cause of to down or stop your application on the production server.
Logging frameworks, such as Log4j, Logback, and SLF4J, provide log levels (INFO, DEBUG, ERROR) to control logging at runtime and also restrict what has to be printed. These logging frameworks allow you to change the logging level and configuration at runtime to control the logging in your application. Sometimes, we can't restrict some log entries because they are required for business analysis and to understand the application's behavior.
In a traditional J2EE monolithic application, we can avoid the problem of disk space and also scale hardware for logging easily, but what happens when we move from the traditional J2EE monolithic application to cloud-based distributed application? A cloud-based deployment doesn't bind with a predefined machine. A distributed cloud-based system can be deployed over multiple virtual machines and containers.
As we discussed in Chapter 12, Containerizing Microservice, containers, such as Docker, are short-lived. So, we can't rely on the container and its persistent state of the disk, because as a container is stopped or restarted, it will lose all logs written to the desk.
In the microservices architecture, a distributed application will be running on several isolated machines, either virtual or physical, which means log files will be generated in all machines separately. So, it is impossible to trace these files' end-to-end transactions, because they are processed by multiple microservices. Let's see the following diagram:
As you can see, microservices are running on separate infrastructure and machines, and each microservice emits logs to that local machine. Suppose one task calls Microservice A followed by calling Microservice E and Microservice F. So, Microservice A, Microservice E, and Microservice F are running different machines and each service writes a log for this task on different log files on different machines. This makes it harder to debug and analyze logs for a particular end-to-end task in the microservice-based application. So, we have to set up a tool for log aggregation at the service level. Let's see in the next section how to achieve a centralized logging solution for the microservices-based distributed application.