For event logging within the Neo4j server, the java.util.logging library of Java is used. You can configure the logging parameters in the conf/logging.properties file. The default level of logging is INFO, and the messages are printed on the terminal, as well as written to a rolling file located at data/log. Depending on the development stage and requirements, you can change the default behavior or even turn off the logging, using:

java.util.logging.ConsoleHandler.level=OFF

This will turn off all console output. The log files have a size limit of 10M after which rotation takes place. The files are named as neo4j.<id>.<rotation sequence #>.log. You can configure the naming scheme, frequency of rotation, and the size of the backlog using the following parameters respectively:

java.util.logging.FileHandler.pattern
java.util.logging.FileHandler.limit
java.util.logging.FileHandler.count

You can check out more about the FileHandler class of logging at https://docs.oracle.com/javase/7/docs/api/java/util/logging/FileHandler.html.

Along with events that occur within the Neo4j server, we can also log the HTTP requests and responses that are serviced by the Neo4j server. To achieve this, we need to configure the logger, location of logging, and the optimal format of the logs. You can enable HTTP logging by appending the following parameters defined in the conf/neo4j-server.properties file:

org.neo4j.server.http.log.enabled=true
org.neo4j.server.http.log.config=conf/neo4j-http-logging.xml

The first parameter indicates to the server that HTTP logging has been enabled for the server. You can toggle the behavior by setting the value to false. The second parameter specifies the format of logging, the file rollover settings that govern how the log output is formatted and stored. By default, an hourly log rotation is used and the generic common log format (http://en.wikipedia.org/wiki/Common_Log_Format).

If the log writes to a file, then the server initially checks whether the directory is existent with appropriate write permissions, failing which a failure is reported and the server does not start.

We can also collect the logs from the garbage collector. In order to enable GC logging, we have to uncomment the following parameter so that the appropriate value is passed on to the server:

wrapper.java.additional.3=-Xloggc:data/log/neo4j-gc.log

GC logging cannot be directed to the terminal; we can find the log statements in data/log/ne4j-gc.log, or the appropriate directory that we had set in the preceding option value.

Logical logs are used as journals for the operations, and prove to be useful in scenarios when a recovery is needed for the database after a crash has occurred. The logs are generally rotated when the size exceeds a threshold (the default is 25M) and you can specify how many logs need to be kept. The reason for storing the logical log history is to serve incremental backups and keep the Neo4j HA clusters operational. When not enabled, the latest populated logical log is stored. We can configure the format using the following parameters:

keep_logical_logs=<true/false>
keep_logical_logs=<amount><type>

Some sample configurations can be specified as follows:

# To indefinitely keep logical logs
keep_logical_logs=true

# To store most recent populated log
keep_logical_logs=false

# To keep logical logs containing committed transactions for past 30 days.
keep_logical_logs=30 days

# To store logical logs that contain the most recent 500,000 transactions
keep_logical_logs=500k txs

The type option supports a few other cases which are listed as follows:

When working with Neo4j, you need several files to be read in a concurrent manner, since the different entities are stored in different files. However, Linux platforms generally define an upper bound on the number of files that can be concurrently opened. You can check the limit for the current system user with:

user@localhost:~$ ulimit -n
1024

The default value (1024) is inadequate for most practical scenarios involving indexed entities or multiple server connections. You can increase this limit to a higher value. Generally, a value over 40000 is recommended, depending on the patterns of use. For the current session, you can change this value using the ulimit command, logging in as the root user for the system. To make a system-wide persistent effect, you need to follow these steps:

In the preceding steps replace neo4j with the name of your current user. If you still see exceptions such as Too many open files or Could not stat() directory then the limit needs to be increased even further.