Logging

To write the different log lines into destination files for later use, it's OK to directly use print method. However, the python standard module logging may provide a better option in some ways here.

Similar to the cluster's logs, the log information provided by logging supports several importance levels, such as CRITICAL, ERROR, WARNING etc. Also, it provides logging.formatter to format the logs with many readily available information, such as time, file name, full path etc. So, it would be much more convenient to use logging to write logs.

Create a Logger

To use logging, first initialize a Logger called log with the name 'Gen':

log = logging.getLogger('Gen')

The Logger names are used to track Loggers' hierarchies (with dot notation).

Logging Level

Then, set the logging level to INFO:

logging.basicConfig(level=logging.INFO)

There are five logging levels in logging (in descending order):

• CRITICAL
• ERROR
• WARNING
• INFO
• DEBUG

The default logging level is WARNING, this means that all log messages that have same or higher levels than WARNING will be written. Here we set it to be INFO, cause we also want to show the heartbeat logs.

In the future, if you find heartbeat logs are so annoying that you only want to see more useful information, you can go back to change this level as you like to silence INFO logs.

File Handler

Now that we have the Logger and have set the message level, we hope that we can write logs to files in the future.

Every output place of logs is controlled by Handlers. A Logger can have a number of Handler objects to receive and process log records. There are 3 types of Handlers:

• StreamHandler
• FileHandler
• NullHandler

If we want to output our logs to the console, we can choose StreamHandler. Here we take advantages of FileHandler since we output logs to files.:

out = logging.FileHandler(args.o)

Now we have a FileHandler called out. The args.fake_logfile in the above code is the parsed outputfile argument.

At last, we need to attach this FileHandler to the Logger we created before:

log.addHandler(out)

Formatter

The FileHandler uses a Formatter to turn the log records into output messages, here the output messages should be strings which will be written into output files.

Apparently, access logs and error logs have different formats, so we treat them seperatly:

if mode == 'error':
        log_format = logging.Formatter("[%(asctime)s] [%(levelname)s] %(message)s", "%a %b %d %H:%M:%S %Y")
else:
        log_format = logging.Formatter("%(message)s")

The Formatter has several pre-defined attributes for convenience to directly use:

• %(name)s:                 Name of the Logger
• %(levelname)s:         Logging level of this message
• %(pathname)s:         Full pathname of the source file where the logging call was issued (if available)
• %(asctime)s             Time when the log was created
• ...

Some fields are not very easy to be specified through the formatter, so we will deal with them later in their own class methods.

After we create the Formatter, we will attach it to the Handler:

out.setFormatter(log_format)

Issue Logs

Now, we have created the Logger, attached the FileHandler to specify the output file, set the Formatter to specify the log string formats, we are ready to issue logs.

For example, for class fake_error_gen's method warn_lines, issue an ERROR message:

self.log.error("[pid %s:tid %s] [client %s] %s", pid, tid, ip, self.errors[random.randrange(len(self.errors))])

Just look back to the error log format we set before:

"[%(asctime)s] [%(levelname)s] %(message)s", "%a %b %d %H:%M:%S %Y")

"[pid %s:tid %s] [client %s] %s" will serve as the %(message)s part, the final log line will be [time] + [levelname] + message:

[Wed Dec 28 18:55:35 2016] [WARNING] [pid 26712:tid 1300805056] [client 13.195.146.243] My phone is out of battery