G3log: Asynchronous logging the easy way

It is my privilege to present g3log, the successor of g2log that has been brewing since the creation of the g2log-dev.
G3log is just like g2log a “crash-safe asynchronous logger“. It is made to be simple to setup and easy to use with an appealing, non-cluttered API.
G3log is just like g2log blazing fast but with approximately 30% or faster LOG processing. G3log adds some important features:

1. Completely safe to use across libraries, even libraries that are dynamically loaded at runtime
2. Easy to customize with log receiving sinks. This is the major feature of g3log compared to g2log as it gives the users of g3log the ability to tweak it as they like without having to touch the logger’s source code.
3. G3log provides faster log processing compared to g2log. The more LOG calling threads, the greater the performance difference in g3log’s favor.

G3log is just like g2log released as a public domain dedication. You can read more about that here: http://unlicense.org

So, don’t let slow logging bog your performance down. Do like developers, companies and research institutes are doing all over the globe, use g3log. Want to know more? Read on …

If you already know what you get from g2log then continue on to the new features of g3log. If you are new to g2log then you can first read a quick recap of what you get when using g2log/g3log.

G2log and G3log recap

g3log (and g2log) is an asynchronous logging utility made to be efficient and easy to use, understand, and modify. The reason for creating g2log and later g3log was simply that other logging utilities I researched were not good enough API-wise or efficiency-wise.

There are good, traditional, [#reasons] for using a synchronous logger. Unfortunately a traditional synchronous logger is just too slow for high performance code. G2log and g3log satisfy those #reasons while still being asynchronous

To get the essence of g3log it is only needed to read a few highlights:

• It is a logging and design-by-contract framework.
• Slow I/O access commonly associated with logging is done in FIFO order by one or several background threads
• A LOG call returns immediately while the actual logging work is done in the background
• Queued log entries are flushed to the log sinks at shutdown.
• It is thread safe, so using it from multiple threads is completely fine.
• It catches SIGSEGV and other fatal signals and logs them before exiting.
• It is cross platform. For now, tested on Windows7, various Linux distributions and OSX. There is a small difference between the *nix and the Windows version. On Linux/OSX a caught fatal signal will generate a stack dump to the log.
• g2log is used in commercial products as well as hobby projects since early 2011.
• g3log is used in commercial products since fall of 2013.
• The code is given for free as public domain. This gives the option to change, use, and do whatever with it, no strings attached.

Using g3log

g3log uses level-specific logging. This is done without slowing down the log-calling part of the software. Thanks to the concept of active object g3log gets asynchronous logging – the actual logging work with slow disk or network I/O access is done in one or several background threads

Example usage

Optional to use either streaming or printf-like syntax
Conditional LOG_IF as well as normal LOG calls are available. The default log levels are: DEBUG, INFO, WARNING, and FATAL.

LOG(INFO) << "streaming API is as easy as ABC or " << 123;

// The streaming API has a printf-like equivalent
LOGF(WARNING, "Printf-style syntax is also %s", "available");

Conditional logging

LOG_IF(DEBUG, small < large) << "Conditional logging is also available.

// The streaming API has a printf-like equivalent
LOGF_IF(INFO, small > large,
"Only expressions that are evaluated to true %s,
"will trigger a log call for the conditional API");

Design-by-Contract
CHECK(false) will trigger a “fatal” message. It will be logged, and then the application will exit. A LOG(FATAL) call is in essence the same as calling CHECK(false).

CHECK(false) << "triggers a FATAL message"
CHECKF(boolean_expression,"printf-api is also available");

Initialization

A typical scenario for using g3log would be as shown below. Immediately at start up, in the main function, g2::LogWorker is initialized with the default log-to-file sink.

// main.cpp
#include<g2log.hpp>
#include<g2logworker.hpp>
#include <std2_make_unique.hpp>

#include "CustomSink.h" // can be whatever

int main(int argc, char**argv) {
using namespace g2;
std::unique_ptr<LogWorker> logworker{ LogWorker::createWithNoSink() }; // 1
auto sinkHandle = logworker->addSink(std2::make_unique<CustomSink>(),  // 2
&CustomSink::ReceiveLogMessage); // 3
g2::initializeLogging(logworker.get());  // 4

1. The LogWorker is created with no sinks.
2. A sink is added and a sink handle with access to the sinks API is returned
3. When adding a sink the default log function must be specified
4. At initialization the logging must be initialized once to allow for LOG calls

G3log with sinks

Sinks are receivers of LOG calls. G3log comes with a default sink (the same as G2log uses) that saves LOG calls to file. A sink can be of any class type without restrictions as long as it can either receive a LOG message as a std::string or as a g2::LogMessageMover.

The std::string option will give pre-formatted LOG output. The g2::LogMessageMover is a wrapped struct that contains the raw data for custom handling and formatting in your own sink.

Using g2::LogMessage is easy:

// example from the default FileSink. It receives the LogMessage
//        and applies the default formatting with .toString()
void FileSink::fileWrite(LogMessageMover message) {
...
out << message.get().toString();
}

Sink Creation

When adding a custom sink a log receiving function must be specified, taking as argument a std::string for the default log formatting look or taking as argument a g2::LogMessage for your own custom made log formatting.

auto sinkHandle = logworker->addSink(std2::make_unique<CustomSink>(), // 1, 3
&CustomSink::ReceiveLogMessage); // 2

1. A sink is owned by the G3log and is transferred to the logger wrapped in a std::unique_ptr
2. The sink’s log receiving function is given as argument to LogWorker::addSink
3. LogWorker::addSink returns a handle to the custom sink.

Calling the custom sink

All public functions of the custom sink are reachable through the handler.

// handle-to-sink calls are thread safe. The calls are executed asynchronously
std::future<void> received = sinkHandle->call(&CustomSink::Foo, some_param, other);

Code Examples

Example usage where a custom sink is added. A function is called though the sink handler to the actual sink object.

// main.cpp
#include<g2log.hpp>
#include<g2logworker.hpp>
#include <std2_make_unique.hpp>

#include "CustomSink.h"

int main(int argc, char**argv) {
using namespace g2;
std::unique_ptr<LogWorker> logworker{ LogWorker::createWithNoSink() };
auto sinkHandle = logworker->addSink(std2::make_unique<CustomSink>(),
&CustomSink::ReceiveLogMessage);

// initialize the logger before it can receive LOG calls
initializeLogging(logworker.get());
LOG(WARNING) << "This log call, may or may not happend before"
<< "the sinkHandle->call below";

// You can call in a thread safe manner public functions on your sink
// The call is asynchronously executed on your custom sink.
std::future<void> received = sinkHandle->call(&CustomSink::Foo,
param1, param2);

// If the LogWorker is initialized then at scope exit the g2::shutDownLogging() will be called.
// This is important since it protects from LOG calls from static or other entities that will go out of
// scope at a later time.
//
// It can also be called manually:
g2::shutDownLogging();
}

Example usage where a the default file logger is used and a custom sink is added

// main.cpp
#include<g2log.hpp>
#include<g2logworker.hpp>
#include <std2_make_unique.hpp>

#include "CustomSink.h"

int main(int argc, char**argv) {
using namespace g2;
auto defaultHandler = LogWorker::createWithDefaultLogger(argv[0],
path_to_log_file);

// logger is initialized
g2::initializeLogging(defaultHandler.worker.get());

LOG(DEBUG) << "Make log call, then add another sink";

defaultHandler.worker->addSink(std2::make_unique<CustomSink>(),
&CustomSink::ReceiveLogMessage);

...
}

It is easy to start using the logger anywhere in your code base.

// some_file.cpp
#include <g2log.hpp>
void SomeFunction() {
...
LOG(INFO) << "Hello World";
}

g3log API

In addition to the logging API: LOG, LOG_IF, CHECK (and the similar printf-calls) there are a few functions that are helpful for using and tweaking g3log.

Initialization

initializeLogging(...)

Dynamic log levels
I.e. disabling/enabling of log-levels at runtime. Enabling of this feature is done by a #define G2_DYNAMIC_LOGGING.

// g2loglevels.hpp
void setLogLevel(LEVELS level, bool enabled_status);
bool logLevel(LEVELS level); /

Sink Handling
See g2logworker.hpp for adding a custom sink, or accessing the default filesink.

std::unique_ptr<SinkHandle<T>> addSink(...)
static g2::DefaultFileLogger createWithDefaultLogger(...)
static std::unique_ptr<LogWorker> createWithNoSink();

Internal Functions
See g3log.hpp for several internal functions that usually don’t have to be used by the coder but can if using the g3log library to the fullest.

// will be called when the LogWorker goes out of scope
shutDownLogging()

// for unit testing, or customized fatal call handling
void changeFatalInitHandlerForUnitTesting(...)

Where to get it

Please see: https://github.com/KjellKod/g3log

Enjoy
Kjell (a.k.a KjellKod)