Kjellkod’s g2log vs Google’s glog: Are asynchronous loggers taking over?

G2log, the asynchronous logging utility I have been working on in my spare time is finally live and ready to download. An introduction and a performance comparison with Google’s logging library (glog) can be found at www.kjellkod.cc/g2log-efficient-background-io-processign-with-c11

The performance tests were made on a Dell Latitude E6500 laptop, Intel® Core™2 Duo CPU P8600 @ 2.40GHz × 2, with a solid state drive. On another system using a standard hard drive the difference between synchronous (glog) and asynchronous (g2log) would be larger.

The performance comparison is more about comparing the power of asynchronous operations to synchronous operations. Disk file access I/O wait times can be painfully long. This is clearly helped if done asynchronously, and punished if done synchronously

In a worst case scenario with a lot of data congestion the Google’s glog shows wait times of 1/2 second up to a whole second when flushing log entries to disk.

Kjellkod’s g2log gets leverage by doing slow disk access in the background. With the same worst case test setup the maximum wait times for a LOG call to finish was in the range of 13 – 113 milliseconds.

In a not so extreme test, but still with heavy LOG usage (1 thread writing 1 million log entries) g2log has maximum wait times of about 13 milliseconds. Google’s glog peaks at 459 milliseconds.

With these high times it can be good to know that the average time for a LOG call was much better. For g2log the average time was in the range of 6 – 9 microseconds.

For Google’s glog the average time was in the range of 10 – 13 microseconds. The log entries buffering that makes glog pseudo asynchronous clearly pays off. Google’s glog still achieves very good average time for being a synchronous logger.

From my testing it is obvious that an asynchronous logging utility outperforms a synchronous logger. Especially the peak wait times, when doing slow disk access, is mitigated if done in the background.

In addition to outperforming the traditional synchronous loggers g2log also provides some crash security. At shutdown g2log flushes all FIFO queued log entries to file.

In case of a fatal event, such as segmentation fault or floating point exception (and more), a signal handler will receive the fatal signal then notify g2log, which resends the aborting signal after saving to file all the pre-fatal log entries. The resent fatal signal will finally abort the application.

Too see how it is done, and to start using g2log you can go to http://www.kjellkod.cc/g2log-efficient-background-io-processign-with-c11.

G2log’s code is completely free, available as a Public Domain Dedication.

Advertisements

About kjellkod

Software Engineer by trade, (former?) Martial Artist and Champion by strong will and small skill... and a Swede by nationality :)
This entry was posted in C++, Software Engineering. Bookmark the permalink.

70 Responses to Kjellkod’s g2log vs Google’s glog: Are asynchronous loggers taking over?

  1. Luis Esquivel says:

    Hello. G2log seems to be an efficient log, I want to use it from a c# code, do you have an example ?
    Regards.

    • kjellkod says:

      Hi Luis,

      Interesting suggestion,. personally I would take the idea of g2log and “port” that idea to C#. G2log uses macros for instance to glean and do some log twiddling (not much macro but a little) and that would not be of any use if called cross-languages. So my recommendation is to research existing tools and if not good enough to “idea port” g2log.

      g2log is mainly the following
      1. Clean, uncluttered syntax [C# — easy]

      2. Asynchronous, thread safe logging [C# — easy]

      3. At crash due to segmentation fault, floating point exception etc such signals
      will be caught and all previous log entries are flushed to file before shutdown finishes
      [C# — I have no idea how to do this in C#]

      4. Easy “Design-by-Contract” type of functionality.

  2. KIKIJIKI says:

    Hi Kjellkod

    I’m thinking of using the library because I liked it very much, but I have a problem.
    I want to use it inside a dll project, and I don’t understand where I should put the initialization.
    The logger I used before was wrapped in a singleton so I just called it directly.

    Thank you

    • kjellkod says:

      Hi Kikijiki

      I am a bit rusty with windows and dll problems but if I am not remembering wrong the important thing is to only instantiate it once.

      I think the safest way is to instantiate it immediately in your main method, that is the way I do it (Windows/Linux) and that is the way I suggested in my blog-article and at codeproject

      Any clearer?

      • KIKIJIKI says:

        I use the logger only inside the dll, and I don’t have a main.
        I thought of using the DllMain but of course it returns immediately.

      • marlowa42 says:

        I was hoping it might use the nifty counter trick that is employed by std::cout etc so that it works before main is even called. This allows std::cout to be used in static object initialisation.

      • kjellkod says:

        Here’s how it can be resolved in g3log. Similar steps can be taken for g2log of course.


        namespace {
        static std::once_flag g_initLogger;
        static std::unique_ptr g_logger;
        static std::unique_ptr g_loggerHandle;
        } // namespace

        void InitializeLogging(const std::string& logPrefix, const FilePath& logDir, bool timed) {
        std::call_once(g_initLogger, [&]{
        if (!DirectoryExists(logDir))
        CreateDirectory(logDir);

        std::string logPath = SysWideToNativeMB(logDir.AsUTF16Unsafe());
        g_logger = g3::LogWorker::createLogWorker();
        g_loggerHandle = g_logger->addDefaultLogger(logPrefix, logPath, timed);
        g3::initializeLogging(g_logger.get());
        });
        }

        bool IsLoggerEnabled() {
        return g3::internal::isLoggingInitialized();
        }

        void ShutdownLogging() {
        g3::internal::shutDownLogging();
        g_logger.reset();
        }

        Then initializing and later shutting down in a secure way is as easy as:

        BOOL MFCSample::InitInstance() {
        // ....
        InitializeLogging(L"MFCSample", "some_path", true);
        }

        int MFCSample::ExitInstance() {
        ShutdownLogging();
        return CWinAppEx::ExitInstance();
        }

        Thanks to jhlee8804 for providing the code snippets

  3. kjellkod says:

    I see. Well without knowing much about the code is not easy to give you an answer. I can think of a few different scenarious. Maybe the reasoning can help?

    1. Find a good ‘start-up hook’ and initialize the logging there. Such a hook must be guaranteed to get run first for everything in the logger.
    –> Problem: Same as 2) how should the logger ‘worker’ be stored so that it’s destructor is safely called at program exit?

    2. No hook can be found. Instead use the C++11 feature std::call_once to get lazy initialization the first time you make a log call. See the lazy initialization example from Anthony Williams http://www.justsoftwaresolutions.co.uk/threading/multithreading-in-c++0x-part-6-double-checked-locking.html
    –> Problem: Can you find a ‘going-out-of-scope’ hook? It would be nice to have a std::call_once triggered destructor as well, to force the destructor of the logger to safely flush any queued messages before dll is unloaded (program exits).

    3. ‘going-out-of-scope’ is a not an issue. The thing should just keep running. In that case just use example 2) and make it a pure singleton.

  4. kjellkod says:

    btw: several people have requested ‘module/dll/part’ specific logging, so that each ‘part’ gets its own log-file. It is in the pipeline but so far I have not set a timeframe as I think there are more pressing features that should be done first.

    Changing filename at runtime + completely threadsafe timestamps will be released tomorrow. I just finished the testing (x86/x64 Linux/Windows)

  5. JEG says:

    How can I log simultaneously to a file and also something else (like a database)?

  6. mikeseven says:

    g2log doesn’t build with clang. Tested on Mac and Linux.

  7. kjellkod says:

    Hi @mikeseven. g2log supports only gcc and MS VS2012 at the moment. It is fairly easy to get it to run with clang. See the posts and comments at https://kjellkod.wordpress.com/2012/06/02/g2log-now-with-mainstream-c11-implementation/

  8. Andrea says:

    Hi, thanks for your wonderful logging library 🙂
    Is it possible to give a specific file name to log on, with the current code? Also, is it possible to have a singleton instance (as I could do myself by using a static variable, but better if in the lib itself)? Or, even better, a “named” logger instance? So that I can just do something like get_logger_by_name(“my_application”) and log through it?

    Thanks.

    • kjellkod says:

      In g2log you can specify what filename you should use. Currently it will use that and append and prepend it. To get the exact name you can call g2logworker::logFileName().
      You certainly can have a g2logworker object wherever and as many as you want and just call g2logworker::save(std::string). However you do not use the LOG(...) macros in that case.

      In the coming version of g2log-dev (the development branch of g2log) there is the possibility to have whatever log recipient as you want using the concept of “sinks”. I am thinking on expaning the LOG macros
      to be able to use different g2logworkers as well. Something like LOGX(INFO, instance) but that will be after that g2log-dynamic-sinks are released.

      • Andrea says:

        Thanks for your reply. From what I see in the code (g2LogWorker.cpp, fetched via Mercurial on BitBucket), the only point where the file name is set is through createLogFileName, which uses a format like prefix + “.g2log.” + localtime + “.log”. I can’t find, in code or docs, how to set a specific name for the log file, just the output directory.

        Am I missing something?

      • kjellkod says:

        Sorry. Yes you are correct. the actual filename will be decided by g2logger. You can query the logger for what the name was and you can change the location for the file.

        It should be easy enough for you to add another function to do this if for some reason you HAVE to specify the actual filename.

      • kjellkod says:

        … You can check g2LogWorkerImpl::backgroundChangeLogFile for the implementation.

        It sounds like maybe you would want to change this functionality?

        Feel free to do so.

  9. Andrea says:

    Yes, it is easy enough, I was just curious if there was such a feature and I was not aware of. I need a specific file name since we already have our log rotation scheme and I cannot change that for a number of reasons.

    Anyway, is there a better way than blog comments to discuss with you about g2log? Even by email would be less tricky 🙂

    Thanks,
    Andrea

  10. Daniel says:

    I’m getting a few issues on unresolved external symbols during linking.
    “void __cdecl g2::initializeLogging
    “public: __cdecl g2::internal::LogMessage::LogMessage
    “public: virtual __cdecl g2::internal::LogMessage::~LogMessage(void)”
    “public: __cdecl g2LogWorker::g2LogWorker
    “public: virtual __cdecl g2LogWorker::~g2LogWorker(void)”

    What files do I need to link to my project? I’m running VS 2012.

    • Daniel says:

      Also, there was a problem I ran into while building in my project.
      In g2log.h, you have the variable called “const int DEBUG = 0”.
      Turns out, in atldef.h, microsoft has a part in code that says

      #ifdef _DEBUG
      #ifndef DEBUG
      #define DEBUG
      #endif
      #endif

      And I was getting compiling issues because of that. I just changed the name to something else in g2log.h, but I thought I’d just make you aware of that.

    • Daniel says:

      Disregard. I fix that problem and the other problem I posted.
      Bit I have a different problem.

      I’m using g2log in a dll file and call the creation of my log file in a function. As soon as the file is created, it displays “Exiting, log location: [nameOfLogFile]” And then crashes when I try to log another statement in a different function. Why does the file close as soon as I create it?

      • It’s hard to know the exact reasons since you gave no code examples. If you want you can email me at hedstrom at kjellkod dot cc and we can try to solve it together.

        Did you see the example files (g2log/test_example) on how to use it?

        If I take a **guess** at the problem at hand I think it could be something in the area of:
        1) You are using the g2log in a dll file. When it is created it also creates the log file.
        2) After creation it goes out of scope, the g2logworker destructor is called and the internal logger worker will close the file with the last message “Exiting, log location ….”
        3) After the g2logworker has gone out of scope you try to call LOG(…) on it. This will cause a crash.
        4) If you at scope exit for the g2logworker had called the g2::shutdownlogging then you would avoid the crash… but I think that the issue at hand is that you need to ensure that the g2logworker stays active for a longer time.

        **Solution**
        You need to carefully consider
        *) When should the logger be created
        *) How do you make sure it stays alive for as long time as you need to (static?)
        *) How can you make sure that it quits gracefully? I, e most likely you would want to do the following,
        A) call g2::shutdownlogging
        B) Trigger the destruction of the g2logger so that its inner worker exits gracefully.

  11. Dmitry says:

    Hello! Here is my question: in my project I don’t have access to main, but I have an initialization hook. If I will declare std::unique_ptr as the statically-defined variable (so that it will survive the init function scope), will glog still be able to log everything in case of crash?

    • kjellkod says:

      Sure that works fine. No problem whatsoever as long as it is initialized properly.
      If possible I also strongly suggest to make sure that it is destroyed at a given point and not at static instances destruction.

      The example below is from using g3log. G2log would be simpler to use (no handle)


      #include "g2logworker.hpp"
      #include "g2log.hpp"

      // in some cpp file, you can use an anonymous namespace
      namespace {

      // either instantiate it like this or maybe just having a
      //std::unique_ptr and instantiate it in your
      // hook function with a "std::call_once"
      std::unique_ptr gWorker{g2::LogWorker::createWithNoSink()};

      }

      void InitializationHookFunction() {

      // using call_once is probably better to safeguard from
      // multiple calls to instantiate the logger
      // The sinkhandle can either be kept somehow or just
      // let it go out of scope later.

      auto sinkHandle = gWorker->addSink(std2::make_unique(
      ...logNamePrefix..., ...directoryPathToTheFile...),
      &g2::FileSink::fileWrite);

      g2::initializeLogging(gWorker.get());
      }

      // some guaranteed to run deinitialization hook
      void DeinitializeHook() {
      g2::internal::shutDownLogging(); // all further LOG(..) calls will be ignored
      gWorker.reset();
      }

      • Dmitry says:

        All right, thanks. Then the only strange thing I see is that g_logger_instance is not a unique_ptr itself.

        Anyway, thanks for the great lib (and another thanks for using cmake as a build system).

      • kjellkod says:

        The reason why the g2log.cpp::g_logger_instance is not a unique_ptr is that it is not owned by the g2 namespace inside g2log.cpp. It is owned by whomever instantiated it.

        In the normal scenario the g2::Logworker would go out if scope at the end of main(…), thereby shutting down the background worker thread and flushing all LOG calls to file before exiting.

        Your questions yesterday led me to re-evaluate the shutdown. Tonight I will push changes that makes the g2::Logworker call automatically g2::internal::shutdownLoggging when ~Logworker is called. That way it will be safer for both your use case and other use cases.

        Any LOG(…) call after g2::internal::shutdownLoggging is ignored., thus making it safer since it would be impossible to call once Logworker is destroyed.

      • Dmitry says:

        Thanks a lot for that!

        Also there are some questions about your CMake script. Why you define CMAKE_BUILD_TYPE Release? And also later incorporate CMAKE_CXX_FLAGS_DEBUG into CMAKE_CXX_FLAGS (but only for gcc & mingw)? Is there any reasoning behind that?

        Also, am I right that since you are not exporting any symbols, g2log now can’t be linked as shared lib with Visual C++?

      • kjellkod says:

        I have mostly used g2log on Linux. On Linux the symbols are exported automatically.

        I can see why the setup in the CMakeFiles.txt could be confusing. For most use cases it is enough to specify the *build type* in Cmake. See the top of the CMakeFiles.txt for more details.

        There are a number of people who has used g2log in a dll setup on Windows. See other comments here at my blog or at StackOverflow and you will find them. So, yes, it is definitely possible.

        I asked another g2log user who is using g2log in Windows if he had any advice for you.

        I will summarize some of his answers regarding Windows/dll. It is his words below so I cannot give more information If you need more in depth facts.

        His comments, modified for your benefit
        1) On windows you have to export the symbols explicitly. This can be done via a directive a la dllspec or a definitions file.

        2) On both Windows and Linux you have to take care regarding binary safety. There “could be issues” regarding
        a) endianness or alignment of structs if they differ between the dll and the other runtime the dll is used in.

        b) Memory management. You cannot delete memory in one runtime, when it was allocated in another. That makes use of standard library constructs like std::string dangerous.

        [Kjell comment: On Linux I have never experienced a problem with this for g2log. It might be different on Windows?]

        You could solve this by having a c-api with no memory management shared between the binaries, just passing immutable data such as raw const char* and size between them.

        For logging this is not a big deal. You could expose minimal functions,. the rest like macro magic and formatting can be made header only and hence be compiled for each translation unit separately.

        ==============================

        A reference that might be helpful
        1) http://www.cmake.org/Wiki/BuildingWinDLL

      • Dmitry says:

        Actually, there is a problem with that.

        When I am using Visual Studio and storing LogWorker in the static unique_ptr, I get a segfault (and looks like multiple segfaults). One of them, for example, happens, when the lock in shutDownLogging is acquired (probably, because of the fact that g_logging_init_mutex was already destroyed – but if I comment that out, memory is still corrupted and segfault happens quite a while after, during some system calls). So, I will have to find some other way to do that. Do you happen to have any ideas?

  12. kjellkod says:

    You can very easily play with for example test_examples/ main*****.cpp with this.

    I played a little with this earlier. If the static std::unique_ptr was defined at the top of the file then it would always core dump. If it was defined just above the
    int main(int argc, char** argv) then it never crashed.

    static initialization and destruction can be very tricky.
    That is why I mentioned earlier that
    “If possible I also strongly suggest to make sure that it is destroyed at a given point and not at static instances destruction.”

    If you can be sure that a “hook init”,. and a “hook destruct” function will be called then I would try to rely on that instead since you can then make it work pretty easily.

  13. kjellkod says:

    The windows-dll g2log project that the “user” I referred to above made can be found here.
    https://github.com/d-led/g2log-dll maybe that will be helpful for you as well.

    It is missing the macros and formatting ‘magic’ but works.
    Good luck.
    – kjell

    • kjellkod says:

      I have added g3log support for runtime dynamic loading of shared libraries (.so files) on Linux. I.e using dlopen and dlsym

      At this time I have only planned to have this support for g3log, not g2log. For g2log users they can look at the https://github.com/d-led/g2log-dll project or at what changes were done at g3log (in case they want to incorporate that into g2log)

  14. Taqreez Ali says:

    Hi,
    Kudos for the great work!
    I have a doubt that crash handling may not work. In your crashHandler function (in crashhandler_unix.cpp) you are using ‘backtrace_symbols’ function which internally uses ‘malloc’ and malloc may itself crash if heap is already corrupted. Why did not you use ‘backtrace_symbols_fd’ function instead.

    Thanks
    Taqreez

    • kjellkod says:

      Hi Taqreez,

      The crashhandling has worked great for years.

      I will look into backtrace_symbols_fd., Thanks for the suggestion. It is true that if you run out if heap and get bad_alloc issues then the crashhandler AND pushing the last crash message to the log receiver(s) might fail.

      I will look into your suggestion to see if this could improve the crashhandling and make it even more robust

  15. noizex says:

    Since I started using g3log I stopped receiving breaks in Microsoft Visual Studio 2013 in Debug mode. g3log catches it and prints some vague message like:

    ***** FATAL SIGNAL RECEIVED *******

    ***** FATAL TRIGGER RECEIVED *******

    ***** RETHROWING SIGNAL SIGSEGV(11)

    ******* STACKDUMP *******

    Exiting after fatal event (FATAL_SIGNAL). Exiting with signal: SIGSEGV
    Log content flushed flushed sucessfully to sink

    Then application exists and I can’t debug it easily. Before i started using this logger it always ended up at the break where the runtime error occurred.

    Any idea if there is some switch to enable it or I did something wrong?

    • kjellkod says:

      Short answer:
      You can disable the fatal handling in g3log in windows. The “initialize logging” call will instantiate a signal handler. Please comment out the call to the signal handler and you should be good.

      Long answer:
      The readon is that g3log tries to ensure that all queued lig entries are written to file before exiting due to a fatal error.

      On Linux/OSX you will get a stackdump in your lig that hopefully will give you enough clues to find the fatal issue.

      On Windows the fatal handler is not as complete as of yet. Im in the progress of completing the fatal handler in Windows so it should be as good as in *NIX platfoerms. E.t.a for that is within two weeks.

  16. noizex says:

    Thanks for fast reply! Is it possible to have it as some switchable option, either when I initialize logging or by some #define? I mean, for Release (or even RelWithDebInfo) builds it’s great, but for Debug builds where I want a quick break into the problem it forces me to comment out g3log initialization for a moment, check the break, and bring it back.

    I will try to comment that call you suggest, but even if the stackdump works in log file, it won’t be more helpful than break into debugger to inspect. So some switch for Debug builds would be really welcome.

    • kjellkod says:

      I will look into it for the coming update.

      At best it will work in Debug with both Visual Studio breaking point as well as pushing the stackdump to the logfile as well as flushing all log entries to the sink.

      • noizex says:

        Yes, that would be great to have logs of stackdump to logfile (and flushes all other log messages) and proper breaks in debug build in MSVC.

        I checked it in another debugger I use for heavy debugging – WinDbg. Maybe it’ll help somehow to know that it breaks correctly on the exception (using access violation test case from g3log examples), and when I continue it exits app with a properly done log.

        So it may be something MSVC related, but not sure. if you need any additional tests just let me know. For now I commented line with signal handler (which stopped logs from being written when exception happens). If I need both logs and breaks I will fall back to WinDbg as it works better, but is not so convenient to use as MSVC own debugger.

      • kjellkod says:

        Hi I have significantly improved the crash handling for g3log in Windows. You now get stacktrace for signals and exceptions.

        I’ll publish a “release blog” about it shortly.

        Regarding your request for easier debug in Visual Studio. It really is a mess the way Windows handles fatal events.

        I think the best would be to first handle the fatal event, flush logs to file, then let the original error propagate.

        I.e first restore signal handlers to default then send that fatal signal again.

        In terms of exceptions it it is basically the same approach. After flushing the logs the return of the exception handler should trigger “search for next exception handler”.

        It’s not exactly how it’s handled today but I don’t mind changing it to what you wanted… At least when running in Debug.

        In production I think it makes more sense to exit. Otherwise you might have a system that keeps running after a fatal event and the logging is would be shut down

  17. Steven says:

    I really like what you have done, but I am running into trouble with using it in a static library that ends up in a managed C++/CLI application. I cannot include the headers because it complains that threads cannot be used in a managed C++ application compiled with /clr or /clr:pure.

    Is there a way to hide the implementation? I have tried to write a wrapper class, but in some form or another I always have to make LogWorker visible (specifically through the sink handler LogMessageMover), and I can’t seem to get around it.

    Can this be done somehow?

    • kjellkod says:

      Cool. Interesting approach. Have you tried a “pimpl” solution?

      Is it the internal class LogWorkerImpl that messes things up? You could change it to be a pure “pimpl” solution by having by pointer in the LogWorker. Then just put the LogWorkerImpl in the .cpp file. I.e. a normal pimpl solution. This would move the “Active” object to the .cpp and be hidden from the header file.

      • noizex says:

        Great news! 🙂 I will wait then for some info and try it as soon as it’s available.

        Regarding handling fatal errors in Debug – if it can be implemented it will really make things easier. Same goes for things like CHECK() assert – in debug it would be great if it triggered a break exactly where it happened, with proper call stack. Right now it behaves like it should only in Release – it flushes logs and exits app, so I find out my app didn’t pass the CHECK() because it suddenly closes window, then I have to open logfile and see that certain assert was broken, but I don’t even have access to call stack at this point. I think that’s how glog worked (I was using it before I found your library).

      • Steven says:

        That is what I am trying to do; although I did it slightly differently. Essentially, I created a wrapper class and forward declared LogMessage and MoveOnCopy typedef then included g2moveoncopy.hpp. This got around the compiler issues, but now it seems to lock up when calling createWithNoSink. Specifically it locks up in the CreateActive when trying to run the thread. aPtr->thd_ = std::thread(&Active::run, aPtr.get()); I am stuck on figuring this out.

        Not having thread support in C++/CLI is excruciatingly frustrating. This is not the only area that I want to do threading but have to write some screwy workarounds to hide it. Sorry for the rant.

        Anyway, I would love to get G3Log to work because it seems to be very robust. If you have any other thoughts, that would be great.

      • kjellkod says:

        Here’s my suggestion.

        Create a fork off of g3log in Github.com/g3log. Make an “example” that uses g3log and is C++/CLI and I will try to work with you on this.

      • noizex says:

        What behaviour should I expect with this version? What I noticed so far is that it as nice callstack when it returns with FATAL log or crash or does CHECK(). But it still doesn’t trigger break in MSVC when contract is broken, just exits silently (which is perfect for any build except Debug where I want to get into debugger straight away).

        The callstack is nice though and helps a lot with tracking what caused the exit. Just not sure if there is something more that should happen (I didn’t add any special #defines so maybe I didn’t enable something).

      • kjellkod says:

        That’s correct.

        The big difference is that before it only got stack dump and “safe shutdown” on fatal signals. Now you also got this on uncaught exceptions and several system/OS fatal exceptions. See the g3log-FATAL-choice “example” that is build by default.

        The “break on Debug” is not done yet. G3log is a night-time coding thing, done for free so I’ve only a limited time to put these things in. ….. But I’ll follow up on it pronto. It’s on the first of my g3log backlog list.

  18. noizex says:

    Somehow my reply appeared under wrong comment, I was replying to kjellkod’s comment from “February 16, 2015 at 10:08 pm”.

    • kjellkod says:

      No problem.

      Please try the latest and greatest for windows. It’s on github.com/g3log in a branch. https://github.com/KjellKod/g3log/tree/windows-vector-exceptions

      I will merge it in master and put it on BitBucket as soon as I can. (Probably tonight)

      • noizex says:

        Also I seem to have some weird FATAL signal that has completly uninformative call stack, and I’m unable to trigger that error in debugger. Debug/release doesn’t matter it will just close the app and write such log:

        ***** Received fatal signal SIGSEGV(11) PID: 47572

        ******* STACKDUMP *******
        stack dump [0]
        stack dump [1] XcptFilter
        stack dump [2] f:\dd\vctools\crt\crtw32\dllstuff\crtexe.c L: 666 __tmainCRTStartup
        stack dump [3] seh_longjmp_unwind4
        stack dump [4] f:\dd\vctools\crt\crtw32\dllstuff\crtexe.c L: 466 mainCRTStartup
        stack dump [5] BaseThreadInitThunk
        stack dump [6] RtlInitializeExceptionChain
        stack dump [7] RtlInitializeExceptionChain

        Not initializing g3log doesn’t help as it internally is initialized and catches these things, I will probably have to remove the lib and includes to see what triggers this exception.

      • kjellkod says:

        I’ll put a help text in the stack dump I think and also write up a “release blog” for Windows g3log users.

        In production mode the library lacks the ” “Source Browser Database” .pdb file that you normally get in Debug mode. This is the database lookup that is needed for matching addresses to symbols.

        You can enable this in Production mode as well. I don’t (yet) know how to make this so in CMake but in Visual Studio you can do it.

        Choose properties on your project. Make sure that the “Active” configuration is Release. Choose: Linker -> Debugging. In “Generate Debug Info”. Select “Yes”.

        After a full rebuild the .pdb file, or “Source Browser Database” is generated. It won’t give a perfect stack trace but it’s still better.

        The thing to remember is that hte .pdb file must be installed where the program is installed otherwise, in case of a crash, it cannot use the “source browser database” to do the symbol lookup.

  19. Rathnadhar K V says:

    I want to use boost::movelib::unique_ptr instead of std::unique_ptr …how can I make the shift while creating the logger object??

    I need this because, rest of the software uses boost libraries instead of std libraries.

  20. Tom808 says:

    Hi Kjellkod,
    Is there a way to remove sink ?

  21. Franck says:

    Hi Kjellkod,
    First, great work and great logger !!
    I wish to use g3log in an apache module, the main constraint I have is to use 1 log file per vhost (a different sink per virtual host), actually I can add a sink per vhost, but all vhosts will log in all sinks. I do not know the best way to achieve this with g3log, maybe having a map of logger and set the current one according to the vhost ? Any hint would be appreciated, I’d love to use g3log in the apache context.
    Thanks a lot
    F.

    • kjellkod says:

      What is a vhost?

      G3log is public domain so you can change whatever with no obligation to push changes back to the community (although it is nice when people do).

      Two options that I see
      1. Use a custom sink with filtering and only allow messages for that specific sink that comes from allowed locations.
      PRO: no changes needed to G3log code
      CON: you have to adapt an existing sink (see git repo g3sinks) or write your own
      CON: The queues will contain messages that are dropped later (only the right sink will allow them) so that’s overhead

      Another option:
      Change G3log source code so that the macros take a G3log worker (or sink!) and immediately push to that instead.
      PRO: you get what you want and can custom make it accordingly
      PRO: Most of the code is already written you only change and adapt it
      CON: you would likely make it hard to merge back future G3log changes automatically and would have to do it manually — G3log is mature at this point so I don’t think that’s much of an issue.
      CON/PRO?: I would personally rename your fork to something else. The “ancestry” is obvious but it will add a functionality and design approach that G3log doesn’t have nor will have. I think a fair number of people do want it so if you make it public I can add a link to it from G3log.

      • Franck says:

        Thanks for the suggestions ! I actually tried a custom sink that could do the filtering per vhost (virtual host, http://www.example.com / http://www.example1.com would run on same Apache using the module but the need is to keep separated example / example1 separate log), apache owns the “main”, the module can just ‘hook’ to Apache.
        To be able to filter on virtual host, I need to have a the information in the message that is to be logged as well. Haven’t had time yet to check if can use a derived message.
        Thanks
        F.

      • kjellkod says:

        File information is part of the LogMessage struct. If you sink receive that instead of the me std::string to your sink then that information should be easy to access. Please see the readme and the API doc as well as tests for the details.

  22. rxduty says:

    Have you got any plan for g4log yet?

    • kjellkod says:

      I got plans but am short of time. I get $0 for G3log and don’t know how to make money out of g4log so normal dev work for my employer has priority

      • rxduty says:

        From my own experience with g3log, I think a few factors could be considered to attract more people to use it (might be wrong and mean no offense):
        1) Better documentation (not necessary to contain every detail, but should be clean and easy to follow)
        2) Cleaner organization of code/configuration files. For example, I feel the cmake files are too verbose and could be much more concise.
        3) Easier customization and integration. Assume most users don’t want to modify and maintain a different fork.
        I recently tried spdlog (and switched back to g3log again). But it handles the above points much better IMO. I think that’s why it gets much attentions. I do like the crashsafe feature and I truly believe you know what you’re doing (how to design a logging library). But cool tech doesn’t always mean great product. If you still have passion for continuing this project, I would be glad to exchange more ideas and contribute as far as I could (I have to admit my knowledge on this topic is rather limited.)

      • kjellkod says:

        No offense taken.

        G2log and G3log was partly written to get into c++11. As well as fulfilling a need for a great logger. At the time of writing them there were no great asynchronous loggers out there.

        It’s funny that you mention customizations as an issue with G3log. I believe writing a custom sink is about as easy as it can be with G3log. It was designed with custom sinks and calling into custom sinks as one key feature. Logging to “whatever” as well as custom formatting in a sink is dead simple with plenty of doc, examples and unit tests in G3log As well as in the g3sink repos. Maybe you are thinking about other types and areas of customizations?

        G3log could of course be a header only library. As it is now it is a library that you link with. I’ve heard no complaints before that it is hard to integrate with so that’s something I would like you to elaborate on.

        —–

        Obviously writing g4log would take into consideration previous knowledge so if/when that happens it’ll be a completely different beast.
        —-
        G3log improvements at this point, with just a single main contributor, relies on community aid. I give bugfixes high priority and invest personal time when those are discovered. Significantly improving doc would be nice but to be honestly most questions and “issues” raised are from people not bothering to read the API doc or the readme.

        Refactoring CMake files, changing the queues, separating fifos for sink commands and log messages wouldn’t be very hard but I have no professional motivation nor monetary incentive to do so.

        What could change is the advent of c++17, it has some very cool stuff that I would l like to work with. As tech lead/manager at work I might not be able to satisfy that curiosity at my current company and if that happens I know from past experiences that I like to get creative with side projects.

        Feedback is always welcomed so shoot away ideas here or as suggestions for improving g3log; it might be useful if I decide to write g4log

      • rxduty says:

        1. Customizations: when talking about “easier”, I’m not implying that it’s currently difficult to do so, just could be even more convenient. For most use cases, users shouldn’t even need to touch the library code. For example, the log format, it’s convenient to just set a format string to get what you want. Of course, there are limitations in this method, but it should work “out-of-box” for many users and many applications. Making changes to the library itself usually means you need to maintain the part you customized and have to resolve any merging issues for future update/releases.
        2. CMake and integration. Similar experience with customization for me. It’s easy enough to tweak CMakeLists to fit the library into my code base. But still, I need some “tweaks”. Header only library is a plus but I wouldn’t say it’s a must-have if it takes significantly more development time. I fully understand how difficult it is to develop and maintain a side project.
        3. Documentation. I think a wiki like the one spdlog has is good enough, clean and organized.

        I keep referring to spdlog just because I tried it recently. Probably I’m not qualified to comment the strength of each in the core logging functions. But for the user experience/interface part, I personally prefer spdlog. As an open source library on GitHub, you lose a lot of users and potential developers if they don’t feel right at the first place. Again I believe your expertise on this topic, but to make it a successful product (and possibly make some money to you), you have to consider these factors.

      • kjellkod says:

        The readme on G3log is presented directly as the user looks at the repo. It could benefit from more code and less text m. Apart from that I don’t see a big difference in explanation or text between spedlog and G3log.

        Please check the documentation for G3log on how to add your own sinks. There is no reason to touch/change any G3log library code at all.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s