Imagine that you have to use a data structure. It is nothing fancy, only a storage for items that you will use now and then and some later on. These items will be inserted, then removed intermittently. They will have some internal order, or significance, to them and be arranged accordingly. Sometimes insertions and removal will be at the beginning or the end but most of the times you will have to find the location and then do an insertion or removal somewhere in between. An absolute requirement is that the insertion and removal of an element is efficient, hopefully even O(1).

Now, what data structure should you use?

At a time like this it is good advice to double check with books and/or online resources to verify that your initial hunch was correct. In this situation maybe you recollect that vectors are generally fast for accessing an item but can be slow in modifying operators since items not at the end that are inserted/removed will cause part of the array to be shifted. Of course you also know that an insertion when the vector is full will trigger a re-size. A new array storage will be created and all the items in the vector will be copied and moved to the new storage. This seems intuitively slow.

A linked list on the other hand might have slow O(n) access to the item but the insertion/removal is basically just switching of pointers so this O(1) operations are very much appealing. I double check with a few different online resources and make my decision. Linked-list it is…

BEEEEEEEEP. ERROR. A RED GNOME JUMPS DOWN IN FRONT OF ME AND FLAGS ME DOWN. HE TELLS ME IN NO UNCERTAIN WAYS THAT I AM WRONG. DEAD WRONG.

Wrong? How can this be wrong?
This is what the online resources say:

www.cplusplus.com/reference/stl/list Regarding std::list [...] Compared to other base standard sequence containers (vectors and deques), lists perform generally better in inserting, extracting and moving elements in any position within the container, and therefore also in algorithms that make intensive use of these, like sorting algorithms.

www.cplusplus.com/reference/stl/vector Regarding std::vector [...] vectors are generally the most efficient in time for accessing elements and to add or remove elements from the end of the sequence. [...] For operations that involve inserting or removing elements at positions other than the end, they perform worse than deques and lists.

http://en.wikipedia.org/wiki/Sequence_container_(C%2B%2B)#List Vectors are inefficient at removing or inserting elements other than at the end. Such operations have O(n) (see Big-O notation) complexity compared with O(1) for linked-lists. This is offset by the speed of access — access to a random element in a vector is of complexity O(1) compared with O(n) for general linked-lists and O(log n) for link-trees.

In this example most insertions/removals will definitely not be at the end, this is already established. So that should mean that the linked-list would be more effective than the vector, right? I decide to double check with Wikipedia, Wiki.Answers and Wikibooks on Algorithms. They all seem to agree and I cannot understand what the RED GNOME is complaining about.

I take a break to watch Bjarne Stroustrup’s Going Native 2012 keynote. At time 01:44 in the video-cast and slide 43, something interesting happens. I recognize what the Red Gnome has tried to to tell me. It all falls into place. Of course. I should have known. Duh. Locality of Reference.

[ Import Stuff comes now]

1. It does not matter that the linked-list is faster than vector for inserting/removing an element. In the slightly larger scope that is completely irrelevant. Before the element can be inserted/removed the right location must be found. And finding that location is extremely slow compared to a vector. In fact, if both linear search is done for vector and for linked-list, then the linear search for vector completely, utterly and with no-contest beats the list.
2. The extra shuffling, copying overhead on the vector is time-wise cheap. It is dirt cheap and can be completely ignored if compared to the huge overhead for traversing a linked-list.

Why,you may ask? Why is the linear search so extremely efficient for vector compared to the supposedly oh-so-slow linked-list linear search? Is not O(n) for linked-list comparable to O(n) for vector?

In a perfect world perhaps, but in reality it is NOT! It is here that Wiki.Answers and the other online resources are wrong! as the advice from them seem to suggest to use the linked-list whenever non-end insertion/removals are common. This is bad advice as they completely seem to disregard the impact of “locality of reference“.

The linked-list have items in disjoint areas of memory. To traverse the list the cache lines cannot be utilized effectively. One could say that the linked-list is cache line hostile, or that the linked-list maximizes cache line misses. The disjoint memory will make traversing of the linked-list slow because RAM fetching will be used extensively.

A vector on the other hand is all about having data in adjacent memory. An insertion or removal of an item might mean that data must be shuffled, but this is cheap for the vector. Dirt cheap (yes I like that term). The vector, with its adjacent memory layout maximizes cache utilization and minimizes cache line misses. This makes the whole difference, and that difference is huge as I will show you.

Let us test this by doing insertions of random integer values. We will keep the data structure sorted and to get to the right position we will use linear search for both vector and the linked-list. Of course this is silly to do for the vector but I want to show how effective the adjacent memory vector is compared to the disjoint memory linked-list.

[ The Code ]

/* "NumbersInVector &randoms" is pre created random values that we just put into a vector. This way the same random values can be used for testing and comparing the vector to the linked-list.

Example:
0, 1, 8, 4, 1     will get sorted to:
0, 1, 1, 4, 8
*/
template <typename Container>
void linearInsertion(const NumbersInVector &randoms, Container &container)
{
 std::for_each(randoms.begin(), randoms.end(),
 [&](const Number &n)
 {
    auto itr = container.begin();
    for (; itr!= container.end(); ++itr)
    {
       if ((*itr) >= n) { break; }
    }
    container.insert(itr, n); // sorted insert
 });
}

/* Measure time in milliseconds for linear insert in a std container */
template <typename Container> 
TimeValue linearInsertPerformance(const NumbersInVector &randoms, Container &container)
{
 g2::StopWatch watch;
 linearInsertion(std::cref(randoms), container);
 auto time = watch.elapsedMs().count();
 return time;
}

The time tracking (StopWatch that I wrote about it here) is easy with C++11.. Now all that is needed is the creation of the random values and keeping track of the measured time for a few sets of random numbers. We measure this from short sets of 10 numbers all the way up to 500,000. This will give a nice perspective

void listVsVectorLinearPerformance(size_t nbr_of_randoms)
{
  std::uniform_int_distribution distribution(0, nbr_of_randoms);
  std::mt19937 engine((unsigned int)time(0)); // Mersenne twister MT19937

  auto generator = std::bind(distribution, engine);
  NumbersInVector     values(nbr_of_randoms);

  // Generate n random values and store them
  std::for_each(values.begin(), values.end(), [&](Number &n)
                { n = generator();});

  NumbersInList      list;
  TimeValue list_time = linearInsertPerformance(values, list);
  NumbersInVector    vector;
  TimeValue vector_time = linearInsertPerformance(values, vector);

  std::cout << list_time << ", " << vector_time << std::endl;
}

Calling this for values going all the way up to 500,000 gives us enough values to plot them in a graph and which shows exactly what Bjarne Stroustrup told us at the Going Native 2012. For removal of items we get a similar curve but that is not as time intensive. I have google doc collected some timing results that I made on IdeOne.com, my x64 Windows7 / Ubuntu Linux laptop for you to see.

So 500,000 sorted insertions in a linked-list took some 1 hour and 47 minutes. The same number of elements for the vector takes 1 minute and 18 seconds. You can test this yourself. The code is available at http://ideone.com/pi2Od.

[ Locality of Reference ]

The processor and cache architecture has a lot of influence on the result obviously. On a Linux OS the cache level information can be retrieved by

grep . /sys/devices/system/cpu/cpu*/index*/cache

On my x64 quad-core laptop that gives each core-pair 2x L1 cache a 32 KBytes, 2x L2 cache a 256 KBytes. All cores share a 3 MBytes L3 cache.

Conceptual drawing of quad core Intel i5

I thought it would be interesting to compare my quad-core against a simpler and older computer. Using the CPU-Z freeware software I got from my old dual-core Windows7 PC that it has 2xL1 caches a 32KBytes and 1xL2 cache a 2048 KBytes.

Testing both with a smaller sets of values shows the cache significance clearly (disregarding different bus bandwidth and CPU Speed )

x86 old dual-core PC

x64 quad-core Intel i5

[ Devil's Advocate ]

The example above is of course somewhat extreme. This integer example brings out another bad quality of linked-list. The linked-list need 3 * size of integer to represent one item when the vector only need 1. So for small data types it makes even less sense to use a linked-list. For larger types you would see the same behavior as shown above but obviously the times would differ.

[What to do now?]

Donald Knuth made the following statement regarding optimization:

“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil” .

Meaning that “Premature Optimization” is when the software developer thinks he is doing performance improvements to the code without knowing for a fact that it is needed or that it will greatly improve performance. The changes made leads to a design that is not as clean as before, incorrect or hard to read. The code is overly complicated or obfuscated. This is clearly an anti-pattern.

On the other hand we have the inverse of the extreme Premature Optimization anti-pattern, this is called Premature Pessimization. Premature pessimization is when the programmer chose techniques that are known to have worse performance than other set of techniques. Usually this is done out of habit or just because.

Example 1: Always copy arguments instead of using const reference.
Example 2: Using the post increment ++ operator instead of pre increment ++. I.e value++ instead of ++value.

For incrementing native types pre- or post-increment is of little difference but for class types the performance difference can have an impact. So using value++ always is a bad habit that when used for another set of types actually is degrading performance. For this reason it is always better to just do pre increments. The code is just as clear as post increments and performance will always be better (albeit usually only a little better).

Using linked-list in a scenario when it has little performance benefit (say 10 elements only touched rarely) is still worse compared to using the vector. It is a bad habit that you should just stay away from.

Why use a bad performance container when other options are available? Options that are just as clean and easy to work with and have no impact on code clarity.

Stay away from Premature Optimization but be sure to not fall into the trap of Premature Pessimization! You should follow the best (optimal) solution, and, of course, common sense without trying to optimize and obfuscate the code and design. Sub-optimal techniques such as linked-list should be avoided since there are clearly better alternatives out there.

[ C++ only? ]

Of course this is true for other languages as well. I leave it as a fun test for whomever feel challenged to test for C#.

Anyhow,.. I decided to put down some random pieces of code and thoughts that were running through my head in the past,. maybe a code snippet or two is useful to you as well?

1. Time

OK, so with C++11 we get this whole nice feature to deal with time in different ways. Previously this was not cross-platform standardized and even thread unsafe for the ANSI functions. Time and time again (pun intended) we allways run into the task of measuring the time between two points,. and sometimes, just sometimes, it is nice to force a “wait/sleep” either for fun or for testing. This would be easy to do with C++11 but due to its very verbose syntax it can be confusing to get it right. Perhaps these snippets might help?

Check out in my artificial examples, like the silly_sleep_ms(…) and you will see what I mean

#include <iostream>
#include <chrono>
void silly_sleep_ms(unsigned int ms)
{
  typedef std::chrono::high_resolution_clock clock;
  typedef std::chrono::milliseconds milliseconds;

  clock::time_point t0 = clock::now();
  std::this_thread::sleep_for(milliseconds(ms));
  clock::time_point t1 = clock::now();
  milliseconds total_ms = std::chrono::duration_cast<milliseconds>(t1 - t0);

  std::cout <<"this_thread_sleep (" << ms <<") milliseconds: "
            << total_ms.count() << "ms\n";
}

Running silly_sleep_ms(50) three times yield:
this_thread_sleep 50 milliseconds: 52ms
this_thread_sleep 50 milliseconds: 51ms
this_thread_sleep 50 milliseconds: 51ms

On my ancient PC (Intel Core 2-CPU 6400 @ 2.13GHz) running Windows7 it is usually slightly worse performance than when I boot up for Ubuntu. But on windows7 this seemed to be typical with a couple of milliseconds difference. No big deal, right?.

Of course repeating all the typedefs and template arguments all the time is kind of tedious. A simple StopWatch utility makes things easier,. so with an attempt at refactoring it.

namespace whatever // formatted for blog readability
{
  typedef std::chrono::high_resolution_clock clock;
  typedef std::chrono::microseconds microseconds;
  typedef std::chrono::milliseconds milliseconds;

  clock::time_point now(){return clock::now();}

  microseconds intervalUs(const clock::time_point& t1, const clock::time_point& t0)
 {
    return std::chrono::duration_cast<microseconds>(t1 - t0);
 }

  milliseconds intervalMs(const clock::time_point& t1,const clock::time_point& t0)
 {
    return std::chrono::duration_cast<milliseconds>(t1 - t0);
 }



class StopWatch
{
   clock::time_point start_;
 public:
   StopWatch() : start_(clock::now()){}
   clock::time_point restart() { start_ = clock::now(); return start_;}
   microseconds elapsedUs()    { return intervalUs(now(), start_);}
   milliseconds elapsedMs()    { return intervalMs(now(), start_);}
};

} // whatever

This cuts down the verbose syntax needed for the silly_sleep_ms(…) function

void silly_sleep_ms(unsigned int ms)
{
 using namespace whatever;
   StopWatch measure;
   std::this_thread::sleep_for(g2::milliseconds(ms));

   std::cout <<" 2this_thread_sleep (" << ms <<") milliseconds: "
             << measure.elapsedMs().count() << "ms\n";
}

Running it instead for microseconds (us) the overhead is proportionally much higher (not as bad if running on Linux).

void silly_sleep_us(unsigned int us)
{
 using namespace whatever;
    StopWatch measure;
    std::this_thread::sleep_for(microseconds(us));

    std::cout <<" this_thread_sleep (" << us <<") microseconds: "
              << measure.elapsedUs().count() << "us\n";
}

Running silly_sleep_us(50) three times yield:
this_thread_sleep (50) microseconds: 1792us
this_thread_sleep (50) microseconds: 1151us
this_thread_sleep (50) microseconds: 1921us

Whoa. The overhead just went from some meager 4% overhead to 38 times the expected time and actually getting closer to 2 milliseconds for some calls. Repeating this even more times just confirms that this result was not unusual but what is to be expected!

Being Windows rusty and all and doing this a late night last year together with packing (for paternal leave in the USA) made my sleepy brain more befuddled than usual.

Mental note to self:
Do not try to do snippet programming examples and pack at the same time

For some reason I actually thought I had seen a bug since I tested the same code on Linux and the overhead was not nearly as high. In a temporary state of confusion I emailed Anthony Williams (creator of just::thread and the chrono library that I use). He very nicely replied to my email explaining that
“… the smallest timer tick granularity on Windows is 1 millisecond, so it is hard to sleep for less than that, and the standard says that sleep_for sleeps for *at least* the specified time.”

Duh! Reading it on the plane made me feel more than a little silly. Of course. Anthony was even so nice as to suggest a snipped for shorter sleep. Using this_thread::yield in a loop with the high_resolution_clock would get better result. The tradeoff being that the operation on Windows would in itself be a time consuming operation,. maybe even yielding as bad result as before.

template<typename Duration>
void short_sleep(Duration d) // thanks to Anthony Williams for the suggestion of short_sleep
{
   clock::time_point const start=clock::now(), stop=start+d;
   do{
       std::this_thread::yield();
     } while(clock::now()<stop);
}

// ... updated for using the new sleep utility
void silly_improved_short_sleep_us(unsigned int us)
{
  using namespace whatever;
  StopWatch measure;
  short_sleep(microseconds(us));
  auto elapsed_us = measure.elapsedUs().count();

  std::cout <<"this_thread_sleep (" << us <<") microseconds: "
            << elapsed_us << "us\n";
}

Running a modified version of the silly_sleep_us, using the short_sleep gave for three tries
this_thread_sleep (50) microseconds: 51us
this_thread_sleep (50) microseconds: 295us
this_thread_sleep (50) microseconds: 51us

Running it even more times show that the result is more inconsistent then what I first expected. Just like Anthony suggested using the high_resolution_clock is sometimes as fickle as the original this_thread_sleep.

So way better than before but still shaky. Shaky is still better than always way out there I suppose. But what is better is naturally the improved syntax with or without the StopWatch. Instead of writing very verbose like as shown before:

typedef std::chrono::high_resolution_clock clock;
typedef std::chrono::milliseconds milliseconds;

  clock::time_point t0 = clock::now();
  // ... some operation to time estimate
  clock::time_point t1 = clock::now();

  milliseconds total_ms = std::chrono::duration_cast<milliseconds>(t1 - t0);

All you need to do is to use a StopWatch or similar grouped functionality and you cut it down to

StopWatch measure;
// ... some operation to time estimate

auto elapsed_us = measure.elapsedUs().count();

Or just using the namespace whatever as used above

clock::time_point t0 = now();
// ... some operation to time estimate
clock::time_point t1 = now();

auto elapsed_ms = intervalMs(t1, t0).count();

There you go, with C++11 it is a snap to pin down elapsed time no matter if you are on Windows or on Linux. Unfortunately we have this verbose template syntax that will scare away many newcomers and till it gets simpler I suggest you to use something like the StopWatch or the whatever functionality above.

The last 11 years took me, then my family, over two continents and three countries in search of job and home. It was an interesting journey and will continue to be so I suspect.

After 11 years and a few different jobs I have made some conclusions, mental notes that I figured was worth mentioning, at least to my self. Maybe in writing it down I will get some clarity in all of this. A lot of times that is how I work. In telling someone else, or just writing it down the pieces just fall into place.

And yes this blog is kind of longish. It is an entry actually written for myself. You are not intruding if you are reading it – but it is long and rambling so beware if you continue to read it. You might get bored. Also this story is completely egocentric – after all it is my story.

In fact I suggest you do something else with your time. If you really want to read interesting blogs then I suggest you check out Steve Yegge a blog virtuoso who is stealing way too much of my spare time since I discovered him last week

If I have any points to make apart from memory reminiscing they will probably be scattered in hard to find places, or not at all, so this is a good time to stop reading before you have lost too much of your own personal time.

Still reading? Do not say I did not warn you.

I graduated from the Lulea University of Technology in early 2001. I was all green, eager to understand what working was all about. I mean working with CS. All my previous work experience had mostly entailed manual labor. Some of my previous working experience was just silly hard,. and even mentioning it now will make you sneer and think I am being ludicrous.

Even so. I really got my first work experience during some winter weekends with my dad and brother Hedas. Our house in Alvik was heated with a huge (well I thought so then) incinerator. As the temperature crept down to -30C (-22 Fahrenheit), even touching – 40C sometimes, we needed a lot of firewood. My dad did this lumberjack-hobbyist-for-fun-and-for-necessity every winter weekend until we had enough firewood for the next season. It was manual work, chainsaw (for my dad, not for me), saw and axe. The cut down trees were stripped of branches until they were just timber. Then my brother and I took these logs and piled them. It was a lot of dragging, pulling, lifting. I always tried to take as big logs as my 2 year older brother. I probably did not succeed very often, but I tried. I was 7 when I started helping out in the forest.

I am not saying that my life was hard. I was happy, this was fun. I am just trying to build up some character here so you (I) understand why things happened later as it happened. Why I am the way I am.

At 13 I got my first summer job. I was slaving away, well at least working long days, with painting buildings, laying cement building foundations, cleaning pig stables, bringing in the hay and other pretty hard tasks for a 13 year old. I continued to work some summers at the pig farm. It totally sucked and the pay was low, the people there mostly made fun of the scrawny young kids that awkwardly tried to do grown-up jobs but finding a summer job was not so easy so this is what was available.

At 27 my past work experiences encompassed not only farm work but also working with no-safety sawmills ( I am so glad I did not loose any fingers), summer nights at a bakery doing the last touch on Danish pastry and surveing huge ovens and their outgoing bread lines., I did monotonic work in the assembly lines and technical fix-it support for a local healthcare company. At later years when I was at the university I also got some work at a day-care center and of course being the ice-cream truck man for Hemglass rocked (Hemglass is the Swedish version of the “Schwans truck”).

So back in 2001 I knew what manual labor type of work was all about. Unfortunately I knew absolutely zero about real software engineering. Just a year previous I really sucked, yes really sucked at programming. My lab partner Pär Hellström was then my constant programmer buddy. We collaborated about what do do, but when it came down to coding the loops, the ; and the if’s then else he was just so much faster than I was. Whenever I tried it was just too many forgotten ; or slow-to-look-up library calls that made it too staccato for our mutual enjoyment. Out of convenience for both of us Pär took charge of all the coding. It gave me good practice at reading code, following and pointing out improvements and thinking ahead for the what-to-do-next instructions but actually coding, really coding, was slow going for me.

Then during my way-too-long Master Thesis work for professor Roch Guerin at his lab at UPenn something happened. Maybe it was cracking away at OSPF server code, maybe it was failing miserably at becoming good at C++ templates or maybe it was seeing glimpses of what really good C++ object oriented server code could be in contrast to messy #ifdef cluttered C-code. Either way I got boosted. I realized I really liked programming. C++ was fun. Code and how it all made sense in a higher design and that source code could be beautiful in a way that I never even came close to glimpse during my university studies.

So when I handed in my Master’s Thesis (probably only a stumbling approach to be academic) it felt like a fundamental life-step. I was ready, eager and anticipating a change. A life change. Professional life change. I so badly wanted to work with my new found skills. To prove myself, to learn and do do something useful. Now all I had to do was to find my job.

… Not as easy as it sounds. For me it took almost 2 years before I reached what I was looking for.

2001 – A kindling fire

The job market in Luleå was just not for a young aspiring C++ programmer. I tried and tried and no one got back to me. All my sent out resumes and cold calls was just rewarded with the wall of silence. Not even a written rejection letter from the companies I contacted. .. Well maybe one or two letters. Hey, this is my story I can choose to forget, mix-up, misplace and distort – if that helps in getting it out, . .

… I reconsidered. Maybe the www tech had something to offer? I had dabbled with php at the University and applied for a web-editing job at the local newspaper, Kuriren. Out of over 300 applicants, a few got selected for an interview. I was one of the lucky few. All eager, all ready I was quickly shot downed, burned and rejected. All of the applicants had enough php experience – what the team really wanted was the applicant to be a good golfer and an ardent fly fisher. I was neither.

Someone, I do not remember who, told me to stop sending out my resume and just go to the companies and visit them in person. I looked over my list of companies that I had applied to. All had so far given me the silent treatment. I started to work down the list.

At my second try, at (Swedish: “Förenkla Din Tillvaro“) FDT I got to talk to the president. From behind his desk the CEO Kent Forss looked up at me and said

Kent: “Oh. So you are Kjell? I just deleted your CV from my inbox”
Silence. Awkward silence for an eternity.

I jumped-started my brain and manage to squeak out something about that
Me: “ it is a Good Thing I had another CV with me ...”

As Kent and I was talking the lead programmer Peter Joki walked by and got interested. It turned out the he was in dire need of more people and once the president talked to him I had in just 5 minutes landed my first job. Yeeaaay!

Months later and happy coding in Visual Basic 6, porting old code from Visual Basic 3, laying down the foundation and coding a whole new piece of software and playing with software design I did something unexpected. I quit my job. The job was fun, the company and the colleagues had fun together but in the end it was not for me. I wanted more, or at least I wanted something else.

I wanted to work with C++ and I wanted to learn on a whole different level than I could at FDT. I wanted to work with the best and the brightest. I am not saying that the develpers at FDT were not the best and brightest but it was too close to home, and it was not the cutting-edge, world changing software that I dreamed about. After 8+ months in the USA I wanted more of the world than Luleå. I wanted UPenn at a company-level. So I pondered and pondered. In Sweden the job market for Software Engineers seemed shot. Ericsson had already started to lay off engineers and in the two years that followed the company lay-offs basically killed the job market for newly graduated software wannabees. The repercussions of that we can still feel today. I blame (probably completely biased and wrongfully) Ericsson as the No.1 reason why computer science went from being popular at the universities to something obscure. I decided to leave the country.

In retrospective I think it was good that I tried to get away from Sweden. Judging from the reactions of friends and acquaintances I did something crazy, ballsy and utterly loco. I went to the bank, took out a loan of 100.000 SEK and went abroad. I had no security for the loan, no job but somehow I managed to persuade the banker of my golden opportunities.

After a short vacation in Spain I was back in Philadelphia. I was going to find a job in the USA. A potentially serious relationship waited there and I thought I could pull it off. My skunkwork agenda was to find a company that would sponsor my visa, then go back to Sweden and apply for the sponsored-visa. It was gutsy, not how normal people would have done it and in the end it came down hard on me. I learned a lot in the process and have never, ever, regretted it.

In the end the relationship went sour and I was just feeling miserable in Philadelphia. Sure enough feeling miserable does not improve anyone’s job hunting chances, but what really killed my chances was Mr terrorist, Bin Laden himself. On September 11th, 2001 I had my second, maybe final job interview for a company in the international bank market. My profile was perfect for them as they for their Operations department wanted someone European who also knew at least one Nordic language. I think I almost had the job. As we were discussing salary the interview got interrupted by the secretary. The towers were hit. You all know what happened after that… All my chances in the USA were killed that day, my relationship had already gone from sour to rancid and now the whole world, not only the USA was in turmoil.

A few weeks later I was back in Luleå. Most of my borrowed money spent and with a loan that demanded monthly payments I had to get creative quickly. Lots of luck and a mismatch of odd-jobs helped me. For almost a year I got by as a substitute teacher, on-and-off assignments for the vampire run StudentConsulting with everything from teaching Visual Studio IDE and Visual Basic programming to making small ActiveX components and attempts at www intranet hacking (hacking in the meaning of creating, not destroying).

My last odd-job was doing the dishes at a local Chinese/Pizzeria restaurant. Oddly enough I never considered that to be an all-time low. In fact I think I learned a lot from Waldorf and I think of that time as something profound where I finally could get some much needed mental meditation as I was cleaning up and washing cups, plates and fragile things that I tried not to break or chip.

October 2002. The Dawn of the Software Engineer

Lots and lots and lots of cold calls, company visits, job applications, travel to Stockholm and back to Luleå again, a few job interviews, loosing myself with loser childhood delinquent friends. Loosing the same friends. Loosing other friends. Gaining some friends.
It was a negative spiral only made slightly positive by my refusal to give up. I was desperately trying to learn about myself – and in the same time I involuntarily learned some about the beauty and cruelty of the world and the people in it. I tried to desperately understand and learn from my mistakes and was hoping and striving for a better tomorrow. A better me. That one year I spent was a year in clarifying and purifying turmoil. It was my personal inferno, my purgatory.

Then out of the blue it came. As I was vacuuming at the restaurant someone called me. Finally. A Company. A reply. Park Air Systems (PAS) in Horten, Norway. It was my dream. It was C++. It was something that mattered. They made a difference. I had a stumbling job interview over the phone. Norwegian and Swedish are supposed to be similar. For me it was not. To this day I cannot fathom how I made it past the first stumbling phone interview, but I did.

Some time later I was in Horten for my second and final job interview.The day before I had walked through the town all the way to the company location. I wanted to know exactly where it was. What the company looked like from the outside, how long time would it take me to walk from the hotel to PAS. In the afternoon I tried to grasp what the community of Horten was like and close to the end of the work day I remember that I called the secretary and asked about dress code. Good thing I did.

Me: “Should I wear a suit or be more casual dressed”.

The secretary told me with a laugh and with no room for misunderstanding
Her: “Hvis du ønsker jobben må du IKKE ha på deg en dress, du må komme uformell kledd” (If you want the job, DO NOT wear a suit, you must come dressed casual)

She was right. Just maybe I also had something to offer in addition to not wearing a suit? Either way I got the job as a Software Systems Engineer for Operations working with Air Traffic Surveillance. It was very exciting. Very important. And I was finally and officially an Engineer albeit a very junior one at that.

Park Air Systems (PAS) had basically only senior skilled-level engineers working at Operations and Development. After 4 years there I had worked as a junior developer, done my best as a Software Responsible for AMB Chile and Heathrow old and new tower. I was doing problem solving, system integration, performance and system tuning more than I was doing programming. I wanted to do more with software and I realized or at least felt that I would always be junior at PAS and always only do junior software developing assignments, if any. I was simply too good at being a Software Responsible in comparison to my skills as a relatively bleak (junior) software developer. There was no need or desire for the company to let me explore my software coding skills. In the end we, my wife and I (yes I managed to get married too, but that’s a whole other story) decided it was time to move back to Sweden.

In retrospective the Norwegian era was amazing. I had found a new belief in myself, managed to do some good work, whetted my appetite for coding and I got glimpses of how very good software could look like. My coding style is definitely colored by the experiences I made at PAS, by the C++ gurus I met. They helped to create the image of how I wanted to be as a software engineer.
…
After four years in Horten I could look back on that day in 2002 when I had gone to Norway with a hope of a software future, a woman that loved me, a job that was promising and a hint of martial art in the background. I am very grateful for my time in Norway and Horten, PAS and friends will always have a special place in my heart.

2006, kindred spirits and an appetite to learn

At HiQ-Stockholm I found kindred spirits. They had a passion for software that helped me exhume half-buried coding skills. It fed my desire to excel, to prove to myself that I had it in me. The whole company felt vibrant and positive. You could actually feel it at the company. It was in the atmosphere. It was felt from within, throughout from the core to its client-based consultants. Even today when I no longer work for HiQ I am amazed by what the CEO Lars Stugemo has achieved. I am convinced that his positive nature has created the HiQ company soul. The “If it is to be, it is up to me” motto we were using back in the days was not only an empty company value and vision but became very personal for me and many other employees.

I still try to live by that motto – if there is a change that I desire, that change should start with me. Be a doer, be active, to show by example, lots of guts and will was always in me but it really bloomed at HiQ. I owe a huge debt of gratitude from the software virtuosos and inspiring people that I met there.
At this time I really feel like dropping names. If nothing else then just for the reason that these guys are inspiring and should get some praise. I hope you don’t mind
Martin Vilcans who unknowingly inspired me, and probably many more, to teach at HiCollege.
Ivar Gaitan as another code and challenge seeker who had visions and shared them with me
And finally Manne Fagerlind who opened up the world of code quality and who showed us all that testing was inseparable to software development.

If you work in Sweden in the Stockholm area then maybe you will get lucky and come across these three. If you do, pay heed – you might just learn something!
OK I better stop now with the flattering or it will just be too much brown nosing. I only mention a few but seriously both at HiQ and at assignments I met so many ingenious people that all made huge impressions, inspired and taught me about software.

OK, after this Oscar-feel-nominee list it is time to continue… I am way off the subject, if there ever was one.

The Great Colleague Buddy Gurarantee

I learned some important lessons from my four years at HiQ. Apart from being able to immerse myself in C++, dabbling in other languages (Java, C#, Python), design principles, patterns, anti-patterns awareness, agile methodology and just having lot and lots of fun I had some very important recruiting experiences. From past experiences in Sweden I would say that normally the job-interviewing is a two step process.

1. A manager and a team leader or engineer will meet you for an interview. If you pass you get to step
2. Two managers meet you. This time some more exploratory questions and finally the salary discussion takes place. In the end it is the managers that not only is leading the whole interview process but also have the final say.

At HiQ it was also (usually) a two-or-three step process, but upside-down. First a group manager or two would meet the candidate. If the lucky candidate managed to get through the eye-of-the-needle then the most important step would come. The colleague evaluation!

It really is a simple concept. Only hire someone you would like to have as a colleague yourself. Not who the manager would like as a colleague, but who the other consultants would like to work with. From a number of interviews that I helped out with I can summarize it as following

Two consultants, potential colleagues, will sit down and talk with the nervous applicant. Sometimes testing his/her knowledge and sometimes only doing some relaxed talking about whatever,. but I think usually a mix of the two. In the end it is what the two potential colleagues thought of the applicant that mattered. Their decision was usually adhered to, and a no from them was always absolute. The colleagues had veto power and they used it carefully.

This is such an improvement from the manager-only-deciding interview process that I have seen at other companies. I believe that a lot of HiQ’s success is connected to their job-interview process and how it gives some kind of
Great Colleague Buddy Guarantee.

I think the order of steps is also important: step 1 manager, step 2 potential colleagues. The manager can weed out by experience. The consultant employees can use gut feeling, tests or whatever to grasp the what and who about the candidate. The manager(s) only let through the most promising applicants and only a few are selected for consultant interviewing. This made the consultants doing the second interview feel that it was important what they were doing. It was important of course. Doing the second and not the first interview also made it more interesting for the interviewers since anyone up for a second interview was bound to have skills, personality and a story … (well an interesting background). In short someone that would be nice to have as a colleague.

The bonus of this interview process is maybe already clear to you. Employees with a certain type of personality and passion for what they do will most likely pick candidates that have personalities that fit in and who share similar passion. They would also veto out the candidates that are not just felt the way they should feel, even if that candidate would have all the necessary know-how. Gut feeling, first impression and knowledge. These tools can build you an employee workforce that is unstoppable. They can achieve anything because they have vital strengths in common, they share a passion for what they do and they enjoy each others company.

This is also a risk of course. If you manage to build a formidable workforce like the one described above you will also risk that they at some point break away to form their own companies – maybe even persuading other people to join them as they leave. Then again, it might not be a bad thing. As these people leave they might just continue to be ambassadors and continue to talk well about the company.
Judging from what I just wrote above that is sure true of me.

Hmmm. “Gut feeling, first impression and knowledge”
Come to think of it, maybe in 2001 that was what stopped me from being a web developer,. then again if only fly fishing and golfing were the qualifying criteria then I was probably better off somewhere else
…

Hey. Did you apply to HiQ and did not get the job? Sorry if I bummed you out. If you have applied to HiQ and did not get hired then I am not saying that you do not have a passion for software. Or lack of a winning personality. Heck, you are probably a coder virtuoso that love code like Romeo loved Juliette, dream, sleep, eat and sh*t software like no one else. You probably have a personality and attitude that shine like the sun and would put all others to shame if we even compared us to you… Please know that just as with everything else there are false negatives that can stop the most suitable candidate. People will have off-days, the moon, girlfriend or too much lactose intake can affect a very human process.

Either way, from my personal experience, this is the best way I have come across to get the employees that are just awesome.

Or maybe this is just common practice everywhere? I think Google practices something similar in their rigorous 4-step hiring process. Although of course at Google you have to be exceptional not only with coding but with math, algorithm thinking in your veins and just Über at everything (including being humble, down-to-earth and not trying to show-off as über).

Sometimes a good thing will end

If I was professionally married to HiQ I think I started to fall out of love with the company. Some middle-management mishaps that affected me personally did its part and 2 hours commuting every day did not help either. I am just not a jammed-traffic-waiting-to-avoid-next-crazy-driver kind of guy. I will probably never fall down, all D-Fens style but driving all those hours in my loud Jazz was wearing me down. That much commuting, never knowing if I would be 1 hour, or more, delayed due to traffic is not for me. Especially not with a kid that soon would start preschool and demanded regular drop and pick-up times.

So once more it was time for a change…
2010, Critical and Preventive care with Medical Technology

I wanted to make a difference also. No more military spy planes for Pakistan. Seriously.
Let me say that again. I would like to make a personal statement now:

No more military technology assignments for me!

Hey. I am not saying it is bad. I am saying I am done with it. After the mandatory 15+ months as a 2nd Lieutenant and some 3+ years developing military technology I have done my part in this area. Especially since I feel that my karma took a big hit working with military technology… But who am I kidding really? I might have a preference but in the end it all comes down to paying the bills. At least it felt nice making that statement

I decided medical technology was my tech field of choice for the next adventure. After a long search I decided that Prevas was the right choice for me. Of all the consultant companies in Uppsala they seemed to have the strongest focus on the Uppsala market. Uppsala is where I live at the moment so that was important to me. Limiting commuting time was a must. Today I do not really know how I managed to get hired by them. What made them decide to hire me? Lots of luck and maybe a beneficial job market?

In my attempt to keep up my salary (which was likely to be significantly lower than before), to prove that I was good at consulting I believe I came off as arrogant, egocentric and not a person I would hire myself following the previous mentioned
Great Colleague Buddy Guarantee. God, that just sounded terrible. I hope it was not that bad. Either way here I am. Prevas – in all its glory, from within its core it shows (to me) the very essence of the Swedish Software Engineer. All competent, all experienced, many senior level engineers. I look forward to getting to know Prevas and its soul and learning more about creating great software, and maybe also about myself in the process.

2011, paternal leave for two months in the USA did wonders

I feel rejuvenated and am boosting with energy and inspiration. My agony, obsession, emptiness and pleasure all mixed together after publishing the G2log article at codeproject is finally behind me.

Now, back in Sweden again I look back on weeks with my in-laws when I could ponder, meditate, changing diapers and just hang out with my kids. All those hour long stroller walks while trying to get my daughter to take her post-lunch nap were great. All of these everyday life events while not messing around (much) with C++11, concurrency intricate designs or g2log were good. I decided to write this blog entry for once,. maybe more personal than I usually write. But that is what I really write the blog for anyway: for myself, personal record keeping and whomever else that might be reading it.

If I am lucky then maybe my war stories can help someone. Not giving up, trying harder, doing it differently, are all good life rules, at least for me and maybe also for you.

To be continued for sure …

Many thanks to the people who followed g2log and emailed me improvement suggestions. It made it a whole lot better!

Apart from some formatting issues, a missing word in the introduction and that the downloads are only available on BitBucket and not through CodeProject I think it looks OK. Hopefully the editors will fix the rough edges in a couple of days.

The performance tests were made on a Dell Latitude E6500 laptop, Intel® Core™2 Duo CPU P8600 @ 2.40GHz × 2, with a solid state drive. On another system using a standard hard drive the difference between synchronous (glog) and asynchronous (g2log) would be larger.

The performance comparison is more about comparing the power of asynchronous operations to synchronous operations. Disk file access I/O wait times can be painfully long. This is clearly helped if done asynchronously, and punished if done synchronously

In a worst case scenario with a lot of data congestion the Google’s glog shows wait times of 1/2 second up to a whole second when flushing log entries to disk.

Kjellkod’s g2log gets leverage by doing slow disk access in the background. With the same worst case test setup the maximum wait times for a LOG call to finish was in the range of 13 – 113 milliseconds.

In a not so extreme test, but still with heavy LOG usage (1 thread writing 1 million log entries) g2log has maximum wait times of about 13 milliseconds. Google’s glog peaks at 459 milliseconds.

With these high times it can be good to know that the average time for a LOG call was much better. For g2log the average time was in the range of 6 – 9 microseconds.

For Google’s glog the average time was in the range of 10 – 13 microseconds. The log entries buffering that makes glog pseudo asynchronous clearly pays off. Google’s glog still achieves very good average time for being a synchronous logger.

From my testing it is obvious that an asynchronous logging utility outperforms a synchronous logger. Especially the peak wait times, when doing slow disk access, is mitigated if done in the background.

In addition to outperforming the traditional synchronous loggers g2log also provides some crash security. At shutdown g2log flushes all FIFO queued log entries to file.

In case of a fatal event, such as segmentation fault or floating point exception (and more), a signal handler will receive the fatal signal then notify g2log, which resends the aborting signal after saving to file all the pre-fatal log entries. The resent fatal signal will finally abort the application.

Too see how it is done, and to start using g2log you can go to http://www.kjellkod.cc/g2log-efficient-background-io-processign-with-c11.

G2log’s code is completely free, available as a Public Domain Dedication.

I think the best one, at least the easiest read,  so far is “Active Object the C++11 way“.  It was previously named “… C++0x way”.

I put this stuff up mostly because I think it’s fun, but once in a blue moon I get an email from someone who is using this, or another snipped and just wanted to say thanks. Those days I just get a big mental reward and a silly smile.

It’s nice to make a difference.

This is good, quick online help when starting out with C++11.

I’m sure to look over this and of course Bjarne Stroustrup’s more in depth C++11FAQ as g2log moves closer to being released.

Several days now into the September moose hunt in Northern Sweden. Our elkhound Aslan is on “vacation” with it’s owner till Friday. Outside the rain has poured down for 2 days and we’ve taken the opportunity to clear the cabin, stock up on our supplies and we’re to return before sunrise tomorrow at the cabin – mayhap it has cleared up by then?

Anyhow, this entry is really about a type of hunting, moose stalking, that I’m getting pretty good at, although (as you’ll read) I still have lots to learn.

Three days ago I was scouting out our most northern area for tracks. This is an area where we normally don’t hunt for a number of different reasons. I got surprised when I saw lots, and lots of moose tracks. A few of them fresh and with two tracks from a moose cow and it’s calf. The tracks were made this night or maybe just recently.

I decided to stalk them and got lucky, or well, almost. The moose tracks were relatively easy to follow. The tracks went alongside a tiny path in the forest, making sneaking easy as it was less brush to stay clear of and also making the prints of the moose easier to spot. 1 hour and 45 minutes later I had followed them for approximately 1 km. In front of me was trickier terrain, I could continue on the path but had to dodge 4 trees with low branches that went over the path, or I could walk around the trees but the high brush would also make that much slower and increase the possibilities for mistakes. I decided to dodge the branches.

The first tree went just fine. Some 17 years of martial arts have helped my balance and some ridiculous slow acrobatics moved me over a tiny spruce, under and sideways of some branches until I was in the clear. I took my time and surveyed the surroundings. Up ahead I could see the clearing, a bog, that I was expecting. I had planned to stay put at the clearing for 20 minutes and then head back again since the sun has started to get low.

I guess I was too focused on the clearing and if any moose was grazing far ahead. I just completely missed what was almost in front of me. Just to the side of the next trees to dodge, behind a thin spruce was a moose cow. A sudden, but not too sudden movement of gray alerted me. I froze in place. I heard the calf up ahead. Had they heard me? Had they seen me? The cow had been very close, ludicrously close even. I was hoping they they hadn’t but my mind was a blank. I did not try to go through, or sidestep the spruces I only waited for them to show up on either side.

After what felt like an eternity, but was probably just a couple of minutes. I heard the brush rustle as they moved away. Maybe a trick of the wind had given them my scent? Or maybe they were just moving away. Either way they had stayed for those minutes, just beyond the spruces and close enough for me to act on, if my mind would’ve worked…

I moved up and checked out the area. Yes, it was the moose and the calf that I had tracked. No doubt about it.

I went back to where I had stood and measured with steps the distance from my position to where the moose cow had hidden. It was a distance of 12 steps. 12 steps! That would be approximately (for me) 10 meters or 33 feet. Amazing. And I missed it.

The next day I just had to return. I measured the steps again. Checked the tracks and took a picture with my IPhone. Of course in the morning light it looks impossible to miss a huge moose behind the thin spruce (just left of the center). Use your imagination,. the trees to the right of it was casting a dark shadow. As the moose was in the deep shade and hard to see I still think it was short of a miracle that I missed it. Funny, Crazy, Sad and Unbelieving.

I know that hunters often come very close to moose. Usually when dogs are involved and the moose is on the move they can almost run over the hunter. So 10 meters away isn’t remarkable that way. However, 10 meters away, when walking straight at the moose THAT my friend is remarkable. The closest I got before was 25-30 meters, but then I knew the moose was there, now I didn’t and was just trying to be “sneaky” (if you get my drift)

Tomorrow is a new day. Rain or not I’m gonna stalk the bogs, the clearings and the forest. This time …

2011-09-25 update: Too bad the rain just kept pouring down. Of the hunting days all of 4+ days just rained away. 160mm of rain in 4+ days. Flooded swamps, roads and just very, very wet. At least I got a good last day. Tracking a bull moose for almost 4km,. then he made a semi-circle and if I would’ve continued he would’ve gotten wind of me so this year (so far) no moose meat in the freezer. But in 10 days I go hunting again

…

So how do I do moose stalking? Below I’ve written down my ideas how I do the moose stalking. I’m sure I’ve lots to learn but it seems to be going quite OK.

Stalking is painstakingly slow. Apart from the wind catching your scent and signaling the moose the animal also has great hearing and although it has pretty bad eyesight it’s still good at spotting movement . So for a successful stalk you need to basically have the wind in your favor, being virtually completely silent and moving so slow that even if the moose looks at you it won’t easily see your movement.

This type of stalking is something I’ve practiced the last 4 years I’ve been hunting. My neighbor told me how he hunts and inspired me, hopefully I’ve even stepped it up a notch. Basically it’s very, very, very slow moving. Each step being placed carefully, slow, deliberately with a precise shifting of my weight on the next foot. This way you can stop mid-motion in case there’s a hidden branch under your foot that you missed – you can change the placing of the step and avoid a disastrous “crack” that would alert the prey.

Every 10 steps or so, or whenever the terrain is suitable I slowly lower myself so that I can see under the branches, getting a different look at your surroundings is important.

Every now and then, say 30 steps or whenever suitable I just freeze in place. Take in the forest sounds even more and try to penetrate visually what’s beyond the spruces.

Of course moose have a sense of hearing that is just short of incredible. They can move their ears in any direction without moving their head which helps them pinpoint very accurate where a sound comes from. So how do I avoid getting noticed by sound? Well, I use “active hearing protection” (see image of a similar product) with this I can magnify sound a lot (I think approx. 17-18dB). This is great for hearing animals in the forest but above all it makes it very obvious how much sound I make.

By magnifying all the sounds nearest. i.e. my sounds I can really hear the rustling of my clothes, the sound of a tree branch sweeping across my pants or the crushing of a leaf under the boot. This will make you learn very fast to control your movements until the sounds you make are less than the natural sounds of the forest.

What about mistakes? Yes. They do happen, but by taking care and following the advice above they should be minimized. When a mistake happens, a twig cracks, or you *whatever* … then the solution is simple. Just freeze in place. One sound is no sound. After all the forest is full of sounds, it’s only when consecutive sounds (mistakes) come from you that the animal will react.

By using this tactic I have successfully (knock-on-wood) being able to sneak close to moose, spotting them and moving closer – hopefully for a killing shot- 3 out of 4 years (this hunt is still ongoing,. and I guess it’s debatable whether or not I managed it so far this year — top section)

The Cilk plus seems very promising. Intel bought up the Cilk Arts company and is working now towards fully commercializing it , for now free as a “technology preview” — and maybe more in focusing on getting the revenues from the Cilk behind the curtains technology such as CilkScreen that guarantees to find the parallel code determinacy races.

In short Cilk Plus provides an easy way for the programmer to expose the parallelism. This used to be most used, and most efficient for library writers, low lewel, embedded type of optimization. But now since Intel is stepping it up a notch and is really promoting Cilk Plus and especially its many advantages to OpenMP I believe Cilk Plus has a real chance in making it into the application software as well.

A word about Cilk Plus vs OpenMP. OpenMP is an open standard so to compete with that Intel has made Cilk Plus open as well and welcome other compiler vendors to implement it. In fact it’s well in progress for GCC and a patch to support the most important Cilk features was released as a GCC 4.7 patch this August. Cilk Plus also scaled much better than OpenMP and does not penalize other processes as OpenMP does.

Two affiliates with Intel was at SICS Multicore Days to promote Cilk plus. Dr. Arch Robinson from Intel who showed practical easy-to-use programming examples (here’s his cilk plus blog series and Intel blogs for Cilk Plus). And the Cilk Plus keynote from Professor Charles E Leiserson from MIT (a Cilk founder). I don’t think the multicore days presentations are available yet but I found this presentation that was borrowed from Charles’ lecture and it seems to be almost the same: http://developer.amd.com/afds/assets/presentations/2080_final.pdf

Let’s cut to the chase let’s see a classic simple example of Cilk Plus. The example is the fibonacci calculation as shown in this Intel presentation.

int fib(int n)
{
   if (n < 2) return n;
   int x = fib(n-1);
   int y = fib(n-2);
return x+y;
}

Which with nested parallelism would look like this

int fib(int n)
{
   if (n < 2) return n;
   int x = cilk_spawn fib(n-1);
   int y = fib(n-2);
 cilk_sync;
  return x+y;

Another nice feature is the cilk_for that for array and matrix operations many times can replace the normal for-loop. At scope end the cilk_for will synchronize the parallel calls. This is implied implicitly and can also, as shown above, be forced wherever with cilk_sync

Here’s another view of Cilk Plus from Dr.Dobb, unfortunately it does not describe CilkScreen which I think goes with Cilk Plus as hand-in-glove

As soon as the moose hunt is done with I’m sure to check out Cilk Plus

Today I read a somewhat dated post regarding the coming Qt5 which is the next big Qt API change. The move from Qt3 to Qt4 was huge and it wasn’t done in a snap to convert Qt3 made software to Qt4 – but this time seems that most of the API:s will stay intact. So I hope that Qt4 based software can do this transaction without too much headache for the programmers.

With Qt5 the Qt Quick technology is definitely put in focus. As Qt Quick is a web development inspired technology GUI and widget making should get easier and faster. But the change of focus from Qt/C++ to Qt/Quick might not be altogether positive. I see a trend that companies are choosing not to adopt to the Qt Quick technology for a few different reasons.

1. Transparency. In Qt/C++ most *magic* behind the curtains can be crystal clear with just a small effort on the developer’s side. If problems occur you can debug your way into the C++ core of Qt itself. With the Javascript-ish QML that transparency is gone or at least muddled thanks to the extra layers for handling QML.

2. Performance. Already today Qt is suffering from performance issues. Putting QML on top befuddles even more the reason why some operations are sluggish. It’ll make it much harder to see if it’s the Qt/C++ part or the QML part that’s the culprit.

Already now it’s pretty darn hard to find out these things, especially when you throw in graphic driver dependencies as well.
…

Let me give you an example of the performance issues I’ve seen with Qt. Maybe this is actually the core of today’s blog:

Especially with the native graphical engine on Linux-types of OS:es can Qts performance issues be manifested. This is evident on Intel’s Atom platforms with integrated graphical chips.

Inte’s drivers should support all the necessary bells and whistles but even standard 2D rendering operations are sluggish with X using up most of the CPU resources. This is very easy to verify with Qt’s example software “concentric circles“.

Use the native graphical engine, hack the concentriccircles example so it runs 10.000 “circle operations” and measure the time for this. Then run the same concentriccirles example but this time with the runtime flag for using Qt’s raster graphical engine. With -graphicssystem raster the graphics will be software (Qt) rendered and not through X11/Intels drivers. All of a sudden, by bypassing the Inte’s drivers the rendering performance gets a boost. On three different Intel Atom Linux projects I’ve heard about, or worked with firsthand the difference was in the area of 80% X11/CPU utilization (native) vs 2-4% X11/CPU utilization (raster).

Here Qt probably says it’s Intels driver’s fault, where Intel blames it on Qt’s lack of OpenVG support. The solution is however easy, no matter whose fault it is. Just use -graphicssystem raster and your system should work great.

According to testing of Qt and communication with Intel I’ve gleaned the following.

-graphicssystem raster works great. Low CPU overhead and high speed for most (all) operations. Software rendering is done in the Qt libraries (?) instead at the driver.

-graphicssystem native (the default) will make your Linux ( in case of Intel Atom with integrated graphical chip) very sluggish due to the CPU (X11) overhead.The fault of this is according to Intel how Qt approximates a sine wave using the native X calls. This is highly inefficient and resulting in a multitude of extra calls into the Intel driver for creating/deleting pixmaps and drawing trapezoids.

Whether this is true or not I’ve not even tried to confirm. I know that raster works so that is my recommendation to go with. A few years ago (2009?) raster wasn’t that stable but I hope it is now. However it might be for good reasons that the trolls haven’t made raster the default on X11, so who knows there might be other pitfalls when using raster that I do not know about.

I guess my blog lost focus a bit. My point being. Adding a new layer on top of Qt for the GUI work can be great for decreasing developing time. But this will definitely come at a cost whenever the software engineers doing the work run into performance issues – are those issues due to Qt or QML? In the end, if these issues occur or cannot be understood or worked around it may be a show stopper for the project! forcing the project to switch graphical technology or hardware altogether.

I guess I’ll have “fun” debugging tasks in a year or so