c++11 std::array

Here's a quickie on the std::array. its a nice wrapper on the old c-style lookup table approach that gives it a std container like feel to it with very little fuss.

#include <iostream>
#include <array>

int main()
{
    std::array<float,3> a = {1,2,3};

    for(auto& b: a)
        std::cout << b << ' ';
    std::cout << "\n";

    std::cout << a[1] << "\n";

    std::array<std::array<float,3>, 3> b
        = {{ {1,2,3},
             {2,4,5},
             {3,6,9} }};

    for(auto& c: b)
        for(auto& d: c)
            std::cout << d << ' ';
    std::cout << "\n";

    std::cout << b[1][2] << "\n";
}

hashed RTTI and Mixin's for Semi-UUID taging class

Quite a while back I posted some code that uses RTTI to UUID tag classes. In the post i mention that a hash on the RTTI info can be used to also ID the class. Here ere are a few implementations that use std::hash to get a semi-unique ID and automatically apply it to a class.

There are 2 main methods demoed here;

• A template constructor with pointer approach called "AutoIDPtr" which is what you want if your classes have many layers of inheritance like the "CChild" example
  
• A CRTP Mixin approach call "AutoIDMixin" is also possible and cuts out the mess on the stack at runtime, BUT this prevents any more derivation on the class. It also means that you have to keep typing out the instantiated class name in all the constructors as well which gets annoying.
  
• A final way that is not shown is to use pass down a summary of results in a AutoIDBase class and not derive from the AutoID.. classes at all.
  

Keep in might that hash has a chance of clashing, if this happens you can always specialize the Attribs/ClassDictionary class. Also the ClassDictionary is completely optional. You can simply delete it use the Attrib Traits ID inplace of the REG_ID at get a working system.

#include <iostream>
#include <iomanip>
#include <stdint.h>
#include <typeinfo>
#include <functional>
#include <map>

/****************************************************************
 ****************OPTIONAL INFO GATHERING PLACE*******************
 ****************************************************************/

struct ClassDictionary
{
    struct Reg
    {
        uint32_t    id_;
        std::string name_;
        uint32_t    hash_;
    };

    typedef std::map<std::string, Reg> RegMap;

    template <typename T>
    static uint32_t reg()
    {
        std::string key  = typeid(T).name();
        uint32_t    hash = std::hash<std::string>()(typeid(T).name());

        std::cout << "Registering hash:"
                  << std::hex << std::setw(8) << hash << std::dec
                  << " as "  << key
                  << "\n";

        return regCore(key,hash);
    }

    static uint32_t regCore(std::string key, uint32_t hash)
    {
        RegMap::iterator rit = regMap_.find(key);
        if (rit != regMap_.end())
            return rit->second.hash_;

        regMap_[key].id_ = gID++;
        regMap_[key].name_ = key;
        regMap_[key].hash_ = hash;

        return regMap_[key].hash_;
    }

    static void printList()
    {
        std::cout << "ClassDictionary has " << gID << " entries\n";
        for (const auto& e : regMap_ )
        {
            std::cout << "Entry:" << e.second.id_
                      << " Hash:"
                      << std::hex << std::setw(8) << e.second.hash_ << std::dec
                      << " Name:" << e.second.name_
                      << "\n";
        }
    }

    static uint32_t gID;
    static RegMap regMap_;
};

uint32_t                ClassDictionary::gID = 0;
ClassDictionary::RegMap ClassDictionary::regMap_;

/****************************************************************
 ***************************THE CORE*****************************
 ****************************************************************/

template <typename T>
struct Attrib
{
    static const uint32_t REG_ID;
    static const char*    NAME;
    static const uint32_t ID;
    enum { SIZE = sizeof(T) } ;
};

template<typename T>
const char* Attrib<T>::NAME = typeid(T).name();

template<typename T>
const unsigned Attrib<T>::ID = std::hash<std::string>()(typeid(T).name());

template<typename T>
const uint32_t Attrib<T>::REG_ID = ClassDictionary::reg<T>();

class Base
{
public:
    Base(uint32_t type, uint32_t size) :
        type_(type),
        size_(size)
    {}

    void whoami()
    {
        std::cout << "I am type:"
                  << std::hex << std::setw(8) << type_ << std::dec
                  << " size:" << size_
                  << "\n";
    }

private:
    uint32_t type_;
    uint32_t size_;
};

class AutoIdPtr : public Base
{
public:
    template <typename T>
    AutoIdPtr(T* kid) :
      Base(Attrib<T>::REG_ID, Attrib<T>::SIZE)
    {}
};

template <typename T>
class AutoIdMixin : public Base
{
public:
    AutoIdMixin() :
      Base(Attrib<T>::REG_ID, Attrib<T>::SIZE)
    {}
};

/****************************************************************
 ***************************TEST IT******************************
 ****************************************************************/

class A : public AutoIdMixin<A>
{
public:
    A() : AutoIdMixin<A>()
    {}
};

class B : public AutoIdPtr
{
public:
    B() :
        AutoIdPtr(this)
    {}

    int var;
};

class C : public AutoIdPtr
{
public:
    C() :
        AutoIdPtr(this)
    {}

    template <typename T>
    C(T* child) :
        AutoIdPtr(child)
    {}

    char stuff[10];
};

class CChild : public C
{
    CChild() : C(this)
    {}
};

class IHaveALongAndMessyName : public AutoIdPtr
{
public:
    IHaveALongAndMessyName() :
        AutoIdPtr(this)
    {}
};

int main()
{
    A a;
    B b;
    C c;
    C c2;

    a.whoami();
    b.whoami();
    c.whoami();

    ClassDictionary::printList();
}

The output looks like

Registering hash:85b5dc34 as 1A
Registering hash:60a08ea3 as 1B
Registering hash:d9107b5c as 1C
Registering hash:59e2da5b as 22IHaveALongAndMessyName
Registering hash:52593138 as 6CChild
I am type:85b5dc34 size:8
I am type:60a08ea3 size:12
I am type:d9107b5c size:20
ClassDictionary has 5 entries
Entry:0 Hash:85b5dc34 Name:1A
Entry:1 Hash:60a08ea3 Name:1B
Entry:2 Hash:d9107b5c Name:1C
Entry:3 Hash:59e2da5b Name:22IHaveALongAndMessyName
Entry:4 Hash:52593138 Name:6CChild

Great resouce on good code design

This post was a nice run down on the design principles used for reusable and reworkable designs.
http://net.tutsplus.com/tutorials/how-to-write-code-that-embraces-change/

And there are less excuses now with C++11. Decoupling interfaces is a blast with lamdbas, Of course the older c++ versions could use boost::bind, it was just as effective. In my post a week or 2 back u can see the almost full decoupling of a producer consumer system via the bind method Here - Check for the "setCallback" function.

logging top CPU and MEM usage

Hacked a script to dump a log of the top CPU and MEM processing for a machine a while back.. All the info for it was nicked from here:
http://unixskylab.wordpress.com/2009/01/05/my-favorite-aix-top-resource-consuming-commands/

I figure its a rather useful little script to keep handy... so here is a quick reproduction but I don't have an AIX machine at home so its 50-50 that the AIX version will work off the bat... And after a couple of tweeks its ready and tested for ubuntu...

For AIX (untested)

while [ 1 ]; do 
  DATE=`date "+%Y%m%d"` 
  LOG=cpu_mem.log.$DATE 
  if [ 1 ]; then 
    echo; 
    date; 
    ps aux | head -1; 
    echo "TOP MEM"; 
    ps aux | sort -rn +3 | head; 
    echo "TOP CPU"; 
    ps aux | sort -rn +2 | head -10; 
    sleep 10; 
  fi | tee -a $LOG 
done 

For Unbuntu

while [ 1 ]; do 
  DATE=`date "+%Y%m%d"` 
  LOG=cpu_mem.log.$DATE 
  if [ 1 ]; then 
    echo; 
    date; 
    ps aux | head -1; 
    echo "TOP MEM"; 
    ps aux | sort -rnk4 | head; 
    echo "TOP CPU"; 
    ps aux | sort -rnk3 | head -10; 
    sleep 10; 
  fi | tee -a $LOG 
done 

Git show the version prior to the commit hash

getting the verion prior to the current hash point in git is yet another one of the annoying little bits of pointless info that makes people hate unix tools...

http://stackoverflow.com/questions/338436/is-there-a-quick-git-command-to-see-an-old-version-of-a-file

The secret is the "~1" which means prior to the commit hash number you gave. EG

git show commithash~1:path/to/file 

A basic async queueless and lockless multi threaded boost::asio producer consumer example

Sean Parents talk at this years GoingNative2013 event about "Tasks" and his critic of futures got me thinking about my use of multi threaded async asio responders

Seans talk is online at http://channel9.msdn.com/Events/GoingNative/2013/Cpp-Seasoning and it is well worth the time to watch it and the rest of the series.

I dont quite see how Seans "Tasks" idea can work without any kind of mutex/locks but i might have miss-understood him.. The asio io_service's post method is thread safe and im not certain(and to lazy to check) but it seems like that would suggest that the posted message queue inside the asio should be mutex protected... The sample code i have created here has no locks at all..

Besides that my code is unfortunately old(i just cut it down for a post out).. it still uses boosts threads and binds instead of the newer std threads and lamdbas. But hey whats a blog post without bugs, typos and caveats..

Basically this code is all about passing messages (ie a request for some jobs sub task to be done) between a producer thread and a consumer thread using the asios "post" method.

The code is a multi threaded async event processing model so it can be confusing to read. Boosts Asio offers a busy system for io_service (called boost::asio::io_service::work) to keep the threads alive but i prefer to use a async heartbeat event fired by a timer.. that way you can periodically monitor the threads health and take timed actions related to it eaiser... The core operation is that threads call into "run" and boot off the "start" to generate the initial post messages for send_msg and then the task goes on generates more "post" for both the producer and consumer sides of the job, until it reaches the end point at 3mil messages..

It uses no locks to achieve this and can pass the 3million messages(which are just ints) in about 9-10 secs on a intel i5-2450 notebook.

UPDATE: forgot to mention there is no backoff throttle between the producer and consumer so the producer can flood out the system with messages. If you want to fix that just add a Queue or Circular buffer for the messages between them. The producer times out when the queue level hits an upper threshold, and posts are sent only if the consumer is at the lower threashold/idling/or a check is done at each heartbeat.

Another option is to do this via a virtual queue(like the counters from the circular buffer) e.g use 2 message counters one for consumed and one for produced.. the delta of them is the queue message dept.. youll note that this double counter option is also lockless..

#include <iostream>

#include <boost/bind.hpp>
#include <boost/asio.hpp>
#include <boost/thread/thread.hpp>

#include <boost/thread/mutex.hpp>
#include <boost/function.hpp>

/* RESULTS
DEBUG 116 send msg:3000000
DEBUG 56 recv msg:3000000
DEBUG 101 tock
DEBUG 90 run ended ??
DEBUG 42 tick
DEBUG 32 run ended ??

real    0m9.050s
user    0m0.000s
sys     0m0.031s
 */

const int EXIT_COUNT=3000000;

typedef uint32_t Message;

class Consumer
{
public:
    Consumer() :
        timer_(io_service_)
    {}

    void run()
    {
        // WARNING THREAD CORE LOOP!
        std::cout << "DEBUG " << __LINE__ << " run started\n";

        running_ = true;
        start();

        io_service_.run();
        running_ = false;

        std::cout << "DEBUG " << __LINE__ << " run ended ??\n";
    }

    void start()
    {
        io_service_.post(boost::bind(&Consumer::do_heartbeat, this));
    }

    void do_heartbeat()
    {
        std::cout << "DEBUG " << __LINE__ << " tick\n";
        if (running_)
        {
            timer_.expires_from_now(boost::posix_time::seconds(1));
            timer_.async_wait(boost::bind(&Consumer::do_heartbeat, this));
        }
    }

    void recv_msg(Message msg)
    {
        if (msg >= EXIT_COUNT)
            running_ = false;

        if (msg % 1000 == 0)
            std::cout << "DEBUG " << __LINE__ << " recv msg:" << msg << "\n";
    }

    void check_que(Message msg)
    {
        //WARNING external thread entry point
        io_service_.post(boost::bind(&Consumer::recv_msg, this, msg));
    }

    boost::asio::io_service io_service_;
    boost::asio::deadline_timer timer_;
    bool running_;
};


class Producer
{
public:
    Producer() :
        timer_(io_service_),
        count_(0)
    {}

    void run()
    {
        // WARNING THREAD CORE LOOP!
        std::cout << "DEBUG " << __LINE__ << " run started\n";

        running_ = true;
        start();

        io_service_.run();
        running_ = false;

        std::cout << "DEBUG " << __LINE__ << " run ended ??\n";
    }

    void start()
    {
        io_service_.post(boost::bind(&Producer::do_heartbeat, this));
        io_service_.post(boost::bind(&Producer::send_msg, this));
    }

    void do_heartbeat()
    {
        std::cout << "DEBUG " << __LINE__ << " tock\n";
        if (running_)
        {
            timer_.expires_from_now(boost::posix_time::seconds(1));
            timer_.async_wait(boost::bind(&Producer::do_heartbeat, this));
        }
    }

    void send_msg()
    {
        count_++;

        callback_(count_);

        if (count_ % 1000 == 0)
            std::cout << "DEBUG " << __LINE__ << " send msg:" << count_ << "\n";

        if (count_ > EXIT_COUNT)
            running_ = false;
        else
            io_service_.post(boost::bind(&Producer::send_msg, this));
    }

    template <class C>
    void setCallback(C callback)
    {
        callback_ = callback;
    }

    boost::function<void (Message)> callback_; // reciver for signaling..

    uint32_t count_;

    boost::asio::io_service io_service_;
    boost::asio::deadline_timer timer_;
    bool running_;
};

int main(int argc, char* argv[])
{
    try
    {
        Producer p;
        Consumer c;

        p.setCallback(boost::bind(&Consumer::check_que, &c, _1));

        boost::thread pThread(boost::bind(&Producer::run, &p ));
        boost::thread cThread(boost::bind(&Consumer::run, &c ));

        pThread.join();
        cThread.join();
    }
    catch (std::exception& e)
    {
        std::cerr << "Exception: " << e.what() << "\n";
    }

    return 0;
}

C++11 no floats in templates??? WTF???

Sigh... i have no idea why c++11 doesn't allow floats in templates...

template <float number>
struct Value
{
  static const float VALUE = number;
};

The crazier part is that they have allowed enough other syntax sugar that you can fake it.. So why not just support it in the first place??

#include <iostream>

template <int NUM, int DEN>
struct Float
{
    constexpr static float value() { return (float)NUM / (float)DEN; }
    static constexpr float VALUE = value();
};

template <class GRAD, class CONST>
struct LinearFunc
{
    static float func(float x) { return GRAD::VALUE*x + CONST::VALUE; }
};

int main()
{
    // Y = 0.333 x + 0.2
    // x=2, y=0.866
    std::cout << " func(2) = "
              << LinearFunc<Float<1,3>, Float<1,5> > ::func(2) << std::endl;
}

Arrays and sizeof fun.

Sizeof arrays is a rather large land mine in c/c++. Even the most experienced coders will often hack out a piece of code and suddenly come to a screeching halt half way in when they realize they are being an idiot(hopeful they realize...).

The problem is that arrays in C are supposed to be well arrays.. but people often use them as shortcut to define some small non-standard chunk of data, and then you have a nice little surprise waiting to happen when your later upgrading or re-factoring or just using that piece of code. (otherwise know as the "30 seconds saved, 3 hours wasted" coding style..)

Here is what happens when you use arrays as a replacement type:

#include <iostream>

void func(char c[5])
{
  std::cout << "sizeof(param,5):" << sizeof(c) << std::endl;
}

void func2(char c[14])
{
  std::cout << "sizeof(param,14):" << sizeof(c) << std::endl;
}

typedef char Typedefed[5];

void func3(Typedefed c)
{
  std::cout << "sizeof(typedef,5):" << sizeof(c) << std::endl;
}

int main()
{
  char a[5];

  std::cout << "sizeof(local,5):" << sizeof(a) << std::endl;
  func(a);
  func2(a);
  func3(a);
}

Result (yes it does compile even with the char[14] array being handed a char[5], segv surprise.. yummy!):

sizeof(local,5):5
sizeof(param,5):4
sizeof(param,14):4
sizeof(typedef,5):4

So whats going on.. well this is the unfortunate consequence a design decision in the Ansi C standard "6.2.2.1 Lvalues and function designators".

"Except when it is the operand of the sizeof operator or the unary & operator, or is a character string literal used to initialize an array of character type, or is a wide string literal used to initialize an array with element type compatible with wchar_t, an lvalue that has type ``array of type'' is converted to an expression that has type ``pointer to type'' that points to the initial element of the array and is not an lvalue. "

So to work around it your forced to take other measures. Generally I believe its a good idea to do it the right way first time and simply create a trivial struct for your primitive type, consider it a slightly bloated version of a typedef. Your alternatives to this are always passing a ref to the array, or always passing its size with it.

#include <iostream>
void wasteful(char* c, std::size_t s)
{
  std::cout << "sizeof(c):" << sizeof(c) << std::endl;
  std::cout << "sizeof(s):" << sizeof(s) << std::endl;
  std::cout << "wasted  :" << sizeof(c) + sizeof(s) - s << std::endl;

  std::cout << "used  :" << s << std::endl;
}

void easytoforget(char (&e)[5])
{
  std::cout << "sizeof(e):" << sizeof(e) << std::endl;  
};

struct Structed
{
  Structed(char* a)  { memcpy(a_,a,sizeof(a_)); }

  char a_[5];
};

void fixed(Structed e)
{
  std::cout << "sizeof(e):" << sizeof(e) << std::endl;  
};

int main()
{
  char a[5];
 
  std::cout << std::endl << "Solutions:" << std::endl;
  wasteful(a, sizeof(a));
  easytoforget(a);
  fixed(a);
}