Showing posts with label algorithms. Show all posts
Showing posts with label algorithms. Show all posts

Monday, June 11, 2012

C++11 lockless queues

I have been messing around with the new c++11 threading in order to write a post on it... as per usual i ended up side tracked on something else.. while coding it i got thinking about the lockless vs lock implementations. Basically lockless implementations require you to design to very hard restrictions. They are "write mastering" and "obstruction free updating"

Write-Mastering
Write Mastering is where a data variable is updated by a single thread. Its often surprising how easy this is to do. Eg for a Queue the head is mastered from the producer side and the tail is mastered from the consumer side. (this is the example below)

Obstruction-freedom
When write mastering is not possible an obstruction-free check can be used This technique doesn't explicitly lock a data structure but instead uses a pair of "consistency markers" that are updated the sequance of read, update check 1, copy, copy update, (reread)check, write back copy(or rollback), update check 2.

To fully explain it: When an update is planed the first consistency marker is compared to the second. If they dont match enter a spin lock until they do. If/When they match the structure is free for an update. The first marker is scrambled and written to something new. The update proceeds on a copy and once its done the markers are rechecked to see if they are still scrambled as the thread choose. If both markers are cleared then nothing else changed them so then the update writes back the data copy and then updates the secondary marker to match the first. If the compare fails then the updated copy is tossed and the process starts again.

Lockless designs tend to be big cpu and memory wasters when they get into the lockless spins or are constantly clashing over updating shared structures. As a result the system running them should be balanced, tuned and as large as possible so that there is always data ready to be processed in each thread with as few conflicts as possible. You can get a taste of how cpu intensive they are by running the following example and watching your CPU hit the roof.

//complie with 
//g++ -std=c++11 lockless_queue.cpp -o lockless_queue.exe
#include <stdint.h>
#include <thread>
#include <iostream>

class LocklessQueueSys
{
  enum { size=1000};

public:
  LocklessQueueSys() :
    head(0),
    tail(0)
  {}
  
  void producer()
  {
    uint32_t count= 0;
    while(1)
      {
 while(((head+1)%size)== tail); //spin lock
 msg[head] = count++;
 head = (head + 1) % size;
      }
  }
  
  void consumer()
  {
    uint32_t expect=0;
    while(1)
      {
 while(head == tail); //spin lock
 if(expect != msg[tail])
    std::cout << "Error:" << expect  <<  "\n";
 if(expect%10000000 == 0)
    std::cout << "check:" << expect  <<  "\n";
 expect=msg[tail]+1;
 tail = (tail + 1) % size;
      }
  }
private:
  int32_t msg[size];
  int32_t head;
  int32_t  tail;
};

int main()
{
  //Use a member function in a thread
  LocklessQueueSys x;
  std::thread tpro(&LocklessQueueSys::producer, &x);
  std::thread tcon(&LocklessQueueSys::consumer, &x);
  
  tpro.join();
  tcon.join(); 
}

Friday, May 4, 2012

c++11 lambda

c++11 lambdas

Lambdas are an excellent and long awaited addition to the c++ language. Lambdas where simply not possible before but the Boost lib offered a rather passable set of macros that let you get away in-lining a kin of lambda. (basically it was a series of functional objects)

The lambda syntax however makes my skin itch a bit. It is yea another reuse of the bracket characters - [](){} - and one that is potentially lowering the readability of the code and introducing coding errors as a result. I would have preferred something a bit more clear and obvious.

Firstly the basic lambda syntax is [capture](parameters)->return-type {body}. This is often short handed the most basic form when people blog and type about it which is []() { .... }


The first [] pair forms a "capture". The capture is run one time at and in the context of the lambdas creation(ie the point where you put the code). There several convenient shortcuts that grab the various variables from the current context automatically. Consider these to be members (and constructor params) of the resulting lambda object.

The next () pair forms the "parameters" these are the same as the parameters of a function definition. This resembles the operator(...) function call of a functional object or the parameter list of a function pointer. Note that these ARE surprisingly optional, but generally they are not left out.

The difference between capture and parameters is rather subtle (or obvious depending on your point of view or experience with it to date). It can at can seem like overkill or just outright fluff when you are constantly using lambdas in the local context. However the key point to realize is that if you pass the resulting lambda out of the current context without executing it then the "capture" has already completed for the context that you made it in and passed it out of. As a result the parameters in the capture must still be IN CONTEXT at the point of the Lambdas actual execution (or the capture was made by value). Parameters on the other hand are filled in when the Lambda is actually executed.

The -> forms the return type. It is optional and often left out. However it can become quite important when templates are used with lambdas otherwise the compiler can get terribly confused with which of the specializations are supposed to be put in place. I assume this is just be a problem with the maturity level of the current gen of c++11 compilers.

And of course the {} forms the body of the code. The same scoping rules apply here as if you coded the object as a completely separate member function of a functional object. So keep in mind that you can not reach outside to variables that where not passed in through the capture or params.

Ok an example: The little known summing of numbers.. zzZZZ.. ha wadaimiss...


// comile with 
//  g++  -std=c++11 lambda.cpp

#include <iostream>
#include <algorithm>
#include <vector>
#include <functional>

void examplePassByValue(int* start, int* end)
{
  int stuff[]={1,2,3,4,5,6,7,8,9};
  int sum = 0; 
  std::for_each(stuff, stuff+(sizeof(stuff)/sizeof(int)), [sum] (int v) 
    {
      std::cout << v << "\n";
    // This is passed by const value (os its BAD.. if you try to modify it.. )
    //sum += v;
    });
}

void examplePassByRef(int* start, int* end)
{
  int sum = 0; 
  std::for_each(start, end, [&sum] (int v )
  {
    std::cout << v << "\n";
    sum += v;
  });
  std::cout << " sum is : "<< sum << "\n";
}

void examplePassByRefImplicatScoped(int* start, int* end)
{
  return;

  // pass by ref of the entire implicat scope
  int sum = 0; 

  std::for_each(start, end, [&] (int v) 
  {
    std::cout << v << "\n";
    sum += v;
  });
  std::cout << " sum is : "<< sum << "\n";
}

// and now the fun...
template <class T>
void filterTo(T* start,
       T* end,
       std::function<bool (const T&)> filter,
       std::function<void (const T&)> action
       )
{ 
  std::for_each(start, end, [=] (T v) { if ( filter( v ) ) action( v ); });
}

void exampleStdFunctionLambdaNoClosure(int* start, int* end)
{
  //lambda compiling to a function pointer compatible with std:function
  std::function<bool (const int&)> filter
    = [](const int& v)->bool { return v%2; };
  std::function<void (const int&)> action
    = [](const int& v)->void { std::cout << "filt odd: " << v << "\n"; };

  filterTo(start, end,
    filter, action);
}

void exampleLambdaNoFunctionTemplateWeakness(int* start, int* end)
{
//  //This is the same as above and should work however the curent 
//  // compiler is  very poor at solving this.. so it dies
//  filterTo(start, end,
//    [](const int& v)->bool { return v%1; },
//    [](const int& v)->void { std::cout << v << "\n"; });
}

void exampleStdFunctionLambdaClosure(int* start, int* end)
{
  //evil: lambda closure is creates a class object not a function pointer
  // this can still wrap up in the std::function 
  int sum = 0;
  std::function<bool (const int&)> filter
    = [](const int& v)->bool { return v%2; };
  std::function<void (const int&)> action
    = [&sum](const int& v)->void {  sum += v; std::cout << "run sum:" << sum << "\n"; };

  //lambda compiling to a function pointer compatible with std:function
  filterTo(start, end,
    filter, action);

  std::cout << "Tot sum:" << sum << std::endl;
}

int main()
{
  int stuff[]={1,2,3,4,5,6,7,8,9};
  int* end = stuff + (sizeof(stuff)/sizeof(int)); //warning this is pointer arithmetic
 
  //suming examples
  examplePassByValue(stuff, end);
  examplePassByRef(stuff, end);
  examplePassByRefImplicatScoped(stuff, end);

  //find all odds..
  exampleStdFunctionLambdaNoClosure(stuff, end);
  exampleStdFunctionLambdaClosure(stuff, end);
  
  return 0;
}

Since this is a bit of a mess to run here is the make to build and run it (set BASE) to match your mingw 4.7 install path
BASE=/c/tools/mingw47
export PATH:=${BASE}/i686-w64-mingw32/lib:${BASE}/bin:${PATH}

all: run

lambda.exe: lambda.cpp
 g++ -std=c++11 lambda.cpp -o lambda.exe

run: lambda.exe
 lambda.exe



Refer: http://www.cprogramming.com/c++11/c++11-lambda-closures.html

Thursday, June 17, 2010

Reservoir Sampling - K samples from an infinite stream

Question;
You have a stream of infinite queries (ie Google searches). Describe how you would find K uniformly distributed samples and write code for it.

Reservoir Sampling is an algorithm designed to take K samples from a stream of samples of unknown size in linear time.

How it works.
The idea is simple. Firstly note that until the samples appear you don't know the total number of samples. To avoid this problem you fill up a reservoir of samples and then adjust it as each new sample appears.

First record samples until you have filled the reservoir(K samples). Once you have the first K samples the probability of each sample being in the final result was K/K or 100%.

Now when the K+i sample appears the probability of the new sample appearing in one _PARTICULAR_ output slot becomes 1/K+i. Since there are K potential output slots there is K/(K+i) chance of it being in the output and i/(K+i) chance that it wont be.

So choosing whether or not to add the new sample is easy we toss the dice. If we decide that the new sample is in the output then one of the existing items needs to be removed. Since all the items that where added to output and have equal chance of being present they also have equal chance of being removed. Hence we can simply choose to remove one of the existing samples with a 1/K probability.

This make sense logically but how do we prove it.
  • Assuming the prior stage worked correctly each element in the output should have K/(K+i-1) chance of being present.
  • There is i/(K+i) chance that prior output will be unchanged this round. Hence each element has Ki/(K+i)(K+i-1) chance of being output if the unchanging choice is made for this stage.
  • There is K/(K+i) chance that this something in the output will change
    • And for each element there is (K-1)/K chance it will not be replaced thus a total of (K-1)/K * K/(K+i) * K/(K+i-1) chance it will not be removed.
    • So conversely there is a 1/K * K/(K+i) * K/(K+i-1) chance that it will be replaced.

Hence the net chance of the existing elements making it to the next round is the sum of the chance that nothing changed with the chance that it was not the one that changed.
= Ki/(K+i)(K+i-1) + (K-1)K/(K+i-1)(K+i)
=> (Ki + (K-1)K)/(K+i-1)(K+i)
=> K(i+K-1)/(K+i-1)(K+i)
=> K/(K+i)
Q.E.D.

And of course the chance of the new element making to the output was K/(K+i). As you can see it has nicely balanced out for the possible output items for this stage.

BUT HERE IS THE CATCH.. Note that the mathematics/algorithm considered only the choice between output and non-output state of the samples. IT MADE NO STATEMENT AS TO THE OUTPUTS ORDERING. Hence the output samples need to be randomly shuffled in the final stages of the algorithm or there will be an issue with the ordering of the samples.

The code:
//compile using g++
#include <iostream>
using namespace std;

int reservoirSample(int sample, int* samples, int size, int count)
{
  if(count < size)
    samples[count] = sample;
  else
    if((rand()%count) < size)
      samples[rand()%size] = sample;
  
  return ++count;
}

#define SIZE 10
// #define STREAM_AVERAGE 300
#define STREAM_AVERAGE 10
int main()
{
  int count = 0;
  int samples[SIZE];
  int sample;
  int i = 0;

  srand(time(NULL));

  cout << "Sample Stream: " << endl;
  while(
        (count < SIZE) || 
        (rand()%STREAM_AVERAGE > 0)
        )
    {
      sample = rand()%1000;
      cout << sample << " ";

      count = reservoirSample(sample, samples, SIZE, count);
    }
  cout << endl;
  cout << "Total samples: " << count << endl;

  cout << "Output samples: " << endl;
  for(i = 0;i < SIZE;i++)
    cout << samples[i] << " ";
  cout << endl;
}

Sunday, June 13, 2010

SLT algorithm basics

STL algorithms are very powerful. They have 2 fundamental components.

The Algorithms:
  • These are the outer flow of control functions that direct the action of the operator classes. In general the internals are some form of for loop that passes over a series of STL compatible iterators and performs an action.
  • A list of them is here: http://www.cplusplus.com/reference/algorithm/

The Function/Operator classes
  • These are a class that implement the operator() overloader to provide the listed functionally.
  • Within this group of objects are an important subgroup of wrapper operators. These wrappers are designed to expand or covert the abilities of the other operators and to adapt existing function and class members for use with the STL algorithms.
  • A list of them is here: http://www.cplusplus.com/reference/std/functional/

I think I need to break this down and approach it in smaller chunks and apologies for the template heavy code. It cut down my typing.
#include <iostream>
#include <list>
#include <string>

#include <algorithm>
#include <functional>
#include <iterator>
using namespace std;

class RandN
{
  int n;
public:
  RandN(const int _n) { n = _n; }

  int operator() () const { return rand()%n; }
};

void readInAFIle(string filename)
{
  vector<int> values;
  ifstream file (filename);
  if (file)
    copy(istream_iterator<string> (file), istream_iterator<string>(), back_inserter (values));
}

template<class itor>
itor filterToThreshold(itor start, itor end, int threshold)
{
  //move to the end if... why call did they call it "remove"...
  return remove_if(start, end, bind2nd(greater<int>(), threshold));
}

template<class itor>
bool checkIfPresent(itor start, itor end, int value)
{
  return find(start, end, value) != end;
}

template<class itor>
void createRandSamples(itor start, int size, int limit)
{
  generate_n(start, size, RandN(limit));  
}

template<class itor>
void printOut(itor start, itor end)
{
  // *copy* the data into the output stream
  copy (start, end, ostream_iterator<int>(cout, " ")); 
  cout << endl;
}

template<class itor> 
void doDemo(itor start, itor end, string kind)
{
  cout << endl;
  cout << "Working on a " << kind << "..." << endl;
  printOut(start, end);
  cout << (checkIfPresent(start, end, 5) ? "there is a 5" : "No 5's") << endl;
  
  itor splitPoint = filterToThreshold(start, end, 5);
  int count = distance(start, splitPoint);
  cout << kind << " was filtered to a count of " << count << endl;
  printOut(start, splitPoint);
}

int main()
{
  const int SIZE = 20;
  list<int> valuesList;
  int valuesArray[SIZE];

  //create a rand list of stuff...
  srand(time(NULL));
  createRandSamples(valuesArray, SIZE, 10);
  createRandSamples(back_inserter(valuesList), SIZE, 10);

  doDemo(valuesArray, valuesArray + SIZE, "Array");
  doDemo(valuesList.begin(), valuesList.end(), "List");
}

Saturday, June 5, 2010

Sorting billions of numbers

There are several key facts to consider when sorting a massive number of numbers

Key factors to consider
  • * The memory usage profile and efficiency of the sort algorithm
    • - sort algorithms that can run back and forth over pages while sorting 1 number are out
      • -- that means bubble sort, insertion sort, selection sort, quicksort are out
    • - a sort algorithm that can naturally deal with chucked groups of data is ideal
      • -- the best candidate is merge sort. Using 4 chucks of memory 2 input and 2 output chunks
      • -- use the insertion sort when the but of items in the merge list are below 32 since is more efficient
  • * FileIO speed and rates
    • - fileIO is slow waiting for it is inefficent create a 2 threads that:
      • -- reads in chucks of data when the input buffer drops below a threshold.
      • -- writes out chucks of data when the output buffer contains data.
      • -- these threads may need to handle multiple buffer files as well as the final in/out file.
  • * the number of cores and virtual cores on the system
    • -- create inter/intra sorting worker threads equal to the number of virtual cores in the system so that the cpu is fully utilized.
  • * the physical paging sizing of memory and the allocation time and size of memory chunks
    • - dont repeatedly allocate and free memory instead keep a buffer pool with chucks of memory available
    • - allocate chucks of memory are below the size of a memory page.
    • - allocate chucks of memory only when the buffer pool is almost empty
    • - free chucks of memory if the buffer pool excess an upper limit.

Pesudo Code;
So the basic merge sort needs be to modified into a couple of ways to make it work
  • - First the FileIO thread will load buffers into the input buffers, these buffer have unsorted data inside chucks
  • - using multiple threads and if the instage buffer is running low, perform intra sort on each chuck, ie sort the data in the chucks and move it to the instage buffer pool marking them as a sequence of 1 chuck, ie mark the last chuck in the sequence with an end flag.
  • - using multiple threads, each thread will take wait for 2 input out of sequences from the instage buffers
    • -- remove the instage sequance from the instage buffer
    • -- perform inter merge between next inline 2 chucks of the input sequences until one of the sequences final chucks are full processed
    • -- when an input sequences final chuck is found move all the remain input chucks over to the output sequence (note the final chuck is left marked final)
    • -- once the first output chuck of the new sequance is complete the sequence can be moved in the instage buffer, The final chuck is not needed.
    • -- as each input chucks merge is completed move it to the empty buffer pool.
    • -- as each output chuck is completed append it to output sequence and get a new chunk from the empty buffer pool
    • -- if both final chucks of the input sequences are merged the output chuck needs to be marked as final

Thursday, June 3, 2010

In place Merge sort

In place merge sort is kinda like shoting your self in the foot before a race.. memory is cheap so the additional movement of data that this thing creates is meaningless. Also this version isnt very efficient because it implements the rotate of the data array via 3 reversal steps.
The alternate is done with a gray counter and a modulas of the counters remainder (ie the greatest common denominator between the total size left and right halves)... clearly Im not about to waste more time on it....

How it Works;
1) divide the list in half
2) perform mergesort on the left and right parts
3) merge the left and right parts
3a) to merge step through the array until a swap with the right is needed
3b) circular rotate the data around in the array by a shift right equal to the size of the right.
3c) set the point to the start of the old left list.

Properties;
Best, Average and Worst case speed is O(nlog(n)).
Memory is 1
In general it is unstable but stable versions do exist.

//compile with g++
#include <iostream>
#include <iomanip>
using namespace std;

void print(int* data, int size);

void reverse(int* data, int size)
{
  int temp;
  for(int i = 0; i < size/2; i++)
    {
      temp = data[i];
      data[i] = data[size-i-1];
      data[size-i-1] = temp;
    }
  //print(data, size);
}

void inPlaceMergeSort(int* data, int size)
{
  if(size < 2)
    return;

  int mid       = size/2;

  inPlaceMergeSort(data,       mid);
  inPlaceMergeSort(&data[mid], size - mid);

  int left      = 0;
  int out       = 0;
  while(left != mid)
    {
      if(data[mid] < data[left]) 
 {
   //cout << "orginal:" << left << " " << mid << " " << size << endl;
   //print(data, size);
   int size_right = size - mid;
   reverse(&data[left], mid  - left);
   reverse(&data[mid],  size - mid);
   reverse(&data[left], size - left);
   mid = size - ( mid - left);
 }
      else                         
 { left++; }
    }
}

//test
#define SIZE 20
#define SCRAMBLE(x, y) ((0xa57b & ~y) + ((0x3829 & x) << 1))

bool check(int data[], int size)
{
  for(int i = 1; i < size; i++)
    if(data[i] < data[i-1])
      {
        cout << "FAIL!" << endl;
        return false;
      }
  cout << "PASS" << endl;
  return true;
}

void print(int* data, int size)
{
  for(int i = 0; i < size; i++)
    cout << setw(5) << data[i] << " ";
  cout << endl;
}

bool test(int* data, int size)
{
  print(data, SIZE);
  inPlaceMergeSort(data, SIZE);
  print(data, SIZE);
  return check(data, SIZE);
}

int main()
{
  int data[SIZE];
  bool pass = true;

  //easy data
  data[0] = 1;
  for(int i = 0; i < SIZE; i++)
    data[i] = SIZE - i;
  pass &= test(data, SIZE);

  //semi repeated data
  data[0] = 1;
  for(int i = 1; i < SIZE; i++)
    data[i] = SCRAMBLE(i, data[i-1]);
  pass &= test(data, SIZE);

  //the sort killer
  for(int i = 0; i < SIZE; i++)
    data[i] = 5;
  pass &= test(data, SIZE);

  //and some randoms to catch anything i missed
  srand ( time(NULL) );

  for(int j = 0; j < 100; j++)
    {
      for(int i = 0; i < SIZE; i++)
        data[i] = (int)((float)(j+1)*((float)rand()/(float)RAND_MAX));
      pass &= test(data, SIZE);
    }

  if(pass)
    cout << "ALL PASSED" << endl;
  else
    cout << "FAILED" << endl;
}

Saturday, May 29, 2010

Shell Sort

Shell sort

Basically the idea of shell sort is to move numbers in a large step first, and slow reduce this step down to 1. The idea summarizes as simulated annealing. The whole reason for the additionally outer loop is to reduce the overall randomness in inital sort the data by quickly and roughly grouping low value items to the left and high value items to the right.

The inner core of it is insertion sort which will correct the sorting of the data irrespective of what your choice of outer steps are so long as the last step is 1.

How it Works;
1) choose a step by which to do your insertion sort
2) preform insertion sort as normal but using the step to determine which data is side by side.

The understandable version of the algorithm works as follows:
1) break the list of data into rows and columns
2) preform insertion sort on each column
3) repeat with more less columns until we have a single column.

In my code example I have have created an easy to understand implementation of shell sort it explicitly names out the columns and rows


Properties;
Worst case speed is O(n^2) or O(nlog^2n) if you use the special sequence of steps.
Memory is 1
It is Unstable (due the interleaving of the data several neigbour items are skipped)

//compile with g++
#include <iostream>
#include <iomanip>
using namespace std;

void print(int data[], int size);

void shellSort(int data[], int size)
{
  int tmp;
  int i; 
   // int step = 1; //comment the for and uncomment this to see that shell is just insertation sort!
   for(int step = size/2; step > 0; step = step/2)
      for(int j = step; j < size; j++)
        {
          tmp = data[j];
          i = j;
          while(
                (i >= step) &&
                (data[i-step] > tmp)
                )
            {
              data[i] = data[i-step];
              i -=step;
            }

          if(i != j)
            data[i] = tmp;
        }
}

void shellSortUnderstandable(int data[], int size)
{
  int tmp;
  int srow;
  for(int step = size/2; step > 0; step = step/2)
    {
      for(int column= 0; column < step; column++)
        {
          int rows = (int)((size-column)/step) + 1;
          for(int row = 1; row < rows-1; row++)
            {
              tmp = data[row*step + column];

              for(srow = row-1; srow >= 0; srow--)
                if(data[srow*step + column] > tmp)
                  data[(srow+1)*step + column] = data[(srow*step) + column];
                else
                  break;
              
              if(srow+1 != row)
                data[(srow+1)*step + column] = tmp;
            }
        }
    }
}

//test
#define SCRAMBLE(x, y) ((0xa57b & ~y) + ((0x3829 & x) << 1))

bool check(int data[], int size)
{
  for(int i = 1; i < size; i++)
    if(data[i] < data[i-1])
      {
        cout << "FAIL!" << endl;
        return false;
      }
  cout << "PASS" << endl;
  return true;
}

void print(int data[], int size)
{
  for(int i = 0; i < size; i++)
    cout << setw(5) << data[i] << " ";
  cout << endl;
}

bool test(int data[], int size)
{
  print(data, size);
  shellSort(data,size);
  print(data, size);
  return check(data, size);
}

#define SIZE 300
int main()
{
  int data[SIZE];
  bool pass = true;

  //easy data
  data[0] = 1;
  for(int i = 0; i < SIZE; i++)
    data[i] = SIZE - i;
  pass &= test(data, SIZE);

  //semi repeated data
  data[0] = 1;
  for(int i = 1; i < SIZE; i++)
    data[i] = SCRAMBLE(i, data[i-1]);
  pass &= test(data, SIZE);

  //the sort killer!
  for(int i = 0; i < SIZE; i++)
    data[i] = 5;
  pass &= test(data, SIZE);

  //and some randoms to catch anything i missed
  srand ( time(NULL) );

  for(int j = 1; j < 100; j++)
    {
      for(int i = 0; i < 3*j; i++)
        data[i] = (int)((float)(j+1)*((float)rand()/(float)RAND_MAX));
      pass &= test(data, 3*j);
    }

  if(pass)
    cout << "ALL PASSED" << endl;
  else
    cout << "FAILED" << endl;
}

Insertion Sort

How it Works;
0) Sorted data will be at the front of the array, starting 1 place after the start of the data,
1) take your current value out of the array, creating an empty point
2) set your insertion point to the tail of the sorted data.
3a) if the current item is lower then the insertion point item move the insertion points item up to the empty place in the array repeat with the next possible insertion point
3b) if the current item is equal to or greater than the insertion point place the current item back into the arrays empty place and repeat from 1 with the next unsorted data.

Properties;
Worst and average case speed is O(n^2)
Memory is 1
It is Stable (unless you implement it badly)

//compile with g++
#include <iostream>
#include <iomanip>
using namespace std;

void print(int data[], int size);

void insertionSort(int data[], int size)
{
  int tmp;
  int i;
  for(int j = 1; j < size; j++)
    {
      tmp = data[j];
      i = j;
      while(
            (i > 0) &&
            (data[i-1] > tmp)
            )
        {
          data[i] = data[i-1];
          i--;
        }

      if(i != j)
        data[i] = tmp;
    }
}

//test
#define SIZE 20
#define SCRAMBLE(x, y) ((0xa57b & ~y) + ((0x3829 & x) << 1))

bool check(int data[], int size)
{
  for(int i = 1; i < size; i++)
    if(data[i] < data[i-1])
      {
        cout << "FAIL!" << endl;
        return false;
      }
  cout << "PASS" << endl;
  return true;
}

void print(int data[], int size)
{
  for(int i = 0; i < size; i++)
    cout << setw(5) << data[i] << " ";
  cout << endl;
}

bool test(int data[], int size)
{
  print(data, SIZE);
  insertionSort(data,SIZE);
  print(data, SIZE);
  return check(data, SIZE);
}

int main()
{
  int data[SIZE];
  bool pass = true;

  //easy data
  data[0] = 1;
  for(int i = 0; i < SIZE; i++)
    data[i] = SIZE - i;
  pass &= test(data, SIZE);

  //semi repeated data
  data[0] = 1;
  for(int i = 1; i < SIZE; i++)
    data[i] = SCRAMBLE(i, data[i-1]);
  pass &= test(data, SIZE);

  //the sort killer!
  for(int i = 0; i < SIZE; i++)
    data[i] = 5;
  pass &= test(data, SIZE);

  //and some randoms to catch anything i missed
  srand ( time(NULL) );

  for(int j = 0; j < 100; j++)
    {
      for(int i = 0; i < SIZE; i++)
        data[i] = (int)((float)(j+1)*((float)rand()/(float)RAND_MAX));
      pass &= test(data, SIZE);
    }

  if(pass)
    cout << "ALL PASSED" << endl;
  else
    cout << "FAILED" << endl;
}

Selection Sort

The algorithm works as follows:
How it Works;
1) Find the minimum value in the list
2) Swap the minimum item with first item
3) Repeat for all remain positions

Properties;
Worst and average case speed is O(n^2)
Memory is 1
It is Stable (unless you implement it badly)

//compile with g++
#include <iostream>
#include <iomanip>
using namespace std;

void print(int data[], int size);

void selectionSort(int data[], int size)
{
  int min;
  int tmp;
  for(int j = 0; j < size; j++)
    {
      min = j;
      for(int i = j;i < size;i++)
        if(data[i] < data[min])
          min = i;

      if(min != j)
        {
          tmp = data[min];
          data[min] = data[j];
          data[j] = tmp;
        }
    }
}

//test
#define SIZE 20
#define SCRAMBLE(x, y) ((0xa57b & ~y) + ((0x3829 & x) << 1))

bool check(int data[], int size)
{
  for(int i = 1; i < size; i++)
    if(data[i] < data[i-1])
      {
        cout << "FAIL!" << endl;
        return false;
      }
  cout << "PASS" << endl;
  return true;
}

void print(int data[], int size)
{
  for(int i = 0; i < size; i++)
    cout << setw(5) << data[i] << " ";
  cout << endl;
}

bool test(int data[], int size)
{
  print(data, SIZE);
  selectionSort(data,SIZE);
  print(data, SIZE);
  return check(data, SIZE);
}

int main()
{
  int data[SIZE];
  bool pass = true;

  //easy data
  data[0] = 1;
  for(int i = 0; i < SIZE; i++)
    data[i] = SIZE - i;
  pass &= test(data, SIZE);

  //semi repeated data
  data[0] = 1;
  for(int i = 1; i < SIZE; i++)
    data[i] = SCRAMBLE(i, data[i-1]);
  pass &= test(data, SIZE);

  //the sort killer!
  for(int i = 0; i < SIZE; i++)
    data[i] = 5;
  pass &= test(data, SIZE);

  //and some randoms to catch anything i missed
  srand ( time(NULL) );

  for(int j = 0; j < 100; j++)
    {
      for(int i = 0; i < SIZE; i++)
        data[i] = (int)((float)(j+1)*((float)rand()/(float)RAND_MAX));
      pass &= test(data, SIZE);
    }

  if(pass)
    cout << "ALL PASSED" << endl;
  else
    cout << "FAILED" << endl;
}

Bubble Sort

Bubble sort

How it works;
Basically bubble sort works by stepping through the data a swapping the current and next data items if they are out of order.

Properties;
Best case and worst case speed are both O(n^2)
Memory is 1 (the temp swap location)


//compile with g++
#include <iostream>
#include <iomanip>
using namespace std;

void print(int data[], int size);

void bubblesort(int data[], int size)
{
  bool change = true;
  int tmp;
  while(change)
    {
      change = false;
      for(int i = 0;i < size-1;i++)
        if(data[i] > data[i+1])
          {
            tmp = data[i];
            data[i] = data[i+1];
            data[i+1] = tmp;
            change = true;
          }
    }
}

//test
#define SIZE 20
#define SCRAMBLE(x, y) ((0xa57b & ~y) + ((0x3829 & x) << 1))

bool check(int data[], int size)
{
  for(int i = 1; i < size; i++)
    if(data[i] < data[i-1])
      {
        cout << "FAIL!" << endl;
        return false;
      }
  cout << "PASS" << endl;
  return true;
}

void print(int data[], int size)
{
  for(int i = 0; i < size; i++)
    cout << setw(5) << data[i] << " ";
  cout << endl;
}

bool test(int data[], int size)
{
  print(data, SIZE);
  bubblesort(data,SIZE);
  print(data, SIZE);
  return check(data, SIZE);
}

int main()
{
  int data[SIZE];
  bool pass = true;

  //easy data
  data[0] = 1;
  for(int i = 0; i < SIZE; i++)
    data[i] = SIZE - i;
  pass &= test(data, SIZE);

  //semi repeated data
  data[0] = 1;
  for(int i = 0; i < SIZE; i++)
    data[i] = SCRAMBLE(i, data[i-1]);
  pass &= test(data, SIZE);

  //the sort killer!
  for(int i = 0; i < SIZE; i++)
    data[i] = 5;
  pass &= test(data, SIZE);

  //and some randoms to catch anything i missed
  srand ( time(NULL) );

  for(int j = 0; j < 100; j++)
    {
      for(int i = 0; i < SIZE; i++)
        data[i] = (int)((float)(j+1)*((float)rand()/(float)RAND_MAX));
      pass &= test(data, SIZE);
    }

  if(pass)
    cout << "ALL PASSED" << endl;
  else
    cout << "FAILED" << endl;
}

Friday, May 28, 2010

Quick sort

UPDATE: I noticed a lot o traffic coming to this entry.. since it seems popular ill clean up and simplify the code some time later this week(currently its 2010/06/17). Then repost it in a new entry and note the new link here..


Quick sort..

How it works.
1) First choose a pivot value.
2) Then divide the list into 2 sections, do the division by finding the left most item the exceeds the pivot and swapping it with the right most item that is less than the pivot.
3) Then recurse on the 2 halves.

There are several hidden fail points
1) if a bad pivot value is chosen then the list wont divide into 2
2) if the data becomes to similar, or even completely identical then the chances of a bad pivot increase.

As a result of the fail point care needs to be taken to correctly find the right and left swap items and where that the the list is be split.

Speed:
Quick sort is O(nlog(n)) on average. and O(n^2) in the worst case.

//compile with g++
#include <iostream>
#include <iomanip>
using namespace std;

void print(int data[], int size);

void quicksort(int data[], int sectionLow, int sectionHigh)
{
  if(sectionLow >= sectionHigh-1)
    return;

  int low  = sectionLow;
  int high = sectionHigh-1;
  int mid  = (low+high)/2;

  int pivotValue = data[mid];

  while(low < high)
    {
      while(
            (low < high) &&
            (data[low]  < pivotValue) &&
            (data[high] > pivotValue)
            )
        {
          //O(N^2) avoidance.. move the pointers inorder or 1 will start to dominate 
          // when the data starts to get too similar
          low++;
          high--;
        }
      
      //one or both of these wont run.. the other will find its end...
      while((low < high) && (data[low]  < pivotValue)) low++;
      while((low < high) && (data[high] > pivotValue)) high--;

      cout << low << "<>" << high << " ";

      //do the swap
      if(low < high)
        {
          int temp = data[low];
          data[low]  = data[high];
          data[high] = temp;

          //step away from it otherwise it will get stuck in case that these are both pivotValues
          low++;
          high--;
        }
    }

  cout << endl;

  cout << low << " ";
  //quick sort boundary conditions are problem matic
  //make certian of where the end of the data was
  if(data[low] < pivotValue)   mid = low+1;
  else                         mid = low;

  cout << mid << " ";

  //make certain that this isnt the start or end edge or we will end in an infinite loop
  if(mid == sectionHigh) mid--;
  if(mid == sectionLow)  mid++;

  cout << mid << endl;

  cout << "stage result: " << sectionLow
       << "<->" << mid
       << "<->" << sectionHigh << " pivot:" << pivotValue << endl;
  print(&(data[sectionLow]), sectionHigh-sectionLow);

  quicksort(data, sectionLow, mid);
  quicksort(data, mid, sectionHigh);
}

//test
#define SIZE 20
#define SCRAMBLE(x, y) ((0xa57b & ~y) + ((0x3829 & x) << 1))

bool check(int data[], int size)
{
  for(int i = 1; i < size; i++)
    if(data[i] < data[i-1])
      {
        cout << "FAIL!" << endl;
        return false;
      }
  cout << "PASS" << endl;
  return true;
}

void print(int data[], int size)
{
  for(int i = 0; i < size; i++)
    cout << setw(5) << data[i] << " ";
  cout << endl;
}

bool test(int data[], int size)
{
  print(data, SIZE);
  quicksort(data,0, SIZE);
  print(data, SIZE);
  return check(data, SIZE);
}

int main()
{
  int data[SIZE];
  bool pass = true;

  //easy data
  data[0] = 1;
  for(int i = 0; i < SIZE; i++)
    data[i] = SIZE - i;
  pass &= test(data, SIZE);

  //semi repeated data
  data[0] = 1;
  for(int i = 0; i < SIZE; i++)
    data[i] = SCRAMBLE(i, data[i-1]);
  pass &= test(data, SIZE);

  //the sort killer!
  for(int i = 0; i < SIZE; i++)
    data[i] = 5;
  pass &= test(data, SIZE);

  //and some randoms to catch anything i missed
  srand ( time(NULL) );

  for(int j = 0; j < 100; j++)
    {
      for(int i = 0; i < SIZE; i++)
        data[i] = (int)((float)(j+1)*((float)rand()/(float)RAND_MAX));
      pass &= test(data, SIZE);
    }
  
  if(pass)
    cout << "ALL PASSED" << endl;
  else
    cout << "FAILED" << endl;
}