Computer Science 8751
Machine Learning and Knowledge Discovery in Databases

Extra Credit Programming Assignment
Genetic Algorithms (35 extra credit points)
Due Friday, December 12, 2003 * NO LATE PROGRAMS *


In this lab you will be implementing a genetic algorithm for producing a set of rules to predict a concept.

Hypotheses: Representing the Bit String

The format of your hypothesis as a bit string should be based on the GABIL algorithm with one generalization, you should allow your rules to reference continuous variables as well.

To represent clauses corresponding to discrete features use the standard GABIL representation, use one bit for each value of that feature, where a 1 indicates that value is allowable and a 0 indicates the value is not allowed. For example, for feature A with three possible values (a1, a2, a3), three bits would be used. If the three bits were set to 101 in a hypothesis that would indicate a condition of (A = a1 or A = a3).

For continuously valued attributes, you should use a representation with 2 bits for an operation and 32 bits to represent a floating point value. If both of the operation bits are set to 1, then any value of the continuously valued feature is allowable. If the 2 operations bits are 10 then values that are less than or equal to the floating point value are allowed, and if the 2 operation bits are 01 then values that are greater than or equal to the floating point value are allowed. For example, if the 34 bits for continuous feature B were 01 followed by the 32 bits to represent 1.5, this would indicate a test of B >= 1.5.

Genetic Operators

You should plan on implementing one cross-over operator, plus the point mutation, AddAlternative, and DropCondition operators. For cross-over, you should use a variation on the GABIL cross-over operator as follows:

  1. Pick two parents stochastically based on the fitness of each parent divided by the sum of fitness across all members of the population.
  2. Pick two cross-over points as follows from the first parent: Then repeat this process to pick the second cross-over point in the first parent (note, do not allow the system to pick the same cross-over point). Order the two cross-over points so that the one occuring earlier in the string is the first.
  3. Pick two cross-over points for the second parent. This should be based on the cross-over points from the first parent. Determine the number of possible combinations of cross-over points in the second parent that match in the attribute and bit, but not in rule (and are ordered correctly) and pick randomly among these.


The fitness of individual points should be the correctness (on the training set) of each hypothesis.

General Genetic Algorithm

Your code should take a dataset and several parameters. The following parameters should be set by the user when calling the program:

A sample run of your code on a data set should produce show the top 11 hypotheses (the rules for the 11 highest fitness members of the population) after evolution has completed.

You should set up your code so that it is able to save the top 11 hypotheses in a file. You should then be able to use the resulting hypotheses to predict the class for a separate set of data (a test file) and produce a confusion matrix for that data. Your output should look something like this:

Confusion matrix for hypothesis 1 from file XXX on data in file XXX:

               Actual Class
                  0   1 
Predicted   0 |   5   1 
Class       1 |   0  10 

  Accuracy =  93.75%

Confusion matrix for hypothesis 2 from file XXX on data in file XXX:

What to Turn In

Print out a copy of all of your code files. You should hand in printouts demonstrating how your program works by running your program on several data sets, including your own. For the rules produced from your data set try to analyze the resulting rules and determine how accurate you think the rules are at capturing the concept expressed by your data.

You should also write up a short report (at least one page, no more than three) discussing your design decisions in implementing the genetic algorithm and how your version of the code works.

You must also submit your code electronically. To do this go to the link and follow the directions for uploading a file (you can do this multiple times, though it would be helpful if you would tar your files and upload one file archive).

To make your code easier to check and grade please use the following procedure for collecting the code before uploading it: