Computer Science 8751
Advanced Machine Learning
Homework Assignment 3 (20 points)
Due March 12, 2008

1. Given a neural network with three input units (i1, i2, i3), two hidden units (h1 and h2) and one output unit (o1) and a threshold unit (a unit whose activitity is always 1) and weights connecting each input unit plus the threshold unit to the hidden units and weights connecting each hidden unit plus the threshold unit to the output unit where all of the weights are initially set to 0.01. Assuming a learning rate of 0.25, that each hidden and output unit is sigmoidal, and that backpropagation is being used (but no momentum term is in use), what will the weights be after each of these examples is presented, where the weights at the start are the ones after the previous example:

         inputs     target output
         i1 i2 i3   o1
         1  0  1    0
         1  1  1    1
         0  1  0    0
         1  1  0    1
       

2. Exercise 4.9 on page 125.
3. The curse of dimensionality is a critical problem for the nearest neighbor method. One way of dealing with this curse is to focus more on key features and less on others. How might you create a genetic algorithm that would address this issue? What would be your hypothesis representation? What would be your fitness function?