CS 5541 Study Questions and Homework 5

Homework (15 points) due Wednesday, December 10, 2003.

For the final exam, also see the first four sets of study questions (Review 1, Review 2, Review 3, Review 4). The final exam will be cumulative.

Homework assignment

For this Homework, turn in your answers for the following study questions:

Q1 [2 pts]
Q6 [1 pts]
Q12 [2 pts]
Q14 [2 pts]
Q15 [2 pts]
Q18 [2 pts]
Q19 [2 pts]
Q20 [2 pts]

Study questions

  1. For the following data set, give a single layer neural net architecture suited to attempting approximation of the data set:
    Input Output
    0 0 0 0 0
    1 0 0 0 1
    0 1 1 0 0
    0 0 0 1 1
  2. In the last question, what function do you think is given in the table?
  3. Use a spreadsheet like that used in class to iterate the delta rule through 20 epochs (cycles through the data) to attempt to find weights that solve this problem for your ANN in question #1. Use the sigmoidal activation function.
  4. Were you able to find weights that approximate the function? Explain your answer.
  5. Give four additional inputs (binary valued vectors of length 4) for the network, and run them with the weights you found in #3. Give the outputs of your network. Explain your results.
  6. What mathematical idea does the delta rule rely on? (You don't have to give the derivation, just state the idea).
  7. Show that without an activation function, a neural network always outputs 0 when the inputs are 0.
  8. What problem does backpropagation solve that is not solved by the delta rule?
  9. Is the delta rule a supervised training method? Explain.
  10. Is backpropagation a supervised training method? Explain.
  11. What would an unsupervised method that implements association learning in neural networks look like?
  12. Is a self-organizing map (a) supervised, or (b) unsupervised? Explain.
  13. Draw a diagram in terms of neural units, and connection weights for the general self-organizing map (SOM) architecture.
  14. A given SOM has 3 inputs and two outputs. Draw and label the architecture of the SOM.
  15. How many categories can the SOM in question #14 represent? Explain your answer.
  16. How do we test for generalization in a SOM?
  17. In the algorithm for training (establishing) the weights in a SOM, before the weight update rule can be applied, what is done?
  18. Give the weight update rule for the SOM algorithm. Make sure to explain your notation.
  19. For a SOM with three inputs and two outputs, one input data sample is: (1, 1, 0). Assuming the weight vectors are (.5 .2 .3) for unit 1 in the output, and (.8 .9 0) for unit 2 in the output, compute the weight update for that data sample.
  20. For a SOM with four inputs and two outputs, and weight vector (0 0 .5 1.0) for output unit 1, and (1.0 .5 0 0) for output unit 2, what category does the input vector: (0 1 0 0) fall into?
  21. On what basis do SOMs achieve categorization?