- For the following dataset (in C4.5 format):
data.names:
no, yes. | classes
a: true, false
b: true, false
c: continuous
d: true, false
e: continuous
data.data:
false,true,15,false,20,yes
true,false,1,true,5,no
false,false,10,true,10,no
false,false,8,false,15,yes
true,true,13,true,16,yes
false,true,9,false,8,no
false,false,1,true,5,no
true,false,12,false,13,no
true,true,15,false,6,yes
true,true,15,true,10,yes
false,true,13,true,7,no
false,true,3,false,5,yes
- What decision tree would be learned using ID3? [15 points]
- What rules would be learned using the sequential covering algorithm
assuming we start from the most general rules and we use the
adaptation discussed in the homework to allow the rules to incorporate
continuous features? [15 points]
- For each of the following data points, what class would be predicted
using the 3-Nearest Neighbor algorithm using a Manhattan distance
measure where the two continuous features are scaled to be values
between 0 and 1? [15 points]
- true,true,10,true,15 class?
- false,true,3,true,9 class?
- false,false,15,false,3 class?
- What predictions would be made using the Naive Bayes learning method
for the data points shown in the previous section? [15 points]
- Using the agglomerative single link clustering method, determine the
clusters that would be produced from the data points in data.data
assuming we ignore the class (the no or yes value), that our distance is
measured as nearest neighbor question above, and where we have the following threshold
values (two points are considered to be connected if their distance is
*less* than these thresholds):
- Threshold=1.2 [5 points]
- Threshold=2.2 [5 points]
- Threshold=3.2 [5 points]
- Describe the inductive bias for each of the following methods:
- Version Spaces [5 points]
- Decision Trees produced using ID3 [5 points]
- Neural Networks using Backpropagation [5 points]
- One mechanism proposed for learning not only the weights of a neural
network but also the structure (i.e., number of hidden units, connection
patterns of units, etc.) of the neural network involves genetic programming.
A set of random neural networks are generated and then trained using
backpropagation on a set of training data. Then the "fittest" of the
networks reproduce (as if the networks were genetic programs) and the resulting new networks are trained again.
Fill in the details of this algorithm. Explain how the term fittest
might be determined. Also, indicate how networks can reproduce to
generate new networks that may have different network structures. Can
you see any possible problems with this algorithm? [20 points]
- How does the Minimum Description Length principle relate to the notion
of Occam's razor? How could one incorporate MDL into the process of
selecting a particular hypothesis (i.e., preferring a decision tree,
a neural network, etc.)? [15 points]
- Assume a problem has 9 states numbered 0 to 8. Further assume that in
each state one can take one of two actions, L or R. In states 1 to 7,
taking the action L in state I puts you in state I-1 (taking action L in
state 5 puts you in state 4). In state 0 taking action L leaves you in
state 0. In state 8, taking action L leaves you in state 8. For each
of the states 0 to 7, taking action R in state I puts you in state
(I + 3) mod 9 (e.g., taking R in state 2 puts you in state 5, taking R
in state 7 puts you in state 1, etc.). Taking action R in state 8 leaves
you in state 8. Any action that takes you from a different state and
leaves you in state 8 has a reward of 100. All other actions have a
reward of 0. Assume a discount factor of 0.8.
Draw the state diagram for this problem.
Show the V*(s) values for these states.
Show the optimal Q(s,a) values for this problem.
Show an optimal policy for this problem. [20 points]
- For the following EBL domain theory:
A(?x,?y), B(?y,?x,?z) -> C(?x,?y,?z)
D(?x,?y), E(?x,?y) -> B(?x,1,?y)
F(?x,?x), F(?y,?y) -> D(?x,?y)
Assume the following facts are asserted:
A(1,2) A(3,1) E(2,3) F(3,2) F(2,2)
A(2,2) A(3,2) E(1,2) F(3,3) F(2,3)
Explain with a proof tree that C(1,2,3) is true. [3 points]
Assuming the predicates A, E, and F are operational, what rule would
EGGS learn? [12 points]
Assuming that predicate B is also operational, what rule would be learned?
[5 points]
- Explain the fundamental difference between the Bagging and Ada-Boosting
ensemble learning methods? How do these notions relate to the concept of
generating a good ensemble? What are the advantages and disadvantages of
each method? [15 points]
- What neural network would be generated by KBANN from the following rules
assuming the output predicate is J and the input predicates are A, B, C
and D? For each unit generated you should connect it to any input unit
that it is not already directly connected to with a small weight link.
[20 points]
A, B -> E
A, C, D -> E
E, A -> F
not B, D -> F
not E, not F -> G
A, F -> G
E, not F -> H
E, G, H -> J
- Given a neural network with 3 input units (A, B, C), two hidden units
(D, E), one output unit (F) and one unit that always has an activation
value of 1 (ONE) and the following weight connections:
ONE->D: 0.5
A->D: -0.5
B->D: 0.0
C->D: 1.0
ONE->E: -0.5
A->E: 0.0
B->E: 0.0
C->E: 0.5
ONE->F: 0.0
D->F: -0.5
E->F: 0.5
What would be the weights after each of the following points is presented
(in the sequence shown) assuming a learning rate of 0.25 and a momentum
term of 0.9. Assume the hidden and output units use a sigmoidal activation
function and that the weights are changed using backpropagation. [15 points]
A B C F
Point 1: 1 0 1 1
Point 2: 0 1 1 0
Point 3: 1 1 1 1
- Show the initial G and S sets and the G and S sets after each of the
data points shown below is presented to the Version Space (Candidate
Elimination) algorithm: [15 points]
1,2,1,3,1,+
2,1,2,2,2,-
1,2,1,1,2,-
1,1,2,3,1,-
1,2,2,3,2,+
- What do you think is the most interesting algorithm we discussed in class
and why? [10 points]
- What do you think is the most important issue facing machine learning and
why? [10 points]