I have set up a script to automatically test your assignment 2 submissions. It only really tests the tmi and ll3 portions of your assignment of course, since you all used different measures for user2 and user3. You should get an automatically generated message in the near future that shows the results of my script running your program on my test cases. The output will show the test case, your answer, and the proposed "correct" answer. Hopefully it makes sense, and you can see how your programs did. If your program did not run according to specifications then you will see whatever error messages NSP might have generated. A few words on the test cases: For the 2x2 case I chose the following (pictured as a 2x2 table, not in NSP format) W2 -W2 W1 2 1 | 3 -W1 2 95 |97 --------- 4 96 100 (TMI = .0742) (LL = 10.2850) The above is just a general purpose table that looks alot like "typical" bigram data. What I mean by that is that it's rather messy, W1 and W2 occur together, but they also occur separately. The number of times they occur together is dwarfed by the same size. In real data of course the sample size would be much bigger, and the skewness of the contingency table would be even more so. 1 0 | 1 0 99 |99 --------- 1 99 100 (TMI = .0808) (LL = 11.2003) 10 0 |10 0 90 |90 --------- 10 90 100 (TMI = .4690) (LL = 65.0166) These two cases are rather extreme, and generally speaking I think we'd agree that the second case (where the bigram occurs 10 times) would be a "stronger" bigram than the first case (where it only occurs 1 time). You'll note that the TMI and LL scores reflect this. I will check and see how user2 (your proposed measure) differentiates between these two. Certainly they should be scored differently, and I think it would be hard to argue that the first case (of the bigram occurring 1 time) would deserve a higher score than the second. I am always willing to hear arguments on these points of course. Here are the test cases for the 2x2x2 case. (I hope the tables make a bit of sense): W3 W3 -W3 -W3 W2 -W2 W2 -W2 W1 1 0 0 0 -W1 0 0 0 99 (LL3 = 22.4006) W3 W3 -W3 -W3 W2 -W2 W2 -W2 W1 10 0 0 0 -W1 0 0 0 90 (LL3 = 130.0332) These are similar to the cases for 2x2, where the first has a bit less evidence in favor of dependence, and note that the ll3 scores reflect this point. Again, I will be looking at how your user3 handles this same situation. Ted Pedersen October 22, 2002