iSenseStream

Usage Notes for iSenseStream
Christopher G. Prince (chris@cprince.com)
26 August 2005
Revised: 9/20/05; 10/8/05; 10/11/05; 10/13/05

First you need to install IKAROS on your Mac OS X system. If you haven't already, download the latest version from http://www.cprince.com/PubRes/IKAROS/iSenseStream
I've compiled the software on my Mac OS 10.4.2 system, and haven't tested it with any other Mac OS X version. To install IKAROS, unarchive the .tgz file. You should be able to do this by double clicking on the .tgz file and waiting for it to create the IKAROS folder. Place the IKAROS folder in a location (i.e., folder) where you want it. I keep mine on my Desktop. Your IKAROS folder will have a version number after its name. E.g., your IKAROS folder may be called "IKAROS.0.8.0"
Next, you need to set up the Path Preference for the IKAROS Viewer. To do this, launch the IKAROS Viewer program within the IKAROS folder. Then under the IKAROS Viewer menu, go to the Preferences... submenu. Delete any character contents in the Path field of the Preferences window. Now, with the IKAROS folder open in the Mac Finder and the IKAROS Viewer Preferences also in view, drag the file named "IKAROS" into the Path field. This will place the path name into the Path field of the IKAROS Viewer Preferences window. You should now see something like:

/Users/chris/Desktop/IKAROS.0.8.0/IKAROS

in the Path field of the IKAROS Viewer Preferences, except of course with the specifics from your computer. It should look like:
Now, click on the "OK" button of the IKAROS Viewer Preferences window.

iSenseStream_Original

Now you are ready to start iSenseStream. To do this, within the already running IKAROS Viewer application, use

File -> Open ...

and open the following IKAROS XML file--

IKAROS.0.8.0 -> UserModules -> iSenseStream -> Examples -> iSenseStream_Original.xml
You should now see the open iSenseStream_Original.xml XML file. It should look like this:
See the Filenames section below for file names you may want to change before starting the iSenseStream program.
To start running iSenseStream_Original program once you have edited the iSenseStream_original file to put in your own file names, click on the fourth button from the left in the upper right region of the open XML file. The button looks like this:
When the program runs, after running for a period of time, it should give graphical output that looks like:
To obtain synchrony estimates and centroid values for an entire digital video file, start iSenseStream running on your video file, and when it finishes (i.e., there is no more movement on the screen), stop the program running (e.g., Quit the application), and the output files should now contain your data.

iSenseStream_OriginalPlusQT

If you also want to create Quicktime movies of the mixelgrams and the associated audio from the input, instead open the following IKAROS XML file--

IKAROS.0.8.0 -> UserModules -> iSenseStream -> Examples -> iSenseStream_OriginalPlusQT.xml

Filenames

The iSenseStream XML files provide, in part, the description of a series of modules. Particular input and output file names are given in particular module descriptions. In order to use different input or output files, you need to edit the text of the iSenseStream XML file. This can be done in the IKAROS Viewer application, before starting to run iSenseStream. You can also edit the text of the XML file with a normal text editor (be careful about using word processors though-- you'll need to make sure to save the XML file as "text only"). The file names and the module where the file names are given are listed below.

I recommend you use files with extensions (e.g., .dv, .mov). So far, files without extensions have not worked for me.

Usage of File	Module
Dital video file to be used as input. This file should be placed in the Examples folder where the iSenseStream_Original.xml file is located. Quicktime (.dv and .mov) and .avi files should work as input files. So far, the MPEG files I've tried don't work.	InputQTMovie
Dital video file to be used as input (should be exactly the same name as used with the InputQTMovie module)	InputQTAudio
An output file which will contain the synchrony estimates from the edge detection method.	OutputFile, edgeDetectOutput
An output file which will contain the (x, y) coordinates of the centroid of the mixelgrams. (0, 0) is the upper left of each image and mixelgram.	OutputFile, centroidOutput
Output files which will contain the Quicktime movies visual processed results and the audio from the original video file. Only mono-audio output is presently written to the output Quicktime movies. 10/13/05: There are now two output Quicktime movies created by iSenseStream_OriginalPlusQT. One of these is the mixelgram/audio output. The other is the sobel edge detected/audio output.	OutputQTAudioVisual

Parameters

Similar to file names, the modules in iSenseStream need parameters of various kinds. In order to use different parameter values for particular modules, you also need to edit the text of the iSenseStream XML file. This can also be done in the IKAROS Viewer application, before starting to run iSenseStream, or using a text editor. The parameter names and the module where the parameter are given are listed below.

Module

Parameter

Usage of Paramter

OutputQTAudioVisual

visual_time_scale and visual_sample_duration

These two parameters should be set to reflect the visual frame rate of the input video file.

If the visual frame rate of the input video file is 27.97 frames per second, use--

visual_time_scale = "2997"
visual_sample_duration = "100"

If the visual frame rate of the input video file is 30 frames per second, use--

visual_time_scale = "600"
visual_sample_duration = "20"

The visual_time_scale parameter indicates the number of time units that pass per second in the video, and the visual_sample_duration parameter indicates the number of time units that pass per visual frame.

To determine the visual frame rate of the input file (on Mac), you can use the Quicktime Player. First, launch the video file in Quicktime player. Next, use Window -> Show Movie Info. FPS gives the visual frames per second.

OutputQTAudioVisual

visual_sample_duration

This parameter should be set to the audio sampling rate in samples per second. For example, if the audio sampling rate is 44.1 kHz, this parameter should be "44100".

10/13/05: The parameters for the OutputQTAudioVisual module should be the same for both module instances. That is, the parameters should be the same for outputting the mixelgram/audio Quicktime movie and outputting the sobel edge detected results/audio.

Running iSenseStream From the Command Line

It may be convenient for you to run iSenseStream from the command line. First, you should edit the iSenseStream XML file so it contains the file names and parameters you want to use. Your video files, just like when running iSenseStream from the IKAROS Viewer, should still be contained in the Examples folder of iSenseStream. Next, follow the procedure below.

1) Open a terminal window by launching the Terminal application. On my system, the Terminal application is in the Applications -> Utilities folder.

2) Make sure you are running a "bash" shell in the terminal window. To ensure this, type "bash" <return>

3) In the terminal window, type:

export PATH=$PATH:/Users/<UserName>/Desktop/IKAROS.0.8.0 <return>

Note: You should replace "<UserName>" with your user name. This assumes that IKAROS is placed on your Desktop.

To figure out what your user name is, type (there is a space before the "~" character in the following).

echo ~ <return>

4) Now, you need to change your current working directory in the Terminal window into the Examples folder of iSenseStream. To do this, type:

cd ~/Desktop/IKAROS.0.8.0/UserModules/iSenseStream/Examples <return>

Note: This needs to be typed in the Terminal window.

5) Now, you are ready to start iSenseStream running on your XML file. To do this, type:

IKAROS iSenseStream_OriginalPlusQT.xml <return>

Note: This assumes you are working from the XML file-- iSenseStream_OriginalPlusQT.xml

Here is an image of my terminal window just before I started running iSenseStream--

KNOWN BUGS

1) The centroid cross-bars on the centroid.IMAGE output often don't appear. I'm working on this.

2) When running in the IKAROS Viewer, the OutputQTAudioVisual module will not properly close its file if you let the IKAROS Viewer run to completion on the input file. If you run from the command-line, this module will properly close its output file in this case.

BUG FIXES

1) On 10/5/05, I fixed the problem that was in the InputQTMovie module-- it hadn't been computing grayscale correctly. Previously it was just adding the R, G, and B components and dividing by three.