Output compared to Nyström's Matlab Algorithm --- needs a dataset with 'x' 'y' coordinates to run #8

New issue

Closed

opened 2018-08-27 18:37:14 +00:00 by AsimHDar · 40 comments

AsimHDar commented 2018-08-27 18:37:14 +00:00 (Migrated from github.com)

Copy link

So for some reason all the sources of the input data from the author seem to be corrupt --- the input is basically two columns with the x and y coordinates and you can feed in specific data(screen size, viewing distance, etc). Tried it with a sample from one of our subjects by removing everything other than the x and y coordinates and the algorithm worked fine. (Note: this was just a test, and so the screen size, etc aren't according to the actual ones used in our study)

I've attached the outputs (and input: randomsample.csv) currently trying to find out what these event labels stand for (They are labelled 0-4) @AdinaWagner Any ideas?

DetectionResults.zip

So for some reason all the sources of the input data from the author seem to be **corrupt** --- the input is basically two columns with the x and y coordinates and you can feed in specific data(screen size, viewing distance, etc). Tried it with a sample from one of our subjects by removing everything other than the x and y coordinates and the algorithm worked fine. (Note: this was just a test, and so the screen size, etc aren't according to the actual ones used in our study) I've attached the outputs (and input: randomsample.csv) currently trying to find out what these event labels stand for (They are labelled 0-4) @AdinaWagner Any ideas? [DetectionResults.zip](https://github.com/psychoinformatics-de/studyforrest-data-eyemovementlabels/files/2325154/DetectionResults.zip)

adswa commented 2018-08-28 06:03:19 +00:00 (Migrated from github.com)

Copy link

The 1, 2, 3, & 4 represent classifications of eyemovements, apparently. The file "write2csv.m" contains the code

D_temp(logical(ETparams.fixationIdx(1, i).Idx), 4) = 1;
D_temp(logical(ETparams.saccadeIdx(1,i).Idx), 4) = 2;
D_temp(logical(ETparams.glissadeIdx(1, i).Idx), 4) = 3;
D_temp(logical(ETparams.nanIdx(1, i).Idx), 4) = 4;

I haven't found anything relating to the 0, but I suspect those could simply be left "unlabeled". write2csv starts by initializing 5 columns of zeros

D_temp = zeros(nRows, 5);

and then seems to overwrite only these rows where an event was detected. If no event was detected/classified, it should keep the initial zero.

The 1, 2, 3, & 4 represent classifications of eyemovements, apparently. The file "write2csv.m" contains the code > D_temp(logical(ETparams.fixationIdx(1, i).Idx), 4) = 1; > D_temp(logical(ETparams.saccadeIdx(1,i).Idx), 4) = 2; > D_temp(logical(ETparams.glissadeIdx(1, i).Idx), 4) = 3; > D_temp(logical(ETparams.nanIdx(1, i).Idx), 4) = 4; I haven't found anything relating to the 0, but I suspect those could simply be left "unlabeled". write2csv starts by initializing 5 columns of zeros > D_temp = zeros(nRows, 5); and then seems to overwrite only these rows where an event was detected. If no event was detected/classified, it should keep the initial zero.

👍 1

AsimHDar commented 2018-08-28 07:35:31 +00:00 (Migrated from github.com)

Copy link

Awesome! So would it be helpful to compare our algorithm to theirs?

We can use both algorithms on a common dataset (can the unit test generate an output with just x,y coordinates in a csv?) to see the differences. And then we can either make similar plots (see attached in OP) and compare quickly, or I can dig out number of events.

If we have a small sample we can also compare the differences in detection of events (one of the output shows velocity against time, showing saccade onset, and gliccade detection) to get an intuitive idea of what we/they may be missing, etc.

Awesome! So would it be helpful to compare our algorithm to theirs? We can use both algorithms on a common dataset (can the unit test generate an output with just x,y coordinates in a csv?) to see the differences. And then we can either make similar plots (see attached in OP) and compare quickly, or I can dig out number of events. If we have a small sample we can also compare the differences in detection of events (one of the output shows velocity against time, showing saccade onset, and gliccade detection) to get an intuitive idea of what we/they may be missing, etc.

👍 1

adswa commented 2018-08-28 08:23:29 +00:00 (Migrated from github.com)

Copy link

Sounds good to me!
I think both eyeballing plots and comparing/assessing differences in event detection would be good. How about using both algorithms on one of our real data files for a start (the unit test takes sub-02, run 5)?
I'll email you the file with only x and y data (I can't upload a tsv file and Alex has just warned me to never zip data because then the same kind of corruption as with Nyströms input data could happen)

Sounds good to me! I think both eyeballing plots and comparing/assessing differences in event detection would be good. How about using both algorithms on one of our real data files for a start (the unit test takes sub-02, run 5)? I'll email you the file with only x and y data (I can't upload a tsv file and Alex has just warned me to never zip data because then the same kind of corruption as with Nyströms input data could happen)

aqw commented 2018-08-28 09:07:36 +00:00 (Migrated from github.com)

Copy link

@ElectronicTeaCup @AdinaWagner @mih It gets more interesting than that...

Not only is the .zip file corrupt, but on a hunch, I downloaded the file 5 times. None of the md5sums match. I then looped over each one, extracting them into the same git repo, comitting, and comparing changes. The contents change, a lot, every time, and entire files are added and removed. So, that web server simply can't be trusted.

I looked on Archive.org to see if they have cached copies of the data; they don't. And google doesn't turn up anyone hosting copies either. The guy has a GitHub profile, but doesn't really share anything there. :-/

I'll see if I can get in contact with the sysadmins of the faulty server, and will see if I can get a response.

@ElectronicTeaCup @AdinaWagner @mih It gets more interesting than that... Not only is the .zip file corrupt, but on a hunch, I downloaded the file 5 times. None of the md5sums match. I then looped over each one, extracting them into the same git repo, comitting, and comparing changes. The contents change, a lot, every time, and entire files are added and removed. So, that web server simply can't be trusted. I looked on Archive.org to see if they have cached copies of the data; they don't. And google doesn't turn up anyone hosting copies either. The guy has a GitHub profile, but doesn't really share anything there. :-/ I'll see if I can get in contact with the sysadmins of the faulty server, and will see if I can get a response.

AsimHDar commented 2018-08-28 11:46:44 +00:00 (Migrated from github.com)

Copy link

@aqw Wow, that's pretty funky! So it there a simple solution to compress/bundle files and immortalize them? (At least, locally for long periods of time)

Here are the outputs I've gotten so far:

matlabfigure.zip --- change extension to tar if it doesn't work directly

@aqw Wow, that's pretty funky! So it there a simple solution to compress/bundle files and immortalize them? (At least, locally for long periods of time) Here are the outputs I've gotten so far:   [matlabfigure.zip](https://github.com/psychoinformatics-de/studyforrest-data-eyemovementlabels/files/2327809/matlabfigure.zip) --- change extension to tar if it doesn't work directly

aqw commented 2018-08-28 11:49:31 +00:00 (Migrated from github.com)

Copy link

@ElectronicTeaCup It's a complex problem, but simply using things like git can certainly make things more apparent.

I got a response from Lund's IT; they've forwarded me to the correct people. Hopefully they respond.

@ElectronicTeaCup It's a complex problem, but simply using things like git can certainly make things more apparent. I got a response from Lund's IT; they've forwarded me to the correct people. Hopefully they respond.

👍 1

AsimHDar commented 2018-08-28 12:46:34 +00:00 (Migrated from github.com)

Copy link

Trying to make a main sequence but currently the amplitudes seem to be a bit nutty.

The current counts are (sub2_run5):

Saccades: 848
Gliccades: 356
Fixations: 552

Trying to make a main sequence but currently the amplitudes seem to be a bit nutty. The current counts are (sub2_run5): Saccades: 848 Gliccades: 356 Fixations: 552

AsimHDar commented 2018-08-28 13:06:38 +00:00 (Migrated from github.com)

Copy link

This is the current output:

Going to wait if we can get our hands on the sample data. Amp shouldn't be more than 20 (!)

This is the current output:  Going to wait if we can get our hands on the sample data. Amp shouldn't be more than 20 **(!)**

aqw commented 2018-08-28 13:59:52 +00:00 (Migrated from github.com)

Copy link

@ElectronicTeaCup @AdinaWagner Awwwwwww yeah. Sysadmin underground FTW [1]. I got the data. I feel like a member of the twilight bark [2].

The data (just InputData/ , none of the .m files) are now in inputs/event_detector_1.1/ of the test branch. I wasn't quite sure where you wanted it, but it seemed logical.

Happy spelunking through non-corrupted data!

[1] If this thread ever becomes public: shout out to Nils Holmberg of Lund University for being one cool dude.
[2] https://www.youtube.com/watch?v=Ed_Kxyqgp1c

@ElectronicTeaCup @AdinaWagner Awwwwwww yeah. Sysadmin underground FTW [1]. I got the data. I feel like a member of the twilight bark [2]. The data (just ``InputData/`` , none of the ``.m`` files) are now in ``inputs/event_detector_1.1/`` of the ``test`` branch. I wasn't quite sure where you wanted it, but it seemed logical. Happy spelunking through non-corrupted data! [1] If this thread ever becomes public: shout out to Nils Holmberg of Lund University for being one cool dude. [2] https://www.youtube.com/watch?v=Ed_Kxyqgp1c

AsimHDar commented 2018-08-28 14:16:07 +00:00 (Migrated from github.com)

Copy link

@aqw Thanks a bunch!

@aqw Thanks a bunch!

AsimHDar commented 2018-08-28 14:50:31 +00:00 (Migrated from github.com)

Copy link

Fresh out of the box (with inputs/event_detector_1.1/) :

Saccades: 321
Glissades: 132
Fixations: 286

(What is the deal with the amplitude?)

EDIT: messed up the axis previously

Fresh out of the box (with inputs/event_detector_1.1/) : Saccades: 321 Glissades: 132 Fixations: 286    (What is the deal with the amplitude?) EDIT: messed up the axis previously

mih commented 2018-08-30 18:47:23 +00:00 (Migrated from github.com)

Copy link

Looks all good so far. I have our algorithm ready to test. To make a comparison with their data possible, I'd need a px2deg factor and the sampling rate. Do you have this info?

Looks all good so far. I have our algorithm ready to test. To make a comparison with their data possible, I'd need a px2deg factor and the sampling rate. Do you have this info?

mih commented 2018-08-30 19:03:46 +00:00 (Migrated from github.com)

Copy link

Found it

Tparams.screenSz = [1024 768]; % pixels
ETparams.screenDim = [0.516, 0.326]; % meters
ETparams.viewingDist = 0.60; % meters
ETparams.samplingFreq = 500; % Hz
ETparams.blinkVelocityThreshold = 1000;             % if vel > 1000 degrees/s, it is noise or blinks
ETparams.blinkAccThreshold = 100000;               % if acc > 100000 degrees/s^2, it is noise or blinks
ETparams.peakDetectionThreshold = 100;              % Initial value of the peak detection threshold. 

ETparams.minFixDur = 0.040; % in seconds
ETparams.minSaccadeDur = 0.010; % in seconds

Found it ``` Tparams.screenSz = [1024 768]; % pixels ETparams.screenDim = [0.516, 0.326]; % meters ETparams.viewingDist = 0.60; % meters ETparams.samplingFreq = 500; % Hz ETparams.blinkVelocityThreshold = 1000; % if vel > 1000 degrees/s, it is noise or blinks ETparams.blinkAccThreshold = 100000; % if acc > 100000 degrees/s^2, it is noise or blinks ETparams.peakDetectionThreshold = 100; % Initial value of the peak detection threshold. ETparams.minFixDur = 0.040; % in seconds ETparams.minSaccadeDur = 0.010; % in seconds ```

👍 1

mih commented 2018-08-30 19:04:47 +00:00 (Migrated from github.com)

Copy link

For some reason I thought someone said the data was 1250 Hz, but this seems to suggest otherwise.

For some reason I thought someone said the data was 1250 Hz, but this seems to suggest otherwise.

AsimHDar commented 2018-08-30 19:10:04 +00:00 (Migrated from github.com)

Copy link

Yes, I actually remember that too.

"The experimental setup consisted of one control computer
running SMI’s recording software iViewX with default settings
and communicating with another computer running
MATLAB to display the texts in full screen. Text images
were shown on a Samsung 19-in. flat screen with a size of
377  300 mm and viewed from a distance of 670 mm.
After a 13-point calibration, the participants were left to
read the text at their own pace. They could only move forward
through the text (by clicking the mouse). Gaze positions
were recorded monocularly (although viewing was
binocular) with the SMI HiSpeed eyetracker at 1250 Hz
throughout the experiment."

[1]https://link.springer.com/content/pdf/10.3758%2FBRM.42.1.188.pdf

Yes, I actually remember that too. "The experimental setup consisted of one control computer running SMI’s recording software iViewX with default settings and communicating with another computer running MATLAB to display the texts in full screen. Text images were shown on a Samsung 19-in. flat screen with a size of 377  300 mm and viewed from a distance of 670 mm. After a 13-point calibration, the participants were left to read the text at their own pace. They could only move forward through the text (by clicking the mouse). Gaze positions were recorded monocularly (although viewing was binocular) with the SMI HiSpeed eyetracker at **1250 Hz** throughout the experiment." [1]https://link.springer.com/content/pdf/10.3758%2FBRM.42.1.188.pdf

mih commented 2018-08-30 19:15:06 +00:00 (Migrated from github.com)

Copy link

Hmm, maybe these data are from a different source. Will look into the actual speeds tomorrow.

Hmm, maybe these data are from a different source. Will look into the actual speeds tomorrow.

AsimHDar commented 2018-08-30 19:28:17 +00:00 (Migrated from github.com)

Copy link

Maybe phoney labeling? I dug out the original (http://dev.humlab.lu.se/www-transfer/people/marcus-nystrom/EventDetector1.0.rar) and the data set is called "1250Hz_3_Participants.mat". But it's corrupt.

I'll run their algorithm with the different sampling rate and see what I get.

Maybe phoney labeling? I dug out the original (http://dev.humlab.lu.se/www-transfer/people/marcus-nystrom/EventDetector1.0.rar) and the data set is called "1250Hz_3_Participants.mat". But it's corrupt. I'll run their algorithm with the different sampling rate and see what I get.

mih commented 2018-08-30 19:30:19 +00:00 (Migrated from github.com)

Copy link

thx

thx

AsimHDar commented 2018-08-30 20:02:01 +00:00 (Migrated from github.com)

Copy link

Parameters:
ETparams.screenSz = [1024 768]; % pixels
ETparams.screenDim = [0.377, 0.300]; % meters
ETparams.viewingDist = 0.67; % meters
ETparams.samplingFreq = 1250; % Hz
ETparams.blinkVelocityThreshold = 1000;
ETparams.blinkAccThreshold = 100000;
ETparams.peakDetectionThreshold = 100;

ETparams.minFixDur = 0.040; % in seconds
ETparams.minSaccadeDur = 0.010; % in seconds

Counts are the same

Saccades: 321
Glissades: 132
Fixations: 286

Parameters: `ETparams.screenSz = [1024 768]; % pixels` `ETparams.screenDim = [0.377, 0.300]; % meters` `ETparams.viewingDist = 0.67; % meters` `ETparams.samplingFreq = 1250; % Hz` `ETparams.blinkVelocityThreshold = 1000; ` `ETparams.blinkAccThreshold = 100000; ` `ETparams.peakDetectionThreshold = 100; ` `ETparams.minFixDur = 0.040; % in seconds` `ETparams.minSaccadeDur = 0.010; % in seconds`    Counts are the same > Saccades: 321 > Glissades: 132 > Fixations: 286

mih commented 2018-08-31 06:22:21 +00:00 (Migrated from github.com)

Copy link

https://github.com/psychoinformatics-de/studyforrest-data-eyemovementlabels/pull/1/commits/c8078accd4b3364f750f4f05f0535bc27541e043 adds a test that runs our code on a trial from the Nystrom data. Looks pretty good. Let me know what you think.

There are a few cases, where I think we should add a rule that prevents labeling too close to a missing data boundary -- or we strip more data during preprocessing.

Let me know what you think.

https://github.com/psychoinformatics-de/studyforrest-data-eyemovementlabels/pull/1/commits/c8078accd4b3364f750f4f05f0535bc27541e043 adds a test that runs our code on a trial from the Nystrom data. Looks pretty good. Let me know what you think. There are a few cases, where I think we should add a rule that prevents labeling too close to a missing data boundary -- or we strip more data during preprocessing. Let me know what you think.

adswa commented 2018-08-31 09:05:33 +00:00 (Migrated from github.com)

Copy link

output after latest of your pushs looks picture-perfect for the Nyström data for me. Main sequence looks absolutely fine.

output after latest of your pushs looks picture-perfect for the Nyström data for me. Main sequence looks absolutely fine.

mih commented 2018-08-31 09:11:33 +00:00 (Migrated from github.com)

Copy link

@AdinaWagner @ElectronicTeaCup if you get a chance, please run that test on all the other trials of the nystrom data and carefully go through the event sequence to see if there are ANY issues.

@AdinaWagner @ElectronicTeaCup if you get a chance, please run that test on all the other trials of the nystrom data and carefully go through the event sequence to see if there are ANY issues.

mih commented 2018-08-31 10:04:10 +00:00 (Migrated from github.com)

Copy link

@ElectronicTeaCup can you get me a list of all events of all types detected from one of the nystrom data samples using the matlab code? I want to validate against that.

@ElectronicTeaCup can you get me a list of all events of all types detected from one of the nystrom data samples using the matlab code? I want to validate against that.

AsimHDar commented 2018-08-31 10:18:14 +00:00 (Migrated from github.com)

Copy link

1_2.csv.tar.zip (rename to .tar)

This is the output that the algorithm generates for 1_2.csv. It's a bit meh since it goes frame by frame saying what that event in that particular moment is.

But i can get a count in a bit if you want, and the plot of velocities and with time, denoting which event is what (fig 9, nystrom). It's a matlab fig attached earlier (matlabfigure.zip --- change extension to tar if it doesn't work directly)

[1_2.csv.tar.zip](https://github.com/psychoinformatics-de/studyforrest-data-eyemovementlabels/files/2339660/1_2.csv.tar.zip) (rename to .tar) This is the output that the algorithm generates for 1_2.csv. It's a bit meh since it goes frame by frame saying what that event in that particular moment is. But i can get a count in a bit if you want, and the plot of velocities and with time, denoting which event is what (fig 9, nystrom). It's a matlab fig attached earlier ([matlabfigure.zip](https://github.com/psychoinformatics-de/studyforrest-data-eyemovementlabels/files/2327809/matlabfigure.zip) --- change extension to tar if it doesn't work directly)

mih commented 2018-08-31 10:33:18 +00:00 (Migrated from github.com)

Copy link

what settings did run run that with?

what settings did run run that with?

AsimHDar commented 2018-08-31 10:49:06 +00:00 (Migrated from github.com)

Copy link

The stock ones. And it's for the whole set from what I recall; was super slow when I was looking at it.

The stock ones. And it's for the whole set from what I recall; was super slow when I was looking at it.

mih commented 2018-08-31 11:26:22 +00:00 (Migrated from github.com)

Copy link

Ok. The file you sent seems to be only for one trial. Processing that trial with our code takes about 5s on my machine. Which part was slow for you?

Ok. The file you sent seems to be only for one trial. Processing that trial with our code takes about 5s on my machine. Which part was slow for you?

mih commented 2018-08-31 11:58:41 +00:00 (Migrated from github.com)

Copy link

The stock ones.

That is still ambigous (stock from paper, or stock from code), please give the numbers.

Here is the info I extract from their output (assuming: deg_per_pixel(0.377, 0.67, 1024), sampling_rate=1250.0):

And here the same info from our code:

> The stock ones. That is still ambigous (stock from paper, or stock from code), please give the numbers. Here is the info I extract from their output (assuming: `deg_per_pixel(0.377, 0.67, 1024), sampling_rate=1250.0`):   And here the same info from our code:  

adswa commented 2018-08-31 12:05:21 +00:00 (Migrated from github.com)

Copy link

I have run test_nystrom.py on all csv files. I have used the interactive version of the tests to look at the plots, that is my version of all code is ffd4041 . If any of the issues I point out has been fixed in later pushes, just ignore them. I feel like I'm being a bit overly detailed, but since you asked for ANY issues...

Overall, everything looks good. The one thing I found which resembled something "systematic" was what I would call a misclassification of either noise or saccades after/before missing data as a pursuit (and there we less than a dozen cases in total). (see first bullet point)

• There are cases in which saccadic-like patterns are classed as pursuit if they happen prior or after phases of data loss. See e.g. 1_3.csv at ~34.5 & ~69, 2_2.csv at ~67, 2_3.csv at ~63, 2_4.csv at ~6, ~41, ~44
• 1_9.csv has in my opinion a saccade where a pursuit was detected at ~48.8
• 1_3 & 1_4 each have two stray datapoints in the top left corner of their main sequence.
• data files 2_1, 2_2, 2_3 have somewhat of a gap in their main sequence. In all 1_*.csv files there is also not much data but it isn't a real gap (I attached a picture of the main sequence of 2_2.csv). Not sure whether this is maybe just due to few data points
  

In file 2_4.csv I encountered the following warnings when testing test_nystrom.py:

code/tests/test_nystrom.py::test_real_data
/home/adina/Documents/MastersThesis/Asim/studyforrest-data-eyemovementlabels/code/detect_events.py:361: RuntimeWarning: Mean of empty slice.
sacc_start, sacc_end, peakvels.mean())
/usr/lib/python3/dist-packages/numpy/core/_methods.py:80: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)

-- Docs: http://doc.pytest.org/en/latest/warnings.html

thats all I encountered. To me, all of that looked good.

I have run test_nystrom.py on all csv files. I have used the interactive version of the tests to look at the plots, that is my version of all code is ffd4041 . If any of the issues I point out has been fixed in later pushes, just ignore them. I feel like I'm being a bit overly detailed, but since you asked for ANY issues... Overall, everything looks good. The one thing I found which resembled something "systematic" was what I would call a misclassification of either noise or saccades after/before missing data as a pursuit (and there we less than a dozen cases in total). (see first bullet point) - There are cases in which saccadic-like patterns are classed as pursuit if they happen prior or after phases of data loss. See e.g. 1_3.csv at ~34.5 & ~69, 2_2.csv at ~67, 2_3.csv at ~63, 2_4.csv at ~6, ~41, ~44 - 1_9.csv has in my opinion a saccade where a pursuit was detected at ~48.8 - 1_3 & 1_4 each have two stray datapoints in the top left corner of their main sequence. - data files 2_1, 2_2, 2_3 have somewhat of a gap in their main sequence. In all 1_*.csv files there is also not much data but it isn't a real gap (I attached a picture of the main sequence of 2_2.csv). Not sure whether this is maybe just due to few data points  In file 2_4.csv I encountered the following warnings when testing test_nystrom.py: > code/tests/test_nystrom.py::test_real_data /home/adina/Documents/MastersThesis/Asim/studyforrest-data-eyemovementlabels/code/detect_events.py:361: RuntimeWarning: Mean of empty slice. sacc_start, sacc_end, peakvels.mean()) /usr/lib/python3/dist-packages/numpy/core/_methods.py:80: RuntimeWarning: invalid value encountered in double_scalars ret = ret.dtype.type(ret / rcount) > -- Docs: http://doc.pytest.org/en/latest/warnings.html thats all I encountered. To me, all of that looked good.

mih commented 2018-08-31 12:08:13 +00:00 (Migrated from github.com)

Copy link

gap in their main sequence

I think this is due to the stimulus they used. AFAIK this was a reading task, hence many small saccades between words and a few big ones at the end of each line.

> gap in their main sequence I think this is due to the stimulus they used. AFAIK this was a reading task, hence many small saccades between words and a few big ones at the end of each line.

adswa commented 2018-08-31 12:09:08 +00:00 (Migrated from github.com)

Copy link

that makes sense. That also explains an almost complete lack of smooth pursuits very well

that makes sense. That also explains an almost complete lack of smooth pursuits very well

mih commented 2018-08-31 12:11:46 +00:00 (Migrated from github.com)

Copy link

There should not be any. In the unit test I have set the max_fixation_amp to 4.0 deg to pratically suppress any pursuit labeling.

There should not be any. In the unit test I have set the `max_fixation_amp` to 4.0 deg to pratically suppress any pursuit labeling.

adswa commented 2018-08-31 12:19:44 +00:00 (Migrated from github.com)

Copy link

there are the ones I noted above. Not many, but very few. In my opinion could be ignored. To make double sure I just checked for 2_4.csv with the most recent pushes you made, I get e.g.

5.9032 -> 6.1360: PURS (4.7257803935172085)

there are the ones I noted above. Not many, but very few. In my opinion could be ignored. To make double sure I just checked for 2_4.csv with the most recent pushes you made, I get e.g. > 5.9032 -> 6.1360: PURS (4.7257803935172085)

mih commented 2018-08-31 12:21:31 +00:00 (Migrated from github.com)

Copy link

We could think about making a switch that disables this, but I think more information is better. There should not be any pursuits, because the stimulus is static, hence a pursuit could be a drift of the measurement -- but we are not in the position to tell.

We could think about making a switch that disables this, but I think more information is better. There should not be any pursuits, because the stimulus is static, hence a pursuit could be a drift of the measurement -- but we are not in the position to tell.

AsimHDar commented 2018-09-01 21:57:20 +00:00 (Migrated from github.com)

Copy link

Update: So I can get the main sequences and other data from the Nystrom algorithm after a lot of messing around (weird structs that I have not seen before) and the huge numbers were there probably due to the lack of conversion from pixel to degree (adjusting for the factor made the outputs in the proper range of values for amp)

I can't get TST to work. On python 2 I get an error mid script, and on python 3 I get a weird numpy error (which I have seen before --- so probably due to my environment?) I cloned it on two other computers but could not get the thing to work for me. Does it work on python 2? Will go on the debugger some more tomorrow, ugh.

Update: So I can get the main sequences and other data from the Nystrom algorithm after a lot of messing around (weird structs that I have not seen before) and the huge numbers were there probably due to the lack of conversion from pixel to degree (adjusting for the factor made the outputs in the proper range of values for amp) I can't get TST to work. On python 2 I get an error mid script, and on python 3 I get a weird numpy error (which I have seen before --- so probably due to my environment?) I cloned it on two other computers but could not get the thing to work for me. Does it work on python 2? Will go on the debugger some more tomorrow, ugh.

mih commented 2018-09-02 07:59:23 +00:00 (Migrated from github.com)

Copy link

@ElectronicTeaCup Just post the error messages and we'll figure it out. Both py2 and py3 work fine for me.

Note that the TST code already has an importer for the nystrom output CSV file that you provided. There should be no need to edit their code in any way -- this has the potential for breakage and could introduce differences that we do not want.

@ElectronicTeaCup Just post the error messages and we'll figure it out. Both py2 and py3 work fine for me. Note that the TST code already has an importer for the nystrom output CSV file that you provided. There should be no need to edit their code in any way -- this has the potential for breakage and could introduce differences that we do not want.

AsimHDar commented 2018-09-02 08:18:18 +00:00 (Migrated from github.com)

Copy link

Okay, thanks. After a new pull I get this

python test_nystrom.py
Traceback (most recent call last):
  File "test_nystrom.py", line 2, in <module>
    from . import utils as ut
ValueError: Attempted relative import in non-package

And

python3 test_nystrom.py
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 16, in <module>
    from . import multiarray
ImportError: cannot import name 'multiarray'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test_nystrom.py", line 1, in <module>
    import numpy as np
  File "/usr/lib/python2.7/dist-packages/numpy/__init__.py", line 142, in <module>
    from . import add_newdocs
  File "/usr/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/usr/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/usr/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 24, in <module>
    raise ImportError(msg)
ImportError: 
Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control).  Otherwise reinstall numpy.

If i force the initial lines to work by placing the needed files (detect events) into the working directory and removing the from . then I get this:

 > /home/asimio/CleanEyes/studyforrest-data-eyemovementlabels/code/tests/test_nystrom.py(67)<module>()
     65     pl.show()
     66 
---> 67 def test_real_data():
     68     data = np.recfromcsv(
     69         'inputs/event_detector_1.1/1_2.csv',


ipdb> s
--Return--
Traceback (most recent call last):

  File "<ipython-input-6-19dd949130e6>", line 1, in <module>
    runfile('/home/asimio/CleanEyes/studyforrest-data-eyemovementlabels/code/tests/test_nystrom.py', wdir='/home/asimio/CleanEyes/studyforrest-data-eyemovementlabels/code/tests')

  File "/home/asimio/.local/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "/home/asimio/.local/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 94, in execfile
    builtins.execfile(filename, *where)

  File "/usr/lib/python2.7/bdb.py", line 53, in trace_dispatch
    return self.dispatch_return(frame, arg)

  File "/usr/lib/python2.7/bdb.py", line 91, in dispatch_return
    if self.quitting: raise BdbQuit

BdbQuit

Okay, thanks. After a new pull I get this ``` python test_nystrom.py Traceback (most recent call last): File "test_nystrom.py", line 2, in <module> from . import utils as ut ValueError: Attempted relative import in non-package ``` And ``` python3 test_nystrom.py Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 16, in <module> from . import multiarray ImportError: cannot import name 'multiarray' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "test_nystrom.py", line 1, in <module> import numpy as np File "/usr/lib/python2.7/dist-packages/numpy/__init__.py", line 142, in <module> from . import add_newdocs File "/usr/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in <module> from numpy.lib import add_newdoc File "/usr/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 8, in <module> from .type_check import * File "/usr/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 11, in <module> import numpy.core.numeric as _nx File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 24, in <module> raise ImportError(msg) ImportError: Importing the multiarray numpy extension module failed. Most likely you are trying to import a failed build of numpy. If you're working with a numpy git repo, try `git clean -xdf` (removes all files not under version control). Otherwise reinstall numpy. ``` If i force the initial lines to work by placing the needed files (detect events) into the working directory and removing the `from .` then I get this: ``` > /home/asimio/CleanEyes/studyforrest-data-eyemovementlabels/code/tests/test_nystrom.py(67)<module>() 65 pl.show() 66 ---> 67 def test_real_data(): 68 data = np.recfromcsv( 69 'inputs/event_detector_1.1/1_2.csv', ipdb> s --Return-- Traceback (most recent call last): File "<ipython-input-6-19dd949130e6>", line 1, in <module> runfile('/home/asimio/CleanEyes/studyforrest-data-eyemovementlabels/code/tests/test_nystrom.py', wdir='/home/asimio/CleanEyes/studyforrest-data-eyemovementlabels/code/tests') File "/home/asimio/.local/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 705, in runfile execfile(filename, namespace) File "/home/asimio/.local/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 94, in execfile builtins.execfile(filename, *where) File "/usr/lib/python2.7/bdb.py", line 53, in trace_dispatch return self.dispatch_return(frame, arg) File "/usr/lib/python2.7/bdb.py", line 91, in dispatch_return if self.quitting: raise BdbQuit BdbQuit ```

mih commented 2018-09-02 08:36:59 +00:00 (Migrated from github.com)

Copy link

To run the tests, stay in the root of the repository and execute something like this:

python3 -m pytest -s -v  code --pdb

The first could also be python, and you can also give code/tests or . or a filename, or even a single function. The --pdb will invoke a debugger if something fails.

To run the tests, stay in the root of the repository and execute something like this: ``` python3 -m pytest -s -v code --pdb ``` The first could also be `python`, and you can also give `code/tests` or `.` or a filename, or even a single function. The `--pdb` will invoke a debugger if something fails.

AsimHDar commented 2018-09-02 16:17:54 +00:00 (Migrated from github.com)

Copy link

Thanks, I had totally forgotten about that piece of code you sent me.
I got the outputs from the Nystrom algorithm and the numbers look great, and so do the main sequences! I added the outputs of the Nystrom algorithm and included a readme to state the settings used (the ones from the paper were used), also included is the number of saccades the Matlab algorithm detected before writing the csv output.

Using the test script: I found that the number of saccades when compared (using the Nystrom data set) are very similar; usually just differing by 1-5 saccades except for 1_3 in which our algorithm gave us 273 saccades as opposed to 247 (a difference of 26) from Nystom algorithm output.

@mih I don't know if it's important at this time, but looking at the variables when running the matlab script, the number of events differs a bit on some files when compared to the number seen using the test script. For example in 1_2 I would get 259 saccade events, and when I process the csv with the test script I get 248. The maximum difference was by 43 (in 1_7)

@AdinaWagner

1_3 & 1_4 each have two stray datapoints in the top left corner of their main sequence.
data files 2_1, 2_2, 2_3 have somewhat of a gap in their main sequence. In all 1_*.csv files there is also not much data but it isn't a real gap (I attached a picture of the main sequence of 2_2.csv). Not sure whether this is maybe just due to few data points

Both cases are the same in Nystrom algorithms outputs too. The gap is more in 2_1 and 2_2 but 2_3 has a few data points:

Thanks, I had totally forgotten about that piece of code you sent me. I got the outputs from the Nystrom algorithm and the numbers look great, and so do the main sequences! I added the outputs of the Nystrom algorithm and included a readme to state the settings used (the ones from the paper were used), also included is the number of saccades the Matlab algorithm detected before writing the csv output. Using the test script: I found that the number of saccades when compared (using the Nystrom data set) are very similar; usually just differing by 1-5 saccades except for 1_3 in which our algorithm gave us 273 saccades as opposed to 247 (a difference of 26) from Nystom algorithm output. @mih I don't know if it's important at this time, but looking at the variables when running the matlab script, the number of events differs a bit on some files when compared to the number seen using the test script. For example in 1_2 I would get 259 saccade events, and when I process the csv with the test script I get 248. The maximum difference was by 43 (in 1_7) @AdinaWagner > 1_3 & 1_4 each have two stray datapoints in the top left corner of their main sequence. > data files 2_1, 2_2, 2_3 have somewhat of a gap in their main sequence. In all 1_*.csv files there is also not much data but it isn't a real gap (I attached a picture of the main sequence of 2_2.csv). Not sure whether this is maybe just due to few data points Both cases are the same in Nystrom algorithms outputs too. The gap is more in 2_1 and 2_2 but **2_3** has a few data points: 

mih commented 2019-04-23 07:08:28 +00:00 (Migrated from github.com)

Copy link

I think we are good on this aspect. https://github.com/psychoinformatics-de/paper-remodnav

I think we are good on this aspect. https://github.com/psychoinformatics-de/paper-remodnav

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

master

git-annex

No results found.

Labels

Clear labels   

bug

Something isn't working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature or request

  

good first issue

Good for newcomers

  

help wanted

Extra attention is needed

  

invalid

This doesn't seem right

  

question

Further information is requested

  

wontfix

This will not be worked on

No labels

bug

duplicate

enhancement

good first issue

help wanted

invalid

question

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

studyforrest/data-eyemovementlabels#8

No description provided.