πŸ”¬ research-publications


user-3f0708 03 October, 2018, 12:15:46

I would like to know if you have any article that explains the processes of tracking the pupil abs open source platform, from image acquisition, conversion to gray scale and pupil detection through pupil reflection due to infrared illumination. Because I am doing a job using the pupil it is necessary to mention how the process works tracking the movements of the eyes.

user-af87c8 05 October, 2018, 08:29:53

@user-3f0708 did you look at the pupil labs original paper? https://arxiv.org/abs/1405.0006

user-8779ef 10 October, 2018, 20:29:20

Are there any published studies that characterize the pupil system's latency, accuracy, and precision, or a portion fo these metrics? I'm looking for a peer reviewed citation. I understand that these numbers will evolve with the system, but need an estimate to motivate a project proposal.

user-af87c8 11 October, 2018, 08:06:13

2018-11-17: Important: We found a bug in our analysis that influenced the accuracy measurements. The updated numbers are in the text

@user-8779ef (not pupil labs affiliated) we are currently evaluating the system on a large set of eye movement tests simultaneously with an Eyelink 1000. It's not peer reviewed/published. Using n=15, a 49-point grid, 13 point 2D calibration we found an accuracy of ~0.8Β° (eyelink 0.5Β°), precision of 0.12Β° (eyelink 0.02Β°). Delay was ~10ms compared to Eyelink but we do not evaluate this, formaly (we could, but there are some other timing issues so I personally would not use it for gaze dependent measurements). Decay of performance occurs with pupil labs with an average of 1.1Β° after ~5minutes (much worse in some subjects, much less decay in others). I'm not aware of any peer reviewed papers, there is one preprint but it has only n=3. The results presented here are preliminary (!). I do not expect them to change much [comment: ha! good I had that disclaimer ;)), they are not very dependent on fixation algorithm used (we use velocity based engbert, but HMM gives similar results but has some further issues).

user-af87c8 11 October, 2018, 08:07:32

of course the final paper will show a much more nuanced view on this, there are many differences between the 20k€ eyelink and 2k€ pupil labs glasses - if you are looking for small saccades you will be completly lost with PL πŸ˜‰ But that's not what the glasses were developed for

user-af87c8 11 October, 2018, 08:08:31

if anyone is interested in the subject-plots etc. I'm happy to give a preliminary view

mpk 11 October, 2018, 08:09:32

Hi @user-af87c8 just curious, could you share how you calculated the system latency?

user-af87c8 11 October, 2018, 08:13:34

@mpk first of all I asked here and this is what was told to me πŸ˜‰ I have trigger based pupil labs data & concurrent eyelink data. But I do a lot of regressing because computer time & eye-cams in PL were sometimes with drift. I would need to look into detail in our own data

mpk 11 October, 2018, 08:14:08

@user-af87c8 regarding drift, after your detailed report we actually fixed this.

mpk 11 October, 2018, 08:14:42

We recently remeasured latency and got around 10ms from start of exposure until the frame was in Pupil Capture.

user-af87c8 11 October, 2018, 08:16:09

I could e.g. cross correlate eyelink with pupil labs - but this ignores the drift I still have in my data. We use the 120Hz (not the new 200Hz) cameras, might that influence the lag?

mpk 11 October, 2018, 08:16:27

it might. We can measure that as well.

mpk 11 October, 2018, 08:17:16

Btw: We really appreciate the thorough evaluation

user-af87c8 11 October, 2018, 08:18:16

You are welcome πŸ˜ƒ I'm happy how well the PL glasses do accuracy wise! This makes me more confident for actual mobile measurements πŸ˜ƒ

user-af87c8 11 October, 2018, 08:18:34

(I'm also not doing this work alone, Katharina Gross, Inga Ibbs & Peter KΓΆnig are with me in this project)

user-af87c8 11 October, 2018, 08:19:42

I edited my message to reflect your results

wrp 11 October, 2018, 08:33:13

@user-8779ef Jeff MacInnes et al.'s paper "Wearable Eye-tracking for Research: Automated dynamic gaze mapping and accuracy/precision comparisons across devices" also provides a comparison of accuracy between mobile systems (these researchers are not affiliated with Pupil Labs. The paper is not peer reviewed - on biorxiv): https://www.biorxiv.org/content/early/2018/06/28/299925

mpk 11 October, 2018, 08:57:49

@user-af87c8 thanks for the info. I look forward to the paper!

user-8779ef 11 October, 2018, 19:38:30

@wrp Thank you for that link.

wrp 12 October, 2018, 02:52:58

Welcome!

user-72a78a 12 October, 2018, 14:35:45

Does anyone know if the eye-tracker can sync with physiological sensors such as heart rate monitors, EEG, or GSR sensors?

user-af87c8 12 October, 2018, 15:56:28

we have the pupil labs synched with EEG using parallel-port triggers. But in order to do so you need pupillabs from source

user-af87c8 12 October, 2018, 15:57:12

alternative would be to send ZMQ triggers from stimulation computer to eyetracking recorder (and in parallel sync your other devices from the stimuulus computer). Thats what we did in another study

user-72a78a 12 October, 2018, 16:55:12

Thank you! Could you clarify what you mean by 'pupillabs from source'?

user-af87c8 15 October, 2018, 08:21:23

there is the possibility to run pupil labs as a binary (e.g. a exe in windows) and you can run it from python (from the source). From the source you need to install many dependencies first, which can be difficult in my experience

user-e79f71 22 October, 2018, 03:41:29

Any further instructions regarding syncing other devices would be greatly appreciated! For example eprime, psychtoolbox, presentation, etc with pupil. For some context, our initial use will be pupillometry only, so timing accuracy is important, but that's about it. PS I hope to use CHAP to do the pupillometry analysis, not sure if anyone has experience with think tool in combination with pupil

http://in.bgu.ac.il/en/Labs/CNL/chap/default.aspx

user-af87c8 24 October, 2018, 10:47:29

@mpk I checked in one subject and your 10ms delay between camera and and computer checks out. Sorry for the confusion, the initial drift between camera and computer we had (which is now hopefully fixed), contained sometimes an offset of 40ms. Cross correlation is difficult because of pupillabs somewhat unsteady samplingrate - but just visually if I add a 10ms delay, it fits our data (disclaimer: checked only 2 subjects at only 2 close saccades) nicely

wrp 24 October, 2018, 10:52:50

thanks for the update @user-af87c8 - looking forward to further feedback/publications from you and your group 😸

End of October archive