So... is it me, or do the eye images arrive to the pupil detection plugin upside down?

Yes, I know that one camera is upside down.

Sorry, I shoudl have expanded.

So, when I view the raw eye videos using VLC ...eye_1 is upside down, and eye_0 is right-side up.

...but, when I view the images inside the plugin, or save them out, it is the opposite.

Known issue? Am I crazy?

Image coordinate system origin is top left not bottom left.

oK, thx.

Trying to decide how to handle this, since our trained neural network expects right-side up images.

https://numpy.org/doc/stable/reference/generated/numpy.flipud.html

Thanks for the pointer. Already on it 🙂

...and, does HMD-Eyes change any pupil detection settings from the default values I see in player?

Not sure

Happy to do some digging. Any idea where I should look?

Easiest way to check would be to subscribe to all notifications and look how eye processes etc are being started

Hurmn.

Ok. Thanks, @papr. As always, very helpful.

Man, it's like 10pm in Germany right now, isn't it? That's dedication.

10:30 😉 happy to help

Hi @papr and everyone, I wonder what method you recommend for synchronizing the eye data with separately recorded task events, after a session. Is there any way to flash detectable markers on the screen (say at the beginning of each trial) so that the timestamps of these events appear in the exported data?

Hey everyone, is there a supported way of exporting lots of videos without dropping every single recording folder into the player gui?

@papr I'm using the fixations and world_timestamps csv files the world_timestamps I can read in without exporting as a numpy array

If you recorded fixations in real time, you could read the fixations.pldata file using this function https://gist.github.com/papr/743784a4510a95d6f462970bd1c23972#file-extract_diameter-py-L71-L96

But you will need to aggregate them, as the online fixation detector output is slightly different from the post-hoc detector.

unfortunately I'm not using the online detector as I tend to get more frames the less plugins I run in the online modus

You can use the recorded gaze data and run the post-hoc fixation detector at the bottom of this tutorial https://github.com/pupil-labs/pupil-tutorials/blob/master/01_load_exported_data_and_visualize_pupillometry.ipynb

As I need to sync the world video with another camera I require to have close to 60 frames at all times

I had a look at the tutroial, it seems that the csv table used for everything is something I get from a 'raw data export'. Is that right? exported_pupil_csv = os.path.join(recording_location, 'exports', '000', 'pupil_positions.csv')

Is there a recommended way of calling the raw_data_export plugin from outside the gui?

Thank you again! I'm gonna have a go at it.

So, I changed the code that you linked, so that I get all the collumns (from pupil.pldata) that I need for the tutorial code. But what I get for the fixations is only one fixation for the whole duration of the video. Any idea what the error might be?

Also your saying that I should compute the fixations directly from the gaze.pldata file? which columns would I need for that? (my guess would be: [confidence, diameter, circle_3d:normal(x,y,z)] and norm_pos(x,y) for the actual point looked at in the world video)

Difficult to tell what the issue might be. I might need to have a quick look at the code to check.

To load gaze data, you need to pass topic="gaze" to load_and_yield_data(). Also, you need to adjust the extracted fields accordingly.

Please run the tutorial as is with the accompanying data and check if the generated fixations match the original output. Afterward, try again with your own recording.

Thanks. Has anyone integrated pupilLab APIs with mWorks?

This reminds me - because there is no parallax error in VR due to an offset of the eyes and scene camera, I question the importance of calibrating to multiple depths.

The only reason to do that would be if the pupil changed size with fixatnoi depth, but due to the fixed focal distance (because the optics are fixed), there is no change in accommodation with changes in depth.

It is!

So, I think you guys are wasting time with the default setting of calibrating to multiple depth planes. My advice? Instead of changing depths, change the clear-color of the background so that you vary gray levels and get a variety of pupil sizes.

I should say that I'm not sure if it has been published that pupil size varies with depth of accommodation, and does not vary with fixation depth under vergence / accommodation conflict. This is an assumptino of mine tested by looking at pupils during calibration at different target depths along the same angular direction.

They are just very, very stable.

Well, I should acknowledge that the fixatino target's position will change subtly within the screen of the left/right eye as it changes in depth, so it's not a total waste to change depth planes, but it would be more effective to just stick to a single plane and vary spatial position more evenly across the field.

Looks like I may wrong about the stability of pupil size when changing fixation depth without accommodation! https://link.springer.com/content/pdf/10.1007/s00417-017-3770-2.pdf ...but, I remain skeptical, because that's just not what I see in these eye videos.

Ok, another issue: when switching between pupil plugins (I am developing one), I often get an error: eye1 - [ERROR] launchables.eye: Process Eye1 crashed with trace: Traceback (most recent call last): File "C:\Users\gabri\Documents\GitHub\pupil\pupil_src\launchables\eye.py", line 776, in eye consume_events_and_render_buffer() File "C:\Users\gabri\Documents\GitHub\pupil\pupil_src\launchables\eye.py", line 272, in consume_events_and_render_buffer p.gl_display() File "C:\Users\gabri\Documents\GitHub\pupil\pupil_src\shared_modules\pupil_detector_plugins\pye3d_plugin.py", line 193, in gl_display draw_eyeball_outline(result) File "C:\Users\gabri\Documents\GitHub\pupil\pupil_src\shared_modules\pupil_detector_plugins\visualizer_2d.py", line 58, in draw_eyeball_outline if pupil_detection_result_3d["model_confidence"] <= 0.0: KeyError: 'model_confidence'

This is solved by erasing pupil player settings.

...however, I can't help but feel this is common enough to be considered a bug. I will post this as an issue...

Thanks for the heads up. I'll double-check.

That was an issue in the past - but I cleaned out the data folder of offline data. I'm surprised that that kind fo info is stored in player settings.

The session settings store which plugins were active and their settings. On launch, it reloads the previously active plugins. If the loaded plugin is faulty, it will crash and not write out new session settings. i.e. the next time, the faulty plugin will be loaded again.

I'm guessing that you have hit the nail on the head, so I'll hold off on posting the issue.

Got it running now on the gaze.pldata file as well! One quick question though: the fixation plot looks rather different when I compute it with the pupil- vs with the gaze-data. Is that normal?

fixations for pupil data

fixations for gaze data

Want to know how EyeTracking data looks like from the Pupil Lens tracker. Is it possible to get a sample file through here?

That is a very valid point, thank you. I'll make sure to do that, if I decide to put such a comparison in my thesis. For now I just wanted to make sure that the output I produced is reasonable. Also if I where to use the pupil export plugin the output I would get should be the one from the gaze data right? Just asking so that I can be sure that it is reasonable to compare my output with the one from the plugin.

hello hello!

I'm currently busy trying to match the gaze data of pupil invisible recordings to video frame data and I'm trying to understand the logged data a little bit better. Could you maybe help me with that?

I've parsed both time files for gaze and video frame values

then I went over all video frame timestamps and looked for the closest matching gaze timestamp

when I then visualize the result, it looks like I am a little ahead on the gaze data

How do you visualize the result?

it's a little app I wrote in unity. could be that I screwed something up in there of course and I'll look at that some more for sure. but just to get a better sense for the logged data

is it normal that I get quite a few extra entries in the gaze timestamp logfile, because they maybe don't finish exactly at the same time?

By logfile, do you mean the .time file?

yeah

maybe 1-3 seconds after my last videoframe timestamp data

This is indeed normal. You can discard the excess timestamps.

alright nice, thanks for the info!

I do think I'm doing the correct thing, but I'm slightly ahead and I don't really have an explanation for that

ahead meaning the gaze values I show to the corresponding closest frame is a few frames ahead of where the companion app shows it should be (and yours is correct too)

So, you are visualizing the data in realtime, correct?

post processing for now, to later add information from other sensor data

just wrote a parser to match the gaze timestamps to video timestamps, let the video play, ask each frame which frame we are at, look up the match, draw the gaze on top

So, you could use Pupil Player to compare the result, correct? Have you tried that? The question would be, which of the sensors is providing the incorrect data.

just a sec, I didn't know about pupil player 🙂

https://github.com/pupil-labs/pupil/releases/latest

really neat, thanks!

so I was trying to reinvent the wheel, in a shoddier version 🙂

would rectifying the video frames result in gaze points not reflecting what you looked at? or I guess one could rectify those as well

not that the camera distortion bothers me, it's just something we tried out with a tool we developed inhouse

but this is perfect, thank you very much. now I can adapt the format of what I have to match yours, see where I went wrong

another thing that happened is that the original files disappeared or got renamed 🙂

I recommend keeping a backup of the original PI recording.

thanks for the tip!

hoooo, there's even plugins that do all kinds of things, how great is this! 🙂

You can also write your own plugins.

might be difficult to do in this project, but looks more appealing than what I had going so far 🙂

time to find out how long the conversion for the maximum invisible recording length takes on this old machine

98 minutes, ~4gb video

Well, Pupil Player is not optimized for long recordings. You might see serious performing issues with this kind of recording lengths. 😕

looks just fine, no worries 🙂

oh, just noticed. I used the eye video overlay

I can flip both vertically and horizontally

but 90° is what it's off by I think?

Correct. Rotation is currently not supported.

but this is great still. already showed a flaw in the way we use some of the external applications

a couple of psych tests that participants had to periodically go through

cognitive tests of a sort, usually on tablets in the field

but where those are positioned matters. if people naturally almost fully close their eyes, hard to get good eyetracking data 🙂

moving to a specific frame in these very long recordings gets a little tougher

it's so cool that I can start exporting from a specific frame on

but can I stop that before the end as well?

oh or does it start from the beginning anyways and I misinterpreted

Yes, there are trim marks left and right of the timeline to adjust the export range. You can change the range in the general settings as well

oh perfect, thank you again 🙂

right I even had it open in a tab already, got a little excited when I had the player open

Hello! Did the removal of PIL happen in this latest release?

Windows 10

@user-3cff0d I have sent you a download link via DM. Please let me know if you run into any issues with this bundle.

sorry, I didn't see this in the docs and I'm not sure if it's there, but is there anyway to only use the 2d pupil detector (for calibration, data collection, etc)?

What's the name of the 2d choreography that I can put into the calibration settings (unity) ?

The current hmd-eyes version only supports 3d calibration.

For that, you will have to use hmd-eyes 0.62 or earlier. It is compatible with Pupil Capture v1.23 or lower. Be aware that this will use a dual-monocular gaze mapper instead of a binocular one.

Feel free to share a recording of your 3d calibration with [email removed] We can have a look and give specific feedback. For example, it could be possible that the 3d eye model is not fit well. That would explain an instable gaze signal.

nvm, I found it, but as for the calibration data, still need to find out about that

ah, i have to use capture; i've been using service with unity

another question: would i be able to receive 2d pupil detection data? if so, i might just try to do my own calibration in unity using raw 2d data

i asked before if i should subscribe to "gaze.2d", but it looks like i should subscribe to "pupil", right? or use a PupilListener to parse pupilData whenever it comes in and extract the position in camera 3d space

Has anyone here worked with the application AttentionVisualizer that can stream eye tracking data from Invisible and Core?

so, how would i go about getting gaze data from the 2d model?

I'm having some trouble with this; in the GazeListener.cs script, it subscribes to gaze.3d; so would I just subscribe to gaze.2d instead?

Oh, is there no way to get 2D data in the newer version?

Hi, We have recorded 2D data with pupil Lab core. How can we extract pupil diamarer in mm? since we do not have corrected for perspective data. Is there any python code I can use for diameter extraction?

Thank you for your answer. It says "is not corrected for perspective" in the 2D diameter. If I want to use 2D diameter do I need to consider some other parameters (e.g. off distortion axis and so on) or I can use them as is.

I must have eye0.mp4 and eye1.mp4 for post-hoc. Correct?

Because eye0 and eye1.mp4 weren't recorded, the extract size of the pupil in mm must be forgotten. Correct?

Thank you. Is there any tutrial or example of what information should I push to the pye3d-detector?

I have another question regarding the export of data. Is there an way to call the export function from my own code without a need to use the gui (pupil player)? def export(rec_dir, user_dir, min_data_confidence, start_frame=None, end_frame=None, plugin_initializers=(), out_file_path=None, pre_computed={}): :::: from exporter.py

What I now noticed is that I need these corrected previous gaze points, which are computed in the recent_events function of vis_scan_path.py, if I want to draw lines between the gazes from the last few miliseconds for visualisation purposes