I am probably going to ask something very stupid...

I am working in Win 10, I want to modify gaze_from_offline_calibration.py I guess I can make a plugin, correct? I copied the .py, renamed it to MY_gaze_from_offline_calibration.py, and put it in the plugin folder. Then I open Pupil Player, load a recording session and I have no errors, but I don't see the plugin on the right...then if I try to change selection from the popup menu, it crashes...

File "launchables\player.py", line 630, in player File "pyglui\ui.pyx", line 199, in pyglui.ui.UI.update File "pyglui\ui.pyx", line 169, in pyglui.ui.UI.draw File "pyglui\menus.pxi", line 603, in pyglui.ui.Scrolling_Menu.draw File "pyglui\menus.pxi", line 626, in pyglui.ui.Scrolling_Menu.draw File "pyglui\menus.pxi", line 653, in pyglui.ui.Scrolling_Menu.draw_scroll_window_elements File "pyglui\menus.pxi", line 461, in pyglui.ui.Growing_Menu.draw File "pyglui\menus.pxi", line 477, in pyglui.ui.Growing_Menu.draw File "pyglui\ui_elements.pxi", line 395, in pyglui.ui.Selector.draw File "pyglui\ui_elements.pxi", line 437, in pyglui.ui.Selector.draw File "pyglui\pyfontstash\fontstash.pyx", line 102, in pyglui.pyfontstash.fontstash.Context.draw_limited_text (pyglui/pyfontstash/fontstash.c:3050) File "pyglui\pyfontstash\fontstash.pyx", line 19, in pyglui.pyfontstash.fontstash._to_unicode (pyglui/pyfontstash/fontstash.c:1325) MemoryError

player - [INFO] launchables.player: Process shutting down.

Do I need to change anything else?

Thank you for the help!

Hey, I wanted to check if anybody had built a preprocessing pipeline for the pupil diameter data in Python. I've seen some in matlab but no python so far. Thanks!

Hi @user-97997c there are no stupid questions :) There are generally two workflows for modifying/extending Pupil: creating new Plugins and modifying Pupil source code. We recommend to create new Plugins mostly, since this is the easiest and will potentially enable you to re-use the plugin in newer versions of Pupil. Did you also rename the plugin class inside of your copy? Otherwise there might be naming clashes.

Regarding the crash, a quick google search indicates that this could be a problem with special characters in the path to pupil or the plugin. Does your usename contain any special non-english characters? Please try and create a user account with only english characters in this case.

Also on a side node: we are currently reworking the pupil and gaze pipelines inside of Pupil, such that they can be extended/modified more easily. We are almost finished with this rework and plan to release a new version of Pupil in the next weeks. I would recommend you checkout the latest version on the develop branch to take a look at how the new pipeline works. There's a chance the new version might break compatibility with plugins that rely on core parts of the pupil codebase.

@user-2be752 did you check the Pupil Community repository? I don't know if any of those can help you, but might be worth pointing you towards it :) https://github.com/pupil-labs/pupil-community

@user-2be752 I worked with the Pupil about 2 years ago capturing the diameter data, I didn't touch this code over 2 years now but it might be helpful for you: https://github.com/yurikleb/pupil_cognition/tree/master/pupil_recorder

Hi all, to get more insight in how our pupil is moving before we move our head, I wanted to generate a heatmap of a large dataset which represents the turning angles of the eye (teta and phi). For this purpose I want to use the following dataset from MPI which is recorded with pupil labs core: https://www.mpi-inf.mpg.de/departments/computer-vision-and-machine-learning/research/human-activity-recognition/discovery-of-everyday-human-activities-from-long-term-visual-behaviour-using-topic-models/. The dataset contains the normalized xy-coordinates of the pupil and the gaze. But no calibration data. My question is now: is there any possibility to transfer them back into a 3D representation to be able to calculate the eye angles? I tried already out the unprojectPoints-function (https://github.com/pupil-labs/pupil/blob/master/pupil_src/shared_modules/camera_models.py). But I still don’t get a relation between those two planes (pupil and gaze) right? Probably it is easy, but I didn’t come to a good solution on my own.

Hi @user-f40782 does the dataset contain the actual recordings, or just the exported 2D pupil+gaze data? If you just want the 3D pupil data, you could run the offline pupil detection again on the eye videos. There is no way to transform 2D pupil or gaze data into 3D pupil data without the eye videos.

No unfortunately the dataset doesn't contain the actual recordings. I was afraid it wouldn't work. Thanks for your answer!

Hi guys, does pupil work with a computer integrated webcam?

Hi @user-151c9e , please see my corresponding answer in the 👁 core channel

Hi! I'm trying to use fixation detector (fn - https://github.com/pupil-labs/pupil/blob/master/pupil_src/shared_modules/fixation_detector.py#L157-L244) to detect fixations & output the same csv as pupil player-->export--> fixation.csv. Could I get clarification on the capture & gaze_data parameter inputs & if I'm calling the correct function please?

Hi @user-ddc8dd, just to calrify: do I understand correctly that you want to use the linked function outside of Pupil in your own codebase?

In that case you can see here how we execute the function: https://github.com/pupil-labs/pupil/blob/346ccaf71ed6a799ddd593f38c2918704384d7e0/pupil_src/shared_modules/fixation_detector.py#L457-L477 where cap is a mocked capture object containing the frame_size and camera_intrinsics of the world cameras as well as the timestamps corresponding to the gaze_data.

Also please note that detect_fixations returns a generator object. You can easily get a list of the fixations with

fixations = list(detect_fixations(...))

Yes, thank you @user-c5fb8b! When calling list(detect_fixations(...), it seems to be outputting the fixation info in bytes, so how can I retrieve the same values in a dataframe similar to pupil player's fixation export?

@user-ddc8dd The fixation dictionary is serialized with msgpack because we are running the fixation detecion in a background process and you can only transfer serialized data between processes. If you are using the code in your own codebase, I recommend you just skip the serialization by removing this line: https://github.com/pupil-labs/pupil/blob/244661b19c71d105ef2f6946b5b014c0682734f3/pupil_src/shared_modules/fixation_detector.py#L115

That worked, thank you! I've also been running into "MemoryError: Unable to allocate array with shape (10097981328,) and data type float64" It keeps occurring when calling gaze_dispersion in detect_fixations (line 214) --> vector_dispersion (line 152) --> pdist (line 131) https://github.com/pupil-labs/pupil/blob/346ccaf71ed6a799ddd593f38c2918704384d7e0/pupil_src/shared_modules/fixation_detector.py#L214 It seems like an important part of the calculation...any advice on how to get around the MemoryError?

Hi, I don't see documentation for all the different files that pupil-labs outputs from a recording. What do the ".time", ".time_aux", ".intrinsics", ".bin", and ".raw" files contain and how do I read from them?

@user-ddc8dd If I see this correctly, you are trying to allocate an array of float64 with 10097981328 entries. This roughly equals 80GB of memory being allocated in your RAM, which most likely you lack the resources for. Are you really using such a big dataset? If so, you should probably split it into multiple smaller parts for processing.

Hi @user-3f2d35, normally you would not need to read from the files in the recording folder, but use the export function of Pupil Player to transform relevant data into csv files. There is no documentation for the internal files in the recording folder, but we can explain how to read from them in case you have a special use case that requires this. What are you trying to do? Would the csv export be sufficient?

Thank you for your response! I am trying to coordinate the editing of the raw data so that we can remove a portion of the data and still have everything synchronize.

Thank you so much for all the help @user-c5fb8b! I think I'm really close, but my output for detect_fixations() is still not matching pupil player's output. I think it has to do with gaze_data parameter. I'm calling load_pldata_file() from file_methods.py to extract gaze.pldata and assigning this to the variable, gaze. Then, gaze_data = [gp.serialized for gp in gaze.data]. Am I misunderstanding what var gaze.data should be here? Or gaze_data in general?

Hi Pupil - I am getting the same error as described here: https://github.com/pupil-labs/pupil/issues/1819 ... was there a resolution to this issue? My data was captured using offline calibration and I currently have the Raw Data Exporter and Eye Movement Detector plugins enabled. Any information on resolving this error would be amazing. Thank you so much!

@user-2143a5 if you are using the offline calibration, that you might have overlapping gaze mapping sections. Simply adjust the gaze mapping range or delete the duplicated gaze mappers in the offline calibration menu.

@user-ddc8dd this looks correct. How are you matching your output to players output?

@papr I used enumerate() of gaze.data output from load_pldata_file() to compare gaze.pldata to pupil player's export's gaze_positions.csv & found that there were discrepancies by comparing timestamp with the associated data. Although some entries perfectly matched, there were additional ones, entries missing, & others where the confidence matched but norm_pos wouldn't match. A similar discrepancy is happening when comparing our fixations (extracted through detect_fixations()) to pupil player's export's fixations.csv, so I thought it may have something to do with what I'm inputting for gaze_data parameter.

@user-ddc8dd Please be aware that you should compare gaze positions based on base data. The base data is either one (monocular gaze point) or two (binocular gaze point) pupil data keys, where a pupil data key is a tuple of timestamp and eye id.

thank you @papr!

@papr Thank you!

Howdy! I'm trying to put together a simple custom choreography for screen marker calibration / accuracy testing. Is there documentation somewhere for how I would do this? My naive approach would be to subclass ScreenMarkerChoreographyPlugin and override __get_list_of_markers_to_show() to return a list with different coordinates. When I do this, though, my overriding method never gets called. I expect I'm probably making a mistake with some basic feature of Python inheritance; any help would be greatly appreciated. Thanks!

@user-6bc565 The double underscore makes the function private. You are not doing anything wrong. Is the marker placement the only thing you want to change? In this case, we can make this function accessible for subclasses in our next release. As a work around, you can copy the complete ScreenMarkerChoreographyPlugin and modify it as necessary

@papr Yes, the placement is the only thing I'm trying to change. Thank you! Will do.

@user-6bc565 there is an other work around: overwrite def _ScreenMarkerChoreographyPlugin__get_list_of_markers_to_show(mode: ChoreographyMode) -> list

The __ prefix causes the function to be renamed as stated above. This language feature is meant to be used to avoid overwriting unknown super-class features by accident. In this case, this is definitively something someone would want to overwrite.

@papr Thanks again! That's exactly the explanation and solution/feature I needed. Works as intended!