Hi Pupil Labs team! Quick questions: are there any parameters in the Reference Image Mapper that can be changed to make the detection/mapping to the reference image more sensitive? This could be useful for some of my recordings. Also just wondering if the Reference Image Mapper code open-source? Would be cool to play around with it and fine-tune paraments for my use-case. Thanks and keep up the good work!

Hello, can we use a one plus 10 with a Pupil Invisible?

thanks Nadia – I saw that Neon is compatible with One plus 10, was wondering it the documentation was up-to-date. Anyway thanks!

Hi Pupil Team (@user-480f4c and @user-d407c1 ) I have a doubt regarding AOI generation. I referred to the code (https://docs.pupil-labs.com/alpha-lab/gaze-metrics-in-aois/#map-fixations-to-aois) but does it consider the following:

1) What if a participant is moving their head while looking at poster as AOI generation is based on position/coordinates?

2) Does it consider if participant move/walk slightly and analyses the photo/poster etc?

If yes then could you explain which part of code takes care of that? If not, then is there a way to incorporate this?

Hi @user-2b79c7 👋🏽 ! The AOI tutorial works with the data exported after applying the Reference Image Mapper enrichment on Pupil Cloud.

This tool essentially allows you to map gaze data of a 3D environment (e.g., your eye-tracking recording while looking at paintings in a gallery) onto a 2D image (e.g., a photo of the paintings). The mapping applies only when the 2D reference image is visible to the scene camera of your eye-tracking recording.

As you see in the data exported from the Reference Image Mapper, there is a column "fixation detected in reference image" with True/False values that indicates whether the image was detected in the video and by looking into the fixation coordinates (fixation x [px] and fixation y [px]) you can get information of where your participant was looking at a given time point (you can find this info in the table of fixations on the AOI tutorial page: https://docs.pupil-labs.com/alpha-lab/gaze-metrics-in-aois/#map-fixations-to-aois). Therefore, to answer your questions, you will obtain fixation information even if the participant is moving slightly their head, as long as the reference image is detected in the scene camera.

I encourage you to have a look at the Reference Image Mapper documentation to see more examples of how this enrichment works: https://docs.pupil-labs.com/enrichments/reference-image-mapper/

I hope this helps, but please let me know if you have more questions!

Hi! I am searching for a good blink detection alrogithm for my experiment. The participants, however, make quite some vertical eye-movements during the performed task. Therefore, I am a bit hesitant to use your blind detection. We also used the pupil player to get the data from the pupil invisible, since we are not allowed to upload our data to the cloud. I look around for some alternative options, but have not found a solution yet. Do you have any recommondations?

I read that vertical eye-movements may cause many false positive blinks (instead of saccades) with the current version of the blink detection algorithm

I'm not sure I fully understand your question. Let me explain the workflow of how you can use the AOI tutorial:

1) Apply the Reference Image Mapper enrichment on Pupil Cloud.

• This tool basically maps gaze from your eye-tracking recording onto your reference image. If the mapping is applied correctly (see our best practices: https://docs.pupil-labs.com/enrichments/reference-image-mapper/#scanning-best-practices), it should work fine even when you move, i.e., even when the location of AOI changes in the scene camera coordinates.
• In the example video we have in the documentation (https://www.youtube.com/watch?v=IF8f1Z3ZkEo&t=7s), you can see that the AOI as defined by the reference image is sometimes visible and sometimes not, and its location changes in the scene camera depending on the angle that the wearer looks at it. The mapping of the Reference Image Mapper is not affected by this, and you can get the csv file with fixation and gaze mapped on the reference image. Please have a look at the exported data as well: https://docs.pupil-labs.com/export-formats/enrichment-data/reference-image-mapper/

2) Run the AOI tutorial. - This will first present you with the reference image on a pop-up window and then you will be able to define manually your AOIs. - After the AOIs are defined on the image, you will be able to run the remaining code snippets and calculate metrics per AOI.

Therefore, the metrics for the AOI are calculated based on the reference image, rather than the video.

@user-2b79c7 thanks for clarifying. Some points below:

• For optimal mapping output using the Reference Image Mapper, you would need a scanning recording that is similar to your eye-tracking recording (e.g., in terms of scene lighting, features etc.).
• Additionally, the scene needs to have relatively static features in the environment. If there is a lot of movement or the objects change in appearance or shape (like in your motorbike example), the mapping can fail.
• Specifically for the screen example you mentioned, we do offer a tutorial that allows you to use the Reference Image Mapper enrichment and then map and visualise gaze onto a screen with dynamic content, e.g. a video. You can find it here: https://docs.pupil-labs.com/alpha-lab/map-your-gaze-to-a-2d-screen/

Hey! I recorded some videos (not on wifi) but then was able to get on wifi - they should upload correct?

Hi @user-91d7b2 ! Yes! If you have internet access and Cloud upload was enabled, they will start uploading when connected. Is it not the case? If so, would you mind performing the following test https://speedtest.cloud.pupil-labs.com/ to ensure you have enough upload speed to our servers? (some company/institutional networks may have limitations on upload). Also please have a look on the Recordings view of the app in case a recording is stuck.

Upload speed seems fine!

Thank you!

Hi Pupil Team,

I recently followed the tutorial at https://docs.pupil-labs.com/alpha-lab/map-your-gaze-to-a-2d-screen/#running-the-code, and I encountered an issue during the process. After uploading the data and selecting the screen corners, I received an error message in the "dynamic_rim.py" file on this line:

for packet in out_video.encode(out_frame):

Upon inspecting the code, I found that the error occurs when the "out_video.width" parameter (line 418) is an odd number. This leads to a program crash.

To resolve this, I manually adjusted my reference picture dimensions from 1600x1204 to 1599x1204, which worked. However, I'm wondering if there's a better and automatic solution to this problem.

Any guidance or suggestions you can provide would be greatly appreciated.

Thank you, Maayan Bahar

Hi Nadia, I have a few different task, such as cycling in a urban environment, walking in a shopping street. But I think that the task with most vertical eye movements, is a task were my participants have to walk in a hallway and avoid some obstacles on the ground while also detecting targets that are placed on the walls of the corridor. Here, the participants have to make a lot of eye-movements including vertical to see both the ground objects and the targets that are placed more on shoulder level.

Hello! I'm doing image reference mapper in Pupil Cloud on a one minute image. It's been almost three hours and it's still running. Is this normal?

Also, I have a Pupil Invisible and I'm trying to export my recordings. I don't have the gaze.csv in my exported files. Is this something I have to generate manually?

Thank you!

Hi, I'm not sure if this is the right place for my question... We are working with a pupil invisible. However, the invisible creates multiple small videos of different periods in time as it seems. Different segments appear to be indicated by an increasing index. I couldn't play it back in the Pupil Player.

Hi @user-e40297 ! It is normal for the Invisible Companion to split the video onto different video parts, one the reasons for this, is the to size limits on Android. Given that you plan using Pupil Player, I assume you do not want, or you are not allowed to use Pupil Cloud, right? Any specific reason?

Nevertheless, both Pupil Player and Cloud can handle multi-part videos. So, do you see any error while loading the folder to Pupil Player?

Also, and just to be 100% on the same page, are you exporting the recordings from the phone as explained here https://docs.pupil-labs.com/invisible/how-tos/data-collection-with-the-companion-app/transfer-recordings-via-usb.html ?

Our institute is still unsure about using the cloud based on privacy issues... I'll look at your suggestion and ask my colleague

Pupil Cloud is GDPR compliant and follows standard security measurements, you can read more about it here. https://docs.google.com/document/d/18yaGOFfIbCeIj-3_GSin3GoXhYwwgORu9_7Z-grZ-2U/export?format=pdf

What do you mean by map? is the structure of the folder different? Might it be that the folder copied onto the computer is not the Exported one (Documents/Pupil Invisible Export/foldername) but rather the one at Documents/Pupil Invisible/foldername ?

The original recordings made by the Invisible Companion App are located in Internal Storage > Documents > Pupil Invisible, but those are in RAW/binary format. To make them compatible with Pupil Player, you have to :

Export:
For single recordings, the export button is found by clicking on the 3 vertical dots to the right of the cloud symbol
For multiple recordings, click the download symbol at the bottom of the screen

As outlined in the prev. link I shared: https://docs.pupil-labs.com/invisible/how-tos/data-collection-with-the-companion-app/transfer-recordings-via-usb.html , this would generate a new folder in Internal Storage > Documents > Pupil Invisible Export, which is the one you have to copy to the computer.

Please ensure that you follow these steps and let us know if you can still not load it into Pupil Player. If not, please let us know what error message Player prompts you with or share the Pupil Player log file.

Hi @user-2fc592. Responding to your message https://discord.com/channels/285728493612957698/285728493612957698/1161142714696478740 here.

Unfortunately, we don't have a method to correct gaze offsets in Pupil Cloud post-hoc. The current workflow requires these to be set in the Companion App before recording.

May I ask which analysis tools you are using in Cloud or iMotions software? We do have a method to correct gaze offsets for Invisible recordings post-hoc, but this requires our desktop software. Once done, there is no way to reimport them to Cloud or iMotions, which might somewhat limit your analysis.

hello，is the script ready？

Can I ask how long it's supposed to take for the reference mapper to run on a 20 second video?

My reference mapper keeps failing and giving me errors. I thought I followed the instructions. Is it okay if I can talk to someone directly to ask some questions? Thank you!

Reference Image Mapper fail

Hello, I have been trying to make adjustments on the raw data files, then re-calculate the gaze_on_face, however, I couldn't not align the face_detections.csv timestamps to the gaze.csv timestamps. Is there a script for that? or how can we get info to correctly align the two?

thanks @user-c03258 I am having a similar issue, maybe @nmt or @user-d407c1 have ideas? Thanks

Hi @user-c03258 and @user-8a677a ! The face detections csv file contains the timestamps of frames from the scene camera where a face is located. The scene camera goes at a frame rate of 30FPS, while gaze is at 200Hz, meaning there's not a exact match. What you would need is to do is for each gaze row, find the closest scene frame using the timestamp column.

One way to achieve this, is using the pandas function merge_asof, like it is done at the densepose library https://github.com/pupil-labs/densepose-module/blob/main/src/pupil_labs/dense_pose/main.py#L157 or in our tutorial on how to sync with external sensors https://docs.pupil-labs.com/invisible/how-tos/advanced-analysis/syncing-sensors/#match-video-and-heart-rate

that's very helpful! thank you!!

@user-d407c1 thank you!

Thank you for your reply, Neil. I have tried to use the Enrichment tools like the reference mapper and others, but none have proven successful.

@nmt I used real time api to send events to my invisible during a recording: https://docs.pupil-labs.com/invisible/real-time-api/track-your-experiment-progress-using-events/. But when I loaded the recording in Pupil Player, there is only option to export annotation, not the events. I think Pupil Cloud might help me to extract the event information, but I can't upload the data to the cloud. Is there any way that I can extract the event information?

Hi there - I'm trying to export a video from my workspace so that I can edit it side-by-side with a static camera that is showing what the wearer is doing from a distance. When I do that, the scene information (gaze, fixation etc) isn't overlaid on the video. Is there a way for me to export in a way that will let me keep that information 'on' the video? Thanks in advance 🙂

Hi @user-6c7210 ! You will need to generate a video renderer, you can find this option when you create a project, under the analysis tab > "+ New Visualisation"

Hi, some begginer questions... I´m tryingo to export video from Invisible glases. There is no way to do it directly from the phone (with the gaze circle)? I´ve found the Pupil Player, but my recordings are corrupted. - Only1 of 3 captured both eyes. - the one with both eyes is 12 minutes long, but the world wideo is not complete (approx 4:30 is OK), it´s runnig in the Invisible companion (the first part), itˇs running on the cloud, but the player crashes.

What to do?I need this video! thank you. Jan

Hello, I was wondering if there is an example of how to use the Invisible with Unity for real-time input (mapping the gaze input to screen coordinates). I would be happy for any help 🙂

hey👋, does anyone know how to check if a surface is observed in pupil capture via python? Any help would be greatly appreciated 😄

Hello @user-480f4c ! I have a question regarding the synchronization of the gaze cameras both with each other, and with an outside event, which is the clapping of a clapboard. We managed to add an "Event" in the Project Editor in the Pupil Cloud, which we can find in the events.csv file. However we had trouble moving frame-by-frame in the video player to precisely locate the clap. Left/right arrows move by 5 seconds. We were advised to use the , and . keys to scroll left and right frame-by-frame, which is not working. Another question concerns whether speed with which the enrichments are processed depends on your server or the server on which our device is. Many thanks in advance 😊

Hello, just getting started here with Pupil Invisible. 1. Is there a way of exporting a simple video file that displays the World Camera view along with Gaze and Fixation? We were getting ready to try screen recording the playback window of Pupil Cloud as a workaround, but it would be great if we had a simpler way to export this.

1. Is there a way to reduce the number of fixations that remain on the screen during playback? This would be helpful to us to keep the screen less cluttered.

Thanks!

In Pupil Invisible, is there a way to adjust the milliseconds threshold of detection of a fixation? I'm trying to adjust the system so that fewer fixations appear on the Fixation Overlay, to keep the screen less cluttered while we examine moving text that is being read by the subject.

Hi Pupil Team, we have a pretty urgent issue here, as we are conducting an experiment today. The Pupil Invisible don't connect to the controller app (when it is plugged in, the scene and eye cameras do not register, and stay greyed out). Everything was working smoothly yesterday, and nothing notable changed about the setup since then.

We’ve tried the following troubleshooting options already; - restarting the companion device - using alternative USB-C cables - testing the black/other cables on other devices (all work fine) - testing the cables on the companion device (all work to charge the device) - unplugging/replugging the cables between the glasses/device - removing/reattaching the scene camera from the glasses

The specs are as follows: - unit = pupil invisible - companion = OnePlus 8T - software = android 11 - build number = oxygen OS 11.0.6.9.KB05BA - app = 1.4.30-prod

Any help/suggestions would be highly appreciated!

Debugging - cameras not working

Hi, I am currently using the invisible device for research purposes at university but the Neon device will help my research better, is it possible to swap ? This will help my progress massively.

I have problem with downalding one record

also cant find this file on mobile

Thank you!

Nadia, could you please give me a detailed description of how to do that?

When I go to download raw data, I do see Scene Video on the list, but I don't see Timeseries data or a fixations.csv file. Am I maybe looking in the wrong place?

Thank you. That way of downloading worked. If I wanted any fixations below 200 milliseconds to not be displayed, would I basically delete the entire row in Excel from the spreadsheet for each fixation that I didn't want to display?

Dear Pupils Labs Team. I finished the data recording in a field study, in which we investigated eye-tracking parameters in the urban environment. Two questions came up in this process:

(1) I have a recording, which was uploaded to the cloud. This video contains sensitive information, so I wanted to cut off the last 10 minutes of the recording. Is it possible to cut the video in the cloud? I dont want to loose the data by deleting the entire video ...

(2) do you have any background information, regarding the sensitivity and accuracy of the pupils labs invisible? This would be really helpful to argue for the removal of outliers in the recordings. In other words, how accurate is the device in detecting a fixation or an eye-blink? Was this ever tested?

Best regards Anton

Thank you so much! The papers are super helpful 🙂 I upvoted your suggestion

hi, i would like to know if anyone of you has the documentation of Pupil Invisible, i need to understand the type of moviment that can be registered

Hi I am having some problems with my invisible gaze tracker. The red circle was not showing the gaze anymore while wearing the glasses and was stuck in the left corner when attempting to reset the calibration. I do not even see a red circle (the gaze) on the live preview. I wonder what is going on? Any ideas? I had to cancel my experiment and I do also have some planned tomorrow.

sent! thanks @user-480f4c