Hi! I'm pretty new to pupil labs and an intern in a research facility. My advisor said that I should learn how to work with pupil labs. Now we're already have troubles to view/play the recordings in pupil cloud. Although it seemed to worked some time ago. Any tips?

Hello, I have an issue with Pupil Invisible. I don't see the gaze/red circle anymore after the recording and during the calibration. Why? How can I fix it? thanks

Hi @user-092531! Welcome to the Pupil Labs community 🙂 Replying to your message about eye tracking underwater: https://discord.com/channels/285728493612957698/285728493612957698/958790379539415040 This would be a fantastic development! What product are you currently prototyping with? Would it be possible to see an image of the underwater hardware you are developing? You mention "lens over the camera as part of the water proof enclosure" Is this completely sealed and watertight?

It works now! Crazy, may I know what the issue was?

Hi Richard, we are doing the development with the pupil core. Can't give you a photo of the hardware just yet but will be happy to once it's finished. It is completely watertight.

Hi @user-1abb3f Replying to your message from the Core channel here https://discord.com/channels/285728493612957698/285728493612957698/959555127507828826 Pupil Invisible connects to the OnePlus 8T mobile Companion Device that ships with the glasses. Please use the USB cable provided. The companion device runs the Invisible Companion App - which is exclusively used to record and/or stream gaze behavior. You cannot record data by connecting the glasses to a laptop.

really thank you for your help, there's no error report during the running process now, but I still got an issue is that I can't completely output the full-length video after running the code, is there any suggested way to deal with this situation?

https://docs.pupil-labs.com/invisible/how-tos/integrate-with-the-real-time-api/introduction/ has a couple of broken links in it, just in case it's not known.

https://docs.pupil-labs.com/invisible/how-tos/tools/monitor-your-data-collection-in-real-time https://github.com/pupil-labs/pupil-docs/src/invisible/how-tos/integrate-with-the-real-time-api/introduction/ https://docs.pupil-labs.com/invisible/how-tos/applications/track-your-experiment-progress-using-events

thank you!

Hi everyone, my university got Pupil Labs Invisible for a scientific paper. In this one we compare the duration needed to look from one area marked by Aruco markers to another. My question would be, since I am relatively new to working with eye tracking, if there is already an applicable script for this, or one that could be built upon.

Hi @user-f408eb! Did you see the Marker Mapper enrichment in Pupil Cloud yet? Its using AprilTag rather than Aruco, but it is used for exactly the purpose of tracking surfaces with markers and mapping gaze onto them. The export of that enrichment should make it easy to detect and time transitions.

https://docs.pupil-labs.com/invisible/explainers/enrichments/#marker-mapper

thank you for your reply, the situation is like: the full recording is 15 min, but the output avi is only 6 min

@nmt Hi again Neil. Accurate eye tracking is what we need to make our treatment of eye cancer non-invasive. We currently have 4 cameras directed at the patient during treatment. Do you think we can use our information in combination with your network for the estimation of gaze angle?

I see. To integrate your network with our cameras would require some modification of the network, correct? Is there any way we could get to look at your network? We might be able to assign some professionals in adapting your network to our needs given the opportunity

Is there a way to use the pupil invisible for multiple hours? For example, is it feasible to use a USB-C splitter to charge the phone and have the eye tracker connected?

Yes, this is possible using a powered USB-C hub! The hub you use must be OTG compatible and even then some hubs are not compatible with some versions of Android. But we have tested this successfully with e.g. this hub: https://www.amazon.de/gp/product/B08HZ1GGH9/

Hi. When I use raw data export on a recording made in Pupil Invisible, I am looking at the gaze.csv file. I find that some values of gaze x[px] and gaze y [py] are negative. Why does that happen? I am looking at https://docs.pupil-labs.com/invisible/explainers/data-streams/#gaze for the coordinate system.

Hi Marc. My minimum value was -101.9. Is this normal? I tried manually drawing the marker on the scene video using OpenCV but it doesn't seem to work. Should I share my data/code?

Hi, is it possible that you set up offset correction for a previous subject/wearer?

@user-413ab6 In that case it would be great if you could share a recording with [email removed] Then we can take a look if everything looks ok and if offset correction might be a solution!

Sure, will do. Thanks!

Thank you! That worked. Can you also help me with my other query? (the code)

Hi, I would recommend discarding these out-of-bounds data points from your visualization.

Yes, I have checked, thanks. I notice that some users don't face this issue. For me, the offset is large. Then again, I have nystagmus and a host of other eye problems. That may be affecting it!

We need urgent help. We are collecting data using pupil invisible. It was working perfectly. But today, suddenly, it stopped recording. Here is the scenario: we started the recording, and it recorded for approximately one minute, then it stopped working and gave the following error (please check the screenshot). Here are things we tried: we uninstalled and installed the pupil companion app, restarted the device couple of times, and disconnected and connected the pupil invisible glass multiple times. Could you please let me know what the issue is? We had to stop the data collection session for this reason. As we are in the middle of collecting data, any urgent help is appreciated.. Thank you very much.

URGENT

Hi @user-d1072e Confidence values from 0.0-1.0 reflect the certainty of pupil detection for a given eye image with Pupil Core. Pupil Invisible, in contrast, does not rely on Pupil Detection. The eye cameras are off-axis and for some gaze angles, the pupils aren't visible. This is not a problem for the gaze estimation pipeline as higher-level features in the eye images are leveraged by the neural network.

Ah OK, it makes sense now. Thank you both a lot for the explanation

Hello, In Pupil Cloud after creating a new project and a new enrichment, the enrichment does not load, the blue bar remains white (even after several hours for a 10 minute video). Do you know how to solve this problem ? Thank you in advance

Hi. I would like to record the GPS data in Invisible Companion. This doc https://docs.pupil-labs.com/invisible/explainers/data-streams/#gps says that it is there in the settings, but I can't find the setting. Can you help me?

Good evening, I'm working on integrating a pupil invisible eye tracker with eeg (brainvision liveamp). I've found that LSL might be the way to go for time sync and centralizing data gathering. I am using this:

https://pupil-invisible-lsl-relay.readthedocs.io/en/stable/

however, the device is not found. I have the oneplus8 device connected to the same wifi network, the companion app is open and glasses are plugged into the phone. I've verified my connection works as I am using a code example related to extracting surface gaze data in realtime. Is there something I'm missing in my configuration to connect PI tracker to LSL? Thank you in advance.

Unfortunately the docs are a bit ahead of their time 😬 While working on the recent doc update this was already written, but it is not yet implemented in the app and we have mistakenly released that! While it is on the roadmap, an update on this for the app is also not planned within the next couple of weeks. I know there are other 3rd party apps that can track GPS on the phone though, but I can't recommend a specific one.

Sorry for the inconvenience! I will remove the GPS section from the docs now!

@marc hello again! According to the documentation gaze data that is not in the boundaries of the markers is not interpreted in x y coordinaties. Is it really possible to export the time span needed for the transitions from one level to the other defined with markers? Also does the invisible support Aruco markers?

Hello @papr , item 1) is what I'm having issue with.

Sure, it's my private (home) network.

That's correct

Unfortunately, we do not have any logs in the relay yet. To debug this, we will need to run some custom code. Could you please run this example in your terminal? https://pupil-labs-realtime-api.readthedocs.io/en/latest/examples/async.html#device-discovery

I would need you to add the following lines in line 25 (after the if __name__ ... part):

import logging
logging.basicConfig(level=logging.DEBUG)

and share the logs printed to the terminal with us.

This is what I have:

DEBUG:asyncio:Using proactor: IocpProactor Looking for the next best device... DEBUG:pupil_labs.realtime_api.discovery:ServiceStateChange.Added myrouter._http._tcp.local. None

All devices after searching for additional 5 seconds: ()

Starting new, indefinitive search... hit ctrl-c to stop. DEBUG:pupil_labs.realtime_api.discovery:ServiceStateChange.Added myrouter._http._tcp.local.

Ah, thats exactly what I meant. Please excuse the confusion. So if the duration of the transition of the gaze from one surface to the other is to be determined, is this possible with the enrichment? Even if the gaze is reported between the surfaces with values outside the range?

Thank you very much!

That did it...I guess I didn't have auto-update on the companion app. Eek. The device is found now - thank you so much @papr

Hi there, I have a question regarding saccades please. I am using Pupil Core to record eye movements and am interested in the saccade count (number of saccades) and saccade duration. I have read that I can consider that the time during two fixations is a saccade - but what about the blinks ? How can I take those into account / out of the equation to calculate my saccades ? Thanks in advance. Joanna

Hi @user-ee70bf! With Pupil Core this is a bit more complicated than with Pupil Invisible: - Pupil Player does have a blink detection algorithm which you can use to detect blinks. It is less reliable though than the algorithm for Pupil Invisible. - The fixation detection algorithm does currently not take blinks into account and thus blinks may lead to false detections of fixations endings during the blink. - The fixation detection algorithm of Pupil Core does also not compensate for head movements, so for robust detection a setting with little head movement is required. Otherwise small VOR movements will lead to false detections. - There is no detection for smooth pursuit movements, so scenarios that include a lot of those will also lead to false detections.

Besides those points though, one can consider all gaze samples that do not belong to a fixation to be saccade samples.

Thanks @marc for all of those precisions ! This is much clearer. Therefore, I do not need to "substract" the blink durations to calculate my saccades ?

Your welcome! As @papr mentioned the blinks should actually already be compensated in the fixation detection (my initial statement was a bit inaccurate), so no subtraction should be necessary. The potential issue is that some blinks might not get detected and thus not compensated. This could only be fixed by manually annotating the blinks.

(This also assumes that one is not able to blink during a saccade. @nmt do you have further insight into the correctness of this assumption?)

Thanks for the correction! So, assuming we have this case: Fixation -> Saccade -> Blink -> Saccade -> Fixation. Would you count this as one saccade? If yes, would you subtract the blink duration from the saccade duration? Or more interestingly: What would be your approach regarding handling blinks between fixations?

Thank you very much for your insight! @user-ee70bf Does this give you enough input to continue your work?

Hi there, I have a question regarding the remote connection please. I'm using Pupil Invisible and I need to start the registration data with Matlab, using the remote connection. I found this https://github.com/pupil-labs/pupil-helpers/tree/master/matlab, but I have big problem with zmq package. can you help me? Thanks in advance. E

OK, clear, thank you! I need to do an experiment on product packaging using Eye tracking and EEG. Since the researchers I am working with created the project on Matlab, I wanted to add Pupil Invisible. If you need more details or if I didn't explain myself well, I am here.

While in Python I can have eye tracking data in realtime? Because I could use Python and afterwards call it in Matlab

Yes, I wanted to record the data in Matlab in real time. I need to do an experiment where participants observe images for seconds on pc monitor and I want to record their eye movements during the observation. I would like to have a csv file with the fixation coordinates and timestamp, without recording the video and uploading it to the Cloud.

Yes exactly, I'm interested in the coordinates of the pc monitor. Do you recommend me to record the pc screen video and then upload it to the Cloud? The only doubt I had was about the timestamp. Because if I could start the Pupil Invisible inside Matlab the timestamps were consistent. If I start the video there are seconds of difference between my project in Matlab and the recording of the video.

Ok thank you so much for the valuable information and the link you sent me, I will look into it. If I have other doubts I will write to you.

Yes, thank you! You would be doing me a great favor! I wanted to start my experiment in June. Let me know if it's too soon, please.

That is plenty of time!

You might want to start thinking how you want to get the data from Python into your Matlab. Easiest would be storing the data into a file and reading that post-hoc into Matlab.

You can do that via the Python client. Maybe this is an option: https://uk.mathworks.com/products/matlab/matlab-and-python.html

Is it still true that the IMU and gaze data are subject to non-constant temporal offset? There is latency between the two by an amount that changes over time?

Hi Guys, we are trying to sync data from Azure Kinect and pupil invisible. Could you please suggest what is the best way to do that? Any pointer is highly appreciated. Thanks.

I don't know about best, but would writing a piece of software, which receives sync events via the network api suffice? https://docs.pupil-labs.com/invisible/how-tos/integrate-with-the-real-time-api/introduction/

Hi @user-ee70bf This sounds like an eye movement reinstatement paradigm, where fixation-saccade sequences are specifically reproduced when recalling a past experience from episodic memory. If so, I am wondering if analysis of saccades is even necessary. Typically it is the relative direction of eye movements between fixations which is important. See this paper: https://psycnet.apa.org/doi/10.1037/a0026585.

For more info on saccades in mobile eye tracking compared to other contexts, I can highly recommend this article: https://royalsocietypublishing.org/doi/10.1098/rsos.180502

We have a Pupil Invisible which currently runs into problems. Near the connection of the world camera the glasses are overheating. We expect there is a short-circuit inside. Please advise of how to fix this. At this moment we do not dare to open the glasses ourselves (yet) for inspection.

Please contact info@pupil-labs.com in this regard.

Hi Guys, We recorded approx 600 videos, which are uploaded to the pupil cloud. We need to download the PI World Video (Egocentric) of all these recordings. We can download each recording individually and extract the zip file to get the egocentric video data. This approach is very time-consuming. We are wondering is there any easiest way to download all the egocentric video data from the cloud recordings? Thanks.

I think downloading using API will be convenient for us as I think we still need to add each video manually to the project. Could you please provide resources on this? Suggestion: I think it would be great if the cloud has the feature to select and download a set of recordings.

Thanks a lot for the feedback! I can definitely see how this would be very helpful for use-cases like yours! I have asked the engineering team to provide a minimal example of how to achieve this with the API and will forward it to you asap. This may take until tomorrow.

Hi 🙂 My question concerns timestamps. I am conducting a motion capture (Xsens IMUs) + eye-tracking (Pupil invisible) study. I need to sync the data from both system which is not obvious. My goal was to match the acceleration recorded from Xsens' head IMU and glasses' IMU. My experiment is on a trampoline so I have a pretty good idea of the shape of the acceleration profile. Unfortunately, it seems like Pupils' gaze and acceleration data are not synced? Is it possible to know how the timestamp for the IMU in the gasses is given?

Ah zut! I have 16/20 subjects collected. Is it a fixed lag or an unknown lag?

Hello, we are collecting data while our participants are using a touchscreen in a dark environment but the resolution of the world camera on Invisible is not enough to clearly see the components on the screen. Is it possible to adjust the camera settings so that the texts can be read clearly or can we connect an external world camera to Invisible? If yes, do you have any recommendations? We got similar results with Core too. Thanks in advance.

Thanks for your answer @papr. You can understand that I am really disappointed. If I understand well, there is nothing we can do about it?

Hi, I recorded data using pupil invisible now, I am trying to analyze the data. I am not sure how should I start with data analysis, I came across a video explaining to use pupil player in order to create the heatmap but when I am trying to create a surface it shows that no marker is found and I am not sure how to define the markers. I would appreciate if someone can help me

I have another question while we are at it. I saw multiple times the notion of gaze angle appearing. It is an information that I need, I would like to be able to know the orientation of the gaze in terms of angles (side, up/down) relatively to the glasses reference frame. I am guessing I could compute it from gaze_point_3d, eye_center and gaze_normal fields from gaze_position.csv (exported from the app). However, these fields remain empty after export. Am I doing something wrong? Or is there any other way to access gaze angles ?

Please see this example on how to get directional gaze estimates in spherical coordinates based on the 2d pixel locations estimated by Pupil Invisible https://gist.github.com/papr/164e310047f7a73130485694d037abad

Is it possible to get pupil data when using the invisible eyetracker? Meaning, if the invisible eyetracker also provides parameters like the pupil diameter, similar to the core device

Hi, I'm having an issue getting videos from the Pupil Companion app to the cloud. All videos stay at 0%, no matter how long I leave it. I have tried restarting the app and phone, have tried multiple WiFi and data connections, and nothing has worked. I could previously upload videos with no issue.

Thank you, that worked! I'm also having an issue where I'm getting notifications that I can't action, some of them saying there are errors with the glasses. Is there any other way I can action them? One is about calibration and the other is a recording error but the notification is cut off so I can't actually read what the issue is.

Hi,we are researchers from a university in China. We want to buy some of your products, but we have some problems in import licensing and channels. Who can help?

@marcthankyou!

Are you planning to connect "invisible" to psychopy?

Hello friends, I have some issues with my recordings: 1. Are the recordings time sensitive or is there any space limitations, other than the phone itself, for the videos? When I recently try to have recording, although I have the color trail all the time, my video is not recording completely! 2. For audio output, I don’t have the audio after I use the pupil player. Is that normal? 3. The main concern is with pupil player. I try to have the merged video, and I usually can see it while it’s playing in the player. But when I try to download it, I have a mp4 file that cannot be played or when I push the download button it says the 2d or 3d data for visualization is missing!

~~Hey, are you using the iMotions exporter by any chance?~~ This should not be possible for Invisible recordings.

Player is v3.5.1

My app is 8 I believe

Yes, the newer one

That is much easier to use

Could you please share a recording (not the export/download) that is supposed to have audio with [email removed]

Hey I have a question regarding the PI Timestamp. Is it denoting the time the image was taken or received on the mobile device?

Thank you!