Okay. I will report here again if the app still works funky after we replace the module with the new one. Please keep me updated on the surface issue!

Hi pupil team! How can i get the coordinate values from the cloud?

Hey Pupil labs team, I'd like to have an offline version of the way pupil cloud show the video footage with the gaze data . Is it possible to download this video or is there a script that I may use to add gaze to video? Thank you in advance for your response.

Hello Pupil team. I'm trying to find the gaze direction on the real-time API. Is there a way to obtain this or the depth of gaze (coordinate z)?

Hello! Does Neon have the Prescription Lens Kit similar to the Invisible?

Is it being sold as a bundle “I can see clearly now”? together with the prescription lenses kit?

Thank you, Miguel

Hello! I was wondering if you would recommend merging the blink dataframe with the gaze dataframe in an attempt to create a closer number of gaze outputs across people? There just seems to be a huge inconsistency across people even when shown the same stimlus.

Hi I have a few questions that regarding Neon before purchasing that I hope you can help me with: 1) Does the real-time Python API use the deep-learning gaze estimation? If no, then what is it doing? Something similar to Invisible? 2) Is accelerometer and IMU data accessible through the python API? And what about pupil-data once that is ready?

Hi, @user-a64a96 - sorry about that! Pupil Cloud offers the "Gaze Overlay" visualization. If you run that on the recordings in your project, you'll then be able to download scene videos with the gaze circle displayed on top of it

Hi Pupil labs team, we do not get the “Gaze Overlay” as an option in our Pupil Cloud. Is there a reason for this?

The "bounding box" seems to change like this. I had orignally thought that if i define the QR codes at the beginning of the stimulus, it should hold until the end since they are constantly in view.

Hi, I'm wondering which data types are used as inputs for the gaze estimation neural network implemented in neon, to generate gaze.csv. Does the network only use binocular IR camera videos as inputs, or does it also use other types of data (e.g., scene camera) as inputs for better estimation?

Hi, we found in our data set that gaze data and blinks overlap each other in terms of time, which is somewhat counterintuitive because there is no source of information with closed eyes. I suppose it is due to the fact that the gaze estimation is continuously made, even when the source of information is temporarily missing. If this is the case, can we properly interpret the gaze values as estimated gaze directions “behind the closed eye lids”? Additionally, it would be appreciated If you could inform us of the start and end criteria of blinks (e.g. touch of the upper and lower eyelid or x % of eyeballs covered with eye lids etc.) so that we can interpret the results more thoroughly!

Hi @user-44d9d4! It's accurate to say that raw gaze data may contain unexpected coordinates during blink events. It is conceivable to filter these out using the detected blinks' start and end timestamps. These are reported in the 'blinks.csv' export.

However, it's important to understand the strengths and limitations of the blink detector. For instance, under typical viewing conditions, it performs well. That is to say, the start and end timestamps can be considered relatively accurate in correlating with the actual start and end of a blink. That said, in activities that trigger very dynamic eye and head movements, such as football, we've observed false positive blink detections due to the way (I expect) the blink detector leverages optical flow patterns in its computation.

Currently, in Cloud, there's no definitive method to verify the quality of the blink detection result. But soon, we will introduce an eye overlay visualisation that should significantly aid manual quality control.

You can find a much more detailed overview of the blink detector in this whitepaper, which should help you interpret the results better: https://docs.google.com/document/d/1JLBhC7fmBr6BR59IT3cWgYyqiaM8HLpFxv5KImrN-qE/export?format=pdf

hi Pupil Labs I was recently trying the neon monitor and found that the stream continues even after quitting the companion app. Is the stream not related to the companion app? How can I trigger the stream on/off then?

Hi @marc I appreciate your helpful reply! Just to make sure I understand correctly, the network can do "calibration" implicitly for the raw gaze estimation only using IR eye image pairs, as it does not need to solve correspondences between the world and the eyes (as normal camera calibration does) - since the relative positions of the world cam and the eye cams are fixed within a device, and the network learned the correspondences under this configuration. Do I understand it correctly?

I have one more follow-up question: is there any temporal integration in the gaze estimation process? In other words, is the network doing the mapping (IR_cam1_t, IR_cam2_t) -> gaze_t for each time point t independently, without using data at adjacent time points, say (t-1) or (t+1)? I'm curious about this because I'm analyzing the autocorrelations of the gaze data.

Hi Pupil labs

i have a question. 1. what do you mean by timestamp in gaze.csv? What's the standard? 2. can i know the standard of coordinates in gaze.csv? 3. There is start time, end time, what is the unit and when is the time from? in fixation.csv

@user-648ceb - please note my updated response to your question regarding the availability of pupillometry data over the realtime API: https://discord.com/channels/285728493612957698/1047111711230009405/1149297253257859144

Hi Pupil teams! I was wondering whether you know about mismatch between the number of frame images with the world_timestamp. I have a recording that the video has 3026 timestamps (in the world_timestamps.csv). When I checked and pulled the video using openCV, the number of frame is correctly 3026, but then I realized that the recording actually ends at the frame number 1975 (and is correctly the end scene of the recording). From the frame number 1976 to 3026, the frame image could not be retrieved. I checked that my other recordings also show the same problem, although the gap between the two numbers is not that huge. For example, in my other recording, the number of world_timestamp and the number of frame numbers is 2490, while frames could not be retrieved from 2035 frame. Any suggestions for how to fix this issue?

Hello. I see that the "worn" algorithm is not yet ported over to the neon yet. Is there another value we can use in place of this to know when the system is detecting the presence (or lack thereof) of eyes in the eye cameras? It would be valuable for us to be able to know this data in realtime. Thanks!

Hi @user-594678! OpenCV is known to drop frames and so that would explain the mismatch you're seeing. Is there a reason why you're trying to grab frames with OpenCV?

Hey Pupil Labs team, I have a quesiton concerning the surface positions, overlapping surfaces, surface warping, and how gaze coordinates is converted from 3D space into 2D space. Is there someone I can contact?

Hey @nmt , I did some more research and found this library called "decord" and with that all frames are correctly retrieved! I'm leaving the message here so that other users can look for if they encounter the same issue 🙂

Hi team again! I was wondering whether there is a way that I can easily convert the gaze points in pixels to visual angle (Are you planning on developing this feature by any chance?). I don't know much about Azimuth and Elevation, but those values do not seem like the same with the visual angle. I also tried to compute visual angle from the gaze pixels based on the scene camera's visual angles, but it looks like that I need to do some non-linear computation to correctly get the values. Do you have any suggestions that I can try?

Hi @user-594678 ! Azimuth and elevation are gaze angles in degrees from a cyclopean eye that has its origin on the scene camera. Are you looking for the kappa angle? to provide you with some kind of angle like that (as in how much it subtends one gaze point from the optical axis), one needs to know the location of the eyeball's centre or an estimation of it. We plan to include new features such as pupillometry and eye state estimation soon. In fact, by the end of September, these features will be available on the Cloud, and with the help of the eye state estimation, you will be able to get some sort of visual angle (depending on what you are looking for)

Is there a way I can play the video frame-by-frame in Pupil Cloud or with another Pupil software?

Oh, I see. Is the maximum neon sampling rate 30 frames per second? Thanks!

just wanted to confirm that audio is just extracted via the audio channel in the exported scene video?

Hi pupil team. As i know, the origin is in the top-left corner of the image(gaze.csv). if the screen moves, dose the origin move too? If so, is there a way to fix the origin without fixing the eyetracker?

Hi,

I just did two consecutive recordings with Neon and I get an empty file from the imu.csv in the cloud, I recall this was an old error, but that you could pull the file manually and the data might be there. Is there such an alternative way to check?

Thanks,

Hi Wee Kiat Apologies for the delay

Hi, is marker mapper only available in neon monitor app?

Hi What tool can I use to scan the QR code on Neon? Thank you

Thank you for your reply. I would like to check the serial number for registration purposes, but I have used various tools to scan QR codes, but there is no information displayed

An alternative way to get the serial number is to open the info pane in the Neon Companion App: Connect the Hardware, open the app and click on the 'i' icon.

Thanks

Hi @user-d407c1, thanks for your reply and sorry for my late response. I think azimuth and elevation are what I was looking for. Thanks for the clarification.

I have a following question. I plotted the gaze in pixels together with azimuth, and found out that they are not completely overlap. Do you know why this is the case and how I can fix this?

Also, I wonder whether if I directly convert a pixel coordinates in a frame into a visual angle using the camera's field of view (132 x 81 degress), would that be good to compare with azimuth and elevation values? I want to calculate a point in a frame in visual angle and compare it with the gazes of an observer.

Hello Nadia. Thank you for your response. I am using version 2.6.33-prod. Attached is a pic of the .json file.

Hey, is it possible to apply the gaze offset correction after the recording has been uploaded to the cloud?

Hi @user-a0cf2a ! This is currently not possible in Cloud, you will need to download the data and apply it yourself directly over the data. If you would like to see this function on Cloud, you can upvote it here https://feedback.pupil-labs.com/pupil-cloud/p/post-hoc-offset-correction

I just sent an email with the attached file. Thanks!

Hi @user-480f4c . Attached in a new email is the log from a previous recording where the timestamp in the imu.csv does not change (time frozen) while the gaze data has a changing timestamp. What might be happening? The log I mentioned previously (empty imu data) came immediately after this one I am mentioning in this message).

Hi team! We are testing out how we can access Neon Monitor on a browser to monitor our data collection in real time, but we are having trouble getting it to work reliably. It will work on some computers but not others, or work in one instance but when we open another instance a minute later on the same computer it no longer shows up. We are doing internal testing to see what might be the winning combination, but I was wondering if you had some troubleshooting tips? I assume it has something to do with some combination of security or internet connection settings. Thank you!

Hi! I am conducting an experiment to observe eye movements after rotation. My computer is Mac, and the glass is Neon. I have a few questions regarding Neon. Q1: Is it necessary to connect Neon to a computer and use Pupil Capture to collect eye movement data such as pupil position data? If Neon is only connected to a mobile phone (Pupil Companion), can we only obtain a video of eye movement? Is that correct? Q2: How should I use the raw data exported from Pupil Cloud? I am unable to import them into Pupil Player as it indicates that I need to update the version of Pupil Player, but I cannot find the installation package for the latest version.

Nadia hi!

You've mentioned about the pupilometry data will be added in the upcoming weeks to Neon. I wonder, will It be necessary to change any hardware to make it available or just a software update ? Thank you !

Thank you very much! It really helps!

Thank you for the suggestion! We are currently having an issue where Monitor will load once, but if we refresh the page or try a second time, it will no longer load. Do you have any idea why that might be? Thanks again!