Hi Team, I ran 6 RIMs for a project I am working on and the heatmaps got generated for 5 out 6 of them. Can anyone help me identify why the heatmap wasn't generated for that particular RIM? Thanks.

We are still able to start the recording manually during this issues.

Hello! Has new version of Neon Companion been released? I can not update it through the store.

Hey Team, The code for the scanpath image and video seems to be taking a very long time for one single RIM. Is there a workaround or is this how it is going to be? I ran a single RIM which took 7 hours to complete on python. The RIM had 5 viewers in total. Does anyone know how long it took for the demo project to run? I am talking about the example showed on the Scanpath Image and Video section on the Alpha Labs page.

Does 7 hours sound about right?

Hi, How can I get saccade data? duration amplitude etc?

Thank You. That's why I wrote. I'm thinking about buying a neon eye tracker. Currently, in the absence of saccade parameters, I need pass

Hey Pupil Labs, I need to have a high resolution of scene camera video. The video resolution is 1600X1200 and it is around 700 MB but the scene video in the "raw android data" is 1.1 GB. I was wondering why there is this difference. Is it because of different encoding? I also was wondering if 1600X1200 is the max resolution for the neon. Thank you in advance for your response.

hi Pupil Labs, I wanted to ask about the "worn" bool that's returned with the realtime API for Neon glasses. It seems to be always true even when the user is not wearing the glasses. Is this still in development? should we just ignore it then?

Hi Team, I tried running the following code from the Alpha Labs Page to map gaze onto a screen recording - https://docs.pupil-labs.com/alpha-lab/map-your-gaze-to-a-2d-screen/ The output I got was at a 0.5x speed. Is that by default? The example shown is in normal speed but the output I got was 0.5x speed.

I will try doing that! Thanks Miguel!

Hi! I asked a question some time ago I did not follow up on (sorry) but as it turns out my colleague almost immediatly found where the problem was. I have another question (more of a clarification): the docs say pupil cloud accounts for distortion, but is that only when you explicitly chose to undistort, and only for video? If I download the raw data, there is no option to get the undistorted gaze directly, I have to undistort it with OpenCV? Thanks!

Hello， when I finished uploading the recording and went to check the cloud, I found an error in one of the recordings. Should I keep waiting or export the footage directly from my phone to my computer myself? As I want to deal with this footage as soon as possible, if I need to go and export it to my computer myself before analysing it on the cloud, how should I do it?

My OnePlus 10 Pro upgraded to Android 13. Instructions to rollback or downgrade to the previous version are only available for OnePlus 8 and 8T. Can someone provide me the instructions and official Rollback APK for the OnePlus 10 Pro?

Hey Pupil Labs team, Is there a method to perform offline drift correction for Neon? To the best of my knowledge, there is no specific calibration for Neon. However, when we examined the scene video with gaze overlay, we noticed that there is some drift between the "target" and gaze. Thank you in advance for your help.

Hi there, We are conducting a study that involves getting participants to read text displayed on a smartphone while wearing the Neon eye tracker.

We'd like to know how to extract two parameters from the timeseries data (.csv files) that's downloaded from Pupil Cloud. 1) How fast the eyes are moving/scanning while reading (speed). 2) How much the eyes have moved (amplitude) when reading.

Could you provide some information for the above, any codes or methods would be helpful. Thanks.

Hello. Regarding the units of yaw (-180,180), would it be right to say 0 is the magnetic north?

During a subject, the eye-trackers stopped recording because I am out of storage on my phone. If I delete video data on the phone, will it still exist on the cloud? I'm assuming yes, but I also wanted to make sure. Additionally, how much cloud storage do we have?

Face mapper enrichment to data

From the Pupil Cloud, I downloaded the "raw android data" of a recording. In that data folder, there is eye-video named "Neon Sensor Module v1 ps1.mp4", and also a file named "Neon Sensor Module v1 ps1.time", I assume this is the timestamp file of the eye-video. But how to open this .time file?

I experience difficulties with the world cam. I see a red dot (gaze) on a black screen.

Hey, short questions: is it possible to process the NEON recordings with the PupilPlayer? So just copying it from the mobile device to the computer and running PupilPlayer, e.g., to get the gaze overlay. Or is this only possible by using the Cloud?

We previously integrated the PupilInvisible Device by using the "zyre" library in C++, so we were able to directly revceive the eye streams, gaze data, world video etc. in our system based on C++. I assume, that this is no longer possible, so we have to switch to using the HTTP REST API, websocket connections etc.?

Sorry to keep bugging you about this, but any updates on the availability of pupillometry? Thanks!

Hi,

I'm having issues using the cloud to produce heatmaps via the Marker Mapper. When running the % doesn't go above 0 and I also keep getting Internal Server Errors. Enrichment ID: 9f2aa5f1-a695-48dc-b5e9-166f1ccec08b

Hi,

I'm currently working on my thesis on eye tracking where we use pupil cloud. The enrichment 'Face Mapper' is very interesting for our research, although, I was wondering if it is possible to determine where on the face a fixation falls (eye region, mouth region, nose region). When downloading the enrichment data to an Excel file, coordinates of the eyes, mouth and nose are given, but not really whether the fixation falls in these regions?

Hey @&288503824266690561 , quick question about the marker mapper. When computing surface coordinates does pupil labs apply an M-estimator Sample Consensus (MSAC) or similar algorithm that filters out spurious measurements? In that case can I assume that any time a 'gaze detected on surface' row reads true that spurious measurements have already been filtered out or do I need to do my own filtering? I have notices that in the pupil cloud ocassionally the surface spazes out and seems way off but I am unsure if each time this happens it is assosiated with a "false" value as it would seem the website seems to think that the surface is detected even if ocassionally the estimated surface varies wildly in size and shape from the true surface. I would send a screenshot to show you what I am talking about but I cannot seem to be able to log in to the pupil cloud right now. I ordinarily log in with a google account.

Hello! I cant sign in to my pupil cloud with google, what can I do?

me neither

Yes, it works. Thank you very much for your help

Hi, im ken. I want to know how to split long recordings into 3min part to conduct reference image mapper.

Hi, I did some recordings which are now fully uploaded to the cloud. I am trying to watch the video rendering with the fixation and gaze overlay, but these keep dropping out (although both are selected) or don't show up at all. A notice for 'internal server error' keeps popping up. Is there anything I could be doing to avoid this problem?

hello! my pupil cloud always report error:'{"code":500,"message":"internal server error","request_id":"3b8df27c19c0ca4a89d2334d1ff18f1b","status":"error"}", what should I do?

Hi @user-4724d0 and @user-d714ca ! We have experienced some issues with our servers, but our team is working on this, I will let you know as soon as they fix it, apologies for the inconvinience

@user-4724d0 @user-d714ca @user-3c0775 this has been solved, if you are still experiencing issues please let us know.

Working great, thank you!

Hi, anyone have experienced an error when using OpenCV package to adjust the RGB mode? I was trying to define Area of Interest.

oh my bad. It's folder. But when I change it to the actual image file, this happens (it's loading and wont run)

Okay. Sorry

Error found - the Neon module was not properly connected in the frame Hi I have just received my "Neon - Just Act Natural" package with the One+ 10 Pro, but the gasses will not connect when plugged in.

Hey, @&288503824266690561 question: for the enrichment files how come the gaze.csv and the surfacepositions.csv files are of different lengths? Should they not have the same number of data points?

Hi Pupil Labs I had a question about the async and sync methods in pupil-labs-realtime-api. Does the sample rate of gaze differ in two i.e. would async method provide a higher resolution of gaze data stream than sync? I can see it in the api implementation that the sync method pops out the last frame from a queue which might theoretically result in frame drops. Does the async method ensure that we stream every frame from the queue to the connection?

Some data points are missing corner locations in the surface_positions.csv and in the case when the gaze is not detected on the surface there is a “false” in the “gazeDetectedOnSurface” field so I doubt that would explain the disparity.

Hi @user-6c4345 ! We have received the email and we will contact you promptly with a solution.

@marc why is that the frames are flickery? sometimes it works but immediately after few seconds it's flickery.

Hello, I have been trying to use the pupil neon with pupil capture v3.6.0 (my experiment needs real time head tracking) but I can't manage to get a to gaze position. I have tried to set ROI for my eyes and the neon calibration. The gaze is really shaky and and I often observe a low confidence level on one eyes. Any tips or documentation I good follow ? thanks

Hello! Currently I am not able to see the eyes camera in the app. Is there any setting to reactivate it?

Ahhhh I think I see what's going on, when I inverted the time difference to get update frequency it appears that the gaze updates as fast as the infared eye tracking does (150-400Hz) while the surface_positions updates at the camera frame rate (30Hz and under)

Ah, good eye. If you wanted/needed those surface gazes at 200Hz, you could interpolate. That's about the best you could do anyway given the frame rate limit of the scene camera

Hi Nadia, Thank you for the help! I have tried to apply and use the tips, but I still stumble over 1 aspect. The fixation data in 'fixations_on_face' (fixation in world camera pixel coordinates) are different from the fixation data in 'face_positions' (fixation in image coordinates in pixels). I don't really understand how I can compare these to determine which fixations fall in the mouth/eye/nose region? My apologies for the many questions, is there by chance a detailed manual/information somewhere? This one on Pupil Labs Neon is fairly common.

Hi @user-91a92d 👋 ! Do you happen to have recording you can share? We'll be better equipped to provide you with feedback if you share a rec that includes the calibration choreography.

Hi @user-8825ab ! I am answering to you here, since it seems like you are using Neon or Invisible, could you please try to log out and log in again in the app and see if this resolves your issues? https://docs.pupil-labs.com/neon/troubleshooting/#recordings-are-not-uploading-to-pupil-cloud-successfully

thank you

it works now

Hi team again, I uploaded my recordings on icloud but it shows an error on the recording, i wonder what shall i do to make sure it is fine

Thank you

hi Miguel, it's me again, I wonder if we can export the recordings to our personal storge as the data confidential policy, and we don't have a gaze overlay function in our app, I wonder if this is something we should pay separately?

Hi @user-8825ab ! Pupil Cloud is secure and GDPR compliant. You can read more about the privacy policies and security measurements in this document https://docs.google.com/document/d/18yaGOFfIbCeIj-3_GSin3GoXhYwwgORu9_7Z-grZ-2U/export?format=pdf

As of now, we don't offer localised versions of Cloud to be run on your servers, but we do have a basic offline app that will allow you to explore the data https://github.com/pupil-labs/neon-player.

OK THANK YOU, can we download the video with the gaze information(i meant the red circle) on?

when i try to add it, i didn see the overlayed function here

is there some way we can add this function

Hi! We will soon make it available there, but until we modify the UI, you need to click on "Analysis" at the bottom left, and then on the "+ New Visualisation".

ok, then can we download the recording ?

then can we download the recording with the gaze overlay?

ok great, thank you very much for your support

Hello! Where can I find the option to play back the recorded session and stream it using Python? The Neon player can handle playback but lacks a network API. Additionally, why does the recorded data downloaded from the cloud not include eye videos?

Hello! I have a recording with a failed fixation pipeline, can we fix it?

Hi @user-9c53ac ! Reg. your first question, there is no possibility right now to stream back a recorded session, could you please develop what would be the use-case for it?

Reg. sour second question, currently, if you want to download the eye videos, you need to download the original files.

To do so in Cloud, you will need to enable the RAW sensor data download in the workspace.

See this message on how to enable it: https://discord.com/channels/285728493612957698/1047111711230009405/1108647823173484615 Kindly note that this RAW sensor data is the same that you will obtain by exporting the recordings directly from the phone.

Hello, I'm trying to use Neon Monitor but when I try to access the Monitor app through the URL it stops and doesn't show me the page, which tags as 'Not Secure'. It says: "The site doesn’t use a private connection. Someone may be able to view and change the information you send and get through this site. To resolve this issue, the site owner must secure the site and your data with HTTPS."

Can you please help?

Hi @user-7413e1 ! That's probably your firewall, antivirus or some setting in your network. Try using the IP address below the neon.local:8080 or using a different network

Unfortunately, Eduroam is known to be quite restrictive, I would recommend using a different network if possible or setting up a hotspot https://support.microsoft.com/en-us/windows/use-your-windows-pc-as-a-mobile-hotspot-c89b0fad-72d5-41e8-f7ea-406ad9036b85#:~:text=your%20data%20plan.-,Select%20Start%20%2C%20then%20select%20Settings%20%3E%20Network%20%26%20Internet%20%3E%20Mobile,Internet%20connection%20with%20other%20devices.

If using a different network, these are the requirements https://docs.pupil-labs.com/neon/how-tos/data-collection/monitor-your-data-collection-in-real-time.html#connection-problems

Hello everyone, we use the eye tracking glasses in an autonomous driving setup. We have big problems with the exposure. Is there a way to control the camera parameters via the Python API? Is it possible to tell the Backlight Compensation which image area it should pay attention to?

Hi, I'm working with the neon glasses and just made some footage. But right now the upload has been going on for a few hours and it's still 0% uploaded to the server. The loading bar is turning, but not doing much more. Is this normal? Or is there maybe another way to upload the footage to pupil clouds?

Hi @user-7413e1! What device are you using Neon Monitor with?

HI there, I'm wanting to quantify the x,y gaze/fixation co-ordinates in cm units, the raw data files downloaded from pupil cloud gives these co-ordinates as pixels. I was wondering what's the best way for me to convert the px unit generated by Neon world camera to physical units? Thanks

Hi @user-d2d759 👋. May I ask what the goal is here? Do you mean to map gaze to some object in physical space, such that you can quantify gaze shifts in terms of cm relative to the object?

or may be share a gist to playback

thanks but i want finer access to individual frames and its timestamps locally without having to use cloud. but its ok, i will develop it myself

We can recommend using Chrome Browser on your PC for the best experience. The connection is local - no internet is needed. Other devices on the local network can also view the stream, so that's likely why you got that message.

Hi Neil, thanks for the quick reply. We work with real vehicles on real roads. More specifically, we do image stitching between the vehicle camera and the scene camera of the eye tracking glasses. For this we do feature matching and if the image is overexposed or underexposed then unfortunately this does not work.

Yes, we combine it with Semantic Segmentation, Depth etc. to anwere the qestion What the driver was looking at.

Monitor app

Hello! I am also having a problem during live-stream recordings. I am using a hotspot from my iphone to connect both my macbook and the neon phone companion. This works fine to launch the recording in the monitor app, but I sometimes lose the live gaze tracking: meaning that I still have the live world camera feed, but the live gaze indicator (red circle) is stuck in the center of the screen. Is there anything I can do to be better able to monitor the recordings live?

Hi there. I tried to use the basic offline app by dragging the folder directory over. However, when I dropped it onto the program, it loads for a few seconds then crashes. Why would this be?

Monitor App 2

Hi everyone

I want to buy the Neon model. However, I have a few questions. When will the eye state and pupil diameter feature arrive? What is the ease of access to data? It is important that I have access to raw data. Also, will there be any problems with updates?

Hello Everybody! we are currently doing a gaze tracking study with Neon glass. In our experiments, we want to use the Marker mapper to map the gaze data in the defined surface we want. For that, we have fixed AprilTag markers from tag36h11 family in the four corners of the screen (like one instance in the screenshot) in static positions. After we did some recordings and then wanted to run the marker mapper enrichment in pupil cloud, it doesn’t work. It is not able to define a surface at all although all the markers are visible in the screen. Could you please help what could be the possible reason behind it? Could it be because of the size or position of the marker or something else we are missing? Thanks. I am providing one recording id as well if that helps - 2975987a-d462-4981-86f0-5b3251dbc51c

Hi @user-186965 👋🏽 ! Could you please share more information on what happens when you try to create and run the enrichment? When creating the enrichment in your project, you should be able to see whether the markers are detected or not. If they are detected, they are displayed in green colour (see the example I'm attaching). Is that the case? If not, do you see any error?

Hi. I have been looking at the 3D state ".csv" file. If I understand correctly optical axis xyz are the x,y and z component of the directional vector of the eye in the coordinate system described in the documentation, but there is no Torsional information. Am I right? Is there a way to get information on torsion eye movements?

Hi @user-b55ba6! Torsional eye movements as in rotation about the line of sight are not reported. To be able to get information about that, one would need a really detailed view of the Iris, such that you can track some landmark, then define the baseline and track it.

Could you please share the ID of the affected enrichment? You can get it by locating the enrichment in Pupil Cloud, right-clicking on it, and selecting 'Copy enrichment ID.'

Hi @user-186965! Our Cloud team has looked into your enrichment and it should run fine now - could you please have a look and confirm that it has been completed?

Neon with Core pipeline and Pupil Capture

Hi team. I am currently working on a project in the cloud. I have 11 RIMs in the enrichment sections. None of the RIMs have been processed till now. They have been running for almost 3 hours now. Any suggestions? I've tried refreshing the page multiple times. Doesn't seem to work. The data is 2 days old.

Yes we habe tried diffrent settings. I think that the autoexposure doesnt work "correctly" for our usecase because the windscreen just covers a small fraction of the image so the exposure gets adjusted to the dark cockpit. I think that it would be realy helpful if i could adjust the area to which the autoexposure attends to. So to focus on the windscreen. Isn't there any possebility? Thanks!:)

Hey Pupil Labs, I'm interested in generating a gaze + scene video similar to the cloud player on my local machine. I'm aware that some sensors may start a few seconds after the video begins. Could you please direct me to the synchronization documentation or the respected Neon Player GitHub code? Unfortunately, I couldn't find either, and I thought there might be documentation on your website. Thank you!

Backlight 0 works finde but we have to adjust the exposure manually. Autoexposure doesnt work well.

Hi. I am looking at the "Neon Sensor" eye video. I am interested in getting the timestamp of the eye frames. Eye video has 19476 frames, but gaze.csv has 19473 datapoints. The first 40 frames, however are saturated and very white, which raises the doubt if the 19473 datapoints align to the 19476 frames. Do they?

Hey. Does anybody have a suggestion on what would be best to calculate eye movement velocity (to detect saccades). I know there are some algorithms that are pixel based, but I am interested in applying a threshold of 65 deg/s. I see some suggestion to use azimuth and elevation angles. There is also the 3d cartesian coordinates from the 3d_eye_states.csv file. Anybody?

Hi - I am doing data collection with children, so the neon is not always worn properly during the study. The exported 3d eye states and gaze files have data for every frame, even when the eye is not in view. Similar to @user-b55ba6, I want to use the eye video recordings that are in the export folder - but the number of frames in this video do not match the number of frames in the 3d eye state and gaze files (by ~10 frames). I have a few questions: 1) Is there a way to get the timestamps of the eye videos? 2) Do you have any recommendations for other ways to identify frames when the 3d eye state and gaze data are wrong (because there is no eye in view)? and 3) Will there be a way in the future to export the eye videos as an overlay on the world video? Thanks!

Eye frames timestamps

Hi, I have a question regarding the pupilometry update. After updating the companion app, I can see the 3d states file when I export the data after uploading it to the cloud, but not the pupil diameter. Is the output a csv file as well?

Thank you for the quick reply! The Neon player is great, and I can use that to identify frames where the tracker is not being worn. Our main concern is that there is gaze and eye state data for all frames of the recording (and a vis circle being generated) - even when the eye is not in view of the camera. For example, the glasses were pushed so that the participant's forehead was in view instead.

Hi, pupil labs! I have a problem about runing the anaylysis through enrichment by uploeading the reference image mapper. I tried it with two recordings I made, but it all said error that "The enrichment could not be computed based on the selected scanning video and reference image. Please refer to the setup instructions and try again with a different video or image." Can you please tell me where I did it wrong here please?

Hi there! A while back I had asked a question surrounding the cloud softwear and our new neon glasses, when one of the staff messaged back and mentioned i could get a free tutorial session since we purchased the neon glasses, how do I go about scheduling that?

Hello. Regarding the new pupil diameter data streams in cloud, you state that "The algorithm does not provide independent measurements per eye but reports a single value for both eyes." A few questions. 1) How does the algorithm arrive at a single value? Are you taking the average of both eyes to compute this value? Something else? 2) Are there plans to eventually provide different values for left and right eye diameter? If so, is there an ETA for that?

Also, for pupil core data collection through Pupil Capture, the CSV outputs for pupil diameter included "confidence" and "model confidence" values. I don't see these in the pupil cloud pupil data csv files. Is there a confidence value being computed somewhere? Is it automatically trimming bad data or including it? Thanks!