Hi, i have recieved a folder from my colleague that contains the following files in image 01.png, I do wish to visualize the gaze on the world.mp4 video. I have checked your documentation but the gaze data is available in .csv file with world index being available, in my case that isnt available. could you guide me how can i get the gaze into the video.

Hi @user-26cf70 ! That screenshot looks like the raw data, you would want to open it with Pupil Player, where you can already visualize the gaze on top or export the data onto convenient CSV files.

Hello everyone,

I’m currently working on a project that involves head pose estimation using AprilTags, and I have a question regarding the Euler angle convention used in this context. Could someone please clarify which convention is applied in pupil lab for the yaw, pitch, and roll angles? Any details about the coordinate system or the specific order of rotations would be greatly appreciated.

Thank you in advance for your help!

If you're using rotation_x|y|z, these are actually a Rodrigues' rotation vector. If you're using pitch|yaw|roll, then the coordinate system is relative to your origin marker

One way to achieve this is by loading that folder into Pupil Player and exporting the data. This will generate several CSV files, including gaze_positions.csv in the export folder.

With that, you could follow this tutorial for frame extraction.

Hello, I am working on a project using Pupil Labs, where we track participants' gaze as they complete a browser based maze game. I have a question about calibration. Previously I have been incorporating (a) one large calibration before the experiment, consist of around 30 calibration markers which i display on the screen covering the area of the maze in detail, plus (b) 3/4 markers before each trial to maintain calibration. I have then been using 2d gaze mapping post recording in Pupil Player.

Due to time limits, we now need to get rid of the calibration markers before each trial. However, ideally I want to maintain using 2d gaze mapping because 3d has too low an accuracy for what we want to do. Thus I am planning to do (a) the same large initial calibration (b) slippage test before each trial (comparing live data to a location on the screen). I think this means I have to do the gaze mapping in real time now instead of post recording, to do the slippage test. Thus, I have been trying to do a calibration using Pupil Core which involves many markers on the screen (the 5 marker default is not fine grained enough). Do you have any advice on how to do this?

I have managed to find a way to calibrate to different locations on the screen but not to detect markers. I have attached my code to do this.

Thank you!

Hi so in earlier version of neon player, the downloaded CSV files world_index

Thank you! This is very helpful! Is there any way to trigger the validation choreography through Pupil Remote (analogous to socket.send_string("C") but for validation rather than calibration)?

Great thank you! Just one more question, how can I return the recorded accuracy from that validation?

Thank you again. I can't seem to find the angular accuracy in validation.data (mine is returning a dict with keys subject, gazer_class_name, gazer_params, pupil_list, ref_list, timestamp, record, topic and i can't seem to find it in any of these)

Hello, What are the optimum working distances for the ocular camera? I thought I read something in the doc but I can't find it.

Hi, what is the best way to calculate glance duration from a dataset where I have labeled gaze directions or glance proportions

Hello! I am currently attempting to use the LSL plugin with pupil capture (Pupil Core), however I am not able to. Specifically this is what the command prompt is giving as an output. Thanks

Hi, we want to have a world camera recording without distortion, so that we can get a more accurate surface gaze point mapping and position estimation in Pupil Player. We are using Pupil Cam1 ID2 with a resolution of 1280*720. Could someone help?

Hi @user-f43a29 , thanks for you information. Because the recorded world camera video is distorted, I don't know if the position is accurate or not.

We are now trying to use OBS Studio to record the screen, and use the surface gaze data to draw each gaze point on the OBS recording. But we are worried that the distortion shown in the pupil player video will impact the surface gaze estimation. Because the QR codes are also distorted and we cannot make a plat surface.

We tried to use "Camera intrinsics estimation -- show undistorted image" in the pupil capture, which undistorted the camera very well. But the recorded video shown in the pupil player is still distorted. We also want the exported world camera video to be undistorted with surface and gaze mapping, so we can present. Is this possible? Thanks!

Hi @user-f43a29, as you can see in the figure, the screen cannot be fully captured by the pink boundary due to distortion. Because I need to map gaze to a surface, which is basically the full screen, so it’s better to have a more accurate gaze estimation. Is there any solution? Thanks

Hello! I have recorded a video of left and right eye frames using neon glasses. I have a set of frames of size 192 x 384. Then I take only the left eye frames. So I have frames of a shape 192 x 192. Then I want to detect a pupil in 2D (pixel) and 3D (in camera reference frame) So I take it as: pupil_2D = result_3d["ellipse"]["center"] pupil_3D = result_3d["circle_3d"]["center"]

where I get result_3d as follows:

grayscale_array = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
result_2d = detector_2d.detect(grayscale_array)
result_2d["timestamp"] = timestamp
result_3d = detector_3d.update_and_detect(result_2d, grayscale_array)

when I draw the first point on image it fits a pupil, when I draw projected pupil_3D (using camera_matrix) the result is not precise and stable. On the attached image the green point is pupil_2D and the blue point is a projected pupil_3D.

Q1: Do you have a video that shows that more accurate 3D pupil tracking is possible with these detectors

from pupil_detectors import Detector2D
from pye3d.detector_3d import CameraModel, Detector3D, DetectorMode

Q2: Do you have a minimal piece of code which demonstrates how I can get 3D pupil coordinates so that I cold try it on my data.

Greetings. I'm assisting a patient with multiple sclerosis complicated by a series of strokes.

I've tried off the shelf eye tracking hardware. Unfortunately monitor mounted eye trackers are problematic for those with limited mobility and wheelchair-bound. From a positional standpoint there's a lot of variability relative to a monitor.

• Muscle fatigue causing posture changes such as head droop or leaning torso

• A person's position in the chair such as tilt or recline

• Chair alignment to the monitor

• Prescription glasses can interfere with tracking

With this constellation of variables, it's hard to provide reliable eye tracking.

Accessibility is so important for those that don't have much. I'm trying to do my part to bring a piece of the world to a few people in my life.

This brings me to Pupil Labs. So far the stack seems the open compared to other products. I would appreciate insights with eye tracking hardware and software that could be leveraged for computer control.

Hi,

I am trying to do custom automated data loading and processing from recording of Pupil Core. (gaze.pldata and fixation.pldata). Specifically, I am borrowing this function: load_pldata_file() from this module: https://github.com/pupil-labs/pupil/tree/master/pupil_src/shared_modules.

I have observed the following two problems:

1. The data recorded is not alternatve between left eye and right eye (this is judged from the topics data field, with left eye marks as id 0 and right eye marks as id 1)
2. There is an uneven number of data sample from left eye and right eye so that the total number of sample recorded is not always an even number
3. The timestamp of individual data entry doesn't make sense, 1. left eye timestamp might starts at 4.03 while right eye timestamp starts at 0.08. 2. Some timestamp is not monotomically increasing, meaning it will jump from 4.87 to -0.05 then to 0.03...

I have the following questions:

1. Are these expected behavior, or could originate from software or hardware defects?
2. Assuming that these are expected behavior, and cleaning of it is integrated software wise, is it limited to using the raw data exporter plugin in Pupil Player? Is there any suggestion to preserve these data cleaning steps while not having to manually click the raw data exporter plugin to achieve custom automated data export
3. Regarding of misalignment in timestamps, I use the Pupil realtime API to set timestamp to starts at 0 before starting recording, could this be the cause of misaslignment? It seems that at least the right eye camera follows the instruction and have timestamps starting around 0.

Thank you.

Hi Neil, thanks for the speedy response, you are correct, actual timestamp is not needed as long as the relative ordering within each eye is preserved

A potential implementation for my case will be to create a list of left eye and right eye separately, by reading the eye id section to distinguish between the two, and then truncate them to obtain the same number of data points. However, there is an important underlying assumption: the relative order is correct and preserved from .pldata and the corresponding row from the two table are happening simultaneously in time(within acceptable range)

Do you think this is probable? Thanks

Hello, I would like to know the intrinsic matrix of the eye camera in Pupil Core.

Hi, @user-dbab7a - you can find those in the Pupil Core source code

Hello, I would like to ask why the scene camera (the camera that captures the world) does not perform camera distortion correction? Wouldn't not correcting it make the gaze error larger?

Hi, I am running an experiment where I calibrate at the start, and then check accuracy periodically throughout. I am using 2d gaze mapper. At the moment, it is set that if accuracy > 2 (or more generally a threshold) calibration is triggered at that point in the experiment. Unfortunately in piloting this has led to participants needing to calibrate a lot, often back to back. However, if I increase the threshold I find that accuracy can be poor in the data due to slippage. Is there a way to automatically shift calibration based on accuracy data throughout so I can posthoc do this instead of the participant needing to recalibrate throughout? (Would it be accuracte to do this -- is the calibration mapping linear)?

Hi I am using pupil-core hardware with pupils-labs GitHub source code to run the SDK and trying to communicate through another oy script to get gaze coordinate right now the problem I am facing is the the world camera feed is unable to open as the uvc.backend-utility is throwing init error I posted this issue on GitHub one month back but not getting any help can someone help me out from this problem

Hi there, I have completed my eye-tracking experiment using the Pupil Core and want to obtain time-to-first-fixation (TTFF) data. What are the possible methods to achieve this? Thank you~

Hello, I would like to know the rotation and translation matrices between the eye camera and the scene camera of the Pupil Core product. Where can I find these parameters?

Hi comunity i am using Logitech c615 webcame camera as my world feed but the pupil capture through is throwing initialisation error in log What should I do Do even Logitech c615 camera support uvc As in my windows default camera the feed is able to come But not in pupil capture can any one help me out

@user-11d287 , this could be an indication that the camera is not UVC compatible, but you may want to check with Logitech about that.

Hi Rob, I mean to measure the accuracy periodically to see how much the calibration has shifted, and then use this to update the calibration mapping to reshift the whole mapping through a translation. So do a full calibration at the start, assume the only source of noise is translational drift, measure accuracy periodically to see how much translational drift there is, then shift the preexisting calibration to account for this drift. I will try to share a folder with you asap

furthermore, is there any way I can access the homography matrix used for the gaze mapping?

Hi, I have received the replacement part on the left eye camera, it is connecting to the computer and data coming through I am running into a weird problem on consistent offset on norm_pos of left and right pupil

In this task, we are measuring x and y position using the norm_position attributes on the IPC backbone The participant is instructed to stare at the middle of the screen which we can guaranteed but as you can see, there is a consistent offset of the x position and y position, and it differs between right and left camera We make sure to perform calibration before each collection Here is the code we use to generate this result: This is a snippet of code that generate the result:

We tested across different participant, and go through calibration, the error persist Could you help us in resolving this issues?

Hello! I was wondering what is your intended way to remove blinks from the gaze data. I figured not every blink timestamp actually has a correspondant time stamp for the gaze, probably due to the irregular sampling that we observed in the data, which makes it difficult to remove blinks. How would you suggest tackeling that problem? Also, we were discussing thresholds for blinks and were wondering if you'd suggest keeping the defaults in the blick detection plugin for a design where we mainly want to observe micromovement of the eyes and expect no larger eye movements during the experiment.

Hello. Please what is the minimum XY deviation possible after calibration and validation using the 3D pipeline? The best I got today is 0.092 after calibration and 1.282 after validation

Hello, One of the eye camera is not showing the eye image. I am getting error : USB device not recognized

It is showing USB malfunction. Only the world window, and eye 1 are working?