Hi! I'm currently using the pupil labs fork of PyUVC with a pupil core. I notice that some of the camera modes do not seem to be functioning, and I was hoping to figure out a solution for them. For instance, if I use the 120 FPS mode, I receive valid images from the camera. If I use the 200 FPS mode, I receive black images. Monitoring the pixel averages, there is some movement of the pixel average (so not entirely 0) in the 200 FPS mode, but if I put it directly up to my hand it maxes out at 1.0 or 2.0, whereas the 120 FPS mode correctly maxes out at 255. Any thoughts on fixing this?

I am using one of the eye camera arms

It was purchased mid to late last year (2024), and it appears to be Pupil Cam2 or Pupil Cam3. The resolution options I have are 400x400 and 192x192. When using 200 FPS, I was using the 192x192 mode and for 120FPS I was using 400x400

Sure, thanks for your help! I can't use pupil capture as I'm running the camera from custom software on a raspberry pi, so I'd like to minimize the computation required. I also need to package the frames with other sensors' information (also controlled by the same program)

Sure! I'll record that today for you

Hi, I'm curious about gaze mapping algorithms. Could you provide detailed information on the calculation method and the code for it?

Yeah, that's what I'm looking for

Thanks, I'll check it out

I mean how can I calculate the sphere (or plane) equation of the front camera using the gaze_vector, sphere_center, and reference gaze points (after calibration) on a 640x480, 120-degree FOV camera. The goal is to later determine the gaze point by finding the intersection between the gaze_vector line and this equation.

Yes, sorry for my lack of clarity.

Yes, I'm working on monocular gaze estimation using my DIY glasses (with one eye camera and one scene camera), but I'm struggling a bit with calculating the gaze point. 🥲

My glasses are running pye3d, but I'm not sure how to obtain the gaze point. I have an idea to improve slippage prevention for longer usage, and I plan to develop it once I can calculate the gaze point.

So, is finding the intersection of the gaze vector with the image plane not a viable solution? Could you provide guidance on the best approach for this?

I have trouble with camera intrinsic calibration. I think it's either because the program didn't use my usb camera, or it couldn't recognize the pattern recorded from the usb camera. I'm trying to calibrate on a third party camera. https://streamable.com/bre78p

I don't think you can have two apps use the same camera simultaneously. I'd close everything, then start Pupil Capture by itself and select your camera within the settings. The view from your camera should then be visible in the Pupil Capture window, so you should not need an extra app to see what the camera sees.

https://streamable.com/h41hsz

Here's my capture.log file

Thank you for the instructions

I stumbled across this example code and I am still confused how to export custom data gathered in pupil capture. I filled in my custom fields and created my events[CUSTOM_TOPIC] but how do I export this data as a csv after recording? Is that automatically done? Sorry I am not well versed with programming.

Hey Rob! Uhm, I just used gaze data to calculate my own vergence angle and I wanted to output timestamp + the angle + maybe the other variables used for the vergence calculation in a csv for further processing.

Thanks a lot Rob for helping me out so quickly. Suppose I would love to have it in real-time. These 2 columns are actually already used in my vergence calculations which is then used for other calculations in my "recent_events" method. Currently I only output my values as a log in a logger in there which works in real-time. However, what would you suggest could I add or write to have it then exported to a csv or some kind of other output? Would it make sense to recalculate everything after runtime in a seperate script or is there a cleaner way to have all my calculation data stored after recording?

I'm thinking of streaming the gaze data from Pupil Labs Neon to Unreal. To confirm: Am I right that I only need a WebSockets connection for this and not rtsp? Also what will the incoming data look like, is it JSON?

Also, is Pupil Labs working on an Unreal Plugin? At the Max Planck Institute for Psycholinguistics we use multiple of your eye trackers heavily and are super interested in an Unreal Plugin. Thanks for all your great work, great stuff! I'm looking forward to your reply

Hi, I have the calibration data (prerecorded_calibration_result) of my monocular system

{
  "gazer_class_name": "Gazer3D",
  "timestamp": -861452.059011,
  "params": {
    "left_model": {},
    "right_model": {
      "eye_camera_to_world_matrix": [
        [-0.3484695425289404, -0.724353620096769, -0.5948788204183916, -130.66384023297988],
        [-0.7581322961235649, 0.5910203060031228, -0.27555474858254025, 203.71910832784144],
        [0.5511845421490723, 0.3549744088588379, -0.7551084488676022, 435.419631967043],
        [0.0, 0.0, 0.0, 1.0]
      ],
      "gaze_distance": 500
    },
    "binocular_model": {}
  },
  "record": true,
  "version": 2,
  "subject": "calibration.result.v2"
}

I'm curious about the input and output of eye_camera_to_world_matrix. Is this how gaze mapping works? If so, since I'm using 1280x720 resolution for the front camera, will it calculate the specific gaze point within this resolution?

Thank you a lot for your reply. I'm trying to run gaze_mapping algorithms with pye3d-detectors example to point out the gaze point in the front camera window in real-time. It's a part of my project, and I need a minimal software setup for this (I'm using my DIY monocular glasses). I figured out that I need calibration data to accurately map the gaze point, so I did a calibration on Pupil Capture and got 'prerecorded_calibration_result', I think the 'eye_camera_to_world_matrix' plays the role of transforming some inputs calculated by pye3d to the 3d gaze point. Am I correct in this approach? If not, how can I create a simple gaze pointer algorithm in Python based on pye3d without having to install the entire Pupil software suite?

The data in eye_camera_to_world_matrix does play a role - it's the basis for the gaze ray. Conceptually, the gaze point in 3D space exists along this ray. In a monocular pipeline, one might assume that the view target exists at a fixed distance from the viewer. With the gaze ray and an assumed or known distance, you can calculate the gaze point in 3D, and then project that onto the image plane of the scene camera.

The _predict_single method in the Model3D_Monocular class is probably the most relevant piece of code for your question

Hello team, is there any guidance or examples for mapping real-world gaze data into a 3D immersive virtual world? I want to use motion capture to track the position of the glasses, and we also know the position of our projectors. Any advice on how I should proceed? Thank you in advance

https://github.com/MinhHixn/pye3d-detector/blob/master/examples/newMapping.py

Here is my basic code for gaze mapping of my monocular gaze tracking system, it worked but the accuracy was bad, can you take a look at this and tell me what's wrong about the approach please?

Hi, @user-287b79 - reviewing and troubleshooting a project like this would probably require a custom consultancy package. However, I can provide some basic advice - if the mapping appears to be at least go in the correct direction (e.g., when you gaze upwards the mapped point moves upwards even if it's not where you're looking), then my first guess would be poor calibration

Thanks for the help! I succeeded in writing an actor that receives gaze data in Unreal.

Hello Pubil Labs

Problem: Can't run "main.py" Neon-Player I'm working on a project and utilizing the Neon glasses and for this I wanted to see the recordings in the offline neon-player. But since I'm working on a "work" computer I can't install the program. This led to trying to download the source code and just run it based on the instructions at https://github.com/pupil-labs/neon-player. (download source code -> instal requirements -> run main.py)

Running it on pyhton 3.12.3 gives an error in installing of requirements on a package for pupil_labs_uvc. So instead I tried to run it via python 3.11.5, which could successfully install requirements but fails on python main.py. I keep getting this git error, I don't get it.

Error message

(venv) PS C:\NeonPlayer\neon-player-4.6.1\pupil_src> python main.py
Error calling git: "Command '['git', 'describe', '--tags', '--long']' returned non-zero exit status 128." 
 output: "b'fatal: not a git repository (or any of the parent directories): .git\n'"
Traceback (most recent call last):
  File "C:\NeonPlayer\neon-player-4.6.1\pupil_src\main.py", line 38, in <module>
    app_version = get_version()
                  ^^^^^^^^^^^^^
  File "C:\NeonPlayer\neon-player-4.6.1\pupil_src\shared_modules\version_utils.py", line 84, in get_version
    version_string = pupil_version_string()
                     ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\NeonPlayer\neon-player-4.6.1\pupil_src\shared_modules\version_utils.py", line 58, in pupil_version_string     
    raise ValueError("Version Error")
ValueError: Version Error

So wanted to ask - what is up with that? - How can I run it without running the default installer and just running the project? - if that is even possible?

Downloaded source code from latest release. (Source code zip -> neon-player-4.6.1.zip) https://github.com/pupil-labs/neon-player/releases/tag/v4.6.1

do you think I can get away with copying my venv to avoid having to install requrements again 👀

Hi Pupil Labs support, could you please re-open my ticket " ticket-332969685706473472"? It contained crucial information and I am still in the process of taking steps to solve our issue. Unfortunately I did not save the content of the chat locally.