Hi Neil, thank you for the feedback. I will see to it that I can procure a recording that may be of help.
In my pupil capture settings, the lowest resolution available for the eye cameras to me is (320,240). And the only framerate available is 120fps.
I also recently discovered in one attempt that on one eye, the pye3d model slowly drifts during the experiment. I did not notice this until it was time to perform a validation after the block was finished. E.g. Before calibration and starting, the participants is asked to roll their eyes to stabalize the 3d model and it all works well throughout calibration. At the end of the experiment block, I noticed that the 3d model (blue ring) has shifted from the bottom left to the top right (still a solid blue color). What could be a potential cause for this? And is there any way to recover the gaze data from this?
Hi @user-4c4fec , I am briefly hopping in for @user-4c21e5 . Could you share this recording with [email removed] Then, we can take a closer look.
Regarding changing the intrinsics for a recording, while possible, may I first ask why you need to do this?
As a follow up question as well: Is it possible to change camera intrinsics in a recording?
Hello! I am student researcher attempting to do pupillometry with Pupil Core. In my experiment, participants in a dimly lit room facing a computer with a black screen and a fixation cross listen to different piano tones and I want to observe the percent from baseline change in the pupil diameter. I am currently following all the best practices on the pupil core documentation (the pye3d model looks stable and well-fit, calibrating and validating before starting), but I am finding very low pupil confidence in my recordings. When I adjust the eye tracker for each participant and run calibration, it shows good accuracy (<1.5) and the pupil confidence appear to be holding from about 0.95-1. However, when I run the experiment in PsychoPy, the pupil confidence drops dramatically. Upon pre-processing it looks like over 80% of the data has pupil confidence under 0.8 across the few participants I have run. Do you have any insights into how I might fix this?
Hi @user-7d3edd , thanks for sharing the recording with us. I have taken a look. The eye cameras have rather high exposure and may I ask if the participant was wearing mascara? It seems that may have been the case and it is confusing the dark pupil detection method for one eye. You can see this by activating the Eye Overlay in Pupil Player. If you want to still use this data for studying pupil diameter, you could at least use the data from the one eye that consistently has a decent confidence rating.
Going forward, you could also reset the eye cameras to manual exposure in Pupil Capture.
You can also check the Confidence in Pupil Capture, before you start recording, and make sure that the yellow and red circles sit properly and stably on both pupils.
Hi @user-7d3edd , could you share an example recording with [email removed] Then, we can take a closer look and provide better feedback.
@user-f43a29 Hi Rob. I'm having trouble importing the hmd-eyes package into a Unity project. Basically, there are bunch in unresolved external associated with the LZ4Codec Resolvers. Any ideas?
Hi @user-be1dcc , no need to worry about tagging us. We will see the message.
May I ask what version of Unity you are using and what Microsoft Development Packages you have installed (found by opening Visual Studio Installer and clicking Modify on the relevant version)?
Hi good morning. I was just wondering if you could help me with a pupil player problem. I have created a gaze mapper successfully, the fixations are all there and properly recorded. However, when I click on the validation to see the accuracy result it just says 0.0° from 0 / 0 samples. I have tried rerunning everything in 3 different computers and I have the same result for this file. Also, I tried several recordings and it seems only this one has this problem. When I export the data everything is fine, I just have a problem with the validation and I really need the accuracy values. Could you help me with this?
Hi @user-660f48 , was a validation routine run during this recording? If you want, you can share the recording with data@pupil-labs.com and then we can take a closer look.
[email removed] we did not do the routine validation, only two 5-point calibrations at start and end. These recordings were done a while back in 2023. I am happy to share the data with you.
Thanks. That would be helpful.
Thanks I will do this. The funny thing is I only have problems with this one recording. With all the others we can get the accuracy from the Gaze Mapper without any issues.
Just so you know, I just sent it to the email you provided.
Thanks, I will take a look.
Hi @user-660f48 , I have taken a look. It works here. I simply set the trim marks (see Item 4) to the time segment for the second calibration and used the Set from Trim Marks button. You may have to wait a moment when you push the button for it to update. I attached images showing the results for the second calibration.
Also, just to help, you can improve your use of AprilTags and Surface Tracking. The AprilTags should all ideally be flat and in the same plane. The Surface Tracker assumes that they all lie in the same plane. That should also be uniformly illuminated
You can use the image here as a reference for a good AprilTag setup. Note how they also all have the same orientation and align with the boundaries of the fitted rectangle. This will also make things easier. If you change the arrangement, you do not need to cover the display surface of the monitor with them. You can simply tape them to the edges of the monitor.
Ah that is fantastic, many thanks for this. Great advice.
Hi Rob, thanks for the reply. I apologize for taking so long to get back to you as I'm currently running the experiments. I will upload the recordings after I ensure that there aren't any sensitive content taken by accident and send the link via email. There have been more questions that have popped up that I hope will be able to be answered.
Regarding the intrinsics, I was looking at the undistorted view and thought that it looked a bit off so I was asking regarding it. It may not be a big problem in the long run, but i was curious if adjustment is possible.
Thanks. Internally, both Pupil Capture and Pupil Player apply the camera intrinsics and undistort the image before doing critical processing. Is that sufficient or could you also include an example of what seems a bit off in your email?
Hello! I am working with this GitHub project (https://github.com/Lifestohack/pupil-video-backend) and attempting to use a Raspberry Pi, connected to the same WiFi network as my laptop, to transmit data from a Pupil Core device, with the goal of enabling wireless use of the Pupil Core.
The issue I am encountering is that when I start the camera stream on the Raspberry Pi, the terminal indicates that everything is running normally and that frames are being transmitted. However, no video appears in Pupil Capture on my laptop. Has anyone experienced this issue, or are there more robust solutions for making Pupil Core portable?
Hi @user-7c9706 , this is a third-party plugin, with which we do not have direct experience. In addition to posting here, you may want to reach out to the original author.
In any event, it seems as if you have setup everything correctly and it looks like that video backend is successfully connecting to Pupil Capture, so I am not sure exactly what has gone wrong. You could also share the console log of Pupil Capture, if you wish.
With respect to alternative ways to make Pupil Capture portable, you could also connect it to a laptop in a backpack. Otherwise, Pupil Capture does not run on ARM-based systems, like Raspberry Pi, but x86 single-board computers, like LattePanda, should be able to support it.
i am facing grey screen problem in tbhe pupil capture software the cameras are plugged in still it says the camera is blocked
please anyone help i am working to resolve this problem since 3 weeks
Hi @user-5e0300 , I have moved your messages to the Pupil Core channel. Could you first try to restart Pupil Capture with default settings? The button is found in the General Settings tab
i triedd but still i am facing the problem
do i have to change any camera resolution
its opening in the resolution of 190x190
Hi @user-f43a29,thank you for the reply!
I have a couple of follow-up questions:
You mentioned that I could share the console log of Pupil Capture. Could you let me know how I can access or export the console log from Pupil Capture?
I also encountered a rather strange behavior. If I start Pupil Capture first and then start Pupil Service, Pupil Capture immediately crashes. However, if I start Pupil Service first and then open Pupil Capture, there is no crash.
Additionally, if I run only Pupil Capture, the project does not work. But if I run only Pupil Service, the streaming pipeline on the Raspberry Pi seems to run normally.
I am not sure whether this behavior indicates a configuration issue or something related to how the backend connects.
Also, just to clarify: the purpose of this third-party plugin is not to run Pupil Capture on the Raspberry Pi. The Raspberry Pi is only used as a relay device to stream the video data over WiFi, while Pupil Capture still runs on my laptop. In this case, would the ARM limitation still affect this setup?
Thanks again for your help!
Hi @user-7c9706 , you are welcome.
Here are my responses:
C:\Users\<username>\pupil_capture_settings.Hi, I am writing a python script for an experiment with three devices including pupil core. I was wondering how to publish the pupil and gaze data into a specific folder instead of manually copying and pasting it from recordings. I read the network api doc but I'm still confused (sorry I am a coding novice)
Hi @user-80f716 ! Do you want it programmatically copied in realtime or post-hoc?
Post hoc
Great, in that case you do would not need the Network API.
I assume you load the recordings with Pupil Player? Is that right? Then, you would have your gaze_positions and pupil_positions CSV files inside the exports folder of the recording.
May I ask is this what you want to copy somewhere else?
If you change of opinion and want to do it in realtime here is an example subscribing to gaze.
Sorry what is the exports folder? I see the folder called recordings, which then has the date and then sub folders for every trial from that day. Inside i only see the csv file for user info.
I haven't tried loading the recordings, just make a few with pupil capture
Let's get one step back then, you would need to first open them on Pupil Player and export the data.
How would I do this with zmq?
And thank you for the realtime example, since we might do this for future studies.
I see... your goal is to bypass the GUI interaction completely, correct? Pupil Player does not have a headless mode perse
and the ZMQ is only a network protocol.
If you want to work without the Pupil Player GUI, you’ll need to read the recorded data directly using msgpack, as shown here:
https://gist.github.com/papr/81163ada21e29469133bd5202de6893e
That said, I’d recommend first familiarizing yourself with the software and its interface before working directly with the raw data.
Yes. Ok, I'll check it out, thanks for your advice
@user-f43a29
Thank you for your responses.I’m happy to share that I have successfully implemented wireless transmission for the Pupil Core using a Raspberry Pi as a relay with this project. The command that worked for me is:
[email removed] $ python3 main.py -d world -i 192.168.91.110 -p 50020 -vs 0 -vp 320,240,30
This command includes the following information:
-d world: specifies the world camera
-i 192.168.91.110 -p 50020: the computer’s IP address (192.168.91.110) and the listening port (50020) used by Pupil Capture
-vs 0: the device index (0) of the WORLD camera on the Raspberry Pi
-vp 320,240,30: output resolution is 320×240 with a frame rate of 30 FPS
However, this only works for the world camera. When I switch to eye0 or eye1, I cannot reproduce the result. Therefore, I have the following questions:
1.When using remote streaming in Pupil Capture, do the three cameras use different listening ports?
2.Are there differences in image format between the eye cameras and the world camera?
3.As you can see, due to the performance limitations of the Raspberry Pi, the transmitted video quality from the world camera is relatively low. For gaze estimation and calibration, are there minimum requirements for image resolution?
Hi @user-7c9706 , great to hear about your progress and thanks for the explanation!
Seeing as I am not very familiar with that video backend and we have never tested it ourselves, I cannot really provide support. I still recommend also reaching out to the original author.
To answer your questions:
Hello, I am using the Pupil Core glasses and I am experiencing an error that I haven't seen before. When I connect the Pupil Core to my laptop, an error apears on the screen. I wonder if you could help me with it. The error is shown on the picture. Thank you in advance.
Hi @user-85778d , are you on a machine that only has an Intel integrated GPU?
In the past it has worked properly, but I think it has
Ok, if your system has recently undergone any updates, that might explain it. It appears that Intel released an update for their integrated GPUs in recent months that is incompatible with the expectations of the Pupil software. If you have a dedicated GPU in the machine, then you can force Windows to start Pupil Capture with that GPU instead, which should resolve the issue.
Ok, I will try this. Thank you very much for your help
I recently installed psychopy, may this have afected?
That should be unrelated. As mentioned, it is most likely related to GPU drivers and whether your machine is choosing an integrated or dedicated GPU when running Pupil Capture.
Hello, would the Pupil capture software work with a device like the jetson orin nano from nvidia?
Hi @user-da7106 , one of the main concerns with such devices is if they use x86 CPUs or not. That device has an ARM CPU, so it will not be able to run Pupil Capture.
okay thanks!
I have encountered a new issue: after using a Raspberry Pi as a relay for wireless transmission on a certain network (where I modified the "Connect remotely" setting in Pupil Capture's network API to the computer's IP address), Pupil Capture cannot open properly if my laptop is on a different network (the issue also occurs if it is on the same network but with a different IP address). The run log is as follows:
This issue came out after successful wireless transmission.
Hi @user-7c9706 , if the computer is on a different network, then it will need to be assigned a public IP address. The 192.168.x.x form of addresses are for private, local addresses, not externally accessible addresses.
Otherwise, details about WinError 10049 are here. Essentially, there is no device on your local network with the address 192.168.91.110 or if there is, then it has no service running on port 50020.
Also, are you trying to use the Hololens relay? As that is where the error is happening.
Thank you for the clarification, I now understand the issue. Aside from reinstalling, is there a way to directly change the Capture port so that it can open properly?
Hello again, I tried this but it still doesnt work. I started Pupil Capture with both integrated and dedicated GPU but I see the same error. Do you know any other options that i can try?
Does it fail with the same error with both the integrated and dedicated GPU?
Hello, I am interested in analysing pupil diameter and I have some data collected using pupil core and pupil invisible. I would like to check if the pupil diameter calculation methods in pupil core/invisible have the correction method for pupil foreshortening as Neon does (as indicated in the white paper) or would we have to implement one?
Hi @user-ebd8d5 👋 ! Firs, kindly note that pupil diameter is not reported with Pupil Invisible. See https://discord.com/channels/285728493612957698/633564003846717444/1408826828998578208
Pupil Core reports pupil diameter in mm provided by the pye3d model in the diameter_3d variable, and in pixels as observed in the eye videos in the diameter one.
Pupil size in pixels is dependent on the eye-camera to pupil distance and is not corrected for perspective. Pye3d, on the other hand, accounts for differences in eye-camera to pupil distances and corrects for perspective. See more here.
Note that on Pupil Core, you would need to have a proper model fitting and avoid slippage as much as possible.
Hello @user-d407c1 , thanks for the clarification. Are the perspective correction results of pye3d comparable to the pupil foreshortening corrections of Neon?
They are different methods to detect the pupil from start but also the eyeball model, thus not they are not directly comparable 1 to 1, may I ask what are you trying to achieve?
I am trying to evaluate the usability of pupil diameter as a metric for my analysis. And I have some data from pupil core, so wondered if I had to apply a model to correct for pupil foreshortening or not. From my understanding, Neon's outputs work very well even when we move our eyes away from the center
In short, no. Pye3D already builds and continuously updates a 3D eyeball model to account for slippage and PFE.
However, for pupillometry, you typically want to rely on a consistent eye model throughout the entire session, rather than one that keeps updating during recording.
This is why it’s common to have the subject perform eye movements initially to properly fit the model, and then freeze it. The trade-off is that a frozen model becomes more susceptible to slippage, since it no longer adapts over time.
Lemme know if that clarifies it.
@user-d407c1 : Thank you. How would you freeze the data in pupil player for "post-processing" (https://docs.pupil-labs.com/core/best-practices/#pupillometry) ?
Additionally, when I re-export the data from pupil player, my outputs are very different to the ones in exports/000 folder (which came with the data). Specifically, confidence level is only either 0.1 or 1. Why would this happen?
One last question. Plotting eye diameter, I get large jumps in diameter value (mm) (like either side of 4mm). Is this normal?
Hi @user-ebd8d5 👋 ! To “freeze” the model for pupillometry, you’ll do so on the eye windows, in case of Pupil Player, you need to reprocess the pupil data:
Select Post-Hoc Pupil Detection from the dropdown → This will reopen the eye windows and rerun detection automatically. Let this run while the subject performed rotational eye movements (this helps properly fit the model).
Then, pause the detection and on each eye window, go to Pye3D Detector (bottom of the sidebar with a 3D label) and enable Freeze model
You’ll need to do this for both eyes.
Regarding the differences on the export, this is likely due to export settings.
First, in the Raw Data Exporter, there is an option: - Include low confidence data
If this is disabled, samples below the default threshold (~0.6) are filtered out.
If your re-export looks very different from the original:
- Double-check export settings
- Ensure you didn’t unintentionally change detection parameters
- Visually inspect the eye model to confirm it looks reasonable
If in doubt, you can reset to defaults via the General plugin.
About these large jumps, have you already filtered by blinks? These actually look like blinks or partial occlusions, which typically shows up as sudden jumps or spikes in diameter.
I’d recommend removing samples during blinks / low confidence periods
@user-d407c1 : Thanks, I havent filtered the blinks out yet. I was curious why the signal jumps to either side of 4, for example in the first 100 samples. That is a jump of almost 0.3 mm
ohhh you mean not the greater jumps, but the small ones, which eye are you plotting, is there a chance you are plotting both?
Sorry, yes I was. I did not notice that column. Sorry
Np!
Hello,
We are currently carrying out some experiments using Pupil Core and we have the following situation.
We are monitoring two air traffic controllers who switch roles every 30 minutes. We only have time at the very beginning of the session, and once the experiment starts it cannot be interrupted. For this reason, we are considering placing the glasses on both participants at the same time (one pair each) and calibrating them during the five minutes prior to the start of the trial.
Our concern is the following: during calibration, each participant will be calibrated while looking at the monitor positioned in front of them. However, halfway through the experiment they will switch seats, meaning they will no longer be in exactly the same position in which they were calibrated.
We understand that this change will likely affect the precision and accuracy of gaze and fixation data. However, we would like to know whether pupil diameter measurements would also be affected by this change, or whether pupil size data is robust and largely independent of the distance and position from which the calibration was performed.
Thank you very much in advance for your help!
Hi @user-d03324 , if the Pupil Core headset slips, then all measures are affected. However, the pupil diameter measurements depend on the fitted 3D eye model, which can be re-fit when they sit back down, but would require some operator intervention.
To clarify, if pupil diameter is most important for you, then a Best Practice is to freeze the eye model at the beginning of data collection. But, this makes it especially sensitive to headset slippage. As such, when they sit back down, you would want to unfreeze the model, re-fit it, and then freeze it again. Otherwise, you are likely to get inaccurate data, at least in the second half of the recording.
I am not sure if this would be feasible in an air traffic controller context?
Hi, thanks for the answer. Would it be more suitable to do an offline calibration/detection using Pupil Player?
This could be done, by following the instructions here. You could also do a second calibration after they are seated. For pupil diameter estimates, they should just be sure to rotate their eyes 2 or 3 times after changing seats, so that the eye model can be fit anew.
As I am not sure of the constraints of your study, perhaps give it a test run first, so you have the workflow practiced. Using the trim marks (see item 4 under Timeline) will probably be helpful in this case.
Quick question, is the hardware for pupil core open source?
Hi @user-8f6642 👋 ! Do you mean the software? Yes, it is open source, or were you looking for the DYI version?
Never mind, found the guide!
Has anyone used Pupil Core eye-tracker to record optokinetic responses?
Hi @user-92a86b 👋 ! Here you have some examples, but looking at Google Scholar, PubMED or other repositories may yield more results.
or Pupil Core based VR Addons: - Josupeit 2023
Okay, thanks a lot. We are still planning the protocol to follow. Because the monitors they work with are very constrained, we will not be able to perform the calibration on them. Therefore, we are planning to mainly extract pupil diameter.
I take note of your suggestion about freezing the 3D model at the beginning of each test. Also, since they will be seated, I do not think headset slippage will be a major issue, except for the moment when they swap places.
If I have understood correctly, in order to re-fit the model it would be enough for them to make circular eye movements until the circumference turns dark blue and is properly adjusted again?? Thanks again 🙂
Hi @user-d03324 , that is correct. You are welcome!
Hi @user-3bcb3f 👋 ! Yes! That's totally feasible though it would imply modifying the screen marker plugin and the controller to invert the colors and to invert the image sent to the circle detector.
Hi @user-d407c1 could you guide me which lines to change exactly? Would it effect anything else ?
We can definitely provide some high-level guidance on how to approach this, but if you’re looking for a more tailored solution, like for example, us developing a custom plugin for Pupil Capture to handle this workflow or specific guidance on code changes, that would fall under our custom consultancy services.
With that in mind, you would like to have a look at this lines:
on marker_window_controller.py
- Marker colors invert the value
- Background : below 272 add something like:
gl.glClearColor(0.0, 0.0, 0.0, 1.0)
gl.glClear(gl.GL_COLOR_BUFFER_BIT)
And then on the screen marker plugin under __detect_reference_circle _marker do a bitwise operation like inverted_gray_img = cv2.bitwise_not(gray_img) and pass that image to the tracker.update function.
As for what else would affect, text for example you would need to change it and as it is changing the src, you would need to run it from source.
Sorry, I have another question. Comparing the world video from what was recorded during data collection compared to what is exported from Pupil Player, the video exported from pupil player is longer (and I am assuming the timestamps would be also). Could I understand why is the video from Pupil Player longer?
Hi @user-ebd8d5 👋 ! Do you mean between Pupil Capture and Pupil Player?
Hello @user-d407c1 , yes (I think so). I have a dataset containing subfolders of offline_data and export. So I am assuming the data in the parent folder is data collected from Pupil Capture.
Pupil Player can not export data longer than what it was captured but it could be that an export window was defined on Pupil Player, in which case data could be shorter on Pupil Player's exported data. Could that be the case?
Hi there, does anyone know if it is possible to define dynamic aois with core?
Hi @user-c62860 , the Pupil software does not have dynamic AOI support built-in. What kind of AOIs do you wish to track?
thank you for your quick response! 🙂 I would like to investigate eye movements while a person is working, for example with kitchen utensils, like bottles as AOIs
You are welcome. So, for that, a common solution would be to use a computer vision algorithm, such as YOLO or Meta's SAM 2 or 3 to detect and segment the objects. Then, you can have a script that registers when gaze is on the segmented object. You could write a Pupil Player plugin for this purpose.
Although written for Neon, you can see how we did similar for our latest eyetracker in these two Alpha Lab guides:
Thank you! So it has not yet been done for core, right?
I should say, we have not done it for Pupil Core, but as mentioned, the principles and even a substantial portion of the code will be the same as for Neon.
It is possible that a member of the community has done it and they should certainly feel free to hop in!
While we have not done it ourselves, you can also check our Publication List to see what others have done with Pupil Core.
okay, thank you very much, have a good evening 🙂
Thanks, wishing you the same!