hi. I can't get my pupil core to work. Grey screens only. Working on macOS Monterey12.2. Downloaded software from docs.pupil-labs.com today. Please help

thanks, that worked. Do I have to do that every time when starting, or can I set some startup options?

also: only one camera works. Error message: Eye0 can't set value

Hi Pupil Labs Team 🙂 I was wondering if there is an option to export the surface video as in the preview you can see in the analysis tool. When I export the data, it only seems to export csv-files, however, for further analysis it would be helpful for me to have direct access to the video file containing only the surface area. Thanks for your help in advance!

Hi, thank you for the help yesterday, also, I have a problem with the calibration of Pupil Core. I have been following the steps on the web but I am not able to get it. What can I do?

I am doing the calibration with "Screen Market" and with "Single Market", but when I finish the calibration the point of the gaze in the screen isn't where I am watching. The cameras to my eyes are detecting the pupils every moment. I don't know why it doesn't work.

The blue circle around the eye ball is right, I move the eye and it is adapted to my eye ball

Okay. I will send it as soon as possible. Thank you

Hello, I was working with the Pupil core and now the world camera is not working. I only see a gray screen when I open Pupil Capture and even the computer doesn't detect that camera. The others are working good. Some advice?

and even the computer doesn't detect that camera Are you referring to the device not being listed in Windows Device Manager? If that is the case, please write another email to info@pupil-labs.com in this regard. I have reviewed your shared recording and will follow up via email.

I only found an old answer in the discord from 2020 were it was not a feature yet. Did this change by now?

Hi! I have a problem with pupil labs. I was doing a registration and suddenly pupil capture didn't work anymore.

In particular the world camera. Instead the eye's camera work

Hi! I am having trouble with opening my pupil core data through pupil player. When I drag the output file onto pupil player the terminal error is regarding allow_pickles = False in the npyio.py file. I've gone through and changed allow_pickles = True and reinstalled with the new file, but I'm still receiving the error. Does anyone know a solution to having pupil player read file as allow_pickles = True or have an idea of what other scripts may be getting pulled in to create this error?

Hi, Is it possible to add an external video stream (e.g. from a webcam) to a recording when using Pupil Capture?

Otherwise, Pupil Capture does not have the built-in possibility to record a 4th video source.

@papr hi! I have a problem with pupil, can you help me?

@papr Hi! I have a problem with pupil labs. I was doing a registration and suddenly pupil capture didn't work anymore. In particular the world camera appear grey

Yes

Exactly

Eye cameras continue working

Hi, I have the same problem, eye cameras still working but the world camera view in the Pupil capture is gray. Did you solve the problem?

Please see the attached screen

One of the eye cameras on my pupil core device has stopped working properly. The image is so blurry. The reasons I can think of are either that the participant that I was recording took off the headset a little harshly, or that the experimenter has disinfected the headset and has put some gel on the camera. Is there anything I can do about this?

Hello, I have a question about Pupil data. Is there any problem about 3d_diameter when participant wear contact lens?

Hi 👋 The distortion introduced by contact lenses should be negligible, so no problem. 🙂

Hey I have a question about the process after the ellipse detection. I have an Asian friend filling in as a subject for a study and when I check the pupil windows the pupils are detected almost perfectly for every frame. But somehow, despite that, the POG that is detected is terribly incorrect. Is there any way for me to correct this post-hoc? I believe there are some physiological averages that the software uses which might be messing the results up in this case. Is there any way to correct for this?

Hey guys, I've recorded some goal directed saccadic eye movements and tried to measure the kinematic (main sequence i.e.) as a control of the eye quality. I've included only eye traces of highest quality with low noise levels and no blinks etc as benchmark for further recordings. I notice some curiosities in the data (see figure). 1) max peak velocities saturate quite early (pk velocities do not surpass 300-350 degrees/second). 2) the gain of the pkvel*saccade duration seems to be 1, suggesting a peak velocities that is also equal to the mean velocity somehow (?). What I suspect is going on here is that a too low and too long estimation of my saccade peak velocity and duration distort the gain potentially due to over filtering. However, these parameters have been extracted from unfiltered data (on my end that is, so if any it should be noisy but an over estimation of the peak velocity, right?). So I was wondering, is the data collected by pupillabs/lsl plugin already being filtered with a cut off frequency of below 75Hz? I feel the 200 Hz sampling frequency should have the potential to describe the full saccade kinematics. Note, figure is produced from one participant, I'll collect data from others to make sure he isn't just 'slow'.

Here are some figures to show the quality of the recording. See example saccade detection. Data was recorder at 200Hz. Saccades are detected by passing the 20 D/s velocity threshold, corrected onset/offset (with a max shift in 5 samples either way), and then evaluated based on some parameters (such as a window of high confidence > 0.9, minimum / maximum saccade duration and amplitudes). Only the cleanest examples are kept by the algorithm (i tried to be rigorous here) and then visually inspected to make sure they are of high standard.

@user-0b16e3 can you have a look at the above? Could there be something in py3d that acts as a filter?

Hi, in Lsl-recorded data, does anyone know how the 3d gaze data (e.g. in the streams gaze_point_3d_x/y/z) was converted into the normalized streams norm_pos_x/y ? I tried a simple normalization equation on the 3d gaze data but that did not really match the normalized streams.

Hello! Is it possible to run pupil core without running the programm as an admin? Or is it necessary?

Hi I am getting this error when running pupil-capture.exe. Anyone know how to fix it ?

the chinese in the log means permission denied

yes and I confirmed it

In this case, it is not clear to me why the permission was denied. Is this your personal PC or is it managed by your company/institution? One possibility I could think of is that there are additional security policies installed on the device, prohibiting the driver installation.

I thought it may be because of admin privileges. I tried to open it with admin but it showed this again. The software just doesn't detect cameras.

Hi, currently it is only necessary to do so on macOS. On Windows, once the drivers are installed, you can run the software without administrator rights. @papr How do I check if the drivers are installed? I installed the programm via the webpage, but it still asks me everytime to run as an admin (win 10)

They are just grey.

@user-619e31 could you please try following steps 1-7 of these instructions https://github.com/pupil-labs/pyuvc/blob/master/WINDOWS_USER.md

Okay thanks.

Hi Papr, Can you share the re-calibration steps please? I tried post-hoc re-calibration, but nothing has changed so why would this work? I am not sure. Do we need to change any physiological or other parameters? Also, there is no constant offset that I can remove using the manual offset step. It varies based on the gaze direction I believe.

Tja sk> Yes, this is expected. The software checks for the existence of the drivers and will try to install them if they were not installed previously. @papr Thanks!!

Do I understand it correctly that the id0 and id1 confidence parameter tells me how "good" the eyetracvking device is adjusted to the respective eye?

Yes, one could say that. Or more precisely: If the eye cameras are not adjusted correctly, confidence will be low. There are other reasons for low confidence though, e.g. during blinks. @papr thank you so much!!!!

I have a go/no-go task with four fractal images, during which participants should associate each of the four fractals to a decision, and all along I record pupil size. I need to make sure that the changes to pupil size come only from the mental workload and the learning process, and not the fractals differences in luminosity, etc. What are the image characteristics I need to check, for the fractals to be equally presented? Things like red and green and blue histogram? and luminosity histogram? Any references/papers about this?

Hi Papr, Thank you! Currently velocity is computed based on the norm_pos_x and norm_pos_y values (i.e. pl.Data(2,:) and pl.Data(3,:)). These traces are calibrated/mapped to known real world coordinates (to a 0.1 degree precision) using a simple neural network. We asked participants to look at targets and press a button to indicate they fixate their gaze to these targets (15 different positions throughout the field of view). 200 ms of eye traces are then sampled, and we map the median of these values to the actual known world fixed coordinates. The network has 2 inputs (x,y) and 2 outputs (norm(az), norm(el)), which we then scale to actual azimuth and elevation. The network has 1 hidden layer with a few units (3 or so). We then iteratively map each sample to azimuth / elevation, which perfectly predicts the location the participant is looking at. (As we do not use any memory in the network, it seems unlikely that any unwanted filtering may occur on this level). We then compute a saccade amplitude, R, using Pythagoras (i.e. hypot(az,el)). From R we take the two-point central difference differentiation algorithm to compute the velocity.

Targets are being presented in the real world, each target is an LED on a sphere at exactly 1 meter away from the eyes. The participant's head is fixed to not allow for any translational movement.

Hi, depending on the display size you might have issues with within-image illuminosity differences. These will be difficult to get rid of. Also, different light spectra cause different pupil dilation [1]. You might be able to work around that by converting the images into grey images. But I do not know if your experiment allows that. Could you tell us a bit more about the details?

[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6634360/

Hi Papr, we try to place 4 Apriltag markers on each corner of the screen to identify it as a surface as the means to get the gaze point on the screen, but the detection of some markers was not very stable, is there any way to improve the marker detection in post-hoc analysis?

Could you please replicate the same graphs (same filtering process) but instead of using your custom norm-pos-to-az-el conversion, use the gaze_normal0/1_x/y/z values of the same samples that you had chosen previously? To calculate the velocity, you can either calculate the cosine distance between two vectors (yields saccade amplitude directly) or use

r = np.sqrt(x ** 2 + y ** 2 + z ** 2)
el = np.arccos(y / r)  # for elevation angle defined from Z-axis down
az = np.arctan2(z, x)

Note that the functions above yield radians, not degrees.

Sure, do you know which streams those are; idx that is?

Ok thank you. I'll post them once i've finalised the figures! 🙂

No. What values are you getting?

Here is the full error message. I've tried multiple files and they all produce the same error. I realize that the system I am using is slightly modified, so I would like to allow pickles, even if that isn't conventional for the code.

What do you mean by you tried different files? Do refer to recordings?

Could you please list the files of recording folder that you are trying to open?

Hi It's been a while since I was last on here. I have a few more questions about using the core for my experiment. Specifically I wanted to double check a few things about circle_3d_normal_[x,y,z].

1) Is circle_3d_normal on a linear scale? e.g. if the value changes from 0 to 0.1, is this the same amount of eye movement as if it changed from 0.9 to 1?

2) Are the values of circle_3d_normal_[x,y,z] extracted from each eye camera comparable to each other? For instance, if both eyes move by a circle_3d_normal value of 0.2, does this mean both eyes moved the same amount in a given direction (eg. they both moved 2 deg)?

3) I guess this one depends on the answer to 1) and 2), but can the values of circle_3d_normal_[x,y,z] be used to calculate the angle between the two eyes (assuming eye gaze is not perfectly parallel)? For context, I currently use circle_3d_normal_[x,y,z] to calculate the angle between the position of the same eye at two different time points, but we would also like to calculate the angle between the position/direction of the two separate eyes at the same time point.

I also have a few question about extracting direction information but they can wait for now. Many thanks!

Hi, circle_3d_normal is part of the uncalibrated pupil data (in eye camera coordinates). Re 2 and 3: You want to use gaze_normal0/1_x/y/z and eye_center0/1_x/y/z from the gaze data. The former corresponds to circle_3d_normal but in scene camera coordinates. Re 1: Yes, these are in an undistorted euclidian space.

Hello papr, I am running the post-hoc pupil detection on Pupil player, but once I start this procedure the gaze point on the the image dispears, do you have any idea to make the gaze point show again? thanks

Hello @papr Is it possible to get spectrogram of the raw signal (possibly eye movements ) recorded from Pupil Core ? Otherwise, do you have recommendation of obtaining the spectrogram of the raw signal ?

Thanks, but I after I clicked "Dectect References, Calculate All Calibrations and Mappings" button, there still no gaze point showed up. I am wondering is there anything I missed?

Hi @user-205d47. It would be helpful for us to learn more about your aims with regards to frequency analysis of the raw data. What exactly do you intend to do with a spectrogram?

@papr Hi there, I've repeated the main sequence+ and "skewness" relationships for the same saccades i've posted earlier, albeit computed from de gaze_norm0/1_x/y/z streams. Note that the left (blue) and right (pink) are pretty much the same. Correlation between the R (dimensionless movement) of both eyes is almost 1. Overal tendencies between both graphs from the different pupil labs-LSLS data streams are very similar I would say. Peak velocities remain low (even a bit lower then before). while ideally, these should be near 750d/s for longer saccades. I've also include the relation ship between accelerated duration and saccade duration as a function of amplitude and duration overal (blue and pink completely overlap).

Thank you Pablo for the explanations. I tried cleaning the lens and it's a little better now, but the red circle around the pupil keeps jumping all around and I guess it needs to be focused. I got these messages when I open Pupil Capture, and it seems that I have the Pupil Cam1, whose focus is adjustable, right? If so, can you please tell me how to adjust the focus? I didn't find the adjusting screw thing shown in the videos.

Is there any CPU restriction for Pupil Player 3.5.8? Is the eye detection multi-threaded - i.e. can it use any more than 2 cores? I am trying to speed up my data analysis using a virtual machine on Linux and despite having lots of CPU, it appears even slower than on my desktop computer

Hi we are running an image viewing task in Inquisit. The task will have components from which we don't need eye tracking data, but the AOIs that temporally appear about 1/3 of time, do need to be analyzed for AOI gaze time and blink rate. The Pupil Core recording will take a while to process (60 images per participant) if I use Pupil Player to individually find/export each images recorded AOI surface gaze data. Is there a way to batch download a participant's AOI gaze data for all AOI's (60) at once?

Thanks for sharing the log. It is important to look at the IDx suffix. 0 and 1 refer to the eye cameras. ID2 refers to the scene camera.

eye1 - [INFO] video_capture.uvc_backend: Found device. Pupil Cam2 ID1. eye0 - [INFO] video_capture.uvc_backend: Found device. Pupil Cam2 ID0. You have the 200 Hz non-adjustable-focus eye cameras. I will ask my colleagues for further cleaning tips.

Do I have to check for any size or material specifications so that the lens pen fits in the lens and properly cleans it?

@user-98789c Otherwise, before purchasing such a lens pen, please contact info@pupil-labs.com s.t. we can set up a video call and a colleague from the hardware team can look at it.

Hello! We are looking into purchasing the pupil core with the mounted scene camera and were wondering whether the calibration works with curved surfaces? We have a large parabolic screen (height x diameter: 3.5 x 3 m). Can the matching be done by the provided software or would we have to do it ourselves?

Hello! I am trying to use remote command ""T {time}" to resets current pupil time. But after this command, I am not able to give another remote command like "t". the error report is "ZMQError: Operation cannot be accomplished in current state", can anyone help me on this ?

@papr @user-92dca7 @nmt Hi guys, wasn't sure wether you caught my previous post with the figures you asked for. If not see above 😉. Well anyways, I hope to hear from you with regard to max saccadic velocities and wether there may be any process that acts as (LP) filter of sorts. Kind regards, Jesse

@nmt Splitting the screen into flat regions might be sufficient for our needs and would save us some trouble. Either way will have to try it out in practice and see if extrapolating rays from the eyes in the world view and projecting onto the curved surface would be necessary then. Thanks a lot for the suggestion, I appreciate your help!

Hello ! 🙂 Yes, I am surface tracking and using April Tags to define the computer screen and post-hoc manually using the "add surface" feature to digitally define image AOIs. The image AOI's occur about every 12-19s, then not again for 8-10s, so there is parts of the recorded world video for which gaze time doesn't need to be analyzed. It's taking a long time to go through to define each "add surface" then export data for each. (I have 6000+ images lol) Is there a way to annotate and batch download the data for each participant world video recording (100 images) without averaging out all the gaze time for that recording?

Right, thanks for reminding me!

it's okay for me to use circle_3d_normal_[x,y,z] (taken from from a frozen eye model) to calculate the angle between the position of the same eye at two different time points, but I can't use circle_3d_normal_[x,y,z] to calculate the angle between the position of the two different eyes? Correct, if you want to look at absolute orientations! You can easily compare within-eye differences between models.

Hi @papr, thanks for coming back on this. Potentially i've found the error already. However, I'm not sure I fully understand how lsl extracts and writes data from pupil labs particularly when recording with two eyes. It appears when I look at the pupil-labs-lsl timestamps they seem to be consistently separated 2.5 ms apart (opposed to the expected 5ms at 200 Hz). I assume this has to do with two eyes recording simultaneously and alternating samples represent the 2 eyes(?). But then how do I know which sample belongs to which eye? and also are both eye cameras then really recording in opposite phase at exactly 2.5ms interval?

Hi @papr @nmt I am using the VR add of Pupil Labs for measuring pupil dilation in a VR environment. Subject fixates on a red dot in presence of a grey background. But we are having troubles with getting clean and stable recordings. Pupil detection flickers (see attached image and link for the video ) and yields low confidence. We also see a few reflections in the eye, not sure of the source yet but could this be the reason for back tracking and do you have any suggestions on how to get a clean and stable recordings from the VR addon of pupil labs ? https://filesender.surf.nl/?s=download&token=4c883088-d356-4e1c-986e-7be2bcd4cfe3

Hi @papr , what effect does the size and width paramters in the surface definition GUI have? What is the "real world size" mentioned in the documentation? Is it cm,m,inch,miles? Is it the real real world measures, like measured with a ruler? Thanks a lot

Hi there! I have a question related to the setup of the software which is I think is just a problem due to the IT department and the restrictions we have... I install the software and after a while they are blocked by the IT software called Carbon Black Cloud. The IT guys told me: "Please reach out to the software vendor and have them provide an anti-virus exclusion list for us to whitelist." Is this something that you can provide about this??? Thanks for your help!

It is meant for scaling of the heatmap and gaze data. Gaze is mapped primarily to a normalized coordinate system. You can enter the most sensible unit, e.g. pixels for a screen. See also https://docs.pupil-labs.com/core/terminology/#surface-aoi-coordinate-system

It would be helpful to know about the format of that list. If you could point us to corresponding documentation that would be helpful. I can look into it, too, but earliest next week.

Hi @user-205d47. Thanks for sharing the video capture. It looks like contrast between the pupil and iris is low, which is hindering the pupil detection. Please see this message for steps you can take to improve pupil detection (for VR eye tracking, steps 2–5 are relevant): https://discord.com/channels/285728493612957698/285728493612957698/943090995917103144

@user-205d47 If you haven't already, I'd also recommend reading our Pupillometry best practices: https://docs.pupil-labs.com/core/best-practices/#pupillometry

Hello I am currently running an experiment where participants will view visual stimuli through a mirror stereoscope (a different image presented to each eye separately through each mirror). I will simultaneously track their eyes using Pupil Core. Since they will be wearing the glasses (and therefore the mirrors won't block the infrared light) does it matter what type of mirrors we have (hot vs cold mirrors)? Thank you!

good morning. Has anyone managed to open the mp4 files from Matlab on Mac? I only get errors

or, alternatively, is there a way of reading the pupil dilation from matlab? Or is the only way to run the pupil_player first and then export the results to a csv? I only need the dilations, ideally directly from pupil_capture

@user-ed3a2a @user-027014 By default, Capture records mjpeg data and Quicktime Player does not know how to play that back properly. They should work correctly using VLC player. See also https://docs.pupil-labs.com/developer/core/recording-format/#video-player-compatibility

is there a way to 'remote control' pupil_player? No offence, but the documentation is somewhat lacking and hasn't been updated since 2016

it looks as if this file only works from already generated cvs files?. I assume there is no Matlab solution?

thanks! A little more help please, as there is 0 documentation. I worked out how to start and stop a recording. How do I query the pupil size?

What works! Ideally online via send and receive notificaitons.

matlab via zmq via the given 'pupil_remote_control' example.

Ideally I just need to extend that to query some other info

thanks, I will do that and come back to report if it works! Will take a few hours probably.

it seems to work so far, but another question in the middle. every day or two pupil_capture stops working. Terminal throws 'world - [ERROR] video_capture.uvc_backend: Could not connect to device! No images will be supplied.' I then have to reinstall software from scratch. Any idea?

no, MAC

ok, point taken. Just having the situation again. Restart doesn't help, same error.

Do they need to cool down or something between uses? is matlab changing anything?

Should we quickly jump into a video call via the Discord voice channel? Then I can check it out.

yes please

how?

You should see a channel called pupil-voice on the left. Just click on it and you will join the voice/video chat.

Hello Everyone, I was trying to connect my Pupils Eye tracker to LSL but it’s not connecting does anyone know how to fix it? Thanks 😊

Hi Neil thanks for replying me, Basically, I used this blogspot to follow the steps https://ws-dl.blogspot.com/2019/07/2019-07-15-lab-streaming-layer-lsl.html?m=1, and I ended up here without having any error but once I go to install there’s no any lab recorder.exe file generated.

yes, I was about to tell you that thing as well I installed it but the pupil's eye tracker is not linking to the lab recorder, I was wondering how to link the eye tracker device so it can show under the "Record from streams"? Thank you.

Great. The next two steps are: 1. Install pylsl and copy or symlink to the Pupil Capture plugin directory 2. Install our LSL Plugin to the Pupil Capture plugin directory Detailed instructions available here: https://github.com/labstreaminglayer/App-PupilLabs/tree/master/pupil_capture#pupil-capture-lsl-relay-plugin

I also noticed that the tutorial you were following linked to an old version of Pupil Capture. Be sure to use the most recent version: https://github.com/pupil-labs/pupil/releases/tag/v3.5

Thanks, Neil, I did that steps as well but still it's the same issue pupil's eye tracker is not linking with the lab recorder and once I enter start there's an issue coming as " Missing: Qset()"

Could anyone be able to help me with this issue please?

Hello

Please eye cameras have a sampling rate of what ? We calculated 140Hz. although the laptop is corel i5, the memory is really low. We used a resolution of 400 X 400

I can see that sampling frequency 200Hz at 192X192 is in the spec. I chose 400 x 400

The programs are installed into C:\Program Files (x86)\Pupil-Labs, more specifically

C:\Program Files (x86)\Pupil-Labs\Pupil <version>\Pupil <app name> <version>

e.g.

C:\Program Files (x86)\Pupil-Labs\Pupil v3.5.1\Pupil Capture v3.5.1

The executables are:

pupil_capture.exe
pupil_player.exe
pupil_service.exe
PupilDrvInst.exe

Hello, I have download the core program from the website and when I plugged the glasses, it says that the device is found but there is no image. Could you please help me? I am working from a Mac...

can somebody help me?

Hello, I am facing couple of issus (1) I am doing the conversion from Pupil time stamp to System time stamp . After the conversion I can see that the System time is actually 3 hrs lesser then the exact system time of my computer. I am doing the whole conversion with by help of the tutorial provided on the Pupil Labs website.

(2) I am running an experiment where I need to send tags for every image presented on the screen( Image start time, Image end time and Image No.) Can someone help me out and let me know how I can do that using Pupil Labs eye tracker. (3) At last but not least, can we connect Pupil Labs eye tracker with Psychopy?

It will really helpful if anyone can help me out to figure out the solutions. Thank you so much.

Good morning, Is there someone that can help me with my problem? I need guidance to know what I am doing wrong!

The maximum sampling rates are 200 Hz at 192x192 120 Hz at 400x400

The real sampling rate might be lower due to insufficient cpu resources.

The sampling rate of the gaze (combined data from both eyes) might have an higher sampling rate.

Hi, thank you for sharing the video. As you can see, the 2d pupil detection (yellow circle) is still being fitted to the iris instead of the pupil. Please reduce the pupil max parameter in the 2D Pupil Detector settings until this does no longer happen. You might need to reduce the Pupil Min value, too, but this is difficult to tell from the video.

thank you for your answer could you plese tell me where are those release notes?

I will have to come back to you in this regard. I will ask my colleagues for best practices.

The reflection only become a problem if they lie on top of the pupil border. In these cases, the confidence values will be lower than they usually would be, but the gaze estimation should be affected much. Regarding the eye camera positioning: One option is to use the VR headset's built-in IPD-adjustment functionality. Based on the shared video, the positioning is ok, though. The eye ball is fairly centred. 👍

Hello Papr, thanks for replying, Here is the capture.log file.

Sure, Tell me what do.

to*

@user-e29c16 could you please replace the outlet.py file with this one? Afterward, start Capture. The LSL Relay plugin should be listed in the plugin manager. Enabling it should cause the stream to appear in the LSL Recorder app.

Great, thanks for testing! I will release the fix now

Hi papr, after running a few trails there's the issue that comes with camera, It's like if I turned on both the eye0 and eye1 the main camera is getting disconnected if I turned off the eye 0 and eye1 the main camera is getting activated again, could you be able to help, please? Thank you.

So, first I start capture after that everything works smootly and i tried enabling eye0 and eye 1, once i enable them the main cam disconnected. Yes i'm using pupil core headset.

Now, it's keep saying WORLD: Camera disconected. Reconnecting...

Please try restart with defaults settings. Both eye windows should appear and all three cameras should work as expected.

Would it be possible for you to share a raw 1-minute-long Pupil Capture recording with this exposure setting to [email removed] Having access to the raw eye video data allows me to give more concrete recommendations regarding the parameters.

Could anyone be able to help with reading the data recorded from the lab recorder by using matlab? Thanks.

Thanks for replying, I tried that one and I gave the correct path but after trying to load, it gives me an error like this "Unrecognized function or variable 'load_xdf'."

This is the file I'm trying to load.

Thank you so much again, I'll check with that.

Hi @papr I am currently having a pupil headset and trying to use it with a zed stereo camera. I know that pupil capture can time sync all the nodes in one pupil group automatically. I tried to open the zed camera by pupil capture and create a pupil capture instance, but it failed.

Hi again! I would like to add reflective markers for motion tracking to the pupil-core frame. Do you have any experience with certain shapes? 3D prints? suggestions? Thanks!

@papr Hi! I want to make an app that shows gaze points in real time . I wonder if this can be done with MATLAB

Hello all,

I noticed that the eye camera image for the left eye is blurry. However, the focus wheel is clued in the case of the PupilLabs Core.

Is it possible that the focus adjustment wheel moved somehow? No matter how I adjust the eye camera, the confidence of the left eye is always a bit worse.

Can anyone help me with this ? Look at eye 1 - the lashes

Kind regards,

Cobe

Hi, are there any requierements for downloading the pupil core Software? Like how much ram etc is needed

And is there a recommended software for analyzing the Data?

pupil player is pretty great...

it gives you all the csv you will need

The required RAM is difficult to estimate as this heavily depends on the recording length and the detection features being used. @papr Thanks! So there are no specific requirements that my device should have?

For Pupil Capture, a fast CPU is most important (>2.8 GHz Intel Core i5). For Player, RAM is most critical. We recommend 8GB or more. @papr 👍

On the website it is written, that in order to properly adjust the eyetracker, a green circle should appear in the eye camera. however only a light or dark Blue circle and a smaller red one with a red dot appear. Where did I go wrong?

Thank you so much, this is incredibly helpful! Is there anyway to get the amount of eyemovements made (so saccades essentially) and the pupil diameter? Or do I have to import a plugin for that?

Hi Papr, I Just did some recordings using lab recorder but seems like none of them recorded any numerical data, after installing the EEG toolkit to Matlab and tried to read but when I checked the cells their nothing. Could you be able to help me with this? I'll attach the .xdf file and could you tell me is it ok or is there anything missing with the file? Thank you.

I just wanna make sure the data is recoding properly from lab recoder and saving as .xdf file without missing any?

Has anyone encountered a "moov atom not found" error when trying to view their world.mp4? Does anyone know how to troubleshoot this other than trying to repair the file post-process?

I have tried to reboot my Android device but still face this issue.

Hi, Thanks for replying, I just used the default settings and I didn't change anything. if I have to change before recording what should I need to change?

Im having the issue, that fixations are not detected with the respective plugin. Although pupil confidence is almost 1.0 for both eyes, and I am consciously fixating a spot for a few seconds, the plugin doesnt detect any fixation. What could be the mistake?

The detection can take a moment to complete. On the right, you should see a progress indicator when the detection is performed. Can you confirm that? To reinit the detection, change the gaze source temporarily from "Gaze from recording" to "post-hoc calibration" (or vice versa) @papr Wow thanks! My mistake was using the posthoc option and not the recorded one. Thank you so much!

Just to clarify: Using the post-hoc calibration can be a valid option! Just make sure to set it up and calculate some gaze first. Without gaze data, the fixation detector won't have any input. @papr Makes sense! I will try some options to work things out!

Does anyone have any idea why my world camera is blurry?

I'm just getting this message.

@papr Hi, I would like to know the unit of the "gaze_point_3d_xyz" in "gaze_positions", is it in "mm" or "pixel"?

Many Thanks~

Hi all. I trested several times the calibration in 3D mode and 2d mode. and all the time we had better results with 2D.

Is 2D more preferrable? or it could be related to usage of older (2.3.0) version of software

(i have data uploaded if needed)

we made many tests and 3d (on 2.3.0) is always worse.

I also wated to know if surface tracking data is sent via network api if plugin is enabled

are there any network api changes which can cause backward compatibility issues?

Thank you. I hope we'll be able to adapt without many issues to latest releases.

I've been having a lot of trouble with my Pupil Core picking up the right eye's pupil. It's seems to be consistently bad across participants. I'm not sure what else I can do besides adjusting the 2D settings and resetting the 3D model every now and then. Any help would be appreciated

Hello, is it possible to calibrate one eye at a time. Im using an older version of pupil capture. Do you simply just close one of the cameras?

Hi, new calibrations overwrite each other, i.e. with your approach only one eye at a time could be calibrated. Could you clarify the requirements a bit further? 1) Do you have one or two target coordinate systems (example: single scene camera vs dual-display hmd) 2) Are your subjects to fixate the gaze target(s) with both eyes simultaneous?

There are some custom calibration choreographies + algorithms but they all require Pupil Capture 2.0 or newer.

Hi I am currently using the pupil core device and I would like to prototype a wireless use by sending the video of the three cameras to a computer, is it possible to do the pupil tracking on a recorded video using pupil capture or another software ?

These issues have been happening for about a ~year (maybe longer). The issue is that the algorithm just seems to be having a ton of trouble locating the pupil, especially when the eye moves towards the nose bridge. I can get an image, but I'll have to get one of my research associates to do it as I don't have access to the hardware atm

Hey is there any way I can get the specs on the camera module for the VR/Ar camera module? I’m writing a research paper on it

Hi again, has anyone ever used the Pupil Core device without usb wire, as I am trying to mount in on Hololens 2 I would need my user to be able to move anywhere without cable ? I thought that I could also mount a Raspberry on the headset to do the pupil tracking with pupil capture on the Rapsberry and then send it to my computer, yet if someone has a better solution I would be glad to here it. Thank you

Also is it possible to order only the connection wire (no cameras no frame) ? as I already have a pupil core device

Yes, I only want to achieve time synchronization between the Pupil and the zed video stream

Hello, I have pupil core product with world camera and eye0 camera. I am trying to use pupil core with raspberry pi. Raspberry pi is just for the streaming video over WiFi to another PC where pupil capture is runnig. Is there a way or example project about this? I tried Pupil Video Backend, but I could not run the program. Somehow it crashes. Any help will be very beneficial for me. Thanks in advance.

Hello, I bought Pupil Core. Where can I check the serial number?

Do you think a Raspberry could compute the pupil tracking ?

Okay, maybe send the video to a computer through the raspberry might be a better idea ?

Thanks a lot !

then this is for you https://gist.github.com/papr/fcbafd5cf748c9b11e64a4dd37ec8e9a

No, 2.0 or newer is required 🙂

what is your reason for using this old/ancient version? 🙂

The default calibration method assumes binocular vision. I do not know if that would be ok for your. Can you tell us more about what you are trying to measure? And how about running Capture on a separate computer?

Hello. I have access to a pair of the vr-ar googles. But I just want to use then for eye tracking on a computer monitor. And the world cam is giving me some issues. I have gotten it to work by attaching the googles to a pair of glasses, and using an old webcam as the world cam, but this is a very awkward setup. Do you have any ideas as to how I could spoof my monitor output as the world cam, or how to circumvent the world cam altogether? Again, for my use I only need to track gaze on a screen with no head movement. Thanks in advance.

I think assuming binocular vision is ok. I'm just not sure what the best way is to calibrate through the mirrors on the two separate monitors

Hi Papr, Because of your Help I finally go the data for that thank you so much! , So in Gaze --> we have 22 channel count data points for each time point and in Fixations --> we have 7 channel count data points for each time point, I was wondering if you could answer me the difference of gaze and fixations data stream and also wondering what each of these data points represent. Thank you so much again kindly, for helping to understand.

hi~I have a question about the code in the pye3d/base.py in the image. Why does the right gaze_2d point towards the sphere center? I found that it’s stated in the paper that we should disambiguate the right gaze vector to be the one pointing away from the eye ball center.

Hi @user-371233 👋. This should be possible, but there are certain constraints.

In order to understand where the wearer is looking, it's necessary to calibrate the eye tracker. During calibration, the wearer fixates or follows a known target (reference location) while Pupil Capture records and matches pupil data to these locations. Normally, these reference locations can be displayed on the monitor and picked up by the scene camera.

With no scene camera, you could write a custom calibration plugin for Pupil Capture that collects reference data from the system that also drives the monitor. Important to note: The monitor would need to have a fixed spatial relationship with the wearer in order for the calibration to be accurate – like a head-mounted VR system.

An alternative approach to make the system less cumbersome might be to de-case your webcam. Then you can use the default calibration, and subsequently April Tag markers to map gaze onto your screen: https://docs.pupil-labs.com/core/software/pupil-capture/#surface-tracking

Hello, will the HTC Vive binocular add on work with these virtual reality glasses? https://www.shinecon.com/vr-glasses/vr-glasses-for-mobile-phone/vr-shinecon-hot-seller-vr-headsets-vr-glasses.html

Hi, thanks for your question. I discussed it with @user-92dca7 and you are right, the correct 2D gaze vector, v, has to point in the opposite direction of the line from its 2D starting point, p, to the estimated 2D sphere center, c. Mathematically that means: v*(c-p)<0. Note, there is a typo with respect to sign in the original Swirski paper. In our code, we are effectively testing for an equivalent condition, namely v*(p-c)>0. For this note, when dot_products<0, we use the 2D gaze line which was NOT used to calculate dot_products in the first place, but the other one (np.where(dot_producs, 1, 0)).

That depends if the VR headset has the same geometry as the HTC Vive. The add-on clips on https://pupil-labs.com/_nuxt/img/htcvive_e200b.adba5ce.png

You can buy the cameras + cable tree without the casing if you want to engineer your own casing.

Hello, I was just wondering if there was anything already created that allowed the glasses to be used in real time? I've been using the core for a Senior capstone project and I have somewhat of a working program but thought I ask before diving into a rabbit hole.

Thank You so much for your response again, I used MATLAB to load my .xdf files for that I installed eeg libraries, but after loading the .xdf file it does gives me the data but it's not showing what it represents is there a way to get it correctly without using MATLAB or how you suggests me to get this in MATLAB. And also for the visualizing what kind of tools should I need to use, is there a way to get live visualization if yes how can you describe the process for me please? Thank you!

Hi @user-99bb6b 👋. Would you be able to elaborate on you use case? When Core is connected to Pupil Capture, our desktop software, both gaze and pupil data can be recorded and streamed all in real-time.

I can't do it at the same time, but do you think it would be sufficient to just do a binocular calibration on just one of the monitors (each monitor is identical to each other in size and distance)? Would this give me a meaningful calibration?

I guess I was just imagining that they directly look at one display instead of viewing the display through the mirrors?

Is there any advice on how to set up the glasses in a way that allows to capture the pupil in extreme angles? I have really been testing all kinds of stuff, but when looking down as far as I can, the confidence for both eyes always decreases below 0.5 I have been watching the yt Videos but still feel like I am doing something wrong. Is it just that "hard" or what could be the issue?

It is likely that the eye lids are occluding the pupil in this configuration. You can use the orange arm extenders to place the eye cameras more frontal and further down to the eye. This should give you a better view on the pupil in the described situations. @papr

Thanks! But is it possible that there are simply angles that cannot be captured because of the eye-lid und eye lashes? Or is the camera made to capture every angle?

The former. Pupil detection quality will always depend on the actual eye camera position. While Pupil Core is designed to be as flexible as possible regarding eye camera placement, it is still limited by the length of the eye camera arm, the range of the ball joint, and lastly the length of the eye camera cable.

You can find the arm extender geometry here https://github.com/pupil-labs/pupil-geometry/blob/master/Pupil%20Headset%20triangle%20mount%20extender.stl You are free to modify it and 3d printing your own arm extenders that are more suitable to your use case (remember the cable length constraint though!). @papr

Thank you so much!

But in your code, you use dot_products<0 to find the gaze_2d direction, and you said that v*(p-c)>0 should be the right condition. 🤣 😂 🙂

Maybe, I should really just change the few lines of code to avoid any future confusion about this.

Ok so thats what the dual display HMD calibration does? I'm working on setting up a separate computer so I can use these newer plugins. I'll definitley look into these plugins and will probably come back with more questions 🙂 thank you so much for your help, i'm very new to pupil core!

Sure, no worries. 🙂 Happy to help!

Hello everyone, We have tried using the plugin pupil_capture_lsl_relay. The LSL stream is detected by our program but no data is then received. We believe it could be because of the error that we have when activating the plugin in Pupil Capture, i.e., "Couldn't create multicast responder for 224.0.0.1 (set_option: An operation was attempted on a host impossible to reach)". Have you ever had this error? Do you know what is its cause? Best, Puzzled researchers

Hello all! I wanted to know if itt is possible to set up a config for pupil capture which will always set same settings every time pupil capture starts

Every time we start the app - it uses same defaults - with resolution 1280-720 instead of fullhd, manual exposure, wrong settings for network api and calibration mode.

so any start of the app causes the operator to set it up the same way.

If there is any settings or config file to make settings permanent - it would be extremely welcome

Are there any way to remove incompatibility issues and re-save latest correct config

Hello, I would like to know if it is possible make the calibration of the gaze from Pupil Mobile directly, without using Pupil Capture

So for example, if I want to use Pupil Mobile to record a video and I want to know where I am looking at, then first I have to make the choreography in the video and after that the gaze will be detect in the rest of the video, right?

Okay, thank you

Another question, for IOS is it available Pupil Mobile?

is Pupil Mobile available?*

Then, is there another app mobile that I can use?

Okay, do you have any documentation about Raspberry Pi with Pupil Capture?

Okay, thank you

Hello, I want to know whether we can set AOI using heat map?

The existing client is just meant as a template/example implementation on how to interact with Pupil Capture and how to generate reference data. I highly recommend adapting the script to do two things: 1. Display a target stimuli on each display (ideally looks like one for the subject) 2. Send the corresponding target locations for each display to Capture (see https://gist.github.com/papr/fcbafd5cf748c9b11e64a4dd37ec8e9a#file-dual_display_choreography_client-py-L137-L150)

As a result, you will get gaze estimates for each display separately.

The base implementation will be inaccurate as the target locations are not reproducible (e.g. "center" will mean something different for each subject).

Hi @papr quick question; How similar should gaze traces: {'gaze_normal0_x'} {'gaze_normal0_y'} {'gaze_normal0_z'} {'gaze_normal1_x'} {'gaze_normal1_y'} {'gaze_normal1_z'} for the left and right eye be? I'm analysing some data and i find for all participants fairly large differences. Overal shape between left and right eye seems to match, yet scale does not and also i notice an offset between both eyes when i compute gaze with these traces.

by shape i mean the general rough estimates of where the eyes looked. See image. Right is orange blue is left. Targets were at 1m distance, so convergence shoudlnt realy play a role here. Gaze computation was as described by you in a previous post:

r = sqrt(x.^2 + y.^2 + z.^2); el = acosd(y./r); az = atan2d(z,x);

So what I mean here is, yes, they do look similar but also they differ quite a lot in terms of their offset and scale. So which eye is correct?

Thanks for getting back. Ok, so I take it that these differences are to be expected and reasonable when comparing the individual eye cameras? Regarding your question; data is not filtered in anyway. Only NaN values are removed in order to compute gaze.

Some differences are expected. The amount of difference will depend on the factors mentioned above. If you want, I can try to reproduce the graph with my own data the week after next.

The former is correct. If you don't need to rely on the calibration (don't need scene camera coordinates) or are planning on building your own calibration, have a look at the circle_3d_normal pupil data.

I am not sure if I am just not looking in the right place or if it just a so simple I haven't figured it out but is it possible to receive the data from the glasses while they are recording(basically real-time data capture)? Or a way to basically open pupil capture and press export for a previously recorded file?

Hi! Does the HTC vive eye-tracker use corneal reflections to track gaze? It seems like Pupil Labs is using "dark pupil" detection is that correct? Then what is the use of the IR LEDs and IR camera's, in our eye-image I see 5 dots around the pupil, which I assume are coming from the IR LEDs? Thanks for the help!

Hi, I'm Hani, I'm a new user of pupil core. May i know, if i want to get the Area of interest result, did i need to set the marker before recording or after recording ? and Could u please advice me how to set up it ?

Hello, I am wondering how can I run 2 Pupil trackers (using Pupil capture) in a same PC? Also, for saving the computational cost, is it possible for me to firstly use Pupil capture to perform the calibration, then I can run 2 Pupil capture using python APIs (and linking to the corresponding calibration files)

Hi, can you download gaze overlay footage? The description of it in the product overview says you can, but it doesn't seem to be an option

It's recommended to have two PCs, if possible. That said, you can open two instances of Pupil Capture on the same PC. To reduce processing requirements, I'd recommend ensuring that everything is set up correctly (eye camera positioning etc.), then disable real-time pupil detection. Calibration and pupil detection can be performed post-hoc. Note that you'll need to record a calibration choreography, one for each eye tracker. Read more about post-hoc pupil detection and calibration here: : https://docs.pupil-labs.com/core/software/pupil-player/#pupil-data-and-post-hoc-detection

I am using the glasses to capture a user reading text, whether that be a sign or a book. I then take that image or video and convert the words into text. I am trying to test the software that gets the words from the image or video in 3 different way. The first being with images from the world.mp4, then just the world video and finally in real-time. Once I get them working I can optimize the real-time accuracy of the other software to make it faster. The issue I would like to start with is exporting the data without opening pupil player and pressing export. I was maybe thinking of using CMD commands, or if could use the pupil remote to just export it similar to how you would stop and start a recording.

Hi I am still working on the wireless connection with pupil core, it appears that the github backend that you send me seems a bit slow. I am streaming the video from the raspberry to the port of my computer, the video has no latency when I watch it, yet when I use the backend to send it on capture it becomes REALLLLY slow, I don't know if it is my CPU but I don't think so, I think the github backend is not very well optimized, I would like to add a direct link between capture and my video stream on port but I don't know what file I should modify to do this in your github files ? Do you have any idea ?

have no response from pupil-labs staff! trying to buy core product and get tech data on NIR led wavelength (850nm?) custom app. anyone has a phone or email contact for pupil-labs tech/sales staff?

Well I made some changes and it is now better, I decreased the image resolution, the thing is that the problems comes from the backend solution, I guess that comes from the notification system, sending a payload with a "high resolution" image (640x480) might be hard sometimes I don't know why, it is working pretty well with (320x240)... I would still prefer to configure my own source over the network rather than use the notification system, I tried to use usbip to connect over IP, capture detect the USB but then when I select it... capture crash... If there is no other solution I will keep working with video backend plugin i think

You can indeed perform a calibration first to check that it worked okay. Important note: Make sure that you record the calibration, as disabling real-time pupil detection will also disable real-time gaze estimation. You'll need to run pupil detection and calibration post-hoc.

Hello, I would like to know does pupil cloud charge?

Hi all, I'm new here. Hoping someone can help me with a technical issue with my Pupil Core. I own two Pupil Core headsets, both with bilateral pupil cameras and world cameras. One headset works just fine, the other displays the pupil cameras but returns the attached error when I start Pupil Capture. My inclination is to think that the world cam is broken here, but I'm not sure how to check this. I'm seeing (2) Pupil Cam2 ID0 cameras in my device manager under "libusbK USB Devices" but can't seem to identify a world camera. Any advice here?

Hi guys，I want to calculate the actual distance of eye gaze from one point to another point on the screen(ie. which is calibrated), but the norm_pos information obtained from gaze datum is based on the world coordinate system rather than based on the coordinate system of my screen, how can I get the result I want? 1. Is norm_pos multiplied by the actual size of the scene captured by the world camera? If so, how can this actual size be obtained? 2. Is it possible to get norm_pos based on the coordinate system of my screen, so that I can directly multiply norm_pos by the actual size of the screen to get the data I want.

Hi @user-64deb6. You can use the Surface Tracker Plugin + AprilTag markers to map gaze from scene camera coordinates to screen-based coordinates. This will get you norm_pos relative to your screen. Read more about that here: https://docs.pupil-labs.com/core/software/pupil-capture/#surface-tracking

Hello, I'm trying to use Pupil Core for an Intel Macbook (Monterey 12.3). However, while the device is detected on Pupil Capture, I only see a grey screen. What should I do to get it working?

@user-b0efbb, you'll need to run Pupil Capture with administrative privileges on MacOS Monterey. Please see the release notes for detailed instructions: https://github.com/pupil-labs/pupil/releases/tag/v3.5

No problem 🙂. Yes, you'll need to calibrate. Calibration is a necessary step to obtain gaze data. Surface tracking is an additional step to map that gaze to surface-based coordinates.

Not sure if this is the right channel to ask in, but I'm working on a research project with underwater eye tracking. We have a lens over the camera as part of the water proof enclosure. Is there anyway to get a read out on what effect this has for the IR? Whether it's filtering wave lengths or reducing the intensity?

In the gaze_positions.csv export, the website says that the norm_pos_x and y positions are normalized coordinates. What are these normalized coordinates? Like what range of coordinates are there across the whole image?

Hi All. I can start pupil capture (mac monterey) using sudo and the world camera shows - but for some reason the eye camera is gray. The problem is apparently the following : eye1 - [ERROR] video_capture.uvc_backend: Could not connect to device! No images will be supplied. Wonder if anyone could suggest. Glasses work fine on a windows PC

Hi all, recently my pupil core stopped working, when I launch pupil capture it says video_capture.uvc_backend: could not connect to device! No images will be supplied.

I followed these instructions https://docs.pupil-labs.com/core/software/pupil-capture/#windows to no avail

I found the drivers, uninstalled them, rebooted, ran pupil capture with admin privileges but now the drivers don't even show up in device manager anymore

could it be my device is broken?

Hi @user-b7f1bf. We have responded in our email thread 🙂

Normalized space is ~~coordinates are~~ bound between 0 and 1, e.g. image centre: x=0.5, y=0.5 . You can read more about Pupil Core's coordinate systems here: https://docs.pupil-labs.com/core/terminology/#coordinate-system

I made a slight edit to my previous response. The normalized scene camera frame is bound between 0 and 1. However, gaze can be estimated outside of the scene camera's field of view. So normalized gaze coordinates exceeding those bounds are not unexpected.

Hi all! I am using one Pupil core with a Macbook. I have also used it as a dual eye-tracking set up with another Macbook, and Pupil Groups and Time Sync. Now I intend to buy a new computer for the second tracker. What do you recommend? Should I go for a Macbook to secure that the two get connected easily, or would Windows work fine? Does Windows editions have any benefits over Mac? (I am particularly interested about recording sound, which is not any more avaliable for iOs versions)

Thanks, Neil, great, good to know with which system is works stable. . With the audio, I intend to follow the recommendations that you have put here: https://discordapp.com/channels/285728493612957698/285728493612957698/887783672114208808 Do you still recommend exactly this? Do you expect it will work on macOS Monterey? (with my current machine I have not been successful: AudioCapture wants me to use later version that I have, but when I use later version, than PupilLabs doesn't work any more... 🙂 )

No, I am afraid I cannot, since I downgraded back. Also, I am using old version of Pupil Core (1.12.17), because I have further processing adapted for it.

I see. I post process the data from two eye-trackers to overlay gaze paths on the world camera video with an external utility, which might not work with the updates. (but I get that I better solve it 🙂 )

Yet the question about sound: what would be your advice: go for Mac, put Monterey, and try the steps from previous messages?

Oops. Thanks a lot for getting seriously in my situation. What if I use LSL plugin on the Clock Master? Would then LSL adjust the Master's time, and Pupil Time Synch adjust the time on the second machine?

Do I understand it right that the outcome I can (1) approximately synchronise with exported video e.g. by ffmpeg, (2) use exact timestamps to draw audiogram on the frames of this video?

Last question: is there any version of old Pupil Capture + iOs system where old synchronisation with sound works? (actually, this is not only my problem, I know also other researchers who really keep their old versions to keep sound recording working)

Hello, could someone point me to the original screen marker choreography code/plugin? Or any default screen marker or calibration choreography code. I found the custom nine-point calibration plugin but was hoping I could get the base code? I am trying to implement displaying a visual target in my dual_display_choreography_ client (not too sure how to do this since its not a plugin that I'm subclassing and rather a client that enables a plugin). Thank you!