Hello, I am looking for some advice on aligning in time and just wondering if my ideas are feasible. I see Pupil Core is compatable with a "lab streaming layer" to align in time... I am not too familiar with this. My experiment will involve torque, EEG, and EMG data where there are all aligned using a DAQ sending a second of +5v blip to signal start of recording. This data aquisition is initialized in MATLAB. Ideally... I can use MATLAB to integrate the start to the data and simultaneously start the pupilcore recording, or at least signify the start of the trial in the pupil core data. As of now, I am only interested in pupil diameter. What are your thoughts and suggestions? I notices pupil has matlab integration, and I also noticed the LSL files were all python.

Hi, I have a very important question. I am using the Pupil Labs Core headset but the front camera is broken. To replace it, as a temporary solution, I have put a logitech C615 camera attached to the front camera (the position between the camera and the eye cameras is now different but the difference is small). To calculate the intrinsic parameters I run the plugin "camera intrinsic estimation" with radial distortion and I see the parallel lines. To have a good gaze accuracy I do the calibration with 5 markers. My question is, I don't have to indicate in any file the new position between the cameras because I understand that with the calibration the problem of having the cameras in different positions is solved?

Another question to see if I am understanding how calibration works. I perform the calibration with 5 markers at a certain distance using a display. If I don't move, the calibration is good but if I move and get closer to the screen or move away the gaze position is no longer correct. If I want to use the glasses while moving, how do I get a good gaze position?

Hello. i used the divice normally on friday and now camera for eye 0 is not working, i've not changed anything in settings, now it says "local usb: camera disconnected!"

Thank you @user-d407c1 !! I understand. How can I consider the accommodation/convergence ratio in Pupil Labs app?

One way would be to make the calibration at the distance at which you would perform the task. Besides the screen markers, you can also use the physical single marker https://docs.pupil-labs.com/core/software/pupil-capture/#single-marker-calibration-choreography and move it around, or you could use the natural features. https://docs.pupil-labs.com/core/software/pupil-capture/#natural-features-calibration-choreography

Also, if you plan to record at diff distances, you may want to present the marker at diff distances and use the 3D gaze mapper. https://docs.pupil-labs.com/core/software/pupil-capture/#gaze-mapping-and-accuracy

Thanks for all @user-d407c1 ! Have a good day 😃

Hello! Maybe I'm asking a very rudimentary question, but I can't seem to complete the screen marker calibration. I'm running Pupil Capture on Windows 10. When I select the Screen Marker and start the calibration I get a flashing screen with a marker at the centre. Is there a way to solve this?

Hello! I had a problem that I had not been able to figure out. I wanted to call the foreground camera of the Pupil Core in python to get the image information for further processing, but I had not been able to find a way to solve this problem. How can I successfully call Pupil Core’s foreground camera in python? 🙂

Hi, @user-fafdae - you can receive video frames from the Pupil Core system using the network API. Here's a sample script: https://github.com/pupil-labs/pupil-helpers/blob/master/python/recv_world_video_frames_with_visualization.py

Thanks for your answer! I will try it later, hope it'll work! have a nice day

@user-cdcab0

😃 👍

Hello i am new in the pupil lab community. I am trying for a university project to make an android application that takes image from the pupil core glasses.. does anyone know how can I connect the glasses with kotlin or java code with my android app in android studio?

Hello, I am currently in the process of labeling all my surfaces to my pupil core recording in pupil player. Is there a limit to how many surfaces you can label in pupil player? I have run into a problem where adding 130+ surface markers (doing a social interaction experiment with many markers) disables my recording from being open in pupil player after I finish add the surfaces and want to re-open the recording. Sometimes pupil player will just open the eye1 video (to download the fixations), but nothing else before shutting down. Other times, pupil player will just not open if the pupil core recording includes the file with my 130+ surfaces. If I just do it with a small amount of surface trackers (4 surfaces), pupil player opens my recording just fine. I used the april tags for my surfaces.

Hello, I am working on research that requires eye tracking, If I purchase the Pupil Core, is there a way I can make it mobile. I mean connecting to a phone rather than a computer to collect my data?

Secondly, if I purchase a Neon (from my understanding it connect to a mobile device), would the software be free?

Thanks

Hello! I tried using my own camera. Previously, I had a 640400 30fps camera, and after changing the driver to libusbK, it could perform pupil recognition. Now I've switched to a 640400 240fps camera. After changing the driver, I found that it cannot run properly. It gives the following error message. What could be the issue?

Hi Team, I have two questions concerning Pupil Player using Pupil Core: 1. If I use 2 (more than one) map gazers with 2 individual calibrations mapped against one section of the data for validation, the accuracy result will show as “nan degrees from 0/33 samples”, so how can I get the exact degrees? 2. Once I can get the exact validation result, how do I apply that to tweak my old fixation points/coordinates? And how big the accuracy/precision result will make it unreliable? 3. After the gaze mapping task, for some reason I could only have the fixation points and their number shown for the section I have gaze-mapped. How can I have the all the fixations again and with post-hoc correction? Thank you very much in advance!

Hi ! Could you tell me where is the gaze confidence computed in the pupil software ? I am having some difficulties understanding the exact relation between the pupil confidences and the gaze confidence

Hi! I'm playing around with Pupil Core and have carried out some tests in orden to graphic the gaze results. Would you say these plots make sense? I'm a bit worried because of the wakes and the proximity between ayes and wondered if this is just normal or I might have been doing something wrong... Thanks!

I hope to receive your assistance. Thank you very much!🙌

Hello ! When plotting the confidence for 3d model (high light intensity condition), there seams that there is a threshold 0.7. Why is this ?

Hi @user-52bb4d 👋. Thanks for sharing the plot. Would you be able to share a bit more information about the tests you have performed and the data you have plotted? Otherwise it's difficult to say whether the data are as expected.

No, for example, for my left eye, i have a decent long term model (blue) and 2d detection (yellow) but my red ellipse that keep sliding

Here is an example :

I'll do, thanks you

Hi Neil, I am not sure I am understanding you question as well? As I mentioned in my previous post, I median filtered and then interpolated the x_norm and y_norm surface gaze values. Now, we are trying to re-indentify fixation using the I-DT alogorithm as well as detect saccades. When trying re-identify fixations, we need to calculate dispersion using teh gaze point values (visual angle). But, the visual angle is only outputted in the gaze_positions.csv file. In the surface data, we have both gaze and world (multiple) timestamps. So, I am just wondering how to best reconcile cases where you only have visual angle for two gaze timestamps, but not the rest (for a specific world index and/or world timestamp). I hope that makes more sense?

PsychoPy, Core and Saccades

Anyone familiar with using ZMQ to realtime print pupillometry data? I have coppeid the same scrips in the documnetation from the IBC back bone... But I am getting the error : ZMQError: Address already in use ....Is this the address for my Port?

HI Dom, Yes. I am getting my info from this documentation: https://docs.pupil-labs.com/developer/core/network-api/

I am using two screens in my experiment. One will be my computer, where I hope to control the recording of the pupil lab device. The other will be the visual feedback for the participant shown on a 32 inch flatscreen TV approximetely 5 feet infront of them. Is it possible to brodcast the calibration onto a screen that is not connected to the default computer screen responsible for controlling the device?

I have conducted several experiments and the confidence is always less than 0.6. What should I do?

pupil data is here. @user-cdcab0 thank you

Hello. Is there a way to produce a notification when an event begins (i.e fixation begun) and not only when it ends? As I see it now, the fixation detector produces a fixation message only when it ends.

I haven't looked at this directly, but it makes sense when you consider how fixation detection works. The algorithm really can't know that a fixation has started until at least some amount of fixation has already occurred

It's feasible that one could modify the code to emit a fixation notification earlier than the end of the fixation, but it will never be able to be emitted at the moment the fixation started. It will always be late and retroactively timestamped

hello 🙂 I have a question concerning the eye camera window. What does the blue on the side of the window represent? I guess it is the intensity histogramm of the image and the cyan blue line represents the intensity treshold you can choose in the settings ("Pupil intensity range"). what are the red and white lines? Also what do the yellow regions around the ROI mean?

Hi, we just bought a new core device, but when i plug it in my laptop and start the pupil capture, it can't get the input from the cameras, even though it did keep showing "found the camera". Is it the problems of settings or the device itself ?

I found the resolution from the website, it turns out that you have to start adiministrator privileges to get access to the camera feed first, thanks 😃

Hi, I want to ask about the camera device. I have 2 C922 webcams, when I run Pupil Capture, the software cannot open 2 webcams simultaneously. Is it because the camera name is the same? and if that's the case, how do you change the camera name to Pupil Cam3 ID0 or Pupil Cam2 ID0 and so on?

Hi! I found out that after gaze mapping two gazer mappers with two individual calibrations, and turning on the "Activate gaze" button for both gaze mappers, there is one fixation point shown (with the order of the numer) but not overlapped with either of the gaze mapped points. My questions is how do I know which gaze mapper is the fixation point processed from? Thank you very much in advance!

Hey, @user-6072e0 - I don't have two identical webcams, so I can't explore this myself unfortunately, but it does look like the names would need to be unique. However, I don't think there's a way to make two identical devices have different names.

Looking at the Windows driver installation procedure for our headset (https://github.com/pupil-labs/pupil/blob/c309547099099bdff9fac829dc97b546e270d1b6/pupil_src/shared_modules/video_capture/uvc_backend.py#L161), the three cameras each have a unique product ID, and so they can receive unique names. If your cameras are identical, they will likely have identical product IDs

Hi, @user-daaa64 - I tinkered with this a bit, and it looks like the fixation detector uses the average of the active gaze mappers. So if you have two, the fixation point shows halfway between them.

Do you mean to have multiple gaze mappers active at the same time though? Typically you'd want to adjust the start/stop time of each gaze mapper for just the sections they apply to. To do that, set the trim marks by dragging the ends of the green rectangle on the timeline at the bottom of the screen so that the rectangle encloses the section of time a gaze mapper should be used, then, with that gaze mapper selected, press the "Set From Trim Marks" button to apply.

Hello. I have some world.mp4 files from which I want to extract frames automatically. I'm using opencv and passing the frame number as it shows in pupil player. However, I get different frames. The first image is a capture from pupil player (frame 4434). The second one, frame 4434 from the video. My question is: what is the correspondence between pupil player frames and the actual video frames? I imagined it would be a 1 to 1 correspondence, but maybe not?

thank you, @nmt . I am exporting frames with ffmpeg now and I still have the same issue. However, openCV and ffmpeg do extract different video frames when provided with the same frame index. My question is: is there any encoding that occurs when one exports the videos from pupil player? Please note what I am using the world.mp4 file that is in the recording folder, and not an exported version.

indeed, using the exported world.mp4 works

Hello, I have several recordings that range from 12 to 15 minutes in length, and I tried to export data, specifically fixations on surfaces, blinks, and iMotions data, using Pupil Player v3.3.0. While it successfully works for some recordings, it throws an exception for others ( check out the attached log where it happens). Any clue why this is happening and how I can fix it?

Hi, @user-8f48c2 👋🏽 - doing a quick comparison of the code from 3.3 with the most recent release (3.5), it looks like a bug that was already fixed. The code now includes an explicit check for for that case

Hello all, what is the minimum time required for calibration with the Single Marker to achieve the best possible results? Thank you in adavance!

Hi @user-dfd547 👋 Achieving the best possible results with a Single Marker calibration is not solely a matter of time; it's more about capturing a sufficient number of gaze angles and the extent of the field of view covered. To learn more about this calibration choreography, I encourage you to explore our documentation, available at this link: https://docs.pupil-labs.com/core/software/pupil-capture/#single-marker-calibration-choreography

hello ，I have a question，how to transfer gaze position to fixation data

Hi @user-735418 ! In Core there are already two fixation detectors included. An online detector for Capture and an offline detector for Player, which you would have to enable on the Plugin's manager (https://docs.pupil-labs.com/core/software/pupil-capture/#plugins) Here you can find more information about how these fixation detectors work https://docs.pupil-labs.com/core/terminology/#fixations

Ah, I see. If you want to compare fixations in that case, you'd have to do them one at a time

Is there a streamlined method to accurately quantify the fixation count and duration time on an AOI (Area of Interest) from an exported file of eye-tracking data? I'm seeking an efficient approach to avoid manual counting of participants' eye fixations. Any insights or tools that can expedite this process would be greatly appreciated!

two AOIs

Thanks for your reply！ I want to obtain the fixation count and duration time of each AOI. Is there a simple way？

Thank you very much, I understand what you mean. I attempted to define a surface for each AOI and then generated two files，“fixations_on_surface_Surface 1” and “fixations_on_surface_Surface 2”. I filtered the data for "on_surf=TRUE", see the screenshot.

Now, I have a new question. Do you mean that each row represents a fixation point? Because I noticed that there are multiple rows of data for the same "fixation_id".

@user-d407c1Hi,If I want to see how pupil core calibration is calculated, which specific website should I look at? For example: Code for specifically calculating the calibration visual degree.

It's a valid strategy, but potentially a lot of work if you have a lot of subjects/recordings to go through

Hi, I'm doing a reading experiment, and one of the matrices I'm calculating is visual span. Has anyone tried to calculate it using pupil labs software? I have been trying to create an API Plugin (https://docs.pupil-labs.com/developer/core/plugin-api/) that does OCR on World Camera to find out the location of each letter also uses fixation to calculate it.
But maybe because I used light OCR module (tesserocr) so the detection results were not good. Can we get a visual span from the data published by the pupil software? maybe like fixation, blinking, etc. Thank you in advance.

I am trying to get rid of any delay associated with computing pupil data in the pupil capture software. I assume it takes time to compute pupil diameter from video recorded by "eye". Is there a way to record JUST the raw data (which would allow me to align in time with other device equipment) and then LATER analyze the pupil data to find pupil diameter?

Hello, I bought the DIY pupil lab eye tracker and am in the process of setting it up. I tried to install the pupil software (Linux version) on Raspbian but it would not install. Could you guide me on how to get it installed on raspbian?

]

Hi @user-6072e0 👋 ! OCR in the scene camera can be challenging and highly dependable on the lightning conditions, but perhaps you can use the surface mapper with Apriltags to define a page and define sub AOIs inside. You can still use OCR to detect the characters location on the image and use the fixations on surface csv for example.

Here you have a tutorial on how to aggregate data over an image surface https://github.com/pupil-labs/pupil-tutorials/blob/master/02_load_exported_surfaces_and_visualize_aggregate_heatmap.ipynb, I would also recommend that you have a look at https://github.com/pupil-labs/pupil-tutorials on how to work with the data post-hoc.

So, in a nutshell: 1) place AprilTags to your reading document and record it with the Surface Tracker https://docs.pupil-labs.com/core/software/pupil-player/#surface-tracker 2) localise the characters coordinates on the image using whatever approach you find more suitable, eg. OCR 3) use the fixations and gaze on surface files to determine when the subject looks at the page and using the on surface coordinates define what character/word is the subject looking at.

Let us know if you got stuck at some point

Hi I have a few questions about Pupil Core Question1: Is the 3D Gaze point coordinate in the exported file in the world camera coordinate system? Question2: Is the eye center 3D coordinate in the pupil camera coordinate system? Question3: How to obtain camera parameters. Question4: How can the files in the red circle in the following figure be opened? Thanks

Hi @user-4bc389 !

1) gaze_point_3d_x - x position of the 3d gaze point (the point the subject looks at) in the world camera coordinate system as defined in our documentation (https://docs.pupil-labs.com/core/software/pupil-player/#gaze-positions-csv). You can read more about the coordinates systems we use here https://docs.pupil-labs.com/core/terminology/#coordinate-system 2) In world camera coordinates, please refer to https://docs.pupil-labs.com/core/software/pupil-player/#gaze-positions-csv 3) Which camera parameters would you like to obtain? 4) Please refer to this previous message on how to read .intrinsincs files https://discord.com/channels/285728493612957698/446977689690177536/913512054667747389 or https://github.com/pupil-labs/pupil/blob/c309547099099bdff9fac829dc97b546e270d1b6/pupil_src/shared_modules/camera_models.py#L455

thanks

Hi After replacing the World camera, are the Gaze coordinates also under the new World camera coordinates? thanks

Thank you

That means the coordinate system won't change, it's still under the world camera, right?

Can I please also ask that why the validation seems not working sometimes? There are fixations detected in other gaze mappers (with the same calibration but longer time span) that include the same time section in this trimmed period. It can also happen that the validation shows result after I set the trim marks and sometimes it just detected way less samples or no smaples at all. Please see attached. Do you know how can I fix it? Thanks in adavnce!

Hello! reaching out for hardware help. I would Ideally like to measure the diameter of two pupils. My core's right infrared camera only occasionally works. I can rely on left infrared camera working, and working well. Any suggestions for troubleshoot? Or repair processes? Additionally, when I try to open the right eye camera (eye 0) it says "Local USB Camera Disconnected!"

Usually that indicates a bad/poor calibration. I'm not sure why two gaze mappers with the same calibration would have different accuracy results. Do the gaze mapper time spans include the calibration routine?

I suppose the different accuracy results could be potentially related to slippage since they are mapped against towards different stages in an experiment? And yes, the gaze mappers in these two screenshots include screen marker and single marker calibrations. All my gaze mappers contain screen marker and/or single marker processes and I used these mappers to map against the formal experiment procedures. Sometimes I will check if the gaze mapping is working by observing if the fixation points are "on point" enough in the natural feature calibration which is sometimes part of my "Map gaze in" span. Also, I found out that the issue of "no values" or "no samples found" (shown in the screenshots) often occurs when I set a particular section to be gaze-mapped in "Map gaze in" or be validated in "Validate in". If the span for being mapped gazed or validated happens to be the full video, the accuracy results can normally be generated. 🧐

Hi! I am conducting an experiment to observe eye movements after rotation. My computer is Mac, and the glass is Neon. I have a few questions regarding Neon. Q1: Is it necessary to connect Neon to a computer and use Pupil Capture to collect eye movement data such as pupil position data? If Neon is only connected to a mobile phone (Pupil Companion), can we only obtain a video of eye movement? Is that correct? Q2: How should I use the raw data exported from Pupil Cloud? I am unable to import them into Pupil Player as it indicates that I need to update the version of Pupil Player, but I cannot find the installation package for the latest version.

Good morning, we are going to buy a computer for our experiments. Do you consider anything else necessary in addition to the following specifications? Apple Silicon (M1 or M2), 16 GB RAM, Intel i7 CPU. Thank you so much!

Hi does anyone have any idea how to calculate fixations from gazes (i.e formula) any guides

Hi @user-eb6164 ! Please see my previous message below about the fixation detector algorithms employed in Core https://discord.com/channels/285728493612957698/285728493612957698/1153961497702191104

The algorithm code is open source and you can find it here:

https://github.com/pupil-labs/pupil/blob/master/pupil_src/shared_modules/fixation_detector.py

Ohh! I see, well it is an implementation of a dispersion-based method. Perhaps this would help you:

Dario D. Salvucci and Joseph H. Goldberg. 2000. Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the 2000 symposium on Eye tracking research & applications (ETRA '00). Association for Computing Machinery, New York, NY, USA, 71–78. https://doi.org/10.1145/355017.355028 https://dl.acm.org/doi/pdf/10.1145/355017.355028

Hello, I am trying to integrate labstreaming layer with pupil core. I placed the necessary folders from github in my pupil capture plugins. But, My plugin manager is not giving me LSL as an option. I am using a mac. How do I get pupil capture to recognize?

Hi, can I just check what the units in the gaze detection refers to? for example we have this data

(-0.7101088235564816, -108.72276477823718, 442.24575165054046) (148.14312675270114, 34.23232986872651, -13.173346733526909)

which corresponds to the gaze_point_3d and the eye_center_3d, what is the origin point and what units are the readings measured in?