Hello, I am working on streaming gaze parameters for a project. Right now, i have the eyetracker connected to the companion device and stream the values from there to my mac wirelessly. Is there a way to bypass the companion device and directly stream to the mac?

Hi @user-b8a876 , the Companion Device is necessary, as NeonNet runs in the Neon Companion app on the phone. However, are you doing a gaze contingent paradigm or are you only streaming gaze values to collect data for later analysis?

thanks for letting me know @rob. this is a gaze contingent paradigm

I see. And why do you want to bypass the phone? Do you want low latency or are you experiencing network issues?

For lower latency

Ok, in that case, you can connect to Neon via Ethernet cable. Using a local, dedicated router (does not need WiFi or Internet) to connect both the PC and Neon via Ethernet cable is the easiest method.

I see, thank you!

Also, if you have the Samsung S25 Companion Device and a PC with a WiFi 7 card, then you can connect them via a WiFi 7 router and get much lower latency than older WiFi generations, although an Ethernet connection will be the lowest latency.

Hey hey folks, working on a project exploring AOI tracking using the Neon - We've been exploring a number of functionalities in Alpha Labs but were wondering -

Is there something in AlphaLabs or otherwise which would allow dynamic AOI allocation (Say, to a webpage), which would then track if a user was to move the webpage or that window between their screens? We have managed to get detection of screens working as AOIs but not to the granularity of tracking pages.

Hi @user-7e50cc 👋 ! Do you mean something like https://docs.pupil-labs.com/alpha-lab/web-aois/ ?

Hello! So I've been playing around with the dynamic-rim screen-mapping(https://docs.pupil-labs.com/alpha-lab/map-your-gaze-to-a-2d-screen/#map-and-visualise-gaze-onto-a-display-content-using-the-reference-image-mapper) feature, and needed some advice.

I have been experimenting with my (dynamic) scene on the monitor, just like the example video. The thing about this feature was that it required enrichment result from Pupil Cloud. When the scene was stable, I could get a correct enrichment result and run the screen mapping. However, when I tested out with a dynamic scene (that is, the scene on the monitor was changing rapidly), the enrichment practically failed, without any mapping at all.

Is there a strategy using the glasses/screen mapping that I could use in order to make this work? For instance, when running enrichment on the cloud, should I use a screenshot of my monitor? Or use a camera picture of my monitor as a reference image input?

Hi @user-94700a , are you using a scanning recording where there is dynamic content on the screen?

Hi @user-d407c1 ,

Thanks for your reply. This is very helpful. Our system is sensitive to tiny magnetic field changes (order of fT)

We are using the Neon Companion App. When we open the app and stay on the home screen, we do not observe the 76 Hz peak. However, when we start a recording or enable the World Camera stream or IR stream, the 76 Hz peak becomes clearly visible.

I was wondering whether this is because the IMU becomes active only when one of these streams is enabled or a recording starts, or if some other electronic component is activated during streaming/recording.

Regarding the Quest 3, are the world cameras physically absent, or are they physically present but disabled by default?

Also, if I would like to purchase a customized eye tracker, should I contact the team through the Integrators website?

Finally, do you know the sampling frequency of the IMU in Neon?

Thanks again for your help!

@user-55220f Thanks for following up. The IMU samples at 110 Hz and is always running. Even if you don't explicitly stream it, the magnetometer is still physically present. That said, if you're not observing interference from it, then it's probably not something to worry about.

Given your observations, I would recommend selectively enabling one stream at a time (e.g. world camera, eye cameras, etc.) to confirm that the scene camera is indeed the component responsible for the 76 Hz peak.

Regarding the Quest 3, the scene camera is still physically present on the module, but when the Quest 3 frame is detected it is disabled by default and the system ignores the scene camera sensor, does not starts it.

If you're interested in a customized solution, feel free to reach out to [email removed] We can discuss your application and see what level of customization might be possible.

Hi pupil labs team . Is there an updated version of the neon companion apk installation package? thanks.

Hi @user-bd5142 ! Updates of the Neon Companion App are released through the Play Store. The latest version can be found https://discord.com/channels/285728493612957698/733230031228370956/1522489198077022259 and the link below should take you to the Play Store to update it.

thanks, Is there a neon companion apk available? Due to network issues, I am unable to download it from Play Store

Hi @user-bd5142 , we have passed this request onto the team.

Hi! I am aligning my LSL Neon data with the Neon data from the cloud, and I saw that there was a offset in the x position between them. Do you guys know why this is? I did not perform any offset correction in the cloud

Hi @user-fea89c , when you say an offset in x position, do you mean a temporal offset?

No, I mean an offset in the position, for example, the Neon data is at 800px, while the LSL is at 780px.

Hello! I have conducted an experiment involving different nurses and patients communicating with the help of AI on an iPad and some written documents. I have include AprilTag markers onto the iPad and written docs but Pupil Cloud failed to detect them. I wonder if it's possible to create enrichments for each recording, instead of using one reference image with hand-drawn AOIs for all recordings?

Hi @user-52c504 , can you share an example with [email removed] It is possible that the AprilTags were too small or perhaps their brightness/contrast was not optimal.

With respect to your question about an Enrichment per recording, yes, that is possible. Simply make unique Events per recording and then choose those in the Temporal Selection on the left when creating the Enrichtment. Also, depending on your experiment design, be sure to use those Events to also make appropriate Enrichment Sections to greatly improve processing speed.

Our lab has one pupil lab account and at the moment we are hosting data from 2 different experiments which were carried out by different researcher groups. For the sake of compliance we don't want one group of researchers to be able to access the data from other experiment. Is there a way to provide a role based access so that each group can only access recordings of their respective experiments?

Hi @user-fc2c9f , yes, you want to make use of separate Workspaces and Access Controls.

It sounds though like you already uploaded all data to a single Workspace?

yes we uploaded it to the same workspace

I see. It is not currently possible to split a subset of recordings out into another Workspace, nor is it possible to do per recording access control; it only operates at the Workspace level. Going forward, be sure to make use of separate Workspaces.

As an interim solution, you can download the data for the groups and host them on a secure shared service of your choice, under separate accounts/folders.

Ok thanks for the input, we will implement this in our future experiments

Hello, I'm using the Reference Image Mapper as well as the heatmap and I feel like the python script is only considering the number of fixations (their x and y coordinates actually) but not their durations ? Why is that ?

Hi @user-664cca , may I ask which Python script you are referring to?

Hi. I haven't used pupil cloud for a while and now you made it that I cant access the files anymore without paying. Is there an easy way to download everything to my PC?

Hi @user-e74b04 , you still have access to your data. With a free account, you have access to the oldest 2 hours of recordings. You can download them in chunks and then once they are safely backed up, you can delete them to free up the quota.

Please note that deleting them from Pupil Cloud permanently deletes them and it is not possible to re-upload to Pupil Cloud, so make sure you do indeed have a backup before deleting.

So I have to go through ~100 files download, delete, wait until your server realises it and repeat the same spiel, instead of just 1 button that says "download everything, then delete"

Sorry for the late reply (things suddenly turned pretty busy and completely forgot that I posted this question)!

When you say scanning recording, you mean using Reference Image Mapper, right? I had used one of the screen capture of the monitor display as the reference image, but it looks like it was not enough because mapping failed. Do you think I should use the photoshoot of monitor display and surroundings instead?

Hi @user-94700a , you need to choose a Scanning Recording, in addition to providing a Reference Image. I would recommend first reviewing the tutorial video here.

Yes, I used scanning recording too! But it still required me to upload the reference image, and I wished to learn what was the best option that I could use for that. (that is, better to use the screen capture or the photoshoot of the monitor)

It is best to use the following:

• A Scanning Recording that is slow and smooth and which covers the full area of the monitor and desk from multiple different vantage points and orientations. A 1.5 minute Scanning Recording would probably be more than enough in your case.
• A Reference Image that is from the typical point of view of an observer during your experiment. You can use a cellphone to make the image or take a screen capture from the Neon recording.

Our experience is a Scanning Recording and a Reference Image with static content on the screen works best. You could even turn off the screen when making the Scanning Recording and Reference Image.

If you are open to temporarily inviting us to your Workspace as Collaborators [email removed] then we can take a closer look and provide better feedback. Just note that then we can see all data in your Workspace.

Oh I didn't think about turning off the monitor screen! I can also make the scene static for a while before participant starts the expriment to enhance the mapping. I will try the strategies you suggested first, and if it doesn't work out I will let you know. Thank you!

@user-94700a You do not need to make a separate Scanning Recording and Reference Image for every participant. You just do it one time for the environment/desk, so long as the environment/desk remains the same for all participants.

Hello, We are trying to analyse how fixation durations and AOI hits differ across the events that we labelled in our recordings using Pupil Cloud. The issue is that when we export the AOI data from the Enrichments menu, the exported CSV does not indicate which labelled event each AOI hit belongs to. I've attached an example showing the events we created within a recording. However, the export only allows us to obtain data for the entire recording (recording.start to recording.end) or for a single selected event, rather than providing AOI hits for all of our labelled events in one export. Is there a way to export a CSV that includes each AOI hit (or fixation) together with the corresponding labelled event or Time of Interest (TOI) that it occurred within? This would allow us to compare fixation durations and AOI metrics across all of our events without having to export each one individually.

Hi @user-737ef5 👋 ! Are you using the AOI Fixations CSV export? If so, one approach is to correlate the recording ID and fixation ID from that export with the fixations.csv time series from the recording.

That fixations.csv file contains the fixation start and end timestamps [ns], which you can then compare against the timestamps in the events.csv file. This lets you determine which event (or interval between events) each fixation belongs to by checking whether its timestamps fall within the corresponding event boundaries.

Alternatively, if loading multiple files sounds complex, and your goal is simply to cluster fixations to aois across event boundaries, you can define Enrichment Sections.

In this case, you can specify that enrichments should always run from crossing.start to crossing.end across all recordings. That avoids manually trimming each recording, and the aoi_fixations would be only in that section. Still, it generates one enrichment per section rather than adding the event label directly into the exported CSV.

Hi pupil labs team ,I have two questions about the Neon eye tracker . One is whether the Neon USB protocol is 2.0 or 3.0. What is the bandwidth required for raw data transmission and what is the format? Also, does the Neon eye tracker require a separate video receiver driver for video acquisition? Or is this part of the camera driver free? Thanks

Hi @user-bd5142 , is it alright if I first inquire about the end goal? Then, we can better assist.

To briefly answer:

• USB 2.0
• You may want to look into pl-neon-usb and our other Open Source libraries

Thanks

Another question is, is Neon's Type-C (USB) only responsible for capturing scene videos and eye tracking videos?

It transfers the data from all the sensors in Neon:

• Scene camera
• Both Eye cameras
• 9-DoF IMU
• Microphone

Thank you

Sure, I was referring to this one : fixation_histogram2d = hist2d(fixation_x, fixation_y, bins=[nbins_x, nbins_y]) fixation_heatmap = GaussianBlur(fixation_histogram2d, 0.01 * scale) fixation_heatmap /= max(fixation_heatmap) final_heatmap = resize(fixation_heatmap, (ref_img_width, ref_img_height), interpolation=LANCZOS) from this page : https://docs.pupil-labs.com/neon/pupil-cloud/visualizations/heatmap/

Hi @user-664cca , thanks for the clarification. Just a note, that is not a full Python script, but rather a conceptual overview of the process.

To answer your question, we are following standard practice, where fixation heatmaps show the relative number of fixations that land in a given portion of the Reference Image. This is similar to the "Total Fixations" option in the AOI Heatmap Visualizations.

Do you need a classic-style heatmap that instead visualizes total fixation duration?

Thank you for your reply. Indeed, I'd like to give more importance to the fixations duration.

You are welcome. In that case, you could use AOI Heatmaps, but if you need a traditional heatmap showing such a metric, then a custom script would be required. You could use the pl-neon-recording library and reference the examples. If you would like to see such a heatmap in Pupil Cloud, then feel free to add it as a 💡 feature-requests .

Hi! My name is Lauren Slone and I'm new to this discord community. I just purchased 2 Neons for psychology research with toddlers and adults. I have a couple of questions for getting set up with LSL. I tried searching "LSL" in the existing chats, but didn't find what I was looking for (but apologies if this info is indeed already somewhere in this channel).

Background: I can successfully sync the 2 neon trackers via the realtime API from Pupil Labs. I’d like to also sync them with 2 additional 3rd-person view cameras. I’m planning to do this using LSL and I would appreciate any recommendations/advice before I purchase the external cameras, as I am new to using an LSL setup.

Plan: My current plan is to purchase 2 additional android phones to use as cameras, download an LSL app to them, and then connect them to the same local network as the Neon Companion apps. My understanding is that when I then install and run “lsl_relay“ (https://pupil-labs-lsl-relay.readthedocs.io/en/stable/) via my Mac computer Terminal, I’ll be able to discover and send commands to all devices (i.e., both Neons plus the two additional android camera phones).

Here are my questions:

1. Does what I propose above sound right? Do others have alternate setups or alternate camera options that are working well that they would recommend?

2. Pupil labs docs mentioned that I should consider purchasing a dedicated router. I'll primarily be collecting data at a small college that has a wifi network that I can use now without a dedicated router. Are there good reasons to get the router? Is anyone syncing multiple devices successfully in a similar setting without a dedicated router?

Many thanks in advance!

Hi @user-376984 , welcome!

First, to clarify, the lsl_relay is an older tool for our previous eyetracker, Pupil Invisible. With Neon, the LSL integration is built directly into the Neon Companion app and you will want to reference its Documentation here.

To use LSL to discover and record from devices, it is typical to use the standard LabRecorder software provided by the LSL team. This is installed on a central recording PC.

With respect to supported devices, the LSL team maintains a page for that. Their community would be the best to reach out to regarding third-party cameras, as we can only provide explicit support for our Neon<->LSL integration.

Also, LSL is not really used for sending commands to devices. It is primarily used for collecting and automatically time syncing data from disparate sources.

Regarding the router, yes, it is still a good idea to get a dedicated router. Even small universities will have many users on the WiFi network at the same time, potentially saturating it and their activity will interfere with Neon's signal. Also, university networks often have restrictions that can block proper connection with Neon and/or LSL. You can also experience packet drops over such networks. For these reasons, a dedicated router is the recommend option.

When it comes to syncing the cameras of Android devices over LSL, I think there was a community developed LSL app for Android, but I do not know much beyond that. This would also be something good to check with the LSL community.

Hi! I have a question about the offset correction tool in the Neon Player. What units is it in? I'm writing a paper where we use the Neon and I want to be transparent about us using the offset correction tool, but it would be more informative for the reader if we could say what units they are in. Forgive me if there is an obvious answer to this, but I couldn't find it in the documentation.

Hi @user-ac970e , nothing to apologize about. It is in units of pixels in the scene camera image. It is a linear offset.

Thank you!!

Good evening. I am having a hard time with Neon Player (v6.0.13). I can't load a record folder, both in Linux and Windows machines. Video folder exported from Neon companion Android app and copied to a local directory. Am I missing something? Do I have to open an issue in Github?

neon_player_linux.txt is a linux console record for the player app. neon_player.log is a player's log from windows

PS. I am not allowed to post on the #troubleshooting channel.

Hi @user-99cc44 👋 ! are you running it from source?

Hi Pupil Labs,

I am currently working with Pupil Labs Neon via LSL together with a motion tracking system. The goal is to combine head movement from motion tracking with eye movement from Neon in order to estimate the final gaze direction. Our current application is a sound localization setup. The speakers are arranged on a ring, currently in steps of 5 degrees. Each speaker has a visual marker in the center, and participants are instructed to look directly at this marker during the measurement. At the moment, we are evaluating the measurement accuracy. The motion tracking data seems to be correct. With our current processing pipeline, we obtain around 2.5° MAE over several recordings. I assume there might still be room for improvement. I tried two different approaches for the Neon eye-tracking data: Using the LSL gaze point x/y from the scene camera image. Using the Optical Axis values of the left and right eye separately.

My expectation was that the gaze point x/y should give better results, because it should represent the actual estimated gaze in the scene camera image, while the Optical Axis is an eye-state signal?. However, in most of our recordings, the MAE is slightly higher when using the x/y gaze point compared to the Optical Axis approach.

My questions are:

For this type of application, what is the recommended Neon signal to use for estimating gaze direction? If using LSL gaze x/y, is the correct approach to convert the 2D gaze point into a 3D gaze ray using the scene camera intrinsics from scene_camera.json? Could there be a reason why Optical Axis gives slightly better results in our setup, even though we expected x/y gaze to be more appropriate? Are there any recommended steps for combining Neon gaze with external motion tracking data, especially regarding the transformation between the scene camera coordinate system and the motion tracking rigid body?

Any advice would be greatly appreciated.

Best regards

Hi @user-18f9b3 , may I first ask which Motion Capture system you are using?

Hi @rob , we are using the qualisys motion capture System with 4 cameras. For the tracking we use the "every move you make frame"

Hi @user-18f9b3 , thanks. Please check our Motion Capture Alpha Lab guide. It contains scripts for spatial and temporal synchronization with Qualisys and your frame. It makes use of Qualisys' and Neon's LSL integrations, and uses that to synchronize the raw, full fidelity data. If anything is unclear after that, just let us know.

Hello! Let me know if this is the correct channel for this question. We're using Neon systems with deaf signing toddlers and their parents. We want to do some automatic detection of where/when the parents hands are available on-screen (to help us figure out if/when the toddler can perceive their parent's sign langauge input). From the description on the PupilLabs website, it seems like Densepose will draw a bounding box around the person, and then identify what part of the person the gaze point is on.... To other folks who have played around with densepose, do you have a sense of whether we could use it to automatically draw bounding boxes around each individual hand? (parent left/right and child left/right)

Hi @user-409176 , my colleague, @user-d407c1 , is the best reference to ask, but in the meantime, have you seen our Dynamic AOI Tracking With Neon and SAM2 Segmentation and Map Gaze Onto Anything guides? They offer more fine-grained and up-to-date methods. You could, for instance, integrate the newer Sapiens 2 model into the Map Gaze Onto Anything framework, which is more robust than Densepose.

This looks helpful - thank you! One of the dynamic aoi examples is even with a hand!

Hello, I wished to ask a quick question regarding Neon Player using Neon glasses. I've been trying to gather the experiments in a single pipeline. I used realtime API feature (https://docs.pupil-labs.com/neon/real-time-api/) so that I can control the recording and events.

After getting the raw data from the Neon glasses, however, I got a little confused on where I should be going from here. Using Pupil Cloud, while it provided rich enrichments, made my pipeline rather diverged, since I had to upload my files to the cloud. I discovered there was a Neon Player that enabled running the raw data locally. But that led me to several questions, especially on:
(1) Do Neon Player (possibly using plug ins?) support all the features provided in Pupil Cloud? (2) Are there any downsides that I should be aware of, when I use Neon Player instead of using the cloud?

Any suggestions or advice or would be greatly appreciated!

Hi @user-94700a 👋 !

Neon Player and Pupil Cloud don’t currently have one-to-one feature parity.

Pupil Cloud is better suited for workflows that involve data aggregation across multiple recordings and some of the more computationally demanding enrichments (like Auto Image Mapper or Reference Image Mapper). It is also great if you want to use it on a low end computer, or if you like simplicity.

Neon Player, on the other hand, runs locally, is fully open source, and can be extended through plugins, so it gives you more flexibility if you want to build a custom workflow.

If you prefer something more programmatic, you can also use pl-neon-recording, which is the underlying library used by Neon Player, to access and process recordings directly in Python.

The main trade-off is therefore convenience and built-in analysis in Cloud versus local control and extensibility in Neon Player.

Could you share a bit more about what you want to analyze or automate? Knowing which outputs or enrichments you need will help us point you toward the most suitable workflow.

Thank you for the reply, Miguel!

So to give you a gist of my current issue, the reason I tried to integrate it into a single pipeline locally was because I needed to conduct an experiment that involved gathering multiple data in one place in a synchronized manner (e.g., Neon's gaze recording, user interaction data, the type of tasks the user was performing, refernce mapping, various events, etc.)

Initially I was using a Pupil Cloud to export the gaze data from the recording, but I realized tracking different data got quickly tedious because of all the uploading/downloading procedures. Since I was planning to do the research with many participants, I thought I needed to avoid using Pupil Cloud.

I saw that there was a real time API and Neon Player available, so I thought I could run enrichment locally on my desktop, but I was not sure how much Pupil Cloud features were also provided in Neon Player too.

Since you mentioned that they were not one-to-one, say if I wish to run reference image mapping or auto image mapping, those features should be implemented by myself. Is my understanding correct here? 🤔

Hi Rob. So I have downloaded all files of all my data. Now I want to use it. I can easily open all gaze etc raw files but there are no eye states. So I assume they have to be computed first. Why is this not done automatically? Anyway. I tried opening the files with the Neon Player and export the eye states.

"AttributeError: No eyeball data found

The above exception was the direct cause of the following exception:"

I know I had pupil dilation in the pupil cloud. What now?

Hi @user-e74b04 , did you have Compute eyestate disabled in the Neon Companion app at the time of recording?

Checking... Yes. I didn't need it before because before I had complete cloud access.

Ok, pupil diameter is part of eyestate. The Native Recording Data that you are loading into Neon Player comes directly from the phone or from Pupil Cloud?

I still have all the mp4s

both. I had to export some from the phone because it was getting full so they weren't uploaded.

But they have the same structure

That's what I have

Hi @user-94700a ! Thanks for the additional context, that helps clarify what you're aiming for.

May I ask what makes Pupil Cloud cumbersome in your workflow? Uploads from the Companion Device are automatic, and downloads can be automated through the Cloud API. Cloud also lets you batch process recordings in enrichments, which is particularly useful for larger studies.

By contrast, if you move to a fully local workflow, you'll still need to transfer the recordings from the Companion Device to your computer (typically over USB) before processing them.

Regarding your question: your understanding is correct. Neon Player is designed to inspect and process recordings locally (post-hoc), but it does not currently support enrichments such as Reference Image Mapper or Auto Image Mapper. Those enrichments are computationally intensive and currently only available in Pupil Cloud. Depending on the hardware, they may not even be practical to run locally, let alone in realtime.

If your experiment requires realtime reference mapping, you may want to have a look at Surface Mapper, which can run locally in realtime.

So it really depends on which enrichments you need. If your workflow relies heavily on Auto Image Mapper or Reference Image Mapper, Cloud is currently the only option.

Oh, got it! In fact, after some thoughts I was starting to leaning toward using the cloud! The main reason I thought local workflow was necessary was because I was expecting that there will be a lot of segments of events of each recording (as well as all the different data that needed to be synced up).

Instead of editing each event in the cloud, I used realtime API to record the events timestamp, and it came to my mind that I wished to keep my workflow locally to leave it as simple as possible.

But if the Neon player doesn't support the same enrichment features I think it makes more sense my workflow partially dependent on the cloud. (besides, it looked like it was a significant work to implement the algorithm myself 😅)

Thank you for the help!! 😄

Dear team, we record paradigms displayed on a monitor in a dark room. We use april tags to map coordinates. We suffer from overexposed videos, often april tags are not recognized. Is there any way to tune exposure behaviour of the scene camera?

We're using neon with One+ phones on stock rom and current app version.

Hi @user-4b18ca , have you already tried the different Exposure Settings, including manual exposure?

Hey folks, have a question about the pupilCloud Reference Image Mapper enrichment - Is the 3 minute limitation for recording length only for the free version? We are exploring it using research usecases of hour long videos, and splitting it into 3 minute sections can be done (With some effort) but worry it could come with some loss of information when rejoining. Thankyou for the help with our last question too.

We are mainly trying to use the reference image mapper for its 3D world mapping onto 2D surface reference point output, as we have other scripts for AOI detection.

Hey @user-7e50cc 👋🏻 The 3-minute limitation applies only to the scanning recording, which must not exceed three minutes. The recordings you want to analyze with the Reference Image Mapper can be of any duration, even on the free tier.

As a general recommendation, I suggest using Events to mark the portions of the recording where the object or area shown in the reference image is actually visible in the scene video. This allows you to run the enrichment only on the relevant segments of the recording, which not only speeds up processing but also avoids analyzing periods where the reference image is not present.

Hi everyone,

I'm having an issue with my eye-tracking data, and I'm not sure where it's coming from. I'm processing everything through Neon Player.

In the attached image, the red circles are the fixation positions (in pixels; their size represents fixation duration), while the blue lines are the raw gaze samples/scanpath (also in pixels).

What confuses me is that the fixation positions don't seem to match the gaze data. Visually, many fixations appear to be shifted relative to where the gaze samples were actually recorded. I would expect each fixation to be centered around the corresponding cluster of gaze samples, but instead they often appear offset.

Has anyone experienced something similar? Any suggestions on what I should check would be greatly appreciated! Thank you 🙂

Hi! I had a question about the IMU sensor. Are the IMU intrinsic calibration and noise parameters (e.g., gyroscope/accelerometer noise density, bias, or similar) available in any documentation or configuration file? Or is there a specific calibration procedure that needs to be performed to estimate these parameters (in order to put the correct values in a Visual-Inertial Odometry pipeline)? Thanks a lot!

Hi @user-a4aa71 ! Neon is equipped with the ICM-20948 9DOF inertial measurement unit (IMU) by InvenSense, the calibration routine is meant to lock the magnetometer to the north.

Hi- which file do I find the camera intrinsics in within the recording folder?

Hi @user-f89b02 👋! You can find the camera intrinsics in the calibration.bin file. The pl-neon-recording library demonstrates how to read and parse it here.

If you are already using pl-neon-recording you can:

import pupil_labs.neon_recording as nr
rec = nr.open("/path/to/recording")
print(rec.calibration)

Hi @user-d407c1 , if I lost my companion phone (On OnePlus 10 Pro: Android 12 and 13) for Neon tracker, can I use my own phone with the same model (not bought via Pupil Labs) with the Neon Companion apps?

Hi @user-29f76a 👋 ! Here you can find the list of compatible devices with Neon.

Thank you

Hi! My name is Oliver and I am doing undergraduate research that is using pupil labs SAM (segment anything) extension. I was however finding that the areas I specified to mask were actually in reverse? and only one frame of the video actually had that reverse mask effect the rest were empty. I will provide clarification/screen shots if needed! Any help would be appreciated :)

Hi @user-a06211 , if you could provide any logs or error codes from the Google Colab or Python console that would be helpful. You can also share a screenshot if you wish, if you think it would not reveal any private information.

Hi everyone, our neon glasses are not responding when we connect them to the phone nor the companion app on the phone show any response. Event the white LED on the glasses are not turning on while it was turning on immediately. We have visually inspected the usb wire there is no sings of being stretch/tear/disconnection, we also move the wire and wiggled it to detect any possible internal tear/disconnection. Have you faced with the same issue? Could it be the Neon module that is broken? (We bought this recently and have used it for only 10 recordings so far)

Hi @user-38ac69 , could you open a Support Ticket about this in 🛟 troubleshooting ? A member of the team will reach out to you in the morning.

There weren't any error codes that I could see but when I placed by positive clicks the program would mask everything but those areas? As well as when I downloaded that video only the first few seconds were masked at all. I am not sure if I am meant to do it frame by frame? If so when I attempted it wouldn't let me add different positive clicks on a different frame

Thank you for the support!

Dear @rob , Thanks for the response. The open ticket function is not working and it produces this message: "The application didn't respond in time"

Thanks, @user-38ac69 , then please send an email to info@pupil-labs.com for now.

Hi! Is the exact image sensor model used in the Neon scene camera publicly available? Also, besides exposure, are any other scene camera parameters (e.g., gain/ISO or white balance) configurable or automatically controlled?

Hi @user-c6dc34 👋 ! The camera sensor model is not publicly disclosed. Aside from exposure, the camera controls are not currently exposed through the Companion App.

If there are additional camera settings like gain or white balance as you mentioned, that you would like to configure, feel free to submit a feature request in 💡 feature-requests so the team can evaluate it.

Quick question: for the Neon fixation detection algorithm, I am just making the section in our manuscript to concisely summarize the fixation detection process outlined in the PDF provided by Pupil Labs, however I just want to make sure I am correct in saying that the position of the fixation is the median of the gaze coordinates during its respective interval? Or is that not the case?

Hi @user-ac970e , it is the mean of the gaze coordinates for the respective interval.

No. I 've downloaded the binaries for linux and windows.

If you can open a ticket at 🛟 troubleshooting we can definitely follow up there. It would be tremendously helpful if you can also share the recording to data@pupil-labs.com and share any additional logs you see.

Hi! While checking the gaze sampling frequency (on average) of the recordings from one of my experiments, I noticed that it is quite variable. However, when considering the median, it is consistently around 200 Hz. I also noticed that in one of the recordings where the average sampling frequency was much lower than 200 Hz, the Neon dropped a frame (the camera briefly turned grey). In another recording from a different day, the video (and the data) froze for several seconds, and as a result the average sampling frequency was only 77 Hz 😩 . Has this ever happened before? Do you have any idea what it could be due to? Thanks a lot!

Hi! While checking the gaze sampling

Hi! I am looking into the reaction of the eyes to played sounds in a office like environment. I am using psychopy for my experiment and to play sounds over different speakers. For each sound that is played psychopy sends directly a marker to the Neon device, following your suggestions (https://docs.pupil-labs.com/neon/data-collection/psychopy/). I would like to do a timing test to figure out what the delay is between when the marker arrives at the eye-tracker and when the sound is actually played. I used the microphone data of the Neon mobile phone to record the audio. I have a script for detecting the onsets in Matlab. However, if I compare the audio microphone data with the event data, I see a drift in the signal. This drift shows that the difference between the microphone onsets of the sounds and the events onsets become larger over time, with the microphone onsets being behind the onsets of the events. I tried to use the audio data loaded by the find clap example (https://pupil-labs.github.io/pl-neon-recording/latest/), but that did not change anything. Do you guys have any suggestion on how to fix this?

Hi @user-fea89c , using the audio signal is not a reliable way to do time sync. Rather, you want to make use of the Events that you sent. Since you were using the PsychoPy plugin, it internally uses our Time Offset Estimator to correct for transmission delay, so you don't need to measure that separately yourself.

Hi! I am not using it to sync my time. I am just looking for the delay in my experiment delay between when the marker arrives in the Neon device and when the sound is played on the speakers. This also includes a delay between sending out the audio from psychopy and when the audio is actually played on these speakers. Is the audio of the mp4 not synced with the neon clock?

Hi @user-fea89c , there will be no registered delay between when the marker arrives at Neon and when it was sent from the PC, because of the automatically applied clock offset, which implicitly mitigates and corrects for delays. Let me know if the diagram in this message clears that up: https://discord.com/channels/285728493612957698/1438132613775622244

And, yes, as with all datastreams from Neon, they are all timestamped from the same high-precison clock, but the audio signal is not the most reliable way to judge relative timing. Rather, offset-corrected Events are our recommended way, and since the PsychoPy plugin automatically uses those, then you do not need to be concerned about delays with the Events.

Hi! We are using different speakers with a different distance to the computer on which we run the psychopy script. This means that the delay depends on which speaker the sound is coming from and it may not always be 2-3ms. I don't think that psychopy can answer this question, as it is not just influenced by the code but also the equipment we use. Therefore, we would like to check the delays. Is it posibble to get the neon clock timestamps for the audio?

However, whether PsychoPy starts the audio signal at approximately the same time as the Event is sent is a question that the PsychoPy team would need to answer, but the difference there should be quite small, perhaps even on the order of less than 2 or 3 ms (just a rough assumption based on experience with other systems).

I mean that the delay between the PC sending the Event and the PC starting to play the audio signal is probably less than 2-3 ms. The audio waveform will of course take a different amount of time to travel to the participant if it originates at different distances.

If you are working with the Native Recording Data, the timestamps provided by pl-neon-recording are Neon's timestamps for the audio. Otherwise, they are simply the timestamps already embedded in the MP4.

Rather than use Neon's microphones to determine the time each audio waveform requires to travel to the wearer, could you use a professional audio microphone and do some measurements in the lab under the same conditions?

We also do a timing test for another experiment where we don't use the eye-tracker. Here we use LSL to combine the markers with an external microphone. However, we can't assume that the delays are the same as we use another plugins to send the markers (lsl vs the neon plugin). Therefore, we decided to do an extra timing test for the neon device with the neon microphone.

Hey, I am using your neon glasses and having some issues. I am getting Sensor failure and it says The neon Module has stopped providing Eye sensor data I tried to unplug it and plug it back in but it did not work. the error mentioned it may indicate a Neon Module or frame issue. Could you please help Thanks a lot

Hi @user-a94388 , could you open a Support Ticket about this in 🛟 troubleshooting ? Thanks.

Sorry, I mean can you run your PsychoPy experiment with something like contact microphones on the body of the speakers and a professional mic where the wearer sits. Then, you should be able to compute the exact delay with high precision & accuracy.

Hi Rob, That's true, but than the clock of this microphone needs to be aligned with the markers send by psychopy. Otherwise we can't calculate any delays. Therefore we use LSL in the other experiment, so that the clock of the microphone aligns with the event markers. However, as the psychopy plugin for sending markers to lsl is not exactly the same as the plugin of the neon, we can't assume that the delays are the same.

Hi team, I want to raise a technical problem involving video-audio synchronization in the scene video of long recordings (>20min). From one of our experiments where we played movie clips, we noticed that audio lags behind the video and the lag linearly increased with the duration of the recording. We then performed a simple experiment where we simply played beeps along with a flash on the screen, and it showed a similar pattern. We wonder if this is something you are aware of, and if there's any solution to this?

Hi @user-a55486 , please see my discussion with Eva20202 above: https://discord.com/channels/285728493612957698/1047111711230009405/1531947113456468049

Hi Rob, I'm referring to a different kind of problem and has nothing to do with cross-device synchronization. It's just how audio is recorded along with video and embedded in the scene video

Hi is there a simple guide to integrate gpt for event annotation . Video would be helpful

Hi @user-d3bdec , have you seen our Automate Event Annotations with Pupil Cloud and GPT and Audio-Based Event Detection With Pupil Cloud and OpenAI's Whisper guides?