Hi team, I want to use this: pl-neon-recording Python library, to export my native recording data to surface-gaze fixationn data. I cannot follow the guide from your website: https://docs.pupil-labs.com/neon/neon-player/surface-tracker/#workflow. Can I use the Surface Tracker with the Python library to export all my recordings in bulk? My recordings are exported locally via USB-C Cable from the phone. It looks like I have to export each recording separately from Neon Player?

Hi! I have a question regarding the companion smartphones for the Neon and the Invisible. We have one Invisible and one Neon which came with their companions smartphones. Can we use the Invisible phone also for the Neon and the Neon phone for the Invisible?

Hi @user-9a1aed , we do not have a batch exporting tool for Neon Player's Surface Tracker plugin. If you are using pl-neon-recording, then it would be necessary to do some programming to create such a tool. If using Neon Player, then correct, you export each recording separately.

Hi @user-f43a29 thanks for your answer! If I understand well, you mean I can use the enrichment functionalities on those recordings separately, not combining with the other recordings in the other workspace, right?

Hi, @user-9a1aed - you don't need to extract values form the .time file yourself, the recording library you mentioned can do that for you quite simply

from pupil_labs import neon_recording as nr

rec = nr.open("path/to/my/recording")
for e in rec.events:
    print(e.ts, e.event)

Thanks for sharing the recording - that makes it much easier to troubleshoot. What you need here are larger tags. I'd probably double their size. You can also use more tags to improve tracking.

Hi! I am inserting some PLEvent components into routines in PsychoPy Builder and am receiving the following error messages. " return computer_time + self._time_offset_estimate.time_offset_ms.mean / 1e3 AttributeError: 'NoneType' object has no attribute 'time_offset_ms' " I have defined the onset/offset of the components so I am not sure where the offset might be getting lost. Any thoughts on troubleshooting whould be helpful. Thanks! -Eric

Hi there! We are in the process of ordering the Pupil Labs Neon eye-tracking glasses, and it is my understanding that they ship with a Samsung Galaxy S25 as the companion device. Our goal is to integrate the glasses with a Qualisys motion capture system.

We are considering using a Linksys MR7350 router for this setup. Could you confirm whether this router would be suitable, or if you would recommend alternative models? We’d appreciate your recommendations regarding router compatibility (like the one posted in a previous message: TP-Link TL-WR841N N300 WLAN router) and performance when paired with the Samsung Galaxy S25 companion device.

Hello, I downloaded the Pupil Labs plugin for Psychopy Builder on a PC. I am using psychopy 2025.1.1 and pupil labs neon recording 1.0.3. However, there are no drop-down boxes available to change the IP address and port (see photo). Any idea what is happening here or how to fix it?

Larger tags would be the biggest help, but more tags can be helpful too.

With tags this small, the detector will have a hard time finding them on any given frame. Any given marker may be detected on one frame but undetectable on the very next one. On some frames, some markers will be detected and others will not. If there's only, say, a 10% chance of a small marker being detected, and you only have 4 markers, then you will probably only detect 0 or 1 marker. If you have 100 markers though, you will likely detect ~10 of them.

So yes, more markers will help, but in your case larger ones will help more.

hello, can the gaze offsett correction on the neoplayer have 3 digits precision?

Hi, @user-f4b730 - sure, that's a pretty simple change. Are you able/willing to run from source? I've already pushed the change, but it's maybe too minor to justify a release by itself

ok, I see, how can I run it from source?

Hi, @user-ff60f7 👋. Yes, that's correct. They'll be shipping with an S25. In terms of router, in principle, the one you mention should work just fine. Although we have had a lot of success with the afformentioned TP Link model. So that's what I recommend.

Hi @user-e3fdf5 , the time_offset_ms mentioned in that error message is distinct from the onset/offset that you are specifying in the components. The error that you are receiving indicates that a network connection was not established with Neon. What kind of network are you using? Is it a university network?

Hi Rob, thanks for getting back to me. I was testing without Neon turned on. I will try it again with Neon active. Should I have the connection to the phone on or should I connect the device directly to the computer?

Hello @user-cdcab0 , for sure my bad, but running from source is not intuitive at all to me from the online readme...I get I may need just to change the steps in the gazemapper but I not able to start neon player from source

Hi, @user-6de021 - I have pushed the update. You should uninstall the plugin, reinstall it, and restart PsychoPy

Hi team, may I receive any feedback on my recordings? It seems that the recorded gaze has a systematic shift toward the bottom. May I know how I can improve the tracking accuracy? here is my recording: https://hkustconnect-my.sharepoint.com/:v:/g/personal/yyangib_connect_ust_hk/EQ56jrm2_o5Alc4ZrS_bIfMBLlciUd3T55GjQW81WRfKZw?e=c7IDcS

Hi all! Have two questions i was hoping someone could help with - Is there a way to shorten an eye tracking video so i can use it as a reference video (i forgot to shoot a proper one) - Does anybody have a smart alternative for pupil lab case for the neon? It is a nice case, but would like to have something smaller so i can put it in my handbag

Good morning! I would like to inquire If in the Neon Eye-tracker whether head rotation is taken into consideration when generating the gaze maps.

Hi @user-9382ce 👋. Just so I'm sure I understand, could you clarify what you mean by gaze maps? Are you referring to the enrichments like Reference Image Mapper in Pupil Cloud?

Ah okay! Yes head movements are accounted for. Participants can move freely in their environment and we can achieve a robust mapping. You can read how we do this in my previous message here: https://discord.com/channels/285728493612957698/446977689690177536/1196459926395044021 and you might also want to watch this video if you haven't already.

Hi there, we ran into a problem with the android app gui disappearing into the background after launching it. It takes a couple of seconds after opening the application until this happens. We tried restarting the device, connecting / disconnecting wifi, manually closing the app etc., but nothing worked. We also recorded the process in a video which I can dm you, if necessary. The app is running on a OnePlus 10T and we did not encounter this behavior in the last couple of months. Do you have any suggestions?

Hi @user-e0a71c 👋. Please try the following: - Disconnect Neon from the phone - Long-press on the companion app icon, select app info, and then clear storage and cache. - Reconnect Neon, open the app and enable all permissions. Does that solve the issue?

Clearing storage and cache in this way does not delete recordings. However, upon restarting the app you'll need to log in again and re-enable access permissions - which are prompted automatically and it's very easy to follow.

Just to confirm. Do the connections need to be the same? Will this only work if Companion Device and Computer are connected to Network A both with wifi? Or will it work if Computer is connected to network A with ethernet and Companion Device connected to Network A with wifi?

Hi @user-e3fdf5 , you can simply use a dedicated, local router. It does not need to be connected to the Internet, nor to the University network. You can even use a phone hotspot.

Hello@user-f43a29, Neon's companion software needs to be downloaded through Google Store, but because Google cannot access it in my area, I would like to ask if it is possible to provide the corresponding APK file? Thank you.

Companion app on Google Play Store

Thread

Dear Pupil Labs team,

I am using Pupil Cloud for eye movement analysis and would like to confirm the exact settings of the fixation and saccade detection algorithm. Specifically, I would like to know:

Which velocity and duration thresholds are used to classify a saccade?

Which criteria (e.g., velocity, dispersion, minimum duration) are applied to define a fixation?

Are these parameters fixed in Pupil Cloud, or can they be adjusted by the user?

In some eye-tracking systems, thresholds around 30°/s for fixations and 40°/s for saccades, with minimum fixation durations of 80 ms, are commonly reported. Could you please confirm whether these values are applied in Pupil Cloud, or clarify which exact thresholds your software uses?

Thank you very much for your attention and support.

Best regards, Alejandro

Hi @user-9382ce 👋 ! The fixation and saccade thresholds, as well as the algorithm details, are reported in our fixation white-paper, which you can find here.

These parameters are not tunable in Cloud; we use the values that we found work best on our dataset.

For convenience, I am attaching the white-paper directly to this message.

Kindly note that you can also get real-time fixations on the device, and the algorithm may have been slightly adjusted to accommodate its usage on the device.

Hi @user-cdcab0, I asked this question a while ago and it wasn't a feature back then. Is real-time blink detection when using neon and Psychopy a feature now? Sorry if this was in one of the updates, I haven't been following lately and I just got back to work

Trying to use neon with psychopy builder. I am receiving RPC_DEVICE_RUNTIME_ERROR on psychopy. I am also receiving "ioHub Server Process Completed With Code: 0". I have neon connected via wifi and neon monitor can view the camera on the browser. Psychopy does not seem to recognize or find neon. I tried lots of things, reinstalling the package, setting the companion address to the ip address (neon monitor can be accessed with neon.local:8080). Looking for help with this last piece of the connection.

Hi team, may i check if I use pupil_labs.neon_recording.open to read the local recordings, what are .gaze.ts and .scene.ts and .event.ts? Particularly, scene.ts and event.ts, are they the onset time? It seems that the onset of gaze is earlier than the onset of the scene

Is this the correct way to map the gaze, video frame index, event using the same timestamp>

Hi, two questions regarding Neon. Firstly has the streaming on the World View camera views to an IOS system been resolved and with our nonprescription lens the magnets on one of the lens have for some strange reason come away so we can’t attach it back on so Im wondering if it’s possible to get a spare lens?

Hi @user-e3fdf5 , are you using PsychoPy Builder or PsychoPy Coder?

Thank you! I want to have a table to store the fixation xy data, the frame indices, and also the corresponding events

Hi @user-9a1aed , this is correct.

Just note that the default sampling method is nearest neighbor, so any gaze data that occur before the first scene camera frame occurs will be sampled as if they happened during that frame. This is also why recording.begin is shown as aligned with the first gaze datum and the first scene camera frame in your DataFrame, which is generally not the case.

For all overlapping data, you don't need to think about this.

To get the duration of each fixation with pl-neon-recording, just use the start_ts and end_ts fields of each fixation datum:

first_fixation_duration_ns = recording.fixations[0].end_ts - recording.fixations[0].start_ts

Thanks @user-480f4c for getting back to me and I will photo the magnet and send it over.

Hi guys 👋 has anyone worked on obtaining the visual axis using Neon or knows of any references on how to calculate it? Also, has anyone tried projecting vectors using the optical axis through eye centers to estimate gaze position data — is this method legit?

Hi guys, I have the automatic upload to cloud on the companion device of the neon lab glasses, but I deleted the video from the cloud workspace by mistake, but it’s still on the phone, how can I reupload it to the cloud?

Hi Rob, Yes, exactly — by gaze position data I mean estimating the 3D point in the world that the person is looking at, rather than just the gaze direction vector.

Hi all! I just downloaded Neon Player and am having trouble finding a way to add or edit AOIs. Is the Neon Player capable of setting AOIs, or is that a function only available in the Pupil Cloud?

This means that I cannot have duration for each gaze data point I have, right?

Hi @user-9a1aed , what is your end goal?

Gaze is not typically thought of as having a duration. Gaze is rather a continuous process; i.e., the ever changing position of where you are looking, which is sampled moment-by-moment. In other words, each gaze datum can be thought of as an instantaneous measurement in time.

When your gaze pauses, that is classified as a fixation and the duration of that pause is the fixation duration. As a result, there are always less fixations than gaze points.

If this is unfamiliar, just let us know.

Hi @user-9a1aed , I am not sure I fully follow, but to get the number of fixations and fixation duration, you do not need to join gaze.ts and fixations.ts.

Our fixation detector has already done all the hard work for you.

You only need the following Python code:

# number of fixations in recording
n_fixations = len(recording.fixations)

# duration of each fixation (in nanoseconds)
fixation_durations_ns = [(fix.end_ts - fix.start_ts) for fix in recording.fixations]

Hi @user-ac970e , with Neon Player, you use the Surface Tracker plugin and draw multiple Surfaces to represent multiple AOIs.

If you need shapes other than quadrilaterals, then in a post-hoc analysis step, you can apply a mask to the surface-mapped gaze coordinates.

More AprilTags can always be helpful for robustness, but if you have a configuration that works for you, then great!

Hi! I have a similar question about setting AOIs in the Neon Player - is there any way to set surfaces across multiple recordings? Or do we have to set surfaces for each individual recording (and setting AOIs across multiple recordings is only a feature available in the Pupil Cloud)?

If all available data (full player-compatible package) are downloaded, saved offline, and then deleted from Pupil Cloud, can these data be re-uploaded to Pupil Cloud at a later time (restoring the original capture)? If so, how?

Separate Question: Is there any common wisdom on how to filter fixations based on the confidence value (0,1) indicated in 'fixation detected on surface' column in the 'fixations_on_surface' output file from Neon Player? And, is there any such confidence that is computed for fixations on AOIs built in Pupil Cloud? In output files for Pupil Cloud, I only find a total list of fixations/AOI, whereas in the output files for Neon Player, I find fixations and associated confidences/surface. Maybe my underlying question is more about the difference between the surface and AOI. Thanks!

Hi @user-763912 👋. Yes, you can reuse surface definitions across multiple recordings. After loading the Surface Tracker plugin and defining your surfaces in an initial recording, hit export. This action saves a surface_definitions file to the neon_player subdirectory of that recording. You can then copy this file and place it into the corresponding subdirectory of any new recording. The surfaces will be applied automatically as long as the same markers are present in the scene video.

Hi @user-999fbb!

It is not possible to re-upload recordings to Pupil Cloud after they have been downloaded and deleted. Recordings deleted from Cloud can be restored from the trash for 30 days, but once permanently removed, they cannot be uploaded again from any source.

Regarding your second question: - Surface Fixation "Confidence": The confidence metric from Neon Player is binary (1.0 for "on surface" or 0.0 for "off surface"). This is a filter to indicate if a fixation occurred on the defined surface area. It is not a graded confidence score of the fixation itself. This is equivalent to the 'fixation detected on surface' column in the Marker Mapper export from Pupil Cloud. - Surfaces vs. AOIs: A "surface" is the entire area you define for tracking, such as a complete computer screen or a poster. An AOI is an additional layer you can define in Pupil Cloud to subdivide a surface into smaller, specific regions, such as an icon on the screen or a face within a poster. - AOI Data in Pupil Cloud: The export file for an AOI from Pupil Cloud lists only the fixations that fall within that specific, user-defined region. Because the list is already filtered to only include fixations on the AOI (and metrics are also generated) a separate "on/off" column is not necessary.

Hi Team, when I tried to run the PsychoPy program, it says calibration with the host IP failed. Later on, when I tried it several times, it worked. May I know what is going on? Sometimes, it takes a while to connect the host device to the laptop's program.

Hi Neil, I had the same question about the fixation-detected-on-surface column that comes from a Neon Player export. There are some non-binary values in my column (e.g., 0.21739130434782608, 0.9166666666666666) and, just to be clear - the csvs that output about fixations and gaze on each specific surface are essentially timeseries of all fixations, just with a column for whether it was on the particular surface or not? (And the x and y coordinates in each csv must be relative to the Surface, correct?)

Hello, when I am running the Psychopy program, the host phone does not connect to the program, and the bug says cannot calibrate. After a while, the program can run smoothly, although I did not fix anything. I wonder why this happens sometimes? Thx!

If you could provide your PsychoPy log, that would be very helpful

The psychopy log is saved to a file in your .psychopy3 folder. It's overwritten every time you launch PsychoPy, so you need to make sure you grab it right after it happens

As for one of the recordings, I calibrated to ensure accuracy using the offset function. Also checked the eyeview to be captured during the recording, but it seems that the red circle is a bit off. I want to check if this is normal. Thx

Do you mean the red annulus displayed on the phone's screen (the one you apply the offset to), or do you mean the red circle displayed in your PsychoPy program?

The only correction for this is the offset correction which, it sounds like you've already done. When you do the offset correction though, it's important that you adjust it for the target viewing distance. If you adjust it using a target distance that's different from your task, it will be inaccurate

for offset, we create small rectangle shapes on the computer screen. so the target viewing distance should be the same too

Hi all! My name is Nitya and I am a Master's student working on trying to identify blinks versus saccades in our collected data as automatic blink detection data is not available. Upon plotting the pupil size relative to time, I noticed that there seemed to be frequent dips in pupil size. I predict that this could correlate to blinking but wanted to check if anyone had any insights as to what could be causing these dips or if there was any other way to detect blinks. Thank you, Nitya

Hi @user-f43a29! Thank you for the reply. Nitya is working with me and I have been talking with @user-cdcab0 regarding blink detection. Our data is streamed from Psychopy and due to IRB regulations we cannot use Pupil Cloud or the raw eye recordings. The time series data produced by Psychopy is the only thing we can work with in this particular study. Blink detection from Paychopy has not been implemented yet so we have to make our own algorithm to do that with what we already have.

And this pupil diameter drops are the only meaningful blink indicator we have seen in the data so far. So we are wondering if you could share any insight as to what could we do to better detect blinks from the data we have

Hi @user-cdcab0, We are doing a classroom study and we set up the AprilTags on the corners of the screen. However, they are not being detected. My suspicion is that it will be hard to include these when individuals are further back. Can you provide some guidance on this? (We need to define AOIs for later data analysis)

Margins and contrast are good, but the markers are too small at that distance.

A subjective judgement here is a reliable measure. If it's hard for you to distinguish the individual squares within a marker, then it's hard for the detector as well.

We have done offset correctly and with the same viewing distance. Is there other reasons/ways that we can improve the eye tracking accuracy? Thx!

Thanks a lot Neil. Does it make sense to perform the post-hoc analysis given that we do not know the groud truth of the participants' gaze?

Hello. We have exchanged our pupillabs neon eyetracking module for a new one due to a defect. Now when using pupil cloud the old module is still registered for unlimited space, while the new module cannot be activated for unlimited space. How can we register the new eye-tracker module in pupil cloud with our old license?

old: 263231 -> new: 595697

the new one also indicates that there is 7h of data recorded, but we have no access to this recorded data in another account (I guess the data was recorded by pupil labs, not us)

Hi @user-f43a29 thank you so much for your quick reply. I will send an email now.

Hi team, when we use an extension hub to connect the eye tracker, charger, and mobile phone after some experiment blocks, the following prompt pops up on the Neon app“Sensor failure The Companion App has stopped recording extimu data! Stop recording, unplug your Neon device and plug it back in. If this behavior persists, it may indicate hardware issue. Please reach out to our support team for help."

So we removed the hub, directly connected the eye tracker to the mobile phone, and closed the small window. The recording display continued recording, and we asked the participants to look at the four corners and the center of the screen to ensure that the eye tracking was functioning. When the recording automatically ends after the experiment finishes, we found out that the recording was only 4 minutes and 58 seconds. But if we do not click into the video, it shows as an 8-minute video.

thanks!

Thank you for the reply @user-cdcab0 Could you please tell us where this data is store in the raw recording? And how I can access it given that we cannot use pupil cloud? Thank you!

Hello. Could you explain the distortion correction for Marker Mapper or Surface Tracker? If 2D barcodes are placed in the environment with tilt relative to the Neon module's IMU, how does this affect the definition of the ROI? Please discuss the tilt along each axis: X (pitch), Y (roll), and Z (yaw).

Hello. I am a new PhD student in the field of rehabilitation robotics and am using Neon to detect objects of interest and calculate viewing distance. In your Pupil Labs fixation detector whitepaper, there is a following explanation of using IMU to calculate optic flow.

Where can a find the corresponding code?

Hi again! I am using the Neon Player to define surfaces and AOIs, and I just notice that when I load a recording, it seems to automatically begin finding the surface, and progress is shown in this cache. Great! But, do I need to open up every recording in the Neon Player before the surfaces can be found, and then export each recording? In other words, as I make new recordings with the same experiment (on-screen video with April Tags) do I still need to individually open each recording in Neon Player in order for the surfaces to be defined?

Hi Rob! I think I see what you mean: I see from a previous question that I can copy the surface_definitions folder into all recordings to transfer the same surface positions across the recordings, but since it still needs to detect them, after I copy the surface_definitions folder into all recordings then I should load them each in Neon Player so that it can do the detection?

Hi guys! I’m having an issue with creating enrichments for a study I’m doing. The reference image mapping enrichments keep erroring out on each of the 10 samples I have. I’ve tried several different scan recordings and reference images but keep having the same issue. Is any one able to assist?

Hi @user-08dfba 👋. If you're able, please invite [email removed] to your workspace. That way we can take a look and provide some concrete feedback 🙂

I can see this has been asked before but I couldn't see a reply, so apologies for the double question!

When adding a gaze offset in Pupil Cloud, is there a way to do it more precisely? We're getting participants to look at a target, and it can be difficult to tell if the gaze is consistently offset in a particular direction and needs correcting or if the variation of the gaze from the target is just expected variation. It can also be difficult to tell exactly how much the gaze needs correcting, because at some points in time the variation will be less than others - is there a way to calculate the average gaze variation from a particular point in the scene camera over a period of time and then correct the gaze by that amount?

Also, is there a way to correct it other than dragging and dropping with the mouse? It's more accurate than doing it in the app, but it would be nice to have finer control by being able to enter coordinates or even being able to nudge it left/right/up/down using arrow keys once it's placed to make it more precise.

I saw another user mention that it would be great if we could use a target (like a QR code) that the glasses can automatically recognise and calculate the gaze offset from that point over a specified period of time (when the participant is looking at it), but I assume that isn't possible?

Does anyone have any suggestions for lining up the timestamps in gaze_positions_on_surface.csvs with the frames from the world video?

gaze_positions_on_surface.csv does not start at exactly the same timestamp as the world_timestamps.csv (the first value in one is 1755530803833957766 and in the other, 1755530802581479680, which is about one second difference or less, but still problematic to not have things lined up exactly especially since exports from Neon Player only give gaze positions and not fixations, so fixations have to be manually calculated), so I'm also unsure why there would be a discrepancy there. Also when I converted those values to Universal Time in R, the time it gave me did not match anything in the export_info.csv either.

Knowing the timing of events and fixations in my experiment is really important, but I am struggling to figure out how to know when in the gaze_positions_on_surface.csv an event occurs in the world video. I can imagine this is something someone else probably figured out how to do though?

Hello,

I have two recordings that are each around 30 minutes long, with events already marked. I just realized that for enrichments, the videos cannot be longer than 3 minutes. Is there a way to crop or trim the videos so I can run enrichments only on the specific event segments?

Thank you for your help!

Hi Pupil Lab team! Would someone be able to clarify the units for the gaze in the x, y, z directions as well as the units for and difference between device_time, logged_time, and time? Thank you!

I brought up the topic of IMUs to define coordinate axes with the intent of clarifying how AprilTag appears in the scene camera, but I couldn't explain it well. Is the attached figure understandable? I want to know the effects of three types of tilt, including Pitch (shown in the upper left of the attached figure), Roll, and Yaw.

Does the system detect that the AprilTag is tilted and perform processing such as trapezoidal correction? If all tags are tilted in the same way, the correction seems likely to work well. However, if only one tag is tilted, as shown in the previous figure, I want to clarify what kind of processing would be performed if correction were performed.

Hi @user-08dfba - stepping in for Neil here. I just reviewed your P1 enrichment on Cloud in one of the two workspaces. A few points:

The enrichment was completed successfully, but the mapping quality is not optimal. This is partly because the menu is mostly white with only a few black words, which does not provide enough visual features for the Reference Image Mapper to work robustly. For best results, the tool needs a feature-rich environment.

That said, you can significantly improve the mapping quality by adjusting the Neon scene camera exposure. In many of your videos, the menu looks overexposed. Changing either the room's illumination or the scene camera exposure settings (see here) will help the camera capture the environment more accurately.

If you'd like to continue working with the current enrichment, you can also use the Mapping Correction functionality to adjust fixations that were not mapped optimally.

I hope this helps! 🙂

Let me ask another question while another thread is being processed. The following page states that Yaw convergence takes time, but is there any reference data available? I understand it is significantly influenced by the surrounding environment, but it would be helpful to have data showing the relationship between elapsed time and error rate. https://docs.pupil-labs.com/neon/data-collection/calibrating-the-imu/

Hi @user-52e548 , we specify the IMU's model number and link to the manufacturer in our Documentation. They have a page with technical details and specifications.

May I ask why it would be significantly influenced by the surrounding environment? Are you referring to when highly magnetic equipment is nearby?

Thank you for the explanation. I had completely forgotten that AprilTag is known to be square. Indeed, based on this, AprilTag's orientation can be estimated. Additionally, I also overlooked the instruction that all AprilTags should be affixed to the same plane. With these two facts, I realized my question was nonsense from the start. I apologize for bothering you.

In practice, situations requiring AprilTags to be placed on non-planar surfaces can be avoided. We're conducting various verifications on the system's behavior, and during that process, we were merely testing placing AprilTags on different planes.

Hi Miguel,

I didn't see the fixations plugin in Neon Player - thank you so much for letting me know that exists! that is great to have fixations already calculated.

And for the timestamp thing, this is what I am doing: In Neon Player, I am setting the Frame index range to export to begin exactly at the frame that my experiment begins (experiment is just a video that plays while a toddler watches). In order to mark experiment events (event 1, say, always begins 30 seconds into the video) I need to know the timestamp that the video begins - would this be the first timestamp in world_timestamps.csv? If that's true, then I can set trial event 1 to be exactly 30 seconds after the first timestamp in world_timestamps.csv, and then eventually align the timestamps in world_timestamps.csv with the timestamps in the fixations_on_surface.csvs.

The reason why I am confused is that when I convert the timestamps, it gives me a universal time, but in Neon Player I can see that my export begins at frame 1905 or relative time 01:03.496, which is not at the same scale as the timestamps, so I just want to vet that fixations_on_surface.csv really begins from the start of the export I set (at frame 1905) and not at the very beginning of the actual recording (frames 1-1905 is noise). This is such a new software for me and there are so many csvs haha so I want to find a way to know what timestamp the experiment begins. (Also, I see that surface_events.csv does indeed say the first frame is 1905, but the first value in the world_timestamp column is 64.56998008000001, which is different from all the other timestamp numbers 😭 😂 it must be on yet a different scale?)

This is how I am converting the UTC timestamps into a universal time (in R), I'm first dividing by 1e9 to scale it back to seconds: fixations_surface2$start.timestamp..ns. <- as.POSIXct(fixations_surface2$start.timestamp..ns. / 1e9, origin = "1970-01-01", tz = "UTC")

Hi @user-f43a29! That is correct!

@user-08dfba just to make sure we're talking about the same enrichments: Are you referring to the enrichment named "Test" as the errored one compared to "P1" that worked 9 months ago?

@user-08dfba upon confirming with the Cloud team, this was due to a transient issue affecting yesterday's enrichment - this is expected to be fixed tomorrow. Thanks for your understanding 🙂

Out of curiosity, may I ask why did you run the same enrichment (with the same image/scanning rec) twice?

@user-08dfba thanks for sharing more context. Just to clarify, since the enrichments Test and P1 are identical, and P1 has already been completed successfully, is there anything specific preventing you from using the output of P1?

are there any other enrichments that haven't been completed?

@user-480f4c so that YardHouse workspace was a past project that I was trying to test things on. The current project is that second workspace I shared access to is where we first experienced the issues. So that new test enrichment was just a troubleshooting test

Hi! Is there a way to change gaze data capture rate post collection in pupil cloud / on neon player app? say I recorded gaze data at 200hz and wanted to downsample to 100hz, or vice versa

[email removed] ! Let’s try to break it down

Hello, I am still having some trouble downloading the data from the project in the cloud. I manage to download it eventually, but only after several failed attempts. The speed also continues to fluctuate a lot. I was looking at other options, such as the Pupil Cloud API (https://api.cloud.pupil-labs.com/v2). However, when I downloaded the files, they did not correspond to the ones available in the Timeseries (different files and different columns). Is there any web endpoint (API) that provides access to all the files available through the cloud download? And where is this documented? Thank you

Hi Miguel, that sounds great. Would love if you could put together something to show how it would work, no problem if it's next week.

Would this add the gaze offset correction in real time, so that we wouldn't need to adjust it in the app or in Pupil Cloud?

Hi, I want to do a sanity check. We are developing some cognitive aids for paramedics to use in the field. They will be accessed via a tablet. We want to do some user testing in a simulated real world environment. Looking through the Alpha Labs I have come across the "Map to Dynamic Screen Content", "Gaze on Phones" and "Website AOIs". After reading these my plan is to record the screen on the tablet at the same time as the eye tracking. Create a Reference Image Mapper enrichment. Export data for offline analysis, to generate scanpath etc..? Just wanted to double check there isn't an updated way to track dynamic content on a phone or tablet?

Hi, I'm running a video renderer for a project, but I keep getting an error message even after trying multiple times. I've also emailed the cloud support team. Is it possible to get help resolving this?

Hi @user-1423fd ! That’s not possible to change after the fact, the sampling rate is fixed during recording.

Worth noting though: if Cloud detects that the data is below ~185 Hz, it will automatically reprocess the recording to achieve 200 Hz.

Hi! I'm having difficultly deleting wearer profiles. I am not synced to pupil cloud at all, and the wearer has no recordings associated with it, but when I try and delete the wearer I get "Cannot delete wearer. Wearer is either synced with pupil cloud web interface or has recordings"

@user-f43a29 The wearer isn't on the workspace on pupil cloud. I've basically never uploaded anything to pupil cloud

Thank you for the very helpful reply! I was able to see the blinks with Unix timestamps in the file names blink ps1.time exported from the companion phone. But it looks like this blink ps1.time file only existed for recording made roughly after this April. Is this normal?

Hi, do you have any guide on how to use Neon Player? I’m trying to figure out the zone of interest of a golf player on the golf course.

Hi, I have a question about timestamps again. This time about surf_positions_[Name].csv exported from Neon Player. The only documentation I found is from here: https://github.com/pupil-labs/pupil-tutorials/blob/master/04_visualize_surface_positions_in_world_camera_pixel_space.ipynb According to this world_index - The frame number of the world camera to which this sample is correlated. world_timestamp - The world camera timestamp at which the sample was recorded. However, in my exported file, for world_index 0 the world_timestamp is > 2.9. So this would mean that the first frame has a timestamp of 2.9 seconds, which doesn't seem right to me. So how do those times correlate to the other times exported from Neon Player and to the video? It seems a bit strange that this is the only file that doesn't use a system timestamp, instead the timestamps and the world_index (which doesn't exist in any other file) look like straight out of the old Pupil Player.

Hi we have been asked to be involved in a research project involving individuals that are deaf or significant loss of hearing. It is looking at different methods of communication in television programmes involving sign language and subtitles.

We want to eyetracking them with Neon under controlled conditions and I know you that it’s possible to map gaze onto dynamic screen content however what I woukd also like to know is if it is possible to select AOI on the dynamic screen such as the location of the subtitles. Furthermore is also possible from the video to select different body features such as face, arms and hands to identify them and to map the gaze onto them.

Thanks Gary

hi @user-d407c1 Thanks for getting back. Do you think it would be possible to identify body parts on a video rather than from world view camera if we wanted to identify which part of the body the gaze was on someone who was using sign language on a television program.

Hi!

We are working in synchronizing the Neon with external devices (an external camera with the Neon camera and an external motion sensor with the IMU) and a delay appeared within NEON itself, between the NEON IR CAM, and the NEON IMU.

The question is: should I expect the timestamps from the IMU and IR eye cam to reflect the same clock and alignment with insignificant delays between sensors?

Basically we tested it this way: - Turned on Neon with no motion, on a table - Quickly moved Neon - Find the frame from the IR eye cam that corresponds to motion onset (you can see the background FLOW on the IR cam) - Find the onset of the movement in the IMU. - Compare the timestamps , each coming from their corresponding sensor timestamps on imu.csv and gaze.csv

I did 3 tests and got : - Test 1: IMU shows motion 200 ms BEFORE it can be seen in the IR cam - Test 2: IMU shows motion 333 ms BEFORE it can be seen in the IR cam - Test 3: IMU shows motion 60 ms AFTER it can be seen in the IR cam

Could this be possible? Thanks

Thanks @user-d407c1 most appreciated

Hi! I am using the Neon and integrating it with Psychtoolbox. To map my computer screen gaze coordinates, I render markers which are correctly identified (see pic below). I have run the surface mapping, but it is not allowing me to download the remapped coordinates, even though the process has run successfully. I have successfully done this a few times before, so I am unsure what could be going wrong. Thank you very much

@user-f43a29 The difference between the first value in world_timestamps.csv and info.json/start_time (could not find recording.begin) matches the 2.9 seconds. So I think this is perhaps a miscommunication - world_timestamp is not the "world camera timestamp", but the global timestamp of the recording at which this world camera frame was recorded. Regarding your second answer "you are looking at timestamps relative to recording.begin" makes it sound like I have the choice of looking at system timestamps instead. If this is the case it would be the best solution for me. Is there an option to switch this to system timestamps?

When I say system time, I mean absolute (Unix epoch) as opposed to relative timestamp. The distinction you made between unix epoch, recording relative and world relative is very helpful, so let's stick with this. My one question is: Is it possible to get unix epoch timestamps into surf_positions...csv instead of relative timestamps? The other question is: If not, is the world_timestamp in surf_positions...csv recording relative or world relative? It seems to me now that it is recording relative (I have to subtract start_time), even though the name and the documentation suggest that it is world relative.

Thank you for the feedback, Miguel! I tried that approach, and it doesn't work. When I defined my markers, it was okay, and I could run the mapping.

Hello, We recently purchased a Neon device and have been testing it. The cloud was not syncing (I could only view the recordings on the app, but not on the laptop). I checked the website, which suggested logging out and back in again. However, after doing this, all of my data disappeared. Could you please advise on how I can recover the data? Thank you.

Hi @user-b92f77 👋 Could you double-check that the account you used on the web is the same one logged into on the Companion Device? Also, have a look at the recordings list on the device, do you see a little Cloud icon next to them?

If everything matches and the issue persists, please open a ticket in 🛟 troubleshooting so we can look into it privately. Sharing that info there will help us sort it out faster. \

Thanks. I still think that it is a misnomer, as it is clearly a recording relative timestamp an not a world relative timestamp. I was initially very happy that pupil player exports everything as unix epoch times, which makes it easier for me, so I don't understand why surf_positions...csv is the exception to this rule. To be honest, I think It would be very helpful if there were a document where all the different kinds of times were clearly defined, and how to convert between them. I would claim to be very familiar with these kinds of conversions and synchronizations (we support many devices and synchronizations between them) but in this regard I find Pupil Labs to be quite confusing. So far I have been given the following information: - If the data is exported from pupil cloud, in order to convert from unix epoch time to world relative time, one needs to subtract start_time from info.json - If the data is exported from Neon Player, in order to convert from unix epoch time to world relative time, on needs to subtract the first timestamp in world_timestamps.csv, but not the start_time from info.json. - The world_timestamp in surf_positions...csv is world relative, if the file was exported from Pupil Player - The world_timestamp in surf_position...csv is recording relative, if the file was exported from Neon Player (even though it is still called world_timestamp and this is not documented anywhere as far as I can tell).

I cannot imagine that I'm the only one who finds this confusing, but using the search function I have the impression that I'm the only one inquiring about this, so perhaps I am.

I'm sorry, but this makes it even more confusing.

start_time does not convert to world relative time. It converts to relative to recording start. Yes, this is what I wrote. Except if it is an export from Pupil Cloud (at least this is what I have been told twice in this Discord).

Subtracting the first timestamp in world_timestamps.csv also does not convert to world relative time. It converts to relative to start of the scene/world camera feed. What is the difference between a world relative time and "relative to start of scene/world camera feed"? In your reply above you said "World relative timestamps -> Origin is shifted to first scene camera frame". To me this sounds exactly the same.

The differences between Pupil Player and Neon Player happened naturally in the course of development, as we took everything we learned from Pupil Core and found ways to make it simpler and more streamlined for Neon. So you learned from Pupil Player to make everything Unix epoch relative except for surf_positions, which switched from world relative to recording relative without changing the name and without documenting it? It looks more like an oversight to me, which is totally fine, happens to me all the time. 😅 I just think it's a bit silly to pretend that this is an improvement.

The first time I inquired about start_time for Pupil Cloud was here: https://discord.com/channels/285728493612957698/1047111711230009405/1184517071321694378 I then re-enquired here and it was confirmed that it is indeed different between Cloud and Neon Player: https://discord.com/channels/285728493612957698/1047111711230009405/1364243824620343326

The goal is naturally to make things improved and simpler as time goes on. That one got past us. However, on the whole, yes, I would say things have improved I agree, and as I said, I appreciate that for the other files everything is unix epoch relative now.

To clarify though, we would still probably call them something that indicates world timestamps, as they are timestamps for world camera frames. This does not make sense to me. All timestamps in Pupil Player export (with the exception of surf_positions) are simply called "timestamp [ns]", regardless of whether they are for world camera or eye video frame time. In fact that is another thing I think should be in that document, because it is no longer possible to distinguish between world and gaze time from the column name (as used to be the case), it is only possible to find out by comparing to the timestamps in world_timestamps.csv and gaze_positions.csv.

The first time I inquired about start_

Hello again! I noticed that the video and the gaze data do not always have the same length. In fact, even when comparing the difference between world_timestamps and gaze_timestamps, it’s always non-zero—although usually very small. However, I have a session where, even from the Neon player, you can see that the video starts (with some gray frames at the beginning, but I saw here on Discord that others already mentioned this as well) and only afterwards the gaze data appears. When replotting the frames with the gaze overlaid in MATLAB, there’s a clear delay, and indeed when checking world_timestamps and gaze_timestamps there is a difference of about 3 seconds. I tried to realign the data streams based on this offset, but there still seems to be an additional mismatch, and I have no idea how to determine its exact value.

Have you ever come across this issue? Is there a way to reliably realign the video and gaze data streams? Thanks a lot!!

Hi @user-a4aa71 , this is not an issue and is actually expected. Everything is also already aligned.

The different sensors in Neon run in parallel and have non-deterministic start times. Sometimes gaze can start first, sometimes the scene camera starts first. They also end at different times when the recording is stopped & saved.

If you have used Pupil Cloud, this explains the gray frames that you see at the start & end of recordings. They help to visualize the moment between pressing the white record button and the sensors starting to record data.

Since all of Neon's data are timestamped from the same high-precision clock, you can simply work with the timestamps as-is. For example, if you want the gaze data for the first scene camera frame, then just take the gaze data with timestamps between scene camera frame 1 and scene camera frame 2, without applying any offset.

Hi! I had a question about audiostreams. I currently have an applicaiton that captures gaze using the real time API. I would like to look at audio through this application as well. I have a couple of way I am thinking about doing this, which come down to two questions:

1. Is there a way to extract an audiostream/audio using the real time API?
2. If I use the realtime API to stop and save a recording, is there a way for me to access that new recording using the neon API?

Thanks!

Hi everyone, I need help with re-uploading recordings to Pupil Cloud. I had recordings uploaded to Cloud but accidentally deleted them. The recordings are still present on my phone inside the Neon Companion app, but the app marks them as already uploaded, so I cannot re-upload them again. I tried using Export, but the exported files are only raw data (.raw, .time, .bin, .mp4), and they don’t include the processed CSV files (like gaze.csv).

My question: Is there a way to force the app to re-upload a recording that was deleted from Cloud?

Thanks a lot for any advice 🙏

thank you for your response and for the example. i will take a look at them!

Hello, I am working with the pupil neon for my masters thesis. For my MS thesis, I need to use the 3D left and right eye gaze direction vectors from NEON. Can you anyone help me with this? I am free to use any of the neon API version - python, unity, etc.

Hello team, I want to check if there are any products from the company that can tie the monitoring phone to the participant wearing the eye-tracking glasses? This is because our participant may walk and jump around in a room and interact with others. Ideally, we expect a vest for participants to wear so that it can hold the phone while we collect eye tracking data from the participants. thanks a lot!

Hi @user-b55ba6! 🙂 Just to make sure everything is up to date, could you please do 1-2 more tests after updating to the latest app version, 2.9.16-prod? https://discord.com/channels/285728493612957698/733230031228370956/1422080447264854089

Hi @user-ce065d , could you describe in more detail what you need to achieve?

To clarify, Pupil Cloud does not introduce a misalignment or time mismatch. It uses the same timestamps from the raw data on the phone.

Would be able to share both sets of data with [email removed] Then we can check it.

Hi, thank for your answer, unfortunately I can't recover them from the trash folder, Is It possibile to upload them again from the app?