Hi, I’m seeing a possible Neon IMU timestamp bug with pupil-labs-realtime-api 1.9.0.

Raw IMU test:

• RTSP/RTCP timestamp vs Companion: median +3.5 ms
• ImuPacket.tsNs vs RTSP: fixed offset -822.960 s

Example: RTSP = 1788278307.721242 tsNs = 1788277484.760913

Earlier I also observed a different fixed offset of about -860 s, so it changes after restart/reconnection.

The Python API builds IMUData.timestamp_unix_seconds from imu_packet.tsNs / 1e9, while the RTSP timestamp looks correct.

Is this a known issue? For real-time synchronization, should I use the IMU RTSP/RTCP timestamp instead of IMUData.timestamp_unix_seconds?

Hi @user-4a5aa2 , could you open a Support Ticket in 🛟 troubleshooting about this? A member of the team will reach out in the morning.

Hi, I was reading over the white paper for the pupil labs blink detector (https://assets.pupil-labs.com/pdf/Pupil_Labs_Blink_Detector.pdf) and am not quite understanding point 6.1 "Blink onset events that are not followed by a blink offset event within 30 ms are removed. Likewise, blink offset events that are not preceded by a blink onset event within 30 ms are also removed."

This seems incompatible with point 6.2 "The remaining pairs of temporally linked blink onset and blink offset events are combined into single blink events. Blink events with a duration of less than 100 ms are subsequently removed."

I'm unsure if 6.1 was meant to say 300 ms or if I'm misunderstanding something

Hi @user-ad7ce5!

If I understand you correctly, you are asking how an onset and offset can occur within 30 ms when the resulting blink must last at least 100 ms, and whether 30 ms should therefore be 300 ms.

The 30 ms value is correct. It refers to the maximum gap between the end of the eyelid-closing event and the start of the eyelid-opening event. The 100 ms threshold refers to the complete blink, measured from the start of closing to the end of reopening.

I hope that clears up the distinction!

Hello Pupil Labs team,

We are using Neon for XR experiments with Unity and are currently evaluating LSL as a common synchronization layer for Neon, headset data and other sensors. We have a few questions about the current Neon Companion LSL implementation:

1. Current Neon LSL implementation

We found the open-source pupil-labs/lsl-relay, originally developed for Pupil Invisible and later extended with official support for Neon (https://github.com/pupil-labs/pupil-invisible-lsl-relay),(https://pupil-invisible-lsl-relay.readthedocs.io/en/stable/history.html). We also found in your documentation that LSL support for Neon has changed and is now built directly into the Neon Companion app (https://docs.pupil-labs.com/neon/data-collection/lab-streaming-layer/).

Is the current built-in implementation derived from this relay, or is it a different implementation? Is the source code of the current LSL implementation or of the Android Neon Companion app available?

2. Data exposed through LSL

The current Neon Gaze LSL documentation exposes gaze coordinates and part of the eye-state data (https://docs.pupil-labs.com/neon/data-collection/lab-streaming-layer/), while the Real-Time API exposes additional data such as worn, monocular gaze, optical axes and eyelid data (https://pupil-labs.github.io/pl-realtime-api/dev/methods/async/streaming/gaze/).

Are there plans to expose the full set of real-time Neon data through LSL?

3. Neon Gaze timestamp

What exactly does the timestamp attached to a Neon Gaze LSL sample represent?

Your recording-format documentation states that a Neon gaze timestamp is equal to the timestamp of the eye-video frame from which the gaze sample was generated (https://docs.pupil-labs.com/neon/data-collection/data-format/). Is this timestamp converted into the Companion's lsl_local_clock() domain and used for the LSL sample, or is the LSL timestamp assigned at another stage?

Any implementation details or roadmap information would be very useful for us to understand what we can rely on today and how to scope our integration accordingly.

Thank you!

Hi @user-8db985 👋

1. Current Neon LSL implementation

LSL functionality is built directly into the Neon Companion app using the standard Android LSL implementation. The former external lsl-relay is not involved in this workflow and can safely be ignored. The source code for the Companion app integration is not publicly available.

1. Data exposed through LSL

We currently have no plans to add the remaining Real-Time API fields to the LSL stream. To synchronise these additional Neon streams, you can send basic Events during the recording and record the Neon Event Data Outlet with LabRecorder. You can then use the matching events in the Neon and LSL timelines to place Neon’s full recorded data on the LSL time axis.

1. Neon Gaze timestamp

Your understanding is correct. The timestamp associated with the source eye-camera frame is converted into the Companion device’s lsl_local_clock() domain and used as the LSL sample timestamp.

Thank you for the detailed answer, this is helpful !