Hello. I'm building a multimodal rig with Neon + EEG + gloves + webcams, everything going into LSL on one PC. Trying to get all streams aligned to within a few ms.

We're pulling the scene camera over RTSP and using Time Echo for the offset. I think we've got a sign inversion somewhere, we're seeing a consistent ~332 ms error against our other streams.

I wanted to ask:

1. Can you confirm the exact formula? Is it device_time + offset = client_time, or the other way round?

2. Is the timestamp on an RTSP scene frame the sensor capture time, or when it was encoded/sent? And is it the same timebase as the phone's native recording?

3. For send_event, is the event stamped on the device or on the client? Can I pass my own timestamp in the device timebase?

4. Does the offset drift over a session? Should we re-run Time Echo periodically rather than once at the start?

We're on wired Ethernet to the phone already. Let me know if you need any more information, our main aim is to sync up each of these modalities as much as we can based on the frame rate.

Hi @user-a67b22 👋 ! That sounds great. If you're using LSL, the easiest option is to enable Stream to LSL in the Companion App:

https://docs.pupil-labs.com/neon/data-collection/lab-streaming-layer/

This gives you the data directly in your LabRecorder session, including gaze data and event timestamps.

If you're using RTSP to pull the scene camera stream, each frame already contains its capture timestamp in Unix epoch nanoseconds. You can use the clock offset to convert between the Companion Device clock and your PC host clock.

As, for your specific questions:

1. Can you confirm the exact formula? Is it device_time + offset = client_time, or the other way around?

Yes. To convert a timestamp from the Companion Device to the PC timebase, you take the RTSP timestamp and add the offset:

device_time + offset = client_time

For the opposite direction, when converting from PC time to Neon time, you subtract the offset.

See the manual clock offset correction example here

1. Is the timestamp on an RTSP scene frame the sensor capture time, or when it was encoded/sent? And is it the same timebase as the phone's native recording?

It's the capture time, using the Companion Device's clock in UNIX epoch nanoseconds, same as recorded one.

1. For send_event, is the event stamped on the device or on the client? Can I pass my own timestamp in the device timebase?

You can choose either approach, let the system timestamp the event on arrival, or provide your own timestamp.

See: https://pupil-labs.github.io/pl-realtime-api/dev/methods/simple/remote-control/#with-manual-clock-offset-correction

1. Does the offset drift over a session? Should we re-run Time Echo periodically rather than once at the start?

Yes, the offset can drift, so you should estimate it regularly for the best timing accuracy.

If you stay within LSL, though, LSL already handles this clock synchronization for you.

Thank you Miguel. Will give these a shot in my current code base : )