yall... why does the real-time-api use an older version of protobuf and the pl-rec-export use a newer version? Is there a way to use both on one version of protobuf....

I believe I will be able to resolve this issue using python environments, but when trying to use both the real-time API and pl-rec-export I run into version issues as pl-rec-export requires an updated version of protobuf where as real-time API requires the older 3.18.0 version. Was wondering if this was the intended way of resolving this problem or if there was a better option?

Hi I'm getting started analysis data from a Neon device. I was trying to follow the tutorial provided here https://github.com/pupil-labs/pupil-tutorials but as far as I can tell it doesn't work with Neon output. I haven't managed to create a folder output from the cloud that open in the Neon player (v4.1.2) either. I just get an error saying there is no info file in the directory. Is there a similar tutorial for the Neon for basic plotting? Could you advice on some debugging steps for the player? Thanks.

@user-f43a29 @user-4c21e5

I'm getting this error while trying to stream video using this pupil-labs-uvc.

"Can't open stream control - error: Operation not supported"

Platform: Windows

(It's working on Ubuntu though) Not sure what to do.

Really appreciate any help

Hi @user-5bac8b 👋 ! I assume you refer to https://github.com/pupil-labs/pyuvc, do you have the dependencies installed?

@user-d407c1 yup, dependencies are installed

Hi, I have a question regarding an old version of the pupil labs application of densepose. I have run the version 1.0.0b3 (according to the github repo) on some data and would like to use the same version of the code on the rest of the data. However, 'pip install pupil-labs-dense-pose' is not working, it seems that the library is not available on Pypi anymore. I have tried to execute the code by downloading the files from the version 1.0.0b3 from github but I meet this error 'File "/home/aarthuis/densepose/densepose-module-1.0.0b3/src/pupil_labs/dense_pose/main.py", line 12, in <module> from pupil_labs.dynamic_content_on_rim.uitools.ui_tools import ( ModuleNotFoundError: No module named 'pupil_labs'', that I can't solve. Could you help me with this?

All of these mentioned dependencies are installed.

Using pip. And yes, I followed drivers installation steps

Windows 10, Python 3.10.

And I'm not using a virtual environment

we have two apps and want to be able to 1. receive the live stream from the phone and send events, then 2. export the raw files from the phone to the computer for offline processing.

I am trying to build the pupil-labs-realtime-api from source on a Windows 10 machine, but I'm not sure how best to go about it.

there's an exception that gets thrown every once in a while, where the raw data in gaze.py has an unexpected length. I

I'd rather like to try and skip receiving that one unexpectedly different package than close the stream at that point, but maybe that is folly anyways

both the newest as far as I know. I checked today, no newer release available for the realtime api

quickly starting up the phone to check the version of the companion app

2.8.2-prod

Actually, are you okay with opening a 🛟 troubleshooting ticket for this case?

~~pl-rec-export produces an events.csv file, which has the same recording start and stop timestamps that would be found in pupil cloud events.csv file. however, this doesn't make sense to me, because if you're using offline processing methods, the Neon scene camera.mp4 does not have the few seconds of grey shown in the videos downloaded from pupil cloud. So the start/end timestamps should be offset. How can this offset be determined? Is there a file generated that indicates what this offset is?~~ nevermind we put our brains together and figured it out

Hi team, I was wondering if there was a way to get access to audio from the glasses using the real-time API?

Do you know if there are any other workaround to get access to the audio?

Hello everyone. I have one question, during the data analysis by pupil player, sometimes I get this message: “[Error] Marker detection not finished”. When I saw this msg, my data will be extracted without any file for heatmapt. I had never have this problem. Is it any way to resolve it? Thanks in advance

Hi @user-2cc535 👋 ! When you enable the surface tracker, it will try to detect all markers in the scene video. You can see on the bottom a graph with a line showing when a marker is detected. If you attempt to export it before it finishes you will get that error.

That one exactly

Hi everyone! I've just started using Pupil Labs and I'm having difficulty understanding how to use it. I've tried to integrate the gaze tracker into my VR project with HTC Vive, but when I hit play in Unity I get a 'not connected' message. Does anyone know how I could resolve this? I'm not sure if there's something I haven't done to make it work properly.

Hi everyone also from me, I am new here. We have been working with our 6 Neon headsets for some time, always using phone/cloud. Now I am trying to explore connecting directly to PC/raspberry. (1) From https://github.com/pupil-labs/pupil/pull/2299 I gather that Neon support was added to Pupil software, but since the latest release is from 2021, I understand there is only support when running from source. Is that correct? (2) I am trying to run from source on Raspberry Pi 4, working my way through many problems. Has anyone run it with success from RasPi?

Hi @user-224f5a 👋 ! Welcome to the Pupil Labs community! It sounds like you're encountering a connection issue with your setup. To better assist you, could you confirm if you are using the HTC Vive Addon (if so we can follow up here 🥽 core-xr ), or are you attempting a custom integration with 🤿 neon-xr ?

If the first one, are you running Pupil Capture on the same computer or a different one? Checking our documentation (hmd-eyes) might provide some insights or steps you might have missed.

Thanks @user-d407c1 , I will check out the link. We are working in a clinical setup, if the data never leaves the site it can save us some struggle with the ethics commission and data protection officer. Can you provide a link to understand the extend of "calibration-free" and "slippage resistance"? I have used the search function in https://docs.pupil-labs.com/neon/ but I was not able to find it.

Tbh, I think in the end we are going to use it with phone, nonetheless I would like to explore if and how it performs directly with a Windows Computer or Raspberry Pi.

I see, so it is not about better or continuous calibration, but rather the fact that user calibrations cannot be saved, like it can be done conveniently in the app.

I will check out what Neon Player offers compared to the cloud's video player (or rather I delegated this to @user-861e5b 😂 )

Hi Pupil, is there a way to have the Companion App automatically have the recording files available through the Android Filesystem without the need to Export the recordings first? This would definitely improve usability and time for users.

Hi @user-b1e770, be default, all recordings are saved in Documents > Neon > Unique ID folder. From the Companion App, you can also tap the folder icon, select all recordings, and export them in one go.

I was wondering if there's any issues with the pupil labs plugin for psychopy? I have tried on multiple computers, and different versions of psychopy and installing the plugin does not allow pupil labs device to be selected in setttings - eyetracking, and it also seems to break installations of psychopy that were working prior to the installation, any idea what the issue could be? Thanks!

Ah, this was solved by installing using pip rather than the package manager.

Hello @user-bc798f 👋 ! Psychopy's structure and its plugin handling are currently undergoing some changes. I believe the Psychopy team is actively addressing a few challenges that have emerged from these updates. For a more detailed update, my colleague @user-cdcab0 , who has been involved in our plugin development, can provide further insights.

That said, happy to hear that you managed to get it working from pip

Hi Pupil Labs Team, does the Neon Companion App have an intent broadcast for recording start/stop that can be received by another application on the same device?

Hi Pupil Labs, I'm trying to access real time data from Pupil Core with Pupil Capture. I'm having a couple problems. First, I would expect that I should be able to access the most recent frame in the socket, or grab all the data on the socket. I think that possibly recv_multipart() grabs the oldest frame in the socket. Is there a way to only grab the most recent frame, or even better, the whole queue on the socket?

Second, when querying the frame times of the frames that I do get, it appears that frame timestamps do not monotonically increase, but instead fluctuate with some frames having timestamps that are earlier than the prior frame. I just want to make sure that this is expected behavior, and that I can reasonably sort frames by their timestamps without negatively affecting the corresponding gaze data. You can see this in an attached plot of the timestamps.

I've attached some sample code for reproduction. Thanks!

Hi! I bought 2 new pupil core in 2021, and then was on maternity leave and havent had a chance to use them until now. While I was on leave I see that pupil mobile deprecated, so I can no longer use them as I had intended. For people who planned to use pupil core for non-tethered paradigms, what do you suggest we do? I dont have the funds to purchase a Neon and I really want to find a way to use the glasses.

Hi, @user-fcb4e7 - I don't believe so, but you could probably gather this from the web socket stream for the real-time API - ws://localhost:8080

Thanks Dom, that's helpful! We had the same thought - I've tried reading the queue as fast as possible (e.g., without pausing between loops), but it looks like the queue fills faster than recv_multipart() can read the data - its never able to empty the queue on the socket. Do you know of a faster way to read off the socket?

Hi pupil labs, after the API and neon app update, my program isn't able to save events anymore. At first, I was getting an error related to gaze, but this was fixed by updating the API to 1.2.0. After updating the API, now im getting an error every time I try to send an event

This was working completly fine before the update. Do you guys know what was changed in regards to sending events using the async send_event() function?

Hi, @user-2255fa - can you share the error that you're seeing when you send an event?

everything else works fine such as starting a recording, getting gaze position, but for some reason when sending an event the server abruptly disconnects

I'm not able to replicate this using the asynchronous or simple APIs. Can you try a simplified program to narrow down the issue? Here's the send-event example from the docs

import asyncio
import time

from pupil_labs.realtime_api import Device, Network


async def main():
    async with Network() as network:
        dev_info = await network.wait_for_new_device(timeout_seconds=5)
    if dev_info is None:
        print("No device could be found! Abort")
        return

    async with Device.from_discovered_device(dev_info) as device:
        # send event without timestamp
        print(await device.send_event("test event"))

        # send event with current timestamp
        print(
            await device.send_event(
                "test event", event_timestamp_unix_ns=time.time_ns()
            )
        )


if __name__ == "__main__":
    asyncio.run(main())

Hi Dom, I think I've resolved my issue - I realized that unpacking the bit string with msgpack.loads() was the bottleneck. I broke recv_multipart() and msgpack.loads() into separate threads and it seemed to mostly resolve the issue. I still see a sawtooth pattern in terms of the timestamps of the items frames pulled off the queue, but I think that can be easily remedied by sorting the data by timestamp after its been collected.

I've attached sample code in case you have any additional comments or thoughts.