hi if i buy the quest 3 bundle from the website waht will iget

Hi @user-a3714a , you will receive the Neon eyetracking module, a mount for the Quest 3, and a Companion Device (i.e., an Android phone the runs the Neon Companion app for collecting data). It does not include a Quest 3.

May I ask what you intend to do? What would your research project be?

is there a VIVE pro eye tracking add-on?

Hi @user-78c5b3 , not random! While we do not provide a pre-made solution for the HTC Vive Eye Pro, it is possible to 3D print your own mount for the Neon module. What do you plan to study?

i know its random

i was thinking about the eye tracking part, because i am intrested in eye tracking for Fovated rendering. but i'm not too sure what to start with. thx again rob for replying

You are welcome and I see. Is the intention to use it for entertainment or for research purposes?

entertainment tbf

Ok. Then, just to clarify, although Neon could be used for such a purpose, it is designed with research in mind. It is not plug-and-play for foveated rendering in games, for example. At the least, you would need to find games that are moddable and build a foveated rendering plugin with Neon's Real-time API.

You might benefit more by searching for XR eyetracking equipment that is explicitly made for entertainment purposes.

i get it thank you

Hi @rob is there a specific pupil-labs-neon-recording version to get the neon-vr-recorder runnning? Im using Python 3.12 but Im getting this error: neon-vr-recorder.venv\Lib\site-packages\pupil_labs\realtime_api\streaming_init_.py", line 3, in from pupil_labs.neon_recording.stream.imu.imu_pb2 import ImuPacket ModuleNotFoundError: No module named 'pupil_labs.neon_recording.stream' Could´nt find the module in pupil-labs-neon-recording 2.1.4. Do I´ve the wrong version? I used the requirements.txt (Companion App v.2.9.23-prod) Thanks

Hi @user-486277 , can you run the following in that virtual environment:

pip install --upgrade pupil-labs-realtime-api

Does that resolve it?

Thank you. It resolved the module error and after i installed scipy which was missing and wrote the ip of client_gaze in record.py it started but the window looked like in the image and did not record anything. Did i made a mistake? Do I have to calibrate something? We are using the meta3 with the official mount of pupillabs.

You do not need to calibrate anything. How long did you wait when that screen appeared?

It appeared immidiatley or do you mean after I it appeared how long did i put he HMD on? To be honest i took it off after I saw that the image is cut

You don't need to put the HMD on for the tool to work. I mean when the terminal appeared and started showing messages, did you close it immediately when you did not see the VR image being updated or did you wait a few seconds? The messages are saying that it has not yet received a VR image, so it might need some more time to establish that connection. You might need to wait ~15-20 seconds.

i tried it longer now...still no data does it have to be a unity VR image?

It does not need to be a Unity VR image. May I ask what version of MetaOS you have installed?

It`s v.2.4.1034

Thanks, let me check with the team. There have been versions of MetaOS that do not function correctly with the scrcpy tool that is used internally by neon-vr-recorder and workarounds needed to be implemented.

Hi @user-486277 , this seems to be a rather new release of MetaOS. The Neon XR team is looking into it for you.

Hi! I'm using Neon XR mounted in a Quest 3 (via the dedicated mount) for gaze calibration research, and I have two questions about accuracy in the XR setup.

1. Is there a dedicated accuracy test report for Neon when it's mounted in a headset (Quest 3 / Pico)? The Neon Accuracy Test Report I found covers the glasses-frame setup (screen-based + in-the-wild), and the VR/AR tech specs page seems to cite the same 1.8° / 1.3° figures — so I'd like to confirm whether those numbers are expected to hold when the module is mounted in an HMD.

2. If there isn't a dedicated report, could you share a rough expected accuracy for Neon mounted in a Quest 3? I'd like to use it as a reference to check whether what I'm getting in VR is within the expected range or points to a mount/calibration issue on my side.

Thanks!

Hi @user-6671c6 , here are responses to your questions:

1. It is expected to hold, assuming you are using the correct mount calibration for your VR headset. Essentially, when used in a VR headset, it is still the same Neon module, NeonNet pipeline, and Neon Companion app, working with basically the same eye images.
2. Can you share the numbers you are getting and the kind of stimuli/experiment you are running? Did you also run your own mount calibration or are you using the default?

Thanks for the quick reply! Here are the details. My setup is a Neon module mounted in a Meta Quest 3, and the calibration/evaluation scene runs entirely in VR (markers are virtual targets rendered on the Quest 3 display, not on a physical screen).

Stimulus / experiment: a 25-point grid at 5° spacing, with the virtual marker presented at 1 m depth in the VR scene. The participant fixates each marker, and I compute the mean angular error across the points.

Numbers: the mean error is most often around 2.3°.

Calibration: I'm using the gaze ray (visual axis) that Neon outputs — a single ray from the sensor origin — after running the mount calibration from the MRTK3 Templete Project per subject.

I've also confirmed in the Neon Companion app that the eyes are clearly captured by the eye cameras.

Is ~2.3° within the expected range for this kind of in-VR setup, or does it suggest something is off with my mount calibration?

Thanks, @user-6671c6 . When you say you are using the "gaze ray that Neon outputs", are you using the gaze ray from the Neon XR Core Package? In particular, are you using these outputs?

And, how do you ensure that the config.json file with the mount calibration is being used by your custom project?

Yes — I'm using GazeDataProvider.GazeRay (the .origin / .direction outputs you linked).

For the mount calibration: I run the calibration in the MRTK3 sample app, then from the terminal I use an adb command to push the resulting config.json (the one containing the mount calibration) into my own app's config.json location on the headset.

Thanks. And, may I ask if this was done with or without Gaze Offset Correction? If it helps, in a VR context, I find it easiest to do Gaze Offset Correction while wearing the headset and viewing a simple scene with a gaze pointer. I then hold the phone in my head and can drag and update the pointer in real-time.

Thanks! No, I haven't been using the Companion app's Gaze Offset Correction. I thought it wasn't possible in neon-xr because the scene camera doesn't work there — and I had mistakenly assumed that the PL_Calibration scene in the MRTK3 sample was serving that role in VR.

Hi pupillabs team ,What data is recorded by Neon XR? Is the data collected the same as other Neon models?

Hi @user-bd5142 👋 ! If you press Record on the Companion Device or remotely start a recording, everything but the scene camera video would be collected, since the scene camera is physically blocked by the headset.

One important point: gaze and fixations are still initially expressed relative to the Neon scene camera. To use them in your VR environment, you would need to run NeonXR to remap gaze into the VR coordinate space.

Those remapped coordinates would then be stored separately in your Unity environment, or in a separate file if you are using neon-vr-recorder. May I ask which metrics you’re mainly interested in?

Thank you for your reply. I would like to know how Neon XR collects and exports data, as well as what data can be output

If I use a neon VR recorder, then gaze and fixation can also be output. Is that what I mean?

Everything is collected by default. If you start a recording, all data streams except the scene camera video are recorded on the Companion Device when Neon is mounted inside a VR headset. Gaze and fixations are still computed and recorded, but they remain relative to the scene camera coordinate system.

NeonXR (the Unity integration) does not store data by itself. It streams gaze and related data into Unity using a variant of the Realtime API, allowing you to use the information directly within your experiment. It includes utilities for headset calibration (to determine the position of the Neon module relative to the VR headset), remapping gaze into the VR environment, and several MRTK3 examples demonstrating possible interactions and use cases.

How and where the data is stored within Unity is entirely up to your application. You can choose to save it there, start recordings on the Companion Device, or do both depending on your workflow.

neon-vr-recorder, on the other hand, is simply a utility for recording third-party VR content and overlaying gaze data onto the captured footage for later review and analysis.

Thanks

Hello 👋 Everyone

I m testing a pupil labs neon xr with quest 3. Is there an apk or a repo ready to use of an app that i can use in the quest to test the pupil hardware.

Hi @user-81238c! Yes! You can find instructions in our documentation: https://docs.pupil-labs.com/neon/neon-xr/