Hi @user-c9caf9! I'll step in briefly here for Rob. It's the Unity coordinate system!

Hey @user-386d26! Can I just confirm, have you been able to successfully deploy the sample project or not?

Hi @user-8bd1e2! Pupil Core is a calibrated system that requires a controlled environment like a lab (e.g., with constant lighting, and no headset slippage) to achieve the best results. Using the 3d detection pipeline (more info on the gaze mapping pipeline can be found here ), you can achieve accuracy of ~1°–2°.

For a more versatile solution, we'd recommend NeonXR (our VR solution for Neon). I recommend exploring also the NeonXR docs.

Neon employs deep-learning-based gaze estimation, which is calibration-free and highly robust in dynamic environments. It connects to a smartphone via USB, with the phone serving as both the power source and recording unit, ensuring full portability. In terms of accuracy, you can expect 1.8° uncalibrated (and 1.3° with offset-correction). See also the full list of Neon's specs here: https://pupil-labs.com/products/neon/specs

You can also find a high-level description as well as a thorough evaluation of the accuracy and robustness of the algorithm in our white paper: https://zenodo.org/records/10420388

Hi again @user-8bd1e2 - You can find publications using our products in our publications list. Please note that, as NeonXR was launched just a year ago, there may currently be few publications specifically using NeonXR.

We maintain a Neon XR package for Unity. It is a real-time client implementation in Unity and example implementation of transforming the real-time eye tracking data into the virtual coordinate system. The package provides examples to show you how to utilize eye tracking data within Unity very easily. You can find more details here: https://docs.pupil-labs.com/neon/neon-xr/neon-xr-core-package/

To the best of my knowledge, there's no validation work comparing NeonXR to the Meta Quest Pro's internal eye tracker.

Please note that NeonXR uses the same technology as Neon, that is a deep learning approach which provides calibration-free and slippage-invariant gaze data in any environment. You can refer to the specs and Neon's accuracy test report for more details (see my earlier message: https://discord.com/channels/285728493612957698/1187405314908233768/1316750822264148039)

Hey @user-8779ef! I've moved your message here since you're interested in VR.

Regarding gaze, you're right. Neon provides a 2D gaze estimate, which is also expressed as a cyclopean gaze ray (elevation and azimuth [deg]). Depending on the selected gaze mode in-app, Neon's gaze can be generated either from binocular eye image pairs or monocularly from the left or right eye image.

Currently, you can't obtain both left and right eye monocular gaze estimates simultaneously. You might want to suggest this in 💡 features-requests.

Of note: We also generate eye state data with Neon, allowing you to access the optical axes of both eyes. It would probably be feasible for you to implement a custom calibration to transform these into visual axes within a VR environment.

No problem. No, this isn't possible with the current pipeline. The monocular gaze is only computed in real-time on the Companion device.

That said, do you think the eye state measurements could be useful for you?

Hi @user-0c6521 👋! The Pupil Core VR Add-ons, including the HTC Vive Add-on, have been deprecated in favor of NeonXR.

Neon is a much more flexible and robust solution—it’s modular, so you can use it with various headsets or even in a regular frame. Plus, its deep learning approach makes it more robust and easier to use.

We don’t have an off-the-shelf frame for the HTC Vive with NeonXR, but we’ve made the module geometries, PCB board, and a frame design open source so you can easily build your own mount.

That said, if you really want the old HTC Vive mounts based on Pupil Core, feel free to reach out to us at sales@pupil-labs.com to inquire.