🤿 neon-xr


Year

user-0a9f16 07 January, 2025, 22:58:23

@user-f43a29 , thanks for the response. Yeah that is our initial plan: to calibrate for every individual and session to see if it can work out or else we will be considering the custom mount. I got the calibration scene working .

I got few more questions: 1.We will be using PCVR and so the config file is getting saved in AppData folder in the PC. My silly question is: After performing calibration and saving the config, will the next session automatically use the last generated config for offset values? Even if I run my Unity Application (built using OpenXR) with Neon XR core Package after performing the Calibration from the MRTK scene? 2.We primarily want the pupil diameter value but also require the Gaze if possible. Initially, I was using Lab Recorder to get all the metrics for each eye and the XY gaze coordinates. But now I think those values except the pupil diameter wont be useful in VR context and with individual Calibration. What is the way to track the same metrics in the VR space form Unity? What all values can will I be able to record? Can I get an equivalent of Gaze co-ordinate in the VR space? 3. And I am thinking of creating a LSL outlet stream for the VR metrics as we want to synchronize with other sensors. What do you think of this approach?

user-f43a29 08 January, 2025, 10:29:32

Hi @user-0a9f16 , you are welcome!

Here are my responses, split across two messages:

  • PCVR & config files: Not a silly question. Neon XR uses the config file that is saved in the app's Application.persistentDataPath. You can copy the saved config from the MRTK3 Template Project to the corresponding folder that is used by your project. So, it should work the same on your PCVR setup, although PCVR has not been explicitly tested. You could do a quick check by editing the Neon XR code to print the file name and path of the config file that it eventually uses.

  • Obtaining VR gaze coordinates: Obtaining Gaze in VR is certainly possible and is one of the main purposes of Neon XR. And, you are correct that the LSL streamed data will be in Neon's scene camera coordinates, not VR coordinates. The transformation to VR happens within Unity, and yes, you can get those coordinates. For instance, they are the same values used in all the MRTK3 Examples.

user-f43a29 08 January, 2025, 10:29:36
  • Save VR gaze coordinates: If you want to track these metrics in Unity, you could save them into a Unity buffer and then write them to disk at the end of the experiment (or periodically, if your experiment is quite long), but note that you will be limited to the sampling rate of Unity with this approach, if that is relevant.
  • Post-hoc VR transformation: You can also run a recording in the Neon Companion app in parallel. You could then save the pose of the VR camera in Unity over time. Then, after the experiment has finished, you could interpolate the VR camera pose and, using the saved mount calibration, apply it to the gaze data in the Neon recording, similar to how it is done in the link above and as linked in this message (https://discord.com/channels/285728493612957698/1187405314908233768/1326107256936337448). You would then have gaze in VR coordinates at the full sampling rate.

  • Unity LSL outlet: This is certainly a viable approach. It would require some coding, but there should not be anything that would hinder making an LSL outlet for the VR data. Note that also with this approach, you will be limited by Unity's sampling rate, if that is relevant for your use case.

  • Obtaining 3D Eyestate in Unity: This data, including pupil diameter, is also provided in Unity. The eyeball centers and optical axes are even converted to the VR coordinate system. Our fork of the MRTK3 includes a PL_EyeState scene, showing how to visualize this data.

user-aba646 16 January, 2025, 20:54:49

Hello and good afternoon! Just trying my luck; is there a Neon XR custom mount 3d printable file out there for the Oculus Rift S? If not I can give modeling it myself a go.

nmt 20 January, 2025, 03:16:08

Hey @user-aba646! If an Oculus file doesn't already exist, it'd be great if you could eventually share yours in 📸 show-and-tell! I'm sure you're probably already aware, but here's a link to the relevant Neon module and nest geometries that you'll need: https://github.com/pupil-labs/neon-geometry

user-aba646 20 January, 2025, 03:23:05

Thank you. I shall do so!

End of January archive