Hi @user-bd82ac! You can find links to the 3D geometries in this section of the docs. Feel free to ask your other questions here 🙂

Hi @user-bd82ac ! Neon is connected to an Android Device (also when utilised in XR, also known as Companion Device). This device acts as a power, storage and processing unit.

Have a look at the System Overview in our docs.

PS. The battery of this Companion Device while estimating gaze data at 200Hz is around ~4h.

question on the neon will that be availible for the valve index?

Another question can one just get the regular frames like glasses ive seen? If so, mabye an eyewear change?

Hi @user-401946 , you can see all frames that we offer on the Neon product page, including frames that have swappable prescription lenses for participants that wear glasses. However, since you were asking about VR devices, I just want to clarify that while you technically can use the glasses/frames within any VR headset that has a glasses spacer, the data quality could be affected, as the Neon XR code assumes a fixed relationship between the Neon module and the headset. Any slippage of the headset when wearing the glasses will break that assumption and this is likely to occur due to the weight of VR headsets. Also, you will need to re-do the calibration every recording session.

Hi @user-ff2367 ! While it's technically possible to use any frame with NeonXR, the key factor is whether the frame physically fits inside the headset.

Also, it's important to note that if you don't use a dedicated frame designed to maintain a fixed relationship with your VR headset, you'll need to run the calibration each time you use it. This process determines the relationship between Neon's scene camera and the headset's viewport.

Hello @user-f43a29 and @user-d407c1. Thank you very much for the information!

Hello Pupil Labs, we want to use the Neon module in our 3d CAVE system running Unity experiments on Windows. As such, I'm trying to receive real-time eye gaze data into the Unity Editor with the Neon XR Core package, is this possible? If so, how? I followed instructions on this page: https://docs.pupil-labs.com/neon/neon-xr/neon-xr-core-package/ but according to unity console output, there is no device response. I can ping the companion device from my pc, so they are on the same network. Do I need to initiate things first with Python scripts using the real time API as described here : https://docs.pupil-labs.com/neon/real-time-api/tutorials/ or is that not needed with the Neon XR Core package?

Hi @user-ed9dfb ! Two quick questions to help troubleshoot your issue:

1. Are you connected to Eduroam or another institutional network?
2. Have you tried modifying the rtspSettings in your config.json to use a fixed IP? You can find this IP in the Companion App after tapping the streaming icon.

It sounds like the problem might stem from DNS discovery issues.

1. my pc is connected with cable to our institutional network and the companion device is connected through wifi to our institutional network. The companion device is whitelisted so I can ping from my pc to the companion device.
2. How do I do this? When I connect the companion device to my pc, I can't browse any files or folders. I have enabled developer mode and usb debugging and set usb mode to file transfer. I tried using Device File Explorer from Android Studio but I can't locate the config.json file on the device

fyi, the companion device is the motorola edge 40 pro

I can't seem to be able to connect with http://neon.local:8080/ but using the ip adress and port number does start up the neon monitor webinterface. Within the interface, I can only start a recording, but not stop a recording (i need to do this on device). Additionally I don't see any camera feed in the browser.

I tester further with a separate laptop. I created a hotspot on the companion device and connected the laptop to the hotspot. When using the ip adress and port number to open Neon Monitor, I could start AND stop recordings and additionally I could see the gaze circle, but not the video feed

1. Ill see if I can get a dedicated router.
2. after changing the settings like you mentioned, the listening immediately gets aborted

On 2, are you using Eduroam or the hotspot, and could you double-check that the Companion Device did not change its IP address?

Hi @user-083b4f & @user-ed9dfb ! I have been informed that the Unity package uses directly the web-socket port, therefore it should be 8686 and not modified. Please use that port.

Additionally, once you run the project for the first time the config file I mentioned will be copied to Application.persistentDataPath\config.json (which on Windows should be C:\Users\USERNAME\AppData\LocalLow\Mixed Reality Toolkit Organization\MRTK3 Sample\config.json) after that only that one would be evaluated.

Hello @user-d407c1 I can confirm that setting the device ip address and using port 8686 is working, I can now receive eye gaze data within unity editor play mode. Just a note to @user-083b4f , the config.json file location depends on project settings. Easiest thing to do is to click on the file from Unity project window and right click and select Show in Explorer .

Thanks for the help!

Thank you so much! @user-d407c1 @user-ed9dfb It connected successfully!

Hello, Pupil Labs. I'm interested in implementing a feature similar to the one described here: https://docs.pupil-labs.com/alpha-lab/map-your-gaze-to-a-2d-screen/, but within Unity. Do you know if it's possible to perform reference image mapping in Unity?

Hi @user-083b4f 👋 ! The Reference Image Mapper (RIM) is meant more for scenes where you do not have access to the underlying 3D model and the observer's position in it. It derives a 3D model of a scene from the provided scanning recording. Since in Unity you have access to the 3D model and can record exactly where the observer stood/looked in it, you can build heatmaps and perform other analyses without needing a scanning recording + RIM. See our MRTK3 Template Project for some examples.

Thank you @user-f43a29 for your response. I will explore the page you shared!

Hi Pupil Labs, I'm also trying to use Neon with Unity on Windows. I think almost for the same purpose as @user-ed9dfb . After trying all the tips given in the previous conversations, I still run into the same problem. I tried to change the IP addresses using port 8080 or 8686 but without success. I'm using Unity 2021.3.37f1, a OnePlus 10 Pro. The Pupil Monitor works perfectly (video and gaze). Why do you think I still can't connect to the device?

@user-d407c1, Ty for your answer. I haven't installed the MRTK3 package. Is it necessary? I don't want to use a VR HMD.

@user-ce1bfa For me the MRTK3 package was not necessary. Is the IP in the Unity logs correct with your Neon device? If not, try to edit the config file in your (hidden) Windows Appdata folder. I believe it uses this one after you made a build, but I'm not sure.

@user-ed9dfb At last I found this file, which wasn't far away. Thank you very much for your help.

Hi @user-ce1bfa ! Apologies for not getting back to you earlier, and thanks @user-ed9dfb 🙌
That's correct, although is not necessary we recommend the MRTK3 because it contains several examples.

Hi Pupil Labs, I'm having an issue with the device not sending any messages to Unity even though the connection looks fine. Before, when I connected the device, I'd get messages like "2000 messages received" or something similar, but now nothing happens, no matter how long I wait. I didn't change anything, and I'm still using the XR package. Any idea what might be causing this? Would appreciate any guidance!

Hi @user-083b4f ! Did you update to the latest version of the app 2.8.10?

@user-d407c1 Yes. It's 2.8.10-prod

Oh yeah. I see. Thanks for the heads-up!

Dear Pupil @user-d407c1 ,

I am a Ph.D. student in transportation. Due to limited lab resources, I am seeking eye-tracking equipment that can integrate with a VR headset and also function as glasses in real-world settings. My research involves both real-world driving and VR driving simulations, making compatibility with both environments crucial. Could you please assist me in selecting a product from your range that suits my experimental needs?

Thank you.

Hello @user-11daaf ! For using in both real-world and VR environments, I recommend Neon which is modular and can be easily used in both. If you’re interested, please contact us at info@pupil-labs.com to schedule a demo and Q&A session where you can ask all your questions

Hello, I am running into an update issue with my trackers and app. When I open the app and connect the eye trackers, I get this message 'FPGA Update', but the progress bar has been stuck like this for more than 15 minutes. I've also tried changing the network and nothing seems to be working.

Hi @nmt I followed the steps that were stated in that thread and the app is still failing to update properly. I have waited for about 30 minutes and the progress bar is still stuck like the example image I provided last time.

HiNeon team, As mentioned above, I want to use Neon and Unity together. Given that it's possible to read gaze data in real time, is it possible to do the same with IMU data via Unity? Thanks 😉

Thank you for your reply, I'll see if I can adapt the script and suggest my need in the dedicated post.