Hi! Does anyone have a screenshot of a footage shot with neon eye camera?
How long should "processing" take for videos once they have been uploaded to the cloud? I'm consistently getting videos taking upwards of 15 minutes to process and then be able to be worked on. Is this expected?
Hi @user-ac085e π. Thanks for your question! I wanted to provide some insight into the processing that happens when recordings are uploaded to Cloud. - Firstly, the gaze estimation is re-computed. While real-time gaze estimation is done on the phone, there can be fluctuations in the computed sampling rate, e.g. on older phones or during certain 'peak' loads (like other apps running in parallel). However, Neon's cameras always sample at 200 Hz. So, once the recordings are in the Cloud, we re-run the gaze estimation at 200 Hz - In addition to gaze estimation, fixations and scanpath visualisations are also computed, and a blink estimation algorithm is run All of these processes can take a little while to compute, along with the load on our servers at any given time. What you report doesn't sound unreasonable. That said, I understand that it would be nice to have an immediate playback option, especially for performance reviews. I'll pass this feedback on to our Cloud team!
Hey @user-edef2b! Thanks for the clarification! Yes, for the vertex distance there would definitely be some variance depending on how the glasses fit the facial geometry. I measured this just now for my central European facial geometry and got ~20 mm. On people with flatter nose bridges I'd expect this to be smaller, but I can't give you an estimate on the expected range as we have not measured this on a larger population.
The thickness of the plano lenses is about 1.8 mm.
That helps a lot, thank you!
Hello everyone, we have been working with the Pupil Neon for the first time in the last few days and had problems calibrating the glasses. We need a high accuracy at a distance of 30 meters and the small touch screen does not allow us to align the circle very accurately. Is there a possibility to do the calibration via the Python API? Thanks! Best regards William
And in addition, I would be interested to know what the training data for the NeonNet must look like. Is there a possibility to do a fine tuning by ourselves?
Hey @user-c6dc34! Great to hear you're up and running with Neon!
It's not possible to perform an offset correction via the monitor app/python api. However, there is a post-hoc offset correction feature request on our feedback board. Such a feature would probably enable more refined control. Feel free to comment/upvote: https://feedback.pupil-labs.com/pupil-cloud/p/post-hoc-offset-correction
The training data comprises a large cohort of wearers with different ages, ethnicities, facial characteristics, inside and outside etc. We are currently working on a white-paper to elaborate on these factors so stay tuned for updates π. It's not currently possible to fine-tune the network.
Hey people at Pupil labs! I'm Wee, a postdoc from Uni Ulm, working under Prof Dr. Anke Huckauf's lab. I'm testing out the pupil neons we've got, and I came across a bug. Not sure if this is the right place for bug reports, but I'll describe what happened.
I am running some long-form recordings of people playing Magic: the Gathering Arena. After a 5 minute game, the glasses were already quite hot (LED light turned on during recording). I waited for about 1 minute before starting the next match, which I then pressed the record button on the app. However, this time, the glasses did not record anything. Instead, the notification message said "Recording error; we have detected an error during recording!". This was followed by a prompt to disconnect and reconnect the glasses. I did that, and then recorded another match, this time with the LED turned off during recording because I did not want the glasses to be hot. Interestingly, the glasses started recording even though I did not press the record button (note: i only noticed this after viewing the recording).
FYI: I am using the early adopter's Neon, and I've also updated the companion app and the firmware for the glasses. Edit2: On Pupil Cloud, I cannot add events to the recording that had the bug. This recording showed up as a 9minute video, presumably from the previous recording which bugged out, lasting until the end of the new recording.
Hey @user-07e923 π. Thanks for getting in touch and for being an early adopter βΒ it's much appreciated!
The module itself does warm up but should not overheat. We recommend leaving a small gap between the module and forehead - this also reduces headset slippage (although that's not really an issue now with Neon's calibration-free pipeline π).
The recording error is unexpected. It looks like the recording actually started the first time (when you got the error) and then continued (even after you disconnected and reconnected the USB cable) until the very end. The app will continue recording even during a disconnect - it's designed to.
Would you be able to share privately the details of that recording in Cloud such that we can investigate why you can't add events? If so, please contact info@pupil-labs.com with the recording id.
Hi! A bit of a long shot, but is there anyone from Auckland, New Zealand that has a Neon that I could see in person? I'm thinking of purchasing it for my research. Thanks π
I would like to inquire if Neon supports lens replacement, just like Invisible, is there a replacement lens that can be replaced? thanks
Hey Roy π ! Yes, we will offer the possibility of using Neon with different prescription lenses. In fact, we will be releasing soon a frame for Neon (we call it "I can see clearly now") that is designed to hold prescription lenses, and it has a quick-change feature to easily switch out the lenses for different prescriptions. This makes it easy to adjust to new subjects quickly. It will cover a wide range of prescription values.
Sorry to bother you, but I am having trouble with the following situation. Do you know how to deal with this?
γΌγΌγΌγΌγΌ I am using both Neon, Pupilinvisible, and a day ago my data stopped uploading to PupilCloud. 2 days ago my data was uploading to PupilCloud fine, but now it is not uploading to PupilCloud.
Cloud Upload is turned on in the smart phone settings.
Hey @user-5c56d0 π ! Sorry to hear that you're having issues with uploading your data to Pupil Cloud. We recommend running a speed test that can be used to track cloud connectivity issues. Could you please open this speed test on your Companion Device phone and send us back a screenshot of the outcome? https://speedtest.cloud.pupil-labs.com/
Thank you, this is it.
Thanks @user-5c56d0 - could you please also try the points mentioned in this older message by @marc? https://discord.com/channels/285728493612957698/633564003846717444/1040244639128633404
Two additional steps that you could try are the following:
2.6.16-prod. You can see your current version in Settings > About Neon Companion.Hello, @user-c8a63d I'm Rebel from Eyesquare. The problem we often have is that some videos don't upload and just get stuck, so we tried everything and it didn't work. cache deleted and neon app deleted and reinstalled and imported the videos again andres wlan unfortunately everything didn't work.
Hi @user-104eed ! Could you share the recording ID of one of the affected recordings? You can find the ID in the info.json file in the recording folders on the phone, which you can access as described in the following guide:
https://docs.pupil-labs.com/neon/how-tos/data-collection-with-the-companion-app/transfer-recordings-via-usb.html
Hello Rebal! Just to make sure, you have automatic upload enabled in the app settings?
Hi, I am having a similar issue with 10 or so recordings failing to upload. The app is updated, the speed test showed decent results, and automatic upload is enabled. How can I fix this? Thanks
Yes 10 videos are uploaded and only the 3 at the top are not
Hi @user-104eed! Can you please share the Android OS version that's currently installed on your companion device: settings -> About phone -> Android version
Hi Can the data collected by Neon be uploaded from the PC to the Pupil cloud instead of directly uploading through WiFiοΌthanks
Hi @user-4bc389! Recordings can only be uploaded to Pupil Cloud directly from the Companion app, to ensure security and stability
Hi Marc, Yes, I have already prepared whether I want to send via Discord or create an upload link for myself
Just recording IDs would suffice. You can send them via DM on Discord to me!
Hi!! Iβve literally just unboxed my neon kit and made a couple short testsβ Iβm having troubles with Pupil cloud (as someone else specified in the invisible channel) The recordings uploaded to the cloud right away but after 20 min they still show processing in the Cloud.
we have the same problem today
Just wondering if this is normal amount of time to wait for processing or if thereβs a hiccup somewhere π
Hi @user-162faa! Sorry for the sub-optimal first impression! As described in the πΆ invisible channel there does seem to be an issue on our end. We are looking into it! π§βπ§
Gotcha! Iβll wait it out!
@user-104eed We have found and resolved the cause of the issue with recording processing. Computation for processing should now finally actually start and the accumulated queue of recordings should be finished within the next 1-2 hours. So check back soon!
We are still looking into the issue where recordings do not start uploading in the first place.
@marc why do I get {'type': 'response', 'status': 500, 'message': 'Internal Server Error'}). Reconnecting... while trying out the realtime api?
Hi @user-25fb27! Could you elaborate a little bit on what you are doing exactly to produce this error?
@marc I just wanted to grab the frames. This is my code
import cv2 from pupil_labs.realtime_api.simple import discover_one_device device = discover_one_device()
try: while True: frame, gaze = device.receive_matched_scene_video_frame_and_gaze() cv2.circle( frame.bgr_pixels, (int(gaze.x), int(gaze.y)), radius=80, color=(0, 0, 255), thickness=15, )
cv2.imshow("Scene camera with gaze overlay", frame.bgr_pixels)
if cv2.waitKey(1) & 0xFF == 27:
break
except KeyboardInterrupt: pass finally: print("Stopping...") device.close()
It throws Error on stream: RTSPResponseError('RTSP request error for PLAY rtsp://IP/', {'type': 'response', 'status': 500, 'message': 'Internal Server Error'}). Reconnecting... exception during session: <class 'aiortsp.rtsp.errors.RTSPResponseError'> REASON: ('RTSP request error for PLAY rtsp://IP/', {'type': 'response', 'status': 500, 'message': 'Internal Server Error'}) RESPONSE: <Response status=500 msg="Internal Server Error"" headers={'server': 'Pupil Invisible RTSP Server', 'cseq': '4', 'content-length': '0'} content-length=0>
Thanks for the clarification! I have not seen this particular error before. An internal server error indicates that something inside of the server hosted by the Companion app crashed (duh!).
Could you try the following? - Force stop the app (press and hold the app's icon on the home screen -> app info -> force stop. - Restart the app and double check using the live preview that everything is working fine on the surface. - Try the real-time API again.
Also, could you see if the Monitor app's server also hosted by the Companion app is running fine? See here for more details on that: https://docs.pupil-labs.com/neon/how-tos/data-collection-with-the-companion-app/monitor-your-data-collection-in-real-time.html
it works in the browser but not via the script. what could be the issue?
Thanks for checking! Interesting to see it works in the browser. Diagnosing this is a bit difficult given the limited information in the error message. I have a couple more questions:
- What operating system are you using?
- Which line in the code throws the error? Device initialization or pulling data?
- Could you let us know the versions of installed Python modules by reporting the output of pip freeze?
- Could you check if the previous release of the API also produces this error? You can install it via `pip install [email removed]
The old API running on ZMQ was deprecated a while ago and is not available for Neon. In theory, the new API is a superset in terms of features and is using standard protocols for wide compatibility. I realize this doesn't help if its not working for you though...
Also isn't there any documentation to do it with pure zmq messages as it was with invisible and core? never got such error previously with those apis
Hi @user-f36bd4! What exactly have you tried and is not working? Correct, recordings made with the Neon Companion app can not be opened in Pupil Player.
any possible fix?
Hi @user-25fb27 ! this might be related to https://discord.com/channels/285728493612957698/446977689690177536/1116862714871021678
I see that you have av installed, would you mind trying this on a virtual environment where you don't have av or you have it installed from source ?
Hi. Have a question about the real-time API. If I ask for "device.receive_imu_datum().timestamp_unix_seconds", how should it compare to "time.time()" in Python? Should it be the same?
Hi, yes it should be (minus latency) if the computer and the phone are set to the same time.
Thanks Moritz for the response! I sometimes get the same time and sometimes with delay. I will change the phone time to see if this happens again.
See also this guide on time synchronization: https://docs.pupil-labs.com/neon/how-tos/data-collection-with-the-companion-app/achieve-super-precise-time-sync.html
Thanks!
Is there an available 3D model of the Better Safe than Sorry frame?
Hi @user-edef2b! No the CAD model for "Better Safe than Sorry" is currently not publicly available. Only "Just act natural" is available here: https://github.com/pupil-labs/neon-geometry
I have been playing around with our new neon. I have two problems:
1. When I use neon with the companion app, everything seems to work. I am able to record. I can open the streaming view but... in the streaming view it is unable to load the scene camera.
2. When I plug the neon in to use with the most recent version of pupil capture (on OSX, 13.2.1, M2 Max), it doesn't load. When I go to the video source to activate the neon manually, it is part of the list of options but when I select it, it indicates that "The selected camera is already in use or blocked".
Suggestions for either issue is much appreciated!
Hi @user-19bba3 π! Please see my responses below: 1. Monitor app - what device and network were you running it on? Was it a local network or an institutional one? 2. Pupil Capture - If you want to run Neon with Pupil Capture, you'll need to use the latest dev branch and run from source: https://github.com/pupil-labs/pupil/tree/neon-support. We're currently updating the docs to reflect this, please bear with us π . Note, that in this case gaze estimation will not be done using NeonNet, but using the Pupil Core's gaze estimation pipeline which requires calibration / is not robust to slippage etc. Let me know how it goes.
Update to #2. Plugged the neon into a linux machine (Ubuntu 22.04) and now I am able to activate the scene camera but it still will not recognize the eyes.
All of this is with capture version 3.5.8
@user-19bba3 hi! with streaming view you mean the network monitor view or do you mean the in app scene camera preview?
The network monitor view. After I can load the recording and see the scene camera.
Hey everyone at Pupil Labs,
I've a quick question concerning Pupil Cloud's Raw data export. Is it possible to exclude the scene_video.mp4 from the download? I have about 241 recordings (each recording at least a minute or so), and this is too many GBs to download at all once. Making several smaller projects seem to be the current solution, but this is quite odd since I will have to repeat the enrichments again for each subset of data.
Thanks for your help! Keep up the great work πππ
Hi @user-07e923! Given that the Raw Data Export enrichment has been removed in the recent UI update, I am assuming you are still using the old UI available at old.cloud.pupil-labs.com?
In both the new and old UI you can multi-select recordings, right-click and select Download -> Timeseries Data. This will download the same data you get from the Raw Data Export enrichment.
In the old UI you can do this in the recoring list in Drive view. In the new UI you can do this from the workspace view (the one you get when clicking the workspace name in the top left).
is there any python api to playback the recording?
Hey @user-25fb27! Not sure I fully understand the aim here. Would you be able to elaborate? E.g. playback in Cloud or something else?
I have download the data from cloud. I wanted to play it locally with python.
I see. There's currently no existing tooling for this. The best bet would be to download your gaze overlay enrichments with Cloud and then play them back offline. Note that we are working behind the scenes on a script to enable playback of Neon recordings offline. We havenβt a set release date just yet. Later in the year, we also plan to have a simple desktop app with similar functionality.
When the pupil diameter measurements finally arrive for the Neon during Q3 2023, will the diameter measurements be streamed in real-time (via realtime-python-api)? Or do we have to use the Pupil Cloud to get the measurements post-hoc?
It will most likely be available in real-time as well.
Hi team, is there a way to allow a collaborator access to only one project within a workspace, or do they get access to the entire workspace by default?
Hey @user-ae76c9 π ! You can invite your colleagues to become members of a workspace. This will give them access to all the data and projects within a workspace (with different rights depending on the role you would like to assign them). You can find all the details about projects and workspaces in our documentation: https://docs.pupil-labs.com/neon/basic-concepts/projects-and-workspaces/#projects-workspaces
I have made few recordings with the device. The saved recording has the world video, blinks, gaze etc.., Where are the feeds from the eye cameras?
Hey @user-25fb27 π ! You can access the eye cameras video by downloading the raw data. To do so, please right-click on the recording > Download > Raw Android Data. The folder you will download will include the eye cameras video (Neon Sensor Module v1 ps1.mp4).
Please note that if the Raw Android Data option is not shown, you will have to enable it in your workspace. To get the data in this format, please go to the Workspace Settings section and then click on the toggle next to "Show Raw Sensor Data". There you can enable this option in the download menu. Please keep in mind that you need to enable this feature for each workspace you'd like to download the raw android data.
I hope this helps!
or do I have to manually enable recording the eye camera feeds as well?
@user-480f4c I see "Dowload as pupil player format" and not "Raw Android Data"
Did you use Neon or Invisible for your recordings?
Hi, I'd like to use the annotation feature to annotate the start of trials in my experiment. I set the annotation key in pupil capture to 'x'. I considered letting my participants press the same key ('x') to initiate trials in my PsychoPy experiment so I'd get an annotation each time a trial starts. The annotation key, however, does not seem to register an annotation when the PsychoPy window is active (covering the entire screen). Is it known if there is a way to bypass this so that the 'x' key both initiates the next trial and annotates simultaneously?
Hi @user-d99595 π just to clarify: are you using Core or Neon along with Pupil Capture?
Hi!
I'm using the Core with Pupil Capture!
Hi everyone, nice to meet you. Can I ask anybody questions about the functionalities of the neon model?
Hi @user-7ad436 π ! Sure! What would you like to know ? You can also write to info@pupil-labs.com and request an online demo.
Thank you! Can you confirm me that the pupillometry functionality will be updated in Q3 2023? I know that it is already written but just to be sure and receive the information from a person. Also, are the software and hardware sold together in the bundle package?
Yes, pupillometry and eye state are still expected by Q3, but we can't offer a more concrete deadline at this moment.
And yes, both, software and hardware are inc. in the bundle.
Thank you so much!
Help or advice. I am exporting eye camera videos from the neon eye camera (both eyes concatenated horizontally), but my video is ending up scrambled at some point. I am wondering what is happening orwhat could be goign wrong here.
Hi @user-b49db3 π ! How are you reading the eye videos? They might be split into multiple videos ps1, ps2, ... that is. normal.
Apols I wasnβt clear we were using the real-time api β¦ everything in the resulting video looks fine initially but then becomes progressively more scrambled.
What operating system are you using? I've noticed that the scene video becomes progressively more artifact-y on my Debian machine than it does on Mac
HI there. Just to hopefully help/update others. The problem was the firewall (PC) . Switching it off seemed to fix