Hi! We are using the pupil core with a latte panda as we disucussed it with you and you told us to do it that way. We were wondering why it is not possibe to compile it in a arm64 architecture in order to use a Raspberry Pi or similar. Another question is related to try to do the Pupil Core wireless; we think we saw the pupil core with a mobile phone connected, could this be? Finally, if we do not calibrate de Pupil Core, we see that gaze data and fixations are not correctly gathered, could you please confirm this? As we are using this hardware in different settings, sometimes calibration is not possible and we wanted to know which data can be gathered correctly in these scenarios. Thanks!
Hi @user-6a6d64 Hello everyone
I’m glad to see someone working on something similar to what I am doing: I am currently based on this project (https://github.com/Lifestohack/pupil-video-backend) to implement using a Raspberry Pi as a relay, so as to enable remote connection with Pupil Core.
However, in my practice process, I have found that this solution has several problems:
The performance of the Raspberry Pi and the speed of remote network transmission limit the quality of the Pupil Core captured video. If using a resolution of 1280×720, the frame rate is very low (about 5 FPS). If using 640×480, although the frame rate is higher, the stability and accuracy are not as good as high resolution, and this may affect subsequent eye-tracking data applications.
Due to the above problem, I have already used grayscale images instead of RGB images, but this problem still exists.
The problems caused by this low transmission quality are not only stability and accuracy, but also frame loss, and loss of pupil detection after large movements.
If you want to use such a microcomputer to connect Pupil Core, I guess you want to perform portable eye-tracking data collection in motion. In this scenario, I found after experiments that the best calibration method is Screen Marker Calibration Choreography, but the default Screen Marker Calibration Choreography calibration points are not enough. Therefore I recommend you create more custom images similar to Screen Marker Calibration Choreography and use single mark mode for calibration.
The GitHub project mentioned above actually has bugs. If you also want to implement remote transmission based on this project, feel free to contact me, we can discuss it together.
I hope my practical experience can help you. If you have any questions, you can also contact me.
Hi @user-6a6d64 👋! The LattePanda uses an x86 architecture, which is one of the reasons it is a more practical option for running Pupil Capture. The recommendation actually stems from the community, as it tends to provide the smoothest experience.
In principle, compiling the Pupil Core software for ARM64 devices such as a Raspberry Pi might be possible, but it is neither a straightforward nor officially supported workflow. Some dependencies do not provide precompiled ARM wheels, so several components need to be built from source. A number of users have attempted this in the past but ultimately given up due to the complexity.
Regarding wireless use, you may have seen Pupil Mobile in the past. However, it has since been deprecated. It only supported the OnePlus 6 and is no longer a recommended or supported workflow for Pupil Core.
As for calibration, yes, Pupil Core **always **requires calibration to produce meaningful gaze and fixation data. Like most traditional eye-tracking systems, calibration establishes the mapping between pupil position and gaze location. The gaze estimate remains valid as long as the headset stays in the same position relative to the eyes. If the glasses slip, are removed, or are repositioned, that mapping is no longer valid and gaze/fixation accuracy will degrade.
Without calibration, you can still record the raw eye videos and pupil detection data, but the gaze position and fixation outputs should not be considered accurate.
If portability is important and calibration is not always feasible in your setup, would you open to consider Neon instead? It is tethered to a Companion Device (phone) and it uses a deep-learning-based gaze estimation pipeline making it calibration-free, and much better suited for mobile or less controlled environments (out of the lab).
Hi Miguel, thanks a lot for your quick answer! We will investigate the option to compile it in an ARM and thanks for the rest of the answers!
Hi, yes! It will be great to exchange. I leave you my email address: [email removed] so that we could further discuss if you do not mind. Thx!!
Hi！ Okay, if you would also like to use a Raspberry Pi as a relay, we can discuss it together. You can contact me via this email: [email removed]
Hi. Does anyone know whether it matters which eye is which. ie. Pupil Capture references eye 0 and eye 1. Should eye 0 be the left or right eye, or does it not make a difference?
Hi @user-be1dcc , when you say "does it not make a difference", do you mean does it not make a difference for the eye tracking pipeline? Or, does your research question depend on comparing the left to the right eye?
And a further question. I am using the hmd-eyes Unity plugin. I'm using the default calibration targets (a circle of points). I'd like to change this configuration but when I do, it seems the eye tracking behaviour is not quite as good. Are there any recommendations for the configuration of calibration target points?
I mean, in terms of eye tracking output quality (stability or binocular/depth output) from Pupil Capture, should eye0 be the left or right eye?
It has no influence and you do not need to do anything with respect to that. The algorithm implicitly accounts for this.
When you say the default calibration, do you mean the Pupil Capture default calibration that displays markers on a flat computer display or do you mean the 3D display of points in a VR scene via hmd-eyes?
I mean the 3D sefault calibration targets offered in the hmd-eyes examples. Consists of 3 rings at different depths.
Ok, great. Thanks Rob.
You are welcome, but please note that if you are using the Z coordinate of the 3D gaze point to estimate depth, then this is unreliable after about 1m of distance from the person's head.
Note that this is different from the 3D gaze ray provided by hmd-eyes. If you raycast that and compute depth from that, then this is reliable.
Currently I use the gazeData.GazeDirection vector and then use trigonometry to get angular gaze values for azimuth and elevation. Does that sound like a good approach?
Yes, this would be the 3D gaze ray provided by hmd-eyes. This approach is reliable.
Thanks Rob.
does anyone know of an issue where eye cameras have varying focal length ? It seems my left eye is more zoomed in than my right eye, which didn't happen before. I try tingling with the lenses but it only affects focus. Could it be a software tweak ?
Hi @user-9a85ca 👋! The eye cameras on Pupil Core are arrested in place with glue to maintain the correct focal length. If that glue has loosened, it's possible that the focus has shifted.
The camera zoom itself is not a configurable parameter in Pupil Capture. Do you happen to have some example images from the eye cameras? That would help us better understand what you're seeing.
Also, keep in mind that the eye camera image can appear different depending on the selected resolution, so I'd recommend checking the eye camera settings in the corresponding eye camera window first.
Other than that, the only adjustment you can make is physically repositioning the eye cameras using the camera arms to obtain a closer/farther view of the eye.
Im trying to download the pupil core software but my computer isnt recognizing the core device. I tried following the instructions to download the driver by running as administrator and etc but that doesn't work. Do I need to download the drivers seperately or is there something else I can do because no instrutions ive followed so far have worked. Thank you.
Im trying to download the pupil core