Hi, I am looking into the intrinsic matrix of Invisible's scene camera (obtained from https://github.com/pupil-labs/pupil/blob/master/pupil_src/shared_modules/camera_models.py) and the focal lengths are about 766 with the principal point at about 550.

Are these in mm? If yes, does this mean that the image plane (physical sensor) is at a distance of 766 mms (76.6 cms) from the lens? [It sounds phishy or I am missing something conceptually crucial] Or is this missing a scaling factor?

https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html

fx, fy are the focal lengths expressed in pixel units.

cx and cy are also in pixel units!

Thank you @papr and @marc . Can you please tell me the actual sensor measurements?

@user-df9629 I can send you the specifications in a private message

@papr , that works! Thank you!

Hi! I'm totally new using pupil invisible. I have trouble launching pupil player on my mac, and also the gaze marker doesn't match my actual gazing area.

@papr

@user-253e26 greetings 👋 - thanks for the feedback. Let's split this into two separate topics so that we can best assist you: 1. Installing Pupil Player on MacOS: What issue are you experiencing? Trouble during install? 2. Gaze marker not matching your gaze area: Can you provide a concrete example? Do you mean that the gaze represented in Pupil Invisible Companion app or Pupil Cloud during playback does not correspond to what you are looking at? Have you tried the "offset correction" feature in Pupil Invisible Companion App?

For the first question, when I try to open pupil player, there is no response at all. There is no problem during installation.

@wrp

@user-253e26 what macOS version? Is it possible that you are being prompted for permissions to run the app on another display/workspace? You might also want to try deleting pupil_capture_settings directory in your home folder and try opening again.

There is nothing being prompted at all. I will try deleting it first. Thanks!

@papr macos catalina (v10.15) potential install issue for Player ☝️ please advise @user-253e26 if this is not resolved via restarting with default settings

@wrp where do I look for pupil_capture_settings directory?

@wrp For the second question, both the gaze represented in Pupil Invisible Companion app and Pupil Cloud during playback does not correspond to what I'm looking at.

I didn't find the "offset correction" feature in Pupil Invisible Companion App

Does this have anything to do with astigmatism?

@user-253e26 what version of Pupil Invisible Companion App are you using? Long press on app icon from home screen and click get info scroll down to bottom to see version number

version0.8. 18-prod

@user-253e26 Notes: 1. Catalina Install issue for Pupil Core software: I think this is an install issue related to macOS Catalina related to Apple's notarization process and our bundlingn process. When you right click Pupil Player and open, you should be prompted with a dialogue to open the app (this dialoge could be on another screen/workspace if you are using multiple workspaces). pupil_player_settings will not exist yet if you are running the app for the first time and hit with permissions issues as noted above. Regardless pupil_player_settings will be in hd/your_user_directory/pupil_player_settings @papr can follow up with you on this topic.

1. Offset correction: Click the wearer in the Home screen of Pupil Invisible Companion App. You will see a section Gaze Offset Correction and then click Adjust and follow on screen instructions. You can see the most recently pinned message written by @marc in this channel for more info re gaze offset correction.

@wrp Thank you very much! I'll try~

Hi This is what I received. Nothing happened after I click "open", even after I try to open it in folder.

@user-253e26 Yes, this dialogue is expected. Please click Open again.

The open button does not show if you simply double click the application. This is why we need the work around with right-clicking the application.

Hi I right-clicked. It doesn't work.

@user-253e26 ok, thank you for letting us know. This is indeed unexpected. Could you please share the Home directory -> pupil_player_settings -> player.log file with us? This should help us to narrow down the issue.

Thank you very much for the screen recordings. They are very helpful. 👍

No problem!

@user-253e26 It looks like the bundle is missing a dependency. I have one last question: Which version are you using?

@user-253e26 great, thank you. I am working on a bundle update right now.

@papr Thanks! I'll be waiting for your good news.

@user-253e26 This new bundle version should fix the issue: https://github.com/pupil-labs/pupil/releases/download/v1.23/pupil_v1.23-5-gbd3c72a8_macos_x64_fix_0.zip

Please be aware, that you will still get the warning regarding Apple not being able to verify the application. Just use the right-click method as before.

@papr It works now!!! Thanks!

Hi, can you please provide details of the IMU in the headset?

@user-df9629 please email info@ pupil-labs and we can follow up with this request via email

@wrp Sure. Thank you!

Hi all, I have a couple questions for you guys regarding the invisible. I'm currently piloting an experiment with longer recordings > 1hr with parents and their kids, with the parents wearing an invisible and the kids wearing a modified core, and I've been running into some problems with the invisible. First, the phone connected to the invisible really burns through the battery in comparison to that connected to the core, and I'm wondering if you guys have any thoughts on that. Second, when battery gets low, sometimes the invisible stops recording, and while there are save files, pupil player throws an error when I try to input them, again, any ideas how to deal with this? Finally, if the battery dies before I stop the recording, there is no recorded data, and again, any ideas/solutions regarding this?

Hi, I have trouble using markers to define AOI. Is there any way that the heat map can be generated without surface defined?

If I put the markers on the screen, they couldn't be detected. I haven't try to print them out. If I need to stick them on the screen, what the optimal size?

Hi @user-253e26 👋 - please ensure that the markers have enough of a white border around them in order to be detected. Please also note that markers are not detected in real-time in Pupil Invisible. You can detect them post-hoc in Pupil Player.

Hi @user-7192ac 👋 - Thanks for the feedback. Some follow up questions: 1. Are you uploading recordings to Pupil Cloud during recording? Have you tried turning off WiFi during recording or ensuring no recordings are being uploaded while recording? 2. Did you turn the screen off (e.g. press the power button to lock the phone and turn screen off)? If you remain in preview mode or with screen on the battery will certainly be consumed faster. 3. There should be a graceful exit if the battery is running low/dies. @user-0f7b55 or @user-8f5305 can you please confirm this behavior and/or follow up on this question?

@wrp Hi Thanks! What's the optimal size of the markers for a 13inch screen to be detected?

@user-253e26 depends on how far you/participants are away from the screen. I would recommend experimenting with the sizes of the markers.

Thanks! Will do! It seems that 2*2 centimetre works!

But I'm a bit confused with those parameters.

@user-253e26 I would like to migrate this discussion to 👁 core as it is related to Pupil Player (core software)

could you repost there and I will respond.

Hi @user-7192ac Each recording requires a file called info.json, if a recording doesn’t have this file, it will not be uploaded to cloud or cannot be imported in pupil player. This file is written when the “save”-button is pushed after a recording has stopped. So when the phone turns off during a recording this file will be missing. To avoid this we implemented automatic stopping when the batterylevel is 5% of lower, this however will only stop the recording, you still have the possibility to edit the template data, and you need to hit the “save”-button to have a valid recording. As a warning for low battery level (less or equal 6%) you can see a red blinking LED light with your right eye when wearing the glasses, and the phone vibrates shortly when the recording was automatically stopped.

Hi @wrp - 1. I've been keeping everything local, but I just double checked and saw that the wifi was on (though not connected), so I'll try a run today with that off.

In regards to 2. I've been pretty careful about that, but I'll make sure to be even more so.

and in regards to [email removed] can you clarify a bit about how to edit the template data? Do you mean that after the recording has stopped but the phone is still on I need to hit the save button, or are you saying I can edit the files after the fact, for example once I put them on my computer?

@user-7192ac @user-8f5305 was referring to the possibility to fill out form templates in the app. If I am not mistaken, the default template simply saves the date as the recording name. Nonetheless, you will have to confirm on the phone that you want to save or discard the recording.

Thanks @papr, so just to clarify, when the phone battery drops below %5 percent the recording stops, then the user needs to press the save button, otherwise if the phone battery dies completely the recording will not have been saved and the data unrecoverable?

@user-7192ac

when the phone battery drops below %5 percent the recording stops, then the user needs to press the save button This is correct.

otherwise if the phone battery dies completely the recording will not have been saved and the data unrecoverable I am not 100% about the details here, but as far as I know, the video and gaze data will be there, but without the necessary meta information (info.json file) about the recording in order to upload it to Pupil Cloud or open it in Pupil Player. The Companion app will also treat the recording as discarded, if I am not mistaken.

Therefore, we recommend to clearly instruct your subjects to save the recording as soon as they see the led to start blinking red. From personal experience, it is very hard to miss it once it starts blinking.

@papr Thanks for the clarification. I'm personally piloting this with my son so I'll give it a go today, but going forward is there any chance you guys could consider adding an option to just auto-save when the battery hits %5 rather than relying on the user to catch it? My data collection is going to be with parents and their children in naturalistic environments, which any parent knows can be quite hectic and so even a very obvious blinking light could be missed...

@user-7192ac I hear you 🙂 I will forward your request to our android development team 🙂

@papr fantastic. I generally love the invisible, with this being my one and only issue atm

@user-df9629 please email info@ pupil-labs and we can follow up with this request via email @wrp , Hi. I am following up on this. I did send an email but didn't hear back.

Hey ! Hey I'm an assitant researcher on Human Factors. I'm looking for a team that's worked with saccades before.In order to validate my calculations which are in pixels per second. We can't convert to degrees because we're on a random placement of the subject. r We can't convert to degrees because we are on a random placement of the subject in relation to the visual area of interest. And it seems to me that the calculation of degree takes into account this distance (we did not control it).

Hi everyone! I recently purchased the Pupil Invisible and I love it! Still, I'm struggling a little with extracting some of the raw data: is it possible to export the gaze position of the two eyes separately? and is it possible to export the pupil diameters? so far I could only find the normalized gaze position (which I'm assuming is binocular)

Hi @user-a350f5! I am glad to hear you are happy with your Pupil Invisible! It is not possible to get gaze positions for the eyes separately. The gaze estimation algorithms takes both eye images as input and only produces a binocular result. Calculating the pupil diameters is not possible. This is something that we are working on to maybe add this in the future, but given the camera setup of Pupil Invisible this is very difficult and not yet possible.

Hi @marc! thanks for your quick reply. I understand that right now is not possible, but in theory would it be possible? (with modifications to the hardware if necessary). I contacted Moritz for a possible partnership between Pupil-Labs and my company and I would be glad to explore this possibilities

@user-a350f5 I don't think it is impossible, but it is work in progress. Moritz is the right person to contact regarding partnerships, he'll get back to you!

Hi all, first is there any new information about releasing the white paper for the pupil invisible? For the Pupil Invisible companion app, does it use the same process for synchronization as the Pupil Mobile App? We are trying to build an android app that will allow for precise clock synchronization of other wearables with the Pupil Invisible world and eye cam. We've messaged with @papr some on the Pupil core thread and he said that it would be possible to sync with the clock used for Pupil Mobile. Any thoughts on solving this issue would be much appreciated. Thanks!

@user-c6717a Pupil Invisible Companion does not use the same clock mechanism as Pupil Mobile does.

Time is counted in nanosecods. We are counting from unix epoch + pupil_specific offset (currently 0). In the Android companion app this is implemented by sampling System.currentTimeMillis() * 1e6 in UTC once at the start of a recording and adding differences using a monotonic nanosecond clock.

Currently, there is no possibility to adjust the pupil_specific from the outside.

Hi all, I just get the pupil invisible, i have one question regarding to it: the way it get the gaze position is not based on pupil detection? As from the eye camera's angle which is fixed, sometimes, it cannot capture the eye part, only part of the view?

@user-8b3b5f Yes, the gaze estimation pipeline of Pupil Invisible is not based on pupil detection. It is based on an end-to-end machine learning algorithm, that was designed to deal with images like this.

Thank you for the answer @marc . So if I want to do some pupil detection (like what pupil core has), then pupil invisible is not suitable?

@user-8b3b5f Yes, Pupil Invisible does currently not output pupil detection results or any other pupillometry related measures.

OK. thank you. I can do the analysis myself. If the glass can capture a better view of the eye video. So it is possible to get eye video like this?

@user-8b3b5f With Pupil Invisible it is not possible. But Pupil Core was designed for this use-case. It delivers high quality images of the pupil and pupil detection as well as pupil diameter estimation is already available in accompanying software.

oh, I see. thank you @marc . Yes, we bought pupil core as well. If so, what I dnt understand is, then what's the function of the two eye camera of the invisible glass?

@user-8b3b5f Although Pupil Invisible does not do pupil detection, it still uses the eye images to do gaze estimation using the machine learning pipeline. To obtain the eye images that are used as input to this pipeline the eye cameras are necessary.

ok, thank you for the patient answer @marc

Sure your welcome! 👍

Hi all,

I made an export of the gaze positions of one of my subjects, a part of the measurement is attached above:

In the second column, there is the 'world index', I assume this corresponds with the frame numbers, is that correct? My main question is: Why are some of the 'world index' numbers double (or even triple) where other values are there just once?

I hope someone can help me! Thanks in advance!

@user-aed714 This is due to gaze estimation having a higher frame rate than the scene camera. Pupil Player tries to match gaze positions to scene video frames by time (indicated by the world index). Sometimes, multiple gaze positions are matched with a single scene frame. This is normal.

Hello Could you answer the following? I look forward to working with you.

About Pupil Invisible: I can use Invisible Conpanion App and Pupil Cloud. How do you get a CSV data of gaze data or blink data?　Will you be using the Pupil Player? Although I can use Pupil Player with Pupil Core, I can't use Pupil Player with Pupil Invisible.

About Invisible Monitor App: https://docs.pupil-labs.com/invisible/user-guide/invisible-monitor.html I can't find out how to use this. Is there any procedure document ?

@user-5c56d0 Hello 👋

You can use Pupil Player today with Pupil Invisible recordings to export raw data, generalize visualizations, and classify fixations for example. You will not be able to classify blinks today for Pupil Invisible recordings.

In the near future we plan to offer raw data export direct from Pupil Cloud (and more+) for Pupil Invisible recordings.

Pupil Invisible Monitor App is used for real-time gaze and scene video Pupil Invisible --> laptop/desktop computer over WiFi. This tool is often used by operators/observers during real-time studies/experiments.

@user-5c56d0 If you have problems opening Pupil Invisible recordings in Pupil Player, please make sure you are using the most recent version of Pupil Player!

@marc @wrp Thank you for your polite reply. I'm having trouble with the following, can you give me some advice? I downloaded the Pupil Player from below. https://docs.pupil-labs.com/core/#_1-put-on-pupil-core

What we did and the results are shown in the following figure.

@papr please see above ☝️

@user-5c56d0 This is very unusual. Pupil Player does not install any drivers. My guess is that the driver installation caused by portaudio trying to playback the recorded audio. portaudio is a cross-plattform library for audio playback and recording. Could you rename the audio.mp4 file to _audio.mp4 and try to open the recording again?

Hi!

I'm hoping someone can help me with this. I've been using the Pupil Invisible with a companion app on a phone. I've have made some recordings and they are listed in the Recordings on the app. The settings tell me that the cloud upload is going automatically, but the recordings are standing still at 0%, so they aren't uploading in the cloud. How can I solve this?

I've also got problems with the wearers. If I'm in the wearers to add some weares the screens only shows me 'syncing wearers'.

@user-27e529 This sounds like a temporary issue with the connection. Can you reach https://api.cloud.pupil-labs.com/ on your phone's browser?

@papr Yes I can reach that. But now I've got another problem. Because I thought the problem was caused due a connection problem, I've restarted the phone. And immediatly he updated the software. Now I cant't use the companion app anymore. Because this version of android is not compatible with the app. What can I do now? I can't even check if the upload is working 😦

Hi I just got Invisible and try to launch monitor app. It seems connected (cmd says say Hello to ......), but I dont see anything from the Monitor App. Could you help on this maater?

@user-27e529 You can find the Android rollback instructions further up https://discordapp.com/channels/285728493612957698/633564003846717444/654683156972175360

@user-3a56c2 could you share the pi_monitor_settings -> pi_monitor.log file with us?

You can find the folder in your home directory

Which home directory? Desktop side or companion device side

@user-3a56c2 Desktop side. This is likely an issue caused by your desktop network settings.

Ok let me check

I am using it from Windows do you know where should I look up to find above?

Try searching for it in the windows search

@user-3a56c2 It looks like it is using your ethernet interface for the network discovery. Please disable the interface in your network settings and try again.

ok I will try that thanks

Hi i have quick question do you have technical spec on gaze accuracy for Invisible?

@user-3a56c2 we are working on a thorough evaluation of gaze accuracy, but it's still a work in progress. We do not yet have precise numbers we can quote.

Got it! Thanks for the info.

Hi all, any updates on the Pupil Invisible white paper? Thanks!

Just tried to do a 20 min recording, but when I go to the recordings folder to see the recording it says "No Scene Video". I know the scene camera was connected the whole time because I would occasionally check the "Preview" to ensure it was still tracking and recording. Also the Scene camera circle seemed to be filled the entire recording. When I try to open this recording in Pupil Player it says "This recording cannot be opened in Player." Any information on why this might have happened? Thanks!