Hi @user-ea78b7 Please download and run the vc_redist.x64.exe file from this official Microsoft support page. Afterwards, Pupil should start up as expected: https://support.microsoft.com/en-ca/help/2977003/the-latest-supported-visual-c-downloads

Hi @user-c5fb8b , worked like a charm! Thanks a lot.

Glad I could help!

Pupil invisible monitor can’t go full screen on Mac OS. We would like full extend for monitoring purposes. any reason the full screen mode is disabled ?

@user-367c1f thanks for the feedback. @papr or @user-c5fb8b could you look at this early this week and/or make an issue.

@user-367c1f I have just successfully tested full screen mode on macOS without a connected device. You can enable it by clicking the green button in the top left corner of the Window. I was not able to test a connected device yet. I assume that the image will be distorted in full screen mode. If this is the case, we will work on a fix this week.

@papr Thanks for the quick feedback. Any source code or release update about that?

@user-367c1f Do you mean in regards to the fix?

Yes

@user-367c1f If a fix is necessary we will create a release for it. First I need to verify if the image is indeed distorted in fullscreen mode. We will update the channel if there is a new release.

@papr Ok looking for it

After being unable to connect the Monitor App to the Companion device through eduroam system at university we tried to connect at home using our home internet network. We tried the monitor app at home and we have even switched off the Kaspersky Internet Security in case it was blocking communication between the Monitor App and the Companion device and there is still no connection as shown below. We have tried everything suggested so far and are still not able to connect the Monitor App to the Companion device.

Is it possible to record scene video with audio? I know pupil invisible has a microphone, but the video which I download form pupil cloud doesn't contain an audio.

@user-bea039 we will release updates to both Pupil Invisible Companion App soon that will enable audio recording. You will be able to hear audio in recordings in Pupil Cloud, once we enable recording in Invisible Companion App. This is coming very soon.

Hi @user-057596 we received your email as well. We will schedule a time to meet with you via voice/video to try and help debug this issue.

@wrp ok, thanks! Is the app released until this year?

Thanks wrp. We are free on Friday the 6th of December from 1pm (13:00 B.S.T , 2pm 14:00 C.E.T) . Regards Gary

@user-bea039 Yes, we plan on releasing it before the holidays!

We have updated the Pupil Invisible Monitor v1.2 release on Windows to include network debug log messages. Users who experience connection issues between Pupil Invisible Monitor and the Companion application are asked to use this updated version and share the pi_monitor.log files in the pi_monitor_settings folder.

These files will help us to identify issues regarding device discovery.

The updated windows release can be downloaded here: https://github.com/pupil-labs/pupil-invisible-monitor/releases/download/1.2/pupil_invisible_monitor_windows_x64_1.2-24-g237215f.zip

/cc @user-057596

@marc thank you! I can't wait!

Thanks papr. I’ve downloaded the update to our computer at university using the eduroam network and have collected pi_monitor.log files for this scenario and will do the same for the laptop at home and our home network when we get home later in the day and then send the files across for analysis. Best regards Gary

@user-057596 Will they be collected on the same laptop? If you have some of the files available, I could have a look already if you want me to.

Hi papr. We have different laptops for collecting data. One is the Medical Education Lab at Heriot-Watt University at the moment and that’s the one that’s one that is connected to the eduroam network. I’ve just collected the pi_monitor.log files from that one and could send them to data@pupil-labs.com email address supplied by Niall? The other laptop is at our home as we work between the university and there and we use both in medical testing and training. Regards Gary

@user-057596 Yes, the address is correct. Thank you for describing the setup in more detail.

No problem papr. I will send the first files from the university laptop connected to the eduroam right now. Regards Gary

Just sent papr. Regards Gary

@user-057596 I have received your email and had a look at the logs. In both cases, the configuration below was used:

{
    "Kaspersky Security Data Escort Adapter": {
        2: {
            "addr": "10.102.37.150",
            "netmask": "255.255.255.252",
            "broadcast": "10.102.37.151",
            "network": "10.102.37.148",
        }
}

This interface seems to be related to this: https://support.kaspersky.com/12665#block4

My suspicion is that this security tool is blocking the relevant communication between Pupil Invisible Monitor and Companion app.

Hi to you all, we have been trying out the Pupil Invisible on ourselves the past week and have been getting consistently inaccurate results - I've tried different lightings, different people (all with perfect eyesight) and also offline pupil detection and gaze mapping, all to no avail. Using the gaze offset in the Companion-app would fix the issue for one point in the scene (we are talking sitting at a desk, reading a piece of paper), but looking anywhere else would become inaccurate again. I know that eyetracking is not 100% accurate, but, as you can see in the attached screenshot, we are not talking normal inaccuracies but far from usable. What could we be doing wrong, as I seem to be the only one having these issues?

So papr are you saying I should enable the secure connection in Kaspersky to enable communications between the Monitor App and the Companion device?

This would be another person.

Hi @user-ea78b7 ! Could you share a representative recording with data@pupil-labs.com so we can take a look?

@user-057596 I do not know how this software works and what it does. 😕 A solution might be to disable/remove it.

Hi @marc, let me see if I find anything without confidential materials in it. If not I will prepare something.

Thanks @user-ea78b7 !

@user-057596 There was an other user who experienced a similar issue with his/her VPN software.

Do you recommend a internet security software to work with the Invisible glasses papr?

@user-057596 I would recommend an isolated and dedicated setup: - dedicated wifi router without internet - computer without security software

This would give you full control over the network while being secure.

@marc Found something! I will send it via Firefox Send to this e-mail address and provide a little detail as to what I was looking at etc.

@user-057596 usually security mechanisms are based on restricting software/network connections. We need the opposite. 😕

@user-ea78b7 Great, thank you!

We were thinking about that papr but I think this has made that far more definite. Thanks for your help it has been most appreciated. Best regards Gary

@user-057596 ok 👍 After sending the home-network log files I will be able to tell if the connection suffers from the same issue at home.

I will do that we get home and once again thanks for your help.

@user-057596 Sure thing, you are welcome!

@user-367c1f Good news! We are on track for our Pupil Invisible Monitor v1.3 release which will feature correct visualization during full screen mode. We are planning on releasing it on Monday!

Hi papr, that is the files for the home internet system just sent with file titled 1Home with the internet security software on and the second file 2Home with the software disabled. Regards Gary

@user-057596 Again, the Kaspersky Security Data Escort Adapter was chosen by the network framework. 😕

Thanks papr. I’m going to take Kaspersky off the laptop and try again. We want a system that basically plug in and play for medical training and feedback so your suggestion of a isolated and dedicated system with a dedicated router not connected to the internet and no internet security would seem the best solution. Thanks again Gary

Hi to all, me again...I am experiencing problems with the Pupil Invisible Monitor. Both the Companion and Monitor are connected to the same, dedicated Wi-Fi-network. I see in the log that the devices find each other and there is a button saying "In" on the grey area that is shown upon start-up of the Invisible Monitor. However, nothing happens when I click it and I cannot display scene or gaze visualizations. Log attached.

Nevermind, I pressed some random buttons and now it's working

Looks like I had to press the "I"-key to make the transmission start.

Hi @user-ea78b7 where you by any change running Pupil Invisible Monitor in fullscreen mode when you tried pressing the button?

@user-c5fb8b no, I had it windowed. The button remains unpressable, but as long as I can start the livestream with the I-key everything's fine

📣 Announcement 📣 We are pleased to announce that we have released Pupil Invisible Companion App v0.8.0. We recommend upating Pupil Invisible Companion app via Playstore if you do not have auto-update enabled.

This update includes a big new feature that many people have been waiting for - audio recording! You will now be able to record audio using the built in mic on your Pupil Invisible Glasses. By default audio recording is disabled. You can enable this feature from the settings menu in app.

Audio playback for all new recordings will also be available in Pupil Cloud!

Note: We noticed that some scene camera modules have audio quality issues. If you are experiencing any issues, please send an email to info@pupil-labs.com along with the serial number of your Pupil Invisible glasses so we can help you resolve the problem.

@user-ea78b7 I had a look at your recording and want to give some feedback! Generally, the error Pupil Invisible makes can be split into 3 categories: 1) physiological parameters, 2) parallax error and 3) errors from the machine learning algorithm.

1) Physiological Parameters: There are a few physiological parameters, like for example the offset between the optical and visual axis of the eye, that influence a persons gaze, but can not be measured from the eye image alone. Pupil Invisible will always only approximate those values with the population average. Usually this is not a problem, but for those subjects that are outliers of this distribution, i.e. if the average is a bad approximation of their parameters, there can be a significant error. This error will however be a constant offset, which can be compensated using the offset calibration. I looked at your sample recording, and there is a small offset error visible in the raw predictions (the prediction is always a bit too high). Thus the offset correction should help a little bit, but it will not make the results perfect.

2) Parallax Error: Since Pupil Invisibles scene camera is located at the side of the device and the (implicit) depth estimation of the machine learning algorithm is not perfect, there is a parallax error at short distances. Past a distance of 1 meter this error is negligible, but for distances smaller than that it becomes increasingly noticeable. In your sample recording you look at print-out of some graphic you hold relatively close infront of your face. I believe that the increased error in this section is due to parallax error.

3) Machine Learning Algorithm: The machine learning algorithm is the reason why Pupil Invisible is robust against slippage of the headset and outdoor lighting conditions and why it does not need a calibration. But at the same time the algorithm is not perfect and its peak accuracy is a bit lower than that of traditional eye trackers in a lab environment. The errors we get from the algorithm are not trivial to compensate via a calibration.

@marc Thanks for the feedback, so especially the distance to the objects of interest are what make the recordings inaccurate. That's good to know. Is there enough data to work around the algorithm and solve this issue?

@user-ea78b7 What data do you refer to in your question? Do you mean if there is a way to remove the errors coming from the machine learning algorithm?

@marc Exactly - would I be able to show the reference targets at the beginning of a demonstration and then conduct offline pupil detection and reference point detection to re-do the gaze mapping?

@user-ea78b7 The gaze estimation pipeline of Pupil Invisible does not utilize pupil detection and gaze mapping in the way Pupil Core does, it works very differently. Due to the camera angles of Pupil Invisible, pupil detection would not work very well and running the Pupil Core pipeline on Pupil Invisible recordings will not give you satisfying results. We are actively working on further improving the performance of both Pupil Core and Pupil Invisible. If your application requires higher accuracy than what Pupil Invisible can currently give you, I recommend to use Pupil Core. This will require a lab environement, but the peak accuracy is higher.

@marc Thanks marc, we'll look into that

@user-ea78b7 You're welcome!

📣 Announcement 📣 We are pleased to announce that we have released Pupil Invisible Monitor v1.3. This update will allow you to use Pupil Invisible Monitor in fullscreen mode without distorting the video stream.

You can download it here: https://github.com/pupil-labs/pupil-invisible-monitor/releases/latest

/cc @user-367c1f @user-ea78b7 This update should fix the button interaction issue that you described above.

🤩

@papr thank you very much

@wrp Great update of companion app! I download it and record a video with an audio. The audio includes some noise but I cannot determine the noise is much terrible than other scene camera module. So is it possible to get an audio from the module which is good condition? I want to compare these audios.

I found a small dust inside of scene camera. Is there any warranty to get new one?

This is a video which I recorded last week, the dust is not shown in a video of scene camera 😦

@papr Updated the Invisible Monitor, now I am not seeing any buttons. Pressing the "I"-Key (or "O"-key with another smartphone) still works to get the stream to show, though.

@user-ea78b7 Could you try resizing the application to check if the buttons reappear?

@user-ea78b7 Also, could you share the pi_monitor.log file in the User folder -> pi_monitor_settings directory? It might help us to find the issue which causes the buttons to be hidden.

@user-bea039 Could you please send a sample recording to [email removed] that includes the audio noise?

@papr Resizing did not work. I will share the log later.

@user-ea78b7 OK, thank you. By the way, is it possible that you have set an other screen/ui scaling than 100%?

@marc I send just now 🙂

We are pleased to announce that we have just updated the Pupil Invisible Monitor v1.3 release. This update fixes UI scaling issues on systems with high-DPI monitor screens. You can download it here: https://github.com/pupil-labs/pupil-invisible-monitor/releases/latest

@user-ea78b7 We hope that this update fixes the missing-icons-issue that you described earlier.

Maybe someone can support - on the left you can see a "successful" recording from the Invisible Companion 0.8 and on the right an "unsuccessful" recording. What happened was that we started recording with the Invisible Companion and wanted to stop after about 30 minutes. When I took the phone from the participant, the home-screen of Android was showing and the app was minimized. Upon opening, there was no option to "Save" or "Discard" and I had thought that I simply misclicked and did not record at all. However, with the next participant, the same thing happened with all conditions equal which made me investigate.

I found the folder you can see on the right on the phone for the first recording and one with similar content and structure for the subsequent recording. I was not able to reconstruct the problem with another smartphone running Companion 0.8 and did not try yet with the device with the failed recording.

Two issues: 1., maybe you could look into that. This has happened after upgrading to 0.8 and enabling Audio-recording. 2. Is there a way to salvage anything of value from the files I still have of these recordings for which I wasn't able to select "Save" in the app? The following conditions applied: Audio-recording on, Cloud-upload disabled (data protection), Recording started and phone screen switched to show a preview, then I pressed phone's power button, Live transmission via Wi-Fi to an Invisible Monitor in an adjacent room. Except for the live transmission I tried reconstructing the issue with another phone and same glasses and settings, but everything worked fine there. When activating the respective wearer on the phone where the recording failed, the app does show the date and time of when I took the phone from the participant to save the recording and there is about 3GB of data per 30 minutes of recording on the phone but it does not show in the app.

Any help would be much appreciated.

@user-ea78b7 Did the participant confirm that they stopped the actual recording?

@user-0f7b55 The participant did not handle the phone at all, I did. I described my approach in my original post. This approach had worked five times before that on two different phones and then failed twice on the same phone after upgrading Companion to 0.8. As described, I did not even get the opportunity to stop the recording - upon unlocking the smartphone, the Android homescreen was showing and upon opening Companion again, I did not see the usual "Discard" and "Save" buttons. Instead it looked as though the recording hadn't even been started. Of course, OTG was enabled and the Companion app was locked, too. I think the problem was that the Companion app was minimized.

@user-ea78b7 was the app locked in the app switcher?

@marc yes, it was.

Oh yeah, you just wrote that!

@marc I cannot reconstruct why Android was showing the homescreen - it shouldn't have done this, because the app was locked. Maybe the problem was that I started the recording, switched to the preview to see that everything was working and then pressed the power button to switch off the screen? This did not pose any problem on previous recordings though... 😦

That's why I thought that maybe you should investigate anything that might've changed with the switch to 0.8, maybe due to enabling audio recording. Not having the app in the front is probably the cause of me not being able to save the recording.

@user-ea78b7 Just one thing to clarify, in a normal procedure, if you start a recording, then 'minimize' the app (tap home, etc ), upon return to the app , you are not expected to see the "Save/Discard" dialog, but the main recording screen with the recording still ongoing.

@user-0f7b55 True, that was what happened before - then I used the back-button of Android to go back from the main recording screen and could tap "Save". However, with the last two sessions I was brought to the main screen of the app with no recording running.

@user-ea78b7 You have not done any android software updates on the device, correct?

@user-0f7b55 My Android-version on the device is OxygenOS 9.0.9

so it seems that it has been updated

Could this be the reason for the problem? I did not do an Android update during the research sessions so it was working well with the same OS version before. So maybe 0.8 of Companion with 9.0.9 of Oxygen is not a good fit?

@user-ea78b7 We will try to reproduce this using those versions!

@marc Thanks, helpful as always. In the meantime, is there anything I can do to save the recordings from the files I was able to extract?

@user-ea78b7 That is a bit difficult to say not knowing what happend and not having the data available. What happens if you try to open this in Pupil Player?

"invalid recording, there is not file in the directory"

I see, that is probably because the info.json has not been generated. Are you on Linux? If so could you try to play the video files using ffmpeg: ffmpeg /path/to/video

Looking at another info.json, I could be able to create that from scratch for the failed recordings, if that is the only thing holding the Player-software back

I agree, you could "fake" them. The video files usually break beyond repair the easiest, so that would be my largest concern.

@marc This could prove viable. I was able to load up the recording after faking an info.json. I do see gaze replay and eye overlay but no world-video. It throws: "Error: moov atom not found". Does this refer to the starting time declared in the info.json? This was the only value I did not update from another info.json

@user-ea78b7 Please note that the info.json is originally created by Pupil Invisible. Upon opening an invisible recording in Pupil Player, the recording will be transformed to have the same layout and files as a Pupil Capture recording. Since we do this non-destructively, we don't change the original files (e.g. info.json) but instead create additional files for the same information converted to the appropriate formats for Pupil Player. Thus you will see a file info.player.json appear after opening the recording once successfully in Pupil Player. After this any change to info.json won't have any effect on Pupil Player anymore, since it will load the meta information from info.player.json.

@user-c5fb8b Thanks for the info. Could I be able to fix that error by adjusting info.player.json? EDIT: Trying again with time since the epoch in nanoseconds adjusted accordingly

@user-ea78b7 Pupil Player will only load video data from between the recordings start time and start time + duration (both values will be loaded from the info.player.json). If the timestamps of the world recording are outside of the [start_time, start_time+duration] interval, the world video will be shown gray as there are no frames to display in this time frame. What you will have to do is to set the start-time and duration to approprate values that reflect your actual recording timestamps. If you send me the PI world v1 ps1.time file I can help you to extract the relevant time informations.

@user-c5fb8b sounds great, I will take my lunchbreak now and then try again - I am optimistic 🙂 Maybe I'll have to adjust a bit to get the first seconds because I only now hour and minute the recording started but this looks promising

@user-ea78b7 Ok in principle you can also set any time slice you want into the info-file. Any frame where there's no data available from the world video will just be displayed gray. Make sure to not use a too-large time period though, as this might eat up your RAM when we generate internal lookup tables for the entire recording.

@user-c5fb8b I would like to get back to your offer of extracting relevant time informations - I wasn't able to get a world-video showing (the PI world v1 ps1.mp4 does not show anything either, though). Where can I send the files?

@user-ea78b7 Could you try playing the world video with VLC player? The error message you quote could also be due to a broken video file. If it is not broken VLC player should be able to play it.

@marc Okay so it is probably broken, VLC is not playing it.

@user-ea78b7 The last hail mary thing you could try is running ffmpeg -err_detect ignore_err -i world.mp4 world_fixed.mp4 to salvage none-broken sections of the video if possible. If this does not work either I am afraid your world video can not be fixed. You could still use the recording without the scene video, if that is of any use to you.

@marc Any way to do this on Windows 10?

Nevermind, Google is my friend. - Okay, I'm positive now that the world.mp4 is broken beyond repair. I cannot do anything with it in ffmpeg, it will always throw "moov atom not found", which, afaik, is not fixable.

@user-ea78b7 Sorry to hear that, but I agree with that assessment.

I'll do a factory reset of the OnePlus6 and not upgrade the OS this time just to be sure for any subsequent research. Thanks for the support!

@user-ea78b7 a factory reset will not downgrade the OS version.

@user-0f7b55 I just noticed 😄 Clumsy me

What OS version is installed when you ship the phones?

Oxygen 5.1.11

And is there some way to downgrade myself?

Thanks.

Hi, is it possible to stream the audio using pupil mobile?

@user-d20f83 streaming audio is currently not possible.

📣 Announcement for Pupil Invisible Users 📣

Please do not upgrade Android/Oxygen OS version.

Explanation: Android version 10 has a framework issue that prevents the use of Pupil Invisible hardware. Google is aware of this problem and is working towards a fix.

You will see prompts/notifications asking you to Update now please do not accept/install Oxygen/Android OS software updates on your Pupil Invisible Companion Devices. We will let you know when it is recommended to perform an update.

Hello ! My lab bought a pair of Pupil Invisible. We are astonished by the "no-calibration" feature, ours researchs are in applied settings so the less dirturbed the operator is, the better. Do you have a technical paper explaining the machine learning pipeline that you developed ? I am curious, it's a real improvement from the 9-dot grid.

@user-487d4f thanks for the feedback - much appreciated! No, we have not published any papers on the machine learning pipeline, but will be happy to share when we do release more info about the pipeline 😸

Hi we have finally connected Pupil Monitor to the Pupil Companion after help from Kaspersky to take off all of their internet security software. The next question is how do you transfer the recorded file to your computer from either the phone or the cloud. We have developed a software analysis package that streamed from the Pupil Capture software and using the Pupil Core glasses and we know want to stream the recordings from the Pupil Invisible and companion device to our software on our laptop.