Hello, I'd like to know if the Pupil Invisible comes with a software to analyze the data ?

Hi @user-969912 👋 - Currently you can use Pupil Player (desktop software) to perform visualization and "light-weight" analysis for Pupil Invisible recordings. We are actively working on adding enrichment functionality to Pupil Cloud, and hope to be able to share an update with the public by end of this quarter/early Q2.

@wrp Thanks for answering. Do you have any software recommendations for market research then ?

@user-969912 it really depends on what you are looking to achieve. There is lots that can be done using Pupil Player already 😺 However, if you are looking for turn key solution today, you might want to consider using Pupil Invisible for data capture and then iMotions for post-hoc analysis. Again, we are working on adding enrichment functionality to Pupil Cloud - so this might also be a viable option for you in the near future.

Announcement: One of our service providers for Pupil Cloud (Digital Ocean) is experiencing latency issues and some service interruptions. As a result, you may have experienced slow or incomplete uploads from Pupil Invisible Companion to Pupil Cloud. Details from the service provider (Digital Ocean) documented here: https://status.digitalocean.com/incidents/qq6wf8ny1c1n

We hope that this will be resolved very soon. And apologize for any inconveniences this has caused you in using Pupil Cloud.

According to the link above, this issue has been resolved. ✅

@wrp @papr a quick question, because I did not find detailed infos on that: I found in the git log that you somhow can get the pupil invisible to stream to the capture software. However, I did not find how to do that. Is that possible with the pupil invisible software on the 1+ or do I need another app? How exactly would I connect it? I have both on the same WiFi, but I do not see an option how to stream from invisible from capture?

ok, I just needed to take more attention to the WiFi config I had (I was by accident in a VPN with all network tunneled on the Laptop, äehm, sorry xD). However: No it connects, but it crashes often. Also I can not switch on the eye cameras. It works stable with the pupil standalone app. Now all I want to have is actually a pupil diameter for now if that is possible. If I take the code of the standalone app, in this gaze info, is there also the pupil diameter for the invisible, or does the invisible not calculate that info like the core?

Hello! I cannot open the recording.zip file downloaded from pupil cloud. It says the compressed folder is invalid. Is it part of the (current) issue with the Cloud service or something else?

Hey @user-bab6ad! Pupil Capture was not designed to be compatible with Pupil Invisible. You can make it work to some degree (as you seem to have done), but it is not recommended. You can use the Pupil Invisible Monitor App to receive data generated by Pupil Invisible on a desktop computer. However there is no pupil dilation signal provided. We are working on adding pupil dilation, but this will still take a while and we do not have a release date yet. The pupil dilation signal you might get through Pupil Capture will not be very reliable. Due to the positioning of the eye cameras in Pupil Invisible, we get very challenging eye images. The pupil detection algorithms in Pupil Capture do not work robustly on those. If you are interested in a pupil dilation signal we would curently recommend using Pupil Core!

@marc thanks a lot for the info, I already thought so. We have an old pupil core, but one of the side cameras broke, so the invisible was a thought of us if we can quickly use it as a replacement. Anyway we are currently order a replacement so that is handled

Hi, can I change the eye tracking sampling frequency?

If not, what is the sampling frequency?

Hi! I downloaded Pupil player from github and am now trying to load a PI recording that I just downloaded from pupil cloud. How do I drop a recording directory to the pupil player window? I tried to drag and drop the downloaded folder but it does not work. On https://docs.pupil-labs.com/core/software/pupil-player/#load-a-recording it says it needs to be a triple digit folder? I did not find one when I downloaded the recording from cloud.

@user-16e6e3 please unzip the downloaded folder and drag the recording folder into Player. The naming scheme is different for recordings downloaded from Cloud vs the example in docs

thanks! That worked

Hi @user-0eb381! The eye cameras record video at 200 Hz. The real-time gaze estimation on the phone works at 55 Hz. We will release a feature on Pupil Cloud in about 1 month, that will allow you to obtain gaze data at the full 200 Hz post-hoc. There are no options to configure the framerates on the phone, they will always be as high as possible.

thanks! @marc

Hi, We're now thinking to use the lab streaming layer to synchronize data from Pupil Invisible and other data sources. I found a github page (https://github.com/labstreaminglayer/App-PupilLabs/tree/master/pupil_invisible_lsl_relay) about the LSL plugin for PI, but unfortunately as I'm new to LSL, it's not very clear to me how to use it. I wonder if there's any introduction on how to use the plugin. Thanks in advance, Toru

@user-a9ff0d LSL works by collecting/recording data in a central place (https://github.com/labstreaminglayer/App-LabRecorder) from multiple sources (LSL applications). The Pupil Invisible LSL Relay is such a LSL application and is meant to be installed/used via the terminal/command prompt. Depending on your operating system, you might need to install Python first (https://www.python.org/downloads/).

Hi! Im Madan recently we got hololens 2 to do research on eye tracking but unfortunately we couldn't access the IR camera due to some security concerns. Hopefully landed in Pupils lab website and got to know about binocular invisible product. Is there any way to attach them in Hololens2?

Hi @user-7aee47! We have not yet had the opportunity to test the compatibility of the Hololens 2 with Pupil Invisible, so we can't give you a definitive answer. AFAIK the Hololens 2 is supposed to be compatible with all regular glasses. Since Pupil Invisible has the form factor of a regular pair of glasses, I think there is a good chance that they are compatible. The scene camera view is probably obstructed to some degree, but the gaze estimation data would be available. You could test the setup yourself within our 30 day trial period after purchase. If it does not work you can send the hardware back and get a refund.

Hi @papr . I have python (version 3.6.8) installed on my local windows pc and followed the installation instruction but I got an error message, saying "fatal: not a git repository (or any of the parent directories): .git"

@user-a9ff0d you might be missing the cd App-PupilLabs/ step. I have just noticed that the git checkout ... step is out of date and not necessary anymore. I have removed it from the readme file.

Hi, separately from the LSL issue, I also have a problem in the gaze offline calibration. I included the calibration process using the Pupil Calibration Marker v0.4 and tried the gaze from offline calibration. Even though it seems that the calib marker is accurately detected, I got the following error:

Create calibration Default Calibration - [INFO] calibration_routines.data_processing: Collected 0 monocular calibration data. Create calibration Default Calibration - [INFO] calibration_routines.data_processing: Collected 0 binocular calibration data. Create calibration Default Calibration - [ERROR] calibration_routines.finish_calibration: Not enough ref points or pupil data available for calibration. Create calibration Default Calibration - [ERROR] gaze_producer.worker.create_calibration: Calibration failed: Not enough ref points or pupil data available for calibration. player - [ERROR] gaze_producer.controller.gaze_mapper_controller: You first need to calculate calibration 'Default Calibration' before calculating the mapper 'Default Gaze Mapper'

Any help?

@user-a9ff0d Please be aware that you do not need to perform Offline Calibration for Pupil Invisible recordings. Just use the "Gaze from recording" option.

@papr I'm aware of it, but the gaze from recording doesn't look alright. During the calib marker is presented, the detected gaze is always positioned a bit off from the marker.

Looks like this

@user-a9ff0d Please see the most recent "pinned message" by @marc in this regard.

@papr I saw the post, and I used the gaze adjust function before recording this video. By the way, when you adjust the gaze using the wearer setting, are both of corrected and uncorrected data saved? Or only corrected data?

@user-a9ff0d When you adjust gaze with Offset Correction in Pupil Invisible Companion App, gaze data file with the offset applied is saved. The original signal is not saved. Offset information is saved within info.json

Good evening! I am still getting familiar with PI. When I use the raw data exporter in PP, the pupil_positions.csv is empty (only the header is there). Do I need to enable something for it to include the raw data?

@user-16e6e3 Hi, Pupil Invisible does not yet generate pupillometry data. If you are looking for the gaze data check the gaze_positions.csv file.

thanks!

Hi all! Is there a way to have stronger compression for the Invisible recording? 10min video 1.7GB file seems excessively large.

Also, the PI is not recording audio - no microphone in there?

If necessary, could the Companion device microphone used?

@user-808722 Hey. Audio recording is an opt-in feature. You can enable it in the settings. @user-0f7b55 might be able to give further insight regarding the video compression.

@papr thanks, my bad, found the sound setting! @user-0f7b55 is there a way to have stonger/better video compression? the goal it to have smaller output files, like 10min - 200-300mb max, not 2GB.

@user-808722 we are working on that.

@user-0f7b55 thank you! also, how can I easily export the video WITH the gaze overlay, to desktop, to edit/cut the video?

you can use pupil player

@papr can help you with that further

@user-0f7b55 found it thanks. will explore the feature. I can only export the full recording, no cutting feature right?

@user-808722 You can use the trim marks to only export parts of the recording. Checkout this section of our docs. This should help you find your way around the application: https://docs.pupil-labs.com/core/software/pupil-player/#player-window

thanks a lot guys!

Hi, Currently in the measurement mobile, the folders (one folder corresponds to a single measurement) are named like this (e.g., ebf4153e-4583-4698-9b2e-1455fbbb10e4). Is there any way of setting the folder naming rules such as recording date and time?

@user-a9ff0d Again, @user-0f7b55 is the expert here, but as a work-around: The folders have a creation date. You should be able to sort the folders by this creation date within your file viewing application.

@user-a9ff0d sorry, folder naming scheme is not going to change.

@papr @user-0f7b55 I am aware that you can sort them by creation date, but we often use some statistical software such as R to read the output files, in which case these folder names cannot be used to limit the folder search with their names... but okay I understand the situation. Thanks!

@user-a9ff0d If you are accessing the recordings programmatically anyway I recommend reading the info.json file. It includes meta information, e.g. the recording date. This should help you to filter recordings efficiently.

@user-a9ff0d you could also write a script that renames the folders based on the recording date found in the info.json, e.g. prepend the date to the uuid

@papr @user-c5fb8b thanks, I see that may be the best option for now to rename the folders. My laziness just drove me to ask you if there were any ways to work it around 😛

@user-a9ff0d We are big fans of laziness. 😄 This task is definitely something that you want to automate to avoid human errors. 🙂

By the way, is it possible to record while it's recharged? We wondered if by using the usb-c-hub or some cables that can divide the line into two, it's possible to record with recharging at the same time.

we do not officially support nor recommend such a practice

@user-0f7b55 I see, we would appreciate if you come up with some solution for this issue given the amount charge the PI uses while recording someday.

Hi! I currently have trouble opening pupil player. All I see is an empty command line and an empty window that I click on, but it doesn't open. Is it my leptop or something else?

@user-16e6e3 This is a known issue on Windows. Unfortunately, we do not know the underlying cause. But there is a workaround. Please close the application, delete the Home directory -> pupil_player_settings -> user_settings_* files, and start Player again.

this worked. thank you!

another question: can you do surface tracking with PI recordings? if yes, how do you set the markers?

The Pupil Invisible Monitor works on my Windows OS but not on my Debian based Linux OS. Is this a known issue? Do I have to install certain packages for the monitor to work?

Output on Debian, it can find my device but still only shows a gray screen on the output window.

@user-acc960 Hey, this is not a known issue. Just to make sure, this is the ip addr of your wifi interface?

Actually, it looks like it selects your ethernet connection instead your wifi interface for broadcasting.

Is it possible for you to disable the ethernet connection? Unfortunately, there is currently no user option to tell Pupil Invisible Monitor which interface to use.

@papr All right thanks, I'll check it out. Oddly enough my own implementation shows the sensors:

@papr You were right indeed. Now it is showing up. Thank you for the help.

@user-acc960 Glad that I was able to help! May I ask what you are working on? 🙂

@papr Yes I am creating an application for a Raspberry PI that streams the data from the NDSI v4 protocol. The incoming data will be processed with OpenGL to show a marker on the scene video. This OpenGL context will be converted to a video of one minute using the h.264 codec. Every time a one minute video is finished, it will be uploaded automatically to our cloud.

Ah nice! Let us know if you are running into any issues using NDSI 🙂

@papr Haha, I will. 😜

Hi, good afternoon! The Pupil Core, namely the Player to play and export Invisible recordings, would work on Windows 7 or ONLY on Windows 10?

So, I want to play and export Invisible recordings, that the only task, can it be done on Windows 7?

@user-808722 We have only tested the applications on Windows 10. It might work on Windows 7, but I cannot guarantee it.

Thank you! I supposed this was the case. So I'll give it a go tomorrow.

@papr FYI, I tried to run the Player on Windows 7 and it gave me numerous errors, did not run. Didn't try any further, upgraded to Win10. 👌

@papr new question: what causes the recording to stop, to loose signal seemingly randomly / no reason, multiple times?

For what reason does this method put out -1 (error code) no matter what I do? The node has been started and joined the v4 group. For some reason it does work properly in the python variant.

Hi @user-acc960, please consider posting code related questions in the 💻 software-dev channel. Anyways, you can always lookup the official zmq API, which is what the Matlab package should implement. As you see here, a return of -1 indicates one of 4 possible errors: (scroll to the very bottom, return value) http://api.zeromq.org/master:zmq-getsockopt You should be able to use zmq_errno() and zmq_strerror() to further debug your issue: http://api.zeromq.org/3-2:zmq-errno http://api.zeromq.org/3-2:zmq-strerror

@user-808722 This happens due to physical disconnects between the phone and the glasses and/or the scene camera. Please make sure, that your USB-C cable is correctly connected to the phone and glasses. If you keep seeing this type of issue please contact info@pupil-labs.com

@papr thank you! so it's certainly a hardware issue, most likely cable connection either end?

@papr I'm not super familiar with the USB C connector, so if it's plugged in, it can still disconnect and reconnect the data transmission easily?

@papr separate question: is it possible to record and charge the Companion  device, from a battery pack for example? so like a USB C splitter?

@user-808722 Difficult to say. If you notice disconnects although both ends of the cable are well connected the disconnect might be caused by different issue. In this case, please contact our hardware support at info@pupil-labs.com

is it possible to record and charge the Companion device, from a battery pack for example? so like a USB C splitter? To cite @user-0f7b55: we do not officially support nor recommend such a practice

@papr thank you

Hi! I am working with hemianopic patients (partial blindness after brain damage) and I was wondering if it's possible to superimpose the area of a patient's visual field deficit as AOI with Pupil Invisible?

Hi @user-16e6e3! Am I assuming correctly that the portion of the visual field that can not be perceived by the patients is moving with the gaze direction? E.g. always the left half of what my eyes are pointing at is not perceived? Pupil Invisible is of course not capable of measuring what portion of the visual field is affected, but if one would know the size and position of that portion, one could potentially visualize that in the scene camera at least approximately. Pupil Invisible is estimating gaze as "cyclopean gaze", i.e. it does not estimate the gaze directions of each eye separately, but estimates a single "gaze ray" that originates in the scene camera. Visualizing a given masked out area that has a fixed relation to this cyclopean gaze direction would be possible.

@marc This sort of thing is typically measured using a perimetry test.

I know that cortical damage from stroke can lead to neglect in dedicated portions of the visual field (see homonynous hemianopia )

...but there are certainly other types of neglect. I would hit google scholar 🙂

@marc Thanks for your answer! Yes, exactly, the blind field moves with the gaze. I am doing perimetry tests beforehand to determine which part of the visual field is impaired. How would you visualize a masked area fixed in relation to gaze (and, if possible, also get raw data referring to that masked area)? Does this work in Pupil Player?

@user-16e6e3 I am not sure what kind of raw data you are looking for. As far as I understood it, the mask would be just a visualization on top of the scene video that is dependent on the current gaze point. Do you know the "Vis Light Points" visualization in Pupil Player? It masks a circular area around the gaze point. Instead, I would mask the area left of the gaze point in your case (assuming the left half of the field of view is impaired). Meaning, if the subject looks at the center of the scene image, the left half of the image would be masked.

Please excuse my naive approach here. I am very sure that masking everything left of the gaze point is an oversimplification and would not be sufficient for your usecase. Do you have an overview on which "shapes" the impairments can take? (if this term is even applicable...)

@user-8779ef Thanks for the input!

@user-16e6e3 This would probably not be something that could be done immediately with the available tools. However the data that is being generated by Pupil Invisible would allow you to implement a visualization like that yourself, or adapt one of the available visualizations in Pupil Player for your needs as @papr mentioned.

thanks for your answers! @papr In many cases the entire left half of the immage being masked would be an accurate depiction of hemianopia. The data I would like to get is the number of fixations/saccades towards that blind half, for example. I have checked out 'Vis Light Points' before. How could I adjust it to mask one half of the image? Does this and @marc 's suggestion involve dealing with the software development? I'm not familiar with it unfortunately 🙈

@user-16e6e3 A very simple approach would be to check the norm_pos_x field in gaze_positions.csv and fixations.csv and test if the value is smaller or bigger than 0.5 (horizontal center of scene camera image). This would not require any special plugins. Regarding the custom visualization, please contact info@pupil-labs.com I think we can come up with a prototype by next week.

@papr cool! I've looked at the norm_pos_x before and it's definitely a valid and simple approach. Was just wondering if a gaze-contingent analysis would also be possible. Will contact [email removed] Thank you!

@user-16e6e3 Do you have a reference on how this would/could work? Is it even possible that a subject fixates the blind area? From what I understood, the blind area moves with the eye, i.e. if the subject looks to the left, the impairment moves left, too.

Also, is this impairment just left/right or is it possible that there are top/bottom impairments, too?

@papr That's true, it moves with the eye, which is why it's important for patients to make eye movements to explore more of what's hidden in their blind field (e.g. with left-sided hemianopia, they should make more eye movements to the left so they can bring the 'hidden' left part of the environment into the normal sighted field and therefore compensate for their deficit). Patients do not have problems to fixate objects in general, because enough of the central part of their visual field is spared for them to do so.

There are also impairments of top or bottom quarter of the visual field called quadrantanopia, it's quite common among my patients as well.

@user-16e6e3 So if you build a heatmap / 2d histogram of the fixation or gaze norm_pos_x/norm_pos_y data, you should see significant more data points in the area of the impairment, correct?

@papr In patients who compensate well, yes. Others would look more towards their sighted field.

I was also thinking about using markers (apriltags) on the left side of the room (in case of left hemianopia), this would also enable creating a heat map of gaze distributions, right? Would not be gaze-contingent, but could still be an option.

@user-16e6e3 The difference would be the coordinate system. gaze/fixation data is by default in scene camera coordinates (which represents the subject's field of view). If you use the surface tracker to generate heatmaps, you will get data that is relative to a specific region of interest that is independent of the subject's head movement.

that's good to know, thanks!