Is there a lib for converting Pupil Player/Capture raw data to CSV??

@marc Thanks! We aware that it converts raw to csv, but our dataset are divided into blocks and participants. The numbers are too many to individually drag & drop. I was looking for a method that can automatically converts it to csv by programming (python).

Hi, are there any general delays known regarding the timestamps? Maybe a time offset in the world and single eye timestamps?

We accidentally updated the OnePlus 6 operating system (so the app no longer works). Is there a simple method to restore the old system? Should we copy any files before trying?

@papr Event triggers, pupil size, gaze anlge, saccade etc..

Currently it is divided into trial blocks

Each participants have 12 folders each. So.. I'm looking for a method that could automate this process

Hello and thanks for the answer. Are the frame rates from the exported world video and the single eye videos the same as streaming them with the real time api? It appears to be a little time delay (~1 second) in the timestamp we receive from the mobile phone and the reception timestamp of the images in our system. - But only for the single eye video stream.

Yes

I'm receiving the video streams in a custom script and the delay becomes visible while comparing the timestamp from the mobile phone and from the system clock we receive the images.

Hey, please checkout https://gist.github.com/papr/40d332498bfacb5980a754c5692068ec

Hi folks, I have a few questions regarding the smartphone and other ways to use the glasses. My current project involves people wearing the glasses all day. Is there a way to charge the phone while using the glasses? Also, I understand that the amount of data from this collection will be massive, therefore, is it possible to determine the glasses to turn on (say) for 10 minutes every 30 minutes or something? Thanks, Victor

Hi @user-3b5a61! You can connect a "powered USB-C hub" to the phone and connect the Pupil Invisible Glasses and a power source in parallel to record while charging.

The alternative would be to swap phones mid recording.

The app itself does not allow you to set a recording schedule like this. However you can remote control the phone using the real-time API, which allows you to start and stop recordings. This would make it relatively easy to write a custom script to realize such a schedule.

You can find an introduction here: https://docs.pupil-labs.com/invisible/how-tos/integrate-with-the-real-time-api/introduction/

hello pupil team, i have oneplus 6 with android 11 . the invisible companion app does not work . it shows that the version of android is not compatible with invisible companion. it was android 8.1.0 before and it was working but somehow the phone has updated itself and the app is not working anymore. i have tried to delete app data & cache , downgrade the app and restore the mobilephone to factory settings but it could not help any help please 🙂 thanks

i also have oneplus 8 with android 11 and the app is working on it. is there anything which i can do <:whatthefuck:400815732880965632>

the link is not working anymore. but i am looking fot it , thank you so much.

i meant the download link, but it works now : )

thanks

@papr hello! For some reason I have the audio now for my videos, but could you plz let me know what is this type of error as I want to download the video on my pc?

@papr I have the scene and I have the eye cameras, I have the red circle of tracking when I play the merged video on pc, but I can’t download them

So why I cannot download the video that I have on pupil player?! That’s my issue

I have exported the folders for each of my recordings on my laptop. I have installed the pupil player. I drag the folder into the player and I can see the merged scene/ eye camera recordings with the red dot/ circle as an indicator of the point of view. Then, when I want to download (have this complete video) on my own pc, it fails. Sometimes I have the error I sent, sometimes the downloaded video which is mp4, shown as an interrupted file…

True

Hello, I'm getting back to you about a technical query. As I have already indicated, I am currently working on a study in which the time required to look from one area marked with markers to another area marked with markers is surveyed. This will then be compared between the experimental and control groups. Unfortunately, so far I have only been able to mark one area at a time with the Marker Mapper Enrichment when I add an enrichment to the recording. So I am not sure how to get the data of the times between the last viewpoint on one marked area and the first viewpoint on the other marked area. I would be very pleased if you could offer help with this.

Hi, I was wondering how the fixation in the raw data exported from pupillab Cloud were calculated? What is the maximum degree and minimum duration? Can these two values be adjusted？

In older versions of the pupilplayer, the fixation value seemed to be adjustable

Hi @user-64c4d3! Fixations are calculated using a novel velocity based algorithm that is using optical flow to compensate head movements. Its not the same algorithm used in Pupil Player. We are currently working on a pier reviewed paper on the algorithm. For exact values on the velocity thresholds etc I need to ask my colleagues who will be available again on Monday. The values are not adjustable.

Hi! I'm wondering what the FPS from IMU should be. While using ndsi I'm receiving a framerate of around 3.5 - is that correct?

Thanks for the fast reply :). What do you exactly mean by "contained sampe rate"? Is this from the mobile device or are we receiving bigger packages (such as receiving a package three times per second with multiple data)?

Thank you papr I'll take a look at it!

Hi folks, quick question about the 'heat mapping' enrichment feature. In order for this to be applied to a video taking, am I right in thinking that the only way to apply this enrichment is if those marker icons were printed off and stuck to the focus prior to recording using the glasses? Is there no way to add the markers retrospectively?

i would like to create a "heat map" but i didn't print the marker icons ... now i can't do nothing ?

Hi @papr, I get excited every time I communicate with you. The documentation claims: the exact framerate of this signal depends on the model of phone you use as Companion device. On a OnePlus 6 device, the framerate is around 50 Hz, while on a OnePlus 8 it is around 65 Hz. I want to know if this frequency can be increased after I use One plus 9, and whether this frequency can be fixed.

Thank you very much, I will be watching this announcement closely.

This fixationscsv file comes from a recording with Pupil Core. I need to know if and how I can have the same file, most importantly including start_frame_indexand end_frame_indexfor a recodring using Pupil Invisible?

Hello everyone 🙆‍♂️ I’ll be working with the pupil invisible glasses in the upcoming days and wanted to ask, if it’s alright to post any questions regarding error messages/connection issues here on discord? And: Is English the preferred language? 🙂

Have a nice day!

Fixation data for Pupil Invisible can be obtained via Pupil Cloud. Its included in the recording download and the raw data export. The format is a bit different though. The full documentation is here: https://docs.pupil-labs.com/invisible/reference/export-formats.html#fixations-csv

The fixations there have a start and end timestamp corresponding to the contained gaze samples. A frame index is not directly given, but using the fixation timestamps in combination with the scene video timestamps the index can be inferred.

Hi there, I am a researcher using the Pupil Invisible for gaze tracking in my research experiments that last about 1 to 2 hours long. During which the glasses get really warm for the wearer causing discomfort. Are there any solutions for this? If not, I was looking to 3D print an insulating cover to slip on that could reduce the heat transferred to the wearer's skin. To do so, I was hoping I could get some CAD files of just the exterior frame arms to use as a reference for the 3D printed cover.

@marc hi, could you plz LMK what is the issue with such files that make the pupil player crashes? Tnx

I have 4 recordings that make the player crash

This is just the photo I sent. As soon as I drag the folder it comes in two seconds and player crashes by itself

Hi @papr, I found that the scene image obtained by pupil-invisible seems to be somewhat distorted, does this require calibration.

Here is a picture example.

I want to perform object detection on the scene image (in other words: YOLO), do I need to correct the scene image? If Pupil-cloud corrects the image in gaze-mapper, does it mean that I also need to calibrate for object detection.

@paprThis situation happens for all of my recordings

I found this two files:

One thing I forgot to mention: The gaze date is in distorted image coordinates. Since you probably want to relate the gaze data to the YOLO detections, you would need to undistort the gaze points as well, if you distort the video, because YOLO detections would be in undistorted coordinates then.

The gaze overlay hack does unfortunately not handle this for you as well, but you can use the intrinsics as above and the cv2.undistortPoints method. https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html#ga55c716492470bfe86b0ee9bf3a1f0f7e

Thanks for your@marc explanation, I have some doubts, do you mean the gaze data from and Raw data export is distorted?

What I mean is that the gaze data is in the original scene camera image coordinates. If you change the scene camera images by undistorting them, you need to equally change the gaze data as well.

This is might sound more complicated than it is though. In Python pseudo code it would look something like this:

K, D = download_camera_intrinsics()

undistorted_image = cv2.undistort(image, K, D)
undistorted_gaze = cv2.undistortPoints(gaze, K, D)

Plotting gaze onto image would be correct, but the image looks distorted. Plotting undistorted_gaze onto undistorted_image would also be correct and the image does not look distorted.

Plotting gaze onto undistorted_image would be wrong, and the gaze data would be slightly off.

Besides that, I have another question. I want to utilize the eye image images obtained by pupil-invisible, but I found an interesting thing. The number of gaze points and the number of eye images do not match. In other words, eye_image_number (Pupil player)<gaze_points (Pupil cloud-Raw Data Export)<eye_image_number (.mjpeg file).

1.For Pupil Player, I used the exported eye0.mp4.

Is this from the intermediate recording format or exported from Player using the eye video exporter?

1. For mjpeg files, I use Opencv, I counted the number using the following code: *def get_frames(path): ''' :param path: 视频地址 ''' frame_count = 0 frame_buff = [] capture = cv2.VideoCapture(path) while capture.isOpened(): ret, frame = capture.read() if not ret: print("Can't receive frame (stream end?). Exiting ...") break

   frame_count = frame_count + 1
   # frame_buff.append(frame)
   

   return frame_count*

The issue with OpenCV is that it sometimes skips frames and would thus yield a frame count that is too low. This could not explain why you find more eye frames than gaze samples.

@user-a98526

pip install av

import av

container = av.open("eye1.mp4")
count = 0
for frame in container.decode(video=0):
  count += 1
print(count)

In fact, results is as follows: opencv count :3761 ,av count: 3761 "gaze.csv:3566"

Can I upload my data for a detailed analysis?

I have sent an email. In fact, I found that my Pupil Invisible companion produces a 1-2s gap per recording, which is ignored when using Pupil Player but not in Pupil Cloud.

This gap is a solid gray image of 1~2 seconds, and I doubt whether the gaze data during this time is ignored by the Raw Data Exporter.

Yes, I actually did, my remaining doubts are mostly inconsistencies in gaze counts and eye images, and I've shared my data to [email removed] hope this helps.

Thank you very much.

@user-a98526 Please see my notes below: - after pressing the start button, the eye cameras take a short time to start (~0.43 seconds in your recording) - after pressing the stop button, the eye cameras take a short time to stop (0.5-1.0 seconds) - Pupil Cloud matches a right eye frame to every left eye frame as long as it is not further away than 5ms. Left eye frames without a matching right eye frames are not processed for gaze - gaze.csv only contains gaze data between the presses of the start and end buttons

The discrepancy between number of gaze samples in gaze.csv and the number of frames in eye1.mp4 is due to the last point. You can verify this by plotting the timestamps (see attached plot).

What this also means is that there is no issue with the data, all of this is actually expected! 🙂

Is there a way to get the eye image corresponding to the gaze point?

From your@papr explanation, I understand why the number of eye images is greater than the number of gaze points, which solves my doubts.

More precisely, the eye1.time file contains the corresponding timestamps of the left eye frames, which correspond to the gaze ps1.time timestamps. eye0.time contains timestamps of the right eye, which correspond to the timestamps in gaze_right ps1.time.

The timestamps in gaze.csv only contain the timestamps of the left eye (a subset of gaze ps1.time as described above) and would only allow you to match the left eye.

You can open the time files in Python using np.fromfile("gaze ps1.time", dtype="int64").

Okay, I'll try it, thank you very much for your help today. This will be very helpful for my research.

Hi Pupil friends 😉

just a quick question about the Invisible and Markers vs Image Mapper For "art like pictures" an a white background, which method would you recommend for analysis?

Markers or Image Mapper?

Thanks for you opinion Philipp

Hey @user-0f58b0!

We have recently recorded an example dataset (which will be released very soon!) in a gallery, where we used the Reference Image Mapper with great success! So I would for sure recommend it!

It depends a little bit on how the pictures look like though. If it was very uniform art work, say paintings of just solid color, the algorithm will not work well. But for most types of art it should work well!

I can send you a teaser of the dataset if you are interested in comparing it with the setup you are going to have.

@papr

This is what you want?!

@user-b811bd That's not quite what we need! Please first open one of the problematic recordings in Pupil Player, which will log the error in the player.log file. Then, upload the player.log file here. Please upload the actual file rather than taking an image of it!

Hello, can I disable the sound in PupilCloud when creating Gaze Overlay as .mp4?

No, that is not an option currently. You would have to use an external tool to remove the audio from the video.

sounds great, thank you so much

Hi, i need to know if i can use pupil lab open source software with Gaze Point GP3 hardware? thank you so much

Sorry to bother you@papr @marcagain, I tried your method, but I found that the number of data in gaze ps1.time (767, from Down Raw Data) is much smaller than the amount of eye image 1(3761). Also, the timestamp in gaze.csv(from Pupil Cloud Raw Date Export) and the timestamp in eye1.time are not equal. So I'm still not sure how to match them.

Hello everyone! I have a technical problem with pupil invisible. I am using OnePlus 6. The system has been updated by mistake and (as expected) the app was not working anymore. We managed to reinstall the older version (Android version 8.1.0), however, we are still having some issues. Eyes are not detected and the gaze pointer during offset correction is not visible or fixed on the scene video. The recording seems to start correctly, but it will then give an error message. Any idea about how to fix this problem? Please, let me know if you need any more info. Thanks a lot!

Hi @user-4f3037 Sorry to hear about the issues. Could you please contact info@pupil-labs.com in this regard and attach the error message? Ideally, please attach the recording's android.log.zip file as well.

Was the recording uploaded to Pupil Cloud?

Does that mean that you usually copy the recordings from the phone via USB?

Thanks @papr . By the way, I'm trying to use https://api.cloud-staging.pupil-labs.com/v2/ to get the camera parameters, but I don't seem to have access.

Just to clarify, you get the error message when attempting to start the recording, not later after the recording has started already, correct? What is the error message?

ok, please contact info@pupil-labs.com with this information

I tried, but it still doesn't seem to work. My serial number is 7PQ45.

Is it YWSCB?

This is very useful, thanks！

I have a small suggestion that I hope will help you improve your Pupil Invisible experience. The Gaze overlay plugin for Pupil Cloud may add functionality similar to Pupil Player's eye overlay.

Hello everyone 🙆‍♂️ I’m using the pupil invisible glasses for an interview and while checking the recording in the invisible companion app, I’ve noticed that the ET circle doesn’t move at all. Any ideas what I can do to prevent this from happening again? Thank you very much in advance 🙂 EDIT: I just checked the cloud and thankfully the recording seems to be fine there!!!

My application is eye activity classification (fixaition, saccade, smooth pursuit, PSO, etc.), which will require experienced users to label the data. During the tagging process, eye images, gaze points can provide the user with a more accurate judgement. This is why I am so obsessed with matching gaze points to eye images.

In-app and during - but it only appears that way when checking the recording in-app; I just viewed the video in the cloud and it seems to be fine there 🙂

From what age onwards would you recommend using Pupil Invisible?

Hi Kerstin! The official answer would have to be that we have never properly evaluated usage with children and can't make solid recommendations. But we can share some anecdotal advice.

The biggest issue when using Pupil Invisible with children is that the glasses are not sitting well on their small heads. This can be partially circumvented using the head strap, which gives some stability.

In all cases we have tested, the algorithms have been working well even on children as young as 2-3 years. But in that age group the glasses sit very loose. More creative ways of fixating the glasses may help. At maybe 8-10 years I'd expect the issues to stop as the glasses should start to sit well when using the head strap.

Hi @marc, I found an interesting thing. Sometimes an eye0(t) image will be closest to two adjacent eye1(t) and eye1(t+1) images, while eye0(t+1) is not closest to any eye1 image, does this mean that some eye0 （i.e., eye0(t)） images will be used twice in the gaze point estimation.

Thanks for the answer, this is a very interesting situation!

Hi there! I have a problem as well with pupil invisible glasses and the recordings in the invisible companion app. The ET circle seems to stop moving after 10 seconds to 2,5 minutes, even though the recording overall continues. Is there a possibility that I can prevent this from happening so I can check the recordings in the app (not live nor in the cloud) with a working ET circle? Thank you in advance!

Hi @user-bd34eb! Does the gaze circle only stop moving when you play the recording back in the app, or also during playback in Pupil Cloud/Pupil Player/ Live Preview? Would you be able to share the recording with [email removed] so we can take a look?

Thanks @user-bd34eb! After a first look it seems like there is nothing wrong with the recording itself. The gaze data is fully available in the recording. I have forwarded the recording to the Android team to investigate why it might not play correctly in the app. We'll keep you up to date!

Thanks for the report @user-2d66f7! We are still investigating the issue! If you wouldn't mind, could you also share one of the affected recordings with [email removed] so we have another instance to work with?

@user-2d66f7 @user-bd34eb We were able to locate the problem and fixed it with a new release, so please update your apps. The release should become available tomorrow! 🙂 @user-a9589e If you could anyway still send us your example recording that would be great to double check!

Hello. We are small company considering to buy Pupil Labs Invisible. We cannot find anything regarding the warrenty of the products though. Can you inform me of the warrenty? 🙂 Cheers, Frederik

@user-710092 We give 12 months of warranty. In case of a defect we will ship the unit back to us, repair it and ship it back to you for free. Passt warranty we have a repair service, our repair pricing is fair: it covers part and labor cost only.

Is it possible to get the companion app outside the play store ?

You need a google account for that. It would be great if the apk would also be made available outside of the play store.

For a official educational institution such a account binding is difficult. We use the OnePlus in a tight defined way only as (local) recorder of our eyetracking data. So it would be really great to have another option to update the companion app.

I'm not so happy with that but i did that right now.

Hi 🙂 I noticed something weird in my data. I always get a 1.0 confidence score for all of the frames. However, sometimes my participant's eyes are almost closed (not enough to be a blink) and in these moments the gaze estimate in bad laterally. I highly doubt that the gaze orientation is right at these instants. Do you have an idea on how to detect these instant other than I have the feeling that this is not right? Is the confidence score usually reliable?

Thanks for you answer, it explains why it is always 1.0. Do you have an idea on how to detect half closed eyes?

Hi! Is the video frame rate of the pupil invisible world camera variable?

In our videos we assume a slight increase and then decrease to 'normal' of the playback speed. This happens for a few seconds throughout the raw world video.

Could that be possible?

Does anyone know how to resolve issues with uploading to the cloud -- files have been stuck at 0% for several days?

Files/videos of similar length have uploaded to the cloud perfectly fine in the past and the specific files not uploading can be viewed with overlaid gaze by clicking on the file. I have stopped the app completely and rebooted the phone provided by pupil but have not had luck in the files uploading.

Hi @user-6a91a7. Please try logging out of the app and logging back in again.