AOIs in a transit station

I am looking for an easy and non-technical way to stream the gaze data from the pupil lab invivible to a macbook pro. this is mostly for demonstration purpose. I have tried the following link https://docs.pupil-labs.com/invisible/how-tos/data-collection-with-the-companion-app/monitor-your-data-collection-in-real-time.html as suggested on the website, but the page does not seem to open. I would be very grateful for an easy and non-technical solution... many thanks

Hello! I take videos with pupil invisible for my thesis, and I put my videos on pupil clouds but I noticed that the image of some video cuts in the middle of recordings. I would like to select the parts of the video that work but I can not do it on pupil cloud, it takes into account the entire recording.. how can I do?

Hi @user-50cf87! What exactly do you need to get out in the end? - For a gaze overlay video you can cut down the recording using events in the gaze overlay enrichment definition - If it is just about the video, you can cut it using any video editing software. - When exporting all data of a recording, there is not automatic way to crop it down. But the CSV data is ordered by its timestamp, so you could filter it this way.

Head Pose Tracker

Hello, I have a question. I used to use pupil core and in particular the 3d coordinates of the gaze point, can I get that with pupil invisible as well?

Hello folks at Pupil Labs - Last time I checked transferring recordings between workspaces was not possible - Is there any chance this is something in the pipeline for the future? 🥺 . It seems to be a bit of a no-brainer to my mind - to cater for the eventuality of inadvertently capturing recordings while set to the wrong workspace - Unless there is a benevolent reason for not allowing this that I am unaware of.... Plus, there is the small issue of separating concerns for clients post-capture - something I am currently facing...😩

Hi @user-df1f44! This is currently still not possible via the UI. If you have a concrete issue with recordings in the wrong workspace we could try to help you manually! I agree, from a UX standpoint this is a rather obvious feature. It's not quite as trivial to implement as one might expect though, which is why we have deprioritized it so far. Given the interest, we should re-evaluate this!

It took the liberty of creating a feature request for this: https://feedback.pupil-labs.com/pupil-cloud/p/recording-transfer-between-workspaces

Hi@marc，in my use of invisible, it often fails to connect, like this

It usually takes 100 to 200 attempts to connect successfully, and I'm not sure if this is a problem with the invisible or the phone.

Can any Typec-Typec cable be used, as I plan to use other Typec cables.

I'd recommend testing the cable for a couple hours before relying on it!

Specifically, can I try Apple's Typec, which I plan to use?

Or Oneplus' typec.

We have never tried using those cables, so I am not sure. Sorry!

Or can you provide the type of typec you are using?

I think this connection failure problem may come from the cable, and I plan to buy a new cable to test if the problem mentioned above still occurs.

Got it! If the cable is indeed the issue, we'd also be happy to send you a new one!

Because I had this same connection problem with both of my Invisible (unfortunately, I lost a cable, so they used the same one). So I think it's more likely that the cable is faulty.

@marcThanks for your help, I am currently working on head pose estimation work (displacement and pose) and hope to help Invisible in the future.

Hello, I'm a newbie of pupil invisible. And I encounter a trouble(like picture),which always when the video longer than 25mins. Is it overheat? Does anyone know any solution?

I will appreciate it!

@user-d407c1 You means just email to this address about this issue?

Sure, thanks.

Hello, at our institute we are using pupil invisible. I remember when I used it last year for a research project there was a "fixation" plugin available in the plugin manager of pupil player. I started working again with the glasses recently and noticed that there is no fixation plugin available anymore in the version that I have installed now (the newest). In the online user guide I also read now that it is not available for pupil invisible. Was it removed in between versions or do I just don't remember correctly that this option existed? Thank you.

Hello! We are using pupil invisible for a TSST dyadic task (two people) and wanted to know if there was an easy way to set up event markers with psychopy to sync the two recordings

Hey @user-46e93e. Are you using PsychoPy to present stimuli? Check out this tutorial for tracking events using Python: https://docs.pupil-labs.com/invisible/how-tos/integrate-with-the-real-time-api/track-your-experiment-progress-using-events/#track-your-experiment-progress-using-events

Hello, I have a question I understand that pupil invisible only provides 2D line-of-sight coordinates, but is it possible to obtain 3D line-of-sight vectors from this information?

Hi @user-746d07! Yes, a 2D gaze point and a 3D line-of-sight are basically equivalent and you can transform each into the other. To do this you need some camera geometry. Using the inverse of the projection matrix of the camera you can "unproject" the 2D gaze point back into the 3D gaze ray.

This might sound complicated if you have not dealt with this before, but there are lots of resources on the internet about this! You can check this how-to guide on how to obtain the camera intrinsics (which contain the camera matrix and distortion coefficients). https://docs.pupil-labs.com/invisible/how-tos/advanced-analysis/intrinsics/

The guide contains the same math/OpenCV method calls you would need to use. The following line is what it comes down to:

rays_3d = cv2.undistortPoints(
    points_2d.reshape(-1, 2), camera_matrix, dist_coeffs, P=None
)

Let me know if you have questions!

@user-746d07 Yes, the scene camera of Pupil Invisible covers the entire active human FOV. The fields of views of our various scene cameras are as follows (horizontal x vertical):

Core: 99°x53° (by default. When using 1080p it goes up to 139°x83° but has a lot of distortion) Invisible: 82°x82° Neon: 132°x81°

So as you can see the vertical FOV of Pupil Invisible is much larger than for Core. You can not change the direction of the scene camera like for Pupil Core, but given the size of the FOV its also not necessary. Pupil Invisible covers the natural FOV very well and even more extreme scenarios, like e.g. interaction with a smartphone which usually happens at steep downwards angles, are covered well.

Neon has an even larger horizontal FOV, but the motivation of this is rather to cover peripheral vision better than to capture more of the active FOV, which is already covered well. If your timeline allows it I'd recommend purchase Neon over Pupil Invisible.

OK,thank you I understood that pupil invisible has a large FOV and covers, for example, when interacting with a smartphone. In pupil invisible, unlike pupil core, I was told that gaze point supports only 2D output. Considering what you said earlier about camera intrinsics, am I right in understanding that if the orientation and position of the face is known, it is possible to obtain the gaze direction in an absolute coordinate system?

Hi, I just found your company in google. I would like to share with you our idea of research. We are working on visual attention, for this we work with database of heatmap gaze (like MIT1003). We would like to construct our own database which will correspond to our cases of studies (static scenes). We would like also to make some studies about eye fixation on a video. Did you have a solution to propose to us?

Hi, I’m using pupilcloud for my phd and I have a problem to download a gaze overlay enrichment.. nothing happen when I try to download this enrichment (only for this video in particular and I don’t know why). Do you know how I can upload it?

Hi, i am using pupil to have a small eye tracking event. i wish to know how can i calibrate it to the center as it always tends to point to the left of the actual view.

Hi, I've some problem with the uploading of my videos on the cloud with the Pupil Invisible App. I'm connected on internet and the uploading of the videos stay at 0%. I'd tried to delete all the files of my Pupil Cloud to make space on my account but it doasn't change anything. Another solution could be to export the video files locally (from and to the phone) but the option is not available on the app. Does anyone has a solution ?

@user-5b6152, some more debugging tips here: https://docs.pupil-labs.com/invisible/troubleshooting/#recordings-are-not-uploading-to-pupil-cloud-successfully

We are using real time python API (https://pupil-labs-realtime-api.readthedocs.io/en/stable/api/index.html). We are just is there any way to change the recording filename in the cloud when we are starting the recording using python api method (Device.recording_start())?

Hi Neil, would it possible to set up a time to call with someone who might be able to walk us through tracking events for our specific task? I have it set up but still unsure about how to test it. We are also looking to send it to two separate invisible devices at the same time

Hello again, I tried the above mentioned method for parallax correction, tbh one of the best ways of calibration i have yet seen. my only difficulty is, when the calibration is done from a distance or close proximity, the same calibration tends to change when the distance is changed respectively. how can this be fixed or is SOP for calibration which should be followed? and is it better to calibrate with every new user?

So, whether you use the offset correction really depends on things like the experimental task, the participant, whether the non-offset corrected gaze is accurate enough for your needs etc.

I typically do a qualitative assessment with each participant before deciding!

UGREEN USB C Hub 4K60Hz, 5-in-1 Gigabit...

hello. I am currently learning how to get the 3D gaze direction from gaze x,y[px].

Last time you told me about the following code, am I correct that the 3D line of sight direction is a vector with 1 in the z-axis of the value obtained by the following code?

ray=cv2.undistortPoints(
    point_2d.reshape(-1, 2),camera_matrix, distCoeffs=dist_coefs,P = None
)

Yes, that should be correct!

Hello,

I'm looking into acquiring a wearable eye tracker. I'm hoping to get some recommendations based on our requirements. I'm currently using iMotions for on-site user testing of interactive computer video games. I'm using a screen-based eye tracker and Facial Expression Analysis while recording game footage.

I'd like to start testing games with touch controls. But touching the screen causes the user to lean in closer to the screen, so the screen-based tracker loses track. I think a wearable tracker would be better if it can handle this scenario.

Any advice would be appreciated, regarding things to look our for or trade offs to consider. Thank you

Hi @user-3fba95!

Doing screen-based gaze estimation with wearable eye trackers is possible, but it comes with some additional challenges. The output space of a screen-based eye tracker is pixel coordinates on the screen. For a wearable eye tracker the output space is the pixel coordinates of the scene camera. So there is an additional step of converting the gaze estimates from scene camera coordinates to screen coordinates.

In order to do this one has to track the location of the screen in the scene video. Knowing the pose of the screen in the scene video allows you to transform the data. This step will be the major bottleneck for robustness in your setup. The various eye tracking vendors all have solutions for this, but they are all limited.

One option is to do this mapping between coordinate systems manually, i.e. go through the data frame by frame and manually indicate where on the screen the subject is looking at using mouse clicks. This is very labor intensive, but it works. The Tobii software has an interface for this, I am not entirely sure about iMotions. We do not currently have an interface for this, but will add something to Pupil Cloud in ~Q2 2023.

The other option is automatic mapping tools. Most software options offer some implementation of this, but they differ a lot in robustness. The options we have are: the Marker Mapper and the Reference Image Mapper.

The Marker Mapper works robustly but requires markers placed around the screen. If that was acceptable for you, this is probably the most robust solution available for screen tracking. You can find the documentation here: https://docs.pupil-labs.com/invisible/enrichments/marker-mapper/

The Reference Image Mapper only requires only reference image of the screen without markers. Using it to map to a screen is presented in detail here: https://docs.pupil-labs.com/alpha-lab/map-your-gaze-to-a-2d-screen/ I believe that the Reference Image Mapper is the most robust marker-free option currently available anywhere, but its robustness is also limited.

Competing with screen-based eye trackers is not super easy for any wearable eye tracker. I can't promise that our setup would work better in the end than you screen-based system. But I believe there is a chance and its probably the most reliable wearable setup you could try. If you are interested I'd recommend testing the system within our 30 day return period!

Just to add to @marc comment, iMotions has a manual mapping option, with some kind of interpolation between frames. https://imotions.com/blog/learning/product-news/product-release-new-areas-of-interest-aoi-editor-in-imotions-9-0/

Hi invisible community, I'm working on blinking and using Pupil Invisible to catch the moment when we blink. I'd tried to use the raw data in the blink.csv file and convert the start timestamp in frame to compare with the video in Pupil Player. To do it : 1. I've deduct the recording begin data (found in the event.csv) from the start timestamp data to have the moment of the blink from the beginning of the video in nanosecond (and not UTC). 2. I've divided the result by 1/30 because the framerate of the frontal video recorded by the smartglasses is 30 Hrz. I've rounded the result to obtain a frame number. BUT when I compare this frame number with the video in Pupil Player with the eye overlay the eyes are not closed. I check three videos and each time the real blinking on the overlay eyes video in pupil and the supposed blinks from the raw data are not corresponding. For the first video, there is approximately -40 frames between the frame number in the raw data and the real blink in the overlay eyes. For the second video, there is approximately +10 between the frame number in the raw data and the real blink in the overlay eyes. And for the last one, there is absolutely no correspondance between the frame number in the raw data and the real blink in the overlay eyes. Do you have any idea where is the problem ?

Hi @user-5b6152! Your approach makes sense but there are a few caveats! Your technique in 1) is correct, this converts the timestamps from absolute timestamps to relative timestamps since the beginning of the recording. Note however, that the time measured in Pupil Player is relative time since the beginning of the scene video, which is not the same as the beginning of the recording. When the recording starts, all the sensors get initialized, which takes up to a couple of seconds (depending on the sensor) and only then the sensors start recording. The scene video takes usually about 1-2 seconds to initialize (which explains 10-40 frames difference).

So to convert timestamps into "Pupil Player Time", you'd have to subtract the first timestamp of the scene video, rather than the recording.begin event timestamp.

The idea of dividing the result by 30 to get a mapping to world frame indices would only yield a roughly correct mapping. This is because the actual framerate is not perfectly solid 30 FPS. In dark environments the exposure time might be longer than 1/30 of a second, which reduces the framerate. Even in perfect conditions the framerate is not perfectly stable but will fluctuate just slightly. Thus you would accumulate a drift error over time with this strategy.

The recommended strategy to find the corresponding world frames would be to match the timestamps of the events with the scene video timestamps directly to find the closest matches. This can be done with relative easy using Pandas pd.merge_asof method. How this works in detail is described in this how-to guide: https://docs.pupil-labs.com/invisible/how-tos/advanced-analysis/syncing-sensors/

That would be the first timestamp in the world_timestamps.csv file!

I see! I thought the recording.begin event would be generated using the timestamp from the info.json file, my mistake! The other points still hold though, so please check the linked guide!

This depends on the context. The direction is initially just a direction without any units. If you can localize the head/glasses in a 3D coordinate system measured in meters, then you can have a "gaze ray" of a certain length in this direction, and that could be measured in meters.

Okay! We have no other immediate solution. Tomorrow I could provide you with a script that does the required calculations. You'd still need to run it though, but that is fairly easy.

I am not sure I understand the question. The coordinate system in the image seems correct. The 3D gaze point is a point somewhere along the gaze direction originating from the 3D camera position.

G is a vector with no intrinsic physical unit at this point. Its just a direction encoded by the ratio of those numbers. The way undistortPoints works will simply scale this directional vector such that the z coordinate equals 1.

Yes, both is correct!

Hello! I was wondering how i can open a json file ? When I used the time alignment method, it created a time_alignment_parameters.json file. What information can i get from this file ? Thank you in advance!

I know it's for sync between streams of lab recorder, but what does this file tells me? The time difference between the lab recorder and the record from the companion app for example ?

Yea, so basically the cloud_to_lsl intercept , is the time that the companion app was delayed to lsl recording ?

Thank you!

Hi, I have done a series of recordings using pupil invisible. I saw the recordings in pupil cloud, and the uploaded recordings have automatic fixation positions and saccade lines drawn. Is there any documentation of how these are estimated? Also, can I change the fixation threshold myself?

Hello again, recently i had recorded some videos for reference image mapper. but the upload is stuck on 97% and doesnt seem to go futher. i have tried pause and resume it but after pausing the resume is not working. for safety case i have exported the said video to my system through usb. it their a possibility i can upload the same to the cloud from system? as the export has many files i am confused.

@nmt Hi, thanks for your reply. i have checked the mentioned solution. the uploads is enabled and I am also able to access the given site.

@nmt Thank you so much. that worked! 😁

@nmt Hi, i have another doubt. does the companion device need to be on while accessing cloud? cuz as of now i am unable to play any recording on the cloud.

Hi@nmt, I think there is something wrong with my Pupil Invisible, it can't connect to the companion device (Oneplus 6) successfully. I think I need some after-sales service, who do I contact?