πŸ‘ core


user-7fb343 03 September, 2019, 20:31:10

Hi,

Im having an issue with playing the videos im recording, When I drag and drop the video file into pupil player it tells me that its not a valid file. Is there anything I am missing? Im using MacOS if that helps

Any help will be greatly appreciated

papr 03 September, 2019, 20:31:44

@user-7fb343 please drag and drop the folder containing the video file, not the video itself

user-7fb343 03 September, 2019, 20:33:31

Ahhh, working now, thanks a lot, really appreciate it!

user-5543ca 05 September, 2019, 13:01:27

Hello! A recording (done in Feb 2019) was crashed during an experiment and I'm not able to play it now -- when I drop the folder "OOX", the Pupil Player simply crashes. Here is the message I found on the log - "2019-09-05 14:59:02,687 - MainProcess - [DEBUG] root: Unknown command-line arguments: [] 2019-09-05 14:59:02,687 - MainProcess - [DEBUG] os_utils: Disabling idle sleep not supported on this OS version. 2019-09-05 14:59:03,763 - player - [INFO] video_capture: Install pyrealsense to use the Intel RealSense backend 2019-09-05 14:59:26,228 - player - [INFO] launchables.player: Starting new session with '/home/ravi/recordings/2019_02_22/001' 2019-09-05 14:59:26,244 - player - [INFO] update_methods: Updating meta info 2019-09-05 14:59:26,245 - player - [ERROR] launchables.player: Process player_drop crashed with trace: Traceback (most recent call last): File "launchables/player.py", line 774, in player_drop File "shared_modules/update_methods.py", line 45, in update_recording_to_recent File "shared_modules/version_utils.py", line 67, in read_rec_version KeyError: 'Capture Software Version'"

user-5543ca 05 September, 2019, 13:02:06

I would be super grateful if you could help me make this recording usable. Thanks a lot in advance πŸ™‚

papr 05 September, 2019, 13:06:35

@user-5543ca Are you running from source?

user-8779ef 05 September, 2019, 13:27:38

Hello ! I'm seeking some help with a replicable pupil crash / timeout.

user-8779ef 05 September, 2019, 13:27:52

I have both logs and videos to share.

papr 05 September, 2019, 13:28:31

@user-8779ef Can you share the logs here?

user-8779ef 05 September, 2019, 13:28:38

Yep, first, a bit of explanation.

papr 05 September, 2019, 13:29:06

Oh ok, then please send the explanation + logs + videos to data@pupil-labs.com

papr 05 September, 2019, 13:29:12

I do not want this to get lost

user-8779ef 05 September, 2019, 13:29:15

Ok, will do. Thanks!

user-5543ca 05 September, 2019, 13:33:13

@papr No, I'm not running from source.

papr 05 September, 2019, 13:33:45

@user-5543ca ok, which version of Player do you use? Also, could you please share the info.csv file with us?

user-5543ca 05 September, 2019, 13:37:33

version: 1.11.4 (as shown on the Pupil Player when I open some other video).

papr 05 September, 2019, 13:40:13

@user-5543ca since it is a 00X folder, I assume you used Capture to record the recording? Which version of Capture did you use to record?

user-5543ca 05 September, 2019, 13:44:04

Also, version: 1.11.4

papr 05 September, 2019, 13:45:06

@user-5543ca It looks like the info.csv file is not complete. Could you share it in this channel?

user-5543ca 05 September, 2019, 13:45:42

info.csv

papr 05 September, 2019, 13:46:27

@user-5543ca Looks like your recording was not properly terminated.

papr 05 September, 2019, 13:47:01

It is very likely that the videos are not properly terminated either. Can you check by opening them in VLC?

user-5543ca 05 September, 2019, 13:47:19

I did, they didn't play

user-8779ef 05 September, 2019, 13:47:43

Ok @papr . Logs / data sent. FWIW, the issue seems to be that our custom rigs invokes an error in calibration.cpp: "File "shared_modules/tasklib/observers.py", line 28, in raise_exception cv2.error: OpenCV(3.4.5) [email removed] error: (-5:Bad argument) Distortion coefficients must be 1x4, 4x1, 1x5, 5x1, 1x8, 8x1, 1x12, 12x1, 1x14 or 14x1 floating-point vector in function 'cvProjectPoints2'"

user-5543ca 05 September, 2019, 13:48:31

@papr The experiment was a rather long one and we didn't know if the laptop's battery died or the Pupil Player crashed.

papr 05 September, 2019, 13:49:46

@user-5543ca In this case, my guess is that the battery died

papr 05 September, 2019, 13:50:12

@user-5543ca I am sorry, but I do not think we can do much to recover this recording if the videos are broken as well

papr 05 September, 2019, 13:52:07

@user-8779ef Thanks. We will come back to you on Monday or Tuesday.

user-5543ca 05 September, 2019, 13:53:09

@papr Thanks for the information.

user-5543ca 05 September, 2019, 13:53:21

I highly appreciate the swift responses.

user-8779ef 05 September, 2019, 13:54:00

Thanks, @papr I'm happy to provide any information on the hardware that your folks might want to know.

user-8779ef 05 September, 2019, 13:54:17

...as it is a modified version of the V1 HMD plugin.

user-b91aa6 06 September, 2019, 10:12:24

Having this problem, the pupil detectors .pyd files are generated, why is this?

Chat image

user-cde04b 06 September, 2019, 13:54:18

ok thanks

user-8507b7 07 September, 2019, 03:10:17

Hi, I need an eye tracker that can work with active shutter glasses (for a 3D TV). Would the Pupil Core work? I have official Samsung glasses, but also a thicker framed 3active glasses (which I prefer to use as they have a longer battery life and are less fragile). (3active glasses: https://www.amazon.ca/SAMSUNG-Compatible-3ACTIVE-xae-Bluetooth-Rechargeable-TWIN-PACK/dp/B006YWKCTW, Samsung glasses: https://www.amazon.ca/Samsung-SSG-5150GB-3D-Active-Glasses/dp/B00INO6JX2). Thanks!

user-a6cc45 07 September, 2019, 17:58:15

Hello, I have a short Pupil recording and from exported data I can see that number of fixations in fixation_report.csv and eye_movement_by_segment.csv is different. Is this because the algorithm for finding fixations is different?

fixation_report -> 181 fixations eye_movement_by_segment -> only 19 fixations

papr 07 September, 2019, 18:08:09

@user-a6cc45 that is correct, the algorithms are different. We are still evaluating the eye movement detector.

user-a6cc45 07 September, 2019, 18:38:06

One more question: when I add durations of all fixations in eye_movement_by_segment.csv the result is ~56,6, but my recording is 58s long, so is this difference caused by blinking?

papr 07 September, 2019, 18:40:53

@user-a6cc45 not necessarily by blinking only, but low confidence data in general, yes

user-4bf830 09 September, 2019, 15:04:42

Hey, all! I'm working with the Pupil mobile headset for some research, what devices do you recommend for mobile data recording? Hopefully a tablet?

papr 09 September, 2019, 15:05:47

@user-4bf830 I personally can recommend One Plus devices. You can find a list of supported phones here: https://github.com/pupil-labs/pupil-mobile-app/#supported-hardware

user-4bf830 09 September, 2019, 15:06:36

Thanks!

user-96755f 10 September, 2019, 08:56:06

Good morning! How can I replace the fish eye lens? I've got the narrow one, I need to know the correct procedure. Thank you

wrp 10 September, 2019, 08:57:16

carefully unscrew the wide angle lens, and then carefully screw in the narrow lens.

user-8779ef 10 September, 2019, 13:49:51

Hey folks - can someone give me a quote on the cost of the new HMD cameras for custom hardware work? If the LED's arrangement or LED-to-camera power chain has also changed for the new HMD system, please provide the price for those, too.

user-8779ef 10 September, 2019, 13:50:01

Otherwise, we have plenty of first-gen LED rings we can use.

papr 10 September, 2019, 13:50:28

@user-8779ef Please contact info@pupil-labs.com for this kind of information.

user-8779ef 10 September, 2019, 13:51:15

Ok, thanks @papr!

user-8779ef 10 September, 2019, 13:52:38

Same for camera specs?

user-8779ef 10 September, 2019, 13:53:01

Or can you provide those? Specifically, for the second-gen HMD integration eye cameras.

papr 10 September, 2019, 13:53:30

Same for the camera specs.

user-8779ef 10 September, 2019, 13:53:34

Ok, thanks πŸ™‚

papr 10 September, 2019, 13:53:39

Not sure if these are available though.

papr 10 September, 2019, 13:53:57

Please let them know what exact information you are looking for

user-8779ef 10 September, 2019, 13:55:37

Yeah, I understand. I will. By using a hot mirror we are changing the distance of the optical pathway, and worry that, when using the new the eye cameras, the eye will be outside the depth of field.

user-8779ef 10 September, 2019, 13:55:51

TMi, though. I'll ping info πŸ™‚

user-adf88b 10 September, 2019, 18:15:53

Hi, I'm getting the following error when I call StartCoroutine(PupilTools.Connect(true, 5f));:

NullReferenceException: Object reference not set to an instance of an object
Connection.sendData (System.Byte[] data) (at Assets/Plugins/pupil_plugin/Scripts/Networking/Connection.cs:97)
Connection.Initialize () (at Assets/Plugins/pupil_plugin/Scripts/Networking/Connection.cs:41)
PupilTools+<Connect>c__Iterator0.MoveNext () (at Assets/Plugins/pupil_plugin/Scripts/Networking/PupilTools.cs:187)
UnityEngine.SetupCoroutine.InvokeMoveNext (IEnumerator enumerator, IntPtr returnValueAddress) (at C:/buildslave/unity/build/Runtime/Export/Coroutines.cs:17)
user-adf88b 10 September, 2019, 18:16:32

It looks like some sort of error in the unity plugin but I know that the demo scene works

user-adf88b 10 September, 2019, 18:16:41

(And connects fine)

papr 10 September, 2019, 18:16:55

@user-adf88b This is an πŸ₯½ core-xr specific issue. Please move the question to the πŸ₯½ core-xr channel.

user-adf88b 10 September, 2019, 18:18:11

Oh, yes sorry.

user-c87bad 10 September, 2019, 18:32:13

@papr Sorry for the late reply. My version is v1.13. And data is sent to you. Thank you very much. I definitely sure that something wrong with the surface detection.

user-c37dfd 10 September, 2019, 18:49:14

Hi. We used capture and player 0.9.12 for a previous project and now we are getting ready to work on a new project. We would like to update; however, we have previously relied on the manual gaze correction plugin because of our child sample. Will the the latest version of player include the manual gaze correction plugin?

papr 10 September, 2019, 19:05:42

@user-c37dfd yes, it does. I would recommend to give it a try. The ui changed a lot since 0.9.12

papr 10 September, 2019, 19:06:57

@user-c37dfd see our YouTube tutorials on how to use it https://www.youtube.com/playlist?list=PLi20Yl1k_57rlznaEfrXyqiF0sUtZMMLh

user-2143a5 10 September, 2019, 19:09:28

My apologies if I'm in the wrong forum - can anyone here comment on shipping times for the Pupil Core to the US? We're located in the Boston area.

papr 11 September, 2019, 06:40:59

@user-2143a5 This might be a question for @user-97ae1e

user-97ae1e 11 September, 2019, 07:44:35

Hi @user-2143a5, I just sent you a PM

user-97ae1e 11 September, 2019, 08:28:45

We ship with DHL, FedEx, or UPS express services (2 days max to reach most destinations). Shipping charges are as follows: Within Germany- 6 euros In Europe- 15 Euros Out of EU- 35 euros

user-2143a5 11 September, 2019, 11:42:15

Thank you!

user-bd800a 11 September, 2019, 11:51:48

Hi, I would like to use the frame publisher plugin with pupil service, is it implemented or do I need to do it? I can't find the setting

papr 11 September, 2019, 11:54:34

@user-bd800a You will have to start it via a notification, e.g. https://github.com/pupil-labs/pupil-helpers/blob/master/python/recv_world_video_frames.py#L22-L32

user-bd800a 11 September, 2019, 11:54:50

Okay great, thank you!

user-bd800a 12 September, 2019, 09:03:46

Is there a reason for pupil_service not to have the Recorder and Annotations plugins? I imagine that it is due to the timestamps being linked to the world recording which is absent in pupil_service, am I correct?

papr 12 September, 2019, 09:06:31

@user-bd800a This is correct

papr 12 September, 2019, 09:07:32

@user-bd800a Also: Service is meant as low-latency gaze mapping application for interaction with πŸ₯½ core-xr . If you want to make recordings, please use Capture.

user-bd800a 12 September, 2019, 09:07:48

And is there a command to ask the pupil softwares whether it is capture or service?

papr 12 September, 2019, 09:09:48

@user-bd800a I do not think so πŸ€”

papr 12 September, 2019, 09:10:03

Please open a Github issue for that.

user-bd800a 12 September, 2019, 09:17:20

Thank you for the prompt answer

user-c87bad 12 September, 2019, 09:50:53

Hi, there! Is here anyway I can run pupil player by using cmd?

papr 12 September, 2019, 09:51:54

@user-c87bad Do you just want to start it using cmd or do you mean to run Player without UI?

user-c87bad 12 September, 2019, 09:52:46

run without UI

papr 12 September, 2019, 09:52:57

@user-c87bad This is currently not possible

user-c87bad 12 September, 2019, 09:53:07

All right!πŸ˜‚

user-40621b 12 September, 2019, 10:08:50

hi @papr .. How do you calibrate an eye tracker connected to a smartphone? Do you have the tutorial video for this question? thanks..

papr 12 September, 2019, 10:11:13

@user-40621b You basically record the calibration procedure, and process it offline in Player: https://www.youtube.com/watch?v=_Jnxi1OMMTc&list=PLi20Yl1k_57rlznaEfrXyqiF0sUtZMMLh

user-40621b 12 September, 2019, 10:19:06

Thanks @papr .. πŸ˜€

user-c87bad 12 September, 2019, 10:34:12

A quick question: when doing the calibration, if I don't blink eyes, the error would be less. Why that happens? Aren't those data with low confidence discarded??

papr 12 September, 2019, 10:57:58

@user-c87bad yes, low confidence data is filtered, but you might loose your fixation from the calibration marker unwillingly, resulting in less accurate calibrations

user-c87bad 12 September, 2019, 10:59:38

@papr Also some questions about the calibration: 1) I keep my head stationary during the whole process and have calibration for several times. The results or accuracy are different from time to time. Since the calibration result will influence the gaze position and other data, I think this can be really tricky that the result are different. 2) What's the difference between the calibration and calibration test? We can get error of both. Then calibration result can influence the data afterwards but test won't?

user-bf468d 12 September, 2019, 23:33:06

I am looking for simple eye tracking glasses that can detect eye movement and generate in real time X-Y Coordinate 2-Dimensional graph. The output data can be any serial or USB communication format that can be interfaced to an embedded system and not laptop or tablet.

user-c5fb8b 13 September, 2019, 07:50:21

@user-c87bad regarding calibration: 1) The calibration will never be perfect, as there is always noise in the data. Your eyes will never perfectly look at the markers all the time. And then there is completely natural noise in the processing pipelines for both detecting the exact marker positions and detecting your pupil positions. We try to keep these errors within a reasonable range, but reaching a perfect level of accuracy is not possible. Can you please report the errors that were reported at the end of the calibration? Specifically the angular accuracy? If you receive an accuracy that is not good enough for your use case, we can assist you with fine-tuning your set-up to get the most out of the the product. Most importantly there is a distinction between the 2d and 3d pupil detector. The 2d detector can be more accurate, but is very sensible to slippage and movement of the eye-tracker on your head. 2) In the calibration phase we are fitting our model with samples of known relationships between pupil positions and gaze targets. The error you are seeing is the difference between the predicted gaze targets with the new model and the actual gaze targets as detected in the images. The calibration test does not re-fit the model (i.e. it does not influence the data afterwards, yes), but just resamples data and re-calculates the error between predicted vs actual targets. As you see, the targets are at different positions for the calibration test in order to test for generalization of the model to different screen positions as well. Again, the error will be different every time, as there is noise in the data and you will never look at the targets in the exact same way.

user-c87bad 13 September, 2019, 09:55:53

@user-c5fb8b Thank you soooo much!!! The degree error difference can be around 0.5 . Do you think it's a reasonable value? Besides, I also tried some experiments to compare the difference between 2d and 3d. And I am now using the screen marker calibration since my purpose is to track the gaze of audience watching a video. For 2d, the error is around 0.3, and test error is around 0.5. For 3d, the error is around 1.5, and the test error can be around 0.5 too but it also can be around 1.5. It seems 3d is not so stable? Also, I am not sure about the effects when head moves after calibration. For 2d, if I move my head by a large range, the gaze points are totally wrong. But for 3d, the gaze points seems to be similar. Why that happened? So I have to keep my head fixed all the time even after my calibration?

papr 13 September, 2019, 09:59:56

@user-c87bad Do you see the green rectangle after the calibration? This is the calibration area. 2d is very accurate within that area, but looses accuracy outside of it. 3d is able to generalize outside of the rectangle.

user-c87bad 13 September, 2019, 10:33:36

Yes, I saw that. So does it mean that if I want to move my head after calibration, I can only use the 3d mode?

papr 13 September, 2019, 11:05:41

@user-c87bad you can move your head with both, but gaze is only accurate for both if the subject is looking at objects within the calibration area. Note: The area is relative to the scene camera.

user-5543ca 13 September, 2019, 12:26:22

Hello, we are considering the purchase of Pupil Invisible (it looks awesome btw), but I couldn't find info about gaze accuracy and precision on the specifications page.

user-5543ca 13 September, 2019, 12:26:44

Could you please give more information about it before we make the decision to purchase, thanks!

papr 13 September, 2019, 12:40:32

@user-5543ca I think @user-97ae1e is the right person to answer all your questions around a purchase.

user-5543ca 13 September, 2019, 12:41:38

@papr Will @user-97ae1e respond here, or should I rather contact him separately?

user-97ae1e 13 September, 2019, 12:43:54

Hi @user-5543ca πŸ‘‹ if you could contact info@pupil-labs.com we'll be happy to answer any questions you have

user-5543ca 13 September, 2019, 12:49:29

Sure!

user-deafd0 13 September, 2019, 13:32:14

hi I have a question about the surface heat map ,the heat map picture it to small so can any one helping me with that . Thanks

user-deafd0 13 September, 2019, 13:34:04

That the heat map

Chat image

user-deafd0 13 September, 2019, 13:34:22

@papr

papr 13 September, 2019, 13:34:27

@marc

papr 13 September, 2019, 13:43:24

Sorry, the previous message was incomplete. @user-deafd0 Please see the pinned messages. You should see a post by @marc that explains the topic of heatmap sizes and how they relate to the smoothness factor.

user-123d16 13 September, 2019, 13:44:16

Hello, I am having trouble installing the software for the Pupil Core. I've downloaded v1.15 and ran PupilDrvInst.exe as admin, but a window only flashes for a brief second, then nothing. I cannot see the drivers in the Device Manager

user-123d16 13 September, 2019, 13:44:35

On Windows 10, by the way

papr 13 September, 2019, 13:45:09

@user-123d16 Is it possible that your user name contains utf-8 characters or spaces?

user-123d16 13 September, 2019, 13:45:57

My username? For my computer login?

papr 13 September, 2019, 13:46:04

correct

user-123d16 13 September, 2019, 13:46:24

No, my username is only letters

papr 13 September, 2019, 13:46:47

We had a series of issues related to paths on Windows that is why I asked

user-123d16 13 September, 2019, 13:47:08

As far as I know, it is just 4 letters, no spaces

user-123d16 13 September, 2019, 13:48:06

It would be what I see in C:\Users\, correct?

papr 13 September, 2019, 13:48:07

Great! Please try running Capture directly with admin rights. If I remember correctly, one needs to start the PupilDrvInst.exe with a set of arguments. Capture does that for you.

user-123d16 13 September, 2019, 13:48:22

So, pupil_capture.exe?

papr 13 September, 2019, 13:48:26

Correct

user-123d16 13 September, 2019, 13:48:34

I cannot find that file in what I've downloaded

user-123d16 13 September, 2019, 13:48:42

I can find run_capture.bat

wrp 13 September, 2019, 13:48:50

You downloaded source code

papr 13 September, 2019, 13:48:56

Will is correct

user-123d16 13 September, 2019, 13:49:14

How do I download the correct files?

papr 13 September, 2019, 13:49:20

Please find the bundled release here: https://github.com/pupil-labs/pupil/releases

user-123d16 13 September, 2019, 13:50:48

Okay, I see what I did wrong.

user-123d16 13 September, 2019, 13:51:01

I clicked where it said ".zip" on the side.

user-123d16 13 September, 2019, 13:51:09

Thank you

papr 13 September, 2019, 13:51:12

Ah that happens πŸ™‚

user-c5fb8b 13 September, 2019, 14:56:20

@user-c87bad Following up: 0.5 is a good error value. With the 3d detector you might even experience errors as high as 2.0 - 3.0 from time to time, which would be on the upper end of still being reasonably accurate for a lot of tasks. The 3d model needs some time to adjust itself to your eye at the beginning, you are recommended to look around your field of view for a couple of seconds before starting the actual calibration.

user-deafd0 13 September, 2019, 15:45:35

@papr @marc sorry I didn't understand the solution

user-deafd0 13 September, 2019, 15:45:54

can you help me with that

user-c87bad 13 September, 2019, 16:22:14

@papr Just let you know about the questions I sent before about the surface detection. I tried your new version v1.15 and found the same problem happens too.

papr 13 September, 2019, 16:24:59

@user-c87bad please be aware, that our email response said v1.16, not v1.15. πŸ™ƒ

papr 13 September, 2019, 16:25:49

But yes, we were able to reproduce the issue in v1.15, too.

user-c87bad 13 September, 2019, 17:58:17

@papr oh, yeah, I just saw the email. Thank you soooo much for that.

papr 13 September, 2019, 18:01:24

@user-deafd0 We will update our documentation to include a clearer explanation of the new heatmap size mechanism and come back to you with a draft to get your feedback on it.

user-a6cc45 15 September, 2019, 13:41:01

Hi, I have a question about fixation detection algorithm. In Salvucci & Goldberg article there is said that dispersion is calculated like this:

D = [max(x) – min(x)] + [max(y) – min(y)]

but what are the units? Degrees?

papr 15 September, 2019, 20:19:26

@user-a6cc45 The input unit depends on the unit of D. D is a threshold set by the user. So if you want to set a maximum dispersion of e.g. 2 degrees, then x and y need to be in degrees as well.

user-876467 16 September, 2019, 09:49:22

Hello. After, I load the recordings folder into Pupil Player, I obtain an Export folder. Inside that folder there are many .csv files. I have a few queries regarding them. Would anyone please explain these in simpler terms.

1) export_info.csv: Absolute Time range= -30.427443. What does negative value signify?

2) pupil_positions.csv: First entry of pupil_timestamp column= -634260.347985. What do (negative) values like these mean?

3) world_timestamps.csv: First entry of # timestamps [seconds]= -634260.353483

4)export_info.csv: Start_Time (System) = 1568293713.8267 ; Start Time (Synced)= -634260.3791

What can be inferred from 2) , 3) , 4)?

Thanks.

user-c9d205 16 September, 2019, 10:25:40

I canno't seem to set up the glasses so that my pupils are in the frame (even when using the given extendors), Any tips?

user-3e42aa 16 September, 2019, 10:48:38

Is there a schedule for 1.16 bundle release?

user-dfeeb9 16 September, 2019, 14:03:32

Can I just check, is the bytestring to start and stop a recording through zmq still 'R' and 'r'?

user-dfeeb9 16 September, 2019, 14:05:23

Because it seems I can no longer stop recordings with 'r' edit: It seems running recv_string() after each send fixes it, but i'll verify

papr 16 September, 2019, 14:11:19

@user-dfeeb9 yes, receiving after sending is vital in a request - reply pattern, e.g. Pupil Remote

user-dfeeb9 16 September, 2019, 14:13:34

Thanks for the info, that solved it! @papr

user-dfeeb9 16 September, 2019, 14:14:04

haven't updated my setup in a long time so I should've done my due diligence to check, thanks for the help

papr 16 September, 2019, 14:16:03

@user-3e42aa We are waiting for the release notes to be finalized, afterwards we will publish.

papr 16 September, 2019, 14:18:10

@user-c9d205 Please contact info@pupil-labs.com in this regard. Please add screenshots of the eye windows such that we can get a feeling for the setup. Ideally, you could also include a picture of you wearing the device.

user-180ab4 16 September, 2019, 16:46:25

Hi all - the fps indicator for both of the detected cameras in my pupil-hmd device rarely climbs higher than about 30 fps - either with with the default setting of 120Hz or any similar value. I had thought this was a usb3.0 bandwidth limitation but having installed a new usb card I'm wondering if I'm missing something - any help appreciated, cheers!

user-f7489b 16 September, 2019, 17:22:39

Hey @papr, I'm new to pupil and I'm trying to find the exact system requirements for running all the software without a hitch. The only information that i could find related to requirements was that the software needs windows 10 64 bit to run, but there was nothing in regards to hardware recommendations. Would you happen to have any information related to this?

papr 16 September, 2019, 20:41:52

@user-180ab4 this is likely due to insufficient CPU resources

papr 16 September, 2019, 20:44:22

@user-f7489b it is difficult to define a minimum hardware spec since the required performance is dependent on a lot of factors, monocular vs binocular mapping, camera frame rates, active plugins etc. Do you require realtime pupil detection?

user-180ab4 17 September, 2019, 11:10:08

@papr many thanks for the response, I've had a quick look and the CPU doesn't go much above 60% when running the pupil capture and Unity together. Is it possible that there's something else that I could be missing?

papr 17 September, 2019, 11:11:33

@user-180ab4 With which program did you look up the cpu usage?

papr 17 September, 2019, 11:12:27

We also see performance issue when running Unity and Capture in parallel on the same machine. I would recommend a two-computer setup, connected via ethernet

user-dfeeb9 17 September, 2019, 13:42:48

Hi @papr and @user-e7102b , I've had to update my lab setup after windows nuked my drivers for pupil. As a result, I developed a more up to date variant of our previously used pupil_middleman. This time there's no need for intermediate comms with matlab since I think you guys have that natively now. Nonetheless, if you think this might be useful, I can add it to pupil_community.

https://github.com/mtaung/eye_socket

user-dfeeb9 17 September, 2019, 13:43:04

Additionally, can I ask the status of a batch exporter? is somebody working on it/does one exist?

papr 17 September, 2019, 13:50:11

@user-dfeeb9 Nice work. Here some feedback: - I think it would be nice if you added instructions for installing msgpack and zmq. You can copy it from the pupil docs - The readme has a typo in start_recordign - readme is incorrect, the default clock epoch is not fixed

By default, pupil trackers will use UNIX epoch timestamps. - notifications are not triggers in Capture, annotations are. Notifications are a specific type of message that is relayed to all plugins.

user-dfeeb9 17 September, 2019, 13:51:50

Oh some of those are pretty egregious- do notifications still exist in the latest version of pupil? I might have missed the examples in pupil-helper

papr 17 September, 2019, 13:52:41

@user-dfeeb9 Notifications exist yes. Annotations were special notifications in an older version. This changed, They became their own type of messages.

user-dfeeb9 17 September, 2019, 13:53:23

Copy that, I'll read up and implement this properly

user-dfeeb9 17 September, 2019, 13:58:33

regarding batch exporter, if someone's working on it I'd be happy to help. If nobody's made such a tool yet, I'd be happy to work on it but may need to consult you quite a bit through the process

papr 17 September, 2019, 14:04:20

@user-dfeeb9 re batch exporter: We are currently not working on this. Here the reason why we have disabled it:

In previous versions of Player, there was only pre-recorded data that was generated by Capture. The batch exporter could just open a recording and run the export procedure on the pre-recorded data.

With the introduction of Offline Pupil Detection and other offline data generators, the data is not necessarily generated yet, if a recording is exported by the batch exporter.This also means that the batch recorder would not work consistently.

The solution would be an architecture like this:

  • The batch exporter becomes an external script that interacts with Player via the IPC
  • The batch exporter opens one recording at a time
  • The batch exporter loads all plugins based a previous configuration
  • Data generation plugins start offline data generation procedure, potentially triggering a chain of other calculations
  • When all calculations are done, start the configured exporters
  • When all exports are done, proceed from th ebeginning with a new recording
papr 17 September, 2019, 14:07:18

I would be happy to assist you/consult with each of the steps πŸ‘

user-dfeeb9 17 September, 2019, 14:08:12

I'm very interested in getting my feet wet with further dev work outside of my lab stuff. I'll have to double check that i don't get told off for allocating too much time on this, but I want to get back to you about this so I'll stay closely in touch

user-dfeeb9 17 September, 2019, 14:09:50

sorry for the many questions - regarding pupil clock, in my last experimental runs I had epoch timestamps for my recordings and did not exert much effort to use time-sync. What is the recommended standard protocol for handling the internal clock in pupil? My understanding is that they went off of system time assuming you were running everything from one machine. Pupil docs also mentioned in the past that the timestamps ran off of clock_monotonic

user-dfeeb9 17 September, 2019, 14:10:08

If you have standard advice, I'll gladly make sure my docs are in line

papr 17 September, 2019, 14:10:54

This would be a major contribution that would be appreciated by a lot of people in the community.

user-dfeeb9 17 September, 2019, 14:11:13

Yeah I've had at least several people talk to me about a batch processor being very needed

user-dfeeb9 17 September, 2019, 14:11:19

I know @user-e7102b has wanted one in the past

papr 17 September, 2019, 14:17:49

@user-dfeeb9 Re clock: Generally, I would recommend to follow Pupil time, since other plugins require it to follow their own clocks, e.g. LSL Relay. Pupil can't follow two different clocks.

I would mirror hmd-eyes behaviour: https://github.com/pupil-labs/hmd-eyes/blob/master/plugin/Scripts/TimeSync.cs#L66-L71

It calculates the offset between local and pupil time, and applies it to all local clock time measurements

user-180ab4 17 September, 2019, 14:20:29

@papr - this number was from task manager, however I have the same low framerate issue when running pupil Capture without Unity. I have tried both the 2 camera pupil for Vive and the 3 camera mobile version and am getting the same results on this box, about to try with ubuntu to see if this makes a difference but any thoughts in the meantime greatly appreciated.

papr 17 September, 2019, 14:25:28

@user-180ab4 were you recording during these tests? Trying to see if it might be I/O related

user-180ab4 17 September, 2019, 14:51:03

@papr - no, this is just calibration so far. Disclaimer: I'm a complete novice with eye tracking so could be doing something stupid. I've tried switching from 2d to 3d models and playing around with the exposure settings etc. The specific problem that I'm trying to address is poor tracking accuracy, this has led me to question why the framerate was poor (which may or may not be the primary cause)

papr 17 September, 2019, 14:57:56

@user-180ab4 Can you make a screen recording of you doing a calibration with the 3 camera headset (Pupil Core)? Ideally, you would do a manual marker calibration such that one can see the world and eye windows in the screen recording.

Please restart with defaults from the general settings menu before doing the screen recording.

user-180ab4 17 September, 2019, 14:59:15

@papr - OK will do, it's likley that this will be some time next week due to time constraints, thanks for the assistance tho.

papr 17 September, 2019, 14:59:46

@user-180ab4 sure thing πŸ‘

user-b23084 17 September, 2019, 15:36:42

"PLAYER: overread 8" error? is there a repository of error codes somewhere?

papr 17 September, 2019, 15:37:14

@user-b23084 Is this the full error message? I have not encountered this before.

user-b23084 17 September, 2019, 15:37:41

it shows up in Player while viewing the recording

user-b23084 17 September, 2019, 15:39:17

one recording has just this and a "6" version, another recording has this and error y=22 x=11

user-b23084 17 September, 2019, 15:41:02

also, anyone know if you can manually adjust the calibration? It's almost never spot on.

papr 17 September, 2019, 15:44:07

@user-b23084 This looks like ffmpeg/pyav output to me. Probably nothing critical.

Re manual correction: You can rerun the calibration offline if it was included in the recording, and apply a manual offset.

Also, there is an expected gaze estimation error, that depends on the quality of the calibration. The accuracy visualizer menu has text field that tells you the accuracy of your most recent calibration/validation

user-b23084 17 September, 2019, 15:52:03

how to apply the manual offset?

papr 17 September, 2019, 15:55:09

@user-b23084 Please see our video tutorials about offline calibration: https://www.youtube.com/watch?v=_Jnxi1OMMTc&list=PLi20Yl1k_57rlznaEfrXyqiF0sUtZMMLh

user-b23084 17 September, 2019, 16:01:30

video tutorials! perfect! why didn't I think to ask youtube? LOL. I see how to do a calib. after the fact, will look for manual offset video

user-b23084 17 September, 2019, 16:01:32

Thank you

papr 17 September, 2019, 16:02:27

@user-b23084 I do not think there is a dedicated video, but it is mentioned in one of the videos. Sorry that I cannot be more specific right now.

papr 17 September, 2019, 16:08:06

@here We are pleased to announce the latest release of Pupil software v1.16! We highly recommend downloading the latest application bundles: https://github.com/pupil-labs/pupil/releases/tag/v1.16

user-ef48e1 17 September, 2019, 16:37:16

hello everyone I have an issue and I don't know if I am doing something wrong.

user-ef48e1 17 September, 2019, 16:37:40

I am using eye-tracking glasses connected with the pupil lab's mobile. While I am starting my recordings everything looks ok. The cameras works and the location is activated so I proceeded to my recordings but at the end of those no videos played. Although the time of recording is correct the screen of videos is black. Could anyone help me?

papr 17 September, 2019, 16:37:46

This is your issue, right? https://github.com/pupil-labs/pupil/issues/1646

papr 17 September, 2019, 16:38:20

I will respond there for persistence of the response

user-ef48e1 17 September, 2019, 16:38:38

Dear papr I just submit this issue

user-ef48e1 17 September, 2019, 16:38:54

ok thank you I will be waiting

user-fd5a69 17 September, 2019, 20:11:55

Hello everyone, I have an issue with the pupil capture program. I'm using a windows pc. I downloaded and unzipped the pupil labs 1.16-29 release, and an error message showed up when I was trying to open the pupil capture program: DLL load failed: The specified model could not be found. Do I need to install some dependencies before I run the program?

user-31df78 17 September, 2019, 20:12:33

Yep, I came here just for this issue. I was under the impression that after unpacking the .7z from github I could just run the capture exe, but it's not working.

papr 17 September, 2019, 20:13:08

@user-31df78 @user-fd5a69 I will try to replicate it

user-31df78 17 September, 2019, 20:13:29

https://i.imgur.com/qMYVFso.png

user-31df78 17 September, 2019, 20:13:38

here's an ss of when i try to launch the exe as admin

papr 17 September, 2019, 20:18:48

I was not able to reproduce the issue πŸ˜•

user-fd5a69 17 September, 2019, 20:19:01

Yeah, I got the exact same error message. I have a very old version (1.2.7) of pupil labs installed on my computer and we've been using that for a long time. Do you think that may be the cause of this error?

papr 17 September, 2019, 20:19:23

No this should not be the error

user-31df78 17 September, 2019, 20:19:32

Do I need a specific version of Python installed on my computer?

papr 17 September, 2019, 20:19:35

The error messgae indicates a missing dll

papr 17 September, 2019, 20:19:50

No you do not. The bundle should come with everything you need

papr 17 September, 2019, 20:20:16

The question is which one is it.

papr 17 September, 2019, 20:21:40

@user-31df78 @user-fd5a69 Would you be willing to install this tool in order to debug the issue? https://github.com/lucasg/Dependencies/releases

It is able to list all dll dependencies and shows which ones are missing.

user-31df78 17 September, 2019, 20:24:13

I downloaded Release and opened pupil_player in it, if that's correct

papr 17 September, 2019, 20:25:04

@user-31df78 The download includes multiple exe files, which one did you use?

user-31df78 17 September, 2019, 20:25:15

DependenciesGui.exe

papr 17 September, 2019, 20:25:21

Perfect

user-31df78 17 September, 2019, 20:25:31

https://i.imgur.com/FLVDy4p.png

user-31df78 17 September, 2019, 20:25:34

not sure if that's helpful

papr 17 September, 2019, 20:26:07

You should be able to expand these two rows

papr 17 September, 2019, 20:26:10

on the left

papr 17 September, 2019, 20:26:15

Like a tree view

user-31df78 17 September, 2019, 20:26:30

yep yep

user-31df78 17 September, 2019, 20:26:41

not sure what i'm looking for though

papr 17 September, 2019, 20:28:56

ok, instead of Capture/Player.exe please open the detector_3d*.pyd file with the dependencies tool

papr 17 September, 2019, 20:29:12

You can find it in the pupil_detectors subfolder from the Pupil release

user-31df78 17 September, 2019, 20:29:49

https://i.imgur.com/jiDqEHu.png

papr 17 September, 2019, 20:30:07

ok, I got the first two, I got too

papr 17 September, 2019, 20:30:28

The third one seems to be the one in question

user-31df78 17 September, 2019, 20:31:07

seems like a visual c++ runtime library issue

user-31df78 17 September, 2019, 20:31:11

let me try to download that again

papr 17 September, 2019, 20:31:42

@user-31df78 I was about to suggest that. I wonder why I got that installed and you don't πŸ€”

user-fd5a69 17 September, 2019, 20:31:43

That is what I get

Chat image

papr 17 September, 2019, 20:32:00

@user-fd5a69 yeap, same file

user-31df78 17 September, 2019, 20:35:02

Seems to have fixed the problem, it's updating drivers now.

papr 17 September, 2019, 20:35:24

@user-31df78 great! From where did you download the missing file?

user-31df78 17 September, 2019, 20:35:39

The actual capture window is Not Responding but I'm sure it'll fix itself once the drivers are done installing, thanks for the troubleshooting help papr

papr 17 September, 2019, 20:35:56

Yes it should continue afterwards, correct

papr 17 September, 2019, 20:36:29

@user-31df78 Did you download x64 or x86?

user-31df78 17 September, 2019, 20:36:33

Perfect! thanks so much, it's 100% working now.

user-31df78 17 September, 2019, 20:36:43

x64

user-31df78 17 September, 2019, 20:37:05

My OS is 64-bit

user-fd5a69 17 September, 2019, 20:37:30

I downloaded the x64 version and mine's now working. Thank you! @papr

papr 17 September, 2019, 20:38:22

@user-31df78 @user-fd5a69 thank you so much for your helping finding the issue! I will update the release notes and add the link above for users that encounter the same issue

user-31df78 17 September, 2019, 20:42:56

πŸ‘ Thanks for your help again

papr 17 September, 2019, 20:52:44

Release notes updated

user-e7102b 17 September, 2019, 22:20:19

Hey @user-dfeeb9 I've been using some batch processing scripts here https://github.com/PierreYvesH/batchExport to export basic pupil and annotation data.

user-e7102b 17 September, 2019, 22:21:03

I was planning to adapt these to also export pupil position (and surface position) data, but didn't get round to it yet.

user-e7102b 17 September, 2019, 22:21:18

You might find these useful

user-31df78 18 September, 2019, 00:29:35

Sorry, quick question - for the Pupil Core, are there specific USB-C/USB3 cables that I should look for, if I need a longer cable than the one that arrived with the glasses?

user-dfeeb9 18 September, 2019, 10:14:29

Apologies for adding to the question backlog, but suppose someone wanted to access pupil frame data (diameter for example) through an external script in python, my understanding is that I'd have to read from the IPC backbone to achieve that - is that right?

papr 18 September, 2019, 10:15:34

@user-dfeeb9 that is correct

papr 18 September, 2019, 10:16:06

@user-755e9e can you give a recommendation for @user-31df78 ?

Sorry, quick question - for the Pupil Core, are there specific USB-C/USB3 cables that I should look for, if I need a longer cable than the one that arrived with the glasses?

user-755e9e 18 September, 2019, 11:12:02

@user-31df78 The max length i would recommend is 5 meters, since a longer cable might have power loss and require a power repeater.

user-072005 18 September, 2019, 11:55:06

Hey, so I'm trying to run a study outside and have the pupil core glasses hooked up to a laptop, but if I close the laptop the feed from the cameras is gone until I reopen the laptop. Is this just how it is? Do I need to change some setting on my laptop?

papr 18 September, 2019, 12:02:02

@user-072005 This heavily depends on the operating system that you are using. But yes, you should be able to turn off this feature.

user-072005 18 September, 2019, 12:07:07

Oh yes, I found it. Sorry, should've been able to figure that out by myself. Thanks

user-5fa537 18 September, 2019, 14:38:07

Friends- I just received my core + RealSense bracket. I'm using a 415i on macOS for the moment and I'm not getting any good response from the scene camera. I can get the 415 dev software to give me data/images (but, strangely, not the imu under macOS but OK under Windows... strange) - I looked for some 'getting started' notes re the RealSense, but what I found was related to the v1 toolkit that doesn't support the 4xx series. Am I missing something?

user-5fa537 18 September, 2019, 14:39:59

Also- someone must have done a little research on a light, USB-C non-3D scene camera that could be used in place of the heavy sensor for times where that would be desirable? The recommended solution of $900Β± for a second frame, wiring + camera is OK, but maybe a litte too much $ for my budget right now :/

papr 18 September, 2019, 14:40:03

@user-5fa537 Hey, they macOS bundled application does not support the Realsense D400 series

papr 18 September, 2019, 14:40:34

pyrealsense (the underlying driver module for the camera) is not supported on macOS, sadly

user-5fa537 18 September, 2019, 14:40:36

@papr Ah- OK- should I fork/buld/

user-5fa537 18 September, 2019, 14:40:43

Ahhh

user-5fa537 18 September, 2019, 14:41:01

Yeah, I tried to build that and got nothing

papr 18 September, 2019, 14:41:08

Please use Windows or Linux in this case.

user-5fa537 18 September, 2019, 14:41:34

Plus, want the IMU data for the ML-based correction / gaze stuff Gabe & Rakshit did

papr 18 September, 2019, 14:41:59

From the camera?

user-5fa537 18 September, 2019, 14:42:21

OK- so, if I get the pyrealsense 'built' at some point I might be able to fold into the macOS stuff?

user-5fa537 18 September, 2019, 14:42:22

yes

papr 18 September, 2019, 14:43:22

if I get the pyrealsense 'built' at some point I might be able to fold into the macOS stuff? Yes, if you run Pupil from source IMU data for the ML-based correction You will have to gather that data by yourself. I would recommend to write a custom plugin that accesses this data

user-5fa537 18 September, 2019, 15:01:43

ok perfect! that seems like a good learning experience

user-5fa537 18 September, 2019, 15:02:15

(slash 'productive procrastination' vector)

user-5fa537 18 September, 2019, 15:02:59

"Honey- the deck needs power washed and stained before winter snow...",

user-5fa537 18 September, 2019, 15:03:07

"Can't talk now- debugging"

papr 18 September, 2019, 15:03:18

That is the right spirit πŸ˜„

user-5fa537 18 September, 2019, 15:03:50

She's lived with that conversation since about 1984... she's used to it.

user-5fa537 18 September, 2019, 15:04:33

OK- I'll dl the pupil source, see what I can get going wrt the 4xx software in my spare time (need it for another application anyway)

user-5fa537 18 September, 2019, 15:04:46

then fork/pull/etc

user-5fa537 18 September, 2019, 15:06:58

so it looks like you only run a 'master' branch w/ no dev or anything right now?

papr 18 September, 2019, 15:07:17

Correct

user-5fa537 18 September, 2019, 15:08:30

so, new to discord, but, looks like the πŸ’» software-dev channel is more relevant for further stuff on this?

user-5fa537 18 September, 2019, 15:08:48

(so many collaboration environments, so little time...)

papr 18 September, 2019, 15:09:19

Yes, feel free to move dev related questions there

user-5fa537 18 September, 2019, 15:18:20

cool and thanks- my lightweight usb-c camera question still stands πŸ™‚

user-5fa537 18 September, 2019, 15:20:12

Also- I'm a little curious about why the frame with the mount for the sensor was >$ than the frame w/ the scene camera in the first place? Is it the USB-C hub and all that? Not complaining, just counterintuitive

user-fd5a69 18 September, 2019, 20:11:24

[email removed] I'm trying to use the pupil remote plugin but the pupil capture couldn't find my phone (which has the latest version of pupil mobile installed / downloaded from google store). The command prompt says that: Devices with outdated NDSI version found. Pleas update these devices. Is there any newer version of pupil labs that I can download?

user-fd5a69 18 September, 2019, 20:22:16

Sorry I mean newer version of pupil mobile

papr 18 September, 2019, 20:27:24

@user-fd5a69 Just to make sure: You connected a Pupil Core headset to a phone running Pupil Mobile, Pupil Mobile correctly detects all cameras, and the phone is in the same wifi as the computer running Capture?

papr 18 September, 2019, 20:28:34

What color does the Android app icon have?

user-fd5a69 18 September, 2019, 20:29:40

Chat image

user-fd5a69 18 September, 2019, 20:30:56

Yes, the app looks like this. It says it's commspec v3 and it seems that the latest version of pupil capture is detecting v4

papr 18 September, 2019, 20:34:13

I think this error message is a bit confusing. It comes up in my case too but correctly detects my pupil mobile device.

papr 18 September, 2019, 20:34:43

In the Pupil Mobile Manager in Capture, what does it say? No hosts found?

user-fd5a69 18 September, 2019, 20:41:29

Yeah, it says no host found. I also have an old version of pupil capture installed on my pc and it can detect my phone.

papr 18 September, 2019, 20:42:04

The same pc as the one running the newer version?

user-fd5a69 18 September, 2019, 20:42:10

Yes

papr 18 September, 2019, 20:43:55

Since v1.16 is working for me, and the older version is working for you, my best guess is that the newer Capture version did not get the same network rights as the old one.

papr 18 September, 2019, 20:44:24

You know, this network popup that you get when you launch a newly downloaded Capture version for the first time.

papr 18 September, 2019, 20:45:17

Could you try downloading it again, and making sure that you give the appropriate permissions during the first launch?

user-fd5a69 18 September, 2019, 20:57:18

Hi, I just tried and now it works! Thank you very much!

user-9d7bc8 18 September, 2019, 21:41:52

Hello. My team has used the HoloLens Binocular Add-On for a past project, and now we'd like to use Pupil Labs equipment again, but the Core this time. Unfortunately, we're on a limited budget, so I was wondering, is it possible to purchase just the frame for the Core and use the HoloLens Add-On cameras on it? Thank you.

papr 18 September, 2019, 21:42:32

@user-9d7bc8 please contact info@pupil-labs.com in this regard

user-9d7bc8 18 September, 2019, 21:42:59

Will do, thank you.

wrp 19 September, 2019, 02:33:00

@user-9d7bc8 we received your email, will follow up today. Thanks.

user-c87bad 19 September, 2019, 11:45:52

Hi! I've tried the new version v1.16 and it worked really good. But I still find there are several times the screen turned to be like gray without images. Also I found that always happened with low thredsholds(in pupil player even though I have no idea what's that mean). And also in v1.16 there is no info.csv any more. So where is the time information like synced time going to?

user-deafd0 19 September, 2019, 15:49:01

hi, how can I get a heat map for the fixation ?

papr 19 September, 2019, 17:41:02

@user-c87bad If you see gray frames in Player, it means that there was a disconnect or a lack of frames during the recordings. Not sure which thresholds you are referring to. Could you elaborate on them?

v1.16 introduces the info.player.json file. See the release notes for more information.

papr 20 September, 2019, 08:00:52

@user-deafd0 Heatmaps are not generated from fixations but from the raw gaze data. If you wanted heatmaps from fixations, you would have to generate them yourself. For this, you will require the fixations_on_surface_<name>.csv file which can be exported with the surface tracker.

user-c87bad 20 September, 2019, 09:20:50

@papr Thresholds is from pupil-player where you can see the marker cache, progress bar and other information. Sorry, but how can I make sure there is no disconnection during the recordings? What can cause that?

user-deafd0 20 September, 2019, 10:40:24

@papr thank you for the replay ,how can I generated by my self ?

papr 20 September, 2019, 10:41:49

@user-deafd0 By reading the csv data, and using the programming language of your choice. Sorry, but I do not have a code example for this right now.

papr 20 September, 2019, 10:43:28

@user-c87bad I am still not sure which thresholds you mean. Could you make a screen recording to demonstrate your issue?

Frame drops can happen either if the system is under high load, or if there is a physical disconnection, e.g. due to a loose cable.

user-c87bad 23 September, 2019, 08:39:40

@papr Sorry for the late reply. And you can see the term 'thredsholds' at the bottom. As you can see the green line of this term, frame drops happened when the line is thin. (Sorry for that I don't know how to describe it). Do you think the blink detection may influence the frame? Because I saw the thredholds from the blink detction.

Chat image

papr 23 September, 2019, 08:57:16

@user-c87bad Correct, the thresholds are related to blink detection. Blink detection is not responsible for grey frames, which I had difficulties to understand your reference. Blink detection only considers pupil data, which is based on valid frames only. Gray frames are inserted as placeholders for dropped frames. The number of gray frames should not change based on any settings made in Player.

papr 23 September, 2019, 08:58:14

@user-c87bad But maybe, I am having trouble understanding your reference again. Could you post a screen shot of what you are referring to as gray frame. Maybe, I am talking about something else than you. πŸ™‚

user-c87bad 23 September, 2019, 09:06:51

The red circles are when region that frame drops happened. You can compare the thickness of the line with part of good frames.

Chat image

papr 23 September, 2019, 09:10:27

@user-c87bad Could you seek the video to that location, and make an other screenshot?

papr 23 September, 2019, 09:17:35

@user-c87bad You can see the frame drops in the eye0 fps actually much better. I am not sure why the visualization is different for the blink detector, too. As said above, I can assure you, that the offline blink detector does not cause frame drops after the effect.

user-c87bad 23 September, 2019, 09:27:03

@papr I am trying to get a zoom in image for that part. What do you mean eye0 fps? If you mean the left upper terms, as I saw, there is no difference when frame drops happened.

papr 23 September, 2019, 09:29:40

@user-c87bad I am referring to the orange, discontinued timeline at the top, in your last screenshot. The discontinuities indicate frame drops in the eye0 video.

user-c87bad 23 September, 2019, 09:32:27

@papr Yeah, you are right!!!

papr 23 September, 2019, 09:35:40

@user-c87bad btw, this is what I refer to as a gray/dropped frame in the world video.

Chat image

user-c87bad 23 September, 2019, 09:38:32

@papr Yeah, I got it. Thank you.

user-c87bad 23 September, 2019, 10:00:11

@papr Sorry to bother you again. Is here any code that I can use to overlay apriltags into videos like the previous version? I can't find them on the website. Or I just overlay the image is fine?

papr 23 September, 2019, 10:01:45

@user-c87bad How did you overlay the privious markers?

papr 23 September, 2019, 10:02:38

I think, if you just overlay an image of the apriltag marker, you should be fine.

user-c87bad 23 September, 2019, 10:03:37

@papr For previous markers, you provide a 'encode_marker(mId)' function, so I just scale them and overlay into my frames.

papr 23 September, 2019, 10:06:48

@user-c87bad Just use these images https://github.com/AprilRobotics/apriltag-imgs/tree/master/tag36h11

user-c87bad 23 September, 2019, 10:08:04

@papr Yeah, I am now trying to overlay them.πŸ™‚

user-141bcd 23 September, 2019, 13:10:26

@papr : I just updated Pupil Capture to v1.16. (I'm on Wibndows 10.) Whenever I quit a recording, it crashes. Any idea what could cause this? Here's the traceback:

user-141bcd 23 September, 2019, 13:10:39

Chat image

user-141bcd 23 September, 2019, 13:11:09

v1.12 (which I used before) still functions normally

papr 23 September, 2019, 13:11:53

@user-141bcd "Whenever I quit a recording" Do you do so by clicking the red R button? Or by closing the application?

user-141bcd 23 September, 2019, 13:12:27

I can replicate it by clicking the red "R".

papr 23 September, 2019, 13:12:37

@user-c5fb8b any idea what could cause this issue?

user-141bcd 23 September, 2019, 13:13:10

i originally realized though when only interacting with capture via "remote" (from Unity)

papr 23 September, 2019, 13:15:10

@user-141bcd are you using the Unity Streaming feature, i.e. are you streaming the unity content as video to Capture?

user-141bcd 23 September, 2019, 13:15:43

nope

papr 23 September, 2019, 13:16:06

@user-141bcd can you specify which video backend is active during the recording?

user-141bcd 23 September, 2019, 13:16:08

I just tried again, also there it seems to be caused by the ending of the recording.

papr 23 September, 2019, 13:17:24

@user-141bcd At the end of the recording, the video header is written. If it is misconfigured, it crashes. You can see at the top of the screenshot, that it is not able to recognize the yuv420p pixel format

user-141bcd 23 September, 2019, 13:17:26

I did not change the defaults. But where do I look this up?

user-141bcd 23 September, 2019, 13:17:50

(refering to the video backend)

papr 23 September, 2019, 13:19:04

@user-141bcd ok, thanks for the clarification. There is a camera icon on the right. What does the overlay say when you hover it with your mouse

user-141bcd 23 September, 2019, 13:21:00

"Recorder" πŸ˜‰

papr 23 September, 2019, 13:21:43

@user-141bcd I meant the icon above, the icon with the front facing camera

papr 23 September, 2019, 13:22:04

Please also click on it and make a screenshot of the menu

user-141bcd 23 September, 2019, 13:22:29

ah yep:

user-141bcd 23 September, 2019, 13:22:44

Chat image

papr 23 September, 2019, 13:24:22

@user-141bcd I can reproduce the error

user-141bcd 23 September, 2019, 13:25:27

(dunno if that's relevant)

Chat image

papr 23 September, 2019, 13:39:58

@user-141bcd This is the fix for the issue: https://github.com/pupil-labs/pupil/pull/1653 We will update the current bundle to include the fix later today

user-141bcd 23 September, 2019, 13:41:47

πŸ‘Œ that was quick. Then I'll grab the bundle later. Thx!

papr 23 September, 2019, 15:39:00

@here We have updated the current Pupil release. It includes a series of fixes for Pupil Capture and Pupil Player. We highly recommend downloading the latest v1.16-43 bundle: https://github.com/pupil-labs/pupil/releases/tag/v1.16

papr 23 September, 2019, 15:43:39

@user-141bcd Please see the fixed release above.

user-04c0dd 24 September, 2019, 00:54:02

Trying to get pupil capture to build from source on windows 10. I followed the directions in the pupil docs after completing the instructions it appeared the surface tracker module required the pupil apriltags module So i installed pupil's apriltags library However there seems to be a problem with the dll Whenever it tries to open the dll it returns an error "The specified module could not be found" I have verified it is passing in the correct dll file

papr 24 September, 2019, 06:11:26

@user-04c0dd Apriltags depends on the pthreads library. I can recommend to install Apriltags via pip install pupil-apriltags

user-bd800a 24 September, 2019, 12:07:12

Hi, I have had some problem with camerar recognition with an old version of pupil capture (1.8.26): the detected cameras are 2 (world) and twice camera 0. I've tried uninstalling and re-installing the drivers but that never solved it.

papr 24 September, 2019, 12:15:10

@user-bd800a Does your Pupil Core headset actually have two world cameras, or is there an other camera detected as world camera by mistake?

user-04c0dd 24 September, 2019, 12:31:11

@papr I used pip install pupil-apriltags and it looks like that creates a dll: 'libapriltag.dll' that the bindings.py file tries to open. However, when it opens this dll it returns the "specified module could not be found". Do you think this is a pthreads issue? Do you recommend installing pthreads separately?

papr 24 September, 2019, 12:31:41

@user-04c0dd It should be installed automatically πŸ€”

papr 24 September, 2019, 12:32:31

@user-04c0dd Could you please try this:

pip install -U pupil_apriltags==0.dev0
user-04c0dd 24 September, 2019, 12:43:56

@papr that worked! thanks!

papr 24 September, 2019, 12:46:22

@user-04c0dd great, thank you. We are just finalising our first apriltags release. @user-c5fb8b will let you know, as soon as it is available.

user-04c0dd 24 September, 2019, 12:47:07

@papr great thanks!

user-c5fb8b 24 September, 2019, 13:06:10

@user-04c0dd pupil_apriltags v1 has now been officially released. please update once more to the latest version with

pip install -U pupil_apriltags
user-31df78 24 September, 2019, 17:33:24

Hi, does anyone have any suggestions, tips, or anecdotes about calibrating the Core headset for children? So far the single marker calibration has worked best on adult testers but seems dependent on how well they can follow the instructions on head movement.

papr 24 September, 2019, 17:35:53

@user-31df78 I think I have seen approaches that used a toy/rattle and the natural features/manual annotation feature in Offline Calibration.

user-31df78 24 September, 2019, 17:52:05

@papr Thanks! Compared to single marker though, I seem to get worse accuracy the farther their gaze from the center of the world camera. Should the tester's head be stationary? And is there any recommended pattern for how I should be pointing features out?

papr 24 September, 2019, 17:57:14

@user-31df78 Gaze estimation works best within the calibration area (green area visible after a calibration in Capture). 3d gaze estimation is able to generalise outside of this area up to some degree. Remember that this area is relative to the scene camera, and therefore independent of the subject's head movements.

If you expected your subject to perform more extreme eye movements, e.g. their head is fixed by a chin rest, then you should perform a calibration that covers more of the subject's field of view.

If the subject's head is not fixed, you can get away with a smaller calibration area due to the human's center bias. We are more likely to move our head towards the point of attention instead of performing extreme eye movements.

user-31df78 24 September, 2019, 18:07:12

@papr Thanks a lot. I didn't know that the area was relative to the camera and I'll see if the accuracy drop is related to when the gaze travels outside of the calibrated area.

papr 24 September, 2019, 18:07:49

@user-31df78 Accuracy loss also happens if pupil detection fails, e.g. during blinks

papr 24 September, 2019, 18:08:18

See the gaze data's confidence value as a quality indicator.

user-31df78 24 September, 2019, 18:11:43

@papr I haven't looked at the blink detection plugin yet. Does it only track and annotate the recorded data with estimated blink times, or does it have any bearing on accuracy?

papr 24 September, 2019, 18:12:48

@user-31df78 It actually works the other way around. It uses the pupil confidence to find blinks. The assumption is that during a blink the confidence suddenly drops and increases again in both eyes.

papr 24 September, 2019, 18:13:27

@user-31df78 Player has a "minimum data confidence" slide in the general settings. Only data with higher confidence than this threshold will be visualised in Player.

user-31df78 24 September, 2019, 18:21:17

@papr Sweet, that addresses a lot of my issues. As always thanks for your help.

user-deafd0 25 September, 2019, 11:51:23

hi , what the mean by x_norm ,y_norm and x_scaled ,y_scaled in surface gaze data? ,also if I need to generate a heatmap from the gaze data what the data that I need to use it to gnerate the heatmap ? @papr

papr 25 September, 2019, 11:55:12

@user-deafd0 x_norm/y_norm are normalised values, i.e. (0,0) refers to the bottom left of the surface and (1, 1) to the top right of the surface.

The *_scaled are just the *_norm values, multiplied by the surface size.

x_scaled = x_norm * surface_width
y_scaled = y_norm * surface_height
papr 25 September, 2019, 11:56:53

@user-deafd0 To generate a heatmap, you need a 2d histrogram function, e.g. numpy.histogram2d() [1].

Then you pass x_norm and y_norm, and the number of bins of your choice.

[1] https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram2d.html

user-deafd0 25 September, 2019, 11:58:33

Thanks @papr

user-bda130 25 September, 2019, 18:33:28

@papr Hello - I am wondering if with V1.16-43, is there no longer an ability to edit new calibration names and trim markers on Player?

user-bda130 25 September, 2019, 18:34:31

There used to be an option to change the name of the calibration and set specific trim markers

Chat image

papr 25 September, 2019, 18:36:28

@user-bda130 you can only edit them in the original recording. As the description says, this calibration was copied into this recording.

papr 25 September, 2019, 18:36:57

@user-bda130 if you create a new calibration the options to name etc should be available.

user-bda130 25 September, 2019, 18:47:04

@papr This is the original recording from capture being imported into player

user-bda130 25 September, 2019, 18:48:14

When I create a new calibration it calls it a default calibration and has a similar message about being from another recording

Chat image

papr 25 September, 2019, 18:49:05

@user-bda130 I think this might be a bug. Please create a Github issue.

user-bda130 25 September, 2019, 18:49:51

okay, will do! thanks

user-bd800a 26 September, 2019, 07:05:02

@papr There is 1 world camera but one of the pupil cameras is detected as the same as the other pupil camera

papr 26 September, 2019, 07:07:00

@user-bd800a so if you open the uvc manager menu on the right, the drop down lists three cameras, of which one is duplicated?

papr 26 September, 2019, 07:13:34

@user-bd800a and did you have this problem from the start with this headset, or did it appear after using it successfully?

user-bd800a 26 September, 2019, 07:22:36

It appeared after a few uses

user-bd800a 26 September, 2019, 07:22:51

Pupil Camera 0 is duplicated

papr 26 September, 2019, 07:25:52

@user-bd800a I have not encountered this issue before. Maybe the camera names were changed in the driver by accident? Maybe you can rename them in the Device Manager?

user-bd800a 26 September, 2019, 07:26:16

I will try and get back to you

user-c9d205 26 September, 2019, 09:04:55

Can someone point me out to a code example that takes pictures of the main camera just like this one: https://github.com/pupil-labs/pupil-helpers/blob/0df77b47cebd49a6c35b6769da483c115a626836/pupil_remote/recv_world_video_frames.py But with the eye gazes on the image or just the coordinates of the gazes on the image?

papr 26 September, 2019, 09:38:52

@user-c9d205 You would have to render the gaze data yourself into the image.

Use this script to receive gaze data: https://github.com/pupil-labs/pupil-helpers/blob/0df77b47cebd49a6c35b6769da483c115a626836/pupil_remote/filter_messages.py#L24

Change the subscription from pupil. to gaze. in line 23/24.

The gaze's norm_pos field is the normalised position within the world image. Use this with flip_y=True to calculate the pixel location: https://github.com/pupil-labs/pupil/blob/master/pupil_src/shared_modules/methods.py#L535-L546

Then you can use opencv, to draw e.g. circle at that location: https://docs.opencv.org/2.4/modules/core/doc/drawing_functions.html#circle

user-0ee84d 03 February, 2021, 12:59:50

I’m following this

user-c9d205 26 September, 2019, 09:47:19

@papr Thak you very much

user-c9d205 26 September, 2019, 09:48:51

Will I have to synchronize time stamps or something of the sort? If so, how is it done?

papr 26 September, 2019, 09:51:00

Since this is a realtime application, you need to make a trade off between correctness and delay.

The easiest implementation renders the most recent gaze to the most recent world video frame. This generates the lowest delay but might be inaccurate in terms of matching timestamps.

The alternative is to buffer world video images and gaze data and match it before rendering the data.

papr 26 September, 2019, 09:51:48

@user-c9d205 If your goal is an accurate visualization, I would recommend to do the matching after the data has been recorded.

user-c9d205 26 September, 2019, 09:53:33

Is there a way to reset the timestamps so they start from 0 when I want them to?

user-c9d205 26 September, 2019, 09:55:58

Great, thank you so much

user-deafd0 26 September, 2019, 14:04:11

Hi I have the y_norm and y_scaled as a negative value can I know why these values is negative @papr

Chat image

papr 26 September, 2019, 14:16:22

@user-deafd0 In this case, the gaze was not on the surface. Capture/Player attempt to map the corresponding (world) gaze to surface gaze, as soon as the surface is detected. Since the surface is only one part of the subject's field of view, and the subject is free to look to somewhere else, it is possible that the mapped surface gaze is not within the surface bounds (x_norm or y_norm not in range [0, 1]).

papr 26 September, 2019, 14:16:46

The on_srf column tells you if the mapped gaze was within the surface's bounds.

user-deafd0 26 September, 2019, 14:19:13

Thanks one more question how can I get the surface fixation? @papr

papr 26 September, 2019, 14:20:18

@user-deafd0 Fixations on surfaces are exported in the fixations_on_surface csv file. Remember to activate the Offline Fixation detector before running the export.

user-deafd0 26 September, 2019, 14:20:51

Okay thank you so much @papr

user-fed837 26 September, 2019, 14:51:36

I'm a new user and can anyone please tell me how to get values from API for Unity?

papr 26 September, 2019, 14:52:42

@user-fed837 Hi πŸ‘‹ Please see the https://github.com/pupil-labs/hmd-eyes/ documentation for details. If there are further questions, please post them to our dedicated πŸ₯½ core-xr channel. πŸ‘

user-fed837 26 September, 2019, 14:53:05

Thank you!

user-7f9568 26 September, 2019, 16:43:23

Hello, I am trying to assemble the DIY version of Pupil Labs for an academic school project using the Logitech c270 as a world camera and as a Genius Facecam 1000x eye camera, however pupil capture does not recognize either, my question is if they should be necessarily the models cited in the document such as the Logitech C520 for World and the Microsoft HD6000 for eye?

user-7f9568 26 September, 2019, 16:44:49

Chat image

user-7f9568 26 September, 2019, 16:45:30

I would appreciate all your comments about it

user-4bf830 26 September, 2019, 16:55:45

Hey, @papr ! I recently downloaded the new Pupil package, but they won't even open on my computer. Would this be because I still have the old version downloaded as well? The old one still works perfectly for me, at the moment, but I wanted to go ahead and snag the update.

papr 26 September, 2019, 17:00:59

@user-7f9568 Please follow steps 1-7 for all of your cameras: https://github.com/pupil-labs/pyuvc/blob/master/WINDOWS_USER.md

Afterwards launch Capture and the cameras should appear in the UVC Manager.

papr 26 September, 2019, 17:03:04

@user-4bf830 Does only a specific app not work for you, or do all of them not work. Please check the capture.log file.

Please see the release note updates, if the mentioned DLL import error also applies to your case: https://github.com/pupil-labs/pupil/releases/tag/v1.16

user-4bf830 26 September, 2019, 17:03:50

Thanks, I'll start poking around and see what I can figure out. Neither Capture nor Player will open for me right now, but I'll check that note and take a look.

user-121d2c 26 September, 2019, 17:31:01

Hi, I have some questions about the exact settings. I want to capture the highest res, easiest to analyze videos in my clinical trial, which uses the 200 hz pupil add on attached to the vr htc vive, but I’m not sure what settings are best with regards to resolution, exposure, contrast, and frame rate. I want the highest quality videos I can get so that some of the data-orientated folks outside of the hospital can analyze them as best they can. Should I set the resolution to 400, the exposure to auto, and the frame rate to 200? Thanks!

user-7f9568 26 September, 2019, 17:49:00

@papr Tanks, now is working

user-121d2c 27 September, 2019, 00:51:05

Are there any disadvantages to setting resolution to the highest setting possible?

user-121d2c 27 September, 2019, 02:09:58

I see that the website says this: As a first step it is recommended to check the eye camera resolution as some parameters are resolution dependent. For fast and robust pupil detection and tracking we recommend using the default resolution settings. For 200hz eye cameras the default resolution is set to 192x192 pixels. If you have an older 120hz eye camera, the default is 320x240 pixels.

What are the specific disadvantages to setting a Lower resolution?

user-222750 27 September, 2019, 14:05:41

Hi, Is there a way to sync up RGB video data with the Realsense depth data in Pupil software (or a plugin)? I've recorded the RGB part at 720p and 30fps, but the depth video is at 14.71fps or something like that.

user-4bf830 27 September, 2019, 15:37:21

I’m trying my mobile for the first time on an older Nexus, why would this be occurring?

Chat image

user-4bf830 27 September, 2019, 15:54:28

The phone I'm using runs android 8.1.5

user-4f5461 27 September, 2019, 17:48:10

Is it possible to use pupil capture 1.16 with Invisible?

user-4f5461 27 September, 2019, 17:48:55

The pupils are showing up, but the world viewer camera is listed as "unknown" and won't connect. Are the drivers not available yet?

user-deafd0 28 September, 2019, 15:54:53

hi how can I get the fixation number for the surface from the surface fixation file ? @papr

user-ac350a 30 September, 2019, 07:42:52

Hi, can I estimate objects distance to the eye with pupil?

papr 30 September, 2019, 07:59:28

@user-ac350a any object, or do you mean the distance of the gaze point?

user-ac350a 30 September, 2019, 08:03:11

tanks for answer, no only gaze point

papr 30 September, 2019, 08:15:27

@user-ac350a The 3d detection and mapping point includes a field gaze_point_3d. It is the gaze's location in the scene camera's coordinate system. You can calculate it's distance to the subject using the vector norm.

user-deafd0 30 September, 2019, 10:20:39

hi how can I get the fixation number for the surface from the surface fixation file ? @papr

papr 30 September, 2019, 10:23:54

@user-deafd0 My apologies. Do you mean the ID or total number of fixations on the surface?

papr 30 September, 2019, 10:25:33

@user-4f5461 You would have to manually install the drivers. But Capture is not designed to work with Pupil Invisible. Please use the Pupil Invisible Companion device to make recordings with your Pupil Invisible glasses.

user-deafd0 30 September, 2019, 10:26:29

I mean total number and what mean by fixation Id @papr

papr 30 September, 2019, 10:26:59

@user-4bf830 difficult to say what the issue is. What is the billboard device? Have you connected it and the headset via a USB hub?

papr 30 September, 2019, 10:28:17

@user-deafd0 fixations have an ID, just for differentiation. Don't worry about them.

Just count the rows in the file whose on_srf field is true.

user-deafd0 30 September, 2019, 10:29:50

Thanks @papr

user-4bf830 30 September, 2019, 12:25:44

@papr device is a nexus 6p running android 8.1.5, it is running through a usb hub.

papr 30 September, 2019, 12:27:41

@user-4bf830 My guess is that the errors are coming from Pupil Mobile recognising the USB Billboard Device as sensor, and attempting to interact with it, which fails. Either disconnect the device, or ignore the errors. The errors should not interfere with other Pupil Mobile's functionality.

user-4bf830 30 September, 2019, 12:32:08

It actually caused the recording to end and crash after about 5 seconds

user-4bf830 30 September, 2019, 12:32:47

I’ll try finding a workaround for the cable. What do you suggest? Do I need a cable that doesn’t have a USB end to be able to use it?

papr 30 September, 2019, 12:33:18

Oh, in this case, connect the Pupil Core headset directly to the phone using a proper USB-C -> USB-C cable.

user-4bf830 30 September, 2019, 12:33:29

Ok, thank you!

papr 30 September, 2019, 14:08:39

@here We have two announcements we'd like to share!

πŸ“£ New docs.pupil-labs.com πŸ“£ We are happy to share our newly redesigned documentation website - https://docs.pupil-labs.com/ - Highlights and changes: - Structure: We have separated user and developer facing documentation. Hopefully this will make reading docs more user-friendly. We have also organized both user and developer docs by product/product family. Hopefully this makes it easier to navigate and locate relevant information. - Search: One of the things we really liked about the old docs site with everything on a single page was that you could easily do cmd/ctrl + F to search. You can now use the search bar in the website. Currently this is only search through headings. But very soon you will be able to use the search for full-text search across the website. - New product documentation: We have added docs for Pupil Invisible! - Contributions: We welcome feedback and contributions! You can make PRs to https://github.com/pupil-labs/pupil-docs

papr 30 September, 2019, 14:09:57

πŸ“£ Pupil Software Release Update πŸ“£ We updated the current Pupil Core software release. It fixes a recent issue with Offline Calibration reported by @user-bda130. We highly recommend downloading the latest v1.16-71 bundle: https://github.com/pupil-labs/pupil/releases/tag/v1.16

user-121d2c 30 September, 2019, 14:11:27

Hi @papr are there any issues associated with upping resolution to the maximum allowable on a htc vive add on? Would the csvs still generate? The crisper resolution helps with manual analysis but I don’t want to change it it it messes with other components

papr 30 September, 2019, 14:20:52

@user-121d2c Sorry that I have not responded to your previous question yet.

We usually recommend the lower resolution since it requires less resources in terms of USB bandwidth and cpu usage for pupil detection. In cases where pupils exceed a specific pixel size, we even have experienced improved pupil detection using the lower resolutions.

user-f497a5 30 September, 2019, 16:06:29

anyone having probelems opening 2018 files with v1.16-71?? Help! In the meantime, will revert to v1.10...for now. Thanks in advance. Regards

papr 30 September, 2019, 16:06:48

@user-f497a5 Hey πŸ™‚

papr 30 September, 2019, 16:07:05

What is the exact error message that you get?

user-f497a5 30 September, 2019, 16:07:48

H @papr - thanks for quick response. err.. one sec

user-f497a5 30 September, 2019, 16:09:33

@papr

user-f497a5 30 September, 2019, 16:09:50

@papr

Chat image

papr 30 September, 2019, 16:10:05

There you go πŸ™‚

user-f497a5 30 September, 2019, 16:10:11

lol..

user-f497a5 30 September, 2019, 16:10:36

Fair enough. #mustmakenewrecordings.

user-f497a5 30 September, 2019, 16:10:48

@papr thanks though

papr 30 September, 2019, 16:11:05

@user-f497a5 There might be a manual work around.

papr 30 September, 2019, 16:11:28

Also, you should be ok with using Player v1.15 with these recordings

papr 30 September, 2019, 16:11:55

Could you share the info.csv file of one of these 2018 recordings?

user-f497a5 30 September, 2019, 16:12:32

Ah, I see. Sure thing. will look into it... sure why not... 2 ticks

user-450945 30 September, 2019, 21:53:29

Is anyone aware of using pupil to map or quantify heterotropic diplopia?

End of September archive