hi, I'm having some trouble with continuously receiving gaze data from pupil (via zmq/msgpack). I do receive data, but approx. every 5 seconds the stream is "paused" for half a second (blocking call to zmq_socket.recv()). I have a screen capture showing the problem, but it's too large to be uploaded here (8M). Can I send it to you somewhere else?

@user-82e7ab you can send it to data[at]pupil-labs.com

ok, it's on its way.

@user-82e7ab Which version of capture are you running? Are you sending any notifications to Capture?

this is what we send/receive

I did a time measurement exactly around zmq_socket.recv() (blocking call). These are runtimes in microsec: 2 118 3 21491 3 56 1 79 2 21228 2 2 185 185 20868 3 147 129 231 21519 3 2 2 1 196 22053 4 2 2 2 2 21219 2 108 94 154 21048 2 1 2 1 21244 90 394 237 3 124 20915 2 2 2 1 109 20927 213 3 3 2 3 20870 2 1 68 21449 2 2 1 2 1 371371 3 1 1 2 2 1 3 1 2 2 56 2 1 99 63 1 136 1 35 1 2 159 108 4 3 28 115 1 61 56 2 123 2 44 46 2 126 2 1 1 1 66 5109 588 571 671 54 356 3 147 2 456 185 2 421 1 2 2 3 2 1 190 36 40 185 2 455 222 283 210 255 1 1 2 1 44 1 192 33 146 1 126 21947 2 47 317 2 21101 2 1 1 3 1 1 21084 1 1 1

Most are only a few microsec. Some are around 20 millisec. But a few are really long-running.

Any idea what might be the problem?

@user-82e7ab Unfortunately, I did not have the time to reproduce the issue yet.

Did you try using Service instead of Capture?

I tried now, and there is no such issue in Pupil Service. That'll work for me, thx : )

Great!

Hi all,

I am currently facing a problem with a Pupil eye tracker for both eyes. I am trying to track both eyes separately with this tracker on a surface using the built-in surface plugin. The plugin provides a stream with a lot of data in it. In the reference implementation for surface tracking (https://github.com/pupil-labs/pupil-helpers/blob/master/python/filter_gaze_on_surface.py) only the first norm_pos in the stream is used, which is somehow a combination of both eyes. After digging through the code and the provided data stream, I found out, that there are actually norm_pos values for both eyes in the stream, hidden in the base data. My only problem with this is that I was not able to find any documentation about this and that I am not sure if those norm_pos values under base_data are actually the separated eye positions on the given surface. Does anybody have any experiences with tracking both eyes separately on a surface and might be able to help me out?

Best Regards Valkyrie

@user-fab1ad Hi, these are pupil norm_pos values, i.e. normalized to the eye camera coordinate system and therefore not useful for mapping onto a surface. It is required to map them into the world camera space first. You need to modify finish_calibration.py to instantiate a Dual_Monocular_Mapper instead of a Binocular_Mapper. This will result in separately mapped gaze points for each eye which will be mapped automatically onto the surface.

@papr thanks for this quick response, I will try it out. To be clear, do you mean the Binocular_Gaze_Mapper in the calibrate_2d_binocular method?

@@user-fab1ad Depends on the detection&mapping mode selected.

Can you give more detail on the difference between circle_3D_normal and center in pupil position and gaze_normal and eye_center_3d in gaze position data. They seem to be the same thing. Thanks.

@user-6b1b1f they are the same thing but in different coordinate systems.

Hi @papr thanks. So pupil position is each eye’s coordinate frame and gaze is the world’s. How do you know the separation between the eyes in the world?

@user-6b1b1f You mean how we know which pupil datum comes from which eye? Each eye process has an id that is inherited by the data.

Yes I know the ids. I think I know now. You don’t relate the eyes to each other. Each eye is zeroed in the world coordinate frame and the pupil distance between eyes is not needed or known. Correct?

What do you mean by zeroed?

hello, i'm working on Pupil Player 1.9.7 but vis polyline plugin is not working. I can't see any trace and no history of preview fixations. How can I fix it?

Hi guys i want to record the video using hmd eyes vr sample can anyone please share reference for this

@user-96755f 1. Make sure that you did not close the plugin by accident 2. Make sure you have gaze data (is the green dot visible?)

@papr yes, green dot is visible.

Hi, I have a question: I'm running a set up where I display a side by side world view in the HTC Vive, the worldcam is mounted on top of the Vive. I tried all calibration methods except HMD because you need Unity for that. I got the best results with screen calibration and 'detection & mapping mode' set to 2d. The middle is about right but outwards it's way off. This is what the Accuracy Visualizer looks like in 1.9.7. What would you advise to do? I can share a video. Thanks!!

@user-f441d4 Please share a video. I am not quite sure what you mean. You feed the world camera input to Capture and display it simultaneously in the Vive?

I get the world camera input from capture ZMQ, then display that in the vive side by side with opencv. How can I share the video with you?

Please send it to data@pupil-labs.com

Thanks, I sent it.

@user-f441d4 By way off, do you mean that the green frame does not go around the monitor?

@papr When I look at the monitor after calibration, it doesn't follow my eyes in the corners of the monitor, it does however in the middle.

@user-f441d4 try a manual marker calibration with a bigger area of calibration

Thanks, will try this later!

Hello, I am new to development and I'm having trouble running the pupil software.

I am getting this error:

LAPTOP-SOOMPRSR:~/pupil/pupil_src$ pip3 install msgpack Requirement already satisfied: msgpack in /home/jenchi/.local/lib/python3.5/site-packages (0.6.0) LAPTOP-SOOMPRSR:~/pupil/pupil_src$ python3 main.py service service - [ERROR] launchables.service: Process Service crashed with trace: Traceback (most recent call last): File "/home/jenchi/pupil/pupil_src/launchables/service.py", line 91, in service from file_methods import Persistent_Dict File "/home/jenchi/pupil/pupil_src/shared_modules/file_methods.py", line 25, in <module> ), "msgpack out of date, please upgrade to version (0, 5, 6 ) or later." AssertionError: msgpack out of date, please upgrade to version (0, 5, 6 ) or later. service - [INFO] launchables.service: Process shutting down.

Does anyone know the solution to this? Thank you

Hello, dave, this is a bug 😄

The source code is

assert ( msgpack.version[1] == 5 ), "msgpack out of date, please upgrade to version (0, 5, 6 ) or later."

We will fix it asap, or you can install version 0.5.6 of msgpack now

by pip3 install msgpack==0.5.6

BTW, what is your current version of msgpack?

Thank you Windson, my current version is 0.6.0

You are welcome

@user-7f26c3 The new version might not be compatible with our software. It has not been tested yet. Please use pip3 install msgpack==0.5.6 to install the correct version for now.

Hi! A question: for the pupil recorder, if I use the IPC/zeromq, what is the correct parameter to set a target directory when I send a recording.should_start notification?

@user-bab6ad PupilLabsCommands.SendRequestMessage(socket, new Dictionary<string, object> { { "subject", "recording.should_start" }, { "session_name", path }, { "record_eye", true } });

thx! Did not understood that the "session_name" is a path. Perfect, thx!

SendRequestMessage is my own convenience version

😃

😃

How can I pull data from pupil capture to my Java program?

Hello, sorry if I'm asking a silly question. I am trying to run main.py using the service function. This is the error message I am getting:
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ detector_2d.cpp:702:28: fatal error: opencv2/core.hpp: No such file or directory compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 service - [ERROR] launchables.service: Process Service crashed with trace: Traceback (most recent call last): File "/home/jenchi/pupil/pupil_src/launchables/service.py", line 98, in service import pupil_detectors File "/home/jenchi/pupil/pupil_src/shared_modules/pupil_detectors/init.py", line 18, in <module> build_cpp_extension() File "/home/jenchi/pupil/pupil_src/shared_modules/pupil_detectors/build.py", line 32, in build_cpp_extension ret = sp.check_output(build_cmd).decode(sys.stdout.encoding) File "/usr/lib/python3.5/subprocess.py", line 626, in check_output **kwargs).stdout File "/usr/lib/python3.5/subprocess.py", line 708, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['/usr/bin/python3', 'setup.py', 'install', '--install-lib=/home/jenchi/pupil/pupil_src/shared_modules']' returned non-zero exit status 1 service - [INFO] launchables.service: Process shutting down.

Does anyone know the cause of this?

@user-7f26c3 Did you install opencv?

@user-f27d88 yes I have

@user-7f26c3 Please be aware that you need to build Opencv manually if you run Ubuntu 16.04.

@papr I am indeed using ubuntu 16.04, but I did follow the instructions on the development docs here https://docs.pupil-labs.com/#linux-dependencies, so I am not sure why I am still getting this error

The 'fatal error: opencv2/core.hpp: No such file or directory' occurs during this operation when trying to run main.py: 'python3 setup.py install --install-lib=/home/jenchi/pupil/pupil_src/shared_modules'.

My core.hpp file is in ~/opencv/modules/core/include/opencv2. I don't know if the problem is simply that it cannot find the file, but I do not know where the script is looking for the file

if anyone has any insight please help 😥

more specifically, it occurs when building the 'pupil_detectors.detector_2d' extension

@user-7f26c3 Are you sure you install opencv correctly? Try import cv2 in python script

@user-f27d88 yes to my knowledge it is installed correctly, and importing cv2 works

Hi all, is it possible to get the world camera feed before it goes into pupil? I'd like to do some processing with as little lag as possible. Thanks!

@user-f441d4 This is only possible if you adapt https://github.com/pupil-labs/pyndsi/blob/master/examples/uvc-ndsi-bridge-host.py and use the Pupil Mobile backend in Capture.

Great, thanks for the quick answer!

Hi! Two more questions in my quest: If I want to enable specific plugins over the world process (blink and fixation detection) on recording, who do I do that? And what are the zero mq channels to subscribe to do get that data? I did not find it in the documentation and subscribing to all messages by b'' was a bit much chatter....

This has probably already been asked before, but is there a c++ library to produce dictionary style messages like in the unity demo?

does anyone know what this is?

@user-6c7426 this is non-critical. It only shows, that it was impossible to fit an eyemodel. Possibly caused by heavy slippage or no eye at all

So I tried IPC using this provided script: https://github.com/pupil-labs/pupil-helpers/blob/master/python/recv_world_video_frames.py

I ran Pupil Capture with the cameras plugged in to the USB port and receive an image on the Pupil Capture software. I tried modifying the script a bit so that the image will show up on the python interface:

left_cnt = 0 right_cnt = 0 while True: topic, msg = recv_from_sub() if topic == 'frame.world': recent_world = np.frombuffer(msg['raw_data'][0], dtype=np.uint8).reshape(msg['height'], msg['width'], 3) elif topic == 'frame.eye.0': recent_eye0 = np.frombuffer(msg['raw_data'][0], dtype=np.uint8).reshape(msg['height'], msg['width'], 3) left_cnt = left_cnt + 1 if(left_cnt >= 100): left_cnt = 0 cv2.imshow('left',recent_eye0) elif topic == 'frame.eye.1': recent_eye1 = np.frombuffer(msg['raw_data'][0], dtype=np.uint8).reshape(msg['height'], msg['width'], 3) right_cnt = right_cnt + 1 if(right_cnt >= 100): right_cnt = 0 cv2.imshow('right',recent_eye1)

if recent_world is not None and recent_eye0 is not None and recent_eye1 is not None:
        pass  # here you can do calculation on the 3 most recent world, eye0 and ey1 images

However, all I see are blank images (gray) which would suggest that the IPC is not working. I ran Pupil Capture and then ran this script (while leaving Pupil Capture running). Is there something else I am supposed to do? Note that I am doing this on a Windows 10 64-bit machine.