Thanks Neil! I'll look into this and see how it goes 🙂

Attempting to use Python 3.7 virtual env in Anaconda but this error comes up

/any third party camera

Hi PupilTeam!

I may have found a problem in the PupilCloud. I had a browser window open for a few days with an active login. During these days, another person logged into the account and created at least one new workspace.

The problem was that this newly created workspace was not displayed to me when I clicked on the workspace overview.

I logged out and logged back in, and then the workspace was listed. I hope this problem description makes sense.

Best regards!

@marc Yes. I refreshed and since it didn't work i felt forced to log out 😄

Atm I struggel with another issue and I hope you have some ideas what could be the reason.

I installed `pip install pupil-labs-dynamic-rim' on Ubuntu 22.04 + python3.10-tk (Note: we tested the script also on a windows version and it works fine)

The problem is about get_corners.py regarding the cv2 call: cv2.namedWindow('...') While executing this line, it 'seems' that the script just freezes. Every line after is not executed anymore. I tried a simple debug, but also the debugger stops without information on that line.

I used venv and global env. I tried multiple cv2 versions with two different python versions. I tried to change the order of the code. I modified get_corners.py just having a cv2.imshow() which will also cause a freeze/hold

On another computer with same OS version the same issue happens. I assume some problem with the window manager vs cv2 communication?

Any help and ideas are welcome! For now I will try to build a workaround to enter the point coordinates, but I would love to have a fix for this issue.

EDIT: I also tested a simple snippet 'test.py', 1) just importing cv2, 2) creating a namedwindow, 3) showing an Image and using 4) waitKey to verify that cv2 is working. And that worked!

Does anyone know why my powershell script isn't starting pupil capture recording ```ps

$PupilCapturePath = "C:\Users\scrim\OneDrive - Aston University\Documents\Work\Research Associate\pupil\pupil_src" $PupilCaptureScript = "$PupilCapturePath\main.py" $PupilCaptureModule = "$PupilCapturePath\capture"

$pupilCaptureProcess = Start-Process -FilePath python -ArgumentList "$PupilCaptureScript" -PassThru

while (!(Test-NetConnection localhost -Port 50020 -InformationLevel Quiet)) { Start-Sleep -Milliseconds 100 }

$null = Start-Process -FilePath python -ArgumentList "-c", "import sys; sys.path.append('$PupilCaptureModule'); from capture import Pupil_Capture; Pupil_Capture()" -Wait

$null = Start-Process -FilePath python -ArgumentList "-c", "from capture import Pupil_Capture; Pupil_Capture().start_recording()" -Wait

$k4aRecorderPath = "C:\Program Files\Azure Kinect SDK v1.4.1\tools\k4arecorder.exe" Start-Process -FilePath $k4aRecorderPath -ArgumentList "output.mkv"

$duration = 180 # 3 minutes

Start-Sleep -Seconds $duration

$null = Start-Process -FilePath python -ArgumentList "-c", "from capture import Pupil_Capture; Pupil_Capture().stop_recording()" -Wait

Stop-Process -Name "k4arecorder"

Stop-Process -Id $pupilCaptureProcess.Id ```

My third party camera is recording correctly however Pupil Capture isn't recording when I send the arguments over

Sorry I'm a bit confused about the network API, how would I get it to work?

Hi Dom, I just tried what you suggested and everything is working and running very smoothly! Thank you for the help 🙂

Hi! Using Pupil Remote, how can I control if Pupil Core are not connected to the PC?. I tried with NetMQException but it's not working. I am using C#.

Thank you!

Hey@user-6eeda2! Responding to your question https://discord.com/channels/285728493612957698/285728493612957698/1118526143436423219 here. Well done on getting that up and running. The next phase will be to transform the pupil data from eye camera to scene camera coordinates. This will require calibration and gaze mapping. - Calibration marker choreographies are handled in shared_modules > calibration_choreography - Calibration and gaze mapping can be found in shared_modules > gaze_mapping Suggest you start looking in those places. The general order of operations would be something like: 1. Present marker and collect reference locations 2. Run bundle adjustment to find the physical relation of the cameras to each other (for 3d gaze pipeline) 3. Use this relationship to linearly map pupil vectors that are yielded by the 3D eye model. Of course, Pupil Capture software handles all of this automatically, so you need to figure out your own implementation of the source code. Good luck!

Hi Dom, everything was fine but I realised when I tried outputting the recording the camera footage would end up corrupt due to the Powershell script not ending the recording gracefully (CTRL +C to stop recording with Azure Kinect camera), what would you advice? I'll add my code below.

According to https://learn.microsoft.com/en-us/azure/kinect-dk/azure-kinect-recorder, you can specify a recording duration when you startup k4arecorder.exe. You should definitely use that since you have a fixed recording duration

Ahhh true I'll give it a shot thanks

Hey there! Fast question, if I have the pupil labs VR inside an HMD looking at a single wide display (set around 15 cm away from the eyes). Are there ideal settings for this. I'm not getting the best calibration so I am wondering if this is because of the eyes converging? I know my description may not be specific, but what are your general thoughts on this?

Hi @user-8619fb ! Would you be able to share the eye videos, or even better a recording including the calibration choreography, such that we can offer better feedback?

If you prefer, you can share it with data@pupil-labs.com rather than here.

Also could you provide some additional information regarding which HMD are you using, which pipeline you use (2D or 3D), etc, that would be quite helpful.

Generally speaking, and if you are using the 3D gaze pipeline, did you roll your eyes before doing the calibration to obtain a good model fit? Check out this video on how to get this model. https://youtu.be/_1ZRgfLJ3hc

Additionally, and specially in VR, it might be helpful to set a ROI to delimit where the pupil can be, in case you have dark areas in the image that might confuse the algorithm. https://drive.google.com/file/d/1tr1KQ7QFmFUZQjN9aYtSzpMcaybRnuqi/view?usp=sharing

Hello, I want wonder something. I'm unity developer, so never work on these stuff, dont judge me please 😄 I want to change eye tracking algorithm in this pupil software but I want to able to use rest of the program, is it possible if I develop plugin? will it be override pupil detection? I'm looking for easy way to make that because otherwise I have to interfere program and send my data from outside but this is not easy and I'm not sure I can do that in short time

@user-d407c1 Creating a new environment with fresh installation of realtime-api works. However, it receives the video frames only once. It is not able to stream the videos continually from the device. Here is the code which is the async sample that's in the documentation.

async def main():
    async with Network() as network:
        dev_info = await network.wait_for_new_device(timeout_seconds=5)
    if dev_info is None:
        print("No device could be found! Abort")
        return
    print(dev_info)

    async with Device.from_discovered_device(dev_info) as device:
        status = await device.get_status()
        sensor_world = status.direct_world_sensor()
        if not sensor_world.connected:
            print(f"Scene camera is not connected to {device}")
            return

        restart_on_disconnect = True
        async for frame in receive_video_frames(
            sensor_world.url, run_loop=restart_on_disconnect
        ):
            bgr_buffer = frame.bgr_buffer()
            print(bgr_buffer.shape)
            draw_time(bgr_buffer, frame.datetime)
            cv2.imshow("Scene Camera - Press ESC to quit", bgr_buffer)
            if cv2.waitKey(1) & 0xFF == 27:
                return


def draw_time(frame, time):
    frame_txt_font_name = cv2.FONT_HERSHEY_SIMPLEX
    frame_txt_font_scale = 1.0
    frame_txt_thickness = 1

    # first line: frame index
    frame_txt = str(time)

    cv2.putText(
        frame,
        frame_txt,
        (20, 50),
        frame_txt_font_name,
        frame_txt_font_scale,
        (255, 255, 255),
        thickness=frame_txt_thickness,
        lineType=cv2.LINE_8,
    )


if __name__ == "__main__":
    with contextlib.suppress(KeyboardInterrupt):
        asyncio.run(main())

the print(bgr_buffer.shape) is executed only once. any fix?

Thank you again @user-cdcab0 @nmt

How would I find the endpoint address for the usb cameras?

Hello again!!! Instead of the pye3d model I am trying to get the gaze estimation by doing 2d gaze mapping because I have seen that it is easier to obtain. If I understand correctly what gaze mapping does is to obtain a linear regression model with the normal pupil position as input and the reference gaze as target data from the calibration data and then, predict the gaze with the fitted model and the normal eye position from the data provided by pupil-detectors. If I calibrate with the single marker of the pupil labs software I get the file "notify.pldata". My idea is to read the data from this file with a python file in order to get the norm pos and the reference gaze, without having to create a calibration choreography from scratch, and then make the adjustment and predict. My question is, with notify.pdldata can I get the gaze and norm pos data to adjust the model?

thanks it works!

Hi there, I'm currently trying to modify the ui slider to have smaller step sizes from the gaze offset correction plugin here https://gist.github.com/papr/d3ec18dd40899353bb52b506e3cfb433. However, when i set the ui.Slider's step size to 0.001 from 0.01, it doesn't change anything. When I set a value 0.059 for example in the field above the slider in pupil player, it rounds itself to 0.6. How can I change this? Thanks

Hi, there. I am trying to install pyuvc from source (https://github.com/pupil-labs/pyuvc) on a Jetson Orin Nano (Architecture: aarch64, OS: Ubuntu 20.04). I followed the steps to install dependencies including libusb and libturbo-jpeg. I also add the target user to the plugdev group (logout and back in). Then, when I pip install ./pyuvc, I got the following error: ERROR: Could not build wheels for pupil-labs-uvc, which is required to install pyproject.toml-based projects Any ideas to solve this problem? Thanks.

@user-cdcab0 Thank you very much for your quick reply. Here is the full output after pip install ./pyuvc.

Also, out of curiosity, why do you want to run pyuvc from source?

@user-cdcab0 I tried git submodule update --init --recursive and found the libusb and libuvc folders are empty. I manually downloaded them and then successfully installed the pyuvc. Thank you very much. Actually, I wanna run the pupil core on a Embedded System on Module device (Jetson Orin Nano), but no apps are available for aarch64 platform. So I tried to install from source (I successfully installed from source and ran the pupil_src/main.py capture https://github.com/pupil-labs/pupil/blob/master/pupil_src/main.py on a Windows PC.) and manually installed all dependencies following the requirements (https://github.com/pupil-labs/pupil/blob/master/requirements.txt). When I tried to run the main.py capture on the Jetson Orin Nano, however, an error is reported. < Error calling git: "Command '['git', 'describe', '--tags', '--long']' returned non-zero exit status 128." output: "b'fatal: No names found, cannot describe anything.\n'" Traceback (most recent call last): File "main.py", line 38, in <module> app_version = get_version() File "/home/yi/pupil-master/pupil_src/shared_modules/version_utils.py", line 84, in get_version version_string = pupil_version_string() File "/home/yi/pupil-master/pupil_src/shared_modules/version_utils.py", line 58, in pupil_version_string raise ValueError("Version Error") ValueError: Version Error >. Anyway, I will continue to figure it out and I'd like to see if you have any ideas . Thank you very much.

I'm having trouble getting a custom plugin to show up in the Pupil Capture's plugin manager. I've placed it in the pupil_src/shared_modules directory as my_plugin.py, I'm running Python 3.11.3, and I don't see any errors or warnings in the console when starting Pupil Capture.

Any idea why my plugin is not showing up in the plugin manager? Any help would be greatly appreciated!

@user-cdcab0 Yes， you are right. I tired to use git to clone the repo but failed. There seems to be a network problem. error: RPC failed; HTTP 408 curl 22 The requested URL returned error: 408fatal: the remote end hung up unexpectedly . Alternatively, I downloaded the zip files for the repo. Do you think this will be the cause of the error above?

Here are the files I downloaded

Ok it worked, sorry I missed that part in documentation

I have another question, if I write eye tracking detection as a plugin and with same output, when I enabled my plugin will it be override the old one, or it is possible to do that, before deep diving I want to be sure about that. or the best way is change the algorithm from source code?

Okey, first of all I never worked on python or these systems so my keywords might be wrong sorry for that. My professor keep says he wants to change eye tracking algorithm, and we are working on AR optics so I think he wants to know eye looking position so I assume he wants to detect pupil position with his own algorithm. Btw I said to him I can write program just covering for that part using emguCV but he wants to use rest of the program. So I think he wants to change pupil detector

is that make sense to you?

okey I will check that, ty for your help and patience

Hi everyone

I have a problem with PupilCore and Matlab, and I wanted to know if everyone has already dealt with this problem.

I'm using Windows 10 and Matlab 2021b aiming at controlling PupilCore from Matlab. I have followed every step from https://github.com/pupil-labs/pupil-helpers/tree/master/matlab. I have installed correctly zeroMQ and msgpack. I have compiled all mex files correctly using the make.m function.

But when I tried to run the test 'pupil_remote_control.m', is doesn't work. The error is "Invalid MEX-file '...\MexFileName.mexw64': The specified module could not be found". I have make sure that everything that must be in the Matlab path, is there.

I run also the tests of zeroMQ https://github.com/fagg/matlab-zmq, but the same error appears: 'Invalid MEX-file '...\MexFileName.mexw64': The specified module could not be found".

I have tried to install the mex files with different compilers (MinGW, Visual Studio 2022, Visual Studio 2019) but none of them worked.

I have also checked that in python, the code works. It seems that, somehow, it doesn't work only when I tried to use it in Matlab.

Has anyone of you faced this problem before? My intuition is that the issue is very simple to fix, but I have no clue.

Many thanks in advance for your help 🙂 Victor

@user-709f65 - I found a matlab-zmq fork that claims support for Matlab r2022b. Maybe you can give this a try? https://github.com/fpdotmonkey/matlab-zmq/

Hi, @user-cdcab0 - I tried to increase git's buffer, but still cannot git clone the repo (the same error: error: RPC failed; HTTP 408 curl 22 The requested URL returned error: 408). I will greatly appreciate it if you could provide some instructions to initialize a git repo and tag a version number.

thanks @user-cdcab0 I'll try...let's see if it works. I'll let you know. Thanks!!

Hi, @user-cdcab0. It works. The version error was solved. Thank you very much for your kindness and help!👍

Hi Pupil labs, I am currently following the online tutorial to map gaze onto 3D models with NerfStudio. I have created a complete reference image mapper enrichment and created a CONDA environment with all the dependencies etc as per the online instructions on a Windows 11 OS. I have been able to proceed through the tutorial up to the point of running Colmap on the frames with the command python pyflux/run_nerfstudio.py. When the Colmap feature extractor is ran, I am constantly met with the error message ‘qt.qpa.plugin: Could not load the Qt platform plugin "windows" in "" even though it was found. This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem’. I have attempted this on another computer and I am met with the same error message. Searches online have suggested adding platform plugins to PATH and/or the working directories but this has not worked for me. Any help on this would be greatly appreciated. Thanks in advance. Tim

Hi @user-37b952 ! Can you try

conda remove qt
conda remove pyqt 
conda install qt 
conda install pyqt

in your conda environment?

Hi! When I get the images of the eyes with pyuvc I get the left eye flipped horizontally and the right eye flipped vertically and horizontally (like the attached image). With 2d mapping I need to fit the linear model and then predict. I fit the linear model with the data from the notify.pdldata file that I get after doing a calibration and recording in Pupil Capture. This notify file stores the 2D and 3D detection values from the recording. My question is: how is the layout of the images when doing 2d or 3d detection? Is the detection done with the layout of the image I attached or is there any flipping? I need to know this to make the prediction. Thanks in advance! 😃

Did you guys make changes to the running from source tutorial? Seems much more straight forward now, unless I'm looking at a different page

I also forked matlab-zmq to build on the newest MATLAB (at least it works for me with 2023a), and also added precompiled mex files for macOS and Linux: https://github.com/iandol/matlab-zmq -- it works well for me.

For msgpack, I also found that I could get a ~2X performance improvement using the new dictionary type rather than containers.Map -- https://github.com/iandol/opticka/blob/master/communication/parsemsgpack.m#L188 & https://github.com/iandol/opticka/blob/master/communication/dumpmsgpack.m -- for a tight loop in e.g. PTB this is the difference between dropping frames or not...

msgpack parsing is still slower than in Python, I didn't find a good msgpack parsing alterative so far...

Hi, @user-6eeda2 - I dug into the weeds a bit, and I don't see any relevant flipping occurring. I think you can just take the images as they are

Nerf & QT in Win11

Hello, I am trying to use the Cloud API in Python to download zip files of my recordings. I can use the download links provided by api.get_recordings() to download the zip files of the raw data. However, I would like to download the files corresponding to clicking "Download Timeseries Data + Scene Video" in the web interface. How do I go about that using the Cloud API?

Hi, We have booked Pulpil Invisible in old version, but we did not have corresponding mobile phone and software. Is that okay for us to buy a mobile phone locally and buy the software from u?

Hi @user-4b9ddd 👋 ! Yes, you can buy a phone from other sources. Please note though that Pupil Invisible is only compatible with OnePlus 6 (Android 8 or 9) or OnePlus 8/8T (Android 11). The glasses will not work with other devices. As for the software, the Invisible Companion App is freely available on Google Play Store! I hope this helps!

Are you referring to the Invisible Companion App version? If so, when you get your new phone (see my previous message for compatible devices), you can download the Companion App from Google Play Store which should be the latest version of the App: 1.4.30prod.

Because we bought it long time ago but the demand belong to this version. Is compatible for the old version Pulpil Invisible (discontinued) with the the latest version of the App?

Sorry I didn’t notice that “you are unsettling”

So the Neon is not same with PI?

Appreciate for your kind reply!

So can we still book PI now?

If we only buy the related mobile phone, could you provide us 3C certificate for customs cleanrance in China?

Detail

By email？

Appreciate for ur patience and kindness