Hey, I saw that you can receive a response from the mobile device for things like IP address, Battery Level, Serial Number of Glasses and so on (more or less your discover_one_device() functionality in your real time API). Is there anywhere in the documentation noted which commands have to be send to the phone and what messages will be received in response? We can currently start/stop local recordings on the phone remotely from our tooling, but I would like to receive the additional information as well. Thanks!

No, it is a C++ integration which is not using any of your python scripts

The implementation with ndsi (and e.g. zyre related) is in a way outdated and does not support such things I guess?

Okay, thanks. Is there any data when the NDSI integration will no longer be working?

@user-fb5b59 That said, and given that there is no NDSI c++ reference implementation, I would not recommend investing any efforts into building a c++ integration for it.

I am able to connect to the ws://pi.local:8080/api/status and will receive the status message on connection. Is there any message I can send via WebSocket so I get a new status update in response?

Btw, did you open source the client somewhere?

Just one additional question regarding the IMU usage in the future with the new API: will it still be send in chunks of size 60 or directly live streamed?

@user-fb5b59 we will keep sending in chunks of 60 this is actually the IMU doing the chunking when sending the data via i2c to the USB bridge not the companion app or the network api host.

Hi folks, I recently changed from pupil core to pupil invisible. In my studies using pupil core, I could easily use the Pupil Player software to calculate fixation positions. However, with the Invisible that does not seem possible. How to calculate fixations using the Invisible and Pupil Player?

Thanks in advance 🙂

Thank you so much, @marc :)

Dear sir

Thank you for your management. Excuse me. Could you answer the following question? Best regards

My questions are as follows

Q1 What are the steps to follow when connecting the eyewear to a smartphone?　　Is there anything else besides "Enable OTG"?

Q2 Does the eyewear normally work as soon as the eyewear is connected to the smartphone?　My eyewear has been difficult to be recognised as connected since it was first purchased.

Q3. Is there a solution to my problem?

My situation is as follows.

・When the eyewear (Invisible) is connected to the smartphone (smartphone enclosed with the purchase), it takes a very long time for the eyewear to be recognized (It cannot be recognized somedays). Experiments cannot be carried out.

・The Scene Camera Icon and Eye Camera Icon in the attached image do not light up.

・The eyewear gets hot and electricity is running.

・Yesterday morning, eyeware worked after about 10 minutes of connection. In the afternoon it never worked. Today, it also didn't work.

・My use of the Invisible eyewear so far is about one hour or less (total recording time is less than one hour).

Thank you for your answer. The following is my state.

1. What version of Android is running Futami:　 It is 8.1.0.

2. What device OnePlus 6 or OnePlus 8/8T. If OnePlus 6 you will need OTG enabled. OnePlus8/8T does not have this requirement. Futami:　It is OnePlus 6A.

3. Are you using the original cable we supplied to connect the glasses to the phone? Futami:　Yes, it is.

Q4 Is the operation "Enable Application Lock" necessary?

@user-5c56d0 If you are not able to get up and running, I suggest that you contact us at info@pupil-labs.com to follow up so that our Hardware team can help debug and diagnose the issue.

@wrp Thank you.

Could you tell me how to do the operation "Enable Application Lock" necessary? I once did it last year, but I don't know how to do it now.

Thank you very much. I was very saved.

I'm sorry.

Could you give me the following?

In my attached image, how to go from (1) to (2).

Thank you very much.

Is the follwoing action is correct? I have to change from "un-lock" to "Lock"(i.e., from (3) to (4) in my attached Image). My original one was "un-lock".

I'm sorry. This is my intended image.

Thank you

In addition, the other application "Pupil Mobile" shows the following error.

"com. pupillabs. exceptions. Has No Sensor Permission Exception" in my attached image.

@user-5c56d0 Pupil Mobile is not compatible with Pupil Invisible and is depreciated as an application.

Thank you for your reply. I understand.

I’m sorry for my trouble. Could you answer the following?

Q1 In general, how long after connecting the eyewear to the smartphone is the eyewear recognized?

Q2. I have set the application lock to lock ON (i.e. locked), but 10 minutes have passed without the eyewear being recognized. Regarding the operation "Enable Application Lock", is it correct to change the application from unlocked to locked?　The reason I ask this is because the eyewear worked yesterday morning with the application unlocked.

Thank you.

The eyewear was not recognized when a Thunderbolt cable was used. Is USB-C & USB-C cable the appropriate cable? Such as the following products.

https://www.amazon.co.jp/RAMPOW-2x2-20Gbps%E3%83%87%E3%83%BC%E3%82%BF%E8%BB%A2%E9%80%81%E3%80%91PD3-0-QC3-0%E8%B6%85%E9%AB%98%E9%80%9F%E5%85%85%E9%9B%BB-type-c%E3%82%B1%E3%83%BC%E3%83%96%E3%83%AB-Galaxy%E7%AD%89%E3%82%BF%E3%82%A4%E3%83%97C%E5%AF%BE%E5%BF%9C/dp/B0827KJ6DY/ref=sr_1_2_sspa?keywords=USB+Type-C&qid=1657015425&sr=8-2-spons&psc=1&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUFBMldCRDlEQ1JQMEUmZW5jcnlwdGVkSWQ9QTA0OTUyODAzM0JGODRHQ0FWSUdJJmVuY3J5cHRlZEFkSWQ9QTFDOVBCRFVFMkJCWUsmd2lkZ2V0TmFtZT1zcF9hdGYmYWN0aW9uPWNsaWNrUmVkaXJlY3QmZG9Ob3RMb2dDbGljaz10cnVl

Thank you. From the following, I assumed that the problem was with the eyewear. I was also surprised to hear that the eyewear should be recognized as soon as it is connected to the smartphone via USB. The reason is that the eyewear has not been recognized easily since I first purchased it.

・Your original cable could not be used for the eyewear, but my headphones could be charged with it. Therefore, I can assume that your original cable is not broken.

・I tried the following two types. (1) one Thunderbolt cable I have and (2) two Type C cables that are not Thunderbolt cables. As a result, the eyewear was not recognized by the smartphone in all three cases.

Hi! I have questions about setting up an experiment with 3 invisible eye tracking systems in a conference room setting (4.1m x 7.5 m). Three subjects sit in those positions shown in the figure. We want to track their head and eye movements. We have tried putting QR codes (apriltag) on the wall and the table. However, when we threw the data in the Pupil Player and did the camera localization, the camera pose was not fully developed. We want to know how big and how many of them we need in this room size so that we could get solid 3D-model and head pose. Is it possible tracking three goggles at the same time in this big environment? What is your advice working in this big environment with several goggles? Thanks in advance!

Hi! To get a solid 3d model, I recommend making a dedicated "scanning recording" * which you can use to build the head pose tracker's model. Afterward, copy the file to the other recordings. This has the advantage that all three PI recordings will be localized within the same model.

* See the tutorial for reference https://youtu.be/9x9h98tywFI?t=15

hi! I’m using the enrichment named reference image map to convert pupil invisible videos to heat map but it always failed and warned me use another video. I tried several times but still failed. I wonder how to success?

Hi @user-7e5889 👋. Would you be able to describe the environment in which you're using the reference image mapper?

@user-c23af2 You can visualize fixations (rather than raw gaze) as a polyline in Pupil Cloud. You need to open a recording player and then enable the Fixations toggle in the bottom right. (Note: we are only currently releasing this feature and it might take until about next Monday to become available for all recordings). You can however not export this visualization yet.

A visualization of a polyline of the raw gaze is not directly available and you'd have to generate it yourself. You can download the gaze data in CSV format and then visualize it e.g using Python and matplotlib.

Hi @user-7e5889! @nmt 's question was aiming at something different. The Reference Image Mapper does not work in every physical recording environment. It requires the environment to be sufficiently rich in visual features. So e.g. tracking an empty table top in an empty room would probably not work. Could you describe your application and recording environment a little bit? If possible, you could also share your scanning recording with [email removed] so we can take a direct look.

After I created an enrichment and press the 'start' button, then it seems to be loading (p1) but soon went back to the previous status (p2). Then I refreshed the webpage, it seems to be in progress again (p3). But after several minutes, I refreshed the webpage one more time, it showed error 100%(p4).

the detail is shown in the screenshot below and the enrichment ID is 4ff6da06-5525-4aca-8c41-29e4e69afb25

Hi Pupil Labs, Hi Pupil Labs, I am carrying on an experiment and I am using Pupil Invisible. I would like to know how to visualize the gaze positions with a polyline for each gaze position.

Thanks @user-7e5889! This make sit much clearer! If you get this error state, this means that either the environment you're in is unsuitable, or the scanning video of the environment is insufficient. I have not seen your scanning videos, but I think in your case the environment is the issue. With the plain white board, the solid gray floor and the rather texture less background objects and walls the algorithm does not have enough visual features to hold on to. The image you try to track is also a rather fuzzy/blurry one with not too many hard features.

Possible fixes for this would be: 1) make sure the object you are trying to track is more feature-rich. E.g. if it's possible you could swap to a different painting that is less fuzzy. Depending on the research question swapping out the object of interest is of course not always possible.

2) Make sure the object of interest is located in a feature rich environment. If you could move the whiteboard in front of a wall that has large texture that would help. Feature rich objects next the object of interest would also help.

Regarding paintings: we are about to release an example project next week demonstrating the reference image mapper in a gallery with paintings. Maybe that can serve as an example.

Had a recording with the invisible today. Unfortunately, I found out later that no scene video was recorded.

Could there be an error in the export?

I didn't get an error message from the companion app while recording.

that surprises me

I don't use the cloud service. But the two "PI world" files are missing in the exported directory.

But the camera was attached the whole time.

Or do you mean it was disabled in the app?

I can load the recording into the player but when I play the recording I see and hear nothing.

I think that was the problem. OTG was disabled.

I'm surprised I was able to record anyway.

Thank you for that. If things have to be done quickly on site, then unfortunately you make mistakes. I'll pay attention to that in the future.

Also, that means all three subjects do not need to do the scanning process right? We just need to create that ahead of time, them anyone could wear the goggles and walk into the scene. Am I getting this right?

I don't remember the name of the file. Please check the head pose tracker docs. If this information is missing I will update the docs accordingly tomorrow

Let us know if you run into trouble again with the next test! The object of interest must be included in the environment when recording the scanning video!

The painting is suffering from the same problems again: the painting itself is fuzzy and the background is pretty monotone. The test with the table I would expect to succeed though! The table is visually very busy and should work for the algorithm. The computation (if it is successful) can take quite a while, definitely longer than 5 min, so give it some time!

sad.

It depends on the painting! Again, we are releasing a demo dataset using the Reference Image Mapper in an art gallery, where it worked very well. I recommend you check it out once it is out and see what painting we have used. This should be the right tool for you.

Could you share the scanning video you made for the table with [email removed]

We will need to have a look at the scanning recording to tell 🙂

Thanks! I reviewed it and I would recommend the following: In your recording, you hold the glasses in multiple static poses while moving very quickly between them. Please try moving more, but slowly and continuously. This allows the algorithm to leverage more points of view. Holding a single pose is not beneficial. Also, try to include top-down views and side-views as well. This will help the algorithm to build a more stable model.

A bit more variation in distance to the table could also help, i.e. moving a bit closer and further away during the video.

Hi @user-7e5889 👋 Check out the video in our documentation on how to make a scanning recording: https://docs.pupil-labs.com/invisible/explainers/enrichments/#setup-2 I would also recommend watching this explainer video https://www.youtube.com/watch?v=ygqzQEzUIS4 to get a better grasp of how the Reference Image Mapper algorithm works.

@user-7e5889 Hi again! You now see some real examples from an art gallery in our new Demo Workspace: https://docs.pupil-labs.com/invisible/explainers/basic-concepts/#demo-workspace. Check out the Reference Image Mapper Enrichments with paintings, and accompanying scanning recordings there.

thanks! But I am afraid that I can't find this #demo-workspace on this page. Could you send a screenshot to me?

https://docs.pupil-labs.com/invisible/explainers/basic-concepts/#demo-workspace

I think there is no area named 'demo workspace' on this page. Is there any issue about cache or version release

Thanks! I have accessed the page, but there was no reaction after I clicked the 'heatmap'. I recorded a video. I can send it to your email if needed.

yes

I am able to reproduce the issue. I will forward it to our Cloud development team. Thanks for reporting the issue!

I had the use case where I didn't have access to the internet. Is a constant online connection required to use the Companion App or the recording?

Or is it only used for login?

OK. So if I'm logged in, can I use the app offline?

Yes, that is correct

To use the Pupil Invisible Monitor, I tried to make the OnePlus a HotSpot. To connect me with the tablet to this one. Unfortunately, after successfully connecting, I cannot call up the Pupil Invisible Monitor.

Yes, I mean the new web application.

There seems to be a problem with the DNS. Unfortunately, I also have no information about which IP the OnePlus has if it establishes a hotspot itself.

So that I can call IP-OnePlus:8080 in the browser.

Hi, I have 2 invisible glasses. It seems they're intermittently failing to connect between the android and glasses after several sessions of data collection. Most of the time the both scene and camera icons are inactive (grey) when connected to the phone. Otherwise it takes a long time to connect. I've updated the phone and app, not working. What can i do?

Hi @user-95de6f! I have a couple follow up questions:

Are the Android phones you are using OnePlus 6 or OnePlus 8 devices? For OnePlus 6 there would be a couple of things to watch out for, on OnePlus 8 connection should work out of the box.

Are you using the cable that was included in the box to connect the glasses and the phone, or another 3rd party cable?

Do both sets of glasses and phone behave the same? I.e. both loose the connection sometimes?

When you say "it stops collecting data", what exactly does that mean? Does the sensor show up as disconnected in the app and the circles stop filling up on the home screen? Or did you observe something else to come to this conclusion?

I tried that. Unfortunately, the browser returns the message that it cannot access the page.

Are you able to ping the device's ip address from the terminal?

What operating system is running on the tablet?

When I check the IP of the tablet it says something like 192.168.... but the IP of the OnePlus is supposed to be 10.59....etc. This shows me the streaming option in the companion app.

The ip of the tablet does not matter. The ip that matters is the one displayed in the app's Streaming menu.

Does your hotspot have internet access?

Connection Problems with 2 devices

OK. I have now tried, as you said. With the streaming option, it is currently looking for the DNS service. Without the internet, this could be a problem.

It is waiting for DNS service...

I have the IP. However, when I enter it in the browser, I get the message: Website not available. ERR_CONNECTION_REFUSED

I try an HTTP connection to "router" IP:8080

And can you confirm that the app is running?

Now it works! The error was that the RTSP service was not running.

Does this service start automatically?

How did you determine the service was not running? Did you do anything specific to enable it?

I get a notification that the service has started.

I spoke to our Android engineering team. The recommended setup is to have an external wifi of some sort, e.g. another phone providing the hotspot. Hosting the hotspot on the companion device technically works but is not officially supported. For me, the rtsp service is immediately available after setting up the hotspot (without wifi or internet connection) and fully restarting the app (force quit + launch).

But are you able to connect via the router ip and the corresponding port? It is very possible the rtsp service is only launched once requested via the http entrypoint api

You are welcome!

Is anyone aware on how to disable the system update notifications? I tried this, but they still appear :-( https://www.nstec.com/how-to-stop-software-update-notifications-on-android/

I am not aware of a method to disable them unfortunately 😦

hello, I am trying to run a code that has used pupil_apriltags on windows 10. but I have problem installing the package. first it was about "problem building wheel" which got solved by changing a line in setup.py. now I'm getting another error "python setup.py develop did not run successfully. note: This error originates from a subprocess, and is likely not a problem with pip." and my pip is up to date. can anyone help me solve this?

The software is not tested on other OS than the specific OxygenOS versions for Android 8/9 (1+6) and 11 (1+8)

Cool! Thanks for sharing it here @user-28ac9f! 👍

Hey, I don't know if this is the correct channel for my question, but I'd be happy to move it if need be.

We are using Pupil Invisible glasses and Pupil Player for analyses. Our lab needs to have the experimental room video footage in the background with the MET data overlayed on top. Is there anyway to have this setup?

Right now, if we drag the MET data folder into Pupil Player first, we can't resize the video. However, if we drag the background footage first, then we can't get the MET data overlayed on top, we can only see the MET video data without gaze data

Hi guys, I am having some issues with the invisible companion app. When I start recording, it seems to be working until I try to stop the recording. It shows an error message saying "recording error" and then the app freezes. Is anyone able to help troubleshoot what is going on? I have tried clearing cache, resetting the app and turning the phone off and on again.

Attached is a photo of the error message.

The recording seems to be fine for shorter sessions such as 2-3 minutes. However, when participants are spending 16-20 minutes recording, this is when we get the error message.

Hi @user-8d7781 👋. Would you be able to share another photo of the error message? I can't seem to view the current one 🙂

When I received this message, the recording had cut off half way through the experiment. It also did not let me import the recording into pupil player, and said there was no available fixation data.

Thanks for the screenshot @user-8d7781. Could you share the affected recording with [email removed] so we can take a look it?

Hi,

We are planning to use the Invisible Glasses for tracking where the driver was looking while driving a vehicle. We had a few questions regarding that:

1. How long should a person look at a specific point, e.g. a traffic light, or a pedestrian, at least, for the eye tracker to consider this a gaze and to stabilize around that point, and be sufficiently accurate?
2. Is the precision of the identified point influenced by duration i.e. how long we look at that point.
3. Is there a relation of accuracy and time to distance? i.e. if an object is further away, then do we need to look a little bit longer for the same duration?
4. Should we do things like look around in our field of view after putting on the glasses or just start recording, no adjustments necessary?

Thanks. 🙂

And is it possible to connect Pupil Invisible glasses with the Pupil Core Capture software? I would guess not, but just to confirm. 🙂

Hi @user-d4d4bc!

1) + 2) The raw gaze signal is 200 Hz and the estimates are made independently per sample. That means that no stabilization is necessary and looking at the same point for a longer time does not affect the quality of the estimate. For fixation detection the gaze signal needs to remain stable on a target for a minimum duration though.

3) Again, the duration does not influence the accuracy. However, the distance of the object of regard does to a degree. If the subject looks at a target that is < 1 meter away, the gaze estimate will suffer from a "parallax error", which gets worse the closer the target is. This error introduces a constant offset in the predictions. For distances > 1 meter this error does not have an effect.

4) No further setup is necessary, the device works at full quality as soon as you put it on.

5) You can open recordings made with Pupil Invisible in Pupil Player. A bunch of features like obtaining fixations, blinks and gaze mapping are however only available in Pupil Cloud. It is not possible to use Pupil Core's gaze estimation pipeline in Pupil Capture after connecting Pupil Invisible directly to a computer.

Let me know if you have further questions!

hello guys, does invisible calculate pupil diameter ?

Hey, Pupil Invisible's eye camera angles are not suited to do so. Your current option for calculating Pupillometry is Pupil Core.

If you are dependent on pupillometry, then yes. Note that uncontrolled light environments can make it difficult to get clean pupillometry data due to the pupillary light reflex. Maybe there are other non-pupillometry metrics that can help answer your research question (allowing the use of Invisible).

Hello Pupil Labs Team , our pupil invisible oneplus 8 companion app flickers and the screen stays bright , is the issue known ?

@nmt Hey Neil, we have been exporting the raw eye-tracking data from the phones to the computer. So far we have not used cloud services.

For the video footage inside our room, we are running the video data through a video conversion script that was provided by pupil.

Hi Marc, I just shared the google drive folder with that email. The folder was too large to send via email.

I am trying to setup the Monitor. I have to figure out how to do this, in our IT sec setup.

AFAIK creating the hotspot on the companion device will not work at all, as mDNS is not supported in Android 11 (which is mandatory for the connection to the glasses to work) [1][2].

Thus I am trying to use a hotspot from my company phone, but it seems to partially break sometimes as well (the streaming works, but all actions will not; probably due to UDP blocking?)

[1] https://blog.esper.io/android-dessert-bites-26-mdns-local-47912385/ [2] https://issuetracker.google.com/issues/140786115

@papr this process should also work if instead of the cloud data I use the data obtained directly from the device, correct? If I'm not mistaken, I am downgrading from 200Hz (cloud) to ~66Hz (local), and that's pretty much it, right?

Note that since a recent update the real-time gaze estimation is much higher than 66 Hz though! Its >120 Hz now.

Hi, was just wondering what the cloud storage capacity was for the Pupil Invisible Glasses?

Hi @user-3b418f! The storage capacity in Pupil Cloud is unlimited!

Oh cool, would there ever be any plans to change that do you think? If pupil labs got bigger etc? Trying to work out data management from the perspective of higher education 🙂

can I make a request in advance that if you do make plans, to create pop-ups or disclaimers that appear on the cloud website itself? We never check the generic intuitional email we used for the account creation 🤣

Haven't used Invis in a while. I'm delighted to see the new fixation detector 🙂

...at the same time, I see it in the cloud, but not when i download the recording?

What's going on there?

I right click, and select "download recording" (and not "download raw data")

I would have expected the gaze and fixation overlay on the raw video.

It is as if the option to download recording is just returning the raw data.

it returns a downloadable pupil labs folder "raw-data-export"

...and, why can I enable fixations when viewing one video, but not another?

Then once added, you can select the play button to process the new enrichment that should appear on the screen, and then download that.

I have created a project and have tried this. Something seems to be malfunctioning. And what about this issue that downloading only returns raw data, but never the processed video?

It looks like the application of the gaze overlay enrichment stalled for some reason, which may explain why one video lacked the overlay. One source of confusion is that another video had the overlay without being in a project, and without enrichments applied to it.

Hi Papr, thanks for this... I will try this shortly. On a side note, do we know why the PI world v1 ps2.mp4 was corrupted? It keeps happening for every recording now, where when we go to stop any recording, the app freezes and continues to record despite clicking stop. We aren't able to use this device because it keeps producing corrupted files, so would love to figure out why it's happening 😦 Thanks

@user-b14f98 Since the introduction of the fixation detector, fixation data is computed for every recording on-upload to Pupil Cloud. Fixation data has also been calculated post hoc for all recordings already in Pupil Cloud before the release. If you download a recording in CSV format, (i.e. "Download Recording" rather than "Download Raw Data"), the files should include a fixations.csv file that contains the fixation data. Let me know if that is not actually the case for any of your recordings!

The scene video included in this download is intentionally the unaltered original scene video with no gaze overlay visualizations applied. The idea is that you could use it to create any custom visualization on top of it, without having to deal with visualizations already in there. If you are interested in a video with gaze overlay, using the "Gaze Overlay" enrichment as @user-3b418f said would be the recommended way. If the computation of this enrichment is not finishing something might be wrong with it. Could you let me know it's enrichment ID (visible in "View Details") so we can take a look?

Outside of using the gaze overlay enrichment, there is currently no other way of obtaining such a video, except maybe screen recording your browser. The scanpath visualization for fixations which is now available in the recording player can currently not be exported as video. We are aware that the options for exporting scene video with visualizations are still pretty limited. We are currently working on two things to improve on this:

1) we will soon publish a Python library together with guides that make the handling of video, timestamp matching and the manual creation of custom visualizations much easier. 2) A bit further down the road still, but we plan on adding a bigger video rendering tool, which will allow you to create all kinds of visualizations on top of the scene video, including gaze, fixations, eye video overlay, face/surface detections etc. Release will not be before 2023 though.

Sounds like there might be a spelling mistake in there. The renamed file should info.json. If this is already the case, could you please share it here?

Hello, I am using pupil invisible glasses and I want to use the scene video in my code. But I have problem connecting to the device. I can see the video in http://pi.local:8080/ but I get this in python using real-time API: (I use windows 10 and Python 3.10.4)

device = discover_one_device(max_search_duration_seconds=10)

frame, gaze = device.receive_matched_scene_video_frame_and_gaze() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'NoneType' object has no attribute 'receive_matched_scene_video_frame_and_gaze'

Can someone help me with this?

Thanks, Marc. To reiterate, the issue is that the top video in the image below gave me the "fixation" toggle without any additional work, before it was in a project. However, I cannot seem to activate the same "fixation" toggle in the bottom video, with the enrichment ID 22e8850b-e664-48c3-b173-d14f498117ca

After noticing the difference I did add both videos to the same project ("cycling") with the same "gaze overlay" enrichment applied from the start to the end of the video. The issue was not resolved.

Notice that the bottom video has an "!" for an icon. Something may be wrong with it.

@marc Dear Ladies and Gentlemen, After the University of Passau bought the Pupil Lab Invisible, we are now conducting a study with it. The following error message was displayed. I hope you can help us with this:

If device is none that means that no device was discovered. Could you share details about your network setup?

We have restarted the processing of the failed recording, which has fixed the issue. The exclamation mark should now be gone and you should be able to toggle the fixation visualisation!

alternatively to

device = discover_one_device(max_search_duration_seconds=10)

you can also use

from pupil_labs.realtime_api.simple import Device
device = Device("pi.local", 8080)  # or use ip address instead of domain name

We want to use the invisible for user testing on a digital screen. We want to track what the users see first and how they navigate through the prototype. Currently it seems that the glasses have a slight shift to the left if we look at the fixations. Is there a way to improve this? We have tested with several people. Another question we have is if there is a certain distance needed between the glasses and the screen?

Is it possible to get the realtime stream of the gaze data into Unity3D for pupil invisible?

Hi @user-011cbf 👋. Thanks for your questions!

For users looking at a digital screen positioned less than 1 m away, there may be some parallax error. You can use the offset correction feature to compensate for this. To access this feature click on the name of the currently selected wearer on the home screen and then click "Adjust". There are instructions in-app. The offset you set will be saved in the wearer profile and applied to future recordings of this wearer automatically. Important to note: * the offset is valid at the viewing distance it was set, which should* be okay for screen-based work.

Have you also checked out the Marker Mapper Enrichment? You can add April Tag markers to your screen and use the enrichment to obtain gaze positions relative to the screen: https://docs.pupil-labs.com/invisible/explainers/enrichments/#marker-mapper

Thank you for the response. The realtime stream works fine, and I wanted to get the marker id using april tags. However, I have some issues for installing pupil-apriltags, and could not fix it.

C:\Users\Admin>python Python 3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information.

from pupil_apriltags import Detector

at_detector = Detector(families='tag36h11', ... nthreads=1, ... quad_decimate=1.0, ... quad_sigma=0.0, ... refine_edges=1, ... decode_sharpening=0.25, ... debug=0) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Users\Admin\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pupil_apriltags\bindings.py", line 285, in init self.libc = ctypes.CDLL(str(hit)) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3568.0_x64__qbz5n2kfra8p0\lib\ctypes__init__.py", line 374, in init self._handle = _dlopen(self._name, mode) FileNotFoundError: Could not find module 'C:\Users\Admin\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pupil_apriltags\lib\apriltag.dll' (or one of its dependencies). Try using the full path with constructor syntax.

Any tip to fix the issue? I tried to update the binding.py as the recommendations of some guys, but it does not take into effect.

Could you please share the file contents of C:\Users\Admin\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pupil_apriltags\lib\?

I spoke to our Android engineering team

Is it possible to send annotation events over a network to the pupil Invisible? If I remember correctly this was possible with pupil Core via 0MQ, does a similar solution exist for the Invisible companion app?

Hi @user-cc819b! Yes, this is possible via the real-time API. You can find an introduction here: https://docs.pupil-labs.com/invisible/how-tos/integrate-with-the-real-time-api/introduction/

thanks!

Hi,

I had a question. We have an application where we would like to do both, head pose tracking, and surface tracking simultaneously in the same vehicle.

For surface tracking, we were planning to use different sized markers.

For head pose tracking, as I understand, we are required to use the same sized markers.

I was thinking what we could do is, we could use 36h11 for head pose tracking, and run surface tracking only with Circle21h7 markers.

As I understand, head pose tracking is currently compatible only with 36h11 markers.

Would this be okay? As in would there be a problem with the head pose tracking algorithm, owing to different sized Circle21h7 markers placed in the vehicle?

In the future, is it planned that the head pose tracker starts detecting Circle21h7, which could lead to problems with the functionality?

Thanks. 🙂

Hi @user-d4d4bc! This might work just fine, but some of the marker families can have false positive detections with markers of other families. As this is combining square and circular markers, I imagine this would not be the case, but I'd recommend validating this. If we change the compatibility of the head pose tracker to other families, we would most likely allow the user to select the family, so this should not be an issue.

An alternative to using different families would be to split the markers into two sets by their ID. I.e. use constant size markers for head pose tracking with a specific set of IDs, say 1-16. And varying size markers for surface tracking with IDs 17-32. For the surface tracking you can easily select which of the detected markers should be used for surface tracking. For the head pose tracker you'd have to generate the model without the other markers present (or covered).

Hi there,

I have some really annoying issues while using the glasses with Invisible. I think there might be some hardware connection issue, as the glasses seems to randomly disconnect from the phone, with error messages and phone vibrations. This happens approximately 9 times over 10, and my team and I have not been able to record any usable signal from our experimentations.

Is there any way to fix this issue, or do we need a hardware replacement?

Thank you

Thaks Mark for the quick reply.

Thanks Marc. 🙂

Hi! Question about Pupil Cloud: I have created a project and added two enrichments: raw data export and gaze overlay. I have downloaded this data to my local storage. Now when I add a new recording to the project and compute the enrichments, is there any way to download only the enrichments for the newer recording? Is there a way to download enrichments for each recording separately?

Hi @user-413ab6 👋 You will need to download the entire enrichment. For Raw Data Exporter it's likey not that big of a deal because the download size should remain small. However, for Gaze Overlay the download can get quite large as it is videos, which can take a while.

This is good feedback and we will incorporate a way to make more granular downloads in future updates to Cloud.

Hello! Question about the blink detection: is there any chance to get the detected blinks as .csv file for data collected before the introduction of this feature in the Cloud? I am specifically talking for data collected in January, 2022. Thanks for your support! 🙂

Is it possible to use Invisible with Android phone other than OnePlus 8 or 6?

Hi @user-8a4f8b! No, only those two models are currently supported. We have very specific requirements for the hardware and operating system and these things vary wildly between manufacturers and models, so we need to limit our support to just those devices. You can use any OnePlus 6 or 8 device though (assuming a compatible Android version), they do not need to be purchased from us.

@marc or would it be possible to transfer/steam recorded data from OnePlus via USB to PC (i.e. without using wifi nor bluetooth)?

I just found this. so it does seem possible: https://docs.pupil-labs.com/invisible/how-tos/data-collection-with-the-companion-app/transfer-recordings-via-usb.html

Hello, what's the end-to-end sample delay for real-time streaming of gaze data with invisible, and at what frequency can the gaze data be accessed?