After many hours of trying it miraculously throws and exception at the point where the connection stops working.

When running it on my Mac it works, when running it on Android this error occurs after 10-30 seconds.

https://github.com/pupil-labs/hmd-eyes/issues/63

Your AsyncIO is outdated 😉

Thanks for bringing this up @user-54376c.

Hey, can someone shed some insight on what's preventing the use of 3D pupil detection in hmd-eyes?

Why has it never been properly implemented?

Actually we plan to release this in the upcoming week :-) @user-8779ef

Hi guys. I tried recording with heatmap in unity. I noticed that when I pressed h my game start dragging and the folders created were blank. I've been using the market scene for test, am I supposed to use any extra script? .

Any suggestion please?

@mpk That is great

As far as I can tell, it's just a matter of changing the initialization settings

I want to deploy the example project of hmd-eyes v0.5.1 to the Hololens, but it cannot connect to pupil capture... Which IP adress do I have to enter in the PupilSettings? The one of the HololensRelay of pupil capture, the one of my PC or another one? Would be really nice if anyone of you could help me

@user-8779ef we are actually doing a bit more than that. I hope we are still on track for this week, maybe it will be beginning of upcoming week.

I eagerly await! Thanks, MPK.

Hello, in SteamVR cameraRig instead of Camera(eye) for both eyes, I have Camera (left) and Camera(right) for both the eyes.. In this case what change do I need to make to track my eyes? Do I need to anything while calling PupilGazeTracker.Instance.StartVisualizingGaze()? Should I put a script on both the cameras to update marker.localPosition = PupilData._2D.LeftEyePosition and marker.localPosition = PupilData._2D.RightEyePosition ?

If I have 2 cameras per eye(left and right) to create render texture and use them at the same time in the scene.. it gives me "No cameras rendering" error.. Can I just tell pupilGazeTracker to consider only real Camera(left) and Camera(right) for gaze tracking and ignore the other cameras? I have assigned the "Calibration Demo" script to Camera(head).

Hi guys. I'm considering also purchasing the VR addon to put in my HTC Vive but I have a question regarding eye tracking vs gaze tracking. I believe I will need to calibrate the eye position/rotation relative to the screen by having the user view points popping up on each edge of the screen in my VR program and then know the rotation angle of the eyeball to know at what pixel(s) the eye is focus at each frame, correct? I believe the Pupil sourcecode does "2d eye tracking" too but I don't know what those 2d coordinates represent and could be used for and how it differs from "gaze tracking". For my use case I need to know where on the screen or field of view the eye is focused at.

Hello everyone, I'm having major troubles getting started with the Vive add-on, the latest pupil service/ capture and hmd-eyes in Unity. For now, I'm trying to get the example scenes provided in hmd-eyes to work. I can run the 2D and 3D calibration, but when they are done, nothing happens. Since I don't get any message like "calibration done/ failed", I suspect that the problem is not the transition to another scene, but rather properly ending the calibration? I'm using Unity 2017.3 and pupil service, the Unity project is completely empty besides the pupil_plugin and SteamVR. Does anyone have experienced the same problem? Thank a lot for any help!

@user-86c436 Make sure you have the latest release build (not dev build) of both pupil and hmd-eyes.

If you have separate unity scenes for 2D/3D calibration, then you're running an old version of HMD-eyes. The current version has ONLY the 2D market scene demo.

@user-8779ef ah ok, I will look into that. thank you for that tip 😃

No problem! TO be clear, on github, make sure you select the "releases" tab.

If you just go to the front page and hit download, you're getting the dev build.

@user-8779ef I just re-downloaded (and re-imported) the unity package from https://github.com/pupil-labs/hmd-eyes/releases/download/v0.61/Pupil.Import.Package.VR.v0.61.unitypackage - and I still have the same problem. Also, in this package there are scenes called "2D Calibration Demo" and "3D Calibration Demo" as well as "Market with 2D Calibration" and "Market with 3D Calibration".

Hurmn. Ok. And you have the latest version of the pupil software?

v1.11 of pupil service?

https://github.com/pupil-labs/pupil/releases

In any case, thanks for correcting my bit of misinformation. I realize now that I must have recently downloaded 2d_calibration_demo_vr_v0.61.zip, which explains why I thought they had removed the 3D market demo.

This also jived with my experience that the 3D demo had been generally problematic over the past many months. I have been waiting for a fix, or for them to remove it.

In any case, I ran 2d_calibration_demo_vr_v0.61.zip on my Win 10 machine with pupil service 1.11 yesterday, and can confirm that those work on this end.

If yours does not work, I would check https://github.com/pupil-labs/hmd-eyes/issues for an open issue that acknowledges the problem. If none exists, I would create an issue.

I wasn't aware that they just released a new version yesterday. Anyway, tried it - and still didn't work. Not in the editor nor from the build. I have one more idea to try, and if this doesn't work, I guess, I'll open an issue (I didn't see one that describes the same problem). Thanks a lot for your help!

No problem. I didn't know they released a new versino yesterday, either!

So, I should correct my comment, that means I was using V1.10

GOod luck!

hi @user-2ee432, pupil capture already takes care of gaze tracking and with the hmd-eyes plugin for unity you can directly access the 2d/3d gaze estimate and the gaze vectors (after an initial calibration). so no need to do this yourself by hand.

hi @user-86c436, the 3d calibration can take some time to finish (depending on your settings), so you might need to wait 15-30 seconds. but as you are having the same issue in 2d and 3d this must be something else. Could you please try again with hmd-eyes, v0.61 pupil capture 1.10 (not service, not 1.11) and unity 2017.4 LTS (you mentioned 2017.3).

@fxlange Thank you very much. Can you please explain what 2d and not 3d gaze estimate is at all?

hi @user-a5f66b, the current version of hmd-eyes is kind of hard coded to only one camera which must be centered. this is the case when working with the default camera in unity, as the returned transform will be centered between the two actual cameras.

@user-2ee432, these are actual misleading terms. in short we are using two alternative approaches. the "2d" is based on the coordinates of the detected pupil/iris while "3d" builds a 3d representation of your eye balls first and calculates the gaze vectors based on that.

@fxlange Hm, so both can methods provide the same gaze angle transforms but one is possibly more accurate than the other? (my guess is the 3d one?)

Not really, 2d doesn't provide any gaze angles but normalized viewport coordinates you can (in Unity) project onto a plane with fixed depth. Only the 3d approach provides a 3d estimate and gaze vectors. On top it is more stable against hmd slippage. That's also why we will focus on the 3d approach from the next hmd-eyes release on.

I think I understand, with the first method you need to know the distance from the eye to the virtual image created by the HMD exactly to guess its angle?

Yes, in VR the distance between the camera and your gaze targets, with all targets having the same distance.

I see, thank you. One more question, if I wanted to use the Pupil camera addon with my own modified program, can I (is everything open sourced)? And if yes, which classes and methods should I look for?

Oh and second question, 2) How are the eye positions relative to HMD calibrated initially and can it be re-calibrated every n-th minute to counter HMD slippage?

Hi Everyone! I am just started with PupilLab hdm.eyes. I installed unity and both Pupil Service and Capture and downloaded these 4 Assets: - Pupil.Import.Package.VR.v0.61.unitypackage - 2d_calibration_demo_vr_v0.61.zip - Source code (zip) - Source code (tar.gz) than I tried following the instructions in this link: https://github.com/pupil-labs/hmd-eyes/blob/master/Developer.md and opened a new Project folder importing: Pupil.Import.Package.VR.unitypackage but as soon as I put in the Hierachy the Calibration. unity and set the build settings a strange compiler error occurs. It says "The associated script cannot be loaded" on every Script in every GameObject in the project. It basically cannot find the script even if they are in the "Scripts" folder which is inside of the the "PupilPlugin" folder in the assets of Unity. Also I tried to open the script in VisualStudio and Build but nothing happened and whenever I try to run Unity a message comes up on the scene panel saying "All compiler errors need to be fixed "

I think you have so select .NET 4.x in you player settings for "Scripting Runtime Version" and "Api Compability Version"

Hey @fxlange, thank you for your answer. Today it miraculously started working - I didn't change anything of my setup. It now works from inside the editor as well as from a build and with pupil service as well as with pupil capture (both 1.11).

@user-54376c Cynicer thank you so much it helped me a lot and now it works. 😃

I could call my scene after the pupil labs calibration runs. But is it possible to call the calibration scene again from my scene? Do we have guidelines for that?

Hi everyone I´m just getting started with pupil HTC VIVE addon and Unity. Now I have a problem with an error that namespace FFmpegPipe could not be found (see picture) What I did so far: - imported Pupil.Import.Package.VR.v0.61 into Unity Project - changed Project to UWP - set Scripting Runtime Version to .NET 4.x Equivalent -set Scripting Backend to IL2CPP - set Api Compatibility Level to .NET 4.x

Any help will be appreciated

Hello Guys! I am having some problems with the Unity scene "Market with 2D calibration". I already added the new scene in the build settings but whenever I run the scene it skips the calibration part and directly shows the market scene. The laser does not follow my gaze but only the Vive headset position. I have read in the Market Scene Demo>SceneManager Folder that there is a text file saying the scene needs to be loaded from a calibration scene but even after few trials I do not get what that meens. Also I am just started programming and I do not get in the "Accessing Data" part of this article: https://github.com/pupil-labs/hmd-eyes/blob/master/Developer.md what that code is for honestly and where should I store it in the pupil_plugin assets. What should I Do? Thank you in Advance for your help

hi @user-9bc047, the "Market with 2d calibration" scene is supposed to be run after the calibration. You need to start the "Market Scene Demo/2D Calibration Demo" first and follow the instructions to run the calibration. Afterwards (if the calibration succeeded) you will jump to the "Market with 2d calibration" scene automatically.

hi @user-806673, v0.61 is supposed to run on Unity 2017.4 with the default player settings (basically still using .NET 3.5). Any particular reason for switching to UWP and IL2CPP ? Btw we are currently working on a new release which supports Unity 2018.3 and .NET 4.x. (hmd-eyes-alpha)

How are the eye positions relative to HMD calibrated initially and can it be re-calibrated every n-th minute to counter HMD slippage?

@user-9bc047 I was confused about that at first, but I solved it by adding the scenes (2D calibration demo and Market with 2d calibration) in the build settings, and then play the 2d calibration demo scene.

Hey @fxlange, I haven't tested it yet, but just want to share my appreciation for the renewed attention to HMD eyes. Are you the pupil labs employed engineer working on hmd-eyes ?

Hello everyone, Does anyone know which c# code of hmd_eyes project do track the pupil? I am trying to understand the algorithm of tracking pupil.

@user-8779ef yes, @fxlange is working at Pupil Labs - pleased to have him on the team 😸

Awesome

Hi guys. The demo market scene with heat map is not working. The calibration doesn't run. It jump straight to the market scene. I am using the latest builf

Build. Thanks

'''Win32Exception: ApplicationName='F:/UnityProj/Eye tracker test/Assets/StreamingAssets/FFmpegOut/ffmpeg.exe', CommandLine='-y -f rawvideo -vcodec rawvideo -pixel_format rgb24 -video_size 2048x1024 -framerate 30 -loglevel warning -i - -pix_fmt yuv420p "D:/Pupil Recordings/2019_03_08/001/Heatmap.mp4"', CurrentDirectory='' System.Diagnostics.Process.Start_noshell (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process) System.Diagnostics.Process.Start_common (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process) System.Diagnostics.Process.Start (System.Diagnostics.ProcessStartInfo startInfo) FFmpegOut.FFmpegPipe..ctor (System.String name, Int32 width, Int32 height, Int32 framerate, Codec codec) (at Assets/pupil_plugin/FFmpegOut/FFmpegPipe.cs:57) Heatmap.OpenPipe () (at Assets/pupil_plugin/Scripts/Demos/Heatmap.cs:330) Heatmap.LateUpdate () (at Assets/pupil_plugin/Scripts/Demos/Heatmap.cs:275)'''

Do you all have any plans to make a JS SDK? We are planning to use this and want to get it working in AFrame

Hi I have the HTC Vive Pro add on and I have created a VR application in Unity . If anyone has an experience with pupil labs HTC vive add on, would be very helpful if you could tell me the steps to call any functions of the eye tracker in Unity C# scripts? Or are any projects already created in github with Unity and pupil labs?

@user-0eef61 hi. Pupil already have some demo scripts which you can use. What do you want to achieve exactly? I know and I have used the calibration script and the market scene demo

Hi @user-a39804 Thanks for your reply. I have set up the eye tracker in htc vive and I have downloaded the pupil software. Now I downloaded pupil import package from: https://github.com/pupil-labs/hmd-eyes/releases/tag/v0.61 and I am looking the developer documentation: https://github.com/pupil-labs/hmd-eyes/blob/master/Developer.md

From the eye tracker I want to get pupil data, blinks and gaze coordinates. Are those possible?

@user-0eef61 this is one of the issues I also have. I have sent 3 messages already and there is no response. I bet it's possible @papr [email removed] just have to show us

@wrp is there a way to record blink and gaze coordinates in hmd?

Hi @fxlange Thank you for your reply and sorry about the late response. Do you mean that if in Unity it does not say success after the calibration the demo won't run? Cause as I told you it only says "calibration ended" but it does not say success or failure. Moreover when I stop the scene from running it says "Display 1 No camera rendering". I tried adding a camera to the scene but it does not change anything. Thank youin advance

Hi @user-a39804 I am receiving some errors . I cannot run the project because the pupil gaze tracker is not recognized. It says that the Local Builder could not be found. I followed the steps in the developed docs but I still get this error. Have you encountered this before?

@user-0eef61 no I haven't. I'm not sure why @wrp and @papr are not responding. We need to wait for them I guess

Hi @wrp , @papr can you please give me any advice why I am facing this issue? I followed all the steps but when I imported the package from: https://github.com/pupil-labs/hmd-eyes/releases/tag/v0.61 into Unity, I get a lot errors. I cannot open the PupilGazeTracker or PupilSettings. Everything else is running, like my VR headset, pupilcapture.exe What am I forgetting here? Thanks in advance

@fxlange Sorry for late reply. Thanks for your help I got it working now. I´m looking forward to the new release for Unity 2018.3 because we want to use the HD Renderpipeline

Since the resolution and fps for the HMD addons seem to be different from the standard Pupil Labs wearable device [email removed] I have to ask, for the HMD addons, what is their tracking accuracy, precision and speed (latency)? Thank you

Hi again, I actually made it work by changing in player setting scripting runtime version and api compatibility to .NET 4.x

now I had another question, has anyone recorded any data such as pupil diameter, blinks and gaze position?

@user-806673 great to hear!

for Unity 2018.3 support: feel free to already check out our latest hmd-eyes alpha release (hmd-eyes-alpha).

thanks @user-8779ef and yes, that's me. looking forward to your feedback on our latest pre-release 😃

@user-0eef61 you are using hmd-eyes v0.61 but which version of unity and pupil capture are you actually running? by "recorded any data such as ..." do you mean accessing or actually recording data? for data access we have several demos and the documentation you already linked. did you try the Market Scene Demo/Market with 2D Calibration?. for pupil diameter you can subscribe to the pupil topic, which is described here in detail: https://github.com/pupil-labs/hmd-eyes/blob/master/Developer.md#topics-that-do-not-require-calibration. recording is supported as well: https://github.com/pupil-labs/hmd-eyes/blob/master/Developer.md#recording-data

hi @user-9bc047, on the screen it says "ended" (before switching the scene) but in the console you should see "Calibration finished" followed by "notify.calibration.successful" or "notify.calibration.failed"

@user-a39804 you mentioned trying the Heatmap demo. Did you already try the Market Scene Demo/Market with 2D Calibration. Your exception might be related to your Unity Version and/or Scripting Runtime version. Please try hmd-eyes v0.61 with Unity 2017.4 LTS, Scripting Runtime Version Stable .NET 3.5 and Standalone build target (not UWP for VR).

thank you so much @fxlange it worked

@user-2ee432 recalibrating from time to time is supported and can be controlled via your own application.

@fxlange Thank you, but I think there may be a misunderstanding. I do not mean manual recalibration, I mean automatic calibration not requiring anything from the user, running in the background, to compensate headset slippage. I think if the camera can determine where the eye is relative to the camera than it can do the same over time, say every n-th minute.

this is already happening for the 3d approach, yes. this is why the 3d approach is way more robust against slippage (but not for 2d, there you have to do it manually including the user following the calibration markers. from previous questions I was assuming 2d)

Awesome! So there isn’t a need to calibrate initially by looking at specific points on the screen as well, just rotate your eyes around for a while?

May I ask that how do you send the eye Image to the unity demo? Do you send the eye Image data or a python pointer and get the data in unity?

@fxlange

hi @user-2ee432, an initial manual calibration is needed for both approaches - but the slippage correction (adjusting the 3d eye model) is happening automatically - for 3d only.

@user-b91aa6 the eye images are provided via the frame publisher plugin of pupil capture - via message pack as binary data so to say - not via pointers

@fxlange that's a great feature, thanks for adding. Is it part of the Python codebase now?

hi @user-e91538, as far as I know there are currently no plans for JS support but you could build your own SDK similar to hmd-eyes for unity. the communication is based on zeromq and message pack which are probably available for js /nodejs as well. but this is more a topic for software-dev

@user-2ee432 this is all happening on pupil capture side yes (and available for quite a longer time already). hmd-eyes is just receiving the gaze information.

@fxlange ty

@fxlange i tried the headmap in unity 2018.2 and unity2018. 3. I will try it in 2017.4... Thanks

Hello - We are having an issue with the integration of the pupil labs eye tracking in Unity where when we try to exit the eye tracking scene (not the calibration scene) it will hang forever (Unity freezes). The version of Unity we are running this in is 2017.3.1f1. Additionally, we are successfully able to load this test scene from a 2D calibration scene like the one in the 2D Market demo with eye tracking. The issue seems to be coming from PupilTools.cs in the Disconnect() function, specifically from Connection.CloseSockets(). Has anyone encountered a similar issue/solution to this issue? Since this program is run simultaneously with pupil service is there a socket between them that is not being released? Thank you in advance for any help or advice you can offer.

Hello Everyone, I'm new to hmd-eyes and I'm trying to get the demo working on a HoloLens. However, there seems to be a few issues with the code (Or maybe I did something wrong). I found a large amount of compile time errors such as Recorder.cs not having an isProcessing boolean variable but referenced in CameraCapture.cs, No PupilConversion static class referenced in CameraCapture.cs and so on. Could someone advise please

@user-879f69 Have you tried running Pupil Capture instead of Pupil Service... Both will make the HMD eyes work with the eye tracker. I remember seeing a post recommending to use Capture instead of Service...

Try Capture and come back to us with the results !

@user-9fdb5f What version of Unity are you using ? I think that very recent versions of Unity were not working great with HMD Eyes 0.61... For my part, I tested on Unity 2017.3.1f1 and 2018.1.8f1 and it's working fine.

If you have error about "unsafe code": you need to enable the "unsafe code" in your project. To do so, go here: - Open menu "Edit --> Project Settings --> Player" - Open the "Other Settings" section on your right panel on your screen - In the "configuration" section, you will see the option "Allow unsafe code', you need to CHECK that option.

https://cdn.discordapp.com/attachments/285728635267186688/536916269438337054/unsafe_code_option.png

@user-8b1528 Thank you for your suggestion. Once we were able to display the eyes from the HMD eye tracker (get it out of ghost mode) we were able to run the scene using pupil capture and terminate the scene without issue. 👍

are there plans to allow to use the HMD addons without them starting the pupil capture GUI?

hey @user-8b1528, Thanks for getting back to me. I am currently on 2017.4.1f1. I am unsure how the code works in the first place as there are variables and functions missing that are referenced. I also do not get any unsafe code error.

hi @user-9fdb5f for hololens, did you try to follow these instructions: https://github.com/pupil-labs/hmd-eyes#hololens-getting-started. feel free to send me a PM if the instructions are not working for you.

Hi there. I am researcher working on a project that will use Pupil Service. I have a custom head-mounted display that is designed to be super minimal. It consists of a small LED array, but no screen whatsoever. I would like to perform a calibration that involves the user looking at a physical rig that I made. I would like it to work so that when a user looks at the rig at various built-in targets (ie, target on the Left side, Top side, Right side, Bottom side, etc etc), the Pupil Service program can associate coordinates to each respective target. Are there code examples for Pupil Service applications that have calibration routines that do not use AR/VR displays? Anything helps! Thank you kindly

Looks like this is what I need: https://github.com/pupil-labs/hmd-eyes/tree/master/python_reference_client anything else you would recommend for this project? THanks

@user-345bdf There is also a plugin that calibrates to finger location, detected using computer vision filters on the scene camera.

That might be informative.

I think it's here: https://github.com/pupil-labs/pupil/tree/master/pupil_src/shared_modules/calibration_routines/fingertip_calibration

@user-345bdf From what I understand, it's not really an HMD Eyes setup, it's more a regular Pupil Labs setup... If so, maybe transfer the discussion to the appropriate channel (Ex.: pupil)

@user-345bdf ...and, I don't think you need to mess with HMD Eyes. ONly pupil capture. I would implement your code as a plugin. See the developer docs for info on plugin development. @user-8b1528 You beat me to it. @user-345bdf Listen to him.

I see. The reason I started looking here was because I want to use Pupil Service for better performance on my laptop...I have no use for the video streams whatsoever, and I have a head mounted display to deliver feedback to the user, its just not a screen-based feedback. I will go to the Pupil discussion instead though if you feel that it is more fitting. Thank you for the feedback 😃

Hello everyone, I'm using pupil labs in HTC Vive and I frequently has some problem to configure the capture software to obtain a correct pupil detection. It seems that the coarse detection (the cyan rectangles) doesn't find my pupil. Any idea how I could fix that problem?

@user-879f69 How did you solve the problem? Because I have the same issue: Unity crashes every time I stop the scene. I'm using unity 2017.4.20f and Capture 1.11.4

@user-ee433b I think the camera is out of focus. Also your ROI is set to narrow.

@user-ee433b To adjust focus, you have to turn the ring around the camera pin hole to make it focus closer or farther. You may have to turn it 1-2 complete turns sometimes... In case you don't know how to adjust it 😉

Thank you, it's not perfect, but it's better.

@user-ee433b You can always try to adjust the camera images settings, in my case I use those values: • Image Post Processing : o Backlight compensation : 2 o Brightness : 64 o Contrast : 64 o Gain : 0 o Power line frequency : 60 Hz (or whatever power line Hz you have at your place) o Hue : -40 o Saturation : 0 o Sharpness : 0 o Gamma : 144 o White balance temperature : 4100 o White Balance temperature Auto : on

Update me if it helped you out those values...

@user-8b1528 Thank you, it's far better with the parameters you gave me

Great ! I had the same problems when I started playing with Pupil hardware, focus and parameters... Glad I could help you and others having the same problem fine tuning the setup 😉

@user-ee433b the image looks pretty sharp to me. the res of the eye cams is 400x400 max and will never appear crisp. There a various benefits for pupil detection in that. But that's an other thing. Changing the focal length by turning the lens of the eye tracking cam is not recommended by pupil labs. In your image see how the brightest spot is pointed top left of the pupil. if you can point that directly at the pupil, that might help further. cheers

@user-14d189 with the 120hz Vive add-on you can adjust eye camera focus by manually turning the lens.

With the 200hz eye cameras you can not manually adjust the eye camera focus and should not need to.

@wrp thanks for the info.

Hi all. I'm working with the HTC Vive and have a question regarding the calibration space in VR. For the 2D mode, it seems to me that normalized values for the gaze positions don't exceed the range of 0.4 - 0.6 for both x and y direction, when looking at an area that has approx. the same dimensions as the calibration circle. What are the dimensions of the actual calibration space in VR, is it scaled to the camera canvas? I attached a small drawing to clarify what I mean. Thanks in advance!

Hi, I've updated my Unity to 2017.4.24f1 hoping that it would help with the crash but it hasn't

Unity is crashing when I stop the scene.

I'm using the last version of HMD-eyes and when I stop the market demo (or any scene using the 2D calibration+PupilManager), Unity crashes

@user-ee433b What Pupil software you are running ? For the latest HMD Eyes 0.61, it is recommended to use Pupil Capture instead of Pupil Service... Is using Capture still make your Unity crash ?

Hi, I use the last version of capture

I made some tests and it seems that it might come from the connection between capture and Unity. Because if I stop before the connection is established, it doesn't crash, but if it says connected... and I stop... paf, crash

Hi guys, I am running unity 2017.4.24f1 with pupil service. Whenever I am done calibrating it shows this image below without showing calibration successful or failed. So it will just be stucked to that window. What could be the cause?

@user-8b1528 The version I'm using is Capture 1.11-4

Hello everyone I am running untiy 2017.4.23 and pupil capture 1.11-4 and I was trying to open up a scene, that was built on a newer unity version and I get a lot of errors if I open it in the old unity version, in which pupil servie works with unity. Have any of you experienced something similar and do you have any adivece for me? I am gald for any input. Thank you very much. : )

@user-99bf85 Sorry, I don' t understand what you mean by newer unity version and old unity version

@user-ac3779 Have you tried using Pupil Capture instead of Pupil Service ?

@user-ee433b I still have Unity freeze sometimes after a session, not figured out why sometimes it just hang there...

@user-8b1528 Yes. But when I use Pupil Capture, unity hangs and I have to close it from the task manager before I could start it again.

@user-8b1528 For me, it is automatic. If the connection is done, Unity will crash

I'm using Pupil Capture V1.10.20, maybe you can try that one and see if better and could help find why the version 1.11.4 would hang Unity more now...

@user-ac3779 But using Pupil Capture pass the calibration scene OK compared to using Pupil Service ?

@user-8b1528 Once I play the game and about to start calibration, unity will hang with pupil capture running. But it doesn't with pupil service

@user-ee433b Thank you for your reply. The scene was built with a Unity 2018.3.9 version and for Pupil Service to work with Unity I am using Unity 2017.4.23. But once I open the scene in Unity 2017.4.23 I recieve many compiler errors. Does this help to understand my problem ?

@user-99bf85 When you import a scene in Unity, it compiles and modify some files. When you trying to import that same scene in a previous version of Unity, some new info brought by a newer version of Unity can confused and even make the scene not usable in that previous Unity version... Many times in many softwares, trying to open a project saved with a newer version into an older version fails since the structure of the project has changed (unless the software allow you to save your file with the mention "Make compatible with previous version" a bit like Adobe Photoshop does...)

@user-ac3779 I'm still using Pupil Capture V1.10.20 and not tried the V1.11 yet, will try to find a moment to try it and see if it makes my Unity hang or not. Will keep you posted when it will be done...

@user-8b1528 okay. Thanks so much for your response

@user-8b1528 Thank you for your response. Do you know if there would be any way to do this ("make compatible with previous version") in Unity.

@user-99bf85 never seen that option in Unity, I just wanted to clarify that in some software, that option exist

@user-99bf85 Usually what I do when I want to open a project with a new version of Unity, I make a copy of it to keep the original one intact. That why when you start Unity, when you select your project, they provide you with which version was used last to save that project

Hi all, I have a question. I have recorded some data and I am trying to analyse the pupil diameter. I know that I should first clean the data. by looking at the confidence and keep the data that carries a confidence value greater than ~0.6. Then I look at the diameter of pupil column and I believe that I should make some sort of a change there because diameter is for example 54.80 but the diameter should be from 1.5 mm to 8 mm So I am thinking, I should calculate it somehow? (Maybe in my example I change the decimal point ti 5.48?) Lastly I want to analyse it. I have some ideas how but any advice would be helpful.

@user-8b1528 Okay. Thank you again for explaining!

@user-0eef61 I seen a post that maybe can help you in the hmd-eyes-alpha section: https://discordapp.com/channels/285728493612957698/553513210834190367/559646840098914314

They seems to talk about number of pixels...

Have not played with those kind of data yet on my side.

Hi @user-8b1528 Thanks for your advice. From those messages I can see that I have the data in pixels whereas in mm I should have clicked 3d in pupil capture to get the diameter _3d and that's bad news to me because I have already recruited participants and payed them and got data from the eye tracker but I have got only this diameter and not diameter 3d. My hope is to know if there is any chance to use this data for now to analyse anything rather than start recruiting participants again and paying them again. I have asked some people and hopefully they know anything related to this.

@user-8b1528 I finally got the calibration to work with pupil service and it loaded the market scene after it ended. So how can I record the eye tracker data as pupil capture is not working with my unity.

@user-8b1528 I got the recording also.

so, is there any way to get the exported data in realtime (computers system time) timestamps instead of the current session time in seconds from start?

nevermind i think i found the info in pupil channel

epoch time etc

how do you actually quit out of 2d calibration demo vr ???

and is that the only way to calibrate a session for VR (im not making my own unity game or anything. i just want to calibrate for a user, then close the calibration software once it is complete, and start a recording. then do another task in vr. then finish the recording)

every time i try to quit out of the 2d calibration VR demo, it freezes up. there seems to be no quit command, just alt-f4

Hi @user-ee433b, I also experienced this behavior but only rarely. We added a fix for this issue in our current pre-release of hmd-eyes(hmd-eyes-alpha). You might want to give it a try but be aware that it is still in alpha.

hey @user-24270f, the 2d vr demo is just a demo and not intended to be used as a pre calibration application but I understand your use case and the idea of combining vr applications like that is an interesting approach. currently you won't be able to adjust the calibration routine without going into unit. you also need to be aware that the calibration results are very dependent on the camera settings in VR, if both applications aren't using identical cameras the calibration won't be of much use.

Hey,

I have a question about the 2D/3D calibration. It seems, that there is a problem with the accuracy of the calibration. If the Calibration is succesfull I use a scene (e.g market scene demo or my own scene) to check how precise my eyes are tracked. But it seems that there is a always a slight (sometimes bigger) offset. For Example if I look at an Object, the Linerenderer will not be on that exact point, but rather a bit to right/left (the raycast will not collide with the object and I will not be able to tell if the user looked at a certain object). This is especially problematic for objects which are far away. Is there a way to increase the accuracy or do any of you have the same experience? Is that something I should expect? What I did so far is to increase the amount of Circles and their dimensions and also the samples per depth.

I'm currently using pupil capture v1.11-4 and the latest hmd eyes (https://github.com/pupil-labs/hmd-eyes) with a Vive Pro. My Unity Version is 2017.4.0f1.

Thanks

@fxlange @user-8b1528 I've reinstalled the drivers and now it works fine, no more crash when I stop the scene

@fxlange ok. If I understand correctly, there is no way to calibrate pupil capture for a generic VR use for agnostic data collection. The best I will get is to just start the capture and "hope" that I get enough 80% + confidence for meaningful data

Hi @user-24270f, no - for a generic VR this won't be possible. Only if you know all parameters of your VR camera(s) - especially parameters regarding the viewport to 3d transformation - you might be able to setup the hmd-eyes calibration in a way that it matches your VR application. On top you need a way to make sense out of your recording, you would need to capture what the participant is looking at. A screen capture might not be sufficient.

We discussed it internally and definitely see the use case. If anyone has an idea, knows a different approach - happy to hear.

I don't have engine level access with the VR software I'm using, but I have scripting access. I'll try run a basic "calibration" at the start of a game session (follow the moving circle) and use that to calibrate it in post for generating my own heat map. It probably won't work, but I'll try

If you're interested in more details of my project and wouldn't mind throwing ideas around, feel free to PM me at your own convenience

Hi

Sometimes, I have a crash Unity while using pupil-HMD. I kill the process, quit capture, ... And after that, when I restart everything, the HMD-plugin don't want to connect to capture anymore

I have the message: "trying to connect to pupil. Please start Pupil capture/service ...", but nothing happend

usually, if it doesn't manage to connect, it's writing trying to connect in 5 seconds in the terminal

here... nothing at all

any idea?

I tried to debug and I think I've found where the error comes from

Here is the code public static IEnumerator Connect(bool retry = false, float retryDelay = 5f) { UnityEngine.Debug.Log("Wait : " + Connection.isConnected); yield return new WaitForSeconds (3); UnityEngine.Debug.Log("ok");

    while (!IsConnected) 
    {

I add the 2 debug lines and here is what come out of the terminal: wait : false

@user-ee433b and you are not able to reconnect after restarting capture as well?

I've closed everything, I've shut down my computer for lunch time and when I come back, it's still not working

I've created a new Unity project, imported SteamVR and Pupil.Import.Package.VR.v0.62 and it manage to connect without problem

Any idea?

Hi everyone! I was wondering if anyone could help me understanding how I should use the Connection.cs script that is showned in the Accesing data section of the https://github.com/pupil-labs/hmd-eyes/blob/master/Developer.md Should I attach it to any object in the scene I am launching after Calibration? I looked for obejects in the demo scenes that have Connections.cs attached but I couldn't find one. I tried recording data anyway but althought it created the data folders no files were in it. Thank you in advance for your help