You reconstructed the hmd-eye project. It's very exciting. But I have some questions now
@user-e04f56 What went wrong?
@mpk
@user-5ca684
@user-abff35 I checked with the server version of PupilDefinitions.cs and it does not have any reference of 'Calibration' in it. the specified line (126) should actually just have a '{' in it. So my guess is that there was a merging error. Can you compare it to the GitHub version https://github.com/pupil-labs/hmd-eyes/blob/master/unity_pupil_plugin/Assets/pupil_plugin/Scripts/Pupil/PupilDefinitions.cs or send me your file?
it's my file. Thank you
I ran the Calibration.unity first,then i import the Pupil Import Package.unitypackage,Mistakes happen
oh, but good to know.. need to update the Import Package
Is that what I did wrongοΌ
the unitypackage is actually meant for new projects
if you run the GiHub project, you don't need to import it
but I should clarify that in the Readme
ahaha, But if I don't import the unitypackage .There will be other mistakes.
like this
something went wrong with either the merge into the Master branch or the pull request. DLL's and the ffmpeg.exe were ignored for some reason.. I just added them manually
can you pull the newest version?
just did it on my side and imported it as a new Unity project and it compiled without errors
all right π
thanks for the feedback
I've tried to add the pupil gaze tracker object to my scene and am getting his error
@user-a36fbf may I ask in which branch you are working on and when you updates last? We removed GL-based drawing and CalibrationGL.cs (shown in your list of errors) some time ago.
It seems that I'm in a very old branch. I've updated it and it seems that the issue is gone. Thanks for the help
Hi guys. I'm using unity with the pupil labs software and the unity pupil labs plugin. I would like to know where the user looks in the unity world but I don't manage to get this data
@user-c0934a have a look at the Market Demo Scene. It exemplifies how to - after a successful 2D calibration - access the tracking information sent by Pupil. You can use PupilData._2D.GetEyeGaze to get the current viewport position and if you need to, convert it to 3D space with either Camera.ViewportToWorldPoint or Camera.ViewportPointToRay
Hi guys, I'm messing around with the pupil inserts for the Vive in unity here and i'm getting a bug where when i use the "autorun pupil service" option it will open 2 captures for eye 0 and 2 background services that i have to close using the task manager. Any idea what causes this?
@user-c07d29 Pupil service spawns a new process and a new window for each eye that it's processing data for. I think the 'eye 0' label for both windows is just an incorrect label either at the UI level or at the camera driver level, but you should verify that the video feeds in each window are indeed different. If you minimize the windows then the eye data will continue to be processed but you will save a bit of CPU by not rendering the camera images.
sorry for not being clear earlier, it's actually opening 3 capture windows, 2 eye 0 and an eye 1 so the labels are correct. It looks like the gaze tracker auto run is trying to launch the service twice but i haven't been able to find any errors in the code on the github that does this.
If you look in your device manager, do you have 3 cameras installed?
nope, 2 pupil cams but they both have ID0 which could be the issue
no, that's not it. I have the same problem.
(with ID0)
I usually run the service first, before starting up Unity, and I haven't noticed Unity trying to startup yet another service instance. But I don't work with this on a daily basis, there's a student in the lab that does that. He says that he had the same problem as you before the dev branch was merged in (6 days ago).
yea, i'm looking through the code now and i'll probably go through it more thoroughly tomorrow to see if i can find what's causing it
found the error and made a PR to fix
Hi all, does anyone know when the ffmpeg components of the pupil plugin will be tested/working? I have no problems calibrating and seeing the market scene but I can't initiate recording from within Unity due to the ffmpeg missing error and would like to use that functionality
@user-413327#7188 thanks for the PR. we were not able to get to it, today, as we worked on getting out a version with 3d calibration. will work on it, first thing next week, though. initial thoughts: your PR replaces pupil_capture with pupil_service, but I think we will need to check for both to cover all use cases.
@thepod#1280 did you change back to the master branch? I actually thought we fixed the missing files problems with the recent release versions. the dev branch is a little behind, at the moment. will try on our end, as well.
@user-e04f56 please delete the dev brnach if it is behind master.
we will continue with feature branches instead of master/dev
In the hmd unity plugin, is there any way to record the pupil data automatically (like the pickled data the pupil capture creates) or is it just the ffmpeg output?
Hi, we are using unity pupil plugin but we are facing some accuracy issues. What are the good steps to get the accuracy better ? Best, T
@grimeywimey#5261 we currently only have the ffmpeg solution to capture gaze information. Recording/saving data in other formats can be done, but it is a very customer specific feature that varies from use-case to use-case. We have plans for a heatmap demo scene, though, which may be something you are thinking of.
@TLK#2768 could it be a result of the calibration pattern? Accuracy can be improved by adapting the marker placement, e.g. drawing them closer to the center (as this is where most users point their gaze) or adding more points to the process. In case of the DK2, the standard calibration pattern actually needs to be spread out to cover the gazing area.
Hi! Today, I downloaded a new version of unity pupil plugin and tried 2 Demo, 2D calibration demo and 3D calibration demo. 2D calibration demo worked well but in 3D calibration demo there was no visualization for gaze tracking. What should I do? I thought a white colored marker would appear on the scene.
@user-e04f56 Okay thanks you for the answer. We are using the basic calibration pattern currently. We would like to know if you consider the fact that the hmd rendering is done for each lens and it might be some distortion if the corners
I found that two Pupil Service windows disappear after calibration in 3D calibration demo unlike 2D calibration demo. Is the reason why I can't get any result in 3D calibration demo?
@user-56b6a5 please use Pupil Capture instead of Service. I think there might be a problem with Service. @user-e04f56 what software where you using to test/develope?
@mpk Thank you! It's working! But a white marker is flickering, so I can't see the result well. Is it because of inaccurate calibration?
@user-56b6a5 , @mpk I switched to using Pupil Capture, as well. But I will run some tests with Pupil Service to see if I get the same problems
@user-56b6a5 as for 3D calibration: if the 3D scene is shown then at least the calibration succeeding. Once calibrated, can you expand the "Demo" gameobject in the Unity hierarchy and select "Gaze_3D". If the Transform in the Inspector does not update at all, you are not getting any positional updates. Otherwise, the received positions could just be outside the field of view..
@user-e04f56 I just tried the 3d position as well and I noticed that the marker is, mostly, behind the near clipping plane. Any idea what is going wrong? I've tried a couple of times and the calibration seems succesful. One thing I've changed is lowering the y values of the Vector Depth Radius scale to support a display. I used the following y values: 0.18, 0.35, 0.45
@user-e04f56 The white marker is moving well but it repeats appearance / disappearance. It appears, then disappear for a few seconds, and appears again. What will be the reason?
Hi!. I am trying to implement eye tracking in Hololens. since we don't have a working hololens-unity plugin right now, what is the workaround for this issue? Can I try to modify hmd-eyes plugin for hololens or is it too difficult to achieve?
@user-ed036b socket communication proofed to be a big problem with the combination of Unity and HoloLens, so our current approach is to try a UDP-based solution: As Pupil Capture/Service will have to run on a remote PC in any case, we are currently working on a Unity UDP client to run on that machine as well (basically doing everything placed in Communication.cs). The results are then sent to HoloLens.
Hi, I am trying to synchronize head direction with gaze data after recording. Is there a way to get Unity to export the time since windows start in the format you use for time stamps like with this function? Or alternatively is there a way to record the absolute start time (as in info.csv) with more precision than seconds so I can do this matching myself? I think a feature of capturing this info would be really helpful. Thanks!
Like if this could be used in Unity or recorded in info.csv:
cdef double get_sys_time_monotonic(): return timeGetTime() * 1e-3
@user-e04f56 Thanks for the reply. That was good info on the complexity of the problem. Are you targeting a particular date for the release?
hi
setup
"the path is set to C:/Program Files/Pupil Service/pupil_service.exe" in unity?
where is path setting menu in unity?
@B.suke#1636 you can either select the PupilGazeTracker gameobject and set the path in the Inspector
Just click the "Browse" button
or select PupilSettings in pupil_plugin\Resources, which will show this in the Inspector
Under "Pupil Service App" you can set the service path
is the function to pull the confidence from each eye in script in the pupil plugin anywhere? Anytime i call the PupilTools.Confidence with any eye ID it gives me a null reference exception
also, is there a way i can save a calibration so if i'm doing demos of the hardware, just so i don't have to run through the calibration each time? (I understand you're not supposed to but humour me)
Hi.
I think I properly installed all necessary packages in "https://docs.pupil-labs.com/" and want to connect pupil-labs devices with HTC-Vive device.
We want to collect coordinates data where people are looking at in a 360 video using the pupil device and it seems that to do so, we can test it using this library (hmd-eyes).
Right now, we want to make connection between Unity and this pupil device in the HTC-Vive device and followed all steps in "Changing standard settings via PupilGazeTracker".
After playing, the current scene shows us the "Connecting to pupil." message and we can see the message in the HTC-Vive device as well. However, the Unity stops often and it doesn't show the word "Connected" as described in 3 in this section, "Changing standard settings via PupilGazeTracker". If I press a button, "IsConnected", in Main of Pupil Gaze Tracker, it shows me Connected but it works although the device is not really connected even. So, I don't really think they are connected.
Do you guys have any ideas of how to fix it? Thanks.
This is the button I pressed. Even though they are not properly connected, I can see "Connecting to pupil." or press Calibrate button to see a calibration point in the middle.
The calibration point can be seen in the Vive.
Has noone faced the same situation I am having? Where can I get some advice from?
when you hit c to get the calibration point does it throw any errors?
Hey I got a question, in the old Pupil we are trying to get the pupil diameter within Unity, but none of the values there seem to be passed to the Unity client, how can I fix this?
I don't know why but it stops when I play the unity_pupil_plugin in the unity. Is that because of some installation problems?
Pupil capture application runs successfully and I can extract gaze_positions.csv from the recorded files. However, when it comes to this hmd-eyes library, it doesn't work.
I think one of the problems comes from librealsense. When I run example codes like pointcloud, (https://github.com/IntelRealSense/librealsense/tree/master/examples), the program is terminated because of some memory related errors.
What can be the problem? Has anyone faced the same issue?
@khg2478#8260 would you be open to a remote session so we can try to debug if something is going wrong in Unity?
@grimeywimey#5261 regarding the PupilTools.Confidence problem (sorry for the late reply): Did you subscribe to both "pupil.0" and "pupil.1" instead or in addition to "gaze"? This is required for the Confidence method to work. After that you get values for each eye, which are then saved to a respective dictionary. These dictionaries are probably the reason you get the mentioned null reference exception.
@user-72b0ef are you referring to the 'diameter_3d' variable, which is part of the Pupil datum format? To use this in Unity, PupilData simply has to be adapted to look for the value in the received dictionary
@user-e04f56 We are indeed refering to the 'diameter_3d' variable. We can already extract the 'confidence' (for both eyes individually)and write it to an xml datafile. But when wetried getting the 'diameter_3d in the same way it would permanently return '0'. Even though in pupil_capture it would show the current diameter_3d.