vr-ar


user-6e3ee7 02 May, 2017, 07:40:36

Hello!

Greetings, I have question about wearing eyeglasses with HMD. From our experiments, the eye image being generally out of focus (blurry) and a lower contrast ratio with eyeglasses, plus reflections of the LEDs from HMD on the eyeglasses make the pupil detection accuracy drop significantly.

Since Pupil-Labs' doc claims detection works with users wearing contact lenses and eyeglasses so wondering if it's limitation of HMD device? Thanks!

user-41f1bf 02 May, 2017, 12:24:52

To check calibration I am asking participants to look to some dots in a standard layout so that any eye-dot offset is noticeable.

mpk 02 May, 2017, 14:48:20

@user-6e3ee7 for hmds tracking works well for contacts. Eye glasses are very difficult unfortunatelty.

user-64e12b 09 May, 2017, 11:37:11

Hello! Unfortunately I'm having the same problem as vivi, with only one eye appearing in the unity project, I am using the latest one from GitHub, any suggestions?

wrp 09 May, 2017, 11:38:38

@user-64e12b did you try the dev branch with latest pupil capture or service (v096)?

user-64e12b 09 May, 2017, 11:39:26

Ah no, I was on master. Will check dev now. And yes v0.96, thank you

user-64e12b 09 May, 2017, 12:46:45

Hmm, now I seem to be getting 0's on both of the displays

user-64e12b 09 May, 2017, 12:48:47

The service is running, and when I turn on "print msg" and put on the rift I actually see the messages change to confidence:1 and the base_data variable change

user-64e12b 09 May, 2017, 12:49:29

Just wishing to run something to show me where I'm looking live

wrp 09 May, 2017, 12:59:20

Are eye cameras feeds selected and visible in service or capture?

user-64e12b 09 May, 2017, 13:08:53

Yeah, they are both apparent

wrp 09 May, 2017, 14:10:25

@user-64e12b - you should be seeing feedback in the plugin that you are connected and if both eyes processes are detected, then you will see two green icons in the plugin GUI

user-efb835 09 May, 2017, 19:36:54

Hi hmd-eyes channel! I'm trying to get the pupil headset to work with the Epson Moverio. Has anyone tried this before? Right now, when I run it on the Moverio, it only shows a tiny green circle in the left corner. How would I know if both eyes were detected / calibrated?

user-64e12b 10 May, 2017, 08:49:41

@wrp

Chat image

wrp 10 May, 2017, 08:50:35

@user-64e12b - it would appear that both eye processes are detected/connected

user-64e12b 10 May, 2017, 08:59:09

On the plugin's code, I see that OnGazePacket is activated, and even though the _pupilDataDict has values like "confidence = 0.97, norm_pos = [0.49, 0.66]" the values returned in Pupil.values.GazePoint3D are still 0,0,0

user-64e12b 10 May, 2017, 08:59:17

Maybe a calibration issue?

user-64e12b 10 May, 2017, 09:00:01

I'm just trying to actually see the point that I'm looking at on the screen, to know if the device is actually calibrated..

user-5ca684 10 May, 2017, 09:02:08

@user-64e12b you have to calibrate to either 2D or 3D

user-5ca684 10 May, 2017, 09:02:16

you can set it up under the settings

user-5ca684 10 May, 2017, 09:02:26

sorry the documentation is not complete yet

user-5ca684 10 May, 2017, 09:03:56

@user-64e12b is it possible that you have not switchet to 3D mode for calibration?

user-64e12b 10 May, 2017, 09:06:12

@user-5ca684 I did, and I just performed another calibration but the results are the same I'm afraid. (and when support is nice and available the lack of documentation isnt that bad :))

user-64e12b 10 May, 2017, 09:06:42

In the code I also see CurrentCalibrationMode = _3D

user-64e12b 10 May, 2017, 09:07:00

while debugging i mean

user-5ca684 10 May, 2017, 09:07:05

sure

user-5ca684 10 May, 2017, 09:08:50

im pulling up the project now to see what can be the problem, but it seems to me that for some reason you are not getting the data from the service

user-64e12b 10 May, 2017, 09:09:41

Okay thanks. The weird thing is that when printing messages I can actually see stuff change, like confidence, and other variables when I assume are from the service

user-5ca684 10 May, 2017, 09:10:00

no its not wierd

user-5ca684 10 May, 2017, 09:10:24

if the 3D calibration is not complete or failed you will still get some data in

user-5ca684 10 May, 2017, 09:10:31

its just not 3D

user-5ca684 10 May, 2017, 09:10:55

the IPC bacbone is transmitting the pupil data either way

user-5ca684 10 May, 2017, 09:11:09

are you using sercive or capture?

user-64e12b 10 May, 2017, 09:11:25

currently capture

user-64e12b 10 May, 2017, 09:11:31

and I see, that makes sense now

user-5ca684 10 May, 2017, 09:12:03

could you check in capture if its on 3D mode?

user-64e12b 10 May, 2017, 09:13:55

detection & mapping mode?

user-5ca684 10 May, 2017, 09:14:03

yes

user-64e12b 10 May, 2017, 09:14:11

yes it is on 3d

user-5ca684 10 May, 2017, 09:14:17

hmm

user-64e12b 10 May, 2017, 09:14:18

Should I be seeing 3 windows?

user-64e12b 10 May, 2017, 09:14:24

One capture, one for each eye?

user-5ca684 10 May, 2017, 09:14:25

yes

user-64e12b 10 May, 2017, 09:14:37

oh and one terminal

user-5ca684 10 May, 2017, 09:14:37

that all sounds about right

user-5ca684 10 May, 2017, 09:14:43

yep

user-5ca684 10 May, 2017, 09:16:11

im going to check it right now on my machine locally, because lately I have tested it mainly with remote mode, however that shouldnt make a difference

user-5ca684 10 May, 2017, 09:16:32

except of course if there is an issue with the binary version

user-64e12b 10 May, 2017, 09:17:30

Does it make sense that I have to click "detect eye 1, detect eye 2" every time? the windows for the eyes don't open automatically

user-64e12b 10 May, 2017, 09:17:46

not that I really mind, just wondering if it might indicate something

user-5ca684 10 May, 2017, 09:17:51

hmm, no they should

user-5ca684 10 May, 2017, 09:21:07

in your version do you have an Autorun Pupil Service under Settings>>pupil app>>

user-5ca684 10 May, 2017, 09:21:09

?

user-64e12b 10 May, 2017, 09:21:31

yep

user-64e12b 10 May, 2017, 09:21:37

I toggled it on

user-5ca684 10 May, 2017, 09:21:52

try it off and run the service manually

user-5ca684 10 May, 2017, 09:22:30

I mean this is probably not going to solve the data issue, but this did bring the eye detection up for me just now

user-64e12b 10 May, 2017, 09:22:50

on "local" or "remote"?

user-5ca684 10 May, 2017, 09:23:08

im trying on local, because I see you are on local too

user-64e12b 10 May, 2017, 09:24:09

yeah

user-64e12b 10 May, 2017, 09:24:15

well this way I'm not getting "connected" even

user-5ca684 10 May, 2017, 09:24:50

that is kind of wierd

user-5ca684 10 May, 2017, 09:25:08

have you made sure that the Service port matches the one in capture?

user-5ca684 10 May, 2017, 09:26:04

okay im getting the data

user-5ca684 10 May, 2017, 09:27:13

okay a couple of things to check:

user-5ca684 10 May, 2017, 09:27:49
  1. service port stated in Capture has to match the one in Unity in pupilgaze Settings
user-5ca684 10 May, 2017, 09:28:37
  1. try to manually start the Pupil Remote plugin in the Capture (this should be on by default)
user-5ca684 10 May, 2017, 09:29:12
  1. once connected, calibratie for 3D and check the output log in the terminal of the capture
user-5ca684 10 May, 2017, 09:30:55

(Im taking a note to fix the issue with local automatic start, I believe with service it would work. I think the reason to this problem is that the request happens before your machine can bring up the Pupil Capture, so im gonna have to loop until the eyeprocesses are on)

user-64e12b 10 May, 2017, 09:41:24

Okay, gotta go for a bit but will do and update on results. Thank you!

user-5ca684 10 May, 2017, 09:42:25

allright, just send me a PM as well please

user-efb835 10 May, 2017, 13:39:02

! I'm trying to get the pupil headset to work with the Epson Moverio. Has anyone tried this before? Right now, when I run it on the Moverio, it only shows a tiny green circle in the left corner. How would I know if both eyes were detected / calibrated?

wrp 11 May, 2017, 06:18:13

@here - just a friendly reminder that Pupil software on Windows is only for Windows 10 (while you may get things up and running on Windows 8 - it is really not advisable to use anything other than Windows 10)

wrp 11 May, 2017, 06:19:29

@user-efb835 - apologies for not responding earlier - could you please describe your setup a bit more? Are you using the Pupil Labs BT-200 or BT-300 add-on?

user-5b7b52 11 May, 2017, 13:40:42

Universal Windows Platform (UWP) is not yet supported right? NetMQ seems to support UWP since a couple of days (May 6th) https://github.com/zeromq/netmq/commits/master . Are you planning to upgrade to this version soon?

user-efb835 11 May, 2017, 15:28:34

I'm using Windows 10 on an Alienware machine. I have the Pupil Labs BT-200 and do not have the add on. I currently have the pupil headset plugged into the laptop and I'm trying to get the Unity Integration program to run without the headset - just to see if the ports work. I have Pupil Capture running on another machine with the Pupil Remote plugin. Right now, the errors I get in Unity on my Alienware machine have to do with Unity not being able to connect to the pupil remote server.

user-efb835 11 May, 2017, 15:32:09

Both devices are on the same wifi network

user-efb835 11 May, 2017, 15:33:41

I am also using Unity 5.5.1f1

user-5ca684 11 May, 2017, 15:34:54

@user-efb835 could you tell me the exact error message?

papr 11 May, 2017, 15:36:00

Dear @user-efb835 are you using the newest version of the unity integration?

papr 11 May, 2017, 15:36:41

And did you set the IP address of the computer running Pupil Capture in the unity integration?

user-efb835 11 May, 2017, 15:39:04

Yes I set the IP address and Port in the Pupil Listener attribute

user-efb835 11 May, 2017, 15:39:24

The error is Failed to connect the server; Context Terminate; Quit the Thread

user-5ca684 11 May, 2017, 15:39:33

@user-efb835 hmm that doesnt exactly sound right

user-efb835 11 May, 2017, 15:39:46

I downloaded the project off of github's hmd-eyes today so I think I have the most recent verge

user-5ca684 11 May, 2017, 15:40:02

there should be a PupilGaze GameObject

papr 11 May, 2017, 15:40:02

yes, that is the newest version

user-efb835 11 May, 2017, 15:40:05

It works if I have both the pupil capture and unity project on the same computer

user-efb835 11 May, 2017, 15:40:27

I'm using the unity_integration project now; not the unity_integration_callibration

user-5ca684 11 May, 2017, 15:40:35

ahh I see

user-efb835 11 May, 2017, 15:40:58

I'm using the IP and port from Pupil Remote

user-5ca684 11 May, 2017, 15:40:59

I have not updated the unity_integration at all

user-5ca684 11 May, 2017, 15:41:23

all the new work has been done under the unity_integration_calibration

user-5ca684 11 May, 2017, 15:41:30

try to connect with that

user-efb835 11 May, 2017, 15:41:34

okay

user-5ca684 11 May, 2017, 15:42:52

so there should be a PupilGaze GameObject >>Settings>>pupil app

user-efb835 11 May, 2017, 15:44:39

unity integration callibration works when the pupil capture is on the same computer as the unity program. Trying now on separate devices

user-efb835 11 May, 2017, 15:46:22

Still same error

user-5ca684 11 May, 2017, 15:46:43

could you please copy paste the error

user-5ca684 11 May, 2017, 15:46:47

?

wrp 11 May, 2017, 15:46:59

Just to clarify - are you using the dev branch?

user-efb835 11 May, 2017, 15:47:14

Nope - master

user-efb835 11 May, 2017, 15:47:41

Failed to connect the server; PupilGazeTracker:NetMQClient()

wrp 11 May, 2017, 15:49:29

@user-efb835 could you please check out dev branch?

wrp 11 May, 2017, 15:50:09

And try the unity integration calibration plugin

user-efb835 11 May, 2017, 15:50:17

okay

user-efb835 11 May, 2017, 15:51:52

I get a bunch of errors b/c i don't have an HMD headset plugged in. Is it possible to test the remote capabilities first?

user-5ca684 11 May, 2017, 15:52:09

yes it should be

user-efb835 11 May, 2017, 15:53:29

I get Type "UnityEngnie.Camera" does not contain a definition for 'allowHDR' and no extension method can be found. Are you missing an assembly reference?

user-efb835 11 May, 2017, 15:53:50

also "does not contain a definition for 'allowMSAA'

user-5ca684 11 May, 2017, 15:54:08

which version of Unity are you running?

user-5ca684 11 May, 2017, 15:56:14

this might be a Unity 5.6+ feature on the Cameras, if you do not want to update your Unity now could you just comment these two lines out? Later I am going to make this dependent on the Unity version number.

user-efb835 11 May, 2017, 15:56:17

I am using Unity 5.5.1f1

user-5ca684 11 May, 2017, 15:56:55

im sorry for the trouble, didnt know 5.5 didnt have these features

user-efb835 11 May, 2017, 15:57:18

No worries! Now I get OpenVR failed initialization with error code VRInitError_Init_HmdNotFound

user-5ca684 11 May, 2017, 15:57:26

that is fine

user-5ca684 11 May, 2017, 15:57:47

but the other two lines might prevent you from running

user-efb835 11 May, 2017, 15:58:49

Okay - now I get NullReferenceException: PupilGazeTracker.OnDestroy()

user-5ca684 11 May, 2017, 15:59:16

without running?

user-efb835 11 May, 2017, 15:59:50

when I start to run

user-efb835 11 May, 2017, 16:00:13

or rather stop running

user-5ca684 11 May, 2017, 16:00:28

that makes more sense

user-efb835 11 May, 2017, 16:00:30

Also - I don't get any Logs about being connected to server, etc

user-5ca684 11 May, 2017, 16:00:45

I will look into it!

user-efb835 11 May, 2017, 16:00:49

nothing happens when I press play - thank you!

user-efb835 11 May, 2017, 16:00:54

I seriously appreciate your help!

user-5ca684 11 May, 2017, 16:01:18

you might have to set the AutoRun Pupil Service under the PupilGaze

user-5ca684 11 May, 2017, 16:01:55

:

user-5ca684 11 May, 2017, 16:02:06

Local / Remote

user-5ca684 11 May, 2017, 16:02:15

if Remote, the IP address

user-5ca684 11 May, 2017, 16:03:12

you can choose to have AutoRun Pupil Service app or not, if you choose to do it manually, you can start / stop it with the two buttons under the toggle

user-efb835 11 May, 2017, 16:05:43

I see - hold on

user-efb835 11 May, 2017, 16:07:29

Still no luck - I've set the IP and reset the port on capture appropriately

user-5ca684 11 May, 2017, 16:08:17

how about the autorun?

user-efb835 11 May, 2017, 16:10:07

yep that is set as well

user-5ca684 11 May, 2017, 16:10:30

okay, set to ON?

user-5ca684 11 May, 2017, 16:11:09

have you set it ON, before running the application or after? if you toggle it while running, it will take no effect

user-efb835 11 May, 2017, 16:11:36

I have it set to on before the app runs

user-5ca684 11 May, 2017, 16:12:07

okay, and so nothing happens when pressing play in Unity

user-5ca684 11 May, 2017, 16:12:11

no error messages?

user-5ca684 11 May, 2017, 16:13:06

you are sure that its set to Remote?

user-5ca684 11 May, 2017, 16:13:31

the last thing I can think of is some kind of Firewall issue that blocks the connection

user-5ca684 11 May, 2017, 16:14:20

if you dont mind, could you PM me screenshots of your setup?

user-3ad5b4 14 May, 2017, 07:56:08

Hello! How do i apply the project "unity_integration_calibration" on hololens ?

user-f7028f 15 May, 2017, 06:42:57

Hello everyone! I'm newbie in vr and pupil development. I've carefully read all docs but still haven't a general vision how to sort out all of this staff. I'll be really appreciate for any help and tips! The background: 1) I have HTC VIve and Pupil's HTC Vive add-on. Add-on is connected to separate USB port beside Vive. The idea: 2) My goal is to get gaze direction/coordinates to some static picture in HTC Vive (it's first stage of experiment) The experiment: 3) I'm able to run Pupil-capture and see 2 window for 2 eye-cameras. But world camera doesn't show anything and in the list of capture source I see only 2 eye-cameras and several unknown source (is it vive's camera or my laptap camera or what?) The issue: 4) According to docs, first of all I have to calibrate eye-cameras with world camera. How to impelemt that with HTC Vive + Pupil's HTC Vive add-on? Is it possible on Windows 10 or have I setup all environment under Linux with hmd-eye blocks?

It's a common observe of my stuck ) I didn't dig deeply into development of my own plugin or something like this. I need to sort of that general issues first. Thanks in advance!

user-5ca684 15 May, 2017, 09:47:45

@user-f7028f Could you tell me which version of the unity_integration_calibration are you running?

user-5ca684 15 May, 2017, 09:49:29

@user-3ad5b4 I do not have experience with the hololens in Unity, but the plugin should work as long as you have pupil cameras connected to either local or on a remote machine

user-f7028f 15 May, 2017, 09:49:55

I doesn't running any 3rd-party apps but just pupil capture, i can't find any docs about it. Have I run some additional apps to make "connection" of HTC Vive and Pupil's add-on?

user-5ca684 15 May, 2017, 09:50:06

@user-3ad5b4 the documentation for the plugin needs a lot of work yet, but for now I can walk you through if you want

user-5ca684 15 May, 2017, 09:50:51

@user-f7028f Im sorry I might have misunderstood you

user-f7028f 15 May, 2017, 09:51:13

as I said, I'm very begginer in this area and have to learn from the very beggining how it works )

user-5ca684 15 May, 2017, 09:51:19

@user-f7028f I thought you are about to use the unity_integration_calibration

user-1486c3 15 May, 2017, 10:18:12

Hi all, I have some problems with the unity plugin. I am running my game and the data is stream quite well. But after a few minutes (it varies) the data stream just stops. the capture app is still working, the eye cameras are still working and getting good tracking, the game in unity still works, but no data is getting to the game. I basically know of this only when I stop the game, the unity app is not responding, and the eye processes are still on. I also collect the data for later use, and I see that the time course is much shorter then the game was. I tried to look at the capture log in the settings directory, but no notification marks the time it stopped. is this something that you know, or have seen and can tell me something about it? it is really crucial that I will be able to track for about 15 minutes straight. thanks

user-5ca684 15 May, 2017, 10:20:13

@user-1486c3 first of all, I want to make sure that you are using the correct version, which is in the dev branch (unity_integration_calibration)

user-5ca684 15 May, 2017, 10:21:15

@user-1486c3 are you recording with the plugin or you are just simply running it?

user-1486c3 15 May, 2017, 10:30:06

I am not sure what you mean by recording, but I am subscribed to the gaze messages through unity, and save the data that comes in in a csv file. I am not recording with the capture. I am not sure what version it is, as I have downloaded the plugin and made some changes to it, becuase the _pupilData object didn't receive all the data. the changes are these:

Chat image

user-1486c3 15 May, 2017, 10:31:46

just added an envelop class so that the json string will be loaded properly, and not only the norm_pos

user-5ca684 15 May, 2017, 10:33:40

I see, well there is a development in the unity_integration_calibration, I believe the one you are using is not where the new work has been done

user-5ca684 15 May, 2017, 10:34:47

you could try that out, but if you have already made a lot of work in the customization, I understand that might be inconvinant

wrp 15 May, 2017, 10:35:14

@here - lots of work has been done in the dev branch - https://github.com/pupil-labs/hmd-eyes/tree/dev/unity_integration_calibration

wrp 15 May, 2017, 10:35:30

we plan to merge this to master after working out major issues and testing

wrp 15 May, 2017, 10:35:38

feel free to test it out and give us feedback here

user-1486c3 15 May, 2017, 11:19:46

thanks, I will look into it. is there a way to have a look at the dev branch without using git? another thing, I have been using the capture app because while using pupil service, the configurations would reset every time i start the eye process. Is there a way to save the service settings?

user-5ca684 15 May, 2017, 11:20:37

@user-1486c3 you can just download the project as .zip

user-1486c3 15 May, 2017, 11:58:18

this I did, and this version is what I used, except for the envelop class that i added. but it says that it was last updated 2 months ago.

user-f7028f 16 May, 2017, 08:59:32

@user-5ca684 hi! I've made some steps to achive my goals ) I switched from Windows to Linux (Ubuntu 16.04), installed lates Unity3d, Pupil Capture. Now I follow hmd-eyes instruction and probably all going fine (PupilGaze shows eye position and etc) but I can't figure out whether should I see something on HTC Vive headset or not? Now I see there some part of Ubuntu desktop's background

user-5ca684 16 May, 2017, 09:06:08

@user-f7028f To be honest I have only tried local connection with Unity on Win10. I have only used Ubuntu for running the Pupil Capture while connecting to it via network. About Vive and Unity its another question, in order to use the Htc Vive in Unity you will need to have a plugin active, by default we are using the OpenVR plugin. Its should be included in the project we provide on git

user-f7028f 16 May, 2017, 09:13:34

yes, I added OpenVR plugin into project settings --> player ---> other

user-f7028f 16 May, 2017, 09:14:49

Am I right that thats plugin must translate Unity scene to Vive screen?

mpk 16 May, 2017, 09:42:12
user-3ad5b4 17 May, 2017, 05:15:40

@user-5ca684 Hololens run an app on its own CPU instead of CPU on a local PC. So I plan to build a windows store app using unity and Visual studio. After building in unity, I get a .sln profile. Then I compile and release the .sln profile to hololens using Visual Studio. When I build a .sln profile in Unity, I choose the platform Windows Store.

Chat image

user-3ad5b4 17 May, 2017, 05:15:54

But it fails with the error as the picture.

Chat image

user-3ad5b4 17 May, 2017, 05:19:58

Do you have any idea on this error?

user-f7028f 17 May, 2017, 09:45:35

@user-5ca684 why so necessary to run Pupil Capture under Linux or MacOS? Why Windows isn't suggested? I asked this coz I failed to run calibration scene under Ubuntu because of Unity's SteamVR assets doesn't support Linux! >__<

mpk 17 May, 2017, 09:46:04

You can run it on windows just fine. I dont think thats the issue.

user-5ca684 17 May, 2017, 09:47:41

@user-f7028f it is just an option! I have used two computers, one with Win10+Unity. the other one with Ubuntu+Pupil Capture

papr 17 May, 2017, 09:48:16

@user-5ca684 also be aware, that Pupil Capture is independent of your unity envorinment. They communicate over the network. There is no need to run both on the same operating system.

user-5ca684 17 May, 2017, 09:48:18

@user-f7028f I have done it so, because for some reason I could not get the git source version run on win 10

papr 17 May, 2017, 09:48:31

eh, I meant @user-f7028f , sorry

user-5ca684 17 May, 2017, 09:48:39

no worries 😄

user-f7028f 17 May, 2017, 09:51:18

I got it! Now I'm trying to run it under Windows. Bytheway, should I see colibration scene in Vive when I'll run it from Unity? Should I import SteamVR assets or OpenVR plugin in your project is enough for it? )

user-5ca684 17 May, 2017, 09:52:07

@user-f7028f As I mentioned the project should work as it is out of the box with the OpenVR plug that is included

user-5ca684 17 May, 2017, 09:52:26

@user-f7028f you might have to add it though in the Player Settings

user-5ca684 17 May, 2017, 09:52:56

@user-f7028f + yes, you should be able to see the calibration through Vive

user-5ca684 17 May, 2017, 09:53:37

If you do not see anything in your Vive, that there must be some issue with the way your Vive is installed on your system

user-f7028f 17 May, 2017, 09:54:10

Thanks! I worry because under Ubuntu callibration scene running from the box with checked OpenVR plugin didn't show anything in Vive (

user-5ca684 17 May, 2017, 09:54:22

@user-3ad5b4 The plugin uses Threading for the networking and the Recording. I have never tried to deploy as a Store App /Hololens . It seems like that there is a problem with Threading when you are trying to build it as a store App

user-5ca684 17 May, 2017, 09:54:49

@user-f7028f Frankly I have never used Unity on Linux

user-f7028f 17 May, 2017, 09:55:27

ok ) will provide you with my success\fail under Windows )

user-5ca684 17 May, 2017, 09:55:38

@user-f7028f I dont know much about Vive support on Linux, but you shouldnt worry if you are running it on win10 now

user-5ca684 17 May, 2017, 09:55:51

@user-f7028f thank you, thats great!

user-f7028f 17 May, 2017, 10:26:04

It's alive!!!! It's alive!!!! 😆

mpk 17 May, 2017, 11:19:09

awesome @user-f7028f

user-a2fe55 17 May, 2017, 12:32:11

Hello everyone. I was sent here by the support to ask my question. I'm using Windows 10 and the HTC Vive as my setup. Also, I had installed Unity 3D Version 5.4.0f3 and 5.6 seperately. I was able to run the Master-Branch Unity Scene on Unity Version 5.4.0f3 and the Calibration was successful. But I wanted to get information about the Axis, in perticular the z-Axis, and was told to use the Developer-Branch Scene. So I did and started the Scene. But as soon as the Trackers were connected, the marker for the left eye disappeared and the other markers remained static. Surprisingly, I could finish the Calibration process and got no error messages, neither in Unity nor in the Calibration App. But the blue marker was still missing and the other markers weren't moving at all, after the Calibration was finished. So I just wanted to ask, do you know, why this behaviour occured?

user-5ca684 17 May, 2017, 16:18:55

@user-a2fe55 As you said the dev branch contains the latest development progress. Please note that there are two calibraiton modes you can choose from : 2D and 3D. once you press the calibration button in 2D mode the markers should switch to a white marker that slowly dissapears with your gaze. After one is gone comes the next marker position. If this is not the behaviour you experience than something is going wrong

user-a2fe55 17 May, 2017, 20:05:41

@user-5ca684 Thanks for your fast response. In that case, everything seems to work as it should. But I have another question. Is it possible for a simple Windwos Update to have an effect on the drivers, which are important for the Pupil Labs App?

user-5ca684 17 May, 2017, 20:08:27

@user-a2fe55 Frankly im not an expert on the Pupil App related questions nor on the driver of the camera, but I dont think it should do any harm. But... I guess you can always try to reinstall the drivers, re-download the latest binary etc. .. I heard regarding another case that updating to the latest binary actually solved some issues

user-a2fe55 17 May, 2017, 20:10:18

@user-5ca684 Thank you, very much. This sums it up for me, right now

wrp 18 May, 2017, 02:22:56

@user-a2fe55 - yes it is possible that a Windows update will clear user installed drivers. Please check the device manager before re-installing drivers to see if drivers are already installed when cameras are connected in the libusbK category. You may want to check "show hidden devices" just to see what is installed

user-3ad5b4 18 May, 2017, 14:37:37

@user-5ca684 Thank you. But I wonder if there will be solutions to deploying a Windows Store App.

user-4a3b48 18 May, 2017, 17:26:19

Hey Guys, I am trying to integarte the Pupil Eye with Microsoft Hololens. Has anyone tried doing it ?

wrp 19 May, 2017, 05:50:37

Hi @user-4a3b48 - I think we were in touch via email and I provided a response regarding the route some researchers currently take for Pupil + HoloLens development

wrp 19 May, 2017, 05:51:09

I will post this information below here again for reference.

wrp 19 May, 2017, 05:52:00

Researchers that are currently using the HoloLens have a setup that uses a separate socket server that talks through ZMQ Pupil Caputre and communicates using low-level UDP commands to the HoloLens. This is required AFAIK, because HoloLens will not allow one to use ZMQ on the HoloLens (unless there have been updates). Using this method one can send a command to the HoloLens so that it shows a calibration marker with durations for each sample site. The socket server sends the calibration timings using ZMQ to Pupil software. With this setup you can send a record command to your HoloLens setup and to start a synchronized recording on Pupil.

user-4a3b48 19 May, 2017, 07:58:49

yeah the mail came after that..thanks for the info

user-5ca684 19 May, 2017, 11:07:35

@user-3ad5b4 I am looking into the deployment on UWP & especially hololens, there is a serious Threading issue with UWP deployment. I do have a lead, Im going to get back to you if I know more!

user-f7028f 24 May, 2017, 07:27:06

Hi! I'm investigating scripts of unity_integration_calibration and making some development for my own purposes. And I'm interested in PupilData3D class. I understand a meanings of most of class members but I would like to be 100% sure what they exact mean. Can you detailize these members?

user-f181b1 24 May, 2017, 09:02:54

Hi Everyone!

We have a need to plan our own camera mounts for Pupil's eye cameras as we are using Lumus AR glasses. We are trying to find some materials on the best angles and distance the cameras should be located at, in order to optimize the eye tracking.

Currently right&left movements works great, our issue is when gazing up&down.

Thanks!

Itay

user-5ca684 24 May, 2017, 14:22:44

@user-f7028f I am going to get back to you shortly!

user-f7028f 25 May, 2017, 09:18:39

@user-5ca684 Hi! I debug this parameters for one tracked eye and see that Diameter, Theta, Phi. Diameter_3D is always 0 whatever eye detecting well or not. Looking at the code I see that only norm_pos [0] and [1] is using to calculate avarage postion of eye's gaze. My goal is to obtain eye coordinates(x, y, z) relative to scene visible in hmd.

user-5ca684 25 May, 2017, 09:21:16

@user-f7028f the data received is depending on your calibration

user-5ca684 25 May, 2017, 09:22:25

so if you set the Calibration for 2D then after Pupil Service calibrates you will receive the 2D data

user-5ca684 25 May, 2017, 09:22:58

if you want to use the relative position after calibration, you can use

user-5ca684 25 May, 2017, 09:23:24

GetEyeGaze2D(GazeSource.BothEyes).x GetEyeGaze2D(GazeSource.BothEyes).y

user-5ca684 25 May, 2017, 09:23:54

this is NOT in world space

user-5ca684 25 May, 2017, 09:24:09

and is a normalized 0-1 value

user-f13d61 25 May, 2017, 20:31:40

Hi, I am trying to get calibration to work on my Oculus DK2. I attempting to use the pupil-labs/hmd-eye git repository, but I am pretty confused about what I need to run to make things work. So far, I have tried running Calibration.unity in Unity with Pupil Capture open. I am able to get a blue screen with to show up on my computer and on the oculus, but then nothing else happens. I have also tried playing around with running the python code in hmd_calibration, but I don't really see what's going on there. Would anyone have any more specific instructions on how to get a basic screen target calibration up and running on the Oculus? Thanks!

user-1486c3 28 May, 2017, 13:05:51

Hi, I installed the new pupil version 0.9.10 on a different windows computer (because working with unity and pupil in the same time lead to serious frame-drops of the Vive...). when I run pupil capture the frame rate of eyetracking does not go above ~65 for 1 camera and ~55 to the other, even if only one camera is recording. Its a strong computer, but it doesn't have USB3. the CPU usage is 60% at worst, and no more then 50% memory usage. I looked at the USB bandwidth used by the cameras, and it is fixed on 5% each, even if only 1 is recording. is there anything I can do, or it just wont work with USB2? did anyone else face this problem before?

user-f7028f 29 May, 2017, 07:05:09

@user-5ca684 thanks for this. Acctually I don't yet realize how to obtain data about eye coordinates relative to camera? PupilData3D contains only gaze coordinates but no info about detected eye. Have I switch on something in PupilCapture app to get this kind of info in unity_integration_calibration?

user-f7028f 29 May, 2017, 07:13:03

Additional note. In opened Debug window of Pupil Detector 3D I found text about Eyeball center and diameter in mm. Is it relevant info I search?

user-f7028f 29 May, 2017, 08:01:19

That what I see in Debug window

Chat image

user-006924 29 May, 2017, 12:48:56

Hello all, Does the Hololens addon only work on Mac or Linux?

user-41f1bf 29 May, 2017, 12:49:59

I believe that Hololens is for windows only

user-41f1bf 29 May, 2017, 12:50:11

Am I wrong?

user-006924 29 May, 2017, 12:52:13

Microsoft built it but I'm not sure if other OS s can use it or not I was reading hmd-eyes' read me file in github and it only mentioned Mac and Linux.

user-41f1bf 29 May, 2017, 12:52:44

Hummmm, I am not sure

user-006924 29 May, 2017, 12:52:54

I think that part is only for HTC vive probably.

user-a2fe55 30 May, 2017, 16:21:28

Hey there. I have another question. Right now,

user-a2fe55 30 May, 2017, 16:25:07

we are trying to calculate the z-coordinates with the information from the calculated 3D-Eye-Model. Or in other words, we want to get both line of sights from the virtual 3D-Eye-Models to get the coordinates for the gaze. Is there a way to access this information from pupil capture in the HMD-Unity scene (dev-branch)?

mpk 31 May, 2017, 05:20:13

[email removed] if you run 3d calibration you should get a 3d gaze coordinate that contains the depth (magnitude of the vector)

user-a2fe55 31 May, 2017, 07:49:57

@mpk Thank you, for your quick response. The thing is, we know about this and we are very happy about it. But we want to check the accuracy of the z-Axis for a university project. So, we figured, in order to check this, we get the intersection point of both eyes manually and build a seperate unity project in which we put objects with known distances to the camera. But as we looked deeper into the scripts, we got the impression that the Unity Scene can get the gaze-coordinates via json commands from the Pupil Labs App. That's why we wanted to know, if it's possible to get this information in Unity. I hope, I could clarify my objective.

End of May archive