vr-ar


user-c68bba 01 September, 2017, 01:51:42

@mpk double cameras are opened

Chat image

user-a3f147 01 September, 2017, 03:19:40

Hey, we are having difficulty with HMD-Eyes where it won't record an info.csv, has anybody had/resolved this issue?

user-a3f147 01 September, 2017, 03:19:58

(also the resulting mp4 is just an all-blue screen when opened in VLC)

wrp 01 September, 2017, 03:31:42

@user-c68bba it looks liek both cameras are set to the same device in each eye window

user-a3f147 01 September, 2017, 03:32:46

under capture selection, activate source, if we activate the pupil camera stream it then records with the files the pupil player expects

user-a3f147 01 September, 2017, 04:17:12

now we're getting a Unity_VRCamera.mp4 file, but it only contains the skybox, not other objects in the Unity scene

wrp 01 September, 2017, 04:17:38

@user-5ca684 please review the above issue and respond

user-c68bba 01 September, 2017, 06:20:10

@wrp thank you very much. I understood you, but HTV vive add-on only two pupil cameras, not include world camera. How I canfigure it?

Chat image

user-5874a4 01 September, 2017, 06:23:28

I tried using the recording functionality in Unity by selecting Test Image(Fake capture) in pupil capture. After this, I see only one eye process being open. When trying to open the second eye process, it repeatedly gives an exception "Exception: GLFW window failed to create." I tried even restarting the computer, the error simply refuses to go.

Chat image

user-abff35 03 September, 2017, 10:03:27

I am using HMD_Calibration_3D calibrate method. But when I finish calibrating in Unity , the calibration seem's failed.

user-abff35 03 September, 2017, 10:03:37

Chat image

user-abff35 03 September, 2017, 10:03:38

Chat image

user-abff35 03 September, 2017, 10:03:39

Chat image

user-abff35 03 September, 2017, 10:04:20

How I canfigure it?

user-5ca684 03 September, 2017, 18:11:25

@user-abff35 Have you tried 2D calibration? After calibration pupil needs some time to process the data, have you given it some time? Also please try to adjust the eye distance from your lenses. My optimal setup with HTC Vive is one click away from the lens.

user-5ca684 03 September, 2017, 18:13:07

@user-5874a4 Have you tried to make the recording with actual incoming picture data? I haven't tested recording with Test image

user-5ca684 03 September, 2017, 18:18:56

@user-9a0a2b The GazePoint3D is relative to the headset poistion in unity space. You can use the built in TransformPoint method to convert between spaces. What do you exactly mean by "the update rate is not very good"? The update of any data is dependent on the confidence level. If you are getting jumpy data you might want to try recalibrating.

user-5ca684 03 September, 2017, 18:20:42

@user-a3f147 is Unity_VRCamera your main camera, do you have any other cameras in your scene?

user-c68bba 04 September, 2017, 03:50:00

@user-abff35 @user-5ca684 I have same problem. 2D calibration have been tried.

user-c68bba 04 September, 2017, 04:47:07

@user-5ca684

Chat image

user-5ca684 04 September, 2017, 06:41:38

@user-c68bba from this screenshot it is hard to tell where did it go wrong. I think it must have been during the calibration

user-5ca684 04 September, 2017, 06:43:16

@user-c68bba what do you mean by "2D calibration have been tried" ? does 2D calibration work ? (I think this screen is shows 3D calibration)

user-c68bba 04 September, 2017, 06:46:13

@user-5ca684 Yes,the screenshot is shows 3D calibration.

user-c68bba 04 September, 2017, 06:46:50

@user-5ca684 this problem how to configure it?

user-abff35 04 September, 2017, 07:18:24

@user-5ca684 2D calibration have been tried and it worked.

user-abff35 04 September, 2017, 08:48:41

@user-5ca684 But in 2D calibration, it's not accurate, and I want to try the 3D calibration

user-5ca684 04 September, 2017, 08:49:16

@user-abff35 accuracy depends on the calibration qualityI believe

user-5ca684 04 September, 2017, 08:49:28

both in 2D and 3D

user-5ca684 04 September, 2017, 08:49:49

have you tried to adjust the distance from the lens before calibration?

user-abff35 04 September, 2017, 08:58:34

@user-5ca684 yes, i have tried tried to adjust the distance from the lens. Pupil has been as close to the lens as possible

user-abff35 04 September, 2017, 09:00:45

@user-5ca684 I want to use eye tracking in the panorama, the panorama is curved and the 2D calibration is rectangular, there is a big error between these two.So I want to try the 3D calibration

user-5ca684 04 September, 2017, 09:03:04

@user-abff35 you could calculate the normal manually from the camera and the gaze position manually, or use the built in normals. and yes perhaps the best would be to use the 3d normals for this. please let me know how exactly you have been trying to achieve the 3D calibration, so I can reproduce it

user-abff35 04 September, 2017, 09:06:59

@user-5ca684 In previous versions, 3D calibration was worked.So I'm confused now

user-5ca684 04 September, 2017, 09:08:18

There was no new release lately of the unity plugin execept a development version of the refactor on a separate branch

user-5ca684 04 September, 2017, 09:08:45

could you please try the calibration with a previous pupil capture release?

user-5ca684 04 September, 2017, 09:08:59

(in case you have updated)

user-abff35 04 September, 2017, 09:09:48

Ok, I'll try the previous pupil capture release

user-5ca684 04 September, 2017, 09:10:10

thank you !

user-abff35 04 September, 2017, 09:13:44

😩 It has the same problem

user-abff35 04 September, 2017, 09:32:35

@user-5ca684 it will make the mistake after the calibration is complete

Chat image

user-abff35 04 September, 2017, 09:37:12

here is all my config

user-abff35 04 September, 2017, 09:37:26

Chat image

user-abff35 04 September, 2017, 09:37:26

Chat image

user-abff35 04 September, 2017, 09:37:27

Chat image

user-abff35 04 September, 2017, 09:37:27

Chat image

user-abff35 04 September, 2017, 09:37:29

Chat image

user-abff35 04 September, 2017, 09:37:43

@user-5ca684 thank you

user-8ca7ac 04 September, 2017, 11:22:51

Hi, I'm attempting to set up the current master branch of hmd_eyes (unity_plugin) A TCP connection is only attempted when I use the local functionality, and the plugin itself starts pupil_capture or pupil_service. The problem with that is, that both pupil_capture and pupil_service bind to dynamic TCP ports each time they are started up. So I cannot know the port in advance. What am I missing?

user-8ca7ac 04 September, 2017, 11:25:08

Example

Chat image

user-8ca7ac 04 September, 2017, 11:25:44

I'm using pupil_capture v0.9.13

user-8ca7ac 04 September, 2017, 14:11:04

Turns out the plugin was not properly loaded, due to some missing dependencies. When the plugin is loaded, the 50020 appears as configured by default.

user-fe23df 05 September, 2017, 04:28:57

@user-5ca684 I have the same problem as @user-abff35 and @user-c68bba

user-fe23df 05 September, 2017, 04:34:18

and for me 2D calibration doesn't work either: the markers stay at their original places

user-24270f 05 September, 2017, 05:12:51

wow this is a lot of info for a newcomer. is there a location for the latest stuff people are using for VR capture?

user-24270f 05 September, 2017, 05:35:50

any word on CV1 yet? im trying to fit a camera to the CV1 and it just...wont...fit...

wrp 05 September, 2017, 06:39:06

@user-24270f we have a CV1 add-on in the pipeline. All parts are finalized, just waiting for final revisions before public release (estimate 2-3 weeks).

user-5ca684 05 September, 2017, 07:02:06

@user-fe23df Could you ever calibrate before?

user-fe23df 05 September, 2017, 07:02:26

It worked again after a while

user-fe23df 05 September, 2017, 07:02:31

I had this problem before too

user-fe23df 05 September, 2017, 07:03:02

So the calibration would work fine and then at some point calibration/tracking fails, hence the marker at the player position

user-5ca684 05 September, 2017, 07:03:33

Im not sure if I follow

user-5ca684 05 September, 2017, 07:03:45

you mean during the calibration it suddenly fails?

user-fe23df 05 September, 2017, 07:03:58

no

user-fe23df 05 September, 2017, 07:04:07

after calibration

user-fe23df 05 September, 2017, 07:04:30

I can't pinpoint where it fails

user-5ca684 05 September, 2017, 07:04:47

and after this point there is just no tracking anymore?

user-fe23df 05 September, 2017, 07:04:49

but the eyes are not tracked and gazepoint is kept at 0,0,0

user-fe23df 05 September, 2017, 07:04:52

yeah

user-fe23df 05 September, 2017, 07:04:58

it worked again though

user-fe23df 05 September, 2017, 07:05:22

it would be nice to know what the cause could possibly be

user-abff35 05 September, 2017, 08:05:18

@user-5ca684 hello,In order to get better accuracy, how should I adjust the lens to the eye distance, should the closer the better or the far better

user-5ca684 05 September, 2017, 08:06:07

@user-abff35 I think it is different for everyone. In my case the best result is 1 click away from the lens

user-abff35 05 September, 2017, 08:11:36

@user-5ca684 What does “1 click” mean?

Chat image

user-abff35 05 September, 2017, 08:11:58

Is that it?

user-5ca684 05 September, 2017, 08:15:33

I think so, mine doesnt have the markings, but if you put it the closest and then click one away

user-abff35 05 September, 2017, 08:38:37

OK ,thank you

user-abff35 05 September, 2017, 09:25:53

Chat image

user-abff35 05 September, 2017, 09:26:25

@user-5ca684 He doesn't look very accurate😅

user-abff35 05 September, 2017, 09:32:07

Is it possible that I am Asian, with a different eye structure?

wrp 05 September, 2017, 10:23:45

@user-abff35 the camera images seem ok. Please try setting an ROI

user-abff35 06 September, 2017, 12:24:38

@wrp This problem has been solved, but the 3D calibration is still not available

user-24270f 06 September, 2017, 22:27:54

has anyone actually fitted a camera to the CV1 yet?

user-24270f 06 September, 2017, 22:28:16

which configuration should i go for? cable out, cable in, top, bottom, etc?

wrp 07 September, 2017, 14:30:30

@user-24270f we are working on an integration for CV1 - will be out soon!

wrp 07 September, 2017, 14:31:13

Existing Pupil cameras do not fit into CV1 without destructive modification. We have some new stuff in the pipeline - stay tuned 😄

wrp 07 September, 2017, 14:31:35

@user-a49a87 - we can continue the discussion here if you like

wrp 07 September, 2017, 14:31:58

What hardware are you using?

user-a49a87 07 September, 2017, 14:32:16

Yes, but I didn't see this channel, it seems I have a bit to read

user-a49a87 07 September, 2017, 14:32:24

I'm using HTC Vive

wrp 07 September, 2017, 14:32:54

OK, with Pupil Labs eye tracking add-on (just want to confirm)

wrp 07 September, 2017, 14:33:24

If you are not able to achieve robust pupil detection, I would suggest adjusting the HTC Vive depth and ensuring eye cameras are in focus

user-a49a87 07 September, 2017, 14:33:26

Yes, I've installed the pupil labs cam in the vive

wrp 07 September, 2017, 14:33:40

You may also want to set a ROI in the eye windows

wrp 07 September, 2017, 14:33:52

without seeing your eye images I can only provide this kind of general advice

wrp 07 September, 2017, 14:34:03

If you'd like to upload an eye image, we can provide concrete advice

user-a49a87 07 September, 2017, 14:35:28

Ok, is it possible to record the video of the eye and playback afterwards? I'm alone in my office and it's quite complicated to check the quality of the match with the HMD on 😒

wrp 07 September, 2017, 14:36:46

Yes, you can record eye video

wrp 07 September, 2017, 14:37:55

In Pupil Capture in the sidebar go to Recorder > Record eye

wrp 07 September, 2017, 14:38:08

Then with the world window in focus press r on your keyboard

user-a49a87 07 September, 2017, 14:46:01

How should I send you the image?

user-a49a87 07 September, 2017, 14:46:45

Chat image

user-a49a87 07 September, 2017, 15:00:19

Are the images focussed enough?

mpk 07 September, 2017, 15:00:56

eye1 is focused to close to the camera. eye0 is ok.

user-a49a87 07 September, 2017, 15:11:17

Is it better?

Chat image

user-abff35 08 September, 2017, 01:39:27

@user-5ca684 the 3D calibration is still not available

user-5874a4 08 September, 2017, 03:51:56

@user-5ca684 @user-abff35 I posted the calibration error as an issue on github page for Pupil Capture 0.9.14. They acknowledged the bug and closed it after fixing it. Here is the reference to that issue page, https://github.com/pupil-labs/pupil/issues/825. Do anyone know how to use the version with the fixed code?, I see the main download still contains the faulty version.

user-5874a4 08 September, 2017, 05:10:10

Is it possible to record the eye tracking data with the VR scene that the user sees in HTC vive as video input? I can't figure out how to do this. When i try to record, it always says , "Current world capture is offline. Please reconnect or switch to fake capture". What should be my selection for Capture selection in Pupil Capture window? Or am I misunderstanding the recording feature as a whole?

user-5ca684 08 September, 2017, 05:30:52

@user-5874a4 where do you see this error, in Pupil Capture, or in Unity?

user-abff35 08 September, 2017, 05:31:37

in Pupil Capture

user-abff35 08 September, 2017, 05:31:54

Mine was there

user-5ca684 08 September, 2017, 05:33:16

you might want to ask @mpk / @wrp about error messages in Pupil Capture, did you run this capture from the Pupil Capture program?

user-5ca684 08 September, 2017, 05:33:25

or initiated in Unity?

user-5874a4 08 September, 2017, 05:36:13

@user-5ca684 "Current world capture is offline. Please reconnect or switch to fake capture" , that error was in Pupil capture.

user-5874a4 08 September, 2017, 05:38:25

.../2017_09_07/000/Unity_Camera (eye).webm"). Rendered frame count on MainThread : 245. Written out frame count on SecondaryThread : 1. Leftover : 244. This is what I get in Unity whenever i try to record. Nothing ever really gets written to that folder that it says. I am not really sure what's happening in the background. But since it says in Pupil Capture that "World capture is offline", I am surmising that nothing really is recorded. You can enlighten me on this

user-5874a4 08 September, 2017, 05:39:13

@user-5ca684 I always initiate Pupil capture from Unity

user-5ca684 08 September, 2017, 06:37:00

@user-5874a4 have you ever successfully recorded data from Pupil Capture alone? If I remember right, you have altered the framerate in Unity, that might cause issues with the recording. Could you try to download the unity plugin fresh into a separate folder and run it out of the box without any alteration and run the capture?

user-5ca684 08 September, 2017, 06:37:58

@user-5874a4 you also could try to change the codec

user-abff35 08 September, 2017, 07:20:59

@muzammil#8393 9.12 of the pupil capture looks like it can be calibrated in 3D

user-5ca684 08 September, 2017, 08:19:44

@user-fe23df Are you still experiencing calibration issues? If so could you please try to calibrate using the 9.12 version of the pupil capture too, so we can confirm on multiple fronts that it works with that version?

user-fe23df 08 September, 2017, 09:52:53

@user-5ca684 the problem I have has always been with 9.12. It only occurs once in a while though, and somehow miraculously gets solved after a while of desperation

user-5874a4 10 September, 2017, 01:09:53

@user-5ca684 I’ve integrated the original calibration project with my unity project. So, in that sense true, I am not using the default plugin. But I don’t remember changing any framerate during integration. Could it be that you had set something in the original project which didn’t get reflected when copied to my new one?

user-5874a4 10 September, 2017, 01:10:10

@user-5ca684 Sure. I’ll try to download the plugin fresh to see if it works fine independently.

user-5874a4 10 September, 2017, 01:10:30

@user-5ca684 Well, I tried all the three different codecs. But no use, I could only see the difference in the number of recorded frames (from the error). Nothing gets rendered anyways.

user-5ca684 11 September, 2017, 06:10:19

@user-5874a4 please also try the out of the box plugin. Have you perhaps changed on which Display the game is being rendered? (I mean unity display from the dropdown)

user-8ca7ac 11 September, 2017, 10:37:16

Hey, does the unity HMD_Eyes plugin work with only one eye? If so, how would I configure it?

user-13c487 11 September, 2017, 22:27:23

Howdy, I am trying to use your Pupil Labs products to record the eye tracking of research study participants as they view a virtual environment via an HTC Vive HMD. I want to have a recording of their session as seen through the Main Camera in Unity with their eye tracking data visualized over the recording of the session, similar to what your other products do in Pupil Player. Is this currently possible to do with a recording of the virtual environment?

user-5874a4 13 September, 2017, 22:35:51

@user-5ca684 Okay. I am able to record now by setting Capture Selection -> Manager to “Test Image” and clicking the button “Activate Fake Capture”. The data is proper I suppose. But I didn’t get what I wanted for the videos. There are two video files in the folder, “Unity_Camera (eye).mov” and “world.mp4”. The second video “world.mp4” is not useful probably. I was expecting the first video to record the scene as it is seen by the user (HTC vive). But I could only see the skybox, none the virtual objects are to be seen. The red marker (showing the gaze position) is the only exception and appears in that video. Do I need to change something on the unity side to get what I want?

user-5874a4 13 September, 2017, 22:36:42

In that sense, my issue is similar to @user-13c487

user-5a8f39 14 September, 2017, 04:33:28

I'm also having the same issue. Seems to be something to do with the camera settings for the generated RecorderGO - see https://github.com/pupil-labs/hmd-eyes/blob/dd0dabfeb78cd828bd509e522bd94fd5db035f8f/unity_pupil_plugin/Assets/Scripts/PupilGazeTracker.cs#L378

user-5a8f39 14 September, 2017, 04:46:23

Setting c.clearFlags = CameraClearFlags.SolidColor; seems to do something different

user-5a8f39 14 September, 2017, 05:13:50

also, RecorderGO seems to spawn offset from the main camera -

    RecorderGO.transform.localPosition = Vector3.zero;
    RecorderGO.transform.localRotation = Quaternion.Euler(Vector3.zero);

to fix

user-5a8f39 14 September, 2017, 05:30:46

here's a short clip of me looking at stuff in the vive in unity - https://drive.google.com/open?id=0BwtcU_I5yybJdG5tbG95eUNKMHc - source videos look good, seems to be missing pupil data though

user-5ca684 14 September, 2017, 06:25:32

@user-5a8f39 Thanks for your tips & feedback !

user-f1d099 15 September, 2017, 03:16:50

Hello, I am currently working with hmd eyes unity plugin and am wondering if its possible to record a session in unity and utilize pupil player to analyze the eye movements in a virtual environment.

user-f1d099 15 September, 2017, 17:54:23

An additional question. What is the distance between the gaze marker and the main camera?

user-f1d099 15 September, 2017, 17:54:36

in terms of unity

user-fe23df 15 September, 2017, 19:16:23

it depends on the depth of your gaze, if you're doing 3D

user-fe23df 15 September, 2017, 19:16:34

if you're doing 2D I think it's 0.8

user-fe23df 15 September, 2017, 19:16:50

and as for the calibration marker for 3D it's 100

user-f1d099 15 September, 2017, 19:42:28

Sorry I forgot to mention it was for 2D, I ended up doing some very rough measurements 😅 Looks around 0.77~0.78

Chat image

user-f1d099 15 September, 2017, 19:44:21

oops, the scaling was a tad off its closer to 7.5

user-f1d099 15 September, 2017, 19:48:15

I can't seem to get the 3D tracking to work very well, but for the application 2D should be enough

user-f1d099 15 September, 2017, 20:00:33

In the case in which I cannot use the pupil player for this data, I'm trying to raycast from the gaze marker (the vector being from the camera to the marker). If someone a simple solution for this it would be great. (Currently trying to wrap my head around the maths required)

user-f1d099 17 September, 2017, 15:32:49

Sorry to be bombarding the channel with questions. But is the XY-coodinates shown during runtime in respect to unity or the screen?.

user-fe23df 18 September, 2017, 09:18:02

It's relative to the screen I think

user-fe23df 18 September, 2017, 09:18:18

and there's a 0.5 offset for each axis too

user-e5ed96 18 September, 2017, 12:55:01

I'm trying to use a (3D calibrated) center gaze position with a C++ code as we can with the Unity demo. I'm subscribing to "gaze" now but the message I get gives me a norm pos int the world image only for one eye as you can see one msg only contains the data about one eye in the picture below. How can I get a center gaze pos where two eyes' gaze pos are combined into one? What topic do I need to subscribe to or what plug-in do I need to execute necessarily?

Chat image

user-f1d099 18 September, 2017, 17:52:16

hmm i see that offset you mentioned, why is it there tho?

user-f1d099 18 September, 2017, 17:52:56

is it to set the bottom left as the origin?

user-fe23df 19 September, 2017, 23:29:04

It's because pupil labs uses a different coordinate system from unity. It's specified somewhere in the pupil gaze tracker script

user-993cee 20 September, 2017, 19:27:22

Speaking of offsets, does anyone know how to convert the normalized gaze positions into Unity's actual screen space? As in, get the location of the trackers in Unity 3D space instead of the normalized 0-1 2D screen space?

user-fe23df 20 September, 2017, 23:05:45

x-0.5, y-0.5, 0.8 for me

user-fe23df 20 September, 2017, 23:05:59

(local position relative to the headset)

user-f1d099 25 September, 2017, 20:50:16

yeah, like ohoo said, just -.5 to both

user-f1d099 25 September, 2017, 20:51:04

the values for the coordinates are in terms of the plane 7.5 units away from the main camera

user-29e10a 26 September, 2017, 12:08:22

hi, I would like to ask if there are any plans of adopting the new and upcoming microsoft mixed reality lenses? asus and lenovo and many others announced such VR devices and it seems there are all based on the same dimensions. Because they are all inside-out tracking devices, they open up some more mobile applications...

user-13c487 27 September, 2017, 19:12:00

I'm trying to do the same thing @user-f1d099 was trying to do; do a ray cast from the camera through the red 2D marker. I'm using ScreenPointToRay and accessing NormalizedEyePos2D.x and NormalizedEyePos2D.y and multiplying those normalized values by the camera's pixelWidth and pixelHeight. There's an offset and I don't know how to fix it.

user-a49a87 28 September, 2017, 13:21:06

Hello everyone, I'm trying to use the hmd-eye, but I don't manage to get it working.

user-a49a87 28 September, 2017, 13:32:12

It's a bit odd, it seems that it launch twice each cam. When I click play in Unity, there is not 2, but 4 "capture" windows popping up, 2 for the right eye, 2 for the left eye

user-a49a87 28 September, 2017, 13:37:02

Correction... Every time I press play, there is one more "Pupil Capture - eye 0" windows popping

user-90aa66 28 September, 2017, 16:44:57

@Djinn#0083 Maybe you have phantom pupil_service processes running in the background -you could check in task manager before starting again?

End of September archive