@mpk double cameras are opened
Hey, we are having difficulty with HMD-Eyes where it won't record an info.csv, has anybody had/resolved this issue?
(also the resulting mp4 is just an all-blue screen when opened in VLC)
@user-c68bba it looks liek both cameras are set to the same device in each eye window
under capture selection, activate source, if we activate the pupil camera stream it then records with the files the pupil player expects
now we're getting a Unity_VRCamera.mp4 file, but it only contains the skybox, not other objects in the Unity scene
@user-5ca684 please review the above issue and respond
@wrp thank you very much. I understood you, but HTV vive add-on only two pupil cameras, not include world camera. How I canfigure it?
I tried using the recording functionality in Unity by selecting Test Image(Fake capture) in pupil capture. After this, I see only one eye process being open. When trying to open the second eye process, it repeatedly gives an exception "Exception: GLFW window failed to create." I tried even restarting the computer, the error simply refuses to go.
I am using HMD_Calibration_3D calibrate method. But when I finish calibrating in Unity , the calibration seem's failed.
How I canfigure it?
@user-abff35 Have you tried 2D calibration? After calibration pupil needs some time to process the data, have you given it some time? Also please try to adjust the eye distance from your lenses. My optimal setup with HTC Vive is one click away from the lens.
@user-5874a4 Have you tried to make the recording with actual incoming picture data? I haven't tested recording with Test image
@user-9a0a2b The GazePoint3D is relative to the headset poistion in unity space. You can use the built in TransformPoint method to convert between spaces. What do you exactly mean by "the update rate is not very good"? The update of any data is dependent on the confidence level. If you are getting jumpy data you might want to try recalibrating.
@user-a3f147 is Unity_VRCamera your main camera, do you have any other cameras in your scene?
@user-abff35 @user-5ca684 I have same problem. 2D calibration have been tried.
@user-5ca684
@user-c68bba from this screenshot it is hard to tell where did it go wrong. I think it must have been during the calibration
@user-c68bba what do you mean by "2D calibration have been tried" ? does 2D calibration work ? (I think this screen is shows 3D calibration)
@user-5ca684 Yes,the screenshot is shows 3D calibration.
@user-5ca684 this problem how to configure it?
@user-5ca684 2D calibration have been tried and it worked.
@user-5ca684 But in 2D calibration, it's not accurate, and I want to try the 3D calibration
@user-abff35 accuracy depends on the calibration qualityI believe
both in 2D and 3D
have you tried to adjust the distance from the lens before calibration?
@user-5ca684 yes, i have tried tried to adjust the distance from the lens. Pupil has been as close to the lens as possible
@user-5ca684 I want to use eye tracking in the panorama, the panorama is curved and the 2D calibration is rectangular, there is a big error between these two.So I want to try the 3D calibration
@user-abff35 you could calculate the normal manually from the camera and the gaze position manually, or use the built in normals. and yes perhaps the best would be to use the 3d normals for this. please let me know how exactly you have been trying to achieve the 3D calibration, so I can reproduce it
@user-5ca684 In previous versions, 3D calibration was worked.So I'm confused now
There was no new release lately of the unity plugin execept a development version of the refactor on a separate branch
could you please try the calibration with a previous pupil capture release?
(in case you have updated)
Ok, I'll try the previous pupil capture release
thank you !
😩 It has the same problem
@user-5ca684 it will make the mistake after the calibration is complete
here is all my config
@user-5ca684 thank you
Hi, I'm attempting to set up the current master branch of hmd_eyes (unity_plugin) A TCP connection is only attempted when I use the local functionality, and the plugin itself starts pupil_capture or pupil_service. The problem with that is, that both pupil_capture and pupil_service bind to dynamic TCP ports each time they are started up. So I cannot know the port in advance. What am I missing?
Example
I'm using pupil_capture v0.9.13
Turns out the plugin was not properly loaded, due to some missing dependencies. When the plugin is loaded, the 50020 appears as configured by default.
@user-5ca684 I have the same problem as @user-abff35 and @user-c68bba
and for me 2D calibration doesn't work either: the markers stay at their original places
wow this is a lot of info for a newcomer. is there a location for the latest stuff people are using for VR capture?
any word on CV1 yet? im trying to fit a camera to the CV1 and it just...wont...fit...
@user-24270f we have a CV1 add-on in the pipeline. All parts are finalized, just waiting for final revisions before public release (estimate 2-3 weeks).
@user-fe23df Could you ever calibrate before?
It worked again after a while
I had this problem before too
So the calibration would work fine and then at some point calibration/tracking fails, hence the marker at the player position
Im not sure if I follow
you mean during the calibration it suddenly fails?
no
after calibration
I can't pinpoint where it fails
and after this point there is just no tracking anymore?
but the eyes are not tracked and gazepoint is kept at 0,0,0
yeah
it worked again though
it would be nice to know what the cause could possibly be
@user-5ca684 hello,In order to get better accuracy, how should I adjust the lens to the eye distance, should the closer the better or the far better
@user-abff35 I think it is different for everyone. In my case the best result is 1 click away from the lens
@user-5ca684 What does “1 click” mean?
Is that it?
I think so, mine doesnt have the markings, but if you put it the closest and then click one away
OK ,thank you
@user-5ca684 He doesn't look very accurate😅
Is it possible that I am Asian, with a different eye structure?
@user-abff35 the camera images seem ok. Please try setting an ROI
@wrp This problem has been solved, but the 3D calibration is still not available
has anyone actually fitted a camera to the CV1 yet?
which configuration should i go for? cable out, cable in, top, bottom, etc?
@user-24270f we are working on an integration for CV1 - will be out soon!
Existing Pupil cameras do not fit into CV1 without destructive modification. We have some new stuff in the pipeline - stay tuned 😄
@user-a49a87 - we can continue the discussion here if you like
What hardware are you using?
Yes, but I didn't see this channel, it seems I have a bit to read
I'm using HTC Vive
OK, with Pupil Labs eye tracking add-on (just want to confirm)
If you are not able to achieve robust pupil detection, I would suggest adjusting the HTC Vive depth and ensuring eye cameras are in focus
Yes, I've installed the pupil labs cam in the vive
You may also want to set a ROI in the eye windows
without seeing your eye images I can only provide this kind of general advice
If you'd like to upload an eye image, we can provide concrete advice
Ok, is it possible to record the video of the eye and playback afterwards? I'm alone in my office and it's quite complicated to check the quality of the match with the HMD on 😒
Yes, you can record eye video
In Pupil Capture in the sidebar go to Recorder > Record eye
Then with the world window in focus press r
on your keyboard
How should I send you the image?
Are the images focussed enough?
eye1 is focused to close to the camera. eye0 is ok.
Is it better?
@user-5ca684 the 3D calibration is still not available
@user-5ca684 @user-abff35 I posted the calibration error as an issue on github page for Pupil Capture 0.9.14. They acknowledged the bug and closed it after fixing it. Here is the reference to that issue page, https://github.com/pupil-labs/pupil/issues/825. Do anyone know how to use the version with the fixed code?, I see the main download still contains the faulty version.
Is it possible to record the eye tracking data with the VR scene that the user sees in HTC vive as video input? I can't figure out how to do this. When i try to record, it always says , "Current world capture is offline. Please reconnect or switch to fake capture". What should be my selection for Capture selection in Pupil Capture window? Or am I misunderstanding the recording feature as a whole?
@user-5874a4 where do you see this error, in Pupil Capture, or in Unity?
in Pupil Capture
Mine was there
you might want to ask @mpk / @wrp about error messages in Pupil Capture, did you run this capture from the Pupil Capture program?
or initiated in Unity?
@user-5ca684 "Current world capture is offline. Please reconnect or switch to fake capture" , that error was in Pupil capture.
.../2017_09_07/000/Unity_Camera (eye).webm"). Rendered frame count on MainThread : 245. Written out frame count on SecondaryThread : 1. Leftover : 244. This is what I get in Unity whenever i try to record. Nothing ever really gets written to that folder that it says. I am not really sure what's happening in the background. But since it says in Pupil Capture that "World capture is offline", I am surmising that nothing really is recorded. You can enlighten me on this
@user-5ca684 I always initiate Pupil capture from Unity
@user-5874a4 have you ever successfully recorded data from Pupil Capture alone? If I remember right, you have altered the framerate in Unity, that might cause issues with the recording. Could you try to download the unity plugin fresh into a separate folder and run it out of the box without any alteration and run the capture?
@user-5874a4 you also could try to change the codec
@muzammil#8393 9.12 of the pupil capture looks like it can be calibrated in 3D
@user-fe23df Are you still experiencing calibration issues? If so could you please try to calibrate using the 9.12 version of the pupil capture too, so we can confirm on multiple fronts that it works with that version?
@user-5ca684 the problem I have has always been with 9.12. It only occurs once in a while though, and somehow miraculously gets solved after a while of desperation
@user-5ca684 I’ve integrated the original calibration project with my unity project. So, in that sense true, I am not using the default plugin. But I don’t remember changing any framerate during integration. Could it be that you had set something in the original project which didn’t get reflected when copied to my new one?
@user-5ca684 Sure. I’ll try to download the plugin fresh to see if it works fine independently.
@user-5ca684 Well, I tried all the three different codecs. But no use, I could only see the difference in the number of recorded frames (from the error). Nothing gets rendered anyways.
@user-5874a4 please also try the out of the box plugin. Have you perhaps changed on which Display the game is being rendered? (I mean unity display from the dropdown)
Hey, does the unity HMD_Eyes plugin work with only one eye? If so, how would I configure it?
Howdy, I am trying to use your Pupil Labs products to record the eye tracking of research study participants as they view a virtual environment via an HTC Vive HMD. I want to have a recording of their session as seen through the Main Camera in Unity with their eye tracking data visualized over the recording of the session, similar to what your other products do in Pupil Player. Is this currently possible to do with a recording of the virtual environment?
@user-5ca684 Okay. I am able to record now by setting Capture Selection -> Manager to “Test Image” and clicking the button “Activate Fake Capture”. The data is proper I suppose. But I didn’t get what I wanted for the videos. There are two video files in the folder, “Unity_Camera (eye).mov” and “world.mp4”. The second video “world.mp4” is not useful probably. I was expecting the first video to record the scene as it is seen by the user (HTC vive). But I could only see the skybox, none the virtual objects are to be seen. The red marker (showing the gaze position) is the only exception and appears in that video. Do I need to change something on the unity side to get what I want?
In that sense, my issue is similar to @user-13c487
I'm also having the same issue. Seems to be something to do with the camera settings for the generated RecorderGO - see https://github.com/pupil-labs/hmd-eyes/blob/dd0dabfeb78cd828bd509e522bd94fd5db035f8f/unity_pupil_plugin/Assets/Scripts/PupilGazeTracker.cs#L378
Setting c.clearFlags = CameraClearFlags.SolidColor; seems to do something different
also, RecorderGO seems to spawn offset from the main camera -
RecorderGO.transform.localPosition = Vector3.zero;
RecorderGO.transform.localRotation = Quaternion.Euler(Vector3.zero);
to fix
here's a short clip of me looking at stuff in the vive in unity - https://drive.google.com/open?id=0BwtcU_I5yybJdG5tbG95eUNKMHc - source videos look good, seems to be missing pupil data though
@user-5a8f39 Thanks for your tips & feedback !
Hello, I am currently working with hmd eyes unity plugin and am wondering if its possible to record a session in unity and utilize pupil player to analyze the eye movements in a virtual environment.
An additional question. What is the distance between the gaze marker and the main camera?
in terms of unity
it depends on the depth of your gaze, if you're doing 3D
if you're doing 2D I think it's 0.8
and as for the calibration marker for 3D it's 100
Sorry I forgot to mention it was for 2D, I ended up doing some very rough measurements 😅 Looks around 0.77~0.78
oops, the scaling was a tad off its closer to 7.5
I can't seem to get the 3D tracking to work very well, but for the application 2D should be enough
In the case in which I cannot use the pupil player for this data, I'm trying to raycast from the gaze marker (the vector being from the camera to the marker). If someone a simple solution for this it would be great. (Currently trying to wrap my head around the maths required)
Sorry to be bombarding the channel with questions. But is the XY-coodinates shown during runtime in respect to unity or the screen?.
It's relative to the screen I think
and there's a 0.5 offset for each axis too
I'm trying to use a (3D calibrated) center gaze position with a C++ code as we can with the Unity demo. I'm subscribing to "gaze" now but the message I get gives me a norm pos int the world image only for one eye as you can see one msg only contains the data about one eye in the picture below. How can I get a center gaze pos where two eyes' gaze pos are combined into one? What topic do I need to subscribe to or what plug-in do I need to execute necessarily?
hmm i see that offset you mentioned, why is it there tho?
is it to set the bottom left as the origin?
It's because pupil labs uses a different coordinate system from unity. It's specified somewhere in the pupil gaze tracker script
Speaking of offsets, does anyone know how to convert the normalized gaze positions into Unity's actual screen space? As in, get the location of the trackers in Unity 3D space instead of the normalized 0-1 2D screen space?
x-0.5, y-0.5, 0.8 for me
(local position relative to the headset)
yeah, like ohoo said, just -.5 to both
the values for the coordinates are in terms of the plane 7.5 units away from the main camera
hi, I would like to ask if there are any plans of adopting the new and upcoming microsoft mixed reality lenses? asus and lenovo and many others announced such VR devices and it seems there are all based on the same dimensions. Because they are all inside-out tracking devices, they open up some more mobile applications...
I'm trying to do the same thing @user-f1d099 was trying to do; do a ray cast from the camera through the red 2D marker. I'm using ScreenPointToRay and accessing NormalizedEyePos2D.x and NormalizedEyePos2D.y and multiplying those normalized values by the camera's pixelWidth and pixelHeight. There's an offset and I don't know how to fix it.
Hello everyone, I'm trying to use the hmd-eye, but I don't manage to get it working.
It's a bit odd, it seems that it launch twice each cam. When I click play in Unity, there is not 2, but 4 "capture" windows popping up, 2 for the right eye, 2 for the left eye
Correction... Every time I press play, there is one more "Pupil Capture - eye 0" windows popping
@Djinn#0083 Maybe you have phantom pupil_service processes running in the background -you could check in task manager before starting again?