Using HMD Stereo Post Processing Effect without tracking

Hi guys,

So, I'm figuring out if I can setup and HMD with Blend4Web. I can read tracking data from a dk2 into my application and set the orientation of a static camera using that (see here). All I need is the ability to generate the side by side image with the proper distortion. Is there a way to achieve the HMD post processing effect without using WebVR? I don't mind having to dig into the Blend4Web source code to get this working at the moment.

To answer some questions that may come up, the reasons I can't (at least yet) go with using WebVR and the Rift runtime are:

1. I am using Linux and support from Oculus is currently "on-pause". I don't mind at least trying the Rift runtime, but I am trying to build a medium to long term solution for a research lab and would rather not leave them with something that may stop working all of a sudden or may not work for newer versions of the hardare. (I'm using OpenHMD to get at the tracking data btw).

2. I need the tracking data to be read by (and possibly manipulated by) another program before it reaches Blend4Web.

3. Similar to 2, we may start doing positional tracking at a larger scale than what the rift appears to be designed for.

Hello jansoft88 and welcome to our forum.

Ответ на сообщение пользователя jansoft88
All I need is the ability to generate the side by side image with the proper distortion. Is there a way to achieve the HMD post processing effect without using WebVR?

Main purpose of WebVR API is to get sensors data. There is not another way get tracking data in browser (or I just don't know another way).

Engine solves many issues.
1. It generates the side by side image, use set_hmd_params({enable_hmd_stereo: true}) to enable this rendering mode.
2. Set orientation behavior of a camera.
3. Distortion is in our TODO. But experimental build of Chromium does the distortion by default.
"It's highly recommended that you let the browser do the distortion for you."(Brandon Jones)

1. I am using Linux and support from Oculus is currently "on-pause".

Yes, unfortunately, we are using source code older version right now. There is only way to get tracking data in browser.

2. I need the tracking data to be read by (and possibly manipulated by) another program before it reaches Blend4Web.

You can disable default camera behavior (just does not call enable_hmd), and set up your own behavior.

Thank you Kirill,

Main purpose of WebVR API is to get sensors data. There is not another way tracking motion data in browser (or I just don't know another way).

I'm using OpenHMD with a python wrapper to get the tracking data which eventually gets sent over a websocket. I can then read this data using JS. It sounds a little "round about" ish but I need to go this route for my application.

Engine solves many issues.
1. It generates the side by side image, use set_hmd_params({enable_hmd_stereo: true}) to enable this rendering mode.

I tried doing this and it did not create a side by side image. Where in the .js file should I set these properties (I tried it in the load_cb and I get

Uncaught TypeError: Cannot set property 'enable_hmd_stereo' of null

). I tried assigning m_scenes to a global variable and setting these properties from the JS console and got the same error. Other functions work fine.

3. Distortion is in our TODO. But experimental build of Chromium does the distortion by default.
"It's highly recommended that you let the browser do the distortion for you."(Brandon Jones)

I'm assuming this works when WebVR is working properly. Can I use the distortion specific part even if the Rift runtime is not available or a device is not connected?

Finally, the stereo.glslf file alludes to distortion for HMDs. Is this experimental/not yet working?

Thanks :)

Comming version (16.01) of the engine fixes many rendering issues.

I'm using OpenHMD with a python wrapper to get the tracking data which eventually gets sent over a websocket. I can then read this data using JS. It sounds a little "round about" ish but I need to go this route for my application.

Hhmm. I think it is a good decision.

I tried doing this and it did not create a side by side image. Where in the .js file should I set these properties (I tried it in the load_cb and I get

It is my fault. You need to do something like this:

// …
var m_scenes = require("scenes");
// …
var hmd_params = {};
hmd_params.enable_hmd_stereo = true;
m_scenes.set_hmd_params(hmd_params);
// …

I can check the code only tomorrow :-(, but you can try.

Finally, the stereo.glslf file alludes to distortion for HMDs. Is this experimental/not yet working?

Hehe, yes, it does not work right yet.

Can I use the distortion specific part even if the Rift runtime is not available or a device is not connected?

You can set up VR rendering mode (a side by side image) without connected device.

I'm assuming this works when WebVR is working properly.

Browser does not distoriton correction without WebVR
Only Brandon's Chromium build does distortion stuff right now.

You need to do something like this: …

This gives the same error (I always define m_scenes after the

b4w.register("riftTest", function(exports, require) {

line). Maybe my Blend4Web installation is borked. I also can't get the latest Chrome nightly to work on my Linux machine :(. Firefox is working though and the hmd addon reports WebVR is available.

I think I will move to my Windows machine and try to make things work there. I will post my progress here before tomorrow :)

Comming version (16.01) of the engine fixes many rendering issues.

P.S. Is there a date for this?

This gives the same error

Hm… It seems like a bug.

P.S. Is there a date for this?

Around next week

This gives the same error

Try:

b4w.register("riftTest", function(exports, require) {
// …
var m_cfg = require("config");
var m_scenes = require("scenes");
// …
exports.init = function() {
// …
    m_cfg.set("stereo", "HMD");
// …
}
// …
function load_cb(data_id) {
// …
    m_scenes.set_hmd_params({enable_hmd_stereo: true, distortion_scale: 1})
// …
}

}
window.addEventListener("load", function() {b4w.require("riftTest").init();});

It works for me. It is better to call container.resize after scenes.set_hmd_params
Here is the example, it was created with using the project manager.

That worked beautifully. Thank you so much :)

Now all that is left is the distortion. In Chrome you have to pass an HMDVRDevice object to webkitRequestFullscreen. I'm thinking I can save whatever that is for the dk2 and use that for the time being. Do you think that will work?

I also stumbled across webvr-boilerplate which can do the distortion on browsers that don't support WebVR. I tested their example and it looks good but they are working primarily with three.js. Any way you guys can incorporate this?

You can enable distortion (v15.12) for DK2.

m_scenes.set_hmd_params({
    enable_hmd_stereo: true, 
    distortion_scale: 0.666,
    distortion_coefs: [0.22, 0.28]
})

But it does not correcting chromatic aberration (webvr-boilerplate does same).

I'm thinking I can save whatever that is for the dk2 and use that for the time being.

In case of using engine distortion it is better to disable browser distortion.

requestFullScreen({"vrDistortion": false});

Right correction requires right distortion coefficients.
For example,
OculusRiftDK2 — [0.22, 0.28]
CardboardV1 — [0.441, 0.156]
CardboardV2 — [0.34, 0.55]