Azure Remote Rendering doesn't show textured models

Question

I am trying to get the open data provided by the Ministry of Land, Infrastructure, Transport and Tourism of Japan and display the city model of Tokyo using Holorens 2.
I went through several processes to combine multiple small FBX files, create the combined FBX data, convert the files for Azure Remote Rendering, and place them in the cloud. The file is described below.

arrAsset : https://arrdemo20211108.blob.core.windows.net/arrt-convertedmodels/533946.arrAsset
info.json : https://arrdemo20211108.blob.core.windows.net/arrt-convertedmodels/533946.info.json
result.json : https://arrdemo20211108.blob.core.windows.net/arrt-convertedmodels/533946.result.json

This model is about 1 million polygons. The model also includes the texture of the roof and walls of the building.
I tried to watch this model on HoloLens 2, but unfortunately I couldn't see it.
So I created a model without texture. As a result, the model could be seen on HoloLens 2. I was also able to see a model when two small FBX models were combined.
Therefore, it seems that the reason why it cannot be displayed is due to the amount of data of the texture that has become bloated as a result of combining.

Question 1
Is there a way to display a file containing this texture data on HoloLens 2 using Azure Remote Rendering?

Question 2
The instruction manual for using Azure Remote Rendering clearly states the restrictions on the number of polygons, but unlike this time, the amount of texture data is not stated. Are there any restrictions other than the number of polygons?

Question 3
I think that it can be displayed with HoloLens 2 if multiple models are displayed as they are, without combining the models.
However, in that case, I find it difficult to align small models. This is because the coordinate data of FBX disappears when you convert to Azure Remote Rendering.
Is there a way to keep the coordinate data of FBX when converting to Azure Remote Rendering? Even if I erase it, is there a way to reset it?

Answer 1

Hello and thank you for providing detailed information!

I have downloaded the arrAsset and I'm analyzing it now.

Looking at the output.json, the first thing that comes to mind is that the model's box is far off-center:
"boundingBox": {
"min": [
-76.05452728271485,
-0.04086558520793915,
342.1785583496094
],
"max": [
-41.80239486694336,
2.1324777603149416,
370.4200439453125
]

So the center of the scene is somewhere at ~350 meter in z-direction and that's behind the (default) far clip plane of the renderer.
Are you compensating for this offset in your app by transforming the model? Anyway, I'm not sure if that is the real issue here, but I'll find out.

To answer your other questions:

Q1: No, we don't expose this kind of debug functionality out-of-the-box. (Unless of course you do this fully manually through the client API: load the model, find the node, find the material, grab the material's albedo texture handle, then load a small model programmatically, and assign the texture handle to the small model's material)

Q2: There is no hard limit on the number of textures that can be used. There is only a limit on the maximum size of a single texture (16k x 16k), which is in fact a GPU hardware limit. I think I can make the docs more specific here.

Q3: ARR conversion should keep the coordinates as-is, unless you specify the "recenter to origin" option. In that case it's indeed harder to align multiple models because each one recenters to its own average position. However you can look into this section of the json output:

    "recenteringOffset": [
        0.0,
        0.0,
        0.0
    ],

...to see by which amount the object had been moved during conversion (it is (0,0,0) when re-centering was disabled, like in this case). Aligning should be about re-applying this offset to every model.

I'll keep you updated on my findings with the arrAsset.
Let me know if you have more questions!

Cheers,
Florian

Answer 2

Quick update: I was able to see the model when I transformed it to compensate for the offset as indicated by the output bounds.
Specifically, I applied the following translation to the root object after loading: (59, -1, -356).

However the scene graph looks a bit malformed and maybe that is already part of the input fbx: There are thousands of scene graph nodes under the root with no meshes applied, and only the very last node has a mesh that references everything.

Can you confirm that this is a problem with the initial offset?

Answer 3

OK, I see now what you mean.
The difference is indeed between QuickStart and Showcase, not the service.
Quickstart has some code in there that places the object automatically near you when the box extent is far off the origin. The code in question is in file RemoteRendering.cs, line 216 ff:

[...]
    var aabb = (await loadModelResult.Root.QueryLocalBoundsAsync()).toUnity();
    bool tooBig = aabb.extents.magnitude > Camera.main.farClipPlane;
    bool tooFar = aabb.center.magnitude > Camera.main.farClipPlane;
    float scaleFactor = 1.0f;
    String modelMessage = "Model loaded";
    if (tooBig)
    {
        scaleFactor = (2.0f / aabb.extents.magnitude);
        rootGO.transform.localScale = (rootGO.transform.localScale * scaleFactor);
        modelMessage += $", too big (scaled to {(scaleFactor).ToString("P2", CultureInfo.InvariantCulture)})";
    }
    rootGO.transform.localPosition = (rootGO.transform.localPosition - aabb.center * scaleFactor);
    if (tooFar)
    {
        modelMessage += $", center too far (moved by {-aabb.center})";
    }

specifically the line that sets the object's transform to the middle of its bounding box:

rootGO.transform.localPosition = ....

Showcase on the other hand does not have this kind of code, because the placement happens with the hand controllers when you place the blue animated blob. And that position is always near the origin.
Does this explain it? Maybe we can think about how to improve this - as a temporary solution you may add something similar to the Showcase code?

Cheers,
Florian

Answer 4

Sorry for the confusing message.
My problem was that I couldn't see "533946.arrAsset" when using the sample program named "Showcase" on GitHub.

[Showcase]
https://github.com/Azure/azure-remote-rendering/tree/master/Unity/Showcase
[533946.arrAsset]
https://arrdemo20211108.blob.core.windows.net/arrt-convertedmodels/533946.arrAsset

I understand that the origin of the model you pointed out is offset, and I'm sure it still can't be displayed.
If you can, would you please try displaying "533946.arrAsset" in "Showcase"?

Then I tried a sample program named "Quickstart".

[Quickstart]
https://github.com/Azure/azure-remote-rendering/tree/master/Unity/Quickstart

As a result, "Quickstart" was able to display "533946.arrAsset".
From these results, I thought that the reason why "533946.arrAsset" could not be displayed this time was not on the remote rendering service side but on the displaying program side.

So I'm going to compare "Showcase" and "Quickstart" to find out why the model couldn't be displayed.
I will tell you as soon as the result is known.

We are very grateful for the progress made in your research.