HeroEngine Forums

HeroEngine Support => Art & Art Pipeline => Topic started by: gaarzen on Oct 28, 13, 10:45:06 PM

Title: Why do normal maps show incorrectly ?
Post by: gaarzen on Oct 28, 13, 10:45:06 PM
Hey guys,
I've attached pictures of a test I've made.
The first pics show how the normal maps look like inside HE and the next pics show how they look in Maya.
Areas that have smoothed normals appear terrible (the frame and stairs) while areas that have hard edges seem to look fine.
Unfortunately I can not create assets with just hard edges all the time, normals don't bake well in most cases.
Am I missing something? Could it be that the exporter changes all normals to a specific angle and that's why it looks bad?

Thanks a lot !
Title: Re: Why do normal maps show incorrectly ?
Post by: Amanda_Brooks on Oct 29, 13, 10:57:59 AM
Have you tried using the material override function once the models are in the engine to see if that makes a difference?
Title: Re: Why do normal maps show incorrectly ?
Post by: gaarzen on Oct 29, 13, 11:38:04 AM
Have you tried using the material override function once the models are in the engine to see if that makes a difference?
Thanks for the reply, Amanda :)
What can I do in the material overrides ? Everything there seems to be in order.
Title: Re: Why do normal maps show incorrectly ?
Post by: Amanda_Brooks on Oct 29, 13, 12:26:36 PM
I was thinking that you could try applying the material that way, rather than exporting it with the mesh.
Title: Re: Why do normal maps show incorrectly ?
Post by: gaarzen on Oct 29, 13, 12:50:37 PM
I was thinking that you could try applying the material that way, rather than exporting it with the mesh.
I tried messing with it, changing textures. There isn't a way to actually apply materials, it's just the materials that arrive on export.
Doesn't seem to make any difference :(
Title: Re: Why do normal maps show incorrectly ?
Post by: HE-BENNETT on Oct 29, 13, 01:14:05 PM
It would help to see a picture of the normal map itself and a pictures of the asset with a wire frame overlay.

Part of the reason is likely due to the difference in lighting between Max and HeroEngine.
Title: Re: Why do normal maps show incorrectly ?
Post by: gaarzen on Oct 29, 13, 02:07:57 PM
It would help to see a picture of the normal map itself and a pictures of the asset with a wire frame overlay.

Part of the reason is likely due to the difference in lighting between Max and HeroEngine.

I've attached the images. In green the normals are hard edged and in red the normals are soft.
The normal map follows that rule and that's why in certain areas it looks more rigid and in others more spread.
It doesn't seem like a particular lighting issue because the problem occurs exactly on soft edged areas.
Thanks.

Title: Re: Why do normal maps show incorrectly ?
Post by: HE-BENNETT on Oct 29, 13, 02:53:41 PM
From the look of the of normal map, HeroEngine appears to be rendering as I would expect.  I see anomalies in the map consistent with the odd shading on the model.  Am I correct in thinking the maya screenshot is of the hi-poly model?

It looks like you've mirrored some of your UV's, correct?  If you have overlapping UV shells and try to render a normal map it will produce incorrect results. 
Title: Re: Why do normal maps show incorrectly ?
Post by: gaarzen on Oct 29, 13, 03:03:21 PM
From the look of the of normal map, HeroEngine appears to be rendering as I would expect.  I see anomalies in the map consistent with the odd shading on the model.  Am I correct in thinking the maya screenshot is of the hi-poly model?

It looks like you've mirrored some of your UV's, correct?  If you have overlapping UV shells and try to render a normal map it will produce incorrect results.

The screenshot is of the low poly model + normal map actually. It does look as it should in Maya.
I mirrored some of the UVs but they do not overlap, all identical UV parts are above the 1,1 layout as seen in the attached picture, a method that does work with the hard edges areas of this very mesh and also works with our character, although, having checked it it doesn't seem to mind having overlapping UVs either. normal map looks just fine on hard edged areas, and in any case, one side should look right while the other not if that's true.
Those anomalies are simply how Maya or XNormal bake the high res mesh on top of the low res mesh according to normal angles which in this case are different from one area to another.
I guarantee that if I delete all the duplicated areas the problem would remain :-\

Perhaps HE can not handle meshes with different normal angled areas?
Title: Re: Why do normal maps show incorrectly ?
Post by: HE-BENNETT on Oct 29, 13, 04:00:27 PM
That may prevent the normal bake from overwriting between UV shells, but you still end up with a mirrored normal map.  This will never produce correct lighting. There are settings in Maya however that will allow a mirrored normal map to display correctly on both sides, but this will not work with assets exported for HeroEngine.

http://download.autodesk.com/global/docs/maya2014/en_us/index.html?url=files/Polygons_nodes_Tangent_Space.htm,topicNumber=d30e188977,hash=WS17956D7ADBC6E7364737C28117AE335461-7FFF

http://download.autodesk.com/global/docs/maya2014/en_us/index.html?url=files/Asts_Creating_and_using_mirrored_normal_maps_on_a_character_or_object.htm,topicNumber=d30e622216


Going on the assumption that the way Maya is displaying the normal map is deceptive, it seems most likely that the issue lies with the generation of the map itself.

If you would like, send a zip of the asset's scene file and associated textures to support@heroengine.com and I will take a closer look at it.
Title: Re: Why do normal maps show incorrectly ?
Post by: gaarzen on Oct 29, 13, 05:03:27 PM
That may prevent the normal bake from overwriting between UV shells, but you still end up with a mirrored normal map.  This will never produce correct lighting. There are settings in Maya however that will allow a mirrored normal map to display correctly on both sides, but this will not work with assets exported for HeroEngine.

http://download.autodesk.com/global/docs/maya2014/en_us/index.html?url=files/Polygons_nodes_Tangent_Space.htm,topicNumber=d30e188977,hash=WS17956D7ADBC6E7364737C28117AE335461-7FFF

http://download.autodesk.com/global/docs/maya2014/en_us/index.html?url=files/Asts_Creating_and_using_mirrored_normal_maps_on_a_character_or_object.htm,topicNumber=d30e622216


Going on the assumption that the way Maya is displaying the normal map is deceptive, it seems most likely that the issue lies with the generation of the map itself.

If you would like, send a zip of the asset's scene file and associated textures to support@heroengine.com and I will take a closer look at it.

Thanks so much for the offer Bennett, I'll send it right away ! I'm not sure in which format so I'm sending both HGM and OBJ files.
In general nowdays, game models use overlapping UVs in order to conserve UV space and have better looking textures. In maya there is an option to show it that way and that's what I did.
In HE it actually works perfectly fine like that too, I've attached an Image of a character with a temporary color map and although practically half of the character has overlapping and mirrored UVs the normals look just fine.


Title: Re: Why do normal maps show incorrectly ?
Post by: gaarzen on Oct 29, 13, 09:21:53 PM
I tried doing an old trick in which I hardened all edges and split the UV parts between all creased angles.
The normal map does look much better but the UV seams became extremely visible in HE although in Maya it looks fine..

Title: Re: Why do normal maps show incorrectly ?
Post by: HE-Cooper on Oct 30, 13, 01:34:09 AM
How things look in maya are rarely indicative of what they'll look like in an actual environment. Especially when using a number of "maya only" settings or views. You can even see how different the normals are being treated between your two maya examples above. The HE image, as Bennett points out above is rendering the normal map exactly as how I would expect, seams and odd gradients. Bennett might be able to help you see the various settings causing the outcome to be other than you expected, or other than we expected, but I would start your investigation before the pipeline gets involved, as opposed to after, since the normals are all rendering as expected.
Title: Re: Why do normal maps show incorrectly ?
Post by: HE-BENNETT on Oct 30, 13, 01:40:56 PM
Ah, you are correct, I must have mis-remembered some of the testing I've done related to that. 

It is quite possible to get normal mapping that does not have visible seems like that.  Keep trying to tweak the projection and baking process and see what you can get.  Also, send along the Hi-poly model as well so we can take a further look at it. 
Title: Re: Why do normal maps show incorrectly ?
Post by: gaarzen on Oct 30, 13, 04:38:49 PM
The last example actually looks fine in a Maya render and also in a Max render without any special settings.
The examples look completely different because I did the UV and normal baking in 2 different methods.
I would love to know the key to creating those normal maps and/or UVs so I could duplicate those setting to all the assets.
Trying Studio Max and Marmoset and having the normals look fine there, I currently just can't predict the outcome in HE.
Title: Re: Why do normal maps show incorrectly ?
Post by: gaarzen on Nov 14, 13, 10:03:12 PM
Hey guys,
we're still very stuck with this issue :-\
I hope a way to to fixing it is indeed possible.

Thanks
Title: Re: Why do normal maps show incorrectly ?
Post by: Amanda_Brooks on Nov 14, 13, 10:49:49 PM
Can you give more details about the way in which normal maps aren't working?
Title: Re: Why do normal maps show incorrectly ?
Post by: gaarzen on Nov 15, 13, 03:34:57 PM
Can you give more details about the way in which normal maps aren't working?
Hey,
please tell me what exactly to show, I'll do anything :)
My latest test was hardening all the edges of the object and splitting the different areas in the UV layout, creating the attached normal map.
Although it looks good in Studio Max, Maya and Marmoset Toolbag it doesn't look good in HE and the seams are visible and dark like seen in one of my last messages in this thread.

Thanks !!
Title: Re: Why do normal maps show incorrectly ?
Post by: HE-BENNETT on Nov 19, 13, 04:55:56 PM
Many things vie for my attention, but if you're still having an issue I can try and look into this further tomorrow.

To rule out some things I'll probably want to log into your world from here and take a look at the asset.  Can you create a test area with just that asset in it?
Title: Re: Why do normal maps show incorrectly ?
Post by: gaarzen on Nov 19, 13, 05:37:52 PM
Many things vie for my attention, but if you're still having an issue I can try and look into this further tomorrow.

To rule out some things I'll probably want to log into your world from here and take a look at the asset.  Can you create a test area with just that asset in it?
Oh please do ! Unfortunately I still haven't found a way to work it although I tried several different things.
I created a new area called "Normal Map Problem Scene" and in that scene there is the object.
I also change the UV and normal map to have much larger gaps between UV islands to make sure the problem doesn't have to do with them being too close.

Thanks a lot !
Title: Re: Why do normal maps show incorrectly ?
Post by: gaarzen on Nov 20, 13, 05:09:35 PM
SUCCESS !!
Well, almost anyway.
I took the normal map into photoshop and flipped the green channel and now the object looks just as it should in HE ! However, it only looks well without specularity on, specularity causes the creases to be dark and visible ???