Shader FX to Ogre details

I'm trying to get oFusion Pro to export materials and shaders from Shader FX. I'm using 3ds Max 2011, Shader FX 3.51, and oFusion Pro 1.9.73. I've installed the latest HotFix for oFusion (383). I also swapped out the Shader FX script files in the oFusionShaderFXsupport download (from the oFusion site). I'm a programmer and only marginally familiar with Max so I might be missing something obvious (I'm trying to get the process figured out and documented for the artists). I tried to follow the steps Evak outlined in this post:

http://www.ogre3d.org/addonforums/viewtopic.php?f=5&t=8527

Specifically, I create a Shader FX material, assign it to an object, it shows up in the Max perspective viewport. When I export, all I get is the standard material. I've tried a bit to map the Shader FX material to a Max oFusion material, but only made a mess (and didn't get anything new exported).

What I'm hoping to get out of the export is the Ogre .material and .program, the shader parameter values (including textures) in the material file. The shader code would be nice but not necessary since it can be exported from Shder FX.

Can anyone detail the steps I need to get a Shader FX material exported with oFusion?

The export and creation process is automatic, there is no need for any other steps, just apply the shader in your objects as usual

Is the "export shaders" option selected in the Export dialog?
Are the shaders correctly displayed in the preview viewport?

It might be that you are using a procedural node in your shader, those are not supported as they rely on a Direct3D feature that is not supported by Ogre

Is this issue with any or just with specific shaders?

I finally got a Shader FX material exported. It looks like the oFusion viewport needs to be visible otherwise a default material gets exported. I also had a case where the oFusion viewport was visible but default material was getting exported until I moved stuff around in the oFusion viewport (but I haven't verified that case).

I still don't have the Shader FX material showing up properly in Ogre - but I don't think that's an oFusion issue.
Thanks.

I've found that setting the material in shaderFX to Pixel shader 3 and all lights in 1 pass produces the best results. Most of the time Shader model 2 and 1 light per pass doesn't work out the box.

I was getting the following errors with the simplest shader I could build in Shader FX (Standard Material with only diffuse color). I was still trying to figure out why it was failing - but I'll give your suggestion (sm3 and 1 pass lights) a try. Thanks!


15:34:25: WARNING: material StandardMat_3132 has no supportable Techniques and will be blank. Explanation:
Pass 0: Vertex program StandardMat_3132_vs cannot be used - not supported.
Pass 0: Vertex program StandardMat_3132_vs cannot be used - not supported.


The problem was that I was trying to use the exported HLSL shaders with our custom GL-based RenderSystemPlugin. When I use the standard DirectX9 RenderSystemPlugin, the ShaderFX materials show up in Ogre. We've been using Ogre for over a year and all this time I thought Ogre magically made any shader work with any RenderSystem. But looking through the Sample Ogre materials, it's clear that I was dreaming.

In ShaderFX, I switched the Standard Material FX Format from 3dsMax to CGFX but oFusion still exported HLSL. Is there a way to get oFusion to export a OpenGL compatible shader?

Actually HLSL and CG shaders are the same, they have just a few minor differences (including the file extensions, the name of the parameters in the .program file and some stricter code in some situations), but you can just change them and use the shader in OpenGL, that is how all of the shaders in the advanced samples sections were created, they are created in the HLSL and just changed the file extension and the parameter names in the .program file

I had tried this before (treating the hlsl as cg) but was discouraged by the number of errors when compiling as cg:


14:34:46: Parsing script OrangePot.material
14:34:46: OGRE EXCEPTION(7:InternalErrorException): Unable to compile Cg program StandardMat_4099_vs: CG ERROR : "The compile returned an error."
(29) : warning C7023: Extra brace level in initializer being ignored
(30) : warning C7023: Extra brace level in initializer being ignored
(183) : warning C7011: implicit cast from "float4" to "float3"
(125) : error C1101: ambiguous overloaded function reference "mul(float3, float4x4)"
(0) : float3x4 mul(float3x1, float1x4)
(0) : float3x3 mul(float3x1, float1x3)
(0) : float3x2 mul(float3x1, float1x2)
(0) : float3x1 mul(float3x1, float1x1)
(0) : float2x4 mul(float2x1, float1x4)
(0) : float2x3 mul(float2x1, float1x3)
(0) : float2x2 mul(float2x1, float1x2)
(0) : float2x1 mul(float2x1, float1x1)
(0) : float1x4 mul(float1x3, float3x4)
(0) : float1x3 mul(float1x3, float3x3)
(0) : float1x2 mul(float1x3, float3x2)
(0) : float1x1 mul(float1x3, float3x1)
(0) : float1x4 mul(float1x2, float2x4)
(0) : float1x3 mul(float1x2, float2x3)
(0) : float1x2 mul(float1x2, float2x2)
(0) : float1x1 mul(float1x2, float2x1)
(0) : float1x4 mul(float1x1, float1x4)
(0) : float1x3 mul(float1x1, float1x3)
(0) : float1x2 mul(float1x1, float1x2)
(0) : float1x1 mul(float1x1, float1x1)
(0) : fixed1 mul(fixed1, fixed1x1)
(0) : fixed1 mul(fixed2, fixed2x1)
(0) : fixed1 mul(fixed3, fixed3x1)
(0) : fixed2 mul(fixed1, fixed1x2)
(0) : fixed2 mul(fixed2, fixed2x2)
(0) : fixed2 mul(fixed3, fixed3x2)
(0) : fixed3 mul(fixed1, fixed1x3)
(0) : fixed3 mul(fixed2, fixed2x3)
(0) : fixed3 mul(fixed3, fixed3x3)
(0) : fixed4 mul(fixed1, fixed1x4)
(0) : fixed4 mul(fixed2, fixed2x4)
(0) : fixed4 mul(fixed3, fixed3x4)
(0) : half1 mul(half1, half1x1)
(0) : half1 mul(half2, half2x1)
(0) : half1 mul(half3, half3x1)
(0) : half2 mul(half1, half1x2)
(0) : half2 mul(half2, half2x2)
(0) : half2 mul(half3, half3x2)
(0) : half3 mul(half1, half1x3)
(0) : half3 mul(half2, half2x3)
(0) : half3 mul(half3, half3x3)
(0) : half4 mul(half1, half1x4)
(0) : half4 mul(half2, half2x4)
(0) : half4 mul(half3, half3x4)
(0) : float1 mul(float1, float1x1)
(0) : float1 mul(float2, float2x1)
(0) : float1 mul(float3, float3x1)
(0) : float2 mul(float1, float1x2)
(0) : float2 mul(float2, float2x2)
(0) : float2 mul(float3, float3x2)
(0) : float3 mul(float1, float1x3)
(0) : float3 mul(float2, float2x3)
(0) : float3 mul(float3, float3x3)
(0) : float4 mul(float1, float1x4)
(0) : float4 mul(float2, float2x4)
(0) : float4 mul(float3, float3x4)
(125) : error C1115: unable to find compatible overloaded function "mul(float3x3, error)"
in CgProgram::loadFromSource at ..\..\..\ogre-v1-7-0\PlugIns\CgProgramManager\src\OgreCgProgramManagerDll.cpp (line 67)
14:34:46: High-level program StandardMat_4099_vs encountered an error during loading and is thus not supported.
OGRE EXCEPTION(7:InternalErrorException): Unable to compile Cg program StandardMat_4099_vs: CG ERROR : "The compile returned an error."
(29) : warning C7023: Extra brace level in initializer being ignored
(30) : warning C7023: Extra brace level in initializer being ignored
(183) : warning C7011: implicit cast from "float4" to "float3"
(125) : error C1101: ambiguous overloaded function reference "mul(float3, float4x4)"
(0) : float3x4 mul(float3x1, float1x4)
(0) : float3x3 mul(float3x1, float1x3)
(0) : float3x2 mul(float3x1, float1x2)
(0) : float3x1 mul(float3x1, float1x1)
(0) : float2x4 mul(float2x1, float1x4)
(0) : float2x3 mul(float2x1, float1x3)
(0) : float2x2 mul(float2x1, float1x2)
(0) : float2x1 mul(float2x1, float1x1)
(0) : float1x4 mul(float1x3, float3x4)
(0) : float1x3 mul(float1x3, float3x3)
(0) : float1x2 mul(float1x3, float3x2)
(0) : float1x1 mul(float1x3, float3x1)
(0) : float1x4 mul(float1x2, float2x4)
(0) : float1x3 mul(float1x2, float2x3)
(0) : float1x2 mul(float1x2, float2x2)
(0) : float1x1 mul(float1x2, float2x1)
(0) : float1x4 mul(float1x1, float1x4)
(0) : float1x3 mul(float1x1, float1x3)
(0) : float1x2 mul(float1x1, float1x2)
(0) : float1x1 mul(float1x1, float1x1)
(0) : fixed1 mul(fixed1, fixed1x1)
(0) : fixed1 mul(fixed2, fixed2x1)
(0) : fixed1 mul(fixed3, fixed3x1)
(0) : fixed2 mul(fixed1, fixed1x2)
(0) : fixed2 mul(fixed2, fixed2x2)
(0) : fixed2 mul(fixed3, fixed3x2)
(0) : fixed3 mul(fixed1, fixed1x3)
(0) : fixed3 mul(fixed2, fixed2x3)
(0) : fixed3 mul(fixed3, fixed3x3)
(0) : fixed4 mul(fixed1, fixed1x4)
(0) : fixed4 mul(fixed2, fixed2x4)
(0) : fixed4 mul(fixed3, fixed3x4)
(0) : half1 mul(half1, half1x1)
(0) : half1 mul(half2, half2x1)
(0) : half1 mul(half3, half3x1)
(0) : half2 mul(half1, half1x2)
(0) : half2 mul(half2, half2x2)
(0) : half2 mul(half3, half3x2)
(0) : half3 mul(half1, half1x3)
(0) : half3 mul(half2, half2x3)
(0) : half3 mul(half3, half3x3)
(0) : half4 mul(half1, half1x4)
(0) : half4 mul(half2, half2x4)
(0) : half4 mul(half3, half3x4)
(0) : float1 mul(float1, float1x1)
(0) : float1 mul(float2, float2x1)
(0) : float1 mul(float3, float3x1)
(0) : float2 mul(float1, float1x2)
(0) : float2 mul(float2, float2x2)
(0) : float2 mul(float3, float3x2)
(0) : float3 mul(float1, float1x3)
(0) : float3 mul(float2, float2x3)
(0) : float3 mul(float3, float3x3)
(0) : float4 mul(float1, float1x4)
(0) : float4 mul(float2, float2x4)
(0) : float4 mul(float3, float3x4)
(125) : error C1115: unable to find compatible overloaded function "mul(float3x3, error)"
in CgProgram::loadFromSource at ..\..\..\ogre-v1-7-0\PlugIns\CgProgramManager\src\OgreCgProgramManagerDll.cpp (line 67)
14:34:46: OGRE EXCEPTION(2:InvalidParametersException): Named constants have not been initialised, perhaps a compile error. in GpuProgramParameters::_findNamedConstantDefinition at ..\..\ogre-v1-7-0\OgreMain\src\OgreGpuProgramParams.cpp (line 1423)
14:34:46: Compiler error: invalid parameters in OrangePot.material(32): setting of constant failed
14:34:46: OGRE EXCEPTION(2:InvalidParametersException): Named constants have not been initialised, perhaps a compile error. in GpuProgramParameters::_findNamedConstantDefinition at ..\..\ogre-v1-7-0\OgreMain\src\OgreGpuProgramParams.cpp (line 1423)
14:34:46: Compiler error: invalid parameters in OrangePot.material(33): setting of constant failed
14:34:46: OGRE EXCEPTION(2:InvalidParametersException): Named constants have not been initialised, perhaps a compile error. in GpuProgramParameters::_findNamedConstantDefinition at ..\..\ogre-v1-7-0\OgreMain\src\OgreGpuProgramParams.cpp (line 1423)
14:34:46: Compiler error: invalid parameters in OrangePot.material(34): setting of constant failed
14:34:46: OGRE EXCEPTION(2:InvalidParametersException): Named constants have not been initialised, perhaps a compile error. in GpuProgramParameters::_findNamedConstantDefinition at ..\..\ogre-v1-7-0\OgreMain\src\OgreGpuProgramParams.cpp (line 1423)
14:34:46: Compiler error: invalid parameters in OrangePot.material(35): setting of constant failed
14:34:46: OGRE EXCEPTION(2:InvalidParametersException): Named constants have not been initialised, perhaps a compile error. in GpuProgramParameters::_findNamedConstantDefinition at ..\..\ogre-v1-7-0\OgreMain\src\OgreGpuProgramParams.cpp (line 1423)
14:34:46: Compiler error: invalid parameters in OrangePot.material(36): setting of constant failed
14:34:46: OGRE EXCEPTION(2:InvalidParametersException): Named constants have not been initialised, perhaps a compile error. in GpuProgramParameters::_findNamedConstantDefinition at ..\..\ogre-v1-7-0\OgreMain\src\OgreGpuProgramParams.cpp (line 1423)
14:34:46: Compiler error: invalid parameters in OrangePot.material(37): setting of constant failed


But after your post, I dug back into it and it was only a few lines in the vertex shader that needed to be changed. However, the shader conversion to cg doesn't display _anything_. At least I got a white teapot when the cg compile failed. I read somewhere that the mul() order needs to be reversed for hlsl and cg. I hope this is not true as that would be a major tweak to the shader. Below is the only significant change I made to the cg conversion. I'll be happy to include the full shader (94 lines) if that would be useful.


float3
lightVec_func(float3 worldSpacePos, float3 lightVector, float3x3 objTangentXf, int lightType)
{
// HLSL -> CG
//float3 lightVec = mul(objTangentXf, (mul((lightVector - worldSpacePos), worldI).xyz));
float3 lightVec = mul(objTangentXf, (mul(float4((lightVector - worldSpacePos),1), worldI).xyz));
return lightVec;
}


Another data point:

The converted shader (hlsl to cg) works fine under Direct3D9 Rendering Subsystem but not under OpenGL Rendering Subsystem. Under OpenGL no errors are generated but the shaded object doesn't render.

Yes, you will need to modify the matrices and axes order (swizzling) because of the differences between how the render systems are setup

Additionally bear in mind that D3D uses transposed matrices by default, you will need to change them to the non transposed versions (as in D3D worldViewProj_Transposed -> worldViewProj in CG), this is a trivial change, as is the Swizzling order

After you change the constants to the non transposed versions, the object should be rendered and you can experiment with thew component order to see how they are mapped in CG

I used the transpose of the auto parameters. Sill no teapot. I swapped the mul order for transforming the verts to view space. Still no teapot. I reduced the shader down to the bare minimum (vs only transform vert to view space, ps output yellow). Still no teapot. The only thing left to mess with was the profile. I was using vp30 to match the vs_3_0 output from oFusions hlsl. I tried various GL profiles and got some very surprising results:

arbvp1 - yellow teapot (expected)
vp20 - material error (technique not supported)
vp30 - no teapot
vp40 = yellow teapot (expected)

I have a Quadro FX 3700 with the latest nVidia driver (2010.10.04). I always assumed that you could fall back to an earlier profile. I'm a bit baffled by the compile failure for vp20 and the compile-but-fail of vp30. I'll restore all the cg code that I disabled and see if ShaderFX->oFusion->hlsl->cg works with vp40.

Send me the shader files and I will take a look at them at the first available opportunity

Like I said, I've reduced the shader down to the minimally functional vertex+frag shader. So at this point, it has very little to do with oFusion - but if you have insight into why this would be failing under vp30, I would love to hear it. The reduced material and shaders are so short that I can just list them here.

material:

float4x4 wvp;

fragment_program StandardMat_4099_ps_CG cg
{
source StandardMat_4099_ps.cg
entry_point f
profiles ps_3_0 fp30
}

vertex_program StandardMat_4099_vs_CG cg
{
source StandardMat_4099_vs.cg
entry_point v
profiles vs_3_0 vp30
}


material StandardMat_4099
{
technique
{
pass
{
specular 1 1 1 1 0
emissive 1 1 1 1

vertex_program_ref StandardMat_4099_vs_CG
{
param_named_auto wvp worldviewproj_matrix
}
fragment_program_ref StandardMat_4099_ps_CG
{
}
}
}
}


StandardMat_4099_vs.cg:

float4x4 wvp;

struct a2v{
float4 position : POSITION;
};
struct v2f{
float4 position : POSITION;
};

v2f
v(a2v In, uniform float4x4 wvp)
{
v2f Out = (v2f)0;
Out.position = mul(wvp, In.position);
return Out;
}


StandardMat_4099_ps.cg:

struct v2f{
float4 position : POSITION;
};

float4
f(v2f In) : COLOR
{
float4 done = float4(1,1,0, 1);
return done;
}


I converted over the real shader (the one exported from a ShaderFX material) from hlsl to cg by using the transposed versions of the auto parameters, changing the mul() orders, extracting elements from the matrix by column vs row, and other nit-picky stuff. The teapot showed up (with vp40) but the shader was all messed up. I ended up exporting cgfx from ShaderFX to compare to my hand conversion and found a few things I missed. After fixing those the shader is working under OpenGL - yay. But this is far from the push-button oFusion shader export I was looking for. Maybe converting 100+ lines of auto-generated hlsl code to cg is easy for some - but not for mere mortals.

Lioric: Any possibility of an oFusion option to export CG shaders in the foreseeable future?

Yes, we have implemented CG support but due to internal issues with the Max native shader parser (it produces different results between the viewports and its CG support is not very robust), we have not enabled this feature on public releases

We will continue to page Autodesk on this and hope that they correct this issues soon (if only they merge or just update their internal parsers the world will be a better place)