Using a fragment shader to convert YUV to RGB

A place for users of OGRE to discuss ideas and experiences of utilitising OGRE in their games / demos / applications.

Using a fragment shader to convert YUV to RGB

Postby JeDi » Wed Nov 08, 2006 3:17 pm

Hi,

We have some video textures that were H264 encoded, and thus in YUV420 pixel format. As Ogre/graphics hardware can't natively use this pixel format, we have to convert it to RGB(A). I want to do (part of) this in a fragment shader, but as I have no experience in this area, I have some questions.

I've found some cg source of a possible shader in this source file. However, I'm not sure how to convert this to the Ogre world.
I've put the cg source itself (without the quotes and newlines off course) in YUVtoRGB.cg, and created this Ogre material:

Code: Select all
uniform sampler2DRect Ytex;
uniform sampler2DRect Utex,Vtex;

void main(void) {
  float nx,ny,r,g,b,y,u,v;
  vec4 txl,ux,vx;
  nx=gl_TexCoord[0].x;
  ny=576.0-gl_TexCoord[0].y;
  y=texture2DRect(Ytex,vec2(nx,ny)).r;
  u=texture2DRect(Utex,vec2(nx/2.0,ny/2.0)).r;
  v=texture2DRect(Vtex,vec2(nx/2.0,ny/2.0)).r;

  y=1.1643*(y-0.0625);
  u=u-0.5;
  v=v-0.5;

  r=y+1.5958*v;
  g=y-0.39173*u-0.81290*v;
  b=y+2.017*u;

  gl_FragColor=vec4(r,g,b,1.0);
}


Code: Select all
fragment_program YUVtoRGB cg
{
  source YUVtoRGB.cg
  entry_point main
  profiles ps_2_0 arbfp1

  //default_params
  //{
  //}
}

material test/video
{
  technique
  {
    pass
    {
      texture_unit
      {
        texture test_video
        tex_address_mode clamp
      }

      fragment_program_ref YUVtoRGB
      {
      }
    }
  }
}


I have no idea how to continue from here. I'm searching for some good reference/tutorial on using shaders in Ogre, but no success so far.

I don't think the shader is very complex, so the profile I've chosen is probably way too heavy. Which parameters should I bind? Where do the gl_TexCoord and gl_FragColor variables come from? Are they automatically accessible?

What I want to do is feed the YUV data to a dynamic texture and let the fragment shader to the conversion to RGB(A).


Any help appreciated!

Greetz,
JeDi
JeDi
Gnome
 
Posts: 351
Kudos: 3
Joined: 21 Oct 2004
Location: Diepenbeek, Belgium

Postby sinbad » Wed Nov 08, 2006 7:29 pm

That code you have there is GLSL, not cg. Also, it's expecting the 3 YUV channels as 3 separate textures rather than one texture with 3 components.

A slightly adapted version from this site leads to

Code: Select all
float3 main(
    in      float3    texcoords0 : TEXCOORD0, 
    uniform sampler yuvImage : register(s0),
) : COLOR

{

    float3 yuvcolor; // f(xyz) -> yvu

    float3 rgbcolor;

    yuvcolor.x = tex2D(yuvImage, texcoords0).x;
    yuvcolor.z = tex2D(yuvImage, texcoords0).y-0.5;
    yuvcolor.y = tex2D(yuvImage, texcoords0).z-0.5; 

    rgbcolor.r = 2*(yuvcolor.x/2 + 1.402/2 * yuvcolor.z);
    rgbcolor.g = 2*(yuvcolor.x/2 - 0.344136 * yuvcolor.y/2 - 0.714136 * yuvcolor.z/2);
    rgbcolor.b = 2*(yuvcolor.x/2 + 1.773/2 * yuvcolor.y); 

    return rgbcolor;

}



Not sure if that's right, just quickly thrown together!
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
 
Posts: 19261
Kudos: 69
Joined: 06 Oct 2002
Location: Guernsey, Channel Islands

Postby JeDi » Wed Nov 08, 2006 9:05 pm

Thanks! After a bit of research, I came up with about the same shader as you posted. The numbers are a bit different though, and that may be why my results are weird. I'll try this one as soon as possible.
JeDi
Gnome
 
Posts: 351
Kudos: 3
Joined: 21 Oct 2004
Location: Diepenbeek, Belgium

Re: Using a fragment shader to convert YUV to RGB

Postby pragnesh_58 » Fri Jul 14, 2017 3:29 pm

1) I have used this link : http://www.fourcc.org/source/YUV420P-OpenGL-GLSLang.c and convert my YUV420 image to rgb image now i want to read back my processed RGB data back into host memory from CPU memory, how can i do this?

2) It will be more helpful if you provide me a full source code for below code (yuv -> rgb )of yours:

float3 main(
in float3 texcoords0 : TEXCOORD0,
uniform sampler yuvImage : register(s0),
) : COLOR

{

float3 yuvcolor; // f(xyz) -> yvu

float3 rgbcolor;

yuvcolor.x = tex2D(yuvImage, texcoords0).x;
yuvcolor.z = tex2D(yuvImage, texcoords0).y-0.5;
yuvcolor.y = tex2D(yuvImage, texcoords0).z-0.5;

rgbcolor.r = 2*(yuvcolor.x/2 + 1.402/2 * yuvcolor.z);
rgbcolor.g = 2*(yuvcolor.x/2 - 0.344136 * yuvcolor.y/2 - 0.714136 * yuvcolor.z/2);
rgbcolor.b = 2*(yuvcolor.x/2 + 1.773/2 * yuvcolor.y);

return rgbcolor;

}
pragnesh_58
Gnoblar
 
Posts: 1
Kudos: 0
Joined: 14 Jul 2017


Return to Using OGRE in practice

Who is online

Users browsing this forum: Majestic-12 [Bot] and 2 guests