Text Over NPC Head: 3 Questions

Problems building or running the engine, queries about how to use features etc.
Post Reply
daves
Goblin
Posts: 214
Joined: Fri Jan 20, 2006 3:35 pm

Text Over NPC Head: 3 Questions

Post by daves »

Context:
I'm trying to place a "data block" which is text overlayed on a translucent background over the head of an NPC.

I have three questions enumerated at the bottom of this post. Images related to these questions are embedded below.

Details:
I've tried two approaches both of which use billboards to ensure that the text always faces the camera. I've had some success with each mechanism but have not refined either.

The first approach that I've steered away from is to use the MovableText function to overlay a billboardset which has an associated background image. I dont want to do this since the movabletext is largely independent of the background billboardset (though MovableText by itself works very well).

The approach that I'm focused on now uses a background texture that is common to all npcs (they all have the same translucent background) and then to create a unique foreground texture and material that is associated with the unique billboardset that is associated with each NPC.

Whenever I need to change the text that is placed above the NPC, I first write the background texture to the foreground texture, I then use the WriteToTexture function (that is presented within the Wiki) to overlay the text.

I'm trying to work out the kinks and the kink that I'm looking for help with here is that the text "disappears" (though the background image does not) when I rotate or back away from the NPC.

Once I work out some of the basic kinks then I'll work to "clean it up, and pretty it up".

Images:
The following is what the NPC data block looks like with the text showing correctly over the background.


Image

The following is what the datablock looks like if I back away from the npc slightly. Notice that the NPC name (Joe) is gone. This same effect happens if I just rotate the camera slightly

Image


Code:
A Datablock is instantiated for each NPC. Ultimately there will be hundreds of such instantiated NPCS. The following is my Constructor code for creating the Datablock:

Code: Select all

Datablock::Datablock(string name, string caption, Ogre::Font *font, Ogre::ColourValue color) : Ogre::BillboardSet(name + ".bb")
{
	m_db_name = name + ".bb";
	setBillboardOrigin(Ogre::BillboardOrigin::BBO_BOTTOM_LEFT);
	setMaterialName("Datablock"); // Use a material file

	createBillboard(0,0,0);

	std::string txt_name = m_db_name + ".txt";
	std::string mat_name = m_db_name + ".mat";

	Ogre::TextureManager *text_mgr = Ogre::TextureManager::getSingletonPtr();
	//Ogre::ResourcePtr rp = text_mgr->getByName("bounding_box.png");
	mBackgroundTexture = (Ogre::Texture *) text_mgr->getByName("bounding_box.png").getPointer();
	mForegroundTexture = (Ogre::TexturePtr) text_mgr->createManual(txt_name, "General",
		Ogre::TEX_TYPE_2D, 256, 256, Ogre::MIP_UNLIMITED, Ogre::PF_A8R8G8B8, Ogre::TU_DEFAULT);
	
	mMaterial = Ogre::MaterialManager::getSingleton().create(mat_name, Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);
	mMaterial->getTechnique(0)->getPass(0)->createTextureUnitState(txt_name);
	mMaterial->getTechnique(0)->getPass(0)->setSceneBlending(Ogre::SBF_SOURCE_ALPHA, Ogre::SBF_ONE_MINUS_SOURCE_ALPHA);
	mMaterial->setDepthBias(false);
    mMaterial->setDepthCheckEnabled(false);
    mMaterial->setDepthWriteEnabled(true);
	setMaterialName(mat_name);

	mFont = ConfigurationManager::getSingleton().getFont();
	assert(mFont);
	//this->showOnTop(true);

	m_background_state = true;

	m_string.assign(caption);
}
The following is my code for updating the datablock whenever the NPC name changes:

Code: Select all


void Datablock::Set(string string)
{
	mForegroundTexture->getBuffer()->blit(mBackgroundTexture->getBuffer());

	writeToTexture(string, mForegroundTexture, Ogre::Image::Box(0,127,255,255), mFont, Ogre::ColourValue(0.0, 1.0, 0.0, 1.0), 'c', true);

	m_string = string;
}

The following is the writeToTexture function for the sake of reference: This can be found on the wiki. Its not my code:

Code: Select all


void Datablock::writeToTexture(std::string &str, Ogre::TexturePtr destTexture, Ogre::Image::Box destRectangle, Ogre::Font* font, Ogre::ColourValue &color, char justify,  bool wordwrap)
{
	using namespace Ogre;
	if (destTexture->getHeight() < destRectangle.bottom)
		destRectangle.bottom = destTexture->getHeight();
	if (destTexture->getWidth() < destRectangle.right)
		destRectangle.right = destTexture->getWidth();

	if (!font->isLoaded())
		font->load();

	TexturePtr fontTexture = (TexturePtr) TextureManager::getSingleton().getByName(font->getMaterial()->getTechnique(0)->getPass(0)->getTextureUnitState(0)->getTextureName());
	
	HardwarePixelBufferSharedPtr fontBuffer = fontTexture->getBuffer();
	HardwarePixelBufferSharedPtr destBuffer = destTexture->getBuffer();


	//When locking destBuffer with HBL_NORMAL there are somn issues with OGL but but with HBL_READ_ONLY it works fine -- very strange .. isn't it?
	PixelBox destPb = destBuffer->lock(destRectangle,HardwareBuffer::HBL_READ_ONLY);
	PixelBox fontPb = fontBuffer->lock(Image::Box(0,0,fontTexture->getWidth(),fontTexture->getHeight()),HardwareBuffer::HBL_READ_ONLY);

	uint8* fontData = static_cast<uint8*>( fontPb.data );
	uint8* destData = static_cast<uint8*>( destPb.data );

	const size_t fontPixelSize = PixelUtil::getNumElemBytes(fontPb.format);
	const size_t destPixelSize = PixelUtil::getNumElemBytes(destPb.format);

	const size_t fontRowPitchBytes = fontPb.rowPitch * fontPixelSize;
	const size_t destRowPitchBytes = destPb.rowPitch * destPixelSize;

	Box *GlyphTexCoords;
	GlyphTexCoords = new Box[str.size()];

	Real u1,v1,u2,v2;
	size_t charheight = 0;
	size_t charwidth = 0;

	for(unsigned int i = 0; i < str.size(); i++)
	{
		if ((str[i] != '\t') && (str[i] != '\n') && (str[i] != ' '))
		{
			font->getGlyphTexCoords(OgreChar(str[i]),u1,v1,u2,v2);
			GlyphTexCoords[i].left = u1 * fontTexture->getSrcWidth();
			GlyphTexCoords[i].top = v1 * fontTexture->getSrcHeight();
			GlyphTexCoords[i].right = u2 * fontTexture->getSrcWidth();
			GlyphTexCoords[i].bottom = v2 * fontTexture->getSrcHeight();

			if (GlyphTexCoords[i].getHeight() > charheight)
				charheight = GlyphTexCoords[i].getHeight();
			if (GlyphTexCoords[i].getWidth() > charwidth)
				charwidth = GlyphTexCoords[i].getWidth();
		}

	}	

	size_t cursorX = 0;
	size_t cursorY = 0;
	size_t lineend = destRectangle.getWidth();
	bool carriagreturn = true;
	for (unsigned int strindex = 0; strindex < str.size(); strindex++)
	{
		switch(str[strindex])
		{
		case ' ': cursorX += charwidth;  break;
		case '\t':cursorX += charwidth * 3; break;
		case '\n':cursorY += charheight; carriagreturn = true; break;
		default:
			{
				//wrapping
				if ((cursorX + GlyphTexCoords[strindex].getWidth()> lineend) && !carriagreturn )
				{
					cursorY += charheight;
					carriagreturn = true;
				}
				
				//justify
				if (carriagreturn)
				{
					size_t l = strindex;
					size_t textwidth = 0;	
					size_t wordwidth = 0;

					while( (l < str.size() ) && (str[l] != '\n)'))
					{		
						wordwidth = 0;

						switch (str[l])
						{
						case ' ': wordwidth = charwidth; ++l; break;
						case '\t': wordwidth = charwidth *3; ++l; break;
						case '\n': l = str.size();
						}
						
						if (wordwrap)
							while((l < str.size()) && (str[l] != ' ') && (str[l] != '\t') && (str[l] != '\n'))
							{
								wordwidth += GlyphTexCoords[l].getWidth();
								++l;
							}
						else
							{
								wordwidth += GlyphTexCoords[l].getWidth();
								l++;
							}
	
						if ((textwidth + wordwidth) <= destRectangle.getWidth())
							textwidth += (wordwidth);
						else
							break;
					}

					if ((textwidth == 0) && (wordwidth > destRectangle.getWidth()))
						textwidth = destRectangle.getWidth();

					switch (justify)
					{
					case 'c':	cursorX = (destRectangle.getWidth() - textwidth)/2;
							lineend = destRectangle.getWidth() - cursorX;
							break;

					case 'r':	cursorX = (destRectangle.getWidth() - textwidth);
							lineend = destRectangle.getWidth();
							break;

					default:	cursorX = 0;
							lineend = textwidth;
							break;
					}

					carriagreturn = false;
				}

				//abort - net enough space to draw
				if ((cursorY + charheight) > destRectangle.getHeight())
					goto stop;

				//draw pixel by pixel
				for (size_t i = 0; i <= GlyphTexCoords[strindex].getHeight(); i++ )
					for (size_t j = 0; j <= GlyphTexCoords[strindex].getWidth(); j++)
					{
 						float alpha =  color.a * (fontData[(i + GlyphTexCoords[strindex].top) * fontRowPitchBytes + (j + GlyphTexCoords[strindex].left) * fontPixelSize +1 ] / 255.0);
 						float invalpha = 1.0 - alpha;
 						size_t offset = (i + cursorY) * destRowPitchBytes + (j + cursorX) * destPixelSize;
  						ColourValue pix;
 						PixelUtil::unpackColour(&pix,destPb.format,&destData[offset]);
 						pix = (pix * invalpha) + (color * alpha);
 						PixelUtil::packColour(pix,destPb.format,&destData[offset]);
  					}
 
				cursorX += GlyphTexCoords[strindex].getWidth();
			}//default
		}//switch
	}//for

stop:
	delete[] GlyphTexCoords;

	destBuffer->unlock();
	fontBuffer->unlock();
}


Questions:
1] Why is the NPC name disappearing when I rotate the camera or back the camera away from the displayed datablock?

2] One thing I'm not really comfortable with in this approach is that if I have 1000 NPC's then this approach will require me to create 1000 billboardsets, 1000 materials, and 1000 foreground textures. My discomfort may be largely due to my inexperience with these mechanisms from a performance perspective.

Is my basic approach reasonable? Will there be a problem with my approach from a performance perspective?

3] Additional thoughts on the best (or alternative) way to do what I'm trying to do would also be beneficial.
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

The reason your billboards are rotating when you rotate the camera is that the billboards are aligned along the camera view vector by default (not the actual vector from the object to the camera). You can override this by using accurate-facing billboards....however....

Don't do this with billboards; just write in screen space instead. Project the object's bounding box into screen space, and write your text above the BB in screen space.
daves
Goblin
Posts: 214
Joined: Fri Jan 20, 2006 3:35 pm

Post by daves »

I guess I'm confused by your response. I wasn't asking "why is the billboard rotating"? My first question was "why is the text block disappearing". The reason why i wanted to use billboards is because they are automatically turned towards the camera (which I do want).

The billboard rotation behaviour seems to be working fine.

So again to summarize my three questions were:
1] why is the text disappearing
2] is this approach (which creates a unique billboardset, texture, and material per NPC) one that will have performance issues
3] what are your thoughts about this and alternative approaches.


You do mention an interesting option which is to draw directly in screen space. I would like to investigate and learn about this more. Can someone point me to a tutorial or wiki page where I can start learning about this?
Last edited by daves on Thu Sep 07, 2006 6:43 pm, edited 1 time in total.
User avatar
CaseyB
OGRE Contributor
OGRE Contributor
Posts: 1335
Joined: Sun Nov 20, 2005 2:42 pm
Location: Columbus, Ohio
x 3
Contact:

Post by CaseyB »

I think the first issue is because of the MipMaps. If you get the texture and write on it, then you are only writing on one mipmap and therefore it'll go away when the LOD changes, but I am very interested in Xaviers idea, but I am not quite sure what you mean! Could you give a little more detail?
Image
Image
daves
Goblin
Posts: 214
Joined: Fri Jan 20, 2006 3:35 pm

Post by daves »

Thanks for the info. Your right on the money. Indeed Mipmaps seems to be the "culprit" relative to question 1.

If I change

Code: Select all

   mBackgroundTexture = (Ogre::Texture *) text_mgr->getByName("bounding_box.png").getPointer(); 
   mForegroundTexture = (Ogre::TexturePtr) text_mgr->createManual(txt_name, "General", 
      Ogre::TEX_TYPE_2D, 256, 256, Ogre::MIP_UNLIMITED, Ogre::PF_A8R8G8B8, Ogre::TU_DEFAULT); 
to

Code: Select all

   mBackgroundTexture = (Ogre::Texture *) text_mgr->getByName("bounding_box.png").getPointer(); 
   mForegroundTexture = (Ogre::TexturePtr) text_mgr->createManual(txt_name, "General", 
      Ogre::TEX_TYPE_2D, 256, 256, 1, Ogre::PF_A8R8G8B8, Ogre::TU_DEFAULT); 
then my text no longer dissappears.

Interestingly mipmapping also seemed to cause another issue, which makes me wonder about the correct way to apply mipmaps to this kind of problem (if at all).

If I changed the information displayed about my npc at a particular distance from the camera and then moved towards or away from the npc then the "old" datablock would be displayed. Presumably this is because the mechanism that I'm using only writes the mipmap for the given distance from the camera.

This is demonstrated in the following images:

Correct Display Containing "Joe":

Image

If all I do is move camera away from the Joe NPC I suddenly see an older version of Joes Datablock (has an additional bit of information in it)

Image

Why is there a "stale" mipmap created given the technique that I am using?

If I want to use mipmaps, what do I need to do in order to write an up-to-date image at each level?
daves
Goblin
Posts: 214
Joined: Fri Jan 20, 2006 3:35 pm

Post by daves »

xavier wrote: Don't do this with billboards; just write in screen space instead. Project the object's bounding box into screen space, and write your text above the BB in screen space.
So Xavier, now that I seem to have an answer to my first question (mipmaps)... please help me to understand your caution here.

You say "Dont do this with billboards". Can you please elaborate. To me billboards made sense since they would always orient the text and the background pattern towards the camera. Clearly if I write this textblock in screen space then this will be achieved and perhaps more efficiently.

Are billboards overkill here? Should I be concerned about performance? Please tell me what leads you to suggest this. Could you please tell me what function(s) i'd use to project the objects bounding box into screen space and then what functions would i use to write the text and the background image into screen space.

I have the basic function working now so before I change it to write in screen space I'd just like to understand some of the motivation behind this.

Thanks for additional clarification here!
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

User avatar
Kentamanos
Minaton
Posts: 980
Joined: Sat Aug 07, 2004 12:08 am
Location: Dallas, TX

Post by Kentamanos »

Nice article, that was quick :).
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

Kentamanos wrote:Nice article
Hehe thanks....
that was quick :).
...it might seem that way, but I fought Ogre's overlay metrics for far too many hours today trying to figure out where my text was being displayed. ;)
Slicky
Bronze Sponsor
Bronze Sponsor
Posts: 614
Joined: Mon Apr 14, 2003 11:48 pm
Location: Was LA now France
x 25

Post by Slicky »

Nice article. I'm hoping that I can learn something from this. Previously the only method that I was aware of was billboards.

It looks like if you had a bunch of entities you would have to iterate them to run the text->updates?
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

You would really only need to update() them if you changed the position of the node to which their object is attached, or altered the mesh that would change the bounding box. I put it in for every frame for clarity, but that's really overkill in practice.

The OverlayManager takes care of the actual overlay rendering, so any overlays that are visible and enabled will be rendered for you automatically.
Slicky
Bronze Sponsor
Bronze Sponsor
Posts: 614
Joined: Mon Apr 14, 2003 11:48 pm
Location: Was LA now France
x 25

Post by Slicky »

Very cool - thanks. I am going to try this method instead of some billboards that once in a while had text that disappeared. Also, I'm curious about the renderqueue to see where they are positioned.
daves
Goblin
Posts: 214
Joined: Fri Jan 20, 2006 3:35 pm

Post by daves »

Wow Xavier. You are the man!

I certainly appreciate the effort. It will be very informative to look carefully at your code and to work through an example.

Now, of course since I want my text to have a background similar to the following this code is not quite what I want.

http://i99.photobucket.com/albums/l305/ ... ure1-1.png


Do you think it should be fairly straightforward to provide such a background using your approach?


I'd also still appreciate additional veterans perspectives on the methodology that I've selected (which does indeed use billboards and a material for each npc). Again this methodology is doing what I want it to do, so I'm not sure if there is a good reason to move away from it. One possible reason to move away is captured in Xavier's wiki thoughts (following is snippet from the wiki):
One method is to employ billboards for this purpose, but billboards have their own caveats and gotchas when used for this purpose; for example, maintaining text legibility as distance increases
This is indeed an issue/effect in my current implementation. Indeed its an effect that I can probably live with so unless there are other reasons to move away I may just stick with billboards.

Xavier, Thanks again for taking the time. Very much appreciated.
Slicky
Bronze Sponsor
Bronze Sponsor
Posts: 614
Joined: Mon Apr 14, 2003 11:48 pm
Location: Was LA now France
x 25

Post by Slicky »

I have played with this a little. The text shows up well. However, it doesn't stay anywhere close to the object it is attached to. I have a few cameras in my scene but so far it seems to just move around.
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

@Dave: you can assign a material to OverlayContainers for their backgrounds

@Slicky: Like I said, it might not even be correct. ;) I'll put it in our app (I did that one from scratch and there is no movement in it) and see what's up. Likely it's a missing or misused transform.
Slicky
Bronze Sponsor
Bronze Sponsor
Posts: 614
Joined: Mon Apr 14, 2003 11:48 pm
Location: Was LA now France
x 25

Post by Slicky »

Yeah I understand. It is good work considering no real testing. I thought I'd throw it out there since I don't know how easily I can find it. If I get lucky I'll modify the wiki.
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

Hmm, it works fine for me in our app. You said you have multiple cameras -- did you add a method to reset the camera it uses? If not, it will always use that first camera that you supplied in the constructor.
kneeride
Bugbear
Posts: 807
Joined: Sun May 14, 2006 2:24 pm
Location: Melbourne, Australia

Post by kneeride »

Hi xavier. I have run the code.

For me, the text is mirrored. ie when i walk the object to the left, the text moves to the right. When i walk to the right, the text moves to the left. The text is centred over the object's head when the object is in the middle of the screen.

The height is fine.

I'm not sure if anyone else is getting this behavior.
Slicky
Bronze Sponsor
Bronze Sponsor
Posts: 614
Joined: Mon Apr 14, 2003 11:48 pm
Location: Was LA now France
x 25

Post by Slicky »

I haven't had time to dig into it but my text does move left and right independent of the object when I use the mouse to pan for instance. It may be my implementation that I need to customize this code somehow.
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

I certainly will have a look into it. :)
kneeride
Bugbear
Posts: 807
Joined: Sun May 14, 2006 2:24 pm
Location: Melbourne, Australia

Post by kneeride »

Hi Xavier,
I've been reading over your code because I am geting some unexpected behaviour (described in one of my posts above).

I'm wondering whether the world to screen is correct:

Code: Select all

	// multiply the AABB corner vertex by the view matrix to 
	// get a camera-space vertex
	corner = mat * corner;

	
	// make 2D relative/normalized coords from the view-space vertex
	
	// by dividing out the Z (depth) factor -- this is an approximation
	float x = corner.x / corner.z + 0.5;
	float y = corner.y / corner.z + 0.5;
The reasoning for divide by z is not clear to me. Can you please explain further?

I've found another thread:
http://www.ogre3d.org/phpBB2/viewtopic. ... oordinates

The following is a code snippet from Sinbad. Sinbad uses the projection matrix in his equation:

Code: Select all

// worldPos is a Vector3, but return value is also a Vector3 which is automatically divided by the homogenous 'w' component 

Vector3 hcsPosition = cam->getProjectionMatrix() * cam->getViewMatrix() * worldPos; 

/* HCS stands for 'homogenous clip space' and represents a space from -1 to 1 in all directions which relates to the frustum as distorted by perspective. A point within {-1,1} is within the view frustum, points which are outside this range in any axis are not. An (x,y) of (0,0) is in the centre of the screen. 
If you want to turn this into a pixel you have to factor in the Viewport dimensions. 
*/ 

Another poster created the following function. Which uses Sinbad's formula and then uses the RenderWindow to calc the screen coords:

Code: Select all

// worldToScreenPos 
// 
// returns true if the worldPos is on screen, putting the screen pos into the x & y of the 
// second Vector2 
// returns false if the worldPos is off screen 
// 
bool worldToScreen(Camera* pCamera, RenderWindow* pWindow, Vector3& worldPos, Vector3& screenPos) 
{ 
   Vector3 hcsPosition = pCamera->getProjectionMatrix() * (pCamera->getViewMatrix() * worldPos); 

   if ((hcsPosition.x < -1.0f) || 
      (hcsPosition.x > 1.0f) || 
      (hcsPosition.y < -1.0f) || 
      (hcsPosition.y > 1.0f)) 
      return false; 

   int nCWidth = (pWindow->getWidth()/2); 
   int nCHeight = (pWindow->getHeight()/2); 

   screenPos.x = nCWidth + (nCWidth * -hcsPosition.x); 
   screenPos.y = nCHeight + (nCHeight * hcsPosition.y); 

   return true; 
}
BTW, From reading over the above code, I believe there is a bug:
Should be:
screenPos.x = nCWidth + (nCWidth * hcsPosition.x);
instead of:
screenPos.x = nCWidth + (nCWidth * -hcsPosition.x);

The post then gets confusing because posters say the view to screen forumla is different for d3d and opengl renderers. I think madtulip gets it right in the end, but his code is mixed with app specific code. It would be good to have a specific WorldToScreen function in the wiki. Then such a function could be used for the overlays or other tasks.

I'll have a look at it sometime this week
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

If you beat me to it, by all means feel free to update the code in the Wiki. :)
formula123
Gnoblar
Posts: 7
Joined: Thu Aug 14, 2014 6:23 am

Re: Text Over NPC Head: 3 Questions

Post by formula123 »

Post Reply