[2008 - Accepted] Motion Synthesis based Character Animation

Threads related to Google Summer of Code

Postby tuan kuranes » Fri May 23, 2008 1:46 pm

1. Rather than 'automatic rigging', at first and only if need, 'automatic fitting' would be enough, which means if a skeleton has exaclty the skelton that MOCAP animation uses, but with different names, user can provide a 'mapping' file that help MOCAP apply animation on correct bone.
But that can be left for later, just 'planned by design' could be enough.
'AutoRig' is still an option, but only once everything else done.

2 ogrecollada is just another format. as BVH/AMF are 'industry standard', and converter from one to another already exists, only supporting one fomat is enough.

only animation in .skeleton is replaced

Could rather be 'added', or in another skeleton.

downsample ...keyframing the original animation.


I would first import it totally without downsample or any form of simplification. One Frame of mocap => one Keyframe.

As a second step, with some control parameters do the downsample, smoothness, acceptable error level, and any other stuff.

Those methods of 'downsampling, keyframing, optimising, etc..' could be methods of Ogre::VertexAnimation like existing one 'Ogre::AnimationTrack::optimise()' already does.

That way it could benefits all other form of vertex animation.
User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
 
Posts: 2653
Kudos: 4
Joined: 24 Sep 2003
Location: Haute Garonne, France

Postby Wolfmanfx » Wed May 28, 2008 3:14 pm

Hi westine,
A nice feature for me would be log messages when you make a commit :)
User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
 
Posts: 1525
Kudos: 107
Joined: 03 Feb 2006
Location: Austria - Leoben

Postby westine » Thu May 29, 2008 3:55 pm

Wolfmanfx wrote:Hi westine,
A nice feature for me would be log messages when you make a commit :)

Ok, I will do it when I commit.
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

some problems

Postby westine » Tue Jun 03, 2008 3:14 pm

Now I can add mocap data in bvh mocap data format into Ogre's .skeleton file animation section, but there are some problems when the new added animation is been playing.

the cammand you use to add animation is as follows:

OgreXMLConverter.exe -addanimation rush.bvh jaiqua.skeleton.xml rush.skeleton.xml

rush.bvh is a bvh mocap file which contains the animation you need,
jaiqua.skeleton.xml is a original Ogre file, which is associated with a mesh file, the new merged rush.skeleton.xml contains the new animation.

The problem is that, the bvh has a different bone hierarchy with jaiqua.skeleton's.

bvh_bone id = "0" name = "hip" skeleton_bone id = "33" name ="Spineroot"
bvh_bone id = "1" name = "lhipjoint" skeleton_bone id = "27" name ="Llegroot"
bvh_bone id = "2" name = "lfemur" skeleton_bone id = "16" name ="Lthign"
bvh_bone id = "3" name = "ltibia" skeleton_bone id = "17" name ="Lshin"
bvh_bone id = "4" name = "lfoot" skeleton_bone id = "18" name ="lfoot"
bvh_bone id = "5" name = "ltoes" skeleton_bone id = "20" name ="Ltoe"
bvh_bone id = "6" name = "ltoes_End" skeleton_bone id = "" name =""
bvh_bone id = "7" name = "rhipjoint" skeleton_bone id = "32" name ="Rlegroot"
bvh_bone id = "8" name = "rfemur" skeleton_bone id = "14" name ="Rthigh"
bvh_bone id = "9" name = "rtibia" skeleton_bone id = "15" name ="Rshin"
bvh_bone id = "10" name = "rfoot" skeleton_bone id = "19" name ="Rfoot"
bvh_bone id = "11" name = "rtoes" skeleton_bone id = "21" name ="Rtoe"
bvh_bone id = "12" name = "rtoes_End" skeleton_bone id = "" name =""
bvh_bone id = "13" name = "lowerback" skeleton_bone id = "13" name ="Spine01"
bvh_bone id = "14" name = "upperback" skeleton_bone id = "12" name ="Spine02"
bvh_bone id = "15" name = "thorax" skeleton_bone id = "11" name ="Spine03"
bvh_bone id = "16" name = "lowerneck" skeleton_bone id = "34" name ="neckroot"
bvh_bone id = "17" name = "upperneck" skeleton_bone id = "1" name ="neck"
bvh_bone id = "18" name = "head" skeleton_bone id = "0" name ="head"
bvh_bone id = "19" name = "head_End" skeleton_bone id = "" name =""
bvh_bone id = "20" name = "lclavicle" skeleton_bone id = "26" name ="Lshoulderroot"
bvh_bone id = "21" name = "lhumerus" skeleton_bone id = "23" name ="Larmroot"
bvh_bone id = "22" name = "lradius" skeleton_bone id = "9" name ="Lforearm"
bvh_bone id = "23" name = "lwrist" skeleton_bone id = "25" name ="Lhandroot"
bvh_bone id = "24" name = "lhand" skeleton_bone id = "10" name ="Lhand"
bvh_bone id = "25" name = "lfingers" skeleton_bone id = "22" name ="Lfingers"
bvh_bone id = "26" name = "lfingers_End" skeleton_bone id = "" name =""
bvh_bone id = "27" name = "lthumb" skeleton_bone id = "" name =""
bvh_bone id = "28" name = "lthumb_End" skeleton_bone id = "" name =""
bvh_bone id = "29" name = "rclavicle" skeleton_bone id = "31" name ="Rshoulderroot"
bvh_bone id = "30" name = "rhumerus" skeleton_bone id = "28" name ="Rarmroot"
bvh_bone id = "31" name = "rradius" skeleton_bone id = "3" name ="Rforearm"
bvh_bone id = "32" name = "rwrist" skeleton_bone id = "30" name ="Rhandroot"
bvh_bone id = "33" name = "rhand" skeleton_bone id = "4" name ="Rhand"
bvh_bone id = "34" name = "rfingers" skeleton_bone id = "5" name ="Rfingers"
bvh_bone id = "35" name = "rfingers_End" skeleton_bone id = "" name =""
bvh_bone id = "36" name = "rthumb" skeleton_bone id = "" name =""
bvh_bone id = "37" name = "rthumb_End" skeleton_bone id = "" name =""

this is the video http://www.youtube.com/watch?v=MKzpOE4HzOc
Last edited by westine on Wed Jun 04, 2008 3:54 am, edited 1 time in total.
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

Postby tuan kuranes » Tue Jun 03, 2008 4:36 pm

Three case I guess ?


1) If it's just a bone naming problem, then user would just have to provide a config 'mapbones.cfg' file

[code]
hip=Spineroot
lhipjoint=Llegroot
...
[code]

then cmd line would become
OgreXMLConverter.exe -addanimation rush.bvh jaiqua.skeleton.xml rush.skeleton.xml mapbones.cfg

So for now, user has to provide its mapping config file.

2) If some bones miss, then again mutliple case, either start-bones, end-bones or middle bones. Start bones cannot be discarded, but end-bones can. Middle bones perhaps can have their position either discarded or 'summed' with neighbour bones ? (using config file)

discard can be explicit
[code]
rthumb=0
rthumb_End=0
ltoes_End=0
...
[code]

or implicit, if bone is not listed in the config file.

3) If it's a bone approximation fitting problem, where target bones doesn't fit at all. Then approach is much more complex, 'Skeleton simplification', 'mesh autorig' or even latest amazing 'skeleton extraction' may help there, but anyway, those are out of gsoc scope imho.
User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
 
Posts: 2653
Kudos: 4
Joined: 24 Sep 2003
Location: Haute Garonne, France

Postby westine » Fri Jun 06, 2008 2:43 pm

I use CeguiMeshviewer to view animation with the new added skeleton,
all the original animations are well played, but the rush ( which is new added) is not listed in the animation list.

And I have discovered the difference in character skeleton between bvh and OgreSkeleton,
as bvh is only skeletal animation, so the orientation of each joint is for the direction of its child bone's offset, and there is endeffector which only has offset but not orientation as on more child bone needs to extend from.
When it comes to the Ogre Skeleton, it is for mesh animation, so the position and orientation of each bone is for its own Entity node's local use.

All these concludes that more bone fitting work need to be done, and this probably fall in the 3rd case, but could be more simpler.
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

Postby westine » Wed Jun 18, 2008 3:23 pm

I have been trying to bind a exact matching skeleton this week using Mayaã€
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

Postby sinbad » Thu Jun 19, 2008 8:35 pm

Does it have to be a specific model, or can you use something like the Jaiqua model?
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
 
Posts: 19261
Kudos: 69
Joined: 06 Oct 2002
Location: Guernsey, Channel Islands

Postby braindx » Fri Jun 20, 2008 5:03 am

I wrote a tool for some animation research I was doing with Ogre this semester that might help if I understand your problem correctly...

I use Milkshape 3D to deal with importing and exporting animation and I've found the Valve SMD format easiest to work with. I wrote a tool that given a skeleton and animation in SMD format will export a new animation fit to the skeleton.

You provide a file that maps bones in the animation to bones in the skeleton. The requirement is that the bone structure in the animation must be be as or more complex than the bone structure in the mesh (i.e. every bone in the mesh must be matched to a bone in the animation).

The tool goes through the input animation and transforms the rotation of the bones to world space. It then uses the mapping file to remove unmapped bones, renames the mapped bones to the correct names for the mesh, transforms the rotations back to local space and outputs a new SMD file.

If this is useful, I can try to clean up the code a little bit and send it on over to you.

Additionally, a lot of professional mocap studios use FBX as their format of choice for saving mocap. They will not load properly with the importer provided with Milkshape but I've written one that is compatible if you need. If all your mocap data is in BVH format, Milkshape can convert to/from SMD easily.
braindx
Gnoblar
 
Posts: 24
Kudos: 0
Joined: 19 Aug 2007

Postby tuan kuranes » Fri Jun 20, 2008 9:41 am

every bone in the mesh must be matched to a bone in the animation

I think westine needs exactly that.

Westine, can you give either a sample bvh link or the bone hierarchy you needs?

Meanwhile, perhaps you should stick to a very simple animated mesh/skeleton, like this.
User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
 
Posts: 2653
Kudos: 4
Joined: 24 Sep 2003
Location: Haute Garonne, France

Postby westine » Sat Jun 21, 2008 3:50 am

sinbad wrote:Does it have to be a specific model, or can you use something like the Jaiqua model?

It should be an exactly matching model, because the skeleton in my bvh is not matching the skeleton in any model in the current Ogre.
I tried to match bvh skeleton to Jaiqua and ninja as possible as I can, but because the binding pose is not the same, then the bone weight assignments are different, resulting in angly animations.
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

Postby westine » Sun Jun 22, 2008 4:07 am

I wrote a tool that given a skeleton and animation in SMD format will export a new animation fit to the skeleton.
Does this animation use character model, I means skinning?

You provide a file that maps bones in the animation to bones in the skeleton. The requirement is that the bone structure in the animation must be be as or more complex than the bone structure in the mesh (i.e. every bone in the mesh must be matched to a bone in the animation).
Ogre dose not require every bone in a mesh must be matched to a bone with animation track. I have tried to just bind a small part of bones to animation track, this really works. But my trouble is the Binding Pose.

Additionally, a lot of professional mocap studios use FBX as their format of choice for saving mocap. They will not load properly with the importer provided with Milkshape but I've written one that is compatible if you need. If all your mocap data is in BVH format, Milkshape can convert to/from SMD easily.
All my mocap files are originally asf/amc downloaded from CMU Mocap Lab http://mocap.cs.cmu.edu, and I converted them to bvh for more compatibler use.

Your work is really valuable, now there is no automatically tool for skeleton and mesh binding, we can collaborate and contribute it to the community. Thanks so much.
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

Animation added with braindx' help

Postby westine » Sun Jun 22, 2008 8:54 am

Image pushup

Image
jump

I use simple but probably representable stick man mesh to play motions.
Thanks for braindx' timely and quick help. :wink:
An associate video
http://www.youtube.com/watch?v=fEu1n9Xa1g4
Last edited by westine on Tue Jul 15, 2008 2:11 pm, edited 2 times in total.
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

Postby braindx » Sun Jun 22, 2008 6:57 pm

Does this animation use character model, I means skinning?

I'm not sure what you mean. The SMD format can be used to hold mesh data, but it can also be used to hold only animation data, like a bvh file.

Ogre dose not require every bone in a mesh must be matched to a bone with animation track. I have tried to just bind a small part of bones to animation track, this really works. But my trouble is the Binding Pose.

The binding pose is only an issue because the animation is based off of local rotations starting from the binding pose. The original binding pose doesn't matter because my tool moves everything to world space rotations first. Only the binding pose of the new skeleton that you pass in to it is used when saving the converted animations.

All my mocap files are originally asf/amc downloaded from CMU Mocap Lab http://mocap.cs.cmu.edu, and I converted them to bvh for more compatibler use.

These bvh files can be converted to SMD format using Milkshape 3D.
braindx
Gnoblar
 
Posts: 24
Kudos: 0
Joined: 19 Aug 2007

mesh deormation based on bvh file

Postby Hel » Thu Jun 26, 2008 8:43 am

Hey guys

I'm quiet new with animation but have to do some face animation for my project.
The problem is I have captured a face using infrared capturing device with about 40 markers on the face(.bvh file) , now I want to map the markers to a face mesh ( generated by FaceGen ) . actually, the goal is to animate the face mesh by the captured data.

I tried to use Maya, Poser and some other softwares bit all seem very confusing, has anybody any idea how to do this mapping of the markers and doing the animation ? HELP I'M CLULESS


HH :(
Hel
Gnoblar
 
Posts: 1
Kudos: 0
Joined: 26 Jun 2008

Postby braindx » Thu Jun 26, 2008 8:53 am

This isn't really the right thread for a post like this Hel, but if you're looking for some direction I would look up tutorials on doing vertex animations using Maya. Typically, facial animation is done through vertex morphs rather than skeletal animation. You're going to need to map the markers in the mocap file to vertices in the face mesh. The easiest way to do that would probably be to write a Maya plugin or something like that. I don't know if you're going to find any easily accessible software to do what you want.
braindx
Gnoblar
 
Posts: 24
Kudos: 0
Joined: 19 Aug 2007

Re: mesh deormation based on bvh file

Postby westine » Thu Jun 26, 2008 1:53 pm

Hel wrote:Hey guys

I'm quiet new with animation but have to do some face animation for my project.
The problem is I have captured a face using infrared capturing device with about 40 markers on the face(.bvh file) , now I want to map the markers to a face mesh ( generated by FaceGen ) . actually, the goal is to animate the face mesh by the captured data.

I tried to use Maya, Poser and some other softwares bit all seem very confusing, has anybody any idea how to do this mapping of the markers and doing the animation ? HELP I'M CLULESS


HH :(

I have been struggling with binding bvh's control joints onto meshs for 2 weeks, although I haven't made any face animation, I should say:
1ã€
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

Postby braindx » Wed Jul 02, 2008 10:08 pm

Hey Westine, just out of curiosity, what algorithm do you use to compare two frames or clips of animation when deciding whether or not you can build a transition between them in the graph?
braindx
Gnoblar
 
Posts: 24
Kudos: 0
Joined: 19 Aug 2007

Postby westine » Sat Jul 05, 2008 5:04 am

braindx wrote:Hey Westine, just out of curiosity, what algorithm do you use to compare two frames or clips of animation when deciding whether or not you can build a transition between them in the graph?

I use
Image
to evaluate the distance between frames.
In a small motion graph, all pairwise distances are calculated in a preprocessing step, then every frame has a number of successors within a low distance metric.
Reference to Charpter 4 of http://pages.cs.wisc.edu/~kovar/thesis/index.html for more detail, it's intuitive and easy to code.

Early demo demonstrates this method is nice in small diameter motion graph.
There are two main difficulties:
1ã€
Last edited by westine on Tue Jul 15, 2008 1:46 pm, edited 1 time in total.
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

Postby braindx » Tue Jul 08, 2008 8:13 am

Makes sense. I think the ability to restrict the motion set used when compiling the graph is definitely a good idea, maybe just the ability to "tag" animations with categories and limit what types of animations can transition to what other types of animations. If the distance threshold is small enough and the frame window is large enough, linear interpolation should be fine for motion blending. I'm not sure what you mean about IK though...

Also, just to clarify, this motion graph will be computed offline and then loaded at run-time, having only traversals calculated in real-time, right?
braindx
Gnoblar
 
Posts: 24
Kudos: 0
Joined: 19 Aug 2007

Postby Assaf Raman » Tue Jul 15, 2008 12:57 pm

@westine: I recommend using http://pages.google.com as a site to upload images to - images you want use in the forum in posts..
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
 
Posts: 3092
Kudos: 79
Joined: 11 Apr 2006
Location: TLV, Israel

Postby westine » Tue Jul 15, 2008 2:18 pm

Assaf Raman wrote:@westine: I recommend using http://pages.google.com as a site to upload images to - images you want use in the forum in posts..

Thanks for your recommendation, I have updated all images to google page.
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

An interactive demo released

Postby westine » Sat Aug 16, 2008 7:52 pm

After fundamental functions of motion graph have been implemented, I made three demos on it.

They are:
1ã€
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

How are scripts written?

Postby westine » Sun Aug 17, 2008 11:08 am

Animation scripts are like :
Code: Select all
States: 7
0,jump
1,walk
2,rush
3,wonder
4,wheel
5,round
6,idle
Transitions: 8
0,'AnimationEnd',walk,1
1,'AnimationEnd',rush,2
2,'AnimationEnd',idle,6
6,'AnimationEnd',wheel,4
4,'AnimationEnd',wonder,3
3,'AnimationEnd',round,5
5,'AnimationEnd',round,5,
5,'Timesup=5',jump,0


Althoung this one doesn't cover the whole semantic, the snippet shows a static animation script.

"Static" means once you design the script, you can not intervene it except "quit".

The State and Transition number is now required, it just gives a help to count numbers in my code, but I think forcing users count number is not friendly. Why I leave this not improved is that motion script will be serialized in binary in OgreResourceManager in the next development step, it will be improved together.

The semantic of motion graph script is very simple,
there are a set of quadruples , which in state machine graph will be more
easy to understand and editor, it is the reason why GUI of script is indeed needed.
Quadruple =
{ SourceState , TriggerType , Action , DestState }


SourceState , DestState: stateNum
TriggerType: the triggerType this SourceState feeds
Action: more param with the trigger

TriggerType now supported:
Code: Select all
   enum TriggerType
      {
         NON_TRIGGER,
         ANIMATION_END,
         DIRECTION_CONTROL,
         SPEED_CHANGE,
         ACTION_IDLE,
         TIMESUP
      };

this is now coded in OgreMotionGraph.h
To extend it more flexibly, editor and external script language such as Python is a suggestion.
Last edited by westine on Sun Aug 17, 2008 4:08 pm, edited 1 time in total.
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

What does motion graph do with these triggers?

Postby westine » Sun Aug 17, 2008 11:22 am

The TriggerType is not only for scripted animation, they include both scripted and interactive Triggers.

First thing to be cleared is what " Trigger " is.

Trigger is like an interrupts in operating system.

Normally Motion graph is continuously taking time step and advancing through scripted animation, and in every time tick it peeks whether there are triggers.
If triggers exsit, process them, this is done in MotionGraph::ProcessTrigger() in
OgreMotionGraph.h
OgreMotionGraph.cpp.
Last edited by westine on Mon Aug 18, 2008 4:53 am, edited 1 time in total.
westine
Google Summer of Code Student
Google Summer of Code Student
 
Posts: 62
Kudos: 0
Joined: 01 Apr 2007

PreviousNext

Return to Google Summer Of Code

Who is online

Users browsing this forum: No registered users and 4 guests