Video capture

feanor91

23-03-2009 09:34:46

Hi

I reach a new level by the diffusion on my soft on the net. By now, I want to add some cool functionnality as real time video capture for example. For the moment, I use FRAPS, but it will be nice if I can do it by myself in my code. So if someone have an idea of how to achieve that, perhaps with a free library, or in any otherway, I will be glad.

feanor91

26-03-2009 09:49:36

Hello

Anybody understand my question, or nobody had a clue?

Nevermind, I found this : http://www.codeproject.com/KB/directx/SimulationRecording.aspx?display=Print perhaps, it's a solution, but how to get diretX info from Mogre?

Beauty

22-04-2009 03:32:30

Ogre itself can take screenshots, but this is very very slow on my computer.
renderWindow.WriteContentsToTimestampedFile(...);

One year ago I wrote a capture class using gdi32.dll. This is much more faster. Basically it works, but needs improvements. Also it's embedded to my main application and not easy to cut it out.

I extracted the basic source code which creates screenshots. I'm not shure if this plain code works.
But maybe it can give ideas for a way how to do.

With this you can capture the content of a control (like a window).
But it needs to be optimized for rows of many screenshots.
Also you have to create a video of it. On the fly (needs cpu power) or later (needs a high write rate to hard disk for images).

[System.Runtime.InteropServices.DllImport("gdi32.dll")]
public static extern long BitBlt(IntPtr hdcDest, int xDest, int yDest, int width, int height, IntPtr hdcSrc, int xSrc, int ySrc, int dwRop);


public System.Drawing.Image CaptureCtrl(Control ctrl)
{
g = this.CreateGraphics();
// Create a new image with the size of the control
System.Drawing.Image newImage = new Bitmap(ctrl.Size.Width, ctrl.Size.Height, g);
Graphics memoryGraphics = Graphics.FromImage(newImage);

// get handle
IntPtr src = currentGraphics.GetHdc();
IntPtr dest = memoryGraphics.GetHdc();

// more information here:
// http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wceui40/html/_celrfternaryrasteroperations.asp
BitBlt(dest, 0, 0, ctrl.ClientRectangle.Width, ctrl.ClientRectangle.Height, src, ctrl.Location.X, ctrl.Location.Y, 13369376);

// release handles
currentGraphics.ReleaseHdc(dest);
memoryGraphics.ReleaseHdc(src);

g.Dispose();
memoryGraphics.Dispose();

return newImage;
} // CaptureCtrl()



An alternative would be something like a codec with workaround for good use.
Interesting could be the MJPEG format (maybe in combination with the code above). It saves jpeg pictures into a stream of one file. But it seems so that this format is not very common and not so many tools are available. The files size is not so nice, but this could be converted to an other format later.
http://en.wikipedia.org/wiki/Motion_JPEG

Also this links can be interesting:
Huffyuv is a very fast, lossless Win32 video codec.
http://neuron2.net/www.math.berkeley.ed ... ffyuv.html

TAKSI: Video capture/Screen capture for 3D graphics
http://taksi.sourceforge.net
With this you can use installed video codecs, but I didn't know which one would be good for high speed. (Maybe Huffyuv?)

Well, more I can't tell you.

feanor91

22-04-2009 05:51:19

Hi

Thanks for answer.

I'm still searching in background.

Do you see the links I put on the post? The only things to know is : how to get the directX info needed from Ogre.

Besides, of coure, I need to get sounds with video, in fact, I need to do FRAPS but in my own code.

So, I will see if I can use your code, and will tell you.

Bekas

22-04-2009 17:21:08

The only things to know is : how to get the directX info needed from Ogre.
See here: http://www.ogre3d.org/addonforums/viewtopic.php?f=8&t=9841&start=0

feanor91

11-05-2009 13:11:47

Hi

I'm jus begining to work on video capture, and I try to adapt your code to my own, but I have a question :

what is currentGraphics, you have no declaration of it. I think it's something linked to the current image, but I can manage to understand how you made it.

Can you help me?

Just for remember, I work under VB.net.

Edit :

Forget it, I replace 'currentgraphics' by 'g' and it works. Besides this, it's slow, needs a lot of optimisation and for now I just write image not creating a movie, but it's only the begining, I continue to dig.

feanor91

14-05-2009 13:16:33

IT WORKS!

I've managed to find a library on the net here : http://www.codeproject.com/KB/audio-video/avifilewrapper.aspx, compiled in C#, I get a dll ans with it, I can create avi file very simply. So I use your proc and the image I get, I passe it to avifile to write avi and it works fine.

Meanwhile I have a problem on capturing the control. I have a part of my interface that appears in the captured bitmap about 24 pixels in the bottom. In fact, it's like my interface was overlaying my control, I'don't understand why, because the height captured is good...Stil looking around.

feanor91

15-05-2009 12:03:37

Ok, I fix the 24 pixel shift. It is about this :

ctrl.Location.Y

it return 24 and is right because my control is at 0,24 but I must put 0 in this value to capture all my contol so now all works fine. Great!

AndroidAdam

15-05-2009 15:11:22

Thanks for this, it's very useful for showing off your work :D

feanor91

15-05-2009 20:59:48

Your welcome.

I've modified my class to introduce the video capture system, I will agrement wiki soon. Besides I use the code here to write avi file :

http://www.codeproject.com/KB/audio-video/avifilewrapper.aspx

It compiles well in C# into a dll the only thing I change is about saving codec info in the registry because each time you launch a record it will ask fot codec.

Meanwhile it crops a lot of FPS, I will see to thrad the avi cration, my attempts were not succesfull at this time, I don't know why. When I start the thread, ogre seems to crash....Still oking at this.

feanor91

03-06-2009 10:12:39

Hi

I'm back.....with problem

The video capture system work fine. I'll code it in framended function but, it's very very very time consuming (about 50% FPS....gulp!). Whatever, I wil want to use multithreading system to create movie in a separate thread but if I do that, the image capture works, but the images captured are all black....They will be 2 weeks I look for and found nothing. Any idea? I use .net background worker.

Beauty

03-06-2009 21:25:16

Thanks for your many posts to talk us your state.
Sorry for no answering - I'm busy with my university and other things.
But nice to see that my basic work was helping you (-:

Multicore using is nice ... if it works.
Maybe split the tasks like this:

* main thread captures the pics
* store the pics in a list of thread 1 (comfortable are generic lists like System.Collections.Generic.List<String>)
* the second thread get access to the list of thread 1 and "move" the pictures (delete from T1, add to a List in T2)
Maybe it's quicker to clone the whole list (from T1 to T2) and then clear that one in T1 (instead of read/create/delete the singe elements)

* important: set a lock while moving data between the threads and use the lock only for the needed lines of code

* then the images can be processed by avi compressing in the second thread without disturbing the first one


In my tests I got a problem. The capturing I put to the second thread. I thought it would be the catched pictures, but somehow it was just a reference to the area which will be recorded. So the result was wrong (wanted saving time != picture). This problem I could fixed and multicore using worked.
I did not understand all the image source code. It was just a hack of a C# cook book for saving a singe screenshot. I could improve it by re-using some objects (image variables) instead of recreating everything for every pic.

Generally with capturing the rendering is more slow, because the data transfer structure is optimized for computer --> graphic card. The other way it's more slow. (this I read) And/or the bi-directional traffic has to share the common bandwith. (I suppose)
So I would say the highly optimized FRAPS capturing software is a good comparison what is a good capturing speed.

If the bandwidth to the hard disk is to big, then it slows down the capturing speed.
If the codec needs to much time for compressing then it needs to much CPU power, which slows down, too.
Fast codecs maybe have bad image quality.
So you need compromize.
But I don't know, which codec is good for such a task.
Maybe you do it in 2 steps:
For realtime capturing a codec which is quick, has acceptable quality level and needs medium size on the hard disk.
The second step is a re-encoding to a high compressable format, which will be started manually somewhen later by the user (when no more captures are needed.)

Ok, more I can't help you in the moment. Just some thoughts.

feanor91

04-06-2009 11:43:00

Hi

Tanks for your answer, I will look in that way

feanor91

04-06-2009 15:28:51

So...I did it.

I try to use list in frameended then creating movie in backgroundworker.

The good news : it works.

The bad news : it's more slowly than creating movie in the frameended function....!!??

The only things I see is that the treratment of the list is to slow to be use that way.

Another thing : My movie is parametered to be done in 24 images/s but the render during capture is about 13. When I look at the movie, it's faster than the render captured so it is like accelerated. Do you think it is because 24 vs 13?

I continue to sneak around....

Beauty

04-06-2009 20:19:23

Maybe the 24fps video speed is a preset and some frames are redundant (or maybe interpolated).

For the bad speed - comment out the encoding and keep the rest.
Then you will see if the speed loose is because of the exchange list.

If the exchange list is the problem, then I suppose the reason is the permanently recreation of new objects for the images.
Maybe there is a way to use native pointers.

The pointer way:
* put the grabbed picture is in an image object
* create a pointer to this object (how??)
* use the exchange list only for transfaring the pointers
* delete the pointer in first thread
* then the second thread has access to the image object and no recreation of an image object is needed
* delete the pointer in second thread

Somehow like this you can try to do. With C++ this is easy (although I don't know very much about C++).
With C# it's possible. Also there are managed and unmanaged pointers.
(I thing this is only possible with unsafe compilation mode.)

The Mogre wrapper uses such things internally, but more I can't tell you about.
Maybe somewhere else would improve the code if you publish the source code or a simple test application.

feanor91

05-06-2009 09:57:57

Hi

I think a lot and the double speed of the video is because the ogre render is about half of 24 images/s.

In fact, the video codec try to put 24 images in 1 seconde but it have only about 12-13 images that arrives for the same time so it must double the images to get the 24 images and so, it double the speed. So, I have to allow to ajust the capture rate of the codec. It's easy, it's one of the parameter of the avimanager create function.

For the code now. Here the capture ine FrameEnded function :


Private Function FrameEnded(ByVal evt As FrameEvent) As Boolean

If bRecordVideo Then

Dim g As Graphics = _CtrlToCapture.CreateGraphics()
Dim MyImage As System.Drawing.Image = New Bitmap(_CtrlToCapture.Size.Width, _CtrlToCapture.Size.Height, g)
Dim memoryGraphics As Graphics = Graphics.FromImage(MyImage)
'get handle
Dim src As IntPtr = g.GetHdc()
Dim dest As IntPtr = memoryGraphics.GetHdc()

' more information here:
' http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wceui40/html/_celrfternaryrasteroperations.asp
'BitBlt(dest, 0, 0, _CtrlToCapture.ClientRectangle.Width, _CtrlToCapture.ClientRectangle.Height, src, _CtrlToCapture.Location.X, _CtrlToCapture.Location.Y, 13369376)
BitBlt(dest, 0, 0, _CtrlToCapture.Size.Width, _CtrlToCapture.Size.Height, src, 0, 0, 13369376)


imgListe.Add(MyImage)

' release handles
g.ReleaseHdc(dest)
memoryGraphics.ReleaseHdc(src)

g.Dispose()
memoryGraphics.Dispose()

End If

Return True
End Function


The Onwork function of the background worker :

Private Sub OnWork(ByVal sender As Object, ByVal e As DoWorkEventArgs)

Dim image As Bitmap = Nothing

While True
If worker.CancellationPending Then
Exit While
End If
If imgListe.Count > 0 Then
image = imgListe(0)
FilmStream.AddFrame(image)
imgListe.RemoveAt(0) [color=#FF0000]<---sometime it crashes here[/color]....
End If
End While
End Sub


To work with pointer in VB, I think it will be hard. Meanwhile, I will try something around.

I will try to see about speed by commenting out the film creation as you say.

And last thing : I try to externalyse the variables of the capture system by making them private in the class, but, it appears that nothing is captured, I get black images....I will look forward if I Have made some errors.
I'll be back later to talk about.

feanor91

05-06-2009 12:06:44

Well, after my search of the holly grail, here what I found.

1-g and memorygraphics variables need to bee released in order to have something in the image, otherwise the image is black so, it is needed to recreating them on each capture. It is for this that we can declare globale this wariables.

2-I have better result in FPS term by doing all in frameended function than capture in it then creating film in a backgroundworker. I thinck that getting data from the main thread in the secondary is more time consuming than doing all in the same thread.

3-the speed of movie, as says before, is directly dependant of the rate of capture. If I reduce the size of my control to get at least 24 images/s during capture, the speed of the movie is good.

and finaly :

4-A fast computer is required to have a fast video capture system. (but how Fraps works???)

I will soon repost my class code on wiki so, everybody will be able to look deeper in my code and so, perhaps somenone will have idea that I didn't.

tdev

29-03-2010 23:06:04

this could be a very interesting solution: http://www.outerra.com/video/index.html

Beauty

02-04-2010 21:20:23

This sounds great!
If somebody made some tries with Mogre, please give us a feedback and useful code snippets :wink:

feanor91

16-04-2010 08:23:34

Hello

A long time since my last post....For sure , I've ended development of my project.

So. If you vant to see what I have done and the result of the capture video system, you can go there : http://www.cramayailes-indoor.org/wiki. Click on '"Référentiel..." link and click on one of the first link of the bnew page. Go to the bottom of the page and look at Kitesimulator part, here you have a video player that show the kite figure, recorded with my soft.

Beauty

15-02-2011 16:49:26

The videos of your kite simulation application are looking nice.
Well done project. :D

After many months now I got again the whish to add a video recording feature to my Mogre application.
The GPU compression project of the outerra website sounds great.

Unfortunately I don't know how to use it.
There is a workaround needed like how to apply the shader code, make settings (e.g. grabbing rectangle), read the created video stream (which has to be written to a file, including a special header).

Did you use this GPU encoding for video recording?
If yes I would be happy if you publish some code snippets (or a tiny demo application) where I can see how you handle with the GPU compressor code.

feanor91

31-03-2011 14:24:54

Hi,

Sorry to didn't see your post. But it's a long time since I end up my project and I came here not so often. I don't remember yet what I use. I grab something on the net that works it used avifile.dll, I didn't remember the site where I found informations but here my code :

Private Sub btnToolRecord_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnToolRecord.Click
FrmSimulation3D.Record = Not FrmSimulation3D.Record
If FrmSimulation3D.Record Then
bEnreg = True
FrmSimulation3D.lblEnregistrement.Visible = True
btnToolPlay_Click(sender, e)
If FrmSimulation3D.mnuDemarrerEnPause.Checked Then
If SimuEnPause Then
For i As Integer = 0 To IIf(mdiMain.nbPistes = 0, 1, mdiMain.nbPistes + 1) - 1
FrmSimulation3D.Team(i).bAnim = True
Next
SimuEnPause = False
End If
FrmSimulation3D.lblPause.Visible = False
If TimerEnPause Then
PauseFin = FrmSimulation3D.myTimer.Milliseconds
FrmSimulation3D.TimerFPS.Enabled = True
TimerEnPause = False
FrmSimulation3D.InitChono = FrmSimulation3D.InitChono + (PauseFin - PauseDebut)
End If
End If
FrmSimulation3D.myOgre.OpenVideoStream(FrmSimulation3D.CheminFilm, mdiMain.NomFilm)
If mdiMain.Musique <> "" Then
StartRecording()
End If
btnToolStop.Enabled = False
btnToolPause.Enabled = False
Else
FrmSimulation3D.lblEnregistrement.Visible = False
FrmSimulation3D.myOgre.CloseVideoStream()
bEnreg = False
If mdiMain.Musique <> "" Then
StopRecording()
btnToolStop_Click(sender, e)
Dim aviManager As AviFile.AviManager = New AviFile.AviManager(FrmSimulation3D.CheminFilm + "\" + mdiMain.NomFilm + "_" + FrmSimulation3D.myOgre.RecordDate + ".avi", True)
aviManager.AddAudioStream(FrmSimulation3D.CheminFilm + "\Film.wav", 0)
aviManager.Close()
FileIO.FileSystem.DeleteFile(FrmSimulation3D.CheminFilm + "\Film.wav")
End If
btnToolStop.Enabled = True
btnToolPause.Enabled = True
End If
'If chkChoixCodec.Checked Then
' chkChoixCodec.Checked = False
'End If
End Sub


Meanwhile, there is a great lose of FPS while recording I have to balance by accelerating the rendering of the scene.

As I do something else, for now, I think I will not be good to helping you on that subject.

Meanwhile, I put this subject on notification, in case....

Edit Be happy : finally I find the site...it's here : http://www.codeproject.com/KB/audio-video/avifilewrapper.aspx

Beauty

31-03-2011 15:11:04

Thanks for your quick and helpful reply :D

Which video codec did you use?
Good could be a fast codec with low compression.
(The video file can be converted to a high compression video format later.)

Did you try to implement the GPU compression method of the outerra website project?


Here is the code, converted to C# (if somebody else need it):

private void btnToolRecord_Click(System.Object sender, System.EventArgs e)
{
FrmSimulation3D.Record = !FrmSimulation3D.Record;
if (FrmSimulation3D.Record) {
bEnreg = true;
FrmSimulation3D.lblEnregistrement.Visible = true;
btnToolPlay_Click(sender, e);
if (FrmSimulation3D.mnuDemarrerEnPause.Checked) {
if (SimuEnPause) {
for (int i = 0; i <= (mdiMain.nbPistes == 0 ? 1 : mdiMain.nbPistes + 1) - 1; i++) {
FrmSimulation3D.Team(i).bAnim = true;
}
SimuEnPause = false;
}
FrmSimulation3D.lblPause.Visible = false;
if (TimerEnPause) {
PauseFin = FrmSimulation3D.myTimer.Milliseconds;
FrmSimulation3D.TimerFPS.Enabled = true;
TimerEnPause = false;
FrmSimulation3D.InitChono = FrmSimulation3D.InitChono + (PauseFin - PauseDebut);
}
}
FrmSimulation3D.myOgre.OpenVideoStream(FrmSimulation3D.CheminFilm, mdiMain.NomFilm);
if (!string.IsNullOrEmpty(mdiMain.Musique)) {
StartRecording();
}
btnToolStop.Enabled = false;
btnToolPause.Enabled = false;
} else {
FrmSimulation3D.lblEnregistrement.Visible = false;
FrmSimulation3D.myOgre.CloseVideoStream();
bEnreg = false;
if (!string.IsNullOrEmpty(mdiMain.Musique)) {
StopRecording();
btnToolStop_Click(sender, e);
AviFile.AviManager aviManager = new AviFile.AviManager(FrmSimulation3D.CheminFilm + "\\" + mdiMain.NomFilm + "_" + FrmSimulation3D.myOgre.RecordDate + ".avi", true);
aviManager.AddAudioStream(FrmSimulation3D.CheminFilm + "\\Film.wav", 0);
aviManager.Close();
Microsoft.VisualBasic.FileIO.FileSystem.DeleteFile(FrmSimulation3D.CheminFilm + "\\Film.wav");
}
btnToolStop.Enabled = true;
btnToolPause.Enabled = true;
}
// if (chkChoixCodec.Checked) {
// chkChoixCodec.Checked = false;
// }
}

feanor91

31-03-2011 15:29:54

Which video codec did you use?

default that work in,window

Which video codec did you use?

Yes but try a lot an only few are working good


Did you try to implement the GPU compression method of the outerra website project?


Not know when I write code


Here is the code, converted to C# (if somebody else need it):


Be carrefull, only the part with avimanager and videostream are usefull, the rest is about internal works of my project. But you can learn more by lookin at url I have given

CodeKrash

01-04-2011 02:05:17

I have a d3d9.dll proxy built with msvc++

With it you can basically overlay objects, and save the screen buffer, and even specific objects in the scene (theory)

Beauty

01-04-2011 10:23:28

Did you use it for screen recording?
If yes, can you post an example code, please?

CodeKrash

02-04-2011 00:38:39

it has a direct view into the directx calls so anything is possible. I imagine copying the screen in this context would be way faster. I was going to use it to overlay ventrilo speakers, but never released the solution. The project has some stuff about reading a text file and overlaying its contents, which should be cleaned out before adapting it for your needs. After compilation, place d3d9.dll in the application folder along side your main exe, and during runtime it will "replace" the one in system32 (or wherever it's supposed to be) I tested it with WoW and it works well. It probably works for all DX9 solutions. I also have a (i think) working DX8 version too if anyone wants.

Beauty

17-04-2011 18:29:22

CodeKrash, I tried to compile your project, but the include file d3dx9core.h (and maybe also d3dx9core.cpp) are missing.
Can you give me the files, please?

CodeKrash

25-04-2011 03:52:15

those are part of the dx9 sdk i believe