Low framerate, am I doing something wrong?

In my scene I start with one large box for the floor, an ellipsoid for my character, and a small sphere in the middle. This gets me about 100 fps and everything feels perfect.

Once I start dropping boxes though things get pretty bad. After dropping 10 small (3x3x3 size, 10 mass) boxes, the fps drops to around 60. Once I get 50 boxes on the ground the fps drops to around 30, and the simulation gets very slow. None of the boxes are touching each other, only the large box for the floor.

I'm using the basic frame listener and setting the desired framerate to 120. I've also verified the boxes freeze when they stop moving.

I'm not sure what I could be doing wrong but I don't think I should be dropping in framerate so incredibly fast.

My system is also relatively new so that shouldn't be a problem.

Here is a screenshot:http://img507.imageshack.us/my.php?image=testpk5.jpg

Any ideas?

edit: I did another test - I created a stack of 20 boxes and my fps dropped to about 17.

Ok I figured it out. I was running in debug and switching to release made a world of difference.