Pickle support?

Entropai

05-02-2008 14:19:16

Hi,

I'm trying to use pickle with some of the python-OGRE's types (Vectors, Quaternions etc.) but whenever i'm trying to turn one in to string representation (dumps) intrepreter gives me error that pickling of these instances is not supported. I'm new to pickling (and in general to Python) and just wondering if this is because it would require some extra job to enable pickling with python-OGRE?

bharling

05-02-2008 16:51:57

Nah, it doesn't work. You probably dont want to pickle a whole Vector3 object anyway, as it will contain not only the xyz data, but also all the functions, which you dont really need to save ( or transfer over a network ).

I'd make up my own simple class to hold the xyz values, and pickle that. That should work. Here's an example that takes an ogre.Vector3 as input:

class v3test:
def __init__(self, vector3):
self.x = vector3.x
self.y = vector3.y
self.z = vector3.z

v = v3test( ogre.Vector3.ZERO )

import pickle

pickle.dumps(v)

>>"(i__main__\nv3test\np0\n(dp1\nS'y'\np2\nF0.0\nsS'x'\np3\nF0.0\nsS'z'\np4\nF0.0\nsb."



Then just re-create the true Vector3 object when the pickled data is loaded or recieved.
although, even that is more data than you'd probably need. The best thing would be just to pickle a tuple of the x y z values, that is IMHO the more efficient way to do it. Compressing data like this is an art form in itself ;)

Entropai

05-02-2008 19:30:22

Thanks for the answer. I wasnt aware that pickling would also pack up the functions too :oops:, so yeah its waste. I'll look in to the tuple solution later on, the class wrapping should suffice for now.

Game_Ender

06-02-2008 01:28:07

bharling, that is not true at all. You have complete freedom with the pickling system on what is saved and how data is loaded. Please go here to learn more.

andy

06-02-2008 02:26:59

@Game_Ender: The pickling you are refering to is the Boost one, however I think that Barling was refering to the one built into Python itself which I think has additional limitations??

Andy

Game_Ender

06-02-2008 03:18:06

I am pretty sure that the Boost.Python docs just describe the normal python system listed here. Pickle uses a stack based language that creates objects, it doesn't seem to have a way to express functions that should be added to those objects.

bharling

06-02-2008 09:36:59

Well, I'm lost now. I was just going on previous experience. Although the initial statement is right, if you import standard pickle an try pickling a ogre.Vector3 object, you'll get an exception. I didn't know that boost had a differing implementation.

This is a bit confusing for my small brain. I've used pickle before to save instances of classes I've created in a previous project, and those instances contain not only the instance data, but also the functions of the class as well. I verified this by calling the methods of the un-pickled instance.

So, am I right in saying that by defining __getstate__ and __setstate__ in your class, you can customize how pickle works on it?


EDIT :---

Sorry, on further reading its a bit clearer. Pickle does not save the methods associated with a class, rather it just saves the data associated with it. When the object is unpickled, the class it represents is actually re-imported, hence the requirement to have the class defined in the same space as where you pickle/unpickle an instance of it. eg: if you try to pickle a class in one script, then load it into another python program which does not have any knowledge of the class of the pickled instance, then you don't get true access to the instance you originally pickled. Here's an illustration of what I've learned! :wink:

pickleSave.py

class pickleTest:
def myMethod(self, aVar):
self.myVar = aVar

import pickle

p = pickleTest()
p.myMethod("hello")
r = pickle.dumps(p)
f = open("test.sav", "w")
f.write(r)
f.close()
f = open("test.sav", "r")
c = pickle.loads(f.read())
c.myMethod("World")


pickleLoad.py

import pickle

f = open("test.sav", "r")
c = pickle.loads( f.read() )
f.close()
c.myMethod('World')


pickleLoad.py fails with an attributeError because it has no idea what 'c' actually is, as myMethod is not saved in test.sav, and also not defined in the namespace.

Yes I can see you all nodding and sarcastically saying 'Well done grasshopper' :lol: but I hope this is helpful to someone! Apologies to Entropai for confusion caused...

Game_Ender

06-02-2008 15:39:36

Yes, in essence pickle is like a nice dynamic version of Boost.Serialization in that it just defines a way to extract the state out of an object, and put that state back into a new copy of the object. With something like C++ and Boost.Serialization you have to define the methods manually. With pure python classes and pickle its done dynamically for you. Of course using extension modules like we do, means that we have to define the methods manually any way.

I believe we could dynamically modify the Ogre classes at runtime like so:
import ogre.renderer.OGRE as ogre

def getstate(self):
# your code here
def setstate(self, state):
# your code here

ogre.Vector3. __getstate_manages_dict__ = 1
ogre.Vector3.__getstate__ = getstate
ogre.Vector3.__setstate__ = setstate