Python: pickle module

简介: The pickle module implements binary protocols for serializing and de-serializing a Python object structure. Pickling is the process whereby a Python object hierarchy is converted into a byte stream, and unpickling is the inverse operation, whereby a byte stream (from a binary file or bytes-like object) is converted back into an object hierarchy.

Module Interface

结合 5.21 序列化Python对象 使用下面接口:

• pickle.dump(obj, file, protocol=None, *, fix_imports=True)
  Write a pickled representation of obj to the open file object file.

• pickle.dumps(obj, protocol=None, *, fix_imports=True)
  Return the pickled representation of the object as a bytes object, instead of writing it to a file.

• pickle.load(file, *, fix_imports=True, encoding="ASCII", errors="strict")
  Read a pickled object representation from the open file object file and return the reconstituted object hierarchy specified therein.

• pickle.loads(bytes_object, *, fix_imports=True, encoding="ASCII", errors="strict")
  Read a pickled object hierarchy from a bytes object and return the reconstituted object hierarchy specified therein.

Pickling Class Instances

In most cases, no additional code is needed to make instances picklable. By default, pickle will retrieve the class and the attributes of an instance via introspection. When a class instance is unpickled, its __init__() method is usually not invoked. The default behaviour first creates an uninitialized instance and then restores the saved attributes. The following code shows an implementation of this behaviour:

def save(obj):
    return (obj.__class__, obj.__dict__)

def load(cls, attributes):
    obj = cls.__new__(cls)
    obj.__dict__.update(attributes)
    return obj

如果需要修改上面默认pikle行为，可以使用下面的魔法方法:

• object.__getstate__()
  Classes can further influence how their instances are pickled; if the class defines the method __getstate__(), it is called and the returned object is pickled as the contents for the instance, instead of the contents of the instance’s dictionary. If the __getstate__() method is absent, the instance’s __dict__ is pickled as usual.

• object.__setstate__(state)
  Upon unpickling, if the class defines __setstate__(), it is called with the unpickled state. In that case, there is no requirement for the state object to be a dictionary. Otherwise, the pickled state must be a dictionary and its items are assigned to the new instance’s dictionary.

举几个简单的栗子：

• 1, 来自 stackoverflow 的Simple example of use of setstate and getstate

class Foo(object):
  def __init__(self, val=2):
     self.val = val
  def __getstate__(self):
     print("I'm being pickled")
     self.val *= 2
     return self.__dict__
  def __setstate__(self, old_dict):
     print("I'm being unpickled with these values: ", old_dict)
     self.__dict__ = old_dict

import pickle
f = Foo()
f_string = pickle.dumps(f)
f_new = pickle.loads(f_string)

输出为:

I'm being pickled
I'm being unpickled with these values:  {'val': 4}

• 2, 这个栗子来自官方文档

class TextReader:
    """Print and number lines in a text file."""

    def __init__(self, filename):
        self.filename = filename
        self.file = open(filename)
        self.lineno = 0

    def readline(self):
        self.lineno += 1
        line = self.file.readline()
        if not line:
            return None
        if line.endswith('\n'):
            line = line[:-1]
        return "%i: %s" % (self.lineno, line)

    def __getstate__(self):
        # Copy the object's state from self.__dict__ which contains
        # all our instance attributes. Always use the dict.copy()
        # method to avoid modifying the original state.
        state = self.__dict__.copy()
        # Remove the unpicklable entries.
        del state['file']
        return state

    def __setstate__(self, state):
        # Restore instance attributes (i.e., filename and lineno).
        self.__dict__.update(state)
        # Restore the previously opened file's state. To do so, we need to
        # reopen it and read from it until the line count is restored.
        file = open(self.filename)
        for _ in range(self.lineno):
            file.readline()
        # Finally, save the file.
        self.file = file

在一个 'hello.txt'中输入

1: Hello world!
2: I am line number two.
3: Goodbye!

进行测试：

>>> reader = TextReader("hello.txt")
>>> reader.readline()
'1: Hello world!'
>>> reader.readline()
'2: I am line number two.'
>>> new_reader = pickle.loads(pickle.dumps(reader))
>>> new_reader.readline()
'3: Goodbye!'

参考:

• Python documentation: pickle

read more

• PyMOTW-3: pickle — Object Serialization