COVER ：https://medium.com/@yzhong.cs/serialize-and-deserialize-complex-json-in-python-205ecc636caa
This post is focused on different ways to interact with JSON with Python using json:
https://docs.python.org/3/library/json.html.
We’ll look at their limitations, run time errors, and etc., with sample code. We start from basic type such as Dictionary and eventually discussed a solution for working with complex Python objects.
Dictionary
Python and the json module is working extremely well with dictionaries. The following is for serializing and deserializing a Python dictionary:
Code:
<pre name="ca79" id="ca79" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">import json</pre>
<pre name="2dd3" id="2dd3" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">student = {
"first_name": "Jake",
"last_name": "Doyle"
}
json_data = json.dumps(student, indent=2)
print(json_data)
print(json.loads(json_data))</pre>
Output:
<pre name="f88b" id="f88b" class="graf graf--pre graf-after--p graf--trailing" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">{
"first_name": "Jake",
"last_name": "Doyle"
}
{'first_name': 'Jake', 'last_name': 'Doyle'}</pre>
Object
However, if we create an object, e.g. Student (below), it doesn’t work out of the box.
Code:
<pre name="3b11" id="3b11" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">import json</pre>
<pre name="4b84" id="4b84" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">class Student(object):
def init(self, first_name: str, last_name: str):
self.first_name = first_name
self.last_name = last_name
student = Student(first_name="Jake", last_name="Doyle")
json_data = json.dumps(student)</pre>
Output:
<pre name="b2a5" id="b2a5" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">TypeError: Object of type Student is not JSON serializable</pre>
But interestingly, there is the dict on any python object, which is a dictionary used to store an object’s (writable) attributes. We can use that for working with JSON, and that works well.
Code:
<pre name="d65c" id="d65c" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">import json</pre>
<pre name="2626" id="2626" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">class Student(object):
def init(self, first_name: str, last_name: str):
self.first_name = first_name
self.last_name = last_name
student = Student(first_name="Jake", last_name="Doyle")
json_data = json.dumps(student.dict)
print(json_data)
print(Student(**json.loads(json_data)))</pre>
Output:
<pre name="53f9" id="53f9" class="graf graf--pre graf-after--p graf--trailing" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">{"first_name": "Jake", "last_name": "Doyle"}
<main.Student object at 0x105ca7278></pre>
The double asterisks ** in the Student(**json.load(json_data) line may look confusing. But all it does is expanding the dictionary. In this case, it is equivalent to:
<pre name="a480" id="a480" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">d = json.loads(json_data)
Student(first_name=d["first_name"], last_name=d["last_name"])</pre>
Complex Object
Things get tricky with complex objects. Now assume that we have a Teamobject, which contains a list of Students.
Code:
<pre name="5181" id="5181" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">from typing import List
import json</pre>
<pre name="39b6" id="39b6" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">class Student(object):
def init(self, first_name: str, last_name: str):
self.first_name = first_name
self.last_name = last_name</pre>
<pre name="1ba5" id="1ba5" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">class Team(object):
def init(self, students: List[Student]):
self.students = students
student1 = Student(first_name="Jake", last_name="Doyle")
student2 = Student(first_name="Jason", last_name="Durkin")
team = Team(students=[student1, student2])</pre>
<pre name="8047" id="8047" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">json_data = json.dumps(team.dict, indent=4)
print(json_data)</pre>
Output:
<pre name="c490" id="c490" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">TypeError: Object of type Student is not JSON serializable</pre>
Alright, it looks like **Student **is not JSON serializable and our dict trick no longer works. But if we look at the dump function’s documentation, there is a deafult setting that we can use:
If specified, default should be a function that gets called for objects that can’t otherwise be serialized. It should return a JSON encodable version of the object or raise a
[**TypeError**](https://docs.python.org/3/library/exceptions.html#TypeError "TypeError"). If not specified,[**TypeError**](https://docs.python.org/3/library/exceptions.html#TypeError "TypeError")is raised.
When serializing, we can use that to serialize the **dict **property of each object instead of the object itself. Simply by replacing this line:
<pre name="79ea" id="79ea" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">json_data = json.dumps(team.dict, indent=4)</pre>
with this,
<pre name="4efb" id="4efb" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">json_data = json.dumps(team.dict, lambda o: o.dict, indent=4)</pre>
and everything is working again for serializing. Basically, all it does is to tell dumps to dump the **dict **of every object instead of the object itself.
However, how about deserializing 🤔? Let’s try this out:
Code:
<pre name="46c0" id="46c0" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">from typing import List
import json</pre>
<pre name="1428" id="1428" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">class Student(object):
def init(self, first_name: str, last_name: str):
self.first_name = first_name
self.last_name = last_name</pre>
<pre name="4a82" id="4a82" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">
class Team(object):
def init(self, students: List[Student]):
self.students = students</pre>
<pre name="fdd2" id="fdd2" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">
student1 = Student(first_name="Jake", last_name="Doyle")
student2 = Student(first_name="Jason", last_name="Durkin")
team = Team(students=[student1, student2])
</pre>
<pre name="f4f2" id="f4f2" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">// Serialization
json_data = json.dumps(team, default=lambda o: o.dict, indent=4)
print(json_data)
</pre>
<pre name="016e" id="016e" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">// Deserialization
decoded_team = Team(**json.loads(json_data))
print(decoded_team)</pre>
Output:
<pre name="4842" id="4842" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">{
"students": [
{
"first_name": "Jake",
"last_name": "Doyle"
},
{
"first_name": "Jason",
"last_name": "Durkin"
}
]
}
<main.Team object at 0x105cd41d0></pre>
It looks like it works, but actually, not quite what we want. If we check the type of students property in the team object, we will quickly notice that they are dictionaries, instead of the defined **Student **object type above 😱.
Code:
<pre name="546e" id="546e" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">type(decoded_team.students[0])</pre>
Output:
<pre name="cfc9" id="cfc9" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">dict</pre>
It is not surprising considering that Python is not a strongly-typed programming language. Luckily, it is not hard to convert a dictionary into a python object as demonstrated above. We do this by adding a helper function from_json in each of the objects for doing any necessary type conversions.
<pre name="6bc4" id="6bc4" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">class Student(object):
# ... other code
@classmethod
def from_json(cls, json_data: dict):
return cls(**json_data)
</pre>
<pre name="6eec" id="6eec" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">class Team(object):
# ... other code</pre>
<pre name="b6b0" id="b6b0" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;"> @classmethod
def from_json(cls, json_data: dict):
students = list(map(Student.from_json, data["students"]))
return cls(students)</pre>
Complete code:
<pre name="6c20" id="6c20" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">from typing import List
import json
</pre>
<pre name="6282" id="6282" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">class Student(object):
def init(self, first_name: str, last_name: str):
self.first_name = first_name
self.last_name = last_name</pre>
<pre name="6202" id="6202" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;"> @classmethod
def from_json(cls, data):
return cls(**data)
</pre>
<pre name="dfba" id="dfba" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">class Team(object):
def init(self, students: List[Student]):
self.students = students</pre>
<pre name="8258" id="8258" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;"> @classmethod
def from_json(cls, data):
students = list(map(Student.from_json, data["students"]))
return cls(students)
</pre>
<pre name="4be2" id="4be2" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;">student1 = Student(first_name="Jake", last_name="Foo")
student2 = Student(first_name="Jason", last_name="Bar")
team = Team(students=[student1, student2])
</pre>
<pre name="a25f" id="a25f" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;"># Serializing
data = json.dumps(team, default=lambda o: o.dict, sort_keys=True, indent=4)
print(data)
</pre>
<pre name="d89a" id="d89a" class="graf graf--pre graf-after--pre" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 0px; background: rgba(0, 0, 0, 0.05); padding: 4px 20px 20px; white-space: pre-wrap;"># Deserializing
decoded_team = Team.from_json(json.loads(data))
print(decoded_team)
print(decoded_team.students)</pre>
Output:
<pre name="fe66" id="fe66" class="graf graf--pre graf-after--p" style="overflow: auto; font-family: Menlo, Monaco, "Courier New", Courier, monospace; font-size: 16px; margin: 43px 0px 0px; background: rgba(0, 0, 0, 0.05); padding: 20px; white-space: pre-wrap;">{
"students": [
{
"first_name": "Jake",
"last_name": "Foo"
},
{
"first_name": "Jason",
"last_name": "Bar"
}
]
}
<main.Team object at 0x10baaa400>
[<main.Student object at 0x10baaa2e8>, <main.Student object at 0x10baaa278>]</pre>
And that’s it. That’s how we serialize and deserialize complex JSON with python objects.
网友评论