UE4对象系统_序列化和uasset文件格式

虚幻的序列化这块是个人比较喜欢的技术点，个人在工作中也山寨了个简化版，UE4延续了UE3的序列化方案。它使用了访问者模式(Vistor Pattern)，将序列化的存档接口抽象化，其中FArchive为访问者, 其它实现了void Serialize( FArchive& Ar )接口的类为被访问者。FArchive可以是磁盘文件访问, 内存统计，对象统计等功能。

FArchive

FArchive的类继承体系如下：

Paste_Image.png

定义接口如下:

/**
 * Base class for archives that can be used for loading, saving, and garbage
 * collecting in a byte order neutral way.
 */
class CORE_API FArchive
{
public:

    /** Default constructor. */
    FArchive();

    /** Copy constructor. */
    FArchive(const FArchive&);

    /**
     * Copy assignment operator.
     *
     * @param ArchiveToCopy The archive to copy from.
     */
    FArchive& operator=(const FArchive& ArchiveToCopy);

    /** Destructor. */
    virtual ~FArchive();

public:

    virtual FArchive& operator<<(class FName& Value);
    virtual FArchive& operator<<(class FText& Value);
    virtual FArchive& operator<<(class UObject*& Value);
    virtual FArchive& operator<<(class FLazyObjectPtr& Value);
    virtual FArchive& operator<<(class FAssetPtr& Value);
    virtual FArchive& operator<<(struct FStringAssetReference& Value);
    virtual FArchive& operator<<(struct FWeakObjectPtr& Value);
    virtual void ForceBlueprintFinalization() {}
public:
    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, ANSICHAR& Value);
    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, WIDECHAR& Value);

    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, uint8& Value);
    template<class TEnum>
    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, TEnumAsByte<TEnum>& Value);
    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, int8& Value);
    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, uint16& Value);
    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, int16& Value);
    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, uint32& Value);
    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, bool& D);

    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, int32& Value);
    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, long& Value);
    FORCEINLINE friend FArchive& operator<<( FArchive& Ar, float& Value);
    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, double& Value);
    FORCEINLINE friend FArchive& operator<<(FArchive &Ar, uint64& Value);
    /*FORCEINLINE*/friend FArchive& operator<<(FArchive& Ar, int64& Value);
    template <
        typename EnumType,
        typename = typename TEnableIf<TIsEnumClass<EnumType>::Value>::Type
    >
    FORCEINLINE friend FArchive& operator<<(FArchive& Ar, EnumType& Value)
    {
        return Ar << (__underlying_type(EnumType)&)Value;
    }

    friend FArchive& operator<<(FArchive& Ar, struct FIntRect& Value);
    friend CORE_API FArchive& operator<<(FArchive& Ar, FString& Value);

public:
    virtual void Serialize(void* V, int64 Length) ;
    virtual void SerializeBits(void* V, int64 LengthBits);
    virtual void SerializeInt(uint32& Value, uint32 Max)

    /** Packs int value into bytes of 7 bits with 8th bit for 'more' */
    virtual void SerializeIntPacked(uint32& Value);
    virtual void Preload(UObject* Object) { }

    virtual void CountBytes(SIZE_T InNum, SIZE_T InMax) { }
    virtual FString GetArchiveName() const;
    virtual class FLinker* GetLinker()
    {
        return nullptr;
    }

    virtual int64 Tell()；
    virtual int64 TotalSize()；
    virtual bool AtEnd()；
    virtual void Seek(int64 InPos) { }
    virtual void AttachBulkData(UObject* Owner, FUntypedBulkData* BulkData) { }
    virtual void DetachBulkData(FUntypedBulkData* BulkData, bool bEnsureBulkDataIsLoaded) { }

    /**
     * Sets mapping from offsets/ sizes that are going to be used for seeking and serialization to what
     * is actually stored on disk. If the archive supports dealing with compression in this way it is 
     * going to return true.
     *
     * @param   CompressedChunks    Pointer to array containing information about [un]compressed chunks
     * @param   CompressionFlags    Flags determining compression format associated with mapping
     *
     * @return true if archive supports translating offsets & uncompressing on read, false otherwise
     */
    virtual bool SetCompressionMap(TArray<struct FCompressedChunk>* CompressedChunks, ECompressionFlags CompressionFlags)；
    virtual void Flush() { }

    virtual bool Close()；
    virtual bool GetError()；
    void SetError() ；

    /**
     * Serializes and compresses/ uncompresses data. This is a shared helper function for compression
     * support. The data is saved in a way compatible with FIOSystem::LoadCompressedData.
     *
     * @param   V       Data pointer to serialize data from/ to
     * @param   Length  Length of source data if we're saving, unused otherwise
     * @param   Flags   Flags to control what method to use for [de]compression and optionally control memory vs speed when compressing
     * @param   bTreatBufferAsFileReader true if V is actually an FArchive, which is used when saving to read data - helps to avoid single huge allocations of source data
     * @param   bUsePlatformBitWindow use a platform specific bitwindow setting
     */
    void SerializeCompressed(void* V, int64 Length, ECompressionFlags Flags, bool bTreatBufferAsFileReader = false, bool bUsePlatformBitWindow = false);


    FORCEINLINE bool IsByteSwapping(); // 平台的大端、小端

    // Used to do byte swapping on small items. This does not happen usually, so we don't want it inline
    void ByteSwap(void* V, int32 Length);

    FORCEINLINE FArchive& ByteOrderSerialize(void* V, int32 Length);

    /** Sets a flag indicating that this archive contains code. */
    void ThisContainsCode();

    /** Sets a flag indicating that this archive contains a ULevel or UWorld object. */
    void ThisContainsMap() ;

    /** Sets a flag indicating that this archive contains data required to be gathered for localization. */
    void ThisRequiresLocalizationGather();

    /** Sets a flag indicating that this archive is currently serializing class/struct defaults. */
    void StartSerializingDefaults() ;
    /** Indicate that this archive is no longer serializing class/struct defaults. */
    void StopSerializingDefaults() ;
    /**
     * Called when an object begins serializing property data using script serialization.
     */
    virtual void MarkScriptSerializationStart(const UObject* Obj) { }

    /**
     * Called when an object stops serializing property data using script serialization.
     */
    virtual void MarkScriptSerializationEnd(const UObject* Obj) { }

    FORCEINLINE bool IsLoading() const
    {
        return ArIsLoading;
    }

    FORCEINLINE bool IsSaving() const
    {
        return ArIsSaving;
    }

    FORCEINLINE bool IsTransacting() const
    {
        if (FPlatformProperties::HasEditorOnlyData())
        {
            return ArIsTransacting;
        }
        else
        {
            return false;
        }
    }

    FORCEINLINE bool WantBinaryPropertySerialization() const
    {
        return ArWantBinaryPropertySerialization;
    }

    FORCEINLINE bool IsForcingUnicode() const
    {
        return ArForceUnicode;
    }

    FORCEINLINE bool IsPersistent() const
    {
        return ArIsPersistent;
    }

    FORCEINLINE bool IsError() const
    {
        return ArIsError;
    }

    FORCEINLINE bool IsCriticalError() const
    {
        return ArIsCriticalError;
    }

    FORCEINLINE bool ContainsCode() const
    {
        return ArContainsCode;
    }

    FORCEINLINE bool ContainsMap() const
    {
        return ArContainsMap;
    }

    FORCEINLINE bool RequiresLocalizationGather() const
    {
        return ArRequiresLocalizationGather;
    }

    FORCEINLINE bool ForceByteSwapping() const
    {
        return ArForceByteSwapping;
    }

    FORCEINLINE bool IsSerializingDefaults() const
    {
        return (ArSerializingDefaults > 0) ? true : false;
    }

    FORCEINLINE bool IsIgnoringArchetypeRef() const
    {
        return ArIgnoreArchetypeRef;
    }

    FORCEINLINE bool DoDelta() const
    {
        return !ArNoDelta;
    }

    FORCEINLINE bool IsIgnoringOuterRef() const
    {
        return ArIgnoreOuterRef;
    }

    FORCEINLINE bool IsIgnoringClassGeneratedByRef() const
    {
        return ArIgnoreClassGeneratedByRef;
    }

    FORCEINLINE bool IsIgnoringClassRef() const
    {
        return ArIgnoreClassRef;
    }

    FORCEINLINE bool IsAllowingLazyLoading() const
    {
        return ArAllowLazyLoading;
    }

    FORCEINLINE bool IsObjectReferenceCollector() const
    {
        return ArIsObjectReferenceCollector;
    }

    FORCEINLINE bool IsModifyingWeakAndStrongReferences() const
    {
        return ArIsModifyingWeakAndStrongReferences;
    }

    FORCEINLINE bool IsCountingMemory() const
    {
        return ArIsCountingMemory;
    }

    FORCEINLINE uint32 GetPortFlags() const
    {
        return ArPortFlags;
    }
private:

    /** Copies all of the members except CustomVersionContainer */
    void CopyTrivialFArchiveStatusMembers(const FArchive& ArchiveStatusToCopy);

public:

    /** Whether this archive is for loading data. */
    uint8 ArIsLoading : 1;

    /** Whether this archive is for saving data. */
    uint8 ArIsSaving : 1;

    /** Whether archive is transacting. */
    uint8 ArIsTransacting : 1;

    /** Whether this archive wants properties to be serialized in binary form instead of tagged. */
    uint8 ArWantBinaryPropertySerialization : 1;

    /** Whether this archive wants to always save strings in unicode format */
    uint8 ArForceUnicode : 1;

    /** Whether this archive saves to persistent storage. */
    uint8 ArIsPersistent : 1;

    /** Whether this archive contains errors. */
    uint8 ArIsError : 1;

    /** Whether this archive contains critical errors. */
    uint8 ArIsCriticalError : 1;

    /** Quickly tell if an archive contains script code. */
    uint8 ArContainsCode : 1;

    /** Used to determine whether FArchive contains a level or world. */
    uint8 ArContainsMap : 1;

    /** Used to determine whether FArchive contains data required to be gathered for localization. */
    uint8 ArRequiresLocalizationGather : 1;

    /** Whether we should forcefully swap bytes. */
    uint8 ArForceByteSwapping : 1;

    /** If true, we will not serialize the ObjectArchetype reference in UObject. */
    uint8 ArIgnoreArchetypeRef : 1;

    /** If true, we will not serialize the ObjectArchetype reference in UObject. */
    uint8 ArNoDelta : 1;

    /** If true, we will not serialize the Outer reference in UObject. */
    uint8 ArIgnoreOuterRef : 1;

    /** If true, we will not serialize ClassGeneratedBy reference in UClass. */
    uint8 ArIgnoreClassGeneratedByRef : 1;

    /** If true, UObject::Serialize will skip serialization of the Class property. */
    uint8 ArIgnoreClassRef : 1;

    /** Whether to allow lazy loading. */
    uint8 ArAllowLazyLoading : 1;

    /** Whether this archive only cares about serializing object references. */
    uint8 ArIsObjectReferenceCollector : 1;

    /** Whether a reference collector is modifying the references and wants both weak and strong ones */
    uint8 ArIsModifyingWeakAndStrongReferences : 1;

    /** Whether this archive is counting memory and therefore wants e.g. TMaps to be serialized. */
    uint8 ArIsCountingMemory : 1;

    /** Whether bulk data serialization should be skipped or not. */
    uint8 ArShouldSkipBulkData : 1;

    /** Whether editor only properties are being filtered from the archive (or has been filtered). */
    uint8 ArIsFilterEditorOnly : 1;

    /** Whether this archive is saving/loading game state */
    uint8 ArIsSaveGame : 1;

    /** Set TRUE to use the custom property list attribute for serialization. */
    uint8 ArUseCustomPropertyList : 1;

    /** Whether we are currently serializing defaults. > 0 means yes, <= 0 means no. */
    int32 ArSerializingDefaults;

    /** Modifier flags that be used when serializing UProperties */
    uint32 ArPortFlags;
            
    /** Max size of data that this archive is allowed to serialize. */
    int64 ArMaxSerializeSize;

protected:

    /** Holds the archive version. */
    int32 ArUE4Ver;

    /** Holds the archive version for licensees. */
    int32 ArLicenseeUE4Ver;

    /** Holds the engine version. */
    FEngineVersionBase ArEngineVer;

    /** Holds the engine network protocol version. */
    uint32 ArEngineNetVer;

    /** Holds the game network protocol version. */
    uint32 ArGameNetVer;
};

通过重载operater <<来实现对数据的访问。

UObject的序列化接口

• void UObject::Serialize( FArchive& Ar )
  UObject通过实现Serialize接口来序列化对象数据。

void UObject::Serialize( FArchive& Ar )
{
    // These three items are very special items from a serialization standpoint. They aren't actually serialized.
    UClass *ObjClass = GetClass();
    UObject* LoadOuter = GetOuter();
    FName LoadName = GetFName();

    // Make sure this object's class's data is loaded.
    if(ObjClass->HasAnyFlags(RF_NeedLoad) )
    {
        Ar.Preload(ObjClass);

        // make sure this object's template data is loaded - the only objects
        // this should actually affect are those that don't have any defaults
        // to serialize.  for objects with defaults that actually require loading
        // the class default object should be serialized in FLinkerLoad::Preload, before
        // we've hit this code.
        if ( !HasAnyFlags(RF_ClassDefaultObject) && ObjClass->GetDefaultsCount() > 0 )
        {
            Ar.Preload(ObjClass->GetDefaultObject());
        }
    }

    // Special info.
    if( (!Ar.IsLoading() && !Ar.IsSaving() && !Ar.IsObjectReferenceCollector()) )
    {
        Ar << LoadName;   // 对象名
        if(!Ar.IsIgnoringOuterRef())
        {
            Ar << LoadOuter;        // Outer对象
        }
        if ( !Ar.IsIgnoringClassRef() )
        {
            Ar << ObjClass;  // UClass对象
        }
    }
    // Special support for supporting undo/redo of renaming and changing Archetype. 编辑器中使用
    else if( Ar.IsTransacting() )
    {
        if(!Ar.IsIgnoringOuterRef())
        {
            if(Ar.IsLoading())
            {
                Ar << LoadName << LoadOuter;

                // If the name we loaded is different from the current one,
                // unhash the object, change the name and hash it again.
                bool bDifferentName = GetFName() != NAME_None && LoadName != GetFName();
                bool bDifferentOuter = LoadOuter != GetOuter();
                if ( bDifferentName == true || bDifferentOuter == true )
                {
                    LowLevelRename(LoadName,LoadOuter);
                }
            }
            else
            {
                Ar << LoadName << LoadOuter;
            }
        }
    }

    // Serialize object properties which are defined in the class.
    // Handle derived UClass objects (exact UClass objects are native only and shouldn't be touched)
        // 判断当前对象是否为UClass的实例, 如果是普通的Object的化就序列化它的所有带UPROPERTY()的成员变量
    if (ObjClass != UClass::StaticClass())
    {
        SerializeScriptProperties(Ar);
    }

    // Keep track of pending kill
    if( Ar.IsTransacting() )  // 编辑中undo/redo时序列化
    {
        bool WasKill = IsPendingKill();
        if( Ar.IsLoading() )
        {
            Ar << WasKill;
            if (WasKill)
            {
                MarkPendingKill();
            }
            else
            {
                ClearPendingKill();
            }
        }
        else if( Ar.IsSaving() )
        {
            Ar << WasKill;
        }
    }

    // Serialize a GUID if this object has one mapped to it
    FLazyObjectPtr::PossiblySerializeObjectGuid(this, Ar);

    // Invalidate asset pointer caches when loading a new object
    if (Ar.IsLoading() )
    {
        FStringAssetReference::InvalidateTag();
    }

    // Memory counting (with proper alignment to match C++)
    SIZE_T Size = GetClass()->GetStructureSize();
    Ar.CountBytes( Size, Size );
}

• void UObject::SerializeScriptProperties( FArchive& Ar ) const
  该函数用来序列化Object体系类中声明为UPROPERTY()的成员变量。

Paste_Image.png

• UStruct::SerializeTaggedProperties
  序列化对象属性，并且加入tag，这个是为了处理对象类发生变化导致属性匹配不上(版本升级和容错)。

void UStruct::SerializeTaggedProperties(FArchive& Ar, uint8* Data, UStruct* DefaultsStruct, uint8* Defaults, const UObject* BreakRecursionIfFullyLoad) const
{
    //SCOPED_LOADTIMER(SerializeTaggedPropertiesTime);

    // Determine if this struct supports optional property guid's (UBlueprintGeneratedClasses Only)
    const bool bArePropertyGuidsAvailable = (Ar.UE4Ver() >= VER_UE4_PROPERTY_GUID_IN_PROPERTY_TAG) && !FPlatformProperties::RequiresCookedData() && ArePropertyGuidsAvailable();

    if( Ar.IsLoading() )
    {
        // Load tagged properties.

        // This code assumes that properties are loaded in the same order they are saved in. This removes a n^2 search 
        // and makes it an O(n) when properties are saved in the same order as they are loaded (default case). In the 
        // case that a property was reordered the code falls back to a slower search.
        UProperty*  Property            = PropertyLink;
        bool        bAdvanceProperty    = false;
        int32       RemainingArrayDim   = Property ? Property->ArrayDim : 0;

        // Load all stored properties, potentially skipping unknown ones.
        while (1)
        {
            FPropertyTag Tag;
            Ar << Tag;  // 读取标记

            if( Tag.Name == NAME_None )
            {
                break;
            }
            if (!Tag.Name.IsValid())
            {
                UE_LOG(LogClass, Warning, TEXT("Invalid tag name: struct '%s', archive '%s'"), *GetName(), *Ar.GetArchiveName());
                break;
            }

            // Move to the next property to be serialized
            if( bAdvanceProperty && --RemainingArrayDim <= 0 )
            {
                Property = Property->PropertyLinkNext;
                // Skip over properties that don't need to be serialized.
                while( Property && !Property->ShouldSerializeValue( Ar ) )
                {
                    Property = Property->PropertyLinkNext;
                }
                bAdvanceProperty        = 0;
                RemainingArrayDim   = Property ? Property->ArrayDim : 0;
            }
            
            // Optionally resolve properties using Guid Property tags in non cooked builds that support it.
            if (bArePropertyGuidsAvailable && Tag.HasPropertyGuid)
            {
                // Use property guids from blueprint generated classes to redirect serialised data.
                FName Result = FindPropertyNameFromGuid(Tag.PropertyGuid);
                if (Result != NAME_None && Tag.Name != Result)
                {
                    Tag.Name = Result;
                }
            }
            // If this property is not the one we expect (e.g. skipped as it matches the default value), do the brute force search.
            if( Property == nullptr || Property->GetFName() != Tag.Name )
            {
                // No need to check redirects on platforms where everything is cooked. Always check for save games
                if ((!FPlatformProperties::RequiresCookedData() || Ar.IsSaveGame()) && !Ar.HasAnyPortFlags(PPF_DuplicateForPIE|PPF_Duplicate))
                {
                    FName EachName = GetFName();
                    FName PackageName = GetOutermost()->GetFName();
                    // Search the current class first, then work up the class hierarchy to see if theres a match for our fixup.
                    UStruct* Owner = GetOwnerStruct();
                    if( Owner )
                    {
                        UStruct* CheckStruct = Owner;
                        while(CheckStruct)
                        {
                            FName NewTagName = UProperty::FindRedirectedPropertyName(CheckStruct, Tag.Name);

                            if (NewTagName != NAME_None)
                            {
                                Tag.Name = NewTagName;
                                break;
                            }

                            CheckStruct = CheckStruct->GetSuperStruct();
                        }
                    }
                }

                UProperty* CurrentProperty = Property;
                // Search forward...
                for ( ; Property; Property=Property->PropertyLinkNext ) // 查找到对应的Property
                {
                    if( Property->GetFName() == Tag.Name )
                    {
                        break;
                    }
                }
                // ... and then search from the beginning till we reach the current property if it's not found. 从头再找，这块利用了大部分情况下UProperty是顺序序列化的，提高查找效率。
                if( Property == nullptr )
                {
                    for( Property = PropertyLink; Property && Property != CurrentProperty; Property = Property->PropertyLinkNext )
                    {
                        if( Property->GetFName() == Tag.Name )
                        {
                            break;
                        }
                    }

                    if( Property == CurrentProperty )
                    {
                        // Property wasn't found.
                        Property = nullptr;
                    }
                }

                RemainingArrayDim = Property ? Property->ArrayDim : 0;
            }
#if WITH_EDITOR
            if (!Property)
            {
                Property = CustomFindProperty(Tag.Name);
            }
#endif // WITH_EDITOR

            FName PropID = Property ? Property->GetID() : NAME_None;
            FName ArrayInnerID = NAME_None;

            // Check if this is a struct property and we have a redirector
            // No need to check redirects on platforms where everything is cooked. Always check for save games
            if (!FPlatformProperties::RequiresCookedData() || Ar.IsSaveGame())
            {
                if (Tag.Type == NAME_StructProperty && PropID == NAME_StructProperty)
                {
                    const FName NewName = FLinkerLoad::FindNewNameForStruct(Tag.StructName);
                    const FName StructName = CastChecked<UStructProperty>(Property)->Struct->GetFName();
                    if (NewName == StructName)
                    {
                        Tag.StructName = NewName;
                    }
                }
                else if ((PropID == NAME_EnumProperty) && ((Tag.Type == NAME_EnumProperty) || (Tag.Type == NAME_ByteProperty)))
                {
                    const FName NewName = FLinkerLoad::FindNewNameForEnum(Tag.EnumName);
                    if (!NewName.IsNone())
                    {
                        Tag.EnumName = NewName;
                    }
                }
            }

            const int64 StartOfProperty = Ar.Tell();
            if( !Property )
            {
                //UE_LOG(LogClass, Warning, TEXT("Property %s of %s not found for package:  %s"), *Tag.Name.ToString(), *GetFullName(), *Ar.GetArchiveName() );
            }
#if WITH_EDITOR
            else if (BreakRecursionIfFullyLoad && BreakRecursionIfFullyLoad->HasAllFlags(RF_LoadCompleted))
            {
            }
#endif // WITH_EDITOR
            // editoronly properties should be skipped if we are NOT the editor, or we are 
            // the editor but are cooking for console (editoronly implies notforconsole)
            else if ((Property->PropertyFlags & CPF_EditorOnly) && !FPlatformProperties::HasEditorOnlyData() && !GForceLoadEditorOnly)
            {
            }
            // check for valid array index
            else if( Tag.ArrayIndex >= Property->ArrayDim || Tag.ArrayIndex < 0 )
            {
                UE_LOG(LogClass, Warning, TEXT("Array bound exceeded (var %s=%d, exceeds %s [0-%d] in package:  %s"), 
                    *Tag.Name.ToString(), Tag.ArrayIndex, *GetName(), Property->ArrayDim-1, *Ar.GetArchiveName());
            }

            else if( !Property->ShouldSerializeValue(Ar) )
            {
                UE_CLOG((Ar.IsPersistent() && FPlatformProperties::RequiresCookedData()), LogClass, Warning, TEXT("Skipping saved property %s of %s since it is no longer serializable for asset:  %s. (Maybe resave asset?)"), *Tag.Name.ToString(), *GetName(), *Ar.GetArchiveName() );
            }

            else if (Property->ConvertFromType(Tag, Ar, Data, DefaultsStruct, bAdvanceProperty))
            {
                if (bAdvanceProperty)
                {
                    continue;
                }
            }

            else if (Tag.Type != PropID)
            {
                UE_LOG(LogClass, Warning, TEXT("Type mismatch in %s of %s - Previous (%s) Current(%s) for package:  %s"), *Tag.Name.ToString(), *GetName(), *Tag.Type.ToString(), *PropID.ToString(), *Ar.GetArchiveName() );
            }
            else
            {
                uint8* DestAddress = Property->ContainerPtrToValuePtr<uint8>(Data, Tag.ArrayIndex);  
                uint8* DefaultsFromParent = Property->ContainerPtrToValuePtrForDefaults<uint8>(DefaultsStruct, Defaults, Tag.ArrayIndex);

                // This property is ok. 读取数据内存          
                Tag.SerializeTaggedProperty(Ar, Property, DestAddress, DefaultsFromParent);

                bAdvanceProperty = true;
                if (!Ar.IsCriticalError())
                {
                    continue;
                }
            }

            bAdvanceProperty = false;

            // Skip unknown or bad property.
            const int64 RemainingSize = Tag.Size - (Ar.Tell() - StartOfProperty);
            uint8 B;
            for( int64 i=0; i<RemainingSize; i++ )
            {
                Ar << B;
            }
        }
    }
    else
    {
        check(Ar.IsSaving() || Ar.IsCountingMemory());

        UScriptStruct* DefaultsScriptStruct = dynamic_cast<UScriptStruct*>(DefaultsStruct);

        /** If true, it means that we want to serialize all properties of this struct if any properties differ from defaults */
        bool bUseAtomicSerialization = false;
        if (DefaultsScriptStruct)
        {
            bUseAtomicSerialization = DefaultsScriptStruct->ShouldSerializeAtomically(Ar);
        }

        // Save tagged properties.

        // Iterate over properties in the order they were linked and serialize them.
        const FCustomPropertyListNode* CustomPropertyNode = Ar.ArUseCustomPropertyList ? Ar.ArCustomPropertyList : nullptr;
        for (UProperty* Property = Ar.ArUseCustomPropertyList ? (CustomPropertyNode ? CustomPropertyNode->Property : nullptr) : PropertyLink;
            Property;
            Property = Ar.ArUseCustomPropertyList ? FCustomPropertyListNode::GetNextPropertyAndAdvance(CustomPropertyNode) : Property->PropertyLinkNext)
        {
            if( Property->ShouldSerializeValue(Ar) )// 判断是否要序列化该属性
            {
                const int32 LoopMin = CustomPropertyNode ? CustomPropertyNode->ArrayIndex : 0;
                const int32 LoopMax = CustomPropertyNode ? LoopMin + 1 : Property->ArrayDim;
                for( int32 Idx = LoopMin; Idx < LoopMax; Idx++ )
                {
                    uint8* DataPtr      = Property->ContainerPtrToValuePtr           <uint8>(Data, Idx);
                    uint8* DefaultValue = Property->ContainerPtrToValuePtrForDefaults<uint8>(DefaultsStruct, Defaults, Idx);
                                        // 判断该属性是否为CDO中的属性值相同，如果相同就不必存了(节约)
                    if( CustomPropertyNode || !Ar.DoDelta() || Ar.IsTransacting() || (!Defaults && !dynamic_cast<const UClass*>(this)) || !Property->Identical( DataPtr, DefaultValue, Ar.GetPortFlags()) )
                    {
                        if (bUseAtomicSerialization)
                        {
                            DefaultValue = NULL;
                        }
#if WITH_EDITOR
                        static const FName NAME_PropertySerialize = FName(TEXT("PropertySerialize"));
                        FArchive::FScopeAddDebugData P(Ar, NAME_PropertySerialize);
                        FArchive::FScopeAddDebugData S(Ar, Property->GetFName());
#endif
                        FPropertyTag Tag( Ar, Property, Idx, DataPtr, DefaultValue );
                        // If available use the property guid from BlueprintGeneratedClasses, provided we aren't cooking data.
                        if (bArePropertyGuidsAvailable && !Ar.IsCooking())
                        {
                            const FGuid PropertyGuid = FindPropertyGuidFromName(Tag.Name);
                            Tag.SetPropertyGuid(PropertyGuid);
                        }
                        Ar << Tag;

                        // need to know how much data this call to SerializeTaggedProperty consumes, so mark where we are
                        int64 DataOffset = Ar.Tell();

                        // if using it, save the current custom property list and switch to its sub property list (in case of UStruct serialization)
                        const FCustomPropertyListNode* SavedCustomPropertyList = nullptr;
                        if(Ar.ArUseCustomPropertyList && CustomPropertyNode)
                        {
                            SavedCustomPropertyList = Ar.ArCustomPropertyList;
                            Ar.ArCustomPropertyList = CustomPropertyNode->SubPropertyList;
                        }

                        Tag.SerializeTaggedProperty( Ar, Property, DataPtr, DefaultValue );

                        // restore the original custom property list after serializing
                        if (SavedCustomPropertyList)
                        {
                            Ar.ArCustomPropertyList = SavedCustomPropertyList;
                        }

                        // set the tag's size
                        Tag.Size = Ar.Tell() - DataOffset;

                        if ( Tag.Size >  0 )
                        {
                            // mark our current location
                            DataOffset = Ar.Tell();

                            // go back and re-serialize the size now that we know it
                            Ar.Seek(Tag.SizeOffset);
                            Ar << Tag.Size;

                            // return to the current location
                            Ar.Seek(DataOffset);
                        }
                    }
                }
            }
        }
        static FName Temp(NAME_None);
        Ar << Temp;
    }
}

• UStruct::SerializeBinEx
  特殊情况才会走这个函数，目前个人认为正常存文件不走这个函数。

void UStruct::SerializeBinEx( FArchive& Ar, void* Data, void const* DefaultData, UStruct* DefaultStruct ) const
{
    if ( !DefaultData || !DefaultStruct )
    {
        SerializeBin(Ar, Data);
        return;
    }

    for( TFieldIterator<UProperty> It(this); It; ++It )
    {
                // Serializes the property with the struct's data residing in Data, unless it matches the default
                // 序列化跟CDO中不一样的属性
        It->SerializeNonMatchingBinProperty(Ar, Data, DefaultData, DefaultStruct);
    }
}

• UStruct::SerializeBin

//
// Serialize all of the class's data that belongs in a particular
// bin and resides in Data.
//
void UStruct::SerializeBin( FArchive& Ar, void* Data ) const
{
    if( Ar.IsObjectReferenceCollector() )
    {
        for( UProperty* RefLinkProperty=RefLink; RefLinkProperty!=NULL; RefLinkProperty=RefLinkProperty->NextRef )
        {
            RefLinkProperty->SerializeBinProperty( Ar, Data );
        }
    }
    else if( Ar.ArUseCustomPropertyList )
    {
        const FCustomPropertyListNode* CustomPropertyList = Ar.ArCustomPropertyList;
        for (auto PropertyNode = CustomPropertyList; PropertyNode; PropertyNode = PropertyNode->PropertyListNext)
        {
            UProperty* Property = PropertyNode->Property;
            if( Property )
            {
                // Temporarily set to the sub property list, in case we're serializing a UStruct property.
                Ar.ArCustomPropertyList = PropertyNode->SubPropertyList;

                Property->SerializeBinProperty(Ar, Data, PropertyNode->ArrayIndex);

                // Restore the original property list.
                Ar.ArCustomPropertyList = CustomPropertyList;
            }
        }
    }
    else
    {
        for (UProperty* Property = PropertyLink; Property != NULL; Property = Property->PropertyLinkNext)
        {
            Property->SerializeBinProperty(Ar, Data);
        }
    }
}

下面为调试时的几张堆栈图:

Paste_Image.png Paste_Image.png

---

uasset文件格式

UE中使用统一的格式存储资源(uasset, umap)，每个uasset对应一个包(package)，存储一个UPackage对象时，会将该包下的所有对象都存到uasset中。UE的uasset文件格式很像Windows下的DLL文件格式(PE格式)，并且使用起来神似(下一节分析Linker)。

UE4文件格式.png

• File Summary 文件头信息
• Name Table 包中对象的名字表
• Import Table 存放被该包中对象引用的其它包中的对象信息(路径名和类型)
• Export Table 该包中的对象信息(路径名和类型)
• Export Objects 所有Export Table中对象的实际数据。

/**
 * A "table of contents" for an Unreal package file.  Stored at the top of the file.
 */
struct FPackageFileSummary
{
    /**
    * Magic tag compared against PACKAGE_FILE_TAG to ensure that package is an Unreal package.
    */
    int32       Tag;

private:
    /* UE4 file version */
    int32       FileVersionUE4;
    /* Licensee file version */
    int32       FileVersionLicenseeUE4;
    /* Custom version numbers. Keyed off a unique tag for each custom component. */
    FCustomVersionContainer CustomVersionContainer;

public:
    /**
    * Total size of all information that needs to be read in to create a FLinkerLoad. This includes
    * the package file summary, name table and import & export maps.
    */
    int32       TotalHeaderSize;

    /**
    * The flags for the package
    */
    uint32  PackageFlags;

    /**
    * The Generic Browser folder name that this package lives in
    */
    FString FolderName;

    /**
    * Number of names used in this package
    */
    int32       NameCount;

    /**
    * Location into the file on disk for the name data
    */
    int32   NameOffset;

    /**
    * Number of gatherable text data items in this package
    */
    int32   GatherableTextDataCount;

    /**
    * Location into the file on disk for the gatherable text data items
    */
    int32   GatherableTextDataOffset;

    /**
    * Number of exports contained in this package
    */
    int32       ExportCount;

    /**
    * Location into the file on disk for the ExportMap data
    */
    int32       ExportOffset;

    /**
    * Number of imports contained in this package
    */
    int32       ImportCount;

    /**
    * Location into the file on disk for the ImportMap data
    */
    int32       ImportOffset;

    /**
    * Location into the file on disk for the DependsMap data
    */
    int32       DependsOffset;

    /**
    * Number of references contained in this package
    */
    int32       StringAssetReferencesCount;

    /**
    * Location into the file on disk for the string asset references map data
    */
    int32       StringAssetReferencesOffset;

    /**
    * Location into the file on disk for the SearchableNamesMap data
    */
    int32       SearchableNamesOffset;

    /**
    * Thumbnail table offset
    */
    int32       ThumbnailTableOffset;

    /**
    * Current id for this package
    */
    FGuid   Guid;

    /**
    * Data about previous versions of this package
    */
    TArray<FGenerationInfo> Generations;

    /**
    * Engine version this package was saved with. For hotfix releases and engine versions which maintain strict binary compatibility with another version, this may differ from CompatibleWithEngineVersion.
    */
    FEngineVersion SavedByEngineVersion;

    /**
    * Engine version this package is compatible with. See SavedByEngineVersion.
    */
    FEngineVersion CompatibleWithEngineVersion;

    /**
    * Flags used to compress the file on save and uncompress on load.
    */
    uint32  CompressionFlags;

    /**
    * Value that is used to determine if the package was saved by Epic (or licensee) or by a modder, etc
    */
    uint32  PackageSource;

    /**
    * Array of compressed chunks in case this package was stored compressed.
    */
    TArray<FCompressedChunk> CompressedChunks;

    /**
    * If true, this file will not be saved with version numbers or was saved without version numbers. In this case they are assumed to be the current version.
    * This is only used for full cooks for distribution because it is hard to guarantee correctness
    **/
    bool bUnversioned;

    /**
    * Location into the file on disk for the asset registry tag data
    */
    int32   AssetRegistryDataOffset;

    /** Offset to the location in the file where the bulkdata starts */
    int64   BulkDataStartOffset;
    /**
    * Offset to the location in the file where the FWorldTileInfo data starts
    */
    int32   WorldTileInfoDataOffset;

    /**
    * Streaming install ChunkIDs
    */
    TArray<int32>   ChunkIDs;

    int32       PreloadDependencyCount;

    /**
    * Location into the file on disk for the preload dependency data
    */
    int32       PreloadDependencyOffset;
};

导入表条目FObjectImport

/**
 * UObject resource type for objects that are referenced by this package, but contained
 * within another package.
 */
struct FObjectImport 
{
    /**
     * The name of the UObject represented by this resource.
     * Serialized
     */
    FName           ObjectName;  // 对象名称

    /**
     * Location of the resource for this resource's Outer.  Values of 0 indicate that this resource
     * represents a top-level UPackage object (the linker's LinkerRoot).
     * Serialized
     */
    FPackageIndex   OuterIndex;  // 对象的Outer的Index

    /**
     * The name of the package that contains the class of the UObject represented by this resource.
     * Serialized
     */
    FName           ClassPackage;  // 该对象的类元数据(UClass对象）所在的包名

    /**
     * The name of the class for the UObject represented by this resource.
     * Serialized
     */
    FName           ClassName;   // 该对象的类元数据名称

// 后面的数据为运行时填写
    /**
     * The UObject represented by this resource.  Assigned the first time CreateImport is called for this import.
     * Transient
     */
    UObject*        XObject;

    /**
     * The linker that contains the original FObjectExport resource associated with this import.
     * Transient
     */
    FLinkerLoad*    SourceLinker;  // 该对象由哪个Linker加载的

    /**
     * Index into SourceLinker's ExportMap for the export associated with this import's UObject.
     * Transient
     */
    int32                SourceIndex;
    bool            bImportPackageHandled;
    bool            bImportSearchedFor;
    bool            bImportFailed;
};

导出表的条目FObjectExport

/**
 * UObject resource type for objects that are contained within this package and can
 * be referenced by other packages.
 */
struct FObjectExport 
{
    /**
     * The name of the UObject represented by this resource.
     * Serialized
     */
    FName           ObjectName;  // 对象名称

    /**
     * Location of the resource for this resource's Outer.  Values of 0 indicate that this resource
     * represents a top-level UPackage object (the linker's LinkerRoot).
     * Serialized
     */
    FPackageIndex   OuterIndex;  // 对象的Outer的Index

    /**
     * Location of the resource for this export's class (if non-zero).  A value of zero
     * indicates that this export represents a UClass object; there is no resource for
     * this export's class object
     * Serialized
     */
    FPackageIndex   ClassIndex;

    /**
    * Location of this resource in export map. Used for export fixups while loading packages.
    * Value of zero indicates resource is invalid and shouldn't be loaded.
    * Not serialized.
    */
    FPackageIndex ThisIndex;

    /**
     * Location of the resource for this export's SuperField (parent).  Only valid if
     * this export represents a UStruct object. A value of zero indicates that the object
     * represented by this export isn't a UStruct-derived object.
     * Serialized
     */
    FPackageIndex   SuperIndex;

    /**
    * Location of the resource for this export's template/archetypes.  Only used
    * in the new cooked loader. A value of zero indicates that the value of GetArchetype
    * was zero at cook time, which is more or less impossible and checked.
    * Serialized
    */
    FPackageIndex   TemplateIndex; // 对象原形的Index

    /**
     * The object flags for the UObject represented by this resource.  Only flags that
     * match the RF_Load combination mask will be loaded from disk and applied to the UObject.
     * Serialized
     */
    EObjectFlags    ObjectFlags;

    /**
     * The number of bytes to serialize when saving/loading this export's UObject.
     * Serialized
     */
    int64           SerialSize;      //对象占据的磁盘大小

    /**
     * The location (into the FLinker's underlying file reader archive) of the beginning of the
     * data for this export's UObject.  Used for verification only.
     * Serialized
     */
    int64           SerialOffset;  // 文件中的偏移

    /**
     * The location (into the FLinker's underlying file reader archive) of the beginning of the
     * portion of this export's data that is serialized using script serialization.
     * Transient
     */
    int32               ScriptSerializationStartOffset;

    /**
     * The location (into the FLinker's underlying file reader archive) of the end of the
     * portion of this export's data that is serialized using script serialization.
     * Transient
     */
    int32               ScriptSerializationEndOffset;

    /**
     * The UObject represented by this export.  Assigned the first time CreateExport is called for this export.
     * Transient
     */
    UObject*        Object;

    /**
     * The index into the FLinker's ExportMap for the next export in the linker's export hash table.
     * Transient
     */
    int32               HashNext;

    /**
     * Whether the export was forced into the export table via OBJECTMARK_ForceTagExp.
     * Serialized
     */
    bool            bForcedExport;   

    /**
     * Whether the export should be loaded on clients
     * Serialized
     */
    bool            bNotForClient;   // 游戏Client是否使用该对象

    /**
     * Whether the export should be loaded on servers
     * Serialized
     */
    bool            bNotForServer;  // 游戏server是否使用该对象

    /**
     * Whether the export should be always loaded in editor game
     * False means that the object is 
     * True doesn't means, that the object won't be loaded.
     * Serialized
     */
    bool            bNotAlwaysLoadedForEditorGame;

    /**
     * True if this export is an asset object.
     */
    bool            bIsAsset;

    /**
     * Force this export to not load, it failed because the outer didn't exist.
     */
    bool            bExportLoadFailed;

    /**
     * Export is a dynamic type.
     */
    enum class EDynamicType : uint8
    {
        NotDynamicExport,
        DynamicType,
        ClassDefaultObject,
    };

    EDynamicType    DynamicType;

    /**
     * Export was filtered out on load
     */
    bool            bWasFiltered;

    /** If this object is a top level package (which must have been forced into the export table via OBJECTMARK_ForceTagExp)
     * this is the GUID for the original package file
     * Serialized
     */
    FGuid           PackageGuid;

    /** If this object is a top level package (which must have been forced into the export table via OBJECTMARK_ForceTagExp)
     * this is the package flags for the original package file
     * Serialized
     */
    uint32          PackageFlags;
};

备注: FPackageIndex表示Linker中Import Table或Export Table中的索引, 分如下情形

1. PackageIndex > 0 表示在Export Table中的索引，实际索引 Index = PackageIndex - 1;
2. PackageIndex < 0 表示在Export Table中的索引，实际索引 Index = -(PackageIndex + 1);
3. PackageIndex == 0 表示当前UPackage对象

FLinkerLoad

负责将uasset文件中的对象加载到内存中，起桥梁作用。相关源码:
Engine\Source\Runtime\CoreUObject\Public\UObject\LinkerLoad.h
Engine\Source\Runtime\CoreUObject\Private\UObject\LinkerLoad.cpp

1. FLinkerLoad::ELinkerStatus FLinkerLoad::Tick( float InTimeLimit, bool bInUseTimeLimit, bool bInUseFullTimeLimit );
   用于解析uasset文件，当bInUseTimeLimit为true时, Tick不会一次性做完解析工作，分时间片进行加载，在一帧里，Tick()不会占用太多时间。
2. UObject* CreateExport( int32 Index );
   创建Export Table中Index位置的对象。
3. void LoadAllObjects(bool bForcePreload);
   加载Export Table中的所有对象。

在阅读CreateExport, CreateImport等对象加载代码时需要明白Outer, Class, PackageIndex这些概念，先根据类型创建出对象，然后才Serialize对象的数据。

注: 读取包中对象时，可以一次性加载所有export table中的对象，也可以按需加载某个对象(比如包P0中的对象A被包P1引用，在加载P1时，可能只会加载P0中的对象A，而不是P0中的所有对象)。

关于异步加载
Engine\Source\Runtime\CoreUObject\Private\Serialization\AsyncLoading.cpp
Engine\Source\Runtime\CoreUObject\Private\Serialization\AsyncLoadingThread.h

FLinkerSave

负责将内存Package中的对象存储到uasset文件。相关源码:
Engine\Source\Runtime\CoreUObject\Public\UObject\LinkerSave.h
Engine\Source\Runtime\CoreUObject\Private\UObject\LinkerSave.cpp
LinkerSave做的活不多， 要特别注意序列化Object时的巧妙.

FArchive& FLinkerSave::operator<<( UObject*& Obj )
{
    FPackageIndex Save;
    if (Obj)
    {
        Save = MapObject(Obj);   // 返回的是PackageIndex
    }
    return *this << Save;
}

在保存一个Package时主要工作是在FSavePackageResultStruct UPackage::Save(UPackage* InOuter, UObject* Base, EObjectFlags TopLevelFlags, const TCHAR* Filename, FOutputDevice* Error, FLinkerLoad* Conform, bool bForceByteSwapping, bool bWarnOfLongFilename, uint32 SaveFlags, const class ITargetPlatform* TargetPlatform, const FDateTime& FinalTimeStamp, bool bSlowTask);函数中。
源码路径:Engine\Source\Runtime\CoreUObject\Private\UObject\SavePackage.cpp

部分值得注意的代码截图:

Paste_Image.png Paste_Image.png Paste_Image.png 写入Name Table Paste_Image.png Paste_Image.png 写入Export Object

结尾

还有以下几点没有讲述

1. uasset文件的压缩
2. uasset的Cook(过滤掉与游戏发布时无关的数据)
3. BulkData(BulkData是指一大块数据, 它也被存在uasset文件中，但是加载对象的时候可以不加载它，等到需要时在问LinkerLoad要数据，例如UTexture2D的纹理数据的加载)。