ART世界探险(20) - Android N上的编译流程

就在我们分析Android M版本的ART还只走出了一小段路的时候，Android N的新ART就问世了。
Android N上的ART还是有不小的改进的。不过做为一个关注细节的系列文章，我们还是从Compile的过程说起。

流程概述

在安装的时候，默认情况下，Android N只做interpret-only的编译，如下命令行所示：

/system/bin/dex2oat --zip-fd=7 --zip-location=base.apk --oat-fd=8 --oat-location=/data/app/vmdl692968727.tmp/oat/arm64/base.odex --instruction-set=arm64 --instruction-set-variant=kryo --instruction-set-features=default --runtime-arg -Xms64m --runtime-arg -Xmx512m --compiler-filter=interpret-only --swap-fd=10 --debuggable, priority: 10, policy: 5:freezer:/,4:name=systemd:/,3:cpuset:/,2:cpu:/bg_non_interactive,1:cpuacct:/,

所以我们更新一下编译时候的时序图：

compile

在安装的时候不编译了，那么就要在运行时通过JIT来编译，这个流程如下：

Jit

而在编译系统应用时，dex2oat还是会生成完整编译的结果的，我们看个高通骁龙820上的例子：

out/host/linux-x86/bin/dex2oatd --runtime-arg -Xms64m --runtime-arg -Xmx512m --runtime-arg -classpath --runtime-arg & --boot-image=out/target/product/msm8996/dex_bootjars/system/framework/boot.art --dex-file=out/target/product/msm8996/obj/APPS/Browser2_intermediates/oat/arm64/package.odex.input --dex-location=/system/app/Browser2/Browser2.apk --oat-file=out/target/product/msm8996/obj/APPS/Browser2_intermediates/oat/arm64/package.odex --android-root=out/target/product/msm8996/system --instruction-set=arm64 --instruction-set-variant=kryo --instruction-set-features=default --include-patch-information --runtime-arg -Xnorelocate --no-generate-debug-info --no-inline-from=core-oj.jar --generate-mini-debug-info --compile-pic

CompilerDriver::CompileAll

比起M上我们分析过的版本，N上的新版本的CompileAll的注释更详细了，结构也更清晰了一点。
我们先来看看Android M版的：

void CompilerDriver::Compile(jobject class_loader, const std::vector<const DexFile*>& dex_files,
                             ThreadPool* thread_pool, TimingLogger* timings) {
  for (size_t i = 0; i != dex_files.size(); ++i) {
    const DexFile* dex_file = dex_files[i];
    CHECK(dex_file != nullptr);
    CompileDexFile(class_loader, *dex_file, dex_files, thread_pool, timings);
  }
  VLOG(compiler) << "Compile: " << GetMemoryUsageString(false);
}

再看看对应的Android N的CompileAll，是不是看起来正规了许多呢？

void CompilerDriver::CompileAll(jobject class_loader,
                                const std::vector<const DexFile*>& dex_files,
                                TimingLogger* timings) {
  DCHECK(!Runtime::Current()->IsStarted());

  InitializeThreadPools();

  VLOG(compiler) << "Before precompile " << GetMemoryUsageString(false);
  // Precompile:
  // 1) Load image classes
  // 2) Resolve all classes
  // 3) Attempt to verify all classes
  // 4) Attempt to initialize image classes, and trivially initialized classes
  PreCompile(class_loader, dex_files, timings);
  // Compile:
  // 1) Compile all classes and methods enabled for compilation. May fall back to dex-to-dex
  //    compilation.
  if (!GetCompilerOptions().VerifyAtRuntime()) {
    Compile(class_loader, dex_files, timings);
  }
  if (dump_stats_) {
    stats_->Dump();
  }

  FreeThreadPools();
}

CompilerDriver::Compile

CompilerDriver的Compile方法，从Android M时代的不足10行，到了Android N上也变成40多行了。

Android M上的CompilerDriver::Compile:

void CompilerDriver::Compile(jobject class_loader, const std::vector<const DexFile*>& dex_files,
                             ThreadPool* thread_pool, TimingLogger* timings) {
  for (size_t i = 0; i != dex_files.size(); ++i) {
    const DexFile* dex_file = dex_files[i];
    CHECK(dex_file != nullptr);
    CompileDexFile(class_loader, *dex_file, dex_files, thread_pool, timings);
  }
  VLOG(compiler) << "Compile: " << GetMemoryUsageString(false);
}

Android N对应的Compile方法：

void CompilerDriver::Compile(jobject class_loader,
                             const std::vector<const DexFile*>& dex_files,
                             TimingLogger* timings) {
  if (kDebugProfileGuidedCompilation) {
    LOG(INFO) << "[ProfileGuidedCompilation] " <<
        ((profile_compilation_info_ == nullptr)
            ? "null"
            : profile_compilation_info_->DumpInfo(&dex_files));
  }

  DCHECK(current_dex_to_dex_methods_ == nullptr);
  for (const DexFile* dex_file : dex_files) {
    CHECK(dex_file != nullptr);
    CompileDexFile(class_loader,
                   *dex_file,
                   dex_files,
                   parallel_thread_pool_.get(),
                   parallel_thread_count_,
                   timings);
    const ArenaPool* const arena_pool = Runtime::Current()->GetArenaPool();
    const size_t arena_alloc = arena_pool->GetBytesAllocated();
    max_arena_alloc_ = std::max(arena_alloc, max_arena_alloc_);
    Runtime::Current()->ReclaimArenaPoolMemory();
  }

  ArrayRef<DexFileMethodSet> dex_to_dex_references;
  {
    // From this point on, we shall not modify dex_to_dex_references_, so
    // just grab a reference to it that we use without holding the mutex.
    MutexLock lock(Thread::Current(), dex_to_dex_references_lock_);
    dex_to_dex_references = ArrayRef<DexFileMethodSet>(dex_to_dex_references_);
  }
  for (const auto& method_set : dex_to_dex_references) {
    current_dex_to_dex_methods_ = &method_set.GetMethodIndexes();
    CompileDexFile(class_loader,
                   method_set.GetDexFile(),
                   dex_files,
                   parallel_thread_pool_.get(),
                   parallel_thread_count_,
                   timings);
  }
  current_dex_to_dex_methods_ = nullptr;

  VLOG(compiler) << "Compile: " << GetMemoryUsageString(false);
}

CompilerDriver::CompileDexFile

Android M版的CompileDexFile:

void CompilerDriver::CompileDexFile(jobject class_loader, const DexFile& dex_file,
                                    const std::vector<const DexFile*>& dex_files,
                                    ThreadPool* thread_pool, TimingLogger* timings) {
  TimingLogger::ScopedTiming t("Compile Dex File", timings);
  ParallelCompilationManager context(Runtime::Current()->GetClassLinker(), class_loader, this,
                                     &dex_file, dex_files, thread_pool);
  context.ForAll(0, dex_file.NumClassDefs(), CompilerDriver::CompileClass, thread_count_);
}

Android N版本增加了一个CompileClassVisitor，访问者模式啊，更加上档次了。

void CompilerDriver::CompileDexFile(jobject class_loader,
                                    const DexFile& dex_file,
                                    const std::vector<const DexFile*>& dex_files,
                                    ThreadPool* thread_pool,
                                    size_t thread_count,
                                    TimingLogger* timings) {
  TimingLogger::ScopedTiming t("Compile Dex File", timings);
  ParallelCompilationManager context(Runtime::Current()->GetClassLinker(), class_loader, this,
                                     &dex_file, dex_files, thread_pool);
  CompileClassVisitor visitor(&context);
  context.ForAll(0, dex_file.NumClassDefs(), &visitor, thread_count);
}

CompileClassVisitor

虽然在Android N上改用了CompileClassVisitor，但是本质上还是跟Android M上的CompilerDriver::CompileClass是一回事情。最终还是要落实到CompileMethod方法上去。
细节就略过不分析了。
我们先过一下CompileClassVisitor的逻辑：

  virtual void Visit(size_t class_def_index) REQUIRES(!Locks::mutator_lock_) OVERRIDE {
    ATRACE_CALL();
    const DexFile& dex_file = *manager_->GetDexFile();
    const DexFile::ClassDef& class_def = dex_file.GetClassDef(class_def_index);
    ClassLinker* class_linker = manager_->GetClassLinker();
    jobject jclass_loader = manager_->GetClassLoader();
    ClassReference ref(&dex_file, class_def_index);
    // Skip compiling classes with generic verifier failures since they will still fail at runtime
    if (manager_->GetCompiler()->verification_results_->IsClassRejected(ref)) {
      return;
    }
    // Use a scoped object access to perform to the quick SkipClass check.
    const char* descriptor = dex_file.GetClassDescriptor(class_def);
    ScopedObjectAccess soa(Thread::Current());
    StackHandleScope<3> hs(soa.Self());
    Handle<mirror::ClassLoader> class_loader(
        hs.NewHandle(soa.Decode<mirror::ClassLoader*>(jclass_loader)));
    Handle<mirror::Class> klass(
        hs.NewHandle(class_linker->FindClass(soa.Self(), descriptor, class_loader)));
    Handle<mirror::DexCache> dex_cache;
    if (klass.Get() == nullptr) {
      soa.Self()->AssertPendingException();
      soa.Self()->ClearException();
      dex_cache = hs.NewHandle(class_linker->FindDexCache(soa.Self(), dex_file));
    } else if (SkipClass(jclass_loader, dex_file, klass.Get())) {
      return;
    } else {
      dex_cache = hs.NewHandle(klass->GetDexCache());
    }

    const uint8_t* class_data = dex_file.GetClassData(class_def);
    if (class_data == nullptr) {
      // empty class, probably a marker interface
      return;
    }

    // Go to native so that we don't block GC during compilation.
    ScopedThreadSuspension sts(soa.Self(), kNative);

    CompilerDriver* const driver = manager_->GetCompiler();

    // Can we run DEX-to-DEX compiler on this class ?
    optimizer::DexToDexCompilationLevel dex_to_dex_compilation_level =
        GetDexToDexCompilationLevel(soa.Self(), *driver, jclass_loader, dex_file, class_def);

    ClassDataItemIterator it(dex_file, class_data);
    // Skip fields
    while (it.HasNextStaticField()) {
      it.Next();
    }
    while (it.HasNextInstanceField()) {
      it.Next();
    }

    bool compilation_enabled = driver->IsClassToCompile(
        dex_file.StringByTypeIdx(class_def.class_idx_));

    // Compile direct methods
    int64_t previous_direct_method_idx = -1;
    while (it.HasNextDirectMethod()) {
      uint32_t method_idx = it.GetMemberIndex();
      if (method_idx == previous_direct_method_idx) {
        // smali can create dex files with two encoded_methods sharing the same method_idx
        // http://code.google.com/p/smali/issues/detail?id=119
        it.Next();
        continue;
      }
      previous_direct_method_idx = method_idx;
      CompileMethod(soa.Self(), driver, it.GetMethodCodeItem(), it.GetMethodAccessFlags(),
                    it.GetMethodInvokeType(class_def), class_def_index,
                    method_idx, jclass_loader, dex_file, dex_to_dex_compilation_level,
                    compilation_enabled, dex_cache);
      it.Next();
    }
    // Compile virtual methods
    int64_t previous_virtual_method_idx = -1;
    while (it.HasNextVirtualMethod()) {
      uint32_t method_idx = it.GetMemberIndex();
      if (method_idx == previous_virtual_method_idx) {
        // smali can create dex files with two encoded_methods sharing the same method_idx
        // http://code.google.com/p/smali/issues/detail?id=119
        it.Next();
        continue;
      }
      previous_virtual_method_idx = method_idx;
      CompileMethod(soa.Self(), driver, it.GetMethodCodeItem(), it.GetMethodAccessFlags(),
                    it.GetMethodInvokeType(class_def), class_def_index,
                    method_idx, jclass_loader, dex_file, dex_to_dex_compilation_level,
                    compilation_enabled, dex_cache);
      it.Next();
    }
    DCHECK(!it.HasNext());
  }

 private:
  const ParallelCompilationManager* const manager_;
};

然后会调到CompilerDriver中的static函数CompileMethod。