上篇文章简析了AGP插件的初始化过程,本篇将会讲解一下AGP的代码编译过程。
任务创建
上篇文章中我们曾经介绍过构建的核心任务都是在TaskManager
中创建的,代码如下:
public void createTasksForVariantScope(@NonNull final VariantScope variantScope) {
//省略部分代码...
// Add a compile task
createCompileTask(variantScope);
}
在项目里面Java代码的编译由compile${variantName}JavaWithJavac
任务来完成,而这个任务就是在createCompileTask
方法里面被创建出来,代码如下:
protected void createCompileTask(@NonNull VariantScope variantScope) {
JavaCompile javacTask = createJavacTask(variantScope);
//省略部分代码...
}
public JavaCompile createJavacTask(@NonNull final VariantScope scope) {
JavaPreCompileTask preCompileTask =
taskFactory.create(new JavaPreCompileTask.ConfigAction(scope));
preCompileTask.dependsOn(scope.getTaskContainer().getPreBuildTask());
//创建JavaCompile Task
final JavaCompile javacTask = taskFactory.create(new JavaCompileConfigAction(scope));
scope.getTaskContainer().setJavacTask(javacTask);
setupCompileTaskDependencies(scope, javacTask);
postJavacCreation(scope);
return javacTask;
}
代码也是比较简单,大概是创建一个JavaCompile Task,然后设置它的依赖关系,通过查看JavaCompileConfigAction
的源码,我们可以得到这个JavaCompile Task本质是个AndroidJavaCompile
类型对象,代码如下
public class JavaCompileConfigAction implements TaskConfigAction<AndroidJavaCompile> {
//省略部分代码...
@NonNull
@Override
public String getName() {
return scope.getTaskName("compile", "JavaWithJavac");
}
@NonNull
@Override
public Class<AndroidJavaCompile> getType() {
return AndroidJavaCompile.class;
}
@Override
public void execute(@NonNull final AndroidJavaCompile javacTask) {
//省略部分代码...
}
}
通过getName
方法我们可以知道,JavaCompile Task的名称格式是compile${variantName}JavaWithJavac
,类型是AndroidJavaCompile
,AndroidJavaCompile
继承于JavaCompile
,其中核心的Java编译工作都是由后者完成的。
Java编译过程
JavaCompile
的compile
方法是Java代码编译的入口,当我们在AS里面点击Run
又或者双击compile${variantName}JavaWithJavac
Task会跑到compile
方法里,代码如下:
protected void compile(IncrementalTaskInputs inputs) {
if (!compileOptions.isIncremental()) {
compile();
return;
}
DefaultJavaCompileSpec spec = createSpec();
Compiler<JavaCompileSpec> incrementalCompiler = getIncrementalCompilerFactory().makeIncremental(
createCompiler(spec),
getPath(),
(IncrementalTaskInputsInternal) inputs,
source,
getEffectiveAnnotationProcessorPath()
);
performCompilation(spec, incrementalCompiler);
}
首先会先判断是否为增量编译,如果不是直接走全量编译,因为全量编译过程比较简单,这里我们只分析它的增量编译,IncrementalTaskInputs
是Gradle提供的API,用来检测文件的修改,这里不多做介绍,我们重点关注makeIncremental
跟createCompiler
,前者返回一个Compiler
对象,并且最终是调用这个对象的execute
来编译代码的,但事实上真正的代码编译是由createCompiler
创建出来的对象来执行的,前者通过代理模式,创建了一些列的代理Compiler
来处理一些编译前工作。
makeIncremental
是IncrementalCompilerFactory对象方法,它代码如下:
public <T extends JavaCompileSpec> Compiler<T> makeIncremental(CleaningJavaCompilerSupport<T> cleaningJavaCompiler, String taskPath, FileTree sources, RecompilationSpecProvider recompilationSpecProvider) {
TaskScopedCompileCaches compileCaches = createCompileCaches(taskPath);
Compiler<T> rebuildAllCompiler = createRebuildAllCompiler(cleaningJavaCompiler, sources);
ClassDependenciesAnalyzer analyzer = new CachingClassDependenciesAnalyzer(new DefaultClassDependenciesAnalyzer(interner), compileCaches.getClassAnalysisCache());
ClasspathEntrySnapshotter classpathEntrySnapshotter = new CachingClasspathEntrySnapshotter(fileHasher, streamHasher, fileSystemSnapshotter, analyzer, compileCaches.getClasspathEntrySnapshotCache(), fileOperations);
ClasspathSnapshotMaker classpathSnapshotMaker = new ClasspathSnapshotMaker(new ClasspathSnapshotFactory(classpathEntrySnapshotter, buildOperationExecutor));
PreviousCompilationOutputAnalyzer previousCompilationOutputAnalyzer = new PreviousCompilationOutputAnalyzer(fileHasher, streamHasher, analyzer, fileOperations);
IncrementalCompilerDecorator<T> incrementalSupport = new IncrementalCompilerDecorator(classpathSnapshotMaker, compileCaches, cleaningJavaCompiler, rebuildAllCompiler, previousCompilationOutputAnalyzer, interner);
return incrementalSupport.prepareCompiler(recompilationSpecProvider);
}
通过上面的代码我们可以看见Compiler是被IncrementalCompilerDecorator
对象的prepareCompiler
方法创建出来的,prepareCompiler
方法源码如下:
public Compiler<T> prepareCompiler(RecompilationSpecProvider recompilationSpecProvider) {
Compiler<T> compiler = getCompiler(recompilationSpecProvider);
return new IncrementalResultStoringCompiler(compiler, classpathSnapshotMaker, compileCaches.getPreviousCompilationStore(), interner);
}
可以看到最终返回给JavaCompile的其实是个IncrementalResultStoringCompiler
对象,IncrementalResultStoringCompiler顾名思义它在Java代码的编译过程中只是用来存储编译结果的,当JavaCompile调用IncrementalResultStoringCompiler
的execute
方法来编译Java代码时,实际上会把编译任务交给了delegate
来完成,代码如下
public WorkResult execute(T spec) {
WorkResult result = delegate.execute(spec);
if (result instanceof RecompilationNotNecessary) {
return result;
} else {
storeResult(spec, result);
return result;
}
}
这个delegate
对象是通过前面的getCompiler
方法创建出来的,代码如下:
//创建delegate对象.
private Compiler<T> getCompiler(RecompilationSpecProvider recompilationSpecProvider) {
if (!recompilationSpecProvider.isIncremental()) {
LOG.info("Full recompilation is required because no incremental change information is available. This is usually caused by clean builds or changing compiler arguments.");
return rebuildAllCompiler;
} else {
PreviousCompilationData data = compileCaches.getPreviousCompilationStore().get();
if (data == null) {
LOG.info("Full recompilation is required because no previous compilation result is available.");
return rebuildAllCompiler;
} else {
PreviousCompilation previousCompilation = new PreviousCompilation(data, compileCaches.getClasspathEntrySnapshotCache(), previousCompilationOutputAnalyzer);
return new SelectiveCompiler(previousCompilation, cleaningCompiler, rebuildAllCompiler, recompilationSpecProvider, classpathSnapshotMaker);
}
}
}
这个delegate其实就是个SelectiveCompiler
对象,编译java代码时会调用它的execute
方法,代码如下:
public WorkResult execute(T spec) {
// 省略全量编译判断代码...
RecompilationSpec recompilationSpec = recompilationSpecProvider.provideRecompilationSpec(currentCompilation, previousCompilation);
if (recompilationSpec.isFullRebuildNeeded()) {
LOG.info("Full recompilation is required because {}. Analysis took {}.", recompilationSpec.getFullRebuildCause(), clock.getElapsed());
return rebuildAllCompiler.execute(spec);
}
recompilationSpecProvider.initializeCompilation(spec, recompilationSpec);
if (Iterables.isEmpty(spec.getSourceFiles()) && spec.getClasses().isEmpty()) {
LOG.info("None of the classes needs to be compiled! Analysis took {}. ", clock.getElapsed());
return new RecompilationNotNecessary();
}
try {
return recompilationSpecProvider.decorateResult(recompilationSpec, cleaningCompiler.getCompiler().execute(spec));
} finally {
Collection<String> classesToCompile = recompilationSpec.getClassesToCompile();
LOG.info("Incremental compilation of {} classes completed in {}.", classesToCompile.size(), clock.getElapsed());
LOG.debug("Recompiled classes {}", classesToCompile);
}
}
很显然的SelectiveCompiler
本身也只是个代理对象,最终它会调用cleaningCompiler
的execute
去编译代码,这个地方我们一会再分析,这个地方我们重点关心两个方法,一个是provideRecompilationSpec
,另外一个是initializeCompilation
,他们都是由RecompilationSpecProvider
接口提供的,而JavaRecompilationSpecProvider
类实现了这个接口,先看看provideRecompilationSpec
,代码如下:
//org.gradle.api.internal.tasks.compile.incremental.recomp.JavaRecompilationSpecProvider
/**
* processClasspathChanges是用来检测classpath是否有被修改的
* 而processOtherChanges就是用来检测Java源码的修改的
*/
public RecompilationSpec provideRecompilationSpec(CurrentCompilation current, PreviousCompilation previous) {
RecompilationSpec spec = new RecompilationSpec();
processClasspathChanges(current, previous, spec);
processOtherChanges(current, previous, spec);
spec.getClassesToProcess().addAll(previous.getTypesToReprocess());
return spec;
}
processClasspathChanges
用来负责检查classpath的改动,这里我们只分析processOtherChanges
,它负责了代码修改的检测以及类依赖关系检索等等工作,源码如下:
private void processOtherChanges(CurrentCompilation current, PreviousCompilation previous, RecompilationSpec spec) {
SourceFileChangeProcessor javaChangeProcessor = new SourceFileChangeProcessor(previous);
AnnotationProcessorChangeProcessor annotationProcessorChangeProcessor = new AnnotationProcessorChangeProcessor(current, previous);
ResourceChangeProcessor resourceChangeProcessor = new ResourceChangeProcessor(current.getAnnotationProcessorPath());
InputChangeAction action = new InputChangeAction(spec, javaChangeProcessor, annotationProcessorChangeProcessor, resourceChangeProcessor, sourceFileClassNameConverter);
inputs.outOfDate(action);
inputs.removed(action);
}
processOtherChanges
里面首先创建了三个Processor用来处理发生了变化的源文件,InputChangeAction
实现了Gradle的Action
接口,outOfDate
removed
都是Gradle提供的API,用来回调被增删修改文件的,InputChangeAction
的代码大概如下:
class InputChangeAction implements Action<InputFileDetails> {
// 省略部分代码...
public void execute(InputFileDetails input) {
if (spec.getFullRebuildCause() == null) {
File file = input.getFile();
if (FileUtils.hasExtension(file, ".java")) {
Collection<String> classNames = sourceFileClassNameConverter.getClassNames(file);
if (classNames.isEmpty()) {
spec.setFullRebuildCause("source dirs are changed", file);
} else {
javaChangeProcessor.processChange(file, classNames, spec);
}
} else if (!FileUtils.hasExtension(file, ".jar") && !FileUtils.hasExtension(file, ".class")) {
resourceChangeProcessor.processChange(input, spec);
} else {
annotationProcessorChangeProcessor.processChange(input, spec);
}
}
}
}
当有源文件被修改了编译时会回调execute
方法,input.getFile()
会返回被修改了的文件的绝对路径,sourceFileClassNameConverter
负责把/
分隔的文件路径转换为.
分隔的类的绝对路径,这里我们重点关心javaChangeProcessor
,它是SourceFileChangeProcessor
类型对象,主要负责检索代码的依赖关系的,processChange
方法源码如下:
public void processChange(File inputFile, Collection<String> classNames, RecompilationSpec spec) {
//把当前被修改文件添加到待编译文件list里面
spec.getClassesToCompile().addAll(classNames);
for (String className : classNames) {
//获取类的依赖关系.
DependentsSet actualDependents = previousCompilation.getDependents(className, IntSets.EMPTY_SET);
if (actualDependents.isDependencyToAll()) {
spec.setFullRebuildCause(actualDependents.getDescription(), inputFile);
return;
}
spec.getClassesToCompile().addAll(actualDependents.getDependentClasses());
spec.getResourcesToGenerate().addAll(actualDependents.getDependentResources());
}
}
通过调用PreviousCompilation
的getDependents
方法获取被修改类的所有依赖,后者其实又调用了ClassSetAnalysis
的getRelevantDependents
方法来获取类依赖,其内部又会调用recurseDependents
方法来深度递归 获取依赖的依赖 把所有依赖都获取出来,代码如下:
public DependentsSet getRelevantDependents(String className, IntSet constants) {
String fullRebuildCause = annotationProcessingData.getFullRebuildCause();
if (fullRebuildCause != null) {
return DependentsSet.dependencyToAll(fullRebuildCause);
}
DependentsSet deps = getDependents(className);
if (deps.isDependencyToAll()) {
return deps;
}
if (!constants.isEmpty()) {
return DependentsSet.dependencyToAll();
}
Set<String> classesDependingOnAllOthers = annotationProcessingData.getGeneratedTypesDependingOnAllOthers();
Set<GeneratedResource> resourcesDependingOnAllOthers = annotationProcessingData.getGeneratedResourcesDependingOnAllOthers();
if (deps.getDependentClasses().isEmpty() && classesDependingOnAllOthers.isEmpty() && resourcesDependingOnAllOthers.isEmpty()) {
return deps;
}
Set<String> resultClasses = new HashSet<String>();
Set<GeneratedResource> resultResources = new HashSet<GeneratedResource>(resourcesDependingOnAllOthers);
//recurseDependentClasses内部会递归的调用, 直到把整个依赖链给检索出来
recurseDependentClasses(new HashSet<String>(), resultClasses, resultResources, deps.getDependentClasses());
recurseDependentClasses(new HashSet<String>(), resultClasses, resultResources, classesDependingOnAllOthers);
resultClasses.remove(className);
return DependentsSet.dependents(resultClasses, resultResources);
}
关于类的依赖的获取ClassSetAnalysis
本质也是调用了ClassSetAnalysisData
对象的getDependents
方法去获取的,ClassSetAnalysisData
会解析.class
结构,从常量池里检索出依赖的类对象来,常量池里面类型为7的就是class类型,代码并不多,如果比较熟悉.class
文件结构的话很容易就读懂,这里就不再分析了。
回到SelectiveCompiler
的execute
方法里,第一步通过调用JavaRecompilationSpecProvider
对象的provideRecompilationSpec
方法,把被修改的java文件,以及它的依赖文件全部都被检索出来了,接着会继续调用它的initializeCompilation
方法来初始化一些编译前工作,包括设置classpath、清理被删除文件、把以.
分隔的类路径转换为以/
分隔的文件绝对路径等等,代码如下:
public void initializeCompilation(JavaCompileSpec spec, RecompilationSpec recompilationSpec) {
//省略部分代码...
Factory<PatternSet> patternSetFactory = fileOperations.getFileResolver().getPatternSetFactory();
PatternSet classesToDelete = patternSetFactory.create();
PatternSet sourceToCompile = patternSetFactory.create();
prepareJavaPatterns(recompilationSpec.getClassesToCompile(), classesToDelete, sourceToCompile);
//把com.nls.exapmle.Test.java格式的类路径转换为com/nls/exapmle/Test.java文件路径.
spec.setSourceFiles(narrowDownSourcesToCompile(sourceTree, sourceToCompile));
//设置classpath,主要是把build/intermediates/javac/debug/compileDebugJavaWithJavac路径添加进来.
includePreviousCompilationOutputOnClasspath(spec);
addClassesToProcess(spec, recompilationSpec);
//处理被删文件
deleteStaleFilesIn(classesToDelete, spec.getDestinationDir());
deleteStaleFilesIn(classesToDelete, spec.getCompileOptions().getAnnotationProcessorGeneratedSourcesDirectory());
deleteStaleFilesIn(classesToDelete, spec.getCompileOptions().getHeaderOutputDirectory());
//省略部分代码...
}
SelectiveCompiler
最后通过调用cleaningCompiler.getCompiler()
的execute
方法来编译代码,这个cleaningCompiler
对象是在JavaCompile
的compile
里面被创建出来的,代码是在JavaCompile.createCompiler()
里,源码如下:
private CleaningJavaCompiler createCompiler(JavaCompileSpec spec) {
Compiler<JavaCompileSpec> javaCompiler = CompilerUtil.castCompiler(((JavaToolChainInternal)getToolChain()).select(this.getPlatform()).newCompiler(spec.getClass()));
return new CleaningJavaCompiler(javaCompiler, getOutputs());
}
可以看到这个cleaningCompiler
就是CleaningJavaCompiler
类对象,但是CleaningJavaCompiler
也仅仅是个代理对象,它并不参与代码的真正编译工作,java代码的编译工作是由newCompiler
方法创建出来的对象来完成的,newCompiler
是ToolProvider
的一个接口,JavaToolProvider
实现了这个接口,负责提供java编译所需要的compiler,代码是在org.gradle.api.internal.tasks.AbstractJavaToolChain
类里,代码如下:
private class JavaToolProvider implements ToolProvider {
//省略部分代码...
public <T extends CompileSpec> Compiler<T> newCompiler(Class<T> spec) {
if (JavaCompileSpec.class.isAssignableFrom(spec)) {
Compiler<T> compiler = AbstractJavaToolChain.this.compilerFactory.create(spec);
return compiler;
} else if (JavadocSpec.class.isAssignableFrom(spec)) {
Compiler<T> compilerx = new JavadocGenerator(AbstractJavaToolChain.this.execActionFactory);
return compilerx;
} else {
throw new IllegalArgumentException(String.format("Don't know how to compile using spec of type %s.", spec.getClass().getSimpleName()));
}
}
//省略部分代码...
}
JavaToolProvider
里面会根据不同的类型创建对应的compiler,这里我们的spec是JavaCompileSpec
类型,所以最终会通过JavaCompilerFactory
接口的create方法来创建出compiler,DefaultJavaCompilerFactory
类实现了这个接口,代码如下:
public Compiler<JavaCompileSpec> create(Class<? extends CompileSpec> type) {
Compiler<JavaCompileSpec> result = createTargetCompiler(type, false);
return new AnnotationProcessorDiscoveringCompiler(new NormalizingJavaCompiler(result), processorDetector);
}
private Compiler<JavaCompileSpec> createTargetCompiler(Class<? extends CompileSpec> type, boolean jointCompilation) {
if (!JavaCompileSpec.class.isAssignableFrom(type)) {
throw new IllegalArgumentException(String.format("Cannot create a compiler for a spec with type %s", type.getSimpleName()));
} else if (CommandLineJavaCompileSpec.class.isAssignableFrom(type)) {
return new CommandLineJavaCompiler(execHandleFactory);
} else {
return (Compiler)(ForkingJavaCompileSpec.class.isAssignableFrom(type) && !jointCompilation ? new DaemonJavaCompiler(workingDirProvider.getWorkingDirectory(), JdkJavaCompiler.class, new Object[]{javaHomeBasedJavaCompilerFactory}, workerDaemonFactory, forkOptionsFactory, classPathRegistry, actionExecutionSpecFactory) : new JdkJavaCompiler(javaHomeBasedJavaCompilerFactory));
}
}
最终返回的compiler是AnnotationProcessorDiscoveringCompiler
类型对象,但AnnotationProcessorDiscoveringCompiler
也仅仅是个代理对象,真正的java编译compiler对象是下面的createTargetCompiler
方法创建出来的,最终被创建出来的compiler是JdkJavaCompiler
类型的对象,这个对象是真正用来编译java代码的compiler,它的execute方法代码如下:
public WorkResult execute(JavaCompileSpec spec) {
LOGGER.info("Compiling with JDK Java compiler API.");
JdkJavaCompilerResult result = new JdkJavaCompilerResult();
JavaCompiler.CompilationTask task = createCompileTask(spec, result);
boolean success = task.call();
if (!success) {
throw new CompilationFailedException();
}
return result;
}
代码也比较简单,首先创建一个CompilationTask
,这是JDK
提供的一个接口,最后通过调用CompilationTask
的call
方法来编译java代码,createCompileTask
方法代码如下:
private JavaCompiler.CompilationTask createCompileTask(JavaCompileSpec spec, JdkJavaCompilerResult result) {
//构造编译参数
List<String> options = new JavaCompilerArgumentsBuilder(spec).build();
//创建compiler
JavaCompiler compiler = javaHomeBasedJavaCompilerFactory.create();
MinimalJavaCompileOptions compileOptions = spec.getCompileOptions();
Charset charset = compileOptions.getEncoding() != null ? Charset.forName(compileOptions.getEncoding()) : null;
StandardJavaFileManager standardFileManager = compiler.getStandardFileManager(null, null, charset);
Iterable<? extends JavaFileObject> compilationUnits = standardFileManager.getJavaFileObjectsFromFiles(spec.getSourceFiles());
boolean hasEmptySourcepaths = JavaVersion.current().isJava9Compatible() && emptySourcepathIn(options);
JavaFileManager fileManager = GradleStandardJavaFileManager.wrap(standardFileManager, DefaultClassPath.of(spec.getAnnotationProcessorPath()), hasEmptySourcepaths);
//创建task
JavaCompiler.CompilationTask task = compiler.getTask(null, fileManager, null, options, spec.getClasses(), compilationUnits);
//省略部分代码...
return task;
}
createCompileTask
方法里调用的大多数是JDK
提供的API了,这里值得关注的就是JavaCompiler
,它是JDK
提供的一个接口,用来负责编译java代码的,它是被JavaHomeBasedJavaCompilerFactory
类的create
方法创建出来,代码如下:
public JavaCompiler create() {
JavaCompiler compiler = findCompiler();
if (compiler == null) {
throw new RuntimeException("Cannot find System Java Compiler. Ensure that you have installed a JDK (not just a JRE) and configured your JAVA_HOME system variable to point to the according directory.");
}
return compiler;
}
private JavaCompiler findCompiler() {
File realJavaHome = currentJvmJavaHomeFactory.create();
return SystemProperties.getInstance().withJavaHome(realJavaHome, systemJavaCompilerFactory);
}
currentJvmJavaHomeFactory
对应的类型是CurrentJvmJavaHomeFactory
,而systemJavaCompilerFactory
对应的类型是SystemJavaCompilerFactory
,定义如下:
public static class CurrentJvmJavaHomeFactory implements Factory<File>, Serializable {
@Override
public File create() {
return Jvm.current().getJavaHome();
}
}
public static class SystemJavaCompilerFactory implements Factory<JavaCompiler>, Serializable {
@Override
public JavaCompiler create() {
return JdkTools.current().getSystemJavaCompiler();
}
}
currentJvmJavaHomeFactory
以及SystemProperties
的withJavaHome
方法会从环境变量配置里面读取JDK
的配置路径,SystemJavaCompilerFactory
则是负责创建JavaCompiler
,JdkTools
会通过反射把JavaCompiler
创建出来,代码大概如下:
public JavaCompiler getSystemJavaCompiler() {
Class<?> clazz;
try {
if (isJava9Compatible) {
clazz = isolatedToolsLoader.loadClass("javax.tools.ToolProvider");
try {
return (JavaCompiler) clazz.getDeclaredMethod("getSystemJavaCompiler").invoke(null);
} catch (IllegalAccessException e) {
cannotCreateJavaCompiler(e);
} catch (InvocationTargetException e) {
cannotCreateJavaCompiler(e);
} catch (NoSuchMethodException e) {
cannotCreateJavaCompiler(e);
}
} else {
clazz = isolatedToolsLoader.loadClass(DEFAULT_COMPILER_IMPL_NAME);
}
} catch (ClassNotFoundException e) {
throw new IllegalStateException("Could not load class '" + DEFAULT_COMPILER_IMPL_NAME);
}
return DirectInstantiator.instantiate(clazz.asSubclass(JavaCompiler.class));
}
整个创建流程大致如下:
最后就是通过JDK提供的JavaCompiler接口来编译java代码。
总结
-
AGP会在apply时创建出compile${variantName}JavaWithJavac任务,负责用来编译java代码,这个任务本质上是个AndroidJavaCompile类型对象,创建过程是在TaskManager里的createTasksForVariantScope方法里。
-
AndroidJavaCompile被执行的时候会调用IncrementalCompilerFactory的makeIncremental方法来创建出compiler,compiler会被多重代理,最终负责编译java代码的是在JdkTools里面创建出来的JavaCompiler,它是JDK提供的接口。除了创建出compiler以外,AndroidJavaCompile还会调用JavaRecompilationSpecProvider的provideRecompilationSpec跟initializeCompilation方法,来检索出所有需要编译的java源文件。
网友评论