美文网首页
presto(一) -- 启动Server流程

presto(一) -- 启动Server流程

作者: lyoungzzz | 来源:发表于2018-08-24 17:39 被阅读329次

Server(presto-0.207)启动涉及到的代码不是很多(这里只讨论大致的执行流程,不涉及具体的实现细节),主要是关于Plugins的加载,读配置文件.properties,进行相应的操作。
首先,根据presto日志server.log定位服务启动的入口。
INFO main com.facebook.presto.server.PrestoServer ======== SERVER STARTED ========
启动类:presto-main/src/main/java/com/facebook/presto/server/PrestoServer.java
server启动的所有流程均在run()中实现:
首先,虚拟机参数相关验证,比如:是否JDK8+、JDK厂商:Oracle or OpenJDK、64-bit JVM版本、x86_64 on Mac OS,或者x86-64、amd64 on Linux系统,是否采用G1的垃圾回收器等。系统时间要求:必须是当前时间。

verifyJvmRequirements();
verifySystemTimeIsReasonable();

接着,创建不可变集合ImmutableList,传入一些启动模块,具体的实现细节,后续有需要再分析。

ImmutableList.Builder<Module> modules = ImmutableList.builder();
        modules.add(
                new NodeModule(),
                new DiscoveryModule(),
                new HttpServerModule(),
                new JsonModule(),
                new JaxrsModule(true),
                new MBeanModule(),
                new JmxModule(),
                new JmxHttpModule(),
                new LogJmxModule(),
                new TraceTokenModule(),
                new JsonEventModule(),
                new HttpEventModule(),
                new ServerSecurityModule(),
                new AccessControlModule(),
                new EventListenerModule(),
                new ServerMainModule(sqlParserOptions),
                new GracefulShutdownModule());

        modules.addAll(getAdditionalModules());
        //使用io.airlift.平台代码库构建的应用程序的入口点,并加载需要的module实例(包含一些启动默认参数)
        Bootstrap app = new Bootstrap(modules.build());

关于启动的默认参数比较多,现截取一部分:

2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   PROPERTY                                                              DEFAULT                           RUNTIME                                  DESCRIPTION
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   event.max-output-stage-size                                           16MB                              16MB
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.client.timeout                                                  5.00m                             5.00m
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.initial-hash-partitions                                         100                               100
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query-manager.initialization-required-workers                         1                                 1                                        Minimum number of workers that must be available before the cluster will accept queries
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query-manager.initialization-timeout                                  5.00m                             5.00m                                    After this time, the cluster will accept queries even if the minimum required workers are not available
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.max-concurrent-queries                                          1000                              1000
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.max-history                                                     100                               100
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.max-length                                                      1000000                           1000000
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.max-queued-queries                                              5000                              5000
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.min-expire-age                                                  15.00m                            15.00m
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.min-schedule-split-batch-size                                   100                               100
2018-08-24T10:54:27.769+0800    INFO    main    Bootstrap   query.execution-policy                                                all-at-once                       all-at-once
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.manager-executor-pool-size                                      5                                 5
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.max-cpu-time                                                    1000000000.00d                    1000000000.00d
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.max-execution-time                                              100.00d                           100.00d
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.max-run-time                                                    100.00d                           100.00d
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.remote-task.max-callback-threads                                1000                              1000
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.remote-task.max-error-duration                                  5.00m                             5.00m
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.remote-task.min-error-duration                                  5.00m                             5.00m
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   query.schedule-split-batch-size                                       1000                              1000
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.client.timeout                                                   2.00m                             2.00m
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.http-response-threads                                            100                               100
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.http-timeout-threads                                             3                                 3
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.info.max-age                                                     15.00m                            15.00m
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.info-update-interval                                             3.00s                             3.00s                                    Interval between updating task data
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.initial-splits-per-node                                          64                                6
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.level-time-multiplier                                            2                                 2                                        Factor that determines the target scheduled time for a level relative to the next
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.max-drivers-per-task                                             2147483647                        2147483647                               Maximum number of drivers a task can run
2018-08-24T10:54:27.770+0800    INFO    main    Bootstrap   task.max-index-memory                                                 64MB                              64MB
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.max-local-exchange-buffer-size                                   32MB                              32MB
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   driver.max-page-partitioning-buffer-size                              32MB                              32MB
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.max-partial-aggregation-memory                                   16MB                              16MB
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.max-worker-threads                                               64                                6
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.min-drivers                                                      128                               12
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.min-drivers-per-task                                             3                                 3                                        Minimum number of drivers guaranteed to run per task given there is sufficient work to do
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.share-index-loading                                              false                             false
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   sink.max-buffer-size                                                  32MB                              1GB
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.split-concurrency-adjustment-interval                            100.00ms                          100.00ms
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.status-refresh-max-wait                                          1.00s                             1.00s
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.concurrency                                                      16                                16                                       Default number of local parallel jobs per worker
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.cpu-timer-enabled                                                true                              true
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.task-notification-threads                                        5                                 5                                        Number of threads used for internal task event notifications
2018-08-24T10:54:27.771+0800    INFO    main    Bootstrap   task.task-yield-threads                                               3                                 3                                        Number of threads used for setting yield signals
2018-08-24T10:54:27.772+0800    INFO    main    Bootstrap   task.verbose-stats                                                    false                             false

开始加载plugins和配置文件

try {
            Injector injector = app.strictConfig().initialize();
            //生成PluginManager实例,载入/data0/presto/data/plugin目录下的.jar
            injector.getInstance(PluginManager.class).loadPlugins();
            //生成StaticCatalogStore实例,载入.properties配置文件
            injector.getInstance(StaticCatalogStore.class).loadCatalogs();

            // TODO: remove this huge hack
            //更新UUID, ServiceAnnouncement
            updateConnectorIds(
                    injector.getInstance(Announcer.class),
                    injector.getInstance(CatalogManager.class),
                    injector.getInstance(ServerConfig.class),
                    injector.getInstance(NodeSchedulerConfig.class));
            //接口,无具体方法实现
            injector.getInstance(SessionSupplier.class).loadConfigurationManager();
            injector.getInstance(ResourceGroupManager.class).loadConfigurationManager();
            //生成AccessControlManager实例,如果路径etc/下存在文件access-control.properties
            //如果检查通过,或者不存在该文件,打印日志信息"-- Loaded system access control allow-all --"
            injector.getInstance(AccessControlManager.class).loadSystemAccessControl();
            //原理过程大致同上
            injector.getInstance(PasswordAuthenticatorManager.class).loadPasswordAuthenticator();
            injector.getInstance(EventListenerManager.class).loadConfiguredEventListener();

            injector.getInstance(Announcer.class).start();

            log.info("======== SERVER STARTED ========");
        }
        catch (Throwable e) {
            log.error(e);
            System.exit(1);
        }
    }

关于AccessControlManager.class下的loadSystemAccessControl()代码分析:

public void loadSystemAccessControl()
            throws Exception
    {
        if (ACCESS_CONTROL_CONFIGURATION.exists()) {
            Map<String, String> properties = new HashMap<>(loadProperties(ACCESS_CONTROL_CONFIGURATION));

            String accessControlName = properties.remove(ACCESS_CONTROL_PROPERTY_NAME);
            checkArgument(!isNullOrEmpty(accessControlName),
                    "Access control configuration %s does not contain %s", ACCESS_CONTROL_CONFIGURATION.getAbsoluteFile(), ACCESS_CONTROL_PROPERTY_NAME);

            setSystemAccessControl(accessControlName, properties);
        }
        else {
            setSystemAccessControl(AllowAllSystemAccessControl.NAME, ImmutableMap.of());
        }
    }

    @VisibleForTesting
    protected void setSystemAccessControl(String name, Map<String, String> properties)
    {
        requireNonNull(name, "name is null");
        requireNonNull(properties, "properties is null");

        checkState(systemAccessControlLoading.compareAndSet(false, true), "System access control already initialized");

        log.info("-- Loading system access control --");

        SystemAccessControlFactory systemAccessControlFactory = systemAccessControlFactories.get(name);
        checkState(systemAccessControlFactory != null, "Access control %s is not registered", name);

        SystemAccessControl systemAccessControl = systemAccessControlFactory.create(ImmutableMap.copyOf(properties));
        this.systemAccessControl.set(systemAccessControl);

        log.info("-- Loaded system access control %s --", name);
    }

由于在集群中没有配置该文件,启动时默认allow-all,直接通过检查。后面两步的加载原理大致也是一样。运行到这里,就可以启动server了。

injector.getInstance(Announcer.class).start();
log.info("======== SERVER STARTED ========");

相关文章

网友评论

      本文标题:presto(一) -- 启动Server流程

      本文链接:https://www.haomeiwen.com/subject/sgsbiftx.html