美文网首页超级大数据
storm(基础二)Scheduler(调度器)

storm(基础二)Scheduler(调度器)

作者: 裘马轻狂大帅 | 来源:发表于2019-04-26 08:49 被阅读1次

    注意:谷歌翻译 仅供参考学习用,建议学习看英文。版本Storm Version: 1.2.2

      Storm现在有4种内置调度程序:DefaultScheduler,IsolationScheduler,MultitenantScheduler,ResourceAwareScheduler。
    Storm now has 4 kinds of built-in schedulers: DefaultScheduler, IsolationScheduler, MultitenantScheduler, ResourceAwareScheduler.

    可插拔调度程序Pluggable scheduler

    您可以实现自己的调度程序来替换默认调度程序,以将执行程序分配给工作程序。您将类配置为在storm.yaml中使用“storm.scheduler”配置,并且您的调度程序必须实现 IScheduler接口。

    You can implement your own scheduler to replace the default scheduler to assign executors to workers. You configure the class to use the "storm.scheduler" config in your storm.yaml, and your scheduler must implement the IScheduler interface.

    隔离计划程序(Isolation Scheduler)

    The isolation scheduler makes it easy and safe to share a cluster among many topologies. The isolation scheduler lets you specify which topologies should be "isolated", meaning that they run on a dedicated set of machines within the cluster where no other topologies will be running. These isolated topologies are given priority on the cluster, so resources will be allocated to isolated topologies if there's competition with non-isolated topologies, and resources will be taken away from non-isolated topologies if necessary to get resources for an isolated topology. Once all isolated topologies are allocated, the remaining machines on the cluster are shared among all non-isolated topologies.
    隔离调度程序使得在许多拓扑中共享集群变得简单而安全。隔离调度程序允许您指定哪些拓扑应该“隔离”,这意味着它们在群集内的一组专用计算机上运行,​​而不会运行其他拓扑。这些隔离拓扑优先于集群,因此如果存在与非隔离拓扑的竞争,则将资源分配给隔离拓扑,并且如果需要,将从非隔离拓扑中获取资源以获得用于隔离拓扑的资源。分配完所有隔离拓扑后,群集中的其余计算机将在所有非隔离拓扑中共享。

    You can configure the isolation scheduler in the Nimbus configuration by setting "storm.scheduler" to "org.apache.storm.scheduler.IsolationScheduler". Then, use the "isolation.scheduler.machines" config to specify how many machines each topology should get. This configuration is a map from topology name to the number of isolated machines allocated to this topology. For example:

    您可以通过将“storm.scheduler”设置为“org.apache.storm.scheduler.IsolationScheduler”来在Nimbus配置中配置隔离调度程序。然后,使用“isolation.scheduler.machines”配置指定每个拓扑应该获得的计算机数。此配置是从拓扑名称到分配给此拓扑的隔离计算机数量的映射。例如:

    isolation.scheduler.machines:     "my-topology": 8    "tiny-topology": 1   "some-other-topology": 3

    提交到未列在其中的群集的任何拓扑都不会被隔离。请注意,Storm用户无法影响其隔离设置 - 这只能由群集管理员允许(这是非常有意的)。
    Any topologies submitted to the cluster not listed there will not be isolated. Note that there is no way for a user of Storm to affect their isolation settings – this is only allowed by the administrator of the cluster (this is very much intentional).

    隔离调度程序通过在拓扑之间提供完全隔离来解决多租户问题 - 避免拓扑之间的资源争用。目的是“生产的”拓扑应该在隔离配置中列出,测试或开发中的拓扑不应该。群集上的其余计算机为隔离拓扑和运行非隔离拓扑提供故障切换的双重角色。

    The isolation scheduler solves the multi-tenancy problem – avoiding resource contention between topologies – by providing full isolation between topologies. The intention is that "productionized" topologies should be listed in the isolation config, and test or in-development topologies should not. The remaining machines on the cluster serve the dual role of failover for isolated topologies and for running the non-isolated topologies.

    相关文章

      网友评论

        本文标题:storm(基础二)Scheduler(调度器)

        本文链接:https://www.haomeiwen.com/subject/qjvxnqtx.html