使用的是imply集成的druid，Imply提供了一套完整的部署方式，包括依赖库，Druid，tranquility、图形化的数据展示页面（pivot），SQL查询组件等。
安装版本： imply_3.0.7
部署规划：
角色机器集群角色
主节点master服务server3Coordinator,Overlord
主节点master服务server4Coordinator,Overlord
数据服务、查询服务server5Historical, MiddleManager, Tranquility,Broker,Pivot
数据服务、查询服务server6Historical, MiddleManager, Tranquility,Broker,Pivot
注：1、本druid集群具有HA特性。
2、server7也安装有imply_3.0.7,以便于以后横向拓展。
启动命令：
以后台挂起方式启动druid：
在server3、server4上执行
nohup bin/supervise -c conf/supervise/master.conf > master.log &
在server5、server5上执行
nohup bin/supervise -c conf/supervise/data-with-query.conf > data-with-query.log &
可视化控制台
overlord 控制页面：http://server3:8090/console.html
druid集群页面：http://server3:8081/index.html#/
数据可视化页面：http://server5:9096/datasets
配置文件
conf/druid/_common/common.runtime.properties
#
# Extensions
#
druid.extensions.directory=dist/druid/extensions
druid.extensions.hadoopDependenciesDir=dist/druid/hadoop-dependencies
druid.extensions.loadList=["mysql-metadata-storage","druid-hdfs-storage","druid-kafka-indexing-service","druid-lookups-cached-global","druid-histogram","druid-datasketches"]
#
# Logging
#
# Log all runtime properties on startup. Disable to avoid logging properties on startup:
druid.startup.logging.logProperties=true
#
# Zookeeper
#
druid.zk.service.host=server3:2181,server4:2181,server5:2181
druid.zk.paths.base=/druid
#
# Metadata storage
#
# For Derby server on your Druid Coordinator (only viable in a cluster with a single Coordinator, no fail-over):
#druid.metadata.storage.type=derby
#druid.metadata.storage.connector.connectURI=jdbc:derby://master.example.com:1527/var/druid/metadata.db;create=true
#druid.metadata.storage.connector.host=master.example.com
#druid.metadata.storage.connector.port=1527
# For MySQL:
druid.metadata.storage.type=mysql
druid.metadata.storage.connector.connectURI=jdbc:mysql://server2:3306/druid
druid.metadata.storage.connector.user=root
druid.metadata.storage.connector.password=root
# For PostgreSQL:
#druid.metadata.storage.type=postgresql
#druid.metadata.storage.connector.connectURI=jdbc:postgresql://db.example.com:5432/druid
#druid.metadata.storage.connector.user=...
#druid.metadata.storage.connector.password=...
#
# Deep storage
#
# For local disk (only viable in a cluster if this is a network mount):
#druid.storage.type=local
#druid.storage.storageDirectory=var/druid/segments
# For HDFS:
druid.storage.type=hdfs
druid.storage.storageDirectory=hdfs://server1:9870/druid/segments
#
# Indexing service logs
#
# For local disk (only viable in a cluster if this is a network mount):
#druid.indexer.logs.type=file
#druid.indexer.logs.directory=var/druid/indexing-logs
# For HDFS:
druid.indexer.logs.type=hdfs
druid.indexer.logs.directory=hdfs://server1:9870/druid/indexing-logs
# For S3:
#druid.indexer.logs.type=s3
#druid.indexer.logs.s3Bucket=your-bucket
#druid.indexer.logs.s3Prefix=druid/indexing-logs
#
# Service discovery
#
druid.selectors.indexing.serviceName=druid/overlord
druid.selectors.coordinator.serviceName=druid/coordinator
#
# Monitoring
#
druid.monitoring.monitors=["org.apache.druid.java.util.metrics.JvmMonitor"]
druid.emitter=logging
druid.emitter.logging.logLevel=debug
注：需将core-site.xml、hdfs-site.xml、mapred-site.xml、yarn-site.xml等Hadoop的配置文件移至_common文件夹下
conf/pivot/config.yaml
# The port on which the Pivot server will listen on.
port: 9096
# runtime directory
varDir: var/pivot
servingMode: clustered
# User management mode
# By default Imply will not show a login screen and anyone accessing it will automatically be treated as an 'admin'
# Uncomment the line below to enable user authentication, for more info see: https://docs.imply.io/on-prem/configure/config-api
#userMode: native-users
# The initial settings that will be loaded in, in this case a connection will be created for a Druid cluster that is running locally.
stateStore:
type: mysql
connection: 'mysql://user:password@server2:3306/druid'
table: 'pivot_state'
initialSettings:
connections:
- name: druid
type: druid
title: My Druid
host: localhost:8887
brokerHosts: ["server5:8082"]
coordinatorHosts: ["server3:8081"]
overlordHosts: ["server3:8090"]
#
# Pivot must have a state store in order to function
# The state (data cubes, dashboards, etc) can be stored in two ways.
# Choose just one option and comment out the other.
#
# 1) Stored in a sqlite file, editable at runtime with Settings View. Not suitable for running in a cluster.
# 2) Stored in a database, editable at runtime with Settings View. Works well with a cluster of Imply servers.
#
#
# 1) File-backed (sqlite) state (not suitable for running in a cluster)
#
#stateStore:
# type: sqlite
# connection: var/pivot/pivot-settings.sqlite
#
# 2) Database-backed state 'mysql' (MySQL) or 'pg' (Postgres)
#
# stateStore:
# location: mysql
# connection: 'mysql://user:password@server2:3306/druid'
创建conf/supercise/data-with-query.conf文件
:verify bin/verify-java
:verify bin/verify-version-check
broker bin/run-druid broker conf
router bin/run-druid router conf
pivot bin/run-pivot conf
historical bin/run-druid historical conf
middleManager bin/run-druid middleManager conf
# Uncomment to use Tranquility Server
#!p95 tranquility-server bin/tranquility server -configFile conf/tranquility/server.json
# Uncomment to use Tranquility Kafka
#!p95 tranquility-kafka bin/tranquility kafka -configFile conf/tranquility/kafka.json
# Uncomment to use Tranquility Clarity metrics server
#!p95 tranquility-metrics-server java -Xms4g -Xmx4g -cp "dist/tranquility/lib/*:dist/tranquility/conf" com.metamx.tranquility.distribution.DistributionMain server -configFile conf/tranquility/server-for-metrics.yaml
创建conf/supercise/master.conf文件
:verify bin/verify-java
:verify bin/verify-version-check
coordinator bin/run-druid coordinator conf
!p80 overlord bin/run-druid overlord conf
注：
1、因端口号冲突，在配置文件中修改了以下端口号：
修改router端口为8887
修改middlemanager端口号为8092
修改pivot端口号为9096
因druid启动有自检端口号机制，需修改启动文件bin/verify-default-ports中检查的端口号
网友评论