美文网首页
The Road of DBA 19_NoSQL_Elastic

The Road of DBA 19_NoSQL_Elastic

作者: Linux_淡忘 | 来源:发表于2019-07-11 19:33 被阅读0次

mongo回顾

1.sql和nosql区别
2.应用场景
- 用户评论
- 游戏装备
- 问卷调查
- 日志
3.mongo特点
4.安装部署
5.配置文件
6.CURD增删改查
7.用户授权
- admin root角色
- 自定义 read write角色
8.副本集
- 创建目录
- 修改配置
- 启动所有节点
- 初始化副本集配置参数
- 测试主从复制是否正常
- 模拟故障转移
- 扩容和收缩
- 权重调整和主库降级
- 仲裁节点
9.备份恢复
- 全备
- oplog备份
- mysql导出csv,导入到mongo
10.模拟误删除操作
- 全备
- 模拟误删除
- 备份oplog
- 切换到local下，查找误删除语句的时间点
- 处理备份文件
- 恢复到误操作时间点以前
- 检验数据是否恢复成功

2、Elasticsearch介绍

2.1

2.2、Elasticsearch功能


1）分布式的搜索引擎和数据分析引擎

2）针对电商搜索

3）对海量数据进行近实时处理

2.3、特点


1）搜索
2）全文搜索
3）分析数据
4）处理海量数据PB
5）高可用高性能分布式搜索引擎数据库

2.4、应用场景


1）网页搜索
2）新闻搜索
3）商品标签
4）日志收集分析展示

3、安装启动


1） 下载安装软件
mkdir  -p /data/es_soft/
cd /data/es_soft/
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.6.0.rpm
#yum install elasticsearch-6.6.0.rpm -y
rpm -ivh elasticsearch-6.6.0.rpm

2） 配置启动
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service
systemctl status elasticsearch.service
systemctl is-active elasticsearch.service

3）检查是否启动成功
ps -ef|grep elastic
lsof -i:9200
curl localhost:9200

4）查看配置文件位置
 rpm -qc elasticsearch 
/etc/elasticsearch/elasticsearch.yml         ---- 主配置文件
/etc/elasticsearch/jvm.options           -----java虚拟机配置
/etc/init.d/elasticsearch         ----init.d启动脚本
/etc/sysconfig/elasticsearch         ----环境变量相关信息，不需要修改
/usr/lib/sysctl.d/elasticsearch.conf           ----环境变量相关
/usr/lib/systemd/system/elasticsearch.service            ----systemd启动脚本

5）如果起不来

调整elasticsearch配置
/etc/elasticsearch/elasticsearch.yml
[root@db01 ~]# grep "^[a-z]" /etc/elasticsearch/elasticsearch.yml
node.name: node-1
path.data: /data/elasticsearch
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: true
network.host: 10.0.0.51 
http.port: 9200

mkdir /data/elasticsearch/ -p 
chown -R elasticsearch:elasticsearch /data/elasticsearch/

systemctl edit elasticsearch
[Service]
LimitMEMLOCK=infinity

systemctl daemon-reload
systemctl start elasticsearch

es-head安装部署


yum install nodejs npm openssl screen -y
node -v
npm  -v
npm install -g cnpm --registry=https://registry.npm.taobao.org
tar zxvf elasticsearch-head.tar.gz -C /opt/
cd /opt/elasticsearch-head
npm run start & 

修改es配置文件
vim /etc/elasticsearch/elasticsearch.yml
http.cors.enabled: true 
http.cors.allow-origin: "*"

重启es
systemctl restart elasticsearch

网页打开
10.0.0.51:9100
输入
http://10.0.0.51:9200
image

插入数据


curl -XPUT  'localhost:9200/vipinfo/user/2?pretty' -H 'Content-Type: application/json' -d' {
    "first_name": "Jane",
    "last_name" : "Smith",
    "age" : 32,
    "about" : "I like to collect rock albums", "interests": [ "music" ]
}'

curl -XPUT  'localhost:9200/vipinfo/user/3?pretty' -H 'Content-Type: application/json' -d' {
    "first_name": "Jane",
    "last_name" : "Smith",
    "age" : 32,
    "about" : "I like to collect rock albums", "interests": [ "music" ]
}'

curl -XPOST  'localhost:9200/vipinfo/user?pretty' -H 'Content-Type: application/json' -d' {
    "first_name": "Jane",
    "last_name" : "Smith",
    "age" : 32,
    "about" : "I like to collect rock albums", "interests": [ "music" ]
}'

curl -XPUT  'localhost:9200/vipinfo/user/2?pretty' -H 'Content-Type: application/json' -d' {
    "first_name": "zhang",
    "last_name" : "ya",
    "age" : 18,
    "about" : "I like to collect rock albums", "interests": [ "music" ]
}'

curl -XPUT  'localhost:9200/vipinfo/user/3?pretty' -H 'Content-Type: application/json' -d' {
    "first_name": "ya",
    "last_name" : "zhang",
    "age" : 28,
    "about" : "I like to collect rock albums", "interests": [ "music" ]
}'

curl -XPOST  'localhost:9200/vipinfo/user?pretty' -H 'Content-Type: application/json' -d' {
    "first_name": "老男孩Linux",
    "last_name" : "学院",
    "age" : 8,
    "about" : "I like to collect rock albums", "interests": [ "music" ]
}'

条件查询数据


curl -XGET 'localhost:9200/vipinfo/user/1?pretty'

curl -XGET 'localhost:9200/vipinfo/user/_search?q=last_name:ya&pretty'
curl -XGET 'localhost:9200/vipinfo/user/_search?q=age:28&pretty'

使用Query-string查询 
curl -XGET 'localhost:9200/vipinfo/user/_search?pretty' -H 'Content-Type: application/json' -d'           
{
  "query" : { 
    "match" : {
        "last_name" : "ya"
     }
  } 
}
'

集群（所有节点执行）


1） 下载安装软件
mkdir  -p /data/es_soft/
cd /data/es_soft/
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.6.0.rpm
#yum install elasticsearch-6.6.0.rpm -y
rpm -ivh elasticsearch-6.6.0.rpm

2）配置配置文件
[root@db01 /server/tools]# cat /etc/elasticsearch/elasticsearch.yml 
cluster.name: linux58
node.name: node-1
path.data: /data/elasticsearch
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: true
network.host: 10.0.0.51,127.0.0.1
http.port: 9200
discovery.zen.ping.unicast.hosts: ["10.0.0.51", "10.0.0.52"]
discovery.zen.minimum_master_nodes: 1
http.cors.enabled: true 
http.cors.allow-origin: "*"

[root@db02 /etc/elasticsearch]# cat /etc/elasticsearch/elasticsearch.yml 
cluster.name: linux58
node.name: node-2
path.data: /data/elasticsearch
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: true
network.host: 10.0.0.52,127.0.0.1
http.port: 9200
discovery.zen.ping.unicast.hosts: ["10.0.0.51", "10.0.0.52"]
discovery.zen.minimum_master_nodes: 1
http.cors.enabled: true 
http.cors.allow-origin: "*"

[root@db03 /etc/elasticsearch]# cat /etc/elasticsearch/elasticsearch.yml 
cluster.name: linux58
node.name: node-3
path.data: /data/elasticsearch
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: true
network.host: 10.0.0.53,127.0.0.1
http.port: 9200
discovery.zen.ping.unicast.hosts: ["10.0.0.51", "10.0.0.52", "10.0.0.53"]
discovery.zen.minimum_master_nodes: 1
http.cors.enabled: true 
http.cors.allow-origin: "*"

3）创建数据目录并授权
mkdir /data/elasticsearch/ -p 
chown -R elasticsearch:elasticsearch /data/elasticsearch/

4） 配置启动
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service
systemctl status elasticsearch.service
systemctl is-active elasticsearch.service

5）检查是否启动成功
ps -ef|grep elastic
lsof -i:9200
curl localhost:9200

6）添加
systemctl edit elasticsearch
[Service]
LimitMEMLOCK=infinity

systemctl daemon-reload
systemctl restart elasticsearch
curl 10.0.0.51:9200
curl 10.0.0.52:9200
curl 10.0.0.53:9200

查看节点数


curl -XGET 'http://localhost:9200/_cat/nodes?human&pretty'

增加index和设置分片、副本

curl -XPUT 'localhost:9200/index2?pretty' -H 'Content-Type: application/json' -d'       
{
    "settings" : { 
        "number_of_shards" : 3, 
        "number_of_replicas" : 1
    } 
}'

安装kibana


1）下载安装

2）修改配置文件
[root@db01 /data/soft]# grep "^[a-z]" /etc/kibana/kibana.yml 
server.port: 5601
server.host: "10.0.0.51"
server.name: "db01"
elasticsearch.hosts: ["http://localhost:9200"]

3）网站连接elasticsearch
image image

安装分词器(1.版本要对应 2.所有的ES节点都需要安装 3.所有es节点都需要重启)


未使用分词器之前查询中文数据
curl -XPOST http://localhost:9200/index2/fulltext/1 -H 'Content-Type:application/json' -d'
{"content":"美国留给伊拉克的是个烂摊子吗"}
'
curl -XPOST http://localhost:9200/index2/fulltext/2 -H 'Content-Type:application/json' -d'
{"content":"公安部：各地校车将享最高路权"}
'
curl -XPOST http://localhost:9200/index2/fulltext/3 -H 'Content-Type:application/json' -d'
{"content":"中韩渔警冲突调查：韩警平均每天扣1艘中国渔船"}
'
curl -XPOST http://localhost:9200/index2/fulltext/4 -H 'Content-Type:application/json' -d'
{"content":"中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"}
'

curl -XPOST http://localhost:9200/index2/fulltext/_search?pretty  -H 'Content-Type:application/json' -d'
{
    "query" : { "match" : { "content" : "中国" }},
    "highlight" : {
        "pre_tags" : ["<tag1>", "<tag2>"],
        "post_tags" : ["</tag1>", "</tag2>"],
        "fields" : {
            "content" : {}
        }
    }
}


1）安装
cd /usr/share/elasticsearch/bin
./elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.6.0/elasticsearch-analysis-ik-6.6.0.zip

2）重启
systemctl restart elasticsearch

分词器测试


创建索引
curl -XPUT http://localhost:9200/index

创建映射
curl -XPOST http://localhost:9200/index/fulltext/_mapping -H 'Content-Type:application/json' -d'
{
        "properties": {
            "content": {
                "type": "text",
                "analyzer": "ik_max_word",
                "search_analyzer": "ik_max_word"
            }
        }
}'

创建一些文档
curl -XPOST http://localhost:9200/index/fulltext/1 -H 'Content-Type:application/json' -d'
{"content":"美国留给伊拉克的是个烂摊子吗"}
'
curl -XPOST http://localhost:9200/index/fulltext/2 -H 'Content-Type:application/json' -d'
{"content":"公安部：各地校车将享最高路权"}
'
curl -XPOST http://localhost:9200/index/fulltext/3 -H 'Content-Type:application/json' -d'
{"content":"中韩渔警冲突调查：韩警平均每天扣1艘中国渔船"}
'
curl -XPOST http://localhost:9200/index/fulltext/4 -H 'Content-Type:application/json' -d'
{"content":"中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"}
'

查询
curl -XPOST http://localhost:9200/index/fulltext/_search?pretty  -H 'Content-Type:application/json' -d'
{
    "query" : { "match" : { "content" : "中国" }},
    "highlight" : {
        "pre_tags" : ["<tag1>", "<tag2>"],
        "post_tags" : ["</tag1>", "</tag2>"],
        "fields" : {
            "content" : {}
        }
    }
}
'

相关文章

网友评论

      本文标题：The Road of DBA 19_NoSQL_Elastic

      本文链接：https://www.haomeiwen.com/subject/nsrtkctx.html

      延伸阅读

      深度阅读

      • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

      栏目导航

      热点阅读

      关于我们|服务条款|联系我们|The Road of DBA 19_NoSQL_Elastic|投稿指南|网站地图|RSS订阅|排版工具|手机版

      提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

      Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

      备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

      本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

      所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！