Elasticsearch 6.5.4 自定义排序插件

作者: 孙瑞锴 | 来源:发表于2019-10-02 21:26 被阅读0次

Elasticsearch 6.5.4 自定义排序插件
Elasticsearch 6.5.4 自定义过滤器插件
elasticsearch 自定义排序插件
elasticsearch 问题01-启动报错
Skywalking 6.0环境搭建
filebeat+logstash+elasticsearch+
ElasticSearch插件编写-Similarity插件
ElasticSearch第二篇:安装插件
ElasticSearch
docker 安装 Elasticsearch:6.5.4 并

1. 借鉴

无

2. 开始

示例插件：按照指定日期的最小价格排序

2.1 创建索引

PUT /demo

POST /demo/_mapping/demo
{
    "properties":{
        "name":{
            "type":"text",
            "analyzer":"ik_max_word",
            "search_analyzer":"ik_smart"
        },
        "counts":{
            "type":"nested",
            "include_in_parent":true,
            "properties":{
                "details":{
                    "type":"nested",
                    "include_in_root":true,
                    "properties":{
                        "level":{
                            "type":"keyword"
                        },
                        "price":{
                            "type":"double"
                        }
                    }
                },
                "date":{
                    "type":"date",
                    "format":"yyyy-MM-dd"
                }
            }
        }
    }
}

2.2 添加测试数据

POST /demo/demo/1
{
  "name": "测试1",
  "counts": [
    {
      "details": [
        {
          "level": "0",
          "price": "10"
        },
        {
          "level": "1",
          "price": "20"
        },
        {
          "level": "2",
          "price": "30"
        }],
        "date": "2019-08-25"
    },
    {
      "details": [
        {
          "level": "0",
          "price": "20"
        },
        {
          "level": "1",
          "price": "30"
        },
        {
          "level": "2",
          "price": "40"
        }],
        "date": "2019-08-26"
    }]
}

POST /demo/demo/2
{
  "name": "测试2",
  "counts": [
    {
      "details": [
        {
          "level": "0",
          "price": "20"
        },
        {
          "level": "1",
          "price": "30"
        },
        {
          "level": "2",
          "price": "40"
        }],
        "date": "2019-08-25"
    },
    {
      "details": [
        {
          "level": "0",
          "price": "50"
        },
        {
          "level": "1",
          "price": "60"
        },
        {
          "level": "2",
          "price": "70"
        }],
        "date": "2019-08-26"
    }]
}

2.3 plugin.xml

需要注意的是plugin.xml文件必须放在assembly文件夹中，与main同级目录

<?xml version="1.0"?>
<assembly>
    <id>plugin</id>
    <formats>
        <format>zip</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <fileSets>
        <fileSet>
            <directory>${project.basedir}/src/main/resources</directory>
            <outputDirectory>/minPriceSortPlugin</outputDirectory>
        </fileSet>
    </fileSets>
    <dependencySets>
        <dependencySet>
            <outputDirectory>/minPriceSortPlugin</outputDirectory>
            <useProjectArtifact>true</useProjectArtifact>
            <useTransitiveFiltering>true</useTransitiveFiltering>
            <excludes>
                <exclude>org.elasticsearch:elasticsearch</exclude>
                <exclude>org.apache.logging.log4j:log4j-api</exclude>
            </excludes>
        </dependencySet>
    </dependencySets>
</assembly>

2.4 plugin-descriptor.properties

#有些属性已经过时了，可以参考官网
description=minPriceSortPlugin
version=1.0
name=minPricePlugin
#site=${elasticsearch.plugin.site}
#jvm=true
classname=com.ruihong.MinPriceSortPlugin
java.version=1.8
elasticsearch.version=6.5.4
#isolated=${elasticsearch.plugin.isolated}

2.5 pom.xml 配置（关键部分）

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
  </properties>

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.12</version>
      <scope>test</scope>
    </dependency>

    <dependency>
      <groupId>org.elasticsearch.client</groupId>
      <artifactId>transport</artifactId>
      <version>6.5.4</version>
      <scope>compile</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-assembly-plugin</artifactId>
        <version>2.3</version>
        <configuration>
          <appendAssemblyId>false</appendAssemblyId>
          <outputDirectory>${project.build.directory}/releases/</outputDirectory>
          <descriptors>
            <descriptor>${basedir}/src/assembly/plugin.xml</descriptor>
          </descriptors>
        </configuration>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>single</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
        </configuration>
      </plugin>
    </plugins>
  </build>

2.6 com.ruihong.MinPriceSortPlugin

package com.ruihong;

import com.ruihong.ex01.MinPriceSortEngine;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.plugins.Plugin;
import org.elasticsearch.plugins.ScriptPlugin;
import org.elasticsearch.script.ScriptContext;
import org.elasticsearch.script.ScriptEngine;

import java.util.Arrays;
import java.util.Collection;

public class MinPriceSortPlugin extends Plugin implements ScriptPlugin
{
    @Override
    public ScriptEngine getScriptEngine(Settings settings, Collection<ScriptContext<?>> contexts)
    {
        System.out.println(String.format("contexts: %s", Arrays.toString(contexts.toArray())));
        return new MinPriceSortEngine();
    }
}

2.7 com.ruihong.ex02.MinPriceSortEngine

package com.ruihong.ex01;

import org.apache.lucene.index.LeafReaderContext;
import org.elasticsearch.script.ScriptContext;
import org.elasticsearch.script.ScriptEngine;
import org.elasticsearch.script.SearchScript;
import org.elasticsearch.search.lookup.SearchLookup;

import java.io.IOException;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Comparator;
import java.util.Date;
import java.util.List;
import java.util.Map;

public class MinPriceSortEngine implements ScriptEngine
{
    private static final ThreadLocal<SimpleDateFormat> threadLocal = new
            ThreadLocal<SimpleDateFormat>();

    @Override
    public String getType()
    {
        return "min-price";
    }

    @Override
    public <FactoryType> FactoryType compile(String name, String code, ScriptContext<FactoryType> context, Map<String, String> params)
    {
        System.out.println(String.format("MinPriceSortEngine use params the scriptName %s, scriptSource %s, context %s, params %s", name, code, context.name, params.entrySet()));

        if (!context.equals(SearchScript.SCRIPT_SORT_CONTEXT))
        {
            throw new IllegalArgumentException(getType() + " scripts cannot be used for context [" + context.name + "]");
        }

        if ("min-price-sort".equals(code))
        {
            SearchScript.Factory factory = (p, lookupl) -> new SearchScript.LeafFactory() {
                @Override
                public SearchScript newInstance(LeafReaderContext ctx) throws IOException
                {
                    return new SearchScript(p, lookupl, ctx) {
                        @Override
                        public double runAsDouble()
                        {
                            try
                            {
                                final Date start = covertDateStrToDate(p.get("time-gte").toString(), "yyyy-MM-dd");
                                final Date end = covertDateStrToDate(p.get("time-lt").toString(), "yyyy-MM-dd");
                                final String level = p.get("level").toString();

                                Double minPrice = ((List<Map<String, Object>>) lookupl.source().get("counts"))
                                        .stream().filter(item ->
                                        {
                                            Date sellDate = covertDateStrToDate(item.get("date").toString(), "yyyy-MM-dd");
                                            return sellDate.compareTo(start) >= 0 && sellDate.compareTo(end) < 0;
                                        })
                                        .flatMap(item -> ((List<Map<String, Object>>) item.get("details")).stream())
                                        .filter(item -> item.get("level").toString().equals(level))
                                        .map(item -> Double.valueOf(item.get("price").toString()))
                                        .min(Comparator.naturalOrder())
                                        .orElse(-1.0);
                                return minPrice;
                            }
                            catch (Exception ex)
                            {
                                ex.printStackTrace();
                            }
                            return 0;
                        }
                    };
                }

                @Override
                public boolean needs_score()
                {
                    return false;
                }
            };
            return context.factoryClazz.cast(factory);
        }

        throw new IllegalArgumentException("Unknown script name " + code);
    }

    public static Date covertDateStrToDate(String dateStr, String format)
    {
        SimpleDateFormat sdf = null;
        sdf = threadLocal.get();
        if (sdf == null)
        {
            sdf = new SimpleDateFormat(format);
        }

        Date date = null;
        try
        {
            date = sdf.parse(dateStr);
        } catch (ParseException e)
        {
            e.printStackTrace();
        }
        return date;
    }
}

2.8 打包

maven clean install -Dmaven.test.skip=true
将releases目录下的zip放到es的plugins目录下，解压zip并重启es

2.9 测试

使用kibana

GET /hotelsearch_alias/_search
{
  "sort": [
    {
       "_script": {
          "type": "number",
          "order": "asc",
          "script": {
              "source": "min-price-sort",
              "lang": "min-price",
              "params": {
              "level": "0",
              "time-gte": "2019-08-13",
              "time-lt": "2019-08-14"
            }
          }
        }
    }
  ]
}

3. 大功告成

Elasticsearch 6.5.4 自定义排序插件
1. 借鉴无 2. 开始示例插件：按照指定日期的最小价格排序 2.1 创建索引 2.2 添加测试数据 2.3 ...
Elasticsearch 6.5.4 自定义过滤器插件
1. 借鉴 Elasticsearch自定义过滤插件实现复杂逻辑过滤还有其他技术文档，比如官网，google上面的...
elasticsearch 自定义排序插件
http://www.w3cschool.tech/s?btnG=&w=elastic+%E6%8E%92%E5%...
elasticsearch 问题01-启动报错
elasticSearch 启动报错 (1)使用windos32系统启动elasticsearch6.5.4版本 ...
Skywalking 6.0环境搭建
下载： elasticsearch6.5.4 https://artifacts.elastic.co/downl...
filebeat+logstash+elasticsearch+
0.环境说明 filebeat、logstash、elasticsearch版本为6.5.4。 1.filebea...
ElasticSearch插件编写-Similarity插件
ElasticSearch插件编写-Similarity插件 1.ElasticSearch插件是什么 Elast...
ElasticSearch第二篇:安装插件
ElasticSearch插件安装一：ElasticSearch-headElasticSearch插件安装二：k...
ElasticSearch
ElasticSearch插件 ELASTICSEARCH-HEAD（需要在ELASTICSEARCH CONFI...
docker 安装 Elasticsearch:6.5.4 并
拉镜像查看镜像启动一个容器elasticsearch/jvm.options 默认配置 -Xms2g - Xm...