使用Elasticsearch做关键字自动补全，支持拼音和首字拼音以及模糊匹配等

2020-12-18 20:06:40 编辑 0 阅读量:1417

和Elasticsearch自带的Completion Suggester的原理不同，这里的原理是，新建一个字段的时候，把拼音、拼音首字和分词后的文本保存起来，然后当用户输入时，每次都用multi_search查询是否有相关的关键字。这样做有个优点就是支持多字段而且支持模糊查询，不像completion suggester只支持前缀补全。做法如下：

## 一、 安装分词器
参考文章:[elasticsearch集群的搭建以及常用插件（分词器、web页面）的安装](http://cblog.xyz/article/227)

## 二、设置好分词器

```
PUT /test
{
    "settings": {
        "analysis": {
            "analyzer": {
                "pinyin_completion_analyzer": {
                    "tokenizer": "full_pinyin_letter"
                }
            },
            "tokenizer": {
                "full_pinyin_letter": {
                    "type" : "pinyin",
                    "keep_first_letter" : false,
                    "keep_full_pinyin" : true,  # 重要参数
                    "keep_none_chinese" : true,
                    "keep_separate_first_letter" : true, # 重要参数
                    "keep_original" : false,
                    "limit_first_letter_length" : 16,
                    "lowercase" : true,
                    "trim_whitespace" : true,
                    "keep_none_chinese_in_first_letter" : true
                }
            }
        }
    },
    "mappings": {
        "test": {
            "properties": {
                "name": {
                    "type": "keyword",
                    "fields": {
                        "hanzi": {
                            "type": "text",
                            "analyzer": "jcseg_search"
                        },
                        "pinyin": {
                            "type": "text",
                            "analyzer": "pinyin_completion_analyzer"
                        }
                    }
                }
            }
        }
    }
}
```

## 三、测试分词器
```
GET test/_analyze
{
  "text": "深圳市南山法院,
  "analyzer": "pinyin_completion_analyzer"
}
# 结果
{
  "tokens": [
    {
      "token": "s",
      "start_offset": 0,
      "end_offset": 1,
      "type": "word",
      "position": 0
    },
    {
      "token": "shen",
      "start_offset": 0,
      "end_offset": 1,
      "type": "word",
      "position": 0
    },
    {
      "token": "z",
      "start_offset": 1,
      "end_offset": 2,
      "type": "word",
      "position": 1
    },
    {
      "token": "zhen",
      "start_offset": 1,
      "end_offset": 2,
      "type": "word",
      "position": 1
    },
    {
      "token": "s",
      "start_offset": 2,
      "end_offset": 3,
      "type": "word",
      "position": 2
    },
    {
      "token": "shi",
      "start_offset": 2,
      "end_offset": 3,
      "type": "word",
      "position": 2
    },
    {
      "token": "n",
      "start_offset": 3,
      "end_offset": 4,
      "type": "word",
      "position": 3
    },
    {
      "token": "nan",
      "start_offset": 3,
      "end_offset": 4,
      "type": "word",
      "position": 3
    },
    {
      "token": "s",
      "start_offset": 4,
      "end_offset": 5,
      "type": "word",
      "position": 4
    },
    {
      "token": "shan",
      "start_offset": 4,
      "end_offset": 5,
      "type": "word",
      "position": 4
    },
    {
      "token": "f",
      "start_offset": 5,
      "end_offset": 6,
      "type": "word",
      "position": 5
    },
    {
      "token": "fa",
      "start_offset": 5,
      "end_offset": 6,
      "type": "word",
      "position": 5
    },
    {
      "token": "y",
      "start_offset": 6,
      "end_offset": 7,
      "type": "word",
      "position": 6
    },
    {
      "token": "yuan",
      "start_offset": 6,
      "end_offset": 7,
      "type": "word",
      "position": 6
    }
  ]
}

```

## 四、效果展示
```
# 新建数据
PUT /test/test/1
{
  "name": "浙江省人民法院"
}

PUT /test/test/2
{
  "name": "菏泽法院"
}

PUT /test/test/3
{
  "name": "深圳市南山法院"
}

PUT /test/test/3
{
  "name": "深圳市男闪法院"
}

# 效果展示
POST /test/test/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "match_phrase": {
            "name.pinyin": "南山"
          }
        },
        {
          "match": {
            "name.hanzi": {
              "query": "南山"
            }
          }
        }
      ]
    }
  }
}
# 结果
{
  "took": 3,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 0.7203747,
    "hits": [
      {
        "_index": "website",
        "_type": "suggest",
        "_id": "4",
        "_score": 0.7203747,
        "_source": {
          "name": "深圳市男闪法院"
        }
      },
      {
        "_index": "website",
        "_type": "suggest",
        "_id": "3",
        "_score": 0.7203747,
        "_source": {
          "name": "深圳市南山法院"
        }
      }
    ]
  }
}
```
结果不太符合心中的预期，所以需要设置一个优先级，让汉字的权重上升

```
POST /test/test/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "match_phrase": {
            "name.pinyin": "南山"
          }
        },
        {
          "match": {
            "name.hanzi": {
              "query": "南山",
              "boost": 100
            }
          }
        }
      ]
    }
  }
}

# 结果
{
  "took": 3,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 30.604605,
    "hits": [
      {
        "_index": "website",
        "_type": "suggest",
        "_id": "3",
        "_score": 30.604605,
        "_source": {
          "name": "深圳市南山法院"
        }
      },
      {
        "_index": "website",
        "_type": "suggest",
        "_id": "4",
        "_score": 1.8363955,
        "_source": {
          "name": "深圳市男闪法院"
        }
      }
    ]
  }
}
```
嗯，结果还是可以的。试试用首字母：
```
POST /test/test/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "match_phrase": {
            "name.pinyin": "ns"
          }
        },
        {
          "match": {
            "name.hanzi": {
              "query": "ns",
              "boost": 100
            }
          }
        }
      ]
    }
  }
}

# 结果
{
  "took": 4,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 0.7203747,
    "hits": [
      {
        "_index": "website",
        "_type": "suggest",
        "_id": "4",
        "_score": 0.7203747,
        "_source": {
          "name": "深圳市男闪法院"
        }
      },
      {
        "_index": "website",
        "_type": "suggest",
        "_id": "3",
        "_score": 0.7203747,
        "_source": {
          "name": "深圳市南山法院"
        }
      }
    ]
  }
}
```
结果没毛病，分数相同时按照id排序

---
参考：
- [1][我想用这个拼音插件做完成建议（completion suggester），请问应该如何配置呢？ #91](https://github.com/medcl/elasticsearch-analysis-pinyin/issues/91)
- [2][Completion Suggest如何实现汉字、拼音、简拼搜索？ #43](https://github.com/medcl/elasticsearch-analysis-pinyin/issues/43)
- [3][Elasticsearch实现类似 like '?%' 搜索](http://www.cnblogs.com/clonen/p/6674492.html)
- [4][ElasticSearch拼音插件elasticsearch-analysis-pinyin使用介绍](https://my.oschina.net/xiaohui249/blog/214505)
- [5][elasticsearch集群的搭建以及常用插件（分词器、web页面）的安装](http://cblog.xyz/article/227)
- [6][Elasticsearch Suggester详解 ](https://elasticsearch.cn/article/142)
- [7][Elasticsearch - edgeNGram自动补全](http://blog.csdn.net/xifeijian/article/details/51167786)
- [8][请教使用elasticsearch-analysis-pinyin分词实现suggest字段关键字自动补全问题](https://elasticsearch.cn/question/1035)

使用Elasticsearch做关键字自动补全，支持拼音和首字拼音以及模糊匹配等

附件: