ElasticSearch Index 질의(w/ python)

특정 검색 결과를 질의해서 반환받아보자. 

from elasticsearch import helpers, Elasticsearch
import csv, sys

es = Elasticsearch(sys.argv[1])
res = []
# Process hits here
def process_hits(hits, results):
    for item in hits:
        results.append(item)
    return results
data = es.search(
         index=sys.argv[2],
         size=10000,
         scroll='5m',
         body={
                 "query":{
                    "query_string":{
                       "default_field":"data",
                       "query":"*데이터*"
                    }
                 }
              }
       )


# Get the scroll ID
sid = data['_scroll_id']
scroll_size = len(data['hits']['hits'])

# Before scroll, process current batch of hits
res = process_hits(data['hits']['hits'], res)

# To Csv File
csv_file = sys.argv[3]
field_names = ['data', '@timestamp', 'level']

while scroll_size > 0:
    print("Scrolling... x" + repr(scroll_size))
    data = es.scroll(scroll_id=sid, scroll='5m')

    # Process current batch of hits
    res = process_hits(data['hits']['hits'], res)

    # Update the scroll ID
    sid = data['_scroll_id']

    # Get the number of results that returned in the last scroll
    scroll_size = len(data['hits']['hits'])

with open(csv_file, 'w') as csv_file:
    writer = csv.DictWriter(csv_file, fieldnames=field_names)
    writer.writeheader()
    for document in [x['_source'] for x in res]:
        writer.writerow(document)

 

===

 

The client noticed that the server is not a supported distribution of Elasticsearch

 

위와 같은 에러가 발생하면 임시 방편으로 version < 7.14.0 미만으로 설치한다.

ES client 는 7.14.0 부터 apache 2.0 에 오픈 프로토콜을 지원하지 않아 발생한다.