Django-haystack 与 AWS Opensearch

问题描述 投票：0回答：1

我正在尝试实现一个在本地运行但想在 AWS 上实现的项目。

我在 Docker 容器中本地运行，并为该项目运行一个 ElasticSearch 实例。在生产中我想运行 AWS OpenSearch。当我使用 docker 容器弹性搜索运行 

manage.py update_index
时，一切正常。当我尝试使用 OpenSearch 在 AWS 上实施时，我得到以下结果。

GET https://opensearch-instance.eu-west-2.es.amazonaws.com:443/haystack-local/_mapping [status:401 request:0.151s]
Undecodable raw error response from server: Expecting value: line 1 column 1 (char 0)
PUT https://opensearch-instance.eu-west-2.es.amazonaws.com:443/haystack-local [status:401 request:0.087s]
Undecodable raw error response from server: Expecting value: line 1 column 1 (char 0)
POST https://opensearch-instance.eu-west-2.es.amazonaws.com:443/haystack-local/modelresult/_bulk [status:400 request:0.061s]
POST https://opensearch-instance.eu-west-2.es.amazonaws.com:443/haystack-local/modelresult/_bulk [status:400 request:0.060s]
POST https://opensearch-instance.eu-west-2.es.amazonaws.com:443/haystack-local/modelresult/_bulk [status:400 request:0.058s]
POST https://opensearch-instance.eu-west-2.es.amazonaws.com:443/haystack-local/modelresult/_bulk [status:400 request:0.058s]
POST https://opensearch-instance.eu-west-2.es.amazonaws.com:443/haystack-local/modelresult/_bulk [status:400 request:0.058s]
--- Logging error ---
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/haystack/management/commands/update_index.py", line 119, in do_update
    backend.update(index, current_qs, commit=commit)
  File "/usr/local/lib/python3.11/site-packages/haystack/backends/elasticsearch_backend.py", line 239, in update
    bulk(
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/helpers/__init__.py", line 257, in bulk
    for ok, item in streaming_bulk(client, actions, **kwargs):
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/helpers/__init__.py", line 188, in streaming_bulk
    for data, (ok, info) in zip(
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/helpers/__init__.py", line 99, in _process_bulk_chunk
    raise e
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/helpers/__init__.py", line 95, in _process_bulk_chunk
    resp = client.bulk('\n'.join(bulk_actions) + '\n', **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
    return func(*args, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/client/__init__.py", line 1173, in bulk
    return self.transport.perform_request('POST', _make_path(index,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/transport.py", line 312, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/connection/http_urllib3.py", line 129, in perform_request
    self._raise_error(response.status, raw_data)
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: <exception str() failed>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/logging/__init__.py", line 1110, in emit
    msg = self.format(record)
          ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/logging/__init__.py", line 953, in format
    return fmt.format(record)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/logging/__init__.py", line 687, in format
    record.message = record.getMessage()
                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/logging/__init__.py", line 377, in getMessage
    msg = msg % self.args
          ~~~~^~~~~~~~~~~
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/exceptions.py", line 55, in __str__
    cause = ', %r' % self.info['error']['root_cause'][0]['reason']
                     ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
TypeError: string indices must be integers, not 'str'
Call stack:
  File "/app/src/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.11/site-packages/haystack/management/commands/update_index.py", line 297, in handle
    self.update_backend(label, using)
  File "/usr/local/lib/python3.11/site-packages/haystack/management/commands/update_index.py", line 342, in update_backend
    max_pk = do_update(
  File "/usr/local/lib/python3.11/site-packages/haystack/management/commands/update_index.py", line 147, in do_update
    LOG.error(error_msg, error_context, exc_info=True)
Message: 'Failed indexing %(start)s - %(end)s (retry %(retries)s/%(max_retries)s): %(exc)s (pid %(pid)s): %(exc)s'
Arguments: {'start': 1, 'end': 3, 'retries': 5, 'max_retries': 5, 'pid': 1, 'exc': RequestError(400, 'no handler found for uri [/haystack-local/modelresult/_bulk] and method [POST]', {'error': 'no handler found for uri [/haystack-local/modelresult/_bulk] and method [POST]'})}
[ERROR/MainProcess] Error updating events using default 
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/haystack/management/commands/update_index.py", line 297, in handle
    self.update_backend(label, using)
  File "/usr/local/lib/python3.11/site-packages/haystack/management/commands/update_index.py", line 342, in update_backend
    max_pk = do_update(
             ^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/haystack/management/commands/update_index.py", line 119, in do_update
    backend.update(index, current_qs, commit=commit)
  File "/usr/local/lib/python3.11/site-packages/haystack/backends/elasticsearch_backend.py", line 239, in update
    bulk(
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/helpers/__init__.py", line 257, in bulk
    for ok, item in streaming_bulk(client, actions, **kwargs):
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/helpers/__init__.py", line 188, in streaming_bulk
    for data, (ok, info) in zip(
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/helpers/__init__.py", line 99, in _process_bulk_chunk
    raise e
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/helpers/__init__.py", line 95, in _process_bulk_chunk
    resp = client.bulk('\n'.join(bulk_actions) + '\n', **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
    return func(*args, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/client/__init__.py", line 1173, in bulk
    return self.transport.perform_request('POST', _make_path(index,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/transport.py", line 312, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/connection/http_urllib3.py", line 129, in perform_request
    self._raise_error(response.status, raw_data)
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: <exception str() failed>
Traceback (most recent call last):
  File "/app/src/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/haystack/management/commands/update_index.py", line 297, in handle
    self.update_backend(label, using)
  File "/usr/local/lib/python3.11/site-packages/haystack/management/commands/update_index.py", line 342, in update_backend
    max_pk = do_update(
             ^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/haystack/management/commands/update_index.py", line 119, in do_update
    backend.update(index, current_qs, commit=commit)
  File "/usr/local/lib/python3.11/site-packages/haystack/backends/elasticsearch_backend.py", line 239, in update
    bulk(
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/helpers/__init__.py", line 257, in bulk
    for ok, item in streaming_bulk(client, actions, **kwargs):
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/helpers/__init__.py", line 188, in streaming_bulk
    for data, (ok, info) in zip(
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/helpers/__init__.py", line 99, in _process_bulk_chunk
    raise e
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/helpers/__init__.py", line 95, in _process_bulk_chunk
    resp = client.bulk('\n'.join(bulk_actions) + '\n', **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
    return func(*args, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/client/__init__.py", line 1173, in bulk
    return self.transport.perform_request('POST', _make_path(index,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/transport.py", line 312, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/connection/http_urllib3.py", line 129, in perform_request
    self._raise_error(response.status, raw_data)
  File "/usr/local/lib/python3.11/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: <exception str() failed>

我意识到我应该在我的 Docker 容器中安装 OpenSearch（但我正在接管别人的项目，所以会到达那里）。我不知道如何让 

django-haystack
与 OpenSearch 一起使用。我想我必须更改 Haystack 连接设置，目前是

HAYSTACK_CONNECTIONS = {
    "default": {
        "ENGINE": "haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine",
        "URL": os.environ.get("ELASTICSEARCH_URL"),
        "INDEX_NAME": f"white-eagle-lodge-{DEPLOY_ENV}",  # noqa: F405
        "INCLUDE_SPELLING": True,
        "TIMEOUT": 180,
    }
}

我担心 Haystack 没有为 OpenSearch 提供 

ENGINE
。 任何人都可以帮助如何设置它。谢谢

django amazon-web-services django-haystack
1个回答
0
投票

我注意到有几个类似的问题没有答案。我有这个工作。

在

requirements.txt
中：
elasticsearch<7.14

在

settings.py
：

import elasticsearch
from requests_aws4auth import AWS4Auth
awsauth = AWS4Auth(
    os.environ.get('AWS_ACCESS_KEY_ID', '<AWS_ACCESS_KEY>'),
    os.environ.get('AWS_SECRET_ACCESS_KEY', '<AWS_SECRET_KEY>'),
    'eu-west-2',
    'es')

HAYSTACK_CONNECTIONS = {
    "default": {
        "ENGINE": "haystack.backends.elasticsearch7_backend.Elasticsearch7SearchEngine",
        "URL": os.environ.get("ELASTICSEARCH_URL"),
        "INDEX_NAME": f"haystack-{DEPLOY_ENV}",  # noqa: F405
        "INCLUDE_SPELLING": True,
        "TIMEOUT": 180,
        'KWARGS': {
            'port': 443,
            'http_auth': awsauth,
            'use_ssl': True,
            'verify_certs': True,
            'connection_class': elasticsearch.RequestsHttpConnection,
            'build_flavor': 'default'
        }
    }
}

这对我有用:)

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.