我正在尝试从 python 脚本在 redshift 中运行查询,但出现错误:
sqlalchemy.exc.InternalError: (psycopg2.InternalError) ALTER EXTERNAL TABLE cannot run inside a transaction block
这是我的代码:
engine = create_engine(SQL_ENGINE % urlquote(REDSHIFT_PASS))
partition_date = (date.today() - timedelta(day)).strftime("%Y%m%d")
query = """alter table {table_name} add partition (dt={date_partition}) location 's3://dft-dwh-files/raw_data/google_analytics/revenue_per_channel/{date_partition}/';""".format(date_partition=partition_date,table_name=table_name)
conn = engine.connect()
conn.execute(query).execution_options(autocommit=True)
我该如何解决这个问题?
对于 PostgreSQL,您需要将 隔离级别 设置为
AUTOCOMMIT
,而不是 SQLAlchemy autocommit
:
conn.execution_options(isolation_level="AUTOCOMMIT").execute(query)
这个解决方案对我来说适用于使用sqlalchemy:session而不是:conn作为@univerio答案
我在这里引用他们的答案
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
engine = create_engine('postgresql+psycopg2://USER:[email protected]:5432/DB_OR_TEMPLATE')
session = sessionmaker(bind=engine)()
session.connection().connection.set_isolation_level(0)
session.execute('CREATE DATABASE test')
session.connection().connection.set_isolation_level(1)
如果您没有任何数据库,您应该使用
template1
"""Isolation level values."""
ISOLATION_LEVEL_AUTOCOMMIT = 0
ISOLATION_LEVEL_READ_COMMITTED = 1
ISOLATION_LEVEL_SERIALIZABLE = 2
另一种选择是在删除或添加索引之前手动运行
COMMIT
:
from sqlalchemy.orm import Session
with Session(get_engine()) as session:
session.execute("COMMIT")
session.execute("DROP INDEX CONCURRENTLY IF EXISTS <my_index>;")