我正在使用 google api python 客户端和在 google cloud 为我托管的模型在 google cloud 机器学习 API 上运行在线预测。 当我预测发送一张图像时,服务器(包括所有流量)大约需要 40 秒。当我发送两张图像时,一段时间后,我收到消息:
timeout: The read operation timed out
我想将超时设置为其他值,但我没有找到如何设置。
这是我的代码:
import base64
import io
import time
from PIL import Image
from oauth2client.service_account import ServiceAccountCredentials
from googleapiclient import discovery
SCOPES = ['https://www.googleapis.com/auth/cloud-platform']
SERVICE_ACCOUNT_FILE = 'mycredentialsfile.json'
credentials = ServiceAccountCredentials.from_json_keyfile_name(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
ml = discovery.build('ml', 'v1', credentials=credentials)
projectID = 'projects/{}'.format('projectID') + '/models/{}'.format('modelID')
width = 640
height = 480
instances = []
for image in ["image5.jpg", "image6.jpg"]:
img = Image.open(image)
img = img.resize((width, height), Image.ANTIALIAS)
output_str = io.BytesIO()
img.save(output_str, "JPEG")
instance = {"b64": base64.b64encode(output_str.getvalue()).decode("utf-8") }
output_str.close()
instances.append(instance)
input_json = {"instances": instances }
request = ml.projects().predict(body=input_json, name=projectID)
print("Starting prediction")
start_time = time.time()
response = request.execute()
print("%s seconds" % (time.time() - start_time))
是的。我同意上面Shohei的回答。我花了一段时间才找到这个简单而优雅的解决方案。只需要在代码中添加以下内容即可
import socket
timeout_in_sec = 60*3 # 3 minutes timeout limit
socket.setdefaulttimeout(timeout_in_sec)
# then you could create your ML service object as usually, and it will have the extended timeout limit.
ml_service = discovery.build('ml', 'v1')
我找到了一种方法,研究来自 github 上的 google api python 客户端的示例并尝试相同的更改。
使用httplib2进行身份验证可以设置超时时间。
修改后的代码如下:
import base64
import io
import time
from PIL import Image
# Need: pip install google-api-python-client
import httplib2
from oauth2client.service_account import ServiceAccountCredentials
from googleapiclient import discovery
SCOPES = ['https://www.googleapis.com/auth/cloud-platform']
# API & Services -> Credentials -> Create Credential -> service account key
SERVICE_ACCOUNT_FILE = 'mycredentialsfile.json'
credentials = ServiceAccountCredentials.from_json_keyfile_name(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
http = httplib2.Http(timeout=200)
http = credentials.authorize(http)
ml = discovery.build('ml', 'v1', http=http)
projectID = 'projects/{}'.format('projectID ') + '/models/{}'.format('modelID')
width = 640
height = 480
instances = []
for image in ["image5.jpg", "image6.jpg"]:
img = Image.open(image)
img = img.resize((width, height), Image.ANTIALIAS)
output_str = io.BytesIO()
img.save(output_str, "JPEG")
instance = {"b64": base64.b64encode(output_str.getvalue()).decode("utf-8") }
output_str.close()
instances.append(instance)
input_json = {"instances": instances }
request = ml.projects().predict(body=input_json, name=projectID)
print("Starting prediction")
start_time = time.time()
response = request.execute()
print("%s seconds" % (time.time() - start_time))
我认为只需进行一些修改,您就可以使用它为 python 客户端中的几乎所有谷歌云 API 设置超时。
我希望这有帮助。
你已经解决了问题,但我找到了另一种方法来做到这一点。
import socket
socket.setdefaulttimeout(150)
如果在没有http的情况下调用
discovery.build
,则http客户端由build_http
方法中的build
实例化。
正如您在这里所看到的,
build_http
如果在创建http客户端之前设置了超时,则会创建一个带有超时的http客户端实例。
所以你所要做的就是通过
socket.setdefaulttimeout
设置这个值:)
我不同意已接受的答案,因为这可能会导致使用 api 时受到限制。我在这里找到了相应的代码:https://github.com/googleapis/google-api-python-client/blob/main/googleapiclient/http.py#L1933-L1962
这就是里面发生的事情:
def build_http():
"""Builds httplib2.Http object
Returns:
A httplib2.Http object, which is used to make http requests, and which has timeout set by default.
To override default timeout call
socket.setdefaulttimeout(timeout_in_sec)
before interacting with this method.
"""
if socket.getdefaulttimeout() is not None:
http_timeout = socket.getdefaulttimeout()
else:
http_timeout = DEFAULT_HTTP_TIMEOUT_SEC
http = httplib2.Http(timeout=http_timeout)
# 308's are used by several Google APIs (Drive, YouTube)
# for Resumable Uploads rather than Permanent Redirects.
# This asks httplib2 to exclude 308s from the status codes
# it treats as redirects
try:
http.redirect_codes = http.redirect_codes - {308}
except AttributeError:
# Apache Beam tests depend on this library and cannot
# currently upgrade their httplib2 version
# http.redirect_codes does not exist in previous versions
# of httplib2, so pass
pass
return http
正如您所看到的,在 try 块中的 http 实例中添加了更多配置。所以我会推荐@Shohei Miyashita 使用套接字的方法。
话虽这么说,我也不认为这是一个好的解决方案,因为这是一个低级别的设置,意味着它可以应用于其他 http 客户端。我建议在实例化 API 客户端后将其设置回默认值:
import socket
socket.setdefaulttimeout(150)
# instantiate API client here
socket.getdefaulttimeout()