我已将模型部署为 Kserve 中的 docker 镜像。当我尝试从模型 pod 中卷曲请求模型时,我收到类似“ConnectionResetError (104, 'Connection reset by peer)”的错误。这是我的 handler.py 文件
import os
import json
import logging
from PIL import Image
import io
import torch
from transformers import DetrForObjectDetection
from ts.torch_handler.base_handler import BaseHandler
from detectron2.structures import Boxes, ImageList, Instances, BitMasks, PolygonMasks
from detectron2.modeling import META_ARCH_REGISTRY, build_backbone, detector_postprocess
from detectron2.utils.visualizer import Visualizer, VisImage
from detectron2.layers import batched_nms
import torchvision.transforms as T
import torch.nn.functional
import numpy as np
import base64
这是我的推理服务
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
name: custom-model
namespace: user1
spec:
predictor:
containers:
- name: kserve-container
image: <Dockerhub>/<Docker image >
env:
- name: "MODEL_NAME"
value: "table_det"
- name: "PROTOCOL"
value: "v1"
ports:
- containerPort: 8085
readinessProbe:
failureThreshold: 3
periodSeconds: 100
successThreshold: 1
tcpSocket:
port: 8085
timeoutSeconds: 10
initialDelaySeconds: 120
periodSeconds: 30
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: "1"
memory: 2Gi
i have deployed the Infrernece service file with Docker image.
i got the services like
Inference service {true}
ksvc {true}
Revision {true]
route {true}
virtualservice {true}
All service created by the service is created successfully without any error but i'm not able to establush the connection between the ingress of istio and the model . give me the best solution : thank you
I need solution and how the connection happens between the kserve and model deployed. i want to set perfect connection between the model and kserve.