如何使用自定义 HTML 和 JavaScript 将图像加载到 Gradio 的画布上？

问题描述 投票：0回答：1

我愿意通过自己创建html和js来在gradio中实现边界框编辑器（用户可以移动bbox，缩放和绘制新的bbox）。但我找不到将图像加载到画布的方法。我将不胜感激任何形式的帮助

到目前为止我完成的最好的事情是将图像加载到图像 html 元素：

import gradio as gr
import numpy as np
import base64
from PIL import Image
from io import BytesIO

def image_to_base64(img):
    buffered = BytesIO()
    img.save(buffered, format="PNG")
    img_str = base64.b64encode(buffered.getvalue()).decode("utf-8")
    return img_str

def image_uploaded(img):
    img_pil = Image.fromarray(img)
    img_str = image_to_base64(img_pil)

    # HTML content with a canvas element
    html_content = f"""
    <html lang="en">
    <body>
        <img src="data:image/png;base64,{img_str}" alt="">
        <canvas id="canvas"></canvas>
    </body>
    </html>
    """
    
    return img, gr.HTML(html_content)

with gr.Blocks(js=None) as demo:
    uploaded_image = gr.Image(type="numpy", show_label=False, interactive=True, show_download_button=False)
    original_image = gr.Image(type="numpy", interactive=False, show_label=False, show_download_button=False)

    bounding_box_editor = gr.HTML()
    
    uploaded_image.upload(image_uploaded, inputs=uploaded_image, outputs=[original_image, bounding_box_editor])

demo.launch(share=False)

附注如果还有其他实现 - 我会很高兴看到它们（用户可以移动 bbox、缩放并绘制新的 bbox）

我需要帮助的是 JavaScript 部分，以实际将上传的图像绘制到画布上。任何指导或示例将不胜感激。

javascript python html bounding-box gradio
1个回答
0
投票

你可以在很多问题或教程中看到它

你必须获取画布，接下来获取它的上下文

.getContext('2d')
，然后使用
.drawImage(img, x, y, width, height)

放置图像

类似这样的事情

var image  = document.getElementById("image");
var canvas = document.getElementById("canvas");
var ctx = canvas.getContext('2d');
ctx.drawImage(image, 0, 0);

但我发现问题是

gr.HTML()
无法运行
<script>
，这使得如何运行它成为问题。

但是您可以将脚本添加到 

gr.Block(js=script)
，然后使用分配给按钮的 
onclick=".."
或分配给图像的 
onload="..."
。

完整代码：

import gradio as gr
import numpy as np
import base64
from PIL import Image
from io import BytesIO

def image_to_base64(img):
    buffered = BytesIO()
    img.save(buffered, format="PNG")
    img_str = base64.b64encode(buffered.getvalue()).decode("utf-8")
    return img_str

def image_uploaded(img):
    img_pil = Image.fromarray(img)
    img_str = image_to_base64(img_pil)

    img_str = f'data:image/png;base64,{img_str}'
    #img_str = f'https://upload.wikimedia.org/wikipedia/en/7/7d/Lenna_%28test_image%29.png'
    
    # HTML content with a canvas element
    html_content = f"""
<h1>Uploaded: img</h1>

<img id="image" src="{img_str}" alt="" onload="draw_image();">

<h1>Uploaded: canvas</h1>

<canvas id="canvas" width="600"></canvas>

<!-- <button onclick="draw_image();" style="background-color:#eee;padding:25px">Draw Image</button>  -->
"""

    return img, html_content

script = """
/* this function is executed automatically at start (not when you load canvas) */
/* so draw_image() has to be inside (not in place of `async ()`) */
async () => {
    globalThis.draw_image = () => {
        var image  = document.getElementById("image");
        var canvas = document.getElementById("canvas");
        var ctx = canvas.getContext('2d');
        ctx.drawImage(image, 0, 0);
    }
}
"""

with gr.Blocks(js=script) as demo:
    uploaded_image = gr.Image(type="numpy", show_label=False, interactive=True, show_download_button=False)
    original_image = gr.Image(type="numpy", interactive=False, show_label=False, show_download_button=False)

    bounding_box_editor = gr.HTML()
    
    uploaded_image.upload(image_uploaded, inputs=uploaded_image, outputs=[original_image, bounding_box_editor])
    
demo.launch(share=False)
最新问题
© www.soinside.com 2019 - 2024. All rights reserved.