Vertex AI - 分类文本可能会因包含不当内容而被阻止

Question

我在 Firebase 函数中实现了 Vertex AI，以便对用户创建的内容进行分类。我应该是“成人”的类别之一，但是当 Vertex AI 检测到不适当的文本时，它会阻止响应并且不会对任何内容进行分类。它只是抛出此错误： 响应被阻止，因为输入或响应可能包含暴力、性主题或其他贬义内容的描述。请尝试重新表述您的提示。

const textModel = 'gemini-1.5-flash';

const vertexAI = new VertexAI({ project: projectId, location: 'us-central1' });

// Instantiate Gemini models
const generativeModel = vertexAI.getGenerativeModel({
    model: textModel,
});

const textToCategorize = `Categorize this content: "${contentDocumentCategories}". Print only 1 of these categories and don't include anything else: Games, Movies & TV, Music, Celebrities, Humor, Pets & Animals, Food & Drink, Vehicles, Technology, Science, Travel & Transportation, Sports, Online Communities, Books & Literature, Anime & Manga, Comics, Shopping, Nature, Random, Toys, Memes, Business, Interests, People, Lifestyle, Politics, Adult.`;
const vertexRequest = {
    contents: [{ role: 'user', parts: [{ text: textToCategorize }] }],
};
const vertexResult = await generativeModel.generateContent(vertexRequest);crashing
const vertexResponse = vertexResult.response;

// Response
if (vertexResponse)
   console.log('Category: ', vertexResponse.candidates[0].content.parts[0].text);

我怎样才能绕过这个？我希望它能够检测不适当的文本并按照查询中的指示对其进行分类。

Answer 1

您可以尝试在代码中添加文本预筛选，如下所示

const textModel = 'gemini-1.5-flash';
const vertexAI = new VertexAI({ project: projectId, location: 'us-central1' });

// Instantiate Gemini models
const generativeModel = vertexAI.getGenerativeModel({
    model: textModel,
});

// Example pre-screening function (simple regex for illustration)
function checkForSensitiveContent(text) {
    const sensitivePatterns = /explicit words or patterns here/i; // Define sensitive content patterns
    return sensitivePatterns.test(text);
}

const textToCategorize = `Categorize this content: "${contentDocumentCategories}"`;

if (checkForSensitiveContent(textToCategorize)) {
    console.log('Category: Adult');
} else {
    const vertexRequest = {
        contents: [{ role: 'user', parts: [{ text: textToCategorize }] }],
    };

    try {
        const vertexResult = await generativeModel.generateContent(vertexRequest);
        const vertexResponse = vertexResult.response;

        if (vertexResponse) {
            console.log('Category: ', vertexResponse.candidates[0].content.parts[0].text);
        }
    } catch (error) {
        console.error('Error processing with Vertex AI: ', error.message);
    }
}