ChatCompletion call with image as data url not working with com.azure:azure-ai-openai:1.0.0-beta.9

Question

I am trying to send an image for ChatCompletion using the Microsoft provided Azure OpenAi java sdk "com.azure:azure-ai-openai:1.0.0-beta.9".

val cco = ChatCompletionsOptions(
        mutableListOf(
        ChatRequestSystemMessage("You are an assistant which does Optical Character Recognition    on scanned medical documents."),
        ChatRequestUserMessage(mutableListOf<ChatMessageContentItem>(
            ChatMessageTextContentItem("Read text from the scanned medical document. Only respond with the text content."),
            ChatMessageImageContentItem(ChatMessageImageUrl("data:${contentType};base64,${Base64.getEncoder().encodeToString(bytes)}"
        ))
    ))))

The content is however not being sent correctly. It seems that the Microsoft provided client is not encoding the content correctly. Can someone confirm if this implementation works. The same code works with kotlin-openai and other libraries.

"error": {
    "message": "This model's maximum context length is 128000 tokens. However, your messages resulted in 291246 tokens. Please reduce the length of the messages.",
    "type": "invalid_request_error",
    "param": "messages",
    "code": "context_length_exceeded"
}

Someone to help with a issued faced during coding nd confirm if the specified library does indeed work.

What is the model you are using?

Jaya Shankar G S
– Jaya Shankar G S

2024-07-03 05:42:10 +00:00
Commented Jul 3, 2024 at 5:42 — Jaya Shankar G S
– Jaya Shankar G S, Commented Jul 3, 2024 at 5:42

Nicolas R · Accepted Answer · 2024-07-17 08:54:56Z

0

The issue has been raised on Github and agreed as a bug in azure-ai-openai SDK for Java, version 1.0.0-beta.9 - see github issue: https://github.com/Azure/azure-sdk-for-java/issues/40900

The fix has been pulled: https://github.com/Azure/azure-sdk-for-java/pull/40687 and based on the last comment, it looks like it is now included in version 1.0.0-beta.10 (see in maven)

answered Jul 17, 2024 at 8:54

Nicolas R

14.7k2 gold badges32 silver badges61 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

ChatCompletion call with image as data url not working with com.azure:azure-ai-openai:1.0.0-beta.9

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related