Ollama Connector: Question about IImageToTextService Implementation

[russor45]

Has there been any consideration for implementing IImageToTextService for the Ollama connector?

I can only find the Hugging Face implementation in the codebase, but I might have missed it if it exists elsewhere. I noticed that OllamaSharp does include support for this feature.

One interesting observation is that while OllamaSharp supports providing prompts with image analysis, the current IImageToTextService interface doesn't include this capability. This prompt functionality could be valuable across different connectors, enabling more flexible image analysis - from controlling response structure to asking specific questions about image content.

I've been experimenting with a local implementation and was wondering if this is something that might be valuable to the project. If there's interest or existing plans for this feature, I'd be happy to learn more or potentially contribute to the effort.

Looking forward to any thoughts or guidance on this.

[RogerBarreto]

@russor45 Thanks for your input, currently our I<Modality>To<Modality>Service are specific for models that take the first as an input and gives back the second as the output.

For the scenario you are mentioning above, we recommend using ChatCompletion models, that take a ChatHistory with messages that can have multiple contents (images) in it and you can ask questions about the images you provided in the images, something very similar how you experience in a ChatGPT experience, providing attachments as part of the message.