How to run latest Gemma3 models with Ollama WebUI? 500 Internal Server Error fix

p.kaczmarek2 4875 0

TL;DR

Gemma 3 multimodal models in Ollama WebUI trigger a 500 Internal Server Error when the Docker-based Ollama core is too old.
The fix is to stop the Docker Ollama container, download Ollama from GitHub Releases, and run ollama.exe serve on the same port.
The broken container reported client version 0.6.1, while the standalone replacement uses 0.6.2.
After switching cores, WebUI can reach the new backend and Gemma 3 runs locally again, though models must be redownloaded.
The 1B model cannot handle images, so 4B or larger is recommended; 27B worked best in the tests.

Generated by the language model.

Report a violation of the law

Reply Cool? Ranking DIY | New topic

Notify about new articles

📢 Listen (AI):

Topic author Helpful post? (+1)

Post #1
21488942 21 Mar 2025 10:31

Are you trying to run latest Gemma3 multimodal AI models, but keep getting error 500 in Ollama WebUI?
Here's a solution, but first few words about Gemma 3. Gemma 3 is collection of lightweight, open models built from the same research and technology that powers Gemini 2.0 models. Gemma 3 models are designed to run fast, directly on devices and come in a range of sizes (1B, 4B, 12B and 27B), allowing you to choose the best model for your specific hardware and performance needs.

These models are very easy to download from Ollama Library and run, but Ollama Docker Package comes with obsolete Ollama version 0.6.1, so you can't run new them directly, at least until the Docker package is updated. I'll show a simple work-around here.

Error 500 issue
So, I'm assuming you already have a Docker setup like that:

You have Ollama core and Ollama Web interface both running in docker.
If not, you can get Ollama WebUI here.
You have also probably already downloaded Gemma3 in your Ollama Web UI, but when you try to run it, you'll getting:
Code: text Expand Select all Copy to clipboard
500: Ollama: 500, message='Internal Server Error', url='http://host.docker.internal:11434/api/chat'

Just as on the screenshot.

The cause of the problem
This is caused by Docker using obsolete Ollama core, namely version 0.6.1, at least in my case. You can check it by running:
Code: text Expand Select all Copy to clipboard
C:\Users\user>docker run --rm ghcr.io/open-webui/open-webui:ollama ollama --version Warning: could not connect to a running Ollama instance Warning: client version is 0.6.1

I've tried to update it, but found no way to do so. Luckily, there is a workaround...

Easiest solution
Download Ollama directly from the Releases tab:
https://github.com/ollama/ollama/releases/tag/v0.6.2
Choose package for your OS, in my case it was ollama-windows-amd64.zip.

Shutdown ollama in Docker first:

Extract it and run, as in:
Code: text Expand Select all Copy to clipboard
ollama.exe serve

Now, as long as port settings are matching, your Ollama WebUI from docker should be able to reach the new ollama core. You can also check it's version:
Code: text Expand Select all Copy to clipboard
W:\TOOLS\ollama-windows-amd64>ollama.exe --version Warning: could not connect to a running Ollama instance Warning: client version is 0.6.2

So, now you're running a newer Ollama.
This will mean that you'll have to redownload AI models. I've downloaded only smallest Gemma so far.

You can also download a bigger model:

Let's check if it works

Now, a word of warning - smallest 1b model with not work with images, so I suggest starting with 4b.

Some first Gemma 3 tests
It's time for some little Gemma 3 testing. I've played around with it a bit and decided to showcase 27b model, as it seems more reliable than smaller ones, obviously. Yet I can still run it on my 7 years old ROG gaming notebook.

Nice, it can read the time correctly.

Not bad, it has even noticed the slight damage of the bulb.
Let's try something harder.

Well, unfortunatelly it still makes mistakes and can give confusing results, but it's still better than LLaVA which I tested in the past...

Summary
It turns out that it is very easy to run new Gemma 3 models locally. The only issue that I encountered was the obsolete Ollama version in Docker, but hopefully, Docker package will be updated soon as well, so you won't encounted this problem in the future.
Regarding Gemma 3 itself, it seems very promising, especially the larger versions. They seem better than LLaVa at the first glance, but now I'm going to perform more tests.
I'll leave them for another topic.
Did you also try to run Gemma 3, and if so, what are your experiences there?
If you're more interested in Gemma 3, you can also just post here an image or a prompt and I'll test Gemma with it.

Cool? Ranking DIY
Helpful post? Buy me a coffee.
About Author
p.kaczmarek2 p.kaczmarek2

Moderator Smart Home
Offline

Joined: 26 Dec 2014

Posts: 14403

Help: 650

Posts rating: 12334

Points: 140936
p.kaczmarek2 wrote 14403 posts with rating 12334, helped 650 times. Been with us since 2014 year.
ADVERTISEMENT
Create an account, log in here. You will receive points by participating in discussions.
Join this discussion.

Install Elektroda application

Didn't find an answer? Ask Artificial Intelligence

*I agree to send the question to OpenAI, Anthropic PBC, Perplexity AI, Inc., Kagi Inc., Google LLC - owners of language models in order to prepare the best response. The companies may monitor and log information entered into the form.

*I agree to publicly display my question and answer. The question and answer will be publicly available to everyone. The process may take a few minutes. Upon completion, you will be redirected to the page with the answer.

Wait...(2min)

Reply Cool? Ranking DIY | New topic

Notify about new articles

📢 Listen (AI):

Report a violation of the law

FAQ

TL;DR: If Gemma 3 fails with a 500 error in Ollama WebUI, the fix is simple: replace Docker’s Ollama 0.6.1 with host-based 0.6.2. As the post says, "The cause of the problem" is the outdated core inside the container. This FAQ helps local AI users run newer Gemma 3 models, including multimodal variants, through Open WebUI without changing their whole setup. [#21488942]

Why this matters: A single version mismatch can block new Gemma 3 models entirely, even when Open WebUI and Docker appear to be working normally.

Option	Ollama version	Gemma 3 status	Notes
Docker-based Ollama core	0.6.1	Fails with 500 error	Obsolete version in the shown setup
Host-installed Ollama	0.6.2	Works	Replaces the backend while WebUI stays in Docker
Gemma 3 1B	Runs, but limited	Not suitable for images	Author recommends starting higher
Gemma 3 4B+	Better starting point	Better for multimodal tests	27B looked more reliable

Key insight: The failure is not a Gemma 3 model bug. Open WebUI works once it points to a newer Ollama core on the host, with matching ports. [#21488942]

Quick Facts

Gemma 3 is presented as a family of lightweight models in 1B, 4B, 12B, and 27B sizes, so users can match model size to available hardware and performance goals. [#21488942]
The shown failure happens at http://host.docker.internal:11434/api/chat, returning 500: Internal Server Error when WebUI calls an outdated Ollama backend. [#21488942]
The container check reports client version 0.6.1, while the working standalone Windows install reports client version 0.6.2 after extraction and launch. [#21488942]
The host-side replacement uses ollama.exe serve after stopping the Docker Ollama service, so Docker Open WebUI can talk to the newer core on the same port. [#21488942]
In early local tests, the author says the 27B Gemma 3 model felt more reliable than smaller versions and still ran on a 7-year-old ROG gaming notebook. [#21488942]

1. Why does Ollama WebUI show a 500 Internal Server Error when I try to run the latest Gemma 3 model in Docker?

Ollama WebUI shows the 500 error because the Docker setup uses an outdated Ollama core, version 0.6.1, which the post says cannot run the newer Gemma 3 models correctly. The failing request shown is /api/chat on host.docker.internal:11434, so WebUI is reachable, but the backend version is too old for that model. [#21488942]

2. How can I fix the Gemma 3 error 500 issue in Ollama WebUI when the Docker image uses Ollama 0.6.1?

Fix it by replacing the Docker-based Ollama backend with a newer standalone Ollama 0.6.2 install on the host. 1. Stop the Ollama service in Docker. 2. Download and extract Ollama 0.6.2 for your OS. 3. Start it with ollama.exe serve and keep the same port mapping so Open WebUI can reach it. [#21488942]

3. What is Gemma 3, and how is it different from other lightweight multimodal AI models?

Gemma 3 is a collection of lightweight open models built from the same research and technology that powers Gemini 2.0 models. The post lists four sizes—1B, 4B, 12B, and 27B—and highlights fast on-device use plus multimodal capability, which makes it practical for local text-and-image tests on varied hardware. [#21488942]

4. What is Ollama WebUI, and how does it connect to the Ollama core running locally or in Docker?

"Ollama WebUI" is a web interface that sends chat requests to an Ollama backend, usually over a local HTTP endpoint, and can run separately from the core service inside Docker or on the host. In the shown setup, WebUI calls http://host.docker.internal:11434/api/chat, so the interface and model runtime are connected over a port, not bundled into one process. [#21488942]

5. Which Ollama version is required to run newer Gemma 3 models correctly?

The working version in the post is Ollama 0.6.2. Version 0.6.1 inside the Docker image triggered the 500 error, while the host-installed 0.6.2 backend allowed the author to launch Gemma 3 successfully through the same WebUI. [#21488942]

6. How do I check the Ollama version inside the open-webui Docker container?

Run docker run --rm ghcr.io/open-webui/open-webui:ollama ollama --version to check the Ollama version bundled with that container image. In the post, that command prints a warning about no running instance and then reports client version is 0.6.1, which confirms the outdated backend. [#21488942]

7. What are the exact steps to replace the outdated Docker-based Ollama core with Ollama 0.6.2 on Windows?

Use a host-side Windows install and leave Open WebUI in Docker. 1. Stop Ollama in Docker. 2. Download the Windows amd64 ZIP for Ollama 0.6.2 and extract it. 3. Run ollama.exe serve, then verify with ollama.exe --version, which in the post reports client version is 0.6.2. [#21488942]

8. Where can I download the newer Ollama release needed for Gemma 3, and which package should I choose for Windows amd64?

Download it from the Ollama Releases page linked in the thread, specifically the 0.6.2 release. For a 64-bit Windows machine, the post says to choose ollama-windows-amd64.zip, extract it, and run the executable directly. [#21488942]

9. Why do I need to shut down the Ollama Docker service before starting ollama.exe serve on the host machine?

You shut down the Docker Ollama service to avoid backend conflict and let Open WebUI talk to the newer host-based Ollama instead. The post states that, once port settings match, Docker WebUI can reach the new core, so leaving the old container service active could keep traffic pointed at version 0.6.1. [#21488942]

10. How do port settings between Docker Open WebUI and a host-installed Ollama affect whether Gemma 3 works?

Gemma 3 works only if Open WebUI can reach the correct Ollama backend on the expected port. The post uses host.docker.internal:11434, and explicitly says the Docker WebUI should reach the newer host core "as long as port settings are matching," so a wrong port leaves WebUI connected to the failing backend or no backend at all. [#21488942]

11. What causes previously downloaded AI models to need redownloading after switching from Docker Ollama to a newer standalone Ollama?

Previously downloaded models need redownloading because the author switched from the Docker-based Ollama environment to a separate standalone Ollama installation. That new host install has its own model storage context, so the post notes that moving to the newer backend means downloading the AI models again. [#21488942]

12. Why doesn’t the Gemma 3 1B model work with images, and why is the 4B model a better starting point for multimodal tests?

The post says the smallest Gemma 3 1B model will not work with images, so it is a poor choice for multimodal testing. The author recommends starting with the 4B version instead, because it supports the image-oriented tests shown in Open WebUI and avoids that specific 1B limitation. [#21488942]

13. Gemma 3 vs LLaVA: which model gives better local image understanding results based on early tests?

Gemma 3 looked better than LLaVA in the author’s early local image tests. The post says the 27B Gemma 3 model seemed more reliable, correctly read a clock image, noticed slight bulb damage, and still made some mistakes, but the author judged it better than LLaVA at first glance. [#21488942]

14. What kind of hardware is practical for running larger Gemma 3 models like 27B locally, especially on an older gaming laptop?

A larger Gemma 3 model such as 27B can still be practical on older enthusiast hardware. The author reports running the 27B model on a 7-year-old ROG gaming notebook, which suggests local testing is possible on aging gaming-class laptops, though the post does not provide exact RAM or GPU figures. [#21488942]

15. What have users experienced when running Gemma 3 locally through Ollama WebUI, especially with image prompts and larger models?

Users can get Gemma 3 running locally through Ollama WebUI after fixing the backend version, and larger models give better image results. In the post, the 27B model handled simple image prompts well, including reading time and spotting slight bulb damage, while the author still warns that results can be confusing on harder tasks. [#21488942]