In order to run Ollama in your local system, the best way is to use docker in the Windows Subsystem for Linux. This is a tutorial going over what needs to be done.

🐧 Installing WSL and Docker on Windows

This guide will walk you through setting up Windows Subsystem for Linux (WSL) and Docker inside WSL on a Windows machine.

1. Enable WSL

Open PowerShell as Administrator and run:

wsl --install

This installs Ubuntu as the default distribution. If WSL 2 isn't already enabled, this command handles it too.

Reboot Your PC

This will install Ubuntu. Once done, restart your PC to complete the setup.

Set Up Your Linux Distribution

After reboot, WSL will launch and prompt you to create a username and password.

If it doesn't launch automatically, run:

wsl

Or launch Ubuntu from the Start Menu.

Update Ubuntu

Inside WSL terminal:

sudo apt update && sudo apt upgrade -y

2. 🐳Install Docker in WSL (Ubuntu)

Instructions taken from Docker Ubuntu Installation.

Add the repository to Apt sources:

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

Install the Docker packages.

To install the latest version, run:

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Run Docker without sudo

(Optional but recommended):

sudo usermod -aG docker $USER

Then restart WSL or your terminal:

exit

And open it again. Test with:

docker run hello-world

The following screen should be visible

3. 🦙 Installing Ollama in the WSL Ubuntu Environment

We will install and run Ollama using Docker, following the instructions on the official to access the docker Image Ollama Docker page.

Since I’m running this on a laptop without GPU support, I opted for the CPU version. If you have a GPU, using the GPU version is highly recommended for better performance.

1. Run the Docker container:

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

2. Run a model (e.g., llama3):

docker exec -it ollama ollama run llama3

This will download and run the llama3 model (or whichever model you specify). You can now interact with it.

Use the command below to see help and available options:

/?

3. Starting Ollama on future sessions:

When you reopen WSL, you can start the container using:

docker start ollama

Then, re-run your desired model:

docker exec -it ollama ollama run llama3

4. Accessing via browser or client:

Once running, visit:

http://localhost:11434

You should see a response like:

{"message":"Ollama is running"}

5. To stop Ollama:

docker stop ollama