Run AI models locally for free without any limitations : A Step-by-Step Guide for Windows, Mac, and Linux

jay75chauhan
4 min readAug 11, 2024

--

Running AI models locally offers increased control and privacy over your data. This guide provides detailed steps for securely installing and running AI models, including Ollama with the Llama 3.1 model, Docker setup, and Stable Diffusion, on Windows, Mac, and Linux systems.

1. Install and Run Ollama with Llama 3.1

For Windows:

  1. Download Ollama
    Go to the Ollama download page and get the Windows installer.
  2. Install Ollama
    Execute the installer and follow the prompts to complete the installation.
  3. Open Command Prompt
    Search for “cmd” in the Start menu and open Command Prompt.
  4. Pull the Llama 3.1 Model
    Execute:
ollama pull llama3.1

For Mac:

  1. Download Ollama
    Visit the Ollama download page and download the Mac installer.
  2. Install Ollama
    Open the .dmg file and drag Ollama into your Applications folder.
  3. Open Terminal
    Navigate to Applications > Utilities > Terminal.
  4. Pull the Llama 3.1 Model
    Execute:
ollama pull llama3.1

For Linux:

  1. Download Ollama
    Access the Ollama download page and select the Linux package.
  2. Install Ollama
    Follow the installation instructions provided on the website, which may involve using a .deb or .rpm package.
  3. Open Terminal
    Launch your terminal emulator.
  4. Pull the Llama 3.1 Model
    Execute:
ollama pull llama3.1

2. verify if Ollama is running

To verify if Ollama is running properly, you can visit the URL http://127.0.0.1:11434 in your web browser. If Ollama is running correctly, you should see a response from the service, such as a status message, an API response, or some other indicator that the server is operational.

3. Monitor GPU Performance (Optional)

For Windows:

  1. Open Command Prompt
    Search for “cmd” in the Start menu and open Command Prompt.
  2. Install NVIDIA Drivers
    Ensure NVIDIA drivers are up to date via the NVIDIA website.
  3. Monitor GPU Performance
    Execute:
nvidia-smi

For Mac:

  1. Open Terminal
    Navigate to Applications > Utilities > Terminal.
  2. Install GPU Monitoring Tools
    MacOS does not natively support nvidia-smi. Consider using third-party applications like iStat Menus.
  3. Monitor GPU Performance
    Use the third-party tool to track GPU performance.

For Linux:

  1. Open Terminal
    Launch your terminal emulator.
  2. Install NVIDIA Drivers
    Install the necessary drivers:
sudo apt-get install nvidia-driver-<version>

3. Monitor GPU Performance
Execute:

watch -n 0.5 nvidia-smi

4. Install and Configure Docker

For Windows:

  1. Download Docker Desktop
    Visit the Docker download page and download Docker Desktop for Windows.
  2. Install Docker Desktop
    Run the installer and follow the setup instructions.
  3. Start Docker Desktop
    Open Docker Desktop from the Start menu.

For Mac:

  1. Download Docker Desktop
    Visit the Docker download page and download Docker Desktop for Mac.
  2. Install Docker Desktop
    Open the .dmg file and drag Docker into the Applications folder.
  3. Start Docker Desktop
    Open Docker Desktop from the Applications folder.

For Linux:

  1. Add Docker’s Official GPG Key
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

2. Add Docker Repository to Apt Sources

echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

3. Install Docker

sapt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

4. Run Open WebUI In Docker Container

  1. Open Docker CLI or Terminal
    Use Docker CLI (Command Line Interface) or Terminal.
  2. Run the Container
    Execute:
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

To verify if open-webui is running properly, you can visit the URL http://127.0.0.1:3000 in your web browser

select mode & star using

ollama run <model name>

you can run multiple AI model

--

--

No responses yet