Run AI models locally for free without any limitations : A Step-by-Step Guide for Windows, Mac, and Linux

Running AI models locally offers increased control and privacy over your data. This guide provides detailed steps for securely installing and running AI models, including Ollama with the Llama 3.1 model, Docker setup, and Stable Diffusion, on Windows, Mac, and Linux systems.

1. Install and Run Ollama with Llama 3.1

For Windows:

1. Download Ollama
   Go to the Ollama download page and get the Windows installer.
2. Install Ollama
   Execute the installer and follow the prompts to complete the installation.
3. Open Command Prompt
   Search for “cmd” in the Start menu and open Command Prompt.
4. Pull the Llama 3.1 Model
   Execute:

ollama pull llama3.1

For Mac:

1. Download Ollama
   Visit the Ollama download page and download the Mac installer.
2. Install Ollama
   Open the .dmg file and drag Ollama into your Applications folder.
3. Open Terminal
   Navigate to Applications > Utilities > Terminal.
4. Pull the Llama 3.1 Model
   Execute:

ollama pull llama3.1

For Linux:

1. Download Ollama
   Access the Ollama download page and select the Linux package.
2. Install Ollama
   Follow the installation instructions provided on the website, which may involve using a .deb or .rpm package.
3. Open Terminal
   Launch your terminal emulator.
4. Pull the Llama 3.1 Model
   Execute:

ollama pull llama3.1

2. verify if Ollama is running

To verify if Ollama is running properly, you can visit the URL http://127.0.0.1:11434 in your web browser. If Ollama is running correctly, you should see a response from the service, such as a status message, an API response, or some other indicator that the server is operational.

3. Monitor GPU Performance (Optional)

For Windows:

1. Open Command Prompt
   Search for “cmd” in the Start menu and open Command Prompt.
2. Install NVIDIA Drivers
   Ensure NVIDIA drivers are up to date via the NVIDIA website.
3. Monitor GPU Performance
   Execute:

nvidia-smi

For Mac:

1. Open Terminal
   Navigate to Applications > Utilities > Terminal.
2. Install GPU Monitoring Tools
   MacOS does not natively support nvidia-smi. Consider using third-party applications like iStat Menus.
3. Monitor GPU Performance
   Use the third-party tool to track GPU performance.

For Linux:

1. Open Terminal
   Launch your terminal emulator.
2. Install NVIDIA Drivers
   Install the necessary drivers:

sudo apt-get install nvidia-driver-<version>

3. Monitor GPU Performance
Execute:

watch -n 0.5 nvidia-smi

4. Install and Configure Docker

For Windows:

1. Download Docker Desktop
   Visit the Docker download page and download Docker Desktop for Windows.
2. Install Docker Desktop
   Run the installer and follow the setup instructions.
3. Start Docker Desktop
   Open Docker Desktop from the Start menu.

For Mac:

1. Download Docker Desktop
   Visit the Docker download page and download Docker Desktop for Mac.
2. Install Docker Desktop
   Open the .dmg file and drag Docker into the Applications folder.
3. Start Docker Desktop
   Open Docker Desktop from the Applications folder.

For Linux:

1. Add Docker’s Official GPG Key

sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

2. Add Docker Repository to Apt Sources

echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

3. Install Docker

sapt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

4. Run Open WebUI In Docker Container

1. Open Docker CLI or Terminal
   Use Docker CLI (Command Line Interface) or Terminal.
2. Run the Container
   Execute:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

To verify if open-webui is running properly, you can visit the URL http://127.0.0.1:3000 in your web browser

select mode & star using

ollama run <model name>

you can run multiple AI model