Run AI models locally for free without any limitations : A Step-by-Step Guide for Windows, Mac, and Linux
Running AI models locally offers increased control and privacy over your data. This guide provides detailed steps for securely installing and running AI models, including Ollama with the Llama 3.1 model, Docker setup, and Stable Diffusion, on Windows, Mac, and Linux systems.
1. Install and Run Ollama with Llama 3.1
For Windows:
- Download Ollama
Go to the Ollama download page and get the Windows installer. - Install Ollama
Execute the installer and follow the prompts to complete the installation. - Open Command Prompt
Search for “cmd” in the Start menu and open Command Prompt. - Pull the Llama 3.1 Model
Execute:
ollama pull llama3.1
For Mac:
- Download Ollama
Visit the Ollama download page and download the Mac installer. - Install Ollama
Open the.dmg
file and drag Ollama into your Applications folder. - Open Terminal
Navigate to Applications > Utilities > Terminal. - Pull the Llama 3.1 Model
Execute:
ollama pull llama3.1
For Linux:
- Download Ollama
Access the Ollama download page and select the Linux package. - Install Ollama
Follow the installation instructions provided on the website, which may involve using a.deb
or.rpm
package. - Open Terminal
Launch your terminal emulator. - Pull the Llama 3.1 Model
Execute:
ollama pull llama3.1
2. verify if Ollama is running
To verify if Ollama is running properly, you can visit the URL http://127.0.0.1:11434
in your web browser. If Ollama is running correctly, you should see a response from the service, such as a status message, an API response, or some other indicator that the server is operational.
3. Monitor GPU Performance (Optional)
For Windows:
- Open Command Prompt
Search for “cmd” in the Start menu and open Command Prompt. - Install NVIDIA Drivers
Ensure NVIDIA drivers are up to date via the NVIDIA website. - Monitor GPU Performance
Execute:
nvidia-smi
For Mac:
- Open Terminal
Navigate to Applications > Utilities > Terminal. - Install GPU Monitoring Tools
MacOS does not natively supportnvidia-smi
. Consider using third-party applications likeiStat Menus
. - Monitor GPU Performance
Use the third-party tool to track GPU performance.
For Linux:
- Open Terminal
Launch your terminal emulator. - Install NVIDIA Drivers
Install the necessary drivers:
sudo apt-get install nvidia-driver-<version>
3. Monitor GPU Performance
Execute:
watch -n 0.5 nvidia-smi
4. Install and Configure Docker
For Windows:
- Download Docker Desktop
Visit the Docker download page and download Docker Desktop for Windows. - Install Docker Desktop
Run the installer and follow the setup instructions. - Start Docker Desktop
Open Docker Desktop from the Start menu.
For Mac:
- Download Docker Desktop
Visit the Docker download page and download Docker Desktop for Mac. - Install Docker Desktop
Open the.dmg
file and drag Docker into the Applications folder. - Start Docker Desktop
Open Docker Desktop from the Applications folder.
For Linux:
- Add Docker’s Official GPG Key
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
2. Add Docker Repository to Apt Sources
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
3. Install Docker
sapt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
4. Run Open WebUI In Docker Container
- Open Docker CLI or Terminal
Use Docker CLI (Command Line Interface) or Terminal. - Run the Container
Execute:
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
To verify if open-webui is running properly, you can visit the URL http://127.0.0.1:3000
in your web browser
select mode & star using
ollama run <model name>
you can run multiple AI model