Look, most people who are fine with AI surrounding their daily routine would rather use publicly available LLM services like ChatGPT or Gemini due to their convenience and powerful features, but not all want that, especially if you’re dealing with sensitive data, or just a privacy advocate person who wants self-controlled programs whenever applicable.
In any case and for any reason, if you want to set up your own LLM service, provided that you’re running an NVIDIA RTX GPU (GeForce or professional model), here’s something to start you off.
The first one is Ollama, an open-source app that makes running and interacting with LLMs pretty seamless. Think drag-and-drop PDFs, conversational chat, and even multimodal prompts that mix text and images. NVIDIA’s collaboration with Ollama has led to big improvements like faster performance on models such as gpt-oss-20B and Google’s Gemma 3, better memory management, stability across multiple GPUs, and support for efficient retrieval-augmented generation.
And if you want something more developer-focused, Ollama plays well with other apps. For example, AnythingLLM can sit on top of it, letting you build your own AI assistant that pulls from your documents and knowledge bases. Students can use this to turn lecture slides into flashcards, ask context-based questions tied to their notes, or even generate practice quizzes. With RTX acceleration, responses feel snappy and unrestricted — no waiting for servers, no usage limits.
Another popular route is LM Studio, built on llama.cpp, which provides a clean interface for running models locally. NVIDIA has tuned LM Studio as well, adding support for models like Nemotron Nano v2 9B, default Flash Attention for a big performance boost, and CUDA kernel optimizations for extra speed. For hobbyists and devs, this means real-time chats, local API endpoints for custom projects, and a smoother experience overall.
Beyond productivity and study tools, NVIDIA’s also experimenting with AI assistants for gaming PCs, and this one is even more straightforward, with Project G-Assist, an AI helper that takes simple voice or text commands to tweak system settings. The latest update adds laptop-specific controls like BatteryBoost adjustments for longer battery life, WhisperMode to reduce fan noise by half, and app profiles that balance performance and efficiency depending on whether you’re plugged in. With the Plug-In Builder and Plug-In Hub, users can even extend G-Assist with their own commands and integrations.
