Most AI chatbots require a ton of processing power, so much so that they usually live in the cloud. Sure, you can run ChatGPT on your PC or even a Raspberry Pi, but the local program is sending your prompts over the Internet to OpenAI’s servers for a response. Some LLMs (Large Language Models) can run locally, but require a powerful GPU with a lot of VRAM. You couldn’t possibly run a chatbot locally on a mobile device? Or could you?
A brand new open-source project called MLC LLM is lightweight enough to run locally on just about any device, even an iPhone or an old PC laptop with integrated graphics. And, once you have MLC LLM installed, you can turn off the Internet, because all of the data and processing is happening on your system.
