Локальные ИИ-модели на Android: эксперимент с офлайн-ассистентом вместо облачной подписки

I've been hearing from many experts in the artificial intelligence field since last year predicting that local LLMs (Large Language Models) will be the future of AI. I don't make such sweeping predictions, especially in tech, but the growing number of voices favoring local LLMs pushed me hard to get my hands on them. After months of delay, I finally ran a local LLM on my phone. While this was an experiment, I treated the local LLM as my primary AI assistant for performing everyday AI tasks. I knew the limitations of local LLMs and set the expectations accordingly, but the one I tried still surprised me. I might not have felt that way had I not used an AI cloud subscription, which I've grown accustomed to all these years. Although I knew it wouldn't work, I skipped the AI cloud subscription by running a local LLM on my Android phone. Here is what I noticed in my experiment. Running local LLMs on Android Trivia challenge Think you know your tokens from your tensors? Test your knowledge of running AI models right on your Android phone. Which app is widely regarded as one of the first popular tools for running local LLMs directly on an Android device? What component in a modern Android phone is most critical for accelerating local LLM inference? Which model format is commonly used to run quantized LLMs efficiently on resource-constrained devices like Android phones? What does 'quantization' mean in the context of running LLMs on Android? Google's Gemini Nano is notable for being the first version of Gemini designed for on-device use. On which Android feature did it first debut publicly? What is the typical RAM requirement to run a capable 7-billion parameter LLM in 4-bit quantized form on an Android device? Which open-source inference engine, originally developed for desktop use, has been ported to Android and powers many local LLM apps on the platform? Which Qualcomm chip generation was the first to officially support Qualcomm's AI Hub platform for on-device generative AI workloads on Android phones? Your Score Thanks for playing! My Android phone became my AI assistant without a cloud subscription A local model has answers to most of what you need I didn't complicate my setup. I downloaded and installed an app called PocketPal from the Google Play Store and then downloaded a local model from Hugging Face to run it on my phone. The PocketPal app allows you to deploy open source models from Hugging Face without leaving the app. It took only a few minutes to complete the setup, though it was worth investing time. I could easily use the E2B variant of the Gemma 4 model when not connected to the internet and still generate answers to complex questions. For example, I use it as a decision-maker tool. Instead of reading a complete phone review, I copy and paste the text and then ask it to tell me the strengths, weaknesses, and who should avoid buying the handset. Similarly, I can give it the data and ask it to organize it in a table. It can also summarize text, highlight key points, and break down complex topics so that everyone can understand the basics. Since it can't pull data from the internet, you need to copy and paste the information in the prompt. It can do almost everything that an AI model running in the cloud can do, except browse the web. While the lack of web browsing capability is a disadvantage, its offline capability puts it in a unique position. No matter how powerful a cloud-based LLM is, it can't compete with a local model in giving you access to AI when you can't access the internet. Since it doesn't send your data to the cloud servers, all your information stays on your phone. So, a cloud-based model is no match for local ones when it comes to protecting users' privacy. My local LLM is far from being perfect Its flaws are hard to ignore I won't rant about the knowledge cutoff date here. I accept the limitation, but it bothers me when it gives me wrong information when I ask it about events that took place well before its knowledge cutoff date. I asked the E2B variant of the Gemma 4 to list all the cricket teams that have won the Indian Premier League (IPL), and it gave me teams that don't exist in franchise cricket. That's how outrageous the answer was. I even confronted it and said that it was wrong, but to no avail. I tried to give it multiple clues but still couldn't get the right answer. This is not just about cricket. The local LLM I tested gave me bad responses most of the time when I asked about past events. A series of bad responses can significantly lower your confidence. You'll keep doubting the answers for topics you don't know, but what makes it worse is that you can't fact-check that information using the internet without leaving the app. I also noticed it gave me unnecessarily long answers even for the simplest of queries. It lacks awareness of when to keep things concise. Instead of requiring me to intervene and ask for a concise answer when needed, I would've preferred a more automatic approach, where it would know when to be comprehensive and when to keep things short. You'll sometimes need a lot of patience to get the best results. A local LLM can't replace a cloud AI, but it can help you rethink your subscription A cloud AI subscription gives you access to many powerful features. But most people don't need them. Instead, drafting emails, writing to a friend, and helping set the right tone in writing have broader appeal. There is no need to pay a fee to use AI for these kinds of tasks — a local LLM is good enough. For daily productivity tasks, a local LLM works better because of its offline capability. For the way I use AI, a local LLM can't replace a cloud AI subscription. But after this experiment, I'm more open to the idea of ditching mine.