Журналист заменил Perplexity локальной нейросетью на Android: плюсы и минусы эксперимента

I have eight years of experience covering Android, with a focus on apps, features, and platform updates. I love looking at even the minute changes in apps and software updates that most people would scroll past, and sharing my opinion unabashedly. My background in tracking Android updates and app ecosystems gives me an instinct for what's worth paying attention to, especially the changes that companies don't always openly talk about. When I need to search the internet, I don't open Google anymore. Instead, I use Perplexity, though it isn't my newfound love — it replaced Google Search on all my devices a long time ago. Although I feel Perplexity hasn't improved at the pace it should have, thus lagging behind its competitors, I keep coming back to it for its awesome web search results. However, that's not the only instance when I use Perplexity. I often use Perplexity for questions I don't require a web search for. I needed to break that habit, so I came up with a plan to see whether something else could take its place for questions I don't need a web search for. A local Large Language Model (LLM) was the only solution. I did it on my Android phone. I replaced Perplexity with a local LLM on Android to get instant answers without an internet connection. Here's what I found in my experiment. Quiz 8 Questions · Test Your Knowledge Perplexity and local LLMs Trivia challenge Think you know your AI search engines and on-device language models? Put your knowledge to the test. AI SearchLocal LLMsModelsPrivacyTech 01 / 8 AI Search What is Perplexity AI primarily designed to do? Correct! Perplexity AI acts as an 'answer engine,' combining live web search with large language model reasoning to deliver sourced, conversational responses. Unlike a traditional search engine, it synthesizes information rather than just listing links. Not quite. Perplexity AI is best described as an answer engine — it searches the web in real time and uses LLMs to present cited, conversational answers rather than raw search results. Image generation and audio transcription are handled by other specialized tools. 02 / 8 Local LLMs Which open-source tool is most widely used to run large language models locally on a personal computer? Correct! Ollama has become the go-to command-line tool for pulling and running LLMs locally, making models like Llama, Mistral, and Gemma accessible with just a few terminal commands. Its simplicity and broad model library have made it a community favorite. Not quite. While LM Studio and Jan.ai are excellent GUI-based options, Ollama is the most widely adopted tool for running local LLMs, especially among developers who prefer a lightweight command-line workflow. It supports a huge library of models and integrates easily with other apps. 03 / 8 Models Meta's open-weight model family, which powers many local LLM setups, is known as what? Correct! Meta's Llama (Large Language Model Meta AI) series — including Llama 2 and Llama 3 — has become the backbone of the local AI movement. Its open-weight releases allowed developers and researchers worldwide to fine-tune and deploy powerful models without cloud dependency. Not quite. The answer is Llama, Meta's open-weight model family. While Falcon (from TII), Mistral (from Mistral AI), and Gemma (from Google) are all prominent open models used locally, it is Meta's Llama series that ignited the local LLM revolution when it was first leaked and later officially released. 04 / 8 Privacy What is one of the biggest privacy advantages of running an LLM locally rather than using a cloud-based service like Perplexity? Correct! When you run an LLM locally, all processing happens on your own hardware, meaning sensitive prompts, documents, and conversations stay entirely on your device. This makes local LLMs especially appealing for handling confidential business data or personal information. Not quite. The key privacy benefit of local LLMs is that your data never leaves your machine — no prompts are sent to external servers, and no company can log or analyze your queries. Cloud services like Perplexity, while convenient, transmit your input to remote servers to generate responses. 05 / 8 AI Search Perplexity AI controversially introduced a feature in 2024 that summarized web pages without sending traffic to publishers. What was this feature called? Correct! Perplexity's 'Pages' feature generated fully formatted articles from web research, drawing criticism from publishers who argued it replicated their content without driving referral traffic. The controversy reignited broader debates about AI and fair use in the media industry. Not quite. The feature in question was called 'Pages,' and it allowed Perplexity to produce polished, article-style summaries of topics sourced from the web. Publishers like Condé Nast and News Corp raised objections, arguing it scraped and repackaged their content without compensation or traffic. 06 / 8 Tech What does the term 'quantization' mean in the context of running local LLMs? Correct! Quantization compresses a model by representing its weights in lower bit formats — such as 4-bit instead of 16-bit — dramatically reducing RAM requirements with minimal quality loss. This technique is what makes it possible to run surprisingly capable models on consumer hardware like a gaming PC or even a MacBook. Not quite. Quantization refers to reducing the precision of a model's numerical weights (for example, from 32-bit floats down to 4-bit integers). This shrinks the model's memory footprint significantly, enabling large models to run on hardware that would otherwise struggle, at the cost of only a small reduction in output quality. 07 / 8 Models Which company developed the Mistral 7B model, one of the most celebrated efficient open-weight LLMs? Correct! Mistral AI, a Paris-based startup founded in 2023 by former DeepMind and Meta researchers, released Mistral 7B and stunned the AI community by outperforming much larger models on several benchmarks. The company has since released Mixtral and other models that continue to push the frontier of efficient open-weight AI. Not quite. Mistral 7B was created by Mistral AI, a French startup that launched in 2023 with backing from prominent European investors. Its founders came from DeepMind and Meta, and the company quickly gained a reputation for releasing highly capable models that punch well above their weight class. 08 / 8 Local LLMs Apple Silicon Macs have become popular for running local LLMs. What hardware feature makes them particularly well-suited for this task? Correct! Apple Silicon's unified memory architecture means the CPU, GPU, and Neural Engine all share the same high-bandwidth memory pool. This allows a Mac with 32GB or 64GB of unified RAM to load and run surprisingly large quantized models at usable speeds, making it one of the best consumer platforms for local AI inference. Not quite. The key advantage is unified memory — Apple Silicon chips share a single fast memory pool across the CPU and GPU rather than having separate pools. This means a MacBook Pro with 64GB RAM can effectively use all of that memory for model weights, something traditional PC setups with discrete GPUs and their limited VRAM cannot easily match. Challenge Complete Your Score / 8 Thanks for playing! My local LLM was good at little things that don't need the internet It was still unreliable Google's Edge Gallery app is, hands down, the easiest way to run local LLMs on your Android phone, so I didn't waste time looking elsewhere and installed the app on my phone. Installing Gemma 4 E2B was just as easy. All I had to do was download it from the Edge Gallery app, and it was ready to answer my questions. After struggling for a few minutes to decide which prompt to use, I gave it a bunch of high school math problems. While it passed the test with flying colors, it wasn't easy to read its equation because it wasn't properly formatted. I also noticed that it over-answers even the simple prompts. I asked it to write a quadratic equation, and it responded with several forms of the equation, which I clearly didn't ask for. I haven't checked, but I'm sure Perplexity would give me a much shorter answer with an explanation of what it means. I don't do math anymore, so it isn't a deal-breaker for me. But what was most disappointing is how much it hallucinates, even when answering factual questions. Since its knowledge cutoff date is January 2025, I asked about events in 2024 or prior, and to my surprise, it gave me wrong answers multiple times. In my next prompt, I uploaded an image of my balcony and asked it to suggest colors to paint my walls. I really loved the comprehensive answer this time, as it gave me all the options for different moods based on the color of the floor tiles and the surroundings. I can surely consider Gemma 4 E2B my go-to for interior tips when I need to give my bedrooms and balcony a makeover in the future. I won't need Perplexity for this. I also love its ability to summarize long text. It's also good at pulling specific information from a block of text and brainstorming concepts. I am never going back to Perplexity for these types of tasks. I couldn't stop reaching for Perplexity Even when I didn't need the web Old habits die hard, and I kept reaching for Perplexity out of habit in a week-long experiment more times than I would've liked. However, I won't blame it entirely on my habit. The local LLM didn't immediately replace Perplexity because it simply wasn't as good. It performed well sometimes, and gave me wrong and painfully long answers in other instances. So, it was a combination of habit that I had formed and the limitations of the local LLM. Another major problem with the local LLM was that it wasn't a seamless experience to resume chats. When I restart or switch to it from another app, it takes some time to initialize the model. Also, finding your old chats isn't as easy as it should be. I also had to switch to Perplexity in the middle of a conversation while using the local LLM when I wanted to ask follow-up questions requiring current information. This suggests that I must limit the local LLM usage to specific tasks, like tips to do something, summarizing texts, drafting emails, and brainstorming ideas. I still need Perplexity, but not for everything A local LLM can't replace an AI model that browses the internet. However, local LLMs can handle many things that don't require a web search. Sure, it has its flaws, but I'm hopeful that its accuracy will improve with future updates. This doesn't mean I'll wait for it to get better. Instead, I'll keep using it for everything that it is good at right now. Even if it doesn't improve, its ability to run locally and not send your data to the cloud makes it worth using, especially if you take your privacy seriously like me. I'm staying.