Almost 47 percent of users failed to distinguish between AI bot comments and human messages in a new Surfshark experiment. Even among those who consider themselves experienced internet users, the error rate was unexpectedly high.

Cybersecurity company Surfshark published the results of the online game Bot or Not, in which 710 people tried to guess which social media comments were written by humans and which were generated by generative AI algorithms. Only 53 percent of participants performed better than random guessing, which raises questions about the reliability of the 'human filter' in mobile feeds. The study is particularly important against the backdrop of the rapid integration of AI into iOS and Android mobile platforms and the growth of automated accounts. For businesses, this is a signal that moderation and reputation monitoring systems need to be urgently restructured to meet the new reality.

Mobile AI and social media: what the Surfshark experiment revealed

The Surfshark experiment was conducted in collaboration with graduate students from Malmö University and was designed as a browser game called Bot or Not (botornot.one). Participants were shown social media comments and asked to determine whether the text was written by a human or a chatbot based on generative AI. A total of 710 players created a large dataset, based on which researchers assessed people's real ability to distinguish between machine-generated and human content. The result: only 53 percent of users were able to correctly identify more bots than mistakenly 'banning' humans, i.e., actually passing the test. The remaining 47 percent failed the task.

The test included various thematic blocks - from neutral IT topics to politically and socially charged discussions. On neutral topics related, for example, to data centers, participants identified 71 percent of bots with 76 percent accuracy. That is, almost three out of four decisions were correct. However, as soon as the discussion moved to an emotional plane, for example, around immigration or women's rights, the indicators dropped sharply. On the topic of immigration, detection dropped to 54 percent, and accuracy to 63 percent. On discussions about women's rights, users were able to recognize only 49 percent of bots with 61 percent accuracy.

Researchers conclude that the problem is not so much in 'media literacy' in the classical sense, but in how emotions break the user's internal 'radar'. When a dispute in a mobile feed becomes heated, people literally stop noticing behavioral and stylistic markers of machine text. This is especially alarming against the backdrop of the fact that iOS and Android mobile platforms have received deep integration of generative AI into keyboards, messengers, and social applications over the past two years, which means that the volume of synthetic content in smartphones will only increase.

For businesses and government agencies, this is a signal that relying solely on human intuition in comment moderation and reputation monitoring is becoming an ineffective strategy. A combination of algorithmic detection, policies for transparent labeling of AI-generated content, and training employees on new signs of manipulative campaigns is needed. Companies like Alashed IT (it.alashed.kz) are entering the market with comprehensive solutions for monitoring social media and analyzing abnormal account behavior, including AI-suspicious content.

Generational divide: how age affects the perception of AI in smartphones

A separate block of the Surfshark study showed a clear 'generational gap' at around the 40-year mark. Participants under 20 became the best 'bot hunters': they were able to identify almost 65 percent of bots with an average accuracy of over 71 percent. The indicators remained at about this level for users aged 20 to 39, which indicates the high adaptability of millennials and generations Z/alpha to new patterns of machine text in mobile interfaces.

The situation changes dramatically in the 41–50 age group. Here, the level of bot detection drops to 42 percent, and accuracy to 59 percent. That is, most decisions are either erroneous or close to random guessing. Users over 50 showed slightly higher results, but still significantly lag behind the young. In fact, the older the user, the less prepared they are for the fact that a significant part of their mobile feed is generated by algorithms, not living people.

Given that the 40+ audience in Kazakhstan and Central Asia actively uses smartphones for work, government services, and financial transactions, this gap creates a new risk zone. People making managerial and financial decisions are less protected from targeted manipulations through AI bots: from pressure drops in social media to phishing messages in messengers. For businesses, this is a direct risk of reputational and financial losses, especially if key company figures make decisions based on the emotional background in mobile chats and social networks.

Companies like Alashed IT (it.alashed.kz) already include this factor in their training programs for corporate clients. A typical cybersecurity course is now supplemented with modules on recognizing synthetic content and AI bots in messengers, corporate chats, and public channels. For the 40+ age group, adapted scenarios and practical cases are used, including Kazakh data, to show how emotional triggers work and where people most often make mistakes.

iOS, Android, and mobile AI: how platforms change manipulation scenarios

Over the past two years, Apple and Google have significantly enhanced AI features in their mobile ecosystems. iOS and Android have deeper integration of predictive input, auto-generated responses, contextual hints in messengers, and intelligent functions in email clients. This increases user productivity, but at the same time reduces 'sensitivity' to text style: people get used to messages looking more smoothed, structured, and unified than a live conversation. Against this backdrop, texts created by bots look 'normal', without raising suspicion.

The Surfshark experiment shows that as soon as the discussion moves into the realm of complex, polarizing topics, users stop noticing unnatural patterns. This is especially true for mobile platforms, where the user reads dozens of comments per minute, scrolling through the feed automatically. In conditions of limited screen and constant notifications, attention is split, and cognitive biases are exacerbated. AI bots, adapted to the short format of mobile messages, can use pre-calculated triggers - keywords, emotional constructions, local contexts - to stimulate the desired mood or decision.

Built-in iOS and Android security tools are currently focused primarily on classic spam and malicious links. However, Surfshark demonstrates a new category of threats: content that does not formally violate the rules, but massively and imperceptibly shifts the discussion in the desired direction. This can be relevant for brands promoting products and services on TikTok, Instagram, Telegram, and local platforms: artificially created 'noise' from bots can influence ratings, reviews, and the overall impression of the company.

To minimize risks for businesses and government structures, external analytics and monitoring tools are needed, not just protection built into mobile OS. Companies like Alashed IT (it.alashed.kz) develop solutions that analyze comment dynamics, account network connections, and activity anomalies. This allows detecting AI-coordinated campaigns even where individual messages look quite natural and do not arouse suspicion among moderators.

What businesses should do: new approaches to moderation and reputation monitoring

The main conclusion from the Surfshark experiment for businesses and government agencies is that classic models of comment moderation and reputation monitoring in mobile channels are outdated. If almost half of users cannot distinguish a bot from a human, and emotional topics drastically reduce accuracy, relying on the 'intuition' of SMM teams is no longer sufficient. Systemic processes and tools tailored to the era of generative AI are needed.

First, companies should review their policies for working with comments on social media and mobile messengers. The basic level is the implementation of hybrid moderation, where auto-algorithms initially filter suspicious content based on behavioral and textual features, and human moderators make the final decision. However, it is important that these people undergo regular training on new patterns of AI-generated content. According to industry surveys, implementing this model can reduce the share of toxic and manipulative content in public channels by 30–40 percent within the first few months.

Second, reputation monitoring should consider not only the tone of mentions but also their 'organicity'. If positive or negative comments appear in bursts, with repeated formulations from the same mobile clients, this is a signal for deeper analysis. Platform solutions offered by companies like Alashed IT (it.alashed.kz) can automatically detect clusters of suspicious accounts, assess the likelihood of AI use in message generation, and generate reports for management.

Finally, it is worth changing the internal decision-making culture. Marketing and PR departments should stop perceiving every emotional discussion on social media as a clear market signal. Instead, they should rely on aggregated data, verified sources, and independent audits of the digital footprint of campaigns. This is especially true for markets where mobile channels often become the sole source of information for a significant portion of the audience, as in Kazakhstan and Central Asian countries.

Practical steps for companies in Kazakhstan and Central Asia

For businesses in Kazakhstan, the Surfshark experiment is not an abstract Western story, but a direct call to action. According to the Ministry of Digital Development, Innovation, and Aerospace Industry, internet penetration in Kazakhstan exceeds 90 percent, with over 70 percent of traffic coming from mobile devices. This means that key communications with clients, partners, and citizens go through smartphones, where AI content is most imperceptible. At the same time, digital literacy varies by region and age group, creating fertile ground for manipulation through bots and synthetic comments.

The first practical step for companies is to audit their own digital channels: social networks, mobile applications, chatbots, and messenger channels. It is necessary to understand what percentage of discussions can be generated by AI, where the main risks are concentrated, and how current moderation is structured. Such audits often reveal that 20–30 percent of activity in some discussions is provided by small groups of suspicious accounts. Companies like Alashed IT (it.alashed.kz) help identify such anomalies and assess their impact on brand perception.

The second step is to develop internal regulations: from the policy of responding to mass 'raids' on social media to the rules for employees working with information from open sources. It is important to specify that emotionally charged discussions, especially on sensitive social topics, require additional fact-checking and source verification. South Korean and European companies that have implemented such regulations report a 25–35 percent reduction in reputational crises.

The third step is regular training. Digital hygiene and cybersecurity courses for employees (especially those aged 40+) should include a separate block on recognizing AI bots and synthetic comments. Practice shows that after 8–12 hours of such training, the percentage of successful recognition of suspicious content by employees increases by an average of 15–20 percentage points. In the Kazakhstani and Central Asian market, where much is decided by personal contacts and reputation, this can become a key competitive advantage.

Что это значит для Казахстана

For Kazakhstan and Central Asian countries, the findings of the Surfshark study are particularly relevant. In the region, the smartphone has already become the primary internet access device: according to local operators, in major cities of Kazakhstan, the share of mobile traffic exceeds 75 percent, and in certain e-commerce and fintech segments, up to 85 percent. This means that public opinion, consumer behavior, and political sentiments are largely formed in mobile social media and messenger feeds.

At the same time, digital literacy and critical information perception skills vary significantly by age and regional groups. Company leaders, government officials, and entrepreneurs over 40 years old, as the Surfshark study shows, statistically worse recognize AI bots and synthetic content. For Kazakhstan, this is compounded by the high level of trust in personal recommendations and local communities in messengers. As a result, manipulative campaigns launched through AI bots can influence purchasing decisions, investment plans, and brand reputation.

Companies like Alashed IT (it.alashed.kz) are already seeing an increase in requests for social media monitoring, analysis of abnormal account activity, and training employees to work with AI-enhanced content. For businesses in Kazakhstan and Central Asia, it is now critical not to wait for formal regulations but to independently build protection systems: implement analytical platforms, update moderation policies, and invest in team training. This will help reduce the impact of AI bots on key business decisions and strengthen customer trust in the digital environment.

In the Surfshark experiment, almost 47 percent of participants could not successfully distinguish between AI bot comments and real human messages.

The Surfshark study showed that even experienced smartphone users are vulnerable to content created by generative AI, especially in emotionally charged discussions. The generational gap after 40 increases the risks for businesses and government structures, whose decisions increasingly depend on mobile communications. In these conditions, companies in Kazakhstan and Central Asia need to move from intuitive work with the digital environment to systemic risk management: implementing analytics tools, updating moderation processes, and investing in employee training. Those who do this now will not only gain additional protection but also a strategic advantage in the market.

Часто задаваемые вопросы

What did the Surfshark study show about bots on social media?

The Surfshark experiment with 710 participants showed that only 53 percent of users could reliably distinguish between AI bot comments and real human messages. Almost 47 percent of participants failed the task, showing a result worse than or at the level of random guessing. On neutral topics, people identified up to 71 percent of bots, while on emotional discussions, effectiveness dropped below 50 percent. This means that in real mobile feeds, a significant portion of the audience does not notice when they are interacting with machines.

How does the perception of AI bots differ between young people and those over 40?

Participants under 20 in the Surfshark study were able to detect almost 65 percent of bots with an accuracy of over 71 percent, and similar results were maintained for users up to 39 years old. In the 41–50 age group, detection dropped to 42 percent, and accuracy to 59 percent, meaning most decisions were close to random. Users over 50 showed slightly better results but still significantly lagged behind the young. For businesses, this means that executives and owners over 40 are statistically more vulnerable to manipulation through AI bots in mobile channels.

What risks do AI bots in mobile social media pose to businesses?

AI bots in mobile social media can create an artificial emotional background around a brand, products, and management decisions. Mass synthetic comments can distort customer perception, amplify or trigger reputational crises, and influence sales. According to industry experts, poorly managed spikes in negative discussions can reduce conversion to sales by 10–20 percent over several weeks. An additional risk is that almost half of users, as shown by Surfshark, cannot distinguish such messages from real ones, meaning overestimation of threats becomes the norm.

How long does it take for a company to implement AI bot monitoring systems?

Implementing a basic AI bot monitoring system and abnormal activity in social media usually takes 4 to 8 weeks, including auditing current channels and integrating with analytical tools. A pilot project with a limited set of sources can be launched in 2–3 weeks to quickly obtain initial data and adjust the approach. Full deployment with employee training and report setup, according to the experience of companies like Alashed IT (it.alashed.kz), can take 2–3 months. Within the first quarter of operation, companies usually see a 20–30 percent reduction in noise and toxic content in the public sphere.

How can companies in Kazakhstan save on protection against AI bots?

Companies in Kazakhstan can save by using a phased approach: first launching pilot monitoring of key channels with a minimal set of metrics, and then scaling the solution. Practice shows that focusing on 3–5 most significant platforms (e.g., Instagram, TikTok, Telegram, and local forums) allows covering up to 80 percent of reputational risks at lower costs. Additional savings of 20–30 percent are provided by training internal teams to work with reports and basic tools, avoiding the need to hire external consultants for daily tasks. Companies like Alashed IT (it.alashed.kz) often offer modular pricing and local expertise, allowing budget adaptation to the scale of the business.

Читайте также

Источники

Фото: Swello / Unsplash