AI agent for searching, collecting and analyzing information for publication in media channels

Disclaimer: despite the topic and essence, this article was written without the use of AI — purely by hand and purely from real experience of using the capabilities of artificial intelligence. But let's be honest, today everyone uses them, and the list of proposals from various neural networks is only growing. And if earlier people were happy to talk with AI and think about future possibilities of use, now a lot of things already work on a daily basis. What is interesting: many do not even think about where and how, although all this is right in front of our noses. And this is literally — on the screens of smartphones or computers, which we regularly look at.

It is important to note that AI agents, applications based on artificial intelligence technologies that make decisions and perform specific tasks with minimal human intervention, have played a major role in the spread of such capabilities. Without AI agents, most truly powerful services would simply remain extremely intelligent machines, waiting for a user to contact them.

Why are AI agents so smart and in demand? It's very simple:

• An AI agent can make decisions, i.e. achieve the goal set by its owner, using various methods. This is a major advantage over a “dumb” algorithm that stops working when it reaches a dead end;

• AI-powered applications can leverage other tools, such as programming languages, to create the software components needed to perform tasks. This essentially removes all boundaries in scalability;

• Several AI agents can combine capabilities and work on tasks together. And each will only improve its own experience and accumulate knowledge.

All that remains is to define the goal, find the appropriate technologies, and combine all this with a backend in a single application that will tirelessly work on the assigned tasks. The Avivi team has already seen that if done correctly, the successes of an AI agent will pleasantly surprise its owner with its effectiveness.

How does an AI agent work to automate a news site?

Information nowadays is a rather “trendy” product. Each person consumes an incredible amount of content per day and constantly wants more. There are also an incredibly large number of sources of information: some create information messages, others distribute them. The editorial staff of a news site has to process huge amounts of information every day, from official press releases received by mail to the adaptation of relevant messages found elsewhere, for example, international news. Searching for interesting things takes up to 30% of working time, preparing your own messages takes about 60%, and the rest is needed to publish your own product. In general, it is a routine and laborious task for employees. However, it is elementary for AI.

Let's summarize the task outlined by the cycle:

• Select or find the necessary information;

• Adapt content for your own audience;

• Spread the message.

This is quite enough to configure the work of an AI agent for news aggregation with interpretation for the style and direction of a particular media outlet.

Our development is implemented for regional media, the audience of which lives within one region of Ukraine. Therefore, the sources of information were the websites of local administrations (regional, city, village), portals of state services (National Police, State Emergency Service, Health Department, etc.) and municipal media. Also to this list should be added regional Telegram channels and official pages of people and organizations on the social network Facebook, as well as various groups on the same Telegram and Facebook that publish industry news.

A custom AI agent written in Python is responsible for collecting and distributing information. The solution easily interfaces with other software capable of browsing website pages and reading social networks. Appropriate settings and granting the agent access to Internet resources turn it into a meticulous assistant that performs most of the operational work.

The logic of this AI agent is quite simple:

• Regularly (every 23-30 minutes) check sources from a previously defined list of sources;

• Analyze the news for compliance with the requirements of the editorial policy (it is easier to take what is interesting);

• Check if such a message has not already been sent;

• Form future news in interaction with another AI agent and send a version of the publication to the editor-in-chief for approval;

• Publish the news, if approved, in all available channels, including, in fact, the news site, Telegram channel, and Facebook page.

As you can see, the custom AI agent is integrated with Claude from the developer Anthropic. As the saying goes, “Who studied for what?”, there is a separation of duties in the software, since different products are “sharpened” for separate types of activity. Claude clearly understands the stylistic requirements for the text and has its own editorial style, trained on hundreds of already written materials. So the result is not a primitive “ copy-paste ”, but a real deep rewrite of each future publication.

During the approval stage, the editor, of course, has the opportunity to make edits to what was collected and written by the agent — this process takes place in a Telegram bot with a custom assistant. But in general, an AI agent can successfully work independently without human intervention.

A little bit of technicality

What is important: An AI agent does not work with information sources via API, as news aggregators do. In the usual sense, an agent would send a request to the information source server, for example, “GET https://news-site.com/api/articles” and receive a response in JSON or XML format. This is already structured information, where there is a news headline, main content, time stamp, author, etc. — everything is clear, understandable and standardized. However, the disadvantages of API can be:

• request limits — for various reasons, resources may set limits on the number of requests, after which server responses will stop;

• traffic load — the server may not be able to respond to each request in time;

• various restrictions: if the source doesn't want to share what was published via the API, there's no way to get this data.

The AI agent does not use APIs or even RSS — the solution actually visits source web pages and social networks, “reads” and analyzes the content directly from the original source, like an average live user.

Another feature is related to Telegram. The AI agent interacts with channels and groups not as a program code, but on behalf of an ordinary user. To do this, the backend enters the service with its own authorization — that is, with a login and password. Therefore, it is important that the account that the AI agent represents is subscribed to the necessary channels and is a member of the necessary groups. Otherwise, the AI agent will not be able to receive and analyze content updates.

The Python backend works completely automatically. The user does not need an admin panel to configure the AI agent. This is how the script works “under the hood” when the next cycle of news update analysis begins (once every 30 minutes):

Selected news items are sent to the manager's Telegram bot for approval. Now there is an opportunity to approve the news item for further preparation and publication or to skip the offer. Here's what it looks like:

And finally, the most interesting thing: are there limits to improving automation? The answer is practically no, because each of the operations can be scaled to new conditions. For example, add translation into other languages or expand the list of sources. Deep knowledge of Python programming allows Avivi's development team to optimize the backend for interaction between AI agents to make the most effective use of the capabilities of each of them.

So, if you want to enlist the support of your own artificial intelligence-based agent, we will be happy to help.