Yandex News Scraper: Operational Guide, Configurations, and Playbooks

Direct answer

Yandex News Scraper extracts Russian and CIS media coverage by scraping story clusters, individual article outlets, and homepage trending topics from Yandex News via dzen.ru. It supports keyword searches, regional category browsing, and real-time trend capture across five regional domains (Russia, Kazakhstan, Belarus, Uzbekistan, and International). Media analysts, researchers, and PR teams use this Actor to transform unstructured Cyrillic news feeds into structured JSON records containing headlines, summaries, sources, publication dates, and cluster metrics. Because Yandex restricts non-Russian IP addresses, execution requires routing requests through Apify Residential proxies configured with Russian IP addresses.

Architecture and operational modes

Yandex News functions as Russia's primary news aggregator, grouping articles from thousands of publisher outlets around common event clusters. Yandex News Scraper provides access to this aggregated dataset across three dedicated execution modes designed for specific data collection goals.

The searchNews mode executes keyword queries against the aggregator. It handles search strings in Russian or English (such as 'Яндекс', 'Ukraine', or 'Apple') and returns matching story clusters alongside linked publisher articles.

The browseCategory mode navigates designated rubric sections on dzen.ru/news/rubric/. Validated rubric slugs include politics, world, computers, sport, business, science, society, culture, auto, and health. Legacy news.yandex.ru category URLs are converted automatically during execution.

The getTrending mode scrapes the Yandex News homepage panel, collecting real-time trending topic keywords from the 'что обсуждают' panel together with top story clusters.

Practical use cases

These use cases come from Yandex News Scraper's published documentation. Each is expanded into an operating pattern so the Yandex News Scraper output has a purpose beyond collection.

Use case 1: Brand monitoring

Outcome: track Russian and CIS press coverage of your company, product, or executives.

Question to answer: Looking only at confirmed differences, which ones are worth a response and which are just formatting noise?

Configure: Start with maxStories (Maximum number of story clusters to scrape per query or category (1-200).), mode (Scraping mode: search by keyword, browse a category, or get trending stories.), searchQueries (One or more search queries (required for searchNews mode). E.g. 'Яндекс', 'Ukraine', 'Apple'.). Use the narrowest Yandex News Scraper values that represent the real task, keep the first result set small, and record why each selected input matters to the Yandex News Scraper outcome.

Working method: Store every run under its collection date, key records by a durable identifier rather than position or display text, and only escalate a difference once it has been confirmed against two consecutive runs.

Deliverable: Create a change log entry per run listing what appeared, what disappeared, and what changed enough to matter. Include the Yandex News Scraper source identifier and the collected fields behind every Yandex News Scraper decision.

Stop condition: Pause when the same field flips back and forth across runs without a clear cause, or the source's structure shifted mid-comparison. Fix the Yandex News Scraper question, comparison rule, or configuration before expanding the Yandex News Scraper run.

Use case 2: Competitive intelligence

Outcome: see which stories dominate Russian media on any topic.

Question to answer: Once the acceptance rule is fixed, which records satisfy it outright, and which sit in a gray zone?

Configure: Start with mode (Scraping mode: search by keyword, browse a category, or get trending stories.), maxStories (Maximum number of story clusters to scrape per query or category (1-200).), proxyConfiguration (Required. Yandex/Dzen only serves Russian IPs. Use Apify Residential proxy with country=RU for reliable access.). Use the narrowest Yandex News Scraper values that represent the real task, keep the first result set small, and record why each selected input matters to the Yandex News Scraper outcome.

Working method: Write the acceptance rule down before the first record is reviewed, apply it consistently across the batch, and change only one rule or input between batches so any shift in the result has a clear cause.

Deliverable: Create a reviewed queue where every record keeps its raw form alongside the rule that accepted, excluded, or flagged it as uncertain. Include the Yandex News Scraper source identifier and the collected fields behind every Yandex News Scraper decision.

Stop condition: Pause when the same record is classified differently on a repeat pass, or a required field is missing often enough to change outcomes. Fix the Yandex News Scraper question, comparison rule, or configuration before expanding the Yandex News Scraper run.

Use case 3: Journalism & academic research

Outcome: systematic collection of Russian-language news for media analysis.

Question to answer: Which side-by-side comparisons hold up once outliers are set aside, and which depend entirely on them?

Configure: Start with categoryUrls (Yandex News category page URLs (required for browseCategory mode). Use dzen.ru/news/rubric/ URLs. Confirmed working slugs: computers, politics, world, sport, business, science, society, culture, auto, health. Legacy news.yandex.ru/.html URLs are auto-converted.), proxyConfiguration (Required. Yandex/Dzen only serves Russian IPs. Use Apify Residential proxy with country=RU for reliable access.), searchQueries (One or more search queries (required for searchNews mode). E.g. 'Яндекс', 'Ukraine', 'Apple'.). Use the narrowest Yandex News Scraper values that represent the real task, keep the first result set small, and record why each selected input matters to the Yandex News Scraper outcome.

Working method: Set the comparison rule before seeing the results, group records against that rule rather than after the fact, and treat any record that resists grouping as information, not noise to discard.

Deliverable: Create a side-by-side comparison table with cohort labels, normalized fields, and a short note on what the gap implies. Include the Yandex News Scraper source identifier and the collected fields behind every Yandex News Scraper decision.

Stop condition: Pause when the comparison rule shifted mid-analysis, or a single record is skewing an entire cohort's average. Fix the Yandex News Scraper question, comparison rule, or configuration before expanding the Yandex News Scraper run.

Step-by-step extraction workflow

1. Select the operational mode ('searchNews', 'browseCategory', or 'getTrending') based on whether your pipeline requires keyword monitoring, category browsing, or homepage trend tracking.
2. Configure input parameters including searchQueries for keyword runs, categoryUrls for section browsing, regional domain selection ('ru', 'kz', 'by', 'uz', 'com'), and proxyConfiguration set to Apify Residential proxy with country code RU.
3. Execute the Actor on the Apify platform and receive structured JSON outputs.
4. Separate output objects by filtering on recordType ('story', 'article', or 'trending').
5. Relate individual article records back to parent news events by matching storyId keys across the dataset.
6. Export structured JSON records to downstream analytics tools or databases for continuous monitoring.

Configuring input schema and proxy requirements

The input schema controls execution behavior and target routing through the following parameters:

• mode (string): Operational mode. Select 'searchNews' for keywords, 'browseCategory' for rubric sections, or 'getTrending' for homepage trends. Default is 'searchNews'.
• searchQueries (array): List of keyword strings required for 'searchNews' mode (e.g. ["Яндекс", "Apple"]).
• categoryUrls (array): List of Yandex News category page URLs required for 'browseCategory' mode. Must use dzen.ru/news/rubric/* URLs (such as politics, world, computers, sport, business, science, society, culture, auto, health). Legacy news.yandex.ru/*.html URLs convert automatically.
• domain (string): Regional Yandex News edition to target ('ru' for Russia, 'kz' for Kazakhstan, 'by' for Belarus, 'uz' for Uzbekistan, 'com' for International). Default is 'ru'.
• maxStories (integer): Maximum number of story clusters to extract per query or category (1 to 200). Default is 20.
• sortOrder (string): Sorting order for search and category results ('newest' or 'popular'). Default is 'newest'.
• language (string): Sets content language preference and Accept-Language header ('ru', 'kk', 'be', 'uz', 'en'). Default is 'ru'.
• proxyConfiguration (object): Required proxy settings. Yandex and Dzen only serve Russian IP addresses. Configure Apify Residential proxy with apifyProxyCountry: "RU".

Output schema fields and data relationships

The Actor returns three distinct record types linked by shared identifiers:

Story records represent aggregated news events. Fields include recordType ("story"), storyId, headline, summary, category, topicTags, articleCount, domain, storyUrl, and scrapedAt.

Article records represent individual media publications within a cluster. Fields include recordType ("article"), storyId, headline, summary, sourceName, sourceUrl, articleUrl, publishedAt, domain, and scrapedAt.

Trending records capture real-time homepage topics from the 'что обсуждают' panel. Fields include recordType ("trending"), topics array, domain, and scrapedAt.

The articleCount field reports the total number of publications Yandex registers for a story cluster and is not limited by maxStories. Discrepancies between articleCount and the number of extracted article records occur when individual articles are paywalled, geo-blocked, or de-listed by Yandex.

Frequently asked questions

How do story and article records link together?

Every article record includes a storyId field matching the storyId of its parent story record. Downstream systems can filter outputs by recordType and join articles to parent stories on this shared string key.

Which regional Yandex News domains are supported?

The Actor supports five regional edition domains: ru (Russia), kz (Kazakhstan), by (Belarus), uz (Uzbekistan), and com (International).

Why is proxy configuration mandatory for Yandex News scraping?

Yandex and Dzen enforce strict geographic routing and block non-Russian IP addresses. Setting proxyConfiguration to use Apify Residential proxy with apifyProxyCountry: "RU" ensures stable extraction.

What does the articleCount field represent?

The articleCount field indicates the total number of publications Yandex reports for a news cluster, independent of maxStories. Extracted article records may be fewer than articleCount because Yandex omits paywalled, geo-blocked, or de-listed articles from the rendered view.

Can I search using English keywords?

Yes. You can input English search strings in searchQueries and set language to en. The scraper will return English-language publications indexed by Yandex News.

Resources

• Yandex News Scraper on Apify