Hello everyone! My name is Alexey Malafeev, and I am a Senior SEO Specialist formerly at Detskiy Mir. I have also worked as a lead at KazanExpress (currently Magnit Market). In this article, I aim to showcase practical case studies on how to simplify and automate work processes within the SEO operations of a large e-commerce project. This includes tasks such as monitoring the technical health of a website, analyzing traffic patterns, handling large volumes of data, and managing category seasonality.
In my experience, various projects present both common and recurring tasks as well as unique challenges arising from specific business characteristics, site architecture, or internal processes. However, one common issue with large-scale projects is the overwhelming amount of data that surpasses manual handling capacity. Large e-commerce sites comprise hundreds of thousands to millions of pages, numerous queries, URLs, technical parameters, and regular updates. Standard tools eventually reach their limits.
In my case, a simple moment triggered my transition to more advanced methods: the need to export all indexed pages of a website no longer fit on a single Excel sheet, which has a limitation of 1,048,576 rows. Initially, tasks were manageable with Power Query. As challenges grew in complexity over time, requiring data processing, saving history, conducting regular checks, cross-referencing varied sources, and obtaining results automatically, I gradually shifted towards Python, SQL databases, APIs, creating dashboards in Yandex DataLens, and Power BI.
Today, in addition to the existing toolset, AI tools expedite specific task resolution. The article will illustrate some results derived from this evolution. One of my initial Python scripts sorted a list of URLs based on their page types concerning the URL structure. This categorization was essential for distinguishing the myriad types of pages found on larger websites beyond mere listings and products. Having a quick way to sort and identify these pages based on their types is crucial for problem resolution efficiency.
For instance, when dealing with technical issues highlighted in Google Search Console under the Pages section, extracting a full list of pages from the interface is not feasible due to row limitations. However, analyzing even a few hundred or thousand URLs can offer insights into technical errors such as redirects, 404 errors, canonical issues, etc. The script delineates page types, page quantities, and their percentage share of the total pages, aiding in prioritizing tasks accordingly.
On larger projects, automating repetitive tasks like monitoring sitemap.xml changes is crucial. The disappearance of numerous pages from sitemap.xml without apparent reasons signals trouble. Therefore, I implemented a script to parse sitemap.xml daily, storing data in a database to track changes over time and detect anomalies by comparing with other datasets.
In managing SEO tasks, the ultimate goal is to support business objectives. Collaborating and sharing data with relevant departments is vital. Different product categories exhibit diverse seasonal patterns, and timely identification of demand peaks aids in strategic planning. A structured approach involving semantic templates, historical data analysis, and dashboard creation facilitates seasonality analysis at various levels, benefiting not only the SEO team but the entire organization.
In conclusion, by focusing on identifying problems, determining required data, and selecting suitable tools for solutions, a tailored approach can streamline operations effectively. Leveraging simple scripts or SQL queries can save time, enhance monitoring, and ensure regular oversight of critical aspects. The presented strategies aim to enhance technical SEO practices for large-scale e-commerce projects.
About the Author
