Magento 301 Redirect Mapping: Ultimate Guide (2020)

301 redirect mapping for Magento ecommerce stores overall follows the same steps like for any other ecommerce platform. Therefore, in this guide you will find suggestions which might be useful regardless of the CMS and some tips and tricks to take into consideration when working specifically with Magento 2 platform.

What is a 301 redirect

301 redirect is a rule which forwards bots and users to another URL. 301 redirect is also called a permanent redirect as it suggests to search engines that the old URL is gone for good and link equity should be passed to the redirected page.

If you are looking for additonal information on various types of status codes besides 301 redirect can check Google documentation on Errors and Status messages.

Why proper 301 redirects are important for ecommerce

If somebody moves to another city and doesn’t update his address all the correspondence still will be sent to the old house and most likely won’t reach the recipient. Probably the postbox mostly will be filled with random promo materials, however, what about bills, official documents, maybe even a handwritten letter from some older relative etc.? Not suggested to ignore these.

Same goes with URLs. If the content of the page is moved to another URL and there is no 301 redirect in place the website visitors and crawlers will land on 404 error (Not found) page meaning they won’t reach the product they might want to buy, category they want to explore or blog post they want to read. It directly influences user experience and conversion rate as well.

If 301 redirect mapping is neglected all traffic channels might be harmed:

Organic traffic. 301 redirects pass the value of backlinks to the new destination URL. Various sources suggest that 90-95% of the backlink link value is passed via 301 redirects which is definetely a significant amount. If 301 redirects are not implemented correctly old pages will return 404 error and sooner or later will disappear from SERP. At the same time new URLs even thou having the same content may not reach top positions.
Paid traffic. If not updated instantly paid advertisements can also have outdated URLs and lead to 404 pages. Same applies to, e.g., email marketing. If on Monday there was sent a newsletter filled with links to products and offers and on Tuesday URLs were updated, then people who might open Newsletter on Wednesday will get 404.
Referrals. Social media posts, blog posts, press releases, forums etc. all might end up having outdated links.
Direct Traffic. Clicks on bookmarks, browser history etc. are attributed to Direct traffic in Google analytics.

TIP. For ecommerce stores that often refresh their product stock and end up having 404 pages it is suggested to invest time in 404 page optimisation. E.g., include bestseller products, related products etc. to make sure user is quickly guided to something similar they tried to find initially.

Here is an example of website migration to Magento 2 which was performed without taking care of 301 redirect mapping.

First, we can see how overall traffic decreased in all channels after Go-live, The website especially suffered from huge descrease of the organic traffic. Need to note that in general M2 version of the specific website initially was filled with bugs, errors etc., which affected the overall performance of the website and SEO on top of missing 301 mapping.

Besides the direct loss of the traffic we can see an immediate spike in 404 errors for those visitors who still visited the page. Therefore, conversion rate decreased as well.

When you need to think about 301 redirect mapping

In general whenever URLs are affected you need to evaluate if 301 redirect mapping is required. The most common situations when you might need to implement 301 redirects are following:

Migration to new platform – Magento to Magento 2, Shopify to Magento etc.
Domain name change (includes also subdomain or subdirectory) changes – example.de to example.com, example123.com to example.com, etc.
Menu changes or website structure updates.
Http to https migration

TIP. It’s also suggested to always consider a possibility to preserve existing URLs at least for top performing pages to minimize the fluctuation in rankings.

Magento 2 level versus server level redirects

There are 3 levels of redirects you can apply:

Magento level redirects in the admin panel
Server level redirects (Nginx, Apache etc.)
Client side redirects

1. Magento or app level redirects

When to use?
When you want to redirect one or few pages within the existing domain name. For example, you are merging 2 categories, one of the categories should be deleted and 301 redirect set to another category.

Pro

Simply done and in future can be easily edited if needed, no developer help is required
It’s also possible to enable automatic redirects in Magento 2 if existing categories or products have changes in URL keys.

Cons

By default Magento 2 allows to map only HTML pages. Meaning you can’t map URLs which have file/extension as part of the URL, e.g., URL path media/photo1.jpg can’t be redirected to media/photo2.jpg
With the help of developers a larger list of redirects can uploaded directly in database, however, if handling manually it can take quite a lot of time to enter each redirect independently. There is a possibility of human error which can lead to redirect chains or wrong URLs included in mapping.

2. Server level redirects.

Possible to redirect URLs one by one or set a rule using regex which can help redirect big number of URLs at once.

When to use?
Some popular cases include:

In case of migration to another domain or subdomain.
Big amount of URLs with clear rules, e.g., all URLs containing subdirectory /2019/ should be redirected to the same URL without subdirectory. Or all URLs ending in .html should be redirected to version without.
Migrations from http to https

Pro’s

Effective way to handle extensive changes in URL structure

Con’s

Complicated to edit
Require knowledge of regex and help of developers

Client side redirects

This type of redirect is performed in the level of a web browser (client side). Such redirects include JS redirects, meta refresh etc. In general it’s suggested to avoid usage of such redirects as search engine crawlers might not follow such redirects.

301 redirect mapping steps in case of sitewide migration

Overall, if changes in URL structure are implemented, all indexable URLs should be redirected using 301 redirects. The bare minimum is to make sure that at least the following crucial URLs won’t return a 404 error page:

Landing pages which are receiving traffic, especially organic. Mapping such pages will support both SEO and overall user experience. Not only organic traffic, but also referral traffic, social media, bookmarks, email campaigns and other sources of traffic might rely on old URLs. If the user lands on a 404 page instead of the correct one, it will cause friction and could be a reason for not completing the intended purchase.
URLs with valuable backlinks. Each backlink from a resource with high authority adds value to a website from an SEO perspective. More resources pointing to a specific page make it more trusted and authoritative in the eyes of Google. If a backlink leads to a 404 error page, the value is lost. Therefore, in cases when an URL has a lot of backlinks, even if the page doesn’t receive much traffic and won’t be present after the migration is done, there should be a 301 redirect in place.

How to handle Magento 301 Redirect Mapping

There are 6 main steps for coming up with a redirect map:

Understanding the scope.
What, when and how is being migrated. Good understanding of changes in URL structure as well as principles how new URLs are generated will help be more efficient.
Gather. the full list of new URLs and the full list of old URLs.
Compare. Place the two lists side by side in a single sheet (Google Spreadsheet or Open Office).
Map. Now, match each old URL with its corresponding new URL.
Hand Off.Once you’ve developed a full 301 redirect map, give it to the developers to import in Back-end
Test before and after go-live.One missed page can directly lead to a decrease in website traffic and conversion rate.

Preparations for Magento 301 Redirect Mapping

Software and accesses.

Make sure you have the following:

Screaming Frog or any other SEO crawler
Open Office (or Excel)is quite good software to work with a big amount of cells (when working with large ecommerce stores with many products). Google spreadsheet and Excel may crash or/and be slow.
SEO tool with information about backlinks, e.g., Ahrefs, Semrush, Moz or other
Project-specific accesses. Google Analytics and Google Search Console accesses of the project. For a full analysis, it’s possible to analyze also server log files.

Information on project

To avoid surprises, it’s very important to collect information about the scope of the work. Especially if SEO specialists are involved closer to the end of the migration, instead of working since the very beginning of the project.

Migration type. What type of migration is planned and does it involve major changes in URL structure (e.g. address change from example.eu to example.com).
The scope of work. Exactly which domains, subdomains, language versions, related websites (landing pages, blogs etc.) are planned to be migrated.
Changes in content. Is it planned to migrate all content? Maybe some set of pages will be deleted? What about product categories? Blog posts? Often not all products are being migrated as well, it’s good to clarify approx. percentage of how many products will be actually migrated.
Changes in naming. Are category names staying the same? Are subcategory titles the same? CMS pages? etc.
URL structure. Are there major changes planned regarding URL structure? For example, www.example.com/category/product1-123.html to www.example.com/product1. Pay attention to the endings of URLs and beginnings of URLs. If the old website had .html at the end of each URL and now it no longer does, there should be redirects set for all pages even if, in general, page titles and URL structure stay the same. The same applies to situations when example.com/gb is changed to example.com/en/ and even when example.com/gb is changed to example.com/gb/. Regarding changes which affect the homepage and/or the whole website — consult with developers. They should be able to effortlessly create server-side redirect rules.
Existing maps. There already is often some pre-made map used in the planning phase.
Existing and Staging website walkthrough. Understand how they are different.
Note, even if it’s said that product names will stay the same and as a result, all URLs will stay the same you still need to crosscheck every URL.

Overall Principles of Magento 301 redirect mapping

In the case of ecommerce stores, it’s easier to map URLs grouped by type. The typical structure of Magento stores includes — category pages, subcategory pages, product pages, CMS pages, home page, customer account, and cart/checkout related pages.

Collecting URLs for Magento 301 redirect mapping

You can use the following sources to get URLs:

For products and categories URLs — can ask developers to export URLs from the database
XML Sitemap
Whole website crawl
Google Analytics — top landing pages
SEO tools— URLs with backlinks

Mapping product pages

Example of Magento 2 Product page with the standard URL (https://www.elekcig.de/joyetech-ego-aio-kit)

If a store has thousands of products, they probably had some standard approach to product URL structure for both the old and new websites. In case of Magento 2, it’s typically www.example.com/product1 (product name right after domain name). If the old website had the same approach, then mapping won’t be needed (need to test though if it’s really, really the same using formulas in Excel)

If there are any changes then:

ask developers to export the product URL list from the old website and from the new website
adjust the URLs in excel e.g, If you had www.example.com/products/product1 remove products/ (can use Find and Replace for that) so that structure matches.
compare both lists using vlookup formula to check if all products from the new website can be found in the old one.
If yes, then it’s great! Prepare the document listing old PDP and correlating new PDP URLs.

This method applies if you are sure that the product titles stay more or less the same and the majority of products are being migrated. If the naming of the products doesn’t match, try to find some principles how they are created and try to make them similar. Or use some additional data. e.g., if there is the SKU or ID in the old URL, and this data is being migrated to the new website, ask developers to export SKU next to new URLs and you will get the opportunity to map them www.example.com/products/product1-2939 www.example.com/products/product12

Try to avoid manual mapping of product pages! If it’s only about changes in directories, then the redirects can be applied by developers as one specific rule which applies to all URLs. No need to upload a list with all product URLs in Magento Back-End.

Mapping category pages

If there are changes in category URLs then again try to look for some principles that have changed overall, e.g., directory is excluded or added. Similarly, as with products, it’s important to crosscheck how the naming of categories has changed. There might be some manual mapping required as well.

CMS pages

Usually, eCommerce stores don’t have many custom CMS pages so you might map them manually. You can get such pages by crawling the old and new websites, excluding previously mapped categories as well as product pages. URLs which are left can be mapped.

Others

Make sure the homepage is mapped, if there are changes! Also, blog articles if there are any.

Top Landing Pages

GA is also a great source of URLs that need to be redirected — there might be URLs which are not internally linked and still get some traffic. The top landing pages, which are receiving traffic and are not included in the original redirects map, might need additional 301 redirects.

Select a period of 1–3 months (can be exceptions due to the seasonality of business etc.)
We won’t need Paid traffic as a source, however, there might be URLs related to organic, referral, social and direct traffic worth mapping. So can use these channels as a source of information. The minimum is organic traffic. You can’t be sure that GA data is 100% accurate (it can be broken due to the wrong setup), but you can quickly crosscheck if Google/organic as a source is approximately matching
Navigate to Landing page report and export Landing pages to .csv file
Export pages to the same Spreadsheet Open Office file where the Sitemap is.
Compare the list using VLOOKUP formula.
If some additional URLs are discovered, crawl them in Screaming Frog to check the status of pages (It’s suggested to create 301 redirects also if there is no 404 error)
To crawl specific URLs in Screaming Frog change Mode to List. Click on Upload => Enter Manually…

URLs with valuable backlinks

Identify pages which have backlinks from external sources (more than few and preferably with higher scores) using some SEO tool:

Export the list of URLs
Compare with the existing list of data from sitemap and GA
Crosscheck newly discovered URLs via Screaming Frog
Add to list

XML sitemap

Example of an XML Sitemap

Usually, the sitemap includes those URLs which are considered valuable from an SEO perspective, therefore they should be mapped in the 301 map. In case of Magento stores, if there weren’t used any extensions to generate XML, the sitemap probably will be missing CMS pages, homepage etc. Don’t consider XML sitemaps to be reliable by default!

However, it’s a good way for gathering all product and category URLs, especially if you can’t gather data directly from a database.

To make sure that URLs included in the sitemap are real and existing (no 404 errors, no 301 etc.) the sitemap should be crawled using Screaming Frog. Before downloading the sitemap, you can check the health of the sitemap in the Search Console (SC => Crawl => Sitemap). If SC is reporting some major problems with the sitemap, e.g. 404 errors, the sitemap might be outdated and can’t be trusted.

How to do sitemap crawling:

Find the location of the XML sitemap. Typically sitemaps should be listed in www.example.com/robots.txt. If the sitemap is not included in robots.txt file, check the Search Console (the sitemaps should be submitted there and you will get the specific URL). If you have access to the back-end of the store, you can easily find sitemap there.
Open Screaming Frog. Change the Mode (main navigation menu) to List
Choose Download Sitemap

Crawl sitemap
Export the results of the crawl as a .CSV file
Import to Open Office or Google Spreadsheet.
Delete unneeded columns. Keep — URLs, titles, meta descriptions, response codes, meta robots 1
Hide pages which have other response codes that 200

Full website crawl

To spot all internally linked URLs, it’s possible to do the whole website crawl in Screaming Frog. Be careful as canonicalized pages, pages with noindex tag, URLs with parameters etc. could be included — pages which don’t need 301 redirects, because they weren’t indexed. If there are no external URLs pointing to the URL with parameters and no incoming traffic, it’s possible to exclude it from 301 mapping.

Identifying redirect chains

When you have collected the list of URLs, run an additional crawl and create a redirect chain report in Screaming Frog.

Export the redirect chain report to Open Office/Google Spreadsheet to identify redirect chains with more than 2 URLs.

In such case, you will need to map both the 1st and 2nd URL in the chain.

Compare

After the full list of URLs from the old website is compiled and there is an accessible list of new URLs (can crawl the staging site after the content migration is done) they should be placed in one spreadsheet.

Map

Map means to match old URL with corresponding new URL. If there is no exact match can look for the closest alternative. If some product is missing, can map it to category page etc.

Handoff

Developers should receive one spreadsheet with 2 columns — Old URL and new URL.

Magento 301 Redirect Testing

Run 2 tests:

When URL redirects are implemented in the BE of the staging website, crawl the list to see if all redirects are working as expected.
Create a redirect chain report to make sure that they all report 301 in the first step and 200 status in the second step on the staging website.

After the go-live, run another crawl of your 301 redirects list to check if everything is still working as planned.

Bugs

You still might miss some valuable pages so monitor 404 reports in GA and add additional redirects if needed in the upcoming 2–3 weeks after go-live!

Check out our Magento development and migration services here! Looking for more tips and tricks on Magento 301 redirect mapping? Perhaps you have some you’d like to share with us? Drop us a line at [email protected]!

This article is produced by scandiweb

Lina Ivanova

Magento 301 Redirect Mapping: Ultimate Guide

What is a 301 redirect

Why proper 301 redirects are important for ecommerce

When you need to think about 301 redirect mapping