Taming the Giant: SEO Lessons from a 200,000+ URL Website Migration

Website migrations are complex enough. Add 200,000+ eCommerce URLs into the mix, and it reaches a whole new level of difficulty.

But, with the right team by your side, it doesn’t have to be impossible.

Today, we’ll walk you through a recent website migration project with one of our biggest eCommerce clients, sharing the unique challenges faced by this brand and the lessons learned along the way for managers of similarly sized sites.

Let’s get started:

Get help with your site’s migration by partnering with our SEO specialists today.

Client Background

Fire Mountain Gems and Beads is an eCommerce business that sells jewelry supplies, gemstones, and high-quality beads at wholesale prices. 

Its large product inventory means its website hosts more than 200,000 URLs, which makes any SEO efforts that much more complicated.

In early 2023, FMGB approached Inflow about an upcoming website migration. They wanted an expert eye during their move to Salesforce Commerce Cloud, with goals to streamline their site and retain as much organic traffic as possible post-migration.

So, we rolled up our sleeves to get started.

Yes, Size Does Matter

No website migration is easy — but, when you throw hundreds of thousands of URLs into the process, it quickly becomes a beast of a project for even the most talented of internal marketing teams.

Here at Inflow, we follow a detailed website migration checklist for every client, regardless of website size. This helps us find and fix the most common errors affecting most sites during and after a migration is complete.

However, we often identify unique website challenges along the way — and FireMountainGems.com was no exception.

Below, we’ll detail a few of the specific roadblocks we encountered during FMGB’s migration. The common thread? In one way or another, they were all a result of the site’s massive size.

Challenge #1: Prioritization

With a site the size of FireMountainGems.com, reviewing every page before a migration is impossible — and not nearly worth the time it would take to do it well.

Instead, we started with the site’s biggest hitters, which would have a disproportionate effect on performance after the migration was completed.

Using a Screaming Frog scan that took several days to run, we segmented the site’s top performers by:

• Organic traffic
• Organic conversions
• Organic visibility
• Backlinks

From there, we created a priority list of the top 20,000 URLs — the pages we would focus our attention on when planning and implementing the migration.

Challenge #2: Redirects Upon Redirects

In the original iteration of FireMountainGems.com, URL structures were fairly simple. But the updated architecture of the new site didn’t translate one-to-one, which meant redirections were in order.

An example: FMGB’s original category page for gold-filled beads was /shop/gold_filled-beads.

Its corresponding URL on the new site was built out a little differently, including the URL folder pass and refinements (i.e. category facets).

This mismatch applied across many of FMGB’s category pages. Setting up 301 redirects was crucial — but needed to be done on a much larger scale than for a smaller site.

With much attention and care, our team implemented the redirects. After an initial period in which Google recrawled the redirects, traffic began to recover (and, in the case of this specific category page, surpass original traffic levels!).

Challenge #3: Maximizing Crawl Budget

When reviewing the updated site in staging, we encountered a new challenge: a faceted navigation feature that created numerous unique URLs.

When a user selected a filter on a category page, the CMS would generate a new URL that incorporated that selected filter. As additional filters were applied, the CMS would continue to create new URLs based on these selections.

In turn, each category could theoretically have an unlimited number of URL variations, depending on the combinations of filters chosen.

Not only was this messy to deal with from an organizational perspective, but it also could have serious effects on the website’s crawlability and SEO rankings.

Google limits the number of pages that are crawled and indexed on each site, which is crucial when it’s dealing with massive websites like FireMountainGems.com. However, if that crawl budget is not directed properly by the website managers, it can be wasted on pages that drive little traffic — leaving key product and category pages behind and resulting in a direct hit to site revenue.

This was a real concern for Fire Mountain Gems and Beads and a high priority for our team.

As we did before, we started by identifying the most important pages with applied refinements, reviewing key metrics like organic traffic and revenue. We discovered that our priority pages all fell within two refinements (the main page with an applied color/material filter).

So, we added a disallow within our robots.txt file to prevent Google from crawling any URLs with three or more refinements.

This way, we could better control the number of pages that Google saw on the site and prevent wasted budget on URLs of secondary importance.

Learn more about best practices for eCommerce crawl budget here.

Challenge #4: Overlooked Duplicates

When building a proper URL structure for these endless refinements, we uncovered another error in staging:

As those category URLs with conditions (color, size, etc.) were generated, title tags were not automatically updated — leading to hundreds of pages with the same title tags.

To prevent the headache of manually updating these category pages’ metadata in the future, we caught and fixed the problem by appending the refinement into the title tag to make them unique from the moment of creation.

Setting the Stage for Future SEO Growth With a Smart Migration

Even the smallest of errors during migration can lead to massive amounts of cleanup when you’re dealing with a site the size of FireMountainGems.com.

Fortunately, FMGB’s team had the insight to bring along an SEO agency from the beginning to minimize these mistakes, which will pay off in dividends as the new site gains more organic visibility in the months to come.

With these errors resolved, the FMGB team will have more time to devote to on-page SEO — and our team will be there to help them every step of the way.

Have an upcoming migration of your own? Avoid common mistakes like these and more by partnering with Inflow from the start.

Schedule your free consultation now to learn more.