Comparing Methods For Moving Sharepoint Content To New Site Collections

The Problem of Content Fragmentation

Over time, SharePoint content has the tendency to become fragmented across different site collections as new sites and libraries are created for various teams, projects, and initiatives. This decentralized approach leads to siloed islands of information that become difficult to manage, secure, search, and collaborate across.

Some of the key challenges with SharePoint content fragmentation include:

• Difficulty finding information – Users struggle to locate documents stored in disparate sites and libraries they don’t have access to or are unaware of.
• Security and compliance risks – With content spread across different sites, securing confidential data and applying consistent policies and retention rules becomes an issue.
• Lack of cohesive taxonomy – Different metadata models applied in isolated sites makes surfacing related content through search problematic.
• Inability to apply global modifications – Making sitewide changes to features, design, and functionality is impeded by distributed structure.

To address these problems, organizations need to restructure and consolidate SharePoint content into unified sites and libraries that bring related information assets together under common governance and access models. This allows users to more easily discover and retrieve documents from one destination while also enabling streamlined security and compliance control.

Methods for Content Migration

There are several techniques available for migrating SharePoint content from multiple source locations into new centralized site collections and libraries:

Manual Migration via SharePoint Interface

SharePoint includes embedded functionality to manually move documents and folders between locations using drag-and-drop or cut-and-paste in the web interface. This allows for ad hoc migration of small sets of files:

• Simple and intuitive for basic moves
• Moves metadata and version histories by default
• Labor intensive and time consuming at scale
• Errors and timeouts likely with very large migrations

Scripted Migration with PowerShell

SharePoint’s PowerShell module includes cmdlets like Move-SPOFile that enable scripting migration jobs to move entire libraries and offline processing for large batches of content:

• Scalable using scripts with logic checking and error handling
• Fine grained control over metadata, versions etc.
• Requires PowerShell expertise to build custom scripts
• Only moves content from accessible SharePoint sites

Migration Tools: ShareGate Migration Manager

Third-party migration tools like ShareGate simplify large and complex migrations through easy-to-use interfaces and additional functionality like query builders and scheduling:

• Easy to use graphical interface for mapping moves
• Automates complex large scale migrations
• Provides detailed logging and auditing
• Added licensing cost for the tool

SharePoint Migration API

Microsoft’s migration API exposes a REST endpoint to build migration jobs that move SharePoint content programmatically:

• Build custom migration apps with robust API
• Supports incremental syncs after initial move
• Only available in SharePoint Online currently
• Requires software development skills

Overall, factors like the scale, frequency, and complexity of the migration drive tool selection. For one-time moves, manual methods or PowerShell work well while large recurring syncs benefit automated tools or custom apps.

Preparing for a Migration

Planning and analysis is key before starting any SharePoint content migration. Essential upfront preparation tasks include:

Analyze Existing Infrastructure

Document sources and destinations:

• Map source sites, libraries, folders, with metadata models, permissions, features etc.
• Design new info architecture and site structure aligned to business goals
• Plan where legacy content will reside in new structure

This analysis ensures nothing gets overlooked during migration to the new layout.

Consolidation Plan

Identify consolidation opportunities across identified source repositories based on factors like common users, content types etc. This prevents moving duplicate or overlapping information.

Destination Site Collection Setup

Configure central repositories in SharePoint Online or OneDrive based on the desired information architecture to receive migrated content with appropriate:

• Sites, libraries, content types, metadata
• Views, columns, permissions
• Information management policies

All infrastructure should be ready before starting document transfers.

Other Considerations

Some other key checks:

• Version histories – determine what gets retained
• Security trimming – avoiding unauthorized access
• Managed metadata – retaining or transforming taxonomies
• Transforming content types

Handle information governance facets ahead of time for a smooth result.

Executing a Migration

Once analysis and planning completes, the next phase is executing the SharePoint content migration. Here are key considerations for tool selection and processes.

PowerShell Migration Steps

For scripted PowerShell migrations:

1. Connect to SharePoint Online in PowersShell using access token in secret variable
2. Retrieve source and destination site URLs stored in parameter variables
3. Map source structure into defined content type for destination library
4. Recurse folders and enumerate all files in source folder tree
5. Filter child items that match defined filemasks to migrate
6. Construct destination folder structure mirroring source
7. Iterate files and execute Move-SPOFile to new location
8. Capture errors for post-migration review
9. Email migration logs and outputs

This generic pattern flexibly moves different hierarchies.

Migration API Process

If using the SP Migration API:

1. Register apps with required OAuth scopes in Azure AD
2. Acquire access tokens from AAD for auth flow
3. Discover source and destination sites via API
4. Create migration job with source and dest URLs
5. Start incremental copy batches with page size throttles
6. Poll job status endpoint for completion signals
7. Fetch reporting metrics from logs for analysis

The API handles heavy lifting after initial setup.

Large Migrations

Regardless of tools, large migrations should:

• Parallelize work across batches using throttles to prevent timeouts
• Route failures into dedicated queues for retry with exponential backoff
• Surface migration dashboard showing real-time progress and stats

Automate handling edge cases and failures.

Validation & Testing

Post migration:

• SP migration tool reports should show no errors
• Sample documents from source should migrate successfully
• Metadata, versions preserved for sampled items
• Document totals should match between source and destination

Root cause any data integrity gaps before finalizing work.

Post-migration Considerations

After executing the SharePoint migration, finishing tasks include:

URL Redirects

Configure 301 redirects from old locations to new libraries so bookmarked links keep working.

Ongoing Syncs

Schedule periodic migration syncs to capture new source content on a rolling window:

• Daily, weekly, monthly sync schedules
• Sync only changed documents since last run
• Keep sync metadata like last updated timestamps

This automates migration of new stuff long term.

Decommissioning

As legacy sites get deprecated, considerations like:

• Final backup/archives before shutdown
• Removing site collection ownership assignments
• Retaining sites in read-only mode if needed
• Communicating impending changes to site owners

This cleans up obsolete containers after migration.

User Training

Drive adoption of new consolidated sites via:

• Emails, portals banners showing changes
• Updated process documentation around new info architecture
• Self-help guidance on finding moved content
• Digital literacy skills building for search, metadata etc.

Ongoing enablement ensures community thrives.