The bigger picture: Strategizing your data warehouse migration

By Ravi Dharnikota

If your organization is moving its data warehouse to the cloud, you can be confident you’re in good company. And if you read my last blog post about the six-step migration process, you can be even more confident that the move will go smoothly. However, don’t pull the trigger just yet. You’ve got a bit more planning to do, this time at a more strategic level.

First, let’s recap the migration process I covered in my last post, of the data warehouse itself. In that blog post, I broke down all the components of this diagram:

Data Warehouse Migration Process

Now, as you can see in the diagram below, the data warehouse migration process itself is part of a bigger picture of migration planning and strategy. Let’s take a look at the important pre-migration steps you can take help to ensure success with the migration itself.

Migration Strategy and Planning

Step 1: Define Goals and Business Case. Start the planning process with a clear picture of the business reasons for migrating your data warehouse to the cloud. Common goals include:

  • Agility in terms of both the business and the IT organization’s data warehousing projects.
  • Performance on the back end, to ensure timeliness and availability of data, and on the front end, for fast end-user query response times.
  • Growth and headroom to ease capacity planning; the elastic scalability of cloud resources mitigates this problem.
  • Cost savings on hardware, software, services, space, and utilities.
  • Labor savings from reduced needs for database administration, systems administration, scheduling and operations, and maintenance and support.

Step 2: Assess the current data warehouse architecture. If the current architecture is sound, you can plan to migrate to the cloud without redesign and restructuring. If architecturally sufficient for BI but limited for advanced analytics and big data integration, you should review and refine data models and processes as part of the migration effort. If the current architecture struggles to meet current BI requirements, plan to redesign it as you migrate to the cloud.

Step 3: Define the migration strategy. A “lift and shift” approach is tempting, but it rarely succeeds. Changes are typically needed to adapt data structures, improve processing, and ensure compatibility with the chosen cloud platform. Incremental migration is more common and usually more successful.

As I mentioned in my last blog post, a hybrid strategy is another viable option. Here, your on-premises data warehouse can remain operating as the cloud data warehouse comes online. During this transition phase, you’ll need to synchronize the data between the old on-premises data warehouse and the new one that’s in the cloud.

Step 4: Select the technology including the cloud platform you’ll migrate to, and which tools you’ll need for the migration. There are many types of tools and services that can be valuable:

  • Data integration tools are used to build or rebuild ETL processes to populate the data warehouse. Integration platform as a service (iPaaS) technology is especially well suited for ETL migration.
  • Data warehouse automation tools like WhereScape can be used to deconstruct legacy ETL, reverse engineer and redesign ETL processes, and regenerate ETL processes without the need to reconstruct data mappings and transformation logic.
  • Data virtualization tools such as Denodo provide a virtual layer of data views to support queries that are independent of storage location and adaptable to changing data structures.
  • System integrators and service providers like Atmosera can be helpful when manual effort is needed to extract data mappings and transformation logic that is buried in code.

Using these tools and services individually or in combination can make a remarkable difference in your data warehouse, serving to speed and de-risk the migration process.

Step 5: Migrate and operationalize; start by defining test and acceptance criteria. Plan the testing, then execute the migration process to move schema, data, and processing. Execute the test plan and, when successful, operationalize the cloud data warehouse and migrate users and applications.

Learn more at SnapLogic’s upcoming webinar

To get the full story on data warehouse cloud migration, join me for an informative SnapLogic webinar, “Traditional Data Warehousing is Dead: How digital enterprises are scaling their data to infinity and beyond in the Cloud,” on Wednesday, August 16 at 9:00am PT. I’ll be presenting with Dave Wells, Leader of Data Management Practice, Eckerson Group, and highlighting tangible business benefits that your organization can achieve by moving your data to the cloud. You’ll learn:

  • Practical best practices, key technologies to consider, and case studies to get you started
  • The potential pitfalls of “cloud-washed” legacy data integration solutions
  • Cloud data warehousing market trends
  • How SnapLogic’s Enterprise Integration Cloud delivers up to a 10X improvement in the speed and ease of data integration

Register today!

Ravi Dharnikota is Chief Enterprise Architect at SnapLogic. Follow him on Twitter @rdharn1

SnapLogic Sits Down with theCUBE at AWS re:Invent to Talk Self-Service Cloud Analytics

SnapLogic was front-and-center at AWS re:Invent last week in Las Vegas, with our team busier than ever meeting with customers and prospects, showcasing our solutions at the booth, and networking into the evening with event-goers interested in all things Cloud, AWS integration and SnapLogic.

Ravi Dharnikota, SnapLogic’s Head of Enterprise Architecture and Big Data Practice, took time out to stop by and visit with John Furrier, co-founder of the live video interview show theCUBE.  Ravi was joined by Matt Glickman, VP of Products with our partner Snowflake Computing, for a wide-ranging discussion on the changing customer requirements for effective data integration, SaaS integration, warehousing and analytics in the cloud.  

The roundtable all agreed — organizations need fast and easy access to all data, no matter the source, format or location — and legacy solutions built for a bygone era simply aren’t cutting it.  Enter SnapLogic and Snowflake, each with a modern solution designed from the ground-up to be cloud-first, self-service, fully scalable and capable of handling all data. Customers using these solutions together — like Kraft Group, owners of the New England Patriots and Gillette Stadium — enjoy dramatic acceleration in time-to-value at a fraction of the cost by eliminating manual configuration, coding and tuning while bringing together diverse data and taking full advantage of the flexibility and scalability of the cloud.

To make it even easier for customers, SnapLogic and Snowflake recently announced tighter technology integration and joint go-to-market programs to help organizations harness all data for new insights, smarter decisions and better business outcomes.

To watch the full video interview on theCUBE, click here.

SnapLogic CTO James Markarian on DisrupTV

SnapLogic CTO James Markarian recently appeared as a guest on DisrupTV, a weekly live-interview web-series produced by analyst firm Constellation Research and hosted by R “Ray” Wang and Vala Afshar. The trio discussed a variety of enterprise topics including modern data management, data lake strategy considerations and big data analytics.

Continue reading “SnapLogic CTO James Markarian on DisrupTV”

Customer Webinar with Box: Connecting SaaS Apps & Data

Join us on Thursday, September 22nd during this live webinar to hear SnapLogic customer Alan Leung of Box talk about how he architected and implemented Box’s connected infrastructure of SaaS applications like Salesforce and Zuora and cloud data stores such as Amazon Web Services.

Continue reading “Customer Webinar with Box: Connecting SaaS Apps & Data”

Customer Spotlight: AstraZeneca

It’s always a treat to be able to share our customers’ inspiring success stories. MWD Advisors, a UK-based industry research and advisory services firm, recently spent time with our customer, global pharmaceuticals giant AstraZeneca. MWD published the results of that interview to their research subscribers in a report titled “AstraZeneca supports strategic cloud shift with ‘citizen integrator’ platform” (reprint provided here).

AstraZeneca early iPaaS success
AstraZeneca’s early iPaaS success

AstraZeneca started their journey with SnapLogic as part of an IT initiative to make their business process infrastructure more agile, and to incorporate cloud-based platforms and applications. They wanted to efficiently integrate SaaS systems and on-premises platforms without investing in a large, specialized integration team.

Continue reading “Customer Spotlight: AstraZeneca”

SnapLogic Live Summer 2016: Hybrid Cloud Integration for Salesforce

After a brief hiatus, we’re kicking off our Summer 2016 SnapLogic Live series next week with Hybrid Cloud Integration for Salesforce. As all SnapLogic Live webinars go, this session will feature a live demo of SnapLogic integration in action, this time with a focus on Salesforce.

SL-Live-SFDC

Continue reading “SnapLogic Live Summer 2016: Hybrid Cloud Integration for Salesforce”

iPaaS for iRobot: Connect Faster with SnapLogic

In this video, Jim Teal from iRobot talks about how SnapLogic allows his organization to be more agile and operate as a real-time business supporting their global manufacturing operation.

“It’s the ease of use and flexibility of the platform that really is a huge accelerator in productivity.”

You can learn more about why iRobot initially chose SnapLogic in this recorded webinar. To learn more about SnapLogic customers, visit: SnapLogic.com/Customers.