Fact Sheet

Data Migration Services


No Small Undertaking

Data migration typically involves moving data from one platform to another. This process assumes a special significance in the payments industry due to the criticality and sensitivity of various key data elements that are generated and stored by various applications during the payment lifecycle. The sheer data volume generated during the payment lifecycle – from authorization, clearing and settlement, risk and fraud, disputes, loyalty, and billing – means that any migration of this data can potentially have a material impact on a company’s business.

One of the primary challenges faced by an organization that initiates a data migration project is to identify all the components that will be involved in the process and assess the accuracy and integrity of the data contained in them. Failure to do so generally leads to scope creep that will impact project timelines and cost. In addition, impact to dependent applications during the migration can cause significant business risk. To avoid these pitfalls, a seamless and proven migration methodology and framework must be deployed to insure a project is completed on time and on budget.

Based on its 25 years of experience in the payments industry and numerous data migration projects, RS Software has developed a framework designed to migrate data from legacy systems to open source solutions. The RS Software framework covers all phases of data migration projects including migration conceptualization, strategy identification, and customization of the solution, using business domain knowledge and tools for increasing efficiencies when moving data from the legacy source to the target open systems.

The RS Software approach protects organizations from the risk associated with the obsolescence of proprietary products by migrating their data to platforms that are flexible, scalable and open. This approach can lower the total cost of ownership of the data platform by up to 40 percent. The RS Software framework can be tailored to meet specific customer requirements with minimal reengineering of existing applications. This has allowed many of our customers to improve their time to market for data products by up to 20 percent.

Managing the Specific Challenges

In the payments domain, transaction processing (i.e., authorization, clearing and settlement) is supported by multiple applications associated with compliance, enterprise data warehousing, loyalty, merchant services, and risk mitigation. Traditionally, most of these applications, along with the transaction processing application itself, have been operated on mainframes. Most data migration projects involving these key appications are either open systems (e.g., Unix) or big data platforms (e.g., HDFS, NO SQL, NEW SQL, etc.). These migrations have been undertaken to lower overhead costs, improve performance, address system consolidation, enhance system performance and gain a competitive edge in the marketplace.

All the systems mentioned above must meet the strict requirements of payment industry compliance. Therefore, when choosing the new platform to be used to store data, a key attribute to review thoroughly concerns ability to meet regulatory standards. Also, any down-time that maybe involved while switching to the new platform must be carefully considered based on the criticality of relevant systems. If data migration impacts systems that support real-time risk evaluation or real-time updates in loyalty systems, special care needs to be taken to minimize down-time. Running systems in parallel during the migration can address concerns over down-time allowing for the completion of the project before going live with the new platform. Lastly, if an organization utilizes multi-data center application hosting, the new data platform will need to have cross data center synchronization features.

How RS Software Ensures Success

RS Software has successfully completed numerous data migration projects using frameworks designed specifically for clients working in the payments industry. The diagram below represents the main components of the RS Software framework:


Data Migration Pre-processing Components:

  • Data Migration Specific Study
    Studies of several key elements are conducted in order to identify the variables that might increase risk to the project. These studies help project and program management teams size related information and create project plans.

    • Data Migration Purpose Identification: Though it may seem basic, identifying and clearly stating the reasons an organization has decided to initiate a data migration project forms the most important element in the framework. All other elements, stages and steps in the project will be based on these reasons. Generally, reasons for the projects may include the desire to decrease overhead, increase operational efficiency, consolidate a multi-platform data housing strategy to a single platform, improved functionality within the platform and/or gain a competitive advantage.
    • Data Migration Type Identification: There are different types of data migration that vary from simple to complex. The project can be as simple as single-source to single-target or as complex as multi-source to single-target or multi-source to multi-target.
    • System Type Identification: The number of steps involved in a migration varies based on a system’s data ingestion process. If the system is involved in providing OLTP services and data gets populated using web or desktop applications, a one-time data migration step will suffice. However, if the project involves an OLAP system, it will need enhancement to ETL scripts along with one-time migration scripts. Also, if data migration involves real-time or near-real-time applications, additional steps need to be taken to ensure minimum to zero downtime of the applications during the project.
  • Data Profiling and Data Quality Measurement of Source Systems: RS Software recommends that the health of the data within the source systems be evaluated using a variety of quality metrics. This eliminates the proliferation of aged, incorrect data and provides a baseline to validate the data captured by the target system.
  • Target System Evaluation: It is crucial to identify all features that are currently supported by source systems to insure that the possible target systems evaluated provide the same features.
  • Downtime to Switch: This is another important factor that helps determine what migration tools and approaches are used in the project. If an application requires zero downtime, there are tools from vendors like Oracle, IBM and Golden Gate that are built to support zero downtime migrations.

Data Migration Processing Components:

  • Data Migration Process Activities – Building the process that will support the data migration initiative is the most time consuming of the steps that must be taken during the project.
    • Data Migration Blueprint: High-level representation of the data migration process that is accepted by all project stakeholders.
    • Data Migration Technology Stack: The evaluation of the tools to be used in the data migration process. Some organizations do this step as a pre-processing component. RS Software has extensive experience in using the open source and licensed stacks.
    • Conflict Resolution: This step will be primarily for multi-source to single-source data migration. All stakeholders and business owners should agree on each step that is involved in resolving conflicts that may arise.
    • Data Lineage: Data lineage also will need to be addressed in the migration as required based on the current metadata management system.
    • Data Security: It is mandatory to involve data governance and security team members in the project to make sure all data security standards are followed or enhanced as per established guidelines.
    • Documentation Requirements: The level and extent of documents as required by an organization’s standards must be defined and incorporated.
    • Test Cases for Data Migration Validation: Test cases are identified based on the elements in pre-processing and conflict resolution. Test cases should include:
      • Data Loss, Corruption and Integrity: These are the primary concerns in any data migration project. The case studies should address basic data validation, mitigation censoring and truncation.
      • Performance: Test cases should address performance-specific testing. Existing test cases can be utilized if the application provides OLTP services.
      • Semantic Risk: These can be automated test cases or random samplings with structure identification and outlier detection.
      • Data Imputing: Based on past experience, it is crucial that previous platforms need some data imputing because of application related issues or due to multi-platform to single-platform consolidation. Data profiling test cases should cover test cases for data imputing.
    • Development: Using all the elements above, development involves what is required for data migration as well as enhancements to existing data ingestion script.

Data Migration Post-Processing Components

  • Testing – Testing and sign-off on all approved test cases.
  • Performance Measurement: This step is mandatory in most of the data migrations to confirm that performance measurements meet the defined thresholds and SLAs.

About RS Software

rslogo copy

RS Software is a leading custom software development, testing and implementation company for the payments industry. With more than 25 years of experience in this industry, we have helped create, test and implement products and services that have transformed the marketplace.

The development of our data migration framework and the components that are included are the result of work for numerous companies facing the need to upgrade from legacy systems to more modern, less costly platforms. We would welcome the opportunity to assist you with identifying and successfully completing your next data migration project.