{"id":6908,"date":"2022-09-23T11:25:00","date_gmt":"2022-09-23T11:25:00","guid":{"rendered":"http:\/\/rssoftware.com\/?p=6908"},"modified":"2022-11-23T10:07:16","modified_gmt":"2022-11-23T10:07:16","slug":"digital-payments-impacting-everyday-life-what-must-happen-to-keep-them-reliable-3","status":"publish","type":"post","link":"http:\/\/rssoftware.com\/rswordpress\/digital-payments-impacting-everyday-life-what-must-happen-to-keep-them-reliable-3\/","title":{"rendered":"Digital payments impacting everyday life \u2013 What must happen to keep them reliable?"},"content":{"rendered":"\n<h3>Part 3<\/h3>\n\n\n\n<p>In Part 1 we had discussed resiliency as the underlying principle that drives the design of mission-critical payment systems. In Part 2 we discussed performance as one of the essential cornerstones for such systems.<\/p>\n\n\n\n<p>In this article, we would like to discuss scalability as one of the key desirable features for such systems. Although we had briefly touched upon scalability in Part 2, it needs an in-depth treatment.<\/p>\n\n\n\n<p>As per an <span class=\"has-inline-color has-vivid-cyan-blue-color\">article<\/span> in the Business Standard (01-Sep-2022), based on the data published by the National Payments Corporation of India (NPCI), Unified Payment Interface (UPI) transactions grew 85 percent by volume and 67.85 percent by value year-on-year (YoY). In August 2022, UPI crossed the 6-billion mark in volumes, the first time since its inception in 2016.<\/p>\n\n\n\n<p>According to the <span class=\"has-inline-color has-vivid-cyan-blue-color\">Annual Report<\/span> 2021 of Visa, the transaction volume stood at 164.7 billion, up from 140.8 billion in 2020, which is a year-on-year (YoY) rise of 16.97 percent.<\/p>\n\n\n\n<p>This rise in the volume of such mission-critical systems needs technology diligence during design, development, maintenance, planning and upgrade in a deliberate and concerted manner.<\/p>\n\n\n\n<h4>Why build for scalability?<\/h4>\n\n\n\n<p>All businesses including payments, plan for growth. Competition in the payment business tends to drive down margins, hence the growth must be substantial in order to ensure success in business. Payments is a business of scale. Adding one more customer \u2013 consumer or organization or merchant to the network \u2013 should be possible at a negligible incremental cost. Therefore, from inception it must be built with scalability as an architectural principle.<\/p>\n\n\n\n<h4>How do you design for scalability?<\/h4>\n\n\n\n<p>Let us look at some of the components that are needed to build a payment system, and how scalability would impact these components.<\/p>\n\n\n\n<ul><li><strong>Online Components:<\/strong> These components interface with the external world. External systems invoke payments through these components and the payment system needs to provide a response in real-time. This form of invoking a request and waiting for a response in real-time is called a synchronous process. To increase scalability, the system should be able to (a) increase the performance of each point of interaction, termed vertical <strong>scalability<\/strong>, and (b) support the addition of more points of interaction, called <strong>horizontal scalability.<\/strong> The processing that happens could be <strong>stateless<\/strong>, that is, it does not need to look up any data elsewhere as all information is available in the transaction it processes; or it can be <strong>stateful<\/strong>, where it needs to refer to data stored from past flows. Horizontal scalability for a stateful execution is a challenge and needed to be resolved skillfully.<\/li><li><strong>Offline Components:<\/strong> These do not come in the path of online execution; however, often an online component triggers an offline execution. Online components are designed for speed, offline components are not. It is typical to have <strong>buffers<\/strong> and <strong>queues<\/strong> between the two components for speed adaptation. The offline components could be <strong>long-running<\/strong> or <strong>short-running<\/strong> processes. From the scalability perspective, interfacing a high-speed online component with a long -running offline process needs special care.<\/li><li><strong>Batch Processing Components: <\/strong>We just saw that online components trigger offline processing. However, it could be that the invocation could come from a batched set of transactions that are fired offline. Here, the process reads items from the batch files and invokes the processing offline. It is typical that there are upper limits to the <strong>number of items in the batch<\/strong> and the<strong> number of batches <\/strong>that can be processed. Scalability impacts the batch size and number of batches. Whether horizontal or vertical scalability will serve this best depends on the structure of the processing.<\/li><li><strong>Event Processing:<\/strong> Unlike online processing of synchronous processes, events are <strong>asynchronous<\/strong> processes, that is the process notifies the requester after the process gets over &#8211; the requester need not \u201cwait\u201d for the response. This saves resources. Asynchronous processes become an important tool to enable scalability of systems by using the resources optimally.<\/li><li><strong>Dashboard Components:<\/strong> Dashboards typically operate as near-real-time (NRT) processes. After a transaction is processed, it is expected that the transaction \u201cappears\u201d on the dashboard almost immediately. Thus, it is an NRT process. When online components are scaled up, the NRT components need to be appropriately scaled as well, to avoid the feedback of \u201csluggish dashboards\u201d.<\/li><li><strong>Reporting Components: <\/strong>All systems are expected to generate human-readable reports to understand how the business components and technology components are performing. These are typically purely offline, and they are often fired by a scheduler (timer-driven invocation) and shared with recipients \u2013 directly or as a link to a shared copy \u2013 using email, message, or other mechanisms. As volume increases due to scale up, it can impact report performance. We have seen situations where the generation of the report takes so much time that, if the process fails, there is not enough time to rerun the reports fast enough for the business to take decisions in time, leading to a missed deadline! Scale-up of reporting typically calls for <strong>intelligent preprocessing<\/strong>, should the need arise.<\/li><\/ul>\n\n\n\n<p>After teams build newer, scaled-up versions of the payment platform, they need to verify it. This process is called benchmarking. The challenge here is to pump in real-life like transactions at a very high rate to simulate a future state and collect all operational statistics to observe and refactor the kink points so that it is ready for tomorrow.<\/p>\n\n\n\n<p>Having said this, life is not always that scary. Constant innovation is underway in all these components and many more that have not been covered in this introduction. These technology upgrades make life easier for the technology team designing or upgrading payment systems.<\/p>\n\n\n\n<p>If you are interested to read about \u201c<strong>Designing a Payment System<\/strong>\u201d, you can visit the <a href=\"https:\/\/newsletter.pragmaticengineer.com\/p\/designing-a-payment-system\" data-type=\"URL\" data-id=\"https:\/\/newsletter.pragmaticengineer.com\/p\/designing-a-payment-system\" target=\"_blank\" rel=\"noreferrer noopener\">blog<\/a> of the same name authored by Gergely Orosz. He writes <strong>The Pragmatic Engineer Newsletter<\/strong> and is an advisor at mobile.dev who previously worked at Uber, Microsoft, Skype, and Skyscanner.<\/p>\n\n\n\n<p><em>To be continued \u2013 stay tuned for part 4.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Part 3 In Part 1 we had discussed resiliency as the underlying principle that drives the design of mission-critical payment systems. In Part 2 we discussed performance as one of the essential cornerstones for such systems. In this article, we would like to discuss scalability as one of the key desirable features for such systems. [&hellip;]<\/p>\n","protected":false},"author":9,"featured_media":6911,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[89],"tags":[],"_links":{"self":[{"href":"http:\/\/rssoftware.com\/rswordpress\/wp-json\/wp\/v2\/posts\/6908"}],"collection":[{"href":"http:\/\/rssoftware.com\/rswordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/rssoftware.com\/rswordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/rssoftware.com\/rswordpress\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"http:\/\/rssoftware.com\/rswordpress\/wp-json\/wp\/v2\/comments?post=6908"}],"version-history":[{"count":0,"href":"http:\/\/rssoftware.com\/rswordpress\/wp-json\/wp\/v2\/posts\/6908\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/rssoftware.com\/rswordpress\/wp-json\/wp\/v2\/media\/6911"}],"wp:attachment":[{"href":"http:\/\/rssoftware.com\/rswordpress\/wp-json\/wp\/v2\/media?parent=6908"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/rssoftware.com\/rswordpress\/wp-json\/wp\/v2\/categories?post=6908"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/rssoftware.com\/rswordpress\/wp-json\/wp\/v2\/tags?post=6908"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}