Smoke Automation and Regression Testing on a peer-to-peer lending Website with the Data-DrivenTesting Method

Software testing is considered as one of the most important processes in software development as it checks whether the system meets the requirements and specifications of the users. The testing process manually or automatically aims to ensure that the main features are not errors and function during development. The manual process frequently causes the publication time of a feature could not to be punctual. This study aims to create an automation script with the data-driven testing method during smoke and regression testing to ensure the quality, especially the main functions or features, can run normally and not be disturbed by the development of new features that are easily understood. One of Various decent automation tools for testing web applications is Katalon Studio which is based on the Selenium tool. The results of the research that have been carried out show that the process of automation of software testing by applying a script automation tool made with Katalon Studio which applies the data-driven testing method is very good with an achievement rate of 80,21%. The automation tools that are built are easy to use, can be learned quickly, are not too complicated and make users want to use it again.


Introduction
Software testing is considered as one of the most important processes in software development because it checks whether the system meets user requirements and specifications [1] [2]. Software testing is a process to evaluate the functionality of a software application with a view to finding whether the developed software meets the specified requirements or not and to ensure that the product is free from defects or bugs to produce a quality product. Software testing usually consists of dynamically verifying that a program runs as expected with a finite set of test cases, properly selected from infinite execution space [3]. Unit testing is one of the most important and costly steps in software development to ensure software quality. Testing is often overlooked because it takes a lot of time and effort for software developers to create tests. [4], [5]. Despite the efforts of the software testing research community over the last few decades, various studies show that many practitioners in the industry are still reporting challenges for software testing tasks. Management and automation test are considered by practitioners to be the most difficult of all testing activities and many software testing require further investigation by the research community [6].
Software testing can be done manually or automated testing. Manual testing is the process by which software testers (Software Quality Assurance/SQA) must have an end-user perspective to ensure all features or functions work as stated in the requirements document without resorting to automation tools. Automated testing is a software testing method using automation tools to find defects in software. In this process SQA runs test scripts and generates test results automatically using automation tools. Automated testing provides software testers with the facility to automate the software testing process, which is more time efficient, cost effective and user friendly [7]. There are many automated testing tools available, both open source and commercial. Not all working SQA understand well about the coding script automation process, for that it is necessary to choose tools that are user friendly or easy to use by QA. Various decent automation tools for testing web applications have emerged, one of which is Katalon Studio which is based on the Selenium tool. Katalon studio is a tool for testing web-based applications in an automated way [8]. Many types of software testing are usually run by SQA such as stress testing, load testing, sanity testing, smoke testing and regression testing.
Peer-to-peer lending, also known as P2P lending, is a method of lending money to an individual or company. The person who grants this credit is called the lender or investor. Profit/income received by the investor from the interest paid by the borrower. On the other hand, for borrowers, peer-to-peer financing means applying for a loan for personal/business purposes. Basically, P2P lending connects lenders and borrowers online. With a P2P loan, anyone can apply for a variety of purposes without resorting to banking services. Basically, this P2P lending system is very similar to the concept of an online marketplace that provides a meeting place between buyers and sellers. However, in P2P lending, the existing system connects the borrower with the party providing the loan. In other words, P2P lending is a market for lending and borrowing money. Instead of applying for a loan through public institutions such as banks, co-operatives, credit services and governments, the process is much more complicated. The community can instead use the P2P system to apply for loans sponsored by others [9]. Online P2P lending platforms represent a emerging but rapidly expanding phenomenon in the context of digital finance [10].
Currently, the software testing process is not only done manually, but there has been an increase with automated testing. Depends on pre-scripted tests that run automatically, its function is to compare the expected results with the actual results. So that you can find out whether the application is running as expected, using automation testing can be done repeatedly. So, if the results are not the same as expected, you will get a bug. While automated testing will give the best results if used for regression testing because coding changes regularly and is highly dependent on timing, load testing, repeated execution, and performance testing. But now there are tools to perform automated testing that are also user friendly.
Automation testing is used where the product developed has started to stabilize or for products or applications where the features do not have many changes anymore. Test automation is starting to take over the manual testing process as it can reduce test time and generate test reports and don't need to be generated manually.
As software applications develop, more and more test cases (test cases) are produced by SQA to test software applications. For the Software Development Life Cycle (SDLC) methodology using agile scrum, each sprint consists of 2 weeks (10 days) where the developer work time is 6-7 days and the time for testing developed features is only 3 days including smoke an [11] d regression. testing. If it is considered, SQA only has less than 1 day to do smoke and regression testing because 2 days are used to test new features developed independently. So sometimes the product release time is often delayed because smoke and regression testing are still done manually. If you look at the number of test cases consisting of 300-500 test cases, the SQA team spent about 2 days to do it, lack of Human Resources (HR). Automated smoke testing and regression testing processes are needed to reduce testing time, so important features can be immediately published to users. The estimated time for testing if using automation is one day and this will be very helpful for SQA work and the process time for publishing software features.
Smoke testing and regression testing are types of testing that are used as parameters for whether the product is published to users. Smoke and regression testing are always run every 2 weeks by SQA. This smoke and regression testing process is carried out repeatedly because every 2 weeks there are always new features or functions published to users according to stakeholder requests. Smoke Testing is a type of testing that is carried out after the new functions of the software are integrated with the old functions and ensure that the important functions of the software work properly. It is executed "before" any detailed functional or regression tests are run on the application being built. The goal is to reject software that has broken down early in development, so that the SQA team doesn't waste time installing and testing software applications. Smoke testing becomes more productive and efficient if it is automated, or if it is a combination of manual and automated [12]. Regression Testing is testing existing software applications to ensure that changes or additions to functionality have not violated the functions of previous software applications [13]. Its main purpose is to detect and catch defects or bugs that may inadvertently spread into a software application. Software regression is a condition where a previously running software application goes wrong because of changes or additions to one or more parts of the software. The process of selecting test cases for priority test cases used for regression testing is very profitable compared to if the test cases are not prioritized. This is because if there is a hasty regression testing process, the software application testing team or SQA can use priority test cases.
Testing with smoke and regression testing that is done manually is very difficult, time consuming, and requires more effort and a lot of human resources [14]. Software application publication time becomes long because the testing process is carried out completely manually. Therefore, smoke testing and regression testing automation processes are needed to reduce testing time, so that important features can be immediately published to users. The current situation is that most SQA do not understand well about the coding script automation process, for that we need tools that are user friendly or easy to use.
Data-driven testing is a testing technique in which the test case that has been created by SQA will be made in the form of external data parameters defined in the form of a file. The implementation of data-driven testing will make it easier for SQA to perform automated tests using the Katalon Studio tool, even this is very helpful for those who don't know much about coding because SQA doesn't need to update its script code if you want to add new test cases or change existing test cases. For a long time, they simply update or add their test cases to the data-driven testing file that will be provided later. In addition, later the author will update the test case used for smoke and regression testing for the purposes of the automation process. Because there are some test cases that should still be done manually or can't be done manually Automation testing relies on pre-scripted tests that run automatically, the function of which is to compare the expected results with the actual results to find out whether the application or web is running as expected.
Using automated testing can be done repeatedly, so if the results are not the same as expected, we will get bugs that we can report to the development team to be fixed. Katalon is a testing tool that is being widely used for testing both from mobile apps and desktop browsers. These testing tools can record steps, capture web elements in web applications, play automatically for existing test cases and for reporting (record and play). Katalon can be used for beginners or experts in the QA field. The comparison of the Katalon Studio automation tool with other automation tools can be seen in table 1.
The process of publishing new functions to users of software applications that are usually done every two weeks before new functions are published, QA must do smoke and regression testing first, and takes two days. The time given to SQA to carry out the entire testing process including testing every new feature and doing a series of smoke and regression testing is only three days i.e. two days for testing each new feature and one day for doing smoke and regression testing and one day is not enough for doing smoke and regression testing manually. The person dedicated to doing smoke and regression testing is only one person who is rotated from among the SQA team, so it is not possible to do the process in one day or even less than one day. The smoke and regression testing which is still done manually makes the process of publishing new functions in software applications, not on time. Not all SQA can code scripts, so an automation testing technique is needed where SQA is not in direct contact with code such as data-driven testing.
Testing is an integral part of any successful software project. The type of test (guided or automated) depends on a variety of factors, including project requirements, budget, time, expertise, and suitability. The three critical factors of any project are time, cost, and quality (all three are interrelated). The goal of any successful project is to reduce the cost and time required to complete it successfully while maintaining the quality of the output. When it comes to testing, one type can achieve this goal better than the other. Software testing automation aims to reduce the time of testing that is carried out manually, but the automation of tests carried out by previous researchers is not pegged to a certain type of testing. Only against the types of test cases that are usually used. is to create script and find out whether the software testing automation process by applying a script automation tool made with Katalon Studio that applies the data-driven testing method is easy to understand by SQA so that it can reduce smoke and regression testing work time and the publication time of new functions to users is not too late.

Research Methods
This research methodology is carried out by carrying out research steps that begin with collecting information about the smoke process and regression testing, which is still being done manually, summarizing all information related to smoke and regression testing, then looking for automation testing information based on previous studies to obtain tools. automation that should be used by. Researchers also study the web or application that will be used for automation testing so that researchers can conclude what test cases can be automated from the web or application. In this research, a script that is easy to use by Quality Assurance is created to speed up the smoke process and regression testing which is still done manually. The SDLC method applied is Scrum. Scrum is a software engineering technique that applies the principles of an agile approach that harnesses the power of team collaboration, incremental products, and iterative processes to achieve end results. Sprints are part of the Scrum process. Sprints are either user-ready product development timeboxes or brought into production as part of software development and are completed within 30 days. The time limit for each sprint is 2 weeks or 10 days. Within 10 days, the development team will develop new features in 6-7 days, and the SQA team will test each new feature and smoke and regression testing process for 3 days.
The stages of developing new features until they are published as shown in figure 1 begin with the product team gathering information with stakeholders. After the information was collected, the product team studied the concept of the system and the problems to be solved to answer the question of whether the new system was realistic in terms of costs, timing, and differences from the current system. The product team explains the problems to be solved or the features to be developed to the development team or developers and the SQA team by making them into more detailed explanations. Next, the development team implements new features. After the development team has finished implementing each new feature, then the SQA team will test each new feature on the local PC of each SQA before each feature is integrated into the staging environment. If some errors are found during testing, the development team immediately makes improvements so that the feature can be tested again. After the testing process for each feature is completed on each SQA PC, then each feature will be integrated into the staging environment for retesting. After testing each feature in the staging environment, smoke and regression testing are carried out. Next, the SQA team informs the product team and the developer team, that new features are ready to be published, the development team publishes new features, and the product team makes a report on the results of the publication of new features.
The object of this study is a peer-to-peer fintech lender that connects lenders and borrowers on an online platform by transforming traditional systems into easier and more affordable financial access to meet each other's financial needs. Existing platforms offer rigorous and comprehensive credit checks, real-time payments with virtual accounts, and state-of-the-art innovative technologies that can improve the user experience for all users, both lenders and borrowers. The platform allows all lenders to choose where to use their funding capital based on interest rates, terms, risks, and objectives. This means that the lender can choose his borrower.
Data was collected by interview to determine the condition of the manual testing process for smoke and regression testing run by the SQA team and questionnaires to respondents from the team involved in the testing process, namely the SQA team of 5 people, the development team of 10 people and the product team of 4 people, the total respondents are 19 people. After the relevant parties have completed filling out the distributed questionnaires, the data analysis related to the questionnaire results using a Likert scale is scored as 5 = Strongly Agree (SA), 4 = Agree (A), 3 = Neutral (N), 2 = Disagree(D) and 1 = Strongly Disagree (DA) [16].

Results and Discussions
The flow of the peer-to-peer lending system that is used as the object of testing is shown in Figure 2. The online platform of the peer-to-peer lending system can reduce the costs and inefficiencies of the traditional financial system in general and make access to finance easier and more affordable. XYZ provides a strict and comprehensive creditworthiness assessment, real-time payments using virtual accounts, as well as the latest and innovative technologies that can improve the user experience for all users, both lenders, and borrowers.
All borrowers can apply for loans for business purposes, business capital, education, health, or employee loans completely online and will be met with Lenders on an online platform after the loan application is approved by the team. In the platform, each lender can choose for themselves where the funding capital will be allocated based on the interest rate, tenor, risk, to the purpose of the loan (lenders can choose who the borrower is). The lack of experience of the SQA team in automation testing caused several problems, including the test time given was only one day, where one day was not enough if smoke and regression were done manually, publication time was not timely due to smoke and regression testing was carried out more than the specified time because many test cases to run. HR SQA team is not enough to do smoke and regression testing in one day manually, sometimes the feature fixes carried out by the development team are long enough so that the time for smoke and regression testing is less than one day, however the experience of the SQA team is not enough to create test scripts automation.
Problems that arise related to smoke and regression testing, which are still done manually and at the time of feature publication, are overcome by creating an automation script with the data driven testing method to solve these problems. The process of creating an automation testing script begins with creating scripts for all test cases that are used during smoke and regression testing using the Katalon Studio tool. An excel sheet is provided for use by the SQA team as external test data so that the SQA team does not need to code when entering test data or test data. When the SQA team wants to do smoke and regression, they only need to make all test cases that have been scripted automation into a test suite and run them using these tools. Automation script will reduce smoke and regression testing time so that there is no delay in publishing new features. Furthermore, smoke series and regression testing reports will be emailed to the SQA team, making it easier for them to know the percentage of their testing process that was successful and failed.
In the testing process, the steps carried out are initiated by the SQA team by making test cases or test scenarios including positive scenarios and negative scenarios.
The SQA Lead will review the test cases made by the SQA team. After the test case review process is completed by the SQA Lead and the development team has completed the implementation, the SQA team will start the manual testing process on a local PC or local environment. When an error is found during testing, the development team must fix the feature as soon as possible. The SQA team retests the features fixed by the development team in the local environment. Next, the SQA team will test all features in the staging environment and do smoke and regression testing.

System built
To test the automation script built by SQA, create test data for each test case that will be run in an excel file. Test data is data that has been specifically identified for use in software testing. The excel file will be filled in by the SQA team according to the test case being run.

Implementation
Implementation of script automation for all smoke test cases and regression testing using the Katalon Studio tools. The programming language used to develop script automation is Java Groovy. Groovy is a dynamic object-oriented programming language for the Java virtual machine (JVM) that can be used anywhere Java is used.
Test Case registration is one of the main features of the system being tested, this test case is a test case that must be run when doing smoke and regression testing. The test case registration directory is shown in Figure 6.
Order loans are the most important feature in the peerto-peer lending process. Because with this feature, users or lenders can fund loans submitted by borrowers. This feature is included in smoke and regression testing. The test case order loan directory is as shown in figure 7.
The withdrawal test case is a test case related to the process of withdrawing money from lenders at XYZ. The test case withdrawing money directory is as shown in figure 8. For every investment in a loan/product that a lender makes, complete details about the lender's funding will be listed on the Portfolio page. Both the development or performance of the loan, repayment, reviewing details about the borrower, schedule and estimated monthly repayments, and other product details based on tenor, grade, interest rate, and so on, you can see on the Portfolio page. The Portfolio test case directory is as shown in figure 9.

Automation Script Execution
The script execution stage is carried out by all related parties until the process of publishing the new feature process. Based on the results of the tests carried out, the  Percentage of Achievement = Total score/maximum score * 100% = 762/950*100% = 80.21 % Based on the results provided by the respondents, it can be concluded that the automated testing built is easy to use, can be learned quickly, is not too complicated and encourages the SQA team to use it often. The result of the calculation of achievement based on the respondents' assessment is 80.21% which means very good (80% -100%). This means that the automation testing carried out using Katalon studio with the datadriven testing method is in the very good category.

Conclusion
Tests carried out with the script automation tool made the publication time of new features not delayed because SQA no longer needed to do smoke and regression tests manually, which took more than one day. Katalon studio tools and automation testing can support the application testing process in companies by making it easier for testers to do smoke and regression testing that is run automatically. The data-driven testing method really helps in making test data tests when running the automation script without touching the script code at all. This fund is very helpful for those who do not have experience in making script code. People who are dedicated to doing smoke and regression testing can use their time to control other work or check other features while the automation script is running which greatly helps the productivity of the SQA team. Based on the results of the research that has been carried out, it is concluded that the process of automation of software testing by applying a script automation tool made with Katalon Studio which applies the data-driven testing method is very good with an achievement rate of 80, 21%. The automation tool built is easy to use, can be learned quickly, is not too complicated, and makes users want to use it again.