Indian Journal of Science and Technology

Abstract

Objectives: Software testing is no more “Essential” requirement but it is “Critical” for the software application or product survival. It has become mental discipline and the tone of an organization mindset especially in the context of emerging DevOps practice. A few limitations of agile methodology from operational and business readiness perspective are addressed in DevOps. DevOps got emerged out of continuous software delivery which captures the market opportunities and reduces the feedback time. However, the new DevOps processes are influencing profoundly on QA and Testing functions. The present study is to understand different motivational factors driving testing in DevOps Environment and attempts to identify key technical, cultural and managerial factors of testing in DevOps setting. Methods/Statistical Analysis: We conducted extensive literature survey on academic and industry reports in the context of DevOps testing. Finally we screened 29 most relevant articles out of 295 found among 5 major journal databases. Further we extended manual survey including whitepapers and industry reports. Findings: It is evident that most of the articles connected to DevOps testing got published after year 2011 and surged after 2014. This phenomenon shows the nascence of the research progress in this domain. We observed that DevOps in testing is closely associated with automation of test cases. In its advanced stage, it is attributed to auto generation of test cases through model driven frameworks. This has been stressed in more than 50% of review articles. More than 50% of articles connected to cloud, virtualization, simulation techniques of DevOps Testing. The factors like agility, scale, metric driven process, reduction of complexity and cost appeared in more than 16 review articles. DevOps demands alternative metrics for better collaboration and communication between various stakeholders of the system. DevOps testing is face of organization culture and human resource mindset. This has been stressed in most of the review articles. Application/Improvements: This research is constrained from couple of biases (authority and publication). Research valued opinions of the other researchers. The search string used (DevOps Testing) may have multiple synonyms but we de-risked this threat by manually verifying each and every result. Further we extended manual survey using this search string. 
Keywords: Agile Testing, Continuous Integration, Continuous Testing, DevOps Testing