Upgrade data migration validation !

Purpose:  To make sure  data consistency in upgraded environment.

Thoughts : Dump data base before and after upgrade and check data diff for these 2 data bases.

Available Tools:

1. dbForge Data Compare for SQL Server
2. DBDiff 
3. Pgdatadiff tool 

Environment Setup:

1. dbForge Data Compare for SQL Server

Steps 1: Dump sql file before / after upgrade

docker exec -t vaidio pg_dump --format=c  -U ainvr> dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql

docker exec -t your-db-container pg_dump --format=c -U ainvr > dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql

Steps 2: Create Empty Database

createdb -U postgres -W ainvr;
createdb -U postgres -W ainvr2;

Steps3 : Restore data to created empty database

psql -h localhost -p 5433 -U postgres -f  D:\dump_18-10-2021_13_23_57_before.sql ainvr
psql -h localhost -p 5433 -U postgres -f  D:\dump_18-10-2021_14_33_09_after.sql ainvr2

2. DBDiff Tool                               

DBDiff is an automated database schema and data diff tool.

○     It compares two databases, local or remote, and produces a migration file of the differences automatically.

○     Command-Line API

Source Code: https://dbdiff.github.io/DBDiff/

Comparison result

Executed Command:

dbdiff postgres://postgres:@localhost:5433/ainvr postgres://postgres:@localhost:5433/ainvr2

Result:

-- DROP TABLE "public"."scene_object_2021_07_26";

-- DROP TABLE "public"."scene_object_2021_07_27";

-- DROP TABLE "public"."scene_object";

-- DROP TABLE "public"."scene_object_2021_07_28";

3. Pgdatadiff Tool

1. Firstly, compares the row count in both tables.
2. If the row count is the same, it instructs postgres to create MD5 sums from data.

The MD5 sums are based on the data being cast to varchar. 

If you have data types that don't cast to varchar properly then the behavior probably not reliable. 

Source Code: https://github.com/dmarkey/pgdatadiff

Comparison result

Executed command:

pgdatadiff --firstdb=postgres://postgres:@localhost:5433/ainvr --seconddb=postgres://postgres:@localhost:5433/ainvr2

Future

• Integrate with Jenkins through ssh set up environment
• Execute db diff scripts from Jenkins

Recommdation

•  Pgdatadiff Tool

Firstly it compares the row count in both tables.

If the row count is the same, it instructs postgres to create MD5 sums of "chunks" of the table in both DBs and compares them. This way no data is actually read directly by pgdatadiff, it also means that pgdatadiff is relatively fast but is puts a moderate amount of pressure on the DB as it calculates the MD5 sums of large amounts of data. 

If There Isn’t Enough Time To Test !

What to do when there is not enough time for a critical bug fix test just a few days before release?

Partway through your test cycle, everything going smooth and all tasks accomplished at a high success rate but suddenly just before few days to release,

Your manage or alpha tester and internal ad hoc tester found a critical issue and without that fix, the release could not make it.

Set up a meeting, Impact Analysis,

Actors: Project Manger/lead, QA manager/lead, Development manager/lead

It involves analyzing the impact of changes made in features/modules in the application. 

• Analyzing modules with the help of impact analysis documents. It will find the risks associated with any kind of changes in a module/product due to a fixed defect.
• It also helps to implement a prototype for the developers and testers to experience the effects in the system.

Conclusion:

1. Find out the critical fix area.

- Point out the responsibility of the fixed function(s) and associations of same fixed function(s) with other functions within the same module i.e. green area. 

2. Find out the impacted areas/modules due to the critical fix.

- Point out the how many function(s) are affected due to green area fixed function(s) in orange and purple area and what are the effected function(s) responsibilities.

If area/module(s) are equally impacted;

Maybe both impacted modules are sharing the same fixed function.

Test Plan Approach:

Green Module: Smoke Test or End-To-End Test.

Orange or Purple Module: Sanity Test

Here we can customized out Sanity Test:

1. Sanity Test = Regression Test Cases for only effecting function.

2. Sanity Test = Whole system Regression Test - Whole system Negative Regression Test Cases. 

3. Sanity Test = Regression Test Cases for only effecting function - Negative Regression Test Cases for only effecting function.

If area/module(s) are partially impacted;

Maybe partially impacted modules are sharing the return value or function parameter(s) from the highly impacted module due to fixed-function in green module.

Test Plan Approach:

Green Module: Smoke Test or End-To-End Test.

Orange Module : Sanity Test

Purple Module: Smoke Test or End-To-End Test or Ad-Hoc Test.

Here we can customized out Sanity Test as before:

1. Sanity Test = Regression Test Cases for only effecting function.

2. Sanity Test = Whole system Regression Test - Whole system Negative Regression Test Cases. 

3. Sanity Test = Regression Test Cases for only effecting function - Negative Regression Test Cases for only effecting function.

Conclusion:

Impact Analysis is a mandatory phase and crucial point where we can derive our test plan in a limited amount of time to minimize our release risk.

Test plan selection could be changed or customized with the same approach.

Task prioritization, how?

Many real-life decision-making situations have several conflicting criteria that need to be considered at the same time. 

It may be hard to choose, but if my team is great at prioritizing tasks, I can deal with a lot of missed tasks, some distractions, and too many meetings.

The better managers in this world know how to get the best from their teams. They can analyze their team’s strengths and weaknesses, and delegate tasks to team members who have the capabilities to carry them out. 

A Prioritization Matrix is a decision-making method that allows selecting the most appropriate alternative after evaluating multiple conflicting criteria.

This is what we had implemented in our organization and within my Team.

In other way,

Urgent Vs. Important?

Urgency of a task is largely governed by deadlines. Belongs To DO.

Importance of a task is decided by the impact or significant change it can have on your life or an organization.

Urgency is driven by external factors like deadlines.

Importance is more of an introspective(thoughtful) exercise

What you consider important can differ from what others consider as important.

What we got call as: Eisenhower Matrix,

 In case everything is urgent and important? 

Hear comes: "Prioritization"

The action or process of deciding the relative importance or urgency of a thing or things.

Set Priority Methods,

Ultimately prioritize your tasks will depend on the nature of your job and your personal work style, but there are a few common task prioritization methods that might work for you.

1. Eisenhower Decision Matrix

• You sort tasks only based on urgency and importance but it’s not tells the order.

2. Eat the Frog 
3. ABCDE Method
4. Chunking

Eat the frog,

When you have two frogs to eat, eat the ugliest one first or

“If it’s your job to eat a frog, it’s best to do it first thing in the morning.”

Get the biggest, ugliest task out of the way as soon as you can, and the rest will come easily. 

ABCDE method

Separate tasks with similar priorities using the ABCDE method.

Differentiating between tasks that feel like they’re on the same level of importance

Here’s how it works,

Go through your list and give every task a letter from A to E (A being the highest priority).

For every task that has an A, give it a number which dictates the order you’ll do it in.

Repeat until all tasks have letters and numbers.

How to set priority?

Here’s where Brian Tracy’s ABCDE method works wonders. Instead of keeping all tasks on a single level of priority, this method offers two or more levels for each task.

Combined Eat The Frog and ABCDE Method,

The ABCDE method works well when combined with Eat the Frog.

Eat The Frog says “Get the biggest, ugliest task first”

ABCDE says “Assign each task a letter value according to its level of importance”

If you start your day off with your A and B tasks, then you can spend the rest of your day on tasks with lower importance, like C, D, and E.

How To set level of importance:

Ugliest task

Dependent task

Lets, we have following task’s and complete in 10 days sprint – 

According To Eisenhower Matrix all are “DO” tasks i.e. important and urgent,

• Essential Testing
• Defect Verification
• Regression Testing
• Execute API Auto 
• Creating Test Cases for new features
• Upgrade Testing
• Installation Testing
• Device App. Testing

Live workshop: Eat The Frog,

Live workshop: ABCDE Method,

Live workshop: Combined Method,

Live workshop: Final Combined Method,

Conclusion:

- Use Eat the Frog Method

• Based on weighted and complexity.

- Use ABCDE Method 

• Always put the dependent task’s on priority.
• Assign each task a letter value.

- Combine both to set your priority.

When you clearly prioritize your work, you can increase productivity, better manage your time, and feel confident that you’ll hit your deadlines—every time.

Setup Automated Mobile Test Build Parameters in Jenkins !

Jenkins File -

properties([ 

disableConcurrentBuilds(),

buildDiscarder(logRotator(numToKeepStr: '25')),

parameters([

string(defaultValue: "localhost", description: 'Server Login IP address', name: 'LOGIN-IP-VALUE'),

choice(choices: ['All','Smoke','Regression'], description: 'Test Suite Name', name: 'Test_Suite_Name'),

choice(choices: ['emulator','device'], description: 'Device Type', name: 'DEVICE_TYPE'),

extendedChoice(name: 'Slave', value: 'Manish,Evan,Max', defaultValue: 'Manish', description: 'Slave Machine Name', type: 'PT_MULTI_SELECT', visibleItemCount: 2)

])

])

node("${env.Slave}") {

stage('Checkout') {

timestamps {

checkout scm

}

}

stage('Test') {

timestamps {

try {

bat 'python.exe -m robot --variable LOGIN-IP-VALUE:%LOGIN-IP-VALUE% --variable DEVICE_TYPE:%DEVICE_TYPE% -d Output -i %Test_Suite_Name% --loglevel TRACE Tests/'

} catch (Exception e) {

emailSubject = "Jenkins build Failure: ${JOB_NAME} #${BUILD_NUMBER}"

emailRecipients = [[$class: 'DevelopersRecipientProvider'], [$class: 'RequesterRecipientProvider']]

emailBody = """

<style>
table {border:1px solid #ccc}
th {border:1px solid #ccc}
td {border:1px solid #ccc}
</style>
<table>
<tr><th>Result</th><td>${e}</td></tr>
<tr><th>Name</th><td>${currentBuild.fullDisplayName}</td></tr>
<tr><th>Job URL</th><td><a href="${JOB_URL}">${JOB_URL}</a></td></tr>
<tr><th>Build URL</th><td><a href="${BUILD_URL}">${BUILD_URL}</a></td></tr>
</table>
""".stripIndent()

emailext attachLog: true, mimeType: 'text/html', subject: emailSubject, body: emailBody, recipientProviders: emailRecipients

throw e

} finally {

stage('Generate report') {

publishHTML([allowMissing: false,

alwaysLinkToLastBuild: true,

keepAll: true,

reportDir:'Output',

reportFiles:'report.html',

reportName:'Mobile Integration Test Report'

])

}

                        }
                }
        }
}

Preview in Jenkins -

As per my assumption out tests should be enough capable to run on the given environment -

LOGIN-IP-VALUE - Execute test's on given IP address.

Test_Suite_Name - Execute tests based on selected types of scenarios.

DEVICE_TYPE - Using emulator or on the real device connected via USB.

Slave - Execute test on different machines based on user identity.

Setup Mobile Automation With Robot Framework And Appium !

 Robot Framework  [only for android]

 Is an open-source test automation framework for acceptance testing and acceptance test-driven development. It follows different test case styles − keyword-driven, behavior-driven, and data-driven for writing test cases.

Appium is an open-source tool to automate native, web and hybrid mobile application.

Setup needed in Windows?

1. Java 
2. Android Studio
3. Android Emulator 
4. Node.js 
5. Appium
6. Robot Framework 
7. Python - IDE PyCharm

Java Setup -

Android Studio Setup -

Android Emulator Setup -

Node JS Setup -

Appium Setup -

Robot Framework Setup – Install Python -

Robot Framework Setup – Install Robot -

Robot Framework Setup – Install Pycharm - 

Execute Test - 

robot -d Output --loglevel TRACE Tests/Essential/Login.robot

robot -d Output --loglevel TRACE -i Regression Tests/*

robot -d Output --loglevel TRACE -i All Tests/*

Mobile Device Testing using AWS Device Farm !

Device Farm is an app testing service that you can use to test and interact with your Android, iOS, and web apps on real, physical phones and tablets that are hosted by Amazon Web Services (AWS) since 2015.

  There are two main ways to use Device Farm:

• Automated testing of apps using a variety of testing frameworks.

• Remote access of devices onto which you can load, run, and interact with apps in real time.

Automated Testing Vs. Remote Access

Problems will solve?

• The cost and complexity of effective testing is growing
• Maintain device on lab
• Manual testing on each device is not practical
• Record to trace the testing issue

Free Edition Limits in AWS Device Farm?

• Mobile App package size < 4 GB .
• Remote Access/ Automated testing only allow 1 hour at most each Session.

Pricing?

Webdriver Wait for window loads !

Wait for window loads and then select the window.

/// Wait for window till it gets load on the page.

///

/// The name of the window.

/// This is time to wait.

/// if the window not loads in the specified interval of time.

/// If the window not exists on the page.

        protected void WaitUntilWindowLoads(string windowName, int timeOut = -1)

        {

            // Intialization of bool 'isWindowPresent' for checking window present

            Stopwatch stopWatch = new Stopwatch();

            stopWatch.Start();

            if (timeOut == -1)

            {

                timeOut = this.waitTimeOut;

            }

            while (stopWatch.Elapsed.TotalSeconds < timeOut)

            {

                try

                {

                    //Wait for window

                    if (WaitUntilWindow(windowName) == true) break;

                }

                catch (Exception)

                {

                    // For Any exceptions catch value is assinged false

                }

            }

        }

  

///

/// Is Window Opened In the Specified Interval of Time or not.

///

/// This is the name of the window.

/// This is the time to wait for window get open.

/// True if the window is opened otherwise false.

/// If the window not able to find in the specified time.

/// If the window not exists on the page.

        protected bool WaitUntilWindow(string windowName, int timeOut = -1)

        {

            Stopwatch stopWatch = new Stopwatch();

            stopWatch.Start();

            if (timeOut == -1)

            {

                timeOut = this.waitTimeOut;

            }

            while (stopWatch.Elapsed.TotalSeconds < timeOut)

            {

                if (WebDriver.WindowHandles.Any(item =>                       WebDriver.SwitchTo().Window(item).Title == windowName))

                {

                    return true;

                }

            }

            stopWatch.Stop();

            return false;

        }