TRIGGERBLOCK

Saturday, August 10, 2019

XML to HTML report using xsl

Sample junit xml report

<testsuite errors="0" failures="2" hostname="Deepaks-MacBook-Air.local" name="xxx.scalaTests.common.DefaultColumnAdditionSpec" tests="4" time="7.66" timestamp="2019-08-10T09:58:56">

<testcase name="1To validate resultant df, when input datatype is of type string. 1To validate resultant df, when input datatype is of type string." classname="xxx.scalaTests.common.DefaultColumnAdditionSpec" time="7.625"> </testcase>

<testcase name="3To validate resultant df, when input datatype is of type string. 3To validate resultant df, when input datatype is of type string." classname="xxx.scalaTests.common.DefaultColumnAdditionSpec" time="0.003"> </testcase>

<testcase name="4To validate resultant df, when input datatype is of type string. 4To validate resultant df, when input datatype is of type string." classname="xxx.scalaTests.common.DefaultColumnAdditionSpec" time="0.008">
<failure message="org.scalatest.exceptions.TestFailedException was thrown." type="class org.scalatest.exceptions.TestFailedException">...</failure>
</testcase>

<testcase name="5To validate resultant df, when input datatype is of type string. 5To validate resultant df, when input datatype is of type string." classname="xxx.scalaTests.common.DefaultColumnAdditionSpec" time="0.0">
<failure message="org.scalatest.exceptions.TestFailedException was thrown." type="class org.scalatest.exceptions.TestFailedException">...</failure>
</testcase>

</testsuite>

----save as test.xsl--------
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
<html>
<body>
<h2>Result</h2>
<table border="1">
<tr bgcolor="#9acd32">
<th style="text-align:left">Test Name</th>
<th style="text-align:left">Result</th>
</tr>
<xsl:for-each select="testsuites /testsuite/testcase">
<tr>
<td><xsl:value-of select="@name"/></td>
<xsl:choose>
<xsl:when test="failure"><td>FAIL</td></xsl:when>
<xsl:otherwise><td>PASS</td></xsl:otherwise>
</xsl:choose>
</tr>
</xsl:for-each>
</table>
</body>
</html>
</xsl:template>
</xsl:stylesheet>
----------------

STEPS:

Save the above xsl code as test.xsl
Open terminal
cd to juit report file.
Run below code

MAC : xsltproc -o result.html test.xsl junit_result.xml
Win : msxsl sourcefile.xml test.xsl -o junit_result.html

Sunday, August 4, 2019

ScalaTest : testng

ScalaTest : TestNG

Pre-Req :

Os - Linux ubuntu / debian
Java /jdk 1.8 +
scala 2.11 +
Intellij
Install sbt plugin in Intelli
Create an scala project in Intelli using sbt

Commands:
scala -version
sbt sbtVersion

----build.sbt (add below line)--------
scalaVersion := "x.xx.0"
libraryDependencies += "org.scalatest" % "scalatest_x.xx" % "3.0.8" % "test"
libraryDependencies += "org.testng" % "testng" % "6.10"

--- install sbt----
open terminal
echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823
sudo apt-get update
sudo apt-get install sbt
---------------------

Location : src/test/scala/ExampleTestNGSuite.scala

//https://searchcode.com/codesearch/view/16471591/
//http://doc.scalatest.org/1.8/index.html#package

import org.scalatestplus.testng.TestNGSuite
import org.testng.annotations.Test
import org.testng.annotations.BeforeMethod
import org.testng.annotations.BeforeClass
import org.testng.annotations.BeforeSuite
import org.testng.annotations.AfterMethod
import org.testng.annotations.AfterClass
import org.testng.annotations.AfterSuite
import org.testng.annotations.DataProvider

class ExampleTestNGSuite extends TestNGSuite {

// @AfterSuite def failAfterSuite(){ throw new Exception("fail in before method") }
@BeforeMethod def passBeforeMethod(){println("b3")}
@BeforeClass def passBeforeClass(){println("b2")}
@BeforeSuite def passBeforeSuite(){println("b1")}
@AfterMethod def passAfterMethod(){println("a2")}
@AfterClass def passAfterClass(){println("a3")}
@AfterSuite def passAfterSuite(){println("a1")}
@Test(invocationCount = 10)
def thisTestRunsTenTimes = {}
/*
@Test(groups = Array("runMe"))
def testWithException(){
throw new Exception("exception!!!")
}
@Test(groups = Array("runMe"))
def testWithAssertFail = assert( 1 === 2, "assert fail!!!" )

@Test(dependsOnMethods = Array("testWithException"))
def testToGetSkipped = {}
*/
@DataProvider(name = "any_name")
def andValues = {
val test_var = Array("0", "1")
for( x <- test_var; y <- test_var ) yield Array(x,y)
}

@Test(dataProvider = "any_name")
def testAndStates(a: String, b: String){
println("a="+ a + ",b="+ b)
}
}

-----Run Sbt in terminal -----
cd to projects >
run command "sbt test"

------------------------

Result :
A testNG report has to be generated in the project folder .

Sunday, July 28, 2019

Testing with Scala / SBT

Pre-Req :
Os - Linux ubuntu / debian
Java /jdk 1.8 +
scala 2.11 +
Intellij
Install sbt plugin in Intelli
Create an scala project in Intelli using sbt

Commands:
scala -version
sbt sbtVersion

----build.sbt (add below line)--------
scalaVersion := "2.xx.0"
libraryDependencies += "org.scalatest" % "scalatest_2.xx" % "3.0.8" % "test"
libraryDependencies +="org.pegdown" % "pegdown" % "1.4.2" % "test"
libraryDependencies += "org.mockito" % "mockito-all" % "1.8.4"
libraryDependencies += "org.scalamock" %% "scalamock" % "4.3.0" % "test"
testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-h", "target/test-reports")
(click on auto-import link when it appears)

--- install sbt----
open terminal
echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823
sudo apt-get update
sudo apt-get install sbt

---- Config sbt in IntelliJ----
to Debug/Run test in sbt:
1. Edit Configurations
2. "+" Add Scala Test
3. In the new file , Select the test file (test.scala)
4. Check "use sbt"
5. Apply and OK
6. rt ck on test file in File structure/ editor / tool bar - ck run/Debug
-----Run Sbt in terminal -----
cd to projects >
run command "sbt test"
---------------------

Src File : src/main/scala/sqCalculator.scala
object sqCalculator extends App {
def sq(x: Int) = {
x * x
}
}

Test File : test/scala/sqCalculatorTest.scala
import org.scalatest.FunSuite
class sqCalculatorTest extends FunSuite {
test("sqCalculator.cube") {
if(sqCalculator.sq(3) === 9){
println ("passed")
assert (true)
}
else{println (false)}
}
}
---------------------//Mockito----------------------------
//test/scala/LoginService.scala

trait LoginService {
def login(name: String, password: String): String
}

//test/scala/LoginServiceTest.scala
import org.scalatest.FunSuite
import org.scalatest.BeforeAndAfter
import org.scalatestplus.mockito.MockitoSugar
import org.mockito.Mockito._

class LoginServiceTest extends FunSuite with BeforeAndAfter with MockitoSugar {
before {
println("Start")
}
after {
println("Ends")
}

test ("test login service"){
val service = mock[LoginService]
when(service.login("johndoe", "secret")).thenReturn(("johndoe"))
when(service.login("joehacker", "secret")).thenReturn("an")
val johndoe = service.login("johndoe", "secret")
val joehacker = service.login("joehacker", "secret")
if(johndoe == "johndoe"){
assert(true)
}
if(joehacker == None)
assert(false)
else
assert(true)
}
}

----------------ScalaMock--------------------------------
//test/scala/LoginServiceTest2.scala
import org.scalatest.{BeforeAndAfter, FunSpec, FunSuite}
import org.scalamock.scalatest.MockFactory

class LoginServiceTest2 extends FunSpec with BeforeAndAfter with MockFactory {
before{
println("LoginServiceTest2 - start")
}
it("val test") {
val any_name: LoginService = mock[LoginService]
(any_name.login _) expects("a","a") returning "hi"
println(any_name.login("a","a"))
}
after{
println("LoginServiceTest2 - end")
}
}

Monday, July 8, 2019

Python : Pytest-selenium without setup/installation of driver

Python : Pytest-selenium without setup/installation of driver

project/requirements.txt

pandas >= 0.24.2
numpy >= 1.16.3
pytest >= 3.10.1
pytest-html >= 1.20.0
pytest-metadata >= 1.8.0
pytest-profiling >= 1.6.0
pyderman >= 1.3.0
pytest-selenium >= 1.16.0

project/tests/test_case1.py

import pytest
from lib import csv_reader
import os
import pyderman as dr
from pathlib import Path



link="https://www.stats.govt.nz/assets/Uploads/Electronic-card-transactions/Electronic-card-transactions-May-2019/Download-data/electronic-card-transaction-may-2019-csv.zip"
import pytest
@pytest.fixturedef firefox_options(firefox_options):
    #firefox_options.binary = '/path/to/firefox-bin'

       destination = str(Path(__file__).resolve().parents[1]) + "/temp/"    destination="/home/deepak/PycharmProjects/csv_test/temp"

   if(os.path.exists(destination)):
        print(destination)
        print("paht exists")
    destination=destination.replace("\\","\\\\")
    firefox_options.add_argument('-foreground')
    firefox_options.set_preference('browser.download.folderList', 2)  # custom location    firefox_options.set_preference('browser.download.manager.showWhenStarting', False)
    firefox_options.set_preference('browser.download.dir', destination)
    firefox_options.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/zip")
    return firefox_options

def test_deep():
    if(1==1):
        path = dr.install(browser=dr.firefox)
        print('Installed geckodriver driver to path: %s' % path)
        print("yes")
        assert True    else:
        print("no")
        assert False

def test_example(selenium):
    selenium.get(link)

--------------------------

To RUn :

1. open terminal

2. cd inside project

3. pip install requirements.txt

4. pytest

Python : Pytest logger

Create file : /project/pytest.ini

[pytest]

testpaths = tests/
python_paths = ./

console_output_style = progress
addopts = -rsxX -q

log_cli = 1
log_cli_level = INFO
log_cli_format = %(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)
log_cli_date_format=%Y-%m-%d %H:%M:%S

log_file_date_format = %Y-%m-%d %H:%M:%S
log_file_format = %(asctime)s %(levelname)s %(message)s
log_date_format = %Y-%m-%d %H:%M:%S
log_file=simple_logger.log
log_print = True

python_files =
test_*.py
*_test.py

Using without class :
LOGGER = logging.getLogger(__name__) #Decalared as a global
LOGGER.error("Unable to make connection to DB")#to log fails inside/ouside def

Usage in a class :
self.LOGGER = logging.getLogger(__name__)#inside a constructor
self.LOGGER.error("Unable to make connection to DB")#to log fails

Thursday, July 4, 2019

Python : Config /Properties/ ini file

Python : Config/ini/Properties file

-----------credentials.config-----------

[db]
dbname= dbname
hostname= host_anme
port= 1234
username= usr_name
password= pass

----------------------

Python : Config /Properties/ ini file

from configparser import RawConfigParser

config_path = os.getcwd()+'/properties/credentials.config'
confparser = RawConfigParser()
with open(config_path, "r") as config_file:
confparser.read_file(config_file)

KWARGS = {
"dbname": confparser["db"]["dbname"],
"hostname": confparser["db"]["hostname"],
"port":confparser["db"]["port"],
"username": confparser["db"]["username"],
"password": confparser["db"]["password"]
}
"""

Friday, June 21, 2019

MongoDb : MongoDb no SQL

Monday, May 27, 2019

pandas : Lambda ,Map and Eval Conditions

Lambda ,Map and eval in Pandas

1. Simple

import pandas as pd

a=pd.DataFrame({"a":["deea","123","deep","deepak"]})

b=pd.DataFrame({"b":["deeap","1234","deepak","deepa"]})

c=pd.DataFrame(map(lambda x,y:x in y,a['a'],b['b']))

print(c)

result :

0 True

1 True

2 True

3 False

2. Using Dynamic Formula :

import pandas as pd

a=pd.DataFrame({"a":["deea","123","deep","deepak"]})

b=pd.DataFrame({"b":["deeap","1234","deepak","deepa"]})

c=pd.DataFrame(map(eval(input("enter -lambda x,y:x in y")),a['a'].values,b['b'].values))

print(c)

0 True

1 True

2 True

3 False

3)Using Dynamic Formula and 1st columns

import pandas as pd

a=pd.DataFrame({"a":["deea","123","deep","deepak"]})

b=pd.DataFrame({"b":["deeap","1234","deepak","deepa"]})

c=pd.DataFrame(map(eval(input("enter -lambda x,y:x in y")),a.iloc[:,0],b.iloc[:,0]))

print(c)

Result:

0 True

1 True

2 True

3 False

"""

Column Names

data.iloc[:,0] # first column of data frame (first_name)
data.iloc[:,1] # second column of data frame (last_name)
data.iloc[:,-1] # last column of data frame (id)

Rows and Columns
data.iloc[0:5] # first five rows of dataframe
data.iloc[:, 0:2] # first two columns of data frame with all rows
data.iloc[[0,3,6,24], [0,5,6]] # 1st, 4th, 7th, 25th row + 1st 6th 7th columns.
data.iloc[0:5, 5:8] # first 5 rows and 5th, 6th, 7th columns of data frame
"""

Thursday, May 23, 2019

Numpy : Lambda / map / vectorize

import numpy as np

def myfunc(a, b):

if a > b:

return a - b

else:

return a + b

n1=np.array([1,2,3])

n2=np.array([5,4,3])

vfunc = np.vectorize(myfunc)

print(vfunc(n1,n2))

Output :

[6 6 6]

Saturday, May 4, 2019

pandas : Import snippet

TC_ID	TC_NAME	QUERY1	QUERY2	CONDITION	BASICTEST(QUERY1/QUERY2)
TC0001	TC_NAME1	Select col1 from table1	Select col1 from table2	1	11
TC0002	TC_NAME2	Select col1 from table2	Select col1 from table3	2	12
TC0003	TC_NAME3	Select col1 from table3	Select col1 from table4	3	13
TC0004	TC_NAME4	Select col1 from table4	Select col1 from table5	4	14
TC0005	TC_NAME5	Select col1 from table5	Select col1 from table6	5	15
TC0006	TC_NAME6	Select col1 from table6	Select col1 from table7	6	16

To select row and Traverse cells from selected rows
import pandas as pd
df=pd.read_csv("/home/deepak/PycharmProjects/csv_test/test_cases.csv")
O_row=(df[df["TC_ID"]=="TC0006"])
print(O_row["TC_NAME"].values[0])

Output:
TC_NAME6

To convert CSV to list of dictionaries:
(Using Pandas)
import pandas as pd
import csv
df = pd.read_csv(sPath)
list_of_data = df.to_dict(orient='records')#dropna().to_dict(orient='recorddel df
return list_of_data

(Using CSV)
import pandas as pd
import csv
with open(sPath, "r") as f:
reader = csv.DictReader(f)
list_dict = list(reader)
return list_dict

Saturday, April 27, 2019

mac setup

Mac Setup

1. To install "brew" - Ref https://brew.sh/

Run below command:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

2. brew install ranger
3. Change terminal theme to Brew
Terminal -> Preferences -> Settings -> Click desired Profile -> Click "Default" at the bottom of the frame

4. Install sublime text from google
5. Run below command in mac

ln -s "/Applications/Sublime Text.app/Contents/SharedSupport/bin/subl" /usr/local/bin/subl

6.run command "subl" to open sublime text
7. open ranger , navigate to any text file
8. run command "!subl xx.txt"
9.alias vlc='/Applications/VLC.app/Contents/MacOS/VLC'
10. vlc input.mp4

Wednesday, April 24, 2019

ETL : Testing Basics

SQL types:

DML (Data Manuipulation) - Insert , update , Delete , Merge
DDL (Data Definition)-Create , alter,Drop,Truncate
DRL (Data Retrival )- Select
TCL (Transaction Lan)- Commit Rollback
DCL (Data Control)- grant ,revoke

Data WareHouse :

It is a DataBase
Contains data for Analysis

OLTP (Online Transaction Processing System)

Used in Day to Day Transaction
Ex : ATM , Net Banking
Data is Captured into either DB , Flat File (xml) etc.,

DataMart:

It is a smaller part of DataWarehouse
It is specific to a department eg: Finance , HR etc.,
It is used by middle level management to retrieve data wrt to their deptartment.

ETL ( Extract , Transform ,Load):

Process of read data from OLTP and inserting into Data Warehouse
The Data from Data Warehouse is used in Analytics report

Ex: Company A has following departments Eg: Sales , Services department.

Assume Sales departments has records of customer transaction in DB and Services has transaction in an csv file or flat file ( service number , service mechnaic name ,amount paid , spares replaced ,Date etc.,)

If management wants to find out the total transaction of a customer both sales and services how will they do ?

So they need to extract only relavent information from both data sources and load it into a common Destination .This is called ETL.

Hetrogenous Data : Data coming from different Data Sources like Flat file , Orcale DB, Mongo, My Sql ,txt,json etc.,

Mapping Document ( http://dwbitips.blogspot.com/2012/12/what-is-etl-mapping-document-real-time.html ) :

Requirement Document
Contains src , dest table and Column Details
Conatins transformation query or forumula applied for each column

ETL testing = Data Integration Testing + BI testing (Report)

Generic Data Integration Testing :

Scheme match wrt requirement
Count match with src and destination
Duplicate records (loaded twice),
Null Validation
Untransformed Data should match source and destination data (1:1).
Mapping is correct ie., data is going to correct cols as per requirment
Old data already residing is not affected

Late Arriving Dimension or Early Arriving Fact

Happens when you get fact data before the dimension data arrives
example: New employee just onboarded and had an accident before insurance forms could be completed and processed by the insurance company. The hospital will create a medical claim record to be paid by the insurance company, but the insurance company does not yet have a person to associate the claim.
-1 = UNKNOWN, -2 = N/A, -3 = Not Provided etc.,

Ref:

https://www.softwaretestinghelp.com/etl-testing-data-warehouse-testing/

https://www.guru99.com/utlimate-guide-etl-datawarehouse-testing.html

https://www.tutorialspoint.com/etl_testing/index.htm

Sunday, April 21, 2019

pytest : All about pytest

https://drive.google.com/file/d/1-yL5U1ZQaH6NNv2XkCb2yXe_Nq0Piiw1/view?usp=sharing

Index :

Installation x 2
Project structure
Sample code
Important commands x 7
Run specific test case
Raise pytest exception (pytest.raises)
labels (pytest.mark.label_name ,pytest.mark.skip,pytest.mark.skip_if)
Parameterize (pytest.mark.parametrize('x,y,res',[(1,2,3),(0,3,3)]) )
Fixture (pytest.fixture(scope="module") )
conftest
pdb

Installation:
Using pip

pip3 install pytest

pip install selenium

Using requirements.txt : project>requirements.txt

selenium >= 3.0

pytest >= 3.10.1

pytest-html >= 1.20.0

pip install -r requirements.txt

pipenv install -r requirements.txt #Inside virtual env
Project structure

add file : - project>tests>test_filename.py

Sample Code

from selenium import webdriver

def test_1():

driver=webdriver.Chrome("chromedriverpath")

driver.get(url)

objs=driver.find_elements_by_xpath("//button")

for i in objs:

i.click()

open terminal
cd to project
excute command "pytest"

Important commands:

1. pytest --pdb #runs the debugger when an err is encountered

2. pytest -s #to "print" result in console

3. pytest -v #verbose

4. pytest --maxfail=2 # limit no of fails

5. py.test --html=1.html

6. pipenv run pytest

7. pipenv run py.test --html=1.html

pytest -v -k "add or something" # runs only methods with add ,something

pytest -h # help

pytest -v -x #to stop after 1st fail

pytest -v -x --tb=no # do not display stack trace

pytest -q # quiet mode

pytest -rsx #report skipped tests

pytest --lf #runs only tests that failed on last attempt

# in eclipse Window > Preferences > Pydev> PyUnit = Py.testRunner and add above Parameters

Execute specific method of class

pytest test_filename.py::test_method

Exception
def test_mytest():

with pytest.raises(ZeroDivisionError):
1/0

Markers(labels)

@pytest.mark.label_name" .To run "pytest -v -m label_name"

@pytest.mark.skip(reason="any reason") #skip methods

@pytest.mark.skip_if(sys.version_info < (3,3),reason="some reason")

Parametrize

import pytest

@pytest.mark.parametrize('x,y,res',[(1,2,3),(0,3,3)])

def test_add(x,y,res):

assert classname.add(x,y,res)

Fixtures(similar to @Before annotations in Junit test used for - db connection , etc.,)
@pytest.fixture(scope="module") #scope =module/session/function def fix():
    pass    #setup code
    yield   #run till here
    print("done") # tear down code

def fn(fix):
    #fix runs 1st

example:
from selenium import webdriver
@pytest.fixture()
def test_setup():
    global driver
    driver =webdriver.Chrome(executable_path="C:/Users/driver.exe")# Windows
    driver.implicitly.wait(5)
    driver.maximise()
    yield()
driver.close()
driver.quit()

conftest.py
You can put all fixtures inside this file and this file will execute before test starts

Debugger :

import pdb;

pdb.set_trace() # to break

Others

Eclipse Setup :

(if u are using eclipse : Window->Preferences --> pydev --> PyUnit --> Change the Test runner to "Py.test runner".)

Right Click over the file. Run As --> Python Unit-Test

Or press Ctrl+F9:-

It will prompt you to select the test

User can use fixture or setup-teardown
setup
import pytest
def setup_module(module):

tiredown
import pytest
def teardown_module(module):

Allure report
pip install allure-pytest
pip list | grep -i allure

pytest --alluredir=/Users/Documents/reports
allure generate /Users/Documents/reports

setup.py
#Add this to setup.py file:
from setuptools import setup
setup(
    # ...,
    setup_requires=["pytest-runner", ...],
    tests_require=["pytest", ...],
    # ...,
)

#And create an alias into setup.cfg file:
[aliases]
test=pytest
[tool:pytest]
addopts = --verbose
python_files = testing/*/*.py

#If you now type:
python setup.py test

Links

#https://docs.pytest.org/en/latest/goodpractices.html#goodpractices

https://docs.pytest.org/en/latest/parametrize.html
https://docs.pytest.org/en/latest/example/parametrize.html#paramexamples
https://pypi.org/project/pytest-html/
https://pypi.org/project/pytest-csv/

important read
https://bytes.yingw787.com/posts/2018/12/10/data_driven_testing_three/
https://gitlab.com/qalabs/blog/pytest-parametrize-example

important watch **
https://www.youtube.com/watch?v=o9pEzgHorH0
https://www.youtube.com/watch?v=RdE-d_EhzmA

hooks
https://stackoverflow.com/questions/21930858/pytest-parameterize-row-from-csv-as-a-testcase?rq=1

Running pytest inside eclipse ide
Ref :https://stackoverflow.com/questions/37985589/how-to-debug-or-run-pytest-scripts-using-eclipse