Saturday, April 27, 2019

mac setup

Mac Setup



1. To install "brew" - Ref https://brew.sh/
Run below command:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

2. brew install ranger
3. Change terminal theme to Brew
Terminal -> Preferences -> Settings -> Click desired Profile -> Click "Default" at the bottom of the frame

4. Install sublime text from google
5. Run below command in mac

ln -s "/Applications/Sublime Text.app/Contents/SharedSupport/bin/subl" /usr/local/bin/subl

6.run command "subl" to open sublime text
7. open ranger , navigate to any text file
8. run command "!subl xx.txt"
9.alias vlc='/Applications/VLC.app/Contents/MacOS/VLC'
10. vlc input.mp4

Wednesday, April 24, 2019

ETL : Testing Basics

ETL : Testing Basics 

 SQL types:
  • DML (Data Manuipulation) - Insert , update , Delete , Merge
  • DDL (Data Definition)-Create , alter,Drop,Truncate
  • DRL (Data Retrival )- Select
  • TCL (Transaction Lan)- Commit Rollback
  • DCL (Data Control)- grant ,revoke
Data WareHouse :
  1. It is a DataBase
  2. Contains data for Analysis
OLTP (Online Transaction Processing System
  •  Used in Day to Day Transaction
  • Ex : ATM , Net Banking 
  • Data is Captured into either DB , Flat File (xml) etc.,
DataMart:
  • It is a smaller part of DataWarehouse
  • It is specific to a department eg: Finance , HR etc.,
  •  It is used by middle level management to retrieve data wrt to their deptartment.
ETL ( Extract , Transform ,Load):
  • Process of read data from OLTP and  inserting into Data Warehouse
  • The Data from Data Warehouse is used in Analytics report
Ex: Company A has following departments Eg: Sales , Services department. 
Assume Sales departments has records of customer transaction in DB and Services has transaction in an csv file or flat file ( service number , service mechnaic name ,amount paid , spares replaced ,Date etc.,)
If management wants to find out the total transaction of a customer both sales and services how will they do ?
So they need to extract only relavent information from both data sources and load it into a common Destination .This is called ETL.
Hetrogenous Data : Data coming from different Data Sources like Flat file , Orcale DB,  Mongo, My Sql  ,txt,json etc.,
Mapping Document ( http://dwbitips.blogspot.com/2012/12/what-is-etl-mapping-document-real-time.html ) :
  1. Requirement Document
  2. Contains src , dest table and Column Details
  3.  Conatins transformation query or forumula applied for each column

ETL testing = Data Integration Testing + BI testing (Report)
Generic Data Integration Testing : 
  1. Scheme match wrt requirement
  2. Count match with src and destination
  3. Duplicate records (loaded twice), 
  4. Null Validation
  5. Untransformed Data should match source and destination data (1:1).
  6. Mapping is correct ie., data is going to correct cols as per requirment
  7. Old data already residing is not affected

Late Arriving Dimension or Early Arriving Fact

  • Happens when you get fact data before the dimension data arrives 
  • example:  New employee just onboarded and had an accident before insurance forms could be completed and processed by the insurance company. The hospital will create a medical claim record to be paid by the insurance company, but the insurance company does not yet have a person to associate the claim.
  •  -1 = UNKNOWN, -2 = N/A, -3 = Not Provided etc., 



 Ref:
https://www.softwaretestinghelp.com/etl-testing-data-warehouse-testing/
https://www.guru99.com/utlimate-guide-etl-datawarehouse-testing.html
https://www.tutorialspoint.com/etl_testing/index.htm

Sunday, April 21, 2019

pytest : All about pytest

pytest : All about pytest




Index :
  1.  Installation x 2
  2.  Project structure
  3. Sample code
  4. Important commands x 7
  5. Run specific test case
  6. Raise pytest exception (pytest.raises)
  7. labels (pytest.mark.label_name ,pytest.mark.skip,pytest.mark.skip_if)
  8. Parameterize (pytest.mark.parametrize('x,y,res',[(1,2,3),(0,3,3)]) )
  9. Fixture (pytest.fixture(scope="module") )
  10. conftest
  11. pdb
Installation:
Using pip
pip3 install pytest
pip install selenium

Using requirements.txt : project>requirements.txt
selenium >= 3.0
pytest >= 3.10.1
pytest-html >= 1.20.0

pip install -r requirements.txt
or
pipenv install -r requirements.txt #Inside virtual env
Project structure

add file : - project>tests>test_filename.py

Sample Code
from selenium import webdriver
def test_1():
 driver=webdriver.Chrome("chromedriverpath")
 driver.get(url)
 objs=driver.find_elements_by_xpath("//button")
 for i in objs:
  i.click()
  1. open terminal
  2. cd to project
  3. excute command "pytest"
Important commands:
1.    pytest --pdb #runs the debugger when an err is encountered
2.    pytest -s #to "print" result in console
3.    pytest -v #verbose 
4.    pytest --maxfail=2 # limit no of fails
5.    py.test --html=1.html
6.    pipenv run pytest
7.    pipenv run py.test --html=1.html

pytest -v -k "add or something" # runs only methods with add ,something
pytest -h # help
pytest -v -x #to stop after 1st fail
pytest -v -x --tb=no # do not display stack trace
pytest -q # quiet mode
pytest -rsx #report skipped tests 
pytest --lf #runs only tests that failed on last attempt
# in eclipse Window > Preferences > Pydev> PyUnit = Py.testRunner and add above Parameters 

Execute specific method of class
pytest test_filename.py::test_method

Exception
def test_mytest():
with pytest.raises(ZeroDivisionError):
  1/0
  
Markers(labels)
@pytest.mark.label_name" .To run "pytest -v -m label_name"
@pytest.mark.skip(reason="any reason") #skip methods
@pytest.mark.skip_if(sys.version_info < (3,3),reason="some reason")

Parametrize
import pytest
@pytest.mark.parametrize('x,y,res',[(1,2,3),(0,3,3)])
def test_add(x,y,res):
    assert classname.add(x,y,res)

Fixtures(similar to @Before annotations in Junit test used for - db connection , etc.,)

@pytest.fixture(scope="module") #scope =module/session/function def fix():
    pass    #setup code
    yield   #run till here
    print("done") # tear down code

def fn(fix):
    #fix runs 1st

example:
from selenium import webdriver
@pytest.fixture()
def test_setup():
    global driver
    driver =webdriver.Chrome(executable_path="C:/Users/driver.exe")# Windows
    driver.implicitly.wait(5)
    driver.maximise()
    yield()

    driver.close()
    driver.quit()

conftest.py
You can put all fixtures inside this file and this file will execute before test starts


Debugger :
import pdb;
pdb.set_trace() # to break

Others 

Eclipse Setup :
 (if u are using eclipse : Window->Preferences --> pydev --> PyUnit --> Change the Test runner to "Py.test runner".)
Right Click over the file. Run As --> Python Unit-Test
Or press Ctrl+F9:-
It will prompt you to select the test

User can use fixture or setup-teardown
setup
import pytest
def setup_module(module):


tiredown
 
import pytest
def teardown_module(module):

Allure report
pip install allure-pytest
pip list | grep -i allure


pytest --alluredir=/Users/Documents/reports
allure generate /Users/Documents/reports

setup.py
#Add this to setup.py file:
from setuptools import setup
setup(
    # ...,
    setup_requires=["pytest-runner", ...],
    tests_require=["pytest", ...],
    # ...,
)


#And create an alias into setup.cfg file:
[aliases]
test=pytest
[tool:pytest]
addopts = --verbose
python_files = testing/*/*.py


#If you now type:
python setup.py test



Links 
#https://docs.pytest.org/en/latest/goodpractices.html#goodpractices
https://docs.pytest.org/en/latest/parametrize.html
https://docs.pytest.org/en/latest/example/parametrize.html#paramexamples
https://pypi.org/project/pytest-html/
https://pypi.org/project/pytest-csv/

 important read
 https://bytes.yingw787.com/posts/2018/12/10/data_driven_testing_three/
 https://gitlab.com/qalabs/blog/pytest-parametrize-example

important watch **
https://www.youtube.com/watch?v=o9pEzgHorH0
https://www.youtube.com/watch?v=RdE-d_EhzmA

hooks
https://stackoverflow.com/questions/21930858/pytest-parameterize-row-from-csv-as-a-testcase?rq=1 

Running pytest inside eclipse ide
Ref :https://stackoverflow.com/questions/37985589/how-to-debug-or-run-pytest-scripts-using-eclipse


Friday, April 5, 2019

mac : Adding custom paths

Adding custom paths


Permanently

>nano ~/.profile
any_name="/path"
save and close
>$any_name



Temporary (only for the current session)

>alias any_Name="cd /path"
>any_Name