Friday, January 31, 2020

scala : Factory design / Create objects using string

Factory design / Create objects using string



object testing extends App{
val jmap:Map[String,dime] = Map("x" -> new curr)
println(jmap("x").dimensionName)
}

class dime{
val dimensionName: Option[String] = None
}

class curr extends dime{
override val dimensionName: Option[String] = Some("curr")
val y=3432
}
class currex extends dime{
override val dimensionName: Option[String] = Some("currex")
}

Thursday, January 30, 2020

Scala : Provide column names as Seq to select statement in Scala

Scala : Provide column names as Seq to select statement in Scala 



Sample Code :

val df1: DataFrame = Seq((1,1,"kjj"), (2,2,"987987")).toDF("col1", "col2","col3")
val columnName = Seq("col1", "col3")
val DFFiltered = df1.select(columnName.map(name => col(name)): _*)
DFFiltered.show()

Full Code :
project sbt version : 2.1
------------------------build.sbt------------------------------

name := "scala_sample"
version := "0.1"
scalaVersion := "2.11.12"
libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % "2.1.0",
  "org.apache.spark" %% "spark-sql" % "2.1.0")
------------------------testing.scala------------------------------

import org.apache.spark.sql.{DataFrame,SparkSession}
import org.apache.spark.sql.functions._

object testing extends App{
  val spark = SparkSession.builder().master("local").appName("spark-shell").getOrCreate()
  import spark.implicits._
  val df1: DataFrame = Seq((1,1,"kjj"), (2,2,"987987")).toDF("col1", "col2","col3")
  val seq:Seq[String]=Seq("col1","col3")
  val obj:DF=new DF()
  obj.selectCols(df1,seq).show()
}


class DF{
  def selectCols(dataFrame: DataFrame,seq: Seq[String]): DataFrame ={
    val DFFiltered = dataFrame.select(seq.map(name => col(name)): _*)
    DFFiltered
  }
}

Result:
+----+------+
|col1|  col3|
+----+------+
|   1|   kjj|
|   2|987987|
+----+------+

Anti Pattern / Code Smells


Anti-patterns/Design smell

"Design smells are certain structures in the design that indicate violation of fundamental design principles and negatively impact design quality”. 

Tips :
  • Functions should be as small as possible.
  • Avoid if else / Case statements instead used Class and inheritance (Open Closed principle).
  • The number arguments sent to function should be not more than 2 . If exceeds either create a class and send as an object or send as a map.
  • Never send boolean into a function ( as usually it contain if else to validate the same).


There are many design smells classified based on violating design principles:

Abstraction smells

  1. Missing Abstraction: When clumps of data or encoded strings are used instead of creating a class or an interface.
  2. Imperative Abstraction:  When an operation is turned into a class.
  3. Incomplete Abstraction:When an abstraction does not support complementary or interrelated methods completely.
  4. Multifaceted Abstraction: When an abstraction has more than one responsibility assigned to it.
  5. Unnecessary Abstraction: When an abstraction that is actually not needed .
  6. Unutilized Abstraction: When an abstraction is left unused (either not directly used or not reachable).
  7. Duplicate Abstraction: When two or more abstractions have identical names or identical implementation or both.

Encapsulation smells

  1. Deficient Encapsulation: When the declared accessibility of one or more members of an abstraction is more permissive than actually required.
  2. Leaky Encapsulation: When an abstraction “exposes” or “leaks” implementation details through its public interface.
  3. Missing Encapsulation: When implementation variations are not encapsulated within an abstraction or hierarchy.
  4. Unexploited Encapsulation: When client code uses explicit type checks (using chained if-else or switch statements that check for the type of the object) instead of exploiting the variation in types already encapsulated within a hierarchy.

Modularisation smells

  1. Broken Modularisation: When data and/or methods that ideally should have been localized into a single abstraction are separated and spread across multiple abstractions.
  2. Insufficient Modularisation: When an abstraction exists that has not been completely decomposed, and a further decomposition could reduce its size, implementation complexity, or both.
  3. Cyclically-Dependent Modularisation: When two or more abstractions depend on each other directly or indirectly (creating a tight coupling between the abstractions).
  4. Hub-Like Modularisation: When an abstraction has dependencies (both incoming and outgoing) with a large number of other abstractions.

Hierarchy smells

  1. Missing Hierarchy: When a code segment uses conditional logic (typically in conjunction with “tagged types”) to explicitly manage variation in behavior where a hierarchy could have been created and used to encapsulate those variations.
  2. Unnecessary Hierarchy: When the whole inheritance hierarchy is unnecessary, indicating that inheritance has been applied needlessly for the particular design context.
  3. Unfactored Hierarchy: This smell arises when there is unnecessary duplication among types in a hierarchy.
  4. Wide Hierarchy: When an inheritance hierarchy is “too” wide indicating that intermediate types may be missing.
  5. Speculative Hierarchy: When one or more types in a hierarchy are provided speculatively (i.e., based on imagined needs rather than real requirements).
  6. Deep Hierarchy: When an inheritance hierarchy is “excessively” deep.
  7. Rebellious Hierarchy: When a subtype rejects the methods provided by its supertype(s).
  8. Broken Hierarchy:When a supertype and its subtype conceptually do not share an “IS- A” relationship resulting in broken substitutability.
  9. Multipath Hierarchy: When a subtype inherits both directly as well as indirectly from a supertype leading to unnecessary inheritance paths in the hierarchy.
  10. Cyclic Hierarchy: When a supertype in a hierarchy depends on any of its subtypes.

Other Code Smells:

  1. Duplicated Code and Logic Code Smell:refractor part into a separate method 
  2. Long Methods and Classes Code Smell:The perfect method should be between 4 to 20 lines.
  3. Duplicated Methods in the Same or Different Class Code Smell:when you have two methods that do the same functionality. 
  4. Class Divergent Change Code Smell: Not following single responsibility principle
  5. Shotgun Surgery Code Smell: For example, you need to create a new user rule such as ‘Supper-Admin’ then you found yourself must edit some methods in Profile, Products and Employees classes. In that case, consider grouping these methods in one single class so this new class will have a single reason to change.
  6. Feature Envy Code Smell:A method in your class that extensively makes use of another class , then refractor that method into a different class.
  7. Data Clumps Code Smell:A list of functions accepts same group of large parameters.In that case its better to create a class which accepts parameters and send an object instead of parameter group.
  8. Switch Statement Code Smell: Case statements should not have lot of statements instead delegate call other methods/functions or return factory method
  9. Temporary Fields Code Smell:When you have a class instance variables that have been used only sometimes. 
  10. Message Chains Code Smell:When you have a class that uses another class which uses another class and so on.
  11. (in scala : Make your code cleaner by shortening the chain to Employee->Config
  12. Middle Man Code Smell:Sometimes you find many methods in one class do nothing but delegating to another method in a different class.t would be better to get rid of it.
  13. Alternative Classes with Different Interfaces Code Smell:two different classes are created which do the same thing 
  14. Incomplete Library Class Code Smell:Sometimes when you need to retrieve all documents of a particular user? Remember that it is horrible if you tried to edit third-party classes on your own. In this case, you need to extend the functionality of the Document class without editing the original class. Well, the decorator design pattern can be helpful here
  15. Comments Code Smell : Comments is a code smell
  16. Speculative Generality Code Smell : 
    1. * Don’t over plan your code.
    2. * Don’t try to cover a case that likely has 1% chance to happen in the future.
    3. * Sacrifice some speed in order to make your algorithm simpler, especially if you don’t need a real-time result from your application.
    4. * Optimise for speed when your application is actually slow not when you only have 100 users.

Ref :Refactoring for Software Design Smells: Managing Technical Debt  by Girish Suryanarayana

Sunday, January 19, 2020

Scala vs Python

Scala vs Python

----------------VAR / CONST ------------
val a:String="123" // Scala
var a:String="123" // Scala

a="123"# Python
--------------COMMENTS------------------
// this is a commented line in Scala
/* and this is a multiline comment,
...just choose! */

#Hi # Python
""" # Python
multi
"""
or
''' # Python
multi Line
'''
---------------PRINT-----------------
print("whatever you want") #  Python
println("whatever you want") // Scala

a="hi"
print(f"{a}")

val a="hi"
println(s"$a")

---------------LIST/ARRAY-----------------
Declaration :
var myArray = Array(2, 5, "apple", 78) // Scala
my_list = [2, 5, 'apple', 78] # Python

Indexing
myArray(1) // Scala 
my_list[1] # Python 

Slicing
myArray.slice(0,3) // Scala
my_list[0:3] # Python 


myArray.last    // Scala last element
myArray.max     // Scala maximum element
myArray.min     // Scala minimum element

my_list[-1]    # Python last element
max(my_list)   # Python maximum element
min(my_list)   # Python minimum element

sum(my_list) # Python- sum

myArray.sum    // scala sum
myArray.product  // multiplying elements in array

my_list.append('last words') # Python
myArray :+= "last words" 

---------------LOOP-----------------
for i in my_list:print(i)# Python
for (i <- myArray){ println(i)}// Scala
val list = List(1, 2, 3)
list.foreach(x => println(x)) 


--------------MAP------------------
list(map(lambda x:x*3,my_list)) # map in Python
myArray.map(i => i*3) // map in Scala

---------------FILTER-----------------
list(filter(lambda x:x%3==0,my_list)# filter in Python
myArray.filter(i => i%3 == 0) // filter in Scala

---------------REDUCE-----------------
from functools import reduce # Python
print(reduce(lambdax,y:x+y,(1,2,3))) #Python

val lst = List(1, 2, 3) // Scala
list.reduce((x, y) => x + y) // => 6 // Scala

---------------LIST COMPREHENSION-----------------
[i*3 for i in my_list if i%2 == 0] # list comprehension
myArray.filter(i => i%2 == 0).map(i => i*3) // Scala

---------------SORT-----------------------
sortBy(f: T => K): Seq[T]

---------------DICTIONARY-----------------
Declaration:
my_dict = {'first_name': 'Emma','age': 18}# Python
var myMap = (("firstName", "Emma"),("age", 18)) // Scala

New key:
my_dict['country_of_origin'] = 'Italy' # Python
myMap += ("countryOfOrigin" -> "Italy") // Scala

Access:
my_dict['first_name'] # Python
myMap("firstName") // Scala

Loop:
for key, value in my_dict.items(): # Python
    print(key)
    print(value)

for ((key, value) <- myMap){ // Scala
    println(key)
    println(value)
}
--------------TUPLE------------------
my_tup = (1, 2, 3) # Python
my_tup[0] # Python


myTup = (1, 2, 3) // Scala
myTup._1 // Scala
// the indexing is way different than arrays!

-------------SET-------------------
my_set = {1, 3, 5, 1} # Python
mySet = Set(1, 3, 5, 1) // Scala

-------------FUNCTION-------------------
def chop_string(input_string): # Python
    return input_string[0:5]

def chopString(inputString: String): String = { // Scala
    inputString.slice(0, 5)
}
--------------NAMED TUPLE/CASE CLASS------
from collections import namedtuple
n=namedtuple("n",["a","b"])
n2=n(10,20)
print(n2.a)

case class n(a:String,b:Int)
val n2: n = n("name", 1)
println(n2.a)
----------------SORTED 1---------------------
List(10, 5, 8, 1, 7).sortWith(_ < _) // Scala asc
sorted([10, 5, 8, 1, 7] , reverse =false) # Python

----------------SORTED 2---------------------
val list = List(('b',30),('c',10),('a',20))// Scala
println(list.sortBy(_._2))

l=[[1,2],[2,1]] # Python
sorted(l,key=lambdax:x[1])

----------------SORTED 3---------------------
List("banana", "pear", "apple", "orange").sortWith(_.length < _.length) // Scala
sorted(["banana", "pear", "apple", "orange"],key=lambda x:len(x)) # Python

output:(pear, apple, banana, orange)

-----------------Switch / Case ----------------
val result =1
result match {
case -1 =>{println("-1")}
case 0 => { println("asd")}
case x:Int if(x>0) =>
{ println(x)}
}

------------------OPTION = Some/getorelse/match---------------------
To eliminate NullPointerExceptions.

val age:Option[Integer] = None //Some(28)
val x=age.getOrElse(throw new Exception("error"))