Table of Contents

Common Python notes

Magic methods

Methods with “dunder” or “__” like “__init__” are magic methods. Magic methods are related to OOP classes in Python and one of the biggest advantages of using Python’s magic methods is that they provide a simple way to make objects behave like built-in types. That means you can avoid ugly, counter-intuitive, and nonstandard ways of performing basic operators. like some languages will have you do:

if instance.equals(other_instance):
    # do something

You could certainly do this in Python, too, but this adds confusion and is unnecessarily verbose. Different libraries might use different names for the same operations, making the client do way more work than necessary. With the power of magic methods, however, we can define one method (__eq__, in this case), and say what we mean instead:

if instance == other_instance:
    #do something

Making classes iterable

Iterators

Iterable classes are a great way to implement the lazy loading pattern. Here all the data is not returned in one go. Instead one needs to keep calling __next__ until it raises exception when there are no more items to return.

To make a class object iterable, you need to implement “__iter__” and “__next__” (next in Python 2) method.

class Yes(collections.Iterator):

    def __init__(self, stop):
        self.x = 0
        self.stop = stop

    def __iter__(self):
        return self

    def __next__(self):
        if self.x < self.stop:
            self.x += 1
            return 'yes'
        else:
            # Iterators must raise when done, else considered broken
            raise StopIteration

    next = __next__ # Python 2 compatibility

Generators

Generators are a subtype of Iterators. In the sense that all generators are iterators but not all iterators are generators. As you can see above, iterators need you to maintain state(self.x). In most scenarios, this can be simplified with syntactic sugar so that this state management is done for you out of the box. This is exactly what generators do. If state-management is more complex, then one needs to write an iterator and generator is not possible.

A generator function is a function with yield in it. A function with yield in it is still a function, that, when called, returns an instance of a generator object. Simply put, yield gives you a generator. You’d use yield where you would normally use a return in a function. Again:

yield is only legal inside of a function definition, and the inclusion of yield in a function definition makes it return a generator.

Same thing as above using generators:

def yes(stop):
    for _ in range(stop):
        yield 'yes'

Or perhaps simpler, a Generator Expression (works similarly to list comprehensions):
* Generator Expressions are implemented using single parentheses ()

yes_expr = ('yes' for _ in range(stop))

They can all be used in the same way:

>>> stop = 4             
>>> for i, y1, y2, y3 in zip(range(stop), Yes(stop), yes(stop), 
                             ('yes' for _ in range(stop))):
...     print('{0}: {1} == {2} == {3}'.format(i, y1, y2, y3))
...     
0: yes == yes == yes
1: yes == yes == yes
2: yes == yes == yes
3: yes == yes == yes

Two EXCELLENT resources for yield/generators:
https://jeffknupp.com/blog/2013/04/07/improve-your-python-yield-and-generators-explained/

https://jeffknupp.com/blog/2018/06/04/a-common-misunderstanding-about-python-generators/

getattr

getattr(object, 'x') is completely equivalent to object.x.

There’s only in two reasons to use it:

you can’t write object.x, because you don’t know in advance which attribute you want (it comes from a string). very useful for meta-programming.
you want to provide a default value. object.y will raise an AttributeError if there’s no y. But getattr(object, 'y', 5) will return 5.

functools.partial

functools is a library dealing with higher order functions. partial is part of functools.
What functools.partial does is:

Makes a new version of a function with one or more arguments already filled in.
New version of a function documents itself.

Example:

def power(base, exponent):
    return base ** exponent

from functools import partial

square = partial(power, exponent=2)
cube = partial(power, exponent=3)

def test_partials():
    assert square(2) == 4
    assert cube(2) == 8

A great explanation to partial functions is here: https://www.pydanny.com/python-partials-are-fun.html

A great usage of partial functions is here: https://github.com/tkaemming/django-subdomains/blob/master/subdomains/utils.py#L65-L72

Context Managers

Context managers help us to manage resources efficiently so that we can specify what and howto setup or teardown when working with certain objects. For example the with open() for file read.

with open('sample.txt', 'w') as f:
    f.write('Lorem Ipsum')

# this is equivalent to:
f = open('sample.txt', 'w')
f.write('Lorem Ipsum')
f.close()

In more general terms, we need a context manager when we need to perform an action before and /or after an operation

Common scenarios:

Closing a resource after you’re done with it (file, network connection)
Perform cleanuup before/after a function call

Example – Feature Flags
We want to turn features of an application on and off easily

Uses of feature flags:
– A/B testing: we want to see whether a feature turned on generates more customer conversion vs when turned off
– Rolling eleases: we want to turn a feature on only to say 5% of our users
– Show beta version to users opted-in to Beta Testing Program

Let’s see this in a FeatureFlag class:

class FeatureFlags:
    '''
    example class which stores Feature Flags and  their state.
    we only have on feature flag which is called SHOW_BETA
    and this tells us to show beta version or not.
    Defalt, flag is True.
    I have two methods, one to show whether the flag is on or off
    and another to toggle the flag value.
    '''

    SHOW_BETA = "Show beta version of the home page"

    flags = {
        SHOW_BETA: True
    }

    @classmethod
    def is_on(cls, name):
        return cls.flags[name]
    
    @classmethod
    def toggle(cls, name, on):
        cls.flags[name] = on


# instantiate the class
feature_flag = FeatureFlags()

How do we temporarily turn features on and off when testing flags?

Want:

with feature_flag(FeatureFlags.SHOW_BETA):
    assert '/beta' == get_homepage_url()

We can do this by making FeatureFlags into a context manager by implementing __enter__ and __exit__ magic methods

class FeatureFlags:
    '''
    example class which stores Feature Flags and  their state.
    we only have on feature flag which is called SHOOW_BETA
    and this tells is to show beta version or not.
    Defalt, flag is True.
    I have two methods, one to show whether the flag is on or off
    and another to toggle tha flag value.
    '''

    SHOW_BETA = "Show beta version of the home page"

    flags = {
        SHOW_BETA: True
    }

    def __init__(self, name, on=True):
        self.name = name
        self.on = on
        self.old_value = FeatureFlags.is_on(name)

    def __enter__(self):
        FeatureFlags.toggle(self.name, self.on)

    def __exit__(self):
        FeatureFlags.toggle(self.name, self.old_value)

    @classmethod
    def is_on(cls, name):
        return cls.flags[name]
    
    @classmethod
    def toggle(cls, name, on):
        cls.flags[name] = on



# instantiate the class
feature_flag = FeatureFlags()


with feature_flag(FeatureFlags.SHOW_BETA):
    assert '/beta' == get_homepage_url()

There is an even better way to implement context managers via the @contextmanager decorator. That is going to allow us to create a factory function for with statement without having to explicitly insert the __enter__ and __exit__ methods in the FeatureFlags class.

class FeatureFlags:
    '''
    example class which stores Feature Flags and  their state.
    we only have on feature flag which is called SHOOW_BETA
    and this tells is to show beta version or not.
    Defalt, flag is True.
    I have two methods, one to show whether the flag is on or off
    and another to toggle tha flag value.
    '''

    SHOW_BETA = "Show beta version of the home page"

    flags = {
        SHOW_BETA: True
    }

    @classmethod
    def is_on(cls, name):
        return cls.flags[name]
    
    @classmethod
    def toggle(cls, name, on):
        cls.flags[name] = on


from contextlib import contextmanager

@contextmanager
def feature_flag(name, on=True):
    old_value = FeatureFlags.is_on(name)
    FeatureFlags.toggle(name, on)
    yield
    FeatureFlags.toggle(name, old_value)


with feature_flag(FeatureFlags.SHOW_BETA):
    assert '/beta' == get_homepage_url()

A very useful link: https://jeffknupp.com/blog/2016/03/07/python-with-context-managers/

Decorators

Decorators are higher order functions that allow us to modify an underlying function. By higher order function, we mean that decorator is a callable that returns a callable.
We often use decorators to:

Wrap a function in another function
Do something:
- before the call
- after the call
- with provided arguments
- modify the return value or arguments

A very simple decorator

def say_after(func):
  def wrapper(*args, **kwargs):
    func(*args, **kwargs)
    print('Nice to meet you')
  return wrapper

@say_after
def say_hello(name):
  print(f'hello {name}')

say_hello('sam')

Will create this output

hello sam
Nice to meet you

Another simple example that shows some more inner workings:

def decorator_function(func):
  def wrapper(*args, **kwargs):
    print(f'args passed to wrapper:{args}')
    print(f'Executed before function "{func.__name__}"')
    func(*args, **kwargs)
    print(f'Executed after function "{func.__name__}"')
  return wrapper

@decorator_function
def say_something(arg1:str, arg2:str) -> None:
  print(f'{arg1} {arg2}')

say_something('Hello', 'World')

Will create this output:

args passed to wrapper:('Hello', 'World')
Executed before function "say_something"
Hello World
Executed after function "say_something"

Closures

Before talking about decorators with arguments, lets take a quick look at closures

def multiply_base(num):
  def wrapper_multiplication(x):
    return x*num
  return wrapper_multiplication

multiply_by_6 = multiply_base(6) # this is a function wrapper_multiplication

result = multiply_by_6(7)

print(result)

result is 42.

Here “num” is still available inside function wrapper_multiplication because of the closure context.

Decorators that take arguments

So in decorators with arguments the concept is very similar to above closures, the outermost function is just a factory that creates a decorator function. Lets see an example for clarity

def decorator_factory(some_arg_passed):
  def decorator_function(func):
    def wrapper(*args, **kwargs):
      print(f'{some_arg_passed}:args passed to wrapper:{args}')
      print(f'Executed before function "{func.__name__}"')
      func(*args, **kwargs)
      print(f'Executed after function "{func.__name__}"')
    return wrapper
  return decorator_function

@decorator_factory('passed to decorator')
def say_something(arg1:str, arg2:str) -> None:
  print(f'{arg1} {arg2}')

say_something('Hello', 'World')

Will create this output:

passed to decorator:args passed to wrapper:('Hello', 'World')
Executed before function "say_something"
Hello World
Executed after function "say_something"

Here, note the parentheses in the @decorator_factory('passed to decorator'). In the earlier example, the decorator looked like so @decorator_function i.e. no parentheses. This is the key when using decorators with arguments. In this case, I am passing argument of ‘passed to decorator’ inside parentheses to the decorator factory. This essentially runs the decorator_factory function and returns the inner decorator_function. It is this decorator_function that will actually do the decoration for the function say_something(). The argument ‘passed to decorator’ is stored in a closure. This variable some_arg_passed that contains this argument is still accessible from this closure context from inside the wrapper function.

Decorator Class

We can also have decorators that are implemented using classes. Often times, you will see these decorators when modifying a class interface. But these are filled with gotchas and need very careful considerations. Particularly when decorators that are implemented using classes are used to decorate class methods.

Some issues with Decorators

Losing context with Decorator

Since decorators are a wrapper they cause the wrapped function to lose some context.

def decorator_func(func):
  def wrapper(*args, **kwargs):
    func(*args, **kwargs)
  return wrapper

@decorator_func
def hello_world():
  ''' docstring for hello world'''
  print('Hello world')

>>>hello_world.__name__
   'wrapper'
>>>hello_world.__doc__
   #...empty

As you can see that hello_world.__name__ points to ‘wrapper’ and the docstring associated with hello_world is gone.
To fix this, use wraps library

from contextlib import wraps

def decorator_func(func):
  @wraps(func)
  def wrapper(*args, **kwargs):
    func(*args, **kwargs)
  return wrapper

@decorator_func
def hello_world():
  ''' docstring for hello world'''
  print('Hello world')

>>>hello_world.__name__
   'hello_world'
>>>hello_world.__doc__
   ' docstring for hello world'

Common uses for decorators

logging
timing
validation
rate_limiting
mocking/patching

Context Decorators

A very cool trick in using the @contextmanager decorator for creating context manager is that the context manager can also be used as a decorator as well as a context manager inside of a with statement. This feature is available in core library as of Python 3.2. Note, this was called ContextDecorator before but the contextmanager ises ContextDecorator so the context managers it creates can be used as decorators as well as in with statements.

class FeatureFlags:
    '''
    example class which stores Feature Flags and  their state.
    we only have on feature flag which is called SHOOW_BETA
    and this tells is to show beta version or not.
    Defalt, flag is True.
    I have two methods, one to show whether the flag is on or off
    and another to toggle tha flag value.
    '''

    SHOW_BETA = "Show beta version of the home page"

    flags = {
        SHOW_BETA: True
    }

    @classmethod
    def is_on(cls, name):
        return cls.flags[name]
    
    @classmethod
    def toggle(cls, name, on):
        cls.flags[name] = on


from contextlib import contextmanager

@contextmanager
def feature_flag(name, on=True):
    old_value = FeatureFlags.is_on(name)
    FeatureFlags.toggle(name, on)
    yield
    FeatureFlags.toggle(name, old_value)

# as context manager
with feature_flag(FeatureFlags.SHOW_BETA):
    assert '/beta' == get_homepage_url()

# as decorator
@feature_flag(FeatureFlags.SHOW_BETA, on=False)
def get_profile_page():
  beta_flag_on = FeatureFlags.is_on(FeatureFlags.SHOW_BETA)
  return 'beta.html' if beta_flag_on else 'profile.html'

assert get_profile_page() != 'profile.html'

Notice the parentheses in the decorator. This tells you that this line @feature_flag(FeatureFlags.SHOW_BETA, on=False) actually executes the code until the first yield, in effect setting the flag off. After the get_profile_page() is ended, it will run the set of lines after yield, in effect, resetting the flag to where it was. This is very good if we just need to test the behavior of get_profile_page().

One more example to finally make it clear:

from contextlib import contextmanager

@contextmanager
def example_manager(message):
  print(f'Before decorated function: {message}')
  yield
  print(f'after decorated function: {message}')

with example_manager('testing') as e:
  print (f'Hello World')

@example_manager('testing')
def some_test_function():
  print (f'Hello World from test_function')


some_test_function()

This will generate the following output:

Before decorated function: testing
Hello World
after decorated function: testing

Before decorated function: testing
Hello World from test_function
after decorated function: testing

Singleton Pattern

In Python, there are two ways to implement singleton pattern.
Using modules

file: Only_instantiated_once.py

Tek Shinobi

Python notes