Python notes


Common Python notes

Magic methods

Methods with “dunder” or “__” like “__init__” are magic methods. Magic methods are related to OOP classes in Python and one of the biggest advantages of using Python’s magic methods is that they provide a simple way to make objects behave like built-in types. That means you can avoid ugly, counter-intuitive, and nonstandard ways of performing basic operators. like some languages will have you do:

You could certainly do this in Python, too, but this adds confusion and is unnecessarily verbose. Different libraries might use different names for the same operations, making the client do way more work than necessary. With the power of magic methods, however, we can define one method (__eq__, in this case), and say what we mean instead:

Making classes iterable

Iterators

Iterable classes are a great way to implement the lazy loading pattern. Here all the data is not returned in one go. Instead one needs to keep calling __next__ until it raises exception when there are no more items to return.

To make a class object iterable, you need to implement “__iter__” and “__next__” (next in Python 2) method.

Generators

Generators are a subtype of Iterators. In the sense that all generators are iterators but not all iterators are generators. As you can see above, iterators need you to maintain state(self.x). In most scenarios, this can be simplified with syntactic sugar so that this state management is done for you out of the box. This is exactly what generators do. If state-management is more complex, then one needs to write an iterator and generator is not possible.

A generator function is a function with yield in it. A function with yield in it is still a function, that, when called, returns an instance of a generator object. Simply put, yield gives you a generator. You’d use yield where you would normally use a return in a function. Again:

yield is only legal inside of a function definition, and the inclusion of yield in a function definition makes it return a generator.

Same thing as above using generators:

Or perhaps simpler, a Generator Expression (works similarly to list comprehensions):
* Generator Expressions are implemented using single parentheses ()

They can all be used in the same way:

Two EXCELLENT resources for yield/generators:
https://jeffknupp.com/blog/2013/04/07/improve-your-python-yield-and-generators-explained/

https://jeffknupp.com/blog/2018/06/04/a-common-misunderstanding-about-python-generators/

getattr

getattr(object, 'x') is completely equivalent to object.x.

There’s only in two reasons to use it:

  1. you can’t write object.x, because you don’t know in advance which attribute you want (it comes from a string). very useful for meta-programming.
  2. you want to provide a default value. object.y will raise an AttributeError if there’s no y. But getattr(object, 'y', 5) will return 5.

functools.partial

functools is a library dealing with higher order functions. partial is part of functools.
What functools.partial does is:

  • Makes a new version of a function with one or more arguments already filled in.
  • New version of a function documents itself.

Example:

A great explanation to partial functions is here: https://www.pydanny.com/python-partials-are-fun.html

A great usage of partial functions is here: https://github.com/tkaemming/django-subdomains/blob/master/subdomains/utils.py#L65-L72

Context Managers

Context managers help us to manage resources efficiently so that we can specify what and howto setup or teardown when working with certain objects. For example the with open() for file read.

In more general terms, we need a context manager when we need to perform an action before and /or after an operation

Common scenarios:

  • Closing a resource after you’re done with it (file, network connection)
  • Perform cleanuup before/after a function call

Example – Feature Flags
We want to turn features of an application on and off easily

Uses of feature flags:
– A/B testing: we want to see whether a feature turned on generates more customer conversion vs when turned off
– Rolling eleases: we want to turn a feature on only to say 5% of our users
– Show beta version to users opted-in to Beta Testing Program

Let’s see this in a FeatureFlag class:

How do we temporarily turn features on and off when testing flags?

Want:

We can do this by making FeatureFlags into a context manager by implementing __enter__ and __exit__ magic methods

There is an even better way to implement context managers via the @contextmanager decorator. That is going to allow us to create a factory function for with statement without having to explicitly insert the __enter__ and __exit__ methods in the FeatureFlags class.

A very useful link: https://jeffknupp.com/blog/2016/03/07/python-with-context-managers/

Decorators

Decorators are higher order functions that allow us to modify an underlying function. By higher order function, we mean that decorator is a callable that returns a callable.
We often use decorators to:

  • Wrap a function in another function
  • Do something:
    • before the call
    • after the call
    • with provided arguments
    • modify the return value or arguments

A very simple decorator

Will create this output

Another simple example that shows some more inner workings:

Will create this output:

Closures

Before talking about decorators with arguments, lets take a quick look at closures

result is 42.

Here “num” is still available inside function wrapper_multiplication because of the closure context.

Decorators that take arguments

So in decorators with arguments the concept is very similar to above closures, the outermost function is just a factory that creates a decorator function. Lets see an example for clarity

Will create this output:

Here, note the parentheses in the @decorator_factory('passed to decorator'). In the earlier example, the decorator looked like so @decorator_function i.e. no parentheses. This is the key when using decorators with arguments. In this case, I am passing argument of ‘passed to decorator’ inside parentheses to the decorator factory. This essentially runs the decorator_factory function and returns the inner decorator_function. It is this decorator_function that will actually do the decoration for the function say_something(). The argument ‘passed to decorator’ is stored in a closure. This variable some_arg_passed that contains this argument is still accessible from this closure context from inside the wrapper function.

Decorator Class

We can also have decorators that are implemented using classes. Often times, you will see these decorators when modifying a class interface. But these are filled with gotchas and need very careful considerations. Particularly when decorators that are implemented using classes are used to decorate class methods.

Some issues with Decorators

Losing context with Decorator

Since decorators are a wrapper they cause the wrapped function to lose some context.

As you can see that hello_world.__name__ points to ‘wrapper’ and the docstring associated with hello_world is gone.
To fix this, use wraps library

Common uses for decorators

  • logging
  • timing
  • validation
  • rate_limiting
  • mocking/patching

Context Decorators

A very cool trick in using the @contextmanager decorator for creating context manager is that the context manager can also be used as a decorator as well as a context manager inside of a with statement. This feature is available in core library as of Python 3.2. Note, this was called ContextDecorator before but the contextmanager ises ContextDecorator so the context managers it creates can be used as decorators as well as in with statements.

Notice the parentheses in the decorator. This tells you that this line @feature_flag(FeatureFlags.SHOW_BETA, on=False) actually executes the code until the first yield, in effect setting the flag off. After the get_profile_page() is ended, it will run the set of lines after yield, in effect, resetting the flag to where it was. This is very good if we just need to test the behavior of get_profile_page().

One more example to finally make it clear:

This will generate the following output:

Singleton Pattern

In Python, there are two ways to implement singleton pattern.
Using modules

file: Only_instantiated_once.py