Python - Advanced Usage

Advanced Usage of Python

Python is renowned for its simplicity and readability, but it is also a powerful and expressive language capable of supporting advanced programming paradigms and techniques. This guide delves into advanced usage in Python, covering essential concepts such as decorators, context managers, metaclasses, generators, coroutines, concurrency, type hints, and more.

1. Advanced Function Usage

1.1 First-Class Functions

In Python, functions are first-class citizens. This means functions can be passed around as arguments, returned from other functions, and assigned to variables.

def greet(name):
    return f"Hello, {name}!"

def execute(func, name):
    return func(name)

print(execute(greet, "Alice"))

1.2 Closures

A closure is a function object that remembers values in enclosing scopes even if they are not present in memory.

def outer(msg):
    def inner():
        print(msg)
    return inner

func = outer("Welcome to Python")
func()

1.3 Decorators

Decorators allow you to modify the behavior of a function or class. They are often used in logging, access control, memoization, and more.

def decorator_function(original_function):
    def wrapper_function(*args, **kwargs):
        print(f"Wrapper executed before {original_function.__name__}")
        return original_function(*args, **kwargs)
    return wrapper_function

@decorator_function
def display():
    print("Display function ran")

display()

2. Generators and Iterators

2.1 Generators

Generators are functions that yield items instead of returning them all at once. They are memory efficient and excellent for working with large data streams.

def countdown(n):
    while n > 0:
        yield n
        n -= 1

for number in countdown(5):
    print(number)

2.2 Generator Expressions

Like list comprehensions but more memory efficient.

squares = (x*x for x in range(10))
for num in squares:
    print(num)

2.3 The `iter` and `next` Functions

These are used to manually control iteration over objects.

my_list = [1, 2, 3]
it = iter(my_list)
print(next(it))  # 1
print(next(it))  # 2

3. Context Managers and the `with` Statement

3.1 Using `with` for Resource Management

Context managers handle resource setup and teardown, like closing files or releasing locks.

with open("sample.txt", "w") as f:
    f.write("Hello, context managers!")

3.2 Creating Custom Context Managers

class MyContext:
    def __enter__(self):
        print("Entering context")
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        print("Exiting context")

with MyContext():
    print("Inside context")

4. Advanced OOP Concepts

4.1 Dunder (Magic) Methods

Python allows operator overloading and custom behaviors via magic methods like __add__, __str__, __len__.

class Vector:
    def __init__(self, x, y):
        self.x = x
        self.y = y
    
    def __add__(self, other):
        return Vector(self.x + other.x, self.y + other.y)
    
    def __str__(self):
        return f"Vector({self.x}, {self.y})"

v1 = Vector(2, 3)
v2 = Vector(1, 4)
print(v1 + v2)

4.2 Classmethods and Staticmethods

class MyClass:
    @staticmethod
    def static_hello():
        return "Hello from static method"

    @classmethod
    def class_hello(cls):
        return f"Hello from {cls.__name__}"

print(MyClass.static_hello())
print(MyClass.class_hello())

4.3 Metaclasses

Metaclasses are classes of classes. They define how classes behave.

class Meta(type):
    def __new__(cls, name, bases, dct):
        print(f"Creating class {name}")
        return super().__new__(cls, name, bases, dct)

class MyClass(metaclass=Meta):
    pass

5. Functional Programming in Python

5.1 `map`, `filter`, `reduce`

from functools import reduce

numbers = [1, 2, 3, 4, 5]
squared = list(map(lambda x: x*x, numbers))
even = list(filter(lambda x: x % 2 == 0, numbers))
summed = reduce(lambda x, y: x + y, numbers)

5.2 `lambda` Functions

Anonymous functions used in higher-order function calls.

double = lambda x: x * 2
print(double(10))

5.3 Comprehensions

List, dict, and set comprehensions offer a concise way to construct collections.

squares = [x*x for x in range(10)]
evens = {x for x in range(20) if x % 2 == 0}
square_dict = {x: x*x for x in range(5)}

6. Concurrency and Parallelism

6.1 Multithreading

Useful for I/O-bound tasks.

import threading

def print_numbers():
    for i in range(5):
        print(i)

t = threading.Thread(target=print_numbers)
t.start()
t.join()

6.2 Multiprocessing

Used for CPU-bound tasks to bypass the Global Interpreter Lock (GIL).

from multiprocessing import Process

def square(n):
    print(n*n)

p = Process(target=square, args=(5,))
p.start()
p.join()

6.3 Asyncio

For asynchronous programming and concurrency.

import asyncio

async def greet():
    print("Hello")
    await asyncio.sleep(1)
    print("World")

asyncio.run(greet())

7. Type Hinting and Annotations

7.1 Basic Type Hints

def add(x: int, y: int) -> int:
    return x + y

7.2 Complex Type Hints

from typing import List, Dict, Union

def process(data: List[Dict[str, Union[int, str]]]):
    for item in data:
        print(item)

7.3 Type Checking Tools

Tools like `mypy` and `pyright` help enforce static type checking.

8. Introspection and Reflection

8.1 Using `dir`, `type`, and `getattr`

Python allows runtime inspection of objects.

class Person:
    def __init__(self, name):
        self.name = name

p = Person("Alice")
print(type(p))
print(dir(p))
print(getattr(p, "name"))

8.2 `inspect` Module

import inspect

def foo():
    pass

print(inspect.getsource(foo))

9. Modules and Packages

9.1 Importing Strategies

Understand absolute vs relative imports and how to manage packages.

from mypackage.module import func  # Absolute
from .module import func          # Relative

9.2 Creating Packages

Use `__init__.py` to make a folder a package and manage imports efficiently.

9.3 The `__main__` Guard

if __name__ == "__main__":
    print("Running as a script")

10. Advanced Exception Handling

10.1 Custom Exceptions

class MyError(Exception):
    pass

raise MyError("Something went wrong")

10.2 `try` with `else` and `finally`

try:
    print("Try block")
except ValueError:
    print("Exception block")
else:
    print("Else block")
finally:
    print("Finally block")

11. Memory Management and Performance

11.1 Generators vs Lists

Generators are lazy and use less memory.

11.2 `__slots__`

Used to reduce memory usage in classes.

class MyClass:
    __slots__ = ['name', 'age']

    def __init__(self, name, age):
        self.name = name
        self.age = age

11.3 Garbage Collection

Handled by Python automatically, but can be fine-tuned with the `gc` module.

Advanced Python usage enables developers to write more concise, readable, and efficient code. By mastering topics like decorators, generators, context managers, metaclasses, and concurrency, one can fully leverage the capabilities of the Python language for complex and large-scale software systems. Whether you're building web applications, data pipelines, or system automation tools, these advanced concepts are crucial in crafting robust and scalable solutions.