backend / python oop / 08_descriptors_in_python.md

Python Descriptors: Complete Interview Guide

4 interview angles 10 min read source

Python Descriptors: Complete Interview Guide

Table of Contents

  1. Introduction to Descriptors
  2. Descriptor Protocol
  3. Types of Descriptors
  4. Built-in Descriptors
  5. Custom Descriptors
  6. Real-world Examples
  7. Best Practices
  8. Common Interview Questions
  9. Advanced Topics

Introduction to Descriptors

What are Descriptors?

Descriptors are Python objects that define how attribute access is handled. They are a fundamental mechanism that powers many of Python’s built-in features like properties, methods, static methods, and class methods.

Key Points:

  • Descriptors are objects that define __get__, __set__, or __delete__ methods
  • They control how attributes are accessed, set, or deleted
  • They enable computed attributes, validation, and other advanced behaviors
  • They are the foundation for properties, methods, and other Python features

Why Use Descriptors?

  1. Computed Properties: Create attributes that are calculated on-the-fly
  2. Validation: Ensure data integrity when setting attributes
  3. Lazy Loading: Defer expensive operations until needed
  4. Caching: Store computed values for performance
  5. Logging/Tracking: Monitor attribute access and changes

Descriptor Protocol

The descriptor protocol consists of three methods:

class Descriptor:
    def __get__(self, obj, objtype=None):
        """Called when attribute is accessed"""
        pass
    
    def __set__(self, obj, value):
        """Called when attribute is set"""
        pass
    
    def __delete__(self, obj):
        """Called when attribute is deleted"""
        pass

Method Signatures

  • __get__(self, obj, objtype=None): Called for attribute access
    • obj: The instance (None if accessed on class)
    • objtype: The class of the instance
  • __set__(self, obj, value): Called for attribute assignment
  • __delete__(self, obj): Called for attribute deletion

Types of Descriptors

1. Data Descriptors

Data descriptors define both __get__ and __set__ methods. They have higher priority than instance dictionaries.

class DataDescriptor:
    def __init__(self, name):
        self.name = name
    
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return f"Data descriptor: {obj.__dict__.get(self.name, 'Not set')}"
    
    def __set__(self, obj, value):
        obj.__dict__[self.name] = value

class MyClass:
    attr = DataDescriptor('attr')

# Usage
obj = MyClass()
obj.attr = "Hello"  # Calls __set__
print(obj.attr)     # Calls __get__

2. Non-Data Descriptors

Non-data descriptors only define __get__. They have lower priority than instance dictionaries.

class NonDataDescriptor:
    def __init__(self, name):
        self.name = name
    
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return f"Non-data descriptor: {obj.__dict__.get(self.name, 'Not set')}"

class MyClass:
    attr = NonDataDescriptor('attr')

# Usage
obj = MyClass()
obj.attr = "Hello"  # Direct assignment to instance dict
print(obj.attr)     # Still calls __get__ (descriptor wins)

Built-in Descriptors

Properties

Properties are the most common use of descriptors:

class Person:
    def __init__(self, first_name, last_name):
        self._first_name = first_name
        self._last_name = last_name
    
    @property
    def full_name(self):
        """Computed property"""
        return f"{self._first_name} {self._last_name}"
    
    @property
    def age(self):
        """Read-only property"""
        return self._age
    
    @age.setter
    def age(self, value):
        """Setter with validation"""
        if value < 0:
            raise ValueError("Age cannot be negative")
        self._age = value

# Usage
person = Person("John", "Doe")
person.age = 30
print(person.full_name)  # "John Doe"
print(person.age)        # 30

Methods

Instance methods are descriptors:

class MyClass:
    def instance_method(self):
        return "Instance method"
    
    @classmethod
    def class_method(cls):
        return "Class method"
    
    @staticmethod
    def static_method():
        return "Static method"

# All of these are descriptors
print(MyClass.instance_method)  # <function MyClass.instance_method>
print(MyClass.class_method)     # <bound method MyClass.class_method>
print(MyClass.static_method)    # <function MyClass.static_method>

Custom Descriptors

1. Validated Attribute Descriptor

class ValidatedAttribute:
    def __init__(self, min_value=None, max_value=None, allowed_types=None):
        self.min_value = min_value
        self.max_value = max_value
        self.allowed_types = allowed_types
        self.name = None
    
    def __set_name__(self, owner, name):
        """Called when descriptor is assigned to a class"""
        self.name = name
    
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return obj.__dict__.get(self.name)
    
    def __set__(self, obj, value):
        # Type validation
        if self.allowed_types and not isinstance(value, self.allowed_types):
            raise TypeError(f"{self.name} must be one of {self.allowed_types}")
        
        # Value validation
        if self.min_value is not None and value < self.min_value:
            raise ValueError(f"{self.name} must be >= {self.min_value}")
        
        if self.max_value is not None and value > self.max_value:
            raise ValueError(f"{self.name} must be <= {self.max_value}")
        
        obj.__dict__[self.name] = value

class Product:
    price = ValidatedAttribute(min_value=0, allowed_types=(int, float))
    name = ValidatedAttribute(allowed_types=(str,))
    stock = ValidatedAttribute(min_value=0, max_value=1000, allowed_types=(int,))

# Usage
product = Product()
product.price = 29.99  # Valid
product.name = "Laptop"  # Valid
product.stock = 50      # Valid

# These would raise exceptions:
# product.price = -10    # ValueError
# product.name = 123     # TypeError
# product.stock = 2000   # ValueError

2. Cached Property Descriptor

class CachedProperty:
    def __init__(self, func):
        self.func = func
        self.name = func.__name__
    
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        
        # Check if value is already cached
        if self.name not in obj.__dict__:
            obj.__dict__[self.name] = self.func(obj)
        
        return obj.__dict__[self.name]

class ExpensiveCalculation:
    def __init__(self, data):
        self.data = data
    
    @CachedProperty
    def expensive_result(self):
        """This expensive calculation will only run once"""
        print("Performing expensive calculation...")
        return sum(x * x for x in self.data)

# Usage
calc = ExpensiveCalculation([1, 2, 3, 4, 5])
print(calc.expensive_result)  # Performs calculation
print(calc.expensive_result)  # Uses cached result

3. Lazy Loading Descriptor

class LazyProperty:
    def __init__(self, func):
        self.func = func
        self.name = func.__name__
    
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        
        # Create the attribute if it doesn't exist
        if self.name not in obj.__dict__:
            obj.__dict__[self.name] = self.func(obj)
        
        return obj.__dict__[self.name]

class DatabaseConnection:
    def __init__(self, connection_string):
        self.connection_string = connection_string
        self._connection = None
    
    @LazyProperty
    def connection(self):
        """Database connection is only created when first accessed"""
        print("Creating database connection...")
        # Simulate database connection
        return f"Connected to {self.connection_string}"

# Usage
db = DatabaseConnection("postgresql://localhost/mydb")
# Connection not created yet
print("Connection not created yet")
print(db.connection)  # Now connection is created
print(db.connection)  # Uses existing connection

Real-world Examples

1. Django Model Fields

Django uses descriptors extensively for model fields:

class CharField:
    def __init__(self, max_length=None, null=False, blank=False):
        self.max_length = max_length
        self.null = null
        self.blank = blank
        self.name = None
    
    def __set_name__(self, owner, name):
        self.name = name
    
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return obj.__dict__.get(self.name)
    
    def __set__(self, obj, value):
        if value is None and not self.null:
            raise ValueError(f"{self.name} cannot be null")
        
        if value == "" and not self.blank:
            raise ValueError(f"{self.name} cannot be blank")
        
        if self.max_length and len(str(value)) > self.max_length:
            raise ValueError(f"{self.name} too long")
        
        obj.__dict__[self.name] = value

class User:
    username = CharField(max_length=50, null=False, blank=False)
    email = CharField(max_length=100, null=False, blank=False)
    bio = CharField(max_length=500, null=True, blank=True)

2. Configuration Management

class ConfigValue:
    def __init__(self, default=None, required=False, validator=None):
        self.default = default
        self.required = required
        self.validator = validator
        self.name = None
    
    def __set_name__(self, owner, name):
        self.name = name
    
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        
        value = obj.__dict__.get(self.name, self.default)
        
        if value is None and self.required:
            raise ValueError(f"Required config value {self.name} not set")
        
        return value
    
    def __set__(self, obj, value):
        if self.validator:
            value = self.validator(value)
        obj.__dict__[self.name] = value

class AppConfig:
    database_url = ConfigValue(required=True)
    debug = ConfigValue(default=False, validator=bool)
    port = ConfigValue(default=8000, validator=int)
    api_key = ConfigValue(required=True)

Best Practices

1. Use __set_name__ for Automatic Naming

class Descriptor:
    def __set_name__(self, owner, name):
        """Automatically called when descriptor is assigned to a class"""
        self.name = name
        print(f"Descriptor {name} assigned to {owner.__name__}")

class MyClass:
    attr = Descriptor()  # __set_name__ is called automatically

2. Handle Class Access Properly

class Descriptor:
    def __get__(self, obj, objtype=None):
        if obj is None:
            # Accessed on class, return descriptor itself
            return self
        # Accessed on instance, return value
        return obj.__dict__.get(self.name)

3. Use Descriptors for Computed Properties

class ComputedProperty:
    def __init__(self, func):
        self.func = func
        self.name = func.__name__
    
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return self.func(obj)

class Rectangle:
    def __init__(self, width, height):
        self.width = width
        self.height = height
    
    @ComputedProperty
    def area(self):
        return self.width * self.height
    
    @ComputedProperty
    def perimeter(self):
        return 2 * (self.width + self.height)

4. Implement Proper Error Handling

class SafeDescriptor:
    def __get__(self, obj, objtype=None):
        try:
            if obj is None:
                return self
            return obj.__dict__.get(self.name)
        except Exception as e:
            raise AttributeError(f"Error accessing {self.name}: {e}")

Common Interview Questions

Q1: What are descriptors and how do they work?

Descriptors are Python objects that define how attribute access is handled through the descriptor protocol (__get__, __set__, __delete__). They control how attributes are accessed, set, or deleted and are the foundation for properties, methods, and other Python features.

Key Points:

  • They define the behavior of attribute access
  • They can be data descriptors (with __set__) or non-data descriptors (only __get__)
  • Data descriptors have higher priority than instance dictionaries
  • They enable computed properties, validation, and other advanced behaviors

Q2: What’s the difference between data and non-data descriptors?

  • Data descriptors: Define both __get__ and __set__ methods. They have higher priority than instance dictionaries and always control attribute access.
  • Non-data descriptors: Only define __get__ method. They have lower priority than instance dictionaries and can be overridden by direct assignment to the instance.

Q3: How do properties work internally?

Properties are implemented using descriptors. The @property decorator creates a descriptor object that:

  • Uses __get__ to call the getter method
  • Uses __set__ to call the setter method (if defined)
  • Uses __delete__ to call the deleter method (if defined)

Q4: When would you use a custom descriptor?

Custom descriptors are useful for:

  • Validation: Ensuring data integrity when setting attributes
  • Computed properties: Creating attributes that are calculated on-the-fly
  • Lazy loading: Deferring expensive operations until needed
  • Caching: Storing computed values for performance
  • Logging/tracking: Monitoring attribute access and changes
  • Configuration management: Managing application settings with validation

Q5: How do you implement a read-only descriptor?

class ReadOnlyDescriptor:
    def __init__(self, value):
        self.value = value
    
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return self.value
    
    def __set__(self, obj, value):
        raise AttributeError("Cannot modify read-only attribute")
    
    def __delete__(self, obj):
        raise AttributeError("Cannot delete read-only attribute")

class MyClass:
    readonly_attr = ReadOnlyDescriptor("initial value")

Q6: What’s the __set_name__ method used for?

The __set_name__ method is automatically called when a descriptor is assigned to a class. It receives the owner class and the attribute name, allowing the descriptor to know what name it was assigned to. This is useful for:

  • Storing the attribute name for later use
  • Setting up internal state based on the attribute name
  • Avoiding the need to manually specify the attribute name

Q7: How do descriptors relate to Python’s method resolution order (MRO)?

Descriptors are part of Python’s attribute lookup mechanism. When accessing an attribute, Python follows this order:

  1. Data descriptors on the class
  2. Instance dictionary
  3. Non-data descriptors on the class
  4. Class dictionary
  5. Parent classes (following MRO)

Q8: Can you give an example of a practical use case for descriptors?

class TypedAttribute:
    def __init__(self, expected_type):
        self.expected_type = expected_type
        self.name = None
    
    def __set_name__(self, owner, name):
        self.name = name
    
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return obj.__dict__.get(self.name)
    
    def __set__(self, obj, value):
        if not isinstance(value, self.expected_type):
            raise TypeError(f"{self.name} must be {self.expected_type}")
        obj.__dict__[self.name] = value

class Person:
    name = TypedAttribute(str)
    age = TypedAttribute(int)
    height = TypedAttribute(float)

Advanced Topics

1. Descriptor Chaining

class LoggingDescriptor:
    def __init__(self, descriptor):
        self.descriptor = descriptor
    
    def __get__(self, obj, objtype=None):
        print(f"Getting {self.descriptor.name}")
        return self.descriptor.__get__(obj, objtype)
    
    def __set__(self, obj, value):
        print(f"Setting {self.descriptor.name} to {value}")
        return self.descriptor.__set__(obj, value)

class ValidatedAttribute:
    def __init__(self, min_value=None, max_value=None):
        self.min_value = min_value
        self.max_value = max_value
        self.name = None
    
    def __set_name__(self, owner, name):
        self.name = name
    
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return obj.__dict__.get(self.name)
    
    def __set__(self, obj, value):
        if self.min_value is not None and value < self.min_value:
            raise ValueError(f"{self.name} must be >= {self.min_value}")
        if self.max_value is not None and value > self.max_value:
            raise ValueError(f"{self.name} must be <= {self.max_value}")
        obj.__dict__[self.name] = value

class MyClass:
    # Chain descriptors together
    value = LoggingDescriptor(ValidatedAttribute(min_value=0, max_value=100))

2. Descriptor Factories

def typed_attribute(expected_type, default=None):
    """Factory function for creating typed attributes"""
    class TypedAttribute:
        def __init__(self):
            self.name = None
        
        def __set_name__(self, owner, name):
            self.name = name
        
        def __get__(self, obj, objtype=None):
            if obj is None:
                return self
            return obj.__dict__.get(self.name, default)
        
        def __set__(self, obj, value):
            if value is not None and not isinstance(value, expected_type):
                raise TypeError(f"{self.name} must be {expected_type}")
            obj.__dict__[self.name] = value
    
    return TypedAttribute()

class Config:
    # Use factory to create typed attributes
    port = typed_attribute(int, default=8000)
    debug = typed_attribute(bool, default=False)
    host = typed_attribute(str, default="localhost")

3. Metaclass Integration

class DescriptorMeta(type):
    """Metaclass that automatically sets up descriptors"""
    def __new__(cls, name, bases, namespace):
        # Find all descriptors and call __set_name__ if needed
        for key, value in namespace.items():
            if hasattr(value, '__set_name__'):
                value.__set_name__(cls, key)
        return super().__new__(cls, name, bases, namespace)

class MyClass(metaclass=DescriptorMeta):
    attr = ValidatedAttribute(min_value=0)
    # __set_name__ is automatically called

Summary

Descriptors are a powerful and fundamental feature of Python that enable:

  • Computed properties and lazy evaluation
  • Data validation and type checking
  • Attribute access control and logging
  • Framework development (like Django ORM)
  • Clean, reusable code patterns

Understanding descriptors is essential for:

  • Advanced Python development
  • Framework and library creation
  • Interview preparation for senior Python positions
  • Understanding how Python’s built-in features work

The key is to recognize that descriptors are everywhere in Python - properties, methods, and many built-in features are all implemented using the descriptor protocol. Mastering descriptors opens up powerful possibilities for creating clean, maintainable, and flexible code.

Interview angle

  • “What is a descriptor?” — an object defining __get__, __set__ or __delete__ that, when assigned as a class attribute, intercepts attribute access on instances. It’s the mechanism behind property, classmethod, staticmethod and ORM fields.
  • “Data versus non-data descriptor?” — a data descriptor defines __set__ or __delete__ and takes precedence over the instance __dict__; a non-data descriptor defines only __get__ and is shadowed by an instance attribute. That precedence is exactly why @property can’t be overwritten on an instance while a cached method can.
  • “What’s the lookup order?” — type-level data descriptor, then instance __dict__, then type-level non-data descriptor, then __getattr__. Knowing this explains most surprising attribute behaviour.
  • “When would you write one?” — reusable attribute behaviour across many fields: validation, type coercion, lazy loading, unit conversion. For a single attribute, @property is simpler. Use __set_name__ so the descriptor learns its own attribute name automatically.
Previous
Data Abstraction in Python