--- title: "Optimize Python Memory Usage with __slots__" lang: "en" author: "Mohammad Abu Mattar" canonical: https://mkabumattar.com/post/python-slots-optimization --- ![Blog post image for Optimizing your python code with \_\_slots\_\_? - Discover how Python \`\_\_slots\_\_\` can reduce memory usage by up to 40% in data-heavy applications. Perfect for MLOps pipelines and big data processing where millions of objects consume precious memory resources.](/_astro/hero.DP_vYsHU_Z2cAFl7.webp) [Home](/)›[Codesnippets](/codesnippets)›[All Categories](/codesnippets/categories)›[Python](/codesnippets/categories/python) Codesnippets [Prev in PythonList S3 Buckets](/codesnippets/post/python-list-s3-buckets) [Python](/codesnippets/categories/python)[Productivity](/codesnippets/categories/productivity)[Python](/codesnippets/python) # Optimizing your python code with **slots**? [Mohammad Abu Mattar](/authors/mohammad-abu-mattar)Published: 16 Jul 202504 Mins read03 Mins listen [Markdown for AI(opens in a new tab)](/post/python-slots-optimization/index.md "Open the plain-Markdown version of this page, for pasting into an AI tool") TL;DR Discover how Python \`\_\_slots\_\_\` can reduce memory usage by up to 40% in data-heavy applications. Perfect for MLOps pipelines and big data processing where millions of objects consume precious memory resources. Series [Programming Languages](/series/programming-languages)1/1 All posts in this series (1) Code Snippets1 1. [Optimizing your python code with \_\_slots\_\_?You are here](/codesnippets/post/python-slots-optimization) ### Optimizing your python code with \_\_slots\_\_? Contents [Memory optimization with `__slots__`](#memory-optimization-with-__slots__)[Understanding the problem](#understanding-the-problem)[Optimizing data models in big data workflows with `__slots__`](#optimizing-data-models-in-big-data-workflows-with-__slots__)[How `__slots__` works](#how-__slots__-works)[Real-world use case: MLOps data pipeline](#real-world-use-case-mlops-data-pipeline)[Customer churn prediction example](#customer-churn-prediction-example)[A data pipeline for MLOps](#a-data-pipeline-for-mlops)[Big data processing benefits](#big-data-processing-benefits)[An MLOps data pipeline with `__slots__`](#an-mlops-data-pipeline-with-__slots__)[Code implementation](#code-implementation)[Class definitions](#class-definitions)[Data generation functions](#data-generation-functions)[Performance testing](#performance-testing)[Results and analysis](#results-and-analysis)[Performance comparison](#performance-comparison)[Practical benefits](#practical-benefits) ## [Memory optimization with `__slots__`](#memory-optimization-with-__slots__) ### [Understanding the problem](#understanding-the-problem) ### [Optimizing data models in big data workflows with `__slots__`](#optimizing-data-models-in-big-data-workflows-with-__slots__) In big data and MLOps workflows, you often work with massive datasets where you create millions of objects to represent data points, features, or model predictions. Traditional Python classes spend extra memory on the hidden `__dict__` attribute. That overhead turns into memory bottlenecks and slower processing, especially in large-scale data pipelines and machine learning models. ### [How `__slots__` works](#how-__slots__-works) `__slots__` reduces the memory footprint of your objects. By defining `__slots__`, you tell Python to allocate a fixed memory space for a class’s attributes instead of a dynamic `__dict__`. Objects get smaller and attribute access gets faster, which matters for high-performance computing tasks in MLOps and big data. ## [Real-world use case: MLOps data pipeline](#real-world-use-case-mlops-data-pipeline) ### [Customer churn prediction example](#customer-churn-prediction-example) ### [A data pipeline for MLOps](#a-data-pipeline-for-mlops) Imagine you’re building a data pipeline for a machine learning model that predicts customer churn. Your pipeline processes millions of customer records, and each record is represented by a Python object. Using a regular class, each customer object would have a memory overhead from its `__dict__`. When you process millions of these objects, the cumulative memory usage grows large enough to crash your application or push you onto more expensive, higher-memory machines. By using a class with `__slots__`, you can create a memory-efficient data model. This approach cuts memory consumption, so you process more data with the same resources and the pipeline runs faster. In MLOps that matters, because resource use drives both your cloud bill and how far the pipeline can scale. ### [Big data processing benefits](#big-data-processing-benefits) ### [An MLOps data pipeline with `__slots__`](#an-mlops-data-pipeline-with-__slots__) In an MLOps pipeline, data is often represented as objects for processing, feature engineering, and model training. When dealing with big data, memory efficiency is what keeps the job from crashing and the cloud bill low. Using `__slots__` reduces the memory footprint of these data objects, which makes the pipeline more reliable and easier to scale. The following full code snippet simulates a big data MLOps workflow where we process a large number of customer records. We will create two versions of a `Customer` data class: one with the default Python behavior and one optimized with `__slots__`. The code then compares the memory usage and the time taken to create a million instances of each. ## [Code implementation](#code-implementation) ### [Class definitions](#class-definitions) customer\_data\_optimization.py ``` 1import sys2import time3import random4import pandas as pd5from typing import List6 7# --- Part 1: Data Model Definitions ---8 9class Customer:10 """11 A regular Python class to represent a customer record.12 This class uses a default __dict__ to store attributes.13 """14 def __init__(self, customer_id: int, age: int, monthly_spend: float, churned: bool):15 self.customer_id = customer_id16 self.age = age17 self.monthly_spend = monthly_spend18 self.churned = churned19 20class OptimizedCustomer:21 """22 An optimized customer class using __slots__ for memory efficiency.23 This class explicitly defines its attributes, eliminating the __dict__ overhead.24 """25 __slots__ = ['customer_id', 'age', 'monthly_spend', 'churned']26 27 def __init__(self, customer_id: int, age: int, monthly_spend: float, churned: bool):28 self.customer_id = customer_id29 self.age = age30 self.monthly_spend = monthly_spend31 self.churned = churned ``` ### [Data generation functions](#data-generation-functions) ``` 1# --- Part 2: Data Generation and Object Creation ---2 3def generate_customer_data(num_records: int) -> List[tuple]:4 """Generates a list of tuples representing raw customer data."""5 data = []6 for i in range(num_records):7 customer_id = i8 age = random.randint(20, 70)9 monthly_spend = round(random.uniform(25.0, 500.0), 2)10 churned = random.choice([True, False])11 data.append((customer_id, age, monthly_spend, churned))12 return data13 14def create_objects(data: List[tuple], class_type: type) -> List:15 """Creates a list of objects from raw data using the specified class."""16 return [class_type(*record) for record in data] ``` ### [Performance testing](#performance-testing) ``` 1# --- Part 3: Performance Comparison ---2 3def run_performance_test(num_records: int):4 """5 Runs a performance test to compare memory and time for both classes.6 """7 print(f"--- Running performance test with {num_records:,} records ---")8 raw_data = generate_customer_data(num_records)9 10 # Test the regular class11 start_time_regular = time.time()12 regular_customers = create_objects(raw_data, Customer)13 end_time_regular = time.time()14 memory_regular = sum(sys.getsizeof(c) for c in regular_customers) + sys.getsizeof(regular_customers)15 16 # Test the optimized class17 start_time_slotted = time.time()18 slotted_customers = create_objects(raw_data, OptimizedCustomer)19 end_time_slotted = time.time()20 memory_slotted = sum(sys.getsizeof(c) for c in slotted_customers) + sys.getsizeof(slotted_customers)21 22 # Print results23 print("\nRegular Class Performance:")24 print(f" - Total Memory: {memory_regular / (1024**2):.2f} MB")25 print(f" - Time Taken: {end_time_regular - start_time_regular:.4f} seconds")26 27 print("\nSlotted Class Performance:")28 print(f" - Total Memory: {memory_slotted / (1024**2):.2f} MB")29 print(f" - Time Taken: {end_time_slotted - start_time_slotted:.4f} seconds")30 31 # Calculate and print the savings32 memory_saved_mb = (memory_regular - memory_slotted) / (1024**2)33 time_saved_s = (end_time_regular - start_time_slotted)34 35 print("\n--- Summary of Savings ---")36 print(f"Memory Saved: {memory_saved_mb:.2f} MB ({ (memory_saved_mb / (memory_regular / (1024**2))) * 100:.2f}%)")37 print(f"⏱Slotted Class Creation Time is faster by: {time_saved_s:.4f} seconds")38 39 # Optional: Demonstrate a simple MLOps task like converting to a DataFrame40 print("\n--- MLOps Task: Converting to a Pandas DataFrame ---")41 # This task is just for demonstration and doesn't show a direct slots benefit here42 df_regular = pd.DataFrame([c.__dict__ for c in regular_customers])43 df_slotted = pd.DataFrame([[c.customer_id, c.age, c.monthly_spend, c.churned] for c in slotted_customers],44 columns=['customer_id', 'age', 'monthly_spend', 'churned'])45 print("Successfully converted both object lists to Pandas DataFrames.")46 print(f"DataFrame head from Slotted Class:\n{df_slotted.head()}")47 48# --- Part 4: Execution ---49if __name__ == "__main__":50 NUM_RECORDS = 1_000_000 # 1 million records51 run_performance_test(NUM_RECORDS) ``` ## [Results and analysis](#results-and-analysis) ### [Performance comparison](#performance-comparison) This code snippet demonstrates how to optimize memory usage in Python by using `__slots__` in a data-heavy application, specifically in an MLOps context. It compares the memory and performance of a regular class versus an optimized class with `__slots__`. On large datasets both memory use and creation time drop. ### [Practical benefits](#practical-benefits) The implementation shows measurable improvements in memory usage and object creation speed, which suits big data processing and MLOps workflows. Was this useful? ## Tags [#Python](/codesnippets/tags/python)[#MemoryOptimization](/codesnippets/tags/memoryoptimization)[#DataScience](/codesnippets/tags/datascience)[#MLOps](/codesnippets/tags/mlops)[#BigData](/codesnippets/tags/bigdata)[#Performance](/codesnippets/tags/performance)[#BestPractices](/codesnippets/tags/bestpractices) ## Share [Facebook](https://facebook.com/sharer/sharer.php?u=https%3A%2F%2Fmkabumattar.com%2Fcodesnippets%2Fpost%2Fpython-slots-optimization "Share on Facebook")[Twitter](https://twitter.com/intent/tweet/?text=Optimizing%20your%20python%20code%20with%20__slots__%3F&url=https%3A%2F%2Fmkabumattar.com%2Fcodesnippets%2Fpost%2Fpython-slots-optimization "Share on Twitter")[LinkedIn](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fmkabumattar.com%2Fcodesnippets%2Fpost%2Fpython-slots-optimization&title=Optimizing%20your%20python%20code%20with%20__slots__%3F&summary=Discover%20how%20Python%20%60__slots__%60%20can%20reduce%20memory%20usage%20by%20up%20to%2040%25%20in%20data-heavy%20applications.%20Perfect%20for%20MLOps%20pipelines%20and%20big%20data%20processing%20where%20millions%20of%20objects%20consume%20precious%20memory%20resources.&source=https://mkabumattar.com "Share on LinkedIn")[WhatsApp](https://wa.me/?text=Optimizing%20your%20python%20code%20with%20__slots__%3F%20https%3A%2F%2Fmkabumattar.com%2Fcodesnippets%2Fpost%2Fpython-slots-optimization "Share on WhatsApp")[Telegram](https://t.me/share/url?url=https%3A%2F%2Fmkabumattar.com%2Fcodesnippets%2Fpost%2Fpython-slots-optimization&text=Optimizing%20your%20python%20code%20with%20__slots__%3F "Share on Telegram")[Reddit](https://www.reddit.com/submit?url=https%3A%2F%2Fmkabumattar.com%2Fcodesnippets%2Fpost%2Fpython-slots-optimization&title=Optimizing%20your%20python%20code%20with%20__slots__%3F "Share on Reddit")[Hacker News](http://news.ycombinator.com/submitlink?u=https%3A%2F%2Fmkabumattar.com%2Fcodesnippets%2Fpost%2Fpython-slots-optimization&t=Optimizing%20your%20python%20code%20with%20__slots__%3F "Share on Hacker News")[Pinterest](https://pinterest.com/pin/create/button/?url=https%3A%2F%2Fmkabumattar.com%2Fcodesnippets%2Fpost%2Fpython-slots-optimization&media=&description=Discover%20how%20Python%20%60__slots__%60%20can%20reduce%20memory%20usage%20by%20up%20to%2040%25%20in%20data-heavy%20applications.%20Perfect%20for%20MLOps%20pipelines%20and%20big%20data%20processing%20where%20millions%20of%20objects%20consume%20precious%20memory%20resources. "Share on Pinterest")[Email]() ## Comments ## You might also enjoy More posts on similar topics [![Python Async HTTP Requests with aiohttp: Fetch Multiple URLs Concurrently](/_astro/hero.DFXqyT8a_Zqhl7o.webp)](/codesnippets/post/python-async-http-aiohttp-concurrent-requests) ## [Python Async HTTP Requests with aiohttp: Fetch Multiple URLs Concurrently](/codesnippets/post/python-async-http-aiohttp-concurrent-requests) - [Mohammad Abu Mattar](/authors/mohammad-abu-mattar) - [Python](/codesnippets/categories/python) - [Async](/codesnippets/categories/async) - [Networking](/codesnippets/categories/networking) Quick Tip Reuse one aiohttp session, fan your requests out with asyncio.gather, and cap them with a semaphore to fetch hundreds of URLs in the time one loop would take. The Problem \*\*The [#Python](/codesnippets/tags/python)[#Aiohttp](/codesnippets/tags/aiohttp)[#Asyncio](/codesnippets/tags/asyncio)+3 tags [read more](/codesnippets/post/python-async-http-aiohttp-concurrent-requests) [![List S3 Buckets](/_astro/hero.BsiJ6hry_1X9XLn.webp)](/codesnippets/post/python-list-s3-buckets) ## [List S3 Buckets](/codesnippets/post/python-list-s3-buckets) - [Mohammad Abu Mattar](/authors/mohammad-abu-mattar) - [Aws](/codesnippets/categories/aws) - [Python](/codesnippets/categories/python) - [Devops](/codesnippets/categories/devops) Overview Multi-profile S3 management Ever juggled multiple AWS accounts and needed a quick S3 bucket inventory across all of them? This Python script handles it. Use case Perfect for or [#Python](/codesnippets/tags/python)[#Boto3](/codesnippets/tags/boto3)[#AWS](/codesnippets/tags/aws)+5 tags [read more](/codesnippets/post/python-list-s3-buckets) [![Redis Caching Patterns: Cache-Aside, Write-Through & Cache Invalidation](/_astro/hero.Ck2oLF89_ZPiYlL.webp)](/codesnippets/post/redis-caching-patterns-architecture) ## [Redis Caching Patterns: Cache-Aside, Write-Through & Cache Invalidation](/codesnippets/post/redis-caching-patterns-architecture) - [Mohammad Abu Mattar](/authors/mohammad-abu-mattar) - [Backend](/codesnippets/categories/backend) - [Performance](/codesnippets/categories/performance) Need to scale your backend without throwing money at servers? Start with Redis caching patterns. Most databases can handle hundreds of queries per second, but thousands? Your app slows to a crawl [#Redis](/codesnippets/tags/redis)[#Caching](/codesnippets/tags/caching)[#Performance](/codesnippets/tags/performance)+6 tags [read more](/codesnippets/post/redis-caching-patterns-architecture) [![Per-App Shell History for Bash](/_astro/hero.Da_6jPH6_Z1EorRD.webp)](/codesnippets/post/bash-per-app-history) ## [Per-App Shell History for Bash](/codesnippets/post/bash-per-app-history) - [Mohammad Abu Mattar](/authors/mohammad-abu-mattar) - [Shell scripting](/codesnippets/categories/shell-scripting) - [Productivity](/codesnippets/categories/productivity) Organize your Bash history per terminal app. Ever jumped between iTerm2, Ghostty, and VS Code's terminal only to have your command history get all mixed up? This Bash snippet keeps things clean b [#Bash](/codesnippets/tags/bash)[#Shell Scripting](/codesnippets/tags/shell-scripting)[#Productivity](/codesnippets/tags/productivity)+3 tags [read more](/codesnippets/post/bash-per-app-history) [![Per-App Shell History for Zsh](/_astro/hero.DRenzVy__1xwzSL.webp)](/codesnippets/post/zsh-per-app-history) ## [Per-App Shell History for Zsh](/codesnippets/post/zsh-per-app-history) - [Mohammad Abu Mattar](/authors/mohammad-abu-mattar) - [Shell scripting](/codesnippets/categories/shell-scripting) - [Productivity](/codesnippets/categories/productivity) Organize your shell history per terminal app. Ever jumped between iTerm2, Ghostty, and VS Code's terminal only to have your command history get all mixed up? This Zsh snippet keeps things clean b [#Zsh](/codesnippets/tags/zsh)[#Shell Scripting](/codesnippets/tags/shell-scripting)[#Productivity](/codesnippets/tags/productivity)+3 tags [read more](/codesnippets/post/zsh-per-app-history) [![AWS EC2 Instance Management with Boto3: Start, Stop, and Query Instances](/_astro/hero.PnHlvJay_ZkFqWG.webp)](/codesnippets/post/aws-ec2-instance-management-boto3-python) ## [AWS EC2 Instance Management with Boto3: Start, Stop, and Query Instances](/codesnippets/post/aws-ec2-instance-management-boto3-python) - [Mohammad Abu Mattar](/authors/mohammad-abu-mattar) - [Cloud](/codesnippets/categories/cloud) - [Aws](/codesnippets/categories/aws) - [Devops](/codesnippets/categories/devops) - [Automation](/codesnippets/categories/automation) If you've ever spent 20 minutes clicking through the AWS Console just to stop a handful of dev instances, you already know the pain. It's tedious, it doesn't scale, and one wrong click can ruin your a [#AWS](/codesnippets/tags/aws)[#EC2](/codesnippets/tags/ec2)[#Boto3](/codesnippets/tags/boto3)+6 tags [read more](/codesnippets/post/aws-ec2-instance-management-boto3-python) 6 related posts