beginnerintermediateadvanced

Python · Python Internals

Memory management

Concepts: 8

How CPython allocates memory for an object, tracks how many references point at it, frees it — including the one case that needs a second mechanism — and the tools for referencing something without keeping it alive, reusing a value, or building an independent copy.

This section

Internals of Python - Easiest In-Depth Explanation of Mutable, Immutable Data Types with CodeKeerti Purswani

Counting and collecting

CPython's two-part strategy for freeing memory — instant reference counting, plus a periodic collector for the one pattern that defeats it.

Reference counting

coreintermediate

Every CPython object carries a count of how many references point at it. A new name, container slot, or argument binding increments it; del, rebinding, or scope exit decrements it. The object is freed the instant the count hits zero.

Think of it as

A ticket counter on a door: every time someone walks in (a new reference), it clicks up by one; every time someone leaves (a reference goes away), it clicks down. The room is cleared out the moment the counter reads zero — no waiting, no schedule, just that one number.

python

import sys
sys.getrefcount(obj)   # int — includes +1 for getrefcount's own argument
del name                # decrements one reference
x = new_value            # decrements the old object's count, increments the new one's

What we're doing: Track how binding a second name, passing an object to a function, and deleting names each change its reference count.

refcount.pypython

import sys

obj = object()
print(sys.getrefcount(obj))

alias = obj
print(sys.getrefcount(obj))

def hold(x):
    print(sys.getrefcount(x))

hold(obj)

del alias
print(sys.getrefcount(obj))

3: obj = object() creates the object — one reference: the name obj.
4: getrefcount adds its own temporary reference while running, so this reports 2, not 1.
6: alias = obj adds a second, permanent reference.
7: Now 3: obj, alias, plus getrefcount's temporary one.
10: Calling hold(obj) binds a third name, x, for the duration of the call.
12: Inside hold, refcount is 4: obj, alias, the parameter x, plus getrefcount's temporary reference.
14: del alias removes one permanent reference, dropping the count back down.

Output

Why this works: Every reference to obj — a name, a function parameter, a container slot — adds one to its count, and every one that goes away subtracts one. sys.getrefcount always reports one more than the "real" number of long-lived references, because passing obj to it as an argument is itself a temporary reference that exists for the duration of the call. Calling hold(obj) adds a fourth reference (the parameter x) only while hold is running; that reference disappears when hold returns, though this example does not print after the call to show that drop.

Forgetting sys.getrefcount always overcounts by one

Wrong

python

import sys
obj = object()
print(sys.getrefcount(obj))   # assumed this prints 1 — it does not

Better

python

import sys
obj = object()
print(sys.getrefcount(obj) - 1)   # subtract getrefcount's own temporary reference

What you see: sys.getrefcount(obj) reports 2 for an object with exactly one real reference (the name obj) — reading that as "two references exist somewhere else" is the wrong conclusion.

Why: Calling sys.getrefcount(obj) passes obj as an argument, which itself is a reference that exists for the duration of the call — the function is, unavoidably, looking at a count that includes its own temporary hold on the object. Subtracting 1 gives the count that existed just before the call.

The count that decides when an object dies

obj = object()

refcount 1

alias = obj

refcount 2 — a second reference

del obj; del alias

refcount 0 — freed instantly

What changes the reference count

What changes the reference count
Action	Effect on refcount
x = obj	increments — a new name references it
lst.append(obj)	increments — a new container slot references it
f(obj)	increments for the duration of the call — the parameter is a new reference
del x	decrements — removes one reference
x = something_else	decrements the OLD object x referenced, increments the new one

Together

python

import sys
obj = object()
print(sys.getrefcount(obj))   # 2 — the name obj, plus getrefcount's own temporary argument
alias = obj
print(sys.getrefcount(obj))   # 3
del alias
print(sys.getrefcount(obj))   # back to 2

Remember: Every object carries a refcount; freed the instant it hits zero. sys.getrefcount(obj) reports one extra, for its own reference. Alone, it cannot free a cycle.

Garbage collection

coreintermediate

CPython frees most objects instantly via reference counting. The gc module adds a separate, periodic collector that finds groups of objects referencing only each other — reference counting's one blind spot — and frees them too.

Think of it as

Reference counting is a doorman who clears a room the instant its last occupant leaves. But two guests who only ever hold each other's hand never individually "leave" — the doorman never sees zero. The gc module is a separate inspector who periodically checks every room for exactly this pattern: occupied, but unreachable from any door in the building.

python

import gc
gc.collect()          # run the collector now; returns objects freed
gc.get_threshold()    # (gen0, gen1, gen2) allocation thresholds
gc.disable()           # turn off automatic runs (rarely needed)

What we're doing: Build a reference cycle, confirm it survives deleting both names, then show gc.collect() is what actually frees it.

gc_demo.pypython

import gc
import weakref

class Node:
    def __init__(self, name):
        self.name = name
        self.other = None
    def __del__(self):
        print(f"{self.name} collected")

n1, n2 = Node("A"), Node("B")
n1.other, n2.other = n2, n1

watch = weakref.ref(n1)
del n1, n2

print("after del, before collect:", watch())
freed = gc.collect()
print("after gc.collect():", watch())
print("objects freed:", freed > 0)

11: n1 and n2 are built, each referencing the other — a two-object cycle.
13: weakref.ref lets us check whether n1 still exists without adding a real reference (see weak references).
14: del n1, n2 removes the only references reachable from this scope — but they still reference each other.
16: watch() still returns the object here — reference counting alone has not freed it.
17: gc.collect() runs the cycle detector, which finds the pair is unreachable from anywhere and frees both.

Output

after del, before collect: <__main__.Node object at 0x...>
A collected
B collected
after gc.collect(): None
objects freed: True

Why this works: n1.other and n2.other each hold a real reference to the other object, so deleting the names n1 and n2 never brings either object's reference count to zero — reference counting alone has no way to notice the pair is otherwise unreachable. gc.collect() runs a separate algorithm that traces which container objects are reachable from actual program roots (names, stacks, module globals) and frees whatever is not, cycle or no cycle — which is exactly what frees n1 and n2 here.

Assuming gc.collect() is required after every del, even for non-cyclic objects

Wrong

python

import gc

class Value:
    def __del__(self):
        print("freed")

v = Value()
del v
gc.collect()   # unnecessary — v was already freed before this line ran

Better

python

class Value:
    def __del__(self):
        print("freed")

v = Value()
del v   # freed here already, by reference counting alone — "freed" prints now

What you see: No error, but calling gc.collect() after every del is needless overhead — it scans the whole object graph looking for cycles that, in this case, do not exist.

Why: Value has no reference to itself or to anything that references it back, so its reference count reaches zero the instant del v runs, and it is freed immediately by reference counting — the primary mechanism, not gc. The cycle-detecting collector is only needed for objects that reference each other in a loop; calling it unconditionally after every deletion defeats the point of reference counting being fast and immediate for the common case.

Two mechanisms, one gap

Reference counting

frees anything whose count hits zero — instantly, always on

A cycle forms

two objects only reference each other — neither count reaches zero

gc.collect()

finds unreachable cycles and frees them, periodically or on demand

Reference counting vs the gc module

Reference counting vs the gc module
Property	Reference counting	gc (cycle collector)
Runs	continuously, on every reference change	periodically, based on allocation thresholds
Frees	any object whose count hits zero	groups of objects unreachable from outside, even mid-cycle
Handles cycles	no — a cycle never reaches zero on its own	yes — this is its entire purpose
Manual control	none — always active, cannot be disabled	gc.enable() / gc.disable() / gc.collect()

Together

python

import gc
gc.isenabled()          # True by default
gc.get_threshold()      # (2000, 10, 0) — generation thresholds
gc.collect()             # forces a collection now, returns count freed

Remember: Reference counting frees most objects instantly. gc separately, periodically finds and frees reference CYCLES — the one case it cannot handle alone.

Cyclic references

standardintermediate

A reference cycle is a group of objects that reference each other in a loop, directly or through intermediaries. It occurs naturally in parent/child links, and defeats reference counting on its own.

Think of it as

Two people who owe each other a favor, and no one else, waiting for the other to release them first. Neither ever will — reference counting can only free something once every reference to it is gone, and each half of a cycle is itself one of the references keeping the other half alive.

python

import weakref

class Child:
    def __init__(self, parent):
        self.parent = weakref.ref(parent)   # breaks the cycle — no real reference back

p = Parent()
c = Child(p)
p.children.append(c)   # still a real reference, this direction only

What we're doing: Build a parent/child cycle, confirm deleting both names does not free either object, then rebuild it with a weak reference to break the cycle outright.

cycles.pypython

import weakref

class Parent:
    def __init__(self):
        self.children = []

class Child:
    def __init__(self, parent):
        self.parent = parent
        parent.children.append(self)
    def __del__(self):
        print("child collected")


p = Parent()
c = Child(p)
watch = weakref.ref(c)

del p, c
print("after del:", watch())

7: Child.__init__ stores a REAL reference back to its parent — this is the cycle-forming line.
9: parent.children.append(self) is the other half of the loop — the parent now references the child too.
15: p.children holds a reference to c; c.parent holds a reference to p — a two-object cycle.
18: del p, c removes the only outside references — but the pair still references each other.

Output

after del: <__main__.Child object at 0x...>

Why this works: c.parent = p and p.children.append(c) each hold a real reference in opposite directions, so removing the names p and c does not bring either object's reference count to zero — each is still referenced by the other. "child collected" never prints in this run, because reference counting alone genuinely cannot resolve this cycle; only gc.collect() (see garbage collection) or breaking the cycle in the first place fixes it.

Building a parent/child link without considering the cycle it creates

Wrong

python

class TreeNode:
    def __init__(self, value, parent=None):
        self.value = value
        self.parent = parent      # real reference UP
        self.children = []
        if parent is not None:
            parent.children.append(self)   # real reference DOWN — a cycle

root = TreeNode("root")
child = TreeNode("leaf", parent=root)
# every node in this tree is now part of a cycle with its parent

Better

python

import weakref

class TreeNode:
    def __init__(self, value, parent=None):
        self.value = value
        self._parent = weakref.ref(parent) if parent else None   # weak — breaks the cycle
        self.children = []
        if parent is not None:
            parent.children.append(self)   # still a real reference, one direction only

    @property
    def parent(self):
        return self._parent() if self._parent else None

root = TreeNode("root")
child = TreeNode("leaf", parent=root)

What you see: A tree that should be freed once discarded lingers in memory until the next gc.collect() run (which does eventually catch it, but only periodically, not instantly) — measurable as memory that grows and only drops in occasional steps.

Why: A bidirectional parent/child link is a reference cycle by construction — every child references its parent, and every parent references its children. This is not a bug (gc.collect() does eventually free it), but it means the whole tree survives, unreclaimed, for however long it takes the cycle collector to run. Making the upward link (child to parent) a weakref removes it from the cycle entirely, so reference counting alone can free each node the moment its last real reference goes.

Where cycles commonly appear

Where cycles commonly appear
Pattern	The cycle
Parent/child tree	child.parent = parent AND parent.children include child
Doubly linked list	each node references both .next and .prev
Observer pattern	subject.observers includes observer, and observer.subject = subject
A closure capturing its own defining object	an instance method stored back onto the instance it belongs to

Together

python

class Parent:
    def __init__(self):
        self.children = []

class Child:
    def __init__(self, parent):
        self.parent = parent
        parent.children.append(self)

p = Parent()
c = Child(p)   # p.children -> [c], c.parent -> p: a cycle

Remember: A cycle is objects referencing each other in a loop — common in parent/child trees. Reference counting alone cannot free one; gc's collector, or a weakref, can.

Allocation and where things live

Where CPython gets the memory for a new object, what every object costs before it holds any data, and the stack/heap model underneath both.

Memory allocation and object overhead

standardintermediate

CPython objects live in a private heap, using pymalloc — an allocator tuned for small objects (512 bytes or less). Every object carries fixed bookkeeping, so even an empty object costs more than its raw data would in C.

Think of it as

pymalloc is a warehouse that pre-slices shelving into small-object-sized bins, so handing out a small crate is fast — no measuring a fresh space each time. Every crate, even an empty one, still needs a shipping label (the reference count and type pointer) stapled to it — that label's weight is the 'overhead' that never disappears, no matter how little the crate holds.

python

import sys
sys.getsizeof(obj)          # bytes obj occupies, including fixed per-object overhead
sys.getsizeof([1, 2, 3])    # grows with contents — but never starts at 0

What we're doing: Measure how object overhead adds up — an empty container versus one with items, and a small int versus a large one.

overhead.pypython

import sys

print(sys.getsizeof(0))
print(sys.getsizeof([]))
print(sys.getsizeof([1, 2, 3]))
print(sys.getsizeof(()))
print(sys.getsizeof((1, 2, 3)))
print(sys.getsizeof(2**64))

3: Even the int 0 costs 28 bytes — a type pointer, refcount, and digit array, before any meaningful value.
4: An empty list already costs 56 bytes — the overhead of a resizable array structure, holding nothing.
5: Three ints added costs 32 more bytes (roughly 8 bytes per pointer slot, plus growth headroom) — the list only stores pointers, not the int objects themselves.
8: A number outside the machine-word range needs extra internal digits, which is visible in its size.

Output

Why this works: Every object pays fixed overhead for being a Python object at all — a reference count and a type pointer at minimum, managed inside CPython's private heap via pymalloc. That overhead exists whether the object holds anything or not, which is why sys.getsizeof([]) is 56, not 0. A list only stores POINTERS to its items (not copies of the objects themselves), so appending three small ints grows the list's own size by 32 bytes — the ints' own 28 bytes each are counted separately, since they exist as independent objects the list only references.

Assuming sys.getsizeof() on a container reports the size of everything it holds

Wrong

python

import sys
big_strings = ["x" * 10_000 for _ in range(100)]
print(sys.getsizeof(big_strings))   # assumed this includes the strings' data — it does not

Better

python

import sys
big_strings = ["x" * 10_000 for _ in range(100)]
container_size = sys.getsizeof(big_strings)
items_size = sum(sys.getsizeof(s) for s in big_strings)
print(container_size, items_size, container_size + items_size)

What you see: sys.getsizeof(big_strings) reports a small number (a few hundred bytes) even though the list references a million characters' worth of string data — nowhere close to the actual memory in use.

Why: A container in CPython stores references (pointers) to its items, not the items themselves. sys.getsizeof() reports only the size of the container structure — the array of pointers plus its own overhead — never recursing into what those pointers point at. Measuring true total memory requires adding up every referenced object separately, which is what sys.getsizeof() deliberately does not do for you.

sys.getsizeof(), a few common empty/small objects

sys.getsizeof(), a few common empty/small objects
Object	Bytes
1 (a small int)	28
"" (empty string)	41
[] (empty list)	56
() (empty tuple)	40
{} (empty dict)	64

Together

python

import sys
sys.getsizeof(1)      # 28 — not 8 bytes, even though a C long would be
sys.getsizeof([])      # 56 — bookkeeping for a resizable array, holding nothing yet
sys.getsizeof({})      # 64 — a hash table's overhead, before a single key exists

Remember: Objects live in CPython's private heap; pymalloc handles small ones (≤ 512 bytes) from arenas. Every object pays fixed overhead even holding nothing.

Stack vs heap (conceptual model)

standardintermediate

Every Python object lives on the heap — CPython's private heap. The call stack holds only frames and the references inside them, never the objects — unlike C, where a local can live directly on the stack.

Think of it as

The call stack is a stack of index cards, one per active function call, each listing which objects that call currently cares about — but every object itself lives in one shared warehouse (the heap), never on the cards. A card being thrown away (a function returning) removes its list of pointers, but the warehouse items themselves only leave when nothing anywhere still points at them.

python

def f():
    x = [1, 2, 3]   # x: a name in f's frame; [1, 2, 3]: an object on the heap
    return x          # f's frame is popped; the list survives because the return value still references it

What we're doing: Confirm a heap object survives its creating function returning, because the reference keeping it alive moves with the return value rather than living on the stack.

stack_heap.pypython

def make_list():
    values = [1, 2, 3]
    return values


result = make_list()
print(result)
print(type(result))

2: values is a name inside make_list's frame — a reference. The list [1, 2, 3] it points to lives on the heap, not inside the frame.
3: Returning values hands the reference to the caller — make_list's frame is popped right after, but the heap object it referenced is not affected by that.
6: result now holds the same heap reference values did — the list was never tied to the frame's lifetime.

Output

[1, 2, 3]
<class 'list'>

Why this works: make_list's frame — the stack entry for that call — is popped the instant the function returns. If Python worked like C, with values allocated directly on the stack, the list would need to be destroyed or copied at that point. Instead, values was only ever a NAME (a reference) inside the frame, pointing at a list object that lives on the heap independently. Returning values copies the reference, not the object, into the caller's scope — the object was never at risk from the frame going away.

Assuming a local variable's data is destroyed when its function returns, the way it would be in C

Wrong

python

def get_config():
    config = {"debug": True}
    return config
    # in C, a stack-local struct returned by reference would be a dangling-pointer bug here

result = get_config()
print(result)   # this works fine in Python — no such bug exists

Better

python

# Correct model: config is a NAME in get_config's frame, referencing a
# dict object that lives on the heap. Returning config hands that heap
# reference to the caller — the dict is unaffected by the frame ending.
def get_config():
    config = {"debug": True}
    return config

result = get_config()
print(result)   # {'debug': True} — completely safe

What you see: There is no bug — the surprise, coming from a language like C, is expecting one. Python code correctly returning local data is not a special case requiring care.

Why: In C, a local variable can be allocated directly on the stack, so returning a pointer to it IS a dangling-pointer bug once the function returns and that stack space is reused. Python never allocates object data on the stack at all — every object lives on the heap from the moment it is created, and a frame only ever holds references into it. There is no stack-lifetime risk to reason about for Python object data.

Two regions, one direction of reference

Call stack

frames — one per active call, holding names/references only

Heap

every actual object lives here, in CPython's private heap

What lives where, conceptually

What lives where, conceptually
Region	Holds
Call stack	frames — one per active function call
Each frame	local names, each a reference into the heap
Heap	every actual object — ints, strings, lists, functions, everything
A function returns	its frame is popped — references it held are released, objects may or may not be freed

Together

python

def make_list():
    values = [1, 2, 3]   # 'values' is a name in this frame; the list itself is on the heap
    return values         # the frame is popped, but the list survives via the returned reference

result = make_list()
print(result)   # the list outlived the function call that created it

Remember: In CPython, every object lives on the heap. The call stack holds only frames and references — returning never destroys heap data like a C stack pop would.

Referencing, reusing, and copying

A reference that does not keep its target alive, CPython reusing one object for a repeated value, and the two ways to build an independent copy of a structure.

Weak references

standardintermediate

weakref.ref(obj) creates a reference that does NOT count toward obj's reference count — obj can still be freed as if it did not exist. Calling the weak reference returns the object if alive, or None if freed.

Think of it as

A regular reference is a claim ticket for the object — hold one, and the object cannot be discarded. A weak reference is a sticky note that just points at the object's current spot, with no claim on it. Check the note whenever you like, but if the object is already gone by then, the note just says 'not here.'

python

import weakref
r = weakref.ref(obj)      # create a weak reference
r()                          # the live object, or None if it has been freed
weakref.WeakValueDictionary()   # a dict whose VALUES are held weakly

What we're doing: Confirm a weak reference does not keep an object alive, then use WeakValueDictionary to see an entry disappear automatically once its value is freed.

weakrefs.pypython

import sys
import weakref

class Session:
    pass

obj = Session()
print(sys.getrefcount(obj))
r = weakref.ref(obj)
print(sys.getrefcount(obj))

print(r() is obj)
del obj
print(r())

cache = weakref.WeakValueDictionary()
val = Session()
cache["a"] = val
print("a" in cache)
del val
print("a" in cache)

8: Before any weak reference, refcount reflects only the real reference obj.
9: weakref.ref(obj) does NOT change the refcount — the weak reference is invisible to reference counting.
12: del obj drops the last real reference, freeing the object.
13: r() now returns None — the object is gone, and the weak reference reports that honestly instead of keeping it alive.
18: The WeakValueDictionary entry exists while val is alive.
20: Once val is deleted (the only real reference to that Session), the entry is automatically removed — no manual cleanup needed.

Output

2
2
True
None
True
False

Why this works: weakref.ref(obj) is deliberately excluded from the reference-counting mechanism — creating one leaves sys.getrefcount(obj) unchanged, which is the entire point: it lets code hold a handle to an object without being one of the things keeping that object alive. Once every REAL reference is gone, the object is freed exactly as if the weak reference never existed, and calling it afterward returns None rather than a stale pointer. WeakValueDictionary applies the same idea to a whole dict's values, automatically dropping entries whose value has since been freed.

Assuming a weak reference keeps its target alive, like a regular one

Wrong

python

import weakref

class Connection:
    pass

def get_pooled_connection():
    conn = Connection()
    return weakref.ref(conn)   # returning ONLY a weak reference

handle = get_pooled_connection()
print(handle())   # assumed to still work — it does not

Better

python

import weakref

class Connection:
    pass

def get_pooled_connection():
    conn = Connection()
    return conn, weakref.ref(conn)   # keep a REAL reference alongside the weak one

conn, handle = get_pooled_connection()
print(handle())   # works — conn itself is what keeps it alive

What you see: handle() returns None immediately — the Connection was freed the moment get_pooled_connection returned, because the weak reference was the ONLY reference and it never counted toward keeping the object alive.

Why: conn, the only real reference to the Connection, goes out of scope when get_pooled_connection returns, dropping the refcount to zero — the weak reference never contributed to that count in the first place. A weak reference is only useful alongside at least one real reference held somewhere else; it cannot be the sole thing keeping an object alive by design.

A reference that does not keep anything alive

obj = Cache()

refcount 1 — one real reference

r = weakref.ref(obj)

refcount still 1 — the weak reference does not count

del obj; r()

returns None — freed as if r never existed

Regular reference vs weak reference

Regular reference vs weak reference
Property	Regular (obj = x)	weakref.ref(x)
Increments refcount?	yes	no
Keeps the object alive?	yes	no
How to access the object	obj (directly)	ref_obj() — call it
If the object is freed	never happens while referenced	ref_obj() returns None

Together

python

import weakref

class Cache:
    pass

obj = Cache()
r = weakref.ref(obj)
print(r())          # the live Cache object
del obj
print(r())          # None — freed, because r never counted as a real reference

Remember: weakref.ref(obj) does not increment obj's reference count. Calling it returns the object while alive, None once freed — reference without keeping alive.

Interning

standardintermediate

Interning is CPython reusing one object for certain repeated immutable values, instead of building a new one each time — ints -5 to 256, and some string literals. sys.intern(s) does it manually, guaranteeing equal strings share one object.

Think of it as

A print shop keeps one master copy of its most-requested small flyers on the shelf and just hands out the same physical copy every time one is requested, rather than reprinting it. Ask for something less common, though, and it prints you a fresh copy each time — even if two customers ask for the exact same rare thing.

python

import sys
interned = sys.intern(some_string)   # forces some_string into the shared pool
interned is sys.intern(other_string)  # True whenever other_string == some_string

What we're doing: Show a runtime-built string failing an is check against an equal literal, then use sys.intern() to make two runtime-built copies share one object.

interning.pypython

import sys
import json

literal = "hello world!"
built_at_runtime = json.loads('"hello world!"')

print(literal == built_at_runtime)
print(literal is built_at_runtime)

interned_a = sys.intern(built_at_runtime)
interned_b = sys.intern("hello world!")

print(interned_a is interned_b)

4: literal is compiled directly into the code — CPython may or may not intern it automatically depending on its shape.
5: json.loads builds a brand-new string object at runtime — genuinely no way for it to share the literal's object without help.
8: is reports False — two distinct objects, despite equal content.
10: sys.intern() registers built_at_runtime in the shared intern pool (or returns the existing entry if an equal one is already there).
11: A second call to sys.intern() with an equal string returns the SAME pooled object.
13: Both interned results now share one object — is reports True, guaranteed by intern(), not by coincidence.

Output

True
False
True

Why this works: json.loads builds a genuinely new string object at runtime — nothing about the compiler's automatic literal-interning applies to it, so it is a different object from the literal "hello world!" even though the two are equal in value. sys.intern() sidesteps that entirely: it explicitly registers a string in CPython's shared pool and returns whatever object is already there for an equal string, so any two strings passed through sys.intern() are GUARANTEED to be the same object if they are equal — not a coincidence of how they were built.

Relying on automatic interning instead of sys.intern() for a deliberate identity check

Wrong

python

def is_active_status(status):
    return status is "active"   # relying on automatic interning to make this work

print(is_active_status("acti" + "ve"))   # may be True or False — implementation-dependent

Better

python

def is_active_status(status):
    return status == "active"   # the reliable check — value comparison, not identity

print(is_active_status("acti" + "ve"))   # True, unconditionally

What you see: status is "active" gives an inconsistent answer depending on exactly how status was built, whether it was compiled as a constant, and which CPython version and optimization level is running — the kind of bug that passes in one context and silently fails in another.

Why: Automatic interning is an implementation detail CPython makes no formal promise about beyond a few specific, documented cases (small ints, some compile-time string constants). Code that depends on it for correctness is depending on an optimization, not a language guarantee — == is the check that always gives the right answer, regardless of how the string was constructed.

What gets interned automatically, in CPython

What gets interned automatically, in CPython
Value	Interned by default?
Small ints, -5 to 256	yes — always, for the life of the process
A short identifier-like string literal ("user_id")	usually — compiled as one shared constant
A string with spaces or punctuation ("hello world!")	not guaranteed — depends on how it was built
Any string built at runtime (concatenation, f-string, parsing)	no — a new object every time, unless interned manually

Together

python

import sys
"user_id" is "user_id"                       # commonly True — identifier-shaped literal
sys.intern("built at runtime") is sys.intern("built at runtime")   # True — forced by intern()

Remember: Interning reuses one object for repeated immutable values — small ints and some literals automatically, any string via sys.intern(). Use == for correctness.

Copying objects: shallow copy vs deep copy

coreintermediate

A shallow copy (copy.copy) builds a new outer container but reuses the SAME nested objects inside it. A deep copy (copy.deepcopy) recursively copies every nested object too, so the two structures share nothing at all.

Think of it as

A shallow copy is a new folder holding the exact same documents as the original — move a paper between folders and both folders' contents change, because it is one physical page. A deep copy is a new folder holding photocopies of every document, all the way down — nothing you do to one folder's papers ever shows up in the other.

python

import copy
copy.copy(x)         # shallow — new outer, shared nested objects
copy.deepcopy(x)      # deep — new outer AND new nested objects, recursively
x[:]                    # a common shortcut for a shallow copy of a list

What we're doing: Copy a nested dict three ways — plain assignment, shallow copy, deep copy — and mutate the nested list to reveal exactly what each one shares.

copying.pypython

import copy

original = {"tags": ["core", "beta"], "count": 2}

alias = original
shallow = copy.copy(original)
deep = copy.deepcopy(original)

print(alias is original)
print(shallow is original)
print(shallow["tags"] is original["tags"])
print(deep["tags"] is original["tags"])

shallow["tags"].append("gamma")

print(original["tags"])
print(shallow["tags"])
print(deep["tags"])

5: alias = original is not a copy — same object, second name.
6: copy.copy() builds a genuinely new dict object.
7: copy.deepcopy() builds a new dict AND recursively copies what it contains.
11: shallow['tags'] is original['tags'] is True — the shallow copy's outer dict is new, but the inner list is the SAME object.
12: deep['tags'] is original['tags'] is False — deepcopy gave the inner list its own copy too.
15: Mutating the inner list through shallow is visible through original too, because they share that exact list object.

Output

True
False
True
False
['core', 'beta', 'gamma']
['core', 'beta', 'gamma']
['core', 'beta']

Why this works: copy.copy() only duplicates the OUTER container — its new dict has its own separate keys and slots, but each value inside is a reference to the exact same object the original holds, which is why shallow['tags'] is original['tags'] is True. copy.deepcopy() instead walks the whole structure recursively, building a brand-new object at every level it finds, so deep['tags'] is a genuinely separate list — appending to shallow['tags'] is visible in original (they share the list), but never in deep (which has its own).

Using copy.copy() and expecting full independence from the original

Wrong

python

import copy

def process_order(order):
    working_copy = copy.copy(order)   # shallow — looked "safe enough"
    working_copy["items"].append("gift-wrap")
    return working_copy

order = {"id": 42, "items": ["book"]}
process_order(order)
print(order["items"])   # assumed untouched — it is not

Better

python

import copy

def process_order(order):
    working_copy = copy.deepcopy(order)   # deep — genuinely independent
    working_copy["items"].append("gift-wrap")
    return working_copy

order = {"id": 42, "items": ["book"]}
process_order(order)
print(order["items"])   # ['book'] — untouched, as intended

What you see: order["items"] ends up with 'gift-wrap' appended, even though the function only ever touched working_copy — because working_copy["items"] and order["items"] were always the same list object.

Why: copy.copy() only makes the OUTER dict independent — working_copy is a different dict object from order, but every value inside it, including the items list, is shared by reference. Appending to working_copy['items'] mutates that shared list in place, which is visible through order too. copy.deepcopy() is required whenever independence has to extend to nested mutable objects, not just the top-level container.

What each copy actually duplicates

copy.copy() — shallow

+Builds a NEW outer container
+Nested objects are the SAME objects, referenced by both
+Mutating a nested item is visible in both copies

copy.deepcopy() — deep

−Builds a new outer container AND new nested objects
−Recurses into every level
−Nothing is shared — mutating one never affects the other

Assignment vs shallow copy vs deep copy, on a nested list

Assignment vs shallow copy vs deep copy, on a nested list
Operation	New outer object?	Nested objects shared?
y = x	no — same object, two names	n/a — it is the same object entirely
y = copy.copy(x)	yes	yes — nested objects are the SAME objects
y = copy.deepcopy(x)	yes	no — nested objects are recursively copied too

Together

python

import copy
original = {"tags": ["core", "beta"]}
shallow = copy.copy(original)
deep = copy.deepcopy(original)

shallow["tags"].append("gamma")
print(original["tags"])   # ['core', 'beta', 'gamma'] — shallow copy shared the inner list
print(deep["tags"])        # ['core', 'beta'] — deep copy has its own, untouched

Remember: copy.copy() shares nested objects with the original — mutating one is visible in the other. copy.deepcopy() copies everything, sharing nothing.