beginnerintermediateadvanced

Python · Linux

Linux

The process model, permissions, and the command-line toolset a backend Python engineer needs from the OS the code actually runs on

os.getpid() / fork modelsubprocess.run() starts a real new process, not a thread
SignalsSIGTERM asks nicely; SIGKILL (-9) cannot be caught or ignored
chmod / owner:groupthree permission triads — owner, group, other
ps/top/lsof/killfind it, see what it holds open, signal it

1.Processes, PIDs, and threads

A process is a running program with its own memory space and a unique process ID (PID) the kernel assigns it. A thread runs inside a process and shares that process's memory with any other threads in it.

Key features

Every process gets a unique PID (process ID) from the kernel when it starts
A process has its own memory space — one process cannot read another's variables directly
A thread runs inside a process and shares that process's memory with its sibling threads
os.getpid() returns the current process's PID; os.getppid() returns its parent's PID
subprocess.run(...) starts a new, separate process — python.concurrency-and-parallelism.threading-and-thread starts a new thread in this one

Process vs. thread, and how Python starts each

Process vs. thread, and how Python starts each
Aspect	Process	Thread
Memory	own address space, isolated	shares its process's memory
Identifier	PID, unique system-wide	no PID of its own — lives inside one
Crash isolation	one process crashing does not take down another	an unhandled exception in one thread can affect the process
Started from Python	subprocess.run(...), multiprocessing.Process(...)	threading.Thread(...)
Cost to start	heavier — new memory space, new PID	lighter — reuses the parent process's memory

One process, two threads sharing its memory

Process (PID 4821)

its own memory space

Thread 1

reads/writes the process's memory

Thread 2

reads/writes the SAME memory

Command

python

import os

os.getpid()    # this process's own PID
os.getppid()   # the PID of the process that started it (its parent)

A process has its own memory and a unique PID; a thread runs inside a process and shares that process's memory — subprocess.run() gets you the former, threading.Thread() the latter.

2.Signals

A signal is a short notification the kernel or another process sends to a running process — SIGTERM asks it to stop, SIGKILL forces it to stop. Python's signal module lets a process run its own code when one arrives instead of just dying.

Key features

SIGTERM (15) is the polite request to stop — the default kill sends this, and a process can catch it to clean up first
SIGKILL (9) cannot be caught, blocked, or ignored — the kernel ends the process immediately, no cleanup runs
SIGINT (2) is what Ctrl+C sends to the foreground process in a terminal
SIGHUP (1) originally meant "the terminal hung up" — daemons commonly repurpose it to mean "reload your config"
signal.signal(sig, handler) registers a Python function to run when that signal arrives, instead of the default action

Signals a backend engineer actually runs into

Signals a backend engineer actually runs into
Signal	Number	Default action	Typical source
SIGTERM	15	terminate (catchable)	kill PID, systemctl stop, container shutdown
SIGKILL	9	terminate (NOT catchable)	kill -9 PID, OOM killer
SIGINT	2	terminate (catchable)	Ctrl+C in a terminal
SIGHUP	1	terminate (catchable)	terminal closed; daemons repurpose it as "reload config"
SIGCHLD	17	ignored by default	a child process exited — the parent is notified

A signal interrupts a running process

kill -TERM 4821

another process sends SIGTERM

kernel delivers it

process 4821 is interrupted

handler runs

save state, close files, exit(0)

Command

python

import signal

def handler(signum, frame):
    ...  # signum is the signal number that arrived; frame is the interrupted stack frame

signal.signal(signal.SIGTERM, handler)   # register
signal.signal(signal.SIGTERM, signal.SIG_DFL)   # restore the default action

SIGTERM is catchable and asks nicely — use it for cleanup logic; SIGKILL cannot be caught and ends the process immediately, with zero cleanup.

3.File permissions, users, and groups

Every Linux file has an owning user, an owning group, and permission bits for each of owner/group/everyone-else — read, write, and execute. chmod 600 secrets.env means only the owner can read or write it, and no one can execute it.

Key features

A permission string like -rwxr-xr-- has three triads: owner (rwx), group (r-x), other (r--)
Octal notation compresses each triad to one digit: rwx=7, rw-=6, r-x=5, r--=4, so rwxr-xr-- is 754
chmod 600 file means owner can read/write, no one else can read, write, or execute it
Every file also has an owning user and an owning group, changed with chown user:group file
os.chmod(path, 0o600) and pathlib.Path.chmod(0o600) set the same bits from Python

Permission bits, octal digits, and what they mean

Permission bits, octal digits, and what they mean
Symbol	Octal	Meaning
r--	4	read only
-w-	2	write only
--x	1	execute (or "enter", for a directory)
rw-	6	read + write
r-x	5	read + execute — typical for a directory or a script others may run
rwx	7	read + write + execute — full access for that triad

Command

python

import os
from pathlib import Path

os.chmod("secrets.env", 0o600)      # owner rw-, nobody else anything
Path("deploy.sh").chmod(0o755)      # owner rwx, group/other rx

Octal permission = owner digit, group digit, other digit — each digit is read(4)+write(2)+execute(1) added together; chmod 600 means owner-only read/write.

4.Environment variables and the filesystem

An environment variable is a named value passed into a process from its shell or supervisor — os.environ["DB_HOST"] reads one. The filesystem is a single tree rooted at /, with conventional directories like /etc for config and /var/log for logs.

Key features

os.environ is a dict-like mapping of every environment variable visible to this process
os.environ.get("KEY", "default") avoids a KeyError when a variable is not set
os.environ["KEY"] = "value" only affects this process and any children it starts — not the parent shell
/etc holds system-wide config, /var/log holds logs, /tmp is wiped on reboot, /home/<user> is a user's own space
pathlib.Path joins path segments with / directly: Path("/var/log") / "app.log"

Standard Linux directories a backend engineer references most

Standard Linux directories a backend engineer references most
Path	What lives there
/etc	system-wide configuration files (nginx.conf, crontab, ssh config)
/var/log	log files most services write to by convention
/var/lib	persistent application state (databases, package manager data)
/tmp	scratch space, cleared on reboot — never store anything that must survive
/home/<user>	a user's own files and dotfiles (~/.bashrc, ~/.ssh)
/proc	a virtual filesystem exposing live kernel/process info — not real files on disk

Command

python

import os

os.environ.get("DATABASE_URL", "postgresql://localhost/dev")  # safe, has a default
os.environ["DATABASE_URL"]                                    # raises KeyError if unset

os.environ.get(key, default) for optional config, os.environ[key] for required config that should fail fast — and PurePosixPath when a path must follow Linux rules regardless of host OS.

5.Process inspection: ps, top, htop, lsof, kill

ps lists processes, top/htop show them live with resource usage, lsof lists the files and network connections a process has open, and kill sends a process a signal — most often to stop it.

Key features

ps aux (BSD-style, no dash) or ps -ef (Unix-style) lists every process on the system, one line each
top and htop refresh live — htop adds colour, mouse support, and a scrollable process tree over plain top
lsof -i :8000 shows which process is listening on port 8000 — the fastest way to answer "what has this port"
kill PID sends SIGTERM by default (a request); kill -9 PID sends SIGKILL (cannot be ignored)
subprocess.run(["ps", "-ef"], capture_output=True, text=True) drives any of these from Python and parses stdout

ps, top, htop, lsof, kill — one row each

ps, top, htop, lsof, kill — one row each
Command	Answers	Common invocation
ps	"what processes exist right now"	ps aux or ps -ef
top	"what is using CPU/memory right now, live"	top (press q to quit)
htop	"same as top, more readable"	htop (not always preinstalled)
lsof	"what files/ports does this process have open"	lsof -i :8000 · lsof -p PID
kill	"stop (or signal) this process"	kill PID · kill -9 PID

Command

python

import subprocess

subprocess.run(["ps", "aux"])                 # BSD-style: no leading dash
subprocess.run(["lsof", "-i", ":8000"])        # what's listening on port 8000
subprocess.run(["kill", "-15", str(pid)])      # SIGTERM, the polite default
subprocess.run(["kill", "-9", str(pid)])       # SIGKILL, cannot be ignored

ps/top/htop answer "what is running"; lsof answers "what does it have open"; kill sends a signal — SIGTERM (default, catchable) before SIGKILL (-9, not catchable).

6.Text processing: grep, awk, sed

grep searches text for lines matching a pattern, awk extracts and processes fields from structured lines, and sed rewrites text by pattern — three small tools that read a log file faster than opening it in Python for a one-off check.

Key features

grep PATTERN file prints matching lines; grep -c counts them; grep -E enables extended regex
awk works a line at a time, splitting on whitespace by default — $1, $2 are the first and second fields
sed 's/OLD/NEW/' file replaces the first match per line; add g to replace every match on the line
sed -n '2p' file prints only line 2 — -n suppresses sed's normal automatic printing
subprocess.run([...], capture_output=True, text=True) drives any of these from Python

grep, awk, sed — one row each

grep, awk, sed — one row each
Command	Job	Example
grep	find lines matching a pattern	grep ERROR app.log
grep -c	count matching lines	grep -c ERROR app.log
awk	extract/process fields from each line	awk '{print $1, $2}' app.log
sed s/../../	substitute text	sed 's/ERROR/FAIL/' app.log
sed -n Np	print only line N	sed -n '2p' app.log

Command

bash

grep -E 'ERROR|WARN' app.log        # extended regex, either pattern
awk -F',' '{print $2}' data.csv     # -F sets the field separator
sed -i 's/DEBUG/INFO/g' app.log     # -i edits the file in place, g = every match

grep finds lines, awk extracts fields from a line, sed rewrites text — sed's s/// only replaces the first match per line unless you add g.

7.curl and wget

curl sends an HTTP request from the shell and prints or captures the response — useful for testing an API or scripting a call. wget is built specifically for downloading a file to disk, with better default retry/resume behavior.

Key features

curl -s prints the response body to stdout — pipe it into a script, or capture it with $(curl -s ...)
curl -I sends a HEAD request and prints only the response headers, not the body
curl -o FILE saves the response body to FILE instead of printing it; curl -O keeps the remote filename
curl -w "%{http_code}" prints just the numeric status code — the standard way to check success from a shell script
wget resumes an interrupted download with -c and retries on failure by default — curl needs --retry and -C - to do the same

curl vs. wget for the same task

curl vs. wget for the same task
Task	curl	wget
Print a response body	curl -s URL	wget -qO- URL
Download to a file	curl -O URL	wget URL
Headers only	curl -I URL	wget --spider -S URL
Resume a partial download	curl -C - -O URL	wget -c URL
Send JSON with a method	curl -X POST -d '{"a":1}' URL	wget --method=POST --body-data='{"a":1}' URL

Command

bash

curl -s https://httpbin.org/get                 # print the body
curl -sI https://httpbin.org/get                # headers only, no body
curl -s -o out.json https://httpbin.org/get     # save body to out.json
curl -s -w "%{http_code}" -o /dev/null URL      # just the status code

curl -s prints the response and is scriptable; curl -w "%{http_code}" -o /dev/null gets just the status. wget defaults to resumable file downloads; curl needs -C - to do the same.

8.Remote access and scheduling: ssh, scp, systemd, cron

ssh opens a remote shell on another machine over an encrypted connection; scp copies files to or from that machine using the same connection. systemd keeps a service process running and restarts it on crash or reboot; cron runs a command on a fixed schedule.

Key features

ssh user@host opens a remote shell; ssh user@host "command" runs one command remotely and returns
scp file.txt user@host:/path/ copies a local file to a remote path over the same protocol ssh uses
A systemd unit file's Restart=on-failure line is what makes a crashed service come back automatically
cron reads a five-field schedule (minute hour day-of-month month day-of-week) from a crontab, then runs the command in a minimal, non-interactive shell
A cron job's environment is NOT your login shell's environment — PATH and other variables are often missing, a common source of "works when I run it, fails from cron"

ssh, scp, systemd, cron — one row each

ssh, scp, systemd, cron — one row each
Tool	Solves	Common invocation
ssh	"open a shell on that other machine"	ssh deploy@10.0.0.5
scp	"copy this file to/from that machine"	scp app.tar.gz deploy@10.0.0.5:/srv/app/
systemd	"keep this service running, restart it if it dies"	systemctl restart myapp.service
cron	"run this command every night at 2am"	0 2 * * * /usr/bin/python3 /srv/app/nightly.py

Command

bash

ssh user@host                              # interactive remote shell
ssh user@host "ls -la /srv/app"            # run one command, then exit
scp local.txt user@host:/remote/path/      # copy a file up
scp user@host:/remote/file.txt .           # copy a file down

systemctl status myapp.service             # is it running, and why not
systemctl restart myapp.service            # restart it now

crontab -l                                 # list the current user's cron jobs
crontab -e                                 # edit them

ssh opens a remote shell, scp copies files over the same connection. systemd keeps a service running and restarts it on crash (Restart=on-failure); cron runs a command on a schedule — always use an absolute interpreter path and redirect output to a log file.

Spotted a mistake on this page? Tell us.

Read Sheet