What are list, dict, and set comprehensions, and why are they often preferred over explicit loops?

A comprehension is an expression that builds a collection from an iterable in one declarative statement: [expr for x in it if cond] for a list, {k: v for ...} for a dict, and {expr for ...} for a set. Structurally it is the same as a for loop with an optional filter and an append/assignment, but it states 'what the result is' rather than 'how to assemble it'. Multiple for clauses read left to right in the same order as the equivalent nested loops, and a conditional expression (x if c else y) in the output position is different from a filter (if c) at the end.

Why they're preferred: (1) readability - for transformations and filters the intent is visible in one line instead of being spread across initialization, loop, condition and append; (2) fewer mutable-state bugs, because there is no half-built list variable hanging around; (3) usually faster than the append loop, because the interpreter uses a dedicated list-append instruction and avoids a method lookup and call per item. The size of the gain varies by version and workload, so I say 'typically faster' and verify with timeit rather than quote numbers. Python 3.12 inlined comprehensions (PEP 709), which reduced their overhead further without changing semantics.

Scope and semantics details: in Python 3 the loop variable of a comprehension does not leak into the enclosing scope (unlike a for statement, and unlike Python 2 list comprehensions). A walrus assignment inside a comprehension, however, binds in the containing scope by design. In a dict comprehension duplicate keys silently keep the last value, which matters when inverting a mapping with non-unique values; a set comprehension has no defined order.

When not to use them, and common mistakes: don't use a comprehension for side effects ([print(x) for x in items] builds a throwaway list - use a for loop); don't cram multiple nested loops, conditions and walrus operators into one line - once it needs more than about two clauses or stops reading like a sentence, use a loop or a helper function; and for very large or unbounded data prefer a generator expression, because a list comprehension materializes everything in memory. Another gotcha is that {} is an empty dict, so an empty set is set(). Alternatives such as map, filter, itertools and numpy have their place, but for general-purpose transformations a comprehension is the most Pythonic default.