What is a generator expression, and how does it differ from a list comprehension in terms of memory usage?

A generator expression has the same syntax as a list comprehension but uses parentheses: (expr for x in it if cond). Instead of building the whole list, it creates a generator object, a lazy iterator that computes one item each time it is asked for the next value and remembers where it left off. A list comprehension is eager: it computes every element immediately and stores them all, so memory is O(n); the generator object itself is a small constant size, and memory use is O(1) as long as the consumer processes items one at a time.

Why this matters: lazy evaluation lets you process data bigger than memory (lines of a huge file, database rows, infinite sequences), lets pipelines stop early (any, all, next, itertools.islice never compute the rest), and avoids allocating a big intermediate list just to feed sum, max, min, join or a for loop. This is why sum(x * x for x in data) is preferred over sum([x * x for x in data]).

Trade-offs: a generator is single-use (a second iteration yields nothing and no error), supports neither len() nor indexing nor slicing, and cannot be rewound. If you need multiple passes, random access, a length, or printing the contents, use a list or tuple. For small data a list comprehension is often faster, since per-item generator resumption has overhead, so don't use generators purely out of habit. Laziness also changes when things happen: exceptions and side effects occur at consumption time, far from where the expression is written, which complicates debugging, and free variables are looked up when the item is computed (see the n example), not when the generator was created. Only the first iterable of the outermost for clause is evaluated immediately.

Common mistakes: wrapping a generator in list() immediately and losing the benefit; returning a generator that depends on a file or connection that has already been closed by a with block (the generator runs after the file is closed); and iterating the same generator twice. For more complex logic, a generator function with yield is the next step up and shares the same lazy semantics; itertools provides many building blocks for pipelines.