Plugin system design: discovery, interfaces, isolation, and versioning
A plugin system needs four things: a stable interface, a discovery mechanism, lifecycle management, and isolation. The interface is usually an abstract base class or a Protocol that plugins implement. Discovery is typically done with setuptools entry points, which let a plugin package declare a name in a group that your application reads with importlib.metadata.entry_points. This avoids hardcoding a list of plugins and lets third parties install their package alongside yours. Lifecycle matters: decide whether plugins are loaded at startup or lazily, how they register themselves, and how you handle initialization failures without crashing the whole application. Isolation is the hardest part. A plugin runs in the same process with the same privileges, so a buggy or malicious plugin can crash the process or read data it should not. For untrusted plugins, consider a subprocess or a separate service with a narrow RPC boundary. Finally, versioning: the plugin interface is a public API. Version it explicitly, provide a compatibility shim, and document what changes are breaking. Common mistakes include importing all plugins eagerly at startup, which slows startup and increases the blast radius of a failure, and not validating that a discovered entry point actually implements the interface.
Interface: ABC or Protocol that plugins implement. Keep it small and stable.
Discovery: importlib.metadata.entry_points(group='myapp.plugins') or a registry populated by decorators.
Lifecycle: load lazily, isolate failures, and provide an explicit enable/disable mechanism.
Isolation: in-process for trusted plugins, subprocess or RPC for untrusted ones.
Versioning: version the plugin API, document breaking changes, and consider a compatibility layer.
Trade-off: entry points are standard and decoupled but require packaging; a decorator registry is simpler but couples plugins to your import path.
Common mistake: importing every plugin at startup, which increases startup time and makes one broken plugin break the app.
Common mistake: exposing internal objects to plugins without a stable interface, which makes refactoring impossible.
Version note: importlib.metadata.entry_points replaced pkg_resources in Python 3.8+. Prefer it for new code.
0-2 years experience
2-5 years experience
5-8 years experience
8+ years experience