What is the difference between Kafka authentication and authorization?

Authentication is about verifying identity: proving that a client is who it claims to be. Authorization is about enforcing permissions: deciding what an authenticated client is allowed to do. In Kafka, authentication happens at connection time when a client establishes a session with a broker. The broker verifies the client's credentials using a configured mechanism such as SASL/SCRAM, SASL/GSSAPI (Kerberos), SASL/OAUTHBEARER, or mTLS. If authentication succeeds, the broker associates the connection with a principal, which is the identity that will be used for all subsequent authorization decisions. If authentication fails, the connection is rejected and no further communication is possible. Authorization happens after authentication and is enforced on every operation: producing to a topic, consuming from a topic, joining a consumer group, creating a topic, deleting a topic, and so on. Kafka uses ACLs (Access Control Lists) to define which principals can perform which operations on which resources.

The mechanism matters because the two are independent and both are necessary. Authentication without authorization means any authenticated client can do anything, which is a common misconfiguration: teams enable SASL, feel secure, and never configure ACLs, so any client with valid credentials can read every topic. Authorization without authentication is impossible in Kafka because ACLs are keyed on principals, and principals come from authentication. The default principal when authentication is not configured is ANONYMOUS, and if you have not set allow.everyone.if.no.acl.found=false, then ANONYMOUS may have broad access. This is why the first step in securing a Kafka cluster is to enable authentication, and the second step is to set authorizer.class.name and configure ACLs. The principal is the bridge between the two: authentication produces it, authorization consumes it.

A common mistake is to think that enabling TLS is enough for security. TLS provides encryption in transit and, with mTLS, can also provide authentication, but it does not provide authorization. Another mistake is to use a single principal for all applications. If every application authenticates as the same user, you cannot write meaningful ACLs because you cannot distinguish one application from another. Each application should have its own principal, whether that is a SASL username, a Kerberos principal, or a client certificate subject. The trade-off is between operational complexity and security granularity. More principals mean more credentials to manage and rotate, but they also mean least-privilege ACLs are possible. Version note: Kafka's authorizer interface has evolved; the older SimpleAclAuthorizer was replaced by AclAuthorizer in Kafka 2.4, and the ACL management APIs are stable. In KRaft mode (Kafka 3.x), ACLs are stored in the metadata log instead of ZooKeeper, but the ACL model itself is unchanged. Always check the version when configuring authorization.