External Auth/ACL Module Status Abnormal Alert

The External Auth/ACL Module Status Abnormal alert indicates that EMQX encountered errors while communicating with a configured external authentication or authorization service, such as HTTP, MySQL, Redis, PostgreSQL, or JWT.

This condition can prevent clients from completing authentication or authorization, resulting in connection failures or access control errors.

Network Connectivity Failure

Symptom

EMQX cannot establish a network connection to the external authentication or authorization service.

Errors such as timeout or connection refused in Deployment Logs usually indicate a connectivity problem between EMQX and the external service.

Common Causes

• VPC peering, PrivateLink, or the NAT gateway is missing or incorrectly configured.
• The external service IP address or port is incorrect.
• A firewall or security group blocks the required port.

Resolution

• Verify that VPC peering, PrivateLink, or the NAT gateway is configured correctly and active.
• Verify the external service IP address and port.
• Check firewall and security group rules to make sure EMQX can reach the external service.
• Use high availability and load balancing for the external authentication service to avoid a single point of failure.

External Service Unavailable or Unhealthy

Symptom

The external authentication or authorization service is unavailable or unhealthy, so EMQX cannot receive a valid response.

Errors such as unrecoverable_error or disconnected in Deployment Logs usually indicate an external service problem.

Common Causes

• The external service is down or its process exited unexpectedly.
• The external service is overloaded and cannot respond in time.
• The database connection pool is exhausted and rejects new connections.

Resolution

• Check the status and logs of the external service, such as MySQL, PostgreSQL, Redis, or an HTTP service.
• Monitor its CPU, memory, and connection usage for resource exhaustion.
• Increase a database service's connection pool capacity when necessary.

Incorrect External Authentication or Authorization Configuration

Symptom

The external service returns data that EMQX cannot parse because its format or content does not meet EMQX requirements.

Errors such as invalid_response or decode error in Deployment Logs usually indicate an incorrect configuration or response format.

Common Causes

• An HTTP API response is missing the result field required by EMQX.
• A database query is invalid or returns no result.
• The configured JWT key does not match the key used by the client.

Resolution

• Verify that the external HTTP API response follows the EMQX authentication or authorization API requirements.
• Verify that database queries are valid and return the expected result.
• Enable debug mode in a test environment and confirm that the external service response can be parsed by EMQX.

JWT Configuration Problems

Symptom

When JWT is used for authentication or authorization, a mismatch between the JWT configuration and the client's token generation method, or an invalid token, can trigger this alert.

Common Causes

• Key mismatch: The client signs the JWT with a different key from the one EMQX uses for verification. Logs usually contain JWT verification failed.
• Algorithm mismatch: The client uses HS256 while EMQX is configured for another algorithm such as RS256. Logs usually contain unsupported algorithm.
• Expired or invalid token: The client uses an expired or invalid JWT. Logs usually contain exp claim is expired.

Resolution

• Validate the token with a JWT decoding tool or the following command:

  jwt decode <token>

• Verify the JWT header, payload, and signature.

• Make sure the EMQX algorithm and public_key settings match the client's JWT configuration.

• Set an appropriate JWT expiration time and refresh tokens regularly on the client.

Troubleshooting

1. Verify network connectivity. Work with EMQX Technical Support to use tools such as curl, ping, nc, or telnet to confirm that EMQX nodes can reach the external service.
2. Check the external service. Verify that the database or HTTP service is running, has not reached its connection limit, and returns the expected JSON format and HTTP status code.
3. Open Deployment Logs, set Error Type to Authentication or Authorization, and use the error details to identify the cause.
4. For JWT, verify that settings such as algorithm and public_key match the client. Decode a test token and check for expiration or missing claims.