# High Volume of Authentication Failures Alert

The High Volume of Authentication Failures alert indicates that many client authentication attempts have failed within a short period. The alert is triggered under the following conditions:

- **Warning**: At least 1,800 authentication attempts occurred in the past 30 minutes, and more than 50% failed.
- **Error**: At least 1,800 authentication attempts occurred in the past 30 minutes, and more than 75% failed.

This condition usually indicates abnormal client authentication activity that may prevent legitimate clients from connecting. Common causes include incorrect client authentication settings, expired or invalid credentials, unauthorized connection attempts, and an unhealthy authentication service.

## Incorrect Client Authentication Configuration

### Symptom

Many clients fail authentication within a short period, and the failures are strongly associated with a specific Client ID, username, or source IP address.

### Common Causes

- Clients use an incorrect username, password, or token.
- Incorrect authentication settings were distributed during a large-scale device upgrade or batch configuration update.

### Resolution

- Verify that the client username, password, or token matches the server-side configuration.
- Validate authentication settings in a test environment before a batch device upgrade or configuration rollout.

## Unauthorized Client Connection Attempts

### Symptom

Many authentication attempts fail within a short period, but they do not originate from legitimate business clients.

Authentication failure logs show requests concentrated among a small number of unfamiliar IP addresses, or the number of new connections rises sharply while the authentication success rate remains very low.

### Common Causes

- Malicious scanning or brute-force attempts.
- Repeated connection attempts with invalid credentials.

### Resolution

- Add suspicious IP ranges to the blocklist.
- Use the connection and authentication protection features provided by EMQX to restrict abnormal connection behavior.
- EMQX enables a reconnection rate limit for the same Client ID by default to help prevent brute-force attacks. For details, see [Expiration Time](../blacklist.md#expiration-time).

## Authentication Service Failure

### Symptom

When an external authentication service, such as an HTTP endpoint or database, experiences latency, timeouts, or failures, authentication can also fail for legitimate clients.

If the increase in authentication failures coincides with a higher error rate or latency in the external authentication service, the service is likely the cause.

### Common Causes

- The external authentication service is unavailable or returns abnormal responses.
- The authentication service is overloaded, causing requests to time out or fail.

### Resolution

- Check the external authentication service status and performance metrics for timeouts or errors.
- Add an appropriate retry or caching strategy to reduce the impact of temporary service instability.
- Add authentication service instances to improve availability and processing capacity.

## Troubleshooting

1. Log in to the EMQX Cloud Console and enter the deployment.

2. Open **Deployment Logs**, set **Error Type** to **Authentication**, and review authentication failure entries to identify the cause.

3. Open **Alerts** and check for authentication-related alerts. Compare their trigger times with the authentication failures.

4. Use the client source, authentication settings, and external authentication service status to determine whether the cause is a client configuration error, unauthorized access attempt, or service failure.

## Impact

- This alert does not normally stop the deployment.
- If the failure rate remains above the threshold, a client connection configuration or authentication service problem may prevent legitimate access.
- Investigate the affected clients, connection behavior, and authentication service promptly to prevent the issue from spreading.