Operations
Deployment administrators maintain production operation of the platform. Follow the delivered runbook for resources, backups, and upgrades. Use the console to inspect runtime status and manage supported settings. Coordinate maintenance windows, expansion, and recovery with the business owner and delivery team.
Daily Checks
| Check | Action and interpretation |
|---|---|
| Runtime and requests | Use Monitoring to inspect task outcomes, queues, cluster resources, and response latency. Compare trends before and after changes. |
| Device connections | Check device online status, MQTT connectivity, and session changes. Determine whether an issue affects a device group or access service. |
| AI usage | Check model calls, voice usage, provider quotas, and Device AI Credits. Distinguish insufficient balances from service failures. |
| License | Check request allowance and expiration in License Management. Arrange expansion or renewal in advance. |
| Storage and backups | Check database, disk, and log retention. Confirm that backups complete and can be restored. |
When monitoring is unavailable or data is missing, check the collection service and connectivity first. Missing metrics do not mean usage or failures are zero. To investigate one request, follow Logs for runtime context.
Adjust Runtime Capacity
When request volume grows, tasks stay queued, or resource usage approaches capacity, use monitoring to assess expansion needs. Check model provider quotas, device connections, and license limits first, then agree on resource changes with the delivery team.
Administrators follow the delivered runbook to expand resources and check storage capacity. Additional runtime resources do not increase License request limits; confirm third-party quotas separately.
After expansion, compare queue time, completion rate, response latency, and resource usage. To improve business continuity, also assess the availability of the deployment environment and external services.
Backup and Recovery
Back up platform business data, configuration, and the materials required for recovery. Administrators follow the runbook to define scope, frequency, and retention, and protect credentials and business information in the backup.
Follow the agreed backup procedure, verify its result and integrity, and store backups separately from production. Schedule a maintenance window and notify users when downtime is required.
Practice recovery regularly in a separate environment to confirm that backups work and estimate recovery time. After recovery, check the workspace, configuration, license, device information, and connections, then verify business requests.
Restoring a backup loses data created after that backup. Database recovery does not undo actions already performed on devices or business systems. Check execution records to avoid manually issuing those actions again. If recovery fails, keep access closed and follow the recovery runbook.
Upgrade the Platform
Upgrades require a maintenance window. Use the target version and recovery plan confirmed by the delivery team. Keep runtime services on one approved version and use the corresponding console release.
- Record the current version, deployment identity, key configuration, and runtime status. Confirm that the pre-upgrade backup and required keys are available.
- Pause new requests at the production entry point and wait for queued and running tasks to finish. Account for the maintenance impact on voice, IM, workflows, and scheduled tasks.
- Administrators complete the version update and status checks through the upgrade runbook.
- Check service status, MQTT, monitoring, and logs. Verify configuration, AI credits, and the license, then test authorized business requests through the production entry point.
- Restore traffic after verification, and continue observing queues, failures, latency, and provider throttling.
Platform upgrades preserve the installed license and configuration. Follow the recovery plan confirmed by the delivery team when a rollback is needed. If an upgrade is interrupted or checks fail, keep traffic paused, preserve logs and backups, and contact the delivery team.
Troubleshooting and Support
Determine whether an incident affects one device, an IoT platform, a model or voice service, or the whole deployment. Use monitoring and logs to record the time and timezone, device or request identifiers, errors, and recent changes.
For support, contact us with the deployment identity, version, impact, and sanitized records. See Logs for viewing and exporting records.