Connect EMQX Agents to Databricks
The Databricks connector lets EMQX Agents work with Unity Catalog, SQL warehouses and statements, jobs, model serving endpoints, and Vector Search. Use this connector to build agents that query operational data, run data workflows, and use Databricks results in automated decisions.
Common Use Cases
- Inspect catalogs, schemas, and tables before generating a SQL workflow.
- Execute SQL statements and use the results in an alert or report.
- Start or inspect a Databricks job from an agent workflow.
- Query a Vector Search index to add enterprise data context to an agent.
Example chat prompt:
text
Use the analytics-databricks connector. List the available catalogs and schemas, find the telemetry table, and query the latest 20 anomaly records.Prerequisites
Before adding the connector, prepare:
- The web address of your Databricks workspace. Use the URL shown in your browser without a trailing slash, for example,
https://dbc-abc123.cloud.databricks.com. - A Databricks personal access token generated in the same workspace.
- Access to the Unity Catalog objects, SQL warehouses, jobs, serving endpoints, or Vector Search resources required by the workflow.
- Any Databricks entitlement or workspace role required by the selected tools.
Add a Databricks Connector
- In the left navigation menu, click Connectors.
- On the Available tab, locate Databricks and click Add.
- Enter a descriptive connector name.
- Click Confirm. EMQX Agents opens the Databricks connection form in the authorization dialog.
- Enter the following information:
- Workspace URL: Enter the Databricks workspace web address without a trailing slash. The URL must identify the workspace where the access token was generated.
- Access Token: Enter a Databricks personal access token, which starts with
dapi. To generate a token in Databricks, click your username and go to Settings -> Developer -> Access tokens -> Generate new token.
- Click Connect Account.
- Wait for Composio to confirm the connection and return you to EMQX Agents.
- On the Added tab, verify that an Authorization successful message appears and that the connector status is Authorized.
Supported Tools
User and Unity Catalog
| Tool | Access | Description |
|---|---|---|
DATABRICKS_IAM_CURRENT_USER_ME | Read | Gets information about the current Databricks user. |
DATABRICKS_LIST_CATALOGS | Read | Lists catalogs in Unity Catalog. |
DATABRICKS_LIST_SCHEMAS | Read | Lists schemas in a catalog. |
DATABRICKS_LIST_TABLES | Read | Lists tables in a catalog and schema. |
SQL Warehouses and Statements
| Tool | Access | Description |
|---|---|---|
DATABRICKS_LIST_SQL_WAREHOUSES | Read | Lists Databricks SQL warehouses. |
DATABRICKS_SQL_QUERIES_GET | Read | Gets details about a SQL query. |
DATABRICKS_SQL_QUERY_HISTORY_LIST | Read | Lists SQL query history. |
DATABRICKS_SQL_STATEMENT_EXEC_CANCEL_EXEC | Write | Cancels a running SQL statement. |
DATABRICKS_SQL_STATEMENT_EXEC_EXECUTE_STATEMENT | Write | Executes a SQL statement in a SQL warehouse. |
DATABRICKS_SQL_STATEMENT_EXEC_GET_RESULT_CHUNK | Read | Gets a result chunk from a SQL statement. |
DATABRICKS_SQL_STATEMENT_EXEC_GET_STATEMENT | Read | Gets the status and result metadata for a SQL statement. |
DATABRICKS_SQL_WAREHOUSES_DELETE | Destructive | Deletes a SQL warehouse. |
DATABRICKS_SQL_WAREHOUSES_EDIT | Write | Updates a SQL warehouse configuration. |
DATABRICKS_SQL_WAREHOUSES_GET | Read | Gets details about a SQL warehouse. |
DATABRICKS_SQL_WAREHOUSES_START | Write | Starts a SQL warehouse. |
Jobs and Runs
| Tool | Access | Description |
|---|---|---|
DATABRICKS_CREATE_JOB | Write | Creates a Databricks job. |
DATABRICKS_JOBS_JOBS_CANCEL_ALL_RUNS | Write | Cancels all active runs for a job. |
DATABRICKS_JOBS_JOBS_CANCEL_RUN | Write | Cancels a job run. |
DATABRICKS_JOBS_JOBS_DELETE_RUN | Destructive | Deletes a job run record. |
DATABRICKS_JOBS_JOBS_GET | Read | Gets details about a job. |
DATABRICKS_JOBS_JOBS_SET_PERMISSIONS | Write | Sets access permissions for a job. |
DATABRICKS_JOB_RUN_BY_ID | Read | Gets job run details by run ID. |
DATABRICKS_LIST_JOBS | Read | Lists Databricks jobs. |
DATABRICKS_LIST_RUNS | Read | Lists Databricks job runs. |
DATABRICKS_SUBMIT_RUN | Write | Submits a one-time Databricks run. |
DATABRICKS_UPDATE_JOB_BY_ID | Write | Updates a job by job ID. |
Model Serving
| Tool | Access | Description |
|---|---|---|
DATABRICKS_LIST_MODEL_SERVING_ENDPOINTS | Read | Lists model serving endpoints. |
DATABRICKS_SERVING_SERVING_ENDPOINTS_CREATE_PROV_THPUT | Write | Creates a provisioned-throughput model serving endpoint. |
DATABRICKS_SERVING_SERVING_ENDPOINTS_DELETE | Destructive | Deletes a model serving endpoint. |
DATABRICKS_SERVING_SERVING_ENDPOINTS_GET | Read | Gets details about a model serving endpoint. |
Vector Search
| Tool | Access | Description |
|---|---|---|
DATABRICKS_LIST_VECTOR_SEARCH_ENDPOINTS | Read | Lists Vector Search endpoints. |
DATABRICKS_VECTORSEARCH_VECTOR_SEARCH_ENDPOINTS_CREATE | Write | Creates a Vector Search endpoint. |
DATABRICKS_VECTORSEARCH_VECTOR_SEARCH_INDEXES_DELETE_INDEX | Destructive | Deletes a Vector Search index. |
DATABRICKS_VECTORSEARCH_VECTOR_SEARCH_INDEXES_QUERY_INDEX | Read | Queries a Vector Search index. |
DATABRICKS_VECTORSEARCH_VECTOR_SEARCH_INDEXES_UPSERT_DATA | Write | Inserts or updates data in a Vector Search index. |
Important Notice
Tools that delete SQL warehouses, job runs, model serving endpoints, or Vector Search indexes can remove resources or operational history. Review the target resource before allowing these tools.
Use the Connector in a Chat
- In the chat input area, click the connector icon.
- Select the authorized Databricks connector.
- In the prompt, identify the external resource and operation that the agent should use.
- Review the generated agent definition and confirm that it contains only the required tools.
Permissions and Data Access
The connector uses the permissions of the authorized Databricks account. Limit that account to the required catalogs, warehouses, jobs, serving endpoints, and Vector Search resources.
Do not include external service credentials or access tokens in the chat prompt or agent instructions.
Limitations
- Databricks API quotas, workspace policies, and compute availability apply to connector operations.
- SQL statements require an available SQL warehouse and the required data permissions.
- Job, serving endpoint, and Vector Search operations can require additional workspace entitlements.
Troubleshooting
| Symptom | Resolution |
|---|---|
| The connector remains Unauthorized. | Verify that the Workspace URL has no trailing slash, the access token belongs to the same workspace and remains valid, and the token has the required permissions. Then retry the connection. |
| A tool returns a permission error. | Verify the authorized account's access to the target resource and operation. |
| A required tool is unavailable. | Verify that the tool is supported and approved for this connector, then select the connector again in the chat. |
| Requests are rejected intermittently. | Check the external service status, API quota, rate limits, and account plan. |