Configure Egress Private Endpoints

On this page

Establish a secure network connection from a CockroachDB Advanced cluster to send changefeeds to your private cloud infrastructure with egress private endpoints. You can use the CockroachDB Cloud API to create and manage egress private endpoints on a CockroachDB Advanced cluster.

CockroachDB Cloud supports egress private endpoints with the following cloud services:

• Amazon Virtual Private Cloud (AWS VPC)
• Amazon Managed Streaming for Apache Kafka (MSK)
• Google Cloud Virtual Private Cloud (GCP VPC)
• Confluent Cloud

Prerequisites

Refer to the following sections for prerequisites that apply to the corresponding cloud service:

AWS VPC endpoints

The CockroachDB Cloud AWS account must be added as a principal on the endpoint service. Using your CockroachDB Cloud account_id, add the arn:aws:iam::<CC_ACCOUNT_ID>:root principal to the endpoint service definition.

You can use the following API call to retrieve your CockroachDB Cloud account_id:

curl --request GET \
  --url https://cockroachlabs.cloud/api/v1/clusters/{cluster_id} \
  --header 'Authorization: Bearer {secret_key}' | jq .account_id

MSK endpoints

The following prerequisites apply to the MSK service:

• The cluster must not use kafka.t3.small instances.
• If the cluster is not using IAM authentication, set the allow.everyone.if.no.acl.found=false ACL.
• Multi-VPC Connectivity must be enabled.

• Using the account_id returned from the GET /api/v1/clusters/{cluster_id}, include the following in the cluster policy:

  {
    "Effect": "Allow",
    "Principal": {
      "AWS": "arn:aws:iam::<CC_ACCOUNT_ID>:root"
    },
    "Action": [
      "kafka:CreateVpcConnection",
      "kafka:GetBootstrapBrokers",
      "kafka:DescribeCluster",
      "kafka:DescribeClusterV2"
    ],
    "Resource": "<CUSTOMER_MSK_CLUSTER_ARN>"
  }
  

Confluent Cloud endpoints

You can configure egress private endpoints to connect to an AWS or GCP private service configured in a Confluent account. Endpoint creation follows the same process and syntax as for AWS or GCP VPCs.

Confluent Cloud requires a custom DNS configuration due to the TLS certificates provisioned for their Kafka clusters. Collect the required domain names from Confluent. After the endpoint is created, configure custom DNS records for the cluster.

Create an egress private endpoint

A user with the Cluster Admin, Cluster Operator, or Cluster Creator roles can create an egress private endpoint by sending a POST request to the /api/v1/clusters/{cluster_id}/networking/egress-private-endpoints endpoint with the following payload information:

• cluster_id: The CockroachDB Cloud cluster ID.
• region: The region code where the target service identifier is located (e.g. us-east-1).
• target_service_identifier: The unique identifier of the target service, which is a different value depending on the service:
  • AWS VPC: The AWS private service name.
  • MSK: The MSK-provisioned cluster's Amazon Resource Name (ARN).
  • GCP VPC: The GCP service attachment.
• target_service_type: Description of the service type, dependent on the service and authentication method:
  • AWS VPC or GCP VPC: Set to PRIVATE_SERVICE.
  • MSK with SASL/SCRAM authentication: Set to MSK_SASL_SCRAM.
  • MSK with IAM access control: Set to MSK_SASL_IAM.
  • MSK with mutual TLS authentication: Set to MSK_TLS.

Once this request is sent, the CockroachDB Cloud cluster enters a maintenance mode where other configuration changes (cluster scaling, feature configuration, upgrades, etc) cannot be made until the operation is complete. The operation is complete when the endpoint status is AVAILABLE and both the endpoint_id and endpoint_address fields are populated.

Example endpoint creation requests

The following example POST requests assume that an API key has been created for a user with the appropriate role, such as Cluster Operator:

AWS private service endpoint

curl https://management-staging.crdb.io/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints \
-X POST \
-H 'Authorization: Bearer {secret_key}' \
-H 'Content-Type: application/json' \
-d '{
  "cluster_id": "{cluster_id}",
  "region": "us-east-1",
  "target_service_identifier": "com.amazonaws.vpce.us-east-1.vpce-svc-example",
  "target_service_type": "PRIVATE_SERVICE"
}'

MSK cluster endpoint

curl https://management-staging.crdb.io/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints \
-X POST \
-H 'Authorization: Bearer {secret_key}' \
-H 'Content-Type: application/json' \
-d '{
  "cluster_id": "{cluster_id}",
  "region": "us-east-2",
  "target_service_identifier": "arn:aws:kafka:us-east-2:example:cluster/msk-example/99bcd320-3af1-42cc-b8cc-example-7",
  "target_service_type": "MSK_SASL_SCRAM"
}'

GCP private service endpoint

curl https://management-staging.crdb.io/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints \
-X POST \
-H 'Authorization: Bearer {secret_key}' \
-H 'Content-Type: application/json' \
-d '{
  "cluster_id": "{cluster_id}",
  "region": "us-east1",
  "target_service_identifier": "projects/cc-example/regions/us-east1/serviceAttachments/s-example-service-attachment-us-east1-d",
  "target_service_type": "PRIVATE_SERVICE"
}'

Example response

{
    "id": "{endpoint_id}",
    "endpoint_connection_id": "",
    "region": "us-east-2",
    "target_service_identifier": "com.amazonaws.vpce.us-east-2.vpce-svc-example",
    "target_service_type": "PRIVATE_SERVICE",
    "endpoint_address": ""
}

Configure custom DNS

If the cloud service has a TLS certificate that requires traffic to be sent from a specific domain, such as Confluent Cloud, you can use the Cloud API to create custom DNS records for a CockroachDB Cloud cluster that points to the endpoint_address of an egress private endpoint.

Before creating a custom DNS record, check that the endpoint status is in the AVAILABLE state. Save the egress_private_endpoints.id value for later use. This ID is distinct from the endpoint_connection_id, which is an external identifier.

Send a PATCH request to the /api/v1/clusters/{cluster_id}/networking/egress-private-endpoints/{endpoint_id}/domain-names endpoint with the following payload information:

• cluster_id: The CockroachDB Cloud cluster ID.
• endpoint_id: The id value of the egress private endpoint.
• domain_names: A list of domain names to resolve to the private endpoint, as required by the cloud service provider.

For example:

curl https://management-staging.crdb.io/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints/{endpoint_id}/domain-names \
-X PATCH \
-H 'Authorization: Bearer {secret_key}' \
-H 'Content-Type: application/json' \
-d '{
  "cluster_id": "{cluster_id}",
  "endpoint_id": "{endpoint_id}",
  "domain_names": ["*.us-east-2.aws.confluent.cloud"]
}'

The cluster enters maintenance mode once more until the DNS setup is complete, which may take a minute or longer. Traffic from the CockroachDB Cloud cluster should now be routed appropriately to the private endpoint.

Check the endpoint status

Send a GET request to the /api/v1/clusters/{cluster_id}/networking/egress-private-endpoints endpoint to review the status of all egress private endpoints on the cluster:

curl https://management-staging.crdb.io/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints \
-H "Authorization: Bearer {secret_key}"

The response lists all egress private endpoints on the cluster. The state field indicates the status of each endpoint:

{
    "egress_private_endpoints": [
        {
            "id": "{endpoint_id}",
            "endpoint_connection_id": "vpce-0460d7c25b1f505dd",
            "region": "us-east-2",
            "target_service_identifier": "com.amazonaws.vpce.us-east-2.vpce-svc-0fb8e1b95f0ade981",
            "target_service_type": "PRIVATE_SERVICE",
            "endpoint_address": "vpce-0460d7c25b1f505dd-onq0bw5q.vpce-svc-0fb8e1b95f0ade981.us-east-2.vpce.amazonaws.com",
            "state": "AVAILABLE"
        }
    ],
    "pagination": null
}

The following list describes all of the possible state values and their meanings:

• PENDING: The endpoint is in the process of being created.
• PENDING_ACCEPTANCE: The endpoint needs to be manually accepted on the cloud provider service.
• AVAILABLE: The endpoint has been created and is available for traffic.
• DELETING: The endpoint is in the process of being deleted from the cluster.
• REJECTED: The endpoint connection was rejected on the cloud provider side.
• FAILED: Something went wrong with the creation of the endpoint in the cloud provider.
• EXPIRED: The connection request expired before it was accepted on the cloud provider service.
• UNSPECIFIED: Could not determine the current state of the endpoint.

Delete a private endpoint

To delete a private endpoint, send a DELETE request to the /api/v1/clusters/{cluster_id}/networking/egress-private-endpoints/{endpoint_id} endpoint specifying the following information:

• cluster_id: The CockroachDB Cloud cluster ID.
• endpoint_id: The id value of the egress private endpoint.

curl https://management-staging.crdb.io/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints/{endpoint_id} \
-X DELETE \
-H "Authorization: Bearer {secret_key}"

The endpoint briefly enters the DELETING state then is removed from the list of endpoints on the cluster.