Private endpoints

Vespa Cloud lets you set up private endpoint services on your application clusters, to provide clients with safe, non-public access to the application. This is supported for AWS deployments through AWS's PrivateLink, and for GCP deployments through GCP's Private Service Connect. This guide takes you through the necessary configuration steps, for either AWS PrivateLink, or for GCP Private Service Connect.

AWS PrivateLink

Required information:

Procedure:

1. Configure a private endpoint service on your Vespa Cloud Container cluster

   Add <endpoint type="private" /> to deployment.xml, allowing access to the container cluster using the designated ARN from your account. The example allows all roles and users under the 123123123123 account to connect to the endpoint service on the my-container cluster, in each region listed under the <prod> tag.

   See endpoint service configuration for details on valid ARNs, and more fine-grained access control.

   The example also shows how to disable the public zone endpoint, by adding the "zone" type endpoint declaration—this is an optional step, and not required to set up the private service

   <deployment version="1.0">
       <prod>
           <region>region-1</region>
           <region>region-2</region>
       </prod>
       <endpoints>
           <endpoint type="zone" container-id="my-container" enabled="false" />
           <endpoint type="private" container-id="my-container">
               <allow with="aws-private-link" arn="arn:aws:iam::123123123123:root" />
           </endpoint>
       </endpoints>
   </deployment>

   Build and submit the application package, and wait for it to deploy to the indicated regions.

2. Find the service ID of your endpoint services

   Navigate to the endpoints tab for your application in the Console, and find the service ID for the deployment to wish to connect to. While there, verify that access to connect to the endpoint was granted to the correct ARNs.

   

3. Create the VPC interface endpoint

   Create a VPC endpoint in your VPC. This is the entry point for clients that need to connect to the endpoint service, which lets them connect to the Vespa cluster through the private network of AWS. For this example, assume your VPC has id vpc-123 and resides in the AWS region us-east-1, and that the service ID of your endpoint service, found in the Console, is com.amazonaws.vpce.us-east-1.vpce-svc-321:

   $ aws ec2 create-vpc-endpoint \
     --region us-east-1 \
     --vpc-id vpc-123 \
     --service-name com.amazonaws.vpce.us-east-1.vpce-svc-321 \
     --vpc-endpoint-type Interface \
     --private-dns-enabled | jq .
   

   Note the value of the VpcEndpointId field, for verification in the below item. This is also where you specify optional security group and subnet IDs; these are omitted here for brevity. If creating the VPC endpoint through the AWS console instead, be sure to check "Enable DNS names"!

4. Verify the VPC endpoint is connected to the Vespa cluster

   Navigate back to the endpoints tab in the Console, and refresh the page. You should now see a new entry representing the connection between your newly created interface endpoint, and the endpoint service on your container cluster. This is the "CONNECTED ENDPOINTS" in the image above. Verify the ID matches the value of the VpcEndpointId field above. The connection is ready when the state is open.

5. Verify your Vespa cluster is reachable from within your VPC

   The zone endpoint of the designated container cluster should now resolve through private DNS, for any AWS resource that is allowed to connect to your VPC endpoint. The easiest way to verify this is to run the following Python 3.9 lambda, using your own zone endpoint, from within your VPC:

   from socket import gethostbyname
   from urllib.request import urlopen
   
   def lambda_handler(event, context):
       return {
           'statusCode': 200,
           'body': urlopen('https://my-container.my-app.my-tenant.region-1.z.vespa-app.cloud/status.html').read(),
           'ip': gethostbyname('my-container.my-app.my-tenant.region-1.z.vespa-app.cloud')
       }
   

   Alternatively, run a couple of commands from a host inside the VPC:

   $ host my-container.my-app.my-tenant.region-1.z.vespa-app.cloud
   $ curl https://my-container.my-app.my-tenant.region-1.z.vespa-app.cloud/status.html
   

   In both cases, the IP should be in one of the private IP ranges, and the HTTP response from the Vespa container endpoint should be OK.

GCP Private Service Connect

Prerequisites:

$ gcloud services enable compute.googleapis.com
$ gcloud services enable dns.googleapis.com
$ gcloud services enable servicedirectory.googleapis.com

Procedure:

1. Configure a private endpoint service on your Vespa Cloud Container cluster

   Add <endpoint type="private"/> to deployment.xml, allowing access to the container cluster from the GCP account with the designated project ID. The example below allows consumer endpoints created under the private-test account to connect to the endpoint service on the my-container cluster, in each region listed under the <prod> tag.

   The example also shows how to disable the public zone endpoint, by adding the "zone" type endpoint declaration—this is an optional step, and not required to set up the private service

   <deployment version="1.0">
       <prod>
           <region>region-1</region>
           <region>region-2</region>
       </prod>
       <endpoints>
           <endpoint type="zone" container-id="my-container" enabled="false" />
           <endpoint type="private" container-id="my-container">
               <allow with="gcp-service-connect" project="private-test" />
           </endpoint>
       </endpoints>
   </deployment>

   Build and submit the application package, and wait for it to deploy to the indicated regions.

2. Find the service ID of your endpoint services

   Navigate to the endpoints tab for your application in the Console, and find the service ID for the deployment to wish to connect to. While there, verify that access to connect to the endpoint was granted to the correct projects.

   

3. Create the service consumer endpoint

   Create a service consumer endpoint in your VPC. This is the entry point for clients that need to connect to the endpoint service, which lets them connect to the Vespa cluster through the private network of GCP. For this example, assume your project is named test-project, with a VPC network named test-network that resides in the GCP region us-central1, with a subnet test-subnet to hold the endpoint, behind an address to be named test-address, and that the service ID of your endpoint service, found in the Console, is projects/vespa-external/regions/us-central1/serviceAttachments/scsa-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.

   Finally, this example uses the endpoint (forwarding rule) name test-endpoint, and the service directory namespace test-namespace; for production setup, we encourage using the container cluster name as the endpoint name, and the tenant-application-instance triplet as the namespace. The latter is to avoid conflict between endpoints for different applications, and the former to have GCP create private DNS entries for the endpoint that match the zone endpoints created by Vespa Cloud for public endpoints, with the caveat that the private endpoint name generated by GCP will include the leading default. segment when this is the cluster name, while the Vespa Cloud zone endpoint name does not.

   Create network:

   $ gcloud compute networks create test-network
   

   Create subnet:

   $ gcloud compute networks subnets create test-subnet \
       --region=us-central1 \
       --network=test-network \
       --range=10.10.0.0/24
   

   Creating the IP address which will be used by the endpoint, for clients inside your VPC:

   $ gcloud compute addresses create test-address \
       --region=us-central1 \
       --subnet=test-subnet
   

   Next, create a forwarding rule to forward traffic to that IP to the service endpoint in Vespa Cloud:

   $ gcloud compute forwarding-rules create test-endpoint \
       --region=us-central1 \
       --network=test-network \
       --address=test-address \
       --target-service-attachment=projects/vespa-external/regions/us-central1/serviceAttachments/scsa-ed2b1c74514321bf432a0a66735d3a19 \
       --service-directory-registration=projects/test-project/locations/us-central1/namespaces/test-namespace
   

   Note the ID of the created resource, for the verification step below.

   Consider some examples, both using the container ID for the endpoint name:

   • tenant: my-tenant, application: my-app, instance: my-instance, container ID: my-container; generated endpoint name: my-container.my-instance.my-app.my-tenant.us-central1-f.z.vespa-app.cloud
   • tenant: my-tenant, application: my-app, instance: my-instance, container ID: default; generated endpoint name: default.my-instance.my-app.my-tenant.us-central1-f.z.vespa-app.cloud
   • tenant: my-tenant, application: my-app, instance: default, container ID: my-container; generated endpoint name: my-container.my-app.my-tenant.us-central1-f.z.vespa-app.cloud
   • tenant: my-tenant, application: my-app, instance: default, container ID: default; generated endpoint name: default.my-app.my-tenant.us-central1-f.z.vespa-app.cloud

4. Verify the VPC endpoint is connected to the Vespa cluster

   Navigate back to the endpoints tab in the Console, and refresh the page. You should now see a new entry representing the connection between your newly created interface endpoint, and the endpoint service on your container cluster. This is the "CONNECTED ENDPOINTS" in the image above. Verify the ID matches the resource ID of the forwarding rule created above. The connection is ready when the state is open.

5. Verify your Vespa cluster is reachable from within your VPC

   The generated endpoint name (see two items up) of the designated container cluster should now resolve through private DNS, for any GCP resource that is allowed to connect to your VPC endpoint. The easiest way to verify this is to launch an instance in your VPC and run a couple of commands from it:

   $ host test-endpoint.my-instance.my-app.my-tenant.us-central1-f.z.vespa-app.cloud
   $ curl https://test-endpoint.my-instance.my-app.my-tenant.us-central1-f.z.vespa-app.cloud/status.html
   

   The resolved IP address should be in one of the private ranges, and the curl command should simply output OK.

   If the endpoint fails to resolve, refer to GCP's troubleshooting documentation.