Vespa Cloud lets you set up private endpoint services on your application clusters,
for exclusive access from your own, co-located VPCs with the same cloud provider.
This is supported for AWS deployments through AWS's
PrivateLink, and for GCP deployments through GCP's
Private Service Connect.
This guide takes you through the necessary configuration steps, for either
AWS PrivateLink,
or for GCP Private Service Connect.
AWS PrivateLink
Required information:
Item
Description
Your IAM account number
The numeric identifier for your AWS account.
VPC ID
The identifier of your AWS VPC where you wish to connect to the service endpoints from.
AWS region name
The name of the AWS region to connect from. Note that you can only connect to a service in the same region,
or, if public endpoints are disabled, in the same AWS availability zone.
Procedure:
Configure a private endpoint service on your Vespa Cloud Container cluster
Add <endpoint type="private" />
to deployment.xml,
allowing access to the container cluster using the designated ARN from your account.
The example allows all roles and users under the 123123123123 account
to connect to the endpoint service on the my-container cluster,
in each region listed under the <prod> tag.
The example also shows how to disable the public zone endpoint, by adding the
"zone" type endpoint
declaration—this is an optional step, and not required to set up the private service
Build and deploy the application package, and wait for it to deploy to the indicated regions.
Find the service ID of your endpoint services
Navigate to the endpoints tab for your application in the Console,
and find the service ID for the deployment to wish to connect to.
While there, verify that access to connect to the endpoint was granted to the correct ARNs.
Create the VPC interface endpoint
Create a VPC endpoint
in your VPC. This is your entry point, which forwards connections to your Vespa application through the private network of AWS.
For this example, assume your VPC has id
vpc-123 and resides in the AWS region us-east-1, and that the service ID of your
endpoint service, found in the Console, is com.amazonaws.vpce.us-east-1.vpce-svc-321:
Note the value of the VpcEndpointId field, for verification in the below item.
This is also where you specify optional security group and subnet IDs; these are omitted here for brevity.
If creating the VPC endpoint through the AWS console instead, be sure to check "Enable DNS names"!
Verify the VPC endpoint is connected to the Vespa cluster
Navigate back to the endpoints tab in the Console, and refresh the page.
You should now see a new entry representing the connection between your newly created interface endpoint,
and the endpoint service on your container cluster.
This is the "CONNECTED ENDPOINTS" in the image above.
Verify the ID matches the value of the VpcEndpointId field above.
The connection is ready when the state is open.
Verify your Vespa cluster is reachable from within your VPC
The zone endpoint of the designated container cluster should now resolve through private DNS, for any AWS resource
that is allowed to connect to your VPC endpoint.
The easiest way to verify this is to run the following Python 3.9 lambda,
using your own zone endpoint, from within your VPC:
from socket import gethostbyname
from urllib.request import urlopen
def lambda_handler(event, context):
return {
'statusCode': 200,
'body': urlopen('https://badc0ffee.deadbeef.z.vespa-app.cloud/status.html').read(),
'ip': gethostbyname('badc0ffee.deadbeef.z.vespa-app.cloud')
}
Alternatively, run a couple of commands from a host inside the VPC:
The string identifier for your GCP account, like resonant-diode-123456
VPC network and subnetwork names
The name of the network and subnetwork to create your consumer endpoint in.
Procedure:
Configure a private endpoint service on your Vespa Cloud Container cluster
Add <endpoint type="private"/>
to deployment.xml,
allowing access to the container cluster from the GCP account with the designated project ID.
The example below allows consumer endpoints created under the private-test account
to connect to the endpoint service on the my-container cluster,
in each region listed under the <prod> tag.
The example also shows how to disable the public zone endpoint, by adding the
"zone" type endpoint
declaration—this is an optional step, and not required to set up the private service
Build and deploy the application package, and wait for it to deploy to the indicated regions.
Find the service ID of your endpoint services
Navigate to the endpoints tab for your application in the Console,
and find the service ID for the deployment to wish to connect to.
While there, verify that access to connect to the endpoint was granted to the correct projects.
Create the service consumer endpoint
Create a service consumer endpoint
in your VPC. This is your entry point, which forwards connections to your Vespa application through the private network of GCP.
In this example the project is named
test-project, has a VPC network named
test-network that resides in the GCP region us-central1, with a subnet test-subnet
to hold the endpoint, behind an address to be named test-address, and the service ID of the
endpoint service (found in the Console) is
projects/vespa-external/regions/us-central1/serviceAttachments/scsa-xxxxxx.
Finally, the endpoint is named badc0ffee, and the service directory namespace is my-tenant-my-app.
See the discussion on generated endpoint names in the last item in this guide.
Note the ID of the created resource, for the verification step below.
Verify the VPC endpoint is connected to the Vespa cluster
Navigate back to the endpoints tab in the Console, and refresh the page.
You should now see a new entry representing the connection between your newly created interface endpoint,
and the endpoint service on your container cluster.
This is the "CONNECTED ENDPOINTS" in the image above.
Verify the ID matches the resource ID of the forwarding rule created above.
The connection is ready when the state is open.
Verify your Vespa cluster is reachable from within your VPC
The generated endpoint name (see last items) of the designated container cluster should now resolve
through private DNS inside your VPC.
The easiest way to verify this is to launch an instance in your VPC, inside the designated subnet,
and run a couple of commands from it:
When a consumer endpoint is created with a Service Directory namespace, GCP automatically creates
a private DNS record for that endpoint, which must be used instead of the IP address (created above) of the endpoint,
as Vespa application containers have web certificates matching specific domain names.
Unfortunately, we are unable to set the final endpoint names for the consumer endpoint;
for a private endpoint service, we can only set a domain name suffix, and GCP then generates private
DNS records matching your endpoint resource name prepended to this suffix.
The Service Directory namespace of these endpoints must also be one-to-one with their domain name suffixes,
lest the automatic setup fail.
The domain name suffix used by Vespa Cloud is
[<tenant-and-app-anonymous-id>.].z.vespa-app.cloud. We therefore encourage using the
<tenant>-<application> pair as the service directory namespace,
as this ensures a one-to-one mapping between suffixes and namespaces, as required by GCP (see above).
The Vespa Cloud web certificates (see above) match any direct descendant of the domain suffix we set for your services;
thus, any endpoint resource name yields a private DNS record that matches the web certificate, with Service Directory.
Moreover, the zone endpoints generated by Vespa Cloud consist of a random, unique cluster-instance-region id.
Using this same ID as the GCP endpoint resource name (as in the example) results in identical domain names for
the private DNS set up by GCP, and the endpoint names generated by Vespa Cloud, visible in our console.