deployment.xml

deployment.xml controls how your application is deployed.

deployment.xml is placed in the root of the application package and specifies which environments and regions the application is deployed to during automated application deployment, as which application instances.

Deployment progresses through the test and staging environments to the prod environments listed in deployment.xml.

Simple example:

<deployment version="1.0">
    <prod>
        <region>aws-us-east-1c</region>
        <region>aws-us-west-2a</region>
    </prod>
</deployment>

More complex example:

<deployment version="1.0">
    <instance id="beta">
        <prod>
            <region>aws-us-east-1c</region>
        </prod>
    </instance>
    <instance id="default">
        <block-change revision="false" days="mon,wed-fri" hours="16-23" time-zone="UTC" />
        <prod>
            <region>aws-us-east-1c</region>
            ...
            <delay hours="3" minutes="7" seconds="13" />
            <parallel>
                <region>aws-us-west-1c</region>
                <steps>
                    <region>aws-eu-west-1a</region>
                    <delay hours="3" />
                    <test>aws-us-west-2a</test>
                </steps>
                ...
            </parallel>
        </prod>
        <endpoints>
            <endpoint container-id="my-container-service">
                <region>aws-us-east-1c</region>
                ...
            </endpoint>
        </endpoints>
    </instance>
    <endpoints>
        <endpoint id="my-weighted-endpoint" container-id="my-container-service" region="aws-us-east-1c">
          <instance weight="1">beta</instance>
          ...
        </endpoint>
    </endpoints>
</deployment>

Some of the elements can be declared either under the <deployment> root, or, if one or more <instance> tags are listed, under these. These have a bold or when listing where they may be present.

deployment

The root element.

instance

In <deployment> or <parallel> (which must be a direct descendant of the root). An instance of the application; several of these may be simultaneously deployed in the same zone. If no <instance> is specified, all children of the root are implicitly children of an <instance> with id="default", as in the simple example at the top.

block-change

In <deployment>, or <instance>. This blocks changes from being deployed to production in the matching time interval. Changes are nevertheless tested while blocked.

By default, both application revision changes and Vespa platform changes (upgrades) are blocked. It is possible to block just one kind of change using the revision and version attributes.

Any combination of the attributes below can be specified. Changes on a given date will be blocked if all conditions are met. Invalid <block-change> tags (i.e. that contains conditions that never match an actual date) are rejected by the system.

This tag must be placed after any <test> and <staging> tags, and before <prod>. It can be declared multiple times.

The below example blocks all changes on weekends, and blocks revisions outside working hours, in the PST time zone:

<block-change days="sat-sun" hours="0-23" time-zone="America/Los_Angeles"/>
<block-change revision="false" days="mon-fri,sat,sun" hours="0-8,16-23" time-zone="America/Los_Angeles"/>

The below example blocks:

• all changes on Sundays starting on 2022-03-01
• all changes in the hours 16-23 between 2022-02-10 and 2022-02-15
• all changes until 2022-01-05

<block-change days="sun" from-date="2022-03-01" time-zone="America/Los_Angeles"/>
<block-change hours="16-23" from-date="2022-02-10" to-date="2022-02-15" time-zone="America/Los_Angeles"/>
<block-change to-date="2022-01-05" time-zone="America/Los_Angeles"/>

upgrade

In <deployment>, or <instance>. Determines the strategy for upgrading the application, or one of its instances. By default, application revision changes and Vespa platform changes are deployed separately. The exception is when an upgrade fails; then, the latest application revision is deployed together with the upgrade, as these may be necessary to fix the upgrade failure.

test

Meaning depends on where it is located:

staging

In <deployment>, or <instance>. If present, the application is deployed to the staging environment, and tested there, even if no prod zones are deployed to. If present in an <instance> element, staging tests are run for that specific instance before any production deployments of the instance may proceed — otherwise, previous staging tests for any instance are acceptable. When specified, staging tests must be preset in the application test package. See guides for getting to production.

prod

In <deployment>, or in <instance>. If present, the application is deployed to the production regions listed inside this element, under the specified instance, after deployments and tests in the test and staging environments.

region

In <prod>, <parallel>, <steps>, or <group>. The application is deployed to the production region with id contained in this element.

dev

In <deployment>. Optionally used to control deployment settings for the dev and perf environments. This can be used specify a different cloud account, tags, and private endpoints.

delay

In <deployment>, <instance>, <prod>, <parallel>, or <steps>. Introduces a delay which must pass after completion of all previous steps, before subsequent steps may proceed. This may be useful to allow some grace time to discover errors before deploying a change in additional zones, or to gather higher-level metrics for a production deployment for a while, before evaluating these in a production test. The maximum total delay for the whole deployment spec is 48 hours. The delay is specified by any combination of the hours, minutes and seconds attributes.

parallel

In <deployment>, <prod>, or <steps>. Runs the contained steps in parallel: instances if in <deployment>, or primitive steps (deployments, tests or delays) or a series of these (see steps) otherwise. Multiple <parallel> elements are permitted. The following example will deploy to us-west-1 first, then to us-east-3 and us-central-1 simultaneously, and, finally to eu-west-1, once both parallel deployments have completed:

<region>us-west-1</region>
<parallel>
    <region>us-east-3</region>
    <region>us-central-1</region>
</parallel>
<region>eu-west-1</region>

steps

In <parallel>. Runs the contained parallel or primitive steps (deployments, tests or delays) serially. The following example will in parallel:

1. deploy to us-east-3,
2. deploy to us-west-1, then delay 1 hour, and run tests for us-west-1, and
3. delay for two hours.

Thus, the parallel block is complete when both deployments are complete, tests are successful for the second deployment, and at least two hours have passed since the block began executing.

<parallel>
    <region>us-east-3</region>
    <steps>
        <region>us-west-1</region>
        <delay hours="1" />
        <test>us-west-1</test>
    </steps>
    <delay hours="2" />
</parallel>

tester

In <test>, <staging> and <prod>. Specifies container settings for the tester application container, which is used to run system, staging and production verification tests.

The allowed elements inside this are <nodes>.

<staging>
    <tester>
        <nodes count="1">
            <resources vcpu="8" memory="32Gb" disk="30Gb" />
        </nodes>
    </tester>
</staging>

endpoints (global)

In <deployment>, without any <instance> declared or in <instance>: This allows global endpoints, via one or more <endpoint> elements; and zone endpoint and private endpoint elements for cloud-native private network configuration.

endpoints (dev)

In <dev>. This allows zone endpoint and private endpoint elements for cloud-native private network configuration for dev and perf deployments.

endpoint (global)

In <endpoints> or <group>. Specifies a global endpoint for this application. Each endpoint will point to the regions that are declared in the endpoint. If no regions are specified, the endpoint defaults to the regions declared in the <prod> element. The following example creates a default endpoint to all regions, and a us endpoint pointing only to US regions.

<endpoints>
    <endpoint container-id="my-container-service"/>
    <endpoint id="us" container-id="my-container-service">
        <region>aws-us-east-1c</region>
        <region>aws-us-west-2a</region>
    </endpoint>
</endpoints>

endpoint (zone)

In <endpoints> or <group>, with type='zone'. Used to disable public zone endpoints. Non-public endpoints can not be used in global endpoints, which require that all constituent endpoints are public. The example disables the public zone endpoint for the my-container container cluster in all regions, except where it is explicitly enabled, in region-1. Changing endpoint visibility will make the service unavailable for a short period of time.

<endpoints>
    <endpoint type='zone' container-id='my-container' enabled='false' />
    <endpoint type='zone' container-id='my-container' enabled='true'>
        <region>region-1</region>
    </endpoint>
</endpoints>

endpoint (private)

In <endpoints> or <group>, with type='private'. Specifies a private endpoint service for this application. Each service will be launched in the regions that are declared in the endpoint. If no regions are specified, the service is launched in all regions declared in the <prod> element, that support any of the declared access types. The following example creates a private endpoint in two specific regions.

<endpoints>
    <endpoint type='private' container-id='my-container'>
        <region>aws-us-east-1c</region>
        <allow with='aws-private-link' arn='arn:aws:iam::123123123123:root' />
    </endpoint>
    <endpoint type='private' container-id='my-container'>
        <region>gcp-us-central1-f</region>
        <allow with='gcp-service-connect' project='user-project' />
    </endpoint>
</endpoints>

allow

In <endpoint type='private'>. Allows a prinicipal identified by the URN to set up a connection to the declared private endpoint service. This element must be repeated for each additional URN. An endpoint service will only consider allowed URNs of a compatible type, and will only be created if at least one compatible access type-and-URN is given:

• For AWS deployments, specify aws-private-link, and an ARN.
• For GCP deployments, specify gcp-service-connect, and a project ID

<endpoint type='private' container-id="my-container">
    <allow with='aws-private-link' arn='arn:aws:iam::123123123123:root' />
    <allow with='aws-private-link' arn='arn:aws:iam::321321321321:role/my-role' />
    <allow with='aws-private-link' arn='arn:aws:iam::321321321321:user/my-user' />
    <allow with='gcp-service-connect' project='my-project' />
</endpoint>

bcp

In <instance> or <deployment>. Defines the BCP structure of this instance: Which zones should take over for which others during the outage of a zone and how fast they must have the capacity ready. Autoscaling uses this information to decide the ideal cpu load of a zone. If this element is not defined, it is assumed that all regions covers for an equal share of the traffic of all other regions and must have that capacity ready at all times.

If a bcp element is specified at the root, and explicit instances are used, that bcp element becomes the default for all instances that does not contain a bcp element themselves. If a BCP element contains no group elements it will implicitly define a single group of all the regions of the instance in which it is used.

See BCP test for a procedure to verify that your BCP configuration is correct.

Example:

<bcp>
    <group deadline="15m">
        <endpoint id="foo" container-id="bar"/>
        <region>us-east1</region>
        <region>us-east2</region>
        <region fraction="0.5">us-central1</region>
    </group>
    <group>
        <region>us-west1</region>
        <region>us-west2</region>
        <region fraction="0.5">us-central1</region>
    </group>
</bcp>

group

In <bcp>. Defines a bcp group: A set of regions whose members cover for each other during a regional outage.

Each region in a group will (as allowed, when autoscaling ranges are configured) provision resources sufficient to handle that any other single region in the group goes down. The traffic of the region is assumed to be rerouted in equal amount to the remaining regions in the group. That is, if a group has one member, no resources will be provisioned to handle an outage in that member. If a group has two members, each will aim to provision sufficient resources to handle the actual traffic of the other. If a group has three members, each will provision to handle half of the traffic observed in the region among the two others which receives the most traffic.

A region may have fractional membership in multiple groups, meaning it will handle just that fraction of the traffic of the remaining members, and vice versa. A regions total membership among groups must always sum to exactly 1.

A group may also define global endpoints for the region members in the group. This is exactly the same as defining the endpoint separately and repeating the regions of the group under the endpoint. Endpoints under a group cannot contain explicit region sub-elements.