album-recommendation-searcher

This sample app adds custom java plugin code, system and staging tests, and is an introduction into integrating with a CI/CD pipeline. It extends the Quick Start with:

A Searcher component for query and result processing.
- See album-recommendation-docproc for a Document Processing example.
Custom config for the searcher, to demonstrate atomic code and config deployment.
A System and staging test used in automated-deployments.

This sample application is great to deploy to both the Dev and Prod Environments in the Free Trial - experience how easy it is to develop and application using Vespa Cloud, and deploy for production serving.

Prerequisites: git, Java 11, mvn 3.6.1 and openssl. See Getting Started for troubleshooting. Security notes:

To deploy, a user API key is required, see step 4.
To read and write to the instance's endpoint, a certificate and a private key is required, see step 2. Find more details in Data Plane, see "Client certificate".

Dev Environment

Download sample apps:

$ git clone https://github.com/vespa-engine/sample-apps.git && \
  cd sample-apps/vespa-cloud/album-recommendation-searcher

Get a X.509 certificate and private key. Create a self-signed certificate / private key:

$ openssl req -x509 -nodes -days 14 -newkey rsa:4096 \
  -subj "/CN=cloud.vespa.example" \
  -keyout data-plane-private-key.pem -out data-plane-public-cert.pem

Add certificate to application package (it must be copied as clients.pem):

$ mkdir -p src/main/application/security && \
  cp data-plane-public-cert.pem src/main/application/security/clients.pem

At console.vespa.ai, choose tenant and click Keys to generate and save the user API key. The key is saved to $HOME/Downloads/USER.TENANTNAME.pem. Then click "Create application" - use any name for the application.
Update tenant and application in pom.xml — use the values entered in the console in 4.
Build the app, deploy your own instance (my-instance here) to the dev environment and wait for it to start — update apiKeyFile to be the file downloaded above:
```
$ mvn clean package vespa:deploy -DapiKeyFile=$HOME/Downloads/USER.TENANTNAME.pem -Dinstance=my-instance
```

Alternatively, put the key in an environment variable:

$ API_KEY=`echo $VESPA_TEAM_API_KEY | openssl base64 -A -a -d`
$ mvn clean package vespa:deploy -DapiKey="$API_KEY" -Dinstance=my-instance

Now is a good time to read automated-deployments, as first time deployments takes a few minutes. The endpoint URL is printed in the Install application section when the deployment is successful - store this (change root to actual name) for later steps and test it (the massive JSON output is expected). Also see using a browser:
```
$ ENDPOINT=https://root.album-rec-searcher.vespa-team.aws-us-east-1c.dev.public.vespa.oath.cloud
$ curl --cert ./data-plane-public-cert.pem --key ./data-plane-private-key.pem $ENDPOINT
```

Feed documents:

$ curl --cert ./data-plane-public-cert.pem --key ./data-plane-private-key.pem \
  -H "Content-Type:application/json" --data-binary @src/test/resources/A-Head-Full-of-Dreams.json \
  $ENDPOINT/document/v1/mynamespace/music/docid/1
$ curl --cert ./data-plane-public-cert.pem --key ./data-plane-private-key.pem \
  -H "Content-Type:application/json" --data-binary @src/test/resources/Love-Is-Here-To-Stay.json \
  $ENDPOINT/document/v1/mynamespace/music/docid/2
$ curl --cert ./data-plane-public-cert.pem --key ./data-plane-private-key.pem \
  -H "Content-Type:application/json" --data-binary @src/test/resources/Hardwired...To-Self-Destruct.json \
  $ENDPOINT/document/v1/mynamespace/music/docid/3

Dump all document using visit:

$ curl --cert ./data-plane-public-cert.pem --key ./data-plane-private-key.pem \
  "$ENDPOINT/document/v1/mynamespace/music/docid?wantedDocumentCount=100"

Recommend albums, send user profile in query:

$ curl --cert ./data-plane-public-cert.pem --key ./data-plane-private-key.pem \
  "$ENDPOINT/search/?ranking=rank_albums&yql=select%20%2A%20from%20sources%20%2A%20where%20sddocname%20contains%20%22music%22%3B&ranking.features.query(user_profile)=%7B%7Bcat%3Apop%7D%3A0.8%2C%7Bcat%3Arock%7D%3A0.2%2C%7Bcat%3Ajazz%7D%3A0.1%7D"

Limit to albums with the term "to" in title:

$ curl --cert ./data-plane-public-cert.pem --key ./data-plane-private-key.pem \
  "$ENDPOINT/search/?ranking=rank_albums&yql=select%20%2A%20from%20sources%20%2A%20where%20album%20contains%20%22to%22%3B&ranking.features.query(user_profile)=%7B%7Bcat%3Apop%7D%3A0.8%2C%7Bcat%3Arock%7D%3A0.2%2C%7Bcat%3Ajazz%7D%3A0.1%7D"

Prod Environment

Vespa Cloud has support for running system and staging tests, used in automated-deployments. These tests are run as JUnit tests. To develop such tests, deploy the application to dev (like above) and run tests like FeedAndSearchSystemTest:
```
$ mvn test \
  -Dtest.categories=system \
  -DdataPlaneKeyFile=data-plane-private-key.pem -DdataPlaneCertificateFile=data-plane-public-cert.pem \
  -DapiKey="$API_KEY"
```
The apiKey is used to fetch the dev instance's endpoints. The data plane key and certificate pair is used by ai.vespa.hosted.cd.Endpoint to access the application endpoint. Find more details in testing and Vespa Cloud API.
To run tests against a deployment running in Docker on localhost (instead of using dev), configure endpoint location:
```
{
    "localEndpoints": {
        "container": "http://localhost:8080/"
    }
}
```
See system tests for details. Refer to album-recommendation-selfhosted for how to create the application package.

When system and staging tests are ready, deploy to production:

$ mvn vespa:compileVersion -DapiKey="$API_KEY"
$ mvn -Dvespa.compile.version="$(cat target/vespa.compile.version)" \
  -DapiKey="$API_KEY" \
  -DsourceUrl="https://github.com/vespa-engine/sample-apps/tree/$(git rev-parse HEAD)/vespa-cloud/album-recommendation-searcher" \
  package vespa:submit