/**
@page guide_developer_automated_tests Automated Tests
@brief Guide to this SDK's automated testing setup.

The SDK's GitHub repo relies on the following tools to provide automated testing of commits and pull requests.
- [Travis CI](https://travis-ci.org/) is the primary service used for testing. <br>
  <i>See:</i> [aws-iot-device-sdk-embedded-C on Travis CI](https://travis-ci.org/aws/aws-iot-device-sdk-embedded-C)
- [Codecov](https://codecov.io/) provides code coverage results. <br>
  <i>See</i>: [aws-iot-device-sdk-embedded-C on Codecov](https://codecov.io/gh/aws/aws-iot-device-sdk-embedded-C)

The following files on the GitHub repo control the automated tests:
- `.travis.yml` <br>
  Provides the job matrix to run on Travis CI.
- `scripts/` directory <br>
  Shell scripts (`sh`) that run on Travis CI servers.
  - The scripts that test the libraries are named after the library they test. For example, `ci_test_mqtt.sh` tests the MQTT library.
  - `ci_test_doc.sh` tests the documentation build. Runs for pull requests only.
  - `ci_test_coverage.sh` generates and submits coverage results to Codecov. Runs for commits only.

@section guide_developer_automated_tests_setup Setup
@brief How to set up the automated testing services.

All of the automated testing services have web interfaces that can be connected to a GitHub account.

@subsection guide_developer_automated_tests_setup_travis Travis CI
@brief How to set up Travis CI.

<b>Link to Travis CI:</b> https://travis-ci.org/

1. Click <i>Sign in with GitHub</i> in the upper right corner to log in with your GitHub account. On your first login, Travis CI will prompt for GitHub permissions access.
2. Once signed in, hover over your GitHub account in the upper right corner. Click <i>Settings</i> in the menu that pops up.
3. On the <i>Settings</i> page, toggle the switch next to your fork of the C SDK to <i>On</i>.
4. Travis CI is now set up. See @ref guide_developer_automated_tests_credentials_policy_travis for adding commit credentials to Travis CI.

@subsection guide_developer_automated_tests_setup_codecov Codecov
@brief How to set up Codecov.

<b>Link to Codecov:</b> https://codecov.io/

1. Click <i>Log In</i> in the upper right corner.  Then select <i>Log in with GitHub</i> to log in with your GitHub account. On your first login, Codecov will prompt for GitHub permissions access.
2. Navigate to codecov.io/gh/<b>your GitHub username</b>/aws-iot-device-sdk-embedded-C
3. Click on <i>Settings</i>, then copy the <i>Repository Upload Token</i> displayed.
4. Set the Travis CI environment variable `CODECOV_TOKEN` to the Repository Upload Token. Travis CI will automatically submit coverage data on commit check builds.

@section guide_developer_automated_tests_commit_pr Commit vs. Pull Request
@brief The automated tests distinguish between GitHub commits and pull requests. Different sets of tests run on commits and pull requests.

Because pull requests come from other repos and contain unknown code, Travis CI will not provide encrypted environment variables for pull requests. Therefore, pull requests are tested against unsecured servers. The following tests are run for pull requests.
- Common (`ci_test_common.sh`): this script tests common components like the task pool, atomics, etc. These tests do not use the network.
- MQTT (`ci_test_mqtt.sh`): the MQTT tests run on an unsecured connection against a locally-installed Mosquitto broker on the Travis CI VM.
- Shadow (`ci_test_shadow.sh`): only the Shadow unit tests are run for a pull request build. These tests do not use the network.
- Documentation (`ci_test_doc.sh`): checks that the documentation builds against the code in the pull request.

For commits, the credentials necessary for connecting to AWS IoT are provided by Travis CI. The following tests are run for commits.
- Common (`ci_test_common.sh`): runs the same tests as the common pull request tests.
- MQTT (`ci_test_mqtt.sh`): runs the same MQTT tests as the MQTT pull request tests, but against the AWS IoT MQTT broker instead of a Mosquitto broker.
- Shadow (`ci_test_shadow.sh`): runs both the Shadow unit tests and the Shadow system tests. Requires connection to AWS IoT.
- Coverage (`ci_test_coverage.sh`): Gathers code coverage data and submits it to Codecov.

@note Rarely, the Shadow tests on a GitHub fork will fail with a `429 TOO MANY REQUESTS` error. If this happens, restart the Shadow tests. The main C SDK GitHub repo has an increased limit for Shadow requests, so it should not encounter this error.

Because different sets of tests are run for commits and pull requests, both commit and pull request checks should be used to validate code changes. The commit checks can be set up to run on a GitHub fork, where they will run whenever a new commit is pushed to the fork. When a pull request is made from the fork to the main repo, the pull request checks will run.

@section guide_developer_automated_tests_credentials_policy AWS Credentials and Policy
@brief How to set up AWS IoT credentials for commit checks.

Forking the main GitHub repo copies the Travis CI scripts, but does not copy any credentials, which are provided by Travis CI's [encrypted environment variable](https://docs.travis-ci.com/user/environment-variables/) feature. AWS IoT credentials are required to run the commit checks. No credentials are required to run pull request checks.

@subsection guide_developer_automated_tests_credentials_policy_formatting Formatting the client certificate and private key
@brief The AWS IoT client certificate and private key must be formatted to be acceptable for use in a CI script.

The credentials are written from Travis encrypted environment variables using the following command:
```
echo -e $AWS_IOT_CLIENT_CERT > credentials/clientCert.pem
```

Therefore, all given credentials must be formatted for `echo -e`. As an example, consider the following certificate:
```
-----BEGIN CERTIFICATE-----
MIIB8TCCAZugAwIBAwIBADANBgkqhkiG9w0BAQQFADBfMRMwEQYDVQQDEwpuZXRh
cHAuY29tMQswCQYDVQQGEwJVUzEJMAcGA1UECBMAMQkwBwYDVQQHEwAxCTAHBgNV
BAoTADEJMAcGA1UECxMAMQ8wDQYJKoZIhvcNAQkBFgAwHhcNMTAwNDI2MTk0OTI4
WhcNMTAwNTI2MTk0OTI4WjBfMRMwEQYDVQQDEwpuZXRhcHAuY29tMQswCQYDVQQG
EwJVUzEJMAcGA1UECBMAMQkwBwYDVQQHEwAxCTAHBgNVBAoTADEJMAcGA1UECxMA
MQ8wDQYJKoZIhvcNAQkBFgAwXDANBgkqhkiG9w0BAQEFAANLADBIAkEAyXrK2sry
-----END CERTIFICATE-----
```

It should be formatted as follows for Travis CI:
```
-----BEGIN\ CERTIFICATE-----\\nMIIB8TCCAZugAwIBAwIBADANBgkqhkiG9w0BAQQFADBfMRMwEQYDVQQDEwpuZXRh\\ncHAuY29tMQswCQYDVQQGEwJVUzEJMAcGA1UECBMAMQkwBwYDVQQHEwAxCTAHBgNV\\nBAoTADEJMAcGA1UECxMAMQ8wDQYJKoZIhvcNAQkBFgAwHhcNMTAwNDI2MTk0OTI4\\nWhcNMTAwNTI2MTk0OTI4WjBfMRMwEQYDVQQDEwpuZXRhcHAuY29tMQswCQYDVQQG\\nEwJVUzEJMAcGA1UECBMAMQkwBwYDVQQHEwAxCTAHBgNVBAoTADEJMAcGA1UECxMA\\nMQ8wDQYJKoZIhvcNAQkBFgAwXDANBgkqhkiG9w0BAQEFAANLADBIAkEAyXrK2sry\\n-----END\ CERTIFICATE-----
```

Note the spaces have an escape character `\` before them, as in `-----BEGIN\ CERTIFICATE-----`. Line breaks have been removed and replaced with `\\n`.

Similar formatting should be applied to the matching private key.

@subsection guide_developer_automated_tests_credentials_policy_travis Setting the credentials on Travis CI
@brief The AWS IoT endpoint, client certificate, and client certificate private key are set on Travis CI's "Settings" page.

To edit the credentials, go to the "Settings" page on Travis CI. This is usually `<repo URL>/settings`; for example, `https://travis-ci.org/aws/aws-iot-device-sdk-embedded-C/settings` for the main repo. Under "Environment Variables", set the following variables:
| Name                  | Value | Notes |
| --------------------- | ----- | ----- |
| AWS_IOT_CLIENT_CERT   | [Formatted client certificate](@ref guide_developer_automated_tests_credentials_policy_formatting) | The AWS IoT client certificate. <br> This value does not need to be hidden in the build log. |
| AWS_IOT_ENDPOINT      | `ABCDEFG1234567.iot.<region>.amazonaws.com` | The AWS IoT endpoint. <br> This value does not need to be hidden in the build log. |
| AWS_IOT_PRIVATE_KEY   | [Formatted private key](@ref guide_developer_automated_tests_credentials_policy_formatting) | The private key matching the AWS IoT client certificate. @attention <b>This value should be hidden in the build log!</b> |
| IOT_IDENTIFIER_PREFIX | `CSDKCI` | This string is prepended to all MQTT client identifiers used by the CI. <br> It may be anything, but must match the client identifier in the AWS IoT policy. <br> The example policy below uses `CSDKCI`. <br> This value does not need to be hidden in the build log. |

@subsection guide_developer_automated_tests_credentials_policy_aws Setting the AWS IoT policy
@brief The certificates used with Travis CI must have an appropriate policy attached.

AWS recommends using the most restrictive policy possible. The following policy should be used for the CI.
@code{json}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "iot:Connect",
      "Resource": [
        "arn:aws:iot:<region>:<account>:client/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "iot:Publish",
        "iot:Receive"
      ],
      "Resource": [
        "arn:aws:iot:<region>:<account>:topic/${iot:ClientId}/*",
        "arn:aws:iot:<region>:<account>:topic/*/LastWillAndTestament",
        "arn:aws:iot:<region>:<account>:topic/$aws/things/${iot:ClientId}/shadow/*",
        "arn:aws:iot:<region>:<account>:topic/$aws/things/${iot:ClientId}/jobs/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": "iot:Subscribe",
      "Resource": [
        "arn:aws:iot:<region>:<account>:topicfilter/${iot:ClientId}/*",
        "arn:aws:iot:<region>:<account>:topicfilter/*/LastWillAndTestament",
        "arn:aws:iot:<region>:<account>:topicfilter/$aws/things/${iot:ClientId}/shadow/*",
        "arn:aws:iot:<region>:<account>:topicfilter/$aws/things/${iot:ClientId}/jobs/*"
      ]
    }
  ]
}
@endcode

This policy only allows the CI to interact with topics that contain a matching client identifier (which is prefixed with the Travis CI variable `IOT_IDENTIFIER_PREFIX`). A few special topics used by the MQTT tests are also present. The policy will need to be updated if any new topics are used by the tests in the future.

@section guide_developer_automated_tests_jobs_and_provisioning Additional Travis CI setup for Jobs and Provisioning system tests
@brief Additional CI settings for running the [Jobs system tests](@ref jobs_tests) and [Provisioning system tests](@ref provisioning_tests).

The Jobs system tests use AWS CLI to create and delete Jobs. Therefore, additional setup is needed in Travis CI.

@pre See @ref jobs_system_tests_setup first to set up an AWS account for the Jobs system tests. Note that the coverage tests require an additional Thing to be registered (but can reuse the same AWS credentials).

Under the "Environment Variables" section of the Travis CI "Settings" page, set the following additional environment variables.
| Name                  | Value | Notes |
| --------------------- | ----- | ----- |
| AWS_ACCESS_KEY_ID   | The AWS access key for the Jobs tests IAM user | @attention <b>This value should be hidden in the build log!</b> |
| AWS_DEFAULT_REGION      | The AWS region to use with the Jobs tests | Should be the region where the Jobs test Thing is registered. <br> This value does not need to be hidden in the build log. |
| AWS_SECRET_ACCESS_KEY   | The AWS secret access key for the Jobs tests IAM user | @attention <b>This value should be hidden in the build log!</b> |
*/
