Mirror with Harbor

Mirror Application Collection container images and Helm charts with Harbor

Application Collection provides all container images and Helm charts in a OCI-compliant registry that is available through the Internet. In certain cases, such as air-gapped environments, it might be desirable to access those images through a proxy registry, to have a better control of all the network requests involved in the deployment of container images and Helm charts within an organization.

In this guide, we will describe how this problem can be solved using Harbor.

Proxy registries with Harbor

Using Harbor as a proxy registry allows you to restrict all required network requests to Application Collection, except by those made by the Harbor instance to proxy the container images and Helm charts. This way, only the Harbor instance would require network access to Application Collection in an organization’s network.

Harbor supports two different methods to provide container images and Helm charts through a proxy registry:

• Proxy Cache registry: A Proxy Cache registry is a type of registry that caches all the OCI artifacts, namely, container images and Helm charts, from a target registry to the local Harbor instance in real-time.
• Replication rules: A replication rule allows you to replicate a set of OCI artifacts, namely, container images and Helm charts, from a source registry to a target registry (such as from Application Collection to the local Harbor instance).

Configure Harbor as a Proxy Cache of Application Collection

First, create a registry endpoint for Application Collection. In your Harbor instance, go to the sidebar and select Registries > New endpoint. Enter the following fields and click OK:

• Provider: Docker Registry
• Name: application-collection
• Endpoint URL: https://dp.apps.rancher.io
• Access ID: A service account username
• Access Secret: The service account authentication token
• Verify Remote Cert: Enabled
• Optionally, click Test Connection to verify the credentials

Then, go to the sidebar and select Projects > New Project. Enter the following fields:

• Project Name: application-collection. This value will determine the URI prefix under which the OCI artifacts will be available through the Harbor registry.
• Proxy Cache: Enabled. Select the application-collection registry that was created previously.

Harbor will now be configured to act as a Proxy Cache for Application Collection.

Pull a container image

To test the Proxy Cache configuration, you can pull a container image from Application Collection to verify that it works as expected.

For example, if your Harbor is living under the core.harbor.domain domain and the Harbor project’s name is application-collection, the following command would allow you to pull the container image dp.apps.rancher.io/containers/etcd:3.5.14-7.7:

docker pull core.harbor.domain/application-collection/containers/etcd:3.5.14-7.7

Deploy a Helm chart

You can also test the Proxy Cache configuration by deploying a Helm chart from Application Collection to verify that it works as expected.

For example, if your Harbor is living under the core.harbor.domain domain and the Harbor project’s name is application-collection, the following command would allow you to deploy the Application Collection etcd Helm chart from oci://core.harbor.domain/application-collection/charts/etcd:

helm install RELEASE-NAME oci://core.harbor.domain/application-collection/charts/etcd \
    --set global.imageRegistry=core.harbor.domain/application-collection \
    --set global.imagePullSecrets={application-collection}

Configure Harbor replication rules for Application Collection

Configuring a Harbor replication rule may be a costly operation if the source resource filter is not properly constrained, regarding disk and network usage. You may also encounter hit Application Collection’s limits with HTTP 429 Too Many Requests errors in this scenario.

First, create a registry endpoint for Application Collection. In your Harbor instance, go to the sidebar and select Registries > New endpoint. Enter the following fields and click OK:

• Provider: Docker Registry
• Name: application-collection
• Endpoint URL: https://dp.apps.rancher.io
• Access ID: A service account username
• Access Secret: The service account authentication token
• Verify Remote Cert: Enabled
• Optionally, click Test Connection to verify the credentials

Then, go to the sidebar and select Replications > New Replication Rule. Enter at least the following fields and click Save:

• Name: application-collection
• Replication mode: Pull-based
• Source registry: application-collection
• Source resource filter:
  • Name: For all OCI artifacts, enter **. For a specific OCI artifact, enter the path of the image, for example: containers/etcd for the container images of etcd, charts/etcd for the Helm charts, or **/etcd for both
  • Tag: For all tags, enter **. For a specific tag pattern, enter the pattern directly
• Destination:
  • Namespace: application-collection
  • Flattening: No Flatting

Once created, force a replication by selecting the newly created replication rule, and click Replicate. In Executions, you can view the progress and logs of the replication.

When the replication succeeds, the replicated OCI artifacts, namely, container images and Helm charts, will be available for use.

Pull a container image

To test the replication rules configurations, you can pull a replicated container image from Application Collection to verify that it works as expected.

For example, if your Harbor is living under the core.harbor.domain domain and the Harbor project’s name is application-collection, the following command would allow you to pull the container image core.harbor.domain/application-collection/containers/etcd:3.5.14-7.7:

docker pull core.harbor.domain/application-collection/containers/etcd:3.5.14-7.7

Deploy a Helm chart

You can also test the replication rules configuration by deploying a replicated Helm chart from Application Collection to verify that it works as expected.

For example, if your Harbor is living under the core.harbor.domain domain and the Harbor project’s name is application-collection, the following command would allow you to deploy the Application Collection etcd Helm chart from oci://core.harbor.domain/application-collection/charts/etcd:

helm install RELEASE-NAME oci://core.harbor.domain/application-collection/charts/etcd \
    --set global.imageRegistry=core.harbor.domain/application-collection \
    --set global.imagePullSecrets={application-collection}