DGCLI

The DGCLI utility (also known as dgCtl) is used for getting license file, downloading, and updating the artifacts required for deploying and updating the On-Premise services.

Installation artifacts include:

• Docker images of services.
• additional data required for the services to work.

To learn more about the life cycle of installation artifacts, see here.

Architecture

DGCLI allows you to split the process of obtaining the installation artifacts into two steps:

1. Downloading the artifacts from the Internet (the pull mode)
2. Deploying services using the downloaded artifacts (the restore mode)

The artifacts can be stored in an S3-compatible storage or in a file system (see the storage.type setting of the configuration file).

It is recommended to separate downloaded Docker images from additional service data by using the --apps-to-registry argument (see the list of command-line arguments). In this case, Docker images of services will be pushed into Docker Registry, and all other artifacts will be stored in the chosen type of storage. This approach gives more control over the placement of installation artifacts and avoids mixing them up.

Operation modes

DGCLI can be launched in one of the following modes: license, pull or restore.

Mode	Internet access	Source location	Target location
license	Required	Urbi public license servers	S3-compatible storage File system
pull	Required	Urbi public update servers	S3-compatible storage and Docker Registry File system and Docker Registry
restore	Not required	File system	S3-compatible storage and Docker Registry File system and Docker Registry

license mode

In the license mode, DGCLI will request a license file from Urbi servers and write it to the selected storage. See Installing License service for more information.

pull mode

In the pull mode, DGCLI will download the latest installation artifacts from the update servers to the selected storage and (if necessary) to the Docker Registry. DGCLI will also write the size of downloaded artifacts (service images, data, and others) to the log. Existing files in the storage are not counted (unless the --overwrite argument is passed).

When using a file system as a storage (the type: fs setting), DGCLI also creates a manifest file called manifests/latest.json containing the information about all the On-Premise services. The manifest file can be used for subsequent runs of DGCLI in the restore mode with the --from-dir or --by-manifest arguments.

If the --services argument is passed in the pull mode, DGCLI will download the installation artifacts for the specified services only. Note that even with this argument, the information in the manifest file will still be updated for all the On-Premise services.

To check the expected size of downloaded artifacts, pass the --dry-run argument in the pull mode. DGCLI will not download artifacts but will write its size to the log. Compatible with --services, --only-apps, and --only-data arguments.

restore mode

In the restore mode, DGCLI will load the installation artifacts that are already stored in the source file system to the selected storage and (if necessary) to the Docker Registry. Installation artifacts must be previously downloaded to the file system using the pull mode.

Note that when using the file system as the target storage (the storage.type: fs setting), launching the utility in restore mode requires two directories to be mounted: a source directory and a target directory. For more information, see the list of command-line arguments.

Dependencies

• On-Premise License that allows you to download the installation artifacts related to the purchased products.

  To get the license, fill out the form at dev.2gis.com/order.

• Host with the Docker Engine installed, which will be used to launch DGCLI.

  To use DGCLI in the license or pull modes, the host must have Internet access to public update servers.

• Depending on the selected storage type:

  • S3-compatible installation artifacts storage.
  • Free disk space on the file system of the DGCLI host.

• Docker Registry, if it will be used to store service images.

Usage scenarios

Below are some examples of configuration files and command-line arguments of DGCLI.

For more complete documentation, including all available settings, see Configuration file and Command-line arguments.

Note:

All passwords in this section are given for illustrative purposes.

For a real deployment, it is recommended to use more strong and secure passwords.

pull to S3-storage

1. Create configuration file dgctl-config.yaml:

   dgctl-config.yaml

   key: DEMO-KEY-DGCTL-AAAAAA-BBBBBB
   log-format: json
   storage:
       type: s3
       host: artifacts.storage.local
       bucket: dgctl-store
       access-key: AKIAIOSFODNN7EXAMPLE
       secret-key: wJalrXUtnFEMIK7MDENGbPxRfiCYEXAMPLEKEY
   docker:
       registry:
           username: registry-user
           password: DOCKERregistryP@ssW0rd
           server-address: http://docker.registry.local:5000
           image-prefix: /
   

2. Run one of the following commands:

   • To download the Docker images into the registry, and all other artifacts into the storage:

     docker run --rm \
         -v $(pwd)/dgctl-config.yaml:/dgctl-config.yaml \
         -v /mnt/dgctl-source:/dgctl-source \
         -v /var/run/docker.sock:/var/run/docker.sock \
         --user $(id -u):$(id -g) \
         2gis/dgctl:latest \
         pull --config=/dgctl-config.yaml --version=1.21.0 --apps-to-registry
     

   • To download all the artifacts including the Docker images into the storage:

     docker run --rm \
         -v $(pwd)/dgctl-config.yaml:/dgctl-config.yaml \
         -v /mnt/dgctl-source:/dgctl-source \
         --user $(id -u):$(id -g) \
         2gis/dgctl:latest \
         pull --config=/dgctl-config.yaml --version=1.21.0
     

pull to file system

1. Create configuration file dgctl-config.yaml:

   dgctl-config.yaml

   key: DEMO-KEY-DGCTL-AAAAAA-BBBBBB
   log-format: json
   storage:
       type: fs
       directory: /dgctl-source
   docker:
       registry:
           username: registry-user
           password: DOCKERregistryP@ssW0rd
           server-address: http://docker.registry.local:5000
           image-prefix: /
   

2. Run one of the following commands:

   • To download the Docker images into the registry, and all other artifacts into the storage:

     docker run --rm \
         -v $(pwd)/dgctl-config.yaml:/dgctl-config.yaml \
         -v /mnt/dgctl-source:/dgctl-source \
         -v /var/run/docker.sock:/var/run/docker.sock \
         --user $(id -u):$(id -g) \
         2gis/dgctl:latest \
         pull --config=/dgctl-config.yaml --version=1.21.0 --apps-to-registry
     

   • To download all the artifacts including the Docker images into the storage:

     docker run --rm \
         -v $(pwd)/dgctl-config.yaml:/dgctl-config.yaml \
         -v /mnt/dgctl-source:/dgctl-source \
         --user $(id -u):$(id -g) \
         2gis/dgctl:latest \
         pull --config=/dgctl-config.yaml --version=1.21.0
     

restore to S3-storage

1. Create configuration file dgctl-config.yaml:

   dgctl-config.yaml

   key: DEMO-KEY-DGCTL-AAAAAA-BBBBBB
   log-format: json
   storage:
       type: s3
       host: artifacts.storage.local
       bucket: dgctl-store
       access-key: AKIAIOSFODNN7EXAMPLE
       secret-key: wJalrXUtnFEMIK7MDENGbPxRfiCYEXAMPLEKEY
   docker:
       registry:
           username: registry-user
           password: DOCKERregistryP@ssW0rd
           server-address: http://docker.registry.local:5000
           image-prefix: /
   

2. Run one of the following commands:

   • To download the Docker images into the registry, and all other artifacts into the storage:

     docker run --rm \
         -v $(pwd)/dgctl-config.yaml:/dgctl-config.yaml \
         -v /mnt/dgctl-source:/dgctl-source \
         -v /var/run/docker.sock:/var/run/docker.sock \
         --user $(id -u):$(id -g) \
         2gis/dgctl:latest \
         restore --config=/dgctl-config.yaml --version=1.21.0 --from-dir=/dgctl-source --apps-to-registry
     

   • To download all the artifacts including the Docker images into the storage:

     docker run --rm \
         -v $(pwd)/dgctl-config.yaml:/dgctl-config.yaml \
         -v /mnt/dgctl-source:/dgctl-source \
         --user $(id -u):$(id -g) \
         2gis/dgctl:latest \
         restore --config=/dgctl-config.yaml --version=1.21.0 --from-dir=/dgctl-source
     

restore to file system

1. Create configuration file dgctl-config.yaml:

   dgctl-config.yaml

   key: DEMO-KEY-DGCTL-AAAAAA-BBBBBB
   log-format: json
   storage:
       type: fs
       directory: /dgctl-target
   docker:
       registry:
           username: registry-user
           password: DOCKERregistryP@ssW0rd
           server-address: http://docker.registry.local:5000
           image-prefix: /
   

2. Run one of the following commands:

   • To download the Docker images into the registry, and all other artifacts into the storage:

     docker run --rm \
         -v $(pwd)/dgctl-config.yaml:/dgctl-config.yaml \
         -v /mnt/dgctl-source:/dgctl-source \
         -v /mnt/dgctl-target:/dgctl-target \
         -v /var/run/docker.sock:/var/run/docker.sock \
         --user $(id -u):$(id -g) \
         2gis/dgctl:latest \
         restore --config=/dgctl-config.yaml --version=1.21.0 --from-dir=/dgctl-source --apps-to-registry
     

   • To download all the artifacts including the Docker images into the storage:

     docker run --rm \
         -v $(pwd)/dgctl-config.yaml:/dgctl-config.yaml \
         -v /mnt/dgctl-source:/dgctl-source \
         -v /mnt/dgctl-target:/dgctl-target \
         --user $(id -u):$(id -g) \
         2gis/dgctl:latest \
         restore --config=/dgctl-config.yaml --version=1.21.0 --from-dir=/dgctl-source