# Testing duplicty

## Introduction
Duplicitys test concept bases on unit test. 
All tests are contained in the /testing folder of the main repository. 

As to see in the following sketch, there are several levels of testing duplicity and each can be used directly.

                                    ┌─────────────────────┐     
                                    │    docker image     │     
                                    ├─────────────────────┴────┐
                                    │                          │
                                    │  ┌──────────────────┐    │
                                    │  │       tox        │    │
                                    │  └──────────────────┘    │
                                    │            │             │
                                    │            ▼             │
                                    │  ┌──────────────────┐    │
                                    │  │    unittests     │    │
                                    │  └──────────────────┘    │
                                    │            │             │
                                    │            ▼             │
                                    │  ┌──────────────────┐    │
                                    │  │    duplicity     │    │
                                    │  └──────────────────┘    │
                                    │                          │
                                    └──────────────────────────┘
1. Testing directly using __setup.py__
Assuming that your machine has all the required dependencies installed, you can start all the unit tests by simply typing

‘setup.py test‘ 

2. Using __tox__
Tox is a generic virtualenv management and test command line tool that is used for checking your package installs correctly with different Python versions and interpreters. It 
runs the tests in each of the environments that are configured in the tox.ini file (see root folder of the repository)

Duplicity uses tox to make it easy to test your code against multiple
environments. Running tests using the commands above will automatically test
code against different supported environments, including the versions of
dependencies used by the Launchpad build system.

A tox run can be started simply by typing 

‘tox‘

from the main duplicity folder. 

You can run specific tests using:
‘tox -- -s [folder].[folder].[file].[class].[test]‘
For example:
‘tox -- -s testing.unit.test_selection‘
or:
‘tox -- -s testing.unit.test_selection.MatchingTest.test_tuple_include‘

You can test against a single environment, e.g.
‘tox -e py27‘
for example if you are working on fixing a bug, but please do a full run-tests
before submitting a merge request.

Note: some tests require rdiff and pylint to be installed on the system for
them to pass.

Please run all tests on your branch (run-tests) before proposing a merge, to
ensure that all tests pass. The decorator @unittest.expectedFailure can be used
to commit a known-failing test case without breaking the test suite, for
example to exhibit the behaviour in a bug report before it has been fixed.

3. Via a __docker__ image 
Testing on a developer's machine can be tricky. Testing duplicity requires a set of dependencies being installed and reacts sensitiviely to changes of the local python configuration. In order to make sure that such interactions do not pose any influence on executing the tests, docker is the technology of choice. 
Along with the tests, a docker image has been created (cf. Dockerfile in root folder of repo) that ensure the following things:
- It bases on a clean Ubunut 16.04
- It installs all the required packages that are needed for testing
- It then branches the repository of duplicty to the folder /duplicty/testing within the docker image
- And installs all the required python packages (as defined in the requirements.txt)
Therewith, the docker image provides a clean and reproducible environment for executing the tests of duplicty. 
In order to get hands on the docker image you simply: 
1) Install Docker on your machine (https://docs.docker.com/engine/installation/)
2) Start the image docker run -it dernils/duplicitytest /bin/bash (if you did not use the image before, it will be downloaded automatically)
3) At the prompt of the docker image type:
‘cd /testing‘
‘tox‘ 
to start a run of the test cases. 

## Dependencies for testing
If you should prefer to execute the tests locally without using docker, the Dockerfile that is checked into the root folder of the repository contains useful information. It contains a section marked "The following packages are needed for testing duplicity". Within this section all dependencies that need to be installed on a machine to execute the test cases are identified. 

## Working with test coverage
Python makes it easy to determine, how well the tests cover the source code. 

You first run the tests __under observation__ of the coverage script:
‘coverage run setup.py test‘
After that, a report can be generated by the use of the command:
‘coverage html --omit="testing/*,/usr/*"‘

The report will be generated and stored in the folder htmlcov. 

## The wider picture - supporting containers for testing
Testing duplicity invokes backends. The backends are the places where the backup data is actually stored (e.g. an ftp server). In order to have the highest degree of control over the testing process, backends that can be set up locally are also operated in separated docker containers. The whole test infrastructure is shown in the following picture. 

┌─────────────────────┐         ┌──────────────────────────────────────────┐   
│docker image         │         │docker image                              │   
│dernils/duplicitytest│         │dernils/duplicity_testinfrastructure_ssh  │   
├─────────────────────┴────┐    ├──────────────────────────────────────────┴──┐
│                          │    │                                             │
│  ┌──────────────────┐    │    │   ┌──────────────────┐                      │
│  │       tox        │    │ ┌──┼──▶│       sshd       │                      │
│  └──────────────────┘    │ │  │   └──────────────────┘                      │
│            │             │ │  │                                             │
│            ▼             │ │  └─────────────────────────────────────────────┘
│  ┌──────────────────┐    │ │  ┌──────────────────────────────────────────┐   
│  │    unittests     │    │ │  │docker image                              │   
│  └──────────────────┘    │ │  │dernils/duplicity_testinfrastructure_ftp  │   
│            │             │ │  ├──────────────────────────────────────────┴──┐
│            ▼             │ │  │                                             │
│  ┌──────────────────┐    │ │  │   ┌──────────────────┐                      │
│  │    duplicity     │◀───┼─┴──┼──▶│    pure-ftpd     │                      │
│  └──────────────────┘    │    │   └──────────────────┘                      │
│            │             │    │                                             │
└────────────┼─────────────┘    └─────────────────────────────────────────────┘
             │                                                                 
             │                                                                 
             │                                                                 
             └────────────┐                                                    
                          │                                                    
       Internet .─────────┼─────────.                                          
         _.────'          │          `─────.                                   
     _.─'                 │                 `──.                               
   ,'                     ▼                     `.                             
  ;             ┌──────────────────┐              :                            
  :             │     Dropbox      │              ;                            
   ╲            └──────────────────┘             ╱                             
    `.                                         ,'                              
      `──.                                 _.─'                                
          `─────.                   _.────'                                    
                 `─────────────────'                                           

The docker images that contain the test infrastructure are defined in the folder /testing/infrastructure. There is a build script to compile the Dockerfile into actual images (build-duplicitiy-test.sh). However, as all images are also published on the docker hub, it is not necessary to build the images before starting testing. Testing can directly be started by using the script setup.sh. If the required docker images are not yet existing, locally, they will be downloaded by Docker. 

