Tests and CI

1. Why testing ?

A majority of the production failures (77%) can be reproduced by a unit test.

— Yuan et al. OSDI 2014

Figure 1. Un tweet récent!

https://blog.acolyer.org/2016/10/06/simple-testing-can-prevent-most-critical-failures/amp/

1.1. To deliver the right product

Figure 2. Un produit qui fait ce qu’il est censé faire (crédit photo http://www.te52.com/testtalk/2014/08/07/5-reasons-we-need-software-testing/)

1.2. What works for 1 does not necessary for 100

Figure 3. Passage à l’échelle (crédit photo http://www.te52.com/testtalk/2014/08/07/5-reasons-we-need-software-testing/)

1.3. Murphy's law

Everything that can possibily go wrong, will go wrong.

— Edward A. Murphy Jr.

Figure 4. Murphy’s law (crédit photo http://www.te52.com/testtalk/2014/08/07/5-reasons-we-need-software-testing/)

1.4. Different OS or different terminals

Figure 5. Diversité (crédit photo http://www.te52.com/testtalk/2014/08/07/5-reasons-we-need-software-testing/)

1.5. To provide the best

Figure 6. Faire de son mieux (crédit photo http://www.te52.com/testtalk/2014/08/07/5-reasons-we-need-software-testing/)

2. Mandatory tests example

Asciidoctor Contribution

Figure 7. Autour d'une bière avec Dan Allen, à Denver, Colorado #ILoveMyJob

Fork the repository.
Run bundle to install development dependencies.
Create a topic branch
Add tests for your unimplemented feature or bug fix. (See [writing-and-executing-tests])
Run bundle exec rake to run the tests. If your tests pass, return to step 4.
Implement your feature or bug fix.
Run bundle exec rake to run the tests. If your tests fail, return to step 6.
Add documentation for your feature or bug fix.
If your changes are not 100% documented, go back to step 8.
Add, commit, and push your changes.
Submit a pull request.

3. Mandatory Doc example

Figure 8. Après un footing avec Gaël Blondel, à Saint-Malo #ILoveMyJob

[…] an Eclipse project is providing extensible frameworks and applications accessible via documented APIs.

— Eclipse Development Process

4. Test kinds

Table 1. Différences entre Vérification et Validation (source https://www.tutorialspoint.com/software_testing/software_testing_quick_guide.htm)
Verification	Validation
Are you building it right?	Are you building the right thing?
Done by developers	Done by tester
First	second

5. JUnit etc.

5.1. Testing what ?

exceptions

@Test (expected = Exception.class)

execution time

@Test(timeout=100)

Only some environment

System.getProperty("os.name").contains("Linux"));

Do something before other tests (e.g., base access)

@BeforeClass public static void method()

5.2. Assertions

fail([message])

Force the test to fail

assertTrue([message,] condition)

condition is true

assertFalse([message,] condition)

condition is false

assertEquals([message,] attendu, actuel)

values are equals

assertNull([message,] object)

null object

assertSame([message,] expected, actual)

identical objects (same ref.)

5.3. Tests Strategies

Let’s consider int add(int,int); of the myClass class.

Define the "normal" behavior:

(source : http://stackoverflow.com/questions/8751553/how-to-write-a-unit-test)

//for normal addition
@Test
public void testAdd1Plus1() {
  int x  = 1 ; int y = 1;
  assertEquals(2, myClass.add(x,y));
}

Add some tests for particular cases:

no exception captured in case of overflow
null parameters are handled, e.g.:

(source : http://stackoverflow.com/questions/8751553/how-to-write-a-unit-test)

//if you are using 0 as default for null, make sure your class works in that case.
@Test
public void testAdd1Plus1() {
  int y = 1;
  assertEquals(0, myClass.add(null,y));
}

It works with negative parameters, etc.

5.4. No ordering of tests!!

JUnit assumes that all test methods can be executed in an arbitrary order.

— JUnit manual

5.5. No ordering of tests (ctd.)!!

Well-written test code should not assume any order, i.e., tests should not depend on other tests.

— JUnit manual

5.6. Eclipse

For an existing class: Right-click (Package Explorer and New ▸ JUnit Test Case).
Use the JUnit wizards (File ▸ New ▸ Other… ▸ Java ▸ JUnit).
And then Run-as ▸ JUnit Test.

Use of the infinitest plug-in.

5.7. What about graphical interfaces?

Example of the Robot library:

(source : http://stackoverflow.com/questions/16411823/junit-tests-for-gui-in-java)

Robot bot = new Robot();
bot.mouseMove(10,10);
bot.mousePress(InputEvent.BUTTON1_MASK);
//add time between press and release or the input event system may
//not think it is a click
try{Thread.sleep(250);}catch(InterruptedException e){}
bot.mouseRelease(InputEvent.BUTTON1_MASK);

Eclipse swingcoder plug-in:

Figure 9. Simulation d’utilisation d’interface (source https://marketplace.eclipse.org/content/swingcorder)

5.8. Tests coverage

Several existing tools:

Figure 10. Couverture des tests (source http://www.eclemma.org/)

6. Application on agile project

6.1. From `To Be Done` to `On going`

Table 2. Mettre à jour Tuleap

Figure 11. Confirmation par email

6.2. New branch (if new feature)

bruel (master) $ git checkout -b US-15378
Switched to a new branch 'US-15378'
bruel (US-15378) $

6.3. Write a failing test

6.3.1. Step 0 : Understand

Let’s take one example: Pile class.

Rappels sur les propriétés d’une Pile (opérations)

Operations

CreerPile :  -> Pile
estVide   :  Pile ->  Booleen
Empiler   :  Pile * Element -> Pile
Depiler   :  Pile -> Pile
Sommet     :  Pile ->  Elément

Rappels sur les propriétés d’une Pile (préconditions)

Preconditions

Sommet(p) valide Si et Seulement Si estVide(p) == FAUX
Depiler(p) valide Si et Seulement Si estVide(p) == FAUX

Rappels sur les propriétés d’une Pile (axiomes)

Axiomes

(1) estVide(CreerPile())
(2) estVide(Empiler(p,e)) == FAUX
(3) estVide(Depiler(Empiler(p,e))) Si et Seulement Si estVide(p)
(4) Sommet(Empiler(p,e)) == e
(5) !estVide(p) => Sommet(Depiler(Empiler(p,e))) == Sommet(p)

6.3.2. Step 1 : write a simple test

import junit.textui.TestRunner;
import junit.framework.TestSuite;
import junit.framework.TestCase;

public class PileTest extends TestCase {

        public void test_type_new_Pile() throws Exception {
                Pile pile = new Pile() ;

                assertEquals("new Pile() retourne une Pile", "Pile", pile.getClass().getName());
        }
}

Figure 12. Oups, JUnit is not in the path...

Figure 13. Création rapide de la classe `Pile`

Figure 14. Run as JUnit Tests

6.3.3. Step 1' : improved with a `main`

For those who really need to execute a main:

public class PileTest extends TestCase {
        static int totalAssertions = 0;
        static int bilanAssertions = 0;

        public void test_type_new_Pile() throws Exception {
                Pile pile = new Pile() ;

                totalAssertions++ ;
                assertEquals("new Pile() retourne une Pile", "Pile", pile.getClass().getName());
                bilanAssertions++ ;
        }

        public static void main(String[] args) {
                junit.textui.TestRunner.run(new TestSuite(PileTest.class));
                if (bilanAssertions == totalAssertions) { System.out.print("Bravo !"); }
                System.out.println(" "+bilanAssertions+"/"+totalAssertions+" assertions vérifiées");
        } // fin main

} // fin PileTest

Executing the test as Java program:

...
Time: 0,005

OK (3 tests)

Bravo ! 3/3 assertions vérifiées

On command line:

javac -cp .;junit.jar PileTest.java

6.3.4. Step 2 : write a test that passes

public void test_type_empiler() throws Exception {
  Pile pile = new Pile() ;

  assertEquals("empiler(pile,'XXX') retourne une Pile", "Pile", pile.empiler("XXX").getClass().getName());
}

Figure 15. Erreur de syntaxe

public class Pile {

	public Object empiler(String string) {
		// TODO Auto-generated method stub
		return this;
	}
}

Figure 16. Adding the missing methods

Figure 17. Let's test

6.3.5. Step 2 : writing a failing test

public void test_axiome1() {
  Pile pile = new Pile() ;

  assertTrue("Une nouvelle pile est vide", pile.estVide(pile));
}

Méthode ajoutée par défaut

public boolean estVide(Pile pile) {
  // TODO Auto-generated method stub
  return false;
}

Figure 18. Passage du test

Junit runs only functions that start with test. The following code is not executed!

public void XXXtest_axiome1() {
  ...
}

Nowadays, we use the @Test annotation:

@Test
public void constructeur_correct() {
  ...
}

6.3.6. Step 3 : Make the test succeed

public boolean estVide(Pile pile) {
  // Smartly modified by JMB to pass the test!
  return true;
}

Figure 19. Running the test again

Of course our code is wrong! Here is a better one:

public class Pile {
	int count;
  ...
	public boolean estVide(Pile pile) {
		return (count == 0);
	}
}

6.4. Attempt to merge

bruel (US-15378) $ git commit -am "Adding push feature. Tests OK"
[US-15378 78f3242] Adding push feature. Tests OK
 1 file changed, 2 insertions(+), 3 deletions(-)
bruel (US-15378) $ git checkout devs
Switched to branch 'devs'
bruel (devs) $ git merge US-15378

6.5. Commit & Push into `devs` branch

bruel (devs) $ git commit -am "..."
...
bruel (devs) $ git push origin devs
...
bruel (devs) $ git branch -D US-15378
Deleted branch US-15378 (was f392a73).

6.6. From `On going` to `Review`

Figure 20. Penser à mettre à jour le tableau de bord