same target and param in check|condition|fill

doc/README.md

@@ -0,0 +1,13 @@
+# Rougail
+
+## Les variables
+
+  - [Les familles](family/README.md)
+  - [Les variables](variable/README.md)
+
+## Les services
+
+## Les contraintes
+
+ - [Les calcules automatiques](fill/README.md)
+ - [Les vérifications des valeurs](check/README.md)

doc/condition.rst

@@ -0,0 +1,10 @@
+FIXME 
+<!ATTLIST variable remove_condition (True|False) "False">
+
+
+<!ELEMENT condition ((target | param)+ )>
+<!ATTLIST condition name (disabled_if_in|disabled_if_not_in|hidden_if_in|hidden_if_not_in|mandatory_if_in|mandatory_if_not_in) #REQUIRED>
+<!ATTLIST condition source CDATA #REQUIRED>
+<!ATTLIST condition fallback (True|False) "False">
+<!ATTLIST condition force_condition_on_fallback (True|False) "False">
+<!ATTLIST condition force_inverse_condition_on_fallback (True|False) "False">

doc/family/README.md

@@ -0,0 +1,5 @@
+# Famille
+
+  - [Une famille](simple.md)
+  - [Famille crée dynamiquement](auto.md)
+

doc/family/auto.md

@@ -0,0 +1,29 @@
+# Famille crée dynamiquement
+
+Pour créer une famille dynamiquement, il faut créer une family fictive lié à une variable.
+Le nom et la description de la famille et des variables qu'elle contient sera en réalité le prefix du nouveau nom/description. Le suffix viendra de la variable liée.
+
+Par exemple :
+
+```
+<family name='my_family'>
+    <variable name='varname' multi="True">
+        <value>val1</value>
+        <value>val2</value>
+    </variable>
+</family>
+<family name='my_dyn_family_' dynamic="varname description="Describe "">
+    <variable name='my_dyn_var_'/>
+</family>
+```
+
+Créera trois familles :
+
+- la famille : "my_family"
+- la famille dynamique : "my_dyn_family_val1" avec la description "Describe val1"
+- la famille dynamique : "my_dyn_family_val2" avec la description "Describe val2"
+
+Dans la famille dynamique "my_dyn_family_val1" on retrouvera une variable "my_dyn_var_val1".
+
+Bien évidement si le contenu de "varname" venait a évolué, de nouvelles familles dynamiques pouvent apparaitre ou des familles dynamiques peuvent disparaître.
+

doc/family/simple.md

@@ -0,0 +1,56 @@
+# Une famille
+
+Une famille est un [conteneur de variables](../variables.md).
+
+Pour décrire une famille il faut mettre au minimum un nom :
+
+```
+<family name="my_family"/>
+```
+
+Cette famille doit être placé dans une balise [variables](../variables.md) :
+
+```
+<variables>
+    <family name="my_family"/>
+</variables>
+```
+
+## Description et aide de la famille
+
+En plus d'un nom, il est possible de mettre une "description" à la famille. C'est une information "utilisateur" qui nous permettra d'avoir des informations complémentaires sur le contenu de cette famille :
+
+```
+<family name="my_family" description="This is a great family"/>
+```
+
+En plus de la description, il est possible de préciser une aide complémentaire :
+
+```
+<family name="my_family" help="This is a great family"/>
+```
+
+## Mode de la famille
+
+Le [mode](../mode.md) par défaut d'une famille correspond au [mode](../mode.md) le plus petite des variables dans cette famille.
+
+Changer le [mode](../mode.md) d'une famille permet de définir le [mode](../mode.md) par défaut des variables inclusent dans cette famille.
+
+Pour définir le [mode](../mode.md) :
+
+```
+<family name="my_family" mode="expert"/>
+```
+
+## Famille invisible
+
+Il est possible de cacher une famille, ainsi toutes les variables inclusent dans cette famille.
+
+Cacher une famille signifie qu'elle ne sera pas visible lorsqu'on modifie la configuration du service.
+Par contre ces variables sont accessibles lorsqu'on va utiliser ces variables.
+
+Pour cacher une famille :
+
+```
+<family name="my_family" hidden="True"/>
+```

doc/fill/README.md

@@ -0,0 +1,7 @@
+# Les variables calculées
+
+Une variable calculée est une variable donc sa valeur est le résultat d'une fonction python.
+
+  - [Valeur calculée de la variable](value.md)
+  - [Paramètre de la fonction](param.md)
+  - [Réfinition](redefine.md)

doc/fill/param.md

@@ -0,0 +1,197 @@
+# Paramètre de la fonction
+
+## Paramètre positionnel
+
+Déclarons un calcul avec paramètre :
+
+```
+<constraints>
+    <fill name="return_value" target="my_calculated_variable">
+        <param>no</param>
+    </fill>
+</constraints>
+```
+
+Créons la fonction correspondante :
+
+```
+def return_value(value):
+    return value
+```
+
+La variable aura donc "no" comme valeur puisque le paramètre aura la valeur fixe "no".
+
+## Paramètre nommée
+
+Déclarons une contrainte avec un paramètre nommée :
+
+```
+<constraints>
+    <fill name="return_value" target="my_calculated_variable">
+        <param name="valeur">no</param>
+    </fill>
+</constraints>
+```
+
+Dans ce cas la fonction return_value sera exécuté avec le paramètre nommé "valeur" dont sa valeur sera "no".
+
+## Paramètre de type texte
+
+Dans l'exemple précédent :
+
+```
+<constraints>
+    <fill name="return_value" target="my_calculated_variable">
+        <param>no</param>
+    </fill>
+</constraints>
+```
+
+Le paramètre est de type texte (ou "string").
+
+## Paramètre de type nombre
+
+Déclarons un calcul avec paramètre avec un nombre :
+
+```
+<constraints>
+    <fill name="return_value_with_number" target="my_calculated_variable">
+        <param type="number">1</param>
+    </fill>
+</constraints>
+```
+
+Créons la fonction correspondante :
+
+```
+def return_value_with_number(value):
+    if value == 1:
+        return 'no'
+    return 'yes'
+```
+
+La variable aura donc "no" comme valeur puisque le paramètre aura la valeur fixe "1".
+
+## Paramètre de type variable
+
+Créons deux variables avec une contrainte de type variable qui contient le nom de la variable dont sa valeur sera utilisé comme paramètre :
+
+```
+<variables>
+    <family name="family">
+        <variable name="my_calculated_variable"/>
+        <variable name="my_variable" type="number" description="My variable">
+            <value>1</value>
+        </variable>
+    </family>
+</variables>
+<constraints>
+    <fill name="return_value_with_number" target="my_calculated_variable">
+        <param type="variable">my_variable</param>
+    </fill>
+</constraints>
+```
+
+Si l'utilisateur laisse la valeur 1 à "my_variable", la valeur par défault de la variable "my_calculated_variable" sera "no".
+Si la valeur de "my_variable" est différent de 1, la valeur par défaut de la variable "my_calculated_variable" sera "yes".
+
+[Les variables meneuses ou suiveuses](../variable/leadership.md) peuvent être utilisé sans soucis commme paramètre.
+
+### Paramètre avec variable potentiellement non existante
+
+Suivant le contexte une variable peut exister ou ne pas exister.
+
+Un paramètre de type "variable" peut être "optional" :
+
+```
+<variables>
+    <family name="family">
+        <variable name="my_calculated_variable"/>
+    </family>
+</variables>
+<constraints>
+    <fill name="return_value" target="my_calculated_variable">
+        <param type="variable" optional="True">unknow_variable</param>
+    </fill>
+</constraints>
+```
+
+Dans ce cas la fonction "return_value" est exécuté sans paramètre.
+
+Si maintenant on créé un nouveau dictionnaire en créant cette variable, la fonction sera exécuté avec le paramètre.
+
+### Paramètre avec variable potentiellement désactivée
+
+Si une variable est désactivé, l'utilisation de cette variable peut poser problème.
+
+Il est possible de ne pas générer d'erreur si une variable est désactivé en utilisant le paramètre "propertyerror" :
+
+```
+<variables>
+    <family name="general">
+        <variable name="condition">
+            <value>no</value>
+        </variable>
+        <variable name="variable1" disabled="True"/>
+        <variable name="variable2"/>
+    </family>
+</variables>
+
+<constraints>
+    <fill name="calc_val" target="variable2">
+        <param type="variable" propertyerror="False">variable1</param>
+    </fill>
+</constraints>
+```
+
+Dans ce cas, le paramètre n'est jamais donnée à la fonction de destination.
+
+### Paramètre avec variable dynamique
+
+Il est possible de faire un calcul avec comme paramètre [une variable d'une famille dynamique](family/auto.md) mais pour une suffix particulier :
+
+```
+<variables>
+    <family name='family'>
+        <variable name='suffixes' type='string' description="Suffixes of dynamic family" multi="True">
+            <value>val1</value>
+            <value>val2</value>
+        </variable>
+        <variable name="my_calculated_variable" type="string" description="My calculated variable"/>
+    </family>
+    <family name='dyn' dynamic="suffixes">
+        <variable name='vardyn' type='string' description="Dynamic variable">
+            <value>val</value>
+        </variable>
+    </family>
+</variables>
+<constraints>
+    <fill name="return_value" target="my_calculated_variable">
+        <param type="variable">vardynval1</param>
+    </fill>
+</constraints>
+```
+
+Dans ce cas, valeur du paramètre de la fonction "return_value" sera la valeur de la variable "vardyn" avec le suffix "val1".
+
+## Paramètre de type information
+
+Le paramètre peut être la valeur est issue d'une information de la configuration.
+
+Créons une variable et la contrainte :
+
+```
+<variables>
+    <family name="family">
+        <variable name="my_calculated_variable" type="string" description="My calculated variable"/>
+    </family>
+</variables>
+<constraints>
+    <fill name="return_value" target="my_calculated_variable">
+        <param type="information">server_name</param>
+    </fill>
+</constraints>
+```
+
+Dans ce cas, l'information de la configuration "server_name" sera utilisé comme valeur de la variable "my_calculated_variable".
+Si l'information n'existe pas, la paramètre aura la valeur "None".

doc/fill/redefine.md

@@ -0,0 +1,60 @@
+# Rédéfinition
+
+## Redéfinition des calcules
+
+Dans un premier dictionnaire déclarons notre variable et notre calcule :
+
+```
+<variables>
+    <family name="family">
+        <variable name="my_calculated_variable"/>
+    </family>
+</variables>
+<constraints>
+    <fill name="return_no" target="my_calculated_variable"/>
+</constraints>
+```
+
+Dans un second dictionnaire il est possible de redéfinir le calcul :
+
+```
+<variables>
+    <family name="family">
+        <variable name="my_calculated_variable" redefine="True"/>
+    </family>
+</variables>
+<constraints>
+    <fill name="return_yes" target="my_calculated_variable"/>
+</constraints>
+```
+
+Dans ce cas, à aucun moment la fonction "return_no" ne sera exécuté. Seul la fonction "return_yes" le sera.
+
+## Redéfinition avec suppression d'un calcul
+
+Il se peut que dans un dictionnaire on décide de définir une valeur par défaut à une variable via un calcul.
+
+Dans un second dictionnaire il est possible de supprimer ce calcul.
+
+Dans un premier dictionnaire déclarons notre variable et notre calcule :
+
+```
+<variables>
+    <family name="family">
+        <variable name="my_calculated_variable"/>
+    </family>
+</variables>
+<constraints>
+    <fill name="return_no" target="my_calculated_variable"/>
+</constraints>
+```
+
+Dans un second dictionnaire supprimer ce calcul :
+
+```
+<variables>
+    <family name="family">
+        <variable name="my_calculated_variable" redefine="True" remove_fill="True"/>
+    </family>
+</variables>
+```

doc/fill/value.md

@@ -0,0 +1,111 @@
+# Valeur calculée de la variable
+
+## Variable avec une valeur par défaut calculée
+
+Créons une variable dont la valeur est retournée par la fonction "return_no" :
+
+```
+<variables>
+    <family name="family">
+        <variable name="my_calculated_variable"/>
+    </family>
+</variables>
+<constraints>
+    <fill name="return_no" target="my_calculated_variable"/>
+</constraints>
+```
+
+Puis créons la fonction "return_no" :
+
+```
+def return_no():
+    return 'no'
+```
+
+Dans ce cas, la valeur par défaut est la valeur retournée par la fonction (ici "no"), elle sera calculée tant que l'utilisateur n'a pas de spécifié une valeur à cette variable.
+
+Attention, si une valeur par défaut est définit dans la variable "my_calculated_variable" :
+
+```
+<variable name="my_calculated_variable">
+    <value>yes</value>
+</variable>
+```
+
+Cette valeur par défaut sera complètement ignorée. C'est le calcul qui en définira la valeur.
+
+## Variable avec une valeur calculée
+
+En ajoutant le paramètre "hidden" à "True" dans la variable précédente, l'utilisateur n'aura plus la possibilité de modifié la valeur. La valeur de la variable sera donc systématiquement calculée :
+
+```
+<variable name="my_calculated_variable" hidden="True"/>
+```
+
+Si une condition "hidden_if_in" est spécifié à la variable, la valeur sera modifiable par l'utilisateur si elle n'est pas cachée mais elle sera systèmatiquement calculée (même si elle a déjà était modifiée) si la variable est cachée.
+
+## Variable meneuse ou suiveuse avec valeur calculé
+
+Une [variable suiveuse](../variable/leadership.md) ne peut pas être calculé automatiquement.
+Une [variable meneuse](../variable/leadership.md) peut être calculé automatiquement.
+Si la variable n'est pas multiple, il ne faut pas que le calcule retourne une liste.
+
+
+## Variable dynamique avec une valeur calculée
+
+Il est également possible de calculer [une variable d'une famille dynamique](family/auto.md) à partir d'une variable standard :
+
+```
+<variables>
+    <family name='family'>
+        <variable name='suffixes' type='string' description="Suffixes of dynamic family" multi="True">
+            <value>val1</value>
+            <value>val2</value>
+        </variable>
+        <variable name="my_variable" type="string" description="My variable">
+            <value>val</value>
+        </variable>
+    </family>
+    <family name='dyn' dynamic="suffixes">
+        <variable name="my_calculated_variable_dyn_" type="string" description="My calculated variable"/>
+            <value>val</value>
+        </variable>
+    </family>
+</variables>
+<constraints>
+    <fill name="return_value" target="my_calculated_variable_dyn_">
+        <param type="variable">my_variable</param>
+    </fill>
+</constraints>
+```
+
+Dans ce cas, les variables dynamiques "my_calculated_variable_dyn_" seront calculés à partir de la valeur de la variable "my_variable".
+Que cela soit pour la variable "my_calculated_variable_dyn_val1" et "my_calculated_variable_dyn_val2".
+
+Par contre, il n'est pas possible de faire un calcul pour une seule des deux variables issues de la variable dynamique.
+Si c'est ce que vous cherchez à faire, il faudra prévoir un traitement particulier dans votre fonction.
+
+Dans ce cas, il faut explicitement demander la valeur du suffix dans la fonction :
+
+```
+<constraints>
+    <fill name="return_value_suffix" target="my_calculated_variable_dyn_">
+        <param type="variable">my_variable</param>
+        <param type="suffix"/>
+    </fill>
+</constraints>
+```
+
+Et ainsi faire un traitement spécifique pour ce suffix :
+
+```
+def return_value_suffix(value, suffix):
+    if suffix == 'val1':
+        return value
+```
+
+## Variable avec valeur calculée obligatoire
+
+Par défaut les variables calculées ne sont pas des variables obligatoires.
+Dans ce cas un calcul peut retourner "None" ou "", mais surtout un utilisateur peut spécifier une valeur nulle à cette variable. Dans ce cas le calcul ne sera plus réalisé.
+

doc/mode.md

@@ -0,0 +1,8 @@
+Mode
+====
+
+Il existe trois "mode" dans Rougail :
+
+- basic : variables indispensables à la mise en place d'un service
+- normal : variables couramment modifié par l'utilisateur
+- expert : variables a manipuler avec précausion et en toute connaissance de cause

doc/variable/README.md

@@ -0,0 +1,6 @@
+# Variable
+
+  - [Une variable](simple.md)
+  - [Variable meneuse ou suiveuse](leadership.md)
+
+FIXME <!ATTLIST variable test CDATA #IMPLIED>

doc/variable/leadership.md

@@ -0,0 +1,74 @@
+# Variable meneuse ou suiveuse
+
+## Variable meneuse
+
+Une variable meneuse est une [variable](simple.md) qui est obligatoirement de type multiple.
+
+Une variable meneuse peut être obligatoire.
+
+## Variable suiveuse
+
+Une variable suiveuse est une variable donc la longueur n'est pas déterminé par elle-même, mais est identique à celle de la variable meneuse dont elle dépend.
+
+Une variable suiveuse est une variable placer juste derrière une variable meneuse ou une autre variable suiveuse.
+
+L'ordre de définition des variables suiveuses est important.
+
+Cette variable peut être de type multiple. Dans ce cas, pour un index determiné, il est possible de mettre plusieurs valeurs à une même variable.
+
+Une variable suiveuse peut être obligatoire. Cela signifie que lorsqu'une variable meneuse est renseigné, il faut obligatoirement que la variable suiveuse est également une valeur à l'index considéré.
+Si aucune valeur n'est définit pour la variable meneuse, aucune valeur n'est a spécifié pour la variable suiveuse.
+
+Le [mode](../mode.md) par défaut d'une variable suiveuse correspond au [mode](../mode.md) de la variable meneuse.
+
+Si une variable meneuse est caché ou désactivé, les variables suiveuses le seront également.
+
+## Définition des variables meneuse et suiveuse
+
+Voici un exemple de définition d'une variable meneuse et de deux variables meneuses :
+
+```
+<variables>
+    <family name="family">
+        <variable name="leader" multi='True'/>
+        <variable name="follower1"/>
+        <variable name="follower2" multi='True'/>
+    </family>
+</variables>
+<constraints>
+    <group leader="leader">
+        <follower>follower1</follower>
+        <follower>follower2</follower>
+    </group>
+</constraints>
+```
+
+Le nom et la description du groupe prendra par défaut le nom et la description de la variable meneuse.
+
+Il est possible d'en définit d'autres :
+
+```
+<constraints>
+    <group leader="leader" name="leadership" description="My leadership">
+        <follower>follower1</follower>
+        <follower>follower2</follower>
+    </group>
+</constraints>
+```
+
+## Ajout d'une nouvelle variable suiveuse
+
+Pour ajouter, dans un nouveau dictionnaire, une variable suiveuse à notre groupe, rien de plus simple, il suffit de redéfinir un groupe avec cette (ou ces) nouvelle variable :
+
+```
+<variables>
+    <family name="family">
+        <variable name="follower3"/>
+    </family>
+</variables>
+<constraints>
+    <group leader="leader">
+        <follower>follower3</follower>
+    </group>
+</constraints>
+```

doc/variable/simple.md

@@ -0,0 +1,210 @@
+# Variable
+
+## Un variable
+
+Une variable est forcement dans une [famille](../family/README.md). Il faut donc déjà avoir créer une [famille](../family/README.md).
+
+Une variable est déjà un nom. C'est à dire qu'on pourra utiliser plus tard la variable via ce nom.
+
+```
+<variable name="my_variable"/>
+```
+
+## Description et aide sur la variable
+
+En plus d'un nom, il est possible de mettre une "description" à la variable. C'est une information "utilisateur" qui nous permettra d'avoir des informations complémentaires sur le contenu de cette variable :
+
+```
+<variable name="my_variable" description="This is a greate variable"/>
+```
+
+En plus de la description, il est possible de préciser une aide complémentaire :
+
+```
+<variable name="my_variable" help="This is a greate variable"/>
+```
+
+## Le type de la variable
+
+Une variable a un type. Ce type permet de définir les valeurs acceptées par cette variable :
+
+- string : chaine de caractère (type par défaut)
+- number : un nombre
+- float : un chiffre flottant
+- boolean : True ou False si aucune valeur n'est défini la valeur par défaut cette variable sera True
+- password : un mot de passe
+- mail : une adresse mail
+- filename : nom de fichier au sens Unix (exemple : '/etc/passwd')
+- date : une date au format "%Y-%m-%d" (exemple : "2021-01-30")
+- unix_user : nom d'utilisateur au sens Unix
+- ip : n'importe quelle adresse IPv4
+- cidr : n'importe quelle adresse IPv4 au format CIDR
+- local_ip : adresse IPv4 sur un réseau local, si l'adresse IPv4 n'est pas local, un warning sera afficher mais la valeur sera accepté tout de même
+- netmask : masque d'une adresse IPv4
+- network : adresse réseau
+- network_cidr : adresse réseau au format CIDR
+- broadcast : adresse de diffusion
+- netbios : nom netbios
+- domain : nom de domaine
+- hostname : nom d'hôte
+- web_address : adresse web (http://www.cadoles.com/)
+- port : port
+- mac : adresse MAC
+- schedule : périodicité du schedule, les valeurs possibles sont "none", "daily", "weekly" ou "monthly"
+- schedulemod : type de schedule, les valeurs possibles sont "pre" ou "post"
+
+Pour définir le type d'une variable :
+
+```
+<variable name="my_variable" type="number"/>
+```
+
+## Variable à valeur multiple
+
+Par défaut une variable ne peut acceuillir qu'une seule valeur. Il peut être utile de pouvoir spécifier plusieurs valeurs à une même variable.
+
+Pour définir une variable à valeur multiple :
+
+```
+<variable name="my_variable" multi="True"/>
+```
+
+## Mode de la variable
+
+Le [mode](../mode.md) par défaut d'une variable correspond au [mode](../mode.md) de la [famille](../family/README.md).
+
+Pour définir le [mode](../mode.md) :
+
+```
+<variable name="my_variable" mode="expert"/>
+```
+
+## Variable invisible
+
+Il est possible de cacher une variable.
+
+Cacher une variable signifie qu'elle ne sera pas visible lorsqu'on modifie la configuration du service.
+Par contre cette variable sont accessibles lorsqu'on va l'utiliser.
+
+Pour cacher une variable :
+
+```
+<variable name="my_variable" hidden="True"/>
+```
+
+## Variable désactive
+
+Il est possible de désactiver une variable.
+
+Désactiver une variable signifie qu'elle ne sera pas visible lorsqu'on modifie la configuration du service mais également lorsqu'on va l'utiliser.
+
+Pour désactiver une variable :
+
+```
+<variable name="my_variable" disabled="True"/>
+```
+
+## Variable obligatoire
+
+Variable dont une valeur est requise :
+
+```
+<variable name="my_variable" mandatory="True"/>
+```
+
+## Valeur par défaut d'une variable
+
+Il est possible de fixer les valeurs par défaut d'une variable :
+
+```
+<variable name="my_variable">
+    <value>value</value>
+</variable>
+```
+
+Pour une variable multiple, il est possible de préciser plusieurs valeurs :
+
+```
+<variable name="my_variable" multi="True">
+    <value>value 1</value>
+    <value>value 2</value>
+</variable>
+```
+
+Si la variable n'est pas pas une [variable meneuse](leadership.md), la première valeur défini dans cette liste sera également la valeur par défaut proposé si on ajoute une nouvelle valeur à cette variable.
+
+Une valeur par défaut peut également être [une valeur calculer](../fill/README.md).
+
+## Redéfinir une variable
+
+Il est possible de définir une variable dans un dictionnaire et de changer son comportement dans une second dictionnaire.
+
+Attention trois attributs ne sont redéfinisable :
+
+- name
+- type
+- multi
+
+Créons notre variable :
+
+    <variable name="my_variable"/>
+
+Et redéfinisons là :
+
+```
+<variable name="my_variable" redefine="True" description="New description"/>
+```
+
+## Créer une variable inexistante
+
+Il est parfois utile de créer une variable si elle n'existe pas dans un autre dictionnaire :
+
+```
+<variable name="my_variable" exists="False"/>
+```
+
+Si cette variable existe dans un autre dictionnaire, elle ne sera pas modifié ni recréé
+
+## Redéfinir une variable si elle existe
+
+Parfois on veut pouvoir redéfinir une variable mais seulement dans le cas où elle existe déjà :
+
+```
+<variable name="my_variable" redefine="True" exists="True" hidden="True"/>
+```
+
+## Variable à valeur automatiquement modifiée
+
+Une variable avec valeur automatiquement modifiée est une variable dont la valeur sera considéré comme modifié quand le serveur sera déployé.
+
+Voici une variable a valeur automatiquement modifiée :
+
+```
+<variable name="my_variable" auto_save="True">
+    <value>my_value</value>
+</variable>
+```
+
+Dans ce cas la valeur est fixée à la valeur actuelle.
+Par exemple, si la valeur de cette variable est issue d'un calcul, la valeur ne sera plus recalculée.
+
+Ces variables sont généralement des variables obligatoires. En effet ces variable ne sont automatiquement modifiées que si elles ont une valeurs.
+
+Une [variable meneuse ou suiveuse](leadership.md) ne peut pas avoir la propriété auto_save.
+
+## Variable à valeur en lecture seule automatique
+
+Une variable avec valeur en lecture seule automatique est une variable dont la valeur ne sera plus modifiable par l'utilisateur quand le serveur sera déployé.
+
+Voici un variable à valeur en lecture seule automatique :
+
+```
+<variable name="my_variable" auto_freeze="True"/>
+```
+
+Dans ce cas la valeur est fixée à la valeur actuelle et elle ne sera plus modifiable par l'utilisateur.
+Par exemple, si la valeur de cette variable est issue d'un calcul, la valeur ne sera plus recalculée.
+
+Ces variables sont généralement des variables obligatoires. En effet ces variable ne sont en lecteur seul que si elles sont une valeurs.
+
+Une [variable meneuse ou suiveuse](leadership.md) ne peut pas avoir la propriété auto_freeze.

doc/variables.md

@@ -0,0 +1,10 @@
+# Le conteneur des variables
+
+La balise "variables" est le conteneur de l'ensemble des [familles](family/README.md) dans laquelle on pourra placer des [variables](variable/README.md) :
+
+```
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables/>
+</rougail>
+```

src/rougail/__init__.py

@@ -1,5 +1,30 @@
-#from .loader import load
+"""Rougail method
-from .objspace import CreoleObjSpace
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from .rougail import Rougail
 from .annotator import modes
 
-__ALL__ = ('CreoleObjSpace', 'modes')
+__ALL__ = ('RougailObjSpace', 'modes')

src/rougail/annotator.py

@@ -1,1237 +0,0 @@
-# coding: utf-8
-from copy import copy
-from typing import List
-
-from collections import OrderedDict
-from os.path import join, basename
-from ast import literal_eval
-import imp
-
-
-from .i18n import _
-from .utils import normalize_family
-from .error import DictConsistencyError
-
-#mode order is important
-modes_level = ('basic', 'normal', 'expert')
-
-
-class Mode(object):
-    def __init__(self, name, level):
-        self.name = name
-        self.level = level
-    def __gt__(self, other):
-        return other.level < self.level
-
-
-def mode_factory():
-    mode_obj = {}
-    for idx in range(len(modes_level)):
-        name = modes_level[idx]
-        mode_obj[name] = Mode(name, idx)
-    return mode_obj
-
-modes = mode_factory()
-
-
-# a CreoleObjSpace's attribute has some annotations
-# that shall not be present in the exported (flatened) XML
-ERASED_ATTRIBUTES = ('redefine', 'exists', 'fallback', 'optional', 'remove_check', 'namespace',
-                     'remove_condition', 'path', 'instance_mode', 'index', 'is_in_leadership',
-                     'level')  # , '_real_container')
-ERASED_CONTAINER_ATTRIBUTES = ('id', 'container', 'group_id', 'group', 'container_group')
-
-FORCE_CHOICE = {'oui/non': ['oui', 'non'],
-                'on/off': ['on', 'off'],
-                'yes/no': ['yes', 'no'],
-                'schedule': ['none', 'daily', 'weekly', 'monthly'],
-                'schedulemod': ['pre', 'post']}
-
-KEY_TYPE = {'variable': 'symlink',
-            'SymLinkOption': 'symlink',
-            'PortOption': 'port',
-            'UnicodeOption': 'string',
-            'NetworkOption': 'network',
-            'NetmaskOption': 'netmask',
-            'URLOption': 'web_address',
-            'FilenameOption': 'filename'}
-
-TYPE_PARAM_CHECK = ('string', 'python', 'variable')
-TYPE_PARAM_CONDITION = ('string', 'python', 'number', 'variable')
-TYPE_PARAM_FILL = ('string', 'number', 'variable')
-CONVERSION = {'number': int}
-
-FREEZE_AUTOFREEZE_VARIABLE = 'module_instancie'
-
-PROPERTIES = ('hidden', 'frozen', 'auto_freeze', 'auto_save', 'force_default_on_freeze',
-              'force_store_value', 'disabled', 'mandatory')
-CONVERT_PROPERTIES = {'auto_save': ['force_store_value'], 'auto_freeze': ['force_store_value', 'auto_freeze']}
-
-RENAME_ATTIBUTES = {'description': 'doc'}
-INTERNAL_FUNCTIONS = ['valid_enum', 'valid_in_network', 'valid_differ', 'valid_entier']
-
-class SpaceAnnotator:
-    """Transformations applied on a CreoleObjSpace instance
-    """
-    def __init__(self, objectspace, eosfunc_file):
-        self.objectspace = objectspace
-        GroupAnnotator(objectspace)
-        ServiceAnnotator(objectspace)
-        VariableAnnotator(objectspace)
-        ConstraintAnnotator(objectspace,
-                            eosfunc_file,
-                            )
-        FamilyAnnotator(objectspace)
-        PropertyAnnotator(objectspace)
-
-
-class GroupAnnotator:
-    def __init__(self,
-                 objectspace,
-                 ):
-        self.objectspace = objectspace
-        if not hasattr(self.objectspace.space, 'constraints') or not hasattr(self.objectspace.space.constraints, 'group'):
-            return
-        self.convert_groups()
-
-    def convert_groups(self):  # pylint: disable=C0111
-        for group in self.objectspace.space.constraints.group:
-            leader_fullname = group.leader
-            leader_family_name = self.objectspace.paths.get_variable_family_name(leader_fullname)
-            leader_name = self.objectspace.paths.get_variable_name(leader_fullname)
-            namespace = self.objectspace.paths.get_variable_namespace(leader_fullname)
-            if '.' not in leader_fullname:
-                leader_fullname = '.'.join([namespace, leader_family_name, leader_fullname])
-            follower_names = list(group.follower.keys())
-            has_a_leader = False
-            ori_leader_family = self.objectspace.paths.get_family_obj(leader_fullname.rsplit('.', 1)[0])
-            for variable in list(ori_leader_family.variable.values()):
-                if has_a_leader:
-                    # it's a follower
-                    self.manage_follower(namespace,
-                                         leader_family_name,
-                                         variable,
-                                         leader_name,
-                                         follower_names,
-                                         leader_space,
-                                         leader_is_hidden,
-                                         )
-                    ori_leader_family.variable.pop(variable.name)
-                    if follower_names == []:
-                        # no more follower
-                        break
-                elif variable.name == leader_name:
-                    # it's a leader
-                    if isinstance(variable, self.objectspace.Leadership):
-                        # append follower to an existed leadership
-                        leader_space = variable
-                        # if variable.hidden:
-                        #     leader_is_hidden = True
-                    else:
-                        leader_space = self.objectspace.Leadership()
-                        leader_is_hidden = self.manage_leader(leader_space,
-                                                              leader_family_name,
-                                                              leader_name,
-                                                              namespace,
-                                                              variable,
-                                                              group,
-                                                              leader_fullname,
-                                                              )
-                    has_a_leader = True
-            else:
-                raise DictConsistencyError(_('cannot found followers {}').format(follower_names))
-        del self.objectspace.space.constraints.group
-
-    def manage_leader(self,
-                      leader_space: 'Leadership',
-                      leader_family_name: str,
-                      leader_name: str,
-                      namespace: str,
-                      variable: 'Variable',
-                      group: 'Group',
-                      leader_fullname: str,
-                      ) -> None:
-        # manage leader's variable
-        if variable.multi is not True:
-            raise DictConsistencyError(_('the variable {} in a group must be multi').format(variable.name))
-        leader_space.variable = []
-        leader_space.name = leader_name
-        leader_space.hidden = variable.hidden
-        if variable.hidden:
-            leader_is_hidden = True
-            variable.frozen = True
-            variable.force_default_on_freeze = True
-        else:
-            leader_is_hidden = False
-        variable.hidden = None
-        if hasattr(group, 'description'):
-            leader_space.doc = group.description
-        elif hasattr(variable, 'description'):
-            leader_space.doc = variable.description
-        else:
-            leader_space.doc = variable.name
-        leader_path = namespace + '.' + leader_family_name + '.' + leader_name
-        self.objectspace.paths.add_family(namespace,
-                                          leader_path,
-                                          leader_space,
-                                          )
-        leader_family = self.objectspace.space.variables[namespace].family[leader_family_name]
-        leader_family.variable[leader_name] = leader_space
-        leader_space.variable.append(variable)
-        self.objectspace.paths.set_leader(namespace,
-                                          leader_family_name,
-                                          leader_name,
-                                          leader_name,
-                                          )
-        leader_space.path = leader_fullname
-        return leader_is_hidden
-
-    def manage_follower(self,
-                        namespace: str,
-                        leader_family_name: str,
-                        variable: 'Variable',
-                        leader_name: str,
-                        follower_names: List[str],
-                        leader_space: 'Leadership',
-                        leader_is_hidden: bool,
-                        ) -> None:
-        if variable.name != follower_names[0]:
-            raise DictConsistencyError(_('cannot found this follower {}').format(follower_names[0]))
-        follower_names.remove(variable.name)
-        if leader_is_hidden:
-            variable.frozen = True
-            variable.force_default_on_freeze = True
-        leader_space.variable.append(variable)  # pylint: disable=E1101
-        self.objectspace.paths.set_leader(namespace,
-                                          leader_family_name,
-                                          variable.name,
-                                          leader_name,
-                                          )
-
-
-class ServiceAnnotator:
-    """Manage service's object
-    for example::
-        <services>
-            <service name="test">
-                <service_access service='ntp'>
-                    <port protocol='udp' service_accesslist='ntp_udp'>123</port>
-                </service_access>
-            </service>
-        </services>
-    """
-    def __init__(self, objectspace):
-        self.objectspace = objectspace
-        self.convert_services()
-
-    def convert_services(self):
-        if not hasattr(self.objectspace.space, 'services'):
-            return
-        if not hasattr(self.objectspace.space.services, 'service'):
-            del self.objectspace.space.services
-            return
-        self.objectspace.space.services.hidden = True
-        self.objectspace.space.services.name = 'services'
-        self.objectspace.space.services.doc = 'services'
-        families = {}
-        for idx, service_name in enumerate(self.objectspace.space.services.service.keys()):
-            service = self.objectspace.space.services.service[service_name]
-            new_service = self.objectspace.service()
-            for elttype, values in vars(service).items():
-                if not isinstance(values, (dict, list)) or elttype in ERASED_ATTRIBUTES:
-                    setattr(new_service, elttype, values)
-                    continue
-                eltname = elttype + 's'
-                path = '.'.join(['services', service_name, eltname])
-                family = self.gen_family(eltname,
-                                         path,
-                                         )
-                if isinstance(values, dict):
-                    values = list(values.values())
-                family.family = self.make_group_from_elts(service_name,
-                                                          elttype,
-                                                          values,
-                                                          path,
-                                                          )
-                setattr(new_service, elttype, family)
-            new_service.doc = new_service.name
-            families[service_name] = new_service
-        self.objectspace.space.services.service = families
-
-    def gen_family(self,
-                   name,
-                   path,
-                   ):
-        family = self.objectspace.family()
-        family.name = normalize_family(name)
-        family.doc = name
-        family.mode = None
-        self.objectspace.paths.add_family('services',
-                                          path,
-                                          family,
-                                          )
-        return family
-
-    def make_group_from_elts(self,
-                             service_name,
-                             name,
-                             elts,
-                             path,
-                             ):
-        """Splits each objects into a group (and `OptionDescription`, in tiramisu terms)
-        and build elements and its attributes (the `Options` in tiramisu terms)
-        """
-        families = []
-        new_elts = self._reorder_elts(name,
-                                      elts,
-                                      )
-        for index, elt_info in enumerate(new_elts):
-            elt = elt_info['elt']
-            elt_name = elt_info['elt_name']
-
-            # try to launch _update_xxxx() function
-            update_elt = '_update_' + elt_name
-            if hasattr(self, update_elt):
-                getattr(self, update_elt)(elt,
-                                          index,
-                                          path,
-                                          service_name,
-                                          )
-
-            idx = 0
-            while True:
-                if hasattr(elt, 'source'):
-                    c_name = elt.source
-                else:
-                    c_name = elt.name
-                if idx:
-                    c_name += f'_{idx}'
-                subpath = '{}.{}'.format(path, c_name)
-                if not self.objectspace.paths.family_is_defined(subpath):
-                    break
-                idx += 1
-            family = self.gen_family(c_name, subpath)
-            family.variable = []
-            listname = '{}list'.format(name)
-            activate_path = '.'.join([subpath, 'activate'])
-            for key in dir(elt):
-                if key.startswith('_') or key.endswith('_type') or key in ERASED_ATTRIBUTES:
-                    continue
-                value = getattr(elt, key)
-                if key == listname:
-                    self.objectspace.list_conditions.setdefault(listname,
-                                                                {}).setdefault(
-                                                                    value,
-                                                                    []).append(activate_path)
-                    continue
-                family.variable.append(self._generate_element(elt_name,
-                                                              key,
-                                                              value,
-                                                              elt,
-                                                              f'{subpath}.{key}'
-                                                              ))
-            # FIXME ne devrait pas etre True par défaut
-            # devrait etre un calcule
-            family.variable.append(self._generate_element(elt_name,
-                                                         'activate',
-                                                          True,
-                                                          elt,
-                                                          activate_path,
-                                                          ))
-            families.append(family)
-        return families
-
-    def _generate_element(self,
-                          elt_name,
-                          key,
-                          value,
-                          elt,
-                          path,
-                          ):
-        variable = self.objectspace.variable()
-        variable.name = normalize_family(key)
-        variable.mode = None
-        if key == 'name':
-            true_key = elt_name
-        else:
-            true_key = key
-        dtd_key_type = true_key + '_type'
-        if key == 'activate':
-            type_ = 'boolean'
-        elif hasattr(elt, dtd_key_type):
-            type_ = KEY_TYPE[getattr(elt, dtd_key_type)]
-        elif key in self.objectspace.booleans_attributs:
-            type_ = 'boolean'
-        else:
-            type_ = 'string'
-        variable.type = type_
-        if type_ == 'symlink':
-            variable.opt = self.objectspace.paths.get_variable_path(value,
-                                                                    'services',
-                                                                    )
-#            variable.opt = value
-            variable.multi = None
-        else:
-            variable.doc = key
-            val = self.objectspace.value()
-            val.type = type_
-            val.name = value
-            variable.value = [val]
-        self.objectspace.paths.add_variable('services',
-                                            path,
-                                            'service',
-                                            False,
-                                            variable,
-                                            )
-        return variable
-
-    def _reorder_elts(self,
-                      name,
-                      elts,
-                      ):
-        """Reorders by index the elts
-        """
-        new_elts = {}
-        # reorder elts by index
-        for idx, elt in enumerate(elts):
-            new_elts.setdefault(idx, []).append(elt)
-        idxes = list(new_elts.keys())
-        idxes.sort()
-        result_elts = []
-        for idx in idxes:
-            for elt in new_elts[idx]:
-                result_elts.append({'elt_name': name, 'elt': elt})
-        return result_elts
-
-    def _update_override(self,
-                         file_,
-                         index,
-                         service_path,
-                         service_name,
-                         ):
-        file_.name = f'/systemd/system/{service_name}.service.d/rougail.conf'
-        # retrieve default value from File object
-        for attr in ['owner', 'group', 'mode']:
-            setattr(file_, attr, getattr(self.objectspace.file, attr))
-        if not hasattr(file_, 'source'):
-            file_.source = f'{service_name}.service'
-        self._update_file(file_,
-                          index,
-                          service_path,
-                          service_name,
-                          )
-
-    def _update_file(self,
-                     file_,
-                     index,
-                     service_path,
-                     service_name,
-                     ):
-        if not hasattr(file_, 'file_type') or file_.file_type == "UnicodeOption":
-            if not hasattr(file_, 'source'):
-                file_.source = basename(file_.name)
-        elif not hasattr(file_, 'source'):
-            raise DictConsistencyError(_('attribute source mandatory for file with variable name '
-                                               'for {}').format(file_.name))
-
-
-class VariableAnnotator:
-    def __init__(self,
-                 objectspace,
-                 ):
-        self.objectspace = objectspace
-        self.convert_variable()
-        self.convert_helps()
-        self.convert_auto_freeze()
-        self.convert_separators()
-
-    def convert_variable(self):
-        def _convert_variable(variable,
-                              variable_type,
-                              ):
-            if not hasattr(variable, 'type'):
-                variable.type = 'string'
-            if variable.type != 'symlink' and not hasattr(variable, 'description'):
-                variable.description = variable.name
-            if hasattr(variable, 'value'):
-                for value in variable.value:
-                    if not hasattr(value, 'type'):
-                        value.type = variable.type
-                    value.name = CONVERSION.get(value.type, str)(value.name)
-            for key, value in RENAME_ATTIBUTES.items():
-                setattr(variable, value, getattr(variable, key))
-                setattr(variable, key, None)
-            if variable_type == 'follower':
-                if variable.multi is True:
-                    variable.multi = 'submulti'
-                else:
-                    variable.multi = True
-
-        def _convert_valid_enum(namespace,
-                                variable,
-                                path,
-                                ):
-            if variable.type in FORCE_CHOICE:
-                check = self.objectspace.check()
-                check.name = 'valid_enum'
-                check.target = path
-                check.namespace = namespace
-                param = self.objectspace.param()
-                param.text = str(FORCE_CHOICE[variable.type])
-                check.param = [param]
-                if not hasattr(self.objectspace.space, 'constraints'):
-                    self.objectspace.space.constraints = self.objectspace.constraints()
-                    self.objectspace.space.constraints.namespace = namespace
-                if not hasattr(self.objectspace.space.constraints, 'check'):
-                    self.objectspace.space.constraints.check = []
-                self.objectspace.space.constraints.check.append(check)
-                variable.type = 'string'
-
-        if not hasattr(self.objectspace.space, 'variables'):
-            return
-        for families in self.objectspace.space.variables.values():
-            namespace = families.name
-            if hasattr(families, 'family'):
-                families.doc = families.name
-                for family in families.family.values():
-                    family.doc = family.name
-                    for key, value in RENAME_ATTIBUTES.items():
-                        if hasattr(family, key):
-                            setattr(family, value, getattr(family, key))
-                            setattr(family, key, None)
-                    family.name = normalize_family(family.name)
-                    if hasattr(family, 'variable'):
-                        for variable in family.variable.values():
-                            if isinstance(variable, self.objectspace.Leadership):
-                                for idx, follower in enumerate(variable.variable):
-                                    if idx == 0:
-                                        variable_type = 'master'
-                                    else:
-                                        variable_type = 'follower'
-                                    path = '{}.{}.{}.{}'.format(namespace, normalize_family(family.name), variable.name, follower.name)
-                                    _convert_variable(follower,
-                                                      variable_type,
-                                                      )
-                                    _convert_valid_enum(namespace,
-                                                        follower,
-                                                        path,
-                                                        )
-                            else:
-                                path = '{}.{}.{}'.format(namespace, normalize_family(family.name), variable.name)
-                                _convert_variable(variable,
-                                                  'variable',
-                                                  )
-                                _convert_valid_enum(namespace,
-                                                    variable,
-                                                    path,
-                                                    )
-
-    def convert_helps(self):
-        if not hasattr(self.objectspace.space, 'help'):
-            return
-        helps = self.objectspace.space.help
-        if hasattr(helps, 'variable'):
-            for hlp in helps.variable.values():
-                variable = self.objectspace.paths.get_variable_obj(hlp.name)
-                variable.help = hlp.text
-        if hasattr(helps, 'family'):
-            for hlp in helps.family.values():
-                variable = self.objectspace.paths.get_family_obj(hlp.name)
-                variable.help = hlp.text
-        del self.objectspace.space.help
-
-    def convert_auto_freeze(self):  # pylint: disable=C0111
-        def _convert_auto_freeze(variable, namespace):
-            if variable.auto_freeze:
-                new_condition = self.objectspace.condition()
-                new_condition.name = 'auto_hidden_if_not_in'
-                new_condition.namespace = namespace
-                new_condition.source = FREEZE_AUTOFREEZE_VARIABLE
-                new_param = self.objectspace.param()
-                new_param.text = 'oui'
-                new_condition.param = [new_param]
-                new_target = self.objectspace.target()
-                new_target.type = 'variable'
-                path = variable.namespace + '.' + normalize_family(family.name) + '.' + variable.name
-                new_target.name = path
-                new_condition.target = [new_target]
-                if not hasattr(self.objectspace.space.constraints, 'condition'):
-                    self.objectspace.space.constraints.condition = []
-                self.objectspace.space.constraints.condition.append(new_condition)
-        if hasattr(self.objectspace.space, 'variables'):
-            for variables in self.objectspace.space.variables.values():
-                if hasattr(variables, 'family'):
-                    namespace = variables.name
-                    for family in variables.family.values():
-                        if hasattr(family, 'variable'):
-                            for variable in family.variable.values():
-                                if isinstance(variable, self.objectspace.Leadership):
-                                    for follower in variable.variable:
-                                        _convert_auto_freeze(follower, namespace)
-                                else:
-                                    _convert_auto_freeze(variable, namespace)
-
-    def convert_separators(self):  # pylint: disable=C0111,R0201
-        if not hasattr(self.objectspace.space, 'variables'):
-            return
-        for family in self.objectspace.space.variables.values():
-            if not hasattr(family, 'separators'):
-                continue
-            if hasattr(family.separators, 'separator'):
-                for idx, separator in enumerate(family.separators.separator):
-                    option = self.objectspace.paths.get_variable_obj(separator.name)
-                    if hasattr(option, 'separator'):
-                        subpath = self.objectspace.paths.get_variable_path(separator.name,
-                                                                           separator.namespace,
-                                                                           )
-                        raise DictConsistencyError(_('{} already has a separator').format(subpath))
-                    option.separator = separator.text
-            del family.separators
-
-
-class ConstraintAnnotator:
-    def __init__(self,
-                 objectspace,
-                 eosfunc_file,
-                 ):
-        if not hasattr(objectspace.space, 'constraints'):
-            return
-        self.objectspace = objectspace
-        self.eosfunc = imp.load_source('eosfunc', eosfunc_file)
-        self.valid_enums = {}
-        if hasattr(self.objectspace.space.constraints, 'check'):
-            self.check_check()
-            self.check_replace_text()
-            self.check_valid_enum()
-            self.check_change_warning()
-            self.convert_check()
-        if hasattr(self.objectspace.space.constraints, 'condition'):
-            self.check_params_target()
-            self.filter_targets()
-            self.convert_xxxlist_to_variable()
-            self.check_condition_fallback_optional()
-            self.check_choice_option_condition()
-            self.remove_condition_with_empty_target()
-            self.convert_condition()
-        if hasattr(self.objectspace.space.constraints, 'fill'):
-            self.convert_fill()
-        self.remove_constraints()
-
-    def check_check(self):
-        remove_indexes = []
-        functions = dir(self.eosfunc)
-        functions.extend(INTERNAL_FUNCTIONS)
-        for check_idx, check in enumerate(self.objectspace.space.constraints.check):
-            if not check.name in functions:
-                raise DictConsistencyError(_('cannot find check function {}').format(check.name))
-            if hasattr(check, 'param'):
-                param_option_indexes = []
-                for idx, param in enumerate(check.param):
-                    if param.type not in TYPE_PARAM_CHECK:
-                        raise DictConsistencyError(_('cannot use {} type as a param in check for {}').format(param.type, check.target))
-                    if param.type == 'variable' and not self.objectspace.paths.path_is_defined(param.text):
-                        if param.optional is True:
-                            param_option_indexes.append(idx)
-                        else:
-                            raise DictConsistencyError(_(f'unknown param {param.text} in check'))
-                    if param.type != 'variable':
-                        param.notraisepropertyerror = None
-                param_option_indexes = list(set(param_option_indexes))
-                param_option_indexes.sort(reverse=True)
-                for idx in param_option_indexes:
-                    check.param.pop(idx)
-                if check.param == []:
-                    remove_indexes.append(check_idx)
-        remove_indexes.sort(reverse=True)
-        for idx in remove_indexes:
-            del self.objectspace.space.constraints.check[idx]
-
-    def check_replace_text(self):
-        for check_idx, check in enumerate(self.objectspace.space.constraints.check):
-            namespace = check.namespace
-            if hasattr(check, 'param'):
-                for idx, param in enumerate(check.param):
-                    if param.type == 'variable':
-                        param.text = self.objectspace.paths.get_variable_path(param.text, namespace)
-            check.is_in_leadership = self.objectspace.paths.get_leader(check.target) != None
-            # let's replace the target by the path
-            check.target = self.objectspace.paths.get_variable_path(check.target, namespace)
-
-    def check_valid_enum(self):
-        remove_indexes = []
-        for idx, check in enumerate(self.objectspace.space.constraints.check):
-            if check.name == 'valid_enum':
-                if len(check.param) != 1:
-                    raise DictConsistencyError(_(f'cannot set more than one param for valid_enum for variable {check.target}'))
-                param = check.param[0]
-                if check.target in self.valid_enums:
-                    raise DictConsistencyError(_(f'valid_enum already set for {check.target}'))
-                if param.type not in ['string', 'python', 'number']:
-                    raise DictConsistencyError(_(f'unknown type {param.type} for param in valid_enum for {check.target}'))
-                variable = self.objectspace.paths.get_variable_obj(check.target)
-                values = self.load_params_in_validenum(param,
-                                                       variable.name,
-                                                       variable.type,
-                                                       )
-                self.valid_enums[check.target] = {'type': param.type,
-                                                  'values': values}
-                self._set_valid_enum(variable,
-                                     values,
-                                     variable.type,
-                                     )
-                remove_indexes.append(idx)
-        remove_indexes.sort(reverse=True)
-        for idx in remove_indexes:
-            del self.objectspace.space.constraints.check[idx]
-
-    def load_params_in_validenum(self,
-                                 param,
-                                 variable_name,
-                                 variable_type,
-                                 ):
-        if not hasattr(param, 'text') and (param.type == 'python' or param.type == 'number'):
-            raise DictConsistencyError(_(f"All '{param.type}' variables shall be set in order to calculate valid_enum for variable {variable_name}"))
-        if variable_type == 'string' and param.type == 'number':
-            raise DictConsistencyError(_(f'Unconsistency valid_enum type ({param.type}), for variable {variable_name}'))
-        if param.type == 'python':
-            try:
-                values = eval(param.text, {'eosfunc': self.eosfunc, '__builtins__': {'range': range, 'str': str}})
-            except NameError:
-                raise DictConsistencyError(_('The function {} is unknown').format(param.text))
-        else:
-            try:
-                values = literal_eval(param.text)
-            except ValueError:
-                raise DictConsistencyError(_(f'Cannot load {param.text} in valid_enum'))
-        if not isinstance(values, list):
-            raise DictConsistencyError(_('Function {} shall return a list').format(param.text))
-        for value in values:
-            if variable_type == 'string' and not isinstance(value, str):
-                raise DictConsistencyError(_(f'Cannot load "{param.text}", "{value}" is not a string'))
-            if variable_type == 'number' and not isinstance(value, int):
-                raise DictConsistencyError(_(f'Cannot load "{param.text}", "{value}" is not a number'))
-        return values
-
-    def check_change_warning(self):
-        #convert level to "warnings_only"
-        for check in self.objectspace.space.constraints.check:
-            if check.level == 'warning':
-                check.warnings_only = True
-            else:
-                check.warnings_only = False
-            check.level = None
-
-    def _get_family_variables_from_target(self,
-                                          target,
-                                          ):
-        if target.type == 'variable':
-            variable = self.objectspace.paths.get_variable_obj(target.name)
-            family = self.objectspace.paths.get_family_obj(target.name.rsplit('.', 1)[0])
-            if isinstance(family, self.objectspace.Leadership) and family.name == variable.name:
-                return family, family.variable
-            return variable, [variable]
-        # it's a family
-        variable = self.objectspace.paths.get_family_obj(target.name)
-        return variable, list(variable.variable.values())
-
-    def check_params_target(self):
-        for condition in self.objectspace.space.constraints.condition:
-            for param in condition.param:
-                if param.type not in TYPE_PARAM_CONDITION:
-                    raise DictConsistencyError(_(f'cannot use {param.type} type as a param in a condition'))
-            if not hasattr(condition, 'target'):
-                raise DictConsistencyError(_('target is mandatory in condition'))
-            for target in condition.target:
-                if target.type.endswith('list') and condition.name not in ['disabled_if_in', 'disabled_if_not_in']:
-                    raise DictConsistencyError(_(f'target in condition for {target.type} not allow in {condition.name}'))
-
-    def filter_targets(self):  # pylint: disable=C0111
-        for condition_idx, condition in enumerate(self.objectspace.space.constraints.condition):
-            namespace = condition.namespace
-            for idx, target in enumerate(condition.target):
-                if target.type == 'variable':
-                    if condition.source == target.name:
-                        raise DictConsistencyError(_('target name and source name must be different: {}').format(condition.source))
-                    try:
-                        target.name = self.objectspace.paths.get_variable_path(target.name, namespace)
-                    except DictConsistencyError:
-                        # for optional variable
-                        pass
-                elif target.type == 'family':
-                    try:
-                        target.name = self.objectspace.paths.get_family_path(target.name, namespace)
-                    except KeyError:
-                        raise DictConsistencyError(_('cannot found family {}').format(target.name))
-
-    def convert_xxxlist_to_variable(self):  # pylint: disable=C0111
-        # transform *list to variable or family
-        for condition_idx, condition in enumerate(self.objectspace.space.constraints.condition):
-            new_targets = []
-            remove_targets = []
-            for target_idx, target in enumerate(condition.target):
-                if target.type.endswith('list'):
-                    listname = target.type
-                    listvars = self.objectspace.list_conditions.get(listname,
-                                                                    {}).get(target.name)
-                    if listvars:
-                        for listvar in listvars:
-                            variable = self.objectspace.paths.get_variable_obj(listvar)
-                            type_ = 'variable'
-                            new_target = self.objectspace.target()
-                            new_target.type = type_
-                            new_target.name = listvar
-                            new_target.index = target.index
-                            new_targets.append(new_target)
-                    remove_targets.append(target_idx)
-            remove_targets.sort(reverse=True)
-            for target_idx in remove_targets:
-                condition.target.pop(target_idx)
-            condition.target.extend(new_targets)
-
-    def check_condition_fallback_optional(self):
-        # a condition with a fallback **and** the source variable doesn't exist
-        remove_conditions = []
-        for idx, condition in enumerate(self.objectspace.space.constraints.condition):
-            # fallback
-            if condition.fallback is True and not self.objectspace.paths.path_is_defined(condition.source):
-                apply_action = False
-                if condition.name in ['disabled_if_in', 'mandatory_if_in', 'hidden_if_in']:
-                    apply_action = not condition.force_condition_on_fallback
-                else:
-                    apply_action = condition.force_inverse_condition_on_fallback
-                if apply_action:
-                    actions = self._get_condition_actions(condition.name)
-                    for target in condition.target:
-                        leader_or_variable, variables = self._get_family_variables_from_target(target)
-                        for action_idx, action in enumerate(actions):
-                            if action_idx == 0:
-                                setattr(leader_or_variable, action, True)
-                            else:
-                                for variable in variables:
-                                    setattr(variable, action, True)
-                remove_conditions.append(idx)
-                continue
-
-            remove_targets = []
-            # optional
-            for idx, target in enumerate(condition.target):
-                if target.optional is True and not self.objectspace.paths.path_is_defined(target.name):
-                    remove_targets.append(idx)
-            remove_targets = list(set(remove_targets))
-            remove_targets.sort(reverse=True)
-            for idx in remove_targets:
-                condition.target.pop(idx)
-        remove_conditions = list(set(remove_conditions))
-        remove_conditions.sort(reverse=True)
-        for idx in remove_conditions:
-            self.objectspace.space.constraints.condition.pop(idx)
-
-    def _get_condition_actions(self, condition_name):
-        if condition_name.startswith('disabled_if_'):
-            return ['disabled']
-        elif condition_name.startswith('hidden_if_'):
-            return ['hidden', 'frozen', 'force_default_on_freeze']
-        elif condition_name.startswith('mandatory_if_'):
-            return ['mandatory']
-        elif condition_name == 'auto_hidden_if_not_in':
-            return ['auto_frozen']
-
-    def check_choice_option_condition(self):
-        # remove condition for ChoiceOption that don't have param
-        remove_conditions = []
-        for condition_idx, condition in enumerate(self.objectspace.space.constraints.condition):
-            namespace = condition.namespace
-            condition.source = self.objectspace.paths.get_variable_path(condition.source, namespace, allow_source=True)
-            src_variable = self.objectspace.paths.get_variable_obj(condition.source)
-            valid_enum = None
-            if condition.source in self.valid_enums and self.valid_enums[condition.source]['type'] == 'string':
-                valid_enum = self.valid_enums[condition.source]['values']
-            if valid_enum is not None:
-                remove_param = []
-                for param_idx, param in enumerate(condition.param):
-                    if param.text not in valid_enum:
-                        remove_param.append(param_idx)
-                remove_param.sort(reverse=True)
-                for idx in remove_param:
-                    del condition.param[idx]
-                if condition.param == []:
-                    remove_targets = []
-                    for target in condition.target:
-                        leader_or_variable, variables = self._get_family_variables_from_target(target)
-                        if condition.name == 'disabled_if_not_in':
-                            leader_or_variable.disabled = True
-                        elif condition.name == 'hidden_if_not_in':
-                            leader_or_variable.hidden = True
-                            for variable in variables:
-                                variable.frozen = True
-                                variable.force_default_on_freeze = True
-                        elif condition.name == 'mandatory_if_not_in':
-                            variable.mandatory = True
-                    remove_targets = list(set(remove_targets))
-                    remove_targets.sort(reverse=True)
-                    for target_idx in remove_targets:
-                        condition.target.pop(target_idx)
-                    remove_conditions.append(condition_idx)
-        remove_conditions = list(set(remove_conditions))
-        remove_conditions.sort(reverse=True)
-        for idx in remove_conditions:
-            self.objectspace.space.constraints.condition.pop(idx)
-
-    def remove_condition_with_empty_target(self):
-        remove_conditions = []
-        for condition_idx, condition in enumerate(self.objectspace.space.constraints.condition):
-            if not condition.target:
-                remove_conditions.append(condition_idx)
-        remove_conditions = list(set(remove_conditions))
-        remove_conditions.sort(reverse=True)
-        for idx in remove_conditions:
-            self.objectspace.space.constraints.condition.pop(idx)
-
-    def convert_condition(self):
-        for condition in self.objectspace.space.constraints.condition:
-            inverse = condition.name.endswith('_if_not_in')
-            actions = self._get_condition_actions(condition.name)
-            for param in condition.param:
-                if hasattr(param, 'text'):
-                    param = param.text
-                else:
-                    param = None
-                for target in condition.target:
-                    leader_or_variable, variables = self._get_family_variables_from_target(target)
-                    # if option is already disable, do not apply disable_if_in
-                    if hasattr(leader_or_variable, actions[0]) and getattr(leader_or_variable, actions[0]) is True:
-                        continue
-                    for idx, action in enumerate(actions):
-                        prop = self.objectspace.property_()
-                        prop.type = 'calculation'
-                        prop.inverse = inverse
-                        prop.source = condition.source
-                        prop.expected = param
-                        prop.name = action
-                        if idx == 0:
-                            if not hasattr(leader_or_variable, 'property'):
-                                leader_or_variable.property = []
-                            leader_or_variable.property.append(prop)
-                        else:
-                            for variable in variables:
-                                if not hasattr(variable, 'property'):
-                                    variable.property = []
-                                variable.property.append(prop)
-        del self.objectspace.space.constraints.condition
-
-    def _set_valid_enum(self, variable, values, type_):
-        # value for choice's variable is mandatory
-        variable.mandatory = True
-        # build choice
-        variable.choice = []
-        choices = []
-        for value in values:
-            choice = self.objectspace.choice()
-            try:
-                choice.name = CONVERSION.get(type_, str)(value)
-            except:
-                raise DictConsistencyError(_(f'unable to change type of a valid_enum entry "{value}" is not a valid "{type_}" for "{variable.name}"'))
-            if choice.name == '':
-                choice.name = None
-            choices.append(choice.name)
-            choice.type = type_
-            variable.choice.append(choice)
-        if not variable.choice:
-            raise DictConsistencyError(_('empty valid enum is not allowed for variable {}').format(variable.name))
-        # check value or set first choice value has default value
-        if hasattr(variable, 'value'):
-            for value in variable.value:
-                value.type = type_
-                try:
-                    cvalue = CONVERSION.get(type_, str)(value.name)
-                except:
-                    raise DictConsistencyError(_(f'unable to change type of value "{value}" is not a valid "{type_}" for "{variable.name}"'))
-                if cvalue not in choices:
-                    raise DictConsistencyError(_('value "{}" of variable "{}" is not in list of all expected values ({})').format(value.name, variable.name, choices))
-        else:
-            new_value = self.objectspace.value()
-            new_value.name = values[0]
-            new_value.type = type_
-            variable.value = [new_value]
-        variable.type = 'choice'
-
-    def convert_check(self):
-        for check in self.objectspace.space.constraints.check:
-            variable = self.objectspace.paths.get_variable_obj(check.target)
-            name = check.name
-            if name == 'valid_entier':
-                if not hasattr(check, 'param'):
-                    raise DictConsistencyError(_('{} must have, at least, 1 param').format(name))
-                for param in check.param:
-                    if param.type not in ['string', 'number']:
-                        raise DictConsistencyError(_(f'param in "valid_entier" must not be a "{param.type}"'))
-                    if param.name == 'mini':
-                        variable.min_number = int(param.text)
-                    elif param.name == 'maxi':
-                        variable.max_number = int(param.text)
-                    else:
-                        raise DictConsistencyError(_(f'unknown parameter {param.text} in check "valid_entier" for variable {check.target}'))
-            else:
-                check_ = self.objectspace.check()
-                if name == 'valid_differ':
-                    name = 'valid_not_equal'
-                elif name == 'valid_network_netmask':
-                    params_len = 1
-                    if len(check.param) != params_len:
-                        raise DictConsistencyError(_('{} must have {} param').format(name, params_len))
-                elif name == 'valid_ipnetmask':
-                    params_len = 1
-                    if len(check.param) != params_len:
-                        raise DictConsistencyError(_('{} must have {} param').format(name, params_len))
-                    name = 'valid_ip_netmask'
-                elif name == 'valid_broadcast':
-                    params_len = 2
-                    if len(check.param) != params_len:
-                        raise DictConsistencyError(_('{} must have {} param').format(name, params_len))
-                elif name == 'valid_in_network':
-                    params_len = 2
-                    if len(check.param) != params_len:
-                        raise DictConsistencyError(_('{} must have {} param').format(name, params_len))
-                check_.name = name
-                check_.warnings_only = check.warnings_only
-                if hasattr(check, 'param'):
-                    check_.param = check.param
-                if not hasattr(variable, 'check'):
-                    variable.check = []
-                variable.check.append(check_)
-        del self.objectspace.space.constraints.check
-
-    def convert_fill(self):  # pylint: disable=C0111,R0912
-        # sort fill/auto by index
-        fills = {fill.index: fill for idx, fill in enumerate(self.objectspace.space.constraints.fill)}
-        indexes = list(fills.keys())
-        indexes.sort()
-        targets = []
-        eosfunc = dir(self.eosfunc)
-        for idx in indexes:
-            fill = fills[idx]
-            # test if it's redefined calculation
-            if fill.target in targets and not fill.redefine:
-                raise DictConsistencyError(_(f"A fill already exists for the target: {fill.target}"))
-            targets.append(fill.target)
-            #
-            if fill.name not in eosfunc:
-                raise DictConsistencyError(_('cannot find fill function {}').format(fill.name))
-
-            namespace = fill.namespace
-            # let's replace the target by the path
-            fill.target = self.objectspace.paths.get_variable_path(fill.target,
-                                                                   namespace,
-                                                                   )
-
-            value = self.objectspace.value()
-            value.type = 'calculation'
-            value.name = fill.name
-            if hasattr(fill, 'param'):
-                param_to_delete = []
-                for fill_idx, param in enumerate(fill.param):
-                    if param.type not in TYPE_PARAM_FILL:
-                        raise DictConsistencyError(_(f'cannot use {param.type} type as a param in a fill/auto'))
-                    if param.type != 'string' and not hasattr(param, 'text'):
-                        raise DictConsistencyError(_(f"All '{param.type}' variables shall have a value in order to calculate {fill.target}"))
-                    if param.type == 'variable':
-                        try:
-                            param.text, suffix = self.objectspace.paths.get_variable_path(param.text,
-                                                                                          namespace,
-                                                                                          with_suffix=True,
-                                                                                          )
-                            if suffix:
-                                param.suffix = suffix
-                        except DictConsistencyError as err:
-                            if param.optional is False:
-                                raise err
-                            param_to_delete.append(fill_idx)
-                            continue
-                    else:
-                        param.notraisepropertyerror = None
-                param_to_delete.sort(reverse=True)
-                for param_idx in param_to_delete:
-                    fill.param.pop(param_idx)
-                value.param = fill.param
-            variable = self.objectspace.paths.get_variable_obj(fill.target)
-            variable.value = [value]
-        del self.objectspace.space.constraints.fill
-
-    def remove_constraints(self):
-        if hasattr(self.objectspace.space.constraints, 'index'):
-            del self.objectspace.space.constraints.index
-        del self.objectspace.space.constraints.namespace
-        if vars(self.objectspace.space.constraints):
-            raise Exception('constraints again?')
-        del self.objectspace.space.constraints
-
-
-class FamilyAnnotator:
-    def __init__(self,
-                 objectspace,
-                 ):
-        self.objectspace = objectspace
-        self.remove_empty_families()
-        self.change_variable_mode()
-        self.change_family_mode()
-        self.dynamic_families()
-
-    def remove_empty_families(self):  # pylint: disable=C0111,R0201
-        if hasattr(self.objectspace.space, 'variables'):
-            for family in self.objectspace.space.variables.values():
-                if hasattr(family, 'family'):
-                    space = family.family
-                    removed_families = []
-                    for family_name, family in space.items():
-                        if not hasattr(family, 'variable') or len(family.variable) == 0:
-                            removed_families.append(family_name)
-                    for family_name in removed_families:
-                        del space[family_name]
-
-    def change_family_mode(self):  # pylint: disable=C0111
-        if not hasattr(self.objectspace.space, 'variables'):
-            return
-        for family in self.objectspace.space.variables.values():
-            if hasattr(family, 'family'):
-                for family in family.family.values():
-                    mode = modes_level[-1]
-                    for variable in family.variable.values():
-                        if isinstance(variable, self.objectspace.Leadership):
-                            variable_mode = variable.variable[0].mode
-                            variable.variable[0].mode = None
-                            variable.mode = variable_mode
-                        else:
-                            variable_mode = variable.mode
-                        if variable_mode is not None and modes[mode] > modes[variable_mode]:
-                            mode = variable_mode
-                    family.mode = mode
-
-    def dynamic_families(self):  # pylint: disable=C0111
-        if not hasattr(self.objectspace.space, 'variables'):
-            return
-        for family in self.objectspace.space.variables.values():
-            if hasattr(family, 'family'):
-                for family in family.family.values():
-                    if 'dynamic' in vars(family):
-                        namespace = self.objectspace.paths.get_variable_namespace(family.dynamic)
-                        varpath = self.objectspace.paths.get_variable_path(family.dynamic, namespace)
-                        family.dynamic = varpath
-
-    def annotate_variable(self, variable, family_mode, path, is_follower=False):
-        # if the variable is mandatory and doesn't have any value
-        # then the variable's mode is set to 'basic'
-        if not hasattr(variable, 'value') and variable.type == 'boolean':
-            new_value = self.objectspace.value()
-            new_value.name = True
-            new_value.type = 'boolean'
-            variable.value = [new_value]
-        if hasattr(variable, 'value') and variable.value:
-            has_value = True
-            for value in variable.value:
-                if value.type == 'calculation':
-                    has_value = False
-                    has_variable = False
-                    if hasattr(value, 'param'):
-                        for param in value.param:
-                            if param.type == 'variable':
-                                has_variable = True
-                                break
-                    if not has_variable:
-                        # if one parameter is a variable, let variable choice if it's mandatory
-                        variable.mandatory = True
-            if has_value:
-                # if has value but without any calculation
-                variable.mandatory = True
-        if variable.mandatory is True and (not hasattr(variable, 'value') or is_follower):
-            variable.mode = modes_level[0]
-        if variable.mode != None and modes[variable.mode] < modes[family_mode] and (not is_follower or variable.mode != modes_level[0]):
-            variable.mode = family_mode
-        if variable.hidden is True:
-            variable.frozen = True
-            if not variable.auto_save is True and 'force_default_on_freeze' not in vars(variable):
-                variable.force_default_on_freeze = True
-
-    def change_variable_mode(self):  # pylint: disable=C0111
-        if not hasattr(self.objectspace.space, 'variables'):
-            return
-        for variables in self.objectspace.space.variables.values():
-            namespace = variables.name
-            if hasattr(variables, 'family'):
-                for family in variables.family.values():
-                    family_mode = family.mode
-                    if hasattr(family, 'variable'):
-                        for variable in family.variable.values():
-
-                            if isinstance(variable, self.objectspace.Leadership):
-                                mode = modes_level[-1]
-                                for idx, follower in enumerate(variable.variable):
-                                    if follower.auto_save is True:
-                                        raise DictConsistencyError(_(f'leader/followers {follower.name} could not be auto_save'))
-                                    if follower.auto_freeze is True:
-                                        raise DictConsistencyError(_('leader/followers {follower.name} could not be auto_freeze'))
-                                    is_follower = idx != 0
-                                    path = '{}.{}.{}'.format(family.path, variable.name, follower.name)
-                                    self.annotate_variable(follower, family_mode, path, is_follower)
-                                    # leader's mode is minimum level
-                                    if modes[variable.variable[0].mode] > modes[follower.mode]:
-                                        follower.mode = variable.variable[0].mode
-                                variable.mode = variable.variable[0].mode
-                            else:
-                                # auto_save's variable is set in 'basic' mode if its mode is 'normal'
-                                if variable.auto_save is True and variable.mode != modes_level[-1]:
-                                    variable.mode = modes_level[0]
-                                # auto_freeze's variable is set in 'basic' mode if its mode is 'normal'
-                                if variable.auto_freeze is True and variable.mode != modes_level[-1]:
-                                    variable.mode = modes_level[0]
-                                path = '{}.{}'.format(family.path, variable.name)
-                                self.annotate_variable(variable, family_mode, path)
-
-class PropertyAnnotator:
-    def __init__(self, objectspace):
-        self.objectspace = objectspace
-        self.convert_annotator()
-
-    def convert_property(self,
-                         variable,
-                         ):
-        properties = []
-        for prop in PROPERTIES:
-            if hasattr(variable, prop):
-                if getattr(variable, prop) == True:
-                    for subprop in CONVERT_PROPERTIES.get(prop, [prop]):
-                        properties.append(subprop)
-                setattr(variable, prop, None)
-        if hasattr(variable, 'mode') and variable.mode:
-            properties.append(variable.mode)
-            variable.mode = None
-        if properties:
-            variable.properties = frozenset(properties)
-
-    def convert_annotator(self):  # pylint: disable=C0111
-        if hasattr(self.objectspace.space, 'services'):
-            self.convert_property(self.objectspace.space.services)
-            for services in self.objectspace.space.services.service.values():
-                self.convert_property(services)
-                for service in vars(services).values():
-                    if isinstance(service, self.objectspace.family):
-                        self.convert_property(service)
-                        if hasattr(service, 'family'):
-                            self.convert_property(service)
-                            for family in service.family:
-                                self.convert_property(family)
-                                if hasattr(family, 'variable'):
-                                    for variable in family.variable:
-                                        self.convert_property(variable)
-        if hasattr(self.objectspace.space, 'variables'):
-            for variables in self.objectspace.space.variables.values():
-                if hasattr(variables, 'family'):
-                    for family in variables.family.values():
-                        self.convert_property(family)
-                        if hasattr(family, 'variable'):
-                            for variable in family.variable.values():
-                                if isinstance(variable, self.objectspace.Leadership):
-                                    self.convert_property(variable)
-                                    for follower in variable.variable:
-                                        self.convert_property(follower)
-                                else:
-                                    self.convert_property(variable)

src/rougail/annotator/__init__.py

@@ -0,0 +1,57 @@
+"""Annotate dictionaries
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from .group import GroupAnnotator
+from .service import ServiceAnnotator
+from .variable import VariableAnnotator, CONVERT_OPTION
+from .check import CheckAnnotator
+from .value import ValueAnnotator
+from .condition import ConditionAnnotator
+from .fill import FillAnnotator
+from .family import FamilyAnnotator, modes
+from .property import PropertyAnnotator
+
+class SpaceAnnotator:  # pylint: disable=R0903
+    """Transformations applied on a CreoleObjSpace instance
+    """
+    def __init__(self, objectspace, eosfunc_file):
+        self.objectspace = objectspace
+        GroupAnnotator(objectspace)
+        ServiceAnnotator(objectspace)
+        VariableAnnotator(objectspace)
+        CheckAnnotator(objectspace,
+                       eosfunc_file,
+                       )
+        ConditionAnnotator(objectspace)
+        FillAnnotator(objectspace,
+                      eosfunc_file,
+                      )
+        ValueAnnotator(objectspace)
+        FamilyAnnotator(objectspace)
+        PropertyAnnotator(objectspace)
+
+
+__all__ = ('SpaceAnnotator', 'CONVERT_OPTION', 'modes')

src/rougail/annotator/check.py

@@ -0,0 +1,228 @@
+"""Annotate check
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from typing import List, Any
+
+from .variable import CONVERT_OPTION
+from .target import TargetAnnotator
+from .param import ParamAnnotator
+
+from ..utils import load_modules
+from ..i18n import _
+from ..error import DictConsistencyError, display_xmlfiles
+
+INTERNAL_FUNCTIONS = ['valid_enum', 'valid_in_network', 'valid_differ', 'valid_entier']
+
+class CheckAnnotator(TargetAnnotator, ParamAnnotator):
+    """Annotate check
+    """
+    def __init__(self,
+                 objectspace,
+                 eosfunc_file,
+                 ):
+        if not hasattr(objectspace.space, 'constraints') or \
+                not hasattr(objectspace.space.constraints, 'check'):
+            return
+        self.objectspace = objectspace
+        self.let_none = True
+        self.only_variable = True
+        self.functions = dir(load_modules(eosfunc_file))
+        self.functions.extend(INTERNAL_FUNCTIONS)
+        self.target_is_uniq = False
+        self.convert_target(self.objectspace.space.constraints.check)
+        self.convert_param(self.objectspace.space.constraints.check)
+        self.check_check()
+        self.check_valid_enum()
+        self.check_change_warning()
+        self.convert_valid_entier()
+        self.convert_check()
+        del objectspace.space.constraints.check
+
+    def check_check(self):  # pylint: disable=R0912
+        """valid and manage <check>
+        """
+        remove_indexes = []
+        for check_idx, check in enumerate(self.objectspace.space.constraints.check):
+            if not check.name in self.functions:
+                msg = _(f'cannot find check function "{check.name}"')
+                raise DictConsistencyError(msg, 1, check.xmlfiles)
+            if hasattr(check, 'param') and check.param == []:
+                remove_indexes.append(check_idx)
+        remove_indexes.sort(reverse=True)
+        for idx in remove_indexes:
+            del self.objectspace.space.constraints.check[idx]
+
+    def check_valid_enum(self):
+        """verify valid_enum
+        """
+        remove_indexes = []
+        for idx, check in enumerate(self.objectspace.space.constraints.check):
+            if check.name != 'valid_enum':
+                continue
+            for target in check.target:
+                if target.name.path in self.objectspace.valid_enums:
+                    check_xmlfiles = display_xmlfiles(self.objectspace.valid_enums\
+                            [target.name.path]['xmlfiles'])
+                    msg = _(f'valid_enum already set in {check_xmlfiles} '
+                            f'for "{target.name.name}", you may have forget remove_check')
+                    raise DictConsistencyError(msg, 3, check.xmlfiles)
+                if not hasattr(check, 'param'):
+                    msg = _(f'param is mandatory for a valid_enum of variable "{target.name.name}"')
+                    raise DictConsistencyError(msg, 4, check.xmlfiles)
+                variable_type = target.name.type
+                values = self._set_valid_enum(target.name,
+                                              check,
+                                              )
+                if values:
+                    if hasattr(target.name, 'value'):
+                        # check value
+                        self.check_valid_enum_value(target.name, values)
+                    else:
+                        # no value, set the first choice has default value
+                        new_value = self.objectspace.value(check.xmlfiles)
+                        new_value.name = values[0]
+                        new_value.type = variable_type
+                        target.name.value = [new_value]
+            remove_indexes.append(idx)
+        remove_indexes.sort(reverse=True)
+        for idx in remove_indexes:
+            del self.objectspace.space.constraints.check[idx]
+
+    def _set_valid_enum(self,
+                        variable,
+                        check,
+                        ) -> List[Any]:
+        # value for choice's variable is mandatory
+        variable.mandatory = True
+        # build choice
+        variable.values = []
+        variable_type = variable.type
+        variable.type = 'choice'
+
+        has_variable = False
+        values = []
+        for param in check.param:
+            if has_variable:
+                msg = _(f'only one "variable" parameter is allowed for valid_enum '
+                        f'of variable "{variable.name}"')
+                raise DictConsistencyError(msg, 5, param.xmlfiles)
+            param_type = variable_type
+            if param.type == 'variable':
+                has_variable = True
+                if param.optional is True:
+                    msg = _(f'optional parameter in valid_enum for variable "{variable.name}" '
+                            f'is not allowed')
+                    raise DictConsistencyError(msg, 14, param.xmlfiles)
+                if not param.text.multi:
+                    msg = _(f'only multi "variable" parameter is allowed for valid_enum '
+                            f'of variable "{variable.name}"')
+                    raise DictConsistencyError(msg, 6, param.xmlfiles)
+                param_type = 'calculation'
+                value = param.text
+            elif param.type == 'nil':
+                value = None
+            else:
+                if 'type' in vars(param) and variable_type != param.type:
+                    msg = _(f'parameter in valid_enum has incompatible type "{param.type}" '
+                            f'with type of the variable "{variable.name}" ("{variable_type}")')
+                    raise DictConsistencyError(msg, 7, param.xmlfiles)
+                if hasattr(param, 'text'):
+                    try:
+                        value = CONVERT_OPTION[variable_type].get('func', str)(param.text)
+                    except ValueError as err:
+                        msg = _(f'unable to change type of a valid_enum entry "{param.text}" '
+                                f'is not a valid "{variable_type}" for "{variable.name}"')
+                        raise DictConsistencyError(msg, 13, variable.xmlfiles) from err
+            values.append(value)
+            choice = self.objectspace.choice(variable.xmlfiles)
+            choice.name = value
+            choice.type = param_type
+            variable.values.append(choice)
+
+        if has_variable:
+            return None
+
+        for target in check.target:
+            self.objectspace.valid_enums[target.name.path] = {'type': variable_type,
+                                                              'values': values,
+                                                              'xmlfiles': check.xmlfiles,
+                                                              }
+        return values
+
+    @staticmethod
+    def check_valid_enum_value(variable,
+                               values,
+                               ) -> None:
+        """check that values in valid_enum are valid
+        """
+        for value in variable.value:
+            if value.name not in values:
+                msg = _(f'value "{value.name}" of variable "{variable.name}" is not in list '
+                        f'of all expected values ({values})')
+                raise DictConsistencyError(msg, 15, value.xmlfiles)
+
+    def check_change_warning(self):
+        """convert level to "warnings_only"
+        """
+        for check in self.objectspace.space.constraints.check:
+            check.warnings_only = check.level == 'warning'
+            check.level = None
+
+    def convert_valid_entier(self) -> None:
+        """valid and manage <check>
+        """
+        remove_indexes = []
+        for check_idx, check in enumerate(self.objectspace.space.constraints.check):
+            if not check.name == 'valid_entier':
+                continue
+            remove_indexes.append(check_idx)
+            if not hasattr(check, 'param'):
+                msg = _(f'{check.name} must have, at least, 1 param')
+                raise DictConsistencyError(msg, 17, check.xmlfiles)
+            for param in check.param:
+                if param.type != 'number':
+                    msg = _(f'param in "valid_entier" must be an "integer", not "{param.type}"')
+                    raise DictConsistencyError(msg, 18, check.xmlfiles)
+                for target in check.target:
+                    if param.name == 'mini':
+                        target.name.min_number = int(param.text)
+                    elif param.name == 'maxi':
+                        target.name.max_number = int(param.text)
+                    else:
+                        msg = _(f'unknown parameter "{param.name}" in check "valid_entier"')
+                        raise DictConsistencyError(msg, 19, check.xmlfiles)
+        remove_indexes.sort(reverse=True)
+        for idx in remove_indexes:
+            del self.objectspace.space.constraints.check[idx]
+
+    def convert_check(self) -> None:
+        """valid and manage <check>
+        """
+        for check in self.objectspace.space.constraints.check:
+            for target in check.target:
+                if not hasattr(target.name, 'validators'):
+                    target.name.validators = []
+                target.name.validators.append(check)

src/rougail/annotator/condition.py

@@ -0,0 +1,367 @@
+"""Annotate condition
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from typing import List, Any
+
+
+from ..i18n import _
+from ..error import DictConsistencyError
+from ..config import Config
+
+from .target import TargetAnnotator
+from .param import ParamAnnotator
+
+FREEZE_AUTOFREEZE_VARIABLE = 'module_instancie'
+
+
+class ConditionAnnotator(TargetAnnotator, ParamAnnotator):
+    """Annotate condition
+    """
+    def __init__(self,
+                 objectspace,
+                 ):
+        self.objectspace = objectspace
+        self.force_service_value = {}
+        if hasattr(objectspace.space, 'variables'):
+            self.convert_auto_freeze()
+        if not hasattr(objectspace.space, 'constraints') or \
+                not hasattr(self.objectspace.space.constraints, 'condition'):
+            return
+        self.target_is_uniq = False
+        self.only_variable = False
+        self.convert_target(self.objectspace.space.constraints.condition)
+        self.convert_param(self.objectspace.space.constraints.condition)
+        self.check_source_target()
+        self.check_condition_fallback()
+        self.convert_xxxlist()
+        self.convert_condition_source()
+        self.check_choice_option_condition()
+        self.remove_condition_with_empty_target()
+        self.convert_condition()
+
+    def convert_auto_freeze(self):
+        """convert auto_freeze
+        only if FREEZE_AUTOFREEZE_VARIABLE == 'oui' this variable is frozen
+        """
+        for variables in self.objectspace.space.variables.values():
+            for family in variables.family.values():
+                if not hasattr(family, 'variable'):
+                    continue
+                for variable in family.variable.values():
+                    if isinstance(variable, self.objectspace.leadership):
+                        for follower in variable.variable:
+                            self._convert_auto_freeze(follower)
+                    else:
+                        self._convert_auto_freeze(variable)
+
+    def _convert_auto_freeze(self,
+                             variable: 'self.objectspace.variable',
+                             ) -> None:
+        if not variable.auto_freeze:
+            return
+        if variable.namespace != Config['variable_namespace']:
+            msg = _(f'auto_freeze is not allowed in extra "{variable.namespace}"')
+            raise DictConsistencyError(msg, 49, variable.xmlfiles)
+        new_condition = self.objectspace.condition(variable.xmlfiles)
+        new_condition.name = 'auto_frozen_if_not_in'
+        new_condition.namespace = variable.namespace
+        new_condition.source = FREEZE_AUTOFREEZE_VARIABLE
+        new_param = self.objectspace.param(variable.xmlfiles)
+        new_param.text = 'oui'
+        new_condition.param = [new_param]
+        new_target = self.objectspace.target(variable.xmlfiles)
+        new_target.type = 'variable'
+        new_target.name = variable.name
+        new_condition.target = [new_target]
+        if not hasattr(self.objectspace.space, 'constraints'):
+            self.objectspace.space.constraints = self.objectspace.constraints(variable.xmlfiles)
+        if not hasattr(self.objectspace.space.constraints, 'condition'):
+            self.objectspace.space.constraints.condition = []
+        self.objectspace.space.constraints.condition.append(new_condition)
+
+    def check_source_target(self):
+        """verify that source != target in condition
+        """
+        for condition in self.objectspace.space.constraints.condition:
+            for target in condition.target:
+                if target.type == 'variable' and \
+                        condition.source in [target.name.name, target.name.path]:
+                    msg = _('target name and source name must be different: '
+                            f'{condition.source}')
+                    raise DictConsistencyError(msg, 11, condition.xmlfiles)
+
+    def check_condition_fallback(self):
+        """a condition with a fallback **and** the source variable doesn't exist
+        """
+        remove_conditions = []
+        for idx, condition in enumerate(self.objectspace.space.constraints.condition):
+            # fallback
+            if condition.fallback is False or \
+                    self.objectspace.paths.path_is_defined(condition.source):
+                continue
+            if condition.name in ['disabled_if_in', 'mandatory_if_in', 'hidden_if_in']:
+                apply_action = not condition.force_condition_on_fallback
+            else:
+                apply_action = condition.force_inverse_condition_on_fallback
+            remove_conditions.append(idx)
+            if apply_action:
+                self.force_actions_to_variable(condition)
+        remove_conditions.sort(reverse=True)
+        for idx in remove_conditions:
+            self.objectspace.space.constraints.condition.pop(idx)
+
+    def force_actions_to_variable(self,
+                                  condition: 'self.objectspace.condition',
+                                  ) -> None:
+        """force property to a variable
+        for example disabled_if_not_in => variable.disabled = True
+        """
+        actions = self.get_actions_from_condition(condition.name)
+        for target in condition.target:
+            main_action = actions[0]
+            if target.type.endswith('list'):
+                self.force_service_value[target.name] = main_action != 'disabled'
+                continue
+            leader_or_var, variables = self._get_family_variables_from_target(target)
+            setattr(leader_or_var, main_action, True)
+            for action in actions[1:]:
+                for variable in variables:
+                    setattr(variable, action, True)
+
+    @staticmethod
+    def get_actions_from_condition(condition_name: str) -> List[str]:
+        """get action's name from a condition
+        """
+        if condition_name.startswith('hidden_if_'):
+            return ['hidden', 'frozen', 'force_default_on_freeze']
+        if condition_name == 'auto_frozen_if_not_in':
+            return ['auto_frozen']
+        return [condition_name.split('_', 1)[0]]
+
+    def _get_family_variables_from_target(self,
+                                          target,
+                                          ):
+        if target.type == 'variable':
+            if not self.objectspace.paths.is_leader(target.name.path):
+                return target.name, [target.name]
+            # it's a leader, so apply property to leadership
+            family_name = self.objectspace.paths.get_variable_family_path(target.name.path)
+            family = self.objectspace.paths.get_family(family_name,
+                                                       target.name.namespace,
+                                                       )
+            return family, family.variable
+        # it's a family
+        variable = self.objectspace.paths.get_family(target.name.path,
+                                                     target.namespace,
+                                                     )
+        if hasattr(variable, 'variable'):
+            return variable, list(variable.variable.values())
+        return variable, []
+
+    def convert_xxxlist(self):
+        """transform *list to variable or family
+        """
+        fills = {}
+        for condition in self.objectspace.space.constraints.condition:
+            remove_targets = []
+            for target_idx, target in enumerate(condition.target):
+                if target.type.endswith('list'):
+                    listvars = self.objectspace.list_conditions.get(target.type,
+                                                                    {}).get(target.name)
+                    if listvars:
+                        self._convert_xxxlist_to_fill(condition,
+                                                      target,
+                                                      listvars,
+                                                      fills,
+                                                      )
+                    remove_targets.append(target_idx)
+            remove_targets.sort(reverse=True)
+            for target_idx in remove_targets:
+                condition.target.pop(target_idx)
+
+    def _convert_xxxlist_to_fill(self,
+                                 condition: 'self.objectspace.condition',
+                                 target: 'self.objectspace.target',
+                                 listvars: list,
+                                 fills: dict,
+                                 ):
+        for listvar in listvars:
+            if target.name in self.force_service_value:
+                listvar.default = self.force_service_value[target.name]
+                continue
+            if listvar.path in fills:
+                fill = fills[listvar.path]
+                or_needed = True
+                for param in fill.param:
+                    if hasattr(param, 'name') and \
+                            param.name == 'condition_operator':
+                        or_needed = False
+                        break
+                fill.index += 1
+            else:
+                fill = self.objectspace.fill(target.xmlfiles)
+                new_target = self.objectspace.target(target.xmlfiles)
+                new_target.name = listvar.path
+                fill.target = [new_target]
+                fill.name = 'calc_value'
+                fill.namespace = 'services'
+                fill.index = 0
+                if not hasattr(self.objectspace.space.constraints, 'fill'):
+                    self.objectspace.space.constraints.fill = []
+                self.objectspace.space.constraints.fill.append(fill)
+                fills[listvar.path] = fill
+                param1 = self.objectspace.param(target.xmlfiles)
+                param1.text = False
+                param1.type = 'boolean'
+                param2 = self.objectspace.param(target.xmlfiles)
+                param2.name = 'default'
+                param2.text = True
+                param2.type = 'boolean'
+                fill.param = [param1, param2]
+                or_needed = len(condition.param) != 1
+            if len(condition.param) == 1:
+                values = getattr(condition.param[0], 'text', None)
+            else:
+                values = tuple([getattr(param, 'text', None) for param in condition.param])
+            param3 = self.objectspace.param(target.xmlfiles)
+            param3.name = f'condition_{fill.index}'
+            param3.type = 'variable'
+            param3.text = condition.source
+            fill.param.append(param3)
+            param4 = self.objectspace.param(target.xmlfiles)
+            param4.name = f'expected_{fill.index}'
+            param4.text = values
+            fill.param.append(param4)
+            if condition.name != 'disabled_if_in':
+                param5 = self.objectspace.param(target.xmlfiles)
+                param5.name = f'reverse_condition_{fill.index}'
+                param5.text = True
+                param5.type = 'boolean'
+                fill.param.append(param5)
+            if or_needed:
+                param6 = self.objectspace.param(target.xmlfiles)
+                param6.name = 'condition_operator'
+                param6.text = 'OR'
+                fill.param.append(param6)
+
+    def convert_condition_source(self):
+        """remove condition for ChoiceOption that don't have param
+        """
+        for condition in self.objectspace.space.constraints.condition:
+            try:
+                condition.source = self.objectspace.paths.get_variable(condition.source)
+            except DictConsistencyError as err:
+                if err.errno == 36:
+                    msg = _(f'the source "{condition.source}" in condition cannot be a dynamic '
+                            f'variable')
+                    raise DictConsistencyError(msg, 20, condition.xmlfiles) from err
+                if err.errno == 42:
+                    msg = _(f'the source "{condition.source}" in condition is an unknown variable')
+                    raise DictConsistencyError(msg, 23, condition.xmlfiles) from err
+                raise err from err  # pragma: no cover
+
+    def check_choice_option_condition(self):
+        """remove condition of ChoiceOption that doesn't match
+        """
+        remove_conditions = []
+        for condition_idx, condition in enumerate(self.objectspace.space.constraints.condition):
+            if condition.source.path in self.objectspace.valid_enums:
+                valid_enum = self.objectspace.valid_enums[condition.source.path]['values']
+                remove_param = [param_idx for param_idx, param in enumerate(condition.param) \
+                                if param.type != 'variable' and param.text not in valid_enum]
+                if not remove_param:
+                    continue
+                remove_param.sort(reverse=True)
+                for idx in remove_param:
+                    del condition.param[idx]
+                if not condition.param and condition.name.endswith('_if_not_in'):
+                    self.force_actions_to_variable(condition)
+                    remove_conditions.append(condition_idx)
+        remove_conditions.sort(reverse=True)
+        for idx in remove_conditions:
+            self.objectspace.space.constraints.condition.pop(idx)
+
+    def remove_condition_with_empty_target(self):
+        """remove condition with empty target
+        """
+        # optional target are remove, condition could be empty
+        remove_conditions = [condition_idx for condition_idx, condition in \
+                             enumerate(self.objectspace.space.constraints.condition) \
+                             if not condition.target]
+        remove_conditions.sort(reverse=True)
+        for idx in remove_conditions:
+            self.objectspace.space.constraints.condition.pop(idx)
+
+    def convert_condition(self):
+        """valid and manage <condition>
+        """
+        for condition in self.objectspace.space.constraints.condition:
+            actions = self.get_actions_from_condition(condition.name)
+            for param in condition.param:
+                text = getattr(param, 'text', None)
+                for target in condition.target:
+                    leader_or_variable, variables = self._get_family_variables_from_target(target)
+                    # if option is already disable, do not apply disable_if_in
+                    # check only the first action (example of multiple actions:
+                    # 'hidden', 'frozen', 'force_default_on_freeze')
+                    main_action = actions[0]
+                    if getattr(leader_or_variable, main_action, False) is True:
+                        continue
+                    self.build_property(leader_or_variable,
+                                        text,
+                                        condition,
+                                        main_action,
+                                        )
+                    if isinstance(leader_or_variable, self.objectspace.variable) and \
+                            (leader_or_variable.auto_save or leader_or_variable.auto_freeze) and \
+                            'force_default_on_freeze' in actions:
+                        continue
+                    for action in actions[1:]:
+                        # other actions are set to the variable or children of family
+                        for variable in variables:
+                            self.build_property(variable,
+                                                text,
+                                                condition,
+                                                action,
+                                                )
+
+    def build_property(self,
+                       obj,
+                       text: Any,
+                       condition: 'self.objectspace.condition',
+                       action: str,
+                       ) -> 'self.objectspace.property_':
+        """build property_ for a condition
+        """
+        prop = self.objectspace.property_(obj.xmlfiles)
+        prop.type = 'calculation'
+        prop.inverse = condition.name.endswith('_if_not_in')
+        prop.source = condition.source
+        prop.expected = text
+        prop.name = action
+        if not hasattr(obj, 'properties'):
+            obj.properties = []
+        obj.properties.append(prop)

src/rougail/annotator/family.py

@@ -0,0 +1,184 @@
+"""Annotate family
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from ..i18n import _
+from ..error import DictConsistencyError
+from ..utils import normalize_family
+
+
+#mode order is important
+modes_level = ('basic', 'normal', 'expert')
+
+
+class Mode:  # pylint: disable=R0903
+    """Class to manage mode level
+    """
+    def __init__(self,
+                 name: str,
+                 level: int,
+                 ) -> None:
+        self.name = name
+        self.level = level
+
+    def __gt__(self,
+               other: int,
+               ) -> bool:
+        return other.level < self.level
+
+
+modes = {name: Mode(name, idx) for idx, name in enumerate(modes_level)}
+
+
+class FamilyAnnotator:
+    """Annotate family
+    """
+    def __init__(self,
+                 objectspace,
+                 ):
+        self.objectspace = objectspace
+        if not hasattr(self.objectspace.space, 'variables'):
+            return
+        self.remove_empty_families()
+        self.family_names()
+        self.change_modes()
+        self.dynamic_families()
+        self.convert_help()
+
+    def remove_empty_families(self) -> None:
+        """Remove all families without any variable
+        """
+        for families in self.objectspace.space.variables.values():
+            removed_families = []
+            for family_name, family in families.family.items():
+                if not hasattr(family, 'variable') or len(family.variable) == 0:
+                    removed_families.append(family_name)
+            for family_name in removed_families:
+                del families.family[family_name]
+
+    def family_names(self) -> None:
+        """Set doc, path, ... to family
+        """
+        for families in self.objectspace.space.variables.values():
+            families.doc = families.name
+            families.path = families.name
+            for family in families.family.values():
+                if not hasattr(family, 'description'):
+                    family.description = family.name
+                family.doc = family.description
+                del family.description
+                family.name = normalize_family(family.name)
+
+    def change_modes(self):
+        """change the mode of variables
+        """
+        for families in self.objectspace.space.variables.values():
+            for family in families.family.values():
+                family_mode = family.mode
+                # default is high level
+                new_family_mode = modes_level[-1]
+                for variable in family.variable.values():
+                    if not isinstance(variable, self.objectspace.leadership):
+                        func = self._change_variabe_mode
+                    else:
+                        func = self._change_variable_mode_leader
+                    func(variable,
+                         family_mode,
+                         )
+                    if modes[new_family_mode] > modes[variable.mode]:
+                        new_family_mode = variable.mode
+                # set the lower variable mode to family
+                family.mode = new_family_mode
+
+    def _change_variabe_mode(self,
+                             variable,
+                             family_mode: str,
+                             is_follower=False,
+                             ) -> None:
+        # auto_save or auto_freeze variable is set to 'basic' mode
+        # if its mode is not defined by the user
+        if 'mode' not in vars(variable) and \
+                (variable.auto_save is True or variable.auto_freeze is True):
+            variable.mode = modes_level[0]
+        # mandatory variable without value is a basic variable
+        if variable.mandatory is True and not hasattr(variable, 'default'):
+            variable.mode = modes_level[0]
+        # none basic variable in high level family has to be in high level
+        if modes[variable.mode] < modes[family_mode] and \
+                (not is_follower or variable.mode != modes_level[0]):
+            variable.mode = family_mode
+
+    def _change_variable_mode_leader(self,
+                                     leadership,
+                                     family_mode: str,
+                                     ) -> None:
+        is_follower = False
+        leader_mode = None
+        for follower in leadership.variable:
+            if follower.auto_save is True:
+                msg = _(f'leader/followers "{follower.name}" could not be auto_save')
+                raise DictConsistencyError(msg, 29, leadership.xmlfiles)
+            if follower.auto_freeze is True:
+                msg = f'leader/followers "{follower.name}" could not be auto_freeze'
+                raise DictConsistencyError(_(msg), 30, leadership.xmlfiles)
+            self._change_variabe_mode(follower,
+                                      family_mode,
+                                      is_follower,
+                                      )
+            if leader_mode is None:
+                leader_mode = leadership.variable[0].mode
+                leadership.variable[0].mode = None
+            else:
+                # leader's mode is minimum level
+                if modes[leader_mode] > modes[follower.mode]:
+                    follower.mode = leader_mode
+            is_follower = True
+        leadership.mode = leader_mode
+
+    def dynamic_families(self):
+        """link dynamic families to object
+        """
+        for families in self.objectspace.space.variables.values():
+            for family in families.family.values():
+                if 'dynamic' not in vars(family):
+                    continue
+                family.suffixes = self.objectspace.paths.get_variable(family.dynamic)
+                del family.dynamic
+                if not family.suffixes.multi:
+                    msg = _(f'dynamic family "{family.name}" must be linked '
+                            f'to multi variable')
+                    raise DictConsistencyError(msg, 16, family.xmlfiles)
+
+    def convert_help(self):
+        """Convert variable help
+        """
+        for families in self.objectspace.space.variables.values():
+            for family in families.family.values():
+                if not hasattr(family, 'help'):
+                    continue
+                if not hasattr(family, 'information'):
+                    family.information = self.objectspace.information(family.xmlfiles)
+                family.information.help = family.help
+                del family.help

src/rougail/annotator/fill.py

@@ -0,0 +1,75 @@
+"""Fill annotator
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from ..utils import load_modules
+from ..i18n import _
+from ..error import DictConsistencyError
+
+from .target import TargetAnnotator
+from .param import ParamAnnotator
+
+
+class FillAnnotator(TargetAnnotator, ParamAnnotator):
+    """Fill annotator
+    """
+    def __init__(self,
+                 objectspace,
+                 eosfunc_file,
+                 ):
+        self.objectspace = objectspace
+        if not hasattr(objectspace.space, 'constraints') or \
+                not hasattr(self.objectspace.space.constraints, 'fill'):
+            return
+        self.functions = dir(load_modules(eosfunc_file))
+        self.target_is_uniq = True
+        self.only_variable = True
+        self.convert_target(self.objectspace.space.constraints.fill)
+        self.convert_param(self.objectspace.space.constraints.fill)
+        self.fill_to_value()
+        del self.objectspace.space.constraints.fill
+
+    def fill_to_value(self) -> None:
+        """valid and manage <fill>
+        """
+        for fill in self.objectspace.space.constraints.fill:
+            for target in fill.target:
+                # test if the function exists
+                if fill.name not in self.functions:
+                    msg = _(f'cannot find fill function "{fill.name}"')
+                    raise DictConsistencyError(msg, 25, fill.xmlfiles)
+
+                # create an object value
+                value = self.objectspace.value(fill.xmlfiles)
+                value.type = 'calculation'
+                value.name = fill.name
+                if target.name.namespace == 'services':
+                    target.name.default = value
+                else:
+                    target.name.value = [value]
+
+                # manage params
+                if hasattr(fill, 'param') and fill.param:
+                    value.param = fill.param

src/rougail/annotator/group.py

@@ -0,0 +1,156 @@
+"""Annotate group
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from ..i18n import _
+from ..error import DictConsistencyError
+
+
+class GroupAnnotator:
+    """Annotate group
+    """
+    def __init__(self,
+                 objectspace,
+                 ):
+        self.objectspace = objectspace
+        if not hasattr(self.objectspace.space, 'constraints') or \
+                not hasattr(self.objectspace.space.constraints, 'group'):
+            return
+        self.convert_groups()
+
+    def convert_groups(self):  # pylint: disable=C0111
+        """convert groups
+        """
+        # store old leaders family name
+        cache_paths = {}
+        for group in self.objectspace.space.constraints.group:
+            if group.leader in cache_paths:
+                leader_fam_path = cache_paths[group.leader]
+            else:
+                leader_fam_path = self.objectspace.paths.get_variable_family_path(group.leader)
+                cache_paths[group.leader] = leader_fam_path
+            follower_names = list(group.follower.keys())
+            leader = self.objectspace.paths.get_variable(group.leader)
+            ori_leader_family = self.objectspace.paths.get_family(leader_fam_path,
+                                                                  leader.namespace,
+                                                                  )
+            has_a_leader = False
+            for variable in list(ori_leader_family.variable.values()):
+                if isinstance(variable, self.objectspace.leadership) and \
+                        variable.variable[0].name == leader.name:
+                    # append follower to an existed leadership
+                    leader_space = variable
+                    has_a_leader = True
+                elif variable.name == leader.name:
+                    # it's a leader
+                    leader_space = self.manage_leader(variable,
+                                                      group,
+                                                      ori_leader_family,
+                                                      )
+                    has_a_leader = True
+                elif has_a_leader:
+                    # it's should be a follower
+                    self.manage_follower(follower_names.pop(0),
+                                         leader_fam_path,
+                                         variable,
+                                         leader_space,
+                                         )
+                    # this variable is not more in ori_leader_family
+                    ori_leader_family.variable.pop(variable.name)
+                    if follower_names == []:
+                        # no more follower
+                        break
+            else:
+                joined = '", "'.join(follower_names)
+                msg = _(f'when parsing leadership, we espect to find those followers "{joined}"')
+                raise DictConsistencyError(msg, 31, variable.xmlfiles)
+        del self.objectspace.space.constraints.group
+
+    def manage_leader(self,
+                      variable: 'Variable',
+                      group: 'Group',
+                      ori_leader_family,
+                      ) -> 'Leadership':
+        """manage leader's variable
+        """
+        if variable.multi is not True:
+            msg = _(f'the variable "{variable.name}" in a group must be multi')
+            raise DictConsistencyError(msg, 32, variable.xmlfiles)
+        if hasattr(group, 'name'):
+            leadership_name = group.name
+        else:
+            leadership_name = variable.name
+        leader_space = self.objectspace.leadership(variable.xmlfiles)
+        leader_space.variable = []
+        leader_space.name = leadership_name
+        leader_space.hidden = variable.hidden
+        if variable.hidden:
+            variable.frozen = True
+            variable.force_default_on_freeze = True
+        variable.hidden = None
+        if hasattr(group, 'description'):
+            leader_space.doc = group.description
+        elif variable.name == leadership_name and hasattr(variable, 'description'):
+            leader_space.doc = variable.description
+        else:
+            leader_space.doc = leadership_name
+        leadership_path = ori_leader_family.path + '.' + leadership_name
+        self.objectspace.paths.add_leadership(variable.namespace,
+                                              leadership_path,
+                                              leader_space,
+                                              )
+        leader_family = self.objectspace.paths.get_family(ori_leader_family.path,
+                                                          ori_leader_family.namespace,
+                                                          )
+        leader_family.variable[variable.name] = leader_space
+        leader_space.variable.append(variable)
+        self.objectspace.paths.set_leader(variable.namespace,
+                                          ori_leader_family.path,
+                                          leadership_name,
+                                          variable.name,
+                                          )
+        return leader_space
+
+    def manage_follower(self,
+                        follower_name: str,
+                        leader_family_name: str,
+                        variable: 'Variable',
+                        leader_space: 'Leadership',
+                        ) -> None:
+        """manage follower
+        """
+        if variable.name != follower_name:
+            msg = _('when parsing leadership, we expect to find the follower '
+                    f'"{follower_name}" but we found "{variable.name}"')
+            raise DictConsistencyError(msg, 33, variable.xmlfiles)
+        self.objectspace.paths.set_leader(variable.namespace,
+                                          leader_family_name,
+                                          leader_space.name,
+                                          variable.name,
+                                          )
+        if leader_space.hidden:
+            variable.frozen = True
+            variable.force_default_on_freeze = True
+        leader_space.variable.append(variable)

src/rougail/annotator/param.py

@@ -0,0 +1,74 @@
+"""Param annotator
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from ..i18n import _
+from ..error import DictConsistencyError
+
+
+class ParamAnnotator:
+    def convert_param(self, objects) -> None:
+        """ valid and convert param
+        """
+        for obj in objects:
+            if not hasattr(obj, 'param'):
+                continue
+            param_to_delete = []
+            for param_idx, param in enumerate(obj.param):
+                if param.type == 'suffix':
+                    if hasattr(param, 'text'):
+                        msg = _(f'"{param.type}" parameter must not have a value')
+                        raise DictConsistencyError(msg, 28, obj.xmlfiles)
+                    for target in obj.target:
+                        if not self.objectspace.paths.variable_is_dynamic(target.name.path):
+                            msg = _(f'"suffix" parameter cannot be set with target "{target.name}"'
+                                    f' which is not a dynamic variable')
+                            raise DictConsistencyError(msg, 53, obj.xmlfiles)
+                elif not hasattr(param, 'text'):
+                    if not param.type == 'nil':
+                        msg = _(f'"{param.type}" parameter must have a value')
+                        raise DictConsistencyError(msg, 27, obj.xmlfiles)
+                elif param.type == 'nil':
+                    msg = _(f'"{param.type}" parameter must not have a value')
+                    raise DictConsistencyError(msg, 40, obj.xmlfiles)
+                if param.type == 'variable':
+                    try:
+                        path, suffix = self.objectspace.paths.get_variable_path(param.text,
+                                                                                obj.namespace,
+                                                                                )
+                        param.text = self.objectspace.paths.get_variable(path)
+                        if suffix:
+                            param.suffix = suffix
+                            family_path = self.objectspace.paths.get_variable_family_path(path)
+                            param.family = self.objectspace.paths.get_family(family_path,
+                                                                             param.text.namespace,
+                                                                             )
+                    except DictConsistencyError as err:
+                        if err.errno != 42 or not param.optional:
+                            raise err
+                        param_to_delete.append(param_idx)
+            param_to_delete.sort(reverse=True)
+            for param_idx in param_to_delete:
+                obj.param.pop(param_idx)

src/rougail/annotator/property.py

@@ -0,0 +1,106 @@
+"""Annotate properties
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from ..i18n import _
+from ..error import DictConsistencyError
+
+
+PROPERTIES = ('hidden', 'frozen', 'auto_freeze', 'auto_save', 'force_default_on_freeze',
+              'force_store_value', 'disabled', 'mandatory')
+CONVERT_PROPERTIES = {'auto_save': ['force_store_value'],
+                      'auto_freeze': ['force_store_value', 'auto_freeze'],
+                      }
+
+
+class PropertyAnnotator:
+    """Annotate properties
+    """
+    def __init__(self, objectspace):
+        self.objectspace = objectspace
+        if hasattr(self.objectspace.space, 'services'):
+            self.convert_services()
+        if hasattr(self.objectspace.space, 'variables'):
+            self.convert_variables()
+
+    def convert_property(self,
+                         variable,
+                         ) -> None:
+        """convert properties
+        """
+        # hidden variable is also frozen
+        if isinstance(variable, self.objectspace.variable) and variable.hidden is True:
+            variable.frozen = True
+            if not variable.auto_save and \
+                    not variable.auto_freeze and \
+                    'force_default_on_freeze' not in vars(variable):
+                variable.force_default_on_freeze = True
+        if not hasattr(variable, 'properties'):
+            variable.properties = []
+        for prop in PROPERTIES:
+            if hasattr(variable, prop):
+                if getattr(variable, prop) is True:
+                    for subprop in CONVERT_PROPERTIES.get(prop, [prop]):
+                        variable.properties.append(subprop)
+                setattr(variable, prop, None)
+        if hasattr(variable, 'mode') and variable.mode:
+            variable.properties.append(variable.mode)
+            variable.mode = None
+        if 'force_store_value' in variable.properties and \
+                'force_default_on_freeze' in variable.properties:  # pragma: no cover
+            # should not appened
+            msg = _('cannot have auto_freeze or auto_store with the hidden '
+                    f'variable "{variable.name}"')
+            raise DictConsistencyError(msg, 50, variable.xmlfiles)
+        if not variable.properties:
+            del variable.properties
+
+    def convert_services(self) -> None:
+        """convert services
+        """
+        self.convert_property(self.objectspace.space.services)
+        for services in self.objectspace.space.services.service.values():
+            self.convert_property(services)
+            for service in vars(services).values():
+                if not isinstance(service, self.objectspace.family):
+                    continue
+                self.convert_property(service)
+                for family in service.family:
+                    self.convert_property(family)
+                    for variable in family.variable:
+                        self.convert_property(variable)
+
+    def convert_variables(self) -> None:
+        """convert variables
+        """
+        for variables in self.objectspace.space.variables.values():
+            for family in variables.family.values():
+                self.convert_property(family)
+                for variable in family.variable.values():
+                    self.convert_property(variable)
+                    if not isinstance(variable, self.objectspace.leadership):
+                        continue
+                    for follower in variable.variable:
+                        self.convert_property(follower)

src/rougail/annotator/service.py

@@ -0,0 +1,257 @@
+"""Annotate services
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from os.path import basename
+from typing import Tuple
+
+from ..i18n import _
+from ..utils import normalize_family
+from ..error import DictConsistencyError
+# a CreoleObjSpace's attribute has some annotations
+# that shall not be present in the exported (flatened) XML
+ERASED_ATTRIBUTES = ('redefine', 'exists', 'fallback', 'optional', 'remove_check', 'namespace',
+                     'remove_condition', 'path', 'instance_mode', 'index',
+                     'level', 'remove_fill', 'xmlfiles', 'type', 'reflector_name',
+                     'reflector_object', 'manage')
+
+
+KEY_TYPE = {'variable': 'symlink',
+            'PortOption': 'port',
+            'UnicodeOption': 'string',
+            'NetworkOption': 'network',
+            'NetmaskOption': 'netmask',
+            'URLOption': 'web_address',
+            'FilenameOption': 'filename',
+            }
+
+
+class ServiceAnnotator:
+    """Manage service's object
+    for example::
+        <services>
+            <service name="test">
+                <service_access service='ntp'>
+                    <port protocol='udp' service_accesslist='ntp_udp'>123</port>
+                </service_access>
+            </service>
+        </services>
+    """
+    def __init__(self, objectspace):
+        self.objectspace = objectspace
+        if hasattr(self.objectspace.space, 'services'):
+            if not hasattr(self.objectspace.space.services, 'service'):
+                del self.objectspace.space.services
+            else:
+                self.convert_services()
+
+    def convert_services(self):
+        """convert services to variables
+        """
+        self.objectspace.space.services.hidden = True
+        self.objectspace.space.services.name = 'services'
+        self.objectspace.space.services.doc = 'services'
+        self.objectspace.space.services.path = 'services'
+        for service_name, service in self.objectspace.space.services.service.items():
+            service.information = self.objectspace.information(service.xmlfiles)
+            service.information.manage = service.manage
+            service.manage = None
+            for elttype, values in dict(vars(service)).items():
+                if not isinstance(values, (dict, list)) or elttype in ERASED_ATTRIBUTES:
+                    continue
+                eltname = elttype + 's'
+                path = '.'.join(['services', normalize_family(service_name), eltname])
+                family = self._gen_family(eltname,
+                                          path,
+                                          service.xmlfiles,
+                                          )
+                if isinstance(values, dict):
+                    values = list(values.values())
+                family.family = self.make_group_from_elts(service_name,
+                                                          elttype,
+                                                          values,
+                                                          path,
+                                                          )
+                setattr(service, elttype, family)
+            service.doc = service.name
+
+    def make_group_from_elts(self,
+                             service_name,
+                             elttype,
+                             elts,
+                             path,
+                             ):
+        """Splits each objects into a group (and `OptionDescription`, in tiramisu terms)
+        and build elements and its attributes (the `Options` in tiramisu terms)
+        """
+        families = []
+        listname = '{}list'.format(elttype)
+        for elt in elts:
+            # try to launch _update_xxxx() function
+            update_elt = '_update_' + elttype
+            if hasattr(self, update_elt):
+                getattr(self, update_elt)(elt,
+                                          service_name,
+                                          )
+            c_name, subpath = self._get_name_path(elt,
+                                                  path,
+                                                  )
+            family = self._gen_family(c_name,
+                                      subpath,
+                                      elt.xmlfiles,
+                                      )
+            family.variable = []
+            activate_obj = self._generate_element('boolean',
+                                                  'activate',
+                                                  True,
+                                                  elt.xmlfiles,
+                                                  '.'.join([subpath, 'activate']),
+                                                  )
+            for key in dir(elt):
+                if key.startswith('_') or key.endswith('_type') or key in ERASED_ATTRIBUTES:
+                    continue
+                value = getattr(elt, key)
+                if key == listname:
+                    self.objectspace.list_conditions.setdefault(listname,
+                                                                {}).setdefault(
+                                                                    value,
+                                                                    []).append(activate_obj)
+                    continue
+                family.variable.append(self._generate_element(self._get_type(key,
+                                                                             elttype,
+                                                                             elt,
+                                                                             ),
+                                                              key,
+                                                              value,
+                                                              elt.xmlfiles,
+                                                              f'{subpath}.{key}'
+                                                              ))
+            family.variable.append(activate_obj)
+            families.append(family)
+        return families
+
+    def _get_name_path(self,
+                       elt,
+                       path: str,
+                       ) -> Tuple[str, str]:
+        # create element name, if already exists, add _xx to be uniq
+        if hasattr(elt, 'source'):
+            name = elt.source
+        else:
+            name = elt.name
+        idx = 0
+        while True:
+            c_name = name
+            if idx:
+                c_name += f'_{idx}'
+            subpath = '{}.{}'.format(path, normalize_family(c_name))
+            try:
+                self.objectspace.paths.get_family(subpath, 'services')
+            except DictConsistencyError as err:
+                if err.errno == 42:
+                    return c_name, subpath
+            idx += 1
+
+    def _gen_family(self,
+                    name,
+                    path,
+                    xmlfiles
+                    ):
+        family = self.objectspace.family(xmlfiles)
+        family.name = normalize_family(name)
+        family.doc = name
+        family.mode = None
+        self.objectspace.paths.add_family('services',
+                                          path,
+                                          family,
+                                          )
+        return family
+
+    def _generate_element(self,
+                          type_,
+                          key,
+                          value,
+                          xmlfiles,
+                          path,
+                          ):  # pylint: disable=R0913
+        variable = self.objectspace.variable(xmlfiles)
+        variable.name = normalize_family(key)
+        variable.mode = None
+        variable.type = type_
+        if type_ == 'symlink':
+            variable.opt = self.objectspace.paths.get_variable(value)
+            variable.multi = None
+        else:
+            variable.doc = key
+            variable.default = value
+            variable.namespace = 'services'
+        self.objectspace.paths.add_variable('services',
+                                            path,
+                                            'service',
+                                            False,
+                                            variable,
+                                            )
+        return variable
+
+    def _get_type(self,
+                  key: str,
+                  elt_name: str,
+                  elt,
+                  ) -> str:
+        if key == 'name':
+            dtd_key_type = elt_name + '_type'
+        else:
+            dtd_key_type = key + '_type'
+        if key in self.objectspace.booleans_attributs:
+            return 'boolean'
+        if hasattr(elt, dtd_key_type):
+            return KEY_TYPE[getattr(elt, dtd_key_type)]
+        return 'string'
+
+    def _update_override(self,
+                         file_,
+                         service_name,
+                         ):
+        file_.name = f'/systemd/system/{service_name}.service.d/rougail.conf'
+        # retrieve default value from File object
+        for attr in ['owner', 'group', 'mode']:
+            setattr(file_, attr, getattr(self.objectspace.file, attr))
+        if not hasattr(file_, 'source'):
+            file_.source = f'{service_name}.service'
+        self._update_file(file_,
+                          service_name,
+                          )
+
+    def _update_file(self,
+                     file_,
+                     service_name,
+                     ):
+        if not hasattr(file_, 'file_type') or file_.file_type == "UnicodeOption":
+            if not hasattr(file_, 'source'):
+                file_.source = basename(file_.name)
+        elif not hasattr(file_, 'source'):
+            msg = _(f'attribute "source" is mandatory for the file "{file_.name}" '
+                    f'"({service_name})"')
+            raise DictConsistencyError(msg, 34, file_.xmlfiles)

src/rougail/annotator/target.py

@@ -0,0 +1,82 @@
+"""Target annotator
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from ..i18n import _
+from ..error import DictConsistencyError
+
+
+class TargetAnnotator:
+    def convert_target(self,
+                       objects,
+                       ) -> None:
+        """ valid and convert param
+        """
+        targets = []
+        for obj in objects:
+            if not hasattr(obj, 'target'):
+                msg = _('target is mandatory')
+                raise DictConsistencyError(msg, 9, obj.xmlfiles)
+            remove_targets = []
+            for index, target in enumerate(obj.target):
+                # test if it's redefined calculation
+                if self.target_is_uniq and target.name in targets:
+                    msg = _(f'A fill already exists for the target of "{target.name}" created')
+                    raise DictConsistencyError(msg, 24, obj.xmlfiles)
+                targets.append(target.name)
+
+                # let's replace the target by the path
+                try:
+                    if target.type == 'variable':
+                        path, suffix = self.objectspace.paths.get_variable_path(target.name,
+                                                                                obj.namespace,
+                                                                                )
+                        target.name = self.objectspace.paths.get_variable(path)
+                        if suffix:
+                            msg = _(f'target to {target.name.path} with suffix is not allowed')
+                            raise DictConsistencyError(msg, 35, obj.xmlfiles)
+                    elif self.only_variable:
+                        msg = _(f'target to "{target.name}" with param type "{target.type}" '
+                                f'is not allowed')
+                        raise DictConsistencyError(msg, 2, obj.xmlfiles)
+                    if target.type == 'family':
+                        target.name = self.objectspace.paths.get_family(target.name,
+                                                                        obj.namespace,
+                                                                        )
+                    elif target.type.endswith('list') and \
+                            obj.name not in ['disabled_if_in', 'disabled_if_not_in']:
+                        msg = _(f'target "{target.type}" not allow')
+                        raise DictConsistencyError(msg, 10, target.xmlfiles)
+                except DictConsistencyError as err:
+                    if err.errno != 42:
+                        raise err
+                    # for optional variable
+                    if not target.optional:
+                        msg = f'cannot found target "{target.name}"'
+                        raise DictConsistencyError(_(msg), 12, target.xmlfiles) from err
+                    remove_targets.append(index)
+            remove_targets.sort(reverse=True)
+            for index in remove_targets:
+                obj.target.pop(index)

src/rougail/annotator/value.py

@@ -0,0 +1,95 @@
+"""Annotate value
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+
+class ValueAnnotator:  # pylint: disable=R0903
+    """Annotate value
+    """
+    def __init__(self,
+                 objectspace,
+                 ) -> None:
+        if not hasattr(objectspace.space, 'variables'):
+            return
+        self.objectspace = objectspace
+        self.convert_value()
+
+    def convert_value(self) -> None:
+        """convert value
+        """
+        for families in self.objectspace.space.variables.values():
+            for family in families.family.values():
+                if not hasattr(family, 'variable'):
+                    continue
+                for variable in family.variable.values():
+                    if isinstance(variable, self.objectspace.leadership):
+                        variable_type = 'leader'
+                        for follower in variable.variable:
+                            self._convert_value(follower,
+                                                variable_type,
+                                                )
+                            variable_type = 'follower'
+                    else:
+                        self._convert_value(variable)
+
+    def _convert_value(self,
+                       variable,
+                       variable_type: str=None,
+                       ) -> None:
+        # a boolean must have value, the default value is "True"
+        if not hasattr(variable, 'value') and variable.type == 'boolean':
+            new_value = self.objectspace.value(variable.xmlfiles)
+            new_value.name = True
+            new_value.type = 'boolean'
+            variable.value = [new_value]
+        """if the variable is mandatory and doesn't have any value
+        then the variable's mode is set to 'basic'
+        """
+        # variable with default value is mandatory
+        if hasattr(variable, 'value') and variable.value:
+            has_value = True
+            for value in variable.value:
+                if value.type == 'calculation':
+                    has_value = False
+                    break
+            if has_value:
+                # if has value without any calculation
+                variable.mandatory = True
+        if not hasattr(variable, 'value'):
+            return
+        if variable.value[0].type == 'calculation':
+            variable.default = variable.value[0]
+        else:
+            if variable.multi:
+                if variable_type != 'follower':
+                    variable.default = [value.name for value in variable.value]
+                if variable_type != 'leader':
+                    if variable.multi == 'submulti':
+                        variable.default_multi = [value.name for value in variable.value]
+                    else:
+                        variable.default_multi = variable.value[0].name
+            else:
+                variable.default = variable.value[0].name
+        del variable.value

src/rougail/annotator/variable.py

@@ -0,0 +1,212 @@
+"""Annotate variable
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+
+
+CONVERT_OPTION = {'number': dict(opttype="IntOption", func=int),
+                  'float': dict(opttype="FloatOption", func=float),
+                  'choice': dict(opttype="ChoiceOption"),
+                  'string': dict(opttype="StrOption"),
+                  'password': dict(opttype="PasswordOption"),
+                  'mail': dict(opttype="EmailOption"),
+                  'boolean': dict(opttype="BoolOption"),
+                  'symlink': dict(opttype="SymLinkOption"),
+                  'filename': dict(opttype="FilenameOption"),
+                  'date': dict(opttype="DateOption"),
+                  'unix_user': dict(opttype="UsernameOption"),
+                  'ip': dict(opttype="IPOption", initkwargs={'allow_reserved': True}),
+                  'local_ip': dict(opttype="IPOption", initkwargs={'private_only': True,
+                                   'warnings_only': True}),
+                  'netmask': dict(opttype="NetmaskOption"),
+                  'network': dict(opttype="NetworkOption"),
+                  'broadcast': dict(opttype="BroadcastOption"),
+                  'netbios': dict(opttype="DomainnameOption", initkwargs={'type': 'netbios',
+                                  'warnings_only': True}),
+                  'domain': dict(opttype="DomainnameOption", initkwargs={'type': 'domainname',
+                                 'allow_ip': False}),
+                  'hostname': dict(opttype="DomainnameOption", initkwargs={'type': 'hostname',
+                                   'allow_ip': False}),
+                  'web_address': dict(opttype="URLOption", initkwargs={'allow_ip': True,
+                                      'allow_without_dot': True}),
+                  'port': dict(opttype="PortOption", initkwargs={'allow_private': True}),
+                  'mac': dict(opttype="MACOption"),
+                  'cidr': dict(opttype="IPOption", initkwargs={'cidr': True}),
+                  'network_cidr': dict(opttype="NetworkOption", initkwargs={'cidr': True}),
+                  }
+
+
+FORCE_CHOICE = {'schedule': ['none', 'daily', 'weekly', 'monthly'],
+                'schedulemod': ['pre', 'post'],
+                }
+
+
+class VariableAnnotator:  # pylint: disable=R0903
+    """Annotate variable
+    """
+    def __init__(self,
+                 objectspace,
+                 ):
+        if not hasattr(objectspace.space, 'variables'):
+            return
+        self.objectspace = objectspace
+        self.convert_variable()
+        self.convert_test()
+        self.convert_help()
+
+    def convert_variable(self):
+        """convert variable
+        """
+        for families in self.objectspace.space.variables.values():
+            for family in families.family.values():
+                if not hasattr(family, 'variable'):
+                    continue
+                for variable in family.variable.values():
+                    if isinstance(variable, self.objectspace.leadership):
+                        # first variable is a leader, others are follower
+                        variable_type = 'leader'
+                        for follower in variable.variable:
+                            self._convert_variable(families.name,
+                                                   follower,
+                                                   variable_type,
+                                                   )
+                            variable_type = 'follower'
+                    else:
+                        self._convert_variable(families.name,
+                                               variable,
+                                               'variable',
+                                               )
+
+    def _convert_variable(self,
+                          namespace: str,
+                          variable,
+                          variable_type: str,
+                          ) -> None:
+        if variable.type != 'symlink' and not hasattr(variable, 'description'):
+            variable.description = variable.name
+        if hasattr(variable, 'value'):
+            value_to_del = []
+            for idx, value in enumerate(variable.value):
+                if not hasattr(value, 'name'):
+                    value_to_del.append(idx)
+                else:
+                    if not hasattr(value, 'type'):
+                        value.type = variable.type
+                    value.name = CONVERT_OPTION.get(value.type, {}).get('func', str)(value.name)
+            value_to_del.sort(reverse=True)
+            for idx in value_to_del:
+                del variable.value[idx]
+            if not variable.value:
+                del variable.value
+        variable.doc = variable.description
+        del variable.description
+        if variable_type == 'follower':
+            if variable.multi is True:
+                variable.multi = 'submulti'
+            else:
+                variable.multi = True
+        self._convert_valid_enum(namespace,
+                                 variable,
+                                 )
+
+    def _convert_valid_enum(self,
+                            namespace,
+                            variable,
+                            ):
+        """some types are, in fact, choices
+        convert this kind of variables into choice
+        """
+        if variable.type in FORCE_CHOICE:
+            if not hasattr(self.objectspace.space, 'constraints'):
+                xmlfiles = variable.xmlfiles
+                self.objectspace.space.constraints = self.objectspace.constraints(xmlfiles)
+                self.objectspace.space.constraints.namespace = namespace
+            if not hasattr(self.objectspace.space.constraints, 'check'):
+                self.objectspace.space.constraints.check = []
+            check = self.objectspace.check(variable.xmlfiles)
+            check.name = 'valid_enum'
+            target = self.objectspace.target(variable.xmlfiles)
+            target.name = variable.path
+            check.target = [target]
+            check.namespace = namespace
+            check.param = []
+            for value in FORCE_CHOICE[variable.type]:
+                param = self.objectspace.param(variable.xmlfiles)
+                param.text = value
+                check.param.append(param)
+            self.objectspace.space.constraints.check.append(check)
+            variable.type = 'string'
+
+    def convert_test(self):
+        """Convert variable tests value
+        """
+        for families in self.objectspace.space.variables.values():
+            for family in families.family.values():
+                if not hasattr(family, 'variable'):
+                    continue
+                for variable in family.variable.values():
+                    if isinstance(variable, self.objectspace.leadership):
+                        for follower in variable.variable:
+                            self._convert_test(follower)
+                    else:
+                        self._convert_test(variable)
+
+    def _convert_test(self,
+                      variable,
+                      ) -> None:
+        if not hasattr(variable, 'information'):
+            variable.information = self.objectspace.information(variable.xmlfiles)
+        if hasattr(variable, 'test'):
+            if variable.test:
+                values = variable.test.split('|')
+                new_values = []
+                for value in values:
+                    if value == '':
+                        value = None
+                    else:
+                        value = CONVERT_OPTION.get(variable.type, {}).get('func', str)(value)
+                    new_values.append(value)
+                variable.information.test = tuple(new_values)
+            del variable.test
+
+    def convert_help(self):
+        """Convert variable help
+        """
+        for families in self.objectspace.space.variables.values():
+            for family in families.family.values():
+                if not hasattr(family, 'variable'):
+                    continue
+                for variable in family.variable.values():
+                    if isinstance(variable, self.objectspace.leadership):
+                        for follower in variable.variable:
+                            self._convert_help(follower)
+                    else:
+                        self._convert_help(variable)
+
+    @staticmethod
+    def _convert_help(variable) -> None:
+        if hasattr(variable, 'help'):
+            variable.information.help = variable.help
+            del variable.help

src/rougail/config.py

@@ -1,19 +1,42 @@
-# -*- coding: utf-8 -*-
 """
-fichier de configuration pour rougail
+Config file for Rougail
 
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 """
-from os.path import join, isfile, abspath, dirname
+from os.path import join, abspath, dirname
-from pathlib import Path
 
-rougailroot = '/var/rougail' 
-patch_dir = join(rougailroot, 'patches')
-manifests_dir = join(rougailroot, 'manifests')
-templates_dir = join(rougailroot, 'templates')
-dtddir = join(dirname(abspath(__file__)), 'data')
-dtdfilename = join(dtddir, 'rougail.dtd')
 
-# chemin du répertoire source des fichiers templates
+ROUGAILROOT = '/var/rougail'
-patch_dir = '/srv/rougail/patch'
+DTDDIR = join(dirname(abspath(__file__)), 'data')
 
-variable_namespace = 'rougail'
+
+Config = {'rougailroot': ROUGAILROOT,
+          'patch_dir': join(ROUGAILROOT, 'patches'),
+          'manifests_dir': join(ROUGAILROOT, 'manifests'),
+          'templates_dir': join(ROUGAILROOT, 'templates'),
+          'dtdfilename': join(DTDDIR, 'rougail.dtd'),
+          'dtddir': DTDDIR,
+          'variable_namespace': 'rougail',
+          }

src/rougail/data/rougail.dtd

@@ -13,7 +13,7 @@
 
 #    Forked by:
 #    Cadoles (http://www.cadoles.com)
-#    Copyright (C) 2019-2020
+#    Copyright (C) 2019-2021
 
 #    distribued with GPL-2 or later license
 
@@ -45,26 +45,26 @@
 
 <!ELEMENT services (service*)>
 
-<!ELEMENT service ((port* | tcpwrapper* | ip* | interface* | package* | file* | digitalcertificate* | override*)*) >
+<!ELEMENT service ((port* | tcpwrapper* | ip* | interface* | package* | file* | override*)*) >
 <!ATTLIST service name CDATA #REQUIRED>
 <!ATTLIST service manage (True|False) "True">
 
 <!ELEMENT port (#PCDATA)>
-<!ATTLIST port port_type (PortOption|SymLinkOption|variable) "PortOption">
+<!ATTLIST port port_type (PortOption|variable) "PortOption">
 <!ATTLIST port portlist CDATA #IMPLIED >
 <!ATTLIST port protocol (tcp|udp) "tcp">
 
 <!ELEMENT ip (#PCDATA)>
 <!ATTLIST ip iplist CDATA #IMPLIED >
-<!ATTLIST ip ip_type (NetworkOption|SymLinkOption|variable) "NetworkOption">
+<!ATTLIST ip ip_type (NetworkOption|variable) "NetworkOption">
-<!ATTLIST ip interface_type (UnicodeOption|SymLinkOption|variable) "UnicodeOption">
+<!ATTLIST ip interface_type (UnicodeOption|variable) "UnicodeOption">
 <!ATTLIST ip interface CDATA #REQUIRED>
-<!ATTLIST ip netmask_type (NetmaskOption|SymLinkOption|variable) "NetmaskOption">
+<!ATTLIST ip netmask_type (NetmaskOption|variable) "NetmaskOption">
 <!ATTLIST ip netmask CDATA "255.255.255.255">
 
 <!ELEMENT file EMPTY>
 <!ATTLIST file name CDATA #REQUIRED >
-<!ATTLIST file file_type (UnicodeOption|SymLinkOption|variable) "UnicodeOption">
+<!ATTLIST file file_type (UnicodeOption|variable) "UnicodeOption">
 <!ATTLIST file variable CDATA #IMPLIED>
 <!ATTLIST file variable_type (variable) "variable">
 <!ATTLIST file source CDATA #IMPLIED>
@@ -75,30 +75,25 @@
 <!ATTLIST file redefine (True|False) "False">
 <!ATTLIST file templating (True|False) "True">
 
-<!ELEMENT digitalcertificate EMPTY>
-<!ATTLIST digitalcertificate name CDATA #REQUIRED >
-<!ATTLIST digitalcertificate digitalcertificate_type (variable) "variable">
-<!ATTLIST digitalcertificate certificate CDATA #REQUIRED >
-<!ATTLIST digitalcertificate certificate_type (variable) "variable">
-<!ATTLIST digitalcertificate type CDATA #REQUIRED >
-<!ATTLIST digitalcertificate ca CDATA #REQUIRED >
-
 <!ELEMENT override EMPTY>
 <!ATTLIST override source CDATA #IMPLIED >
 <!ATTLIST override templating (True|False) "True">
 
-<!ELEMENT variables (family*, separators*)>
+<!ELEMENT variables (family*)>
-<!ELEMENT family (#PCDATA | variable)*>
+
+<!ELEMENT family (variable*)>
 <!ATTLIST family name CDATA #REQUIRED>
 <!ATTLIST family description CDATA #IMPLIED>
+<!ATTLIST family help CDATA #IMPLIED>
 <!ATTLIST family mode (basic|normal|expert) "basic">
 <!ATTLIST family hidden (True|False) "False">
 <!ATTLIST family dynamic CDATA #IMPLIED>
 
-<!ELEMENT variable (#PCDATA | value)*>
+<!ELEMENT variable (value*)>
 <!ATTLIST variable name CDATA #REQUIRED>
-<!ATTLIST variable type CDATA #IMPLIED>
+<!ATTLIST variable type (number|float|string|password|mail|boolean|filename|date|unix_user|ip|local_ip|netmask|network|broadcast|netbios|domain|hostname|web_address|port|mac|cidr|network_cidr|schedule|schedulemod) "string">
 <!ATTLIST variable description CDATA #IMPLIED>
+<!ATTLIST variable help CDATA #IMPLIED>
 <!ATTLIST variable hidden (True|False) "False">
 <!ATTLIST variable disabled (True|False) "False">
 <!ATTLIST variable multi (True|False) "False">
@@ -110,46 +105,40 @@
 <!ATTLIST variable mode (basic|normal|expert) "normal">
 <!ATTLIST variable remove_check (True|False) "False">
 <!ATTLIST variable remove_condition (True|False) "False">
+<!ATTLIST variable remove_fill (True|False) "False">
 <!ATTLIST variable test CDATA #IMPLIED>
 
-<!ELEMENT separators (separator*)>
-
-<!ELEMENT separator (#PCDATA)>
-<!ATTLIST separator name CDATA #REQUIRED>
-
 <!ELEMENT value (#PCDATA)>
 
 <!ELEMENT constraints ((fill* | check* | condition* | group*)*)>
-<!ELEMENT fill (param*)>
-<!ATTLIST fill name CDATA #REQUIRED>
-<!ATTLIST fill target CDATA #REQUIRED>
 
-<!ELEMENT check (param*)>
+<!ELEMENT fill ((target|param)+)>
+<!ATTLIST fill name CDATA #REQUIRED>
+
+<!ELEMENT check ((target|param)+)>
 <!ATTLIST check name CDATA #REQUIRED>
-<!ATTLIST check target CDATA #REQUIRED>
 <!ATTLIST check level (error|warning) "error">
 
-<!ELEMENT condition ((target | param)+ )>
+<!ELEMENT condition ((target|param)+)>
-<!ATTLIST condition name (disabled_if_in|disabled_if_not_in|hidden_if_in|auto_hidden_if_not_in|hidden_if_not_in|mandatory_if_in|mandatory_if_not_in) #REQUIRED>
+<!ATTLIST condition name (disabled_if_in|disabled_if_not_in|hidden_if_in|hidden_if_not_in|mandatory_if_in|mandatory_if_not_in) #REQUIRED>
 <!ATTLIST condition source CDATA #REQUIRED>
 <!ATTLIST condition fallback (True|False) "False">
 <!ATTLIST condition force_condition_on_fallback (True|False) "False">
 <!ATTLIST condition force_inverse_condition_on_fallback (True|False) "False">
 
-<!ELEMENT group (follower+)>
-<!ATTLIST group leader CDATA #REQUIRED>
-<!ATTLIST group description CDATA #IMPLIED>
-
 <!ELEMENT param (#PCDATA)>
-<!ATTLIST param type (string|variable|number|python) "string">
+<!ATTLIST param type (string|number|nil|variable|information|suffix) "string">
 <!ATTLIST param name CDATA #IMPLIED>
-<!ATTLIST param notraisepropertyerror (True|False) "False">
+<!ATTLIST param propertyerror (True|False) "True">
 <!ATTLIST param optional (True|False) "False">
 
 <!ELEMENT target (#PCDATA)>
 <!ATTLIST target type (family|variable|filelist|iplist|portlist) "variable">
 <!ATTLIST target optional (True|False) "False">
 
-<!ELEMENT follower (#PCDATA)>
+<!ELEMENT group (follower+)>
+<!ATTLIST group leader CDATA #REQUIRED>
+<!ATTLIST group name CDATA #IMPLIED>
+<!ATTLIST group description CDATA #IMPLIED>
 
-<!ELEMENT help ((variable* | family*)*)>
+<!ELEMENT follower (#PCDATA)>

src/rougail/error.py

@@ -1,22 +1,58 @@
-# -*- coding: utf-8 -*-
+"""Standard error classes
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from .i18n import _
+
+
+def display_xmlfiles(xmlfiles: list) -> str:
+    """The function format xmlfiles informations to generate errors
+    """
+    if len(xmlfiles) == 1:
+        return '"' + xmlfiles[0] + '"'
+    return '"' + '", "'.join(xmlfiles[:-1]) + '"' + ' and ' + '"' + xmlfiles[-1] + '"'
+
+
 class ConfigError(Exception):
-    pass
+    """Standard error for templating
+    """
+
 
 class FileNotFound(ConfigError):
-    pass
+    """Template file is not found
+    """
+
 
 class TemplateError(ConfigError):
-    pass
+    """Templating generate an error
+    """
+
 
 class TemplateDisabled(TemplateError):
     """Template is disabled.
     """
-    pass
-
-
-class OperationError(Exception):
-    """Type error or value Error for Creole variable's type or values
-    """
 
 
 class SpaceObjShallNotBeUpdated(Exception):
@@ -29,7 +65,8 @@ class DictConsistencyError(Exception):
     """It's not only that the Creole XML is valid against the Creole DTD
     it's that it is not consistent.
     """
-
+    def __init__(self, msg, errno, xmlfiles):
-
+        if xmlfiles:
-class LoaderError(Exception):
+            msg = _(f'{msg} in {display_xmlfiles(xmlfiles)}')
-    pass
+        super().__init__(msg)
+        self.errno = errno

src/rougail/i18n.py

@@ -1,23 +1,28 @@
-# -*- coding: UTF-8 -*-
+"""Internationalisation utilities
-# Copyright (C) 2012-2013 Team tiramisu (see AUTHORS for all contributors)
+Created by:
-#
+EOLE (http://eole.orion.education.fr)
-# This program is free software: you can redistribute it and/or modify it
+Copyright (C) 2005-2018
-# under the terms of the GNU Lesser General Public License as published by the
+
-# Free Software Foundation, either version 3 of the License, or (at your
+Forked by:
-# option) any later version.
+Cadoles (http://www.cadoles.com)
-#
+Copyright (C) 2019-2021
-# This program is distributed in the hope that it will be useful, but WITHOUT
+
-# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+distribued with GPL-2 or later license
-# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
+
-# details.
+This program is free software; you can redistribute it and/or modify
-#
+it under the terms of the GNU General Public License as published by
-# You should have received a copy of the GNU Lesser General Public License
+the Free Software Foundation; either version 2 of the License, or
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+(at your option) any later version.
-#
+
-# The original `Config` design model is unproudly borrowed from
+This program is distributed in the hope that it will be useful,
-# the rough gus of pypy: pypy: http://codespeak.net/svn/pypy/dist/pypy/config/
+but WITHOUT ANY WARRANTY; without even the implied warranty of
-# the whole pypy projet is under MIT licence
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-"internationalisation utilities"
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
 import gettext
 import os
 import sys

src/rougail/objspace.py

@@ -1,529 +1,494 @@
+"""parse XML files and build a space with objects
+it aggregates this files and manage redefine and exists attributes
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 """
-Creole flattener. Takes a bunch of Creole XML dispatched in differents folders
+from typing import Optional
-as an input and outputs a human readable flatened XML
-
-Sample usage::
-
-    >>> from rougail.objspace import CreoleObjSpace
-    >>> eolobj = CreoleObjSpace('/usr/share/rougail/rougail.dtd')
-    >>> eolobj.create_or_populate_from_xml('rougail', ['/usr/share/eole/rougail/dicos'])
-    >>> eolobj.space_visitor()
-    >>> eolobj.save('/tmp/rougail_flatened_output.xml')
-
-The CreoleObjSpace
-
-- loads the XML into an internal CreoleObjSpace representation
-- visits/annotates the objects
-- dumps the object space as XML output into a single XML target
-
-The visit/annotation stage is a complex step that corresponds to the Creole
-procedures.
-
-For example: a variable is redefined and shall be moved to another family
-means that a variable1 = Variable() object in the object space who lives in the family1 parent
-has to be moved in family2. The visit procedure changes the varable1's object space's parent.
-"""
-from lxml.etree import Element, SubElement  # pylint: disable=E0611
 
 from .i18n import _
 from .xmlreflector import XMLReflector
-from .annotator import ERASED_ATTRIBUTES, SpaceAnnotator
-from .tiramisureflector import TiramisuReflector
 from .utils import normalize_family
-from .error import OperationError, SpaceObjShallNotBeUpdated, DictConsistencyError
+from .error import SpaceObjShallNotBeUpdated, DictConsistencyError
 from .path import Path
-from .config import variable_namespace
+from .config import Config
 
-# CreoleObjSpace's elements like 'family' or 'follower', that shall be forced to the Redefinable type
+# RougailObjSpace's elements that shall be forced to the Redefinable type
 FORCE_REDEFINABLES = ('family', 'follower', 'service', 'disknod', 'variables')
-# CreoleObjSpace's elements that shall be forced to the UnRedefinable type
+# RougailObjSpace's elements that shall be forced to the UnRedefinable type
 FORCE_UNREDEFINABLES = ('value',)
-# CreoleObjSpace's elements that shall be set to the UnRedefinable type
+# RougailObjSpace's elements that shall not be modify
 UNREDEFINABLE = ('multi', 'type')
-
+# RougailObjSpace's elements that did not created automaticly
-CONVERT_PROPERTIES = {'auto_save': ['force_store_value'], 'auto_freeze': ['force_store_value', 'auto_freeze']}
+FORCE_ELEMENTS = ('choice', 'property_', 'leadership', 'information')
-
+# XML text are convert has name
-RENAME_ATTIBUTES = {'description': 'doc'}
-
 FORCED_TEXT_ELTS_AS_NAME = ('choice', 'property', 'value', 'target')
 
-CONVERT_EXPORT = {'Leadership': 'leader',
-                  'Variable': 'variable',
-                  'Value': 'value',
-                  'Property': 'property',
-                  'Choice': 'choice',
-                  'Param': 'param',
-                  'Check': 'check',
-                  }
 
 # _____________________________________________________________________________
 # special types definitions for the Object Space's internal representation
-class RootCreoleObject:
+class RootRougailObject:  # pylint: disable=R0903
-    ""
+    """Root object
-
-
-class CreoleObjSpace:
-    """DOM XML reflexion free internal representation of a Creole Dictionary
     """
-    choice = type('Choice', (RootCreoleObject,), dict())
+    def __init__(self,
-    property_ = type('Property', (RootCreoleObject,), dict())
+                 xmlfiles,
-    # Creole ObjectSpace's Leadership variable class type
+                 name=None,
-    Leadership = type('Leadership', (RootCreoleObject,), dict())
+                 ):
-    """
+        if not isinstance(xmlfiles, list):
-    This Atom type stands for singleton, that is
+            xmlfiles = [xmlfiles]
-    an Object Space's atom object is present only once in the
+        self.xmlfiles = xmlfiles
-    object space's tree
+        if name:
-    """
+            self.name = name
-    Atom = type('Atom', (RootCreoleObject,), dict())
-    "A variable that can't be redefined"
-    Redefinable = type('Redefinable', (RootCreoleObject,), dict())
-    "A variable can be redefined"
-    UnRedefinable = type('UnRedefinable', (RootCreoleObject,), dict())
 
 
-    def __init__(self, dtdfilename):  # pylint: disable=R0912
+class Atom(RootRougailObject):  # pylint: disable=R0903
-        self.index = 0
+    """Atomic object (means can only define one time)
-        class ObjSpace:  # pylint: disable=R0903
+    """
+
+
+class Redefinable(RootRougailObject):  # pylint: disable=R0903
+    """Object that could be redefine
+    """
+
+
+class UnRedefinable(RootRougailObject):  # pylint: disable=R0903
+    """Object that could not be redefine
+    """
+
+
+class ObjSpace:  # pylint: disable=R0903
     """
     Base object space
     """
+
+
+class RougailObjSpace:
+    """Rougail ObjectSpace is an object's reflexion of the XML elements
+    """
+
+    def __init__(self,
+                 xmlreflector: XMLReflector,
+                 ) -> None:
         self.space = ObjSpace()
         self.paths = Path()
-        self.xmlreflector = XMLReflector()
-        self.xmlreflector.parse_dtd(dtdfilename)
-        self.redefine_variables = None
-        self.check_removed = None
-        self.condition_removed = None
 
-        # ['variable', 'separator', 'family']
-        self.forced_text_elts = set()
         self.forced_text_elts_as_name = set(FORCED_TEXT_ELTS_AS_NAME)
         self.list_conditions = {}
+        self.valid_enums = {}
         self.booleans_attributs = []
 
-        self.make_object_space_class()
+        self.make_object_space_classes(xmlreflector)
 
-    def make_object_space_class(self):
+    def make_object_space_classes(self,
-        """Create Rougail ObjectSpace class types, it enables us to create objects like:
+                                  xmlreflector: XMLReflector,
+                                  ) -> None:
+        """Create Rougail ObjectSpace class types from DDT file
+        It enables us to create objects like:
         File(), Variable(), Ip(), Family(), Constraints()... and so on.
+        """
 
-        Creole ObjectSpace is an object's reflexion of the XML elements"""
+        for dtd_elt in xmlreflector.dtd.iterelements():
-
-        for dtd_elt in self.xmlreflector.dtd.iterelements():
             attrs = {}
             if dtd_elt.name in FORCE_REDEFINABLES:
-                clstype = self.Redefinable
+                clstype = Redefinable
+            elif not dtd_elt.attributes() and dtd_elt.name not in FORCE_UNREDEFINABLES:
+                clstype = Atom
             else:
-                clstype = self.UnRedefinable
+                clstype = UnRedefinable
-            atomic = dtd_elt.name not in FORCE_UNREDEFINABLES and dtd_elt.name not in FORCE_REDEFINABLES
             forced_text_elt = dtd_elt.type == 'mixed'
             for dtd_attr in dtd_elt.iterattributes():
-                atomic = False
+                if set(dtd_attr.itervalues()) == {'True', 'False'}:
-                if set(dtd_attr.itervalues()) == set(['True', 'False']):
                     # it's a boolean
                     self.booleans_attributs.append(dtd_attr.name)
                 if dtd_attr.default_value:
                     # set default value for this attribute
                     default_value = dtd_attr.default_value
                     if dtd_attr.name in self.booleans_attributs:
-                        default_value = self.convert_boolean(dtd_attr.default_value)
+                        default_value = self.convert_boolean(default_value)
                     attrs[dtd_attr.name] = default_value
                 if dtd_attr.name == 'redefine':
                     # has a redefine attribute, so it's a Redefinable object
-                    clstype = self.Redefinable
+                    clstype = Redefinable
                 if dtd_attr.name == 'name' and forced_text_elt:
                     # child.text should be transform has a "name" attribute
-                    self.forced_text_elts.add(dtd_elt.name)
                     forced_text_elt = False
 
             if forced_text_elt is True:
                 self.forced_text_elts_as_name.add(dtd_elt.name)
-            if atomic:
-                # has any attribute so it's an Atomic object
-                clstype = self.Atom
 
             # create ObjectSpace object
             setattr(self, dtd_elt.name, type(dtd_elt.name.capitalize(), (clstype,), attrs))
+        for elt in FORCE_ELEMENTS:
+            setattr(self, elt, type(self._get_elt_name(elt), (RootRougailObject,), dict()))
 
-    def create_or_populate_from_xml(self,
+    @staticmethod
-                                    namespace,
+    def _get_elt_name(elt) -> str:
-                                    xmlfolders):
+        name = elt.capitalize()
-        """Parses a bunch of XML files
+        if name.endswith('_'):
-        populates the CreoleObjSpace
+            name = name[:-1]
-        """
+        return name
-        for xmlfile, document in self.xmlreflector.load_xml_from_folders(xmlfolders):
-            self.redefine_variables = []
-            self.check_removed = []
-            self.condition_removed = []
-            self.xml_parse_document(document,
-                                    self.space,
-                                    namespace,
-                                    )
 
     def xml_parse_document(self,
+                           xmlfile,
                            document,
                            space,
                            namespace,
                            ):
-        """Parses a Creole XML file
+        """Parses a Rougail XML file and populates the RougailObjSpace
-        populates the CreoleObjSpace
         """
+        if namespace in ['services', Config['variable_namespace']]:
+            msg = _(f'Namespace name "{namespace}" is not allowed')
+            raise DictConsistencyError(msg, 21, xmlfile)
+        if not namespace:
+            namespace = Config['variable_namespace']
+        redefine_variables = []
+        self._xml_parse(xmlfile,
+                        document,
+                        space,
+                        namespace,
+                        redefine_variables,
+                        )
+
+    def _xml_parse(self,  # pylint: disable=R0913
+                   xmlfile,
+                   document,
+                   space,
+                   namespace,
+                   redefine_variables,
+                   ) -> None:
+        # var to check unique family name in a XML file
         family_names = []
         for child in document:
-            # this index enables us to reorder objects
-            self.index += 1
-            # doesn't proceed the XML commentaries
             if not isinstance(child.tag, str):
+                # doesn't proceed the XML commentaries
                 continue
             if child.tag == 'family':
                 if child.attrib['name'] in family_names:
-                    raise DictConsistencyError(_('Family {} is set several times').format(child.attrib['name']))
+                    msg = _(f'Family "{child.attrib["name"]}" is set several times')
+                    raise DictConsistencyError(msg, 44, xmlfile)
                 family_names.append(child.attrib['name'])
-            if child.tag == 'variables':
-                child.attrib['name'] = namespace
-            if child.tag == 'value' and child.text == None:
-                # FIXME should not be here
-                continue
-            # variable objects creation
             try:
-                variableobj = self.generate_variableobj(child,
+                # variable objects creation
+                variableobj = self.get_variableobj(xmlfile,
+                                                   child,
                                                    space,
                                                    namespace,
+                                                   redefine_variables,
                                                    )
             except SpaceObjShallNotBeUpdated:
                 continue
-            self.set_text_to_obj(child,
+            self.set_text(child,
                           variableobj,
                           )
-            self.set_xml_attributes_to_obj(child,
+            self.set_attributes(xmlfile,
-                                           variableobj,
-                                           )
-            self.variableobj_tree_visitor(child,
-                                          variableobj,
-                                          namespace,
-                                          )
-            self.fill_variableobj_path_attribute(space,
                                 child,
-                                                 namespace,
+                                variableobj,
+                                )
+            self.remove(child,
+                        variableobj,
+                        redefine_variables,
+                        )
+            self.set_path(namespace,
                           document,
                           variableobj,
                           )
             self.add_to_tree_structure(variableobj,
                                        space,
                                        child,
+                                       namespace,
                                        )
             if list(child) != []:
-                self.xml_parse_document(child,
+                self._xml_parse(xmlfile,
+                                child,
                                 variableobj,
                                 namespace,
+                                redefine_variables,
                                 )
 
-    def generate_variableobj(self,
+    def get_variableobj(self,
-                           child,
+                        xmlfile: str,
+                        child: list,
                         space,
                         namespace,
-                           ):
+                        redefine_variables,
+                        ):  # pylint: disable=R0913
         """
-        instanciates or creates Creole Object Subspace objects
+        retrieves or creates Rougail Object Subspace objects
         """
-        variableobj = getattr(self, child.tag)()
+        obj = getattr(self, child.tag)
-        if isinstance(variableobj, self.Redefinable):
+        name = self._get_name(child, namespace)
-            variableobj = self.create_or_update_redefinable_object(child.attrib,
+        if Redefinable in obj.__mro__:
+            return self.create_or_update_redefinable_object(xmlfile,
+                                                            child.attrib,
                                                             space,
                                                             child,
+                                                            name,
                                                             namespace,
+                                                            redefine_variables,
                                                             )
-        elif isinstance(variableobj, self.Atom) and child.tag in vars(space):
+        if Atom in obj.__mro__:
-            # instanciates an object from the CreoleObjSpace's builtins types
+            if child.tag in vars(space):
-            # example : child.tag = constraints -> a self.Constraints() object is created
+                # Atom instance has to be a singleton here
-            # this Atom instance has to be a singleton here
                 # we do not re-create it, we reuse it
-            variableobj = getattr(space, child.tag)
+                return getattr(space, child.tag)
-        self.create_tree_structure(space,
+            return obj(xmlfile, name)
+        if child.tag not in vars(space):
+            setattr(space, child.tag, [])
+        return obj(xmlfile, name)
+
+    def _get_name(self,
                   child,
-                                   variableobj,
+                  namespace: str,
-                                   )
+                  ) -> Optional[str]:
-        return variableobj
+        if child.tag == 'variables':
+            return namespace
+        if 'name' in child.attrib:
+            return child.attrib['name']
+        if child.text and child.tag in self.forced_text_elts_as_name:
+            return child.text.strip()
+        return None
 
     def create_or_update_redefinable_object(self,
+                                            xmlfile,
                                             subspace,
                                             space,
                                             child,
+                                            name,
                                             namespace,
-                                            ):
+                                            redefine_variables,
-        """Creates or retrieves the space object that corresponds
+                                            ):  # pylint: disable=R0913
-        to the `child` XML object
+        """A redefinable object could be created or updated
-
-        Two attributes of the `child` XML object are important:
-
-        - with the `redefine` boolean flag attribute we know whether
-          the corresponding space object shall be created or updated
-
-            - `True` means that the corresponding space object shall be updated
-            - `False` means that the corresponding space object shall be created
-
-        - with the `exists` boolean flag attribute we know whether
-          the corresponding space object shall be created
-          (or nothing -- that is the space object isn't modified)
-
-            - `True` means that the corresponding space object shall be created
-            - `False` means that the corresponding space object is not updated
-
-        In the special case `redefine` is True and `exists` is False,
-        we create the corresponding space object if it doesn't exist
-        and we update it if it exists.
-
-        :return: the corresponding space object of the `child` XML object
         """
-        if child.tag in self.forced_text_elts_as_name:
+        existed_var = self.get_existed_obj(name,
-            name = child.text
-        else:
-            name = subspace['name']
-        if self.is_already_exists(name,
-                                  space,
-                                  child,
-                                  namespace,
-                                  ):
-            default_redefine = child.tag in FORCE_REDEFINABLES
-            redefine = self.convert_boolean(subspace.get('redefine', default_redefine))
-            exists = self.convert_boolean(subspace.get('exists', True))
-            if redefine is True:
-                return self.translate_in_space(name,
                                            space,
                                            child,
                                            namespace,
                                            )
-            elif exists is False:
+        if existed_var:
-                raise SpaceObjShallNotBeUpdated()
+            # if redefine is set to object, default value is False
-            raise DictConsistencyError(_(f'Already present in another XML file, {name} cannot be re-created'))
+            # otherwise it's always a redefinable object
-        redefine = self.convert_boolean(subspace.get('redefine', False))
+            default_redefine = child.tag in FORCE_REDEFINABLES
-        exists = self.convert_boolean(subspace.get('exists', False))
+            redefine = self.convert_boolean(subspace.get('redefine', default_redefine))
-        if redefine is False or exists is True:
+            if redefine is True:
-            return getattr(self, child.tag)()
+                if isinstance(existed_var, self.variable):  # pylint: disable=E1101
-        raise DictConsistencyError(_(f'Redefined object: {name} does not exist yet'))
+                    if namespace == Config['variable_namespace']:
-
+                        redefine_variables.append(name)
-    def create_tree_structure(self,
-                              space,
-                              child,
-                              variableobj,
-                              ):  # pylint: disable=R0201
-        """
-        Builds the tree structure of the object space here
-        we set services attributes in order to be populated later on
-        for example::
-
-            space = Family()
-            space.variable = dict()
-            another example:
-            space = Variable()
-            space.value = list()
-        """
-        if child.tag not in vars(space):
-            if isinstance(variableobj, self.Redefinable):
-                setattr(space, child.tag, dict())
-            elif isinstance(variableobj, self.UnRedefinable):
-                setattr(space, child.tag, [])
-            elif not isinstance(variableobj, self.Atom):  # pragma: no cover
-                raise OperationError(_("Creole object {} "
-                                             "has a wrong type").format(type(variableobj)))
-
-    def is_already_exists(self, name, space, child, namespace):
-        if isinstance(space, self.family):  # pylint: disable=E1101
-            if namespace != variable_namespace:
-                name = space.path + '.' + name
-            return self.paths.path_is_defined(name)
-        if child.tag == 'family':
-            norm_name = normalize_family(name)
                     else:
-            norm_name = name
+                        redefine_variables.append(space.path + '.' + name)
-        return norm_name in getattr(space, child.tag, {})
+                existed_var.xmlfiles.append(xmlfile)
+                return existed_var
+            exists = self.convert_boolean(subspace.get('exists', True))
+            if exists is False:
+                raise SpaceObjShallNotBeUpdated()
+            msg = _(f'"{child.tag}" named "{name}" cannot be re-created in "{xmlfile}", '
+                    f'already defined')
+            raise DictConsistencyError(msg, 45, existed_var.xmlfiles)
+        # object deos not exists
+        exists = self.convert_boolean(subspace.get('exists', False))
+        if exists is True:
+            # manage object only if already exists, so cancel
+            raise SpaceObjShallNotBeUpdated()
+        redefine = self.convert_boolean(subspace.get('redefine', False))
+        if redefine is True:
+            # cannot redefine an inexistant object
+            msg = _(f'Redefined object: "{name}" does not exist yet')
+            raise DictConsistencyError(msg, 46, xmlfile)
+        if child.tag not in vars(space):
+            setattr(space, child.tag, {})
+        return getattr(self, child.tag)(xmlfile, name)
 
-    def convert_boolean(self, value):  # pylint: disable=R0201
+    def get_existed_obj(self,
-        """Boolean coercion. The Creole XML may contain srings like `True` or `False`
+                        name: str,
+                        space: str,
+                        child,
+                        namespace: str,
+                        ) -> None:
+        """if an object exists, return it
+        """
+        if child.tag in ['variable', 'family']:
+            name = normalize_family(name)
+        if isinstance(space, self.family):  # pylint: disable=E1101
+            if namespace != Config['variable_namespace']:
+                name = space.path + '.' + name
+            if not self.paths.path_is_defined(name):
+                return None
+            old_family_name = self.paths.get_variable_family_path(name)
+            if space.path != old_family_name:
+                msg = _(f'Variable was previously create in family "{old_family_name}", '
+                        f'now it is in "{space.path}"')
+                raise DictConsistencyError(msg, 47, space.xmlfiles)
+            return self.paths.get_variable(name)
+        # it's not a family
+        children = getattr(space, child.tag, {})
+        if name in children:
+            return children[name]
+        return None
+
+    @staticmethod
+    def convert_boolean(value: str) -> bool:
+        """Boolean coercion. The Rougail XML may contain srings like `True` or `False`
         """
         if isinstance(value, bool):
             return value
         if value == 'True':
             return True
-        elif value == 'False':
         return False
-        else:
-            raise TypeError(_('{} is not True or False').format(value))  # pragma: no cover
 
-    def translate_in_space(self,
+    def set_text(self,
-                           name,
-                           family,
-                           variable,
-                           namespace,
-                           ):
-        if not isinstance(family, self.family):  # pylint: disable=E1101
-            if variable.tag == 'family':
-                norm_name = normalize_family(name)
-            else:
-                norm_name = name
-            return getattr(family, variable.tag)[norm_name]
-        if namespace == variable_namespace:
-            path = name
-        else:
-            path = family.path + '.' + name
-        old_family_name = self.paths.get_variable_family_name(path)
-        if normalize_family(family.name) == old_family_name:
-            return getattr(family, variable.tag)[name]
-        old_family = self.space.variables[variable_namespace].family[old_family_name]  # pylint: disable=E1101
-        variable_obj = old_family.variable[name]
-        del old_family.variable[name]
-        if 'variable' not in vars(family):
-            family.variable = dict()
-        family.variable[name] = variable_obj
-        self.paths.add_variable(namespace,
-                                name,
-                                family.name,
-                                False,
-                                variable_obj,
-                                )
-        return variable_obj
-
-    def remove_check(self, name):  # pylint: disable=C0111
-        if hasattr(self.space, 'constraints') and hasattr(self.space.constraints, 'check'):
-            remove_checks = []
-            for idx, check in enumerate(self.space.constraints.check):  # pylint: disable=E1101
-                if hasattr(check, 'target') and check.target == name:
-                    remove_checks.append(idx)
-
-            remove_checks = list(set(remove_checks))
-            remove_checks.sort(reverse=True)
-            for idx in remove_checks:
-                self.space.constraints.check.pop(idx)  # pylint: disable=E1101
-
-    def remove_condition(self, name):  # pylint: disable=C0111
-        for idx, condition in enumerate(self.space.constraints.condition):  # pylint: disable=E1101
-            remove_targets = []
-            if hasattr(condition, 'target'):
-                for target_idx, target in enumerate(condition.target):
-                    if target.name == name:
-                        remove_targets.append(target_idx)
-            remove_targets = list(set(remove_targets))
-            remove_targets.sort(reverse=True)
-            for idx in remove_targets:
-                del condition.target[idx]
-
-    def add_to_tree_structure(self,
-                              variableobj,
-                              space,
-                              child,
-                              ):  # pylint: disable=R0201
-        if isinstance(variableobj, self.Redefinable):
-            name = variableobj.name
-            if child.tag == 'family':
-                name = normalize_family(name)
-            getattr(space, child.tag)[name] = variableobj
-        elif isinstance(variableobj, self.UnRedefinable):
-            getattr(space, child.tag).append(variableobj)
-        else:
-            setattr(space, child.tag, variableobj)
-
-    def set_text_to_obj(self,
                  child,
                  variableobj,
-                        ):
+                 ) -> None:
-        if child.text is None:
+        """set text
-            text = None
+        """
-        else:
+        if child.text is None or child.tag in self.forced_text_elts_as_name:
+            return
         text = child.text.strip()
         if text:
-            if child.tag in self.forced_text_elts_as_name:
-                variableobj.name = text
-            else:
             variableobj.text = text
 
-    def set_xml_attributes_to_obj(self,
+    def set_attributes(self,
+                       xmlfile,
                        child,
                        variableobj,
                        ):
+        """ set attributes to an object
+        """
         redefine = self.convert_boolean(child.attrib.get('redefine', False))
+        if redefine and child.tag == 'variable':
+            # delete old values
             has_value = hasattr(variableobj, 'value')
-        if redefine is True and child.tag == 'variable' and has_value and len(child) != 0:
+            if has_value and len(child) != 0:
                 del variableobj.value
         for attr, val in child.attrib.items():
             if redefine and attr in UNREDEFINABLE:
-                # UNREDEFINABLE concerns only 'variable' node so we can fix name
+                msg = _(f'cannot redefine attribute "{attr}" for variable "{child.attrib["name"]}"'
-                # to child.attrib['name']
+                        f' in "{xmlfile}", already defined')
-                name = child.attrib['name']
+                raise DictConsistencyError(msg, 48, variableobj.xmlfiles[:-1])
-                raise DictConsistencyError(_(f'cannot redefine attribute {attr} for variable {name}'))
             if attr in self.booleans_attributs:
                 val = self.convert_boolean(val)
-            if not (attr == 'name' and getattr(variableobj, 'name', None) != None):
+            if attr == 'name' and getattr(variableobj, 'name', None):
+                # do not redefine name
+                continue
             setattr(variableobj, attr, val)
-        keys = list(vars(variableobj).keys())
 
-    def variableobj_tree_visitor(self,
+    def remove(self,
                child,
                variableobj,
-                                 namespace,
+               redefine_variables,
                ):
-        """Creole object tree manipulations
+        """Rougail object tree manipulations
         """
         if child.tag == 'variable':
             if child.attrib.get('remove_check', False):
                 self.remove_check(variableobj.name)
             if child.attrib.get('remove_condition', False):
                 self.remove_condition(variableobj.name)
+            if child.attrib.get('remove_fill', False):
+                self.remove_fill(variableobj.name)
         if child.tag == 'fill':
-            # if variable is a redefine in current dictionary
+            for target in child:
-            # XXX not working with variable not in variable and in leader/followers
+                if target.tag == 'target' and target.text in redefine_variables:
-            variableobj.redefine = child.attrib['target'] in self.redefine_variables
+                    self.remove_fill(target.text)
-        if not hasattr(variableobj, 'index'):
-            variableobj.index = self.index
-        if child.tag == 'check' and child.attrib['target'] in self.redefine_variables and child.attrib['target'] not in self.check_removed:
-            self.remove_check(child.attrib['target'])
-            self.check_removed.append(child.attrib['target'])
-        if child.tag == 'condition' and child.attrib['source'] in self.redefine_variables and child.attrib['source'] not in self.check_removed:
-            self.remove_condition(child.attrib['source'])
-            self.condition_removed.append(child.attrib['source'])
-        variableobj.namespace = namespace
 
-    def fill_variableobj_path_attribute(self,
+    def remove_check(self, name):
-                                        space,
+        """Remove a check with a specified target
-                                        child,
+        """
+        remove_checks = []
+        for idx, check in enumerate(self.space.constraints.check):  # pylint: disable=E1101
+            for target in check.target:
+                if target.name == name:
+                    remove_checks.append(idx)
+        remove_checks.sort(reverse=True)
+        for idx in remove_checks:
+            self.space.constraints.check.pop(idx)  # pylint: disable=E1101
+
+    def remove_condition(self,
+                         name: str,
+                         ) -> None:
+        """Remove a condition with a specified source
+        """
+        remove_conditions = []
+        for idx, condition in enumerate(self.space.constraints.condition):  # pylint: disable=E1101
+            if condition.source == name:
+                remove_conditions.append(idx)
+        remove_conditions.sort(reverse=True)
+        for idx in remove_conditions:
+            del self.space.constraints.condition[idx]  # pylint: disable=E1101
+
+    def remove_fill(self,
+                    name: str,
+                    ) -> None:
+        """Remove a fill with a specified target
+        """
+        remove_fills = []
+        for idx, fill in enumerate(self.space.constraints.fill):  # pylint: disable=E1101
+            for target in fill.target:
+                if target.name == name:
+                    remove_fills.append(idx)
+        remove_fills.sort(reverse=True)
+        for idx in remove_fills:
+            self.space.constraints.fill.pop(idx)  # pylint: disable=E1101
+
+    def set_path(self,
                  namespace,
                  document,
                  variableobj,
-                                        ):  # pylint: disable=R0913
+                 ):
         """Fill self.paths attributes
         """
-        if isinstance(space, self.help):  # pylint: disable=E1101
+        if isinstance(variableobj, self.variable):  # pylint: disable=E1101
-            return
-        if child.tag == 'variable':
             family_name = normalize_family(document.attrib['name'])
             self.paths.add_variable(namespace,
-                                    child.attrib['name'],
+                                    normalize_family(variableobj.name),
-                                    family_name,
+                                    namespace + '.' + family_name,
-                                    document.attrib.get('dynamic') != None,
+                                    document.attrib.get('dynamic') is not None,
-                                    variableobj)
+                                    variableobj,
-            if child.attrib.get('redefine', 'False') == 'True':
+                                    )
-                if namespace == variable_namespace:
+        elif isinstance(variableobj, self.family):  # pylint: disable=E1101
-                    self.redefine_variables.append(child.attrib['name'])
+            family_name = normalize_family(variableobj.name)
-                else:
+            if namespace != Config['variable_namespace']:
-                    self.redefine_variables.append(namespace + '.' + family_name + '.' +
-                                                   child.attrib['name'])
-
-        elif child.tag == 'family':
-            family_name = normalize_family(child.attrib['name'])
-            if namespace != variable_namespace:
                 family_name = namespace + '.' + family_name
             self.paths.add_family(namespace,
                                   family_name,
                                   variableobj,
                                   )
-            variableobj.path = self.paths.get_family_path(family_name, namespace)
 
-    def space_visitor(self, eosfunc_file):  # pylint: disable=C0111
+    @staticmethod
-        self.funcs_path = eosfunc_file
+    def add_to_tree_structure(variableobj,
-        SpaceAnnotator(self, eosfunc_file)
+                              space,
-
+                              child,
-    def save(self,
+                              namespace: str,
-             ):
+                              ) -> None:
-        tiramisu_objects = TiramisuReflector(self.space,
+        """add a variable to the tree
-                                             self.funcs_path,
+        """
-                                             )
+        variableobj.namespace = namespace
-        return tiramisu_objects.get_text() + '\n'
+        if isinstance(variableobj, Redefinable):
+            name = variableobj.name
+            if child.tag in ['family', 'variable']:
+                name = normalize_family(name)
+            getattr(space, child.tag)[name] = variableobj
+        elif isinstance(variableobj, UnRedefinable):
+            getattr(space, child.tag).append(variableobj)
+        else:
+            setattr(space, child.tag, variableobj)

src/rougail/path.py

@@ -1,7 +1,33 @@
+"""Manage path to find objects
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
 from .i18n import _
+from .error import DictConsistencyError
+from .config import Config
 from .utils import normalize_family
-from .error import OperationError, DictConsistencyError
-from .config import variable_namespace
 
 
 class Path:
@@ -13,7 +39,8 @@ class Path:
     def __init__(self):
         self.variables = {}
         self.families = {}
-        self.full_paths = {}
+        self.full_paths_families = {}
+        self.full_paths_variables = {}
 
     # Family
     def add_family(self,
@@ -21,184 +48,176 @@ class Path:
                    name: str,
                    variableobj: str,
                    ) -> str:  # pylint: disable=C0111
-        if '.' not in name and namespace == variable_namespace:
+        """Add a new family
+        """
+        if namespace == Config['variable_namespace']:
             full_name = '.'.join([namespace, name])
-            self.full_paths[name] = full_name
+            self.full_paths_families[name] = full_name
         else:
+            if '.' not in name:  # pragma: no cover
+                msg = _(f'Variable "{name}" in namespace "{namespace}" must have dot')
+                raise DictConsistencyError(msg, 39, variableobj.xmlfiles)
             full_name = name
-        if full_name in self.families and self.families[full_name]['variableobj'] != variableobj:
+        if full_name in self.families and \
-            raise DictConsistencyError(_(f'Duplicate family name {name}'))
+                self.families[full_name]['variableobj'] != variableobj:  # pragma: no cover
+            raise DictConsistencyError(_(f'Duplicate family name "{name}"'), 37, variableobj.xmlfiles)
         self.families[full_name] = dict(name=name,
                                         namespace=namespace,
                                         variableobj=variableobj,
                                         )
+        variableobj.path = full_name
 
-    def get_family_path(self,
+    def add_leadership(self,
+                       namespace: str,
+                       path: str,
+                       variableobj: str,
+                       ) -> str:  # pylint: disable=C0111
+        """add a new leadership
+        """
+        self.families[path] = dict(name=path,
+                                   namespace=namespace,
+                                   variableobj=variableobj,
+                                   )
+        variableobj.path = path
+
+    def get_family(self,
                    name: str,
                    current_namespace: str,
-                        ) -> str:  # pylint: disable=C0111
-        name = normalize_family(name,
-                                check_name=False,
-                                allow_dot=True,
-                                )
-        if '.' not in name and current_namespace == variable_namespace and name in self.full_paths:
-            name = self.full_paths[name]
-        if current_namespace is None:  # pragma: no cover
-            raise OperationError('current_namespace must not be None')
-        dico = self.families[name]
-        if dico['namespace'] != variable_namespace and current_namespace != dico['namespace']:
-            raise DictConsistencyError(_('A family located in the {} namespace '
-                                               'shall not be used in the {} namespace').format(
-                                                   dico['namespace'], current_namespace))
-        return dico['name']
-
-    def get_family_obj(self,
-                       name: str,
                    ) -> 'Family':  # pylint: disable=C0111
-        if '.' not in name and name in self.full_paths:
+        """Get a family
-            name = self.full_paths[name]
+        """
+        name = '.'.join([normalize_family(subname) for subname in name.split('.')])
+        if name not in self.families and name in self.full_paths_families:
+            name = self.full_paths_families[name]
         if name not in self.families:
-            raise DictConsistencyError(_('unknown family {}').format(name))
+            raise DictConsistencyError(_(f'unknown option {name}'), 42, [])
         dico = self.families[name]
+        if current_namespace not in [Config['variable_namespace'], 'services'] and \
+                current_namespace != dico['namespace']:
+            msg = _(f'A family located in the "{dico["namespace"]}" namespace '
+                    f'shall not be used in the "{current_namespace}" namespace')
+            raise DictConsistencyError(msg, 38, [])
         return dico['variableobj']
 
-    def family_is_defined(self,
-                        name: str,
-                        ) -> str:  # pylint: disable=C0111
-        if '.' not in name and name not in self.families and name in self.full_paths:
-            return True
-        return name in self.families
-
     # Leadership
     def set_leader(self,
                    namespace: str,
                    leader_family_name: str,
+                   leadership_name: str,
                    name: str,
-                   leader_name: str,
                    ) -> None:  # pylint: disable=C0111
+        """set a variable a leadership member
+        """
         # need rebuild path and move object in new path
-        old_path = namespace + '.' + leader_family_name + '.' + name
+        old_path = leader_family_name + '.' + name
-        dico = self._get_variable(old_path)
+        leadership_path = leader_family_name + '.' + leadership_name
-        del self.variables[old_path]
+        new_path = leadership_path + '.' + name
-        new_path = namespace + '.' + leader_family_name + '.' + leader_name + '.' + name
+        self.variables[new_path] = self.variables.pop(old_path)
-        self.add_variable(namespace,
+        self.variables[new_path]['leader'] = leadership_path
-                          new_path,
+        self.variables[new_path]['variableobj'].path = new_path
-                          dico['family'],
+        self.variables[new_path]['family'] = leadership_path
-                          False,
+        if namespace == Config['variable_namespace']:
-                          dico['variableobj'],
+            self.full_paths_variables[name] = new_path
-                          )
-        if namespace == variable_namespace:
-            self.full_paths[name] = new_path
-        else:
-            name = new_path
-        dico = self._get_variable(name)
-        if dico['leader'] != None:
-            raise DictConsistencyError(_('Already defined leader {} for variable'
-                                         ' {}'.format(dico['leader'], name)))
-        dico['leader'] = leader_name
 
-    def get_leader(self, name):  # pylint: disable=C0111
+    def is_leader(self, path):  # pylint: disable=C0111
-        return self._get_variable(name)['leader']
+        """Is the variable is a leader
+        """
+        variable = self._get_variable(path)
+        if not variable['leader']:
+            return False
+        leadership = self.get_family(variable['leader'], variable['variableobj'].namespace)
+        return leadership.variable[0].path == path
 
     # Variable
-    def add_variable(self,
+    def add_variable(self,  # pylint: disable=R0913
                      namespace: str,
                      name: str,
                      family: str,
                      is_dynamic: bool,
                      variableobj,
                      ) -> str:  # pylint: disable=C0111
+        """Add a new variable (with path)
+        """
         if '.' not in name:
-            full_name = '.'.join([namespace, family, name])
+            full_path = '.'.join([family, name])
-            self.full_paths[name] = full_name
+            if namespace == Config['variable_namespace']:
+                self.full_paths_variables[name] = full_path
         else:
-            full_name = name
+            full_path = name
-            if namespace == variable_namespace:
+        variableobj.path = full_path
-                name = name.rsplit('.', 1)[1]
+        self.variables[full_path] = dict(name=name,
-        self.variables[full_name] = dict(name=name,
                                          family=family,
-                                         namespace=namespace,
                                          leader=None,
                                          is_dynamic=is_dynamic,
-                                         variableobj=variableobj)
+                                         variableobj=variableobj,
+                                         )
 
-    def get_variable_name(self,
+    def get_variable(self,
-                          name,
+                     name: str,
-                          ):  # pylint: disable=C0111
-        return self._get_variable(name)['name']
-
-    def get_variable_obj(self,
-                         name:str,
                      ) -> 'Variable':  # pylint: disable=C0111
-        return self._get_variable(name)['variableobj']
+        """Get variable object from a path
+        """
+        variable, suffix = self._get_variable(name, with_suffix=True)
+        if suffix:
+            raise DictConsistencyError(_(f"{name} is a dynamic variable"), 36, [])
+        return variable['variableobj']
 
-    def get_variable_family_name(self,
+    def get_variable_family_path(self,
                                  name: str,
                                  ) -> str:  # pylint: disable=C0111
+        """Get the full path of a family
+        """
         return self._get_variable(name)['family']
 
-    def get_variable_namespace(self,
-                               name: str,
-                               ) -> str:  # pylint: disable=C0111
-        return self._get_variable(name)['namespace']
-
     def get_variable_path(self,
                           name: str,
                           current_namespace: str,
-                          allow_source: str=False,
-                          with_suffix: bool=False,
                           ) -> str:  # pylint: disable=C0111
-        if current_namespace is None:  # pragma: no cover
+        """get full path of a variable
-            raise OperationError('current_namespace must not be None')
+        """
-        if with_suffix:
         dico, suffix = self._get_variable(name,
                                           with_suffix=True,
                                           )
-        else:
+        namespace = dico['variableobj'].namespace
-            dico = self._get_variable(name)
+        if namespace not in [Config['variable_namespace'], 'services'] and \
-        if not allow_source:
+                current_namespace != namespace:
-            if dico['namespace'] not in [variable_namespace, 'services'] and current_namespace != dico['namespace']:
+            msg = _(f'A variable located in the "{namespace}" namespace shall not be used '
-                raise DictConsistencyError(_('A variable located in the {} namespace '
+                    f'in the "{current_namespace}" namespace')
-                                                   'shall not be used in the {} namespace').format(
+            raise DictConsistencyError(msg, 41, [])
-                                                       dico['namespace'], current_namespace))
+        return dico['variableobj'].path, suffix
-        if '.' in dico['name']:
-            value = dico['name']
-        else:
-            list_path = [dico['namespace'], dico['family']]
-            if dico['leader'] is not None:
-                list_path.append(dico['leader'])
-            list_path.append(dico['name'])
-            value = '.'.join(list_path)
-        if with_suffix:
-            return value, suffix
-        return value
 
     def path_is_defined(self,
-                        name: str,
+                        path: str,
                         ) -> str:  # pylint: disable=C0111
-        if '.' not in name and name not in self.variables and name in self.full_paths:
+        """The path is a valid path
+        """
+        if '.' not in path and path not in self.variables and path in self.full_paths_variables:
             return True
-        return name in self.variables
+        return path in self.variables
+
+    def variable_is_dynamic(self,
+                            name: str,
+                            ) -> bool:
+        """This variable is in dynamic family
+        """
+        return self._get_variable(name)['is_dynamic']
 
     def _get_variable(self,
                       name: str,
                       with_suffix: bool=False,
                       ) -> str:
+        name = '.'.join([normalize_family(subname) for subname in name.split('.')])
         if name not in self.variables:
-            if name not in self.variables:
+            if '.' not in name and name in self.full_paths_variables:
-                if '.' not in name and name in self.full_paths:
+                name = self.full_paths_variables[name]
-                    name = self.full_paths[name]
+            elif with_suffix:
-            if name not in self.variables:
+                for var_name, full_path in self.full_paths_variables.items():
-                for var_name, variable in self.variables.items():
-                    if variable['is_dynamic'] and name.startswith(var_name):
-                        return variable, name[len(var_name):]
-                if '.' not in name:
-                    for var_name, path in self.full_paths.items():
                     if name.startswith(var_name):
-                            variable = self.variables[self.full_paths[var_name]]
+                        variable = self._get_variable(full_path)
                         if variable['is_dynamic']:
                             return variable, name[len(var_name):]
-                raise DictConsistencyError(_('unknown option {}').format(name))
+        if name not in self.variables:
+            raise DictConsistencyError(_(f'unknown option {name}'), 42, [])
         if with_suffix:
             return self.variables[name], None
         return self.variables[name]

src/rougail/rougail.py

@@ -0,0 +1,94 @@
+"""Takes a bunch of Rougail XML dispatched in differents folders
+as an input and outputs a Tiramisu's file.
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+
+Sample usage::
+
+    >>> from rougail import Rougail
+    >>> rougail = Rougail('/usr/share/rougail/rougail.dtd')
+    >>> rougail.create_or_populate_from_xml(['/usr/share/rougail/dicos'])
+    >>> rougail.create_or_populate_from_xml(['/usr/share/rougail/extra1'], 'extra1')
+    >>> rougail.space_visitor('/usr/share/rougail/funcs.py')
+    >>> tiramisu = rougail.save()
+
+The Rougail
+
+- loads the XML into an internal RougailObjSpace representation
+- visits/annotates the objects
+- dumps the object space as Tiramisu string
+
+The visit/annotation stage is a complex step that corresponds to the Rougail
+procedures.
+"""
+
+from typing import List
+from .objspace import RougailObjSpace
+from .xmlreflector import XMLReflector
+from .tiramisureflector import TiramisuReflector
+from .annotator import SpaceAnnotator
+
+
+class Rougail:
+    """Rougail object
+    """
+    def __init__(self,
+                 dtdfilename: str,
+                 ) -> None:
+        self.xmlreflector = XMLReflector()
+        self.xmlreflector.parse_dtd(dtdfilename)
+        self.rougailobjspace = RougailObjSpace(self.xmlreflector)
+        self.funcs_path = None
+
+    def create_or_populate_from_xml(self,
+                                    xmlfolders: List[str],
+                                    namespace: str=None,
+                                    ) -> List[str]:
+        """Parses a bunch of XML files and populates the RougailObjSpace
+        """
+        for xmlfile in self.xmlreflector.load_xml_from_folders(xmlfolders):
+            document = self.xmlreflector.parse_xmlfile(xmlfile)
+            self.rougailobjspace.redefine_variables = []
+            self.rougailobjspace.xml_parse_document(xmlfile,
+                                                    document,
+                                                    self.rougailobjspace.space,
+                                                    namespace,
+                                                    )
+
+    def space_visitor(self,
+                      eosfunc_file: str,
+                      ) -> None:
+        """All XML are loader, now annotate content
+        """
+        self.funcs_path = eosfunc_file
+        SpaceAnnotator(self.rougailobjspace, eosfunc_file)
+
+    def save(self) -> str:
+        """Return tiramisu object declaration as a string
+        """
+        tiramisu_objects = TiramisuReflector(self.rougailobjspace,
+                                             self.funcs_path,
+                                             )
+        return tiramisu_objects.get_text() + '\n'

src/rougail/template.py

@@ -1,61 +1,61 @@
-# -*- coding: utf-8 -*-
+"""Template langage for Rougail
-"""
+
-Gestion du mini-langage de template
+Created by:
-On travaille sur les fichiers cibles
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 """
 
-import imp
 from shutil import copy
 import logging
 from typing import Dict, Any
 from subprocess import call
-from os import listdir, makedirs
+from os import listdir, makedirs, getcwd, chdir
-from os.path import dirname, join, isfile
+from os.path import dirname, join, isfile, abspath, normpath, relpath
 
 from Cheetah.Template import Template as ChtTemplate
 from Cheetah.NameMapper import NotFound as CheetahNotFound
 
 try:
+    from tiramisu3 import Config
+    from tiramisu3.error import PropertiesOptionError  # pragma: no cover
+except ModuleNotFoundError:  # pragma: no cover
     from tiramisu import Config
     from tiramisu.error import PropertiesOptionError
-except:
-    from tiramisu3 import Config
-    from tiramisu3.error import PropertiesOptionError
 
-from .config import patch_dir, variable_namespace
+from .config import Config
 from .error import FileNotFound, TemplateError
 from .i18n import _
-from .utils import normalize_family
+from .utils import normalize_family, load_modules
 
 
 log = logging.getLogger(__name__)
 log.addHandler(logging.NullHandler())
 
 
-class IsDefined:
-    """
-    filtre permettant de ne pas lever d'exception au cas où
-    la variable Creole n'est pas définie
-     """
-    def __init__(self, context):
-        self.context = context
-
-    def __call__(self, varname):
-        if '.' in varname:
-            splitted_var = varname.split('.')
-            if len(splitted_var) != 2:
-                msg = _("Group variables must be of type leader.follower")
-                raise KeyError(msg)
-            leader, follower = splitted_var
-            if leader in self.context:
-                return follower in self.context[leader].follower.keys()
-            return False
-        else:
-            return varname in self.context
-
-
 @classmethod
 def cl_compile(kls, *args, **kwargs):
+    """Rewrite compile methode to force some settings
+    """
     kwargs['compilerSettings'] = {'directiveStartToken' : '%',
                                   'cheetahVarStartToken' : '%%',
                                   'EOLSlurpToken' : '%',
@@ -65,154 +65,155 @@ def cl_compile(kls, *args, **kwargs):
                                   'commentEndToken' : 'µ' * 10,
                                   'multiLineCommentStartToken' : 'µ' * 10,
                                   'multiLineCommentEndToken' : 'µ' * 10}
-    return kls.old_compile(*args, **kwargs)
+    return kls.old_compile(*args, **kwargs)  # pylint: disable=E1101
 ChtTemplate.old_compile = ChtTemplate.compile
 ChtTemplate.compile = cl_compile
 
 
-class CheetahTemplate(ChtTemplate):
+class CheetahTemplate(ChtTemplate):  # pylint: disable=W0223
-    """classe pour personnaliser et faciliter la construction
+    """Construct a cheetah templating object
-    du template Cheetah
     """
     def __init__(self,
                  filename: str,
                  context,
                  eosfunc: Dict,
-                 destfilename,
+                 extra_context: Dict,
-                 variable):
+                 ):
         """Initialize Creole CheetahTemplate
         """
-        extra_context = {'is_defined' : IsDefined(context),
-                         'normalize_family': normalize_family,
-                         'rougail_filename': destfilename
-                         }
-        if variable:
-            extra_context['rougail_variable'] = variable
         ChtTemplate.__init__(self,
                              file=filename,
                              searchList=[context, eosfunc, extra_context])
 
+    # FORK of Cheetah function, do not replace '\\' by '/'
+    def serverSidePath(self,
+                       path=None,
+                       normpath=normpath,
+                       abspath=abspath
+                       ):  # pylint: disable=W0621
 
-class CreoleLeader:
+        # strange...
-    def __init__(self, value, follower=None, index=None):
+        if path is None and isinstance(self, str):
-        """
+            path = self
-        On rend la variable itérable pour pouvoir faire:
+        if path:  # pylint: disable=R1705
-        for ip in iplist:
+            return normpath(abspath(path))
-        print(ip.network)
+#           original code return normpath(abspath(path.replace("\\", '/')))
-        print(ip.netmask)
+        elif hasattr(self, '_filePath') and self._filePath:  # pragma: no cover
-        print(ip)
+            return normpath(abspath(self._filePath))
-        index is used for CreoleLint
+        else:  # pragma: no cover
+            return None
+
+
+class CreoleLeaderIndex:
+    """This object is create when access to a specified Index of the variable
     """
+    def __init__(self,
+                 value,
+                 follower,
+                 index,
+                 ) -> None:
         self._value = value
-        if follower is not None:
+        self._follower = follower
-            self.follower = follower
-        else:
-            self.follower = {}
         self._index = index
 
     def __getattr__(self, name):
-        """Get follower variable or attribute of leader value.
+        if name not in self._follower:
-
+            raise AttributeError()
-        If the attribute is a name of a follower variable, return its value.
+        value = self._follower[name]
-        Otherwise, returns the requested attribute of leader value.
-        """
-        if name in self.follower:
-            value = self.follower[name]
         if isinstance(value, PropertiesOptionError):
             raise AttributeError()
         return value
-        else:
+
-            return getattr(self._value, name)
+    def __str__(self):
+        return str(self._value)
+
+    def __lt__(self, value):
+        return self._value.__lt__(value)
+
+    def __le__(self, value):
+        return self._value.__le__(value)
+
+    def __eq__(self, value):
+        return self._value.__eq__(value)
+
+    def __ne__(self, value):
+        return self._value.__ne__(value)
+
+    def __gt__(self, value):
+        return self._value.__gt__(value)
+
+    def __ge__(self, value):
+        return self._value >= value
+
+    def __add__(self, value):
+        return self._value.__add__(value)
+
+    def __radd__(self, value):
+        return value + self._value
+
+
+class CreoleLeader:
+    """Implement access to leader and follower variable
+    For examples: %%leader, %%leader[0].follower1
+    """
+    def __init__(self,
+                 value,
+                 ) -> None:
+        self._value = value
+        self._follower = {}
 
     def __getitem__(self, index):
         """Get a leader.follower at requested index.
         """
-        ret = {}
+        followers = {key: values[index] for key, values in self._follower.items()}
-        for key, values in self.follower.items():
+        return CreoleLeaderIndex(self._value[index],
-            ret[key] = values[index]
+                                 followers,
-        return CreoleLeader(self._value[index], ret, index)
+                                 index,
+                                 )
 
     def __iter__(self):
         """Iterate over leader.follower.
 
         Return synchronised value of leader.follower.
         """
-        for i in range(len(self._value)):
+        for index in range(len(self._value)):
-            ret = {}
+            yield self.__getitem__(index)
-            for key, values in self.follower.items():
-                ret[key] = values[i]
-            yield CreoleLeader(self._value[i], ret, i)
 
     def __len__(self):
-        """Delegate to leader value
-        """
         return len(self._value)
 
-    def __repr__(self):
+    def __contains__(self, value):
-        """Show CreoleLeader as dictionary.
+        return self._value.__contains__(value)
 
-        The leader value is stored under 'value' key.
+    async def add_follower(self,
-        The followers are stored under 'follower' key.
+                           config,
+                           name: str,
+                           path: str,
+                           ):
+        """Add a new follower
         """
-        return repr({'value': self._value, 'follower': self.follower})
+        self._follower[name] = []
-
+        for index in range(len(self._value)):
-    def __eq__(self, value):
-        return value == self._value
-
-    def __ne__(self, value):
-        return value != self._value
-
-    def __lt__(self, value):
-        return self._value < value
-
-    def __le__(self, value):
-        return self._value <= value
-
-    def __gt__(self, value):
-        return self._value > value
-
-    def __ge__(self, value):
-        return self._value >= value
-
-    def __str__(self):
-        """Delegate to leader value
-        """
-        return str(self._value)
-
-    def __add__(self, val):
-        return self._value.__add__(val)
-
-    def __radd__(self, val):
-        return val + self._value
-
-    def __contains__(self, item):
-        return item in self._value
-
-    async def add_follower(self, config, name, path):
-        if isinstance(self._value, list):
-            values = []
-            for idx in range(len(self._value)):
             try:
-                    values.append(await config.option(path, idx).value.get())
+                value = await config.option(path, index).value.get()
             except PropertiesOptionError as err:
-                    values.append(err)
+                value = err
-        else:
+            self._follower[name].append(value)
-            raise Exception('hu?')
-        self.follower[name] = values
 
 
 class CreoleExtra:
+    """Object that implement access to extra variable
+    For example %%extra1.family.variable
+    """
     def __init__(self,
                  suboption: Dict) -> None:
         self.suboption = suboption
 
     def __getattr__(self,
-                    key: str) -> Any:
+                    key: str,
+                    ) -> Any:
         return self.suboption[key]
 
-    def __repr__(self):
-        return self.suboption.__str__()
-
     def __iter__(self):
         return iter(self.suboption.values())
 
@@ -220,7 +221,7 @@ class CreoleExtra:
 class CreoleTemplateEngine:
     """Engine to process Creole cheetah template
     """
-    def __init__(self,
+    def __init__(self,  # pylint: disable=R0913
                  config: Config,
                  eosfunc_file: str,
                  distrib_dir: str,
@@ -233,115 +234,84 @@ class CreoleTemplateEngine:
         self.distrib_dir = distrib_dir
         eos = {}
         if eosfunc_file is not None:
-            eosfunc = imp.load_source('eosfunc', eosfunc_file)
+            eosfunc = load_modules(eosfunc_file)
             for func in dir(eosfunc):
                 if not func.startswith('_'):
                     eos[func] = getattr(eosfunc, func)
         self.eosfunc = eos
         self.rougail_variables_dict = {}
 
-    async def load_eole_variables_rougail(self,
-                                          optiondescription):
-        for option in await optiondescription.list('all'):
-            if await option.option.isoptiondescription():
-                if await option.option.isleadership():
-                    for idx, suboption in enumerate(await option.list('all')):
-                        if idx == 0:
-                            leader = CreoleLeader(await suboption.value.get())
-                            self.rougail_variables_dict[await suboption.option.name()] = leader
-                        else:
-                            await leader.add_follower(self.config,
-                                                   await suboption.option.name(),
-                                                   await suboption.option.path())
-                else:
-                    await self.load_eole_variables_rougail(option)
-            else:
-                self.rougail_variables_dict[await option.option.name()] = await option.value.get()
-
-    async def load_eole_variables(self,
-                                  namespace,
-                                  optiondescription):
-        families = {}
-        for family in await optiondescription.list('all'):
-            variables = {}
-            for variable in await family.list('all'):
-                if await variable.option.isoptiondescription():
-                    if await variable.option.isleadership():
-                        for idx, suboption in enumerate(await variable.list('all')):
-                            if idx == 0:
-                                leader = CreoleLeader(await suboption.value.get())
-                                leader_name = await suboption.option.name()
-                            else:
-                                await leader.add_follower(self.config,
-                                                       await suboption.option.name(),
-                                                       await suboption.option.path())
-                        variables[leader_name] = leader
-                    else:
-                        subfamilies = await self.load_eole_variables(await variable.option.name(),
-                                                                     variable,
-                                                                     )
-                        variables[await variable.option.name()] = subfamilies
-                else:
-                    variables[await variable.option.name()] = await variable.value.get()
-            families[await family.option.name()] = CreoleExtra(variables)
-        return CreoleExtra(families)
-
     def patch_template(self,
-                       filename: str):
+                       filename: str,
+                       tmp_dir: str,
+                       patch_dir: str,
+                       ) -> None:
         """Apply patch to a template
         """
-        patch_cmd = ['patch', '-d', self.tmp_dir, '-N', '-p1']
+        patch_cmd = ['patch', '-d', tmp_dir, '-N', '-p1']
         patch_no_debug = ['-s', '-r', '-', '--backup-if-mismatch']
 
-        # patches variante + locaux
+        patch_file = join(patch_dir, f'{filename}.patch')
-        for directory in [join(patch_dir, 'variante'), patch_dir]:
-            patch_file = join(directory, f'{filename}.patch')
         if isfile(patch_file):
             log.info(_("Patching template '{filename}' with '{patch_file}'"))
-                ret = call(patch_cmd + patch_no_debug + ['-i', patch_file])
+            rel_patch_file = relpath(patch_file, tmp_dir)
-                if ret:
+            ret = call(patch_cmd + patch_no_debug + ['-i', rel_patch_file])
-                    patch_cmd_err = ' '.join(patch_cmd + ['-i', patch_file])
+            if ret:  # pragma: no cover
-                    log.error(_(f"Error applying patch: '{patch_file}'\nTo reproduce and fix this error {patch_cmd_err}"))
+                patch_cmd_err = ' '.join(patch_cmd + ['-i', rel_patch_file])
-                    copy(filename, self.tmp_dir)
+                msg = _(f"Error applying patch: '{rel_patch_file}'\n"
+                        f"To reproduce and fix this error {patch_cmd_err}")
+                log.error(_(msg))
+                copy(join(self.distrib_dir, filename), tmp_dir)
 
     def prepare_template(self,
-                         filename: str):
+                         filename: str,
+                         tmp_dir: str,
+                         patch_dir: str,
+                         ) -> None:
         """Prepare template source file
         """
-        log.info(_("Copy template: '{filename}' -> '{self.tmp_dir}'"))
+        log.info(_("Copy template: '{filename}' -> '{tmp_dir}'"))
-        copy(filename, self.tmp_dir)
+        copy(filename, tmp_dir)
-        self.patch_template(filename)
+        self.patch_template(filename, tmp_dir, patch_dir)
 
     def process(self,
                 source: str,
                 true_destfilename: str,
                 destfilename: str,
-                filevar: Dict,
+                variable: Any,
-                variable: Any):
+                ):
         """Process a cheetah template
         """
         # full path of the destination file
         log.info(_(f"Cheetah processing: '{destfilename}'"))
         try:
+            extra_context = {'normalize_family': normalize_family,
+                             'rougail_filename': true_destfilename
+                             }
+            if variable:
+                extra_context['rougail_variable'] = variable
             cheetah_template = CheetahTemplate(source,
                                                self.rougail_variables_dict,
                                                self.eosfunc,
-                                               true_destfilename,
+                                               extra_context,
-                                               variable,
                                                )
             data = str(cheetah_template)
-        except CheetahNotFound as err:
+        except CheetahNotFound as err:  # pragma: no cover
             varname = err.args[0][13:-1]
-            raise TemplateError(_(f"Error: unknown variable used in template {destfilename} : {varname}"))
+            msg = f"Error: unknown variable used in template {source} to {destfilename}: {varname}"
-        except Exception as err:
+            raise TemplateError(_(msg)) from err
-            raise TemplateError(_(f"Error while instantiating template {destfilename}: {err}"))
+        except Exception as err:  # pragma: no cover
+            msg = _(f"Error while instantiating template {source} to {destfilename}: {err}")
+            raise TemplateError(msg) from err
 
         with open(destfilename, 'w') as file_h:
             file_h.write(data)
 
     def instance_file(self,
                       filevar: Dict,
-                      service_name: str) -> None:
+                      tmp_dir: str,
+                      dest_dir: str,
+                      ) -> None:
         """Run templatisation on one file
         """
         log.info(_("Instantiating file '{filename}'"))
@@ -355,51 +325,105 @@ class CreoleTemplateEngine:
             if variable:
                 variable = [variable]
         for idx, filename in enumerate(filenames):
-            destfilename = join(self.dest_dir, filename[1:])
+            destfilename = join(dest_dir, filename[1:])
             makedirs(dirname(destfilename), exist_ok=True)
             if variable:
                 var = variable[idx]
             else:
                 var = None
-            source = join(self.tmp_dir, filevar['source'])
+            source = join(tmp_dir, filevar['source'])
             if filevar['templating']:
                 self.process(source,
                              filename,
                              destfilename,
-                             filevar,
+                             var,
-                             var)
+                             )
             else:
                 copy(source, destfilename)
 
     async def instance_files(self) -> None:
         """Run templatisation on all files
         """
+        ori_dir = getcwd()
+        tmp_dir = relpath(self.tmp_dir, self.distrib_dir)
+        dest_dir = relpath(self.dest_dir, self.distrib_dir)
+        patch_dir = relpath(Config['patch_dir'], self.distrib_dir)
+        chdir(self.distrib_dir)
         for option in await self.config.option.list(type='all'):
             namespace = await option.option.name()
-            if namespace == variable_namespace:
+            if namespace == Config['variable_namespace']:
-                await self.load_eole_variables_rougail(option)
+                await self.load_variables_namespace(option)
             else:
-                families = await self.load_eole_variables(namespace,
+                self.rougail_variables_dict[namespace] = await self.load_variables_extra(option)
-                                                          option)
+        for template in listdir('.'):
-                self.rougail_variables_dict[namespace] = families
+            self.prepare_template(template, tmp_dir, patch_dir)
-        for template in listdir(self.distrib_dir):
-            self.prepare_template(join(self.distrib_dir, template))
         for service_obj in await self.config.option('services').list('all'):
-            service_name = await service_obj.option.doc()
             for fills in await service_obj.list('all'):
                 if await fills.option.name() in ['files', 'overrides']:
                     for fill_obj in await fills.list('all'):
                         fill = await fill_obj.value.dict()
                         filename = fill['source']
-                        distib_file = join(self.distrib_dir, filename)
+                        if not isfile(filename):  # pragma: no cover
-                        if not isfile(distib_file):
+                            raise FileNotFound(_(f"File {filename} does not exist."))
-                            raise FileNotFound(_(f"File {distib_file} does not exist."))
+                        if fill['activate']:
-                        if fill.get('activate', False):
                             self.instance_file(fill,
-                                               service_name,
+                                               tmp_dir,
+                                               dest_dir,
                                                )
                         else:
                             log.debug(_("Instantiation of file '{filename}' disabled"))
+        chdir(ori_dir)
+
+    async def load_variables_namespace (self,
+                                        optiondescription,
+                                        ):
+        """load variables from the "variable namespace
+        """
+        for option in await optiondescription.list('all'):
+            if await option.option.isoptiondescription():
+                if await option.option.isleadership():
+                    for idx, suboption in enumerate(await option.list('all')):
+                        if idx == 0:
+                            leader = CreoleLeader(await suboption.value.get())
+                            self.rougail_variables_dict[await suboption.option.name()] = leader
+                        else:
+                            await leader.add_follower(self.config,
+                                                      await suboption.option.name(),
+                                                      await suboption.option.path(),
+                                                      )
+                else:
+                    await self.load_variables_namespace(option)
+            else:
+                self.rougail_variables_dict[await option.option.name()] = await option.value.get()
+
+    async def load_variables_extra(self,
+                                   optiondescription,
+                                   ) -> CreoleExtra:
+        """Load all variables and set it in CreoleExtra objects
+        """
+        families = {}
+        for family in await optiondescription.list('all'):
+            variables = {}
+            for variable in await family.list('all'):
+                if await variable.option.isoptiondescription():
+                    if await variable.option.isleadership():
+                        for idx, suboption in enumerate(await variable.list('all')):
+                            if idx == 0:
+                                leader = CreoleLeader(await suboption.value.get())
+                                leader_name = await suboption.option.name()
+                            else:
+                                await leader.add_follower(self.config,
+                                                          await suboption.option.name(),
+                                                          await suboption.option.path(),
+                                                          )
+                        variables[leader_name] = leader
+                    else:
+                        subfamilies = await self.load_variables_extra(variable)
+                        variables[await variable.option.name()] = subfamilies
+                else:
+                    variables[await variable.option.name()] = await variable.value.get()
+            families[await family.option.name()] = CreoleExtra(variables)
+        return CreoleExtra(families)
 
 
 async def generate(config: Config,
@@ -408,6 +432,8 @@ async def generate(config: Config,
                    tmp_dir: str,
                    dest_dir: str,
                    ) -> None:
+    """Generate all files
+    """
     engine = CreoleTemplateEngine(config,
                                   eosfunc_file,
                                   distrib_dir,

src/rougail/tiramisu.py

@@ -1,13 +1,41 @@
+"""Redefine Tiramisu object
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
 try:
-    from tiramisu import DynOptionDescription
-except:
     from tiramisu3 import DynOptionDescription
+except ModuleNotFoundError:
+    from tiramisu import DynOptionDescription
 from .utils import normalize_family
 
 
 class ConvertDynOptionDescription(DynOptionDescription):
+    """Suffix could be an integer, we should convert it in str
+    Suffix could also contain invalid character, so we should "normalize" it
+    """
     def convert_suffix_to_path(self, suffix):
         if not isinstance(suffix, str):
             suffix = str(suffix)
-        return normalize_family(suffix,
+        return normalize_family(suffix)
-                                check_name=False)

src/rougail/tiramisureflector.py

@@ -1,506 +1,382 @@
 """loader
 flattened XML specific
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 """
-from os.path import isfile
+from json import dumps
-from lxml.etree import DTD
 
-from .config import dtdfilename, variable_namespace
+from .config import Config
-from .i18n import _
+from .annotator import CONVERT_OPTION
-from .error import LoaderError
+from .objspace import RootRougailObject
-from .annotator import ERASED_ATTRIBUTES
 
 
-FUNC_TO_DICT = ['valid_not_equal']
+class Root():  # pylint: disable=R0903
-FORCE_INFORMATIONS = ['help', 'test', 'separator', 'manage']
+    """Root classes
-ATTRIBUTES_ORDER = ('name', 'doc', 'default', 'multi')
+    """
+    path = '.'
 
 
-CONVERT_OPTION = {'number': dict(opttype="IntOption", func=int),
+class BaseElt:  # pylint: disable=R0903
-                  'choice': dict(opttype="ChoiceOption"),
+    """Base element
-                  'string': dict(opttype="StrOption"),
+    """
-                  'password': dict(opttype="PasswordOption"),
+    name = 'baseoption'
-                  'mail': dict(opttype="EmailOption"),
+    doc = 'baseoption'
-                  'boolean': dict(opttype="BoolOption"),
+    path = '.'
-                  'symlink': dict(opttype="SymLinkOption"),
-                  'filename': dict(opttype="FilenameOption"),
-                  'date': dict(opttype="DateOption"),
-                  'unix_user': dict(opttype="UsernameOption"),
-                  'ip': dict(opttype="IPOption", initkwargs={'allow_reserved': True}),
-                  'local_ip': dict(opttype="IPOption", initkwargs={'private_only': True, 'warnings_only': True}),
-                  'netmask': dict(opttype="NetmaskOption"),
-                  'network': dict(opttype="NetworkOption"),
-                  'broadcast': dict(opttype="BroadcastOption"),
-                  'netbios': dict(opttype="DomainnameOption", initkwargs={'type': 'netbios', 'warnings_only': True}),
-                  'domain': dict(opttype="DomainnameOption", initkwargs={'type': 'domainname', 'allow_ip': False}),
-                  'hostname': dict(opttype="DomainnameOption", initkwargs={'type': 'hostname', 'allow_ip': False}),
-                  'web_address': dict(opttype="URLOption", initkwargs={'allow_ip': True, 'allow_without_dot': True}),
-                  'port': dict(opttype="PortOption", initkwargs={'allow_private': True}),
-                  'mac': dict(opttype="MACOption"),
-                  'cidr': dict(opttype="IPOption", initkwargs={'cidr': True}),
-                  'network_cidr': dict(opttype="NetworkOption", initkwargs={'cidr': True}),
-                  }
 
 
 class TiramisuReflector:
+    """Convert object to tiramisu representation
+    """
     def __init__(self,
-                 xmlroot,
+                 objectspace,
                  funcs_path,
                  ):
-        self.storage = ElementStorage()
+        self.index = 0
-        self.storage.text = ["try:",
+        self.text = ["from importlib.machinery import SourceFileLoader",
-                             "    from tiramisu import *",
+                     "from importlib.util import spec_from_loader, module_from_spec",
-                             "except:",
+                     f"loader = SourceFileLoader('func', '{funcs_path}')",
+                     "spec = spec_from_loader(loader.name, loader)",
+                     "func = module_from_spec(spec)",
+                     "loader.exec_module(func)",
+                     "for key, value in dict(locals()).items():",
+                     "    if key != ['SourceFileLoader', 'func']:",
+                     "        setattr(func, key, value)",
+                     "try:",
                      "    from tiramisu3 import *",
+                     "except:",
+                     "    from tiramisu import *",
                      "from rougail.tiramisu import ConvertDynOptionDescription",
-                             "import imp",
-                             f"func = imp.load_source('func', '{funcs_path}')",
                      ]
-        self.make_tiramisu_objects(xmlroot)
+        self.objectspace = objectspace
+        self.make_tiramisu_objects()
+
+    def make_tiramisu_objects(self) -> None:
+        """make tiramisu objects
+        """
+        baseelt = BaseElt()
+        self.set_name(baseelt)
+        basefamily = Family(baseelt,
+                            self.text,
+                            self.objectspace,
+                            )
+        for elt in self.reorder_family():
+            self.populate_family(basefamily,
+                                 elt,
+                                 )
         # parse object
-        self.storage.get('.').get()
+        baseelt.reflector_object.get()  # pylint: disable=E1101
 
-    def make_tiramisu_objects(self,
+    def reorder_family(self):
-                              xmlroot,
+        """variable_namespace family has to be loaded before any other family
-                              ):
+        because `extra` family could use `variable_namespace` variables.
-        family = self.get_root_family()
+        """
-        for xmlelt in self.reorder_family(xmlroot):
+        if hasattr(self.objectspace.space, 'variables'):
-            self.iter_family(xmlelt,
+            if Config['variable_namespace'] in self.objectspace.space.variables:
-                             family,
+                yield self.objectspace.space.variables[Config['variable_namespace']]
-                             None,
+            for elt, value in self.objectspace.space.variables.items():
-                             )
+                if elt != Config['variable_namespace']:
-
-    def get_root_family(self):
-        family = Family(BaseElt('baseoption',
-                                'baseoption',
-                                ),
-                        self.storage,
-                        False,
-                        '.',
-                        )
-        return family
-
-    def reorder_family(self, xmlroot):
-        # variable_namespace family has to be loaded before any other family
-        # because `extra` family could use `variable_namespace` variables.
-        if hasattr(xmlroot, 'variables'):
-            if variable_namespace in xmlroot.variables:
-                yield xmlroot.variables[variable_namespace]
-            for xmlelt, value in xmlroot.variables.items():
-                if xmlelt != variable_namespace:
                     yield value
-        if hasattr(xmlroot, 'services'):
+        if hasattr(self.objectspace.space, 'services'):
-            yield xmlroot.services
+            yield self.objectspace.space.services
-
-    def get_attributes(self, space):  # pylint: disable=R0201
-        for attr in dir(space):
-            if not attr.startswith('_') and attr not in ERASED_ATTRIBUTES:
-                yield attr
-
-    def get_children(self,
-                     space,
-                     ):
-        for tag in self.get_attributes(space):
-            children = getattr(space, tag)
-            if children.__class__.__name__ == 'Family':
-                children = [children]
-            if isinstance(children, dict):
-                children = list(children.values())
-            if isinstance(children, list):
-                yield tag, children
-
-    def iter_family(self,
-                    child,
-                    family,
-                    subpath,
-                    ):
-        tag = child.__class__.__name__
-        if tag == 'Variable':
-            function = self.populate_variable
-        elif tag == 'Property':
-            # property already imported with family
-            return
-        else:
-            function = self.populate_family
-        #else:
-        #    raise Exception('unknown tag {}'.format(child.tag))
-        function(family,
-                 child,
-                 subpath,
-                 )
 
     def populate_family(self,
                         parent_family,
                         elt,
-                        subpath,
                         ):
-        path = self.build_path(subpath,
+        """Populate family
-                               elt,
+        """
-                               )
+        self.set_name(elt)
-        tag = elt.__class__.__name__
         family = Family(elt,
-                        self.storage,
+                        self.text,
-                        tag == 'Leadership',
+                        self.objectspace,
-                        path,
                         )
         parent_family.add(family)
-        for tag, children in self.get_children(elt):
+        for children in vars(elt).values():
+            if isinstance(children, self.objectspace.family):
+                self.populate_family(family,
+                                     children,
+                                     )
+                continue
+            if isinstance(children, dict):
+                children = list(children.values())
+            if isinstance(children, list):
                 for child in children:
-                self.iter_family(child,
+                    if isinstance(child, self.objectspace.property_) or \
-                                 family,
+                            not isinstance(child, RootRougailObject):
-                                 path,
+                        continue
-                                 )
+                    if isinstance(child, self.objectspace.variable):
-
+                        self.set_name(child)
-    def populate_variable(self,
+                        family.add(Variable(child,
-                          family,
+                                            self.text,
-                          elt,
+                                            self.objectspace,
-                          subpath,
-                          ):
-        is_follower = False
-        is_leader = False
-        if family.is_leader:
-            if elt.name != family.elt.name:
-                is_follower = True
-            else:
-                is_leader = True
-        family.add(Variable(elt,
-                            self.storage,
-                            is_follower,
-                            is_leader,
-                            self.build_path(subpath,
-                                            elt,
-                                            )
                                             ))
+                    else:
+                        self.populate_family(family,
+                                             child,
+                                             )
 
-    def build_path(self,
+    def set_name(self,
-                   subpath,
                  elt,
                  ):
-        if subpath is None:
+        """Set name
-            return elt.name
+        """
-        return subpath + '.' + elt.name
+        elt.reflector_name = f'option_{self.index}'
-
-    def get_text(self):
-        return '\n'.join(self.storage.get('.').get_text())
-
-
-class BaseElt:
-    def __init__(self,
-                 name,
-                 doc,
-                 ):
-        self.name = name
-        self.doc = doc
-
-    def __iter__(self):
-        return iter([])
-
-
-class ElementStorage:
-    def __init__(self,
-                 ):
-        self.paths = {}
-        self.text = []
-        self.index = 0
-
-    def add(self, path, elt):
-        self.paths[path] = (elt, self.index)
         self.index += 1
 
-    def get(self, path):
+    def get_text(self):
-        if path not in self.paths or self.paths[path][0] is None:
+        """Get text
-            raise LoaderError(_('there is no element for path {}').format(path))
+        """
-        return self.paths[path][0]
+        return '\n'.join(self.text)
-
-    def get_name(self, path):
-        if path not in self.paths:
-            raise LoaderError(_('there is no element for index {}').format(path))
-        return f'option_{self.paths[path][1]}'
 
 
 class Common:
+    """Common function for variable and family
+    """
     def __init__(self,
                  elt,
-                 storage,
+                 text,
-                 is_leader,
+                 objectspace,
-                 path,
                  ):
-        self.option_name = None
-        self.path = path
-        self.attrib = {}
-        self.informations = {}
-        self.storage = storage
-        self.is_leader = is_leader
-        self.storage.add(self.path, self)
-
-    def populate_properties(self, child):
-        if child.type == 'calculation':
-            action = f"ParamValue('{child.name}')"
-            option_name = self.storage.get(child.source).get()
-            kwargs = f"'condition': ParamOption({option_name}, todict=True), 'expected': ParamValue('{child.expected}')"
-            if child.inverse:
-                kwargs += ", 'reverse_condition': ParamValue(True)"
-            prop = 'Calculation(calc_value, Params(' + action + ', kwargs={' + kwargs + '}))'
-        else:
-            prop = "'" + child.text + "'"
-        if self.attrib['properties']:
-            self.attrib['properties'] += ', '
-        self.attrib['properties'] += prop
-
-    def get_attrib(self, attrib):
-        ret_list = []
-        for key, value in self.attrib.items():
-            if value is None:
-                continue
-            if key == 'properties':
-                if not self.attrib[key]:
-                    continue
-                value = "frozenset({" + self.attrib[key] + "})"
-            elif key in ['default', 'multi', 'suffixes', 'validators']:
-                value = self.attrib[key]
-            elif isinstance(value, str) and key != 'opt' and not value.startswith('['):
-                value = "'" + value.replace("'", "\\\'") + "'"
-            ret_list.append(f'{key}={value}')
-        return ', '.join(ret_list)
-
-    def populate_informations(self):
-        for key, value in self.informations.items():
-            if isinstance(value, str):
-                value = '"' + value.replace('"', '\"') + '"'
-            self.storage.text.append(f'{self.option_name}.impl_set_information("{key}", {value})')
-
-    def get_text(self,
-                 ):
-        return self.storage.text
-
-    def get_attributes(self, space):  # pylint: disable=R0201
-        attributes = dir(space)
-        for attr in ATTRIBUTES_ORDER:
-            if attr in attributes:
-                yield attr
-        for attr in dir(space):
-            if attr not in ATTRIBUTES_ORDER:
-                if not attr.startswith('_') and attr not in ERASED_ATTRIBUTES:
-                    value = getattr(space, attr)
-                    if not isinstance(value, (list, dict)) and not value.__class__.__name__ == 'Family':
-                        yield attr
-
-    def get_children(self,
-                     space,
-                     ):
-        for attr in dir(space):
-            if not attr.startswith('_') and attr not in ERASED_ATTRIBUTES:
-                value = getattr(space, attr)
-                if isinstance(value, (list, dict)):
-                    children = getattr(space, attr)
-                    if children.__class__.__name__ == 'Family':
-                        children = [children]
-                    if isinstance(children, dict):
-                        children = list(children.values())
-                    if children and isinstance(children, list):
-                        yield attr, children
-
-
-class Variable(Common):
-    def __init__(self,
-                 elt,
-                 storage,
-                 is_follower,
-                 is_leader,
-                 path,
-                 ):
-        super().__init__(elt,
-                         storage,
-                         is_leader,
-                         path,
-                         )
-        self.is_follower = is_follower
-        convert_option = CONVERT_OPTION[elt.type]
-        del elt.type
-        self.object_type = convert_option['opttype']
-        self.attrib.update(convert_option.get('initkwargs', {}))
-        if self.object_type != 'SymLinkOption':
-            self.attrib['properties'] = []
-            self.attrib['validators'] = []
         self.elt = elt
+        self.option_name = None
+        self.text = text
+        self.objectspace = objectspace
+        self.elt.reflector_object = self
+        self.object_type = None
 
     def get(self):
+        """Get tiramisu's object
+        """
         if self.option_name is None:
+            self.option_name = self.elt.reflector_name
             self.populate_attrib()
-            if self.object_type == 'SymLinkOption':
-                self.attrib['opt'] = self.storage.get(self.attrib['opt']).get()
-            else:
-                self.parse_children()
-            attrib = self.get_attrib(self.attrib)
-            self.option_name = self.storage.get_name(self.path)
-            self.storage.text.append(f'{self.option_name} = {self.object_type}({attrib})')
             self.populate_informations()
         return self.option_name
 
     def populate_attrib(self):
-        for key in self.get_attributes(self.elt):
+        """Populate attributes
+        """
+        keys = {'name': self.convert_str(self.elt.name)}
+        if hasattr(self.elt, 'doc'):
+            keys['doc'] = self.convert_str(self.elt.doc)
+        self._populate_attrib(keys)
+        if hasattr(self.elt, 'properties'):
+            keys['properties'] = self.properties_to_string(self.elt.properties)
+        attrib = ', '.join([f'{key}={value}' for key, value in keys.items()])
+        self.text.append(f'{self.option_name} = {self.object_type}({attrib})')
+
+    def _populate_attrib(self,
+                         keys: dict,
+                         ) -> None:  # pragma: no cover
+        raise NotImplementedError()
+
+    @staticmethod
+    def convert_str(value):
+        """convert string
+        """
+        return dumps(value, ensure_ascii=False)
+
+    def properties_to_string(self,
+                             values: list,
+                             ) -> None:
+        """Change properties to string
+        """
+        properties = [self.convert_str(property_) for property_ in values
+                      if isinstance(property_, str)]
+        calc_properties = [self.calc_properties(property_) for property_ in values \
+                if isinstance(property_, self.objectspace.property_)]
+        return 'frozenset({' + ', '.join(sorted(properties) + calc_properties) + '})'
+
+    def calc_properties(self,
+                        child,
+                        ) -> str:
+        """Populate properties
+        """
+        option_name = child.source.reflector_object.get()
+        kwargs = f"'condition': ParamOption({option_name}, todict=True), "
+        if child.expected is None or isinstance(child.expected, int):
+            kwargs += f"'expected': ParamValue({child.expected})"
+        elif isinstance(child.expected, str):
+            val = self.convert_str(child.expected)
+            kwargs += f"'expected': ParamValue({val})"
+        else:
+            val = child.expected.reflector_object.get()
+            kwargs += f"'expected': ParamOption({val})"
+        if child.inverse:
+            kwargs += ", 'reverse_condition': ParamValue(True)"
+        return (f"Calculation(func.calc_value, Params(ParamValue('{child.name}'), "
+                f"kwargs={{{kwargs}}}))")
+
+    def populate_informations(self):
+        """Populate Tiramisu's informations
+        """
+        if not hasattr(self.elt, 'information'):
+            return
+        for key, value in vars(self.elt.information).items():
+            if key == 'xmlfiles':
+                continue
+            if isinstance(value, str):
+                value = self.convert_str(value)
+            self.text.append(f"{self.option_name}.impl_set_information('{key}', {value})")
+
+
+class Variable(Common):
+    """Manage variable
+    """
+    def __init__(self,
+                 elt,
+                 text,
+                 objectspace,
+                 ):
+        super().__init__(elt, text, objectspace)
+        self.object_type = CONVERT_OPTION[elt.type]['opttype']
+
+    def _populate_attrib(self,
+                         keys: dict,
+                         ):
+        if hasattr(self.elt, 'opt'):
+            keys['opt'] = self.elt.opt.reflector_object.get()
+        if hasattr(self.elt, 'values'):
+            values = self.elt.values
+            if values[0].type == 'calculation':
+                value = values[0].name.reflector_object.get()
+                keys['values'] = f"Calculation(func.calc_value, Params((ParamOption({value}))))"
+            else:
+                keys['values'] = str(tuple([val.name for val in values]))
+        if hasattr(self.elt, 'multi') and self.elt.multi:
+            keys['multi'] = self.elt.multi
+        for key in ['default', 'default_multi']:
+            if hasattr(self.elt, key) and getattr(self.elt, key) is not None:
                 value = getattr(self.elt, key)
-            if key in FORCE_INFORMATIONS:
+                if isinstance(value, str):
-                if key == 'test':
+                    value = self.convert_str(value)
-                    value = value.split(',')
+                elif isinstance(value, self.objectspace.value):
-                    if self.object_type == 'IntOption':
+                    value = self.calculation_value(value, [])
-                        value = [int(v) for v in value]
+                keys[key] = value
-                self.informations[key] = value
+        if hasattr(self.elt, 'validators'):
-            else:
+            keys['validators'] = '[' + ', '.join([self.calculation_value(val,
-                self.attrib[key] = value
+                                    ['ParamSelfOption()']) for val in self.elt.validators]) + ']'
+        for key in ['min_number', 'max_number']:
+            if hasattr(self.elt, key):
+                keys[key] = getattr(self.elt, key)
+        for key, value in CONVERT_OPTION[self.elt.type].get('initkwargs', {}).items():
+            if isinstance(value, str):
+                value = f"'{value}'"
+            keys[key] = value
 
-    def parse_children(self):
+    def calculation_value(self,
-        if not 'default' in self.attrib or self.attrib['multi']:
+                          child,
-            self.attrib['default'] = []
+                          args,
-        if self.attrib['multi'] == 'submulti' and self.is_follower:
+                          ) -> str:
-            self.attrib['default_multi'] = []
+        """Generate calculated value
-        choices = []
+        """
-        if 'properties' in self.attrib:
-            if self.attrib['properties']:
-                self.attrib['properties'] = "'" + "', '".join(sorted(list(self.attrib['properties']))) + "'"
-            else:
-                self.attrib['properties'] = ''
-        for tag, children in self.get_children(self.elt):
-            for child in children:
-                if tag == 'property':
-                    self.populate_properties(child)
-                elif tag == 'value':
-                    if child.type == 'calculation':
-                        self.attrib['default'] = self.calculation_value(child, [])
-                    else:
-                        self.populate_value(child)
-                elif tag == 'check':
-                    self.attrib['validators'].append(self.calculation_value(child, ['ParamSelfOption()']))
-                elif tag == 'choice':
-                    choices.append(child.name)
-        if choices:
-            self.attrib['values'] = tuple(choices)
-        if self.attrib['default'] == []:
-            del self.attrib['default']
-        elif not self.attrib['multi'] and isinstance(self.attrib['default'], list):
-            self.attrib['default'] = self.attrib['default'][-1]
-        if self.attrib['validators'] == []:
-            del self.attrib['validators']
-        else:
-            self.attrib['validators'] = '[' + ', '.join(self.attrib['validators']) + ']'
-
-    def calculation_value(self, child, args):
         kwargs = []
-        if hasattr(child, 'name'):
         # has parameters
         function = child.name
         if hasattr(child, 'param'):
             for param in child.param:
-                    value = self.populate_param(param)
+                value = self.populate_param(function, param)
                 if not hasattr(param, 'name'):
                     args.append(str(value))
                 else:
                     kwargs.append(f"'{param.name}': " + value)
-        else:
+        ret = f'Calculation(func.{function}, Params((' + ', '.join(args) + ')'
-            # function without any parameter
+        if kwargs:
-            function = child.text.strip()
+            ret += ', kwargs={' + ', '.join(kwargs) + '}'
-        ret = f"Calculation(func.{function}, Params((" + ', '.join(args) + "), kwargs=" + "{" + ', '.join(kwargs) + "})"
+        ret += ')'
         if hasattr(child, 'warnings_only'):
             ret += f', warnings_only={child.warnings_only}'
         return ret + ')'
 
     def populate_param(self,
+                       function: str,
                        param,
                        ):
+        """Populate variable parameters
+        """
         if param.type == 'string':
-            return f'ParamValue("{param.text}")'
+            value = param.text
-        elif param.type == 'number':
+            if value is not None:
+                value = self.convert_str(value)
+            return f"ParamValue({value})"
+        if param.type in ['number', 'boolean']:
             return f'ParamValue({param.text})'
-        elif param.type == 'variable':
+        if param.type == 'nil':
-            value = {'option': param.text,
+            return 'ParamValue(None)'
-                     'notraisepropertyerror': param.notraisepropertyerror,
+        if param.type == 'variable':
-                     'todict': param.text in FUNC_TO_DICT,
+            return self.build_param(param, function)
-                     }
+        if param.type == 'information':
+            return f'ParamInformation("{param.text}", None)'
+        if param.type == 'suffix':
+            return 'ParamSuffix()'
+        raise Exception(f'unknown type {param.type}')  # pragma: no cover
+
+    @staticmethod
+    def build_param(param,
+                    function: str,  # pylint: disable=W0613
+                    ) -> str:
+        """build variable parameters
+        """
+        option_name = param.text.reflector_object.get()
+        params = [f'{option_name}']
         if hasattr(param, 'suffix'):
-                value['suffix'] = param.suffix
+            param_type = 'ParamDynOption'
-            return self.build_param(value)
+            params.extend([f"'{param.suffix}'", f'{param.family.reflector_name}'])
-        raise LoaderError(_('unknown param type {}').format(param.type))
-
-    def populate_value(self,
-                       child,
-                       ):
-        value = child.name
-        if self.attrib['multi'] == 'submulti':
-            self.attrib['default_multi'].append(value)
-        elif self.is_follower:
-            self.attrib['default_multi'] = value
-        elif self.attrib['multi']:
-            self.attrib['default'].append(value)
-            if not self.is_leader:
-                self.attrib['default_multi'] = value
-        elif isinstance(value, int):
-            self.attrib['default'].append(value)
         else:
-            self.attrib['default'].append("'" + value + "'")
+            param_type = 'ParamOption'
-
+        if not param.propertyerror:
-    def build_param(self,
+            params.append('notraisepropertyerror=True')
-                    param,
+        return "{}({})".format(param_type, ', '.join(params))
-                    ):
-        option_name = self.storage.get(param['option']).get()
-        if 'suffix' in param:
-            family = '.'.join(param['option'].split('.')[:-1])
-            family_option = self.storage.get_name(family)
-            return f"ParamDynOption({option_name}, '{param['suffix']}', {family_option}, notraisepropertyerror={param['notraisepropertyerror']}, todict={param['todict']})"
-        return f"ParamOption({option_name}, notraisepropertyerror={param['notraisepropertyerror']}, todict={param['todict']})"
 
 
 class Family(Common):
+    """Manage family
+    """
     def __init__(self,
                  elt,
-                 storage,
+                 text,
-                 is_leader,
+                 objectspace,
-                 path,
                  ):
-        super().__init__(elt,
+        super().__init__(elt, text, objectspace)
-                         storage,
+        if hasattr(self.elt, 'suffixes'):
-                         is_leader,
+            self.object_type = 'ConvertDynOptionDescription'
-                         path,
+        elif isinstance(self.elt, self.objectspace.leadership):
-                         )
+            self.object_type = 'Leadership'
+        else:
+            self.object_type = 'OptionDescription'
         self.children = []
-        self.elt = elt
 
     def add(self, child):
+        """Add a child
+        """
         self.children.append(child)
 
-    def get(self):
+    def _populate_attrib(self,
-        if not self.option_name:
+                         keys: list,
-            self.populate_attrib()
+                         ) -> None:
-            self.parse_children()
+        if hasattr(self.elt, 'suffixes'):
-            self.option_name = self.storage.get_name(self.path)
+            dyn = self.elt.suffixes.reflector_object.get()
-            object_name = self.get_object_name()
+            keys['suffixes'] = f"Calculation(func.calc_value, Params((ParamOption({dyn}))))"
-            attrib = self.get_attrib(self.attrib) + ', children=[' + ', '.join([child.get() for child in self.children]) + ']'
+        keys['children'] = '[' + ', '.join([child.get() for child in self.children]) + ']'
-            self.storage.text.append(f'{self.option_name} = {object_name}({attrib})')
-            self.populate_informations()
-        return self.option_name
-
-    def populate_attrib(self):
-        for key in self.get_attributes(self.elt):
-            value = getattr(self.elt, key)
-            if key in FORCE_INFORMATIONS:
-                self.informations[key] = value
-            elif key == 'dynamic':
-                dynamic = self.storage.get(value).get()
-                self.attrib['suffixes'] = f"Calculation(func.calc_value, Params((ParamOption({dynamic}))))"
-            else:
-                self.attrib[key] = value
-
-    def parse_children(self):
-        if 'properties' in self.attrib:
-            self.attrib['properties'] = "'" + "', '".join(sorted(list(self.attrib['properties']))) + "'"
-        if hasattr(self.elt, 'property'):
-            #self.attrib['properties'] = ''
-            for child in self.elt.property:
-                self.populate_properties(child)
-            if not self.attrib['properties']:
-                del self.attrib['properties']
-
-    def get_object_name(self):
-        if 'suffixes' in self.attrib:
-            return 'ConvertDynOptionDescription'
-        elif not self.is_leader:
-            return 'OptionDescription'
-        return 'Leadership'

src/rougail/utils.py

@@ -1,23 +1,50 @@
+"""Rougail's tools
+
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 """
-utilitaires créole
+from typing import List
-"""
+from unicodedata import normalize, combining
-import unicodedata
+
-from .i18n import _
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
 
 
-def normalize_family(family_name: str,
+def normalize_family(family_name: str) -> str:
-                     check_name: bool=True,
-                     allow_dot: bool=False) -> str:
     """replace space, accent, uppercase, ... by valid character
     """
-    family_name = family_name.replace('-', '_')
+    family_name = family_name.replace('-', '_').replace(' ', '_').replace('.', '_')
-    if not allow_dot:
+    nfkd_form = normalize('NFKD', family_name)
-        family_name = family_name.replace('.', '_')
+    family_name = ''.join([c for c in nfkd_form if not combining(c)])
-    family_name = family_name.replace(' ', '_')
+    return family_name.lower()
-    nfkd_form = unicodedata.normalize('NFKD', family_name)
-    family_name = ''.join([c for c in nfkd_form if not unicodedata.combining(c)])
-    family_name = family_name.lower()
-    if check_name and family_name == 'containers':
-        raise ValueError(_(f'"{family_name}" is a forbidden family name'))
-    return family_name
 
+
+def load_modules(eosfunc_file) -> List[str]:
+    """list all functions in eosfunc
+    """
+    loader = SourceFileLoader('eosfunc', eosfunc_file)
+    spec = spec_from_loader(loader.name, loader)
+    eosfunc = module_from_spec(spec)
+    loader.exec_module(eosfunc)
+    return eosfunc

src/rougail/xml_compare.py

@@ -1,161 +0,0 @@
-try:
-    import doctest
-    doctest.OutputChecker
-except (AttributeError, ImportError): # Python < 2.4
-    import util.doctest24 as doctest
-try:
-    import xml.etree.ElementTree as ET
-except ImportError:
-    import elementtree.ElementTree as ET
-from xml.parsers.expat import ExpatError as XMLParseError
-
-RealOutputChecker = doctest.OutputChecker
-
-
-def debug(*msg):
-    import sys
-    print >> sys.stderr, ' '.join(map(str, msg))
-
-
-class HTMLOutputChecker(RealOutputChecker):
-
-    def check_output(self, want, got, optionflags):
-        normal = RealOutputChecker.check_output(self, want, got, optionflags)
-        if normal or not got:
-            return normal
-        try:
-            want_xml = make_xml(want)
-        except XMLParseError:
-            pass
-        else:
-            try:
-                got_xml = make_xml(got)
-            except XMLParseError:
-                pass
-            else:
-                if xml_compare(want_xml, got_xml):
-                    return True
-        return False
-
-    def output_difference(self, example, got, optionflags):
-        actual = RealOutputChecker.output_difference(
-            self, example, got, optionflags)
-        want_xml = got_xml = None
-        try:
-            want_xml = make_xml(example.want)
-            want_norm = make_string(want_xml)
-        except XMLParseError as e:
-            if example.want.startswith('<'):
-                want_norm = '(bad XML: %s)' % e
-                #  '<xml>%s</xml>' % example.want
-            else:
-                return actual
-        try:
-            got_xml = make_xml(got)
-            got_norm = make_string(got_xml)
-        except XMLParseError as e:
-            if example.want.startswith('<'):
-                got_norm = '(bad XML: %s)' % e
-            else:
-                return actual
-        s = '%s\nXML Wanted: %s\nXML Got   : %s\n' % (
-            actual, want_norm, got_norm)
-        if got_xml and want_xml:
-            result = []
-            xml_compare(want_xml, got_xml, result.append)
-            s += 'Difference report:\n%s\n' % '\n'.join(result)
-        return s
-
-
-def xml_sort(children):
-    tcl1 = {}
-    #idx = 0
-
-    for child in children:
-        if 'name' in child.attrib:
-            key = child.attrib['name']
-        else:
-            key = child.tag
-        if key not in tcl1:
-            tcl1[key] = []
-        tcl1[key].append(child)
-    cl1_keys = list(tcl1.keys())
-    cl1_keys.sort()
-    cl1 = []
-    for key in cl1_keys:
-        cl1.extend(tcl1[key])
-    return cl1
-
-def xml_compare(x1, x2):
-    if x1.tag != x2.tag:
-        print ('Tags do not match: %s and %s' % (x1.tag, x2.tag))
-        return False
-    for name, value in x1.attrib.items():
-        if x2.attrib.get(name) != value:
-            print ('Attributes do not match: %s=%r, %s=%r'
-                     % (name, value, name, x2.attrib.get(name)))
-            return False
-    for name in x2.attrib:
-        if name not in x1.attrib:
-            print ('x2 has an attribute x1 is missing: %s'
-                     % name)
-            return False
-    if not text_compare(x1.text, x2.text):
-        print ('text: %r != %r' % (x1.text, x2.text))
-        return False
-    if not text_compare(x1.tail, x2.tail):
-        print ('tail: %r != %r' % (x1.tail, x2.tail))
-        return False
-
-    cl1 = xml_sort(x1.getchildren())
-    cl2 = xml_sort(x2.getchildren())
-
-    if len(cl1) != len(cl2):
-        cl1_tags = []
-        for c in cl1:
-            cl1_tags.append(c.tag)
-        cl2_tags = []
-        for c in cl2:
-            cl2_tags.append(c.tag)
-        print ('children length differs, %i != %i (%s != %s)'
-                 % (len(cl1), len(cl2), cl1_tags, cl2_tags))
-        return False
-    i = 0
-    for c1, c2 in zip(cl1, cl2):
-        i += 1
-        if not xml_compare(c1, c2):
-            if 'name' in c1.attrib:
-                name = c1.attrib['name']
-            else:
-                name = i
-            print ('in tag "%s" with name "%s"'
-                     % (c1.tag, name))
-            return False
-    return True
-
-
-def text_compare(t1, t2):
-    if not t1 and not t2:
-        return True
-    if t1 == '*' or t2 == '*':
-        return True
-    return (t1 or '').strip() == (t2 or '').strip()
-
-
-def make_xml(s):
-    return ET.XML('<xml>%s</xml>' % s)
-
-
-def make_string(xml):
-    if isinstance(xml, (str, unicode)):
-        xml = make_xml(xml)
-    s = ET.tostring(xml)
-    if s == '<xml />':
-        return ''
-    assert s.startswith('<xml>') and s.endswith('</xml>'), repr(s)
-    return s[5:-6]
-
-
-def install():
-    doctest.OutputChecker = HTMLOutputChecker
-

src/rougail/xmlreflector.py

@@ -1,15 +1,40 @@
-# coding: utf-8
+"""load XML file from directory
-from os.path import join, isfile, basename, isdir
-from os import listdir
-#from io import BytesIO
 
-from lxml.etree import DTD, parse, tostring  # , XMLParser
+Created by:
+EOLE (http://eole.orion.education.fr)
+Copyright (C) 2005-2018
+
+Forked by:
+Cadoles (http://www.cadoles.com)
+Copyright (C) 2019-2021
+
+distribued with GPL-2 or later license
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""
+from typing import List
+from os.path import join, isfile
+from os import listdir
+
+from lxml.etree import DTD, parse, XMLSyntaxError  # pylint: disable=E0611
 
 from .i18n import _
 from .error import DictConsistencyError
 
 
-class XMLReflector(object):
+class XMLReflector:
     """Helper class for loading the Creole XML file,
     parsing it, validating against the Creole DTD,
     writing the xml result on the disk
@@ -34,57 +59,25 @@ class XMLReflector(object):
 
         :returns: the root element tree object
         """
-        # document = parse(BytesIO(xmlfile), XMLParser(remove_blank_text=True))
+        try:
             document = parse(xmlfile)
+        except XMLSyntaxError as err:
+            raise DictConsistencyError(_(f'not an XML file: {err}'), 52, xmlfile) from err
         if not self.dtd.validate(document):
-            raise DictConsistencyError(_("not a valid xml file: {}").format(xmlfile))
+            dtd_error = self.dtd.error_log.filter_from_errors()[0]
+            msg = _(f'not a valid XML file: {dtd_error}')
+            raise DictConsistencyError(msg, 43, xmlfile)
         return document.getroot()
 
-    def load_xml_from_folders(self, xmlfolders):
+    @staticmethod
+    def load_xml_from_folders(xmlfolders: List[str]):
         """Loads all the XML files located in the xmlfolders' list
 
         :param xmlfolders: list of full folder's name
         """
-        documents = []
-        if not isinstance(xmlfolders, list):
-            xmlfolders = [xmlfolders]
         for xmlfolder in xmlfolders:
-            if isinstance(xmlfolder, list) or isinstance(xmlfolder, tuple):
+            filenames = [join(xmlfolder, filename) for filename in listdir(xmlfolder) if \
-                # directory group : collect files from each
+                         filename.endswith('.xml')]
-                # directory and sort them before loading
-                group_files = []
-                for idx, subdir in enumerate(xmlfolder):
-                    if isdir(subdir):
-                        for filename in listdir(subdir):
-                            group_files.append((filename, idx, subdir))
-                    else:
-                        group_files.append(basename(subdir), idx, dirname(subdir))
-                def sort_group(file1, file2):
-                    if file1[0] == file2[0]:
-                        # sort by initial xmlfolder order if same name
-                        return file1[1].__cmp__(file2[1])
-                    # sort by filename
-                    elif file1[0] > file2[0]:
-                        return 1
-                    else:
-                        return -1
-                group_files.sort(sort_group)
-                filenames = [join(f[2], f[0]) for f in group_files]
-            elif isdir(xmlfolder):
-                filenames = []
-                for filename in listdir(xmlfolder):
-                    filenames.append(join(xmlfolder, filename))
             filenames.sort()
-            else:
-                filenames = [xmlfolder]
             for xmlfile in filenames:
-                if xmlfile.endswith('.xml'):
+                yield xmlfile
-                    #xmlfile_path = join(xmlfolder, xmlfile)
-                    documents.append((xmlfile, self.parse_xmlfile(xmlfile)))
-        return documents
-
-    def save_xmlfile(self, xmlfilename, xml):  # pylint: disable=R0201
-        """Write a bunch of XML on the disk
-        """
-        with open(xmlfilename, 'w') as xmlfh:
-            xmlfh.write(tostring(xml, pretty_print=True, encoding="UTF-8", xml_declaration=True).decode('utf8'))

tests/dictionaries/00empty/tiramisu/base.py

@@ -0,0 +1,18 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_2 = OptionDescription(name="tata", doc="tata", children=[])
+option_2.impl_set_information('manage', True)
+option_1 = OptionDescription(name="services", doc="services", children=[option_2], properties=frozenset({"hidden"}))
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/00load_autofreeze/00-base.xml

@@ -0,0 +1,15 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables>
+        <family name="général">
+            <variable name="mode_conteneur_actif" type="string" description="No change" auto_freeze="True">
+                <value>non</value>
+            </variable>
+            <variable name="module_instancie" type="string" description="No change">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/00load_autofreeze/tiramisu/base.py

@@ -0,0 +1,19 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_4 = StrOption(name="module_instancie", doc="No change", default="non", properties=frozenset({"mandatory", "normal"}))
+option_3 = StrOption(name="mode_conteneur_actif", doc="No change", default="non", properties=frozenset({"auto_freeze", "basic", "force_store_value", "mandatory", Calculation(func.calc_value, Params(ParamValue('auto_frozen'), kwargs={'condition': ParamOption(option_4, todict=True), 'expected': ParamValue("oui"), 'reverse_condition': ParamValue(True)}))}))
+option_2 = OptionDescription(name="general", doc="général", children=[option_3, option_4], properties=frozenset({"basic"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/00load_autofreezeexpert/00-base.xml

@@ -0,0 +1,15 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables>
+        <family name="général">
+            <variable name="mode_conteneur_actif" type="string" description="No change" auto_freeze="True" mode="expert">
+                <value>non</value>
+            </variable>
+            <variable name="module_instancie" type="string" description="No change">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/00load_autofreezeexpert/tiramisu/base.py

@@ -0,0 +1,19 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_4 = StrOption(name="module_instancie", doc="No change", default="non", properties=frozenset({"mandatory", "normal"}))
+option_3 = StrOption(name="mode_conteneur_actif", doc="No change", default="non", properties=frozenset({"auto_freeze", "expert", "force_store_value", "mandatory", Calculation(func.calc_value, Params(ParamValue('auto_frozen'), kwargs={'condition': ParamOption(option_4, todict=True), 'expected': ParamValue("oui"), 'reverse_condition': ParamValue(True)}))}))
+option_2 = OptionDescription(name="general", doc="général", children=[option_3, option_4], properties=frozenset({"normal"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/00load_autosave/00-base.xml

@@ -0,0 +1,12 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables>
+        <family name="général">
+            <variable name="mode_conteneur_actif" type="string" description="No change" auto_save="True">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/00load_autosave/tiramisu/base.py

@@ -0,0 +1,18 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_3 = StrOption(name="mode_conteneur_actif", doc="No change", default="non", properties=frozenset({"basic", "force_store_value", "mandatory"}))
+option_2 = OptionDescription(name="general", doc="général", children=[option_3], properties=frozenset({"basic"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/00load_autosaveexpert/00-base.xml

@@ -0,0 +1,12 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables>
+        <family name="général">
+            <variable name="mode_conteneur_actif" type="string" description="No change" auto_save="True" mode="expert">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/00load_autosaveexpert/tiramisu/base.py

@@ -0,0 +1,18 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_3 = StrOption(name="mode_conteneur_actif", doc="No change", default="non", properties=frozenset({"expert", "force_store_value", "mandatory"}))
+option_2 = OptionDescription(name="general", doc="général", children=[option_3], properties=frozenset({"expert"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/00load_comment/00-base.xml

@@ -0,0 +1,13 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables>
+        <family name="général">
+            <!-- this is a comment -->
+            <variable name="mode_conteneur_actif" type="string" description="No change" hidden="True">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/00load_comment/tiramisu/base.py

@@ -0,0 +1,18 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_3 = StrOption(name="mode_conteneur_actif", doc="No change", default="non", properties=frozenset({"force_default_on_freeze", "frozen", "hidden", "mandatory", "normal"}))
+option_2 = OptionDescription(name="general", doc="général", children=[option_3], properties=frozenset({"normal"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/00load_notype/00-base.xml

@@ -0,0 +1,15 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables>
+        <family name="général">
+            <variable name="mode_conteneur_actif" type="string" description="No change" hidden="True">
+                <value>non</value>
+            </variable>
+            <variable name="without_type">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/00load_notype/tiramisu/base.py

@@ -0,0 +1,19 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_3 = StrOption(name="mode_conteneur_actif", doc="No change", default="non", properties=frozenset({"force_default_on_freeze", "frozen", "hidden", "mandatory", "normal"}))
+option_4 = StrOption(name="without_type", doc="without_type", default="non", properties=frozenset({"mandatory", "normal"}))
+option_2 = OptionDescription(name="general", doc="général", children=[option_3, option_4], properties=frozenset({"normal"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/00load_save/00-base.xml

@@ -0,0 +1,12 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables>
+        <family name="général">
+            <variable name="mode_conteneur_actif" type="string" description="No change" hidden="True">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/00load_save/tiramisu/base.py

@@ -0,0 +1,18 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_3 = StrOption(name="mode_conteneur_actif", doc="No change", default="non", properties=frozenset({"force_default_on_freeze", "frozen", "hidden", "mandatory", "normal"}))
+option_2 = OptionDescription(name="general", doc="général", children=[option_3], properties=frozenset({"normal"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/00load_subfolder/99-base.xml

@@ -0,0 +1,12 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables>
+        <family name="général">
+            <variable name="mode_conteneur_actif" type="string" description="No change" hidden="True">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/00load_subfolder/subfolder/00-base.xml

@@ -0,0 +1,13 @@
+<?xml version="1.0" encoding="utf-8"?>
+
+<rougail>
+    <variables>
+        <family name='général'>
+            <variable name='mode_conteneur_actif1' type='string' description="No change" hidden="True">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/00load_subfolder/tiramisu/base.py

@@ -0,0 +1,19 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_3 = StrOption(name="mode_conteneur_actif", doc="No change", default="non", properties=frozenset({"force_default_on_freeze", "frozen", "hidden", "mandatory", "normal"}))
+option_4 = StrOption(name="mode_conteneur_actif1", doc="No change", default="non", properties=frozenset({"force_default_on_freeze", "frozen", "hidden", "mandatory", "normal"}))
+option_2 = OptionDescription(name="general", doc="général", children=[option_3, option_4], properties=frozenset({"normal"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/01auto_autofreeze/00-base.xml

@@ -0,0 +1,21 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables>
+        <family name="general">
+            <variable name="module_instancie" type="string" description="No change">
+                <value>non</value>
+            </variable>
+            <variable name="mode_conteneur_actif" type="string" description="No change" auto_freeze="True" hidden="True">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+    <constraints>
+        <fill name="calc_val">
+            <param>oui</param>
+            <target>mode_conteneur_actif</target>
+        </fill>
+    </constraints>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/01auto_autofreeze/makedict/base.json

@@ -0,0 +1 @@
+{"rougail.general.module_instancie": "non", "rougail.general.mode_conteneur_actif": "oui"}

tests/dictionaries/01auto_autofreeze/tiramisu/base.py

@@ -0,0 +1,19 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_3 = StrOption(name="module_instancie", doc="No change", default="non", properties=frozenset({"mandatory", "normal"}))
+option_4 = StrOption(name="mode_conteneur_actif", doc="No change", default=Calculation(func.calc_val, Params((ParamValue("oui")))), properties=frozenset({"auto_freeze", "basic", "force_store_value", "frozen", "hidden", Calculation(func.calc_value, Params(ParamValue('auto_frozen'), kwargs={'condition': ParamOption(option_3, todict=True), 'expected': ParamValue("oui"), 'reverse_condition': ParamValue(True)}))}))
+option_2 = OptionDescription(name="general", doc="general", children=[option_3, option_4], properties=frozenset({"basic"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/01auto_base/00-base.xml

@@ -0,0 +1,22 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables>
+        <family name="general">
+            <variable name="mode_conteneur_actif" type="string" description="No change" hidden="True">
+                <value>non</value>
+            </variable>
+            <variable name="mode_conteneur_actif1" type="string" description="No change">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+
+    <constraints>
+        <fill name="calc_val">
+            <param type="variable">mode_conteneur_actif1</param>
+            <target>mode_conteneur_actif</target>
+        </fill>
+    </constraints>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/01auto_base/tiramisu/base.py

@@ -0,0 +1,19 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_4 = StrOption(name="mode_conteneur_actif1", doc="No change", default="non", properties=frozenset({"mandatory", "normal"}))
+option_3 = StrOption(name="mode_conteneur_actif", doc="No change", default=Calculation(func.calc_val, Params((ParamOption(option_4)))), properties=frozenset({"force_default_on_freeze", "frozen", "hidden", "normal"}))
+option_2 = OptionDescription(name="general", doc="general", children=[option_3, option_4], properties=frozenset({"normal"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/01auto_withoutparam/00-base.xml

@@ -0,0 +1,21 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables>
+        <family name="general">
+            <variable name="mode_conteneur_actif" type="string" description="No change" hidden="True">
+                <value>non</value>
+            </variable>
+            <variable name="mode_conteneur_actif1" type="string" description="No change">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+
+    <constraints>
+        <fill name="calc_val">
+            <target>mode_conteneur_actif</target>
+        </fill>
+    </constraints>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/01auto_withoutparam/tiramisu/base.py

@@ -0,0 +1,19 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_3 = StrOption(name="mode_conteneur_actif", doc="No change", default=Calculation(func.calc_val, Params(())), properties=frozenset({"force_default_on_freeze", "frozen", "hidden", "normal"}))
+option_4 = StrOption(name="mode_conteneur_actif1", doc="No change", default="non", properties=frozenset({"mandatory", "normal"}))
+option_2 = OptionDescription(name="general", doc="general", children=[option_3, option_4], properties=frozenset({"normal"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/01base_domainname/00-base.xml

@@ -0,0 +1,12 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+    <variables>
+        <family name="general">
+            <variable name="domain" type="domain" description="Description">
+                <value>my.domain.name</value>
+            </variable>
+        </family>
+    </variables>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/01base_domainname/makedict/base.json

@@ -0,0 +1 @@
+{"rougail.general.domain": "my.domain.name"}

tests/dictionaries/01base_domainname/tiramisu/base.py

@@ -0,0 +1,18 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_3 = DomainnameOption(name="domain", doc="Description", default="my.domain.name", type='domainname', allow_ip=False, properties=frozenset({"mandatory", "normal"}))
+option_2 = OptionDescription(name="general", doc="general", children=[option_3], properties=frozenset({"normal"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

tests/dictionaries/01base_file_include/00-base.xml

@@ -0,0 +1,19 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<rougail>
+
+    <services>
+        <service name="test">
+            <file name="/etc/file"/>
+        </service>
+    </services>
+
+    <variables>
+        <family name="general">
+            <variable name="mode_conteneur_actif" type="string" description="Description">
+                <value>non</value>
+            </variable>
+        </family>
+    </variables>
+</rougail>
+<!-- vim: ts=4 sw=4 expandtab
+-->

tests/dictionaries/01base_file_include/makedict/base.json

@@ -0,0 +1 @@
+{"rougail.general.mode_conteneur_actif": "non", "services.test.files.file.group": "root", "services.test.files.file.mode": "0644", "services.test.files.file.name": "/etc/file", "services.test.files.file.owner": "root", "services.test.files.file.source": "file", "services.test.files.file.templating": true, "services.test.files.file.activate": true}

tests/dictionaries/01base_file_include/tiramisu/base.py

@@ -0,0 +1,30 @@
+from importlib.machinery import SourceFileLoader
+from importlib.util import spec_from_loader, module_from_spec
+loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
+spec = spec_from_loader(loader.name, loader)
+func = module_from_spec(spec)
+loader.exec_module(func)
+for key, value in dict(locals()).items():
+    if key != ['SourceFileLoader', 'func']:
+        setattr(func, key, value)
+try:
+    from tiramisu3 import *
+except:
+    from tiramisu import *
+from rougail.tiramisu import ConvertDynOptionDescription
+option_3 = StrOption(name="mode_conteneur_actif", doc="Description", default="non", properties=frozenset({"mandatory", "normal"}))
+option_2 = OptionDescription(name="general", doc="general", children=[option_3], properties=frozenset({"normal"}))
+option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2])
+option_8 = StrOption(name="group", doc="group", default="root")
+option_9 = StrOption(name="mode", doc="mode", default="0644")
+option_10 = StrOption(name="name", doc="name", default="/etc/file")
+option_11 = StrOption(name="owner", doc="owner", default="root")
+option_12 = StrOption(name="source", doc="source", default="file")
+option_13 = BoolOption(name="templating", doc="templating", default=True)
+option_14 = BoolOption(name="activate", doc="activate", default=True)
+option_7 = OptionDescription(name="file", doc="file", children=[option_8, option_9, option_10, option_11, option_12, option_13, option_14])
+option_6 = OptionDescription(name="files", doc="files", children=[option_7])
+option_5 = OptionDescription(name="test", doc="test", children=[option_6])
+option_5.impl_set_information('manage', True)
+option_4 = OptionDescription(name="services", doc="services", children=[option_5], properties=frozenset({"hidden"}))
+option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1, option_4])

tests/dictionaries/01base_file_include/tmpl/file

@@ -0,0 +1 @@
+%include "incfile"