Infra/rougail
9
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge branch 'develop' into dist/risotto/risotto-2.8.0/develop

Browse Source
This commit is contained in:
Emmanuel Garette 2021-03-27 11:08:19 +01:00
commit 42b67ce575
1935 changed files with 29535 additions and 9034 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
README.md Normal file
View File

@ -0,0 +1,27 @@
Rougail
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
[Documentation (french)](doc/README.md)

37
doc/README.md Normal file
View File

@ -0,0 +1,37 @@
# Rougail
Rougail est un bibliothèque python3 qui permet de charger des dictionnaires (fichiers au format XML), de charger les variables dans Tiramisu et de générer des templates.
## La bibliothèque
- [Utiliser la bibliothèque](dev/README.md)
- [Personnaliser la configration de la bibliothèque](dev/config.md)
## Les dictionnaires
- [Les dictionnaires](dictionary/rougail.md)
- [Les dictionnaires extra](dictionary/extra.md)
### Les variables
- [Les familles](family/README.md)
- [Les variables](variable/README.md)
### Les services
- [La gestion d'un fichier](service/file.md)
- [La gestion d'un fichier de service systemd](service/override.md)
- [La gestion d'une ip](service/ip.md)
### Les contraintes
- [Les calcules automatiques](fill/README.md)
- [Les vérifications des valeurs](check/README.md)
- [Les conditions](condition/README.md)
- [Les variables meneuses ou suiveuses](variable/leadership.md)
## Les templates
- Type creole
- Type jinja2
FIXME ^^

23
doc/auto.rst
View File

@ -1,23 +0,0 @@
Valeur automatiquement modifiée
===============================
Une variable avec valeur automatiquement modifiée est une variable dont la valeur sera considéré comme modifié quand le serveur sera déployé.
Voici un variable a valeur automatiquement modifiée :
<variable name="my_variable" type="oui/non" description="My variable" auto_save="True">
Dans ce cas la valeur est fixée à la valeur actuelle.
Par exemple, si la valeur de cette variable est issue d'un calcul, la valeur ne sera plus recalculée.
Valeur en lecture seule automatique
===================================
Une variable avec valeur en lecture seule automatique est une variable dont la valeur ne sera plus modifiable par l'utilisateur quand le serveur sera déployé.
Voici un variable à valeur en lecture seule automatique :
<variable name="my_variable" type="oui/non" description="My variable" auto_freeze="True">
Dans ce cas la valeur est fixée à la valeur actuelle et elle ne sera plus modifiable par l'utilisateur.
Par exemple, si la valeur de cette variable est issue d'un calcul, la valeur ne sera plus recalculée.

5
doc/check/README.md Normal file
View File

@ -0,0 +1,5 @@
# Les vérifications des valeurs
- [Fonction de vérification](function.md)
- [Les variables à choix](valid_enum.md)
- [Réfinition](redefine.md)

53
doc/check/function.md Normal file
View File

@ -0,0 +1,53 @@
# Fonction de vérification
## Vérification stricte des valeurs
Une fonction de vérification est une fonction complémentaire au type qui permet de valider plus précisement le contenu d'une variable.
Voici un exemple simple de validation des valeurs :
```
<variables>
<variable name="my_variable"/>
</variables>
<constraints>
<check name="islower">
<target>my_variable</target>
</check>
</constraints>
```
La [cible (de type variable)](../target/variable.md) de la fonction de vérification est ici "my_variable".
Dans cette exemple, la valeur de la variable "my_variable" va être validé par la fonction islower.
Voici le contenu de la fonction :
```
def islower(value):
if value is None:
return
if not value.islower():
raise ValueError(f'"{value}" is not lowercase string')
```
Une fonction de vérification doit prendre en compte 2 aspects important :
- la valeur peut ne pas être renseigné (même si la variable est obligatoire), la valeur None doit être prise en compte
- si la valeur est invalide, il faut faire un raise de type ValueError avec, si possible, un message explicite.
À partir de maintenant seule None et des valeurs en minuscule seront autorisés.
Il est possible de définir des [paramètres](../param/README.md) à cette fonction.
## Vérification des valeurs avec avertissement
Dans la contrainte, il est possible de spécifier le niveau d'erreur et le mettre en avertissement :
```
<check name="islower" level="warning">
<target>my_variable</target>
</check>
```
Dans ce cas une valeur avec une majuscule sera accepté, mais un message d'avertissement apparaitra.

61
doc/check/redefine.md Normal file
View File

@ -0,0 +1,61 @@
# Rédéfinition
## Redéfinition des vérification
Dans un premier dictionnaire déclarons notre variable et sa fonction de vérification :
```
<variables>
<variable name="my_variable"/>
</variables>
<constraints>
<check name="islower">
<target>my_variable</target>
</check>
</constraints>
```
Dans un second dictionnaire il est possible de redéfinir le calcul :
```
<variables>
<variable name="my_variable" redefine="True"/>
</variables>
<constraints>
<check name="isspace">
<target>my_variable</target>
</check>
</constraints>
```
Dans ce cas, la fonction "islower" exécuté. Si cette fonction ne retourne pas d'erreur, la seconde fonction "isspace" sera exécuté.
## Redéfinition avec suppression d'un calcul
Il se peut que dans un dictionnaire on décide de vérifier la valeur d'une variable.
Dans un second dictionnaire il est possible de supprimer cette vérification.
Dans un premier dictionnaire déclarons notre variable et notre fonction de vérification :
```
<variables>
<variable name="my_variable"/>
</variables>
<constraints>
<check name="islower">
<target>my_variable</target>
</check>
</constraints>
```
Dans un second dictionnaire supprimer cette vérification :
```
<variables>
<family name="family">
<variable name="my_variable" redefine="True" remove_check="True"/>
</family>
</variables>
```

46
doc/check/valid_enum.md Normal file
View File

@ -0,0 +1,46 @@
# Les variables à choix
Une variable à choix est d'abord une variable avec une [fonction check](function.md).
## Les variables à choix simple
Il est possible d'imposer une liste de valeur pour une variable particulière :
```
<check name="valid_enum">
<param>yes</param>
<param>no</param>
<param>maybe</param>
<target>my_variable</target>
</check>
```
Dans ce cas, seule les valeurs proposés sont possible pour cette variable.
Par défaut, cette variable est obligatoire. Cela signifie qu'il n'est pas possible de spécifier "None" à cette variable.
## Les variables à choix avec valeur None
Il y a deux possibilités pour avoir une valeur "None" dans les choix :
- rendre la variable non obligatoire, cela va ajouter un choix "None" dans la liste :
```
<variable name="my_variable" mandatory="False">
```
Ou en ajoutant le paramètre "None" :
```
<check name="valid_enum">
<param>yes</param>
<param>no</param>
<param type='nil'/>
<param>maybe</param>
<target>my_variable</target>
</check>
```
## La valeur par défaut
Si aucune valeur n'est spécifié pour la variable, automatiquement le premier choix va est placé comme valeur par défaut.

5
doc/condition/README.md Normal file
View File

@ -0,0 +1,5 @@
# Les conditions
- [Déclaration d'une condition](condition.md)
- [Les différentes conditions](conditions.md)
- [Réfinition](redefine.md)

76
doc/condition/condition.md Normal file
View File

@ -0,0 +1,76 @@
# Les conditions
## Un condition
Les conditions permettent d'ajouter ou de supprimer des propriétés à une [variable](../variable/README.md), une [famille](../family/README.md), un [service](../service/service.md), un [fichier](../service/file.md) ou une [ip](../service/ip.md) suivant le contexte.
Nous allons nous concentrer ici sur la condition hidden_if_in, mais [il existe d'autre conditions](conditions.md).
La condition hidden_if_in permet de cacher une variable où une famille à l'utilisateur, mais cette variable est toujours accessible dans un calcul, un vérification ou dans un template.
```
<variables>
<variable name="condition" type="boolean"/>
<variable name="my_variable"/>
</variables>
<constraints>
<condition name="hidden_if_in" source="condition">
<param>True</param>
<target>my_variable</target>
</condition>
</constraints>
```
Le [paramètres](../param/README.md) de la condition permet de définir les valeurs que doit avoir la source pour appliquer l'action.
La [cible](../target/README.md) de la condition est ici "my_variable".
Donc ici la variable est caché à l'utilisateur si la variable "condition" est à True (le paramètre).
## Un condition avec plusieurs paramètres
Il est également possible de mettre plusieurs paramètre :
```
<variables>
<variable name="condition"/>
<variable name="my_variable"/>
</variables>
<constraints>
<condition name="hidden_if_in" source="condition">
<param>yes</param>
<param>maybe</param>
<target>my_variable</target>
</condition>
</constraints>
```
## Une condition optionnelle
Il est possible de définir une condition avec une variable source qui n'existe pas dans toutes les contextes.
Dans ce cas, on met la condition en "optionnelle".
Si la variable source existe, la condition s'applique.
Si la variable source n'existe pas :
- si le nom fini en _if_in (par exemple hidden_if_in), l'action est forcée sans condition (les cibles sont hidden)
- si le nom fini en _if_not_in (par exemple hidden_if_not_in), la condition est totalement ignorée
Ces deux comportements peuvent être changé à tout moment avec l'attribut "apply_on_fallback". Dans ce cas :
- si la valeur de "apply_on_fallback" est "True", l'action est forcée sans condition
- si la valeur de "apply_on_fallback" est "False", la condition est totalement ignorée
Exemple :
```
<condition name="hidden_if_in" source="condition" optional="True", apply_on_fallback="False">
<param>yes</param>
<param>maybe</param>
<target>my_variable</target>
</condition>
```

29
doc/condition/conditions.md Normal file
View File

@ -0,0 +1,29 @@
# Les conditions
## Les conditions \_if_in et \_if_not_in
Il existe deux types de conditions :
- les conditions dont le nom fini par \_if_in : dans ce cas si la variable source est égal à un des paramètres, l'action est effective
- les conditions dont le nom fini par \_if_not_in : dans ce cas si la variable source est différents de tous les paramètres, l'action est effective
## Désactivation
Il est possible de désactiver une [variable](../variable/README.md) ou une [famille](../family/README.md) avec les conditions :
- disabled_if_in
- disabled_if_not_in
## Caché
Il est possible de cacher une [variable](../variable/README.md), une [famille](../family/README.md), un [service](../service/service.md), un [fichier](../service/file.md) ou une [ip](../service/ip.md) avec les conditions :
- hidden_if_in
- hidden_if_not_in
## Obligatoire
Il est possible de rendre obligatoire une [variable](../variable/README.md) avec les conditions :
- mandatory_if_in
- mandatory_if_not_in

29
doc/condition/redefine.md Normal file
View File

@ -0,0 +1,29 @@
# Rédéfinition
Il se peut que dans un dictionnaire on décide de définir une condition.
Dans un second dictionnaire il est possible de supprimer cette condition.
Dans un premier dictionnaire déclarons notre variable et notre calcule :
```
<variables>
<variable name="condition" type="boolean"/>
<variable name="my_variable"/>
</variables>
<constraints>
<condition name="hidden_if_in" source="condition">
<param>True</param>
<target>my_variable</target>
</condition>
</constraints>
```
Dans un second dictionnaire supprimer ce calcul :
```
<variables>
<variable name="condition" redefine="True" remove_condition="True"/>
</variables>
```

191
doc/dev/README.md Normal file
View File

@ -0,0 +1,191 @@
# La bibliothèque rougail
Rougail est une bibliothèque simple a utiliser.
Dans les exemples suivant, nous utilisons la configuration par défaut de Rougail. Vous pouvez [personnaliser les répertoires utilisés](config.md).
## Convertisons un dictionnaire en objet tiramisu
Commençons par créer un dictionnaire simple.
Voici un premier dictionnaire /srv/rougail/dictionaries/00-base.xml :
```
<?xml version='1.0' encoding='UTF-8'?>
<rougail>
<variables>
<variable name="my_variable">
<value>my_value</value>
</variable>
</variables>
</rougail>
```
Construisons les objets tiramisu :
```python
from rougail import RougailConvert
rougail = RougailConvert()
rougail.save('example.py')
```
Un nouveau fichier 'example.py' va être créé dans le répertoire local
## Convertisons un dictionnaire extra en objet tiramisu
En plus du dictionnaire précédent, créons un dictionnaire extra /srv/rougail/extra_dictionaries/00-base.xml
```
<?xml version='1.0' encoding='UTF-8'?>
<rougail>
<variables>
<variable name="my_variable_extra">
<value>my_value_extra</value>
</variable>
</variables>
</rougail>
```
Construisons les objets tiramisu :
```python
from rougail import RougailConvert, RougailConfig
RougailConfig['extra_dictionaries']['example'] = ['/srv/rougail/extra_dictionaries/']
rougail = RougailConvert()
rougail.save('example.py')
```
## Templatisons un template
Nous créons un dictionnaire complémentaire pour ajouter notre template /srv/rougail/dictionaries/00-template.xml :
```
<?xml version='1.0' encoding='UTF-8'?>
<rougail>
<services>
<service name="example">
<file name="/etc/example.conf"/>
</service>
</services>
</rougail>
```
Et un template /srv/rougail/templates/example.conf :
```
The value: %%my_variable
The extra value: %%example.my_variable_extra
```
Générons le fichier tiramisu :
```python
from rougail import RougailConvert, RougailConfig
RougailConfig['extra_dictionaries']['example'] = ['/srv/rougail/extra_dictionaries/']
rougail = RougailConvert()
rougail.save('example.py')
```
Créer les répertoires utils pour la templatisation :
```bash
mkdir /srv/rougail/destinations /srv/rougail/tmp
```
Générons le template :
```python
import asyncio
from example import option_0
from tiramisu import Config
from rougail import RougailSystemdTemplate
async def template():
config = await Config(option_0)
engine = RougailSystemdTemplate(config)
await engine.instance_files()
loop = asyncio.get_event_loop()
loop.run_until_complete(template())
loop.close()
```
Le fichier /srv/rougail/destinations/etc/example.conf est maintenant créé avec le contenu attendu suivant :
```
The value: my_value
The extra value: my_value_extra
```
## Appliquons un patch au template
Il peut être intéressant de réaliser un patch à un template pour corriger un problème spécifique à notre environnement, sans attendre que le mainteneur du template n'est fait la correction.
Testons en créant le patch /srv/rougail/patches/example.conf.patch :
```
--- /srv/rougail/templates/example.conf 2021-02-13 19:41:38.677491087 +0100
+++ tmp/example.conf 2021-02-13 20:12:55.525089820 +0100
@@ -1,3 +1,5 @@
The value: %%my_variable
The extra value: %%example.my_variable_extra
+
+Add by a patch
```
Le patch est bien appliquer sur notre fichier /srv/rougail/destinations/etc/example.conf :
```
The value: my_value
The extra value: my_value_extra
Add by a patch
```
Deux choses importantes à savoir sur les patchs :
- le nom du patch est obligatoire le nom du template source + ".patch"
- la deuxième ligne doit toujours commencer par "+++ tmp/" + le nom du template source
## Créons une fonction personnalisé
Nous créons un dictionnaire complémentaire pour ajouter un calcul à la variable "my_variable" dans /srv/rougail/dictionaries/00-fill.xml :
```
<?xml version='1.0' encoding='UTF-8'?>
<rougail>
<constraints>
<fill name="return_no">
<target>my_variable</target>
</fill>
</constraints>
</rougail>
```
Puis créons la fonction "return_no" dans /srv/rougail/functions.py :
```python
def return_no():
return 'no'
```
Après avoir reconverti les dictionnaires et regénérer le template nous avons donc le contenu du fichier /srv/rougail/destinations/etc/example.conf :
```
The value: no
The extra value: my_value_extra
Add by a patch
```
La valeur de la variable "my_variable" est bien calculé à partir de la fonction "return_no".

94
doc/dev/config.md Normal file
View File

@ -0,0 +1,94 @@
# Personnalisons la configuration de Rougail
La configuration de rougail se trouve dans l'objet RougailConfig :
```python
from rougail import RougailConfig
```
C'est un simple dictionnaire python avec différentes clefs.
Pour modifier il suffit de faire :
```python
RougailConfig[key] = value
```
## Les répertoires des dictionnaires
Il existe deux types de répertoires de dictionnaires :
- les dictionnaires principaux avec la clef "dictionaries_dir". La valeur par défaut est ['/srv/rougail/dictionaries']. Cette variable doit contenir la liste des répertoires contenants des dictionnaires.
Les dictionnaires sont chargés dans l'ordre des répertoires. Chaque répertoire est chargé les uns après les autres. A l'intérieur de ces répertoires les fichiers XML seront classés par ordre alphabétique.
Il n'y a pas de classement par ordre alphabétique de l'ensemble des fichiers XML de tous les répertoires.
Les familles et variables de ces dictionnaires sont classés, par défaut, dans l'espace de nom "rougail". Il est possible de changer le nom de cet espace de nom avec la clef "variable_namespace".
- les dictionnaires extra avec la clef "extra_dictionaries". La valeur est un dictionnaire avec l'ensemble des espaces de nom. La clef étant l'espace de nom et la valeur étant une liste de répertoire.
Par exemple pour ajouter l'extra "example" il faut faire :
```
RougailConfig['extra_dictionaries']['example'] = ['/dir1', '/dir2']
```
Les dictionnaires sont chargés dans le même ordre que les dictionnaires principaux.
## La DTD
Rougail a besoin du fichier de DTD pour lire les fichiers dictionnaire.
Par défaut le fichier de la DTD est dans le sous répertoire "data" du répertoire de code. Le nom du fichier est rougail.dtd.
Pour pouvez changer le répertoire de destination de la DTD et le nom du fichier avec la clef "dtdfilename".
## Le fichier de fonction
Le fichier qui contient les fonctions personnalisés est géré dans la clef "functions_file" et a comme valeur par défaut "/srv/rougail/functions.py".
## Le répertoire des templates
Le répertoire des templates est géré dans la clef "templates_dir" et a comme valeur par défaut : "/srv/rougail/templates".
## Le moteur de templates par défaut
Le moteur de template est géré dans la clef "default_engine" et a comme valeur par défaut : "creole". Les valeurs possible sont "none", "creole" ou "jinja2".
## Le répertoire des patchs
Le répertoire des patches est géré dans la clef "patches_dir" et a comme valeur par défaut : "/srv/rougail/patches".
## Le répertoire temporaire
Le répertoire temporaire est utile lors de la génération de template. Il contient une copie des templates avec, éventuellement, les patches appliqués sur les templates.
Le répertoire de temporaire est géré dans la clef "tmp_dir" et a comme valeur par défaut : "/srv/rougail/tmp".
## Le répertoire de destination des fichiers générés
Le répertoire de destination des fichiers générés est géré dans la clef "destinations_dir" et a comme valeur par défaut : "/srv/rougail/destinations".
## La variable auto_freeze
La propriété auto_freeze n'est appliqué que une variable spécifique passe à True. Par défaut le nom de la variable est "instancied_module", mais il est possible de changer le nom de cette variable via la clef "auto_freeze_variable".
## Les modes
Les modes sont personnalisables dans Rougail. Par défaut les modes sont "basic", "normal" et "expert".
Il est possible de changer cette liste via la clef "modes_level".
Si vous changer ces valeurs, penser à changer les modes par défaut des familles et des variables.
## Le mode par défaut pour une famille
Le mode par défaut d'une famille est "basic". Il est possible de changer le mode par défaut d'une famille via la clef "default_family_mode".
## Le mode par défaut pour une variable
Le mode par défaut d'une variable est "normal". Il est possible de changer le mode par défaut d'une variable via la clef "default_variable_mode".
## Le nom des fonctions internes
Il est possible d'ajouter des fonctions interne via la clef "internal_functions".

12
doc/dictionary/extra.md Normal file
View File

@ -0,0 +1,12 @@
# Les dictionnaires extra
Un extra est un espace de nom différent. L'idée et de pouvoir classer les variables par thématique.
Les espaces de nom extra doivent être déclaré au moment [de la configuration de Rougail](../dev/config.md).
Dans cet espace de nom :
- des variables et des familles peuvent avoir le même nom dans différentes familles
- la valeur d'un cible, source, leader ou follower des contraintes doivent être avec un chemin complet
- on ne peut pas déclarer des services dans cet espace de nom
- dans un template il faut utiliser des chemins complet (%%my_extra.my_family.my_variable ou %%my_extra.my_family.leader.follower pour une variable suiveuse)

22
doc/dictionary/rougail.md Normal file
View File

@ -0,0 +1,22 @@
# Les dictionnaires
## Un dictionnaire ?
Un dictionnaire est un fichier XML donc la structure est décrite dans cette documentation.
Un dictionnaire contient en ensemble de variable chargé dans Tiramisu, utilisable à tout moment, notamment dans des templates.
Les familles, les variables et les contraintes peuvent être défini dans plusieurs dictionnaires. Ces dictionnaires s'aggrège alors.
Il est également possible de redéfinir des éléments pour changer les comportement d'une variable ou d'un service.
## L'espace de nom par défaut
L'espace de nom par défaut s'appelle "rougail" ([ce nom est personnalisable](../dev/config.md)).
Cet espace de nom est un peu particulier :
- le nom des variables et des familles doivent être unique pour l'ensemble de cet espace (même si ces variables ou familles sont dans des familles différentes)
- la valeur d'un cible, source, leader ou follower des contraintes peuvent être avec nom de la variable ou de la famille ou leurs chemins complet
- on peut déclarer des services dans cet espace de nom
- dans un template on peut utiliser cette variable sans le chemin complet (%%my_variable) ou avec (%%rougail.my_family.my_variable)

5
doc/family/README.md Normal file
View File

@ -0,0 +1,5 @@
# Famille
- [Une famille](simple.md)
- [Famille crée dynamiquement](auto.md)

27
doc/family/auto.md Normal file
View File

@ -0,0 +1,27 @@
# Famille crée dynamiquement
Pour créer une famille dynamiquement, il faut créer une famille fictive lié à une variable.
Le nom et la description de la famille et des variables qu'elle contient sera en réalité le prefix du nouveau nom/description. Le suffix viendra de la variable liée.
Par exemple :
```
<variable name='varname' multi="True">
<value>val1</value>
<value>val2</value>
</variable>
<family name="my_dyn_family_" dynamic="varname" description="Describe ">
<variable name="my_dyn_var_"/>
</family>
```
Créera deux familles :
- la famille dynamique : "my_dyn_family_val1" avec la description "Describe val1"
- la famille dynamique : "my_dyn_family_val2" avec la description "Describe val2"
Dans la famille dynamique "my_dyn_family_val1" on retrouvera une variable "my_dyn_var_val1".
Bien évidement si le contenu de "varname" venait a évolué, de nouvelles familles dynamiques pouvent apparaitre ou des familles dynamiques peuvent disparaître.
Attention la variable lié à la famille doit être obligatoirement une variable multiple et il n'est pas possible de mettre une famille dans une famille dynamique.

68
doc/family/simple.md Normal file
View File

@ -0,0 +1,68 @@
# Une famille
Une famille est un conteneur de variables. Elle peut contenir également des familles.
Pour décrire une famille il faut mettre au minimum un nom :
```
<family name="my_family"/>
```
Cette famille doit être placé dans une balise [variables](../variables.md) :
```
<variables>
<family name="my_family"/>
</variables>
```
Ou dans une autre famille :
```
<variables>
<family name="my_family">
<family name="second_family"/>
</family>
</variables>
```
Attention, une famille vide sera automatiquement supprimée.
## Description et aide de la famille
En plus d'un nom, il est possible de mettre une "description" à la famille. C'est une information "utilisateur" qui nous permettra d'avoir des informations complémentaires sur le contenu de cette famille :
```
<family name="my_family" description="This is a great family"/>
```
En plus de la description, il est possible de préciser une aide complémentaire :
```
<family name="my_family" help="This is a great family"/>
```
## Mode de la famille
Le [mode](../mode.md) par défaut d'une famille correspond au [mode](../mode.md) du mode le plus petit entre la famille parente, les variables enfants ou des familles enfants qui sont contenu dans cette famille.
Changer le [mode](../mode.md) d'une famille permet de définir le [mode](../mode.md) par défaut des variables ou des familles inclusent dans cette famille.
Pour définir le [mode](../mode.md) :
```
<family name="my_family" mode="expert"/>
```
## Famille invisible
Il est possible de cacher une famille, ainsi que toutes les variables et des familles inclusent dans cette famille.
Cacher une famille signifie qu'elle ne sera pas visible lorsqu'on modifie la configuration du service.
Par contre ces variables sont accessibles lorsqu'on va utiliser ces variables.
Pour cacher une famille :
```
<family name="my_family" hidden="True"/>
```

285
doc/fill.rst
View File

@ -1,285 +0,0 @@
Les variables calculées
=======================
Une variable calculée est une variable donc sa valeur est le résultat d'une fonction python.
Variable avec une valeur par défaut calculée
--------------------------------------------
Créons une variable de type "oui/non" donc la valeur est retournée par la fonction "return_no" :
<variables>
<family name="family">
<variable name="my_calculated_variable" type="oui/non" description="My calculated variable"/>
</family>
</variables>
<constraints>
<fill name="return_no" target="my_calculated_variable"/>
</constraints>
Puis créons la fonction "return_no" :
def return_no():
return 'non'
Dans ce cas, la valeur par défaut est la valeur retournée par la fonction (ici "non"), elle sera calculée tant que l'utilisateur n'a pas de spécifié une valeur à cette variable.
Si l'utilisateur à définit une valeur par défaut à "my_calculated_variable" :
<variable name="my_calculated_variable" type="oui/non" description="My calculated variable">
<value>oui</value>
</variable>
Cette valeur par défaut sera complètement ignorée.
Variable avec une valeur calculée
---------------------------------
En ajoutant le paramètre "hidden" à "True" dans la variable précédente, l'utilisateur n'aura plus la possibilité de modifié la valeur. La valeur de la variable sera donc systématiquement calculée :
<variable name="my_calculated_variable" type="oui/non" description="My calculated variable" hidden="True"/>
Si une condition "hidden_if_in" est spécifié à la variable, la valeur sera modifiable par l'utilisateur si elle n'est pas cachée mais elle sera systèmatiquement calculée (même si elle a déjà était modifiée) si la variable est cachée.
Variable avec valeur calculée obligatoire
-----------------------------------------
Par défaut les variables calculées ne sont pas des varibles obligatoires.
Dans ce cas un calcul peut retourner None, mais surtout un utilisateur peut spécifier une valeur nulle à cette variable. Dans ce cas le calcul ne sera pas réalisé.
Fonction avec une valeur fixe comme paramètre positionnel
---------------------------------------------------------
Déclarons un calcul avec paramètre :
<constraints>
<fill name="return_value" target="my_calculated_variable">
<param>non</param>
</fill>
</constraints>
Créons la fonction correspondante :
def return_value(value):
return value
La variable aura donc "non" comme valeur puisque le paramètre aura la valeur fixe "non".
Paramètre nommée
----------------
Déclarons une contrainte avec un paramètre nommée :
<constraints>
<fill name="return_value" target="my_calculated_variable">
<param name="valeur">non</param>
</fill>
</constraints>
Dans ce cas la fonction return_value sera exécuté avec le paramètre nommé "valeur" dont sa valeur sera "non".
Paramètre avec un nombre
------------------------
Déclarons un calcul avec paramètre avec un nombre :
<constraints>
<fill name="return_value_with_number" target="my_calculated_variable">
<param type="number">1</param>
</fill>
</constraints>
Créons la fonction correspondante :
def return_value_with_number(value):
if value == 1:
return 'non'
return 'oui'
La variable aura donc "non" comme valeur puisque le paramètre aura la valeur fixe "1".
Paramètre dont la valeur est issue d'une autre variable
-------------------------------------------------------
Créons deux variables avec une contrainte de type variable qui contient le nom de la variable dont sa valeur sera utilisé comme paramètre :
<variables>
<family name="family">
<variable name="my_calculated_variable" type="oui/non" description="My calculated variable"/>
<variable name="my_variable" type="number" description="My variable">
<value>1</value>
</variable>
</family>
</variables>
<constraints>
<fill name="return_value_with_number" target="my_calculated_variable">
<param type="variable">my_variable</param>
</fill>
</constraints>
Si l'utilisateur laisse la valeur 1 à "my_variable", la valeur par défault de la variable "my_calculated_variable" sera "non".
Si la valeur de "my_variable" est différent de 1, la valeur par défaut de la variable "my_calculated_variable" sera "oui".
Paramètre dont la valeur est issue d'une information de la configuration
------------------------------------------------------------------------
Créons une variable et la contrainte :
<variables>
<family name="family">
<variable name="my_calculated_variable" type="string" description="My calculated variable"/>
</family>
</variables>
<constraints>
<fill name="return_value" target="my_calculated_variable">
<param type="information">server_name</param>
</fill>
</constraints>
Dans ce cas, l'information de la configuration "server_name" sera utilisé comme valeur de la variable "my_calculated_variable".
Si l'information n'existe pas, la paramètre aura la valeur "None".
Paramètre avec variable potentiellement non existante
-----------------------------------------------------
Suivant le contexte une variable peut exister ou ne pas exister.
Un paramètre de type "variable" peut être "optional" :
<variables>
<family name="family">
<variable name="my_calculated_variable" type="oui/non" description="My calculated variable"/>
</family>
</variables>
<constraints>
<fill name="return_value" target="my_calculated_variable">
<param type="variable" optional="True">unknow_variable</param>
</fill>
</constraints>
Dans ce cas la fonction "return_value" est exécuté sans paramètre.
Paramètre avec variable potentiellement désactivée
--------------------------------------------------
FIXME :
<!ATTLIST param notraisepropertyerror (True|False) "False">
Il n'y a pas spécialement de test !
Les variables suiveuses
-----------------------
FIXME :
- tests/flattener_dicos/10leadership_append/00-base.xml
- tests/flattener_dicos/10leadership_auto/00-base.xml
- tests/flattener_dicos/10leadership_autoleader/00-base.xml
- tests/flattener_dicos/10leadership_autoleader_expert/00-base.xml
Les variables dynamiques
------------------------
Paramètre avec variable dynamique
'''''''''''''''''''''''''''''''''
Il est possible de faire un calcul avec comme paramètre une variable dynamique mais pour une suffix particulier :
<variables>
<family name='family'>
<variable name='suffixes' type='string' description="Suffixes of dynamic family" multi="True">
<value>val1</value>
<value>val2</value>
</variable>
<variable name="my_calculated_variable" type="string" description="My calculated variable"/>
</family>
<family name='dyn' dynamic="suffixes">
<variable name='vardyn' type='string' description="Dynamic variable">
<value>val</value>
</variable>
</family>
</variables>
<constraints>
<fill name="return_value" target="my_calculated_variable">
<param type="variable">vardynval1</param>
</fill>
</constraints>
Dans ce cas, valeur du paramètre de la fonction "return_value" sera la valeur de la variable "vardyn" avec le suffix "val1".
Calcule d'une variable dynamique
''''''''''''''''''''''''''''''''
Il est également possible de calculer une variable dynamique à partir d'une variable standard :
<variables>
<family name='family'>
<variable name='suffixes' type='string' description="Suffixes of dynamic family" multi="True">
<value>val1</value>
<value>val2</value>
</variable>
<variable name="my_variable" type="string" description="My variable">
<value>val</value>
</variable>
</family>
<family name='dyn' dynamic="suffixes">
<variable name="my_calculated_variable_dyn_" type="string" description="My calculated variable"/>
<value>val</value>
</variable>
</family>
</variables>
<constraints>
<fill name="return_value" target="my_calculated_variable_dyn_">
<param type="variable">my_variable</param>
</fill>
</constraints>
Dans ce cas, les variables dynamiques "my_calculated_variable_dyn_" seront calculés à partir de la valeur de la variable "my_variable".
Que cela soit pour la variable "my_calculated_variable_dyn_val1" et "my_calculated_variable_dyn_val2".
Par contre, il n'est pas possible de faire un calcul pour une seule des deux variables issues de la variable dynamique.
Si c'est ce que vous cherchez à faire, il faudra prévoir un traitement particulier dans votre fonction.
Dans ce cas, il faut explicitement demander la valeur du suffix dans la fonction :
<constraints>
<fill name="return_value_suffix" target="my_calculated_variable_dyn_">
<param type="variable">my_variable</param>
<param type="suffix"/>
</fill>
</constraints>
Et ainsi faire un traitement spécifique pour ce suffix :
def return_value_suffix(value, suffix):
if suffix == 'val1':
return value
Redéfinition des calcules
-------------------------
Dans un premier dictionnaire déclarons notre variable et notre calcule :
<variables>
<family name="family">
<variable name="my_calculated_variable" type="oui/non" description="My calculated variable"/>
</family>
</variables>
<constraints>
<fill name="return_no" target="my_calculated_variable"/>
</constraints>
Dans un second dictionnaire il est possible de redéfinir le calcul :
<variables>
<family name="family">
<variable name="my_calculated_variable" redefine="True"/>
</family>
</variables>
<constraints>
<fill name="return_yes" target="my_calculated_variable"/>
</constraints>
Dans ce cas, à aucun moment la fonction "return_no" ne sera exécuté. Seul la fonction "return_yes" le sera.

6
doc/fill/README.md Normal file
View File

@ -0,0 +1,6 @@
# Les variables calculées
Une variable calculée est une variable donc sa valeur est le résultat d'une fonction python.
- [Valeur calculée de la variable](value.md)
- [Réfinition](redefine.md)

58
doc/fill/redefine.md Normal file
View File

@ -0,0 +1,58 @@
# Rédéfinition
## Redéfinition des calcules
Dans un premier dictionnaire déclarons notre variable et notre calcule :
```
<variables>
<variable name="my_calculated_variable"/>
</variables>
<constraints>
<fill name="return_no">
<target>my_calculated_variable</target>
</fill>
</constraints>
```
Dans un second dictionnaire il est possible de redéfinir le calcul :
```
<variables>
<variable name="my_calculated_variable" redefine="True"/>
</variables>
<constraints>
<fill name="return_yes">
<target>my_calculated_variable</target>
</fill>
</constraints>
```
Dans ce cas, à aucun moment la fonction "return_no" ne sera exécuté. Seul la fonction "return_yes" le sera.
## Redéfinition avec suppression d'un calcul
Il se peut que dans un dictionnaire on décide de définir une valeur par défaut à une variable via un calcul.
Dans un second dictionnaire il est possible de supprimer ce calcul.
Dans un premier dictionnaire déclarons notre variable et notre calcule :
```
<variables>
<variable name="my_calculated_variable"/>
</variables>
<constraints>
<fill name="return_no">
<target>my_calculated_variable"</target>
</fill>
</constraints>
```
Dans un second dictionnaire supprimer ce calcul :
```
<variables>
<variable name="my_calculated_variable" redefine="True" remove_fill="True"/>
</variables>
```

112
doc/fill/value.md Normal file
View File

@ -0,0 +1,112 @@
# Valeur calculée de la variable
## Variable avec une valeur par défaut calculée
Créons une variable dont la valeur est retournée par la fonction "return_no" :
```
<variables>
<variable name="my_calculated_variable"/>
</variables>
<constraints>
<fill name="return_no">
<target>my_calculated_variable</target>
</fill>
</constraints>
```
Puis créons la fonction "return_no" :
```
def return_no():
return 'no'
```
La [cible (de type variable)](../target/variable.md) du calcul est ici "my_calculated_variable".
Dans ce cas, la valeur par défaut est la valeur retournée par la fonction (ici "no"), elle sera calculée tant que l'utilisateur n'a pas de spécifié de valeur à cette variable.
Attention, si une valeur par défaut est définit dans la variable "my_calculated_variable" :
```
<variable name="my_calculated_variable">
<value>yes</value>
</variable>
```
Cette valeur par défaut sera complètement ignorée. C'est le calcul qui en définira la valeur.
Il est possible de définir des [paramètres](../param/README.md) à cette fonction.
## Variable avec une valeur calculée
En ajoutant le paramètre "hidden" à "True" dans la variable précédente, l'utilisateur n'aura plus la possibilité de modifié la valeur. La valeur de la variable sera donc systématiquement calculée :
```
<variable name="my_calculated_variable" hidden="True"/>
```
Si une condition "hidden_if_in" est spécifié à la variable, la valeur sera modifiable par l'utilisateur si elle n'est pas cachée mais elle sera systèmatiquement calculée (même si elle a déjà était modifiée) si la variable est cachée.
## Variable meneuse ou suiveuse avec valeur calculé
Une [variable suiveuse](../variable/leadership.md) ne peut pas être calculé automatiquement.
Une [variable meneuse](../variable/leadership.md) peut être calculé automatiquement.
Si la variable n'est pas multiple, il ne faut pas que le calcule retourne une liste.
## Variable dynamique avec une valeur calculée
Il est également possible de calculer [une variable d'une famille dynamique](../family/auto.md) à partir d'une variable standard :
```
<variables>
<variable name='suffixes' type='string' description="Suffixes of dynamic family" multi="True">
<value>val1</value>
<value>val2</value>
</variable>
<variable name="my_variable" type="string" description="My variable">
<value>val</value>
</variable>
<family name='dyn' dynamic="suffixes">
<variable name="my_calculated_variable_dyn\_" type="string" description="My calculated variable"/>
<value>val</value>
</variable>
</family>
</variables>
<constraints>
<fill name="return_value">
<param type="variable">my_variable</param>
<target>my_calculated_variable_dyn_</target>
</fill>
</constraints>
```
Dans ce cas, les variables dynamiques "my_calculated_variable_dyn_" seront calculés à partir de la valeur de la variable "my_variable".
Que cela soit pour la variable "my_calculated_variable_dyn_val1" et "my_calculated_variable_dyn_val2".
Par contre, il n'est pas possible de faire un calcul pour une seule des deux variables issues de la variable dynamique.
Si c'est ce que vous cherchez à faire, il faudra prévoir un traitement particulier dans votre fonction.
Dans ce cas, il faut explicitement demander la valeur du suffix dans la fonction :
```
<constraints>
<fill name="return_value_suffix">
<param type="variable">my_variable</param>
<param type="suffix"/>
<target>my_calculated_variable_dyn_</target>
</fill>
</constraints>
```
Et ainsi faire un traitement spécifique pour ce suffix :
```
def return_value_suffix(value, suffix):
if suffix == 'val1':
return value
```
## Variable avec valeur calculée obligatoire
Par défaut les variables calculées ne sont pas des variables obligatoires.
Dans ce cas un calcul peut retourner "None" ou "", mais surtout un utilisateur peut spécifier une valeur nulle à cette variable. Dans ce cas le calcul ne sera plus réalisé.

10
doc/mode.md Normal file
View File

@ -0,0 +1,10 @@
Mode
====
Par défault, il existe trois "mode" dans Rougail :
- basic : variables indispensables à la mise en place d'un service
- normal : variables couramment modifié par l'utilisateur
- expert : variables a manipuler avec précausion et en toute connaissance de cause
Il est possible de personnaliser les modes dans la [configuration de rougail](dev/config.md)

7
doc/param/README.md Normal file
View File

@ -0,0 +1,7 @@
# Paramètre de la fonction
- [Paramètre positionnel ou nommée](positional.md)
- [Type de paramètre simple](simple.md)
- [Type de paramètre "variable"](variable.md)
- [Type de paramètre "information"](information.md)

24
doc/param/information.md Normal file
View File

@ -0,0 +1,24 @@
# Paramètre de type information
## Les informations de la configuration
Le paramètre peut être la valeur est issue d'une information de la configuration :
```
<param type="information">server_name</param>
```
Dans ce cas, l'information de la configuration "server_name" sera utilisé comme valeur du paramètre.
Si l'information n'existe pas, la paramètre aura la valeur "None".
## Les informations de la cible
Le paramètre peut être la valeur est issue d'une information de la cible du calcul :
```
<param type="target_information">test</param>
<param type="target_information">help</param>
```
Dans ce cas, l'information de la configuration "test" ou "help" sera utilisé comme valeur du paramètre.
Si l'information n'existe pas, la paramètre aura la valeur "None".

26
doc/param/positional.md Normal file
View File

@ -0,0 +1,26 @@
# Paramètre positionnel
Déclarons un paramètre positionnel :
```
<param>no</param>
```
Créons la fonction correspondante :
```
def return_value(value):
return value
```
La variable "value" de la fonction "return_value" aura donc "no" comme valeur puisque le paramètre aura la valeur fixe "no".
# Paramètre nommée
Déclarons maintenant un paramètre nommée :
```
<param name="valeur">no</param>
```
Dans ce cas la fonction return_value sera exécuté avec le paramètre nommé "valeur" dont sa valeur sera "no".

44
doc/param/simple.md Normal file
View File

@ -0,0 +1,44 @@
# Paramètre de type "texte"
Déclarons un paramètre avec une string :
```
<param type="string">no</param>
```
C'est le type par défaut pour un paramètre.
# Paramètre de type "nombre"
Déclarons un paramètre avec un nombre :
```
<param type="number">1</param>
```
Créons la fonction correspondante :
```
def return_value_with_number(value):
if value == 1:
return 'no'
return 'yes'
```
La variable aura donc "no" comme valeur puisque le paramètre aura la valeur fixe "1".
# Paramètre de type "booléen"
Déclarons un paramètre avec un booléen :
```
<param type="boolean">True</param>
```
# Paramètre de type "nil"
Le paramètre peut être une valeur null (None en python) :
```
<param type="nil"/>
```

55
doc/param/variable.md Normal file
View File

@ -0,0 +1,55 @@
# Paramètre de type "variable"
Imaginons que la variable "my_variable" pré-existe. La valeur de la variable sera utilisé comme paramètre :
```
<param type="variable">my_variable</param>
```
[Les variables meneuses ou suiveuses](../variable/leadership.md) peuvent être utilisé sans soucis comme paramètre.
## Paramètre avec variable potentiellement non existante
Suivant le contexte une variable peut exister ou ne pas exister.
Un paramètre de type "variable" peut être "optional" :
```
<param type="variable" optional="True">unknow_variable</param>
```
Si la variable "unknow_variable" n'existe pas, le paramètre ne sera pas passé à la fonction.
Si maintenant on créé un nouveau dictionnaire en créant cette variable, la fonction sera exécuté avec le paramètre.
## Paramètre avec variable potentiellement désactivée
Si une variable est désactivé, l'utilisation de cette variable peut poser problème.
Il est possible de ne pas générer d'erreur si une variable est désactivé en utilisant le paramètre "propertyerror" :
```
<param type="variable" propertyerror="False">variable1</param>
```
Dans ce cas, si la variable est désactivé, le paramètre n'est jamais donnée à la fonction de destination.
## Paramètre avec variable dynamique
Il est possible de faire un calcul avec comme paramètre [une variable d'une famille dynamique](../family/auto.md) mais pour une suffix particulier.
Par exemple :
```
<param type="variable">vardynval1</param>
```
Dans ce cas, la valeur du paramètre de la fonction sera la valeur de la variable "vardyn" pour la famille ayant le suffix "val1".
Il peut être utile de récupérer la valeur du suffix dans la fonction, pour cela il suffit de mettre un paramètre de type suffix :
```
<param type="suffix"/>
```
Dans l'exemple précédent la valeur de ce paramètre sera "val1".

4
doc/service/README.md Normal file
View File

@ -0,0 +1,4 @@
# Les services

168
doc/service/file.md Normal file
View File

@ -0,0 +1,168 @@
# La gestion d'un fichier
## La balise file
La gestion des fichiers se fait dans un conteneur de [service](service.md).
La déclaration du fichier permet de générer un fichier à partir d'un template pour le déposer à l'endroit prévu dans la déclaration de cette élément.
Il est nécessaire, au minimum, de spécifier le chemin complet du fichier :
```
<services>
<service name="squid">
<file>/etc/squid/squid.conf</file>
</service>
</services>
```
Dans ce cas, le nom du template est déduit du nom du fichier, ici cela sera "squid.conf".
Si le template a un nom différent (par exemple si plusieurs template se retrouve avec le même nom), il est possible de changer le nom du template avec l'attribut source :
```
<file source="template-squid.conf">/etc/squid/squid.conf</file>
```
## Les noms de fichiers dynamique
Il est possible également de définir le nom du fichier dans une variable :
```
<services>
<service name="squid">
<file file_type="variable" source="squid.conf">my_variable</file>
</service>
</services>
<variables>
<variable name="my_variable">
<value>/etc/squid/squid.conf</value>
</variable>
</variables>
```
Attention, la variable doit être de type "filename".
Dans le cas des fichiers dynamique, la source est obligatoire.
Il est même possible de définir une variable de type multiple, ce qui génèrera plusiers fichiers :
```
<services>
<service name="squid">
<file file_type="variable" source="squid.conf">my_variable</file>
</service>
</services>
<variables>
<variable name="my_variable" multi="True">
<value>/etc/squid1/squid.conf</value>
<value>/etc/squid2/squid.conf</value>
</variable>
</variables>
```
Dans ce cas là, le fichier source est identique mais les fichiers de destination seront différent.
Il peut être important de personnaliser le contenu du fichier suivant le fichier de destination.
Dans ce cas il y a deux possibilités :
- la variable "rougail_filename" contient le nom de fichier de destination
- l'utilisateur de l'attribut "variable"
En effet, il est possible de passer le contenu d'une variable au template :
```
<services>
<service name="squid">
<file file_type="variable" source="squid.conf" variable="my_variable2">my_variable1</file>
</service>
</services>
<variables>
<variable name="my_variable1" multi="True">
<value>/etc/squid1/squid.conf</value>
<value>/etc/squid2/squid.conf</value>
</variable>
<variable name="my_variable2" multi="True">
<value>squid1</value>
<value>squid2</value>
</variable>
</variables>
```
Dans ce cas, lors de la génération du fichier /etc/squid1/squid.conf on retrouvera la variable "rougail_variable" avec la valeur "squid1" et le fichier /etc/squid2/squid.conf on retrouvera la variable "rougail_variable" avec la valeur "squid2".
Attention : les deux variables "my_variable1" et "my_variable2" doivent être multiple et de même longueur.
## Les droits des fichiers
Par défaut les droits du fichier généré sont "0644" avec comme utilisateur "root" et groupe "root".
Il est possible de définir une autre valeur à un ou plusieurs de ces attributs :
```
<file mode="0640" owner="nobody" group="squid">/etc/squid/squid.conf</file>
```
## Désactiver la génération d'un fichier
Il est possible de définir une [condition](../condition/README.md) de type "disabled_if_in" ou "disabled_if_not_in" sur une balise fichier :
```
<services>
<service name="test">
<file filelist="squid">/etc/squid/squid.conf</file>
</service>
</services>
<variables>
<variable name="condition" type="boolean"/>
</variables>
<constraints>
<condition name="disabled_if_in" source="condition">
<param>False</param>
<target type="filelist">squid</target>
</condition>
</constraints>
```
Dans ce cas, tous les fichiers avec un attribut filelist à "squid" seront désactivés si la variable "condition" est False.
## Redéfinir une fichier
Il est possible de redéfinir les éléments d'un fichier dans un dictionnaire différent en utilisant l'attribut redefine :
```
<file source="template-squid.conf" redefine="True">/etc/squid/squid.conf</file>
```
## Choix du moteur de templating
Par défaut, le moteur de templating est le moteur de templating compatible avec "creole".
Il est possible de désactiver la templatisation du fichier (il sera alors uniquement copié) :
```
<file engine="none">/etc/squid/squid.conf</file>
```
Ou d'utiliser le moteur "jinja2" :
```
<file engine="jinja2">/etc/squid/squid.conf</file>
```
## Inclusion de template
Un attribut "included" permet de définir la nature du fichier. Cet attribut peut avoir trois valeurs :
- "no" : c'est un fichier normal
- "name" : le répertoire de destination est listé dans un autre template, il faut que le fichier soit généré avant cet autre template
- "content" : le contenu du fichier est copié dans un autre template, il faut que le fichier soit généré avant cet autre template et ce fichier n'a pas besoin d'être installé sur le serveur cible.
Exemples :
```
<file included="name">/etc/squid/squid.conf</file>
<file included="content">/etc/squid/squid.conf</file>
```
Bien entendu, c'est au développeur de lister ou d'inclure le contenu de ce template dans le fichier de destination. Cet attribut permet juste de garantir que le fichier sera fait avant l'autre et de ne pas l'installer sur le serveur si ce n'est pas nécessaire.

47
doc/service/ip.md Normal file
View File

@ -0,0 +1,47 @@
# La gestion d'une IP
## La balise IP
La gestion des IP se fait dans un conteneur de [service](service.md).
La déclaration de l'attribut permet d'associer une IP autorisé à accéder au service.
Il est nécessaire, au minimum, de spécifier le nom d'une variable de type "IP" :
```
<ip ip_type="variable">variable_ip</ip>
```
## La gestion d'un réseau
L'adresse peut être de type réseau ("network") :
```
<ip netmask="variable_netmask">variable_ip</ip>
```
Attention, dans ce cas il faut préciser une variable de type "netmask" dans l'attribut netmask.
## Désactiver la génération d'une IP
Il est possible de définir une [condition](../condition/README.md) de type "disabled_if_in" ou "disabled_if_not_in" sur une balise IP :
```
<services>
<service name="test">
<ip iplist="test_ip">variable_ip</ip>
</service>
</services>
<variables>
<variable name="condition" type="boolean"/>
<variable name="variable_ip" type="ip"/>
</variables>
<constraints>
<condition name="disabled_if_in" source="condition">
<param>False</param>
<target type="iplist">test_ip</target>
</condition>
</constraints>
```
Dans ce cas, tous les IP avec un attribut iplist à "test_ip" seront désactivé si la variable "condition" est False.

43
doc/service/override.md Normal file
View File

@ -0,0 +1,43 @@
# Override
## La balise override
La gestion des overrides se fait dans un conteneur de [service](service.md).
La balise override permet de redéfinir facilement un service systemd.
Il suffit d'avoir un template dont le nom est par défaut le nom du service avec l'extension "service" et de déclarer la balise :
```
<services>
<service name="squid">
<override/>
</service>
</services>
```
Dans cette exemple, le template associé doit s'appeler squid.service
Si le fichier service a un nom différent (par exemple si plusieurs template se retrouve avec le même nom), il est possible de changer le nom du template avec l'attribut source :
```
<override source="test.service"/>
```
Dans ce cas le fichier de destination aura le même nom.
## Choix du moteur de templating
Par défaut, le moteur de templating est le moteur de templating compatible avec "creole".
Il est possible de désactiver la templatisation du fichier (il sera alors uniquement copié) :
```
<override engine="none"/>
```
Ou d'utiliser le moteur "jinja2" :
```
<override engine="jinja2"/>
```

50
doc/service/service.md Normal file
View File

@ -0,0 +1,50 @@
# La gestion d'un service
## La base service
Un service est inclut dans un conteneur [services](../services.md).
Cette balise permet de définir tous les éléments ([fichier](file.md), [IP](ip.md), ...) liés à un service ou à démon.
Il faut, à la création du service, préciser son nom :
```
<?xml version='1.0' encoding='UTF-8'?>
<rougail>
<services>
<service name="squid"/>
</services>
</rougail>
```
Un service peut ne pas être géré :
```
<service name="squid" manage="True"/>
```
Un service non géré est généralement une service qui n'existe pas réellement (par exemple si on configure un client).
Un service non géré ne peut conteneur que des fichiers.
## Désactiver la génération d'un service
Il est possible de définir une [condition](../condition/README.md) de type "disabled_if_in" ou "disabled_if_not_in" sur une balise service :
```
<services>
<service name="test">
</service>
</services>
<variables>
<variable name="condition" type="boolean"/>
</variables>
<constraints>
<condition name="disabled_if_in" source="condition">
<param>False</param>
<target type="servicelist">test</target>
</condition>
</constraints>
```
Dans ce cas, tous les services et les éléments qu'il compose ([fichier](file.md), ...) avec un attribut servicelist à "test" seront désactivés si la variable "condition" est False.

14
doc/services.md Normal file
View File

@ -0,0 +1,14 @@
# Le conteneur des services
La balise "services" est le conteneur de l'ensemble des [services](service/service.md).
Il est placé à la racine du dictionnaire :
```
<?xml version='1.0' encoding='UTF-8'?>
<rougail>
<services/>
</rougail>
```
Attention, cette balise ne peut pas être placé dans un dictionnaire "extra".

5
doc/target/README.md Normal file
View File

@ -0,0 +1,5 @@
# La cible
- [De type variable](../target/variable.md)
- [De type famille](../target/family.md)
- [De type \*list](../target/list.md)

13
doc/target/family.md Normal file
View File

@ -0,0 +1,13 @@
# Cible de type "variable"
Une cible peut être de type famille :
```
<target type="family">my_family</target>
```
Mais une target peut être optionnelle. C'est à dire que si la famille n'existe pas, l'action ne sera pas associé à cette famille.
```
<target type="family" optional="True">my_family</target>
```

35
doc/target/list.md Normal file
View File

@ -0,0 +1,35 @@
# Cible de type \*list
## Les différences cible de type \*.list
### servicelist
Une cible peut être de type [service](../service/service.md) :
```
<target type="servicelist">example</target>
```
### filelist
Une cible peut être de type [fichier](../service/file.md) :
```
<target type="filelist">example</target>
```
### iplist
Une cible peut être de type [ip](../service/ip.md) :
```
<target type="iplist">example</target>
```
## La cible optionnelle
Mais une target peut être optionnelle. C'est à dire que si la \*list n'existe pas, l'action ne sera pas associé.
```
<target type="filelist" optional="True">unknown</target>
```

13
doc/target/variable.md Normal file
View File

@ -0,0 +1,13 @@
# Cible de type "variable"
Par défaut une cible est de type variable.
```
<target>my_variable</target>
```
Mais une target peut être optionnelle. C'est à dire que si la variable n'existe pas, l'action ne sera pas associé à cette variable.
```
<target optional='True'>my_variable</target>
```

13
doc/variable.rst
View File

@ -1,13 +0,0 @@
Variable
========
Variable obligatoire
--------------------
Variable dont une valeur est requise :
<variables>
<family name="family">
<variable name="my_variable" type="oui/non" description="My variable" mandatory="True"/>
</family>
</variables>

4
doc/variable/README.md Normal file
View File

@ -0,0 +1,4 @@
# Variable
- [Une variable](simple.md)
- [Variable meneuse ou suiveuse](leadership.md)

102
doc/variable/leadership.md Normal file
View File

@ -0,0 +1,102 @@
# Variable meneuse ou suiveuse
## Variable meneuse
Une variable meneuse est une variable qui va guider la longueur d'autre variables (appelé variables suiveuse).
Une variable meneuse est une [variable](simple.md) qui est obligatoirement de type multiple.
Une variable meneuse peut être obligatoire.
Le [mode](../mode.md) par défaut correspond au plus petit mode définit par l'utilisateur des variables suiveuses.
## Variable suiveuse
Une variable suiveuse est une variable donc la longueur n'est pas déterminé par elle-même, mais est identique à celle de la variable meneuse dont elle dépend.
Une variable suiveuse est une variable placer juste derrière une variable meneuse ou une autre variable suiveuse.
L'ordre de définition des variables suiveuses est important.
Cette variable peut être de type multiple. Dans ce cas, pour un index determiné, il est possible de mettre plusieurs valeurs à une même variable.
Une variable suiveuse peut être obligatoire. Cela signifie que lorsqu'une variable meneuse est renseigné, il faut obligatoirement que la variable suiveuse est également une valeur à l'index considéré.
Si aucune valeur n'est définit pour la variable meneuse, aucune valeur n'est a spécifié pour la variable suiveuse.
Le [mode](../mode.md) par défaut d'une variable suiveuse correspond au [mode](../mode.md) de la variable meneuse.
Si une variable meneuse est caché ou désactivé, les variables suiveuses le seront également.
## Définition des variables meneuse et suiveuse
Voici un exemple de définition d'une variable meneuse et de deux variables meneuses :
```
<variables>
<family name="family">
<variable name="leader" multi='True'/>
<variable name="follower1"/>
<variable name="follower2" multi='True'/>
</family>
</variables>
<constraints>
<group leader="leader">
<follower>follower1</follower>
<follower>follower2</follower>
</group>
</constraints>
```
Le nom et la description du groupe prendra par défaut le nom et la description de la variable meneuse.
Il est possible d'en définit d'autres :
```
<constraints>
<group leader="leader" name="leadership" description="My leadership">
<follower>follower1</follower>
<follower>follower2</follower>
</group>
</constraints>
```
## Définition des variables meneuse et suiveuse dans un dictionnaire extra
Voici un exemple de définition d'une variable meneuse et de deux variables meneuses dans un espace de nom "example" :
```
<variables>
<family name="family">
<variable name="leader" multi='True'/>
<variable name="follower1"/>
<variable name="follower2" multi='True'/>
</family>
</variables>
<constraints>
<group leader="example.family.leader">
<follower>example.family.follower1</follower>
<follower>example.family.follower2</follower>
</group>
</constraints>
```
Le chemin de la variable meneuse est traditionnel, par contre le chemin des variables suiveuses n'est pas le chemin définitif de la variable.
Le chemin d'une variable suiveuse est normalement "example.family.leader.follower1" mais la variable n'est pas encore une variable suiveuse à ce moment là du traitement. C'est pour cela qu'il ne faut, uniquement dans les groupes, mettre le nom de la variable meneuse dans le chemin.
## Ajout d'une nouvelle variable suiveuse
Pour ajouter, dans un nouveau dictionnaire, une variable suiveuse à notre groupe, rien de plus simple, il suffit de redéfinir un groupe avec cette (ou ces) nouvelle variable :
```
<variables>
<family name="family">
<variable name="follower3"/>
</family>
</variables>
<constraints>
<group leader="leader">
<follower>follower3</follower>
</group>
</constraints>
```

255
doc/variable/simple.md Normal file
View File

@ -0,0 +1,255 @@
# Variable
## Un variable
Une variable est forcement dans [variables](../variables.md) ou dans une [famille](../family/README.md).
Une variable est déjà un nom. C'est à dire qu'on pourra utiliser plus tard la variable via ce nom.
```
<variables>
<variable name="my_variable"/>
<family name="my_family">
<variable name="my_family_variable"/>
</variable>
</variables>
```
## Description et aide sur la variable
En plus d'un nom, il est possible de mettre une "description" à la variable. C'est une information "utilisateur" qui nous permettra d'avoir des informations complémentaires sur le contenu de cette variable :
```
<variable name="my_variable" description="This is a greate variable"/>
```
En plus de la description, il est possible de préciser une aide complémentaire :
```
<variable name="my_variable" help="This is a greate variable"/>
```
Cette aide peut être utilisé à tout moment comme valeur [d'un paramètre](../param/information.md).
## Le type de la variable
Une variable a un type. Ce type permet de définir les valeurs acceptées par cette variable :
- string : chaine de caractère (type par défaut)
- number : un nombre
- float : un chiffre flottant
- boolean : "True" ou "False", si aucune valeur n'est défini la valeur par défaut de cette variable sera "True", ces variables sont également obligatoire par défaut
- password : un mot de passe
- mail : une adresse mail
- filename : nom de fichier au sens Unix (exemple : "/etc/passwd")
- date : une date au format "%Y-%m-%d" (exemple : "2021-01-30")
- unix_user : nom d'utilisateur au sens Unix
- ip : n'importe quelle adresse IPv4
- cidr : n'importe quelle adresse IPv4 au format CIDR
- local_ip : adresse IPv4 sur un réseau local, si l'adresse IPv4 n'est pas local, un warning sera afficher mais la valeur sera accepté tout de même
- netmask : masque d'une adresse IPv4
- network : adresse réseau
- network_cidr : adresse réseau au format CIDR
- broadcast : adresse de diffusion
- netbios : nom netbios
- domain : nom de domaine
- hostname : nom d'hôte
- web_address : adresse web (http://www.cadoles.com/)
- port : port
- mac : adresse MAC
- schedule : périodicité du schedule, les valeurs possibles sont "none", "daily", "weekly" ou "monthly"
- schedulemod : type de schedule, les valeurs possibles sont "pre" ou "post"
Pour définir le type d'une variable :
```
<variable name="my_variable" type="number"/>
```
## Variable à valeur multiple
Par défaut une variable ne peut acceuillir qu'une seule valeur. Il peut être utile de pouvoir spécifier plusieurs valeurs à une même variable.
Pour définir une variable à valeur multiple :
```
<variable name="my_variable" multi="True"/>
```
## Variable invisible
Il est possible de cacher une variable.
Cacher une variable signifie qu'elle ne sera pas visible lorsqu'on modifie la configuration du service.
Par contre cette variable sera accessibles lorsqu'on va l'utiliser.
Pour cacher une variable :
```
<variable name="my_variable" hidden="True"/>
```
## Variable désactive
Il est possible de désactiver une variable.
Désactiver une variable signifie qu'elle ne sera pas visible lorsqu'on modifie la configuration du service mais également lorsqu'on va l'utiliser.
Pour désactiver une variable :
```
<variable name="my_variable" disabled="True"/>
```
## Variable obligatoire
Variable dont une valeur est requise :
```
<variable name="my_variable" mandatory="True"/>
```
Les variables booléans sont par défaut obligatoire. Pour qu'une variable booléan ne soit pas obligatoire il faut le préciser explicitement :
```
<variable name="my_variable" type="boolean" mandatory="False"/>
```
Les variables avec une valeur par défaut (non calculée) sont également automatiquement obligatoire.
[Les variables à choix](../check/valid_enum.md) sans choix "None" sont également automatiquement obligatoire.
## Valeur par défaut d'une variable
Il est possible de fixer les valeurs par défaut d'une variable :
```
<variable name="my_variable">
<value>value</value>
</variable>
```
Pour une variable multiple, il est possible de préciser plusieurs valeurs :
```
<variable name="my_variable" multi="True">
<value>value 1</value>
<value>value 2</value>
</variable>
```
Si la variable n'est pas pas une [variable meneuse](leadership.md), la première valeur défini dans cette liste sera également la valeur par défaut proposé si on ajoute une nouvelle valeur à cette variable.
Une valeur par défaut peut également être [une valeur calculer](../fill/README.md).
## Redéfinir une variable
Il est possible de définir une variable dans un dictionnaire et de changer son comportement dans une second dictionnaire.
Attention trois attributs ne sont redéfinisable :
- name
- type
- multi
Créons notre variable :
<variable name="my_variable"/>
Et redéfinisons là :
```
<variable name="my_variable" redefine="True" description="New description"/>
```
## Créer une variable inexistante
Il est parfois utile de créer une variable si elle n'existe pas dans un autre dictionnaire :
```
<variable name="my_variable" exists="False"/>
```
Si cette variable existe dans un autre dictionnaire, elle ne sera pas modifié ni recréé
## Redéfinir une variable si elle existe
Parfois on veut pouvoir redéfinir une variable mais seulement dans le cas où elle existe déjà :
```
<variable name="my_variable" redefine="True" exists="True" hidden="True"/>
```
## Variable à valeur automatiquement modifiée
Une variable avec valeur automatiquement modifiée est une variable dont la valeur sera considéré comme modifié quand la propriété global "force_store_value" de Tiramisu est mise.
Voici une variable a valeur automatiquement modifiée :
```
<variable name="my_variable" auto_save="True">
<value>my_value</value>
</variable>
```
Dans ce cas la valeur est fixée à la valeur actuelle.
Par exemple, si la valeur de cette variable est issue d'un calcul, la valeur ne sera plus recalculée.
Ces variables sont généralement des variables obligatoires. En effet ces variable ne sont automatiquement modifiées que si elles ont une valeurs.
Une [variable meneuse ou suiveuse](leadership.md) ne peut pas avoir la propriété auto_save.
## Variable à valeur en lecture seule automatique
Une variable avec valeur en lecture seule automatique est une variable dont la valeur ne sera plus modifiable par l'utilisateur quand la [variable "server_deployed" passe à "True"](../dev/config.md).
Voici un variable à valeur en lecture seule automatique :
```
<variable name="server_deployed" type="boolean">
<value>False</value>
</variable>
<variable name="my_variable" auto_freeze="True"/>
```
Dans ce cas la valeur est fixée à la valeur actuelle et elle ne sera plus modifiable par l'utilisateur.
Par exemple, si la valeur de cette variable est issue d'un calcul, la valeur ne sera plus recalculée.
Ces variables sont généralement des variables obligatoires. En effet ces variable ne sont en lecteur seul que si elles ont une valeurs.
Une [variable meneuse ou suiveuse](leadership.md) ne peut pas avoir la propriété auto_freeze.
## Information "test"
L'attribut "test" est un attribut spécial qui permet aux concepteurs d'un dictionnaire d'influancer le robot de test en précisant de valeurs utile à tester.
Concrêtement, le contenu de cet attribut est enregister dans une "information" de l'option Tiramisu correspondante.
Exemple :
```
<variable name="my_variable" test="yes"/>
```
Il est possible de préciser plusieurs valeurs avec le séparateur "|" :
```
<variable name="my_variable" test="yes|no"/>
```
Cette valeur peut être utilisé à tout moment comme valeur [d'un paramètre](../param/information.md).
## Mode de la variable
Le [mode](../mode.md) par défaut d'une variable correspond au [mode](../mode.md) de la [famille](../family/README.md).
Cas particuliers :
- une variable à valeur automatiquement modifiée ou une variable en lecture seule automatique est par défaut en mode "basic".
- si la variable n'est pas dans une famille, la variable aura le mode "normal" par défaut.
- une variable obligatoire sans valeur par défaut (calculer ou non) aura le mode "basic".
Pour définir le [mode](../mode.md) :
```
<variable name="my_variable" mode="expert"/>
```

12
doc/variables.md Normal file
View File

@ -0,0 +1,12 @@
# Le conteneur des variables
La balise "variables" est le conteneur de l'ensemble des [familles](family/README.md) et des [variables](variable/README.md).
Il est placé à la racine du dictionnaire :
```
<?xml version='1.0' encoding='UTF-8'?>
<rougail>
<variables/>
</rougail>
```

35
src/rougail/__init__.py
View File

@ -1,5 +1,32 @@
#from .loader import load """Rougail method
from .objspace import CreoleObjSpace
from .annotator import modes
__ALL__ = ('CreoleObjSpace', 'modes') Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from .convert import RougailConvert
from .template.systemd import RougailSystemdTemplate
from .config import RougailConfig
from rougail.update import RougailUpgrade
__ALL__ = ('RougailConvert', 'RougailSystemdTemplate', 'RougailConfig', 'RougailUpgrade')

1293
src/rougail/annotator.py
View File

@ -1,1293 +0,0 @@
# coding: utf-8
from copy import copy
from typing import List
from collections import OrderedDict
from os.path import join, basename
from ast import literal_eval
import imp
from .i18n import _
from .utils import normalize_family
from .error import DictConsistencyError
#mode order is important
modes_level = ('basic', 'normal', 'expert')
class Mode(object):
def __init__(self, name, level):
self.name = name
self.level = level
def __gt__(self, other):
return other.level < self.level
def mode_factory():
mode_obj = {}
for idx in range(len(modes_level)):
name = modes_level[idx]
mode_obj[name] = Mode(name, idx)
return mode_obj
modes = mode_factory()
CONVERT_OPTION = {'number': dict(opttype="IntOption", func=int),
'float': dict(opttype="FloatOption", func=float),
'choice': dict(opttype="ChoiceOption"),
'string': dict(opttype="StrOption"),
'password': dict(opttype="PasswordOption"),
'mail': dict(opttype="EmailOption"),
'boolean': dict(opttype="BoolOption"),
'symlink': dict(opttype="SymLinkOption"),
'filename': dict(opttype="FilenameOption"),
'date': dict(opttype="DateOption"),
'unix_user': dict(opttype="UsernameOption"),
'ip': dict(opttype="IPOption", initkwargs={'allow_reserved': True}),
'local_ip': dict(opttype="IPOption", initkwargs={'private_only': True, 'warnings_only': True}),
'netmask': dict(opttype="NetmaskOption"),
'network': dict(opttype="NetworkOption"),
'broadcast': dict(opttype="BroadcastOption"),
'netbios': dict(opttype="DomainnameOption", initkwargs={'type': 'netbios', 'warnings_only': True}),
'domain': dict(opttype="DomainnameOption", initkwargs={'type': 'domainname', 'allow_ip': False}),
'hostname': dict(opttype="DomainnameOption", initkwargs={'type': 'hostname', 'allow_ip': False}),
'web_address': dict(opttype="URLOption", initkwargs={'allow_ip': True, 'allow_without_dot': True}),
'port': dict(opttype="PortOption", initkwargs={'allow_private': True}),
'mac': dict(opttype="MACOption"),
'cidr': dict(opttype="IPOption", initkwargs={'cidr': True}),
'network_cidr': dict(opttype="NetworkOption", initkwargs={'cidr': True}),
}
# a CreoleObjSpace's attribute has some annotations
# that shall not be present in the exported (flatened) XML
ERASED_ATTRIBUTES = ('redefine', 'exists', 'fallback', 'optional', 'remove_check', 'namespace',
'remove_condition', 'path', 'instance_mode', 'index', 'is_in_leadership',
'level', 'remove_fill', 'xmlfiles')
ERASED_CONTAINER_ATTRIBUTES = ('id', 'container', 'group_id', 'group', 'container_group')
FORCE_CHOICE = {'oui/non': ['oui', 'non'],
'on/off': ['on', 'off'],
'yes/no': ['yes', 'no'],
'schedule': ['none', 'daily', 'weekly', 'monthly'],
'schedulemod': ['pre', 'post']}
KEY_TYPE = {'variable': 'symlink',
'SymLinkOption': 'symlink',
'PortOption': 'port',
'UnicodeOption': 'string',
'NetworkOption': 'network',
'NetmaskOption': 'netmask',
'URLOption': 'web_address',
'FilenameOption': 'filename'}
FREEZE_AUTOFREEZE_VARIABLE = 'module_instancie'
PROPERTIES = ('hidden', 'frozen', 'auto_freeze', 'auto_save', 'force_default_on_freeze',
'force_store_value', 'disabled', 'mandatory')
CONVERT_PROPERTIES = {'auto_save': ['force_store_value'], 'auto_freeze': ['force_store_value', 'auto_freeze']}
RENAME_ATTIBUTES = {'description': 'doc'}
INTERNAL_FUNCTIONS = ['valid_enum', 'valid_in_network', 'valid_differ', 'valid_entier']
class SpaceAnnotator:
"""Transformations applied on a CreoleObjSpace instance
"""
def __init__(self, objectspace, eosfunc_file):
self.objectspace = objectspace
GroupAnnotator(objectspace)
ServiceAnnotator(objectspace)
VariableAnnotator(objectspace)
ConstraintAnnotator(objectspace,
eosfunc_file,
)
FamilyAnnotator(objectspace)
PropertyAnnotator(objectspace)
class GroupAnnotator:
def __init__(self,
objectspace,
):
self.objectspace = objectspace
if not hasattr(self.objectspace.space, 'constraints') or not hasattr(self.objectspace.space.constraints, 'group'):
return
self.convert_groups()
def convert_groups(self): # pylint: disable=C0111
for group in self.objectspace.space.constraints.group:
leader_fullname = group.leader
leader_family_name = self.objectspace.paths.get_variable_family_name(leader_fullname)
leader_name = self.objectspace.paths.get_variable_name(leader_fullname)
namespace = self.objectspace.paths.get_variable_namespace(leader_fullname)
if '.' not in leader_fullname:
leader_fullname = '.'.join([namespace, leader_family_name, leader_fullname])
follower_names = list(group.follower.keys())
has_a_leader = False
ori_leader_family = self.objectspace.paths.get_family_obj(leader_fullname.rsplit('.', 1)[0])
for variable in list(ori_leader_family.variable.values()):
if has_a_leader:
# it's a follower
self.manage_follower(namespace,
leader_family_name,
variable,
leadership_name,
follower_names,
leader_space,
leader_is_hidden,
)
ori_leader_family.variable.pop(variable.name)
if follower_names == []:
# no more follower
break
elif variable.name == leader_name:
# it's a leader
if isinstance(variable, self.objectspace.Leadership):
# append follower to an existed leadership
leader_space = variable
# if variable.hidden:
# leader_is_hidden = True
else:
leader_space = self.objectspace.Leadership(variable.xmlfiles)
if hasattr(group, 'name'):
leadership_name = group.name
else:
leadership_name = leader_name
leader_is_hidden = self.manage_leader(leader_space,
leader_family_name,
leadership_name,
leader_name,
namespace,
variable,
group,
leader_fullname,
)
has_a_leader = True
else:
raise DictConsistencyError(_('cannot found followers {}').format(follower_names))
del self.objectspace.space.constraints.group
def manage_leader(self,
leader_space: 'Leadership',
leader_family_name: str,
leadership_name: str,
leader_name: str,
namespace: str,
variable: 'Variable',
group: 'Group',
leader_fullname: str,
) -> None:
# manage leader's variable
if variable.multi is not True:
raise DictConsistencyError(_('the variable {} in a group must be multi').format(variable.name))
leader_space.variable = []
leader_space.name = leadership_name
leader_space.hidden = variable.hidden
if variable.hidden:
leader_is_hidden = True
variable.frozen = True
variable.force_default_on_freeze = True
else:
leader_is_hidden = False
variable.hidden = None
if hasattr(group, 'description'):
leader_space.doc = group.description
elif hasattr(variable, 'description'):
leader_space.doc = variable.description
else:
leader_space.doc = variable.name
leadership_path = namespace + '.' + leader_family_name + '.' + leadership_name
self.objectspace.paths.add_family(namespace,
leadership_path,
leader_space,
)
leader_family = self.objectspace.space.variables[namespace].family[leader_family_name]
leader_family.variable[leader_name] = leader_space
leader_space.variable.append(variable)
self.objectspace.paths.set_leader(namespace,
leader_family_name,
leader_name,
leader_name,
)
leader_space.path = leader_fullname
return leader_is_hidden
def manage_follower(self,
namespace: str,
leader_family_name: str,
variable: 'Variable',
leader_name: str,
follower_names: List[str],
leader_space: 'Leadership',
leader_is_hidden: bool,
) -> None:
if variable.name != follower_names[0]:
raise DictConsistencyError(_('cannot found this follower {}').format(follower_names[0]))
follower_names.remove(variable.name)
if leader_is_hidden:
variable.frozen = True
variable.force_default_on_freeze = True
leader_space.variable.append(variable) # pylint: disable=E1101
self.objectspace.paths.set_leader(namespace,
leader_family_name,
variable.name,
leader_name,
)
class ServiceAnnotator:
"""Manage service's object
for example::
<services>
<service name="test">
<service_access service='ntp'>
<port protocol='udp' service_accesslist='ntp_udp'>123</port>
</service_access>
</service>
</services>
"""
def __init__(self, objectspace):
self.objectspace = objectspace
self.convert_services()
def convert_services(self):
if not hasattr(self.objectspace.space, 'services'):
return
if not hasattr(self.objectspace.space.services, 'service'):
del self.objectspace.space.services
return
self.objectspace.space.services.hidden = True
self.objectspace.space.services.name = 'services'
self.objectspace.space.services.doc = 'services'
families = {}
for idx, service_name in enumerate(self.objectspace.space.services.service.keys()):
service = self.objectspace.space.services.service[service_name]
new_service = self.objectspace.service(service.xmlfiles)
for elttype, values in vars(service).items():
if not isinstance(values, (dict, list)) or elttype in ERASED_ATTRIBUTES:
setattr(new_service, elttype, values)
continue
eltname = elttype + 's'
path = '.'.join(['services', service_name, eltname])
family = self.gen_family(eltname,
path,
service.xmlfiles,
)
if isinstance(values, dict):
values = list(values.values())
family.family = self.make_group_from_elts(service_name,
elttype,
values,
path,
)
setattr(new_service, elttype, family)
new_service.doc = new_service.name
families[service_name] = new_service
self.objectspace.space.services.service = families
def gen_family(self,
name,
path,
xmlfiles
):
family = self.objectspace.family(xmlfiles)
family.name = normalize_family(name)
family.doc = name
family.mode = None
self.objectspace.paths.add_family('services',
path,
family,
)
return family
def make_group_from_elts(self,
service_name,
name,
elts,
path,
):
"""Splits each objects into a group (and `OptionDescription`, in tiramisu terms)
and build elements and its attributes (the `Options` in tiramisu terms)
"""
families = []
new_elts = self._reorder_elts(name,
elts,
)
for index, elt_info in enumerate(new_elts):
elt = elt_info['elt']
elt_name = elt_info['elt_name']
# try to launch _update_xxxx() function
update_elt = '_update_' + elt_name
if hasattr(self, update_elt):
getattr(self, update_elt)(elt,
index,
path,
service_name,
)
idx = 0
while True:
if hasattr(elt, 'source'):
c_name = elt.source
else:
c_name = elt.name
if idx:
c_name += f'_{idx}'
subpath = '{}.{}'.format(path, c_name)
if not self.objectspace.paths.family_is_defined(subpath):
break
idx += 1
family = self.gen_family(c_name,
subpath,
elt.xmlfiles,
)
family.variable = []
listname = '{}list'.format(name)
activate_path = '.'.join([subpath, 'activate'])
for key in dir(elt):
if key.startswith('_') or key.endswith('_type') or key in ERASED_ATTRIBUTES:
continue
value = getattr(elt, key)
if key == listname:
self.objectspace.list_conditions.setdefault(listname,
{}).setdefault(
value,
[]).append(activate_path)
continue
family.variable.append(self._generate_element(elt_name,
key,
value,
elt,
f'{subpath}.{key}'
))
# FIXME ne devrait pas etre True par défaut
# devrait etre un calcule
family.variable.append(self._generate_element(elt_name,
'activate',
True,
elt,
activate_path,
))
families.append(family)
return families
def _generate_element(self,
elt_name,
key,
value,
elt,
path,
):
variable = self.objectspace.variable(elt.xmlfiles)
variable.name = normalize_family(key)
variable.mode = None
if key == 'name':
true_key = elt_name
else:
true_key = key
dtd_key_type = true_key + '_type'
if key == 'activate':
type_ = 'boolean'
elif hasattr(elt, dtd_key_type):
type_ = KEY_TYPE[getattr(elt, dtd_key_type)]
elif key in self.objectspace.booleans_attributs:
type_ = 'boolean'
else:
type_ = 'string'
variable.type = type_
if type_ == 'symlink':
variable.opt = self.objectspace.paths.get_variable_path(value,
'services',
)
# variable.opt = value
variable.multi = None
else:
variable.doc = key
val = self.objectspace.value(elt.xmlfiles)
val.type = type_
val.name = value
variable.value = [val]
self.objectspace.paths.add_variable('services',
path,
'service',
False,
variable,
)
return variable
def _reorder_elts(self,
name,
elts,
):
"""Reorders by index the elts
"""
new_elts = {}
# reorder elts by index
for idx, elt in enumerate(elts):
new_elts.setdefault(idx, []).append(elt)
idxes = list(new_elts.keys())
idxes.sort()
result_elts = []
for idx in idxes:
for elt in new_elts[idx]:
result_elts.append({'elt_name': name, 'elt': elt})
return result_elts
def _update_override(self,
file_,
index,
service_path,
service_name,
):
file_.name = f'/systemd/system/{service_name}.service.d/rougail.conf'
# retrieve default value from File object
for attr in ['owner', 'group', 'mode']:
setattr(file_, attr, getattr(self.objectspace.file, attr))
if not hasattr(file_, 'source'):
file_.source = f'{service_name}.service'
self._update_file(file_,
index,
service_path,
service_name,
)
def _update_file(self,
file_,
index,
service_path,
service_name,
):
if not hasattr(file_, 'file_type') or file_.file_type == "UnicodeOption":
if not hasattr(file_, 'source'):
file_.source = basename(file_.name)
elif not hasattr(file_, 'source'):
raise DictConsistencyError(_('attribute source mandatory for file with variable name '
'for {}').format(file_.name))
class VariableAnnotator:
def __init__(self,
objectspace,
):
self.objectspace = objectspace
self.convert_variable()
self.convert_helps()
self.convert_auto_freeze()
self.convert_separators()
def convert_variable(self):
def _convert_variable(variable,
variable_type,
):
if not hasattr(variable, 'type'):
variable.type = 'string'
if variable.type != 'symlink' and not hasattr(variable, 'description'):
variable.description = variable.name
if hasattr(variable, 'value'):
for value in variable.value:
if not hasattr(value, 'type'):
value.type = variable.type
value.name = CONVERT_OPTION.get(value.type, {}).get('func', str)(value.name)
for key, value in RENAME_ATTIBUTES.items():
setattr(variable, value, getattr(variable, key))
setattr(variable, key, None)
if variable_type == 'follower':
if variable.multi is True:
variable.multi = 'submulti'
else:
variable.multi = True
def _convert_valid_enum(namespace,
variable,
path,
):
if variable.type in FORCE_CHOICE:
check = self.objectspace.check(variable.xmlfiles)
check.name = 'valid_enum'
check.target = path
check.namespace = namespace
check.param = []
for value in FORCE_CHOICE[variable.type]:
param = self.objectspace.param(variable.xmlfiles)
param.text = value
check.param.append(param)
if not hasattr(self.objectspace.space, 'constraints'):
self.objectspace.space.constraints = self.objectspace.constraints(variable.xmlfiles)
self.objectspace.space.constraints.namespace = namespace
if not hasattr(self.objectspace.space.constraints, 'check'):
self.objectspace.space.constraints.check = []
self.objectspace.space.constraints.check.append(check)
variable.type = 'string'
def _valid_type(variable):
if variable.type not in CONVERT_OPTION:
xmlfiles = self.objectspace.display_xmlfiles(variable.xmlfiles)
raise DictConsistencyError(_(f'unvalid type "{variable.type}" for variable "{variable.name}" in {xmlfiles}'))
if not hasattr(self.objectspace.space, 'variables'):
return
for families in self.objectspace.space.variables.values():
namespace = families.name
if hasattr(families, 'family'):
families.doc = families.name
for family in families.family.values():
family.doc = family.name
for key, value in RENAME_ATTIBUTES.items():
if hasattr(family, key):
setattr(family, value, getattr(family, key))
setattr(family, key, None)
family.name = normalize_family(family.name)
if hasattr(family, 'variable'):
for variable in family.variable.values():
if isinstance(variable, self.objectspace.Leadership):
for idx, follower in enumerate(variable.variable):
if idx == 0:
variable_type = 'master'
else:
variable_type = 'follower'
path = '{}.{}.{}.{}'.format(namespace, normalize_family(family.name), variable.name, follower.name)
_convert_variable(follower,
variable_type,
)
_convert_valid_enum(namespace,
follower,
path,
)
_valid_type(follower)
else:
path = '{}.{}.{}'.format(namespace, normalize_family(family.name), variable.name)
_convert_variable(variable,
'variable',
)
_convert_valid_enum(namespace,
variable,
path,
)
_valid_type(variable)
def convert_helps(self):
if not hasattr(self.objectspace.space, 'help'):
return
helps = self.objectspace.space.help
if hasattr(helps, 'variable'):
for hlp in helps.variable.values():
variable = self.objectspace.paths.get_variable_obj(hlp.name)
variable.help = hlp.text
if hasattr(helps, 'family'):
for hlp in helps.family.values():
variable = self.objectspace.paths.get_family_obj(hlp.name)
variable.help = hlp.text
del self.objectspace.space.help
def convert_auto_freeze(self): # pylint: disable=C0111
def _convert_auto_freeze(variable, namespace):
if variable.auto_freeze:
new_condition = self.objectspace.condition(variable.xmlfiles)
new_condition.name = 'auto_hidden_if_not_in'
new_condition.namespace = namespace
new_condition.source = FREEZE_AUTOFREEZE_VARIABLE
new_param = self.objectspace.param(variable.xmlfiles)
new_param.text = 'oui'
new_condition.param = [new_param]
new_target = self.objectspace.target(variable.xmlfiles)
new_target.type = 'variable'
path = variable.namespace + '.' + normalize_family(family.name) + '.' + variable.name
new_target.name = path
new_condition.target = [new_target]
if not hasattr(self.objectspace.space.constraints, 'condition'):
self.objectspace.space.constraints.condition = []
self.objectspace.space.constraints.condition.append(new_condition)
if hasattr(self.objectspace.space, 'variables'):
for variables in self.objectspace.space.variables.values():
if hasattr(variables, 'family'):
namespace = variables.name
for family in variables.family.values():
if hasattr(family, 'variable'):
for variable in family.variable.values():
if isinstance(variable, self.objectspace.Leadership):
for follower in variable.variable:
_convert_auto_freeze(follower, namespace)
else:
_convert_auto_freeze(variable, namespace)
def convert_separators(self): # pylint: disable=C0111,R0201
if not hasattr(self.objectspace.space, 'variables'):
return
for family in self.objectspace.space.variables.values():
if not hasattr(family, 'separators'):
continue
if hasattr(family.separators, 'separator'):
for idx, separator in enumerate(family.separators.separator):
option = self.objectspace.paths.get_variable_obj(separator.name)
if hasattr(option, 'separator'):
subpath = self.objectspace.paths.get_variable_path(separator.name,
separator.namespace,
)
xmlfiles = self.objectspace.display_xmlfiles(separator.xmlfiles)
raise DictConsistencyError(_(f'{subpath} already has a separator in {xmlfiles}'))
option.separator = separator.text
del family.separators
class ConstraintAnnotator:
def __init__(self,
objectspace,
eosfunc_file,
):
if not hasattr(objectspace.space, 'constraints'):
return
self.objectspace = objectspace
eosfunc = imp.load_source('eosfunc', eosfunc_file)
self.functions = dir(eosfunc)
self.functions.extend(INTERNAL_FUNCTIONS)
self.valid_enums = {}
if hasattr(self.objectspace.space.constraints, 'check'):
self.check_check()
self.check_replace_text()
self.check_valid_enum()
self.check_change_warning()
self.convert_check()
if hasattr(self.objectspace.space.constraints, 'condition'):
self.check_params_target()
self.filter_targets()
self.convert_xxxlist_to_variable()
self.check_condition_fallback_optional()
self.check_choice_option_condition()
self.remove_condition_with_empty_target()
self.convert_condition()
if hasattr(self.objectspace.space.constraints, 'fill'):
self.convert_fill()
self.remove_constraints()
def check_check(self):
remove_indexes = []
for check_idx, check in enumerate(self.objectspace.space.constraints.check):
if not check.name in self.functions:
xmlfiles = self.objectspace.display_xmlfiles(check.xmlfiles)
raise DictConsistencyError(_(f'cannot find check function "{check.name}" in {xmlfiles}'))
if hasattr(check, 'param'):
param_option_indexes = []
for idx, param in enumerate(check.param):
if param.type == 'variable' and not self.objectspace.paths.path_is_defined(param.text):
if param.optional is True:
param_option_indexes.append(idx)
else:
xmlfiles = self.objectspace.display_xmlfiles(check.xmlfiles)
raise DictConsistencyError(_(f'cannot find check param "{param.text}" in {xmlfiles}'))
if param.type != 'variable':
param.notraisepropertyerror = None
param_option_indexes = list(set(param_option_indexes))
param_option_indexes.sort(reverse=True)
for idx in param_option_indexes:
check.param.pop(idx)
if check.param == []:
remove_indexes.append(check_idx)
remove_indexes.sort(reverse=True)
for idx in remove_indexes:
del self.objectspace.space.constraints.check[idx]
def check_replace_text(self):
for check_idx, check in enumerate(self.objectspace.space.constraints.check):
namespace = check.namespace
if hasattr(check, 'param'):
for idx, param in enumerate(check.param):
if param.type == 'variable':
param.text = self.objectspace.paths.get_variable_path(param.text, namespace)
check.is_in_leadership = self.objectspace.paths.get_leader(check.target) != None
# let's replace the target by the path
check.target = self.objectspace.paths.get_variable_path(check.target, namespace)
def check_valid_enum(self):
remove_indexes = []
for idx, check in enumerate(self.objectspace.space.constraints.check):
if check.name == 'valid_enum':
if check.target in self.valid_enums:
raise DictConsistencyError(_(f'valid_enum already set for {check.target}'))
if not hasattr(check, 'param'):
raise DictConsistencyError(_(f'param is mandatory for a valid_enum of variable {check.target}'))
variable = self.objectspace.paths.get_variable_obj(check.target)
values = self.load_params_in_valid_enum(check.param,
variable.name,
variable.type,
)
self._set_valid_enum(variable,
values,
variable.type,
check.target
)
remove_indexes.append(idx)
remove_indexes.sort(reverse=True)
for idx in remove_indexes:
del self.objectspace.space.constraints.check[idx]
def load_params_in_valid_enum(self,
params,
variable_name,
variable_type,
):
has_variable = None
values = []
for param in params:
if param.type == 'variable':
if has_variable is not None:
raise DictConsistencyError(_(f'only one "variable" parameter is allowed for valid_enum of variable {variable_name}'))
has_variable = True
variable = self.objectspace.paths.get_variable_obj(param.text)
if not variable.multi:
raise DictConsistencyError(_(f'only multi "variable" parameter is allowed for valid_enum of variable {variable_name}'))
values = param.text
else:
if has_variable:
raise DictConsistencyError(_(f'only one "variable" parameter is allowed for valid_enum of variable {variable_name}'))
if not hasattr(param, 'text'):
if param.type == 'number':
raise DictConsistencyError(_(f'value is mandatory for valid_enum of variable {variable_name}'))
values.append(None)
else:
values.append(param.text)
return values
def check_change_warning(self):
#convert level to "warnings_only"
for check in self.objectspace.space.constraints.check:
if check.level == 'warning':
check.warnings_only = True
else:
check.warnings_only = False
check.level = None
def _get_family_variables_from_target(self,
target,
):
if target.type == 'variable':
variable = self.objectspace.paths.get_variable_obj(target.name)
family = self.objectspace.paths.get_family_obj(target.name.rsplit('.', 1)[0])
if isinstance(family, self.objectspace.Leadership) and family.name == variable.name:
return family, family.variable
return variable, [variable]
# it's a family
variable = self.objectspace.paths.get_family_obj(target.name)
return variable, list(variable.variable.values())
def check_params_target(self):
for condition in self.objectspace.space.constraints.condition:
if not hasattr(condition, 'target'):
raise DictConsistencyError(_('target is mandatory in condition'))
for target in condition.target:
if target.type.endswith('list') and condition.name not in ['disabled_if_in', 'disabled_if_not_in']:
raise DictConsistencyError(_(f'target in condition for {target.type} not allow in {condition.name}'))
def filter_targets(self): # pylint: disable=C0111
for condition_idx, condition in enumerate(self.objectspace.space.constraints.condition):
namespace = condition.namespace
for idx, target in enumerate(condition.target):
if target.type == 'variable':
if condition.source == target.name:
raise DictConsistencyError(_('target name and source name must be different: {}').format(condition.source))
try:
target.name = self.objectspace.paths.get_variable_path(target.name, namespace)
except DictConsistencyError:
# for optional variable
pass
elif target.type == 'family':
try:
target.name = self.objectspace.paths.get_family_path(target.name, namespace)
except KeyError:
raise DictConsistencyError(_('cannot found family {}').format(target.name))
def convert_xxxlist_to_variable(self): # pylint: disable=C0111
# transform *list to variable or family
for condition_idx, condition in enumerate(self.objectspace.space.constraints.condition):
new_targets = []
remove_targets = []
for target_idx, target in enumerate(condition.target):
if target.type.endswith('list'):
listname = target.type
listvars = self.objectspace.list_conditions.get(listname,
{}).get(target.name)
if listvars:
for listvar in listvars:
variable = self.objectspace.paths.get_variable_obj(listvar)
type_ = 'variable'
new_target = self.objectspace.target(variable.xmlfiles)
new_target.type = type_
new_target.name = listvar
new_target.index = target.index
new_targets.append(new_target)
remove_targets.append(target_idx)
remove_targets.sort(reverse=True)
for target_idx in remove_targets:
condition.target.pop(target_idx)
condition.target.extend(new_targets)
def check_condition_fallback_optional(self):
# a condition with a fallback **and** the source variable doesn't exist
remove_conditions = []
for idx, condition in enumerate(self.objectspace.space.constraints.condition):
# fallback
if condition.fallback is True and not self.objectspace.paths.path_is_defined(condition.source):
apply_action = False
if condition.name in ['disabled_if_in', 'mandatory_if_in', 'hidden_if_in']:
apply_action = not condition.force_condition_on_fallback
else:
apply_action = condition.force_inverse_condition_on_fallback
if apply_action:
actions = self._get_condition_actions(condition.name)
for target in condition.target:
leader_or_variable, variables = self._get_family_variables_from_target(target)
for action_idx, action in enumerate(actions):
if action_idx == 0:
setattr(leader_or_variable, action, True)
else:
for variable in variables:
setattr(variable, action, True)
remove_conditions.append(idx)
continue
remove_targets = []
# optional
for idx, target in enumerate(condition.target):
if target.optional is True and not self.objectspace.paths.path_is_defined(target.name):
remove_targets.append(idx)
remove_targets = list(set(remove_targets))
remove_targets.sort(reverse=True)
for idx in remove_targets:
condition.target.pop(idx)
remove_conditions = list(set(remove_conditions))
remove_conditions.sort(reverse=True)
for idx in remove_conditions:
self.objectspace.space.constraints.condition.pop(idx)
def _get_condition_actions(self, condition_name):
if condition_name.startswith('disabled_if_'):
return ['disabled']
elif condition_name.startswith('hidden_if_'):
return ['hidden', 'frozen', 'force_default_on_freeze']
elif condition_name.startswith('mandatory_if_'):
return ['mandatory']
elif condition_name == 'auto_hidden_if_not_in':
return ['auto_frozen']
def check_choice_option_condition(self):
# remove condition for ChoiceOption that don't have param
remove_conditions = []
for condition_idx, condition in enumerate(self.objectspace.space.constraints.condition):
namespace = condition.namespace
condition.source = self.objectspace.paths.get_variable_path(condition.source, namespace, allow_source=True)
src_variable = self.objectspace.paths.get_variable_obj(condition.source)
valid_enum = None
if condition.source in self.valid_enums and self.valid_enums[condition.source]['type'] == 'string':
valid_enum = self.valid_enums[condition.source]['values']
if valid_enum is not None:
remove_param = []
for param_idx, param in enumerate(condition.param):
if param.text not in valid_enum:
remove_param.append(param_idx)
remove_param.sort(reverse=True)
for idx in remove_param:
del condition.param[idx]
if condition.param == []:
remove_targets = []
for target in condition.target:
leader_or_variable, variables = self._get_family_variables_from_target(target)
if condition.name == 'disabled_if_not_in':
leader_or_variable.disabled = True
elif condition.name == 'hidden_if_not_in':
leader_or_variable.hidden = True
for variable in variables:
variable.frozen = True
variable.force_default_on_freeze = True
elif condition.name == 'mandatory_if_not_in':
variable.mandatory = True
remove_targets = list(set(remove_targets))
remove_targets.sort(reverse=True)
for target_idx in remove_targets:
condition.target.pop(target_idx)
remove_conditions.append(condition_idx)
remove_conditions = list(set(remove_conditions))
remove_conditions.sort(reverse=True)
for idx in remove_conditions:
self.objectspace.space.constraints.condition.pop(idx)
def remove_condition_with_empty_target(self):
remove_conditions = []
for condition_idx, condition in enumerate(self.objectspace.space.constraints.condition):
if not condition.target:
remove_conditions.append(condition_idx)
remove_conditions = list(set(remove_conditions))
remove_conditions.sort(reverse=True)
for idx in remove_conditions:
self.objectspace.space.constraints.condition.pop(idx)
def convert_condition(self):
for condition in self.objectspace.space.constraints.condition:
inverse = condition.name.endswith('_if_not_in')
actions = self._get_condition_actions(condition.name)
for param in condition.param:
if hasattr(param, 'text'):
param = param.text
else:
param = None
for target in condition.target:
leader_or_variable, variables = self._get_family_variables_from_target(target)
# if option is already disable, do not apply disable_if_in
if hasattr(leader_or_variable, actions[0]) and getattr(leader_or_variable, actions[0]) is True:
continue
for idx, action in enumerate(actions):
prop = self.objectspace.property_(leader_or_variable.xmlfiles)
prop.type = 'calculation'
prop.inverse = inverse
prop.source = condition.source
prop.expected = param
prop.name = action
if idx == 0:
if not hasattr(leader_or_variable, 'property'):
leader_or_variable.property = []
leader_or_variable.property.append(prop)
else:
for variable in variables:
if not hasattr(variable, 'property'):
variable.property = []
variable.property.append(prop)
del self.objectspace.space.constraints.condition
def _set_valid_enum(self, variable, values, type_, target):
# value for choice's variable is mandatory
variable.mandatory = True
# build choice
variable.choice = []
if isinstance(values, str):
choice = self.objectspace.choice()
choice.type = 'calculation'
choice.name = values
variable.choice.append(choice)
else:
self.valid_enums[target] = {'type': type_,
'values': values,
}
choices = []
for value in values:
choice = self.objectspace.choice(variable.xmlfiles)
try:
if value is not None:
choice.name = CONVERT_OPTION[type_].get('func', str)(value)
else:
choice.name = value
except:
raise DictConsistencyError(_(f'unable to change type of a valid_enum entry "{value}" is not a valid "{type_}" for "{variable.name}"'))
if choice.name == '':
choice.name = None
choices.append(choice.name)
choice.type = type_
variable.choice.append(choice)
# check value or set first choice value has default value
if hasattr(variable, 'value'):
for value in variable.value:
value.type = type_
try:
cvalue = CONVERT_OPTION[type_].get('func', str)(value.name)
except:
raise DictConsistencyError(_(f'unable to change type of value "{value}" is not a valid "{type_}" for "{variable.name}"'))
if cvalue not in choices:
raise DictConsistencyError(_('value "{}" of variable "{}" is not in list of all expected values ({})').format(value.name, variable.name, choices))
else:
new_value = self.objectspace.value(variable.xmlfiles)
new_value.name = choices[0]
new_value.type = type_
variable.value = [new_value]
if not variable.choice:
raise DictConsistencyError(_('empty valid enum is not allowed for variable {}').format(variable.name))
variable.type = 'choice'
def convert_check(self):
for check in self.objectspace.space.constraints.check:
variable = self.objectspace.paths.get_variable_obj(check.target)
name = check.name
if name == 'valid_entier':
if not hasattr(check, 'param'):
raise DictConsistencyError(_('{} must have, at least, 1 param').format(name))
for param in check.param:
if param.type not in ['string', 'number']:
raise DictConsistencyError(_(f'param in "valid_entier" must not be a "{param.type}"'))
if param.name == 'mini':
variable.min_number = int(param.text)
elif param.name == 'maxi':
variable.max_number = int(param.text)
else:
raise DictConsistencyError(_(f'unknown parameter {param.text} in check "valid_entier" for variable {check.target}'))
else:
check_ = self.objectspace.check(variable.xmlfiles)
if name == 'valid_differ':
name = 'valid_not_equal'
elif name == 'valid_network_netmask':
params_len = 1
if len(check.param) != params_len:
xmlfiles = self.objectspace.display_xmlfiles(check.xmlfiles)
raise DictConsistencyError(_(f'{name} must have {params_len} param in {xmlfiles}'))
elif name == 'valid_ipnetmask':
params_len = 1
if len(check.param) != params_len:
xmlfiles = self.objectspace.display_xmlfiles(check.xmlfiles)
raise DictConsistencyError(_(f'{name} must have {params_len} param in {xmlfiles}'))
name = 'valid_ip_netmask'
elif name == 'valid_broadcast':
params_len = 2
if len(check.param) != params_len:
xmlfiles = self.objectspace.display_xmlfiles(check.xmlfiles)
raise DictConsistencyError(_(f'{name} must have {params_len} param in {xmlfiles}'))
elif name == 'valid_in_network':
if len(check.param) not in (1, 2):
params_len = 2
xmlfiles = self.objectspace.display_xmlfiles(check.xmlfiles)
raise DictConsistencyError(_(f'{name} must have {params_len} param in {xmlfiles}'))
check_.name = name
check_.warnings_only = check.warnings_only
if hasattr(check, 'param'):
check_.param = check.param
if not hasattr(variable, 'check'):
variable.check = []
variable.check.append(check_)
del self.objectspace.space.constraints.check
def convert_fill(self): # pylint: disable=C0111,R0912
# sort fill/auto by index
fills = {fill.index: fill for idx, fill in enumerate(self.objectspace.space.constraints.fill)}
indexes = list(fills.keys())
indexes.sort()
targets = []
for idx in indexes:
fill = fills[idx]
# test if it's redefined calculation
if fill.target in targets and not fill.redefine:
raise DictConsistencyError(_(f"A fill already exists for the target: {fill.target}"))
targets.append(fill.target)
#
if fill.name not in self.functions:
raise DictConsistencyError(_('cannot find fill function {}').format(fill.name))
namespace = fill.namespace
# let's replace the target by the path
fill.target, suffix = self.objectspace.paths.get_variable_path(fill.target,
namespace,
with_suffix=True,
)
if suffix is not None:
raise DictConsistencyError(_(f'Cannot add fill function to "{fill.target}" only with the suffix "{suffix}"'))
variable = self.objectspace.paths.get_variable_obj(fill.target)
value = self.objectspace.value(variable.xmlfiles)
value.type = 'calculation'
value.name = fill.name
if hasattr(fill, 'param'):
param_to_delete = []
for fill_idx, param in enumerate(fill.param):
if param.type not in ['suffix', 'string'] and not hasattr(param, 'text'):
raise DictConsistencyError(_(f"All '{param.type}' variables must have a value in order to calculate {fill.target}"))
if param.type == 'suffix' and hasattr(param, 'text'):
raise DictConsistencyError(_(f"All '{param.type}' variables must not have a value in order to calculate {fill.target}"))
if param.type == 'string':
if not hasattr(param, 'text'):
param.text = None
if param.type == 'variable':
try:
param.text, suffix = self.objectspace.paths.get_variable_path(param.text,
namespace,
with_suffix=True,
)
if suffix:
param.suffix = suffix
except DictConsistencyError as err:
if param.optional is False:
raise err
param_to_delete.append(fill_idx)
continue
else:
param.notraisepropertyerror = None
param_to_delete.sort(reverse=True)
for param_idx in param_to_delete:
fill.param.pop(param_idx)
value.param = fill.param
variable.value = [value]
del self.objectspace.space.constraints.fill
def remove_constraints(self):
if hasattr(self.objectspace.space.constraints, 'index'):
del self.objectspace.space.constraints.index
del self.objectspace.space.constraints.namespace
del self.objectspace.space.constraints.xmlfiles
if vars(self.objectspace.space.constraints):
raise Exception('constraints again?')
del self.objectspace.space.constraints
class FamilyAnnotator:
def __init__(self,
objectspace,
):
self.objectspace = objectspace
self.remove_empty_families()
self.change_variable_mode()
self.change_family_mode()
self.dynamic_families()
def remove_empty_families(self): # pylint: disable=C0111,R0201
if hasattr(self.objectspace.space, 'variables'):
for family in self.objectspace.space.variables.values():
if hasattr(family, 'family'):
space = family.family
removed_families = []
for family_name, family in space.items():
if not hasattr(family, 'variable') or len(family.variable) == 0:
removed_families.append(family_name)
for family_name in removed_families:
del space[family_name]
def change_family_mode(self): # pylint: disable=C0111
if not hasattr(self.objectspace.space, 'variables'):
return
for family in self.objectspace.space.variables.values():
if hasattr(family, 'family'):
for family in family.family.values():
mode = modes_level[-1]
for variable in family.variable.values():
if isinstance(variable, self.objectspace.Leadership):
variable_mode = variable.variable[0].mode
variable.variable[0].mode = None
variable.mode = variable_mode
else:
variable_mode = variable.mode
if variable_mode is not None and modes[mode] > modes[variable_mode]:
mode = variable_mode
family.mode = mode
def dynamic_families(self): # pylint: disable=C0111
if not hasattr(self.objectspace.space, 'variables'):
return
for family in self.objectspace.space.variables.values():
if hasattr(family, 'family'):
for family in family.family.values():
if 'dynamic' in vars(family):
namespace = self.objectspace.paths.get_variable_namespace(family.dynamic)
varpath = self.objectspace.paths.get_variable_path(family.dynamic, namespace)
family.dynamic = varpath
def annotate_variable(self, variable, family_mode, path, is_follower=False):
# if the variable is mandatory and doesn't have any value
# then the variable's mode is set to 'basic'
if not hasattr(variable, 'value') and variable.type == 'boolean':
new_value = self.objectspace.value(variable.xmlfiles)
new_value.name = True
new_value.type = 'boolean'
variable.value = [new_value]
if hasattr(variable, 'value') and variable.value:
has_value = True
for value in variable.value:
if value.type == 'calculation':
has_value = False
has_variable = False
if hasattr(value, 'param'):
for param in value.param:
if param.type == 'variable':
has_variable = True
break
#if not has_variable:
# # if one parameter is a variable, let variable choice if it's mandatory
# variable.mandatory = True
if has_value:
# if has value but without any calculation
variable.mandatory = True
if variable.mandatory is True and (not hasattr(variable, 'value') or is_follower):
variable.mode = modes_level[0]
if variable.mode != None and modes[variable.mode] < modes[family_mode] and (not is_follower or variable.mode != modes_level[0]):
variable.mode = family_mode
if variable.hidden is True:
variable.frozen = True
if not variable.auto_save is True and 'force_default_on_freeze' not in vars(variable):
variable.force_default_on_freeze = True
def change_variable_mode(self): # pylint: disable=C0111
if not hasattr(self.objectspace.space, 'variables'):
return
for variables in self.objectspace.space.variables.values():
namespace = variables.name
if hasattr(variables, 'family'):
for family in variables.family.values():
family_mode = family.mode
if hasattr(family, 'variable'):
for variable in family.variable.values():
if isinstance(variable, self.objectspace.Leadership):
mode = modes_level[-1]
for idx, follower in enumerate(variable.variable):
if follower.auto_save is True:
raise DictConsistencyError(_(f'leader/followers {follower.name} could not be auto_save'))
if follower.auto_freeze is True:
raise DictConsistencyError(_('leader/followers {follower.name} could not be auto_freeze'))
is_follower = idx != 0
path = '{}.{}.{}'.format(family.path, variable.name, follower.name)
self.annotate_variable(follower, family_mode, path, is_follower)
# leader's mode is minimum level
if modes[variable.variable[0].mode] > modes[follower.mode]:
follower.mode = variable.variable[0].mode
variable.mode = variable.variable[0].mode
else:
# auto_save's variable is set in 'basic' mode if its mode is 'normal'
if variable.auto_save is True and variable.mode != modes_level[-1]:
variable.mode = modes_level[0]
# auto_freeze's variable is set in 'basic' mode if its mode is 'normal'
if variable.auto_freeze is True and variable.mode != modes_level[-1]:
variable.mode = modes_level[0]
path = '{}.{}'.format(family.path, variable.name)
self.annotate_variable(variable, family_mode, path)
class PropertyAnnotator:
def __init__(self, objectspace):
self.objectspace = objectspace
self.convert_annotator()
def convert_property(self,
variable,
):
properties = []
for prop in PROPERTIES:
if hasattr(variable, prop):
if getattr(variable, prop) == True:
for subprop in CONVERT_PROPERTIES.get(prop, [prop]):
properties.append(subprop)
setattr(variable, prop, None)
if hasattr(variable, 'mode') and variable.mode:
properties.append(variable.mode)
variable.mode = None
if properties:
variable.properties = frozenset(properties)
def convert_annotator(self): # pylint: disable=C0111
if hasattr(self.objectspace.space, 'services'):
self.convert_property(self.objectspace.space.services)
for services in self.objectspace.space.services.service.values():
self.convert_property(services)
for service in vars(services).values():
if isinstance(service, self.objectspace.family):
self.convert_property(service)
if hasattr(service, 'family'):
self.convert_property(service)
for family in service.family:
self.convert_property(family)
if hasattr(family, 'variable'):
for variable in family.variable:
self.convert_property(variable)
if hasattr(self.objectspace.space, 'variables'):
for variables in self.objectspace.space.variables.values():
if hasattr(variables, 'family'):
for family in variables.family.values():
self.convert_property(family)
if hasattr(family, 'variable'):
for variable in family.variable.values():
if isinstance(variable, self.objectspace.Leadership):
self.convert_property(variable)
for follower in variable.variable:
self.convert_property(follower)
else:
self.convert_property(variable)

57
src/rougail/annotator/__init__.py Normal file
View File

@ -0,0 +1,57 @@
"""Annotate dictionaries
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from .group import GroupAnnotator
from .service import ServiceAnnotator
from .variable import VariableAnnotator, CONVERT_OPTION
from .check import CheckAnnotator
from .value import ValueAnnotator
from .condition import ConditionAnnotator
from .fill import FillAnnotator
from .family import FamilyAnnotator
from .property import PropertyAnnotator
class SpaceAnnotator: # pylint: disable=R0903
"""Transformations applied on a object instance
"""
def __init__(self, objectspace, eosfunc_file):
self.objectspace = objectspace
GroupAnnotator(objectspace)
ServiceAnnotator(objectspace)
VariableAnnotator(objectspace)
CheckAnnotator(objectspace,
eosfunc_file,
)
ConditionAnnotator(objectspace)
FillAnnotator(objectspace,
eosfunc_file,
)
ValueAnnotator(objectspace)
FamilyAnnotator(objectspace)
PropertyAnnotator(objectspace)
__all__ = ('SpaceAnnotator', 'CONVERT_OPTION')

242
src/rougail/annotator/check.py Normal file
View File

@ -0,0 +1,242 @@
"""Annotate check
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from typing import List, Any
from .target import TargetAnnotator
from .param import ParamAnnotator
from ..utils import load_modules
from ..i18n import _
from ..error import DictConsistencyError, display_xmlfiles
INTERNAL_FUNCTIONS = ['valid_enum', 'valid_in_network', 'valid_differ', 'valid_entier']
class CheckAnnotator(TargetAnnotator, ParamAnnotator):
"""Annotate check
"""
def __init__(self,
objectspace,
eosfunc_file,
):
if not hasattr(objectspace.space, 'constraints') or \
not hasattr(objectspace.space.constraints, 'check'):
return
self.objectspace = objectspace
self.let_none = True
self.only_variable = True
self.functions = dir(load_modules(eosfunc_file))
self.functions.extend(INTERNAL_FUNCTIONS)
self.functions.extend(self.objectspace.rougailconfig['internal_functions'])
self.target_is_uniq = False
self.allow_function = True
self.convert_target(self.objectspace.space.constraints.check)
self.convert_param(self.objectspace.space.constraints.check)
self.check_check()
self.check_valid_enum()
self.check_change_warning()
self.convert_valid_entier()
self.convert_check()
del objectspace.space.constraints.check
def valid_type_validation(self,
obj,
) -> None:
variable_type = None
if obj.name == 'valid_enum':
for target in obj.target:
variable_type = target.name.type
return variable_type
def check_check(self): # pylint: disable=R0912
"""valid and manage <check>
"""
remove_indexes = []
for check_idx, check in enumerate(self.objectspace.space.constraints.check):
if not check.name in self.functions:
msg = _(f'cannot find check function "{check.name}"')
raise DictConsistencyError(msg, 1, check.xmlfiles)
if hasattr(check, 'param') and check.param == []:
remove_indexes.append(check_idx)
remove_indexes.sort(reverse=True)
for idx in remove_indexes:
del self.objectspace.space.constraints.check[idx]
def check_valid_enum(self):
"""verify valid_enum
"""
remove_indexes = []
for idx, check in enumerate(self.objectspace.space.constraints.check):
if check.name != 'valid_enum':
continue
for target in check.target:
if target.name.path in self.objectspace.valid_enums:
check_xmlfiles = display_xmlfiles(self.objectspace.valid_enums\
[target.name.path]['xmlfiles'])
msg = _(f'valid_enum already set in {check_xmlfiles} '
f'for "{target.name.name}", you may have forget remove_check')
raise DictConsistencyError(msg, 3, check.xmlfiles)
if not hasattr(check, 'param'):
msg = _(f'param is mandatory for a valid_enum of variable "{target.name.name}"')
raise DictConsistencyError(msg, 4, check.xmlfiles)
variable_type = target.name.type
values = self._set_valid_enum(target.name,
check,
)
if values:
if hasattr(target.name, 'value'):
# check value
self.check_valid_enum_value(target.name, values)
else:
# no value, set the first choice as default value
new_value = self.objectspace.value(check.xmlfiles)
new_value.name = values[0]
new_value.type = variable_type
target.name.value = [new_value]
remove_indexes.append(idx)
remove_indexes.sort(reverse=True)
for idx in remove_indexes:
del self.objectspace.space.constraints.check[idx]
def _set_valid_enum(self,
variable,
check,
) -> List[Any]:
# build choice
variable.values = []
variable.ori_type = variable.type
variable.type = 'choice'
has_variable = False
values = []
has_nil = False
is_function = False
for param in check.param:
if has_variable:
msg = _(f'only one "variable" parameter is allowed for valid_enum '
f'of variable "{variable.name}"')
raise DictConsistencyError(msg, 5, param.xmlfiles)
if param.type == 'function':
is_function = True
choice = self.objectspace.choice(variable.xmlfiles)
choice.name = param.text
choice.type = 'function'
choice.param = []
variable.values.append(choice)
continue
if is_function:
variable.values[0].param.append(param)
continue
param_type = variable.ori_type
if param.type == 'variable':
has_variable = True
if param.optional is True:
msg = _(f'optional parameter in valid_enum for variable "{variable.name}" '
f'is not allowed')
raise DictConsistencyError(msg, 14, param.xmlfiles)
if not param.text.multi:
msg = _(f'only multi "variable" parameter is allowed for valid_enum '
f'of variable "{variable.name}"')
raise DictConsistencyError(msg, 6, param.xmlfiles)
param_type = 'variable'
elif param.type == 'nil':
has_nil = True
values.append(param.text)
choice = self.objectspace.choice(variable.xmlfiles)
choice.name = param.text
choice.type = param_type
variable.values.append(choice)
if is_function:
return None
if 'mandatory' not in vars(variable):
variable.mandatory = not has_nil
elif variable.mandatory is False:
choice = self.objectspace.choice(variable.xmlfiles)
choice.name = None
choice.type = 'nil'
variable.values.append(choice)
if has_variable:
return None
self.objectspace.valid_enums[variable.path] = {'type': variable.ori_type,
'values': values,
'xmlfiles': check.xmlfiles,
}
return values
@staticmethod
def check_valid_enum_value(variable,
values,
) -> None:
"""check that values in valid_enum are valid
"""
for value in variable.value:
if value.name not in values:
msg = _(f'value "{value.name}" of variable "{variable.name}" is not in list '
f'of all expected values ({values})')
raise DictConsistencyError(msg, 15, value.xmlfiles)
def check_change_warning(self):
"""convert level to "warnings_only"
"""
for check in self.objectspace.space.constraints.check:
check.warnings_only = check.level == 'warning'
check.level = None
def convert_valid_entier(self) -> None:
"""valid and manage <check>
"""
remove_indexes = []
for check_idx, check in enumerate(self.objectspace.space.constraints.check):
if not check.name == 'valid_entier':
continue
remove_indexes.append(check_idx)
if not hasattr(check, 'param'):
msg = _(f'{check.name} must have, at least, 1 param')
raise DictConsistencyError(msg, 17, check.xmlfiles)
for param in check.param:
if param.type != 'number':
msg = _(f'param in "valid_entier" must be an "integer", not "{param.type}"')
raise DictConsistencyError(msg, 18, check.xmlfiles)
for target in check.target:
if param.name == 'mini':
target.name.min_number = int(param.text)
elif param.name == 'maxi':
target.name.max_number = int(param.text)
else:
msg = _(f'unknown parameter "{param.name}" in check "valid_entier"')
raise DictConsistencyError(msg, 19, check.xmlfiles)
remove_indexes.sort(reverse=True)
for idx in remove_indexes:
del self.objectspace.space.constraints.check[idx]
def convert_check(self) -> None:
"""valid and manage <check>
"""
for check in self.objectspace.space.constraints.check:
for target in check.target:
if not hasattr(target.name, 'validators'):
target.name.validators = []
target.name.validators.append(check)

369
src/rougail/annotator/condition.py Normal file
View File

@ -0,0 +1,369 @@
"""Annotate condition
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from typing import List
from ..i18n import _
from ..error import DictConsistencyError
from .target import TargetAnnotator
from .param import ParamAnnotator
from .variable import Walk
class ConditionAnnotator(TargetAnnotator, ParamAnnotator, Walk):
"""Annotate condition
"""
def __init__(self,
objectspace,
):
self.objectspace = objectspace
self.force_service_value = {}
if hasattr(objectspace.space, 'variables'):
self.convert_auto_freeze()
if not hasattr(objectspace.space, 'constraints') or \
not hasattr(self.objectspace.space.constraints, 'condition'):
return
self.target_is_uniq = False
self.only_variable = False
self.allow_function = False
self.convert_target(self.objectspace.space.constraints.condition)
self.check_condition_optional()
self.convert_condition_source()
self.convert_param(self.objectspace.space.constraints.condition)
self.check_source_target()
self.convert_xxxlist()
self.check_choice_option_condition()
self.remove_condition_with_empty_target()
self.convert_condition()
def valid_type_validation(self,
obj,
) -> None:
if obj.source.type == 'choice':
return obj.source.ori_type
return obj.source.type
def convert_auto_freeze(self):
"""convert auto_freeze
only if auto_freeze_variable is True this variable is frozen
"""
for variable in self.get_variables():
if not variable.auto_freeze and not variable.auto_save:
continue
#if variable.namespace != self.objectspace.rougailconfig['variable_namespace']:
# msg = _(f'auto_freeze is not allowed in extra "{variable.namespace}"')
# raise DictConsistencyError(msg, 49, variable.xmlfiles)
variable.force_store_value = True
if variable.auto_save:
continue
new_condition = self.objectspace.condition(variable.xmlfiles)
new_condition.name = 'auto_frozen_if_in'
new_condition.namespace = variable.namespace
new_condition.source = self.objectspace.rougailconfig['auto_freeze_variable']
new_param = self.objectspace.param(variable.xmlfiles)
new_param.text = True
new_condition.param = [new_param]
new_target = self.objectspace.target(variable.xmlfiles)
new_target.type = 'variable'
new_target.name = variable.path
new_condition.target = [new_target]
if not hasattr(self.objectspace.space, 'constraints'):
self.objectspace.space.constraints = self.objectspace.constraints(variable.xmlfiles)
if not hasattr(self.objectspace.space.constraints, 'condition'):
self.objectspace.space.constraints.condition = []
self.objectspace.space.constraints.condition.append(new_condition)
def check_source_target(self):
"""verify that source != target in condition
"""
for condition in self.objectspace.space.constraints.condition:
for target in condition.target:
if target.type == 'variable' and \
condition.source.path == target.name.path:
msg = _('target name and source name must be different: '
f'{condition.source.path}')
raise DictConsistencyError(msg, 11, condition.xmlfiles)
def check_condition_optional(self):
"""a condition with a optional **and** the source variable doesn't exist
"""
remove_conditions = []
for idx, condition in enumerate(self.objectspace.space.constraints.condition):
if condition.optional is False or \
self.objectspace.paths.path_is_defined(condition.source):
continue
remove_conditions.append(idx)
if (hasattr(condition, 'apply_on_fallback') and condition.apply_on_fallback) or \
(not hasattr(condition, 'apply_on_fallback') and \
condition.name.endswith('_if_in')):
self.force_actions_to_variable(condition)
remove_conditions.sort(reverse=True)
for idx in remove_conditions:
self.objectspace.space.constraints.condition.pop(idx)
def force_actions_to_variable(self,
condition: 'self.objectspace.condition',
) -> None:
"""force property to a variable
for example disabled_if_not_in => variable.disabled = True
"""
actions = self.get_actions_from_condition(condition.name)
for target in condition.target:
main_action = actions[0]
if target.type.endswith('list'):
self.force_service_value[target.name] = main_action != 'disabled'
continue
leader_or_var, variables = self._get_family_variables_from_target(target)
setattr(leader_or_var, main_action, True)
for action in actions[1:]:
for variable in variables:
setattr(variable, action, True)
@staticmethod
def get_actions_from_condition(condition_name: str) -> List[str]:
"""get action's name from a condition
"""
if condition_name.startswith('hidden_if_'):
return ['hidden', 'frozen', 'force_default_on_freeze']
if condition_name == 'auto_frozen_if_in':
return ['frozen']
return [condition_name.split('_', 1)[0]]
def _get_family_variables_from_target(self,
target,
):
if target.type == 'variable':
if not self.objectspace.paths.is_leader(target.name.path):
return target.name, [target.name]
# it's a leader, so apply property to leadership
family_name = self.objectspace.paths.get_variable_family_path(target.name.path)
family = self.objectspace.paths.get_family(family_name,
target.name.namespace,
)
return family, family.variable
# it's a family
variable = self.objectspace.paths.get_family(target.name.path,
target.namespace,
)
if hasattr(variable, 'variable'):
return variable, list(variable.variable.values())
return variable, []
def convert_xxxlist(self):
"""transform *list to variable or family
"""
fills = {}
for condition in self.objectspace.space.constraints.condition:
remove_targets = []
for target_idx, target in enumerate(condition.target):
if target.type.endswith('list'):
listvars = self.objectspace.list_conditions.get(target.type,
{}).get(target.name)
if listvars:
self._convert_xxxlist_to_fill(condition,
target,
listvars,
fills,
)
elif not target.optional:
msg = f'cannot found target "{target.type}" "{target.name}"'
raise DictConsistencyError(_(msg), 2, target.xmlfiles)
remove_targets.append(target_idx)
remove_targets.sort(reverse=True)
for target_idx in remove_targets:
condition.target.pop(target_idx)
def _convert_xxxlist_to_fill(self,
condition: 'self.objectspace.condition',
target: 'self.objectspace.target',
listvars: list,
fills: dict,
):
for listvar in listvars:
if target.name in self.force_service_value:
listvar.default = self.force_service_value[target.name]
continue
if listvar.path in fills:
fill = fills[listvar.path]
or_needed = True
for param in fill.param:
if hasattr(param, 'name') and \
param.name == 'condition_operator':
or_needed = False
break
fill.index += 1
else:
fill = self.objectspace.fill(target.xmlfiles)
new_target = self.objectspace.target(target.xmlfiles)
new_target.name = listvar.path
fill.target = [new_target]
fill.name = 'calc_value'
fill.namespace = 'services'
fill.index = 0
if not hasattr(self.objectspace.space.constraints, 'fill'):
self.objectspace.space.constraints.fill = []
self.objectspace.space.constraints.fill.append(fill)
fills[listvar.path] = fill
param1 = self.objectspace.param(target.xmlfiles)
param1.text = False
param1.type = 'boolean'
param2 = self.objectspace.param(target.xmlfiles)
param2.name = 'default'
param2.text = True
param2.type = 'boolean'
fill.param = [param1, param2]
or_needed = len(condition.param) != 1
if len(condition.param) == 1:
values = getattr(condition.param[0], 'text', None)
param_type = condition.param[0].type
else:
values = tuple([getattr(param, 'text', None) for param in condition.param])
param_type = None
for param in condition.param:
if param_type is None or param_type == 'nil':
param_type = param.type
param3 = self.objectspace.param(target.xmlfiles)
param3.name = f'condition_{fill.index}'
param3.type = 'variable'
param3.text = condition.source.path
param3.propertyerror = False
fill.param.append(param3)
param4 = self.objectspace.param(target.xmlfiles)
param4.name = f'expected_{fill.index}'
param4.text = values
param4.type = param_type
fill.param.append(param4)
if condition.name != 'disabled_if_in':
param5 = self.objectspace.param(target.xmlfiles)
param5.name = f'reverse_condition_{fill.index}'
param5.text = True
param5.type = 'boolean'
fill.param.append(param5)
if or_needed:
param6 = self.objectspace.param(target.xmlfiles)
param6.name = 'condition_operator'
param6.text = 'OR'
fill.param.append(param6)
def convert_condition_source(self):
"""remove condition for ChoiceOption that don't have param
"""
for condition in self.objectspace.space.constraints.condition:
try:
condition.source = self.objectspace.paths.get_variable(condition.source)
except DictConsistencyError as err:
if err.errno == 36:
msg = _(f'the source "{condition.source}" in condition cannot be a dynamic '
f'variable')
raise DictConsistencyError(msg, 20, condition.xmlfiles) from err
if err.errno == 42:
msg = _(f'the source "{condition.source}" in condition is an unknown variable')
raise DictConsistencyError(msg, 23, condition.xmlfiles) from err
raise err from err # pragma: no cover
def check_choice_option_condition(self):
"""remove condition of ChoiceOption that doesn't match
"""
remove_conditions = []
for condition_idx, condition in enumerate(self.objectspace.space.constraints.condition):
if condition.source.path in self.objectspace.valid_enums:
valid_enum = self.objectspace.valid_enums[condition.source.path]['values']
remove_param = [param_idx for param_idx, param in enumerate(condition.param) \
if param.type != 'variable' and param.text not in valid_enum]
if not remove_param:
continue
remove_param.sort(reverse=True)
for idx in remove_param:
del condition.param[idx]
if not condition.param and condition.name.endswith('_if_not_in'):
self.force_actions_to_variable(condition)
remove_conditions.append(condition_idx)
remove_conditions.sort(reverse=True)
for idx in remove_conditions:
self.objectspace.space.constraints.condition.pop(idx)
def remove_condition_with_empty_target(self):
"""remove condition with empty target
"""
# optional target are remove, condition could be empty
remove_conditions = [condition_idx for condition_idx, condition in \
enumerate(self.objectspace.space.constraints.condition) \
if not condition.target]
remove_conditions.sort(reverse=True)
for idx in remove_conditions:
self.objectspace.space.constraints.condition.pop(idx)
def convert_condition(self):
"""valid and manage <condition>
"""
for condition in self.objectspace.space.constraints.condition:
actions = self.get_actions_from_condition(condition.name)
for param in condition.param:
for target in condition.target:
leader_or_variable, variables = self._get_family_variables_from_target(target)
# if option is already disable, do not apply disable_if_in
# check only the first action (example of multiple actions:
# 'hidden', 'frozen', 'force_default_on_freeze')
main_action = actions[0]
if getattr(leader_or_variable, main_action, False) is True:
continue
self.build_property(leader_or_variable,
param,
condition,
main_action,
)
if isinstance(leader_or_variable, self.objectspace.variable) and \
(leader_or_variable.auto_save or leader_or_variable.auto_freeze) and \
'force_default_on_freeze' in actions:
continue
for action in actions[1:]:
# other actions are set to the variable or children of family
for variable in variables:
self.build_property(variable,
param,
condition,
action,
)
def build_property(self,
obj,
param: 'self.objectspace.param',
condition: 'self.objectspace.condition',
action: str,
) -> 'self.objectspace.property_':
"""build property_ for a condition
"""
prop = self.objectspace.property_(obj.xmlfiles)
prop.type = 'calculation'
prop.inverse = condition.name.endswith('_if_not_in')
prop.source = condition.source
prop.expected = param
prop.name = action
if not hasattr(obj, 'properties'):
obj.properties = []
obj.properties.append(prop)

300
src/rougail/annotator/family.py Normal file
View File

@ -0,0 +1,300 @@
"""Annotate family
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from ..i18n import _
from ..error import DictConsistencyError
from .variable import Walk
class Mode: # pylint: disable=R0903
"""Class to manage mode level
"""
def __init__(self,
level: int,
) -> None:
self.level = level
def __gt__(self,
other: int,
) -> bool:
return other.level < self.level
class FamilyAnnotator(Walk):
"""Annotate family
"""
def __init__(self,
objectspace,
):
self.objectspace = objectspace
if not hasattr(self.objectspace.space, 'variables'):
return
self.modes = {name: Mode(idx) for idx, name in enumerate(self.objectspace.rougailconfig['modes_level'])}
self.remove_empty_families()
self.family_names()
self.change_modes()
self.dynamic_families()
self.convert_help()
def _has_variable(self,
family: 'self.objectspace.family',
) -> bool:
if hasattr(family, 'variable'):
for variable in family.variable.values():
if isinstance(variable, self.objectspace.family):
if self._has_variable(variable):
return True
else:
return True
return False
def remove_empty_families(self) -> None:
"""Remove all families without any variable
"""
for families in self.objectspace.space.variables.values():
removed_families = []
for family_name, family in families.variable.items():
if isinstance(family, self.objectspace.family) and not self._has_variable(family):
removed_families.append(family_name)
for family_name in removed_families:
del families.variable[family_name]
def family_names(self) -> None:
"""Set doc, path, ... to family
"""
for family in self.get_families():
if not hasattr(family, 'description'):
family.description = family.name
family.doc = family.description
del family.description
def change_modes(self):
"""change the mode of variables
"""
modes_level = self.objectspace.rougailconfig['modes_level']
default_variable_mode = self.objectspace.rougailconfig['default_variable_mode']
if default_variable_mode not in modes_level:
msg = _(f'default variable mode "{default_variable_mode}" is not a valid mode, '
f'valid modes are {modes_level}')
raise DictConsistencyError(msg, 72, None)
default_family_mode = self.objectspace.rougailconfig['default_family_mode']
if default_family_mode not in modes_level:
msg = _(f'default family mode "{default_family_mode}" is not a valid mode, '
f'valid modes are {modes_level}')
raise DictConsistencyError(msg, 73, None)
families = list(self.get_families())
for family in families:
self.valid_mode(family)
self._set_default_mode(family)
families.reverse()
for family in families:
self._change_family_mode(family)
def valid_mode(self,
obj,
) -> None:
modes_level = self.objectspace.rougailconfig['modes_level']
if hasattr(obj, 'mode') and obj.mode not in modes_level:
msg = _(f'mode "{obj.mode}" for "{obj.name}" is not a valid mode, '
f'valid modes are {modes_level}')
raise DictConsistencyError(msg, 71, obj.xmlfiles)
def _set_default_mode(self,
family: 'self.objectspace.family',
) -> None:
if hasattr(family, 'mode') and 'mode' in vars(family):
family_mode = family.mode
else:
family_mode = None
if not hasattr(family, 'variable'):
return
for variable in family.variable.values():
self.valid_mode(variable)
if isinstance(variable, self.objectspace.family):
if family_mode and not self._has_mode(variable):
self._set_auto_mode(variable, family_mode)
continue
if isinstance(variable, self.objectspace.leadership):
func = self._set_default_mode_leader
else:
func = self._set_default_mode_variable
func(variable, family_mode)
@staticmethod
def _has_mode(obj) -> bool:
return 'mode' in vars(obj) and not hasattr(obj, 'mode_auto')
def _set_default_mode_variable(self,
variable: 'self.objectspace.variable',
family_mode: str,
) -> None:
# auto_save or auto_freeze variable is set to 'basic' mode
# if its mode is not defined by the user
if not self._has_mode(variable) and \
(variable.auto_save is True or variable.auto_freeze is True):
variable.mode = self.objectspace.rougailconfig['modes_level'][0]
# mandatory variable without value is a basic variable
elif not self._has_mode(variable) and \
variable.mandatory is True and \
not hasattr(variable, 'default') and \
not hasattr(variable, 'default_multi'):
variable.mode = self.objectspace.rougailconfig['modes_level'][0]
elif family_mode and not self._has_mode(variable):
self._set_auto_mode(variable, family_mode)
@staticmethod
def _set_auto_mode(obj, mode: str) -> None:
obj.mode = mode
obj.mode_auto = True
def _set_default_mode_leader(self,
leadership: 'self.objectspace.leadership',
family_mode: str,
) -> None:
leader_mode = None
for follower in leadership.variable:
self.valid_mode(follower)
if follower.auto_save is True:
msg = _(f'leader/followers "{follower.name}" could not be auto_save')
raise DictConsistencyError(msg, 29, leadership.xmlfiles)
if follower.auto_freeze is True:
msg = f'leader/followers "{follower.name}" could not be auto_freeze'
raise DictConsistencyError(_(msg), 30, leadership.xmlfiles)
if leader_mode is not None:
if hasattr(follower, 'mode'):
follower_mode = follower.mode
else:
follower_mode = self.objectspace.rougailconfig['default_variable_mode']
if self.modes[leader_mode] > self.modes[follower_mode]:
if self._has_mode(follower) and not self._has_mode(leadership.variable[0]):
# if follower has mode but not the leader
self._set_auto_mode(leadership.variable[0], follower_mode)
else:
# leader's mode is minimum level
if self._has_mode(follower):
msg = _(f'the follower "{follower.name}" is in "{follower_mode}" mode '
f'but leader have the higher mode "{leader_mode}"')
raise DictConsistencyError(msg, 63, follower.xmlfiles)
self._set_auto_mode(follower, leader_mode)
self._set_default_mode_variable(follower,
family_mode,
)
if leader_mode is None:
if hasattr(leadership.variable[0], 'mode'):
leader_mode = leadership.variable[0].mode
else:
leader_mode = self.objectspace.rougailconfig['default_variable_mode']
if hasattr(leadership.variable[0], 'mode'):
leader_mode = leadership.variable[0].mode
self._set_auto_mode(leadership, leader_mode)
def _change_family_mode(self,
family: 'self.objectspace.family',
) -> None:
if hasattr(family, 'mode'):
family_mode = family.mode
else:
family_mode = self.objectspace.rougailconfig['default_family_mode']
min_variable_mode = self.objectspace.rougailconfig['modes_level'][-1]
# change variable mode, but not if variables are not in a family
if hasattr(family, 'variable'):
for variable in family.variable.values():
if not isinstance(variable, self.objectspace.family):
if isinstance(variable, self.objectspace.leadership):
func = self._change_variable_mode_leader
else:
func = self._change_variable_mode
func(variable,
family_mode,
)
elif not hasattr(variable, 'mode'):
variable.mode = self.objectspace.rougailconfig['default_family_mode']
if self.modes[min_variable_mode] > self.modes[variable.mode]:
min_variable_mode = variable.mode
if isinstance(family, self.objectspace.family) and \
(not hasattr(family, 'mode') or family.mode != min_variable_mode):
# set the lower variable mode to family
if self._has_mode(family):
msg = _(f'the family "{family.name}" is in "{family.mode}" mode but variables and '
f'families inside have the higher modes "{min_variable_mode}"')
raise DictConsistencyError(msg, 62, family.xmlfiles)
self._set_auto_mode(family, min_variable_mode)
def _change_variable_mode(self,
variable,
family_mode: str,
) -> None:
if hasattr(variable, 'mode'):
variable_mode = variable.mode
else:
variable_mode = self.objectspace.rougailconfig['default_variable_mode']
# none basic variable in high level family has to be in high level
if self.modes[variable_mode] < self.modes[family_mode]:
if self._has_mode(variable):
msg = _(f'the variable "{variable.name}" is in "{variable_mode}" mode '
f'but family has the higher family mode "{family_mode}"')
raise DictConsistencyError(msg, 61, variable.xmlfiles)
self._set_auto_mode(variable, family_mode)
if not hasattr(variable, 'mode'):
variable.mode = variable_mode
def _change_variable_mode_leader(self,
leadership,
family_mode: str,
) -> None:
for follower in leadership.variable:
self._change_variable_mode(follower,
family_mode,
)
leadership.variable[0].mode = None
def dynamic_families(self):
"""link dynamic families to object
"""
for family in self.get_families():
if 'dynamic' not in vars(family):
continue
family.suffixes = self.objectspace.paths.get_variable(family.dynamic)
del family.dynamic
if not family.suffixes.multi:
msg = _(f'dynamic family "{family.name}" must be linked '
f'to multi variable')
raise DictConsistencyError(msg, 16, family.xmlfiles)
for variable in family.variable.values():
if isinstance(variable, self.objectspace.family):
msg = _(f'dynamic family "{family.name}" cannot contains another family')
raise DictConsistencyError(msg, 22, family.xmlfiles)
def convert_help(self):
"""Convert variable help
"""
for family in self.get_families():
if not hasattr(family, 'help'):
continue
if not hasattr(family, 'information'):
family.information = self.objectspace.information(family.xmlfiles)
family.information.help = family.help
del family.help

93
src/rougail/annotator/fill.py Normal file
View File

@ -0,0 +1,93 @@
"""Fill annotator
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from ..utils import load_modules
from ..i18n import _
from ..error import DictConsistencyError
from .target import TargetAnnotator
from .param import ParamAnnotator
CALC_MULTI = ('calc_value', 'calc_list', 'get_range', 'calc_val_first_value', 'unbound_filename')
class FillAnnotator(TargetAnnotator, ParamAnnotator):
"""Fill annotator
"""
def __init__(self,
objectspace,
eosfunc_file,
):
self.objectspace = objectspace
if not hasattr(objectspace.space, 'constraints') or \
not hasattr(self.objectspace.space.constraints, 'fill'):
return
self.functions = dir(load_modules(eosfunc_file))
self.functions.extend(self.objectspace.rougailconfig['internal_functions'])
self.target_is_uniq = True
self.only_variable = True
self.allow_function = False
self.convert_target(self.objectspace.space.constraints.fill)
self.convert_param(self.objectspace.space.constraints.fill)
self.fill_to_value()
del self.objectspace.space.constraints.fill
def calc_is_multi(self, variable: 'self.objectspace.variable') -> bool:
multi = variable.multi
if multi is False:
return multi
if multi == 'submulti':
return True
return not self.objectspace.paths.is_follower(variable.path)
def fill_to_value(self) -> None:
"""valid and manage <fill>
"""
for fill in self.objectspace.space.constraints.fill:
for target in fill.target:
# test if the function exists
if fill.name not in self.functions:
msg = _(f'cannot find fill function "{fill.name}"')
raise DictConsistencyError(msg, 25, fill.xmlfiles)
# create an object value
value = self.objectspace.value(fill.xmlfiles)
value.type = 'calculation'
value.name = fill.name
if fill.name not in CALC_MULTI:
is_calc_multi = self.calc_is_multi(target.name)
else:
is_calc_multi = False
value.calc_multi = is_calc_multi
if target.name.namespace == 'services':
target.name.default = value
else:
target.name.value = [value]
# manage params
if hasattr(fill, 'param') and fill.param:
value.param = fill.param

160
src/rougail/annotator/group.py Normal file
View File

@ -0,0 +1,160 @@
"""Annotate group
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from ..i18n import _
from ..error import DictConsistencyError
from ..utils import normalize_family
class GroupAnnotator:
"""Annotate group
"""
def __init__(self,
objectspace,
):
self.objectspace = objectspace
if not hasattr(self.objectspace.space, 'constraints') or \
not hasattr(self.objectspace.space.constraints, 'group'):
return
self.convert_groups()
def convert_groups(self): # pylint: disable=C0111
"""convert groups
"""
# store old leaders family name
cache_paths = {}
for group in self.objectspace.space.constraints.group:
if group.leader in cache_paths:
leader_fam_path = cache_paths[group.leader]
else:
leader_fam_path = self.objectspace.paths.get_variable_family_path(group.leader, group.xmlfiles)
cache_paths[group.leader] = leader_fam_path
follower_names = list(group.follower.keys())
leader = self.objectspace.paths.get_variable(group.leader)
if '.' not in leader_fam_path:
# it's a namespace
ori_leader_family = self.objectspace.space.variables[leader_fam_path]
else:
# it's a sub family
ori_leader_family = self.objectspace.paths.get_family(leader_fam_path,
leader.namespace,
)
has_a_leader = False
for variable in list(ori_leader_family.variable.values()):
if isinstance(variable, self.objectspace.leadership) and \
variable.variable[0].name == leader.name:
# append follower to an existed leadership
leader_space = variable
has_a_leader = True
elif variable.name == leader.name:
# it's a leader
leader_space = self.manage_leader(variable,
group,
ori_leader_family,
)
has_a_leader = True
elif has_a_leader:
# it's should be a follower
self.manage_follower(follower_names.pop(0),
leader_fam_path,
variable,
leader_space,
)
# this variable is not more in ori_leader_family
ori_leader_family.variable.pop(normalize_family(variable.name))
if follower_names == []:
# no more follower
break
else:
joined = '", "'.join(follower_names)
msg = _(f'when parsing leadership, we espect to find those followers "{joined}"')
raise DictConsistencyError(msg, 31, variable.xmlfiles)
del self.objectspace.space.constraints.group
def manage_leader(self,
variable: 'Variable',
group: 'Group',
ori_leader_family,
) -> 'Leadership':
"""manage leader's variable
"""
if variable.multi is not True:
msg = _(f'the variable "{variable.name}" in a group must be multi')
raise DictConsistencyError(msg, 32, variable.xmlfiles)
if hasattr(group, 'name'):
ori_leadership_name = group.name
else:
ori_leadership_name = variable.name
leadership_name = normalize_family(ori_leadership_name)
leader_space = self.objectspace.leadership(variable.xmlfiles)
leader_space.variable = []
leader_space.name = leadership_name
leader_space.hidden = variable.hidden
if variable.hidden:
variable.frozen = True
variable.force_default_on_freeze = True
variable.hidden = None
if hasattr(group, 'description'):
leader_space.doc = group.description
elif variable.name == leadership_name and hasattr(variable, 'description'):
leader_space.doc = variable.description
else:
leader_space.doc = ori_leadership_name
leadership_path = ori_leader_family.path + '.' + leadership_name
self.objectspace.paths.add_leadership(variable.namespace,
leadership_path,
leader_space,
)
ori_leader_family.variable[normalize_family(variable.name)] = leader_space
leader_space.variable.append(variable)
self.objectspace.paths.set_leader(variable.namespace,
ori_leader_family.path,
leadership_name,
normalize_family(variable.name),
)
return leader_space
def manage_follower(self,
follower_name: str,
leader_family_name: str,
variable: 'Variable',
leader_space: 'Leadership',
) -> None:
"""manage follower
"""
if variable.name != follower_name:
msg = _('when parsing leadership, we expect to find the follower '
f'"{follower_name}" but we found "{variable.name}"')
raise DictConsistencyError(msg, 33, variable.xmlfiles)
self.objectspace.paths.set_leader(variable.namespace,
leader_family_name,
leader_space.name,
normalize_family(variable.name),
)
if leader_space.hidden:
variable.frozen = True
variable.force_default_on_freeze = True
leader_space.variable.append(variable)

147
src/rougail/annotator/param.py Normal file
View File

@ -0,0 +1,147 @@
"""Param annotator
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
try:
import tiramisu3 as tiramisu
except ModuleNotFoundError:
import tiramisu
from .variable import CONVERT_OPTION
from ..i18n import _
from ..error import DictConsistencyError
class ParamAnnotator:
"""Param annotator
"""
objectspace = None
@staticmethod
def valid_type_validation(obj) -> None:
"""Function to valid type (redefine in fill/condition/check classes)
"""
return None
def convert_param(self, objects) -> None:
""" valid and convert param
"""
for obj in objects:
if not hasattr(obj, 'param'):
continue
param_to_delete = []
variable_type = self.valid_type_validation(obj)
for param_idx, param in enumerate(obj.param):
if hasattr(param, 'text'):
if param.type in ['suffix', 'index']:
msg = _(f'"{param.type}" parameter must not have a value')
raise DictConsistencyError(msg, 28, obj.xmlfiles)
if param.type == 'nil':
if param.text is not None:
msg = _(f'"{param.type}" parameter must not have a value')
raise DictConsistencyError(msg, 40, obj.xmlfiles)
elif param.type == 'variable':
try:
path, suffix = self.objectspace.paths.get_variable_path(param.text,
obj.namespace,
param.xmlfiles,
)
param.text = self.objectspace.paths.get_variable(path)
if variable_type and param.text.type != variable_type:
msg = _(f'"{obj.name}" has type "{variable_type}" but param '
f'has type "{param.text.type}"')
raise DictConsistencyError(msg, 26, param.xmlfiles)
if suffix:
param.suffix = suffix
family_path = self.objectspace.paths.get_variable_family_path(path)
namespace = param.text.namespace
param.family = self.objectspace.paths.get_family(family_path,
namespace,
)
except DictConsistencyError as err:
if err.errno != 42 or not param.optional:
raise err
param_to_delete.append(param_idx)
elif param.type == 'function':
if not self.allow_function:
msg = _(f'cannot use "function" type')
raise DictConsistencyError(msg, 74, param.xmlfiles)
if not param.text in self.functions:
msg = _(f'cannot find function "{param.text}"')
raise DictConsistencyError(msg, 67, param.xmlfiles)
if param_idx != 0:
msg = _(f'function "{param.text}" must only set has first parameter')
raise DictConsistencyError(msg, 75, param.xmlfiles)
elif variable_type:
self._convert_with_variable_type(variable_type, param)
continue
# no param.text
if param.type == 'suffix':
for target in obj.target:
if not self.objectspace.paths.variable_is_dynamic(target.name.path):
msg = _(f'"{param.type}" parameter cannot be set with target '
f'"{target.name}" which is not a dynamic variable')
raise DictConsistencyError(msg, 53, obj.xmlfiles)
elif param.type == 'index':
for target in obj.target:
if not self.objectspace.paths.is_follower(target.name.path):
msg = _(f'"{param.type}" parameter cannot be set with target '
f'"{target.name.name}" which is not a follower variable')
raise DictConsistencyError(msg, 60, obj.xmlfiles)
elif param.type == 'nil':
param.text = None
elif param.type == 'string':
param.text = ''
if variable_type:
self._convert_with_variable_type(variable_type, param)
else:
msg = _(f'"{param.type}" parameter must have a value')
raise DictConsistencyError(msg, 27, obj.xmlfiles)
param_to_delete.sort(reverse=True)
for param_idx in param_to_delete:
obj.param.pop(param_idx)
@staticmethod
def _convert_with_variable_type(variable_type: str,
param: 'self.objectspace.param',
) -> None:
if 'type' in vars(param) and variable_type != param.type:
msg = _(f'parameter has incompatible type "{param.type}" '
f'with type "{variable_type}"')
raise DictConsistencyError(msg, 7, param.xmlfiles)
try:
option = CONVERT_OPTION[variable_type]
param.text = option.get('func', str)(param.text)
getattr(tiramisu, option['opttype'])('test',
'Object to valid value',
param.text,
**option.get('initkwargs', {}),
)
except ValueError as err:
msg = _(f'unable to change type of value "{param.text}" '
f'is not a valid "{variable_type}"')
raise DictConsistencyError(msg, 13, param.xmlfiles) from err
param.type = variable_type

108
src/rougail/annotator/property.py Normal file
View File

@ -0,0 +1,108 @@
"""Annotate properties
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from ..i18n import _
from ..error import DictConsistencyError
from .variable import Walk
PROPERTIES = ('hidden', 'frozen', 'force_default_on_freeze',
'force_store_value', 'disabled', 'mandatory')
class PropertyAnnotator(Walk):
"""Annotate properties
"""
def __init__(self, objectspace):
self.objectspace = objectspace
if hasattr(self.objectspace.space, 'services'):
self.convert_services()
if hasattr(self.objectspace.space, 'variables'):
self.convert_family()
self.convert_variable()
def convert_property(self,
variable,
) -> None:
"""convert properties
"""
# hidden variable is also frozen
if isinstance(variable, self.objectspace.variable) and variable.hidden is True:
if not variable.auto_freeze:
variable.frozen = True
if not variable.auto_save and \
not variable.auto_freeze and \
'force_default_on_freeze' not in vars(variable):
variable.force_default_on_freeze = True
if not hasattr(variable, 'properties'):
variable.properties = []
for prop in PROPERTIES:
if hasattr(variable, prop):
if getattr(variable, prop) is True:
# for subprop in CONVERT_PROPERTIES.get(prop, [prop]):
variable.properties.append(prop)
setattr(variable, prop, None)
if hasattr(variable, 'mode') and variable.mode:
variable.properties.append(variable.mode)
variable.mode = None
if 'force_store_value' in variable.properties and \
'force_default_on_freeze' in variable.properties: # pragma: no cover
# should not appened
msg = _('cannot have auto_freeze or auto_save with the hidden '
f'variable "{variable.name}"')
raise DictConsistencyError(msg, 50, variable.xmlfiles)
if not variable.properties:
del variable.properties
def convert_services(self) -> None:
"""convert services
"""
self.convert_property(self.objectspace.space.services)
for services in self.objectspace.space.services.service.values():
self.convert_property(services)
for service in vars(services).values():
if not isinstance(service, self.objectspace.family):
continue
self.convert_property(service)
for family in service.family:
self.convert_property(family)
for variable in family.variable:
self.convert_property(variable)
def convert_family(self) -> None:
"""convert families
"""
for family in self.get_families():
self.convert_property(family)
def convert_variable(self) -> None:
"""convert variables
"""
for variable in self.get_variables(with_leadership=True):
if isinstance(variable, self.objectspace.leadership):
for follower in variable.variable:
self.convert_property(follower)
self.convert_property(variable)

303
src/rougail/annotator/service.py Normal file
View File

@ -0,0 +1,303 @@
"""Annotate services
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from os.path import basename
from typing import Tuple
from ..i18n import _
from ..utils import normalize_family
from ..error import DictConsistencyError
from ..config import RougailConfig
# a object's attribute has some annotations
# that shall not be present in the exported (flatened) XML
ERASED_ATTRIBUTES = ('redefine', 'exists', 'optional', 'remove_check', 'namespace',
'remove_condition', 'path', 'instance_mode', 'index',
'level', 'remove_fill', 'xmlfiles', 'type', 'reflector_name',
'reflector_object',)
ALLOW_ATTRIBUT_NOT_MANAGE = ['file']
class ServiceAnnotator:
"""Manage service's object
for example::
<services>
<service name="test">
<service_access service='ntp'>
</service_access>
</service>
</services>
"""
def __init__(self, objectspace):
self.objectspace = objectspace
self.uniq_overrides = []
if 'network_type' not in self.objectspace.types:
self.objectspace.types['network_type'] = self.objectspace.types['ip_type']
if hasattr(self.objectspace.space, 'services'):
if not hasattr(self.objectspace.space.services, 'service'):
del self.objectspace.space.services
else:
self.convert_services()
def convert_services(self):
"""convert services to variables
"""
self.objectspace.space.services.hidden = True
self.objectspace.space.services.name = 'services'
self.objectspace.space.services.doc = 'services'
self.objectspace.space.services.path = 'services'
for service_name, service in self.objectspace.space.services.service.items():
activate_obj = self._generate_element('boolean',
None,
None,
'activate',
True,
service,
'.'.join(['services', normalize_family(service_name), 'activate']),
)
for elttype, values in dict(vars(service)).items():
if elttype == 'servicelist':
self.objectspace.list_conditions.setdefault('servicelist',
{}).setdefault(
values,
[]).append(activate_obj)
continue
if not isinstance(values, (dict, list)) or elttype in ERASED_ATTRIBUTES:
continue
if not service.manage and elttype not in ALLOW_ATTRIBUT_NOT_MANAGE:
msg = _(f'unmanage service cannot have "{elttype}"')
raise DictConsistencyError(msg, 66, service.xmlfiles)
if elttype != 'ip':
eltname = elttype + 's'
else:
eltname = elttype
path = '.'.join(['services', normalize_family(service_name), eltname])
family = self._gen_family(eltname,
path,
service.xmlfiles,
)
if isinstance(values, dict):
values = list(values.values())
family.family = self.make_group_from_elts(service_name,
elttype,
values,
path,
)
setattr(service, elttype, family)
manage = self._generate_element('boolean',
None,
None,
'manage',
service.manage,
service,
'.'.join(['services', normalize_family(service_name), 'manage']),
)
service.variable = [activate_obj, manage]
service.doc = service.name
def make_group_from_elts(self,
service_name,
elttype,
elts,
path,
):
"""Splits each objects into a group (and `OptionDescription`, in tiramisu terms)
and build elements and its attributes (the `Options` in tiramisu terms)
"""
families = []
listname = '{}list'.format(elttype)
for elt in elts:
# try to launch _update_xxxx() function
update_elt = '_update_' + elttype
if hasattr(self, update_elt):
getattr(self, update_elt)(elt,
service_name,
)
c_name, subpath = self._get_name_path(elt,
path,
)
family = self._gen_family(c_name,
subpath,
elt.xmlfiles,
)
family.variable = []
activate_obj = self._generate_element('boolean',
None,
None,
'activate',
True,
elt,
'.'.join([subpath, 'activate']),
)
for key in dir(elt):
if key.startswith('_') or key.endswith('_type') or key in ERASED_ATTRIBUTES:
continue
value = getattr(elt, key)
if key == listname:
self.objectspace.list_conditions.setdefault(listname,
{}).setdefault(
value,
[]).append(activate_obj)
continue
if key == 'name':
dtd_key_type = elttype + '_type'
else:
dtd_key_type = key + '_type'
elt_type = getattr(elt, dtd_key_type, 'string')
if elt_type == 'variable':
elt_type = 'symlink'
family.variable.append(self._generate_element(elt_type,
dtd_key_type,
elttype,
key,
value,
elt,
f'{subpath}.{key}'
))
family.variable.append(activate_obj)
families.append(family)
return families
def _get_name_path(self,
elt,
path: str,
) -> Tuple[str, str]:
# create element name, if already exists, add _xx to be uniq
if hasattr(elt, 'source') and elt.source:
name = elt.source
else:
name = elt.name
idx = 0
while True:
c_name = name
if idx:
c_name += f'_{idx}'
subpath = '{}.{}'.format(path, normalize_family(c_name))
try:
self.objectspace.paths.get_family(subpath, 'services')
except DictConsistencyError as err:
if err.errno == 42:
return c_name, subpath
idx += 1
def _gen_family(self,
name,
path,
xmlfiles
):
family = self.objectspace.family(xmlfiles)
family.name = normalize_family(name)
family.doc = name
family.mode = None
self.objectspace.paths.add_family('services',
path,
family,
None,
)
return family
def _generate_element(self,
type_,
dtd_key_type,
elttype,
key,
value,
elt,
path,
): # pylint: disable=R0913
variable = self.objectspace.variable(elt.xmlfiles)
variable.name = normalize_family(key)
variable.mode = None
variable.type = type_
if type_ == 'symlink':
variable.opt = self.objectspace.paths.get_variable(value)
variable.multi = None
needed_type = self.objectspace.types[dtd_key_type]
if needed_type not in ('variable', variable.opt.type):
msg = _(f'"{value}" in "{elttype}" must be a variable with type '
f'"{needed_type}" not "{variable.opt.type}"')
raise DictConsistencyError(msg, 58, elt.xmlfiles)
else:
variable.doc = key
variable.default = value
variable.namespace = 'services'
self.objectspace.paths.add_variable('services',
path,
'service',
False,
variable,
)
return variable
def _update_override(self,
override,
service_name,
):
if service_name in self.uniq_overrides:
msg = _('only one override is allowed by service, '
'please use a variable multiple if you want have more than one IP')
raise DictConsistencyError(msg, 69, override.xmlfiles)
self.uniq_overrides.append(service_name)
override.name = service_name
if not hasattr(override, 'engine'):
override.engine = RougailConfig['default_engine']
if not hasattr(override, 'source'):
override.source = f'{service_name}.service'
@staticmethod
def _update_file(file_,
service_name,
):
if not hasattr(file_, 'file_type') or file_.file_type == "filename":
if not hasattr(file_, 'source'):
file_.source = basename(file_.name)
elif not hasattr(file_, 'source'):
msg = _(f'attribute "source" is mandatory for the file "{file_.name}" '
f'"({service_name})"')
raise DictConsistencyError(msg, 34, file_.xmlfiles)
if not hasattr(file_, 'engine'):
file_.engine = RougailConfig['default_engine']
def _update_ip(self,
ip,
service_name,
) -> None:
variable = self.objectspace.paths.get_variable(ip.name, ip.xmlfiles)
if variable.type not in ['ip', 'network', 'network_cidr']:
msg = _(f'ip cannot be linked to "{variable.type}" variable "{ip.name}"')
raise DictConsistencyError(msg, 70, ip.xmlfiles)
if variable.type in ['ip', 'network_cidr'] and hasattr(ip, 'netmask'):
msg = _(f'ip with ip_type "{variable.type}" must not have netmask')
raise DictConsistencyError(msg, 59, ip.xmlfiles)
if variable.type == 'network' and not hasattr(ip, 'netmask'):
msg = _(f'ip with ip_type "{variable.type}" must have netmask')
raise DictConsistencyError(msg, 64, ip.xmlfiles)
if hasattr(ip, 'netmask'):
netmask = self.objectspace.paths.get_variable(ip.netmask, ip.xmlfiles)
if netmask.type != 'netmask':
msg = _(f'netmask in ip must have type "netmask", not "{netmask.type}"')
raise DictConsistencyError(msg, 65, ip.xmlfiles)

91
src/rougail/annotator/target.py Normal file
View File

@ -0,0 +1,91 @@
"""Target annotator
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from ..i18n import _
from ..error import DictConsistencyError
class TargetAnnotator:
"""Target annotator
"""
def convert_target(self,
objects,
) -> None:
""" valid and convert param
"""
targets = []
for obj in objects:
if not hasattr(obj, 'target'):
msg = _('target is mandatory')
raise DictConsistencyError(msg, 9, obj.xmlfiles)
remove_targets = []
for index, target in enumerate(obj.target):
# test if it's redefined calculation
if self.target_is_uniq and target.name in targets:
msg = _(f'A fill already exists for the target of "{target.name}" created')
raise DictConsistencyError(msg, 24, obj.xmlfiles)
targets.append(target.name)
# let's replace the target by the path
try:
if target.type == 'variable':
path, suffix = self.objectspace.paths.get_variable_path(target.name,
obj.namespace,
)
target.name = self.objectspace.paths.get_variable(path)
if suffix:
msg = _(f'target to {target.name.path} with suffix is not allowed')
raise DictConsistencyError(msg, 35, obj.xmlfiles)
elif self.only_variable:
msg = _(f'target to "{target.name}" with param type "{target.type}" '
f'is not allowed')
raise DictConsistencyError(msg, 8, obj.xmlfiles)
if target.type == 'family':
if obj.name not in ['disabled_if_in',
'disabled_if_not_in',
'hidden_if_in',
'hidden_if_not_in',
]:
msg = _(f'target "{target.type}" not allow with "{obj.name}"')
raise DictConsistencyError(msg, 51, target.xmlfiles)
target.name = self.objectspace.paths.get_family(target.name,
obj.namespace,
)
elif target.type.endswith('list') and \
obj.name not in ['disabled_if_in', 'disabled_if_not_in']:
msg = _(f'target "{target.type}" not allow with "{obj.name}"')
raise DictConsistencyError(msg, 10, target.xmlfiles)
except DictConsistencyError as err:
if err.errno != 42:
raise err
# for optional variable
if not target.optional:
msg = f'cannot found target "{target.type}" "{target.name}"'
raise DictConsistencyError(_(msg), 12, target.xmlfiles) from err
remove_targets.append(index)
remove_targets.sort(reverse=True)
for index in remove_targets:
obj.target.pop(index)

98
src/rougail/annotator/value.py Normal file
View File

@ -0,0 +1,98 @@
"""Annotate value
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from .variable import Walk
from ..i18n import _
from ..error import DictConsistencyError
class ValueAnnotator(Walk): # pylint: disable=R0903
"""Annotate value
"""
def __init__(self,
objectspace,
) -> None:
if not hasattr(objectspace.space, 'variables'):
return
self.objectspace = objectspace
self.convert_value()
def convert_value(self) -> None:
"""convert value
"""
for variable in self.get_variables(with_leadership=True):
if isinstance(variable, self.objectspace.leadership):
variable_type = 'leader'
for follower in variable.variable:
self._convert_value(follower,
variable_type,
)
variable_type = 'follower'
else:
self._convert_value(variable)
def _convert_value(self,
variable,
variable_type: str=None,
) -> None:
# a boolean must have value, the default value is "True"
if not hasattr(variable, 'value') and variable.type == 'boolean':
new_value = self.objectspace.value(variable.xmlfiles)
new_value.name = True
new_value.type = 'boolean'
variable.value = [new_value]
# if the variable is mandatory and doesn't have any value
# then the variable's mode is set to 'basic'
# variable with default value is mandatory
if hasattr(variable, 'value') and variable.value:
has_value = True
for value in variable.value:
if value.type == 'calculation':
has_value = False
break
if has_value and 'mandatory' not in vars(variable):
# if has value without any calculation
variable.mandatory = True
if not hasattr(variable, 'value'):
return
if variable.value[0].type == 'calculation':
variable.default = variable.value[0]
else:
if variable.multi:
if variable_type != 'follower':
variable.default = [value.name for value in variable.value]
if variable_type != 'leader':
if variable.multi == 'submulti':
variable.default_multi = [value.name for value in variable.value]
else:
variable.default_multi = variable.value[0].name
else:
if len(variable.value) > 1:
msg = _(f'the non multi variable "{variable.name}" cannot have '
'more than one value')
raise DictConsistencyError(msg, 68, variable.xmlfiles)
variable.default = variable.value[0].name
del variable.value

240
src/rougail/annotator/variable.py Normal file
View File

@ -0,0 +1,240 @@
"""Annotate variable
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from ..i18n import _
from ..error import DictConsistencyError
from ..objspace import convert_boolean
CONVERT_OPTION = {'number': dict(opttype="IntOption", func=int),
'float': dict(opttype="FloatOption", func=float),
'choice': dict(opttype="ChoiceOption"),
'string': dict(opttype="StrOption"),
'password': dict(opttype="PasswordOption"),
'mail': dict(opttype="EmailOption"),
'boolean': dict(opttype="BoolOption", func=convert_boolean),
'symlink': dict(opttype="SymLinkOption"),
'filename': dict(opttype="FilenameOption"),
'date': dict(opttype="DateOption"),
'unix_user': dict(opttype="UsernameOption"),
'ip': dict(opttype="IPOption", initkwargs={'allow_reserved': True}),
'local_ip': dict(opttype="IPOption", initkwargs={'private_only': True,
'warnings_only': True}),
'netmask': dict(opttype="NetmaskOption"),
'network': dict(opttype="NetworkOption"),
'broadcast': dict(opttype="BroadcastOption"),
'netbios': dict(opttype="DomainnameOption", initkwargs={'type': 'netbios',
'warnings_only': True}),
'domainname': dict(opttype="DomainnameOption", initkwargs={'type': 'domainname',
'allow_ip': False}),
'hostname': dict(opttype="DomainnameOption", initkwargs={'type': 'hostname',
'allow_ip': False}),
'web_address': dict(opttype="URLOption", initkwargs={'allow_ip': True,
'allow_without_dot': True}),
'port': dict(opttype="PortOption", initkwargs={'allow_private': True}),
'mac': dict(opttype="MACOption"),
'cidr': dict(opttype="IPOption", initkwargs={'cidr': True}),
'network_cidr': dict(opttype="NetworkOption", initkwargs={'cidr': True}),
}
FORCE_CHOICE = {'schedule': ['none', 'daily', 'weekly', 'monthly'],
'schedulemod': ['pre', 'post'],
}
class Walk:
"""Walk to objectspace to find variable or family
"""
objectspace = None
def get_variables(self,
with_leadership: bool=False,
):
"""Iter all variables from the objectspace
"""
for family in self.objectspace.space.variables.values():
yield from self._get_variables(family, with_leadership)
def _get_variables(self,
family: 'self.objectspace.family',
with_leadership: bool
):
if hasattr(family, 'variable'):
for variable in family.variable.values():
if isinstance(variable, self.objectspace.family):
yield from self._get_variables(variable, with_leadership)
continue
if not with_leadership and isinstance(variable, self.objectspace.leadership):
for follower in variable.variable:
yield follower
continue
yield variable
def get_families(self):
"""Iter all families from the objectspace
"""
for family in self.objectspace.space.variables.values():
yield from self._get_families(family)
def _get_families(self,
family: 'self.objectspace.family',
):
yield family
if hasattr(family, 'variable'):
for fam in family.variable.values():
if isinstance(fam, self.objectspace.family):
yield from self._get_families(fam)
class VariableAnnotator(Walk): # pylint: disable=R0903
"""Annotate variable
"""
def __init__(self,
objectspace,
):
if not hasattr(objectspace.space, 'variables'):
return
self.objectspace = objectspace
self.forbidden_name = ['services', self.objectspace.rougailconfig['variable_namespace']]
for extra in self.objectspace.rougailconfig['extra_dictionaries']:
self.forbidden_name.append(extra)
self.convert_variable()
self.convert_test()
self.convert_help()
def convert_variable(self):
"""convert variable
"""
for variable in self.get_variables(with_leadership=True):
if isinstance(variable, self.objectspace.leadership):
# first variable is a leader, others are follower
variable_type = 'leader'
for follower in variable.variable:
self._convert_variable(follower,
variable_type,
)
variable_type = 'follower'
else:
self._convert_variable(variable,
'variable',
)
def _convert_variable(self,
variable,
variable_type: str,
) -> None:
if variable.namespace == self.objectspace.rougailconfig['variable_namespace'] and \
variable.name in self.forbidden_name:
msg = _(f'the name of the variable "{variable.name}" cannot be the same as the name'
' of a namespace')
raise DictConsistencyError(msg, 54, variable.xmlfiles)
if variable.type != 'symlink' and not hasattr(variable, 'description'):
variable.description = variable.name
if hasattr(variable, 'value'):
value_to_del = []
for idx, value in enumerate(variable.value):
if not hasattr(value, 'name'):
value_to_del.append(idx)
else:
if not hasattr(value, 'type'):
value.type = variable.type
value.name = CONVERT_OPTION.get(value.type, {}).get('func', str)(value.name)
value_to_del.sort(reverse=True)
for idx in value_to_del:
del variable.value[idx]
if not variable.value:
del variable.value
variable.doc = variable.description
del variable.description
if variable_type == 'follower':
if variable.multi is True:
variable.multi = 'submulti'
else:
variable.multi = True
self._convert_valid_enum(variable)
def _convert_valid_enum(self,
variable,
):
"""some types are, in fact, choices
convert this kind of variables into choice
"""
if variable.type in FORCE_CHOICE:
if not hasattr(self.objectspace.space, 'constraints'):
xmlfiles = variable.xmlfiles
self.objectspace.space.constraints = self.objectspace.constraints(xmlfiles)
self.objectspace.space.constraints.namespace = variable.namespace
if not hasattr(self.objectspace.space.constraints, 'check'):
self.objectspace.space.constraints.check = []
check = self.objectspace.check(variable.xmlfiles)
check.name = 'valid_enum'
target = self.objectspace.target(variable.xmlfiles)
target.name = variable.path
check.target = [target]
check.namespace = variable.namespace
check.param = []
for value in FORCE_CHOICE[variable.type]:
param = self.objectspace.param(variable.xmlfiles)
param.text = value
check.param.append(param)
self.objectspace.space.constraints.check.append(check)
variable.type = 'string'
def convert_test(self):
"""Convert variable tests value
"""
for variable in self.get_variables():
if not hasattr(variable, 'test'):
continue
if variable.test:
if not hasattr(variable, 'information'):
variable.information = self.objectspace.information(variable.xmlfiles)
values = variable.test.split('|')
new_values = []
for value in values:
if value == '':
value = None
else:
value = CONVERT_OPTION.get(variable.type, {}).get('func', str)(value)
new_values.append(value)
variable.information.test = tuple(new_values)
del variable.test
def convert_help(self):
"""Convert variable help
"""
for variable in self.get_variables():
if not hasattr(variable, 'information'):
variable.information = self.objectspace.information(variable.xmlfiles)
self._convert_help(variable)
@staticmethod
def _convert_help(variable) -> None:
if hasattr(variable, 'help'):
variable.information.help = variable.help
del variable.help

52
src/rougail/config.py
View File

@ -1,22 +1,50 @@
# -*- coding: utf-8 -*-
""" """
fichier de configuration pour rougail Config file for Rougail
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
""" """
from os.path import join, abspath, dirname from os.path import join, abspath, dirname
rougailroot = '/var/rougail' ROUGAILROOT = '/srv/rougail'
dtddir = join(dirname(abspath(__file__)), 'data') DTDDIR = join(dirname(abspath(__file__)), 'data')
Config = {'rougailroot': rougailroot, RougailConfig = {'dictionaries_dir': [join(ROUGAILROOT, 'dictionaries')],
'patch_dir': join(rougailroot, 'patches'), 'extra_dictionaries': {},
'manifests_dir': join(rougailroot, 'manifests'), 'patches_dir': join(ROUGAILROOT, 'patches'),
'templates_dir': join(rougailroot, 'templates'), 'templates_dir': join(ROUGAILROOT, 'templates'),
'dtdfilename': join(dtddir, 'rougail.dtd'), 'destinations_dir': join(ROUGAILROOT, 'destinations'),
'dtddir': dtddir, 'tmp_dir': join(ROUGAILROOT, 'tmp'),
# chemin du répertoire source des fichiers templates 'dtdfilename': join(DTDDIR, 'rougail.dtd'),
'patch_dir': '/srv/rougail/patch', 'functions_file': join(ROUGAILROOT, 'functions.py'),
'variable_namespace': 'rougail', 'variable_namespace': 'rougail',
'auto_freeze_variable': 'server_deployed',
'default_engine': 'creole',
'internal_functions': [],
'modes_level': ['basic', 'normal', 'expert'],
'default_family_mode': 'basic',
'default_variable_mode': 'normal',
} }

109
src/rougail/convert.py Normal file
View File

@ -0,0 +1,109 @@
"""Takes a bunch of Rougail XML dispatched in differents folders
as an input and outputs a Tiramisu's file.
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Sample usage::
>>> from rougail import RougailConvert
>>> rougail = RougailConvert()
>>> tiramisu = rougail.save('tiramisu.py')
The Rougail
- loads the XML into an internal RougailObjSpace representation
- visits/annotates the objects
- dumps the object space as Tiramisu string
The visit/annotation stage is a complex step that corresponds to the Rougail
procedures.
"""
from typing import List
from .i18n import _
from .config import RougailConfig
from .objspace import RougailObjSpace
from .xmlreflector import XMLReflector
from .tiramisureflector import TiramisuReflector
from .annotator import SpaceAnnotator
from .error import DictConsistencyError
class RougailConvert:
"""Rougail object
"""
def __init__(self,
rougailconfig: RougailConfig=None,
) -> None:
if rougailconfig is None:
rougailconfig = RougailConfig
xmlreflector = XMLReflector(rougailconfig)
rougailobjspace = RougailObjSpace(xmlreflector,
rougailconfig,
)
self._load_dictionaries(xmlreflector,
rougailobjspace,
rougailconfig['variable_namespace'],
rougailconfig['dictionaries_dir'],
)
for namespace, extra_dir in rougailconfig['extra_dictionaries'].items():
if namespace in ['services', rougailconfig['variable_namespace']]:
msg = _(f'Namespace name "{namespace}" is not allowed')
raise DictConsistencyError(msg, 21, None)
self._load_dictionaries(xmlreflector,
rougailobjspace,
namespace,
extra_dir,
)
functions_file = rougailconfig['functions_file']
SpaceAnnotator(rougailobjspace,
functions_file,
)
self.output = TiramisuReflector(rougailobjspace,
functions_file,
).get_text() + '\n'
@staticmethod
def _load_dictionaries(xmlreflector: XMLReflector,
rougailobjspace: RougailObjSpace,
namespace: str,
xmlfolders: List[str],
) -> List[str]:
for xmlfile, document in xmlreflector.load_xml_from_folders(xmlfolders):
rougailobjspace.xml_parse_document(xmlfile,
document,
namespace,
)
def save(self,
filename: str,
) -> str:
"""Return tiramisu object declaration as a string
"""
if filename:
with open(filename, 'w') as tiramisu:
tiramisu.write(self.output)
return self.output

89
src/rougail/data/rougail.dtd
View File

@ -13,7 +13,7 @@
# Forked by: # Forked by:
# Cadoles (http://www.cadoles.com) # Cadoles (http://www.cadoles.com)
# Copyright (C) 2019-2020 # Copyright (C) 2019-2021
# distribued with GPL-2 or later license # distribued with GPL-2 or later license
@ -37,7 +37,8 @@
<!-- root element --> <!-- root element -->
<!-- =============== --> <!-- =============== -->
<!ELEMENT rougail (services | variables | constraints | help)*> <!ELEMENT rougail (services|variables|constraints)*>
<!ATTLIST rougail version (0.9) #REQUIRED>
<!-- ============== --> <!-- ============== -->
<!-- files element --> <!-- files element -->
@ -45,26 +46,19 @@
<!ELEMENT services (service*)> <!ELEMENT services (service*)>
<!ELEMENT service ((port* | tcpwrapper* | ip* | interface* | package* | file* | override*)*) > <!ELEMENT service ((ip*|file*|override*)*)>
<!ATTLIST service name CDATA #REQUIRED> <!ATTLIST service name CDATA #REQUIRED>
<!ATTLIST service manage (True|False) "True"> <!ATTLIST service manage (True|False) "True">
<!ATTLIST service servicelist CDATA #IMPLIED>
<!ELEMENT port (#PCDATA)>
<!ATTLIST port port_type (PortOption|SymLinkOption|variable) "PortOption">
<!ATTLIST port portlist CDATA #IMPLIED >
<!ATTLIST port protocol (tcp|udp) "tcp">
<!ELEMENT ip (#PCDATA)> <!ELEMENT ip (#PCDATA)>
<!ATTLIST ip iplist CDATA #IMPLIED> <!ATTLIST ip iplist CDATA #IMPLIED>
<!ATTLIST ip ip_type (NetworkOption|SymLinkOption|variable) "NetworkOption"> <!ATTLIST ip ip_type (variable) "variable">
<!ATTLIST ip interface_type (UnicodeOption|SymLinkOption|variable) "UnicodeOption"> <!ATTLIST ip netmask_type (variable) "variable">
<!ATTLIST ip interface CDATA #REQUIRED> <!ATTLIST ip netmask CDATA #IMPLIED>
<!ATTLIST ip netmask_type (NetmaskOption|SymLinkOption|variable) "NetmaskOption">
<!ATTLIST ip netmask CDATA "255.255.255.255">
<!ELEMENT file EMPTY> <!ELEMENT file (#PCDATA)>
<!ATTLIST file name CDATA #REQUIRED > <!ATTLIST file file_type (filename|variable) "filename">
<!ATTLIST file file_type (UnicodeOption|SymLinkOption|variable) "UnicodeOption">
<!ATTLIST file variable CDATA #IMPLIED> <!ATTLIST file variable CDATA #IMPLIED>
<!ATTLIST file variable_type (variable) "variable"> <!ATTLIST file variable_type (variable) "variable">
<!ATTLIST file source CDATA #IMPLIED> <!ATTLIST file source CDATA #IMPLIED>
@ -73,24 +67,28 @@
<!ATTLIST file group CDATA "root"> <!ATTLIST file group CDATA "root">
<!ATTLIST file filelist CDATA #IMPLIED> <!ATTLIST file filelist CDATA #IMPLIED>
<!ATTLIST file redefine (True|False) "False"> <!ATTLIST file redefine (True|False) "False">
<!ATTLIST file templating (True|False) "True"> <!ATTLIST file engine (none|creole|jinja2|creole_legacy) #IMPLIED>
<!ATTLIST file included (no|name|content) "no">
<!ELEMENT override EMPTY> <!ELEMENT override EMPTY>
<!ATTLIST override source CDATA #IMPLIED> <!ATTLIST override source CDATA #IMPLIED>
<!ATTLIST override templating (True|False) "True"> <!ATTLIST override engine (none|creole|jinja2) #IMPLIED>
<!ELEMENT variables (family*, separators*)> <!ELEMENT variables ((variable*|family*)*)>
<!ELEMENT family (#PCDATA | variable)*>
<!ELEMENT family ((variable*|family*)*)>
<!ATTLIST family name CDATA #REQUIRED> <!ATTLIST family name CDATA #REQUIRED>
<!ATTLIST family description CDATA #IMPLIED> <!ATTLIST family description CDATA #IMPLIED>
<!ATTLIST family mode (basic|normal|expert) "basic"> <!ATTLIST family help CDATA #IMPLIED>
<!ATTLIST family mode CDATA #IMPLIED>
<!ATTLIST family hidden (True|False) "False"> <!ATTLIST family hidden (True|False) "False">
<!ATTLIST family dynamic CDATA #IMPLIED> <!ATTLIST family dynamic CDATA #IMPLIED>
<!ELEMENT variable (#PCDATA | value)*> <!ELEMENT variable (value*)>
<!ATTLIST variable name CDATA #REQUIRED> <!ATTLIST variable name CDATA #REQUIRED>
<!ATTLIST variable type CDATA #IMPLIED> <!ATTLIST variable type (number|float|string|password|mail|boolean|filename|date|unix_user|ip|local_ip|netmask|network|broadcast|netbios|domainname|hostname|web_address|port|mac|cidr|network_cidr|schedule|schedulemod) "string">
<!ATTLIST variable description CDATA #IMPLIED> <!ATTLIST variable description CDATA #IMPLIED>
<!ATTLIST variable help CDATA #IMPLIED>
<!ATTLIST variable hidden (True|False) "False"> <!ATTLIST variable hidden (True|False) "False">
<!ATTLIST variable disabled (True|False) "False"> <!ATTLIST variable disabled (True|False) "False">
<!ATTLIST variable multi (True|False) "False"> <!ATTLIST variable multi (True|False) "False">
@ -99,51 +97,42 @@
<!ATTLIST variable mandatory (True|False) "False"> <!ATTLIST variable mandatory (True|False) "False">
<!ATTLIST variable auto_freeze (True|False) "False"> <!ATTLIST variable auto_freeze (True|False) "False">
<!ATTLIST variable auto_save (True|False) "False"> <!ATTLIST variable auto_save (True|False) "False">
<!ATTLIST variable mode (basic|normal|expert) "normal"> <!ATTLIST variable mode CDATA #IMPLIED>
<!ATTLIST variable remove_check (True|False) "False"> <!ATTLIST variable remove_check (True|False) "False">
<!ATTLIST variable remove_condition (True|False) "False"> <!ATTLIST variable remove_condition (True|False) "False">
<!ATTLIST variable remove_fill (True|False) "False"> <!ATTLIST variable remove_fill (True|False) "False">
<!ATTLIST variable test CDATA #IMPLIED> <!ATTLIST variable test CDATA #IMPLIED>
<!ELEMENT separators (separator*)>
<!ELEMENT separator (#PCDATA)>
<!ATTLIST separator name CDATA #REQUIRED>
<!ELEMENT value (#PCDATA)> <!ELEMENT value (#PCDATA)>
<!ELEMENT constraints ((fill*|check*|condition*|group*)*)> <!ELEMENT constraints ((fill*|check*|condition*|group*)*)>
<!ELEMENT fill (param*)>
<!ATTLIST fill name CDATA #REQUIRED>
<!ATTLIST fill target CDATA #REQUIRED>
<!ELEMENT check (param*)> <!ELEMENT fill ((target|param)+)>
<!ATTLIST fill name CDATA #REQUIRED>
<!ELEMENT check ((target|param)+)>
<!ATTLIST check name CDATA #REQUIRED> <!ATTLIST check name CDATA #REQUIRED>
<!ATTLIST check target CDATA #REQUIRED>
<!ATTLIST check level (error|warning) "error"> <!ATTLIST check level (error|warning) "error">
<!ELEMENT condition ((target|param)+)> <!ELEMENT condition ((target|param)+)>
<!ATTLIST condition name (disabled_if_in|disabled_if_not_in|hidden_if_in|auto_hidden_if_not_in|hidden_if_not_in|mandatory_if_in|mandatory_if_not_in) #REQUIRED> <!ATTLIST condition name (disabled_if_in|disabled_if_not_in|hidden_if_in|hidden_if_not_in|mandatory_if_in|mandatory_if_not_in) #REQUIRED>
<!ATTLIST condition source CDATA #REQUIRED> <!ATTLIST condition source CDATA #REQUIRED>
<!ATTLIST condition fallback (True|False) "False"> <!ATTLIST condition optional (True|False) "False">
<!ATTLIST condition force_condition_on_fallback (True|False) "False"> <!ATTLIST condition apply_on_fallback (True|False) #IMPLIED>
<!ATTLIST condition force_inverse_condition_on_fallback (True|False) "False">
<!ELEMENT param (#PCDATA)>
<!ATTLIST param type (string|number|nil|boolean|variable|function|information|target_information|suffix|index) "string">
<!ATTLIST param name CDATA #IMPLIED>
<!ATTLIST param propertyerror (True|False) "True">
<!ATTLIST param optional (True|False) "False">
<!ELEMENT target (#PCDATA)>
<!ATTLIST target type (variable|family|servicelist|filelist|iplist) "variable">
<!ATTLIST target optional (True|False) "False">
<!ELEMENT group (follower+)> <!ELEMENT group (follower+)>
<!ATTLIST group leader CDATA #REQUIRED> <!ATTLIST group leader CDATA #REQUIRED>
<!ATTLIST group name CDATA #IMPLIED> <!ATTLIST group name CDATA #IMPLIED>
<!ATTLIST group description CDATA #IMPLIED> <!ATTLIST group description CDATA #IMPLIED>
<!ELEMENT param (#PCDATA)>
<!ATTLIST param type (string|number|variable|information|suffix) "string">
<!ATTLIST param name CDATA #IMPLIED>
<!ATTLIST param notraisepropertyerror (True|False) "False">
<!ATTLIST param optional (True|False) "False">
<!ELEMENT target (#PCDATA)>
<!ATTLIST target type (family|variable|filelist|iplist|portlist) "variable">
<!ATTLIST target optional (True|False) "False">
<!ELEMENT follower (#PCDATA)> <!ELEMENT follower (#PCDATA)>
<!ELEMENT help ((variable* | family*)*)>

66
src/rougail/error.py
View File

@ -1,22 +1,58 @@
# -*- coding: utf-8 -*- """Standard error classes
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from .i18n import _
def display_xmlfiles(xmlfiles: list) -> str:
"""The function format xmlfiles informations to generate errors
"""
if len(xmlfiles) == 1:
return '"' + xmlfiles[0] + '"'
return '"' + '", "'.join(xmlfiles[:-1]) + '"' + ' and ' + '"' + xmlfiles[-1] + '"'
class ConfigError(Exception): class ConfigError(Exception):
pass """Standard error for templating
"""
class FileNotFound(ConfigError): class FileNotFound(ConfigError):
pass """Template file is not found
"""
class TemplateError(ConfigError): class TemplateError(ConfigError):
pass """Templating generate an error
"""
class TemplateDisabled(TemplateError): class TemplateDisabled(TemplateError):
"""Template is disabled. """Template is disabled.
""" """
pass
class OperationError(Exception):
"""Type error or value Error for Creole variable's type or values
"""
class SpaceObjShallNotBeUpdated(Exception): class SpaceObjShallNotBeUpdated(Exception):
@ -29,7 +65,13 @@ class DictConsistencyError(Exception):
"""It's not only that the Creole XML is valid against the Creole DTD """It's not only that the Creole XML is valid against the Creole DTD
it's that it is not consistent. it's that it is not consistent.
""" """
def __init__(self, msg, errno, xmlfiles):
if xmlfiles:
msg = _(f'{msg} in {display_xmlfiles(xmlfiles)}')
super().__init__(msg)
self.errno = errno
class LoaderError(Exception): class UpgradeError(Exception):
pass """Error during XML upgrade
"""

45
src/rougail/i18n.py
View File

@ -1,23 +1,28 @@
# -*- coding: UTF-8 -*- """Internationalisation utilities
# Copyright (C) 2012-2013 Team tiramisu (see AUTHORS for all contributors) Created by:
# EOLE (http://eole.orion.education.fr)
# This program is free software: you can redistribute it and/or modify it Copyright (C) 2005-2018
# under the terms of the GNU Lesser General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your Forked by:
# option) any later version. Cadoles (http://www.cadoles.com)
# Copyright (C) 2019-2021
# This program is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS distribued with GPL-2 or later license
# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
# details. This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# You should have received a copy of the GNU Lesser General Public License the Free Software Foundation; either version 2 of the License, or
# along with this program. If not, see <http://www.gnu.org/licenses/>. (at your option) any later version.
#
# The original `Config` design model is unproudly borrowed from This program is distributed in the hope that it will be useful,
# the rough gus of pypy: pypy: http://codespeak.net/svn/pypy/dist/pypy/config/ but WITHOUT ANY WARRANTY; without even the implied warranty of
# the whole pypy projet is under MIT licence MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
"internationalisation utilities" GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
import gettext import gettext
import os import os
import sys import sys

740
src/rougail/objspace.py
View File

@ -1,514 +1,413 @@
"""parse XML files and build a space with objects
it aggregates this files and manage redefine and exists attributes
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
""" """
Creole flattener. Takes a bunch of Creole XML dispatched in differents folders from typing import Optional
as an input and outputs a human readable flatened XML
Sample usage::
>>> from rougail.objspace import CreoleObjSpace
>>> eolobj = CreoleObjSpace('/usr/share/rougail/rougail.dtd')
>>> eolobj.create_or_populate_from_xml('rougail', ['/usr/share/eole/rougail/dicos'])
>>> eolobj.space_visitor()
>>> eolobj.save('/tmp/rougail_flatened_output.xml')
The CreoleObjSpace
- loads the XML into an internal CreoleObjSpace representation
- visits/annotates the objects
- dumps the object space as XML output into a single XML target
The visit/annotation stage is a complex step that corresponds to the Creole
procedures.
For example: a variable is redefined and shall be moved to another family
means that a variable1 = Variable() object in the object space who lives in the family1 parent
has to be moved in family2. The visit procedure changes the varable1's object space's parent.
"""
from lxml.etree import Element, SubElement # pylint: disable=E0611
from .i18n import _ from .i18n import _
from .xmlreflector import XMLReflector from .xmlreflector import XMLReflector
from .annotator import ERASED_ATTRIBUTES, SpaceAnnotator from .utils import valid_variable_family_name
from .tiramisureflector import TiramisuReflector from .error import SpaceObjShallNotBeUpdated, DictConsistencyError
from .utils import normalize_family
from .error import OperationError, SpaceObjShallNotBeUpdated, DictConsistencyError
from .path import Path from .path import Path
from .config import Config
# CreoleObjSpace's elements like 'family' or 'follower', that shall be forced to the Redefinable type # RougailObjSpace's elements that shall be forced to the Redefinable type
FORCE_REDEFINABLES = ('family', 'follower', 'service', 'disknod', 'variables') FORCE_REDEFINABLES = ('family', 'follower', 'service', 'disknod', 'variables')
# CreoleObjSpace's elements that shall be forced to the UnRedefinable type # RougailObjSpace's elements that shall be forced to the UnRedefinable type
FORCE_UNREDEFINABLES = ('value',) FORCE_UNREDEFINABLES = ('value',)
# CreoleObjSpace's elements that shall be set to the UnRedefinable type # RougailObjSpace's elements that shall not be modify
UNREDEFINABLE = ('multi', 'type') UNREDEFINABLE = ('multi', 'type')
# RougailObjSpace's elements that did not created automaticly
FORCE_ELEMENTS = ('choice', 'property_', 'leadership', 'information')
# XML text are convert has name
FORCED_TEXT_ELTS_AS_NAME = ('choice', 'property', 'value',)
CONVERT_PROPERTIES = {'auto_save': ['force_store_value'], 'auto_freeze': ['force_store_value', 'auto_freeze']} FORCE_TAG = {'family': 'variable'}
RENAME_ATTIBUTES = {'description': 'doc'}
FORCED_TEXT_ELTS_AS_NAME = ('choice', 'property', 'value', 'target')
CONVERT_EXPORT = {'Leadership': 'leader',
'Variable': 'variable',
'Value': 'value',
'Property': 'property',
'Choice': 'choice',
'Param': 'param',
'Check': 'check',
}
# _____________________________________________________________________________ # _____________________________________________________________________________
# special types definitions for the Object Space's internal representation # special types definitions for the Object Space's internal representation
class RootCreoleObject: class RootRougailObject: # pylint: disable=R0903
def __init__(self, xmlfiles): """Root object
"""
def __init__(self,
xmlfiles,
name=None,
):
if not isinstance(xmlfiles, list): if not isinstance(xmlfiles, list):
xmlfiles = [xmlfiles] xmlfiles = [xmlfiles]
self.xmlfiles = xmlfiles self.xmlfiles = xmlfiles
if name:
self.name = name
class CreoleObjSpace: class Atom(RootRougailObject): # pylint: disable=R0903
"""DOM XML reflexion free internal representation of a Creole Dictionary """Atomic object (means can only define one time)
"""
class Redefinable(RootRougailObject): # pylint: disable=R0903
"""Object that could be redefine
"""
class UnRedefinable(RootRougailObject): # pylint: disable=R0903
"""Object that could not be redefine
""" """
choice = type('Choice', (RootCreoleObject,), dict())
property_ = type('Property', (RootCreoleObject,), dict())
# Creole ObjectSpace's Leadership variable class type
Leadership = type('Leadership', (RootCreoleObject,), dict())
"""
This Atom type stands for singleton, that is
an Object Space's atom object is present only once in the
object space's tree
"""
Atom = type('Atom', (RootCreoleObject,), dict())
"A variable that can't be redefined"
Redefinable = type('Redefinable', (RootCreoleObject,), dict())
"A variable can be redefined"
UnRedefinable = type('UnRedefinable', (RootCreoleObject,), dict())
def __init__(self, dtdfilename): # pylint: disable=R0912
self.index = 0
class ObjSpace: # pylint: disable=R0903 class ObjSpace: # pylint: disable=R0903
""" """
Base object space Base object space
""" """
self.space = ObjSpace()
self.paths = Path()
self.xmlreflector = XMLReflector()
self.xmlreflector.parse_dtd(dtdfilename)
self.redefine_variables = None
self.fill_removed = None
self.check_removed = None
self.condition_removed = None
# ['variable', 'separator', 'family']
self.forced_text_elts = set() def convert_boolean(value: str) -> bool:
"""Boolean coercion. The Rougail XML may contain srings like `True` or `False`
"""
if isinstance(value, bool):
return value
if value == 'True':
return True
return False
class RougailObjSpace:
"""Rougail ObjectSpace is an object's reflexion of the XML elements
"""
def __init__(self,
xmlreflector: XMLReflector,
rougailconfig: 'RougailConfig',
) -> None:
self.space = ObjSpace()
self.paths = Path(rougailconfig)
self.forced_text_elts_as_name = set(FORCED_TEXT_ELTS_AS_NAME) self.forced_text_elts_as_name = set(FORCED_TEXT_ELTS_AS_NAME)
self.list_conditions = {} self.list_conditions = {}
self.valid_enums = {}
self.booleans_attributs = [] self.booleans_attributs = []
self.has_dyn_option = False
self.types = {}
self.make_object_space_class() self.make_object_space_classes(xmlreflector)
self.rougailconfig = rougailconfig
def make_object_space_class(self): def make_object_space_classes(self,
"""Create Rougail ObjectSpace class types, it enables us to create objects like: xmlreflector: XMLReflector,
) -> None:
"""Create Rougail ObjectSpace class types from DDT file
It enables us to create objects like:
File(), Variable(), Ip(), Family(), Constraints()... and so on. File(), Variable(), Ip(), Family(), Constraints()... and so on.
"""
Creole ObjectSpace is an object's reflexion of the XML elements""" for dtd_elt in xmlreflector.dtd.iterelements():
for dtd_elt in self.xmlreflector.dtd.iterelements():
attrs = {} attrs = {}
if dtd_elt.name in FORCE_REDEFINABLES: if dtd_elt.name in FORCE_REDEFINABLES:
clstype = self.Redefinable clstype = Redefinable
elif not dtd_elt.attributes() and dtd_elt.name not in FORCE_UNREDEFINABLES:
clstype = Atom
else: else:
clstype = self.UnRedefinable clstype = UnRedefinable
atomic = dtd_elt.name not in FORCE_UNREDEFINABLES and dtd_elt.name not in FORCE_REDEFINABLES
forced_text_elt = dtd_elt.type == 'mixed' forced_text_elt = dtd_elt.type == 'mixed'
for dtd_attr in dtd_elt.iterattributes(): for dtd_attr in dtd_elt.iterattributes():
atomic = False if set(dtd_attr.itervalues()) == {'True', 'False'}:
if set(dtd_attr.itervalues()) == set(['True', 'False']):
# it's a boolean # it's a boolean
self.booleans_attributs.append(dtd_attr.name) self.booleans_attributs.append(dtd_attr.name)
if dtd_attr.default_value: if dtd_attr.default_value:
# set default value for this attribute # set default value for this attribute
default_value = dtd_attr.default_value default_value = dtd_attr.default_value
if dtd_attr.name in self.booleans_attributs: if dtd_attr.name in self.booleans_attributs:
default_value = self.convert_boolean(dtd_attr.default_value) default_value = convert_boolean(default_value)
attrs[dtd_attr.name] = default_value attrs[dtd_attr.name] = default_value
if dtd_attr.name.endswith('_type'):
self.types[dtd_attr.name] = default_value
if dtd_attr.name == 'redefine': if dtd_attr.name == 'redefine':
# has a redefine attribute, so it's a Redefinable object # has a redefine attribute, so it's a Redefinable object
clstype = self.Redefinable clstype = Redefinable
if dtd_attr.name == 'name' and forced_text_elt: if dtd_attr.name == 'name' and forced_text_elt:
# child.text should be transform has a "name" attribute # child.text should be transform has a "name" attribute
self.forced_text_elts.add(dtd_elt.name)
forced_text_elt = False forced_text_elt = False
if forced_text_elt is True: if forced_text_elt is True:
self.forced_text_elts_as_name.add(dtd_elt.name) self.forced_text_elts_as_name.add(dtd_elt.name)
if atomic:
# has any attribute so it's an Atomic object
clstype = self.Atom
# create ObjectSpace object # create ObjectSpace object
setattr(self, dtd_elt.name, type(dtd_elt.name.capitalize(), (clstype,), attrs)) setattr(self, dtd_elt.name, type(dtd_elt.name.capitalize(), (clstype,), attrs))
for elt in FORCE_ELEMENTS:
setattr(self, elt, type(self._get_elt_name(elt), (RootRougailObject,), dict()))
def create_or_populate_from_xml(self, @staticmethod
namespace, def _get_elt_name(elt) -> str:
xmlfolders): name = elt.capitalize()
"""Parses a bunch of XML files if name.endswith('_'):
populates the CreoleObjSpace name = name[:-1]
""" return name
for xmlfile, document in self.xmlreflector.load_xml_from_folders(xmlfolders):
self.redefine_variables = []
self.fill_removed = []
self.check_removed = []
self.condition_removed = []
self.xml_parse_document(xmlfile,
document,
self.space,
namespace,
)
def xml_parse_document(self, def xml_parse_document(self,
xmlfile, xmlfile,
document, document,
space,
namespace, namespace,
): ):
"""Parses a Creole XML file """Parses a Rougail XML file and populates the RougailObjSpace
populates the CreoleObjSpace
""" """
redefine_variables = []
self._xml_parse(xmlfile,
document,
self.space,
namespace,
redefine_variables,
)
def _xml_parse(self, # pylint: disable=R0913
xmlfile,
document,
space,
namespace,
redefine_variables,
) -> None:
# var to check unique family name in a XML file
family_names = [] family_names = []
for child in document: for child in document:
# this index enables us to reorder objects
self.index += 1
# doesn't proceed the XML commentaries
if not isinstance(child.tag, str): if not isinstance(child.tag, str):
# doesn't proceed the XML commentaries
continue continue
if child.tag == 'family': if child.tag == 'family':
if child.attrib['name'] in family_names: if child.attrib['name'] in family_names:
raise DictConsistencyError(_(f'Family "{child.attrib["name"]}" is set several times in "{xmlfile}"')) msg = _(f'Family "{child.attrib["name"]}" is set several times')
raise DictConsistencyError(msg, 44, [xmlfile])
family_names.append(child.attrib['name']) family_names.append(child.attrib['name'])
if child.tag == 'variables':
child.attrib['name'] = namespace
if child.tag == 'value' and child.text == None:
# FIXME should not be here
continue
# variable objects creation
try: try:
variableobj = self.generate_variableobj(xmlfile, # variable objects creation
exists, variableobj = self.get_variableobj(xmlfile,
child, child,
space, space,
namespace, namespace,
redefine_variables,
) )
except SpaceObjShallNotBeUpdated: except SpaceObjShallNotBeUpdated:
continue continue
self.set_text_to_obj(child, self.set_text(child,
variableobj, variableobj,
) )
self.set_xml_attributes_to_obj(xmlfile, self.set_attributes(xmlfile,
child, child,
variableobj, variableobj,
) )
self.variableobj_tree_visitor(child, self.remove(child,
variableobj, variableobj,
namespace, redefine_variables,
) )
self.fill_variableobj_path_attribute(space, if not exists:
child, self.set_path(namespace,
namespace,
document, document,
variableobj, variableobj,
space,
) )
self.add_to_tree_structure(variableobj, self.add_to_tree_structure(variableobj,
space, space,
child, child,
namespace,
) )
if list(child) != []: if list(child) != []:
self.xml_parse_document(xmlfile, self._xml_parse(xmlfile,
child, child,
variableobj, variableobj,
namespace, namespace,
redefine_variables,
) )
def generate_variableobj(self, def get_variableobj(self,
xmlfile, xmlfile: str,
child, child: list,
space, space,
namespace, namespace,
): redefine_variables,
): # pylint: disable=R0913
""" """
instanciates or creates Creole Object Subspace objects retrieves or creates Rougail Object Subspace objects
""" """
variableobj = getattr(self, child.tag)(xmlfile) tag = FORCE_TAG.get(child.tag, child.tag)
if isinstance(variableobj, self.Redefinable): obj = getattr(self, tag)
variableobj = self.create_or_update_redefinable_object(xmlfile, name = self._get_name(child, namespace)
if Redefinable in obj.__mro__:
return self.create_or_update_redefinable_object(xmlfile,
child.attrib, child.attrib,
space, space,
child, child,
name,
namespace, namespace,
redefine_variables,
) )
elif isinstance(variableobj, self.Atom) and child.tag in vars(space): if Atom in obj.__mro__:
# instanciates an object from the CreoleObjSpace's builtins types if child.tag in vars(space):
# example : child.tag = constraints -> a self.Constraints() object is created # Atom instance has to be a singleton here
# this Atom instance has to be a singleton here
# we do not re-create it, we reuse it # we do not re-create it, we reuse it
variableobj = getattr(space, child.tag) return False, getattr(space, child.tag)
self.create_tree_structure(space, return False, obj(xmlfile, name)
# UnRedefinable object
if child.tag not in vars(space):
setattr(space, child.tag, [])
return False, obj(xmlfile, name)
def _get_name(self,
child, child,
variableobj, namespace: str,
) ) -> Optional[str]:
return variableobj if child.tag == 'variables':
return namespace
if 'name' in child.attrib:
return child.attrib['name']
if child.text and child.tag in self.forced_text_elts_as_name:
return child.text.strip()
return None
def create_or_update_redefinable_object(self, def create_or_update_redefinable_object(self,
xmlfile, xmlfile,
subspace, subspace,
space, space,
child, child,
name,
namespace, namespace,
): redefine_variables,
"""Creates or retrieves the space object that corresponds ): # pylint: disable=R0913
to the `child` XML object """A redefinable object could be created or updated
Two attributes of the `child` XML object are important:
- with the `redefine` boolean flag attribute we know whether
the corresponding space object shall be created or updated
- `True` means that the corresponding space object shall be updated
- `False` means that the corresponding space object shall be created
- with the `exists` boolean flag attribute we know whether
the corresponding space object shall be created
(or nothing -- that is the space object isn't modified)
- `True` means that the corresponding space object shall be created
- `False` means that the corresponding space object is not updated
In the special case `redefine` is True and `exists` is False,
we create the corresponding space object if it doesn't exist
and we update it if it exists.
:return: the corresponding space object of the `child` XML object
""" """
if child.tag in self.forced_text_elts_as_name: existed_var = self.get_existed_obj(name,
name = child.text xmlfile,
else:
name = subspace['name']
existed_var = self.is_already_exists(name,
space, space,
child, child,
namespace, namespace,
) )
if existed_var: if existed_var:
# if redefine is set to object, default value is False
# otherwise it's always a redefinable object
default_redefine = child.tag in FORCE_REDEFINABLES default_redefine = child.tag in FORCE_REDEFINABLES
redefine = self.convert_boolean(subspace.get('redefine', default_redefine)) redefine = convert_boolean(subspace.get('redefine', default_redefine))
exists = self.convert_boolean(subspace.get('exists', True))
if redefine is True: if redefine is True:
existed_var.xmlfiles.append(xmlfile) if isinstance(existed_var, self.variable): # pylint: disable=E1101
return self.translate_in_space(name, if namespace == self.rougailconfig['variable_namespace']:
space, redefine_variables.append(name)
child,
namespace,
)
elif exists is False:
raise SpaceObjShallNotBeUpdated()
xmlfiles = self.display_xmlfiles(existed_var.xmlfiles)
raise DictConsistencyError(_(f'"{child.tag}" named "{name}" cannot be re-created in "{xmlfile}", already defined in {xmlfiles}'))
redefine = self.convert_boolean(subspace.get('redefine', False))
exists = self.convert_boolean(subspace.get('exists', False))
if redefine is False or exists is True:
return getattr(self, child.tag)(xmlfile)
raise DictConsistencyError(_(f'Redefined object in "{xmlfile}": "{name}" does not exist yet'))
def display_xmlfiles(self,
xmlfiles: list,
) -> str:
if len(xmlfiles) == 1:
return '"' + xmlfiles[0] + '"'
else: else:
return '"' + '", "'.join(xmlfiles[:-1]) + '"' + ' and ' + '"' + xmlfiles[-1] + '"' redefine_variables.append(space.path + '.' + name)
existed_var.xmlfiles.append(xmlfile)
return True, existed_var
exists = convert_boolean(subspace.get('exists', True))
if exists is False:
raise SpaceObjShallNotBeUpdated()
msg = _(f'"{child.tag}" named "{name}" cannot be re-created in "{xmlfile}", '
f'already defined')
raise DictConsistencyError(msg, 45, existed_var.xmlfiles)
# object deos not exists
exists = convert_boolean(subspace.get('exists', False))
if exists is True:
# manage object only if already exists, so cancel
raise SpaceObjShallNotBeUpdated()
redefine = convert_boolean(subspace.get('redefine', False))
if redefine is True:
# cannot redefine an inexistant object
msg = _(f'Redefined object: "{name}" does not exist yet')
raise DictConsistencyError(msg, 46, [xmlfile])
tag = FORCE_TAG.get(child.tag, child.tag)
if tag not in vars(space):
setattr(space, tag, {})
obj = getattr(self, child.tag)(xmlfile, name)
return False, obj
def create_tree_structure(self, def get_existed_obj(self,
space,
child,
variableobj,
): # pylint: disable=R0201
"""
Builds the tree structure of the object space here
we set services attributes in order to be populated later on
for example::
space = Family()
space.variable = dict()
another example:
space = Variable()
space.value = list()
"""
if child.tag not in vars(space):
if isinstance(variableobj, self.Redefinable):
setattr(space, child.tag, dict())
elif isinstance(variableobj, self.UnRedefinable):
setattr(space, child.tag, [])
elif not isinstance(variableobj, self.Atom): # pragma: no cover
raise OperationError(_("Creole object {} "
"has a wrong type").format(type(variableobj)))
def is_already_exists(self,
name: str, name: str,
xmlfile: str,
space: str, space: str,
child, child,
namespace: str, namespace: str,
): ) -> None:
if isinstance(space, self.family): # pylint: disable=E1101 """if an object exists, return it
if namespace != Config['variable_namespace']:
name = space.path + '.' + name
if self.paths.path_is_defined(name):
return self.paths.get_variable_obj(name)
return
if child.tag == 'family':
norm_name = normalize_family(name)
else:
norm_name = name
children = getattr(space, child.tag, {})
if norm_name in children:
return children[norm_name]
def convert_boolean(self, value): # pylint: disable=R0201
"""Boolean coercion. The Creole XML may contain srings like `True` or `False`
""" """
if isinstance(value, bool): if child.tag in ['variable', 'family']:
return value valid_variable_family_name(name, [xmlfile])
if value == 'True': if child.tag == 'variable': # pylint: disable=E1101
return True if namespace != self.rougailconfig['variable_namespace']:
elif value == 'False': name = space.path + '.' + name
return False if not self.paths.path_is_defined(name):
else: return None
raise TypeError(_('{} is not True or False').format(value)) # pragma: no cover old_family_name = self.paths.get_variable_family_path(name)
if space.path != old_family_name:
msg = _(f'Variable "{name}" was previously create in family "{old_family_name}", '
f'now it is in "{space.path}"')
raise DictConsistencyError(msg, 47, space.xmlfiles)
return self.paths.get_variable(name)
# it's not a family
tag = FORCE_TAG.get(child.tag, child.tag)
children = getattr(space, tag, {})
if name in children and isinstance(children[name], getattr(self, child.tag)):
return children[name]
return None
def translate_in_space(self, def set_text(self,
name,
family,
variable,
namespace,
):
if not isinstance(family, self.family): # pylint: disable=E1101
if variable.tag == 'family':
norm_name = normalize_family(name)
else:
norm_name = name
return getattr(family, variable.tag)[norm_name]
if namespace == Config['variable_namespace']:
path = name
else:
path = family.path + '.' + name
old_family_name = self.paths.get_variable_family_name(path)
if normalize_family(family.name) == old_family_name:
return getattr(family, variable.tag)[name]
old_family = self.space.variables[Config['variable_namespace']].family[old_family_name] # pylint: disable=E1101
variable_obj = old_family.variable[name]
del old_family.variable[name]
if 'variable' not in vars(family):
family.variable = dict()
family.variable[name] = variable_obj
self.paths.add_variable(namespace,
name,
family.name,
False,
variable_obj,
)
return variable_obj
def remove_fill(self, name): # pylint: disable=C0111
if hasattr(self.space, 'constraints') and hasattr(self.space.constraints, 'fill'):
remove_fills= []
for idx, fill in enumerate(self.space.constraints.fill): # pylint: disable=E1101
if hasattr(fill, 'target') and fill.target == name:
remove_fills.append(idx)
remove_fills = list(set(remove_fills))
remove_fills.sort(reverse=True)
for idx in remove_fills:
self.space.constraints.fill.pop(idx) # pylint: disable=E1101
def remove_check(self, name): # pylint: disable=C0111
if hasattr(self.space, 'constraints') and hasattr(self.space.constraints, 'check'):
remove_checks = []
for idx, check in enumerate(self.space.constraints.check): # pylint: disable=E1101
if hasattr(check, 'target') and check.target == name:
remove_checks.append(idx)
remove_checks = list(set(remove_checks))
remove_checks.sort(reverse=True)
for idx in remove_checks:
self.space.constraints.check.pop(idx) # pylint: disable=E1101
def remove_condition(self, name): # pylint: disable=C0111
for idx, condition in enumerate(self.space.constraints.condition): # pylint: disable=E1101
remove_targets = []
if hasattr(condition, 'target'):
for target_idx, target in enumerate(condition.target):
if target.name == name:
remove_targets.append(target_idx)
remove_targets = list(set(remove_targets))
remove_targets.sort(reverse=True)
for idx in remove_targets:
del condition.target[idx]
def add_to_tree_structure(self,
variableobj,
space,
child,
): # pylint: disable=R0201
if isinstance(variableobj, self.Redefinable):
name = variableobj.name
if child.tag == 'family':
name = normalize_family(name)
getattr(space, child.tag)[name] = variableobj
elif isinstance(variableobj, self.UnRedefinable):
getattr(space, child.tag).append(variableobj)
else:
setattr(space, child.tag, variableobj)
def set_text_to_obj(self,
child, child,
variableobj, variableobj,
): ) -> None:
if child.text is None: """set text
text = None """
else: if child.text is None or child.tag in self.forced_text_elts_as_name:
return
text = child.text.strip() text = child.text.strip()
if text: if text:
if child.tag in self.forced_text_elts_as_name:
variableobj.name = text
else:
variableobj.text = text variableobj.text = text
def set_xml_attributes_to_obj(self, def set_attributes(self,
xmlfile, xmlfile,
child, child,
variableobj, variableobj,
): ):
redefine = self.convert_boolean(child.attrib.get('redefine', False)) """ set attributes to an object
"""
redefine = convert_boolean(child.attrib.get('redefine', False))
if redefine and child.tag == 'variable':
# delete old values
has_value = hasattr(variableobj, 'value') has_value = hasattr(variableobj, 'value')
if redefine is True and child.tag == 'variable' and has_value and len(child) != 0: if has_value and len(child) != 0:
del variableobj.value del variableobj.value
for attr, val in child.attrib.items(): for attr, val in child.attrib.items():
if redefine and attr in UNREDEFINABLE: if redefine and attr in UNREDEFINABLE:
# UNREDEFINABLE concerns only 'variable' node so we can fix name msg = _(f'cannot redefine attribute "{attr}" for variable "{child.attrib["name"]}"'
# to child.attrib['name'] f' in "{xmlfile}", already defined')
name = child.attrib['name'] raise DictConsistencyError(msg, 48, variableobj.xmlfiles[:-1])
xmlfiles = self.display_xmlfiles(variableobj.xmlfiles[:-1])
raise DictConsistencyError(_(f'cannot redefine attribute "{attr}" for variable "{name}" in "{xmlfile}", already defined in {xmlfiles}'))
if attr in self.booleans_attributs: if attr in self.booleans_attributs:
val = self.convert_boolean(val) val = convert_boolean(val)
if not (attr == 'name' and getattr(variableobj, 'name', None) != None): if attr == 'name' and getattr(variableobj, 'name', None):
# do not redefine name
continue
setattr(variableobj, attr, val) setattr(variableobj, attr, val)
keys = list(vars(variableobj).keys())
def variableobj_tree_visitor(self, def remove(self,
child, child,
variableobj, variableobj,
namespace, redefine_variables,
): ):
"""Creole object tree manipulations """Rougail object tree manipulations
""" """
if child.tag == 'variable': if child.tag == 'variable':
if child.attrib.get('remove_check', False): if child.attrib.get('remove_check', False):
@ -518,64 +417,95 @@ class CreoleObjSpace:
if child.attrib.get('remove_fill', False): if child.attrib.get('remove_fill', False):
self.remove_fill(variableobj.name) self.remove_fill(variableobj.name)
if child.tag == 'fill': if child.tag == 'fill':
# if variable is a redefine in current dictionary for target in child:
# XXX not working with variable not in variable and in leader/followers if target.tag == 'target' and target.text in redefine_variables:
variableobj.redefine = child.attrib['target'] in self.redefine_variables self.remove_fill(target.text)
if child.attrib['target'] in self.redefine_variables and child.attrib['target'] not in self.fill_removed:
self.remove_fill(child.attrib['target'])
self.fill_removed.append(child.attrib['target'])
if not hasattr(variableobj, 'index'):
variableobj.index = self.index
if child.tag == 'check' and child.attrib['target'] in self.redefine_variables and child.attrib['target'] not in self.check_removed:
self.remove_check(child.attrib['target'])
self.check_removed.append(child.attrib['target'])
if child.tag == 'condition' and child.attrib['source'] in self.redefine_variables and child.attrib['source'] not in self.condition_removed:
self.remove_condition(child.attrib['source'])
self.condition_removed.append(child.attrib['source'])
variableobj.namespace = namespace
def fill_variableobj_path_attribute(self, def remove_check(self, name):
space, """Remove a check with a specified target
child, """
remove_checks = []
for idx, check in enumerate(self.space.constraints.check): # pylint: disable=E1101
for target in check.target:
if target.name == name:
remove_checks.append(idx)
remove_checks.sort(reverse=True)
for idx in remove_checks:
self.space.constraints.check.pop(idx) # pylint: disable=E1101
def remove_condition(self,
name: str,
) -> None:
"""Remove a condition with a specified source
"""
remove_conditions = []
for idx, condition in enumerate(self.space.constraints.condition): # pylint: disable=E1101
if condition.source == name:
remove_conditions.append(idx)
remove_conditions.sort(reverse=True)
for idx in remove_conditions:
del self.space.constraints.condition[idx] # pylint: disable=E1101
def remove_fill(self,
name: str,
) -> None:
"""Remove a fill with a specified target
"""
remove_fills = []
for idx, fill in enumerate(self.space.constraints.fill): # pylint: disable=E1101
for target in fill.target:
if target.name == name:
remove_fills.append(idx)
remove_fills.sort(reverse=True)
for idx in remove_fills:
self.space.constraints.fill.pop(idx) # pylint: disable=E1101
def set_path(self,
namespace, namespace,
document, document,
variableobj, variableobj,
): # pylint: disable=R0913 space,
):
"""Fill self.paths attributes """Fill self.paths attributes
""" """
if isinstance(space, self.help): # pylint: disable=E1101 if isinstance(variableobj, self.variable): # pylint: disable=E1101
return if 'name' in document.attrib:
if child.tag == 'variable': family_name = document.attrib['name']
family_name = normalize_family(document.attrib['name'])
self.paths.add_variable(namespace,
child.attrib['name'],
family_name,
document.attrib.get('dynamic') != None,
variableobj)
if child.attrib.get('redefine', 'False') == 'True':
if namespace == Config['variable_namespace']:
self.redefine_variables.append(child.attrib['name'])
else: else:
self.redefine_variables.append(namespace + '.' + family_name + '.' + family_name = namespace
child.attrib['name'])
elif child.tag == 'family': self.paths.add_variable(namespace,
family_name = normalize_family(child.attrib['name']) variableobj.name,
if namespace != Config['variable_namespace']: space.path,
document.attrib.get('dynamic') is not None,
variableobj,
)
elif isinstance(variableobj, self.family): # pylint: disable=E1101
family_name = variableobj.name
if namespace != self.rougailconfig['variable_namespace']:
family_name = namespace + '.' + family_name family_name = namespace + '.' + family_name
self.paths.add_family(namespace, self.paths.add_family(namespace,
family_name, family_name,
variableobj, variableobj,
space.path,
) )
variableobj.path = self.paths.get_family_path(family_name, namespace) elif isinstance(variableobj, self.variables):
variableobj.path = variableobj.name
def space_visitor(self, eosfunc_file): # pylint: disable=C0111 @staticmethod
self.funcs_path = eosfunc_file def add_to_tree_structure(variableobj,
SpaceAnnotator(self, eosfunc_file) space,
child,
def save(self, namespace: str,
): ) -> None:
tiramisu_objects = TiramisuReflector(self.space, """add a variable to the tree
self.funcs_path, """
) variableobj.namespace = namespace
return tiramisu_objects.get_text() + '\n' if isinstance(variableobj, Redefinable):
name = variableobj.name
tag = FORCE_TAG.get(child.tag, child.tag)
getattr(space, tag)[name] = variableobj
elif isinstance(variableobj, UnRedefinable):
getattr(space, child.tag).append(variableobj)
else:
setattr(space, child.tag, variableobj)

281
src/rougail/path.py
View File

@ -1,209 +1,250 @@
"""Manage path to find objects
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from typing import List
from .i18n import _ from .i18n import _
from .error import DictConsistencyError
from .utils import normalize_family from .utils import normalize_family
from .error import OperationError, DictConsistencyError
from .config import Config
class Path: class Path:
"""Helper class to handle the `path` attribute of a CreoleObjSpace """Helper class to handle the `path` attribute.
instance.
sample: path="creole.general.condition" sample: path="creole.general.condition"
""" """
def __init__(self): def __init__(self,
rougailconfig: 'RougailConfig',
) -> None:
self.variables = {} self.variables = {}
self.families = {} self.families = {}
self.full_paths_families = {} self.full_paths_families = {}
self.full_paths_variables = {} self.full_paths_variables = {}
self.variable_namespace = rougailconfig['variable_namespace']
# Family # Family
def add_family(self, def add_family(self,
namespace: str, namespace: str,
name: str, name: str,
variableobj: str, variableobj: str,
subpath: str,
) -> str: # pylint: disable=C0111 ) -> str: # pylint: disable=C0111
if '.' not in name and namespace == Config['variable_namespace']: """Add a new family
full_name = '.'.join([namespace, name]) """
if namespace == self.variable_namespace:
full_name = '.'.join([subpath, name])
if name in self.full_paths_families:
msg = _(f'Duplicate family name "{name}"')
raise DictConsistencyError(msg, 55, variableobj.xmlfiles)
self.full_paths_families[name] = full_name self.full_paths_families[name] = full_name
else: else:
if '.' not in name: # pragma: no cover
msg = _(f'Variable "{name}" in namespace "{namespace}" must have dot')
raise DictConsistencyError(msg, 39, variableobj.xmlfiles)
full_name = name full_name = name
if full_name in self.families and self.families[full_name]['variableobj'] != variableobj: if full_name in self.families and \
raise DictConsistencyError(_(f'Duplicate family name {name}')) self.families[full_name]['variableobj'] != variableobj: # pragma: no cover
msg = _(f'Duplicate family name "{name}"')
raise DictConsistencyError(msg, 37, variableobj.xmlfiles)
if full_name in self.variables:
msg = _(f'A variable and a family has the same path "{full_name}"')
raise DictConsistencyError(msg, 56, variableobj.xmlfiles)
self.families[full_name] = dict(name=name, self.families[full_name] = dict(name=name,
namespace=namespace, namespace=namespace,
variableobj=variableobj, variableobj=variableobj,
) )
variableobj.path = full_name
def get_family_path(self, def add_leadership(self,
namespace: str,
path: str,
variableobj: str,
) -> str: # pylint: disable=C0111
"""add a new leadership
"""
self.families[path] = dict(name=path,
namespace=namespace,
variableobj=variableobj,
)
variableobj.path = path
def get_family(self,
name: str, name: str,
current_namespace: str, current_namespace: str,
) -> str: # pylint: disable=C0111
name = normalize_family(name,
check_name=False,
allow_dot=True,
)
if '.' not in name and current_namespace == Config['variable_namespace'] and name in self.full_paths_families:
name = self.full_paths_families[name]
if current_namespace is None: # pragma: no cover
raise OperationError('current_namespace must not be None')
dico = self.families[name]
if dico['namespace'] != Config['variable_namespace'] and current_namespace != dico['namespace']:
raise DictConsistencyError(_('A family located in the {} namespace '
'shall not be used in the {} namespace').format(
dico['namespace'], current_namespace))
return dico['name']
def get_family_obj(self,
name: str,
) -> 'Family': # pylint: disable=C0111 ) -> 'Family': # pylint: disable=C0111
if '.' not in name and name in self.full_paths_families: """Get a family
"""
name = '.'.join([normalize_family(subname) for subname in name.split('.')])
if name not in self.families and name in self.full_paths_families:
name = self.full_paths_families[name] name = self.full_paths_families[name]
if name not in self.families: if name not in self.families:
raise DictConsistencyError(_('unknown family {}').format(name)) raise DictConsistencyError(_(f'unknown option {name}'), 42, [])
dico = self.families[name] dico = self.families[name]
if current_namespace not in [self.variable_namespace, 'services'] and \
current_namespace != dico['namespace']:
msg = _(f'A family located in the "{dico["namespace"]}" namespace '
f'shall not be used in the "{current_namespace}" namespace')
raise DictConsistencyError(msg, 38, [])
return dico['variableobj'] return dico['variableobj']
def family_is_defined(self,
name: str,
) -> str: # pylint: disable=C0111
if '.' not in name and name not in self.families and name in self.full_paths_families:
return True
return name in self.families
# Leadership # Leadership
def set_leader(self, def set_leader(self,
namespace: str, namespace: str,
leader_family_name: str, leader_family_name: str,
leadership_name: str,
name: str, name: str,
leader_name: str,
) -> None: # pylint: disable=C0111 ) -> None: # pylint: disable=C0111
"""set a variable a leadership member
"""
# need rebuild path and move object in new path # need rebuild path and move object in new path
old_path = namespace + '.' + leader_family_name + '.' + name old_path = leader_family_name + '.' + name
dico = self._get_variable(old_path) leadership_path = leader_family_name + '.' + leadership_name
del self.variables[old_path] new_path = leadership_path + '.' + name
new_path = namespace + '.' + leader_family_name + '.' + leader_name + '.' + name self.variables[new_path] = self.variables.pop(old_path)
self.add_variable(namespace, self.variables[new_path]['leader'] = leadership_path
new_path, self.variables[new_path]['variableobj'].path = new_path
dico['family'], self.variables[new_path]['family'] = leadership_path
False, if namespace == self.variable_namespace:
dico['variableobj'],
)
if namespace == Config['variable_namespace']:
self.full_paths_variables[name] = new_path self.full_paths_variables[name] = new_path
else:
name = new_path
dico = self._get_variable(name)
if dico['leader'] != None:
raise DictConsistencyError(_('Already defined leader {} for variable'
' {}'.format(dico['leader'], name)))
dico['leader'] = leader_name
def get_leader(self, name): # pylint: disable=C0111 def is_leader(self, path): # pylint: disable=C0111
return self._get_variable(name)['leader'] """Is the variable is a leader
"""
variable = self._get_variable(path)
if not variable['leader']:
return False
leadership = self.get_family(variable['leader'], variable['variableobj'].namespace)
return leadership.variable[0].path == path
def is_follower(self, path):
"""Is the variable is a follower
"""
variable = self._get_variable(path)
if not variable['leader']:
return False
leadership = self.get_family(variable['leader'], variable['variableobj'].namespace)
return leadership.variable[0].path != path
# Variable # Variable
def add_variable(self, def add_variable(self, # pylint: disable=R0913
namespace: str, namespace: str,
name: str, name: str,
family: str, family: str,
is_dynamic: bool, is_dynamic: bool,
variableobj, variableobj,
) -> str: # pylint: disable=C0111 ) -> str: # pylint: disable=C0111
"""Add a new variable (with path)
"""
if '.' not in name: if '.' not in name:
full_name = '.'.join([namespace, family, name]) full_path = '.'.join([family, name])
self.full_paths_variables[name] = full_name if namespace == self.variable_namespace:
self.full_paths_variables[name] = full_path
else: else:
full_name = name full_path = name
if namespace == Config['variable_namespace']: variableobj.path = full_path
name = name.rsplit('.', 1)[1] if full_path in self.families:
self.variables[full_name] = dict(name=name, msg = _(f'A family and a variable has the same path "{full_path}"')
raise DictConsistencyError(msg, 57, variableobj.xmlfiles)
self.variables[full_path] = dict(name=name,
family=family, family=family,
namespace=namespace,
leader=None, leader=None,
is_dynamic=is_dynamic, is_dynamic=is_dynamic,
variableobj=variableobj) variableobj=variableobj,
)
def get_variable_name(self, def get_variable(self,
name,
): # pylint: disable=C0111
return self._get_variable(name)['name']
def get_variable_obj(self,
name: str, name: str,
xmlfiles: List[str]=[],
) -> 'Variable': # pylint: disable=C0111 ) -> 'Variable': # pylint: disable=C0111
return self._get_variable(name)['variableobj'] """Get variable object from a path
"""
variable, suffix = self._get_variable(name, with_suffix=True, xmlfiles=xmlfiles)
if suffix:
raise DictConsistencyError(_(f"{name} is a dynamic variable"), 36, [])
return variable['variableobj']
def get_variable_family_name(self, def get_variable_family_path(self,
name: str, name: str,
xmlfiles: List[str]=False,
) -> str: # pylint: disable=C0111 ) -> str: # pylint: disable=C0111
return self._get_variable(name)['family'] """Get the full path of a family
"""
def get_variable_namespace(self, return self._get_variable(name, xmlfiles=xmlfiles)['family']
name: str,
) -> str: # pylint: disable=C0111
return self._get_variable(name)['namespace']
def get_variable_path(self, def get_variable_path(self,
name: str, name: str,
current_namespace: str, current_namespace: str,
allow_source: str=False, xmlfiles: List[str]=[],
with_suffix: bool=False,
) -> str: # pylint: disable=C0111 ) -> str: # pylint: disable=C0111
if current_namespace is None: # pragma: no cover """get full path of a variable
raise OperationError('current_namespace must not be None') """
if with_suffix:
dico, suffix = self._get_variable(name, dico, suffix = self._get_variable(name,
with_suffix=True, with_suffix=True,
xmlfiles=xmlfiles,
) )
else: namespace = dico['variableobj'].namespace
dico = self._get_variable(name) if namespace not in [self.variable_namespace, 'services'] and \
if not allow_source: current_namespace != namespace:
if dico['namespace'] not in [Config['variable_namespace'], 'services'] and current_namespace != dico['namespace']: msg = _(f'A variable located in the "{namespace}" namespace shall not be used '
raise DictConsistencyError(_('A variable located in the {} namespace ' f'in the "{current_namespace}" namespace')
'shall not be used in the {} namespace').format( raise DictConsistencyError(msg, 41, xmlfiles)
dico['namespace'], current_namespace)) return dico['variableobj'].path, suffix
if '.' in dico['name']:
value = dico['name']
else:
list_path = [dico['namespace'], dico['family']]
if dico['leader'] is not None:
list_path.append(dico['leader'])
list_path.append(dico['name'])
value = '.'.join(list_path)
if with_suffix:
return value, suffix
return value
def path_is_defined(self, def path_is_defined(self,
name: str, path: str,
) -> str: # pylint: disable=C0111 ) -> str: # pylint: disable=C0111
if '.' not in name and name not in self.variables and name in self.full_paths_variables: """The path is a valid path
"""
if '.' not in path and path not in self.variables and path in self.full_paths_variables:
return True return True
return name in self.variables return path in self.variables
def variable_is_dynamic(self,
name: str,
) -> bool:
"""This variable is in dynamic family
"""
return self._get_variable(name)['is_dynamic']
def _get_variable(self, def _get_variable(self,
name: str, name: str,
with_suffix: bool=False, with_suffix: bool=False,
xmlfiles: List[str]=[],
) -> str: ) -> str:
if name not in self.variables: name = '.'.join([normalize_family(subname) for subname in name.split('.')])
if name not in self.variables: if name not in self.variables:
if '.' not in name and name in self.full_paths_variables: if '.' not in name and name in self.full_paths_variables:
name = self.full_paths_variables[name] name = self.full_paths_variables[name]
if name not in self.variables: elif with_suffix:
for var_name, variable in self.variables.items(): for var_name, full_path in self.full_paths_variables.items():
if variable['is_dynamic'] and name.startswith(var_name):
if not with_suffix:
raise Exception('This option is dynamic, should use "with_suffix" attribute')
return variable, name[len(var_name):]
if '.' not in name:
for var_name, path in self.full_paths_variables.items():
if name.startswith(var_name): if name.startswith(var_name):
variable = self.variables[self.full_paths_variables[var_name]] variable = self._get_variable(full_path)
if variable['is_dynamic']: if variable['is_dynamic']:
if not with_suffix:
raise Exception('This option is dynamic, should use "with_suffix" attribute')
return variable, name[len(var_name):] return variable, name[len(var_name):]
raise DictConsistencyError(_('unknown option {}').format(name)) if name not in self.variables:
raise DictConsistencyError(_(f'unknown option "{name}"'), 42, xmlfiles)
if with_suffix: if with_suffix:
return self.variables[name], None return self.variables[name], None
return self.variables[name] return self.variables[name]

452
src/rougail/template.py
View File

@ -1,452 +0,0 @@
# -*- coding: utf-8 -*-
"""
Gestion du mini-langage de template
On travaille sur les fichiers cibles
"""
import imp
from shutil import copy
import logging
from typing import Dict, Any
from subprocess import call
from os import listdir, makedirs, getcwd, chdir
from os.path import dirname, join, isfile, abspath, normpath, relpath
from Cheetah.Template import Template as ChtTemplate
from Cheetah.NameMapper import NotFound as CheetahNotFound
try:
from tiramisu3 import Config
from tiramisu3.error import PropertiesOptionError
except:
from tiramisu import Config
from tiramisu.error import PropertiesOptionError
from .config import Config
from .error import FileNotFound, TemplateError
from .i18n import _
from .utils import normalize_family
log = logging.getLogger(__name__)
log.addHandler(logging.NullHandler())
class IsDefined:
"""
filtre permettant de ne pas lever d'exception au cas où
la variable Creole n'est pas définie
"""
def __init__(self, context):
self.context = context
def __call__(self, varname):
if '.' in varname:
splitted_var = varname.split('.')
if len(splitted_var) != 2:
msg = _("Group variables must be of type leader.follower")
raise KeyError(msg)
leader, follower = splitted_var
if leader in self.context:
return follower in self.context[leader].follower.keys()
return False
else:
return varname in self.context
@classmethod
def cl_compile(kls, *args, **kwargs):
kwargs['compilerSettings'] = {'directiveStartToken' : '%',
'cheetahVarStartToken' : '%%',
'EOLSlurpToken' : '%',
'PSPStartToken' : 'µ' * 10,
'PSPEndToken' : 'µ' * 10,
'commentStartToken' : 'µ' * 10,
'commentEndToken' : 'µ' * 10,
'multiLineCommentStartToken' : 'µ' * 10,
'multiLineCommentEndToken' : 'µ' * 10}
return kls.old_compile(*args, **kwargs)
ChtTemplate.old_compile = ChtTemplate.compile
ChtTemplate.compile = cl_compile
class CheetahTemplate(ChtTemplate):
"""classe pour personnaliser et faciliter la construction
du template Cheetah
"""
def __init__(self,
filename: str,
context,
eosfunc: Dict,
destfilename,
variable,
):
"""Initialize Creole CheetahTemplate
"""
extra_context = {'is_defined' : IsDefined(context),
'normalize_family': normalize_family,
'rougail_filename': destfilename
}
if variable:
extra_context['rougail_variable'] = variable
ChtTemplate.__init__(self,
file=filename,
searchList=[context, eosfunc, extra_context])
# FORK of Cheetah fonction, do not replace '\\' by '/'
def serverSidePath(self,
path=None,
normpath=normpath,
abspath=abspath
):
# strange...
if path is None and isinstance(self, str):
path = self
if path:
return normpath(abspath(path))
# return normpath(abspath(path.replace("\\", '/')))
elif hasattr(self, '_filePath') and self._filePath:
return normpath(abspath(self._filePath))
else:
return None
class CreoleLeader:
def __init__(self, value, follower=None, index=None):
"""
On rend la variable itérable pour pouvoir faire:
for ip in iplist:
print(ip.network)
print(ip.netmask)
print(ip)
index is used for CreoleLint
"""
self._value = value
if follower is not None:
self.follower = follower
else:
self.follower = {}
self._index = index
def __getattr__(self, name):
"""Get follower variable or attribute of leader value.
If the attribute is a name of a follower variable, return its value.
Otherwise, returns the requested attribute of leader value.
"""
if name in self.follower:
value = self.follower[name]
if isinstance(value, PropertiesOptionError):
raise AttributeError()
return value
else:
return getattr(self._value, name)
def __getitem__(self, index):
"""Get a leader.follower at requested index.
"""
ret = {}
for key, values in self.follower.items():
ret[key] = values[index]
return CreoleLeader(self._value[index], ret, index)
def __iter__(self):
"""Iterate over leader.follower.
Return synchronised value of leader.follower.
"""
for i in range(len(self._value)):
ret = {}
for key, values in self.follower.items():
ret[key] = values[i]
yield CreoleLeader(self._value[i], ret, i)
def __len__(self):
"""Delegate to leader value
"""
return len(self._value)
def __repr__(self):
"""Show CreoleLeader as dictionary.
The leader value is stored under 'value' key.
The followers are stored under 'follower' key.
"""
return repr({'value': self._value, 'follower': self.follower})
def __eq__(self, value):
return value == self._value
def __ne__(self, value):
return value != self._value
def __lt__(self, value):
return self._value < value
def __le__(self, value):
return self._value <= value
def __gt__(self, value):
return self._value > value
def __ge__(self, value):
return self._value >= value
def __str__(self):
"""Delegate to leader value
"""
return str(self._value)
def __add__(self, val):
return self._value.__add__(val)
def __radd__(self, val):
return val + self._value
def __contains__(self, item):
return item in self._value
async def add_follower(self, config, name, path):
if isinstance(self._value, list):
values = []
for idx in range(len(self._value)):
try:
values.append(await config.option(path, idx).value.get())
except PropertiesOptionError as err:
values.append(err)
else:
raise Exception('hu?')
self.follower[name] = values
class CreoleExtra:
def __init__(self,
suboption: Dict) -> None:
self.suboption = suboption
def __getattr__(self,
key: str) -> Any:
return self.suboption[key]
def __repr__(self):
return self.suboption.__str__()
def __iter__(self):
return iter(self.suboption.values())
class CreoleTemplateEngine:
"""Engine to process Creole cheetah template
"""
def __init__(self,
config: Config,
eosfunc_file: str,
distrib_dir: str,
tmp_dir: str,
dest_dir: str,
) -> None:
self.config = config
self.dest_dir = dest_dir
self.tmp_dir = tmp_dir
self.distrib_dir = distrib_dir
eos = {}
if eosfunc_file is not None:
eosfunc = imp.load_source('eosfunc', eosfunc_file)
for func in dir(eosfunc):
if not func.startswith('_'):
eos[func] = getattr(eosfunc, func)
self.eosfunc = eos
self.rougail_variables_dict = {}
async def load_eole_variables_rougail(self,
optiondescription):
for option in await optiondescription.list('all'):
if await option.option.isoptiondescription():
if await option.option.isleadership():
for idx, suboption in enumerate(await option.list('all')):
if idx == 0:
leader = CreoleLeader(await suboption.value.get())
self.rougail_variables_dict[await suboption.option.name()] = leader
else:
await leader.add_follower(self.config,
await suboption.option.name(),
await suboption.option.path())
else:
await self.load_eole_variables_rougail(option)
else:
self.rougail_variables_dict[await option.option.name()] = await option.value.get()
async def load_eole_variables(self,
namespace,
optiondescription):
families = {}
for family in await optiondescription.list('all'):
variables = {}
for variable in await family.list('all'):
if await variable.option.isoptiondescription():
if await variable.option.isleadership():
for idx, suboption in enumerate(await variable.list('all')):
if idx == 0:
leader = CreoleLeader(await suboption.value.get())
leader_name = await suboption.option.name()
else:
await leader.add_follower(self.config,
await suboption.option.name(),
await suboption.option.path())
variables[leader_name] = leader
else:
subfamilies = await self.load_eole_variables(await variable.option.name(),
variable,
)
variables[await variable.option.name()] = subfamilies
else:
variables[await variable.option.name()] = await variable.value.get()
families[await family.option.name()] = CreoleExtra(variables)
return CreoleExtra(families)
def patch_template(self,
filename: str,
tmp_dir: str,
patch_dir: str,
) -> None:
"""Apply patch to a template
"""
patch_cmd = ['patch', '-d', tmp_dir, '-N', '-p1']
patch_no_debug = ['-s', '-r', '-', '--backup-if-mismatch']
patch_file = join(patch_dir, f'{filename}.patch')
if isfile(patch_file):
log.info(_("Patching template '{filename}' with '{patch_file}'"))
rel_patch_file = relpath(patch_file, tmp_dir)
ret = call(patch_cmd + patch_no_debug + ['-i', rel_patch_file])
if ret:
patch_cmd_err = ' '.join(patch_cmd + ['-i', rel_patch_file])
log.error(_(f"Error applying patch: '{rel_patch_file}'\nTo reproduce and fix this error {patch_cmd_err}"))
copy(join(self.distrib_dir, filename), tmp_dir)
def prepare_template(self,
filename: str,
tmp_dir: str,
patch_dir: str,
) -> None:
"""Prepare template source file
"""
log.info(_("Copy template: '{filename}' -> '{tmp_dir}'"))
copy(filename, tmp_dir)
self.patch_template(filename, tmp_dir, patch_dir)
def process(self,
source: str,
true_destfilename: str,
destfilename: str,
filevar: Dict,
variable: Any,
):
"""Process a cheetah template
"""
# full path of the destination file
log.info(_(f"Cheetah processing: '{destfilename}'"))
try:
cheetah_template = CheetahTemplate(source,
self.rougail_variables_dict,
self.eosfunc,
true_destfilename,
variable,
)
data = str(cheetah_template)
except CheetahNotFound as err:
varname = err.args[0][13:-1]
raise TemplateError(_(f"Error: unknown variable used in template {source} to {destfilename} : {varname}"))
except Exception as err:
raise TemplateError(_(f"Error while instantiating template {source} to {destfilename}: {err}"))
with open(destfilename, 'w') as file_h:
file_h.write(data)
def instance_file(self,
filevar: Dict,
service_name: str,
tmp_dir: str,
dest_dir: str,
) -> None:
"""Run templatisation on one file
"""
log.info(_("Instantiating file '{filename}'"))
if 'variable' in filevar:
variable = filevar['variable']
else:
variable = None
filenames = filevar['name']
if not isinstance(filenames, list):
filenames = [filenames]
if variable:
variable = [variable]
for idx, filename in enumerate(filenames):
destfilename = join(dest_dir, filename[1:])
makedirs(dirname(destfilename), exist_ok=True)
if variable:
var = variable[idx]
else:
var = None
source = join(tmp_dir, filevar['source'])
if filevar['templating']:
self.process(source,
filename,
destfilename,
filevar,
var)
else:
copy(source, destfilename)
async def instance_files(self) -> None:
"""Run templatisation on all files
"""
ori_dir = getcwd()
tmp_dir = relpath(self.tmp_dir, self.distrib_dir)
dest_dir = relpath(self.dest_dir, self.distrib_dir)
patch_dir = relpath(Config['patch_dir'], self.distrib_dir)
chdir(self.distrib_dir)
for option in await self.config.option.list(type='all'):
namespace = await option.option.name()
if namespace == Config['variable_namespace']:
await self.load_eole_variables_rougail(option)
else:
families = await self.load_eole_variables(namespace,
option)
self.rougail_variables_dict[namespace] = families
for template in listdir('.'):
self.prepare_template(template, tmp_dir, patch_dir)
for service_obj in await self.config.option('services').list('all'):
service_name = await service_obj.option.doc()
for fills in await service_obj.list('all'):
if await fills.option.name() in ['files', 'overrides']:
for fill_obj in await fills.list('all'):
fill = await fill_obj.value.dict()
filename = fill['source']
if not isfile(filename):
raise FileNotFound(_(f"File {filename} does not exist."))
if fill.get('activate', False):
self.instance_file(fill,
service_name,
tmp_dir,
dest_dir,
)
else:
log.debug(_("Instantiation of file '{filename}' disabled"))
chdir(ori_dir)
async def generate(config: Config,
eosfunc_file: str,
distrib_dir: str,
tmp_dir: str,
dest_dir: str,
) -> None:
engine = CreoleTemplateEngine(config,
eosfunc_file,
distrib_dir,
tmp_dir,
dest_dir,
)
await engine.instance_files()

385
src/rougail/template/base.py Normal file
View File

@ -0,0 +1,385 @@
"""Template langage for Rougail
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from shutil import copy
import logging
from typing import Dict, Any
from subprocess import call
from os import listdir, makedirs, getcwd, chdir
from os.path import dirname, join, isfile, isdir, abspath
try:
from tiramisu3 import Config
from tiramisu3.error import PropertiesOptionError # pragma: no cover
except ModuleNotFoundError: # pragma: no cover
from tiramisu import Config
from tiramisu.error import PropertiesOptionError
from ..config import RougailConfig
from ..error import FileNotFound, TemplateError
from ..i18n import _
from ..utils import load_modules
from . import engine as engines
ENGINES = {}
for engine in engines.__all__:
ENGINES[engine] = getattr(engines, engine)
log = logging.getLogger(__name__)
log.addHandler(logging.NullHandler())
class RougailLeaderIndex:
"""This object is create when access to a specified Index of the variable
"""
def __init__(self,
value,
follower,
index,
) -> None:
self._value = value
self._follower = follower
self._index = index
def __getattr__(self, name):
if name not in self._follower:
raise AttributeError()
value = self._follower[name]
if isinstance(value, PropertiesOptionError):
raise AttributeError()
return value
def __str__(self):
return str(self._value)
def __lt__(self, value):
return self._value.__lt__(value)
def __le__(self, value):
return self._value.__le__(value)
def __eq__(self, value):
return self._value.__eq__(value)
def __ne__(self, value):
return self._value.__ne__(value)
def __gt__(self, value):
return self._value.__gt__(value)
def __ge__(self, value):
return self._value >= value
def __add__(self, value):
return self._value.__add__(value)
def __radd__(self, value):
return value + self._value
class RougailLeader:
"""Implement access to leader and follower variable
For examples: %%leader, %%leader[0].follower1
"""
def __init__(self,
value,
) -> None:
self._value = value
self._follower = {}
def __getitem__(self, index):
"""Get a leader.follower at requested index.
"""
followers = {key: values[index] for key, values in self._follower.items()}
return RougailLeaderIndex(self._value[index],
followers,
index,
)
def __iter__(self):
"""Iterate over leader.follower.
Return synchronised value of leader.follower.
"""
for index in range(len(self._value)):
yield self.__getitem__(index)
def __len__(self):
return len(self._value)
def __contains__(self, value):
return self._value.__contains__(value)
async def add_follower(self,
config,
name: str,
path: str,
):
"""Add a new follower
"""
self._follower[name] = []
for index in range(len(self._value)):
try:
value = await config.option(path, index).value.get()
except PropertiesOptionError as err:
value = err
self._follower[name].append(value)
class RougailExtra:
"""Object that implement access to extra variable
For example %%extra1.family.variable
"""
def __init__(self,
suboption: Dict) -> None:
self.suboption = suboption
def __getattr__(self,
key: str,
) -> Any:
try:
return self.suboption[key]
except KeyError:
raise AttributeError(f'unable to find extra "{key}"')
def __iter__(self):
return iter(self.suboption.values())
def items(self):
return self.suboption.items()
class RougailBaseTemplate:
"""Engine to process Creole cheetah template
"""
def __init__(self, # pylint: disable=R0913
config: Config,
rougailconfig: RougailConfig=None,
) -> None:
if rougailconfig is None:
rougailconfig = RougailConfig
self.config = config
self.destinations_dir = abspath(rougailconfig['destinations_dir'])
self.tmp_dir = abspath(rougailconfig['tmp_dir'])
self.templates_dir = abspath(rougailconfig['templates_dir'])
self.patches_dir = abspath(rougailconfig['patches_dir'])
eos = {}
functions_file = rougailconfig['functions_file']
if isfile(functions_file):
eosfunc = load_modules(functions_file)
for func in dir(eosfunc):
if not func.startswith('_'):
eos[func] = getattr(eosfunc, func)
self.eosfunc = eos
self.rougail_variables_dict = {}
self.rougailconfig = rougailconfig
self.log = log
self.engines = ENGINES
def patch_template(self,
filename: str,
) -> None:
"""Apply patch to a template
"""
patch_cmd = ['patch', '-d', self.tmp_dir, '-N', '-p1', '-f']
patch_no_debug = ['-s', '-r', '-', '--backup-if-mismatch']
patch_file = join(self.patches_dir, f'{filename}.patch')
if isfile(patch_file):
self.log.info(_("Patching template '{filename}' with '{patch_file}'"))
ret = call(patch_cmd + patch_no_debug + ['-i', patch_file])
if ret: # pragma: no cover
patch_cmd_err = ' '.join(patch_cmd + ['-i', patch_file])
msg = _(f"Error applying patch: '{patch_file}'\n"
f"To reproduce and fix this error {patch_cmd_err}")
self.log.error(_(msg))
copy(join(self.templates_dir, filename), self.tmp_dir)
def prepare_template(self,
filename: str,
) -> None:
"""Prepare template source file
"""
self.log.info(_("Copy template: '{filename}' -> '{self.tmp_dir}'"))
if not isdir(self.tmp_dir):
raise TemplateError(_(f'cannot find tmp_dir {self.tmp_dir}'))
copy(filename, self.tmp_dir)
self.patch_template(filename)
def instance_file(self,
filevar: Dict,
type_: str,
service_name: str,
) -> None:
"""Run templatisation on one file
"""
self.log.info(_("Instantiating file '{filename}'"))
if 'variable' in filevar:
variable = filevar['variable']
else:
variable = None
filenames = filevar.get('name')
if not isinstance(filenames, list):
filenames = [filenames]
if variable and not isinstance(variable, list):
variable = [variable]
if not isdir(self.destinations_dir):
raise TemplateError(_(f'cannot find destinations_dir {self.destinations_dir}'))
for idx, filename, in enumerate(filenames):
if variable:
var = variable[idx]
else:
var = None
func = f'_instance_{type_}'
data = getattr(self, func)(filevar,
filename,
service_name,
variable,
idx,
)
if data is None:
continue
filename, source, destfile, var = data
destfilename = join(self.destinations_dir, destfile[1:])
makedirs(dirname(destfilename), exist_ok=True)
self.log.info(_(f"{filevar['engine']} processing: '{destfilename}'"))
self.engines[filevar['engine']].process(filename=filename,
source=source,
true_destfilename=destfile,
destfilename=destfilename,
destdir=self.destinations_dir,
variable=var,
rougail_variables_dict=self.rougail_variables_dict,
eosfunc=self.eosfunc,
)
async def instance_files(self) -> None:
"""Run templatisation on all files
"""
ori_dir = getcwd()
chdir(self.templates_dir)
for option in await self.config.option.list(type='all'):
namespace = await option.option.name()
is_var_namespace = namespace == self.rougailconfig['variable_namespace']
self.rougail_variables_dict[namespace] = await self.load_variables(option,
is_var_namespace,
)
for template in listdir('.'):
self.prepare_template(template)
for included in (True, False):
for service_obj in await self.config.option('services').list('all'):
service_name = await service_obj.option.name()
if await service_obj.option('activate').value.get() is False:
if included is False:
self.desactive_service(service_name)
continue
for fills in await service_obj.list('optiondescription'):
type_ = await fills.option.name()
for fill_obj in await fills.list('all'):
fill = await fill_obj.value.dict()
if 'included' in fill:
if (fill['included'] == 'no' and included is True) or \
(fill['included'] != 'no' and included is False):
continue
elif included is True:
continue
if fill['activate']:
self.instance_file(fill, type_, service_name)
else:
self.log.debug(_("Instantiation of file '{filename}' disabled"))
self.post_instance_service(service_name)
self.post_instance()
chdir(ori_dir)
def desactive_service(self,
service_name: str,
):
raise NotImplementedError(_('cannot desactivate a service'))
def post_instance_service(self, service_name): # pragma: no cover
pass
def post_instance(self): # pragma: no cover
pass
def _instance_ip(self,
*args,
) -> None: # pragma: no cover
raise NotImplementedError(_('cannot instanciate this service type ip'))
def _instance_files(self,
*args,
) -> None: # pragma: no cover
raise NotImplementedError(_('cannot instanciate this service type file'))
def _instance_overrides(self,
*args,
) -> None: # pragma: no cover
raise NotImplementedError(_('cannot instanciate this service type override'))
async def load_variables(self,
optiondescription,
is_variable_namespace,
) -> RougailExtra:
"""Load all variables and set it in RougailExtra objects
"""
variables = {}
for option in await optiondescription.list('all'):
if await option.option.isoptiondescription():
if await option.option.isleadership():
for idx, suboption in enumerate(await option.list('all')):
if idx == 0:
leader = RougailLeader(await suboption.value.get())
leader_name = await suboption.option.name()
if is_variable_namespace:
self.rougail_variables_dict[await suboption.option.name()] = leader
else:
await leader.add_follower(self.config,
await suboption.option.name(),
await suboption.option.path(),
)
variables[leader_name] = leader
else:
subfamilies = await self.load_variables(option,
is_variable_namespace,
)
variables[await option.option.name()] = subfamilies
else:
if is_variable_namespace:
value = await option.value.get()
self.rougail_variables_dict[await option.option.name()] = value
if await option.option.issymlinkoption() and await option.option.isfollower():
value = []
path = await option.option.path()
for index in range(await option.value.len()):
value.append(await self.config.option(path, index).value.get())
else:
value = await option.value.get()
variables[await option.option.name()] = value
return RougailExtra(variables)

4
src/rougail/template/engine/__init__.py Normal file
View File

@ -0,0 +1,4 @@
from . import none, creole, jinja2, creole_legacy
__all__ = ('none', 'creole', 'jinja2', 'creole_legacy')

135
src/rougail/template/engine/creole.py Normal file
View File

@ -0,0 +1,135 @@
"""Creole engine
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from os.path import abspath, normpath
from Cheetah.Template import Template
from Cheetah.NameMapper import NotFound
from typing import Dict, Any
from ...i18n import _
from ...utils import normalize_family
from ...error import TemplateError
@classmethod
def cl_compile(kls, *args, **kwargs):
"""Rewrite compile methode to force some settings
"""
kwargs['compilerSettings'] = {'directiveStartToken' : '%',
'cheetahVarStartToken' : '%%',
'EOLSlurpToken' : '%',
'commentStartToken' : '#',
'multiLineCommentStartToken' : '#*',
'multiLineCommentEndToken' : '*#',
}
return kls.old_compile(*args, **kwargs) # pylint: disable=E1101
Template.old_compile = Template.compile
Template.compile = cl_compile
class CheetahTemplate(Template): # pylint: disable=W0223
"""Construct a cheetah templating object
"""
def __init__(self,
filename: str,
source: str,
context,
eosfunc: Dict,
extra_context: Dict,
):
"""Initialize Creole CheetahTemplate
"""
if filename is not None:
super().__init__(file=filename,
searchList=[context, eosfunc, extra_context],
)
else:
super().__init__(source=source,
searchList=[context, eosfunc, extra_context],
)
# FORK of Cheetah function, do not replace '\\' by '/'
def serverSidePath(self,
path=None,
normpath=normpath,
abspath=abspath
): # pylint: disable=W0621
# strange...
if path is None and isinstance(self, str):
path = self
if path: # pylint: disable=R1705
return normpath(abspath(path))
# original code return normpath(abspath(path.replace("\\", '/')))
elif hasattr(self, '_filePath') and self._filePath: # pragma: no cover
return normpath(abspath(self._filePath))
else: # pragma: no cover
return None
# Sync to creole_legacy.py
def process(filename: str,
source: str,
true_destfilename: str,
destfilename: str,
destdir: str,
variable: Any,
rougail_variables_dict: Dict,
eosfunc: Dict,
):
"""Process a cheetah template
"""
# full path of the destination file
try:
extra_context = {'normalize_family': normalize_family,
'rougail_filename': true_destfilename,
'rougail_destination_dir': destdir,
}
if variable is not None:
extra_context['rougail_variable'] = variable
cheetah_template = CheetahTemplate(filename,
source,
rougail_variables_dict,
eosfunc,
extra_context,
)
data = str(cheetah_template)
except NotFound as err: # pragma: no cover
varname = err.args[0][13:-1]
if filename:
msg = f"Error: unknown variable used in template {filename} to {destfilename}: {varname}"
else:
msg = f"Error: unknown variable used in file {destfilename}: {varname}"
raise TemplateError(_(msg)) from err
except Exception as err: # pragma: no cover
if filename:
msg = _(f"Error while instantiating template {filename} to {destfilename}: {err}")
else:
msg = _(f"Error while instantiating filename {destfilename}: {err}")
raise TemplateError(msg) from err
with open(destfilename, 'w') as file_h:
file_h.write(data)

157
src/rougail/template/engine/creole_legacy.py Normal file
View File

@ -0,0 +1,157 @@
"""Legacy Creole engine
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from typing import Dict, Any
from Cheetah.NameMapper import NotFound
from .creole import CheetahTemplate as oriCheetahTemplate
from ...i18n import _
from ...utils import normalize_family
from ...error import TemplateError
@classmethod
def cl_compile(kls, *args, **kwargs):
"""Rewrite compile methode to force some settings
"""
kwargs['compilerSettings'] = {'directiveStartToken' : u'%',
'cheetahVarStartToken' : u'%%',
'EOLSlurpToken' : u'%',
'PSPStartToken' : u'µ' * 10,
'PSPEndToken' : u'µ' * 10,
'commentStartToken' : u'µ' * 10,
'commentEndToken' : u'µ' * 10,
'multiLineCommentStartToken' : u'µ' * 10,
'multiLineCommentEndToken' : u'µ' * 10}
return kls.old_compile(*args, **kwargs) # pylint: disable=E1101
oriCheetahTemplate.compile = cl_compile
class IsDefined:
"""
filtre permettant de ne pas lever d'exception au cas où
la variable Creole n'est pas définie
"""
def __init__(self, context):
self.context = context
def __call__(self, varname):
if '.' in varname:
splitted_var = varname.split('.')
if len(splitted_var) != 2:
msg = u"Group variables must be of type master.slave"
raise KeyError(msg)
master, slave = splitted_var
if master in self.context:
return slave in self.context[master].slave.keys()
return False
else:
return varname in self.context
class CreoleClient():
def get(self, path):
path = path.replace('/', '.')
if path.startswith('.'):
path = path[1:]
if '.' not in path:
return self.context[path]
else:
root, path = path.split('.', 1)
obj = self.context[root]
for var in path.split('.'):
obj = getattr(obj, var)
return obj
def is_empty(data):
if str(data) in ['', '""', "''", "[]", "['']", '[""]', "None"]:
return True
return False
class CheetahTemplate(oriCheetahTemplate):
def __init__(self,
filename: str,
source: str,
context,
eosfunc: Dict,
extra_context: Dict,
):
creole_client = CreoleClient()
creole_client.context=context
extra_context['is_defined'] = IsDefined(context)
extra_context['creole_client'] = creole_client
extra_context['is_empty'] = is_empty
extra_context['_creole_filename'] = extra_context['rougail_filename']
super().__init__(filename, source, context, eosfunc, extra_context)
# Sync to creole.py
def process(filename: str,
source: str,
true_destfilename: str,
destfilename: str,
destdir: str,
variable: Any,
rougail_variables_dict: Dict,
eosfunc: Dict,
):
"""Process a cheetah template
"""
# full path of the destination file
try:
extra_context = {'normalize_family': normalize_family,
'rougail_filename': true_destfilename,
'rougail_destination_dir': destdir,
}
if variable is not None:
extra_context['rougail_variable'] = variable
cheetah_template = CheetahTemplate(filename,
source,
rougail_variables_dict,
eosfunc,
extra_context,
)
data = str(cheetah_template)
except NotFound as err: # pragma: no cover
varname = err.args[0][13:-1]
if filename:
msg = f"Error: unknown variable used in template {filename} to {destfilename}: {varname}"
else:
msg = f"Error: unknown variable used in file {destfilename}: {varname}"
raise TemplateError(_(msg)) from err
except Exception as err: # pragma: no cover
if filename:
msg = _(f"Error while instantiating template {filename} to {destfilename}: {err}")
else:
msg = _(f"Error while instantiating filename {destfilename}: {err}")
raise TemplateError(msg) from err
with open(destfilename, 'w') as file_h:
file_h.write(data)

74
src/rougail/template/engine/jinja2.py Normal file
View File

@ -0,0 +1,74 @@
"""Jinja2 engine
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from typing import Any, Dict
from jinja2 import Environment, FileSystemLoader
from jinja2.exceptions import UndefinedError
from ...i18n import _
from ...utils import normalize_family
from ...error import TemplateError
def process(filename: str,
source: str,
true_destfilename: str,
destfilename: str,
destdir: str,
variable: Any,
rougail_variables_dict: Dict,
eosfunc: Dict,
):
"""Process a cheetah template
"""
# full path of the destination file
dir_name, template_name = filename.rsplit('/', 1)
if source is not None: # pragma: no cover
raise TemplateError(_('source is not supported for jinja2'))
rougail_variables_dict['rougail_variable'] = variable
if variable is not None:
var = {'rougail_variable': variable}
else:
var = {}
try:
# extra_context = {'normalize_family': normalize_family,
# eosfunc
env = Environment(loader=FileSystemLoader([dir_name, destdir]))
template = env.get_template(template_name)
data = template.render(**rougail_variables_dict,
rougail_filename=true_destfilename,
rougail_destination_dir=destdir,
**var,
)
except UndefinedError as err: # pragma: no cover
varname = err
msg = f"Error: unknown variable used in template {filename} to {destfilename}: {varname}"
raise TemplateError(_(msg)) from err
if not data.endswith('\n'):
data = data + '\n'
with open(destfilename, 'w') as file_h:
file_h.write(data)

35
src/rougail/template/engine/none.py Normal file
View File

@ -0,0 +1,35 @@
"""None engine
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from typing import Any
from shutil import copy
def process(filename: str,
destfilename: str,
**kwargs
):
copy(filename, destfilename)

163
src/rougail/template/systemd.py Normal file
View File

@ -0,0 +1,163 @@
"""Template langage for Rougail to create file and systemd file
Cadoles (http://www.cadoles.com)
Copyright (C) 2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from typing import Dict, Any
from os import makedirs, symlink
from os.path import dirname, isfile, join
from ipaddress import ip_network
from .base import RougailBaseTemplate
from ..i18n import _
from ..error import FileNotFound
ROUGAIL_IP_TEMPLATE = """[Service]
%for %%ip in %%rougail_variable
IPAddressAllow=%%ip
%end for
IPAddressDeny=any
"""
ROUGAIL_TMPL_TEMPLATE = """%def display(%%file, %%filename)
%if %%filename.startswith('/etc/') or %%filename.startswith('/var/') or %%filename.startswith('/srv/')
C %%filename %%file.mode %%file.owner %%file.group - /usr/local/lib%%filename
z %%filename - - - - -
%end if
%end def
%for %%service in %%services
%if %%service.activate is True and %%hasattr(%%service, 'files')
%for %%file in %%service.files
%if %%file.activate is True and %%file.included != 'content'
%if %%isinstance(%%file.name, list)
%for %%filename in %%file.name
%%display(%%file, %%filename)%slurp
%end for
%else
%%display(%%file, %%file.name)%slurp
%end if
%end if
%end for
%end if
%end for
"""
class RougailSystemdTemplate(RougailBaseTemplate):
def __init__(self, # pylint: disable=R0913
config: 'Config',
rougailconfig: 'RougailConfig'=None,
) -> None:
self.ip_per_service = None
super().__init__(config, rougailconfig)
def _instance_files(self,
filevar: Dict,
destfile: str,
service_name: str,
variable,
idx: int,
) -> tuple:
source = filevar['source']
if not isfile(source): # pragma: no cover
raise FileNotFound(_(f"File {source} does not exist."))
tmp_file = join(self.tmp_dir, source)
#self.instance_file(fill, 'files')
if variable:
var = variable[idx]
else:
var = None
return tmp_file, None, destfile, var
def _instance_overrides(self,
filevar: Dict,
destfile,
service_name: str,
*args,
) -> tuple:
source = filevar['source']
if not isfile(source): # pragma: no cover
raise FileNotFound(_(f"File {source} does not exist."))
tmp_file = join(self.tmp_dir, source)
service_name = filevar['name']
return tmp_file, None, f'/systemd/system/{service_name}.service.d/rougail.conf', None
def _instance_ip(self,
filevar: Dict,
ip,
service_name: str,
var: Any,
idx: int,
*args,
) -> tuple:
if self.ip_per_service is None:
self.ip_per_service = []
if 'netmask' in filevar:
if isinstance(filevar["netmask"], list):
netmask = filevar['netmask'][idx]
else:
netmask = filevar['netmask']
self.ip_per_service.append(str(ip_network(f'{ip}/{netmask}')))
elif ip:
self.ip_per_service.append(ip)
def desactive_service(self,
service_name: str,
):
filename = f'{self.destinations_dir}/systemd/system/{service_name}.service'
makedirs(dirname(filename), exist_ok=True)
symlink('/dev/null', filename)
def post_instance_service(self,
service_name: str,
) -> None: # pragma: no cover
if self.ip_per_service is None:
return
destfile = f'/systemd/system/{service_name}.service.d/rougail_ip.conf'
destfilename = join(self.destinations_dir, destfile[1:])
makedirs(dirname(destfilename), exist_ok=True)
self.log.info(_(f"creole processing: '{destfilename}'"))
self.engines['creole'].process(filename=None,
source=ROUGAIL_IP_TEMPLATE,
true_destfilename=destfile,
destfilename=destfilename,
destdir=self.destinations_dir,
variable=self.ip_per_service,
rougail_variables_dict=self.rougail_variables_dict,
eosfunc=self.eosfunc,
)
self.ip_per_service = None
def post_instance(self):
destfile = '/tmpfiles.d/rougail.conf'
destfilename = join(self.destinations_dir, destfile[1:])
makedirs(dirname(destfilename), exist_ok=True)
self.log.info(_(f"creole processing: '{destfilename}'"))
self.engines['creole'].process(filename=None,
source=ROUGAIL_TMPL_TEMPLATE,
true_destfilename=destfile,
destfilename=destfilename,
destdir=self.destinations_dir,
variable=None,
rougail_variables_dict=self.rougail_variables_dict,
eosfunc=self.eosfunc,
)

34
src/rougail/tiramisu.py
View File

@ -1,13 +1,41 @@
"""Redefine Tiramisu object
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
try: try:
from tiramisu3 import DynOptionDescription from tiramisu3 import DynOptionDescription
except: except ModuleNotFoundError:
from tiramisu import DynOptionDescription from tiramisu import DynOptionDescription
from .utils import normalize_family from .utils import normalize_family
class ConvertDynOptionDescription(DynOptionDescription): class ConvertDynOptionDescription(DynOptionDescription):
"""Suffix could be an integer, we should convert it in str
Suffix could also contain invalid character, so we should "normalize" it
"""
def convert_suffix_to_path(self, suffix): def convert_suffix_to_path(self, suffix):
if not isinstance(suffix, str): if not isinstance(suffix, str):
suffix = str(suffix) suffix = str(suffix)
return normalize_family(suffix, return normalize_family(suffix)
check_name=False)

719
src/rougail/tiramisureflector.py
View File

@ -1,494 +1,391 @@
"""loader """loader
flattened XML specific flattened XML specific
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
""" """
from .config import Config from json import dumps
from .i18n import _ from os.path import isfile
from .error import LoaderError
from .annotator import ERASED_ATTRIBUTES, CONVERT_OPTION from .annotator import CONVERT_OPTION
from .objspace import RootRougailObject
FUNC_TO_DICT = ['valid_not_equal'] class Root(): # pylint: disable=R0903
FORCE_INFORMATIONS = ['help', 'test', 'separator', 'manage'] """Root classes
ATTRIBUTES_ORDER = ('name', 'doc', 'default', 'multi') """
path = '.'
class BaseElt: # pylint: disable=R0903
"""Base element
"""
name = 'baseoption'
doc = 'baseoption'
path = '.'
class TiramisuReflector: class TiramisuReflector:
"""Convert object to tiramisu representation
"""
def __init__(self, def __init__(self,
xmlroot, objectspace,
funcs_path, funcs_path,
): ):
self.storage = ElementStorage() self.index = 0
self.storage.text = ["import imp", self.text = []
f"func = imp.load_source('func', '{funcs_path}')", if funcs_path and isfile(funcs_path):
self.text.extend(["from importlib.machinery import SourceFileLoader",
"from importlib.util import spec_from_loader, module_from_spec",
f"loader = SourceFileLoader('func', '{funcs_path}')",
"spec = spec_from_loader(loader.name, loader)",
"func = module_from_spec(spec)",
"loader.exec_module(func)",
"for key, value in dict(locals()).items():", "for key, value in dict(locals()).items():",
" if key != ['imp', 'func']:", " if key != ['SourceFileLoader', 'func']:",
" setattr(func, key, value)", " setattr(func, key, value)"])
"try:", self.text.extend(["try:",
" from tiramisu3 import *", " from tiramisu3 import *",
"except:", "except:",
" from tiramisu import *", " from tiramisu import *",
"from rougail.tiramisu import ConvertDynOptionDescription", ])
] self.objectspace = objectspace
self.make_tiramisu_objects(xmlroot) self.make_tiramisu_objects()
# parse object if self.objectspace.has_dyn_option is True:
self.storage.get('.').get() self.text.append("from rougail.tiramisu import ConvertDynOptionDescription")
def make_tiramisu_objects(self, def make_tiramisu_objects(self) -> None:
xmlroot, """make tiramisu objects
): """
family = self.get_root_family() baseelt = BaseElt()
for xmlelt in self.reorder_family(xmlroot): self.set_name(baseelt)
self.iter_family(xmlelt, basefamily = Family(baseelt,
family, self.text,
None, self.objectspace,
) )
for elt in self.reorder_family():
def get_root_family(self): self.populate_family(basefamily,
family = Family(BaseElt('baseoption', elt,
'baseoption',
),
self.storage,
False,
'.',
) )
return family self.baseelt = baseelt
def reorder_family(self, xmlroot): def reorder_family(self):
# variable_namespace family has to be loaded before any other family """variable_namespace family has to be loaded before any other family
# because `extra` family could use `variable_namespace` variables. because `extra` family could use `variable_namespace` variables.
if hasattr(xmlroot, 'variables'): """
if Config['variable_namespace'] in xmlroot.variables: if hasattr(self.objectspace.space, 'variables'):
yield xmlroot.variables[Config['variable_namespace']] variable_namespace = self.objectspace.rougailconfig['variable_namespace']
for xmlelt, value in xmlroot.variables.items(): if variable_namespace in self.objectspace.space.variables:
if xmlelt != Config['variable_namespace']: yield self.objectspace.space.variables[variable_namespace]
for elt, value in self.objectspace.space.variables.items():
if elt != self.objectspace.rougailconfig['variable_namespace']:
yield value yield value
if hasattr(xmlroot, 'services'): if hasattr(self.objectspace.space, 'services'):
yield xmlroot.services yield self.objectspace.space.services
def get_attributes(self, space): # pylint: disable=R0201
for attr in dir(space):
if not attr.startswith('_') and attr not in ERASED_ATTRIBUTES:
yield attr
def get_children(self,
space,
):
for tag in self.get_attributes(space):
children = getattr(space, tag)
if children.__class__.__name__ == 'Family':
children = [children]
if isinstance(children, dict):
children = list(children.values())
if isinstance(children, list):
yield tag, children
def iter_family(self,
child,
family,
subpath,
):
tag = child.__class__.__name__
if tag == 'Variable':
function = self.populate_variable
elif tag == 'Property':
# property already imported with family
return
else:
function = self.populate_family
#else:
# raise Exception('unknown tag {}'.format(child.tag))
function(family,
child,
subpath,
)
def populate_family(self, def populate_family(self,
parent_family, parent_family,
elt, elt,
subpath,
): ):
path = self.build_path(subpath, """Populate family
elt, """
) self.set_name(elt)
tag = elt.__class__.__name__
family = Family(elt, family = Family(elt,
self.storage, self.text,
tag == 'Leadership', self.objectspace,
path,
) )
parent_family.add(family) parent_family.add(family)
for tag, children in self.get_children(elt): for children in vars(elt).values():
if isinstance(children, self.objectspace.family):
self.populate_family(family,
children,
)
continue
if isinstance(children, dict):
children = list(children.values())
if isinstance(children, list):
for child in children: for child in children:
self.iter_family(child, if isinstance(child, self.objectspace.property_) or \
family, not isinstance(child, RootRougailObject):
path, continue
) if isinstance(child, self.objectspace.variable):
self.set_name(child)
def populate_variable(self, family.add(Variable(child,
family, self.text,
elt, self.objectspace,
subpath,
):
is_follower = False
is_leader = False
if family.is_leader:
if elt.name != family.elt.name:
is_follower = True
else:
is_leader = True
family.add(Variable(elt,
self.storage,
is_follower,
is_leader,
self.build_path(subpath,
elt,
)
)) ))
else:
self.populate_family(family,
child,
)
def build_path(self, def set_name(self,
subpath,
elt, elt,
): ):
if subpath is None: """Set name
return elt.name """
return subpath + '.' + elt.name elt.reflector_name = f'option_{self.index}'
def get_text(self):
return '\n'.join(self.storage.get('.').get_text())
class BaseElt:
def __init__(self,
name,
doc,
):
self.name = name
self.doc = doc
def __iter__(self):
return iter([])
class ElementStorage:
def __init__(self,
):
self.paths = {}
self.text = []
self.index = 0
def add(self, path, elt):
self.paths[path] = (elt, self.index)
self.index += 1 self.index += 1
def get(self, path): def get_text(self):
if path not in self.paths or self.paths[path][0] is None: """Get text
raise LoaderError(_('there is no element for path {}').format(path)) """
return self.paths[path][0] self.baseelt.reflector_object.get() # pylint: disable=E1101
return '\n'.join(self.text)
def get_name(self, path):
if path not in self.paths:
raise LoaderError(_('there is no element for index {}').format(path))
return f'option_{self.paths[path][1]}'
class Common: class Common:
"""Common function for variable and family
"""
def __init__(self, def __init__(self,
elt, elt,
storage, text,
is_leader, objectspace,
path,
): ):
self.elt = elt
self.option_name = None self.option_name = None
self.path = path self.text = text
self.attrib = {} self.objectspace = objectspace
self.informations = {} self.elt.reflector_object = self
self.storage = storage self.object_type = None
self.is_leader = is_leader
self.storage.add(self.path, self)
def populate_properties(self, child): def get(self):
if child.type == 'calculation': """Get tiramisu's object
action = f"ParamValue('{child.name}')" """
option_name = self.storage.get(child.source).get() if self.option_name is None:
kwargs = f"'condition': ParamOption({option_name}, todict=True), 'expected': ParamValue('{child.expected}')" self.option_name = self.elt.reflector_name
self.populate_attrib()
self.populate_informations()
return self.option_name
def populate_attrib(self):
"""Populate attributes
"""
keys = {'name': self.convert_str(self.elt.name)}
if hasattr(self.elt, 'doc'):
keys['doc'] = self.convert_str(self.elt.doc)
self._populate_attrib(keys)
if hasattr(self.elt, 'properties'):
keys['properties'] = self.properties_to_string(self.elt.properties)
attrib = ', '.join([f'{key}={value}' for key, value in keys.items()])
self.text.append(f'{self.option_name} = {self.object_type}({attrib})')
def _populate_attrib(self,
keys: dict,
) -> None: # pragma: no cover
raise NotImplementedError()
@staticmethod
def convert_str(value):
"""convert string
"""
return dumps(value, ensure_ascii=False)
def properties_to_string(self,
values: list,
) -> None:
"""Change properties to string
"""
properties = [self.convert_str(property_) for property_ in values
if isinstance(property_, str)]
calc_properties = [self.calc_properties(property_) for property_ in values \
if isinstance(property_, self.objectspace.property_)]
return 'frozenset({' + ', '.join(sorted(properties) + calc_properties) + '})'
def calc_properties(self,
child,
) -> str:
"""Populate properties
"""
option_name = child.source.reflector_object.get()
kwargs = (f"'condition': ParamOption({option_name}, todict=True, notraisepropertyerror=True), "
f"'expected': {self.populate_param(child.expected)}")
if child.inverse: if child.inverse:
kwargs += ", 'reverse_condition': ParamValue(True)" kwargs += ", 'reverse_condition': ParamValue(True)"
prop = 'Calculation(calc_value, Params(' + action + ', kwargs={' + kwargs + '}))' return (f"Calculation(func.calc_value, Params(ParamValue('{child.name}'), "
else: f"kwargs={{{kwargs}}}))")
prop = "'" + child.text + "'"
if self.attrib['properties']:
self.attrib['properties'] += ', '
self.attrib['properties'] += prop
def get_attrib(self, attrib):
ret_list = []
for key, value in self.attrib.items():
if value is None:
continue
if key == 'properties':
if not self.attrib[key]:
continue
value = "frozenset({" + self.attrib[key] + "})"
elif key in ['default', 'multi', 'suffixes', 'validators', 'values']:
value = self.attrib[key]
elif isinstance(value, str) and key != 'opt' and not value.startswith('['):
value = "'" + value.replace("'", "\\\'") + "'"
ret_list.append(f'{key}={value}')
return ', '.join(ret_list)
def populate_informations(self): def populate_informations(self):
for key, value in self.informations.items(): """Populate Tiramisu's informations
"""
if not hasattr(self.elt, 'information'):
return
for key, value in vars(self.elt.information).items():
if key == 'xmlfiles':
continue
if isinstance(value, str): if isinstance(value, str):
value = '"' + value.replace('"', '\"') + '"' value = self.convert_str(value)
self.storage.text.append(f'{self.option_name}.impl_set_information("{key}", {value})') self.text.append(f"{self.option_name}.impl_set_information('{key}', {value})")
def get_text(self, def populate_param(self,
param,
): ):
return self.storage.text """Populate variable parameters
"""
if param.type in ['number', 'boolean', 'nil', 'string', 'port']:
value = param.text
if param.type == 'string' and value is not None:
value = self.convert_str(value)
return f'ParamValue({value})'
if param.type == 'variable':
return self.build_option_param(param)
if param.type == 'information':
return f'ParamInformation("{param.text}", None)'
if param.type == 'target_information':
return f'ParamSelfInformation("{param.text}", None)'
if param.type == 'suffix':
return 'ParamSuffix()'
if param.type == 'index':
return 'ParamIndex()'
raise Exception(f'unknown type {param.type}') # pragma: no cover
def get_attributes(self, space): # pylint: disable=R0201 @staticmethod
attributes = dir(space) def build_option_param(param,
for attr in ATTRIBUTES_ORDER: ) -> str:
if attr in attributes: """build variable parameters
yield attr """
for attr in dir(space): option_name = param.text.reflector_object.get()
if attr not in ATTRIBUTES_ORDER: params = [f'{option_name}']
if not attr.startswith('_') and attr not in ERASED_ATTRIBUTES: if hasattr(param, 'suffix'):
value = getattr(space, attr) param_type = 'ParamDynOption'
if not isinstance(value, (list, dict)) and not value.__class__.__name__ == 'Family': family = param.family.reflector_object.get()
yield attr params.extend([f"'{param.suffix}'", f'{family}'])
else:
def get_children(self, param_type = 'ParamOption'
space, if not param.propertyerror:
): params.append('notraisepropertyerror=True')
for attr in dir(space): return "{}({})".format(param_type, ', '.join(params))
if not attr.startswith('_') and attr not in ERASED_ATTRIBUTES:
value = getattr(space, attr)
if isinstance(value, (list, dict)):
children = getattr(space, attr)
if children.__class__.__name__ == 'Family':
children = [children]
if isinstance(children, dict):
children = list(children.values())
if children and isinstance(children, list):
yield attr, children
class Variable(Common): class Variable(Common):
"""Manage variable
"""
def __init__(self, def __init__(self,
elt, elt,
storage, text,
is_follower, objectspace,
is_leader,
path,
): ):
super().__init__(elt, super().__init__(elt, text, objectspace)
storage, self.object_type = CONVERT_OPTION[elt.type]['opttype']
is_leader,
path,
)
self.is_follower = is_follower
convert_option = CONVERT_OPTION[elt.type]
del elt.type
self.object_type = convert_option['opttype']
self.attrib.update(convert_option.get('initkwargs', {}))
if self.object_type != 'SymLinkOption':
self.attrib['properties'] = []
self.attrib['validators'] = []
self.elt = elt
def get(self): def _populate_attrib(self,
if self.option_name is None: keys: dict,
self.populate_attrib() ):
if self.object_type == 'SymLinkOption': if hasattr(self.elt, 'opt'):
self.attrib['opt'] = self.storage.get(self.attrib['opt']).get() keys['opt'] = self.elt.opt.reflector_object.get()
if hasattr(self.elt, 'values'):
values = self.elt.values
if values[0].type == 'variable':
value = values[0].name.reflector_object.get()
keys['values'] = f"Calculation(func.calc_value, Params((ParamOption({value}))))"
elif values[0].type == 'function':
keys['values'] = self.calculation_value(self.elt.values[0], [])
else: else:
self.parse_children() keys['values'] = str(tuple([val.name for val in values]))
attrib = self.get_attrib(self.attrib) if hasattr(self.elt, 'multi') and self.elt.multi:
self.option_name = self.storage.get_name(self.path) keys['multi'] = self.elt.multi
self.storage.text.append(f'{self.option_name} = {self.object_type}({attrib})') for key in ['default', 'default_multi']:
self.populate_informations() if hasattr(self.elt, key) and getattr(self.elt, key) is not None:
return self.option_name
def populate_attrib(self):
for key in self.get_attributes(self.elt):
value = getattr(self.elt, key) value = getattr(self.elt, key)
if key in FORCE_INFORMATIONS: if isinstance(value, str):
if key == 'test': value = self.convert_str(value)
value = value.split(',') elif isinstance(value, self.objectspace.value):
if self.object_type == 'IntOption': value = self.calculation_value(value, [], calc_multi=value.calc_multi)
value = [int(v) for v in value] keys[key] = value
elif self.object_type == 'FloatOption': if hasattr(self.elt, 'validators'):
value = [float(v) for v in value] keys['validators'] = '[' + ', '.join([self.calculation_value(val,
self.informations[key] = value ['ParamSelfOption(whole=False)']) for val in self.elt.validators]) + ']'
else: for key in ['min_number', 'max_number']:
self.attrib[key] = value if hasattr(self.elt, key):
keys[key] = getattr(self.elt, key)
for key, value in CONVERT_OPTION[self.elt.type].get('initkwargs', {}).items():
if isinstance(value, str):
value = f"'{value}'"
keys[key] = value
def parse_children(self): def calculation_value(self,
if not 'default' in self.attrib or self.attrib['multi']: child,
self.attrib['default'] = [] args,
if self.attrib['multi'] == 'submulti' and self.is_follower: calc_multi=False,
self.attrib['default_multi'] = [] ) -> str:
choices = [] """Generate calculated value
if 'properties' in self.attrib: """
if self.attrib['properties']:
self.attrib['properties'] = "'" + "', '".join(sorted(list(self.attrib['properties']))) + "'"
else:
self.attrib['properties'] = ''
for tag, children in self.get_children(self.elt):
for child in children:
if tag == 'property':
self.populate_properties(child)
elif tag == 'value':
if child.type == 'calculation':
self.attrib['default'] = self.calculation_value(child, [])
else:
self.populate_value(child)
elif tag == 'check':
self.attrib['validators'].append(self.calculation_value(child, ['ParamSelfOption()']))
elif tag == 'choice':
if child.type == 'calculation':
value = self.storage.get(child.name).get()
choices = f"Calculation(func.calc_value, Params((ParamOption({value}))))"
else:
choices.append(child.name)
if choices:
if isinstance(choices, list):
self.attrib['values'] = str(tuple(choices))
else:
self.attrib['values'] = choices
if self.attrib['default'] == []:
del self.attrib['default']
elif not self.attrib['multi'] and isinstance(self.attrib['default'], list):
self.attrib['default'] = self.attrib['default'][-1]
if self.attrib['validators'] == []:
del self.attrib['validators']
else:
self.attrib['validators'] = '[' + ', '.join(self.attrib['validators']) + ']'
def calculation_value(self, child, args):
kwargs = [] kwargs = []
if hasattr(child, 'name'):
# has parameters # has parameters
function = child.name function = child.name
new_args = []
if hasattr(child, 'param'): if hasattr(child, 'param'):
for param in child.param: for param in child.param:
value = self.populate_param(function, param) value = self.populate_param(param)
if not hasattr(param, 'name'): if not hasattr(param, 'name'):
args.append(str(value)) new_args.append(str(value))
else: else:
kwargs.append(f"'{param.name}': " + value) kwargs.append(f"'{param.name}': " + value)
if function == 'valid_network_netmask':
new_args.extend(args)
else: else:
# function without any parameter args.extend(new_args)
function = child.text.strip() new_args = args
ret = f"Calculation(func.{function}, Params((" + ', '.join(args) + "), kwargs=" + "{" + ', '.join(kwargs) + "})" ret = f'Calculation(func.{function}, Params((' + ', '.join(new_args) + ')'
if kwargs:
ret += ', kwargs={' + ', '.join(kwargs) + '}'
ret += ')'
if hasattr(child, 'warnings_only'): if hasattr(child, 'warnings_only'):
ret += f', warnings_only={child.warnings_only}' ret += f', warnings_only={child.warnings_only}'
return ret + ')' ret = ret + ')'
if calc_multi:
def populate_param(self, ret = '[' + ret + ']'
function: str, return ret
param,
):
if param.type == 'string':
return f'ParamValue("{param.text}")'
elif param.type == 'number':
return f'ParamValue({param.text})'
elif param.type == 'variable':
value = {'option': param.text,
'notraisepropertyerror': param.notraisepropertyerror,
'todict': function in FUNC_TO_DICT,
}
if hasattr(param, 'suffix'):
value['suffix'] = param.suffix
return self.build_param(value)
elif param.type == 'information':
return f'ParamInformation("{param.text}", None)'
elif param.type == 'suffix':
return f'ParamSuffix()'
raise LoaderError(_('unknown param type {}').format(param.type))
def populate_value(self,
child,
):
value = child.name
if self.attrib['multi'] == 'submulti':
self.attrib['default_multi'].append(value)
elif self.is_follower:
self.attrib['default_multi'] = value
elif self.attrib['multi']:
self.attrib['default'].append(value)
if not self.is_leader:
self.attrib['default_multi'] = value
elif isinstance(value, (int, float)):
self.attrib['default'].append(value)
else:
self.attrib['default'].append("'" + value + "'")
def build_param(self,
param,
):
option_name = self.storage.get(param['option']).get()
if 'suffix' in param:
family = '.'.join(param['option'].split('.')[:-1])
family_option = self.storage.get_name(family)
return f"ParamDynOption({option_name}, '{param['suffix']}', {family_option}, notraisepropertyerror={param['notraisepropertyerror']}, todict={param['todict']})"
return f"ParamOption({option_name}, notraisepropertyerror={param['notraisepropertyerror']}, todict={param['todict']})"
class Family(Common): class Family(Common):
"""Manage family
"""
def __init__(self, def __init__(self,
elt, elt,
storage, text,
is_leader, objectspace,
path,
): ):
super().__init__(elt, super().__init__(elt, text, objectspace)
storage, if hasattr(self.elt, 'suffixes'):
is_leader, self.objectspace.has_dyn_option = True
path, self.object_type = 'ConvertDynOptionDescription'
) elif isinstance(self.elt, self.objectspace.leadership):
self.object_type = 'Leadership'
else:
self.object_type = 'OptionDescription'
self.children = [] self.children = []
self.elt = elt
def add(self, child): def add(self, child):
"""Add a child
"""
self.children.append(child) self.children.append(child)
def get(self): def _populate_attrib(self,
if not self.option_name: keys: list,
self.populate_attrib() ) -> None:
self.parse_children() if hasattr(self.elt, 'suffixes'):
self.option_name = self.storage.get_name(self.path) dyn = self.elt.suffixes.reflector_object.get()
object_name = self.get_object_name() keys['suffixes'] = f"Calculation(func.calc_value, Params((ParamOption({dyn}, notraisepropertyerror=True))))"
attrib = self.get_attrib(self.attrib) + ', children=[' + ', '.join([child.get() for child in self.children]) + ']' keys['children'] = '[' + ', '.join([child.get() for child in self.children]) + ']'
self.storage.text.append(f'{self.option_name} = {object_name}({attrib})')
self.populate_informations()
return self.option_name
def populate_attrib(self):
for key in self.get_attributes(self.elt):
value = getattr(self.elt, key)
if key in FORCE_INFORMATIONS:
self.informations[key] = value
elif key == 'dynamic':
dynamic = self.storage.get(value).get()
self.attrib['suffixes'] = f"Calculation(func.calc_value, Params((ParamOption({dynamic}))))"
else:
self.attrib[key] = value
def parse_children(self):
if 'properties' in self.attrib:
self.attrib['properties'] = "'" + "', '".join(sorted(list(self.attrib['properties']))) + "'"
if hasattr(self.elt, 'property'):
#self.attrib['properties'] = ''
for child in self.elt.property:
self.populate_properties(child)
if not self.attrib['properties']:
del self.attrib['properties']
def get_object_name(self):
if 'suffixes' in self.attrib:
return 'ConvertDynOptionDescription'
elif not self.is_leader:
return 'OptionDescription'
return 'Leadership'

573
src/rougail/update.py Normal file
View File

@ -0,0 +1,573 @@
"""Update Rougail XML file to new version
Cadoles (http://www.cadoles.com)
Copyright (C) 2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from typing import List
from os.path import join, isfile, basename
from os import listdir
from lxml.etree import DTD, parse, XMLParser, XMLSyntaxError # pylint: disable=E0611
from lxml.etree import Element, SubElement, tostring
from ast import parse as ast_parse
from .i18n import _
from .error import UpgradeError
from .utils import normalize_family
VERSIONS = {'creole': ['1'],
'rougail': ['0.9'],
}
def get_function_name(root, version):
version = version.replace('.', '_')
return f'update_{root}_{version}'
FUNCTION_VERSIONS = [(root, version, get_function_name(root, version)) for root, versions in VERSIONS.items() for version in versions]
class RougailUpgrade:
def __init__(self, test=False, upgrade_help=None):
self.test = test
if upgrade_help is None:
upgrade_help = {}
self.upgrade_help = upgrade_help
def load_xml_from_folders(self,
srcfolder: str,
dstfolder: str,
namespace: str,
):
"""Loads all the XML files located in the xmlfolders' list
:param xmlfolders: list of full folder's name
"""
filenames = [filename for filename in listdir(srcfolder) if filename.endswith('.xml')]
filenames.sort()
for filename in filenames:
xmlsrc = join(srcfolder, filename)
xmldst = join(dstfolder, filename)
try:
parser = XMLParser(remove_blank_text=True)
document = parse(xmlsrc, parser)
except XMLSyntaxError as err:
raise Exception(_(f'not a XML file: {err}')) from err
root = document.getroot()
search_function_name = get_function_name(root.tag, root.attrib.get('version', '1'))
function_found = False
for root_name, version, function_version in FUNCTION_VERSIONS:
if function_found and hasattr(self, function_version):
print(f' - convert {filename} to version {version}')
upgrade_help = self.upgrade_help.get(function_version, {}).get(filename, {})
if upgrade_help.get('remove') is True:
continue
root = getattr(self, function_version)(root, upgrade_help, namespace)
if function_version == search_function_name:
function_found = True
with open(xmldst, 'wb') as xmlfh:
xmlfh.write(tostring(root, pretty_print=True, encoding="UTF-8", xml_declaration=True))
# if
# if not self.dtd.validate(document):
# dtd_error = self.dtd.error_log.filter_from_errors()[0]
# msg = _(f'not a valid XML file: {dtd_error}')
# raise DictConsistencyError(msg, 43, [xmlfile])
# yield xmlfile, document.getroot()
def update_rougail_0_9(self,
root: 'Element',
upgrade_help: dict,
namespace: str,
) -> 'Element':
# rename root
root.tag = 'rougail'
root.attrib['version'] = '0.9'
variables_auto_valid_enum = {}
variables_help = {}
families_help = {}
variables = {}
families = {}
valid_enums = {}
current_service = upgrade_help.get('services', {}).get('default', 'unknown')
files = {current_service: {}}
ips = {current_service: []}
servicelists = {}
test_unknowns_variables = set()
autos = []
constraints_obj = None
for subelement in root:
if not isinstance(subelement.tag, str):
# XML comment
continue
if subelement.tag == 'family_action':
root.remove(subelement)
continue
for subsubelement in subelement:
if not isinstance(subsubelement.tag, str):
# XML comment
continue
for subsubsubelement in subsubelement:
if not isinstance(subsubsubelement.tag, str):
# XML comment
continue
if subsubsubelement.tag == 'variable':
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('remove', []):
subsubelement.remove(subsubsubelement)
continue
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('type', {}):
subsubsubelement.attrib['type'] = upgrade_help['variables']['type'][subsubsubelement.attrib['name']]
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('hidden', {}).get('add', []):
subsubsubelement.attrib['hidden'] = 'True'
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('hidden', {}).get('remove', []):
self.remove(subsubsubelement, 'hidden', optional=True)
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('mandatory', {}).get('remove', []):
self.remove(subsubsubelement, 'mandatory')
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('mandatory', {}).get('add', []):
subsubsubelement.attrib['mandatory'] = 'True'
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('type', {}):
subsubsubelement.attrib['type'] = upgrade_help.get('variables', {}).get('type', {})[subsubsubelement.attrib['name']]
if namespace == 'configuration':
path = subsubsubelement.attrib['name']
npath = normalize_family(subsubsubelement.attrib['name'])
else:
path = namespace + '.' + subsubelement.attrib['name'] + '.' + subsubsubelement.attrib['name']
npath = normalize_family(namespace) + '.' + normalize_family(subsubelement.attrib['name']) + '.' + normalize_family(subsubsubelement.attrib['name'])
variables[path] = subsubsubelement
variables[npath] = subsubsubelement
type = subsubsubelement.attrib.get('type')
if type in ['oui/non', 'yes/no', 'on/off']:
variables_auto_valid_enum.setdefault(subsubsubelement.attrib['type'], []).append(path)
del subsubsubelement.attrib['type']
elif type == 'hostname_strict':
subsubsubelement.attrib['type'] = 'hostname'
elif type in ['domain', 'domain_strict']:
subsubsubelement.attrib['type'] = 'domainname'
elif type == 'string':
del subsubsubelement.attrib['type']
if self.test and subsubsubelement.attrib.get('auto_freeze') == 'True':
del subsubsubelement.attrib['auto_freeze']
#if self.test and subsubsubelement.attrib.get('redefine') == 'True':
# subsubsubelement.attrib['exists'] = 'False'
# del subsubsubelement.attrib['redefine']
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('redefine', {}).get('remove', {}):
del subsubsubelement.attrib['redefine']
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('remove_check', []):
subsubsubelement.attrib['remove_check'] = 'True'
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('mode', {}).get('modify', {}):
subsubsubelement.attrib['mode'] = upgrade_help.get('variables', {}).get('mode', {}).get('modify', {})[subsubsubelement.attrib['name']]
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('type', {}).get('modify', {}):
subsubsubelement.attrib['type'] = upgrade_help.get('variables', {}).get('type', {}).get('modify', {})[subsubsubelement.attrib['name']]
type = subsubsubelement.attrib['type']
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('test', {}):
subsubsubelement.attrib['test'] = upgrade_help.get('variables', {}).get('test', {})[subsubsubelement.attrib['name']]
if subsubsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('remove_value', []):
for value in subsubsubelement:
subsubsubelement.remove(value)
if subsubsubelement.attrib['name'] != normalize_family(subsubsubelement.attrib['name']):
if "description" not in subsubsubelement.attrib:
subsubsubelement.attrib['description'] = subsubsubelement.attrib['name']
subsubsubelement.attrib['name'] = normalize_family(subsubsubelement.attrib['name'])
elif subsubsubelement.tag == 'param':
if subsubelement.tag == 'check' and subsubelement.attrib['target'] in upgrade_help.get('check', {}).get('remove', []):
continue
type = subsubsubelement.attrib.get('type')
if type == 'eole':
subsubsubelement.attrib['type'] = 'variable'
type = 'variable'
if type == 'python':
subsubsubelement.attrib['type'] = 'function'
if subsubsubelement.text.startswith('range('):
func_ast = ast_parse(subsubsubelement.text)
subsubsubelement.text = 'range'
for arg in func_ast.body[0].value.args:
SubElement(subsubelement, 'param', type='number').text = str(arg.value)
else:
raise Exception(f'{subsubsubelement.text} is not a supported function')
type = 'function'
elif type in ('container', 'context'):
raise UpgradeError(_(f'cannot convert param with type "{type}"'))
if subsubelement.attrib['name'] == 'valid_entier' and not 'type' in subsubsubelement.attrib:
subsubsubelement.attrib['type'] = 'number'
if (not type or type == 'variable') and subsubsubelement.text in upgrade_help.get('variables', {}).get('remove', []):
subsubelement.remove(subsubsubelement)
continue
if subsubelement.attrib['name'] == 'valid_enum' and not type:
if subsubsubelement.attrib.get('name') == 'checkval':
if subsubelement.attrib['target'] in upgrade_help.get('check', {}).get('valid_enums', {}).get('checkval', {}).get('remove', []):
subsubelement.remove(subsubsubelement)
continue
raise UpgradeError(_('checkval in valid_enum is no more supported'))
if subsubsubelement.text.startswith('['):
for val in eval(subsubsubelement.text):
SubElement(subsubelement, 'param').text = str(val)
subsubelement.remove(subsubsubelement)
self.move(subsubsubelement, 'hidden', 'propertyerror', optional=True)
if type == 'variable' and subsubsubelement.text in upgrade_help.get('variables', {}).get('rename', []):
subsubsubelement.text = upgrade_help.get('variables', {}).get('rename', [])[subsubsubelement.text]
up = upgrade_help.get('params', {}).get(subsubsubelement.text)
if up:
if 'text' in up:
subsubsubelement.text = up['text']
if 'type' in up:
subsubsubelement.attrib['type'] = up['type']
elif subsubsubelement.tag == 'target':
type = subsubsubelement.attrib.get('type')
if type in ['service_accesslist', 'service_restrictionlist', 'interfacelist', 'hostlist']:
subsubelement.remove(subsubsubelement)
elif (not type or type == 'variable') and subsubsubelement.text in upgrade_help.get('variables', {}).get('remove', []):
subsubelement.remove(subsubsubelement)
elif type == 'family' and subsubsubelement.text in upgrade_help.get('families', {}).get('remove', []):
subsubelement.remove(subsubsubelement)
elif type == 'actionlist':
# for family in root.find('variables'):
# family_list = SubElement(subsubelement, 'target')
# family_list.attrib['type'] = 'family'
# family_list.text = namespace + '.' + family.attrib['name']
subsubelement.remove(subsubsubelement)
if upgrade_help.get('targets', {}).get(subsubsubelement.text, {}).get('optional'):
subsubsubelement.attrib['optional'] = upgrade_help.get('targets', {}).get(subsubsubelement.text, {}).get('optional')
has_target = False
for target in subsubelement:
if target.tag == 'target':
has_target = True
break
if not has_target:
subelement.remove(subsubelement)
continue
elif subsubsubelement.tag == 'slave':
if subsubsubelement.text in upgrade_help.get('variables', {}).get('remove', []):
subsubelement.remove(subsubsubelement)
continue
subsubsubelement.tag = 'follower'
subsubsubelement.text = normalize_family(subsubsubelement.text)
if subelement.tag == 'containers':
current_service = self.upgrade_container(subsubsubelement, current_service, files, ips, servicelists, upgrade_help)
subsubelement.remove(subsubsubelement)
if subelement.tag == 'help':
if subsubelement.tag == 'variable':
if not subsubelement.attrib['name'] in upgrade_help.get('variables', {}).get('remove', []):
variables_help[subsubelement.attrib['name']] = subsubelement.text
elif subsubelement.tag == 'family':
if subsubelement.attrib['name'] not in upgrade_help.get('families', {}).get('remove', []):
families_help[subsubelement.attrib['name']] = subsubelement.text
else:
raise Exception(f'unknown help tag {subsubelement.tag}')
subelement.remove(subsubelement)
continue
if subsubelement.tag == 'auto':
if subsubelement.attrib['target'] in upgrade_help.get('variables', {}).get('remove', []):
subelement.remove(subsubelement)
continue
autos.append(subsubelement.attrib['target'])
subsubelement.tag = 'fill'
if subsubelement.tag in ['fill', 'check']:
if subsubelement.tag == 'check' and subsubelement.attrib['name'] == 'valid_enum':
if subsubelement.attrib['target'] in upgrade_help.get('variables', {}).get('remove', []):
subelement.remove(subsubelement)
continue
params = []
for param in subsubelement:
if param.tag == 'param':
params.append(param.text)
key = '~'.join(params)
if key in valid_enums:
SubElement(valid_enums[key], 'target').text = subsubelement.attrib['target']
subelement.remove(subsubelement)
continue
valid_enums['~'.join(params)] = subsubelement
if subsubelement.tag == 'fill' and subsubelement.attrib['target'] in upgrade_help.get('fills', {}).get('remove', []):
subelement.remove(subsubelement)
continue
if subsubelement.tag == 'check' and subsubelement.attrib['target'] in upgrade_help.get('check', {}).get('remove', []):
subelement.remove(subsubelement)
continue
if subsubelement.attrib['target'] in upgrade_help.get('variables', {}).get('remove', []):
subelement.remove(subsubelement)
continue
if subsubelement.attrib['name'] == 'valid_networknetmask':
subsubelement.attrib['name'] = 'valid_network_netmask'
target = SubElement(subsubelement, 'target')
target.text = subsubelement.attrib['target']
if self.test:
target.attrib['optional'] = 'True'
del subsubelement.attrib['target']
elif subsubelement.tag == 'separators':
subelement.remove(subsubelement)
elif subsubelement.tag == 'condition':
if subsubelement.attrib['source'] in upgrade_help.get('variables', {}).get('remove', []):
try:
subelement.remove(subsubelement)
except:
pass
else:
if subsubelement.attrib['name'] == 'hidden_if_not_in':
subsubelement.attrib['name'] = 'disabled_if_not_in'
if subsubelement.attrib['name'] == 'hidden_if_in':
subsubelement.attrib['name'] = 'disabled_if_in'
if subsubelement.attrib['name'] == 'frozen_if_in':
subsubelement.attrib['name'] = 'hidden_if_in'
self.move(subsubelement, 'fallback', 'optional', optional='True')
if subsubelement.attrib['source'] in upgrade_help.get('variables', {}).get('rename', []):
subsubelement.attrib['source'] = upgrade_help.get('variables', {}).get('rename', [])[subsubelement.attrib['source']]
elif subsubelement.tag == 'group':
if subsubelement.attrib['master'] in upgrade_help.get('groups', {}).get('remove', []):
subelement.remove(subsubelement)
else:
self.move(subsubelement, 'master', 'leader')
for follower in subsubelement:
if '.' in subsubelement.attrib['leader']:
path = subsubelement.attrib['leader'].rsplit('.', 1)[0] + '.' + follower.text
else:
path = follower.text
self.remove(variables[path], 'multi', optional=True)
if subsubelement.attrib['leader'] in upgrade_help.get('groups', {}).get('reorder', {}):
for follower in subsubelement:
subsubelement.remove(follower)
for follower in upgrade_help.get('groups', {}).get('reorder', {})[subsubelement.attrib['leader']]:
SubElement(subsubelement, 'follower').text = follower
if subelement.tag == 'files':
current_service = self.upgrade_container(subsubelement, current_service, files, ips, servicelists, upgrade_help)
subelement.remove(subsubelement)
if subsubelement.tag == 'family':
self.remove(subsubelement, 'icon', optional=True)
if subsubelement.attrib['name'] in upgrade_help.get('families', {}).get('mode', {}).get('remove', []) and 'mode' in subsubelement.attrib:
del subsubelement.attrib['mode']
if subsubelement.attrib['name'] in upgrade_help.get('families', {}).get('rename', {}):
subsubelement.attrib['name'] = upgrade_help.get('families', {}).get('rename', {})[subsubelement.attrib['name']]
if subsubelement.attrib['name'] in upgrade_help.get('families', {}).get('remove', []):
subelement.remove(subsubelement)
continue
if subsubelement.attrib['name'] in upgrade_help.get('families', {}).get('hidden', {}).get('add', []):
subsubelement.attrib['hidden'] = 'True'
if subsubelement.attrib['name'] != normalize_family(subsubelement.attrib['name']):
if "description" not in subsubelement.attrib:
subsubelement.attrib['description'] = subsubelement.attrib['name']
subsubelement.attrib['name'] = normalize_family(subsubelement.attrib['name'])
families[subsubelement.attrib['name']] = subsubelement
# if empty, remove
if is_empty(subelement) or subelement.tag in ['containers', 'files', 'help']:
root.remove(subelement)
continue
# copy subelement
if subelement.tag == 'constraints':
constraints_obj = subelement
services = None
service_elt = {}
for variable, obj in upgrade_help.get('variables', {}).get('add', {}).items():
family = next(iter(families.values()))
variables[variable] = SubElement(family, 'variable', name=variable)
if 'value' in obj:
SubElement(variables[variable], 'value').text = obj['value']
for name in ['hidden', 'exists']:
if name in obj:
variables[variable].attrib[name] = obj[name]
files_is_empty = True
for lst in files.values():
if len(lst):
files_is_empty = False
break
if not files_is_empty:
services = Element('services')
root.insert(0, services)
for service_name, lst in files.items():
service = self.create_service(services, service_name, service_elt, servicelists, upgrade_help)
for file_ in lst.values():
self.move(file_, 'name_type', 'file_type', optional=True)
if 'file_type' in file_.attrib and file_.attrib['file_type'] == 'SymLinkOption':
file_.attrib['file_type'] = 'variable'
if variables[file_.text].attrib['type'] == 'string':
variables[file_.text].attrib['type'] = 'filename'
self.remove(file_, 'rm', optional=True)
self.remove(file_, 'mkdir', optional=True)
service.append(file_)
ips_is_empty = True
for lst in ips.values():
if len(lst):
ips_is_empty = False
break
if not ips_is_empty:
if services is None:
services = Element('services')
root.insert(0, services)
for service_name, lst in ips.items():
service = self.create_service(services, service_name, service_elt, servicelists, upgrade_help)
for ip in lst:
if ip.text in upgrade_help.get('ips', {}).get('remove', []):
continue
for type in ['ip_type', 'netmask_type']:
if type in ip.attrib and ip.attrib[type] == 'SymLinkOption':
ip.attrib[type] = 'variable'
if 'netmask' in ip.attrib:
if ip.attrib['netmask'] == '255.255.255.255':
del ip.attrib['netmask']
if ip.text in variables and 'type' not in variables[ip.text].attrib:
variables[ip.text].attrib['type'] = 'ip'
else:
if ip.text in variables and 'type' not in variables[ip.text].attrib:
variables[ip.text].attrib['type'] = 'network'
if ip.attrib['netmask'] in variables and 'type' not in variables[ip.attrib['netmask']].attrib:
variables[ip.attrib['netmask']].attrib['type'] = 'netmask'
elif ip.text in variables and 'type' not in variables[ip.text].attrib:
variables[ip.text].attrib['type'] = 'ip'
self.remove(ip, 'interface')
self.remove(ip, 'interface_type', optional=True)
self.remove(ip, 'service_restrictionlist', optional=True)
service.append(ip)
if ips_is_empty and files_is_empty:
for service_name in files:
if services is None:
services = Element('services')
root.insert(0, services)
if service_name != 'unknown':
self.create_service(services, service_name, service_elt, servicelists, upgrade_help)
if constraints_obj is None:
constraints_obj = SubElement(root, 'constraints')
if variables_auto_valid_enum:
for type, variables_valid_enum in variables_auto_valid_enum.items():
valid_enum = SubElement(constraints_obj, 'check')
valid_enum.attrib['name'] = 'valid_enum'
for val in type.split('/'):
param = SubElement(valid_enum, 'param')
param.text = val
for variable in variables_valid_enum:
target = SubElement(valid_enum, 'target')
target.text = variable
for name, text in variables_help.items():
variables[name].attrib['help'] = text
for name, text in families_help.items():
if name in upgrade_help.get('families', {}).get('rename', {}):
name = upgrade_help.get('families', {}).get('rename', {})[name]
families[normalize_family(name)].attrib['help'] = text
for auto in autos:
if auto in variables:
variables[auto].attrib['hidden'] = 'True'
else:
# redefine value in first family
family = next(iter(families.values()))
variable = SubElement(family, 'variable', name=auto, redefine="True", hidden="True")
if self.test:
del variable.attrib['redefine']
return root
@staticmethod
def move(elt, src, dst, optional=False):
if src == 'text':
value = elt.text
elt.text = None
else:
if optional and src not in elt.attrib:
return
value = elt.attrib[src]
del elt.attrib[src]
#
if dst == 'text':
elt.text = value
else:
elt.attrib[dst] = value
@staticmethod
def remove(elt, src, optional=False):
if optional and src not in elt.attrib:
return
del elt.attrib[src]
@staticmethod
def create_service(services, service_name, service_elt, servicelists, upgrade_help):
if service_name in service_elt:
return service_elt[service_name]
service = SubElement(services, 'service')
service.attrib['name'] = service_name
if service_name == 'unknown':
service.attrib['manage'] = 'False'
if service_name in upgrade_help.get('services', {}).get('unmanage', []):
service.attrib['manage'] = 'False'
service_elt[service_name] = service
if upgrade_help.get('servicelists', {}).get(service_name):
service.attrib['servicelist'] = upgrade_help.get('servicelists', {}).get(service_name)
elif service_name in servicelists:
service.attrib['servicelist'] = servicelists[service_name]
return service
def upgrade_container(self, elt, current_service, files, ips, servicelists, upgrade_help):
if elt.tag == 'file':
self.move(elt, 'name', 'text')
self.remove(elt, 'del_comment', optional=True)
elt.attrib['engine'] = 'creole_legacy'
if (not 'instance_mode' in elt.attrib or elt.attrib['instance_mode'] != 'when_container') and \
elt.text not in upgrade_help.get('files', {}).get('remove', {}):
if elt.attrib.get('filelist') in upgrade_help.get('services', {}).get('filelist_service', {}):
elt_service = upgrade_help.get('services', {}).get('filelist_service', {})[elt.attrib['filelist']]
if elt_service in files:
service = elt_service
else:
service = current_service
else:
service = current_service
files[service][elt.text] = elt
elif elt.tag in ['host', 'disknod', 'fstab', 'interface', 'package', 'service_access']:
pass
elif elt.tag == 'service_restriction':
for restriction in elt:
if restriction.tag == 'ip' and restriction.text != '0.0.0.0':
self.remove(restriction, 'ip_type', optional=True)
self.remove(restriction, 'netmask_type', optional=True)
if elt.attrib['service'] in upgrade_help.get('services', {}).get('rename', {}):
elt_service = upgrade_help.get('services', {}).get('rename', {})[elt.attrib['service']]
else:
elt_service = elt.attrib['service']
if elt_service in ips:
service = elt_service
else:
service = current_service
ips[service].append(restriction)
elif elt.tag == 'service':
new_name = elt.text
if current_service == 'unknown':
if new_name in files:
raise Exception('hu?')
files[new_name] = files[current_service]
del files[current_service]
ips[new_name] = ips[current_service]
del ips[current_service]
elif new_name not in files:
files[new_name] = {}
ips[new_name] = []
current_service = new_name
if 'servicelist' in elt.attrib:
servicelists[current_service] = elt.attrib['servicelist']
else:
raise Exception(f'unknown containers tag {elt.tag}')
return current_service
def is_empty(elt, not_check_attrib=False):
if not not_check_attrib and elt.attrib:
return False
empty = True
for file_ in elt:
empty = False
return empty
def update_creole_1():
print('pfff')

75
src/rougail/utils.py
View File

@ -1,23 +1,66 @@
"""Rougail's tools
Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
""" """
utilitaires créole from typing import List
""" from unicodedata import normalize, combining
import unicodedata import re
from importlib.machinery import SourceFileLoader
from importlib.util import spec_from_loader, module_from_spec
from .i18n import _ from .i18n import _
from .error import DictConsistencyError
NAME_REGEXP = re.compile(r"^[a-z0-9_]*$")
def normalize_family(family_name: str, def valid_variable_family_name(name: str,
check_name: bool=True, xmlfiles: List[str],
allow_dot: bool=False) -> str: ) -> None:
match = NAME_REGEXP.search(name)
if not match:
msg = _(f'invalid variable or family name "{name}" must only contains '
'lowercase ascii character, number or _')
raise DictConsistencyError(msg, 76, xmlfiles)
def normalize_family(family_name: str) -> str:
"""replace space, accent, uppercase, ... by valid character """replace space, accent, uppercase, ... by valid character
""" """
family_name = family_name.replace('-', '_') family_name = family_name.replace('-', '_').replace(' ', '_').replace('.', '_')
if not allow_dot: nfkd_form = normalize('NFKD', family_name)
family_name = family_name.replace('.', '_') family_name = ''.join([c for c in nfkd_form if not combining(c)])
family_name = family_name.replace(' ', '_') return family_name.lower()
nfkd_form = unicodedata.normalize('NFKD', family_name)
family_name = ''.join([c for c in nfkd_form if not unicodedata.combining(c)])
family_name = family_name.lower()
if check_name and family_name == 'containers':
raise ValueError(_(f'"{family_name}" is a forbidden family name'))
return family_name
def load_modules(eosfunc_file) -> List[str]:
"""list all functions in eosfunc
"""
loader = SourceFileLoader('eosfunc', eosfunc_file)
spec = spec_from_loader(loader.name, loader)
eosfunc = module_from_spec(spec)
loader.exec_module(eosfunc)
return eosfunc

161
src/rougail/xml_compare.py
View File

@ -1,161 +0,0 @@
try:
import doctest
doctest.OutputChecker
except (AttributeError, ImportError): # Python < 2.4
import util.doctest24 as doctest
try:
import xml.etree.ElementTree as ET
except ImportError:
import elementtree.ElementTree as ET
from xml.parsers.expat import ExpatError as XMLParseError
RealOutputChecker = doctest.OutputChecker
def debug(*msg):
import sys
print >> sys.stderr, ' '.join(map(str, msg))
class HTMLOutputChecker(RealOutputChecker):
def check_output(self, want, got, optionflags):
normal = RealOutputChecker.check_output(self, want, got, optionflags)
if normal or not got:
return normal
try:
want_xml = make_xml(want)
except XMLParseError:
pass
else:
try:
got_xml = make_xml(got)
except XMLParseError:
pass
else:
if xml_compare(want_xml, got_xml):
return True
return False
def output_difference(self, example, got, optionflags):
actual = RealOutputChecker.output_difference(
self, example, got, optionflags)
want_xml = got_xml = None
try:
want_xml = make_xml(example.want)
want_norm = make_string(want_xml)
except XMLParseError as e:
if example.want.startswith('<'):
want_norm = '(bad XML: %s)' % e
# '<xml>%s</xml>' % example.want
else:
return actual
try:
got_xml = make_xml(got)
got_norm = make_string(got_xml)
except XMLParseError as e:
if example.want.startswith('<'):
got_norm = '(bad XML: %s)' % e
else:
return actual
s = '%s\nXML Wanted: %s\nXML Got : %s\n' % (
actual, want_norm, got_norm)
if got_xml and want_xml:
result = []
xml_compare(want_xml, got_xml, result.append)
s += 'Difference report:\n%s\n' % '\n'.join(result)
return s
def xml_sort(children):
tcl1 = {}
#idx = 0
for child in children:
if 'name' in child.attrib:
key = child.attrib['name']
else:
key = child.tag
if key not in tcl1:
tcl1[key] = []
tcl1[key].append(child)
cl1_keys = list(tcl1.keys())
cl1_keys.sort()
cl1 = []
for key in cl1_keys:
cl1.extend(tcl1[key])
return cl1
def xml_compare(x1, x2):
if x1.tag != x2.tag:
print ('Tags do not match: %s and %s' % (x1.tag, x2.tag))
return False
for name, value in x1.attrib.items():
if x2.attrib.get(name) != value:
print ('Attributes do not match: %s=%r, %s=%r'
% (name, value, name, x2.attrib.get(name)))
return False
for name in x2.attrib:
if name not in x1.attrib:
print ('x2 has an attribute x1 is missing: %s'
% name)
return False
if not text_compare(x1.text, x2.text):
print ('text: %r != %r' % (x1.text, x2.text))
return False
if not text_compare(x1.tail, x2.tail):
print ('tail: %r != %r' % (x1.tail, x2.tail))
return False
cl1 = xml_sort(x1.getchildren())
cl2 = xml_sort(x2.getchildren())
if len(cl1) != len(cl2):
cl1_tags = []
for c in cl1:
cl1_tags.append(c.tag)
cl2_tags = []
for c in cl2:
cl2_tags.append(c.tag)
print ('children length differs, %i != %i (%s != %s)'
% (len(cl1), len(cl2), cl1_tags, cl2_tags))
return False
i = 0
for c1, c2 in zip(cl1, cl2):
i += 1
if not xml_compare(c1, c2):
if 'name' in c1.attrib:
name = c1.attrib['name']
else:
name = i
print ('in tag "%s" with name "%s"'
% (c1.tag, name))
return False
return True
def text_compare(t1, t2):
if not t1 and not t2:
return True
if t1 == '*' or t2 == '*':
return True
return (t1 or '').strip() == (t2 or '').strip()
def make_xml(s):
return ET.XML('<xml>%s</xml>' % s)
def make_string(xml):
if isinstance(xml, (str, unicode)):
xml = make_xml(xml)
s = ET.tostring(xml)
if s == '<xml />':
return ''
assert s.startswith('<xml>') and s.endswith('</xml>'), repr(s)
return s[5:-6]
def install():
doctest.OutputChecker = HTMLOutputChecker

111
src/rougail/xmlreflector.py
View File

@ -1,90 +1,77 @@
# coding: utf-8 """load XML file from directory
from os.path import join, isfile, basename, isdir
from os import listdir
#from io import BytesIO
from lxml.etree import DTD, parse, tostring # , XMLParser Created by:
EOLE (http://eole.orion.education.fr)
Copyright (C) 2005-2018
Forked by:
Cadoles (http://www.cadoles.com)
Copyright (C) 2019-2021
distribued with GPL-2 or later license
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
from typing import List
from os.path import join, isfile
from os import listdir
from lxml.etree import DTD, parse, XMLSyntaxError # pylint: disable=E0611
from .i18n import _ from .i18n import _
from .error import DictConsistencyError from .error import DictConsistencyError
class XMLReflector(object): class XMLReflector:
"""Helper class for loading the Creole XML file, """Helper class for loading the Creole XML file,
parsing it, validating against the Creole DTD, parsing it, validating against the Creole DTD,
writing the xml result on the disk writing the xml result on the disk
""" """
def __init__(self): def __init__(self,
self.dtd = None rougailconfig: 'RougailConfig',
) -> None:
def parse_dtd(self, dtdfilename):
"""Loads the Creole DTD """Loads the Creole DTD
:raises IOError: if the DTD is not found :raises IOError: if the DTD is not found
:param dtdfilename: the full filename of the Creole DTD :param dtdfilename: the full filename of the Creole DTD
""" """
dtdfilename = rougailconfig['dtdfilename']
if not isfile(dtdfilename): if not isfile(dtdfilename):
raise IOError(_("no such DTD file: {}").format(dtdfilename)) raise IOError(_("no such DTD file: {}").format(dtdfilename))
with open(dtdfilename, 'r') as dtdfd: with open(dtdfilename, 'r') as dtdfd:
self.dtd = DTD(dtdfd) self.dtd = DTD(dtdfd)
def parse_xmlfile(self, xmlfile): def load_xml_from_folders(self,
"""Parses and validates some Creole XML against the Creole DTD xmlfolders: List[str],
):
:returns: the root element tree object
"""
# document = parse(BytesIO(xmlfile), XMLParser(remove_blank_text=True))
document = parse(xmlfile)
if not self.dtd.validate(document):
raise DictConsistencyError(_(f'"{xmlfile}" not a valid xml file: {self.dtd.error_log.filter_from_errors()[0]}'))
return document.getroot()
def load_xml_from_folders(self, xmlfolders):
"""Loads all the XML files located in the xmlfolders' list """Loads all the XML files located in the xmlfolders' list
:param xmlfolders: list of full folder's name :param xmlfolders: list of full folder's name
""" """
documents = []
if not isinstance(xmlfolders, list):
xmlfolders = [xmlfolders]
for xmlfolder in xmlfolders: for xmlfolder in xmlfolders:
if isinstance(xmlfolder, list) or isinstance(xmlfolder, tuple): filenames = [join(xmlfolder, filename) for filename in listdir(xmlfolder) if \
# directory group : collect files from each filename.endswith('.xml')]
# directory and sort them before loading
group_files = []
for idx, subdir in enumerate(xmlfolder):
if isdir(subdir):
for filename in listdir(subdir):
group_files.append((filename, idx, subdir))
else:
group_files.append(basename(subdir), idx, dirname(subdir))
def sort_group(file1, file2):
if file1[0] == file2[0]:
# sort by initial xmlfolder order if same name
return file1[1].__cmp__(file2[1])
# sort by filename
elif file1[0] > file2[0]:
return 1
else:
return -1
group_files.sort(sort_group)
filenames = [join(f[2], f[0]) for f in group_files]
elif isdir(xmlfolder):
filenames = []
for filename in listdir(xmlfolder):
filenames.append(join(xmlfolder, filename))
filenames.sort() filenames.sort()
else:
filenames = [xmlfolder]
for xmlfile in filenames: for xmlfile in filenames:
if xmlfile.endswith('.xml'): try:
#xmlfile_path = join(xmlfolder, xmlfile) document = parse(xmlfile)
documents.append((xmlfile, self.parse_xmlfile(xmlfile))) except XMLSyntaxError as err:
return documents raise DictConsistencyError(_(f'not a XML file: {err}'), 52, [xmlfile]) from err
if not self.dtd.validate(document):
def save_xmlfile(self, xmlfilename, xml): # pylint: disable=R0201 dtd_error = self.dtd.error_log.filter_from_errors()[0]
"""Write a bunch of XML on the disk msg = _(f'not a valid XML file: {dtd_error}')
""" raise DictConsistencyError(msg, 43, [xmlfile])
with open(xmlfilename, 'w') as xmlfh: yield xmlfile, document.getroot()
xmlfh.write(tostring(xml, pretty_print=True, encoding="UTF-8", xml_declaration=True).decode('utf8'))

2
tests/dictionaries/00empty/00_base.xml
View File

@ -1,5 +1,5 @@
<?xml version='1.0' encoding='UTF-8'?> <?xml version='1.0' encoding='UTF-8'?>
<rougail> <rougail version="0.9">
<services> <services>
<service name="tata"> <service name="tata">
</service> </service>

10
tests/dictionaries/00empty/makedict/after.json Normal file
View File

@ -0,0 +1,10 @@
{
"services.tata.activate": {
"owner": "default",
"value": true
},
"services.tata.manage": {
"owner": "default",
"value": true
}
}

4
tests/dictionaries/00empty/makedict/base.json Normal file
View File

@ -0,0 +1,4 @@
{
"services.tata.activate": true,
"services.tata.manage": true
}

10
tests/dictionaries/00empty/makedict/before.json Normal file
View File

@ -0,0 +1,10 @@
{
"services.tata.activate": {
"owner": "default",
"value": true
},
"services.tata.manage": {
"owner": "default",
"value": true
}
}

20
tests/dictionaries/00empty/tiramisu/base.py
View File

@ -1,14 +1,18 @@
import imp from importlib.machinery import SourceFileLoader
func = imp.load_source('func', 'tests/dictionaries/../eosfunc/test.py') from importlib.util import spec_from_loader, module_from_spec
loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
spec = spec_from_loader(loader.name, loader)
func = module_from_spec(spec)
loader.exec_module(func)
for key, value in dict(locals()).items(): for key, value in dict(locals()).items():
if key != ['imp', 'func']: if key != ['SourceFileLoader', 'func']:
setattr(func, key, value) setattr(func, key, value)
try: try:
from tiramisu3 import * from tiramisu3 import *
except: except:
from tiramisu import * from tiramisu import *
from rougail.tiramisu import ConvertDynOptionDescription option_3 = BoolOption(name="activate", doc="activate", default=True)
option_2 = OptionDescription(name='tata', doc='tata', children=[]) option_4 = BoolOption(name="manage", doc="manage", default=True)
option_2.impl_set_information("manage", True) option_2 = OptionDescription(name="tata", doc="tata", children=[option_3, option_4])
option_1 = OptionDescription(name='services', doc='services', properties=frozenset({'hidden'}), children=[option_2]) option_1 = OptionDescription(name="services", doc="services", children=[option_2], properties=frozenset({"hidden"}))
option_0 = OptionDescription(name='baseoption', doc='baseoption', children=[option_1]) option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

20
tests/dictionaries/00load_autofreeze/00-base.xml
View File

@ -1,23 +1,13 @@
<?xml version='1.0' encoding='UTF-8'?> <?xml version='1.0' encoding='UTF-8'?>
<rougail> <rougail version="0.9">
<variables> <variables>
<family name="général"> <variable name="myvar" auto_freeze="True">
<variable name="mode_conteneur_actif" type="oui/non" description="No change" auto_freeze="True"> <value>no</value>
<value>non</value>
</variable> </variable>
<variable name="module_instancie" type="oui/non" description="No change"> <variable name="server_deployed" type="boolean">
<value>non</value> <value>False</value>
</variable> </variable>
</family>
<separators/>
</variables> </variables>
<constraints>
</constraints>
<help/>
</rougail> </rougail>
<!-- vim: ts=4 sw=4 expandtab <!-- vim: ts=4 sw=4 expandtab
--> -->

10
tests/dictionaries/00load_autofreeze/makedict/after.json Normal file
View File

@ -0,0 +1,10 @@
{
"rougail.myvar": {
"owner": "forced",
"value": "no"
},
"rougail.server_deployed": {
"owner": "default",
"value": false
}
}

5
tests/dictionaries/00load_autofreeze/makedict/base.json
View File

@ -1 +1,4 @@
{"rougail.general.mode_conteneur_actif": "non", "rougail.general.module_instancie": "non"} {
"rougail.myvar": "no",
"rougail.server_deployed": false
}

10
tests/dictionaries/00load_autofreeze/makedict/before.json Normal file
View File

@ -0,0 +1,10 @@
{
"rougail.myvar": {
"owner": "default",
"value": "no"
},
"rougail.server_deployed": {
"owner": "default",
"value": false
}
}

20
tests/dictionaries/00load_autofreeze/tiramisu/base.py
View File

@ -1,15 +1,17 @@
import imp from importlib.machinery import SourceFileLoader
func = imp.load_source('func', 'tests/dictionaries/../eosfunc/test.py') from importlib.util import spec_from_loader, module_from_spec
loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
spec = spec_from_loader(loader.name, loader)
func = module_from_spec(spec)
loader.exec_module(func)
for key, value in dict(locals()).items(): for key, value in dict(locals()).items():
if key != ['imp', 'func']: if key != ['SourceFileLoader', 'func']:
setattr(func, key, value) setattr(func, key, value)
try: try:
from tiramisu3 import * from tiramisu3 import *
except: except:
from tiramisu import * from tiramisu import *
from rougail.tiramisu import ConvertDynOptionDescription option_3 = BoolOption(name="server_deployed", doc="server_deployed", default=False, properties=frozenset({"mandatory", "normal"}))
option_4 = ChoiceOption(properties=frozenset({'mandatory', 'normal'}), name='module_instancie', doc='No change', multi=False, default='non', values=('oui', 'non')) option_2 = StrOption(name="myvar", doc="myvar", default="no", properties=frozenset({"basic", "force_store_value", "mandatory", Calculation(func.calc_value, Params(ParamValue('frozen'), kwargs={'condition': ParamOption(option_3, todict=True, notraisepropertyerror=True), 'expected': ParamValue(True)}))}))
option_3 = ChoiceOption(properties=frozenset({'auto_freeze', 'basic', 'force_store_value', 'mandatory', Calculation(calc_value, Params(ParamValue('auto_frozen'), kwargs={'condition': ParamOption(option_4, todict=True), 'expected': ParamValue('oui'), 'reverse_condition': ParamValue(True)}))}), name='mode_conteneur_actif', doc='No change', multi=False, default='non', values=('oui', 'non')) option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2, option_3])
option_2 = OptionDescription(name='general', doc='général', properties=frozenset({'basic'}), children=[option_3, option_4]) option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])
option_1 = OptionDescription(name='rougail', doc='rougail', children=[option_2])
option_0 = OptionDescription(name='baseoption', doc='baseoption', children=[option_1])

22
tests/dictionaries/00load_autofreezeexpert/00-base.xml
View File

@ -1,25 +1,13 @@
<?xml version='1.0' encoding='UTF-8'?> <?xml version='1.0' encoding='UTF-8'?>
<rougail> <rougail version="0.9">
<services/>
<variables> <variables>
<family name="général"> <variable name="my_var" auto_freeze="True" mode="expert">
<variable name="mode_conteneur_actif" type="oui/non" description="No change" auto_freeze="True" mode="expert"> <value>no</value>
<value>non</value>
</variable> </variable>
<variable name="module_instancie" type="oui/non" description="No change"> <variable name="server_deployed" type="boolean">
<value>non</value> <value>False</value>
</variable> </variable>
</family>
<separators/>
</variables> </variables>
<constraints>
</constraints>
<help/>
</rougail> </rougail>
<!-- vim: ts=4 sw=4 expandtab <!-- vim: ts=4 sw=4 expandtab
--> -->

10
tests/dictionaries/00load_autofreezeexpert/makedict/after.json Normal file
View File

@ -0,0 +1,10 @@
{
"rougail.my_var": {
"owner": "forced",
"value": "no"
},
"rougail.server_deployed": {
"owner": "default",
"value": false
}
}

5
tests/dictionaries/00load_autofreezeexpert/makedict/base.json
View File

@ -1 +1,4 @@
{"rougail.general.mode_conteneur_actif": "non", "rougail.general.module_instancie": "non"} {
"rougail.my_var": "no",
"rougail.server_deployed": false
}

10
tests/dictionaries/00load_autofreezeexpert/makedict/before.json Normal file
View File

@ -0,0 +1,10 @@
{
"rougail.my_var": {
"owner": "default",
"value": "no"
},
"rougail.server_deployed": {
"owner": "default",
"value": false
}
}

20
tests/dictionaries/00load_autofreezeexpert/tiramisu/base.py
View File

@ -1,15 +1,17 @@
import imp from importlib.machinery import SourceFileLoader
func = imp.load_source('func', 'tests/dictionaries/../eosfunc/test.py') from importlib.util import spec_from_loader, module_from_spec
loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
spec = spec_from_loader(loader.name, loader)
func = module_from_spec(spec)
loader.exec_module(func)
for key, value in dict(locals()).items(): for key, value in dict(locals()).items():
if key != ['imp', 'func']: if key != ['SourceFileLoader', 'func']:
setattr(func, key, value) setattr(func, key, value)
try: try:
from tiramisu3 import * from tiramisu3 import *
except: except:
from tiramisu import * from tiramisu import *
from rougail.tiramisu import ConvertDynOptionDescription option_3 = BoolOption(name="server_deployed", doc="server_deployed", default=False, properties=frozenset({"mandatory", "normal"}))
option_4 = ChoiceOption(properties=frozenset({'mandatory', 'normal'}), name='module_instancie', doc='No change', multi=False, default='non', values=('oui', 'non')) option_2 = StrOption(name="my_var", doc="my_var", default="no", properties=frozenset({"expert", "force_store_value", "mandatory", Calculation(func.calc_value, Params(ParamValue('frozen'), kwargs={'condition': ParamOption(option_3, todict=True, notraisepropertyerror=True), 'expected': ParamValue(True)}))}))
option_3 = ChoiceOption(properties=frozenset({'auto_freeze', 'expert', 'force_store_value', 'mandatory', Calculation(calc_value, Params(ParamValue('auto_frozen'), kwargs={'condition': ParamOption(option_4, todict=True), 'expected': ParamValue('oui'), 'reverse_condition': ParamValue(True)}))}), name='mode_conteneur_actif', doc='No change', multi=False, default='non', values=('oui', 'non')) option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2, option_3])
option_2 = OptionDescription(name='general', doc='général', properties=frozenset({'normal'}), children=[option_3, option_4]) option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])
option_1 = OptionDescription(name='rougail', doc='rougail', children=[option_2])
option_0 = OptionDescription(name='baseoption', doc='baseoption', children=[option_1])

17
tests/dictionaries/00load_autosave/00-base.xml
View File

@ -1,22 +1,13 @@
<?xml version='1.0' encoding='UTF-8'?> <?xml version='1.0' encoding='UTF-8'?>
<rougail> <rougail version="0.9">
<services/>
<variables> <variables>
<family name="général"> <variable name="server_deployed" type="boolean"/>
<variable name="mode_conteneur_actif" type="oui/non" description="No change" auto_save="True"> <family name="general" description="général">
<variable name="mode_conteneur_actif" type="string" description="No change" auto_save="True">
<value>non</value> <value>non</value>
</variable> </variable>
</family> </family>
<separators/>
</variables> </variables>
<constraints>
</constraints>
<help/>
</rougail> </rougail>
<!-- vim: ts=4 sw=4 expandtab <!-- vim: ts=4 sw=4 expandtab
--> -->

10
tests/dictionaries/00load_autosave/makedict/after.json Normal file
View File

@ -0,0 +1,10 @@
{
"rougail.server_deployed": {
"owner": "default",
"value": true
},
"rougail.general.mode_conteneur_actif": {
"owner": "forced",
"value": "non"
}
}

5
tests/dictionaries/00load_autosave/makedict/base.json
View File

@ -1 +1,4 @@
{"rougail.general.mode_conteneur_actif": "non"} {
"rougail.server_deployed": true,
"rougail.general.mode_conteneur_actif": "non"
}

10
tests/dictionaries/00load_autosave/makedict/before.json Normal file
View File

@ -0,0 +1,10 @@
{
"rougail.server_deployed": {
"owner": "default",
"value": true
},
"rougail.general.mode_conteneur_actif": {
"owner": "default",
"value": "non"
}
}

20
tests/dictionaries/00load_autosave/tiramisu/base.py
View File

@ -1,14 +1,18 @@
import imp from importlib.machinery import SourceFileLoader
func = imp.load_source('func', 'tests/dictionaries/../eosfunc/test.py') from importlib.util import spec_from_loader, module_from_spec
loader = SourceFileLoader('func', 'tests/dictionaries/../eosfunc/test.py')
spec = spec_from_loader(loader.name, loader)
func = module_from_spec(spec)
loader.exec_module(func)
for key, value in dict(locals()).items(): for key, value in dict(locals()).items():
if key != ['imp', 'func']: if key != ['SourceFileLoader', 'func']:
setattr(func, key, value) setattr(func, key, value)
try: try:
from tiramisu3 import * from tiramisu3 import *
except: except:
from tiramisu import * from tiramisu import *
from rougail.tiramisu import ConvertDynOptionDescription option_2 = BoolOption(name="server_deployed", doc="server_deployed", default=True, properties=frozenset({"mandatory", "normal"}))
option_3 = ChoiceOption(properties=frozenset({'basic', 'force_store_value', 'mandatory'}), name='mode_conteneur_actif', doc='No change', multi=False, default='non', values=('oui', 'non')) option_4 = StrOption(name="mode_conteneur_actif", doc="No change", default="non", properties=frozenset({"basic", "force_store_value", "mandatory"}))
option_2 = OptionDescription(name='general', doc='général', properties=frozenset({'basic'}), children=[option_3]) option_3 = OptionDescription(name="general", doc="général", children=[option_4], properties=frozenset({"basic"}))
option_1 = OptionDescription(name='rougail', doc='rougail', children=[option_2]) option_1 = OptionDescription(name="rougail", doc="rougail", children=[option_2, option_3])
option_0 = OptionDescription(name='baseoption', doc='baseoption', children=[option_1]) option_0 = OptionDescription(name="baseoption", doc="baseoption", children=[option_1])

17
tests/dictionaries/00load_autosaveexpert/00-base.xml
View File

@ -1,22 +1,13 @@
<?xml version='1.0' encoding='UTF-8'?> <?xml version='1.0' encoding='UTF-8'?>
<rougail> <rougail version="0.9">
<services/>
<variables> <variables>
<family name="général"> <variable name="server_deployed" type="boolean"/>
<variable name="mode_conteneur_actif" type="oui/non" description="No change" auto_save="True" mode="expert"> <family name="general" description="général">
<variable name="mode_conteneur_actif" type="string" description="No change" auto_save="True" mode="expert">
<value>non</value> <value>non</value>
</variable> </variable>
</family> </family>
<separators/>
</variables> </variables>
<constraints>
</constraints>
<help/>
</rougail> </rougail>
<!-- vim: ts=4 sw=4 expandtab <!-- vim: ts=4 sw=4 expandtab
--> -->

10
tests/dictionaries/00load_autosaveexpert/makedict/after.json Normal file
View File

@ -0,0 +1,10 @@
{
"rougail.server_deployed": {
"owner": "default",
"value": true
},
"rougail.general.mode_conteneur_actif": {
"owner": "forced",
"value": "non"
}
}

Some files were not shown because too many files have changed in this diff Show More