Compare commits

..

7 Commits

Author SHA1 Message Date
Matthieu Lamalle 5eac425fda feat(authn) : add case to test multiples CIDR
Cadoles/bouncer/pipeline/pr-develop Build started... Details
2024-09-25 15:15:04 +02:00
wpetit 0b032fccc9 Merge pull request 'Moteur de règles V2' (#40) from rule-engine-2 into develop
Cadoles/bouncer/pipeline/head This commit looks good Details
Reviewed-on: #40
2024-09-25 09:11:46 +02:00
wpetit fea0610346 feat: reusable rule engine to prevent memory reallocation
Cadoles/bouncer/pipeline/pr-develop This commit looks good Details
2024-09-24 18:45:34 +02:00
wpetit f37425018b feat: use shared redis client to maximize pooling usage (#39)
Cadoles/bouncer/pipeline/head This commit looks good Details
2024-09-23 15:16:30 +02:00
wpetit 4801974ca3 fix(queue): prevent metrics update cancellation on aborted http requests (#39)
Cadoles/bouncer/pipeline/head This commit looks good Details
2024-09-23 10:34:24 +02:00
wpetit bf15732935 feat: disable sentry integration when no dsn is defined
Cadoles/bouncer/pipeline/head This commit looks good Details
2024-09-23 10:13:04 +02:00
wpetit 8317ac5b9a feat: add configurable profiling endpoints (#38) 2024-09-23 10:12:42 +02:00
46 changed files with 1190 additions and 245 deletions

View File

@ -17,7 +17,8 @@ GOTEST_ARGS ?= -short
OPENWRT_DEVICE ?= 192.168.1.1 OPENWRT_DEVICE ?= 192.168.1.1
SIEGE_URLS_FILE ?= misc/siege/urls.txt SIEGE_URLS_FILE ?= misc/siege/urls.txt
SIEGE_CONCURRENCY ?= 100 SIEGE_CONCURRENCY ?= 50
SIEGE_DURATION ?= 1M
data/bootstrap.d/dummy.yml: data/bootstrap.d/dummy.yml:
mkdir -p data/bootstrap.d mkdir -p data/bootstrap.d
@ -114,7 +115,7 @@ grafterm: tools/grafterm/bin/grafterm
siege: siege:
$(eval TMP := $(shell mktemp)) $(eval TMP := $(shell mktemp))
cat $(SIEGE_URLS_FILE) | envsubst > $(TMP) cat $(SIEGE_URLS_FILE) | envsubst > $(TMP)
siege -i -b -c $(SIEGE_CONCURRENCY) -f $(TMP) siege -R ./misc/siege/siege.conf -i -b -c $(SIEGE_CONCURRENCY) -t $(SIEGE_DURATION) -f $(TMP)
rm -rf $(TMP) rm -rf $(TMP)
tools/gitea-release/bin/gitea-release.sh: tools/gitea-release/bin/gitea-release.sh:
@ -131,7 +132,7 @@ tools/grafterm/bin/grafterm:
GOBIN=$(PWD)/tools/grafterm/bin go install github.com/slok/grafterm/cmd/grafterm@v0.2.0 GOBIN=$(PWD)/tools/grafterm/bin go install github.com/slok/grafterm/cmd/grafterm@v0.2.0
bench: bench:
go test -bench=. -run '^$$' ./internal/bench go test -bench=. -run '^$$' -benchtime=10s ./internal/bench
tools/benchstat/bin/benchstat: tools/benchstat/bin/benchstat:
mkdir -p tools/benchstat/bin mkdir -p tools/benchstat/bin
@ -150,7 +151,7 @@ run-redis:
-v $(PWD)/data/redis:/data \ -v $(PWD)/data/redis:/data \
-p 6379:6379 \ -p 6379:6379 \
redis:alpine3.17 \ redis:alpine3.17 \
redis-server --save 60 1 --loglevel warning redis-server --save 60 1 --loglevel debug
redis-shell: redis-shell:
docker exec -it \ docker exec -it \

View File

@ -24,6 +24,7 @@
- [(FR) - Ajouter une authentification OpenID Connect](./fr/tutorials/add-oidc-authn-layer.md) - [(FR) - Ajouter une authentification OpenID Connect](./fr/tutorials/add-oidc-authn-layer.md)
- [(FR) - Amorçage d'un serveur Bouncer via la configuration](./fr/tutorials/bootstrapping.md) - [(FR) - Amorçage d'un serveur Bouncer via la configuration](./fr/tutorials/bootstrapping.md)
- [(FR) - Intégration avec Kubernetes](./fr/tutorials/kubernetes-integration.md) - [(FR) - Intégration avec Kubernetes](./fr/tutorials/kubernetes-integration.md)
- [(FR) - Profilage](./fr/tutorials/profiling.md)
### Développement ### Développement

View File

@ -27,8 +27,8 @@ Bouncer utilise le projet [`expr`](https://expr-lang.org/) comme DSL. En plus de
Le comportement des règles par défaut est le suivant: Le comportement des règles par défaut est le suivant:
1. L'ensemble des entêtes HTTP correspondant au patron `Remote-*` sont supprimés ; 1. L'ensemble des entêtes HTTP correspondant au patron `Remote-*` sont supprimés ;
2. L'identifiant de l'utilisateur identifié (`user.subject`) est exporté sous la forme de l'entête HTTP `Remote-User` ; 2. L'identifiant de l'utilisateur identifié (`vars.user.subject`) est exporté sous la forme de l'entête HTTP `Remote-User` ;
3. L'ensemble des attributs de l'utilisateur identifié (`user.attrs`) sont exportés sous la forme `Remote-User-Attr-<name>``<name>` est le nom de l'attribut en minuscule, avec les `_` transformés en `-`. 3. L'ensemble des attributs de l'utilisateur identifié (`vars.user.attrs`) sont exportés sous la forme `Remote-User-Attr-<name>``<name>` est le nom de l'attribut en minuscule, avec les `_` transformés en `-`.
### Fonctions ### Fonctions
@ -36,25 +36,25 @@ Le comportement des règles par défaut est le suivant:
Interdire l'accès à l'utilisateur. Interdire l'accès à l'utilisateur.
##### `add_header(name string, value string)` ##### `add_header(ctx, name string, value string)`
Ajouter une valeur à un entête HTTP via son nom `name` et sa valeur `value`. Ajouter une valeur à un entête HTTP via son nom `name` et sa valeur `value`.
##### `set_header(name string, value string)` ##### `set_header(ctx, name string, value string)`
Définir la valeur d'un entête HTTP via son nom `name` et sa valeur `value`. La valeur précédente est écrasée. Définir la valeur d'un entête HTTP via son nom `name` et sa valeur `value`. La valeur précédente est écrasée.
##### `del_headers(pattern string)` ##### `del_headers(ctx, pattern string)`
Supprimer un ou plusieurs entêtes HTTP dont le nom correspond au patron `pattern`. Supprimer un ou plusieurs entêtes HTTP dont le nom correspond au patron `pattern`.
Le patron est défini par une chaîne comprenant un ou plusieurs caractères `*`, signifiant un ou plusieurs caractères arbitraires. Le patron est défini par une chaîne comprenant un ou plusieurs caractères `*`, signifiant un ou plusieurs caractères arbitraires.
##### `set_host(host string)` ##### `set_host(ctx, host string)`
Modifier la valeur de l'entête `Host` de la requête. Modifier la valeur de l'entête `Host` de la requête.
##### `set_url(url string)` ##### `set_url(ctx, url string)`
Modifier l'URL du serveur cible. Modifier l'URL du serveur cible.
@ -62,7 +62,7 @@ Modifier l'URL du serveur cible.
Les règles ont accès aux variables suivantes pendant leur exécution. Les règles ont accès aux variables suivantes pendant leur exécution.
#### `user` #### `vars.user`
L'utilisateur identifié par le layer. L'utilisateur identifié par le layer.

View File

@ -14,12 +14,12 @@ Les options disponibles pour le layer sont décrites via un [schéma JSON](https
En plus de ces options spécifiques le layer peut également être configuré via [les options communes aux layers `authn-*`](../../../../../internal/proxy/director/layer/authn/layer-options.json). En plus de ces options spécifiques le layer peut également être configuré via [les options communes aux layers `authn-*`](../../../../../internal/proxy/director/layer/authn/layer-options.json).
## Objet `user` et attributs ## Objet `vars.user` et attributs
L'objet `user` exposé au moteur de règles sera construit de la manière suivante: L'objet `user` exposé au moteur de règles sera construit de la manière suivante:
- `user.subject` sera initialisé avec le nom d'utilisateur identifié ; - `vars.user.subject` sera initialisé avec le nom d'utilisateur identifié ;
- `user.attrs` sera composé des attributs associés à l'utilisation (voir les options). - `vars.user.attrs` sera composé des attributs associés à l'utilisation (voir les options).
## Métriques ## Métriques

View File

@ -14,12 +14,12 @@ Les options disponibles pour le layer sont décrites via un [schéma JSON](https
En plus de ces options spécifiques le layer peut également être configuré via [les options communes aux layers `authn-*`](../../../../../internal/proxy/director/layer/authn/layer-options.json). En plus de ces options spécifiques le layer peut également être configuré via [les options communes aux layers `authn-*`](../../../../../internal/proxy/director/layer/authn/layer-options.json).
## Objet `user` et attributs ## Objet `vars.user` et attributs
L'objet `user` exposé au moteur de règles sera construit de la manière suivante: L'objet `vars.user` exposé au moteur de règles sera construit de la manière suivante:
- `user.subject` sera initialisé avec le couple `<remote_address>:<remote_port>` ; - `vars.user.subject` sera initialisé avec le couple `<remote_address>:<remote_port>` ;
- `user.attrs` sera vide. - `vars.user.attrs` sera vide.
## Métriques ## Métriques

View File

@ -16,18 +16,18 @@ Les options disponibles pour le layer sont décrites via un [schéma JSON](https
En plus de ces options spécifiques le layer peut également être configuré via [les options communes aux layers `authn-*`](../../../../../internal/proxy/director/layer/authn/layer-options.json). En plus de ces options spécifiques le layer peut également être configuré via [les options communes aux layers `authn-*`](../../../../../internal/proxy/director/layer/authn/layer-options.json).
## Objet `user` et attributs ## Objet `vars.user` et attributs
L'objet `user` exposé au moteur de règles sera construit de la manière suivante: L'objet `vars.user` exposé au moteur de règles sera construit de la manière suivante:
- `user.subject` sera initialisé avec la valeur du [claim](https://openid.net/specs/openid-connect-core-1_0.html#Claims) `sub` extrait de l'`idToken` récupéré lors de l'authentification ; - `vars.user.subject` sera initialisé avec la valeur du [claim](https://openid.net/specs/openid-connect-core-1_0.html#Claims) `sub` extrait de l'`idToken` récupéré lors de l'authentification ;
- `user.attrs` comportera les propriétés suivantes: - `vars.user.attrs` comportera les propriétés suivantes:
- L'ensemble des `claims` provenant de l'`idToken` seront transposés en `claim_<name>` (ex: `idToken.iss` sera transposé en `user.attrs.claim_iss`) ; - L'ensemble des `claims` provenant de l'`idToken` seront transposés en `claim_<name>` (ex: `idToken.iss` sera transposé en `vars.user.attrs.claim_iss`) ;
- `user.attrs.access_token`: le jeton d'accès associé à l'authentification ; - `vars.user.attrs.access_token`: le jeton d'accès associé à l'authentification ;
- `user.attrs.refresh_token`: le jeton de rafraîchissement associé à l'authentification (si disponible, en fonction des `scopes` demandés par le client) ; - `vars.user.attrs.refresh_token`: le jeton de rafraîchissement associé à l'authentification (si disponible, en fonction des `scopes` demandés par le client) ;
- `user.attrs.token_expiry`: Horodatage Unix (en secondes) associé à la date d'expiration du jeton d'accès ; - `vars.user.attrs.token_expiry`: Horodatage Unix (en secondes) associé à la date d'expiration du jeton d'accès ;
- `user.attrs.logout_url`: URL de déconnexion pour la suppression de la session Bouncer. - `vars.user.attrs.logout_url`: URL de déconnexion pour la suppression de la session Bouncer.
**Attention** Cette URL ne permet dans la plupart des cas que de supprimer la session côté Bouncer. La suppression de la session côté fournisseur d'identité est conditionné à la présence ou non de l'attribut [`end_session_endpoint`](https://openid.net/specs/openid-connect-session-1_0-17.html#OPMetadata) dans les données du point d'entrée de découverte de service (`.wellknown/openid-configuration`). **Attention** Cette URL ne permet dans la plupart des cas que de supprimer la session côté Bouncer. La suppression de la session côté fournisseur d'identité est conditionné à la présence ou non de l'attribut [`end_session_endpoint`](https://openid.net/specs/openid-connect-session-1_0-17.html#OPMetadata) dans les données du point d'entrée de découverte de service (`.wellknown/openid-configuration`).

View File

@ -24,15 +24,15 @@ Bouncer utilise le projet [`expr`](https://expr-lang.org/) comme DSL. En plus de
#### Communes #### Communes
##### `add_header(name string, value string)` ##### `add_header(ctx, name string, value string)`
Ajouter une valeur à un entête HTTP via son nom `name` et sa valeur `value`. Ajouter une valeur à un entête HTTP via son nom `name` et sa valeur `value`.
##### `set_header(name string, value string)` ##### `set_header(ctx, name string, value string)`
Définir la valeur d'un entête HTTP via son nom `name` et sa valeur `value`. La valeur précédente est écrasée. Définir la valeur d'un entête HTTP via son nom `name` et sa valeur `value`. La valeur précédente est écrasée.
##### `del_headers(pattern string)` ##### `del_headers(ctx, pattern string)`
Supprimer un ou plusieurs entêtes HTTP dont le nom correspond au patron `pattern`. Supprimer un ou plusieurs entêtes HTTP dont le nom correspond au patron `pattern`.
@ -40,11 +40,11 @@ Le patron est défini par une chaîne comprenant un ou plusieurs caractères `*`
#### Requête #### Requête
##### `set_host(host string)` ##### `set_host(ctx, host string)`
Modifier la valeur de l'entête `Host` de la requête. Modifier la valeur de l'entête `Host` de la requête.
##### `set_url(url string)` ##### `set_url(ctx, url string)`
Modifier l'URL du serveur cible. Modifier l'URL du serveur cible.
@ -58,7 +58,28 @@ Les règles ont accès aux variables suivantes pendant leur exécution. **Ces do
#### Requête #### Requête
##### `request` ##### `vars.original_url`
L'URL originale, avant réécriture du `Host` par Bouncer.
```js
{
scheme: "string", // Schéma HTTP de l'URL
opaque: "string", // Données opaque de l'URL
user: { // Identifiants d'URL (Basic Auth)
username: "",
password: ""
},
host: "string", // Nom d'hôte (<domaine>:<port>) de l'URL
path: "string", // Chemin de l'URL (format assaini)
rawPath: "string", // Chemin de l'URL (format brut)
raw_query: "string", // Variables d'URL (format brut)
fragment : "string", // Fragment d'URL (format assaini)
raw_fragment : "string" // Fragment d'URL (format brut)
}
```
##### `vars.request`
La requête en cours de traitement. La requête en cours de traitement.
@ -67,61 +88,65 @@ La requête en cours de traitement.
method: "string", // Méthode HTTP method: "string", // Méthode HTTP
host: "string", // Nom d'hôte (`Host`) associé à la requête host: "string", // Nom d'hôte (`Host`) associé à la requête
url: { // URL associée à la requête sous sa forme structurée url: { // URL associée à la requête sous sa forme structurée
"scheme": "string", // Schéma HTTP de l'URL scheme: "string", // Schéma HTTP de l'URL
"opaque": "string", // Données opaque de l'URL opaque: "string", // Données opaque de l'URL
"user": { // Identifiants d'URL (Basic Auth) user: { // Identifiants d'URL (Basic Auth)
"username": "", username: "",
"password": "" password: ""
}, },
"host": "string", // Nom d'hôte (<domaine>:<port>) de l'URL host: "string", // Nom d'hôte (<domaine>:<port>) de l'URL
"path": "string", // Chemin de l'URL (format assaini) path: "string", // Chemin de l'URL (format assaini)
"rawPath": "string", // Chemin de l'URL (format brut) rawPath: "string", // Chemin de l'URL (format brut)
"rawQuery": "string", // Variables d'URL (format brut) raw_query: "string", // Variables d'URL (format brut)
"fragment" : "string", // Fragment d'URL (format assaini) fragment : "string", // Fragment d'URL (format assaini)
"rawFragment" : "string" // Fragment d'URL (format brut) raw_fragment : "string" // Fragment d'URL (format brut)
}, },
rawUrl: "string", // URL associée à la requête (format assaini) raw_url: "string", // URL associée à la requête (format assaini)
proto: "string", // Numéro de version du protocole utilisé proto: "string", // Numéro de version du protocole utilisé
protoMajor: "int", // Numéro de version majeure du protocole utilisé proto_major: "int", // Numéro de version majeure du protocole utilisé
protoMinor: "int", // Numéro de version mineur du protocole utilisé proto_minor: "int", // Numéro de version mineur du protocole utilisé
header: { // Table associative des entêtes HTTP associés à la requête header: { // Table associative des entêtes HTTP associés à la requête
"string": ["string"] "string": ["string"]
}, },
contentLength: "int", // Taille du corps de la requête content_length: "int", // Taille du corps de la requête
transferEncoding: ["string"], // MIME-Type(s) d'encodage du corps de la requête transfer_encoding: ["string"], // MIME-Type(s) d'encodage du corps de la requête
trailer: { // Table associative des entêtes HTTP associés à la requête, transmises après le corps de la requête trailer: { // Table associative des entêtes HTTP associés à la requête, transmises après le corps de la requête
"string": ["string"] "string": ["string"]
}, },
remoteAddr: "string", // Adresse du client HTTP à l'origine de la requête remote_addr: "string", // Adresse du client HTTP à l'origine de la requête
requestUri: "string" // URL "brute" associée à la requêtes (avant opérations d'assainissement, utiliser "url" plutôt) request_uri: "string" // URL "brute" associée à la requêtes (avant opérations d'assainissement, utiliser "url" plutôt)
} }
``` ```
#### Réponse #### Réponse
##### `response` ##### `vars.response`
La réponse en cours de traitement. La réponse en cours de traitement.
```js ```js
{ {
statusCode: "int", // Code de statut de la réponse status_code: "int", // Code de statut de la réponse
status: "string", // Message associé au code de statut status: "string", // Message associé au code de statut
proto: "string", // Numéro de version du protocole utilisé proto: "string", // Numéro de version du protocole utilisé
protoMajor: "int", // Numéro de version majeure du protocole utilisé proto_major: "int", // Numéro de version majeure du protocole utilisé
protoMinor: "int", // Numéro de version mineur du protocole utilisé proto_minor: "int", // Numéro de version mineur du protocole utilisé
header: { // Table associative des entêtes HTTP associés à la requête header: { // Table associative des entêtes HTTP associés à la requête
"string": ["string"] "string": ["string"]
}, },
contentLength: "int", // Taille du corps de la réponse content_length: "int", // Taille du corps de la réponse
transferEncoding: ["string"], // MIME-Type(s) d'encodage du corps de la requête transfer_encoding: ["string"], // MIME-Type(s) d'encodage du corps de la requête
trailer: { // Table associative des entêtes HTTP associés à la requête, transmises après le corps de la requête trailer: { // Table associative des entêtes HTTP associés à la requête, transmises après le corps de la requête
"string": ["string"] "string": ["string"]
}, },
} }
``` ```
##### `request` ##### `vars.request`
_Voir section précédente._
##### `vars.original_url`
_Voir section précédente._ _Voir section précédente._

View File

@ -1,10 +1,24 @@
# Étudier les performances de Bouncer # Étudier les performances de Bouncer
## In situ
Il est possible d'activer via la configuration de Bouncer de endpoints capable de générer des fichiers de profil au format [`pprof`](https://github.com/google/pprof). Par défaut, le point d'entrée est `.bouncer/profiling` (l'activation et la personnalisation de ce point d'entrée sont modifiables via la [configuration](../../../misc/packaging/common/config.yml)).
**Exemple:** Visualiser l'utilisation mémoire de Bouncer
```bash
go tool pprof -web http://<bouncer_proxy>/.bouncer/profiling/heap
```
L'ensemble des profils disponibles sont visibles à l'adresse `http://<bouncer_proxy>/.bouncer/profiling`.
## En développement
Le package `./internal` est dédié à l'étude des performances de Bouncer. Il contient une suite de benchmarks simulant de proxies avec différentes configurations de layers afin d'évaluer les points d'engorgement sur le traitement des requêtes. Le package `./internal` est dédié à l'étude des performances de Bouncer. Il contient une suite de benchmarks simulant de proxies avec différentes configurations de layers afin d'évaluer les points d'engorgement sur le traitement des requêtes.
Voir le répertoire `./internal/bench/testdata/proxies` pour voir les différentes configurations de cas. Voir le répertoire `./internal/bench/testdata/proxies` pour voir les différentes configurations de cas.
## Lancer les benchmarks ### Lancer les benchmarks
Le plus simple est d'utiliser la commande `make bench` qui exécutera séquentiellement tous les benchmarks. Il est également possible de lancer un benchmark spécifique via la commande suivante: Le plus simple est d'utiliser la commande `make bench` qui exécutera séquentiellement tous les benchmarks. Il est également possible de lancer un benchmark spécifique via la commande suivante:
@ -19,7 +33,7 @@ Par exemple:
go test -bench='BenchmarkProxies/basic-auth' -run='^$' ./internal/bench go test -bench='BenchmarkProxies/basic-auth' -run='^$' ./internal/bench
``` ```
## Visualiser les profils d'exécution ### Visualiser les profils d'exécution
Vous pouvez visualiser les profils d'exécution via la commande suivante: Vous pouvez visualiser les profils d'exécution via la commande suivante:
@ -35,7 +49,7 @@ Par exemple:
go tool pprof -web ./internal/bench/testdata/proxies/basic-auth.prof go tool pprof -web ./internal/bench/testdata/proxies/basic-auth.prof
``` ```
## Comparer les évolutions ### Comparer les évolutions
```bash ```bash
# Lancer un premier benchmark # Lancer un premier benchmark

View File

@ -27,7 +27,7 @@ func (s *Server) initRepositories(ctx context.Context) error {
} }
func (s *Server) initRedisClient(ctx context.Context) error { func (s *Server) initRedisClient(ctx context.Context) error {
client := setup.NewRedisClient(ctx, s.redisConfig) client := setup.NewSharedClient(s.redisConfig)
s.redisClient = client s.redisClient = client

View File

@ -6,6 +6,7 @@ import (
"log" "log"
"net" "net"
"net/http" "net/http"
"net/http/pprof"
"forge.cadoles.com/cadoles/bouncer/internal/auth" "forge.cadoles.com/cadoles/bouncer/internal/auth"
"forge.cadoles.com/cadoles/bouncer/internal/auth/jwt" "forge.cadoles.com/cadoles/bouncer/internal/auth/jwt"
@ -155,6 +156,34 @@ func (s *Server) run(parentCtx context.Context, addrs chan net.Addr, errs chan e
}) })
} }
if s.serverConfig.Profiling.Enabled {
profiling := s.serverConfig.Profiling
logger.Info(ctx, "enabling profiling", logger.F("endpoint", profiling.Endpoint))
router.Group(func(r chi.Router) {
if profiling.BasicAuth != nil {
logger.Info(ctx, "enabling authentication on metrics endpoint")
r.Use(middleware.BasicAuth(
"profiling",
profiling.BasicAuth.CredentialsMap(),
))
}
r.Route(string(profiling.Endpoint), func(r chi.Router) {
r.HandleFunc("/", pprof.Index)
r.HandleFunc("/cmdline", pprof.Cmdline)
r.HandleFunc("/profile", pprof.Profile)
r.HandleFunc("/symbol", pprof.Symbol)
r.HandleFunc("/trace", pprof.Trace)
r.HandleFunc("/{name}", func(w http.ResponseWriter, r *http.Request) {
name := chi.URLParam(r, "name")
pprof.Handler(name).ServeHTTP(w, r)
})
})
})
}
router.Route("/api/v1", func(r chi.Router) { router.Route("/api/v1", func(r chi.Router) {
r.Group(func(r chi.Router) { r.Group(func(r chi.Router) {
r.Use(auth.Middleware( r.Use(auth.Middleware(

View File

@ -3,7 +3,6 @@ package proxy_test
import ( import (
"context" "context"
"io" "io"
"log"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/http/httputil" "net/http/httputil"
@ -24,6 +23,7 @@ import (
redisStore "forge.cadoles.com/cadoles/bouncer/internal/store/redis" redisStore "forge.cadoles.com/cadoles/bouncer/internal/store/redis"
"github.com/pkg/errors" "github.com/pkg/errors"
"github.com/redis/go-redis/v9" "github.com/redis/go-redis/v9"
"gitlab.com/wpetit/goweb/logger"
"gopkg.in/yaml.v3" "gopkg.in/yaml.v3"
"forge.cadoles.com/cadoles/bouncer/internal/setup" "forge.cadoles.com/cadoles/bouncer/internal/setup"
@ -39,6 +39,19 @@ func BenchmarkProxies(b *testing.B) {
name := strings.TrimSuffix(filepath.Base(f), filepath.Ext(f)) name := strings.TrimSuffix(filepath.Base(f), filepath.Ext(f))
b.Run(name, func(b *testing.B) { b.Run(name, func(b *testing.B) {
heap, err := os.Create(filepath.Join("testdata", "proxies", name+"_heap.prof"))
if err != nil {
b.Fatalf("%+v", errors.Wrapf(err, "could not create heap profile"))
}
defer func() {
defer heap.Close()
if err := pprof.WriteHeapProfile(heap); err != nil {
b.Fatalf("%+v", errors.WithStack(err))
}
}()
conf, err := loadProxyBenchConfig(f) conf, err := loadProxyBenchConfig(f)
if err != nil { if err != nil {
b.Fatalf("%+v", errors.Wrapf(err, "could notre load bench config")) b.Fatalf("%+v", errors.Wrapf(err, "could notre load bench config"))
@ -78,7 +91,7 @@ func BenchmarkProxies(b *testing.B) {
b.Logf("fetching url '%s'", rawProxyURL) b.Logf("fetching url '%s'", rawProxyURL)
profile, err := os.Create(filepath.Join("testdata", "proxies", name+".prof")) profile, err := os.Create(filepath.Join("testdata", "proxies", name+"_cpu.prof"))
if err != nil { if err != nil {
b.Fatalf("%+v", errors.Wrapf(err, "could not create cpu profile")) b.Fatalf("%+v", errors.Wrapf(err, "could not create cpu profile"))
} }
@ -86,7 +99,7 @@ func BenchmarkProxies(b *testing.B) {
defer profile.Close() defer profile.Close()
if err := pprof.StartCPUProfile(profile); err != nil { if err := pprof.StartCPUProfile(profile); err != nil {
log.Fatal(err) b.Fatalf("%+v", errors.WithStack(err))
} }
defer pprof.StopCPUProfile() defer pprof.StopCPUProfile()
@ -227,7 +240,12 @@ func createProxy(name string, conf *proxyBenchConfig, logf func(format string, a
} }
layers, err := setup.GetLayers(context.Background(), config.NewDefault()) appConf := config.NewDefault()
appConf.Logger.Level = config.InterpolatedInt(logger.LevelError)
appConf.Layers.Authn.TemplateDir = "../../layers/authn/templates"
appConf.Layers.Queue.TemplateDir = "../../layers/queue/templates"
layers, err := setup.GetLayers(context.Background(), appConf)
if err != nil { if err != nil {
return nil, nil, errors.WithStack(err) return nil, nil, errors.WithStack(err)
} }

View File

@ -12,7 +12,7 @@ proxy:
attributes: attributes:
email: foo@bar.com email: foo@bar.com
rules: rules:
- set_header("Remote-User-Attr-Email", user.attrs.email) - set_header(ctx, "Remote-User-Attr-Email", vars.user.attrs.email)
fetch: fetch:
url: url:
user: user:

View File

@ -0,0 +1,10 @@
proxy:
from: ["*"]
to: ""
layers:
queue:
type: queue
enabled: true
options:
capacity: 100
keepAlive: 10s

View File

@ -8,5 +8,5 @@ proxy:
options: options:
rules: rules:
request: request:
- set_host(request.url.host) - set_host(ctx, vars.request.url.host)
- set_header("X-Proxied-With", "bouncer") - set_header(ctx, "X-Proxied-With", "bouncer")

View File

@ -35,12 +35,15 @@ func RunCommand() *cli.Command {
logger.SetLevel(logger.Level(conf.Logger.Level)) logger.SetLevel(logger.Level(conf.Logger.Level))
projectVersion := ctx.String("projectVersion") projectVersion := ctx.String("projectVersion")
flushSentry, err := setup.SetupSentry(ctx.Context, conf.Admin.Sentry, projectVersion)
if err != nil {
return errors.Wrap(err, "could not initialize sentry client")
}
defer flushSentry() if conf.Proxy.Sentry.DSN != "" {
flushSentry, err := setup.SetupSentry(ctx.Context, conf.Proxy.Sentry, projectVersion)
if err != nil {
return errors.Wrap(err, "could not initialize sentry client")
}
defer flushSentry()
}
integrations, err := setup.SetupIntegrations(ctx.Context, conf) integrations, err := setup.SetupIntegrations(ctx.Context, conf)
if err != nil { if err != nil {

View File

@ -4,14 +4,14 @@
<h2>Incoming headers</h2> <h2>Incoming headers</h2>
<table style="width: 100%"> <table style="width: 100%">
<thead> <thead>
<tr> <tr style="text-align: left">
<th>Key</th> <th>Key</th>
<th>Value</th> <th>Value</th>
</tr> </tr>
</thead> </thead>
<tbody> <tbody>
{{ range $key, $val := .Request.Header }} {{ range $key, $val := .Request.Header }}
<tr> <tr style="text-align: left">
<td> <td>
<b>{{ $key }}</b> <b>{{ $key }}</b>
</td> </td>
@ -27,7 +27,7 @@
<h2>Incoming cookies</h2> <h2>Incoming cookies</h2>
<table style="width: 100%"> <table style="width: 100%">
<thead> <thead>
<tr> <tr style="text-align: left">
<th>Name</th> <th>Name</th>
<th>Domain</th> <th>Domain</th>
<th>Path</th> <th>Path</th>
@ -41,7 +41,7 @@
</thead> </thead>
<tbody> <tbody>
{{ range $cookie := .Request.Cookies }} {{ range $cookie := .Request.Cookies }}
<tr> <tr style="text-align: left">
<td> <td>
<b>{{ $cookie.Name }}</b> <b>{{ $cookie.Name }}</b>
</td> </td>

View File

@ -30,12 +30,15 @@ func RunCommand() *cli.Command {
logger.SetLevel(logger.Level(conf.Logger.Level)) logger.SetLevel(logger.Level(conf.Logger.Level))
projectVersion := ctx.String("projectVersion") projectVersion := ctx.String("projectVersion")
flushSentry, err := setup.SetupSentry(ctx.Context, conf.Proxy.Sentry, projectVersion)
if err != nil {
return errors.Wrap(err, "could not initialize sentry client")
}
defer flushSentry() if conf.Proxy.Sentry.DSN != "" {
flushSentry, err := setup.SetupSentry(ctx.Context, conf.Proxy.Sentry, projectVersion)
if err != nil {
return errors.Wrap(err, "could not initialize sentry client")
}
defer flushSentry()
}
layers, err := setup.GetLayers(ctx.Context, conf) layers, err := setup.GetLayers(ctx.Context, conf)
if err != nil { if err != nil {

View File

@ -1,20 +1,22 @@
package config package config
type AdminServerConfig struct { type AdminServerConfig struct {
HTTP HTTPConfig `yaml:"http"` HTTP HTTPConfig `yaml:"http"`
CORS CORSConfig `yaml:"cors"` CORS CORSConfig `yaml:"cors"`
Auth AuthConfig `yaml:"auth"` Auth AuthConfig `yaml:"auth"`
Metrics MetricsConfig `yaml:"metrics"` Metrics MetricsConfig `yaml:"metrics"`
Sentry SentryConfig `yaml:"sentry"` Profiling ProfilingConfig `yaml:"profiling"`
Sentry SentryConfig `yaml:"sentry"`
} }
func NewDefaultAdminServerConfig() AdminServerConfig { func NewDefaultAdminServerConfig() AdminServerConfig {
return AdminServerConfig{ return AdminServerConfig{
HTTP: NewHTTPConfig("127.0.0.1", 8081), HTTP: NewHTTPConfig("127.0.0.1", 8081),
CORS: NewDefaultCORSConfig(), CORS: NewDefaultCORSConfig(),
Auth: NewDefaultAuthConfig(), Auth: NewDefaultAuthConfig(),
Metrics: NewDefaultMetricsConfig(), Metrics: NewDefaultMetricsConfig(),
Sentry: NewDefaultSentryConfig(), Sentry: NewDefaultSentryConfig(),
Profiling: NewDefaultProfilingConfig(),
} }
} }

View File

@ -0,0 +1,15 @@
package config
type ProfilingConfig struct {
Enabled InterpolatedBool `yaml:"enabled"`
Endpoint InterpolatedString `yaml:"endpoint"`
BasicAuth *BasicAuthConfig `yaml:"basicAuth"`
}
func NewDefaultProfilingConfig() ProfilingConfig {
return ProfilingConfig{
Enabled: true,
Endpoint: "/.bouncer/profiling",
BasicAuth: nil,
}
}

View File

@ -10,6 +10,7 @@ type ProxyServerConfig struct {
Debug InterpolatedBool `yaml:"debug"` Debug InterpolatedBool `yaml:"debug"`
HTTP HTTPConfig `yaml:"http"` HTTP HTTPConfig `yaml:"http"`
Metrics MetricsConfig `yaml:"metrics"` Metrics MetricsConfig `yaml:"metrics"`
Profiling ProfilingConfig `yaml:"profiling"`
Transport TransportConfig `yaml:"transport"` Transport TransportConfig `yaml:"transport"`
Dial DialConfig `yaml:"dial"` Dial DialConfig `yaml:"dial"`
Sentry SentryConfig `yaml:"sentry"` Sentry SentryConfig `yaml:"sentry"`
@ -27,6 +28,7 @@ func NewDefaultProxyServerConfig() ProxyServerConfig {
Sentry: NewDefaultSentryConfig(), Sentry: NewDefaultSentryConfig(),
Cache: NewDefaultCacheConfig(), Cache: NewDefaultCacheConfig(),
Templates: NewDefaultTemplatesConfig(), Templates: NewDefaultTemplatesConfig(),
Profiling: NewDefaultProfilingConfig(),
} }
} }

View File

@ -15,6 +15,8 @@ type RedisConfig struct {
WriteTimeout InterpolatedDuration `yaml:"writeTimeout"` WriteTimeout InterpolatedDuration `yaml:"writeTimeout"`
DialTimeout InterpolatedDuration `yaml:"dialTimeout"` DialTimeout InterpolatedDuration `yaml:"dialTimeout"`
LockMaxRetries InterpolatedInt `yaml:"lockMaxRetries"` LockMaxRetries InterpolatedInt `yaml:"lockMaxRetries"`
MaxRetries InterpolatedInt `yaml:"maxRetries"`
PingInterval InterpolatedDuration `yaml:"pingInterval"`
} }
func NewDefaultRedisConfig() RedisConfig { func NewDefaultRedisConfig() RedisConfig {
@ -25,5 +27,7 @@ func NewDefaultRedisConfig() RedisConfig {
WriteTimeout: InterpolatedDuration(30 * time.Second), WriteTimeout: InterpolatedDuration(30 * time.Second),
DialTimeout: InterpolatedDuration(30 * time.Second), DialTimeout: InterpolatedDuration(30 * time.Second),
LockMaxRetries: 10, LockMaxRetries: 10,
MaxRetries: 3,
PingInterval: InterpolatedDuration(30 * time.Second),
} }
} }

View File

@ -28,10 +28,10 @@ func NewDefaultSentryConfig() SentryConfig {
Debug: false, Debug: false,
FlushTimeout: NewInterpolatedDuration(2 * time.Second), FlushTimeout: NewInterpolatedDuration(2 * time.Second),
AttachStacktrace: true, AttachStacktrace: true,
SampleRate: 1, SampleRate: 0.2,
EnableTracing: true, EnableTracing: true,
TracesSampleRate: 0.2, TracesSampleRate: 0.2,
ProfilesSampleRate: 1, ProfilesSampleRate: 0.2,
IgnoreErrors: []string{}, IgnoreErrors: []string{},
SendDefaultPII: false, SendDefaultPII: false,
ServerName: "", ServerName: "",

View File

@ -74,7 +74,7 @@ func (l *Layer) Middleware(layer *store.Layer) proxy.Middleware {
return return
} }
if err := l.applyRules(r, options, user); err != nil { if err := l.applyRules(ctx, r, options, user); err != nil {
if errors.Is(err, ErrForbidden) { if errors.Is(err, ErrForbidden) {
l.renderForbiddenPage(w, r, layer, options, user) l.renderForbiddenPage(w, r, layer, options, user)
return return

View File

@ -1,6 +1,7 @@
package authn package authn
import ( import (
"context"
"net/http" "net/http"
"forge.cadoles.com/cadoles/bouncer/internal/rule" "forge.cadoles.com/cadoles/bouncer/internal/rule"
@ -9,30 +10,32 @@ import (
"github.com/pkg/errors" "github.com/pkg/errors"
) )
type Env struct { type Vars struct {
User *User `expr:"user"` User *User `expr:"user"`
} }
func (l *Layer) applyRules(r *http.Request, options *LayerOptions, user *User) error { func (l *Layer) applyRules(ctx context.Context, r *http.Request, options *LayerOptions, user *User) error {
rules := options.Rules rules := options.Rules
if len(rules) == 0 { if len(rules) == 0 {
return nil return nil
} }
engine, err := rule.NewEngine[*Env]( engine, err := rule.NewEngine[*Vars](
rule.WithRules(options.Rules...), rule.WithRules(options.Rules...),
rule.WithExpr(getAuthnAPI()...), rule.WithExpr(getAuthnAPI()...),
ruleHTTP.WithRequestFuncs(r), ruleHTTP.WithRequestFuncs(),
) )
if err != nil { if err != nil {
return errors.WithStack(err) return errors.WithStack(err)
} }
env := &Env{ vars := &Vars{
User: user, User: user,
} }
if _, err := engine.Apply(env); err != nil { ctx = ruleHTTP.WithRequest(ctx, r)
if _, err := engine.Apply(ctx, vars); err != nil {
return errors.WithStack(err) return errors.WithStack(err)
} }

View File

@ -65,7 +65,7 @@ func (q *Queue) Middleware(layer *store.Layer) proxy.Middleware {
return return
} }
defer q.updateMetrics(ctx, layer.Proxy, layer.Name, options) defer q.updateMetrics(layer.Proxy, layer.Name, options)
cookieName := q.getCookieName(layer.Name) cookieName := q.getCookieName(layer.Name)
@ -217,7 +217,9 @@ func (q *Queue) refreshQueue(ctx context.Context, layerName store.LayerName, kee
} }
} }
func (q *Queue) updateMetrics(ctx context.Context, proxyName store.ProxyName, layerName store.LayerName, options *LayerOptions) { func (q *Queue) updateMetrics(proxyName store.ProxyName, layerName store.LayerName, options *LayerOptions) {
ctx := context.Background()
// Update queue capacity metric // Update queue capacity metric
metricQueueCapacity.With( metricQueueCapacity.With(
prometheus.Labels{ prometheus.Labels{

View File

@ -6,6 +6,9 @@ import (
proxy "forge.cadoles.com/Cadoles/go-proxy" proxy "forge.cadoles.com/Cadoles/go-proxy"
"forge.cadoles.com/Cadoles/go-proxy/wildcard" "forge.cadoles.com/Cadoles/go-proxy/wildcard"
"forge.cadoles.com/cadoles/bouncer/internal/proxy/director" "forge.cadoles.com/cadoles/bouncer/internal/proxy/director"
"forge.cadoles.com/cadoles/bouncer/internal/proxy/director/layer/util"
"forge.cadoles.com/cadoles/bouncer/internal/rule"
ruleHTTP "forge.cadoles.com/cadoles/bouncer/internal/rule/http"
"forge.cadoles.com/cadoles/bouncer/internal/store" "forge.cadoles.com/cadoles/bouncer/internal/store"
"github.com/pkg/errors" "github.com/pkg/errors"
"gitlab.com/wpetit/goweb/logger" "gitlab.com/wpetit/goweb/logger"
@ -13,7 +16,10 @@ import (
const LayerType store.LayerType = "rewriter" const LayerType store.LayerType = "rewriter"
type Layer struct{} type Layer struct {
requestRuleEngine *util.RevisionedRuleEngine[*RequestVars, *LayerOptions]
responseRuleEngine *util.RevisionedRuleEngine[*ResponseVars, *LayerOptions]
}
func (l *Layer) LayerType() store.LayerType { func (l *Layer) LayerType() store.LayerType {
return LayerType return LayerType
@ -39,7 +45,7 @@ func (l *Layer) Middleware(layer *store.Layer) proxy.Middleware {
return return
} }
if err := l.applyRequestRules(r, options); err != nil { if err := l.applyRequestRules(ctx, r, layer.Revision, options); err != nil {
logger.Error(ctx, "could not apply request rules", logger.E(errors.WithStack(err))) logger.Error(ctx, "could not apply request rules", logger.E(errors.WithStack(err)))
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError) http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
@ -66,7 +72,9 @@ func (l *Layer) ResponseTransformer(layer *store.Layer) proxy.ResponseTransforme
return nil return nil
} }
if err := l.applyResponseRules(r, options); err != nil { ctx := r.Request.Context()
if err := l.applyResponseRules(ctx, r, layer.Revision, options); err != nil {
return errors.WithStack(err) return errors.WithStack(err)
} }
@ -75,7 +83,30 @@ func (l *Layer) ResponseTransformer(layer *store.Layer) proxy.ResponseTransforme
} }
func New(funcs ...OptionFunc) *Layer { func New(funcs ...OptionFunc) *Layer {
return &Layer{} return &Layer{
requestRuleEngine: util.NewRevisionedRuleEngine(func(options *LayerOptions) (*rule.Engine[*RequestVars], error) {
engine, err := rule.NewEngine[*RequestVars](
rule.WithRules(options.Rules.Request...),
ruleHTTP.WithRequestFuncs(),
)
if err != nil {
return nil, errors.WithStack(err)
}
return engine, nil
}),
responseRuleEngine: util.NewRevisionedRuleEngine(func(options *LayerOptions) (*rule.Engine[*ResponseVars], error) {
engine, err := rule.NewEngine[*ResponseVars](
rule.WithRules(options.Rules.Response...),
ruleHTTP.WithResponseFuncs(),
)
if err != nil {
return nil, errors.WithStack(err)
}
return engine, nil
}),
}
} }
var ( var (

View File

@ -1,68 +1,93 @@
package rewriter package rewriter
import ( import (
"context"
"net/http" "net/http"
"forge.cadoles.com/cadoles/bouncer/internal/proxy/director"
"forge.cadoles.com/cadoles/bouncer/internal/rule" "forge.cadoles.com/cadoles/bouncer/internal/rule"
ruleHTTP "forge.cadoles.com/cadoles/bouncer/internal/rule/http" ruleHTTP "forge.cadoles.com/cadoles/bouncer/internal/rule/http"
"github.com/pkg/errors" "github.com/pkg/errors"
) )
type RequestEnv struct { type RequestVars struct {
Request RequestInfo `expr:"request"` Request RequestVar `expr:"request"`
OriginalURL URLVar `expr:"original_url"`
} }
type URLEnv struct { type URLVar struct {
Scheme string `expr:"scheme"` Scheme string `expr:"scheme"`
Opaque string `expr:"opaque"` Opaque string `expr:"opaque"`
User UserInfoEnv `expr:"user"` User UserVar `expr:"user"`
Host string `expr:"host"` Host string `expr:"host"`
Path string `expr:"path"` Path string `expr:"path"`
RawPath string `expr:"rawPath"` RawPath string `expr:"raw_path"`
RawQuery string `expr:"rawQuery"` RawQuery string `expr:"raw_query"`
Fragment string `expr:"fragment"` Fragment string `expr:"fragment"`
RawFragment string `expr:"rawFragment"` RawFragment string `expr:"raw_fragment"`
} }
type UserInfoEnv struct { type UserVar struct {
Username string `expr:"username"` Username string `expr:"username"`
Password string `expr:"password"` Password string `expr:"password"`
} }
type RequestInfo struct { type RequestVar struct {
Method string `expr:"method"` Method string `expr:"method"`
URL URLEnv `expr:"url"` URL URLVar `expr:"url"`
RawURL string `expr:"rawUrl"` RawURL string `expr:"raw_url"`
Proto string `expr:"proto"` Proto string `expr:"proto"`
ProtoMajor int `expr:"protoMajor"` ProtoMajor int `expr:"proto_major"`
ProtoMinor int `expr:"protoMinor"` ProtoMinor int `expr:"proto_minor"`
Header map[string][]string `expr:"header"` Header map[string][]string `expr:"header"`
ContentLength int64 `expr:"contentLength"` ContentLength int64 `expr:"content_length"`
TransferEncoding []string `expr:"transferEncoding"` TransferEncoding []string `expr:"transfer_encoding"`
Host string `expr:"host"` Host string `expr:"host"`
Trailer map[string][]string `expr:"trailer"` Trailer map[string][]string `expr:"trailer"`
RemoteAddr string `expr:"remoteAddr"` RemoteAddr string `expr:"remote_addr"`
RequestURI string `expr:"requestUri"` RequestURI string `expr:"request_uri"`
} }
func (l *Layer) applyRequestRules(r *http.Request, options *LayerOptions) error { func (l *Layer) applyRequestRules(ctx context.Context, r *http.Request, layerRevision int, options *LayerOptions) error {
rules := options.Rules.Request rules := options.Rules.Request
if len(rules) == 0 { if len(rules) == 0 {
return nil return nil
} }
engine, err := l.getRequestRuleEngine(r, options) engine, err := l.getRequestRuleEngine(ctx, layerRevision, options)
if err != nil { if err != nil {
return errors.WithStack(err) return errors.WithStack(err)
} }
env := &RequestEnv{ originalURL, err := director.OriginalURL(ctx)
Request: RequestInfo{ if err != nil {
return errors.WithStack(err)
}
vars := &RequestVars{
OriginalURL: URLVar{
Scheme: originalURL.Scheme,
Opaque: originalURL.Opaque,
User: UserVar{
Username: originalURL.User.Username(),
Password: func() string {
passwd, _ := originalURL.User.Password()
return passwd
}(),
},
Host: originalURL.Host,
Path: originalURL.Path,
RawPath: originalURL.RawPath,
RawQuery: originalURL.RawQuery,
Fragment: originalURL.Fragment,
RawFragment: originalURL.RawFragment,
},
Request: RequestVar{
Method: r.Method, Method: r.Method,
URL: URLEnv{ URL: URLVar{
Scheme: r.URL.Scheme, Scheme: r.URL.Scheme,
Opaque: r.URL.Opaque, Opaque: r.URL.Opaque,
User: UserInfoEnv{ User: UserVar{
Username: r.URL.User.Username(), Username: r.URL.User.Username(),
Password: func() string { Password: func() string {
passwd, _ := r.URL.User.Password() passwd, _ := r.URL.User.Password()
@ -90,18 +115,17 @@ func (l *Layer) applyRequestRules(r *http.Request, options *LayerOptions) error
}, },
} }
if _, err := engine.Apply(env); err != nil { ctx = ruleHTTP.WithRequest(ctx, r)
if _, err := engine.Apply(ctx, vars); err != nil {
return errors.WithStack(err) return errors.WithStack(err)
} }
return nil return nil
} }
func (l *Layer) getRequestRuleEngine(r *http.Request, options *LayerOptions) (*rule.Engine[*RequestEnv], error) { func (l *Layer) getRequestRuleEngine(ctx context.Context, layerRevision int, options *LayerOptions) (*rule.Engine[*RequestVars], error) {
engine, err := rule.NewEngine[*RequestEnv]( engine, err := l.requestRuleEngine.Get(ctx, layerRevision, options)
rule.WithRules(options.Rules.Request...),
ruleHTTP.WithRequestFuncs(r),
)
if err != nil { if err != nil {
return nil, errors.WithStack(err) return nil, errors.WithStack(err)
} }
@ -109,42 +133,65 @@ func (l *Layer) getRequestRuleEngine(r *http.Request, options *LayerOptions) (*r
return engine, nil return engine, nil
} }
type ResponseEnv struct { type ResponseVars struct {
Request RequestInfo `expr:"request"` OriginalURL URLVar `expr:"original_url"`
Response ResponseInfo `expr:"response"` Request RequestVar `expr:"request"`
Response ResponseVar `expr:"response"`
} }
type ResponseInfo struct { type ResponseVar struct {
Status string `expr:"status"` Status string `expr:"status"`
StatusCode int `expr:"statusCode"` StatusCode int `expr:"status_code"`
Proto string `expr:"proto"` Proto string `expr:"proto"`
ProtoMajor int `expr:"protoMajor"` ProtoMajor int `expr:"proto_major"`
ProtoMinor int `expr:"protoMinor"` ProtoMinor int `expr:"proto_minor"`
Header map[string][]string `expr:"header"` Header map[string][]string `expr:"header"`
ContentLength int64 `expr:"contentLength"` ContentLength int64 `expr:"content_length"`
TransferEncoding []string `expr:"transferEncoding"` TransferEncoding []string `expr:"transfer_encoding"`
Uncompressed bool `expr:"uncompressed"` Uncompressed bool `expr:"uncompressed"`
Trailer map[string][]string `expr:"trailer"` Trailer map[string][]string `expr:"trailer"`
} }
func (l *Layer) applyResponseRules(r *http.Response, options *LayerOptions) error { func (l *Layer) applyResponseRules(ctx context.Context, r *http.Response, layerRevision int, options *LayerOptions) error {
rules := options.Rules.Response rules := options.Rules.Response
if len(rules) == 0 { if len(rules) == 0 {
return nil return nil
} }
engine, err := l.getResponseRuleEngine(r, options) engine, err := l.getResponseRuleEngine(ctx, layerRevision, options)
if err != nil { if err != nil {
return errors.WithStack(err) return errors.WithStack(err)
} }
env := &ResponseEnv{ originalURL, err := director.OriginalURL(ctx)
Request: RequestInfo{ if err != nil {
return errors.WithStack(err)
}
vars := &ResponseVars{
OriginalURL: URLVar{
Scheme: originalURL.Scheme,
Opaque: originalURL.Opaque,
User: UserVar{
Username: originalURL.User.Username(),
Password: func() string {
passwd, _ := originalURL.User.Password()
return passwd
}(),
},
Host: originalURL.Host,
Path: originalURL.Path,
RawPath: originalURL.RawPath,
RawQuery: originalURL.RawQuery,
Fragment: originalURL.Fragment,
RawFragment: originalURL.RawFragment,
},
Request: RequestVar{
Method: r.Request.Method, Method: r.Request.Method,
URL: URLEnv{ URL: URLVar{
Scheme: r.Request.URL.Scheme, Scheme: r.Request.URL.Scheme,
Opaque: r.Request.URL.Opaque, Opaque: r.Request.URL.Opaque,
User: UserInfoEnv{ User: UserVar{
Username: r.Request.URL.User.Username(), Username: r.Request.URL.User.Username(),
Password: func() string { Password: func() string {
passwd, _ := r.Request.URL.User.Password() passwd, _ := r.Request.URL.User.Password()
@ -170,7 +217,7 @@ func (l *Layer) applyResponseRules(r *http.Response, options *LayerOptions) erro
RemoteAddr: r.Request.RemoteAddr, RemoteAddr: r.Request.RemoteAddr,
RequestURI: r.Request.RequestURI, RequestURI: r.Request.RequestURI,
}, },
Response: ResponseInfo{ Response: ResponseVar{
Proto: r.Proto, Proto: r.Proto,
ProtoMajor: r.ProtoMajor, ProtoMajor: r.ProtoMajor,
ProtoMinor: r.ProtoMinor, ProtoMinor: r.ProtoMinor,
@ -183,18 +230,17 @@ func (l *Layer) applyResponseRules(r *http.Response, options *LayerOptions) erro
}, },
} }
if _, err := engine.Apply(env); err != nil { ctx = ruleHTTP.WithResponse(ctx, r)
if _, err := engine.Apply(ctx, vars); err != nil {
return errors.WithStack(err) return errors.WithStack(err)
} }
return nil return nil
} }
func (l *Layer) getResponseRuleEngine(r *http.Response, options *LayerOptions) (*rule.Engine[*ResponseEnv], error) { func (l *Layer) getResponseRuleEngine(ctx context.Context, layerRevision int, options *LayerOptions) (*rule.Engine[*ResponseVars], error) {
engine, err := rule.NewEngine[*ResponseEnv]( engine, err := l.responseRuleEngine.Get(ctx, layerRevision, options)
rule.WithRules(options.Rules.Response...),
ruleHTTP.WithResponseFuncs(r),
)
if err != nil { if err != nil {
return nil, errors.WithStack(err) return nil, errors.WithStack(err)
} }

View File

@ -0,0 +1,51 @@
package util
import (
"context"
"sync"
"forge.cadoles.com/cadoles/bouncer/internal/rule"
"github.com/pkg/errors"
"gitlab.com/wpetit/goweb/logger"
)
type RuleEngineFactoryFunc[V any, O any] func(ops O) (*rule.Engine[V], error)
type RevisionedRuleEngine[V any, O any] struct {
mutex sync.RWMutex
revision int
engine *rule.Engine[V]
factory RuleEngineFactoryFunc[V, O]
}
func (e *RevisionedRuleEngine[V, O]) Get(ctx context.Context, revision int, opts O) (*rule.Engine[V], error) {
e.mutex.RLock()
if revision == e.revision {
logger.Debug(ctx, "using cached rule engine", logger.F("layerRevision", revision))
defer e.mutex.RUnlock()
return e.engine, nil
}
e.mutex.RUnlock()
e.mutex.Lock()
defer e.mutex.Unlock()
logger.Debug(ctx, "creating rule engine", logger.F("layerRevision", revision))
engine, err := e.factory(opts)
if err != nil {
return nil, errors.WithStack(err)
}
e.engine = engine
e.revision = revision
return engine, nil
}
func NewRevisionedRuleEngine[V any, O any](factory RuleEngineFactoryFunc[V, O]) *RevisionedRuleEngine[V, O] {
return &RevisionedRuleEngine[V, O]{
factory: factory,
}
}

View File

@ -9,7 +9,7 @@ import (
) )
func (s *Server) initRepositories(ctx context.Context) error { func (s *Server) initRepositories(ctx context.Context) error {
client := setup.NewRedisClient(ctx, s.redisConfig) client := setup.NewSharedClient(s.redisConfig)
if err := s.initProxyRepository(ctx, client); err != nil { if err := s.initProxyRepository(ctx, client); err != nil {
return errors.WithStack(err) return errors.WithStack(err)

View File

@ -8,6 +8,7 @@ import (
"net" "net"
"net/http" "net/http"
"net/http/httputil" "net/http/httputil"
"net/http/pprof"
"net/url" "net/url"
"path/filepath" "path/filepath"
"strconv" "strconv"
@ -146,6 +147,34 @@ func (s *Server) run(parentCtx context.Context, addrs chan net.Addr, errs chan e
}) })
} }
if s.serverConfig.Profiling.Enabled {
profiling := s.serverConfig.Profiling
logger.Info(ctx, "enabling profiling", logger.F("endpoint", profiling.Endpoint))
router.Group(func(r chi.Router) {
if profiling.BasicAuth != nil {
logger.Info(ctx, "enabling authentication on metrics endpoint")
r.Use(middleware.BasicAuth(
"profiling",
profiling.BasicAuth.CredentialsMap(),
))
}
r.Route(string(profiling.Endpoint), func(r chi.Router) {
r.HandleFunc("/", pprof.Index)
r.HandleFunc("/cmdline", pprof.Cmdline)
r.HandleFunc("/profile", pprof.Profile)
r.HandleFunc("/symbol", pprof.Symbol)
r.HandleFunc("/trace", pprof.Trace)
r.HandleFunc("/{name}", func(w http.ResponseWriter, r *http.Request) {
name := chi.URLParam(r, "name")
pprof.Handler(name).ServeHTTP(w, r)
})
})
})
}
router.Group(func(r chi.Router) { router.Group(func(r chi.Router) {
r.Use(director.Middleware()) r.Use(director.Middleware())

View File

@ -1,16 +1,28 @@
package rule package rule
import ( import (
"context"
"github.com/expr-lang/expr" "github.com/expr-lang/expr"
"github.com/expr-lang/expr/vm" "github.com/expr-lang/expr/vm"
"github.com/pkg/errors" "github.com/pkg/errors"
) )
type Engine[E any] struct { type Engine[V any] struct {
rules []*vm.Program rules []*vm.Program
} }
func (e *Engine[E]) Apply(env E) ([]any, error) { func (e *Engine[V]) Apply(ctx context.Context, vars V) ([]any, error) {
type Env[V any] struct {
Context context.Context `expr:"ctx"`
Vars V `expr:"vars"`
}
env := Env[V]{
Context: ctx,
Vars: vars,
}
results := make([]any, 0, len(e.rules)) results := make([]any, 0, len(e.rules))
for i, r := range e.rules { for i, r := range e.rules {
result, err := expr.Run(r, env) result, err := expr.Run(r, env)
@ -42,3 +54,26 @@ func NewEngine[E any](funcs ...OptionFunc) (*Engine[E], error) {
return engine, nil return engine, nil
} }
func Context[T any](ctx context.Context, key any) (T, bool) {
raw := ctx.Value(key)
if raw == nil {
return *new(T), false
}
value, err := Assert[T](raw)
if err != nil {
return *new(T), false
}
return value, true
}
func Assert[T any](raw any) (T, error) {
value, ok := raw.(T)
if !ok {
return *new(T), errors.Errorf("unexpected value '%T'", value)
}
return value, nil
}

View File

@ -0,0 +1,31 @@
package http
import (
"context"
"net/http"
"forge.cadoles.com/cadoles/bouncer/internal/rule"
)
type contextKey string
const (
contextKeyRequest contextKey = "request"
contextKeyResponse contextKey = "response"
)
func WithRequest(ctx context.Context, r *http.Request) context.Context {
return context.WithValue(ctx, contextKeyRequest, r)
}
func WithResponse(ctx context.Context, r *http.Response) context.Context {
return context.WithValue(ctx, contextKeyResponse, r)
}
func ctxRequest(ctx context.Context) (*http.Request, bool) {
return rule.Context[*http.Request](ctx, contextKeyRequest)
}
func ctxResponse(ctx context.Context) (*http.Response, bool) {
return rule.Context[*http.Response](ctx, contextKeyResponse)
}

View File

@ -1,20 +1,18 @@
package http package http
import ( import (
"net/http"
"forge.cadoles.com/cadoles/bouncer/internal/rule" "forge.cadoles.com/cadoles/bouncer/internal/rule"
"github.com/expr-lang/expr" "github.com/expr-lang/expr"
) )
func WithRequestFuncs(r *http.Request) rule.OptionFunc { func WithRequestFuncs() rule.OptionFunc {
return func(opts *rule.Options) { return func(opts *rule.Options) {
funcs := []expr.Option{ funcs := []expr.Option{
setRequestURL(r), setRequestURLFunc(),
setRequestHeaderFunc(r), setRequestHeaderFunc(),
addRequestHeaderFunc(r), addRequestHeaderFunc(),
delRequestHeadersFunc(r), delRequestHeadersFunc(),
setRequestHostFunc(r), setRequestHostFunc(),
} }
if len(opts.Expr) == 0 { if len(opts.Expr) == 0 {
@ -25,12 +23,12 @@ func WithRequestFuncs(r *http.Request) rule.OptionFunc {
} }
} }
func WithResponseFuncs(r *http.Response) rule.OptionFunc { func WithResponseFuncs() rule.OptionFunc {
return func(opts *rule.Options) { return func(opts *rule.Options) {
funcs := []expr.Option{ funcs := []expr.Option{
setResponseHeaderFunc(r), setResponseHeaderFunc(),
addResponseHeaderFunc(r), addResponseHeaderFunc(),
delResponseHeadersFunc(r), delResponseHeadersFunc(),
} }
if len(opts.Expr) == 0 { if len(opts.Expr) == 0 {

View File

@ -1,109 +1,155 @@
package http package http
import ( import (
"context"
"fmt" "fmt"
"net/http"
"net/url" "net/url"
"strconv" "strconv"
"strings" "strings"
"time" "time"
"forge.cadoles.com/Cadoles/go-proxy/wildcard" "forge.cadoles.com/Cadoles/go-proxy/wildcard"
"forge.cadoles.com/cadoles/bouncer/internal/rule"
"github.com/expr-lang/expr" "github.com/expr-lang/expr"
"github.com/pkg/errors" "github.com/pkg/errors"
) )
func setRequestHostFunc(r *http.Request) expr.Option { func setRequestHostFunc() expr.Option {
return expr.Function( return expr.Function(
"set_host", "set_host",
func(params ...any) (any, error) { func(params ...any) (any, error) {
host := params[0].(string) ctx, err := rule.Assert[context.Context](params[0])
if err != nil {
return nil, errors.WithStack(err)
}
host, err := rule.Assert[string](params[1])
if err != nil {
return nil, errors.WithStack(err)
}
r, ok := ctxRequest(ctx)
if !ok {
return nil, errors.New("could not find http request in context")
}
r.Host = host r.Host = host
return true, nil return true, nil
}, },
new(func(string) bool), new(func(context.Context, string) bool),
) )
} }
func setRequestURL(r *http.Request) expr.Option { func setRequestURLFunc() expr.Option {
return expr.Function( return expr.Function(
"set_url", "set_url",
func(params ...any) (any, error) { func(params ...any) (any, error) {
rawURL := params[0].(string) ctx, err := rule.Assert[context.Context](params[0])
if err != nil {
return nil, errors.WithStack(err)
}
rawURL, err := rule.Assert[string](params[1])
if err != nil {
return nil, errors.WithStack(err)
}
url, err := url.Parse(rawURL) url, err := url.Parse(rawURL)
if err != nil { if err != nil {
return false, errors.WithStack(err) return false, errors.WithStack(err)
} }
r, ok := ctxRequest(ctx)
if !ok {
return nil, errors.New("could not find http request in context")
}
r.URL = url r.URL = url
return true, nil return true, nil
}, },
new(func(string) bool), new(func(context.Context, string) bool),
) )
} }
func addRequestHeaderFunc(r *http.Request) expr.Option { func addRequestHeaderFunc() expr.Option {
return expr.Function( return expr.Function(
"add_header", "add_header",
func(params ...any) (any, error) { func(params ...any) (any, error) {
name := params[0].(string) ctx, err := rule.Assert[context.Context](params[0])
rawValue := params[1] if err != nil {
return nil, errors.WithStack(err)
}
var value string name, err := rule.Assert[string](params[1])
switch v := rawValue.(type) { if err != nil {
case []string: return nil, errors.WithStack(err)
value = strings.Join(v, ",") }
case time.Time:
value = strconv.FormatInt(v.UTC().Unix(), 10) value := formatValue(params[2])
case time.Duration:
value = strconv.FormatInt(int64(v.Seconds()), 10) r, ok := ctxRequest(ctx)
default: if !ok {
value = fmt.Sprintf("%v", rawValue) return nil, errors.New("could not find http request in context")
} }
r.Header.Add(name, value) r.Header.Add(name, value)
return true, nil return true, nil
}, },
new(func(string, string) bool), new(func(context.Context, string, string) bool),
) )
} }
func setRequestHeaderFunc(r *http.Request) expr.Option { func setRequestHeaderFunc() expr.Option {
return expr.Function( return expr.Function(
"set_header", "set_header",
func(params ...any) (any, error) { func(params ...any) (any, error) {
name := params[0].(string) ctx, err := rule.Assert[context.Context](params[0])
rawValue := params[1] if err != nil {
return nil, errors.WithStack(err)
}
var value string name, err := rule.Assert[string](params[1])
switch v := rawValue.(type) { if err != nil {
case []string: return nil, errors.WithStack(err)
value = strings.Join(v, ",") }
case time.Time:
value = strconv.FormatInt(v.UTC().Unix(), 10) value := formatValue(params[2])
case time.Duration:
value = strconv.FormatInt(int64(v.Seconds()), 10) r, ok := ctxRequest(ctx)
default: if !ok {
value = fmt.Sprintf("%v", rawValue) return nil, errors.New("could not find http request in context")
} }
r.Header.Set(name, value) r.Header.Set(name, value)
return true, nil return true, nil
}, },
new(func(string, string) bool), new(func(context.Context, string, string) bool),
) )
} }
func delRequestHeadersFunc(r *http.Request) expr.Option { func delRequestHeadersFunc() expr.Option {
return expr.Function( return expr.Function(
"del_headers", "del_headers",
func(params ...any) (any, error) { func(params ...any) (any, error) {
pattern := params[0].(string) ctx, err := rule.Assert[context.Context](params[0])
if err != nil {
return nil, errors.WithStack(err)
}
pattern, err := rule.Assert[string](params[1])
if err != nil {
return nil, errors.WithStack(err)
}
r, ok := ctxRequest(ctx)
if !ok {
return nil, errors.New("could not find http request in context")
}
deleted := false deleted := false
for key := range r.Header { for key := range r.Header {
@ -117,6 +163,21 @@ func delRequestHeadersFunc(r *http.Request) expr.Option {
return deleted, nil return deleted, nil
}, },
new(func(string) bool), new(func(context.Context, string) bool),
) )
} }
func formatValue(v any) string {
var value string
switch v := v.(type) {
case []string:
value = strings.Join(v, ",")
case time.Time:
value = strconv.FormatInt(v.UTC().Unix(), 10)
case time.Duration:
value = strconv.FormatInt(int64(v.Seconds()), 10)
default:
value = fmt.Sprintf("%v", v)
}
return value
}

View File

@ -0,0 +1,195 @@
package http
import (
"context"
"net/http"
"testing"
"forge.cadoles.com/cadoles/bouncer/internal/rule"
"github.com/pkg/errors"
)
func TestSetRequestHost(t *testing.T) {
type Vars struct {
NewHost string `expr:"newHost"`
}
engine := createRuleEngine[Vars](t,
rule.WithExpr(setRequestHostFunc()),
rule.WithRules(
"set_host(ctx, vars.newHost)",
),
)
req, err := http.NewRequest("GET", "http://example.net", nil)
if err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
ctx := context.Background()
ctx = WithRequest(ctx, req)
vars := Vars{
NewHost: "foobar",
}
if _, err := engine.Apply(ctx, vars); err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
if e, g := vars.NewHost, req.Host; e != g {
t.Errorf("req.Host: expected '%v', got '%v'", e, g)
}
}
func TestSetRequestURL(t *testing.T) {
type Vars struct {
NewURL string `expr:"newURL"`
}
engine := createRuleEngine[Vars](t,
rule.WithExpr(setRequestURLFunc()),
rule.WithRules(
"set_url(ctx, vars.newURL)",
),
)
req, err := http.NewRequest("GET", "http://example.net", nil)
if err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
ctx := context.Background()
ctx = WithRequest(ctx, req)
vars := Vars{
NewURL: "http://localhost",
}
if _, err := engine.Apply(ctx, vars); err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
if e, g := vars.NewURL, req.URL.String(); e != g {
t.Errorf("req.URL.String(): expected '%v', got '%v'", e, g)
}
}
func TestAddRequestHeader(t *testing.T) {
type Vars struct {
NewHeaderKey string `expr:"newHeaderKey"`
NewHeaderValue string `expr:"newHeaderValue"`
}
engine := createRuleEngine[Vars](t,
rule.WithExpr(addRequestHeaderFunc()),
rule.WithRules(
"add_header(ctx, vars.newHeaderKey, vars.newHeaderValue)",
),
)
req, err := http.NewRequest("GET", "http://example.net", nil)
if err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
ctx := context.Background()
ctx = WithRequest(ctx, req)
vars := Vars{
NewHeaderKey: "X-My-Header",
NewHeaderValue: "foobar",
}
if _, err := engine.Apply(ctx, vars); err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
if e, g := vars.NewHeaderValue, req.Header.Get(vars.NewHeaderKey); e != g {
t.Errorf("req.Header.Get(vars.NewHeaderKey): expected '%v', got '%v'", e, g)
}
}
func TestSetRequestHeader(t *testing.T) {
type Vars struct {
HeaderKey string `expr:"headerKey"`
HeaderValue string `expr:"headerValue"`
}
engine := createRuleEngine[Vars](t,
rule.WithExpr(setRequestHeaderFunc()),
rule.WithRules(
"set_header(ctx, vars.headerKey, vars.headerValue)",
),
)
req, err := http.NewRequest("GET", "http://example.net", nil)
if err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
vars := Vars{
HeaderKey: "X-My-Header",
HeaderValue: "foobar",
}
req.Header.Set(vars.HeaderKey, "test")
ctx := context.Background()
ctx = WithRequest(ctx, req)
if _, err := engine.Apply(ctx, vars); err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
if e, g := vars.HeaderValue, req.Header.Get(vars.HeaderKey); e != g {
t.Errorf("req.Header.Get(vars.HeaderKey): expected '%v', got '%v'", e, g)
}
}
func TestDelRequestHeaders(t *testing.T) {
type Vars struct {
HeaderPattern string `expr:"headerPattern"`
}
engine := createRuleEngine[Vars](t,
rule.WithExpr(delRequestHeadersFunc()),
rule.WithRules(
"del_headers(ctx, vars.headerPattern)",
),
)
req, err := http.NewRequest("GET", "http://example.net", nil)
if err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
vars := Vars{
HeaderPattern: "X-My-*",
}
req.Header.Set("X-My-Header", "test")
ctx := context.Background()
ctx = WithRequest(ctx, req)
if _, err := engine.Apply(ctx, vars); err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
if val := req.Header.Get("X-My-Header"); val != "" {
t.Errorf("req.Header.Get(\"X-My-Header\") should be empty, got '%v'", val)
}
}
func createRuleEngine[V any](t *testing.T, funcs ...rule.OptionFunc) *rule.Engine[V] {
engine, err := rule.NewEngine[V](funcs...)
if err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
return engine
}

View File

@ -1,22 +1,33 @@
package http package http
import ( import (
"context"
"fmt" "fmt"
"net/http"
"strconv" "strconv"
"strings" "strings"
"time" "time"
"forge.cadoles.com/Cadoles/go-proxy/wildcard" "forge.cadoles.com/Cadoles/go-proxy/wildcard"
"forge.cadoles.com/cadoles/bouncer/internal/rule"
"github.com/expr-lang/expr" "github.com/expr-lang/expr"
"github.com/pkg/errors"
) )
func addResponseHeaderFunc(r *http.Response) expr.Option { func addResponseHeaderFunc() expr.Option {
return expr.Function( return expr.Function(
"add_header", "add_header",
func(params ...any) (any, error) { func(params ...any) (any, error) {
name := params[0].(string) ctx, err := rule.Assert[context.Context](params[0])
rawValue := params[1] if err != nil {
return nil, errors.WithStack(err)
}
name, err := rule.Assert[string](params[1])
if err != nil {
return nil, errors.WithStack(err)
}
rawValue := params[2]
var value string var value string
switch v := rawValue.(type) { switch v := rawValue.(type) {
@ -30,20 +41,34 @@ func addResponseHeaderFunc(r *http.Response) expr.Option {
value = fmt.Sprintf("%v", rawValue) value = fmt.Sprintf("%v", rawValue)
} }
r, ok := ctxResponse(ctx)
if !ok {
return nil, errors.New("could not find http response in context")
}
r.Header.Add(name, value) r.Header.Add(name, value)
return true, nil return true, nil
}, },
new(func(string, string) bool), new(func(context.Context, string, string) bool),
) )
} }
func setResponseHeaderFunc(r *http.Response) expr.Option { func setResponseHeaderFunc() expr.Option {
return expr.Function( return expr.Function(
"set_header", "set_header",
func(params ...any) (any, error) { func(params ...any) (any, error) {
name := params[0].(string) ctx, err := rule.Assert[context.Context](params[0])
rawValue := params[1] if err != nil {
return nil, errors.WithStack(err)
}
name, err := rule.Assert[string](params[1])
if err != nil {
return nil, errors.WithStack(err)
}
rawValue := params[2]
var value string var value string
switch v := rawValue.(type) { switch v := rawValue.(type) {
@ -57,19 +82,38 @@ func setResponseHeaderFunc(r *http.Response) expr.Option {
value = fmt.Sprintf("%v", rawValue) value = fmt.Sprintf("%v", rawValue)
} }
r, ok := ctxResponse(ctx)
if !ok {
return nil, errors.New("could not find http response in context")
}
r.Header.Set(name, value) r.Header.Set(name, value)
return true, nil return true, nil
}, },
new(func(string, string) bool), new(func(context.Context, string, string) bool),
) )
} }
func delResponseHeadersFunc(r *http.Response) expr.Option { func delResponseHeadersFunc() expr.Option {
return expr.Function( return expr.Function(
"del_headers", "del_headers",
func(params ...any) (any, error) { func(params ...any) (any, error) {
pattern := params[0].(string) ctx, err := rule.Assert[context.Context](params[0])
if err != nil {
return nil, errors.WithStack(err)
}
pattern, err := rule.Assert[string](params[1])
if err != nil {
return nil, errors.WithStack(err)
}
r, ok := ctxResponse(ctx)
if !ok {
return nil, errors.New("could not find http response in context")
}
deleted := false deleted := false
for key := range r.Header { for key := range r.Header {
@ -83,6 +127,6 @@ func delResponseHeadersFunc(r *http.Response) expr.Option {
return deleted, nil return deleted, nil
}, },
new(func(string) bool), new(func(context.Context, string) bool),
) )
} }

View File

@ -0,0 +1,139 @@
package http
import (
"context"
"io"
"net/http"
"testing"
"forge.cadoles.com/cadoles/bouncer/internal/rule"
"github.com/pkg/errors"
)
func TestAddResponseHeader(t *testing.T) {
type Vars struct {
NewHeaderKey string `expr:"newHeaderKey"`
NewHeaderValue string `expr:"newHeaderValue"`
}
engine := createRuleEngine[Vars](t,
rule.WithExpr(addResponseHeaderFunc()),
rule.WithRules(
"add_header(ctx, vars.newHeaderKey, vars.newHeaderValue)",
),
)
req, err := http.NewRequest("GET", "http://example.net", nil)
if err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
resp := createResponse(req, http.StatusOK, nil)
ctx := context.Background()
ctx = WithResponse(ctx, resp)
vars := Vars{
NewHeaderKey: "X-My-Header",
NewHeaderValue: "foobar",
}
if _, err := engine.Apply(ctx, vars); err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
if e, g := vars.NewHeaderValue, resp.Header.Get(vars.NewHeaderKey); e != g {
t.Errorf("resp.Header.Get(vars.NewHeaderKey): expected '%v', got '%v'", e, g)
}
}
func TestResponseSetHeader(t *testing.T) {
type Vars struct {
HeaderKey string `expr:"headerKey"`
HeaderValue string `expr:"headerValue"`
}
engine := createRuleEngine[Vars](t,
rule.WithExpr(setResponseHeaderFunc()),
rule.WithRules(
"set_header(ctx, vars.headerKey, vars.headerValue)",
),
)
req, err := http.NewRequest("GET", "http://example.net", nil)
if err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
resp := createResponse(req, http.StatusOK, nil)
vars := Vars{
HeaderKey: "X-My-Header",
HeaderValue: "foobar",
}
resp.Header.Set(vars.HeaderKey, "test")
ctx := context.Background()
ctx = WithResponse(ctx, resp)
if _, err := engine.Apply(ctx, vars); err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
if e, g := vars.HeaderValue, resp.Header.Get(vars.HeaderKey); e != g {
t.Errorf("resp.Header.Get(vars.HeaderKey): expected '%v', got '%v'", e, g)
}
}
func TestResponseDelHeaders(t *testing.T) {
type Vars struct {
HeaderPattern string `expr:"headerPattern"`
}
engine := createRuleEngine[Vars](t,
rule.WithExpr(delResponseHeadersFunc()),
rule.WithRules(
"del_headers(ctx, vars.headerPattern)",
),
)
req, err := http.NewRequest("GET", "http://example.net", nil)
if err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
resp := createResponse(req, http.StatusOK, nil)
vars := Vars{
HeaderPattern: "X-My-*",
}
resp.Header.Set("X-My-Header", "test")
ctx := context.Background()
ctx = WithResponse(ctx, resp)
if _, err := engine.Apply(ctx, vars); err != nil {
t.Fatalf("%+v", errors.WithStack(err))
}
if val := resp.Header.Get("X-My-Header"); val != "" {
t.Errorf("resp.Header.Get(\"X-My-Header\") should be empty, got '%v'", val)
}
}
func createResponse(req *http.Request, statusCode int, body io.Reader) *http.Response {
return &http.Response{
Status: http.StatusText(statusCode),
StatusCode: statusCode,
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Body: io.NopCloser(body),
ContentLength: -1,
Request: req,
Header: make(http.Header, 0),
}
}

View File

@ -23,7 +23,7 @@ func init() {
} }
func setupAuthnOIDCLayer(conf *config.Config) (director.Layer, error) { func setupAuthnOIDCLayer(conf *config.Config) (director.Layer, error) {
rdb := newRedisClient(conf.Redis) rdb := NewSharedClient(conf.Redis)
adapter := redis.NewStoreAdapter(rdb) adapter := redis.NewStoreAdapter(rdb)
store := session.NewStore(adapter) store := session.NewStore(adapter)

View File

@ -27,7 +27,7 @@ func SetupIntegrations(ctx context.Context, conf *config.Config) ([]integration.
} }
func setupKubernetesIntegration(ctx context.Context, conf *config.Config) (*kubernetes.Integration, error) { func setupKubernetesIntegration(ctx context.Context, conf *config.Config) (*kubernetes.Integration, error) {
client := newRedisClient(conf.Redis) client := NewSharedClient(conf.Redis)
locker := redis.NewLocker(client, 10) locker := redis.NewLocker(client, 10)
integration := kubernetes.NewIntegration( integration := kubernetes.NewIntegration(

View File

@ -9,7 +9,7 @@ import (
) )
func SetupLocker(ctx context.Context, conf *config.Config) (lock.Locker, error) { func SetupLocker(ctx context.Context, conf *config.Config) (lock.Locker, error) {
client := newRedisClient(conf.Redis) client := NewSharedClient(conf.Redis)
locker := redis.NewLocker(client, int(conf.Redis.LockMaxRetries)) locker := redis.NewLocker(client, int(conf.Redis.LockMaxRetries))
return locker, nil return locker, nil
} }

View File

@ -3,19 +3,11 @@ package setup
import ( import (
"context" "context"
"forge.cadoles.com/cadoles/bouncer/internal/config"
"forge.cadoles.com/cadoles/bouncer/internal/store" "forge.cadoles.com/cadoles/bouncer/internal/store"
redisStore "forge.cadoles.com/cadoles/bouncer/internal/store/redis" redisStore "forge.cadoles.com/cadoles/bouncer/internal/store/redis"
"github.com/redis/go-redis/v9" "github.com/redis/go-redis/v9"
) )
func NewRedisClient(ctx context.Context, conf config.RedisConfig) redis.UniversalClient {
return redis.NewUniversalClient(&redis.UniversalOptions{
Addrs: conf.Adresses,
MasterName: string(conf.Master),
})
}
func NewProxyRepository(ctx context.Context, client redis.UniversalClient) (store.ProxyRepository, error) { func NewProxyRepository(ctx context.Context, client redis.UniversalClient) (store.ProxyRepository, error) {
return redisStore.NewProxyRepository(client, redisStore.DefaultTxMaxAttempts, redisStore.DefaultTxBaseDelay), nil return redisStore.NewProxyRepository(client, redisStore.DefaultTxMaxAttempts, redisStore.DefaultTxBaseDelay), nil
} }

View File

@ -35,6 +35,6 @@ func setupQueueLayer(conf *config.Config) (director.Layer, error) {
} }
func newQueueAdapter(redisConf config.RedisConfig) (queue.Adapter, error) { func newQueueAdapter(redisConf config.RedisConfig) (queue.Adapter, error) {
rdb := newRedisClient(redisConf) rdb := NewSharedClient(redisConf)
return queueRedis.NewAdapter(rdb, 2), nil return queueRedis.NewAdapter(rdb, 2), nil
} }

View File

@ -1,14 +1,38 @@
package setup package setup
import ( import (
"context"
"strings"
"sync"
"time" "time"
"forge.cadoles.com/cadoles/bouncer/internal/config" "forge.cadoles.com/cadoles/bouncer/internal/config"
"github.com/pkg/errors"
"github.com/redis/go-redis/v9" "github.com/redis/go-redis/v9"
"gitlab.com/wpetit/goweb/logger"
) )
var clients sync.Map
func NewSharedClient(conf config.RedisConfig) redis.UniversalClient {
key := strings.Join(conf.Adresses, "|") + "|" + string(conf.Master)
value, exists := clients.Load(key)
if exists {
if client, ok := (value).(redis.UniversalClient); ok {
return client
}
}
client := newRedisClient(conf)
clients.Store(key, client)
return client
}
func newRedisClient(conf config.RedisConfig) redis.UniversalClient { func newRedisClient(conf config.RedisConfig) redis.UniversalClient {
return redis.NewUniversalClient(&redis.UniversalOptions{ client := redis.NewUniversalClient(&redis.UniversalOptions{
Addrs: conf.Adresses, Addrs: conf.Adresses,
MasterName: string(conf.Master), MasterName: string(conf.Master),
ReadTimeout: time.Duration(conf.ReadTimeout), ReadTimeout: time.Duration(conf.ReadTimeout),
@ -16,5 +40,33 @@ func newRedisClient(conf config.RedisConfig) redis.UniversalClient {
DialTimeout: time.Duration(conf.DialTimeout), DialTimeout: time.Duration(conf.DialTimeout),
RouteByLatency: true, RouteByLatency: true,
ContextTimeoutEnabled: true, ContextTimeoutEnabled: true,
MaxRetries: int(conf.MaxRetries),
}) })
go func() {
ctx := logger.With(context.Background(),
logger.F("adresses", conf.Adresses),
logger.F("master", conf.Master),
)
timer := time.NewTicker(time.Duration(conf.PingInterval))
defer timer.Stop()
connected := true
for range timer.C {
if _, err := client.Ping(ctx).Result(); err != nil {
logger.Error(ctx, "redis disconnected", logger.E(errors.WithStack(err)))
connected = false
continue
}
if !connected {
logger.Info(ctx, "redis reconnected")
connected = true
}
}
}()
return client
} }

View File

@ -91,12 +91,14 @@ func WithRetry(ctx context.Context, client redis.UniversalClient, key string, fn
continue continue
} }
return err return errors.WithStack(err)
} }
return nil return nil
} }
logger.Error(ctx, "redis error", logger.E(errors.WithStack(err)))
return errors.WithStack(redis.TxFailedErr) return errors.WithStack(redis.TxFailedErr)
} }

View File

@ -49,6 +49,19 @@ admin:
# Mettre à null pour désactiver l'authentification # Mettre à null pour désactiver l'authentification
basicAuth: null basicAuth: null
# Profiling
profiling:
# Activer ou désactiver les endpoints de profiling
enabled: true
# Route de publication des endpoints de profiling
endpoint: /.bouncer/profiling
# Authentification "basic auth" sur les endpoints
# de profiling
# Mettre à null pour désactiver l'authentification
basicAuth:
credentials:
prof: iling
# Configuration de l'intégration Sentry # Configuration de l'intégration Sentry
# Voir https://pkg.go.dev/github.com/getsentry/sentry-go?utm_source=godoc#ClientOptions # Voir https://pkg.go.dev/github.com/getsentry/sentry-go?utm_source=godoc#ClientOptions
sentry: sentry:
@ -59,7 +72,7 @@ admin:
sampleRate: 1 sampleRate: 1
enableTracing: true enableTracing: true
tracesSampleRate: 0.2 tracesSampleRate: 0.2
profilesSampleRate: 1 profilesSampleRate: 0.2
ignoreErrors: [] ignoreErrors: []
sendDefaultPII: false sendDefaultPII: false
serverName: "" serverName: ""
@ -99,6 +112,19 @@ proxy:
credentials: credentials:
prom: etheus prom: etheus
# Profiling
profiling:
# Activer ou désactiver les endpoints de profiling
enabled: true
# Route de publication des endpoints de profiling
endpoint: /.bouncer/profiling
# Authentification "basic auth" sur les endpoints
# de profiling
# Mettre à null pour désactiver l'authentification
basicAuth:
credentials:
prof: iling
# Configuration de la mise en cache # Configuration de la mise en cache
# locale des données proxy/layers # locale des données proxy/layers
cache: cache:
@ -164,6 +190,8 @@ redis:
writeTimeout: 30s writeTimeout: 30s
readTimeout: 30s readTimeout: 30s
dialTimeout: 30s dialTimeout: 30s
maxRetries: 3
pingInterval: 30s
# Configuration des logs # Configuration des logs
logger: logger:

79
misc/siege/siege.conf Normal file
View File

@ -0,0 +1,79 @@
# Updated by Siege %_VERSION%, %_DATE%
# Copyright 2000-2016 by %_AUTHOR%
#
# Siege configuration file -- edit as necessary
# For more information about configuring and running this program,
# visit: http://www.joedog.org/
#
#
# Verbose mode: With this feature enabled, siege will print the
# result of each transaction to stdout. (Enabled by default)
#
# ex: verbose = true|false
#
verbose = true
#
# Color mode: This option works in conjunction with verbose mode.
# It tells siege whether or not it should display its output in
# color-coded output. (Enabled by default)
#
# ex: color = on | off
#
color = on
#
# Cache revalidation. Siege supports cache revalidation for both ETag
# and Last-modified headers. If a copy is still fresh, the server
# responds with 304. While this feature is required for HTTP/1.1, it
# may not be welcomed for load testing. We allow you to breach the
# protocol and turn off caching
#
# HTTP/1.1 200 0.00 secs: 2326 bytes ==> /apache_pb.gif
# HTTP/1.1 304 0.00 secs: 0 bytes ==> /apache_pb.gif
# HTTP/1.1 304 0.00 secs: 0 bytes ==> /apache_pb.gif
#
# Siege also supports Cache-control headers. Consider this server
# response: Cache-Control: max-age=3
# That tells siege to cache the file for three seconds. While it
# doesn't actually store the file, it will logically grab it from
# its cache. In verbose output, it designates a cached resource
# with (c):
#
# HTTP/1.1 200 0.25 secs: 159 bytes ==> GET /expires/
# HTTP/1.1 200 1.48 secs: 498419 bytes ==> GET /expires/Otter_in_Southwold.jpg
# HTTP/1.1 200 0.24 secs: 159 bytes ==> GET /expires/
# HTTP/1.1 200(C) 0.00 secs: 0 bytes ==> GET /expires/Otter_in_Southwold.jpg
#
# NOTE: with color enabled, cached URLs appear in green
#
# ex: cache = true
#
cache = true
#
# Cookie support: by default siege accepts cookies. This directive is
# available to disable that support. Set cookies to 'false' to refuse
# cookies. Set it to 'true' to accept them. The default value is true.
# If you want to maintain state with the server, then this MUST be set
# to true.
#
# ex: cookies = false
#
cookies = true
#
# Failures: This is the number of total connection failures allowed
# before siege aborts. Connection failures (timeouts, socket failures,
# etc.) are combined with 400 and 500 level errors in the final stats,
# but those errors do not count against the abort total. If you set
# this total to 10, then siege will abort after ten socket timeouts,
# but it will NOT abort after ten 404s. This is designed to prevent a
# run-away mess on an unattended siege.
#
# The default value is 1024
#
# ex: failures = 50
#
failures = -1