From 301f099622db033330d50c42d6e24f39b2adb5bd Mon Sep 17 00:00:00 2001 From: Mathias Petermann Date: Fri, 21 Aug 2020 09:56:46 +0200 Subject: [PATCH 1/7] Fix link for WES-entropy-client --- docs/production/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/production/index.rst b/docs/production/index.rst index 67e97dae..ccace703 100644 --- a/docs/production/index.rst +++ b/docs/production/index.rst @@ -49,7 +49,7 @@ The amount of effort you wish to expend ensuring that Lemur has good entropy to If you wish to generate more entropy for your system we would suggest you take a look at the following resources: -- `WES-entropy-client `_ +- `WES-entropy-client `_ - `haveged `_ For additional information about OpenSSL entropy issues: From 5d7ca8520a1480af31d14e7fbc8424ef9f504e76 Mon Sep 17 00:00:00 2001 From: Mathias Petermann Date: Wed, 16 Sep 2020 10:23:04 +0200 Subject: [PATCH 2/7] Add remark that the WES-entropy-client is now linked to a fork --- docs/production/index.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/production/index.rst b/docs/production/index.rst index ccace703..503cb581 100644 --- a/docs/production/index.rst +++ b/docs/production/index.rst @@ -52,6 +52,8 @@ If you wish to generate more entropy for your system we would suggest you take a - `WES-entropy-client `_ - `haveged `_ +The original *WES-entropy-client* repository by WhitewoodCrypto was removed, the link now points to a fork of it. + For additional information about OpenSSL entropy issues: - `Managing and Understanding Entropy Usage `_ From 6dc4b9877bf11f90e94c80fad0843abfbea7ea4a Mon Sep 17 00:00:00 2001 From: "dependabot-preview[bot]" <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Tue, 22 Sep 2020 06:11:00 +0000 Subject: [PATCH 3/7] Bump boto3 from 1.14.61 to 1.15.2 Bumps [boto3](https://github.com/boto/boto3) from 1.14.61 to 1.15.2. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/boto/boto3/compare/1.14.61...1.15.2) Signed-off-by: dependabot-preview[bot] --- requirements-docs.txt | 6 +++--- requirements-tests.txt | 5 ++--- requirements.txt | 5 ++--- 3 files changed, 7 insertions(+), 9 deletions(-) diff --git a/requirements-docs.txt b/requirements-docs.txt index f3f417bf..5c6fdf92 100644 --- a/requirements-docs.txt +++ b/requirements-docs.txt @@ -17,8 +17,8 @@ bcrypt==3.1.7 # via -r requirements.txt, flask-bcrypt, paramiko beautifulsoup4==4.9.1 # via -r requirements.txt, cloudflare billiard==3.6.3.0 # via -r requirements.txt, celery blinker==1.4 # via -r requirements.txt, flask-mail, flask-principal, raven -boto3==1.14.61 # via -r requirements.txt -botocore==1.17.61 # via -r requirements.txt, boto3, s3transfer +boto3==1.15.2 # via -r requirements.txt +botocore==1.18.2 # via -r requirements.txt, boto3, s3transfer celery[redis]==4.4.2 # via -r requirements.txt certifi==2020.6.20 # via -r requirements.txt, requests certsrv==2.1.1 # via -r requirements.txt @@ -29,7 +29,7 @@ cloudflare==2.8.13 # via -r requirements.txt cryptography==3.1 # via -r requirements.txt, acme, josepy, paramiko, pyopenssl, requests dnspython3==1.15.0 # via -r requirements.txt dnspython==1.15.0 # via -r requirements.txt, dnspython3 -docutils==0.15.2 # via -r requirements.txt, botocore, sphinx +docutils==0.15.2 # via sphinx dyn==1.8.1 # via -r requirements.txt flask-bcrypt==0.7.1 # via -r requirements.txt flask-cors==3.0.9 # via -r requirements.txt diff --git a/requirements-tests.txt b/requirements-tests.txt index 20453852..b2b51cd7 100644 --- a/requirements-tests.txt +++ b/requirements-tests.txt @@ -10,9 +10,9 @@ aws-sam-translator==1.22.0 # via cfn-lint aws-xray-sdk==2.5.0 # via moto bandit==1.6.2 # via -r requirements-tests.in black==20.8b1 # via -r requirements-tests.in -boto3==1.14.61 # via aws-sam-translator, moto +boto3==1.15.2 # via aws-sam-translator, moto boto==2.49.0 # via moto -botocore==1.17.61 # via aws-xray-sdk, boto3, moto, s3transfer +botocore==1.18.2 # via aws-xray-sdk, boto3, moto, s3transfer certifi==2020.6.20 # via requests cffi==1.14.0 # via cryptography cfn-lint==0.29.5 # via moto @@ -22,7 +22,6 @@ coverage==5.3 # via -r requirements-tests.in cryptography==3.1 # via moto, python-jose, sshpubkeys decorator==4.4.2 # via networkx docker==4.2.0 # via moto -docutils==0.15.2 # via botocore ecdsa==0.14.1 # via moto, python-jose, sshpubkeys factory-boy==3.0.1 # via -r requirements-tests.in faker==4.1.3 # via -r requirements-tests.in, factory-boy diff --git a/requirements.txt b/requirements.txt index 27a37a8c..cab2a8ed 100644 --- a/requirements.txt +++ b/requirements.txt @@ -15,8 +15,8 @@ bcrypt==3.1.7 # via flask-bcrypt, paramiko beautifulsoup4==4.9.1 # via cloudflare billiard==3.6.3.0 # via celery blinker==1.4 # via flask-mail, flask-principal, raven -boto3==1.14.61 # via -r requirements.in -botocore==1.17.61 # via -r requirements.in, boto3, s3transfer +boto3==1.15.2 # via -r requirements.in +botocore==1.18.2 # via -r requirements.in, boto3, s3transfer celery[redis]==4.4.2 # via -r requirements.in certifi==2020.6.20 # via -r requirements.in, requests certsrv==2.1.1 # via -r requirements.in @@ -27,7 +27,6 @@ cloudflare==2.8.13 # via -r requirements.in cryptography==3.1 # via -r requirements.in, acme, josepy, paramiko, pyopenssl, requests dnspython3==1.15.0 # via -r requirements.in dnspython==1.15.0 # via dnspython3 -docutils==0.15.2 # via botocore dyn==1.8.1 # via -r requirements.in flask-bcrypt==0.7.1 # via -r requirements.in flask-cors==3.0.9 # via -r requirements.in From 9f66c18e71641ca4070770a801af7ec4beb04f8b Mon Sep 17 00:00:00 2001 From: Mathias Petermann Date: Tue, 22 Sep 2020 14:48:40 +0200 Subject: [PATCH 4/7] Add REDIS_HOST and REDIS_PORT to celery configuration documentation --- docs/production/index.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/production/index.rst b/docs/production/index.rst index 67e97dae..1fdd5dde 100644 --- a/docs/production/index.rst +++ b/docs/production/index.rst @@ -390,6 +390,9 @@ Here are the Celery configuration variables that should be set:: CELERY_IMPORTS = ('lemur.common.celery') CELERY_TIMEZONE = 'UTC' + REDIS_HOST="your_redis_url" + REDIS_PORT="6379" + Do not forget to import crontab module in your configuration file:: from celery.task.schedules import crontab From f97e880fa662b308d43d562c729ac6c56ba634a0 Mon Sep 17 00:00:00 2001 From: Mathias Petermann Date: Wed, 23 Sep 2020 11:06:11 +0200 Subject: [PATCH 5/7] REDIS_PORT as integer, add hint about multiple redis databases --- docs/production/index.rst | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/production/index.rst b/docs/production/index.rst index ade54c48..55752a95 100644 --- a/docs/production/index.rst +++ b/docs/production/index.rst @@ -388,12 +388,16 @@ To enable celery support, you must also have configuration values that tell Cele Here are the Celery configuration variables that should be set:: CELERY_RESULT_BACKEND = 'redis://your_redis_url:6379' - CELERY_BROKER_URL = 'redis://your_redis_url:6379' + CELERY_BROKER_URL = 'redis://your_redis_url:6379/0' CELERY_IMPORTS = ('lemur.common.celery') CELERY_TIMEZONE = 'UTC' REDIS_HOST="your_redis_url" - REDIS_PORT="6379" + REDIS_PORT=6379 + REDIS_DB=0 + +Out of the box, every Redis instance supports 16 databases. The default database (`REDIS_DB`) is set to 0, however, you can use any of the databases from 0-15. Via `redis.conf` more databases can be supported. +In the `redis://` url, the database number needs to be added with a slash after the port. Do not forget to import crontab module in your configuration file:: From cad04885a09ff232ad49ed9ac96b2498070151bb Mon Sep 17 00:00:00 2001 From: Mathias Petermann Date: Wed, 23 Sep 2020 13:17:28 +0200 Subject: [PATCH 6/7] Add celery configuration options, to config documentation --- docs/administration.rst | 48 +++++++++++++++++++++++++++++++++++++++ docs/production/index.rst | 1 + 2 files changed, 49 insertions(+) diff --git a/docs/administration.rst b/docs/administration.rst index a3225fc2..0cec16a0 100644 --- a/docs/administration.rst +++ b/docs/administration.rst @@ -328,6 +328,54 @@ Lemur supports sending certification expiration notifications through SES and SM LEMUR_SECURITY_TEAM_EMAIL_INTERVALS = [15, 2] +Celery Options +--------------- +To make use of automated tasks within lemur (e.g. syncing source/destinations, or reissuing ACME certificates), you +need to configure celery. See :ref:`Periodic Tasks ` for more in depth documentation. + +.. data:: CELERY_RESULT_BACKEND + :noindex: + + The url to your redis backend (needs to be in the format `redis://:/`) + +.. data:: CELERY_BROKER_URL + :noindex: + + The url to your redis broker (needs to be in the format `redis://:/`) + +.. data:: CELERY_IMPORTS + :noindex: + + The module that celery needs to import, in our case thats `lemur.common.celery` + +.. data:: CELERY_TIMEZONE + :noindex: + + The timezone for celery to work with + + +.. data:: CELERYBEAT_SCHEDULE + :noindex: + + This defines the schedule, with which the celery beat makes the worker run the specified tasks. + +Since the celery module, relies on the RedisHandler, the following options also need to be set. + +.. data:: REDIS_HOST + :noindex: + + Hostname of your redis instance + +.. data:: REDIS_PORT + :noindex: + + Port on which redis is running (default: 6379) + +.. data:: REDIS_DB + :noindex: + + Which redis database to be used, by default redis offers databases 0-15 (default: 0) + Authentication Options ---------------------- Lemur currently supports Basic Authentication, LDAP Authentication, Ping OAuth2, and Google out of the box. Additional flows can be added relatively easily. diff --git a/docs/production/index.rst b/docs/production/index.rst index 55752a95..e4dd2e84 100644 --- a/docs/production/index.rst +++ b/docs/production/index.rst @@ -315,6 +315,7 @@ It will start a shell from which you can start/stop/restart the service. You can read all errors that might occur from /tmp/lemur.log. +.. _PeriodicTasks: Periodic Tasks ============== From d5557c1533dd94a11c3edfed31562949c87a414e Mon Sep 17 00:00:00 2001 From: Hossein Shafagh Date: Wed, 23 Sep 2020 09:58:28 -0700 Subject: [PATCH 7/7] Update index.rst adding insight about the default db --- docs/production/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/production/index.rst b/docs/production/index.rst index e4dd2e84..9f90c0cc 100644 --- a/docs/production/index.rst +++ b/docs/production/index.rst @@ -398,7 +398,7 @@ Here are the Celery configuration variables that should be set:: REDIS_DB=0 Out of the box, every Redis instance supports 16 databases. The default database (`REDIS_DB`) is set to 0, however, you can use any of the databases from 0-15. Via `redis.conf` more databases can be supported. -In the `redis://` url, the database number needs to be added with a slash after the port. +In the `redis://` url, the database number can be added with a slash after the port. (defaults to 0, if omitted) Do not forget to import crontab module in your configuration file::