Infra/lemur
5
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge pull request #3425 from jtschladen/fix-docs

Browse Source 
Debug docs
This commit is contained in:
Jasmine Schladen 2021-02-17 15:26:31 -08:00 committed by GitHub
commit 2fb595046f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
13 changed files with 201 additions and 85 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
CHANGELOG.rst
View File

@ -2,7 +2,7 @@ Changelog
=========
0.8.0 - `2020-11-13`
~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~
This release comes after more than two years and contains many interesting new features and improvements.
In addition to multiple new plugins, such as ACME-http01, ADCS, PowerDNS, UltraDNS, Entrust, SNS, many of Lemur's existing
@ -84,7 +84,7 @@ Upgrading
0.7 - `2018-05-07`
~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~
This release adds LetsEncrypt support with DNS providers Dyn, Route53, and Cloudflare, and expands on the pending certificate functionality.
The linux_dst plugin will also be deprecated and removed.
@ -121,8 +121,7 @@ Happy Holidays! This is a big release with lots of bug fixes and features. Below
Features:
* Per-certificate rotation policies, requires a database migration. The default rotation policy for all certificates.
is 30 days. Every certificate will gain a policy regardless of if auto-rotation is used.
* Per-certificate rotation policies, requires a database migration. The default rotation policy for all certificates is 30 days. Every certificate will gain a policy regardless of if auto-rotation is used.
* Adds per-user API Keys, allows users to issue multiple long-lived API tokens with the same permission as the user creating them.
* Adds the ability to revoke certificates from the Lemur UI/API, this is currently only supported for the digicert CIS and cfssl plugins.
* Allow destinations to support an export function. Useful for file system destinations e.g. S3 to specify the export plugin you wish to run before being sent to the destination.
@ -166,13 +165,9 @@ Big thanks to neilschelly for quite a lot of improvements to the `lemur-cryptogr
Other Highlights:
* Closed `#501 <https://github.com/Netflix/lemur/issues/501>`_ - Endpoint resource as now kept in sync via an
expiration mechanism. Such that non-existant endpoints gracefully fall out of Lemur. Certificates are never
removed from Lemur.
* Closed `#551 <https://github.com/Netflix/lemur/pull/551>`_ - Added the ability to create a 4096 bit key during certificate
creation. Closed `#528 <https://github.com/Netflix/lemur/pull/528>`_ to ensure that issuer plugins supported the new 4096 bit keys.
* Closed `#566 <https://github.com/Netflix/lemur/issues/566>`_ - Fixed an issue changing the notification status for certificates
without private keys.
* Closed `#501 <https://github.com/Netflix/lemur/issues/501>`_ - Endpoint resource as now kept in sync via an expiration mechanism. Such that non-existant endpoints gracefully fall out of Lemur. Certificates are never removed from Lemur.
* Closed `#551 <https://github.com/Netflix/lemur/pull/551>`_ - Added the ability to create a 4096 bit key during certificate creation. Closed `#528 <https://github.com/Netflix/lemur/pull/528>`_ to ensure that issuer plugins supported the new 4096 bit keys.
* Closed `#566 <https://github.com/Netflix/lemur/issues/566>`_ - Fixed an issue changing the notification status for certificates without private keys.
* Closed `#594 <https://github.com/Netflix/lemur/issues/594>`_ - Added `replaced` field indicating if a certificate has been superseded.
* Closed `#602 <https://github.com/Netflix/lemur/issues/602>`_ - AWS plugin added support for ALBs for endpoint tracking.
@ -196,12 +191,8 @@ Upgrading
There have been quite a few issues closed in this release. Some notables:
* Closed `#284 <https://github.com/Netflix/lemur/issues/284>`_ - Created new models for `Endpoints` created associated
AWS ELB endpoint tracking code. This was the major stated goal of this milestone and should serve as the basis for
future enhancements of Lemur's certificate 'deployment' capabilities.
* Closed `#334 <https://github.com/Netflix/lemur/issues/334>`_ - Lemur not has the ability
to restrict certificate expiration dates to weekdays.
* Closed `#284 <https://github.com/Netflix/lemur/issues/284>`_ - Created new models for `Endpoints` created associated AWS ELB endpoint tracking code. This was the major stated goal of this milestone and should serve as the basis for future enhancements of Lemur's certificate 'deployment' capabilities.
* Closed `#334 <https://github.com/Netflix/lemur/issues/334>`_ - Lemur not has the ability to restrict certificate expiration dates to weekdays.
Several fixes/tweaks to Lemurs python3 support (thanks chadhendrie!)

16
docs/administration.rst
View File

@ -78,13 +78,13 @@ Basic Configuration
The default connection pool size is 5 for sqlalchemy managed connections. Depending on the number of Lemur instances,
please specify per instance connection pool size. Below is an example to set connection pool size to 10.
::
::
SQLALCHEMY_POOL_SIZE = 10
.. warning::
This is an optional setting but important to review and set for optimal database connection usage and for overall database performance.
This is an optional setting but important to review and set for optimal database connection usage and for overall database performance.
.. data:: SQLALCHEMY_MAX_OVERFLOW
:noindex:
@ -99,7 +99,7 @@ This is an optional setting but important to review and set for optimal database
.. note::
Specifying the `SQLALCHEMY_MAX_OVERFLOW` to 0 will enforce limit to not create connections above specified pool size.
Specifying the `SQLALCHEMY_MAX_OVERFLOW` to 0 will enforce limit to not create connections above specified pool size.
.. data:: LEMUR_ALLOW_WEEKEND_EXPIRATION
@ -174,6 +174,7 @@ Specifying the `SQLALCHEMY_MAX_OVERFLOW` to 0 will enforce limit to not create c
.. data:: PUBLIC_CA_MAX_VALIDITY_DAYS
:noindex:
Use this config to override the limit of 397 days of validity for certificates issued by CA/Browser compliant authorities.
The authorities with cab_compliant option set to true will use this config. The example below overrides the default validity
of 397 days and sets it to 365 days.
@ -185,6 +186,7 @@ Specifying the `SQLALCHEMY_MAX_OVERFLOW` to 0 will enforce limit to not create c
.. data:: DEFAULT_VALIDITY_DAYS
:noindex:
Use this config to override the default validity of 365 days for certificates offered through Lemur UI. Any CA which
is not CA/Browser Forum compliant will be using this value as default validity to be displayed on UI. Please
note that this config is used for cert issuance only through Lemur UI. The example below overrides the default validity
@ -904,10 +906,12 @@ Active Directory Certificate Services Plugin
.. data:: ADCS_START
:noindex:
Used in ADCS-Sourceplugin. Minimum id of the first certificate to be returned. ID is increased by one until ADCS_STOP. Missing cert-IDs are ignored
.. data:: ADCS_STOP
:noindex:
Used for ADCS-Sourceplugin. Maximum id of the certificates returned.
@ -1640,7 +1644,7 @@ Slack
AWS (Source)
----
------------
:Authors:
Kevin Glisson <kglisson@netflix.com>,
@ -1653,7 +1657,7 @@ AWS (Source)
AWS (Destination)
----
-----------------
:Authors:
Kevin Glisson <kglisson@netflix.com>,
@ -1666,7 +1670,7 @@ AWS (Destination)
AWS (SNS Notification)
-----
----------------------
:Authors:
Jasmine Schladen <jschladen@netflix.com>

12
docs/developer/plugins/index.rst
View File

@ -145,8 +145,7 @@ The `IssuerPlugin` doesn't have any options like Destination, Source, and Notifi
any fields you might need to submit a request to a third party. If there are additional options you need
in your plugin feel free to open an issue, or look into adding additional options to issuers yourself.
Asynchronous Certificates
^^^^^^^^^^^^^^^^^^^^^^^^^
**Asynchronous Certificates**
An issuer may take some time to actually issue a certificate for an order. In this case, a `PendingCertificate` is returned, which holds information to recreate a `Certificate` object at a later time. Then, `get_ordered_certificate()` should be run periodically via `python manage.py pending_certs fetch -i all` to attempt to retrieve an ordered certificate::
def get_ordered_ceriticate(self, order_id):
@ -154,9 +153,10 @@ An issuer may take some time to actually issue a certificate for an order. In t
# retrieve an order, and check if there is an issued certificate attached to it
`cancel_ordered_certificate()` should be implemented to allow an ordered certificate to be canceled before it is issued::
def cancel_ordered_certificate(self, pending_cert, **kwargs):
# pending_cert should contain the necessary information to match an order
# kwargs can be given to provide information to the issuer for canceling
def cancel_ordered_certificate(self, pending_cert, **kwargs):
# pending_cert should contain the necessary information to match an order
# kwargs can be given to provide information to the issuer for canceling
Destination
-----------
@ -286,7 +286,7 @@ The `ExportPlugin` object requires the implementation of one function::
Custom TLS Provider
------
-------------------
Managing TLS at the enterprise scale could be hard and often organizations offer custom wrapper implementations. It could
be ideal to use those while making calls to internal services. The `TLSPlugin` would help to achieve this. It requires the

1
docs/guide/index.rst
View File

@ -65,6 +65,7 @@ Import an Existing Certificate
You can add notification options and upload the created certificate to a destination, both
of these are editable features and can be changed after the certificate has been created.
.. _CreateANewUser:
Create a New User
~~~~~~~~~~~~~~~~~

2
docs/production/index.rst
View File

@ -501,7 +501,7 @@ rely on celery to create the DNS record. This will change when we implement mix
To create a HTTP compatible Authority, you first need to create a new destination that will be used to deploy the
challenge token. Visit `Admin` -> `Destination` and click `Create`. The path you provide for the destination needs to
be the exact path that is called when the ACME providers calls ``http://<domain>/.well-known/acme-challenge/`. The
be the exact path that is called when the ACME providers calls `http://<domain>/.well-known/acme-challenge/`. The
token part will be added dynamically by the acme_upload.
Currently only the SFTP and S3 Bucket destination support the ACME HTTP challenge.

10
docs/quickstart/index.rst
View File

@ -148,7 +148,7 @@ Before Lemur will run you need to fill in a few required variables in the config
LEMUR_DEFAULT_ORGANIZATIONAL_UNIT
Set Up Postgres
--------------
---------------
For production, a dedicated database is recommended, for this guide we will assume postgres has been installed and is on the same machine that Lemur is installed on.
@ -186,11 +186,12 @@ In addition to creating a new user, Lemur also creates a few default email notif
Your database installation requires the pg_trgm extension. If you do not have this installed already, you can allow the script to install this for you by adding the SUPERUSER permission to the lemur database user.
.. code-block:: bash
sudo -u postgres -i
psql
postgres=# ALTER USER lemur WITH SUPERUSER
Additional notifications can be created through the UI or API. See :ref:`Creating Notifications <CreatingNotifications>` and :ref:`Command Line Interface <CommandLineInterface>` for details.
Additional notifications can be created through the UI or API. See :ref:`Notification Options <NotificationOptions>` and :ref:`Command Line Interface <CommandLineInterface>` for details.
**Make note of the password used as this will be used during first login to the Lemur UI.**
@ -202,15 +203,16 @@ Additional notifications can be created through the UI or API. See :ref:`Creati
.. note:: If you added the SUPERUSER permission to the lemur database user above, it is recommended you revoke that permission now.
.. code-block:: bash
sudo -u postgres -i
psql
postgres=# ALTER USER lemur WITH NOSUPERUSER
.. note:: It is recommended that once the ``lemur`` user is created that you create individual users for every day access. There is currently no way for a user to self enroll for Lemur access, they must have an administrator create an account for them or be enrolled automatically through SSO. This can be done through the CLI or UI. See :ref:`Creating Users <CreatingUsers>` and :ref:`Command Line Interface <CommandLineInterface>` for details.
.. note:: It is recommended that once the ``lemur`` user is created that you create individual users for every day access. There is currently no way for a user to self enroll for Lemur access, they must have an administrator create an account for them or be enrolled automatically through SSO. This can be done through the CLI or UI. See :ref:`Creating a New User <CreateANewUser>` and :ref:`Command Line Interface <CommandLineInterface>` for details.
Set Up a Reverse Proxy
---------------------
----------------------
By default, Lemur runs on port 8000. Even if you change this, under normal conditions you won't be able to bind to port 80. To get around this (and to avoid running Lemur as a privileged user, which you shouldn't), we need to set up a simple web proxy. There are many different web servers you can use for this, we like and recommend Nginx.

112
lemur/authorities/views.py
View File

@ -132,31 +132,31 @@ class AuthoritiesList(AuthenticatedResource):
Accept: application/json, text/javascript
Content-Type: application/json;charset=UTF-8
{
"country": "US",
"state": "California",
"location": "Los Gatos",
"organization": "Netflix",
"organizationalUnit": "Operations",
"type": "root",
"signingAlgorithm": "sha256WithRSA",
"sensitivity": "medium",
"keyType": "RSA2048",
"plugin": {
"slug": "cloudca-issuer"
},
"name": "TimeTestAuthority5",
"owner": "secure@example.com",
"description": "test",
"commonName": "AcommonName",
"validityYears": "20",
"extensions": {
"subAltNames": {
"names": []
},
"custom": []
}
}
{
"country": "US",
"state": "California",
"location": "Los Gatos",
"organization": "Netflix",
"organizationalUnit": "Operations",
"type": "root",
"signingAlgorithm": "sha256WithRSA",
"sensitivity": "medium",
"keyType": "RSA2048",
"plugin": {
"slug": "cloudca-issuer"
},
"name": "TimeTestAuthority5",
"owner": "secure@example.com",
"description": "test",
"commonName": "AcommonName",
"validityYears": "20",
"extensions": {
"subAltNames": {
"names": []
},
"custom": []
}
}
**Example response**:
@ -218,8 +218,7 @@ class AuthoritiesList(AuthenticatedResource):
:arg parent: the parent authority if this is to be a subca
:arg signingAlgorithm: algorithm used to sign the authority
:arg keyType: key type
:arg sensitivity: the sensitivity of the root key, for CloudCA this determines if the root keys are stored
in an HSM
:arg sensitivity: the sensitivity of the root key, for CloudCA this determines if the root keys are stored in an HSM
:arg keyName: name of the key to store in the HSM (CloudCA)
:arg serialNumber: serial number of the authority
:arg firstSerial: specifies the starting serial number for certificates issued off of this authority
@ -494,23 +493,48 @@ class CertificateAuthority(AuthenticatedResource):
class AuthorityVisualizations(AuthenticatedResource):
def get(self, authority_id):
"""
{"name": "flare",
"children": [
{
"name": "analytics",
"children": [
{
"name": "cluster",
"children": [
{"name": "AgglomerativeCluster", "size": 3938},
{"name": "CommunityStructure", "size": 3812},
{"name": "HierarchicalCluster", "size": 6714},
{"name": "MergeEdge", "size": 743}
]
}
]
}
]}
.. http:get:: /authorities/1/visualize
Authority visualization
**Example request**:
.. sourcecode:: http
GET /certificates/1/visualize HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{"name": "flare",
"children": [
{
"name": "analytics",
"children": [
{
"name": "cluster",
"children": [
{"name": "AgglomerativeCluster", "size": 3938},
{"name": "CommunityStructure", "size": 3812},
{"name": "HierarchicalCluster", "size": 6714},
{"name": "MergeEdge", "size": 743}
]
}
]
}
]
}
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
"""
authority = service.get(authority_id)
return dict(

1
lemur/certificates/views.py
View File

@ -59,6 +59,7 @@ class CertificatesListValid(AuthenticatedResource):
**Example request**:
.. sourcecode:: http
GET /certificates/valid?filter=cn;*.test.example.net&owner=joe@example.com&page=1&count=20
HTTP/1.1
Host: example.com

6
lemur/destinations/service.py
View File

@ -21,7 +21,7 @@ def create(label, plugin_name, options, description=None):
:param label: Destination common name
:param description:
:rtype : Destination
:rtype: Destination
:return: New destination
"""
# remove any sub-plugin objects before try to save the json options
@ -50,7 +50,7 @@ def update(destination_id, label, plugin_name, options, description):
:param plugin_name:
:param options:
:param description:
:rtype : Destination
:rtype: Destination
:return:
"""
destination = get(destination_id)
@ -81,7 +81,7 @@ def get(destination_id):
Retrieves an destination by its lemur assigned ID.
:param destination_id: Lemur assigned ID
:rtype : Destination
:rtype: Destination
:return:
"""
return database.get(Destination, destination_id)

6
lemur/notifications/service.py
View File

@ -94,7 +94,7 @@ def create(label, plugin_name, options, description, certificates):
:param options:
:param description:
:param certificates:
:rtype : Notification
:rtype: Notification
:return:
"""
notification = Notification(
@ -115,7 +115,7 @@ def update(notification_id, label, plugin_name, options, description, active, ce
:param description:
:param active:
:param certificates:
:rtype : Notification
:rtype: Notification
:return:
"""
notification = get(notification_id)
@ -144,7 +144,7 @@ def get(notification_id):
Retrieves an notification by its lemur assigned ID.
:param notification_id: Lemur assigned ID
:rtype : Notification
:rtype: Notification
:return:
"""
return database.get(Notification, notification_id)

3
lemur/plugins/lemur_aws/plugin.py
View File

@ -450,7 +450,8 @@ class S3DestinationPlugin(ExportDestinationPlugin):
def upload_acme_token(self, token_path, token, options, **kwargs):
"""
This is called from the acme http challenge
This is called from the acme http challenge
:param self:
:param token_path:
:param token:

30
requirements-docs.in
View File

@ -3,8 +3,38 @@
# from requirements-docs.txt
# However, dependabot doesn't use `make up-reqs`, so we have to replicate the necessary dependencies here
# Without including these dependencies, the docs are unable to include generated autodocs
acme
arrow
boto3
botocore
CloudFlare
cryptography
dnspython3
dyn
Flask
Flask-Bcrypt
Flask-Cors
Flask-Mail
Flask-Migrate
Flask-Principal
Flask-RESTful
Flask-Script
Flask-SQLAlchemy
flask_replicated
gunicorn
inflection
josepy
logmatic-python
marshmallow-sqlalchemy
marshmallow<2.20.5 #schema duplicate issues https://github.com/marshmallow-code/marshmallow-sqlalchemy/issues/121
pem
pyjwt
pyOpenSSL
raven[flask]
retrying
SQLAlchemy-Utils
tabulate
xmltodict
# docs specific
sphinx

62
requirements-docs.txt
View File

@ -4,36 +4,92 @@
#
# pip-compile --no-index --output-file=requirements-docs.txt requirements-docs.in
#
acme==1.12.0
# manual debug
alabaster==0.7.12
# via sphinx
arrow==0.17.0
# manual debug
babel==2.8.0
# via sphinx
boto3==1.17.7
# manual debug
botocore==1.20.7
# manual debug
certifi==2020.12.5
# via requests
chardet==3.0.4
# via requests
cloudflare==2.8.15
# manual debug
cryptography==3.4.5
# manual debug
dnspython3==1.15.0
# manual debug
dnspython==1.15.0
# manual debug
docutils==0.15.2
# via sphinx
dyn==1.8.1
# manual debug
idna==2.9
# via requests
imagesize==1.2.0
# via sphinx
flask==1.1.2
# manual debug
flask-bcrypt==0.7.1
# manual debug
flask-cors==3.0.10
# manual debug
flask-mail==0.9.1
# manual debug
flask-migrate==2.6.0
# manual debug
flask-principal==0.4.0
# manual debug
flask-replicated==1.4
# manual debug
flask-restful==0.3.8
# manual debug
flask-script==2.0.6
# manual debug
flask-sqlalchemy==2.4.4
# manual debug
gunicorn==20.0.4
# manual debug
inflection==0.5.1
# manual debug
jinja2==2.11.3
# via sphinx
josepy==1.3.0
# manual debug
logmatic-python==0.1.7
# manual debug
markupsafe==1.1.1
# via jinja2
marshmallow-sqlalchemy==0.23.1
# manual debug
marshmallow==2.20.4
# manual debug
packaging==20.3
# via sphinx
pem==21.1.0
# manual debug
pygments==2.6.1
# via sphinx
pyjwt==2.0.1
# manual debug
pyopenssl==20.0.1
# manual debug
pyparsing==2.4.7
# via packaging
pytz==2019.3
# via babel
raven[flask]==6.10.0
# manual debug
retrying==1.3.3
# manual debug
requests==2.25.1
# via sphinx
six==1.15.0
@ -63,8 +119,14 @@ sphinxcontrib-qthelp==1.0.3
# via sphinx
sphinxcontrib-serializinghtml==1.1.4
# via sphinx
sqlalchemy-utils==0.36.8
# manual debug
tabulate==0.8.7
# manual debug
urllib3==1.25.8
# via requests
xmltodict==0.12.0
# manual debug
# The following packages are considered to be unsafe in a requirements file:
# setuptools