lemur/docs/developer/plugins/index.rst

360 lines
15 KiB
ReStructuredText
Raw Normal View History

2015-06-22 22:47:27 +02:00
Several interfaces exist for extending Lemur:
2015-07-08 00:32:55 +02:00
* Issuer (lemur.plugins.base.issuer)
* Destination (lemur.plugins.base.destination)
* Source (lemur.plugins.base.source)
* Notification (lemur.plugins.base.notification)
2015-09-19 19:12:12 +02:00
Each interface has its own functions that will need to be defined in order for
your plugin to work correctly. See :ref:`Plugin Interfaces <PluginInterfaces>` for details.
2015-06-22 22:47:27 +02:00
Structure
---------
A plugins layout generally looks like the following::
setup.py
lemur_pluginname/
lemur_pluginname/__init__.py
lemur_pluginname/plugin.py
The ``__init__.py`` file should contain no plugin logic, and at most, a VERSION = 'x.x.x' line. For example,
if you want to pull the version using pkg_resources (which is what we recommend), your file might contain::
try:
VERSION = __import__('pkg_resources') \
.get_distribution(__name__).version
except Exception as e:
2015-06-22 22:47:27 +02:00
VERSION = 'unknown'
Inside of ``plugin.py``, you'll declare your Plugin class::
import lemur_pluginname
2015-07-08 00:32:55 +02:00
from lemur.plugins.base.issuer import IssuerPlugin
2015-06-22 22:47:27 +02:00
2015-07-08 00:32:55 +02:00
class PluginName(IssuerPlugin):
2015-06-22 22:47:27 +02:00
title = 'Plugin Name'
slug = 'pluginname'
description = 'My awesome plugin!'
version = lemur_pluginname.VERSION
author = 'Your Name'
author_url = 'https://github.com/yourname/lemur_pluginname'
def widget(self, request, group, **kwargs):
return "<p>Absolutely useless widget</p>"
And you'll register it via ``entry_points`` in your ``setup.py``::
setup(
# ...
entry_points={
'lemur.plugins': [
'pluginname = lemur_pluginname.issuers:PluginName'
],
},
)
You can potentially package multiple plugin types in one package, say you want to create a source and
destination plugins for the same third-party. To accomplish this simply alias the plugin in entry points to point
at multiple plugins within your package::
setup(
# ...
entry_points={
'lemur.plugins': [
'pluginnamesource = lemur_pluginname.plugin:PluginNameSource',
'pluginnamedestination = lemur_pluginname.plugin:PluginNameDestination'
],
},
)
2015-06-22 22:47:27 +02:00
Once your plugin files are in place and the ``/www/lemur/setup.py`` file has been modified, you can load your plugin into your instance by reinstalling lemur:
::
(lemur)$cd /www/lemur
(lemur)$pip install -e .
2015-07-08 00:32:55 +02:00
That's it! Users will be able to install your plugin via ``pip install <package name>``.
.. SeeAlso:: For more information about python packages see `Python Packaging <https://packaging.python.org/en/latest/distributing.html>`_
.. SeeAlso:: For an example of a plugin operation outside of Lemur's core, see `lemur-digicert <https://github.com/opendns/lemur-digicert>`_
.. _PluginInterfaces:
2015-07-08 00:32:55 +02:00
Plugin Interfaces
=================
2015-06-22 22:47:27 +02:00
In order to use the interfaces all plugins are required to inherit and override unimplemented functions
of the parent object.
2015-06-22 22:47:27 +02:00
2015-07-08 00:32:55 +02:00
Issuer
------
2015-06-22 22:47:27 +02:00
Issuer plugins are used when you have an external service that creates certificates or authorities.
In the simple case the third party only issues certificates (Verisign, DigiCert, etc.).
2015-09-20 18:49:16 +02:00
If you have a third party or internal service that creates authorities (EJBCA, etc.), Lemur has you covered,
it can treat any issuer plugin as both a source of creating new certificates as well as new authorities.
2015-06-22 22:47:27 +02:00
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
The `IssuerPlugin` exposes four functions functions::
2015-06-22 22:47:27 +02:00
def create_certificate(self, csr, issuer_options):
2015-07-08 00:32:55 +02:00
# requests.get('a third party')
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
def revoke_certificate(self, certificate, comments):
# requests.put('a third party')
def get_ordered_certificate(self, order_id):
# requests.get('already existing certificate')
def canceled_ordered_certificate(self, pending_cert, **kwargs):
# requests.put('cancel an order that has yet to be issued')
2015-06-22 22:47:27 +02:00
Lemur will pass a dictionary of all possible options for certificate creation. Including a valid CSR, and the raw options associated with the request.
If you wish to be able to create new authorities implement the following function and ensure that the ROOT_CERTIFICATE and the INTERMEDIATE_CERTIFICATE (if any) for the new authority is returned::
2015-06-22 22:47:27 +02:00
def create_authority(self, options):
root_cert, intermediate_cert, username, password = request.get('a third party')
# if your provider creates specific credentials for each authority you can associated them with the role associated with the authority
# these credentials will be provided along with any other options when a certificate is created
role = dict(username=username, password=password, name='generatedAuthority')
return root_cert, intermediate_cert, [role]
.. Note::
Lemur uses PEM formatted certificates as it's internal standard, if you receive certificates in other formats convert them to PEM before returning.
If instead you do not need need to generate authorities but instead use a static authority (Verisign, DigiCert), you can use publicly available constants::
2015-06-22 22:47:27 +02:00
2015-07-08 00:32:55 +02:00
def create_authority(self, options):
# optionally associate a role with authority to control who can use it
role = dict(username='', password='', name='exampleAuthority')
# username and password don't really matter here because we do no need to authenticate our authority against a third party
return EXAMPLE_ROOT_CERTIFICATE, EXAMPLE_INTERMEDIATE_CERTIFICATE, [role]
.. Note:: You do not need to associate roles to the authority at creation time as they can always be associated after the fact.
The `IssuerPlugin` doesn't have any options like Destination, Source, and Notification plugins. Essentially Lemur **should** already have
any fields you might need to submit a request to a third party. If there are additional options you need
in your plugin feel free to open an issue, or look into adding additional options to issuers yourself.
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
Asynchronous Certificates
^^^^^^^^^^^^^^^^^^^^^^^^^
An issuer may take some time to actually issue a certificate for an order. In this case, a `PendingCertificate` is returned, which holds information to recreate a `Certificate` object at a later time. Then, `get_ordered_certificate()` should be run periodically via `python manage.py pending_certs fetch -i all` to attempt to retrieve an ordered certificate::
def get_ordered_ceriticate(self, order_id):
# order_id is the external id of the order, not the external_id of the certificate
# retrieve an order, and check if there is an issued certificate attached to it
`cancel_ordered_certificate()` should be implemented to allow an ordered certificate to be canceled before it is issued::
def cancel_ordered_certificate(self, pending_cert, **kwargs):
# pending_cert should contain the necessary information to match an order
# kwargs can be given to provide information to the issuer for canceling
Destination
-----------
Destination plugins allow you to propagate certificates managed by Lemur to additional third parties. This provides flexibility when
different orchestration systems have their own way of manage certificates or there is an existing system you wish to integrate with Lemur.
2015-06-22 22:47:27 +02:00
By default destination plugins have a private key requirement. If your plugin does not require a certificates private key mark `requires_key = False`
in the plugins base class like so::
class MyDestinationPlugin(DestinationPlugin):
requires_key = False
The DestinationPlugin requires only one function to be implemented::
2015-07-08 01:26:37 +02:00
def upload(self, name, body, private_key, cert_chain, options, **kwargs):
# request.post('a third party')
2015-07-08 01:26:37 +02:00
Additionally the DestinationPlugin allows the plugin author to add additional options
that can be used to help define sub-destinations.
For example, if we look at the aws-destination plugin we can see that it defines an `accountNumber` option::
options = [
{
'name': 'accountNumber',
'type': 'int',
'required': True,
'validation': '/^[0-9]{12,12}$/',
'helpMessage': 'Must be a valid AWS account number!',
}
]
By defining an `accountNumber` we can make this plugin handle many N number of AWS accounts instead of just one.
The schema for defining plugin options are pretty straightforward:
- **Name**: name of the variable you wish to present the user, snake case (snakeCase) is preferred as Lemur
will parse these and create pretty variable titles
- **Type** there are currently four supported variable types
- **Int** creates an html integer box for the user to enter integers into
- **Str** creates a html text input box
2017-03-31 06:09:30 +02:00
- **Boolean** creates a checkbox for the user to signify truthiness
- **Select** creates a select box that gives the user a list of options
- When used a `available` key must be provided with a list of selectable options
- **Required** determines if this option is required, this **must be a boolean value**
- **Validation** simple JavaScript regular expression used to give the user an indication if the input value is valid
- **HelpMessage** simple string that provides more detail about the option
.. Note::
DestinationPlugin, NotificationPlugin and SourcePlugin all support the option
schema outlined above.
Notification
------------
Lemur includes the ability to create Email notifications by **default**. These notifications
2020-10-16 19:40:11 +02:00
currently come in the form of expiration and rotation notices. Lemur periodically checks certifications expiration dates and
determines if a given certificate is eligible for notification. There are currently only two parameters used to
determine if a certificate is eligible; validity expiration (date the certificate is no longer valid) and the number
of days the current date (UTC) is from that expiration date.
2020-10-16 19:40:11 +02:00
Expiration notifications can also be configured for Slack or AWS SNS. Rotation notifications are not configurable.
Notifications sent to a certificate owner and security team (`LEMUR_SECURITY_TEAM_EMAIL`) can currently only be sent via email.
There are currently two objects that available for notification plugins. The first is `NotificationPlugin`, which is the base object for
any notification within Lemur. Currently the only supported notification type is an certificate expiration notification. If you
are trying to create a new notification type (audit, failed logins, etc.) this would be the object to base your plugin on.
You would also then need to build additional code to trigger the new notification type.
2020-10-16 19:40:11 +02:00
The second is `ExpirationNotificationPlugin`, which inherits from `NotificationPlugin` object.
You will most likely want to base your plugin on this object if you want to add new channels for expiration notices (HipChat, Jira, etc.). It adds default options that are required by
all expiration notifications (interval, unit). This interface expects for the child to define the following function::
def send(self, notification_type, message, targets, options, **kwargs):
# request.post("some alerting infrastructure")
Source
------
When building Lemur we realized that although it would be nice if every certificate went through Lemur to get issued, but this is not
always be the case. Oftentimes there are third parties that will issue certificates on your behalf and these can get deployed
to infrastructure without any interaction with Lemur. In an attempt to combat this and try to track every certificate, Lemur has a notion of
certificate **Sources**. Lemur will contact the source at periodic intervals and attempt to **sync** against the source. This means downloading or discovering any
certificate Lemur does not know about and adding the certificate to its inventory to be tracked and alerted on.
The `SourcePlugin` object has one default option of `pollRate`. This controls the number of seconds which to get new certificates.
.. warning::
Lemur currently has a very basic polling system of running a cron job every 15min to see which source plugins need to be run. A lock file is generated to guarantee that
2015-08-02 03:31:38 +02:00
only one sync is running at a time. It also means that the minimum resolution of a source plugin poll rate is effectively 15min. You can always specify a faster cron
job if you need a higher resolution sync job.
The `SourcePlugin` object requires implementation of one function::
def get_certificates(self, options, **kwargs):
# request.get("some source of certificates")
.. note::
Oftentimes to facilitate code re-use it makes sense put source and destination plugins into one package.
2015-12-03 01:04:40 +01:00
Export
------
Formats, formats and more formats. That's the current PKI landscape. See the always relevant `xkcd <https://xkcd.com/927/>`_.
Thankfully Lemur supports the ability to output your certificates into whatever format you want. This integration comes by the way
of Export plugins. Support is still new and evolving, the goal of these plugins is to return raw data in a new format that
can then be used by any number of applications. Included in Lemur is the `JavaExportPlugin` which currently supports generating
a Java Key Store (JKS) file for use in Java based applications.
The `ExportPlugin` object requires the implementation of one function::
def export(self, body, chain, key, options, **kwargs):
# sys.call('openssl hokuspocus')
# return "extension", passphrase, raw
.. note::
Support of various formats sometimes relies on external tools system calls. Always be mindful of sanitizing any input to these calls.
2015-06-22 22:47:27 +02:00
Testing
=======
Lemur provides a basic py.test-based testing framework for extensions.
In a simple project, you'll need to do a few things to get it working:
setup.py
--------
Augment your setup.py to ensure at least the following:
.. code-block:: python
setup(
# ...
install_requires=[
2015-06-22 22:47:27 +02:00
'lemur',
]
2015-06-22 22:47:27 +02:00
)
conftest.py
-----------
The ``conftest.py`` file is our main entry-point for py.test. We need to configure it to load the Lemur pytest configuration:
.. code-block:: python
from lemur.tests.conftest import * # noqa
2015-06-22 22:47:27 +02:00
Test Cases
----------
You can now inherit from Lemur's core test classes. These are Django-based and ensure the database and other basic utilities are in a clean state:
.. code-block:: python
import pytest
from lemur.tests.vectors import INTERNAL_CERTIFICATE_A_STR, INTERNAL_PRIVATE_KEY_A_STR
2015-06-22 22:47:27 +02:00
def test_export_keystore(app):
from lemur.plugins.base import plugins
p = plugins.get('java-keystore-jks')
options = [{'name': 'passphrase', 'value': 'test1234'}]
with pytest.raises(Exception):
p.export(INTERNAL_CERTIFICATE_A_STR, "", "", options)
2015-06-22 22:47:27 +02:00
raw = p.export(INTERNAL_CERTIFICATE_A_STR, "", INTERNAL_PRIVATE_KEY_A_STR, options)
assert raw != b""
2015-06-22 22:47:27 +02:00
Running Tests
-------------
Running tests follows the py.test standard. As long as your test files and methods are named appropriately (``test_filename.py`` and ``test_function()``) you can simply call out to py.test:
::
$ py.test -v
============================== test session starts ==============================
platform darwin -- Python 2.7.10, pytest-2.8.5, py-1.4.30, pluggy-0.3.1
cachedir: .cache
plugins: flask-0.10.0
collected 346 items
2015-06-22 22:47:27 +02:00
lemur/plugins/lemur_acme/tests/test_acme.py::test_get_certificates PASSED
2015-06-22 22:47:27 +02:00
=========================== 1 passed in 0.35 seconds ============================
.. SeeAlso:: Lemur bundles several plugins that use the same interfaces mentioned above.