lemur/lemur/certificates/service.py

904 lines
29 KiB
Python
Raw Normal View History

2015-06-22 22:47:27 +02:00
"""
.. module: lemur.certificate.service
2015-06-22 22:47:27 +02:00
:platform: Unix
:copyright: (c) 2018 by Netflix Inc., see AUTHORS for more
2015-06-22 22:47:27 +02:00
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
2018-09-13 19:35:54 +02:00
import arrow
import re
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes, serialization
2018-09-13 19:35:54 +02:00
from flask import current_app
from sqlalchemy import func, or_, not_, cast, Integer
2020-11-05 03:23:39 +01:00
from sqlalchemy.sql.expression import false, true
2015-06-22 22:47:27 +02:00
from lemur import database
from lemur.authorities.models import Authority
from lemur.certificates.models import Certificate
2018-09-13 19:35:54 +02:00
from lemur.certificates.schemas import CertificateOutputSchema, CertificateInputSchema
from lemur.common.utils import generate_private_key, truthiness
from lemur.destinations.models import Destination
from lemur.domains.models import Domain
from lemur.endpoints import service as endpoint_service
2018-09-13 19:35:54 +02:00
from lemur.extensions import metrics, sentry, signals
from lemur.models import certificate_associations
from lemur.notifications.models import Notification
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
from lemur.pending_certificates.models import PendingCertificate
2018-09-13 19:35:54 +02:00
from lemur.plugins.base import plugins
from lemur.roles import service as role_service
2018-09-13 19:35:54 +02:00
from lemur.roles.models import Role
2015-06-22 22:47:27 +02:00
2019-05-16 16:57:02 +02:00
csr_created = signals.signal("csr_created", "CSR generated")
csr_imported = signals.signal("csr_imported", "CSR imported from external source")
certificate_issued = signals.signal(
"certificate_issued", "Authority issued a certificate"
)
certificate_imported = signals.signal(
"certificate_imported", "Certificate imported from external source"
)
2015-06-22 22:47:27 +02:00
def get(cert_id):
"""
Retrieves certificate by its ID.
2015-06-22 22:47:27 +02:00
:param cert_id:
:return:
"""
return database.get(Certificate, cert_id)
def get_by_name(name):
"""
Retrieves certificate by its Name.
2015-06-22 22:47:27 +02:00
:param name:
:return:
"""
2019-05-16 16:57:02 +02:00
return database.get(Certificate, name, field="name")
2015-06-22 22:47:27 +02:00
def get_by_serial(serial):
"""
Retrieves certificate(s) by serial number.
:param serial:
:return:
"""
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
if isinstance(serial, int):
# although serial is a number, the DB column is String(128)
serial = str(serial)
return Certificate.query.filter(Certificate.serial == serial).all()
def get_by_attributes(conditions):
"""
Retrieves certificate(s) by conditions given in a hash of given key=>value pairs.
:param serial:
:return:
"""
# Ensure that each of the given conditions corresponds to actual columns
# if not, silently remove it
for attr in conditions.keys():
if attr not in Certificate.__table__.columns:
conditions.pop(attr)
query = database.session_query(Certificate)
return database.find_all(query, Certificate, conditions).all()
2015-06-22 22:47:27 +02:00
def delete(cert_id):
"""
Delete's a certificate.
:param cert_id:
"""
database.delete(get(cert_id))
def get_all_certs():
"""
Retrieves all certificates within Lemur.
:return:
"""
return Certificate.query.all()
def get_all_valid_certs(authority_plugin_name):
"""
Retrieves all valid (not expired & not revoked) certificates within Lemur, for the given authority plugin names
ignored if no authority_plugin_name provided.
Note that depending on the DB size retrieving all certificates might an expensive operation
:return:
"""
if authority_plugin_name:
return (
Certificate.query.outerjoin(Authority, Authority.id == Certificate.authority_id).filter(
Certificate.not_after > arrow.now().format("YYYY-MM-DD")).filter(
Authority.plugin_name.in_(authority_plugin_name)).filter(Certificate.revoked.is_(False)).all()
)
else:
return (
Certificate.query.filter(Certificate.not_after > arrow.now().format("YYYY-MM-DD")).filter(
Certificate.revoked.is_(False)).all()
)
def get_all_pending_cleaning_expired(source):
"""
Retrieves all certificates that are available for cleaning. These are certificates which are expired and are not
attached to any endpoints.
:param source: the source to search for certificates
2020-03-18 00:59:09 +01:00
:return: list of pending certificates
"""
2019-05-16 16:57:02 +02:00
return (
Certificate.query.filter(Certificate.sources.any(id=source.id))
.filter(not_(Certificate.endpoints.any()))
.filter(Certificate.expired)
.all()
)
2020-04-28 21:16:46 +02:00
def get_all_certs_attached_to_endpoint_without_autorotate():
"""
Retrieves all certificates that are attached to an endpoint, but that do not have autorotate enabled.
:return: list of certificates attached to an endpoint without autorotate
"""
return (
Certificate.query.filter(Certificate.endpoints.any())
2020-11-05 03:23:39 +01:00
.filter(Certificate.rotation == false())
.filter(Certificate.revoked == false())
.filter(Certificate.not_after >= arrow.now())
.filter(not_(Certificate.replaced.any()))
.all() # noqa
)
def get_all_pending_cleaning_expiring_in_days(source, days_to_expire):
"""
Retrieves all certificates that are available for cleaning, not attached to endpoint,
and within X days from expiration.
:param days_to_expire: defines how many days till the certificate is expired
:param source: the source to search for certificates
2020-03-18 00:59:09 +01:00
:return: list of pending certificates
"""
expiration_window = arrow.now().shift(days=+days_to_expire).format("YYYY-MM-DD")
return (
Certificate.query.filter(Certificate.sources.any(id=source.id))
.filter(not_(Certificate.endpoints.any()))
.filter(Certificate.not_after < expiration_window)
.all()
)
def get_all_pending_cleaning_issued_since_days(source, days_since_issuance):
"""
Retrieves all certificates that are available for cleaning: not attached to endpoint, and X days since issuance.
:param days_since_issuance: defines how many days since the certificate is issued
:param source: the source to search for certificates
2020-03-18 00:59:09 +01:00
:return: list of pending certificates
"""
not_in_use_window = (
arrow.now().shift(days=-days_since_issuance).format("YYYY-MM-DD")
)
return (
Certificate.query.filter(Certificate.sources.any(id=source.id))
.filter(not_(Certificate.endpoints.any()))
2020-03-15 04:41:03 +01:00
.filter(Certificate.date_created > not_in_use_window)
.all()
)
def get_all_pending_reissue():
"""
Retrieves all certificates that need to be rotated.
Must be X days from expiration, uses the certificates rotation
policy to determine how many days from expiration the certificate must be
for rotation to be pending.
:return:
"""
2019-05-16 16:57:02 +02:00
return (
2020-11-05 03:23:39 +01:00
Certificate.query.filter(Certificate.rotation == true())
2019-05-16 16:57:02 +02:00
.filter(not_(Certificate.replaced.any()))
2020-11-05 03:23:39 +01:00
.filter(Certificate.in_rotation_window == true())
2019-05-16 16:57:02 +02:00
.all()
) # noqa
def find_duplicates(cert):
2015-06-22 22:47:27 +02:00
"""
Finds certificates that already exist within Lemur. We do this by looking for
certificate bodies that are the same. This is the most reliable way to determine
if a certificate is already being tracked by Lemur.
:param cert:
2015-06-22 22:47:27 +02:00
:return:
"""
2019-05-16 16:57:02 +02:00
if cert["chain"]:
return Certificate.query.filter_by(
body=cert["body"].strip(), chain=cert["chain"].strip()
).all()
else:
2019-05-16 16:57:02 +02:00
return Certificate.query.filter_by(body=cert["body"].strip(), chain=None).all()
2015-06-22 22:47:27 +02:00
2015-11-27 22:27:14 +01:00
def export(cert, export_plugin):
2015-11-25 23:54:08 +01:00
"""
Exports a certificate to the requested format. This format
may be a binary format.
2015-11-27 22:27:14 +01:00
:param export_plugin:
:param cert:
2015-11-25 23:54:08 +01:00
:return:
"""
2019-05-16 16:57:02 +02:00
plugin = plugins.get(export_plugin["slug"])
return plugin.export(
cert.body, cert.chain, cert.private_key, export_plugin["pluginOptions"]
)
2015-11-25 23:54:08 +01:00
def update(cert_id, **kwargs):
2015-06-22 22:47:27 +02:00
"""
2015-11-24 23:53:22 +01:00
Updates a certificate
2015-06-22 22:47:27 +02:00
:param cert_id:
:return:
"""
cert = get(cert_id)
for key, value in kwargs.items():
setattr(cert, key, value)
2015-08-29 22:07:30 +02:00
2015-06-22 22:47:27 +02:00
return database.update(cert)
def cleanup_owner_roles_notification(owner_name, kwargs):
kwargs["roles"] = [r for r in kwargs["roles"] if r.name != owner_name]
2020-10-10 02:57:35 +02:00
notification_prefix = f"DEFAULT_{owner_name.split('@')[0].upper()}"
kwargs["notifications"] = [n for n in kwargs["notifications"] if not n.label.startswith(notification_prefix)]
2020-10-08 20:38:39 +02:00
def update_notify(cert, notify_flag):
"""
Toggle notification value which is a boolean
:param notify_flag: new notify value
:param cert: Certificate object to be updated
:return:
"""
cert.notify = notify_flag
return database.update(cert)
def create_certificate_roles(**kwargs):
# create a role for the owner and assign it
owner_role = role_service.get_or_create(
kwargs["owner"],
2020-10-10 02:57:35 +02:00
description=f"Auto generated role based on owner: {kwargs['owner']}"
)
# ensure that the authority's owner is also associated with the certificate
2019-05-16 16:57:02 +02:00
if kwargs.get("authority"):
authority_owner_role = role_service.get_by_name(kwargs["authority"].owner)
return [owner_role, authority_owner_role]
return [owner_role]
def mint(**kwargs):
2015-06-22 22:47:27 +02:00
"""
Minting is slightly different for each authority.
Support for multiple authorities is handled by individual plugins.
"""
2019-05-16 16:57:02 +02:00
authority = kwargs["authority"]
2015-06-22 22:47:27 +02:00
issuer = plugins.get(authority.plugin_name)
2015-06-22 22:47:27 +02:00
# allow the CSR to be specified by the user
2019-05-16 16:57:02 +02:00
if not kwargs.get("csr"):
csr, private_key = create_csr(**kwargs)
csr_created.send(authority=authority, csr=csr)
else:
2019-05-16 16:57:02 +02:00
csr = str(kwargs.get("csr"))
private_key = None
csr_imported.send(authority=authority, csr=csr)
2015-07-02 21:10:09 +02:00
cert_body, cert_chain, external_id = issuer.create_certificate(csr, kwargs)
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
return cert_body, private_key, cert_chain, external_id, csr
2015-06-22 22:47:27 +02:00
def import_certificate(**kwargs):
"""
Uploads already minted certificates and pulls the required information into Lemur.
This is to be used for certificates that are created outside of Lemur but
2015-06-22 22:47:27 +02:00
should still be tracked.
Internally this is used to bootstrap Lemur with external
certificates, and used when certificates are 'discovered' through various discovery
techniques. was still in aws.
:param kwargs:
"""
2019-05-16 16:57:02 +02:00
if not kwargs.get("owner"):
kwargs["owner"] = current_app.config.get("LEMUR_SECURITY_TEAM_EMAIL")[0]
2015-06-22 22:47:27 +02:00
return upload(**kwargs)
2015-06-22 22:47:27 +02:00
def upload(**kwargs):
"""
Allows for pre-made certificates to be imported into Lemur.
"""
roles = create_certificate_roles(**kwargs)
2019-05-16 16:57:02 +02:00
if kwargs.get("roles"):
kwargs["roles"] += roles
else:
2019-05-16 16:57:02 +02:00
kwargs["roles"] = roles
cert = Certificate(**kwargs)
2019-05-16 16:57:02 +02:00
cert.authority = kwargs.get("authority")
2015-06-22 22:47:27 +02:00
cert = database.create(cert)
2019-05-16 16:57:02 +02:00
kwargs["creator"].certificates.append(cert)
cert = database.update(cert)
certificate_imported.send(certificate=cert, authority=cert.authority)
return cert
2015-06-22 22:47:27 +02:00
def create(**kwargs):
"""
Creates a new certificate.
"""
try:
cert_body, private_key, cert_chain, external_id, csr = mint(**kwargs)
2018-07-20 19:53:47 +02:00
except Exception:
log_data = {
"message": "Exception minting certificate",
"issuer": kwargs["authority"].name,
"cn": kwargs["common_name"],
}
current_app.logger.error(log_data, exc_info=True)
sentry.captureException()
raise
2019-05-16 16:57:02 +02:00
kwargs["body"] = cert_body
kwargs["private_key"] = private_key
kwargs["chain"] = cert_chain
kwargs["external_id"] = external_id
kwargs["csr"] = csr
2015-06-22 22:47:27 +02:00
roles = create_certificate_roles(**kwargs)
2019-05-16 16:57:02 +02:00
if kwargs.get("roles"):
kwargs["roles"] += roles
else:
2019-05-16 16:57:02 +02:00
kwargs["roles"] = roles
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
if cert_body:
cert = Certificate(**kwargs)
2019-05-16 16:57:02 +02:00
kwargs["creator"].certificates.append(cert)
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
else:
2021-01-30 00:45:51 +01:00
# ACME path
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
cert = PendingCertificate(**kwargs)
2019-05-16 16:57:02 +02:00
kwargs["creator"].pending_certificates.append(cert)
2019-05-16 16:57:02 +02:00
cert.authority = kwargs["authority"]
database.commit()
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
if isinstance(cert, Certificate):
certificate_issued.send(certificate=cert, authority=cert.authority)
2019-05-16 16:57:02 +02:00
metrics.send(
"certificate_issued",
"counter",
1,
metric_tags=dict(owner=cert.owner, issuer=cert.issuer),
)
2018-09-13 19:35:54 +02:00
if isinstance(cert, PendingCertificate):
# We need to refresh the pending certificate to avoid "Instance is not bound to a Session; "
# "attribute refresh operation cannot proceed"
pending_cert = database.session_query(PendingCertificate).get(cert.id)
from lemur.common.celery import fetch_acme_cert
2019-05-16 16:57:02 +02:00
2018-11-28 23:27:03 +01:00
if not current_app.config.get("ACME_DISABLE_AUTORESOLVE", False):
2018-11-28 23:41:56 +01:00
fetch_acme_cert.apply_async((pending_cert.id,), countdown=5)
2015-06-22 22:47:27 +02:00
return cert
def render(args):
"""
Helper function that allows use to render our REST Api.
:param args:
:return:
"""
query = database.session_query(Certificate)
show_expired = args.pop("showExpired")
if show_expired != 1:
one_month_old = (
arrow.now()
.shift(months=current_app.config.get("HIDE_EXPIRED_CERTS_AFTER_MONTHS", -1))
.format("YYYY-MM-DD")
)
query = query.filter(Certificate.not_after > one_month_old)
time_range = args.pop("time_range")
2019-05-16 16:57:02 +02:00
destination_id = args.pop("destination_id")
notification_id = args.pop("notification_id", None)
show = args.pop("show")
2015-07-21 22:06:13 +02:00
# owner = args.pop('owner')
# creator = args.pop('creator') # TODO we should enabling filtering by owner
2015-06-22 22:47:27 +02:00
2019-05-16 16:57:02 +02:00
filt = args.pop("filter")
2015-06-22 22:47:27 +02:00
if filt:
2019-05-16 16:57:02 +02:00
terms = filt.split(";")
term = "%{0}%".format(terms[1])
2018-03-28 17:33:43 +02:00
# Exact matches for quotes. Only applies to name, issuer, and cn
if terms[1].startswith('"') and terms[1].endswith('"'):
term = terms[1][1:-1]
2015-11-25 01:07:44 +01:00
2019-05-16 16:57:02 +02:00
if "issuer" in terms:
2015-06-22 22:47:27 +02:00
# we can't rely on issuer being correct in the cert directly so we combine queries
2019-05-16 16:57:02 +02:00
sub_query = (
database.session_query(Authority.id)
.filter(Authority.name.ilike(term))
2015-06-22 22:47:27 +02:00
.subquery()
2019-05-16 16:57:02 +02:00
)
2015-06-22 22:47:27 +02:00
query = query.filter(
or_(
2018-03-28 17:33:43 +02:00
Certificate.issuer.ilike(term),
2019-05-16 16:57:02 +02:00
Certificate.authority_id.in_(sub_query),
2015-06-22 22:47:27 +02:00
)
)
2019-05-16 16:57:02 +02:00
elif "destination" in terms:
query = query.filter(
Certificate.destinations.any(Destination.id == terms[1])
)
elif "notify" in filt:
query = query.filter(Certificate.notify == truthiness(terms[1]))
2019-05-16 16:57:02 +02:00
elif "active" in filt:
query = query.filter(Certificate.active == truthiness(terms[1]))
2019-05-16 16:57:02 +02:00
elif "cn" in terms:
2015-11-25 01:07:44 +01:00
query = query.filter(
or_(
func.lower(Certificate.cn).like(term.lower()),
Certificate.id.in_(like_domain_query(term)),
2015-11-25 01:07:44 +01:00
)
)
2019-05-16 16:57:02 +02:00
elif "id" in terms:
query = query.filter(Certificate.id == cast(terms[1], Integer))
2019-05-16 16:57:02 +02:00
elif "name" in terms:
2018-03-28 17:33:43 +02:00
query = query.filter(
or_(
func.lower(Certificate.name).like(term.lower()),
Certificate.id.in_(like_domain_query(term)),
func.lower(Certificate.cn).like(term.lower()),
2018-03-28 17:33:43 +02:00
)
)
elif "fixedName" in terms:
# only what matches the fixed name directly if a fixedname is provided
query = query.filter(Certificate.name == terms[1])
2015-06-22 22:47:27 +02:00
else:
query = database.filter(query, Certificate, terms)
if show:
2019-05-16 16:57:02 +02:00
sub_query = (
database.session_query(Role.name)
.filter(Role.user_id == args["user"].id)
.subquery()
)
2015-06-22 22:47:27 +02:00
query = query.filter(
or_(
2019-05-16 16:57:02 +02:00
Certificate.user_id == args["user"].id, Certificate.owner.in_(sub_query)
2015-06-22 22:47:27 +02:00
)
)
if destination_id:
2019-05-16 16:57:02 +02:00
query = query.filter(
Certificate.destinations.any(Destination.id == destination_id)
)
2015-06-22 22:47:27 +02:00
if notification_id:
2019-05-16 16:57:02 +02:00
query = query.filter(
Certificate.notifications.any(Notification.id == notification_id)
)
2015-06-22 22:47:27 +02:00
if time_range:
to = arrow.now().shift(weeks=+time_range).format("YYYY-MM-DD")
2019-05-16 16:57:02 +02:00
now = arrow.now().format("YYYY-MM-DD")
query = query.filter(Certificate.not_after <= to).filter(
Certificate.not_after >= now
)
2015-06-22 22:47:27 +02:00
2019-05-16 16:57:02 +02:00
if current_app.config.get("ALLOW_CERT_DELETION", False):
2020-11-05 03:23:39 +01:00
query = query.filter(Certificate.deleted == false())
result = database.sort_and_page(query, Certificate, args)
return result
2015-06-22 22:47:27 +02:00
def like_domain_query(term):
domain_query = database.session_query(Domain.id)
domain_query = domain_query.filter(func.lower(Domain.name).like(term.lower()))
assoc_query = database.session_query(certificate_associations.c.certificate_id)
assoc_query = assoc_query.filter(certificate_associations.c.domain_id.in_(domain_query))
return assoc_query
def query_name(certificate_name, args):
"""
Helper function that queries for a certificate by name
:param args:
:return:
"""
query = database.session_query(Certificate)
query = query.filter(Certificate.name == certificate_name)
result = database.sort_and_page(query, Certificate, args)
return result
def query_common_name(common_name, args):
"""
Helper function that queries for not expired certificates by common name (and owner)
:param common_name:
:param args:
:return:
"""
2019-05-16 16:57:02 +02:00
owner = args.pop("owner")
2021-01-06 20:28:18 +01:00
page = args.pop("page")
count = args.pop("count")
paginate = page and count
query = database.session_query(Certificate) if paginate else Certificate.query
# only not expired certificates
current_time = arrow.utcnow()
2021-01-06 20:28:18 +01:00
query = query.filter(Certificate.not_after >= current_time.format("YYYY-MM-DD"))\
.filter(not_(Certificate.revoked))\
.filter(not_(Certificate.replaced.any())) # ignore rotated certificates to avoid duplicates
if owner:
query = query.filter(Certificate.owner.ilike(owner))
if common_name != "%":
# if common_name is a wildcard ('%'), no need to include it in the query
query = query.filter(Certificate.cn.ilike(common_name))
2021-01-06 20:28:18 +01:00
if paginate:
return database.paginate(query, page, count)
return query.all()
def create_csr(**csr_config):
2015-06-22 22:47:27 +02:00
"""
Given a list of domains create the appropriate csr
for those domains
:param csr_config:
"""
2019-05-16 16:57:02 +02:00
private_key = generate_private_key(csr_config.get("key_type"))
2015-06-22 22:47:27 +02:00
2015-07-02 21:10:09 +02:00
builder = x509.CertificateSigningRequestBuilder()
2019-05-16 16:57:02 +02:00
name_list = [x509.NameAttribute(x509.OID_COMMON_NAME, csr_config["common_name"])]
if current_app.config.get("LEMUR_OWNER_EMAIL_IN_SUBJECT", True):
name_list.append(
x509.NameAttribute(x509.OID_EMAIL_ADDRESS, csr_config["owner"])
)
if "organization" in csr_config and csr_config["organization"].strip():
name_list.append(
x509.NameAttribute(x509.OID_ORGANIZATION_NAME, csr_config["organization"])
)
if (
"organizational_unit" in csr_config
and csr_config["organizational_unit"].strip()
):
name_list.append(
x509.NameAttribute(
x509.OID_ORGANIZATIONAL_UNIT_NAME, csr_config["organizational_unit"]
)
)
if "country" in csr_config and csr_config["country"].strip():
name_list.append(
x509.NameAttribute(x509.OID_COUNTRY_NAME, csr_config["country"])
)
if "state" in csr_config and csr_config["state"].strip():
name_list.append(
x509.NameAttribute(x509.OID_STATE_OR_PROVINCE_NAME, csr_config["state"])
)
if "location" in csr_config and csr_config["location"].strip():
name_list.append(
x509.NameAttribute(x509.OID_LOCALITY_NAME, csr_config["location"])
)
builder = builder.subject_name(x509.Name(name_list))
2015-07-02 21:10:09 +02:00
2019-05-16 16:57:02 +02:00
extensions = csr_config.get("extensions", {})
critical_extensions = ["basic_constraints", "sub_alt_names", "key_usage"]
noncritical_extensions = ["extended_key_usage"]
for k, v in extensions.items():
if v:
if k in critical_extensions:
2019-05-16 16:57:02 +02:00
current_app.logger.debug(
"Adding Critical Extension: {0} {1}".format(k, v)
)
if k == "sub_alt_names":
if v["names"]:
builder = builder.add_extension(v["names"], critical=True)
else:
builder = builder.add_extension(v, critical=True)
if k in noncritical_extensions:
2019-05-16 16:57:02 +02:00
current_app.logger.debug("Adding Extension: {0} {1}".format(k, v))
builder = builder.add_extension(v, critical=False)
2019-05-16 16:57:02 +02:00
ski = extensions.get("subject_key_identifier", {})
if ski.get("include_ski", False):
builder = builder.add_extension(
x509.SubjectKeyIdentifier.from_public_key(private_key.public_key()),
2019-05-16 16:57:02 +02:00
critical=False,
)
2015-07-02 21:10:09 +02:00
2019-05-16 16:57:02 +02:00
request = builder.sign(private_key, hashes.SHA256(), default_backend())
2015-06-22 22:47:27 +02:00
# serialize our private key and CSR
private_key = private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.TraditionalOpenSSL, # would like to use PKCS8 but AWS ELBs don't like it
2019-05-16 16:57:02 +02:00
encryption_algorithm=serialization.NoEncryption(),
).decode("utf-8")
2019-05-16 16:57:02 +02:00
csr = request.public_bytes(encoding=serialization.Encoding.PEM).decode("utf-8")
2015-06-22 22:47:27 +02:00
return csr, private_key
2015-06-22 22:47:27 +02:00
2015-07-21 22:06:13 +02:00
2015-06-22 22:47:27 +02:00
def stats(**kwargs):
"""
Helper that defines some useful statistics about certifications.
:param kwargs:
:return:
"""
2021-02-22 23:56:34 +01:00
# Verify requested metric
allow_list = ["bits", "issuer", "not_after", "signing_algorithm"]
req_metric = kwargs.get("metric")
if req_metric not in allow_list:
raise Exception(
f"Stats not available for requested metric: {req_metric}"
)
if req_metric == "not_after":
2015-06-22 22:47:27 +02:00
start = arrow.utcnow()
end = start.shift(weeks=+32)
2019-05-16 16:57:02 +02:00
items = (
database.db.session.query(Certificate.issuer, func.count(Certificate.id))
.group_by(Certificate.issuer)
.filter(Certificate.not_after <= end.format("YYYY-MM-DD"))
.filter(Certificate.not_after >= start.format("YYYY-MM-DD"))
.all()
)
2015-06-22 22:47:27 +02:00
else:
2021-02-22 23:56:34 +01:00
attr = getattr(Certificate, req_metric)
2015-06-22 22:47:27 +02:00
query = database.db.session.query(attr, func.count(attr))
items = query.group_by(attr).all()
keys = []
values = []
for key, count in items:
keys.append(key)
values.append(count)
2019-05-16 16:57:02 +02:00
return {"labels": keys, "values": values}
def get_account_number(arn):
"""
Extract the account number from an arn.
:param arn: IAM SSL arn
:return: account number associated with ARN
"""
return arn.split(":")[4]
def get_name_from_arn(arn):
"""
Extract the certificate name from an arn.
:param arn: IAM SSL arn
:return: name of the certificate as uploaded to AWS
"""
return arn.split("/", 1)[1]
2016-10-31 19:00:15 +01:00
def calculate_reissue_range(start, end):
"""
Determine what the new validity_start and validity_end dates should be.
:param start:
:param end:
:return:
"""
span = end - start
new_start = arrow.utcnow()
2016-10-31 19:00:15 +01:00
new_end = new_start + span
return new_start, arrow.get(new_end)
2016-10-31 19:00:15 +01:00
def get_certificate_primitives(certificate):
"""
Retrieve key primitive from a certificate such that the certificate
could be recreated with new expiration or be used to build upon.
:param certificate:
:return: dict of certificate primitives, should be enough to effectively re-issue
certificate via `create`.
"""
start, end = calculate_reissue_range(certificate.not_before, certificate.not_after)
2019-05-16 16:57:02 +02:00
ser = CertificateInputSchema().load(
CertificateOutputSchema().dump(certificate).data
)
assert not ser.errors, "Error re-serializing certificate: %s" % ser.errors
data = ser.data
# we can't quite tell if we are using a custom name, as this is an automated process (typically)
# we will rely on the Lemur generated name
2019-05-16 16:57:02 +02:00
data.pop("name", None)
# TODO this can be removed once we migrate away from cn
2019-05-16 16:57:02 +02:00
data["cn"] = data["common_name"]
# needed until we move off not_*
2019-05-16 16:57:02 +02:00
data["not_before"] = start
data["not_after"] = end
data["validity_start"] = start
data["validity_end"] = end
return data
def reissue_certificate(certificate, replace=None, user=None):
"""
Reissue certificate with the same properties of the given certificate.
:param certificate:
:param replace:
:param user:
:return:
"""
primitives = get_certificate_primitives(certificate)
if primitives.get("csr"):
# We do not want to re-use the CSR when creating a certificate because this defeats the purpose of rotation.
del primitives["csr"]
if not user:
2019-05-16 16:57:02 +02:00
primitives["creator"] = certificate.user
else:
2019-05-16 16:57:02 +02:00
primitives["creator"] = user
if replace:
2019-05-16 16:57:02 +02:00
primitives["replaces"] = [certificate]
# Modify description to include the certificate ID being reissued and mention that this is created by Lemur
# as part of reissue
reissue_message_prefix = "Reissued by Lemur for cert ID "
reissue_message = re.compile(f"{reissue_message_prefix}([0-9]+)")
if primitives["description"]:
match = reissue_message.search(primitives["description"])
if match:
primitives["description"] = primitives["description"].replace(match.group(1), str(certificate.id))
else:
primitives["description"] = f"{reissue_message_prefix}{certificate.id}, {primitives['description']}"
else:
primitives["description"] = f"{reissue_message_prefix}{certificate.id}"
# Rotate the certificate to ECCPRIME256V1 if cert owner is present in the configured list
# This is a temporary change intending to rotate certificates to ECC, if opted in by certificate owners
# Unless identified a use case, this will be removed in mid-Q2 2021
ecc_reissue_owner_list = current_app.config.get("ROTATE_TO_ECC_OWNER_LIST", [])
ecc_reissue_exclude_cn_list = current_app.config.get("ECC_NON_COMPATIBLE_COMMON_NAMES", [])
if (certificate.owner in ecc_reissue_owner_list) and (certificate.cn not in ecc_reissue_exclude_cn_list):
primitives["key_type"] = "ECCPRIME256V1"
new_cert = create(**primitives)
return new_cert
def is_attached_to_endpoint(certificate_name, endpoint_name):
"""
Find if given certificate is attached to the endpoint. Both, certificate and endpoint, are identified by name.
This method talks to elb and finds the real time information.
:param certificate_name:
:param endpoint_name:
:return: True if certificate is attached to the given endpoint, False otherwise
"""
endpoint = endpoint_service.get_by_name(endpoint_name)
attached_certificates = endpoint.source.plugin.get_endpoint_certificate_names(endpoint)
return certificate_name in attached_certificates
def remove_from_destination(certificate, destination):
"""
Remove the certificate from given destination if clean() is implemented
:param certificate:
:param destination:
:return:
"""
plugin = plugins.get(destination.plugin_name)
if not hasattr(plugin, "clean"):
info_text = f"Cannot clean certificate {certificate.name}, {destination.plugin_name} plugin does not implement 'clean()'"
current_app.logger.warning(info_text)
else:
plugin.clean(certificate=certificate, options=destination.options)
2020-12-01 05:06:37 +01:00
def revoke(certificate, reason):
plugin = plugins.get(certificate.authority.plugin_name)
plugin.revoke_certificate(certificate, reason)
# Perform cleanup after revoke
return cleanup_after_revoke(certificate)
def cleanup_after_revoke(certificate):
"""
Perform the needed cleanup for a revoked certificate. This includes -
1. Disabling notification
2. Disabling auto-rotation
3. Update certificate status to 'revoked'
4. Remove from AWS
:param certificate: Certificate object to modify and update in DB
:return: None
"""
certificate.notify = False
certificate.rotation = False
certificate.status = 'revoked'
error_message = ""
for destination in list(certificate.destinations):
try:
remove_from_destination(certificate, destination)
certificate.destinations.remove(destination)
except Exception as e:
# This cleanup is the best-effort since certificate is already revoked at this point.
# We will capture the exception and move on to the next destination
sentry.captureException()
error_message = error_message + f"Failed to remove destination: {destination.label}. {str(e)}. "
database.update(certificate)
return error_message
def get_issued_cert_count_for_authority(authority):
"""
Returns the count of certs issued by the specified authority.
:return:
"""
return database.db.session.query(Certificate).filter(Certificate.authority_id == authority.id).count()