lemur/lemur/manage.py

594 lines
18 KiB
Python
Raw Normal View History

#!/usr/bin/env python
2019-05-16 16:57:02 +02:00
from __future__ import unicode_literals # at top of module
2015-06-22 22:47:27 +02:00
import os
import sys
import base64
import requests
import json
2015-06-22 22:47:27 +02:00
from gunicorn.config import make_settings
from cryptography.fernet import Fernet
2015-06-22 22:47:27 +02:00
from flask import current_app
2016-11-23 06:11:20 +01:00
from flask_script import Manager, Command, Option, prompt_pass
from flask_migrate import Migrate, MigrateCommand, stamp
2015-06-22 22:47:27 +02:00
from flask_script.commands import ShowUrls, Clean, Server
from lemur.dns_providers.cli import manager as dns_provider_manager
from lemur.acme_providers.cli import manager as acme_manager
from lemur.sources.cli import manager as source_manager
from lemur.policies.cli import manager as policy_manager
from lemur.reporting.cli import manager as report_manager
from lemur.endpoints.cli import manager as endpoint_manager
from lemur.certificates.cli import manager as certificate_manager
2016-12-08 18:11:19 +01:00
from lemur.notifications.cli import manager as notification_manager
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
from lemur.pending_certificates.cli import manager as pending_certificate_manager
2015-06-22 22:47:27 +02:00
from lemur import database
from lemur.users import service as user_service
from lemur.roles import service as role_service
from lemur.policies import service as policy_service
2015-08-02 04:08:46 +02:00
from lemur.notifications import service as notification_service
2015-06-22 22:47:27 +02:00
from lemur.common.utils import validate_conf
2015-06-22 22:47:27 +02:00
from lemur import create_app
# Needed to be imported so that SQLAlchemy create_all can find our models
2015-07-21 22:06:13 +02:00
from lemur.users.models import User # noqa
from lemur.roles.models import Role # noqa
from lemur.authorities.models import Authority # noqa
from lemur.certificates.models import Certificate # noqa
from lemur.destinations.models import Destination # noqa
from lemur.domains.models import Domain # noqa
from lemur.notifications.models import Notification # noqa
2015-08-02 00:29:34 +02:00
from lemur.sources.models import Source # noqa
from lemur.logs.models import Log # noqa
from lemur.endpoints.models import Endpoint # noqa
from lemur.policies.models import RotationPolicy # noqa
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
from lemur.pending_certificates.models import PendingCertificate # noqa
2018-11-28 23:27:03 +01:00
from lemur.dns_providers.models import DnsProvider # noqa
2015-06-22 22:47:27 +02:00
2019-04-11 00:40:48 +02:00
from sqlalchemy.sql import text
2015-06-22 22:47:27 +02:00
manager = Manager(create_app)
2019-05-16 16:57:02 +02:00
manager.add_option("-c", "--config", dest="config_path", required=False)
2015-06-22 22:47:27 +02:00
migrate = Migrate(create_app)
REQUIRED_VARIABLES = [
2019-05-16 16:57:02 +02:00
"LEMUR_SECURITY_TEAM_EMAIL",
"LEMUR_DEFAULT_ORGANIZATIONAL_UNIT",
"LEMUR_DEFAULT_ORGANIZATION",
"LEMUR_DEFAULT_LOCATION",
"LEMUR_DEFAULT_COUNTRY",
"LEMUR_DEFAULT_STATE",
"SQLALCHEMY_DATABASE_URI",
]
2015-06-22 22:47:27 +02:00
KEY_LENGTH = 40
2019-05-16 16:57:02 +02:00
DEFAULT_CONFIG_PATH = "~/.lemur/lemur.conf.py"
DEFAULT_SETTINGS = "lemur.conf.server"
SETTINGS_ENVVAR = "LEMUR_CONF"
2015-06-22 22:47:27 +02:00
CONFIG_TEMPLATE = """
# This is just Python which means you can inherit and tweak settings
import os
_basedir = os.path.abspath(os.path.dirname(__file__))
THREADS_PER_PAGE = 8
2015-07-22 19:51:55 +02:00
# General
2015-06-22 22:47:27 +02:00
# These will need to be set to `True` if you are developing locally
CORS = False
debug = False
2015-07-21 18:50:33 +02:00
# this is the secret key used by flask session management
SECRET_KEY = '{flask_secret_key}'
2015-06-22 22:47:27 +02:00
# You should consider storing these separately from your config
2015-07-22 19:51:55 +02:00
LEMUR_TOKEN_SECRET = '{secret_token}'
LEMUR_ENCRYPTION_KEYS = '{encryption_key}'
2015-06-22 22:47:27 +02:00
# List of domain regular expressions that non-admin users can issue
2020-10-10 03:04:16 +02:00
LEMUR_ALLOWED_DOMAINS = []
2015-06-22 22:47:27 +02:00
2015-07-22 19:51:55 +02:00
# Mail Server
2015-06-22 22:47:27 +02:00
LEMUR_EMAIL = ''
LEMUR_SECURITY_TEAM_EMAIL = []
# Certificate Defaults
LEMUR_DEFAULT_COUNTRY = ''
LEMUR_DEFAULT_STATE = ''
LEMUR_DEFAULT_LOCATION = ''
LEMUR_DEFAULT_ORGANIZATION = ''
LEMUR_DEFAULT_ORGANIZATIONAL_UNIT = ''
# Authentication Providers
ACTIVE_PROVIDERS = []
# Metrics Providers
METRIC_PROVIDERS = []
2015-07-22 19:51:55 +02:00
# Logging
2015-06-22 22:47:27 +02:00
LOG_LEVEL = "DEBUG"
LOG_FILE = "lemur.log"
2015-07-22 19:51:55 +02:00
# Database
2015-06-22 22:47:27 +02:00
2015-07-22 19:51:55 +02:00
# modify this if you are not using a local database
SQLALCHEMY_DATABASE_URI = 'postgresql://lemur:lemur@localhost:5432/lemur'
2015-06-22 22:47:27 +02:00
2015-07-22 19:51:55 +02:00
# AWS
2015-06-22 22:47:27 +02:00
#LEMUR_INSTANCE_PROFILE = 'Lemur'
2015-07-23 22:46:54 +02:00
# Issuers
2015-06-22 22:47:27 +02:00
# These will be dependent on which 3rd party that Lemur is
# configured to use.
2015-07-23 22:46:54 +02:00
# VERISIGN_URL = ''
# VERISIGN_PEM_PATH = ''
# VERISIGN_FIRST_NAME = ''
# VERISIGN_LAST_NAME = ''
# VERSIGN_EMAIL = ''
2015-06-22 22:47:27 +02:00
"""
2015-07-21 22:06:13 +02:00
2015-06-22 22:47:27 +02:00
@MigrateCommand.command
def create():
2019-05-16 16:57:02 +02:00
database.db.engine.execute(text("CREATE EXTENSION IF NOT EXISTS pg_trgm"))
2015-06-22 22:47:27 +02:00
database.db.create_all()
2019-05-16 16:57:02 +02:00
stamp(revision="head")
2015-06-22 22:47:27 +02:00
@MigrateCommand.command
def drop_all():
database.db.drop_all()
2015-06-22 22:47:27 +02:00
@manager.shell
def make_shell_context():
"""
Creates a python REPL with several default imports
in the context of the current_app
:return:
"""
return dict(current_app=current_app)
def generate_settings():
"""
This command is run when ``default_path`` doesn't exist, or ``init`` is
run and returns a string representing the default data to put into their
settings file.
"""
output = CONFIG_TEMPLATE.format(
# we use Fernet.generate_key to make sure that the key length is
# compatible with Fernet
2019-05-16 16:57:02 +02:00
encryption_key=Fernet.generate_key().decode("utf-8"),
secret_token=base64.b64encode(os.urandom(KEY_LENGTH)).decode("utf-8"),
flask_secret_key=base64.b64encode(os.urandom(KEY_LENGTH)).decode("utf-8"),
2015-06-22 22:47:27 +02:00
)
return output
class InitializeApp(Command):
"""
This command will bootstrap our database with any destinations as
2015-06-22 22:47:27 +02:00
specified by our config.
Additionally a Lemur user will be created as a default user
and be used when certificates are discovered by Lemur.
"""
2019-05-16 16:57:02 +02:00
option_list = (Option("-p", "--password", dest="password"),)
2015-09-03 23:20:51 +02:00
def run(self, password):
2015-06-22 22:47:27 +02:00
create()
user = user_service.get_by_username("lemur")
2019-05-16 16:57:02 +02:00
admin_role = role_service.get_by_name("admin")
if admin_role:
sys.stdout.write("[-] Admin role already created, skipping...!\n")
else:
# we create an admin role
2019-05-16 16:57:02 +02:00
admin_role = role_service.create(
"admin", description="This is the Lemur administrator role."
)
sys.stdout.write("[+] Created 'admin' role\n")
2019-05-16 16:57:02 +02:00
operator_role = role_service.get_by_name("operator")
if operator_role:
sys.stdout.write("[-] Operator role already created, skipping...!\n")
else:
# we create an operator role
2019-05-16 16:57:02 +02:00
operator_role = role_service.create(
"operator", description="This is the Lemur operator role."
)
sys.stdout.write("[+] Created 'operator' role\n")
2019-05-16 16:57:02 +02:00
read_only_role = role_service.get_by_name("read-only")
if read_only_role:
sys.stdout.write("[-] Read only role already created, skipping...!\n")
else:
# we create an read only role
2019-05-16 16:57:02 +02:00
read_only_role = role_service.create(
"read-only", description="This is the Lemur read only role."
)
sys.stdout.write("[+] Created 'read-only' role\n")
2015-06-22 22:47:27 +02:00
if not user:
2015-09-03 23:20:51 +02:00
if not password:
sys.stdout.write("We need to set Lemur's password to continue!\n")
password = prompt_pass("Password")
password1 = prompt_pass("Confirm Password")
2015-09-03 23:20:51 +02:00
if password != password1:
2015-09-03 23:20:51 +02:00
sys.stderr.write("[!] Passwords do not match!\n")
sys.exit(1)
2015-06-22 22:47:27 +02:00
2019-05-16 16:57:02 +02:00
user_service.create(
"lemur", password, "lemur@nobody.com", True, None, [admin_role]
)
sys.stdout.write(
"[+] Created the user 'lemur' and granted it the 'admin' role!\n"
)
2015-06-22 22:47:27 +02:00
else:
2019-05-16 16:57:02 +02:00
sys.stdout.write(
"[-] Default user has already been created, skipping...!\n"
)
2015-06-22 22:47:27 +02:00
2019-05-16 16:57:02 +02:00
intervals = current_app.config.get(
"LEMUR_DEFAULT_EXPIRATION_NOTIFICATION_INTERVALS", []
)
sys.stdout.write(
"[!] Creating {num} notifications for {intervals} days as specified by LEMUR_DEFAULT_EXPIRATION_NOTIFICATION_INTERVALS\n".format(
2019-05-16 16:57:02 +02:00
num=len(intervals), intervals=",".join([str(x) for x in intervals])
2015-08-02 04:08:46 +02:00
)
)
2015-08-02 04:08:46 +02:00
2019-05-16 16:57:02 +02:00
recipients = current_app.config.get("LEMUR_SECURITY_TEAM_EMAIL")
sys.stdout.write("[+] Creating expiration email notifications!\n")
2019-05-16 16:57:02 +02:00
sys.stdout.write(
"[!] Using {0} as specified by LEMUR_SECURITY_TEAM_EMAIL for notifications\n".format(
recipients
)
)
notification_service.create_default_expiration_notifications(
"DEFAULT_SECURITY", recipients=recipients
)
2015-08-02 04:08:46 +02:00
2019-05-16 16:57:02 +02:00
_DEFAULT_ROTATION_INTERVAL = "default"
default_rotation_interval = policy_service.get_by_name(
_DEFAULT_ROTATION_INTERVAL
)
if default_rotation_interval:
2019-05-16 16:57:02 +02:00
sys.stdout.write(
"[-] Default rotation interval policy already created, skipping...!\n"
)
else:
days = current_app.config.get("LEMUR_DEFAULT_ROTATION_INTERVAL", 30)
2019-05-16 16:57:02 +02:00
sys.stdout.write(
"[+] Creating default certificate rotation policy of {days} days before issuance.\n".format(
days=days
)
)
policy_service.create(days=days, name=_DEFAULT_ROTATION_INTERVAL)
2015-06-22 22:47:27 +02:00
sys.stdout.write("[/] Done!\n")
class CreateUser(Command):
"""
This command allows for the creation of a new user within Lemur.
2015-06-22 22:47:27 +02:00
"""
2019-05-16 16:57:02 +02:00
2015-06-22 22:47:27 +02:00
option_list = (
2019-05-16 16:57:02 +02:00
Option("-u", "--username", dest="username", required=True),
Option("-e", "--email", dest="email", required=True),
Option("-a", "--active", dest="active", default=True),
Option("-r", "--roles", dest="roles", action="append", default=[]),
Option("-p", "--password", dest="password", default=None),
2015-06-22 22:47:27 +02:00
)
def run(self, username, email, active, roles, password):
2015-06-22 22:47:27 +02:00
role_objs = []
for r in roles:
role_obj = role_service.get_by_name(r)
if role_obj:
role_objs.append(role_obj)
else:
sys.stderr.write("[!] Cannot find role {0}\n".format(r))
2015-06-22 22:47:27 +02:00
sys.exit(1)
if not password:
password1 = prompt_pass("Password")
password2 = prompt_pass("Confirm Password")
password = password1
2015-06-22 22:47:27 +02:00
if password1 != password2:
sys.stderr.write("[!] Passwords do not match!\n")
sys.exit(1)
2015-06-22 22:47:27 +02:00
user_service.create(username, password, email, active, None, role_objs)
sys.stdout.write("[+] Created new user: {0}\n".format(username))
class ResetPassword(Command):
"""
This command allows you to reset a user's password.
"""
2019-05-16 16:57:02 +02:00
option_list = (Option("-u", "--username", dest="username", required=True),)
def run(self, username):
user = user_service.get_by_username(username)
if not user:
sys.stderr.write("[!] No user found for username: {0}\n".format(username))
sys.exit(1)
sys.stderr.write("[+] Resetting password for {0}\n".format(username))
password1 = prompt_pass("Password")
password2 = prompt_pass("Confirm Password")
if password1 != password2:
sys.stderr.write("[!] Passwords do not match\n")
sys.exit(1)
user.password = password1
user.hash_password()
database.commit()
2015-06-22 22:47:27 +02:00
class CreateRole(Command):
"""
This command allows for the creation of a new role within Lemur
"""
2019-05-16 16:57:02 +02:00
2015-06-22 22:47:27 +02:00
option_list = (
2019-05-16 16:57:02 +02:00
Option("-n", "--name", dest="name", required=True),
Option("-u", "--users", dest="users", default=[]),
Option("-d", "--description", dest="description", required=True),
2015-06-22 22:47:27 +02:00
)
def run(self, name, users, description):
user_objs = []
for u in users:
user_obj = user_service.get_by_username(u)
if user_obj:
user_objs.append(user_obj)
else:
sys.stderr.write("[!] Cannot find user {0}".format(u))
sys.exit(1)
role_service.create(name, description=description, users=users)
sys.stdout.write("[+] Created new role: {0}".format(name))
class LemurServer(Command):
"""
This is the main Lemur server, it runs the flask app with gunicorn and
uses any configuration options passed to it.
You can pass all standard gunicorn flags to this command as if you were
running gunicorn itself.
For example:
lemur start -w 4 -b 127.0.0.0:8002
Will start gunicorn with 4 workers bound to 127.0.0.0:8002
"""
2019-05-16 16:57:02 +02:00
description = "Run the app within Gunicorn"
2015-06-22 22:47:27 +02:00
def get_options(self):
settings = make_settings()
options = []
for setting, klass in settings.items():
if klass.cli:
if klass.action:
2019-05-16 16:57:02 +02:00
if klass.action == "store_const":
options.append(
Option(*klass.cli, const=klass.const, action=klass.action)
)
else:
options.append(Option(*klass.cli, action=klass.action))
else:
options.append(Option(*klass.cli))
2015-06-22 22:47:27 +02:00
return options
def run(self, *args, **kwargs):
from gunicorn.app.wsgiapp import WSGIApplication
app = WSGIApplication()
# run startup tasks on an app like object
validate_conf(current_app, REQUIRED_VARIABLES)
2019-05-16 16:57:02 +02:00
app.app_uri = 'lemur:create_app(config_path="{0}")'.format(
current_app.config.get("CONFIG_PATH")
)
2015-06-22 22:47:27 +02:00
return app.run()
@manager.command
def create_config(config_path=None):
"""
Creates a new configuration file if one does not already exist
"""
if not config_path:
config_path = DEFAULT_CONFIG_PATH
config_path = os.path.expanduser(config_path)
dir = os.path.dirname(config_path)
if not os.path.exists(dir):
os.makedirs(dir)
config = generate_settings()
2019-05-16 16:57:02 +02:00
with open(config_path, "w") as f:
f.write(config)
sys.stdout.write("[+] Created a new configuration file {0}\n".format(config_path))
@manager.command
def lock(path=None):
"""
Encrypts a given path. This directory can be used to store secrets needed for normal
Lemur operation. This is especially useful for storing secrets needed for communication
with third parties (e.g. external certificate authorities).
Lemur does not assume anything about the contents of the directory and will attempt to
encrypt all files contained within. Currently this has only been tested against plain
text files.
Path defaults ~/.lemur/keys
:param: path
"""
if not path:
2019-05-16 16:57:02 +02:00
path = os.path.expanduser("~/.lemur/keys")
dest_dir = os.path.join(path, "encrypted")
sys.stdout.write("[!] Generating a new key...\n")
key = Fernet.generate_key()
if not os.path.exists(dest_dir):
sys.stdout.write("[+] Creating encryption directory: {0}\n".format(dest_dir))
os.makedirs(dest_dir)
2019-05-16 16:57:02 +02:00
for root, dirs, files in os.walk(os.path.join(path, "decrypted")):
for f in files:
source = os.path.join(root, f)
dest = os.path.join(dest_dir, f + ".enc")
2019-05-16 16:57:02 +02:00
with open(source, "rb") as in_file, open(dest, "wb") as out_file:
f = Fernet(key)
data = f.encrypt(in_file.read())
out_file.write(data)
2019-05-16 16:57:02 +02:00
sys.stdout.write(
"[+] Writing file: {0} Source: {1}\n".format(dest, source)
)
sys.stdout.write("[+] Keys have been encrypted with key {0}\n".format(key))
@manager.command
def unlock(path=None):
"""
Decrypts all of the files in a given directory with provided password.
This is most commonly used during the startup sequence of Lemur
allowing it to go from source code to something that can communicate
with external services.
Path defaults ~/.lemur/keys
:param: path
"""
key = prompt_pass("[!] Please enter the encryption password")
if not path:
2019-05-16 16:57:02 +02:00
path = os.path.expanduser("~/.lemur/keys")
dest_dir = os.path.join(path, "decrypted")
source_dir = os.path.join(path, "encrypted")
if not os.path.exists(dest_dir):
sys.stdout.write("[+] Creating decryption directory: {0}\n".format(dest_dir))
os.makedirs(dest_dir)
for root, dirs, files in os.walk(source_dir):
for f in files:
source = os.path.join(source_dir, f)
dest = os.path.join(dest_dir, ".".join(f.split(".")[:-1]))
2019-05-16 16:57:02 +02:00
with open(source, "rb") as in_file, open(dest, "wb") as out_file:
f = Fernet(key)
data = f.decrypt(in_file.read())
out_file.write(data)
2019-05-16 16:57:02 +02:00
sys.stdout.write(
"[+] Writing file: {0} Source: {1}\n".format(dest, source)
)
sys.stdout.write("[+] Keys have been unencrypted!\n")
@manager.command
def publish_verisign_units():
"""
Simple function that queries verisign for API units and posts the mertics to
Atlas API for other teams to consume.
:return:
"""
from lemur.plugins import plugins
2019-05-16 16:57:02 +02:00
v = plugins.get("verisign-issuer")
units = v.get_available_units()
metrics = {}
for item in units:
2019-05-16 16:57:02 +02:00
if item["@type"] in metrics.keys():
metrics[item["@type"]] += int(item["@remaining"])
else:
2019-05-16 16:57:02 +02:00
metrics.update({item["@type"]: int(item["@remaining"])})
for name, value in metrics.items():
metric = [
{
"timestamp": 1321351651,
"type": "GAUGE",
"name": "Symantec {0} Unit Count".format(name),
"tags": {},
2019-05-16 16:57:02 +02:00
"value": value,
}
]
2019-05-16 16:57:02 +02:00
requests.post("http://localhost:8078/metrics", data=json.dumps(metric))
def main():
2015-06-22 22:47:27 +02:00
manager.add_command("start", LemurServer())
2019-05-16 16:57:02 +02:00
manager.add_command("runserver", Server(host="127.0.0.1", threaded=True))
2015-06-22 22:47:27 +02:00
manager.add_command("clean", Clean())
manager.add_command("show_urls", ShowUrls())
manager.add_command("db", MigrateCommand)
manager.add_command("init", InitializeApp())
2015-07-21 01:13:42 +02:00
manager.add_command("create_user", CreateUser())
manager.add_command("reset_password", ResetPassword())
2015-07-21 01:13:42 +02:00
manager.add_command("create_role", CreateRole())
manager.add_command("source", source_manager)
manager.add_command("certificate", certificate_manager)
2016-12-08 18:11:19 +01:00
manager.add_command("notify", notification_manager)
manager.add_command("endpoint", endpoint_manager)
manager.add_command("report", report_manager)
manager.add_command("policy", policy_manager)
Async Certificate Issuing using Pending Certificates (#1037) * Add PendingCertificate model This change creates a DB table called pending_certificates and associated mapping relationship tables from pending certificate to roles, rotation policy, destination, sources, etc. The table is generated on initialization of Lemur. A pending certificate holds most of the information of a Certificate, while it has not be issued so that it can later backfill the information when the CA has issued the certificate. Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0 Tickets: PBL-36499 * Create a PendingCertificate if cert is empty IssuePlugins should return empty cert bodies if the request failed to complete immediately (such as Digicert). This way, we can immediately return the certificate, or if not just place into PendingCertificates for later processing. + Fix relation from Certificate to Pending Certificate, as view only. There is no real need for anything more than that since Pending cert only needs to know the cert to replace when it is issued later. + Made PendingCertificate private key be empty: UI does not allow private key on 'Create' but only on 'Import'. For Instart, we require the private key but upstream does not necessarily need it. Thus, if someone at Instart wants to create a CSR / key combo, they should manually issue the cert themselves and import later. Otherwise you should let Lemur generate that. This keeps the workflow transparent for upstream Lemur users. Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26 Tickets: PBL-36343 * Fix empty private_key when create Pending Cert On creation of a certificate with a CSR, there is no option for private key. In this case, we actually have a dictionary with private_key as key, but the value is None. This fixes the strip() called on NoneType. Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199 Tickets: PBL-36499 * Source sync finds and uses pending certificate When a source syncs certificates, it will check for a pending certificate. If that is found via external_id (given by digicert as order_id) then it will use the found Pending Certificate's fields to create a new certificate. Then the pending certificate is deleted. Tickets: PBL-36343 Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9 * Add Lemur static files and views for pending certs This adds the basic static files to view pending certificates in a table. Tickets: PBL-36343 Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9 * Add CLI and plugin based pending fetch This change uses the adds a new function to issuer plugins to fetch certificates like source, but for one order. This way, we can control which pending certificates to try and populate instead of getting all certificates from source. Tickets: PBL-36343 Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6 * Revert source using Pending Certificate Tickets: PBL-36343 Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51 * Fix PendingCertificate init getting authority id Should get authority id from authority.id instead of the authority_id key in kwargs. Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623 Tickets: n/a * Add fixtures and basic test for PendingCertificate Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34 Tickets: PBL-36343 * Add User to create_certificate parameters create_certificate now takes a User, which will be used to populate the 'creator' field in certificates.service.upload(). This allows the UI populate with the current user if the owner does not exist in Lemur. + Fix chain being replaced with version from pending certificate, which may be empty (depends on plugin implementation). Change-Id: I516027b36bc643c4978b9c4890060569e03f3049 Tickets: n/a * Fix permalink and filters to pending certs Fixes the permalink button to get a single pending certificate Add argument filter parsing for the pending certificate API Fix comment on API usage Added get_by_name for pending_certificate (currently unused, but useful for CLI, instead of using IDs) Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db Tickets: PBL-36910 * Update displayed fields for Pending Certificates There are a number of unused / unpopulated fields from Certificate UI that does apply to Pending Certificates. Those ones were removed, and added other useful fields: Owner, number of attempts to fetch and date created Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b Tickets: PBL-36910 * Add common name (cn) to Pending Certificate model Fixes the UI missing the CN for Pending Certificate, as it was originally being parsed from the generated certificate. In the case of pending certificate, the CN from the user generates the request, which means a pending cert can trust the original user putting in the CN instead of having to parse the not-yet-generated certificate. There is no real possibility to return a certificate from a pending certificate where the CN has changed since it was initially ordered. Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab Tickets: PBL-36910 * Fix missing imports for service filter + Removed duplicate get_by_name function from old merge Change-Id: I04ae6852533aa42988433338de74390e2868d69b Tickets: PBL-36910 * Add private key viewing to Pending Certificates Add private key API for Pending Certificates, with the same authorization as Certificates (only owner, creator or owner-roles can view private key). Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b Tickets: PBL-36910 * Add edit capability to pending certificates Like editing certificates, we should be able to modify some parts of a pending certificate so the resulting certificate has the right references, owner, etc. + Added API to update pending certificate + Fix UI to use pending certificate scope instead of reusing Certificate + Change pending_certificate.replaces to non-passive association, so that updates do affect it (similar to roles/notifications/etc) Tickets: PBL-36910 Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6 * Add common_name parsing instead using kwargs To fix tests where common name may not be passed in, use the CSR generated to find the official common name. Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4 Tickets: PBL-36343 * Add Cancel to pending certificates and plugins This allows pending certificates to be cancelled, which will be handled by the issuer plugin. Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9 Tickets: PBL-36910 * Add API for Cancelling Pending Certificate Added the DELETE handler for pending_certificates, which will cancel and delete the pending certificate from the pending certs table on successful cancellation via Issuer Plugin. + Add UT for testing cancel API Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4 Tickets: PBL-36910 * Remove Export from Pending Certificates Pending Certificates doesn't need an export since it should just be fetched by Lemur via plugins, and the CSR is viewable via the UI. Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae Tickets: PBL-36910 * Add cancel button functionality to UI This adds the Cancel option to the dropdown of pending certificates. + Adds modal window for Note (may not be required for all issuers, just Digicert) + Add schema for cancel input + Fix Digitcert plugin for non-existant orders When an order is actually issued, then attempting to cancel will return a 403 from Digicert. This is a case where it should only be done once we know the pending cert has been sitting for too long. Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0 Tickets: PBL-36910 * Fix test_pending_cancel UT This change creates and injects a pending cert, which will then be used for the ID so it can be canceled by the unit test. Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346 Tickets: PBL-36343 * Fix test_digicert on non-existent order cancelling a non-existent order is fine since we're cancelling it Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c Tickets: PBL-36343 * Add migrations for PendingCertificates Added revision for Pending Certificates table and foreign key mapping tables. Change-Id: Ife8202cef1e6b99db377851264639ba540b749db Tickets: n/a * Fix relationship copy from Pending to Certificate When a Pending Certificate is changed to a full Certificate, the relationship fields are not copied via vars() function, as it's not a column but mapped via association table. This adds an explicit copy for these relations. Which will properly copy them to the new Certificate, and thus also update destinations. Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685 Tickets: PBL-36343 * Fix renaming of certificates and unit tests The rename flag was not used to rename certificates on creation as expected. Fixed unit test, instead of expunging the session, just copy the pending_certificate so we don't have a weird reference to the object that can't be copied via vars() function. Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0 Tickets: PBL-36343 * Updated developer docs for async certs Added blurb for implementing new issuer functions. Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c Tickets: n/a
2018-02-22 17:13:16 +01:00
manager.add_command("pending_certs", pending_certificate_manager)
manager.add_command("dns_providers", dns_provider_manager)
manager.add_command("acme", acme_manager)
2015-06-22 22:47:27 +02:00
manager.run()
2016-11-21 23:29:20 +01:00
if __name__ == "__main__":
main()