Infra
/
lemur
5
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge branch 'master' into feature/acme-http-challenge

This commit is contained in:
Mathias Petermann 2020-11-03 09:36:37 +01:00
parent 23e1700fad 5569c9d8e1
commit bc564b574d
44 changed files with 676 additions and 286 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.travis.yml
View File

@ -47,4 +47,7 @@ after_success:
notifications: notifications:
email: email:
ccastrapel@netflix.com recipients:
- lemur@netflix.com
on_success: never
on_failure: always

59
docs/administration.rst
View File

@ -28,6 +28,13 @@ Basic Configuration
LOG_FILE = "/logs/lemur/lemur-test.log" LOG_FILE = "/logs/lemur/lemur-test.log"
.. data:: LOG_UPGRADE_FILE
:noindex:
::
LOG_UPGRADE_FILE = "/logs/lemur/db_upgrade.log"
.. data:: DEBUG .. data:: DEBUG
:noindex: :noindex:
@ -269,7 +276,7 @@ Certificates marked as inactive will **not** be notified of upcoming expiration.
silence the expiration. If a certificate is active and is expiring the above will be notified according to the `LEMUR_DEFAULT_EXPIRATION_NOTIFICATION_INTERVALS` or silence the expiration. If a certificate is active and is expiring the above will be notified according to the `LEMUR_DEFAULT_EXPIRATION_NOTIFICATION_INTERVALS` or
30, 15, 2 days before expiration if no intervals are set. 30, 15, 2 days before expiration if no intervals are set.
Lemur supports sending certification expiration notifications through SES and SMTP. Lemur supports sending certificate expiration notifications through SES and SMTP.
.. data:: LEMUR_EMAIL_SENDER .. data:: LEMUR_EMAIL_SENDER
@ -285,6 +292,25 @@ Lemur supports sending certification expiration notifications through SES and SM
you can send any mail. See: `Verifying Email Address in Amazon SES <http://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-email-addresses.html>`_ you can send any mail. See: `Verifying Email Address in Amazon SES <http://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-email-addresses.html>`_
.. data:: LEMUR_SES_SOURCE_ARN
:noindex:
Specifies an ARN to use as the SourceArn when sending emails via SES.
.. note::
This parameter is only required if you're using a sending authorization with SES.
See: `Using sending authorization with Amazon SES <https://docs.aws.amazon.com/ses/latest/DeveloperGuide/sending-authorization.html>`_
.. data:: LEMUR_SES_REGION
:noindex:
Specifies a region for sending emails via SES.
.. note::
This parameter defaults to us-east-1 and is only required if you wish to use a different region.
.. data:: LEMUR_EMAIL .. data:: LEMUR_EMAIL
:noindex: :noindex:
@ -664,6 +690,20 @@ If you are not using a metric provider you do not need to configure any of these
Plugin Specific Options Plugin Specific Options
----------------------- -----------------------
ACME Plugin
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. data:: ACME_DNS_PROVIDER_TYPES
:noindex:
Dictionary of ACME DNS Providers and their requirements.
.. data:: ACME_ENABLE_DELEGATED_CNAME
:noindex:
Enables delegated DNS domain validation using CNAMES. When enabled, Lemur will attempt to follow CNAME records to authoritative DNS servers when creating DNS-01 challenges.
Active Directory Certificate Services Plugin Active Directory Certificate Services Plugin
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -1436,7 +1476,7 @@ Slack
Adds support for slack notifications. Adds support for slack notifications.
AWS AWS (Source)
---- ----
:Authors: :Authors:
@ -1449,7 +1489,7 @@ AWS
Uses AWS IAM as a source of certificates to manage. Supports a multi-account deployment. Uses AWS IAM as a source of certificates to manage. Supports a multi-account deployment.
AWS AWS (Destination)
---- ----
:Authors: :Authors:
@ -1462,6 +1502,19 @@ AWS
Uses AWS IAM as a destination for Lemur generated certificates. Support a multi-account deployment. Uses AWS IAM as a destination for Lemur generated certificates. Support a multi-account deployment.
AWS (SNS Notification)
-----
:Authors:
Jasmine Schladen <jschladen@netflix.com>
:Type:
Notification
:Description:
Adds support for SNS notifications. SNS notifications (like other notification plugins) are currently only supported
for certificate expiration. Configuration requires a region, account number, and SNS topic name; these elements
are then combined to build the topic ARN. Lemur must have access to publish messages to the specified SNS topic.
Kubernetes Kubernetes
---------- ----------

13
docs/developer/plugins/index.rst
View File

@ -215,18 +215,21 @@ Notification
------------ ------------
Lemur includes the ability to create Email notifications by **default**. These notifications Lemur includes the ability to create Email notifications by **default**. These notifications
currently come in the form of expiration notices. Lemur periodically checks certifications expiration dates and currently come in the form of expiration and rotation notices. Lemur periodically checks certificate expiration dates and
determines if a given certificate is eligible for notification. There are currently only two parameters used to determines if a given certificate is eligible for notification. There are currently only two parameters used to
determine if a certificate is eligible; validity expiration (date the certificate is no longer valid) and the number determine if a certificate is eligible; validity expiration (date the certificate is no longer valid) and the number
of days the current date (UTC) is from that expiration date. of days the current date (UTC) is from that expiration date.
There are currently two objects that available for notification plugins the first is `NotficationPlugin`. This is the base object for Expiration notifications can also be configured for Slack or AWS SNS. Rotation notifications are not configurable.
any notification within Lemur. Currently the only support notification type is an certificate expiration notification. If you Notifications sent to a certificate owner and security team (`LEMUR_SECURITY_TEAM_EMAIL`) can currently only be sent via email.
There are currently two objects that are available for notification plugins. The first is `NotificationPlugin`, which is the base object for
any notification within Lemur. Currently the only supported notification type is a certificate expiration notification. If you
are trying to create a new notification type (audit, failed logins, etc.) this would be the object to base your plugin on. are trying to create a new notification type (audit, failed logins, etc.) this would be the object to base your plugin on.
You would also then need to build additional code to trigger the new notification type. You would also then need to build additional code to trigger the new notification type.
The second is `ExpirationNotificationPlugin`, this object inherits from `NotificationPlugin` object. The second is `ExpirationNotificationPlugin`, which inherits from the `NotificationPlugin` object.
You will most likely want to base your plugin on, if you want to add new channels for expiration notices (Slack, HipChat, Jira, etc.). It adds default options that are required by You will most likely want to base your plugin on this object if you want to add new channels for expiration notices (HipChat, Jira, etc.). It adds default options that are required by
all expiration notifications (interval, unit). This interface expects for the child to define the following function:: all expiration notifications (interval, unit). This interface expects for the child to define the following function::
def send(self, notification_type, message, targets, options, **kwargs): def send(self, notification_type, message, targets, options, **kwargs):

3
gulp/build.js
View File

@ -237,7 +237,7 @@ gulp.task('addUrlContextPath',['addUrlContextPath:revreplace'], function(){
.forEach(function(file){ .forEach(function(file){
return gulp.src(file) return gulp.src(file)
.pipe(gulpif(urlContextPathExists, replace('api/', argv.urlContextPath + '/api/'))) .pipe(gulpif(urlContextPathExists, replace('api/', argv.urlContextPath + '/api/')))
.pipe(gulpif(urlContextPathExists, replace('angular/', argv.urlContextPath + '/angular/'))) .pipe(gulpif(urlContextPathExists, replace('/angular/', '/' + argv.urlContextPath + '/angular/')))
.pipe(gulp.dest(function(file){ .pipe(gulp.dest(function(file){
return file.base; return file.base;
})) }))
@ -256,7 +256,6 @@ gulp.task('addUrlContextPath:revreplace', ['addUrlContextPath:revision'], functi
var manifest = gulp.src("lemur/static/dist/rev-manifest.json"); var manifest = gulp.src("lemur/static/dist/rev-manifest.json");
var urlContextPathExists = argv.urlContextPath ? true : false; var urlContextPathExists = argv.urlContextPath ? true : false;
return gulp.src( "lemur/static/dist/index.html") return gulp.src( "lemur/static/dist/index.html")
.pipe(gulpif(urlContextPathExists, revReplace({prefix: argv.urlContextPath + '/', manifest: manifest}, revReplace({manifest: manifest}))))
.pipe(gulp.dest('lemur/static/dist')); .pipe(gulp.dest('lemur/static/dist'));
}) })

8
lemur/authorities/models.py
View File

@ -93,9 +93,11 @@ class Authority(db.Model):
if not self.options: if not self.options:
return None return None
for option in json.loads(self.options): options_array = json.loads(self.options)
if "name" in option and option["name"] == 'cab_compliant': if isinstance(options_array, list):
return option["value"] for option in options_array:
if "name" in option and option["name"] == 'cab_compliant':
return option["value"]
return None return None

35
lemur/certificates/service.py
View File

@ -6,6 +6,7 @@
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com> .. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
""" """
import arrow import arrow
import re
from cryptography import x509 from cryptography import x509
from cryptography.hazmat.backends import default_backend from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes, serialization from cryptography.hazmat.primitives import hashes, serialization
@ -560,20 +561,21 @@ def query_common_name(common_name, args):
:return: :return:
""" """
owner = args.pop("owner") owner = args.pop("owner")
if not owner:
owner = "%"
# only not expired certificates # only not expired certificates
current_time = arrow.utcnow() current_time = arrow.utcnow()
result = ( query = Certificate.query.filter(Certificate.not_after >= current_time.format("YYYY-MM-DD"))\
Certificate.query.filter(Certificate.cn.ilike(common_name)) .filter(not_(Certificate.revoked))\
.filter(Certificate.owner.ilike(owner)) .filter(not_(Certificate.replaced.any())) # ignore rotated certificates to avoid duplicates
.filter(Certificate.not_after >= current_time.format("YYYY-MM-DD"))
.all()
)
return result if owner:
query = query.filter(Certificate.owner.ilike(owner))
if common_name != "%":
# if common_name is a wildcard ('%'), no need to include it in the query
query = query.filter(Certificate.cn.ilike(common_name))
return query.all()
def create_csr(**csr_config): def create_csr(**csr_config):
@ -778,6 +780,19 @@ def reissue_certificate(certificate, replace=None, user=None):
if replace: if replace:
primitives["replaces"] = [certificate] primitives["replaces"] = [certificate]
# Modify description to include the certificate ID being reissued and mention that this is created by Lemur
# as part of reissue
reissue_message_prefix = "Reissued by Lemur for cert ID "
reissue_message = re.compile(f"{reissue_message_prefix}([0-9]+)")
if primitives["description"]:
match = reissue_message.search(primitives["description"])
if match:
primitives["description"] = primitives["description"].replace(match.group(1), str(certificate.id))
else:
primitives["description"] = f"{reissue_message_prefix}{certificate.id}, {primitives['description']}"
else:
primitives["description"] = f"{reissue_message_prefix}{certificate.id}"
new_cert = create(**primitives) new_cert = create(**primitives)
return new_cert return new_cert

1
lemur/certificates/views.py
View File

@ -1155,6 +1155,7 @@ class NotificationCertificatesList(AuthenticatedResource):
) )
parser.add_argument("creator", type=str, location="args") parser.add_argument("creator", type=str, location="args")
parser.add_argument("show", type=str, location="args") parser.add_argument("show", type=str, location="args")
parser.add_argument("showExpired", type=int, location="args")
args = parser.parse_args() args = parser.parse_args()
args["notification_id"] = notification_id args["notification_id"] = notification_id

3
lemur/destinations/schemas.py
View File

@ -31,6 +31,9 @@ class DestinationOutputSchema(LemurOutputSchema):
def fill_object(self, data): def fill_object(self, data):
if data: if data:
data["plugin"]["pluginOptions"] = data["options"] data["plugin"]["pluginOptions"] = data["options"]
for option in data["plugin"]["pluginOptions"]:
if "export-plugin" in option["type"]:
option["value"]["pluginOptions"] = option["value"]["plugin_options"]
return data return data

9
lemur/destinations/service.py
View File

@ -41,12 +41,14 @@ def create(label, plugin_name, options, description=None):
return database.create(destination) return database.create(destination)
def update(destination_id, label, options, description): def update(destination_id, label, plugin_name, options, description):
""" """
Updates an existing destination. Updates an existing destination.
:param destination_id: Lemur assigned ID :param destination_id: Lemur assigned ID
:param label: Destination common name :param label: Destination common name
:param plugin_name:
:param options:
:param description: :param description:
:rtype : Destination :rtype : Destination
:return: :return:
@ -54,6 +56,11 @@ def update(destination_id, label, options, description):
destination = get(destination_id) destination = get(destination_id)
destination.label = label destination.label = label
destination.plugin_name = plugin_name
# remove any sub-plugin objects before try to save the json options
for option in options:
if "plugin" in option["type"]:
del option["value"]["plugin_object"]
destination.options = options destination.options = options
destination.description = description destination.description = description

1
lemur/destinations/views.py
View File

@ -338,6 +338,7 @@ class Destinations(AuthenticatedResource):
return service.update( return service.update(
destination_id, destination_id,
data["label"], data["label"],
data["plugin"]["slug"],
data["plugin"]["plugin_options"], data["plugin"]["plugin_options"],
data["description"], data["description"],
) )

1
lemur/manage.py
View File

@ -120,6 +120,7 @@ METRIC_PROVIDERS = []
LOG_LEVEL = "DEBUG" LOG_LEVEL = "DEBUG"
LOG_FILE = "lemur.log" LOG_FILE = "lemur.log"
LOG_UPGRADE_FILE = "db_upgrade.log"
# Database # Database

16
lemur/migrations/versions/1db4f82bc780_.py
View File

@ -10,11 +10,21 @@ Create Date: 2018-08-03 12:56:44.565230
revision = "1db4f82bc780" revision = "1db4f82bc780"
down_revision = "3adfdd6598df" down_revision = "3adfdd6598df"
import logging
from alembic import op from alembic import op
log = logging.getLogger(__name__) from flask import current_app
from logging import Formatter, FileHandler, getLogger
log = getLogger(__name__)
handler = FileHandler(current_app.config.get("LOG_UPGRADE_FILE", "db_upgrade.log"))
handler.setFormatter(
Formatter(
"%(asctime)s %(levelname)s: %(message)s " "[in %(pathname)s:%(lineno)d]"
)
)
handler.setLevel(current_app.config.get("LOG_LEVEL", "DEBUG"))
log.setLevel(current_app.config.get("LOG_LEVEL", "DEBUG"))
log.addHandler(handler)
def upgrade(): def upgrade():

40
lemur/migrations/versions/c301c59688d2_.py
View File

@ -7,8 +7,9 @@ the rest of the keys, the certificate body is parsed to determine
the exact key_type information. the exact key_type information.
Each individual DB change is explicitly committed, and the respective Each individual DB change is explicitly committed, and the respective
log is added to a file named db_upgrade.log in the current working log is added to a file configured in LOG_UPGRADE_FILE or, by default,
directory. Any error encountered while parsing a certificate will to a file named db_upgrade.log in the current working directory.
Any error encountered while parsing a certificate will
also be logged along with the certificate ID. If faced with any issue also be logged along with the certificate ID. If faced with any issue
while running this upgrade, there is no harm in re-running the upgrade. while running this upgrade, there is no harm in re-running the upgrade.
Each run processes only rows for which key_type information is not yet Each run processes only rows for which key_type information is not yet
@ -31,15 +32,28 @@ down_revision = '434c29e40511'
from alembic import op from alembic import op
from sqlalchemy.sql import text from sqlalchemy.sql import text
from lemur.common import utils
import time import time
import datetime import datetime
from flask import current_app
log_file = open('db_upgrade.log', 'a') from logging import Formatter, FileHandler, getLogger
from lemur.common import utils
log = getLogger(__name__)
handler = FileHandler(current_app.config.get("LOG_UPGRADE_FILE", "db_upgrade.log"))
handler.setFormatter(
Formatter(
"%(asctime)s %(levelname)s: %(message)s " "[in %(pathname)s:%(lineno)d]"
)
)
handler.setLevel(current_app.config.get("LOG_LEVEL", "DEBUG"))
log.setLevel(current_app.config.get("LOG_LEVEL", "DEBUG"))
log.addHandler(handler)
def upgrade(): def upgrade():
log_file.write("\n*** Starting new run(%s) ***\n" % datetime.datetime.now()) log.info("\n*** Starting new run(%s) ***\n" % datetime.datetime.now())
start_time = time.time() start_time = time.time()
# Update RSA keys using the key length information # Update RSA keys using the key length information
@ -50,8 +64,7 @@ def upgrade():
# Process remaining certificates. Though below method does not make any assumptions, most of the remaining ones should be ECC certs. # Process remaining certificates. Though below method does not make any assumptions, most of the remaining ones should be ECC certs.
update_key_type() update_key_type()
log_file.write("--- Total %s seconds ---\n" % (time.time() - start_time)) log.info("--- Total %s seconds ---\n" % (time.time() - start_time))
log_file.close()
def downgrade(): def downgrade():
@ -61,6 +74,7 @@ def downgrade():
"update certificates set key_type=null where not_after > CURRENT_DATE - 32" "update certificates set key_type=null where not_after > CURRENT_DATE - 32"
) )
op.execute(stmt) op.execute(stmt)
commit()
""" """
@ -69,18 +83,18 @@ def downgrade():
def update_key_type_rsa(bits): def update_key_type_rsa(bits):
log_file.write("Processing certificate with key type RSA %s\n" % bits) log.info("Processing certificate with key type RSA %s\n" % bits)
stmt = text( stmt = text(
f"update certificates set key_type='RSA{bits}' where bits={bits} and not_after > CURRENT_DATE - 31 and key_type is null" f"update certificates set key_type='RSA{bits}' where bits={bits} and not_after > CURRENT_DATE - 31 and key_type is null"
) )
log_file.write("Query: %s\n" % stmt) log.info("Query: %s\n" % stmt)
start_time = time.time() start_time = time.time()
op.execute(stmt) op.execute(stmt)
commit() commit()
log_file.write("--- %s seconds ---\n" % (time.time() - start_time)) log.info("--- %s seconds ---\n" % (time.time() - start_time))
def update_key_type(): def update_key_type():
@ -95,9 +109,9 @@ def update_key_type():
try: try:
cert_key_type = utils.get_key_type_from_certificate(body) cert_key_type = utils.get_key_type_from_certificate(body)
except ValueError as e: except ValueError as e:
log_file.write("Error in processing certificate - ID: %s Error: %s \n" % (cert_id, str(e))) log.error("Error in processing certificate - ID: %s Error: %s \n" % (cert_id, str(e)))
else: else:
log_file.write("Processing certificate - ID: %s key_type: %s\n" % (cert_id, cert_key_type)) log.info("Processing certificate - ID: %s key_type: %s\n" % (cert_id, cert_key_type))
stmt = text( stmt = text(
"update certificates set key_type=:key_type where id=:id" "update certificates set key_type=:key_type where id=:id"
) )
@ -106,7 +120,7 @@ def update_key_type():
commit() commit()
log_file.write("--- %s seconds ---\n" % (time.time() - start_time)) log.info("--- %s seconds ---\n" % (time.time() - start_time))
def commit(): def commit():

157
lemur/notifications/messaging.py
View File

@ -30,7 +30,7 @@ from lemur.plugins.utils import get_plugin_option
def get_certificates(exclude=None): def get_certificates(exclude=None):
""" """
Finds all certificates that are eligible for notifications. Finds all certificates that are eligible for expiration notifications.
:param exclude: :param exclude:
:return: :return:
""" """
@ -42,6 +42,7 @@ def get_certificates(exclude=None):
.filter(Certificate.not_after <= max) .filter(Certificate.not_after <= max)
.filter(Certificate.notify == True) .filter(Certificate.notify == True)
.filter(Certificate.expired == False) .filter(Certificate.expired == False)
.filter(Certificate.revoked == False)
) # noqa ) # noqa
exclude_conditions = [] exclude_conditions = []
@ -62,7 +63,8 @@ def get_certificates(exclude=None):
def get_eligible_certificates(exclude=None): def get_eligible_certificates(exclude=None):
""" """
Finds all certificates that are eligible for certificate expiration. Finds all certificates that are eligible for certificate expiration notification.
Returns the set of all eligible certificates, grouped by owner, with a list of applicable notifications.
:param exclude: :param exclude:
:return: :return:
""" """
@ -87,29 +89,30 @@ def get_eligible_certificates(exclude=None):
return certificates return certificates
def send_notification(event_type, data, targets, notification): def send_plugin_notification(event_type, data, recipients, notification):
""" """
Executes the plugin and handles failure. Executes the plugin and handles failure.
:param event_type: :param event_type:
:param data: :param data:
:param targets: :param recipients:
:param notification: :param notification:
:return: :return:
""" """
function = f"{__name__}.{sys._getframe().f_code.co_name}" function = f"{__name__}.{sys._getframe().f_code.co_name}"
log_data = { log_data = {
"function": function, "function": function,
"message": f"Sending expiration notification for to targets {targets}", "message": f"Sending expiration notification for to recipients {recipients}",
"notification_type": "expiration", "notification_type": "expiration",
"certificate_targets": targets, "certificate_targets": recipients,
} }
status = FAILURE_METRIC_STATUS status = FAILURE_METRIC_STATUS
try: try:
notification.plugin.send(event_type, data, targets, notification.options) current_app.logger.debug(log_data)
notification.plugin.send(event_type, data, recipients, notification.options)
status = SUCCESS_METRIC_STATUS status = SUCCESS_METRIC_STATUS
except Exception as e: except Exception as e:
log_data["message"] = f"Unable to send expiration notification to targets {targets}" log_data["message"] = f"Unable to send {event_type} notification to recipients {recipients}"
current_app.logger.error(log_data, exc_info=True) current_app.logger.error(log_data, exc_info=True)
sentry.captureException() sentry.captureException()
@ -150,36 +153,27 @@ def send_expiration_notifications(exclude):
notification_data.append(cert_data) notification_data.append(cert_data)
security_data.append(cert_data) security_data.append(cert_data)
if send_notification( if send_default_notification(
"expiration", notification_data, [owner], notification "expiration", notification_data, [owner], notification.options
): ):
success += 1 success += 1
else: else:
failure += 1 failure += 1
notification_recipient = get_plugin_option( recipients = notification.plugin.filter_recipients(notification.options, security_email + [owner])
"recipients", notification.options
)
if notification_recipient:
notification_recipient = notification_recipient.split(",")
# removing owner and security_email from notification_recipient
notification_recipient = [i for i in notification_recipient if i not in security_email and i != owner]
if ( if send_plugin_notification(
notification_recipient "expiration",
notification_data,
recipients,
notification,
): ):
if send_notification( success += 1
"expiration", else:
notification_data, failure += 1
notification_recipient,
notification,
):
success += 1
else:
failure += 1
if send_notification( if send_default_notification(
"expiration", security_data, security_email, notification "expiration", security_data, security_email, notification.options
): ):
success += 1 success += 1
else: else:
@ -188,37 +182,36 @@ def send_expiration_notifications(exclude):
return success, failure return success, failure
def send_rotation_notification(certificate, notification_plugin=None): def send_default_notification(notification_type, data, targets, notification_options=None):
""" """
Sends a report to certificate owners when their certificate has been Sends a report to the specified target via the default notification plugin. Applicable for any notification_type.
rotated. At present, "default" means email, as the other notification plugins do not support dynamically configured targets.
:param certificate: :param notification_type:
:param notification_plugin: :param data:
:param targets:
:param notification_options:
:return: :return:
""" """
function = f"{__name__}.{sys._getframe().f_code.co_name}" function = f"{__name__}.{sys._getframe().f_code.co_name}"
log_data = { log_data = {
"function": function, "function": function,
"message": f"Sending rotation notification for certificate {certificate.name}", "message": f"Sending notification for certificate data {data}",
"notification_type": "rotation", "notification_type": notification_type,
"certificate_name": certificate.name,
"certificate_owner": certificate.owner,
} }
status = FAILURE_METRIC_STATUS status = FAILURE_METRIC_STATUS
if not notification_plugin: notification_plugin = plugins.get(
notification_plugin = plugins.get( current_app.config.get("LEMUR_DEFAULT_NOTIFICATION_PLUGIN", "email-notification")
current_app.config.get("LEMUR_DEFAULT_NOTIFICATION_PLUGIN", "email-notification") )
)
data = certificate_notification_output_schema.dump(certificate).data
try: try:
notification_plugin.send("rotation", data, [data["owner"]], []) current_app.logger.debug(log_data)
# we need the notification.options here because the email templates utilize the interval/unit info
notification_plugin.send(notification_type, data, targets, notification_options)
status = SUCCESS_METRIC_STATUS status = SUCCESS_METRIC_STATUS
except Exception as e: except Exception as e:
log_data["message"] = f"Unable to send rotation notification for certificate {certificate.name} " \ log_data["message"] = f"Unable to send {notification_type} notification for certificate data {data} " \
f"to owner {data['owner']}" f"to target {targets}"
current_app.logger.error(log_data, exc_info=True) current_app.logger.error(log_data, exc_info=True)
sentry.captureException() sentry.captureException()
@ -226,82 +219,49 @@ def send_rotation_notification(certificate, notification_plugin=None):
"notification", "notification",
"counter", "counter",
1, 1,
metric_tags={"status": status, "event_type": "rotation"}, metric_tags={"status": status, "event_type": notification_type},
) )
if status == SUCCESS_METRIC_STATUS: if status == SUCCESS_METRIC_STATUS:
return True return True
def send_rotation_notification(certificate):
data = certificate_notification_output_schema.dump(certificate).data
return send_default_notification("rotation", data, [data["owner"]])
def send_pending_failure_notification( def send_pending_failure_notification(
pending_cert, notify_owner=True, notify_security=True, notification_plugin=None pending_cert, notify_owner=True, notify_security=True
): ):
""" """
Sends a report to certificate owners when their pending certificate failed to be created. Sends a report to certificate owners when their pending certificate failed to be created.
:param pending_cert: :param pending_cert:
:param notification_plugin: :param notify_owner:
:param notify_security:
:return: :return:
""" """
function = f"{__name__}.{sys._getframe().f_code.co_name}"
log_data = {
"function": function,
"message": f"Sending pending failure notification for pending certificate {pending_cert}",
"notification_type": "failed",
"certificate_name": pending_cert.name,
"certificate_owner": pending_cert.owner,
}
status = FAILURE_METRIC_STATUS
if not notification_plugin:
notification_plugin = plugins.get(
current_app.config.get(
"LEMUR_DEFAULT_NOTIFICATION_PLUGIN", "email-notification"
)
)
data = pending_certificate_output_schema.dump(pending_cert).data data = pending_certificate_output_schema.dump(pending_cert).data
data["security_email"] = current_app.config.get("LEMUR_SECURITY_TEAM_EMAIL") data["security_email"] = current_app.config.get("LEMUR_SECURITY_TEAM_EMAIL")
notify_owner_success = False
if notify_owner: if notify_owner:
try: notify_owner_success = send_default_notification("failed", data, [data["owner"]], pending_cert)
notification_plugin.send("failed", data, [data["owner"]], pending_cert)
status = SUCCESS_METRIC_STATUS
except Exception as e:
log_data["recipient"] = data["owner"]
log_data["message"] = f"Unable to send pending failure notification for certificate {pending_cert.name} " \
f"to owner {pending_cert.owner}"
current_app.logger.error(log_data, exc_info=True)
sentry.captureException()
notify_security_success = False
if notify_security: if notify_security:
try: notify_security_success = send_default_notification("failed", data, data["security_email"], pending_cert)
notification_plugin.send(
"failed", data, data["security_email"], pending_cert
)
status = SUCCESS_METRIC_STATUS
except Exception as e:
log_data["recipient"] = data["security_email"]
log_data["message"] = f"Unable to send pending failure notification for certificate {pending_cert.name} " \
f"to security email {pending_cert.owner}"
current_app.logger.error(log_data, exc_info=True)
sentry.captureException()
metrics.send( return notify_owner_success or notify_security_success
"notification",
"counter",
1,
metric_tags={"status": status, "event_type": "failed"},
)
if status == SUCCESS_METRIC_STATUS:
return True
def needs_notification(certificate): def needs_notification(certificate):
""" """
Determine if notifications for a given certificate should Determine if notifications for a given certificate should currently be sent.
currently be sent For each notification configured for the cert, verifies it is active, properly configured,
and that the configured expiration period is currently met.
:param certificate: :param certificate:
:return: :return:
@ -331,7 +291,6 @@ def needs_notification(certificate):
raise Exception( raise Exception(
f"Invalid base unit for expiration interval: {unit}" f"Invalid base unit for expiration interval: {unit}"
) )
if days == interval: if days == interval:
notifications.append(notification) notifications.append(notification)
return notifications return notifications

4
lemur/notifications/service.py
View File

@ -104,12 +104,13 @@ def create(label, plugin_name, options, description, certificates):
return database.create(notification) return database.create(notification)
def update(notification_id, label, options, description, active, certificates): def update(notification_id, label, plugin_name, options, description, active, certificates):
""" """
Updates an existing notification. Updates an existing notification.
:param notification_id: :param notification_id:
:param label: Notification label :param label: Notification label
:param plugin_name:
:param options: :param options:
:param description: :param description:
:param active: :param active:
@ -120,6 +121,7 @@ def update(notification_id, label, options, description, active, certificates):
notification = get(notification_id) notification = get(notification_id)
notification.label = label notification.label = label
notification.plugin_name = plugin_name
notification.options = options notification.options = options
notification.description = description notification.description = description
notification.active = active notification.active = active

1
lemur/notifications/views.py
View File

@ -340,6 +340,7 @@ class Notifications(AuthenticatedResource):
return service.update( return service.update(
notification_id, notification_id,
data["label"], data["label"],
data["plugin"]["slug"],
data["plugin"]["plugin_options"], data["plugin"]["plugin_options"],
data["description"], data["description"],
data["active"], data["active"],

11
lemur/plugins/bases/notification.py
View File

@ -20,6 +20,15 @@ class NotificationPlugin(Plugin):
def send(self, notification_type, message, targets, options, **kwargs): def send(self, notification_type, message, targets, options, **kwargs):
raise NotImplementedError raise NotImplementedError
def filter_recipients(self, options, excluded_recipients):
"""
Given a set of options (which should include configured recipient info), filters out recipients that
we do NOT want to notify.
For any notification types where recipients can't be dynamically modified, this returns an empty list.
"""
return []
class ExpirationNotificationPlugin(NotificationPlugin): class ExpirationNotificationPlugin(NotificationPlugin):
""" """
@ -50,5 +59,5 @@ class ExpirationNotificationPlugin(NotificationPlugin):
def options(self): def options(self):
return self.default_options + self.additional_options return self.default_options + self.additional_options
def send(self, notification_type, message, targets, options, **kwargs): def send(self, notification_type, message, excluded_targets, options, **kwargs):
raise NotImplementedError raise NotImplementedError

87
lemur/plugins/lemur_acme/acme_handlers.py
View File

@ -18,6 +18,7 @@ import time
import OpenSSL.crypto import OpenSSL.crypto
import josepy as jose import josepy as jose
import dns.resolver
from acme import challenges, errors, messages from acme import challenges, errors, messages
from acme.client import BackwardsCompatibleClientV2, ClientNetwork from acme.client import BackwardsCompatibleClientV2, ClientNetwork
from acme.errors import TimeoutError from acme.errors import TimeoutError
@ -35,8 +36,9 @@ from retrying import retry
class AuthorizationRecord(object): class AuthorizationRecord(object):
def __init__(self, host, authz, dns_challenge, change_id): def __init__(self, domain, target_domain, authz, dns_challenge, change_id):
self.host = host self.domain = domain
self.target_domain = target_domain
self.authz = authz self.authz = authz
self.dns_challenge = dns_challenge self.dns_challenge = dns_challenge
self.change_id = change_id self.change_id = change_id
@ -270,19 +272,18 @@ class AcmeDnsHandler(AcmeHandler):
self, self,
acme_client, acme_client,
account_number, account_number,
host, domain,
target_domain,
dns_provider, dns_provider,
order, order,
dns_provider_options, dns_provider_options,
): ):
current_app.logger.debug("Starting DNS challenge for {0}".format(host)) current_app.logger.debug(f"Starting DNS challenge for {domain} using target domain {target_domain}.")
change_ids = [] change_ids = []
dns_challenges = self.get_dns_challenges(host, order.authorizations) dns_challenges = self.get_dns_challenges(domain, order.authorizations)
host_to_validate, _ = self.strip_wildcard(host) host_to_validate, _ = self.strip_wildcard(target_domain)
host_to_validate = self.maybe_add_extension( host_to_validate = self.maybe_add_extension(host_to_validate, dns_provider_options)
host_to_validate, dns_provider_options
)
if not dns_challenges: if not dns_challenges:
sentry.captureException() sentry.captureException()
@ -290,15 +291,20 @@ class AcmeDnsHandler(AcmeHandler):
raise Exception("Unable to determine DNS challenges from authorizations") raise Exception("Unable to determine DNS challenges from authorizations")
for dns_challenge in dns_challenges: for dns_challenge in dns_challenges:
# Only prepend '_acme-challenge' if not using CNAME redirection
if domain == target_domain:
host_to_validate = dns_challenge.validation_domain_name(host_to_validate)
change_id = dns_provider.create_txt_record( change_id = dns_provider.create_txt_record(
dns_challenge.validation_domain_name(host_to_validate), host_to_validate,
dns_challenge.validation(acme_client.client.net.key), dns_challenge.validation(acme_client.client.net.key),
account_number, account_number,
) )
change_ids.append(change_id) change_ids.append(change_id)
return AuthorizationRecord( return AuthorizationRecord(
host, order.authorizations, dns_challenges, change_ids domain, target_domain, order.authorizations, dns_challenges, change_ids
) )
def complete_dns_challenge(self, acme_client, authz_record): def complete_dns_challenge(self, acme_client, authz_record):
@ -307,11 +313,11 @@ class AcmeDnsHandler(AcmeHandler):
authz_record.authz[0].body.identifier.value authz_record.authz[0].body.identifier.value
) )
) )
dns_providers = self.dns_providers_for_domain.get(authz_record.host) dns_providers = self.dns_providers_for_domain.get(authz_record.target_domain)
if not dns_providers: if not dns_providers:
metrics.send("complete_dns_challenge_error_no_dnsproviders", "counter", 1) metrics.send("complete_dns_challenge_error_no_dnsproviders", "counter", 1)
raise Exception( raise Exception(
"No DNS providers found for domain: {}".format(authz_record.host) "No DNS providers found for domain: {}".format(authz_record.target_domain)
) )
for dns_provider in dns_providers: for dns_provider in dns_providers:
@ -339,7 +345,7 @@ class AcmeDnsHandler(AcmeHandler):
verified = response.simple_verify( verified = response.simple_verify(
dns_challenge.chall, dns_challenge.chall,
authz_record.host, authz_record.target_domain,
acme_client.client.net.key.public_key(), acme_client.client.net.key.public_key(),
) )
@ -355,12 +361,24 @@ class AcmeDnsHandler(AcmeHandler):
authorizations = [] authorizations = []
for domain in order_info.domains: for domain in order_info.domains:
if not self.dns_providers_for_domain.get(domain):
# If CNAME exists, set host to the target address
target_domain = domain
if current_app.config.get("ACME_ENABLE_DELEGATED_CNAME", False):
cname_result, _ = self.strip_wildcard(domain)
cname_result = challenges.DNS01().validation_domain_name(cname_result)
cname_result = self.get_cname(cname_result)
if cname_result:
target_domain = cname_result
self.autodetect_dns_providers(target_domain)
if not self.dns_providers_for_domain.get(target_domain):
metrics.send( metrics.send(
"get_authorizations_no_dns_provider_for_domain", "counter", 1 "get_authorizations_no_dns_provider_for_domain", "counter", 1
) )
raise Exception("No DNS providers found for domain: {}".format(domain)) raise Exception("No DNS providers found for domain: {}".format(target_domain))
for dns_provider in self.dns_providers_for_domain[domain]:
for dns_provider in self.dns_providers_for_domain[target_domain]:
dns_provider_plugin = self.get_dns_provider(dns_provider.provider_type) dns_provider_plugin = self.get_dns_provider(dns_provider.provider_type)
dns_provider_options = json.loads(dns_provider.credentials) dns_provider_options = json.loads(dns_provider.credentials)
account_number = dns_provider_options.get("account_id") account_number = dns_provider_options.get("account_id")
@ -368,6 +386,7 @@ class AcmeDnsHandler(AcmeHandler):
acme_client, acme_client,
account_number, account_number,
domain, domain,
target_domain,
dns_provider_plugin, dns_provider_plugin,
order, order,
dns_provider.options, dns_provider.options,
@ -399,11 +418,10 @@ class AcmeDnsHandler(AcmeHandler):
def finalize_authorizations(self, acme_client, authorizations): def finalize_authorizations(self, acme_client, authorizations):
for authz_record in authorizations: for authz_record in authorizations:
self.complete_dns_challenge(acme_client, authz_record) self.complete_dns_challenge(acme_client, authz_record)
for authz_record in authorizations: for authz_record in authorizations:
dns_challenges = authz_record.dns_challenge dns_challenges = authz_record.dns_challenge
for dns_challenge in dns_challenges: for dns_challenge in dns_challenges:
dns_providers = self.dns_providers_for_domain.get(authz_record.host) dns_providers = self.dns_providers_for_domain.get(authz_record.target_domain)
for dns_provider in dns_providers: for dns_provider in dns_providers:
# Grab account number (For Route53) # Grab account number (For Route53)
dns_provider_plugin = self.get_dns_provider( dns_provider_plugin = self.get_dns_provider(
@ -411,14 +429,14 @@ class AcmeDnsHandler(AcmeHandler):
) )
dns_provider_options = json.loads(dns_provider.credentials) dns_provider_options = json.loads(dns_provider.credentials)
account_number = dns_provider_options.get("account_id") account_number = dns_provider_options.get("account_id")
host_to_validate, _ = self.strip_wildcard(authz_record.host) host_to_validate, _ = self.strip_wildcard(authz_record.target_domain)
host_to_validate = self.maybe_add_extension( host_to_validate = self.maybe_add_extension(host_to_validate, dns_provider_options)
host_to_validate, dns_provider_options if authz_record.domain == authz_record.target_domain:
) host_to_validate = challenges.DNS01().validation_domain_name(host_to_validate)
dns_provider_plugin.delete_txt_record( dns_provider_plugin.delete_txt_record(
authz_record.change_id, authz_record.change_id,
account_number, account_number,
dns_challenge.validation_domain_name(host_to_validate), host_to_validate,
dns_challenge.validation(acme_client.client.net.key), dns_challenge.validation(acme_client.client.net.key),
) )
@ -437,23 +455,26 @@ class AcmeDnsHandler(AcmeHandler):
:return: :return:
""" """
for authz_record in authorizations: for authz_record in authorizations:
dns_providers = self.dns_providers_for_domain.get(authz_record.host) dns_providers = self.dns_providers_for_domain.get(authz_record.target_domain)
for dns_provider in dns_providers: for dns_provider in dns_providers:
# Grab account number (For Route53) # Grab account number (For Route53)
dns_provider_options = json.loads(dns_provider.credentials) dns_provider_options = json.loads(dns_provider.credentials)
account_number = dns_provider_options.get("account_id") account_number = dns_provider_options.get("account_id")
dns_challenges = authz_record.dns_challenge dns_challenges = authz_record.dns_challenge
host_to_validate, _ = self.strip_wildcard(authz_record.host) host_to_validate, _ = self.strip_wildcard(authz_record.target_domain)
host_to_validate = self.maybe_add_extension( host_to_validate = self.maybe_add_extension(
host_to_validate, dns_provider_options host_to_validate, dns_provider_options
) )
dns_provider_plugin = self.get_dns_provider(dns_provider.provider_type) dns_provider_plugin = self.get_dns_provider(dns_provider.provider_type)
for dns_challenge in dns_challenges: for dns_challenge in dns_challenges:
if authz_record.domain == authz_record.target_domain:
host_to_validate = dns_challenge.validation_domain_name(host_to_validate)
try: try:
dns_provider_plugin.delete_txt_record( dns_provider_plugin.delete_txt_record(
authz_record.change_id, authz_record.change_id,
account_number, account_number,
dns_challenge.validation_domain_name(host_to_validate), host_to_validate,
dns_challenge.validation(acme_client.client.net.key), dns_challenge.validation(acme_client.client.net.key),
) )
except Exception as e: except Exception as e:
@ -462,3 +483,15 @@ class AcmeDnsHandler(AcmeHandler):
metrics.send("cleanup_dns_challenges_error", "counter", 1) metrics.send("cleanup_dns_challenges_error", "counter", 1)
sentry.captureException() sentry.captureException()
pass pass
def get_cname(self, domain):
"""
:param domain: Domain name to look up a CNAME for.
:return: First CNAME target or False if no CNAME record exists.
"""
try:
result = dns.resolver.query(domain, 'CNAME')
if len(result) > 0:
return str(result[0].target).rstrip('.')
except dns.exception.DNSException:
return False

1
lemur/plugins/lemur_acme/plugin.py
View File

@ -19,7 +19,6 @@ from acme.errors import PollError, WildcardUnsupportedError
from acme.messages import Error as AcmeError from acme.messages import Error as AcmeError
from botocore.exceptions import ClientError from botocore.exceptions import ClientError
from flask import current_app from flask import current_app
from lemur.authorizations import service as authorization_service from lemur.authorizations import service as authorization_service
from lemur.dns_providers import service as dns_provider_service from lemur.dns_providers import service as dns_provider_service
from lemur.exceptions import InvalidConfiguration from lemur.exceptions import InvalidConfiguration

27
lemur/plugins/lemur_acme/tests/test_acme_dns.py
View File

@ -40,6 +40,19 @@ class TestAcmeDns(unittest.TestCase):
result = yield self.acme.get_dns_challenges(host, mock_authz) result = yield self.acme.get_dns_challenges(host, mock_authz)
self.assertEqual(result, mock_entry) self.assertEqual(result, mock_entry)
def test_strip_wildcard(self):
expected = ("example.com", False)
result = self.acme.strip_wildcard("example.com")
self.assertEqual(expected, result)
expected = ("example.com", True)
result = self.acme.strip_wildcard("*.example.com")
self.assertEqual(expected, result)
def test_authz_record(self):
a = AuthorizationRecord("domain", "host", "authz", "challenge", "id")
self.assertEqual(type(a), AuthorizationRecord)
@patch("acme.client.Client") @patch("acme.client.Client")
@patch("lemur.plugins.lemur_acme.acme_handlers.current_app") @patch("lemur.plugins.lemur_acme.acme_handlers.current_app")
@patch("lemur.plugins.lemur_acme.plugin.len", return_value=1) @patch("lemur.plugins.lemur_acme.plugin.len", return_value=1)
@ -67,7 +80,7 @@ class TestAcmeDns(unittest.TestCase):
iterator = iter(values) iterator = iter(values)
iterable.__iter__.return_value = iterator iterable.__iter__.return_value = iterator
result = self.acme.start_dns_challenge( result = self.acme.start_dns_challenge(
mock_acme, "accountid", "host", mock_dns_provider, mock_order, {} mock_acme, "accountid", "domain", "host", mock_dns_provider, mock_order, {}
) )
self.assertEqual(type(result), AuthorizationRecord) self.assertEqual(type(result), AuthorizationRecord)
@ -85,7 +98,7 @@ class TestAcmeDns(unittest.TestCase):
mock_authz.dns_challenge.response = Mock() mock_authz.dns_challenge.response = Mock()
mock_authz.dns_challenge.response.simple_verify = Mock(return_value=True) mock_authz.dns_challenge.response.simple_verify = Mock(return_value=True)
mock_authz.authz = [] mock_authz.authz = []
mock_authz.host = "www.test.com" mock_authz.target_domain = "www.test.com"
mock_authz_record = Mock() mock_authz_record = Mock()
mock_authz_record.body.identifier.value = "test" mock_authz_record.body.identifier.value = "test"
mock_authz.authz.append(mock_authz_record) mock_authz.authz.append(mock_authz_record)
@ -109,7 +122,7 @@ class TestAcmeDns(unittest.TestCase):
mock_authz.dns_challenge.response = Mock() mock_authz.dns_challenge.response = Mock()
mock_authz.dns_challenge.response.simple_verify = Mock(return_value=False) mock_authz.dns_challenge.response.simple_verify = Mock(return_value=False)
mock_authz.authz = [] mock_authz.authz = []
mock_authz.host = "www.test.com" mock_authz.target_domain = "www.test.com"
mock_authz_record = Mock() mock_authz_record = Mock()
mock_authz_record.body.identifier.value = "test" mock_authz_record.body.identifier.value = "test"
mock_authz.authz.append(mock_authz_record) mock_authz.authz.append(mock_authz_record)
@ -330,11 +343,9 @@ class TestAcmeDns(unittest.TestCase):
result = provider.create_certificate(csr, issuer_options) result = provider.create_certificate(csr, issuer_options)
assert result assert result
@patch( @patch("lemur.plugins.lemur_acme.plugin.AcmeDnsHandler.start_dns_challenge", return_value="test")
"lemur.plugins.lemur_acme.plugin.AcmeDnsHandler.start_dns_challenge", @patch("lemur.plugins.lemur_acme.acme_handlers.current_app", return_value=False)
return_value="test", def test_get_authorizations(self, mock_current_app, mock_start_dns_challenge):
)
def test_get_authorizations(self, mock_start_dns_challenge):
mock_order = Mock() mock_order = Mock()
mock_order.body.identifiers = [] mock_order.body.identifiers = []
mock_domain = Mock() mock_domain = Mock()

55
lemur/plugins/lemur_aws/plugin.py
View File

@ -32,13 +32,14 @@
.. moduleauthor:: Mikhail Khodorovskiy <mikhail.khodorovskiy@jivesoftware.com> .. moduleauthor:: Mikhail Khodorovskiy <mikhail.khodorovskiy@jivesoftware.com>
.. moduleauthor:: Harm Weites <harm@weites.com> .. moduleauthor:: Harm Weites <harm@weites.com>
""" """
from acme.errors import ClientError from acme.errors import ClientError
from flask import current_app from flask import current_app
from lemur.extensions import sentry, metrics
from lemur.plugins import lemur_aws as aws from lemur.extensions import sentry, metrics
from lemur.plugins import lemur_aws as aws, ExpirationNotificationPlugin
from lemur.plugins.bases import DestinationPlugin, ExportDestinationPlugin, SourcePlugin from lemur.plugins.bases import DestinationPlugin, ExportDestinationPlugin, SourcePlugin
from lemur.plugins.lemur_aws import iam, s3, elb, ec2 from lemur.plugins.lemur_aws import iam, s3, elb, ec2, sns
def get_region_from_dns(dns): def get_region_from_dns(dns):
@ -406,3 +407,51 @@ class S3DestinationPlugin(ExportDestinationPlugin):
self.get_option("encrypt", options), self.get_option("encrypt", options),
account_number=self.get_option("accountNumber", options), account_number=self.get_option("accountNumber", options),
) )
class SNSNotificationPlugin(ExpirationNotificationPlugin):
title = "AWS SNS"
slug = "aws-sns"
description = "Sends notifications to AWS SNS"
version = aws.VERSION
author = "Jasmine Schladen <jschladen@netflix.com>"
author_url = "https://github.com/Netflix/lemur"
additional_options = [
{
"name": "accountNumber",
"type": "str",
"required": True,
"validation": "[0-9]{12}",
"helpMessage": "A valid AWS account number with permission to access the SNS topic",
},
{
"name": "region",
"type": "str",
"required": True,
"validation": "[0-9a-z\\-]{1,25}",
"helpMessage": "Region in which the SNS topic is located, e.g. \"us-east-1\"",
},
{
"name": "topicName",
"type": "str",
"required": True,
# base topic name is 1-256 characters (alphanumeric plus underscore and hyphen)
"validation": "^[a-zA-Z0-9_\\-]{1,256}$",
"helpMessage": "The name of the topic to use for expiration notifications",
}
]
def send(self, notification_type, message, excluded_targets, options, **kwargs):
"""
While we receive a `targets` parameter here, it is unused, as the SNS topic is pre-configured in the
plugin configuration, and can't reasonably be changed dynamically.
"""
topic_arn = f"arn:aws:sns:{self.get_option('region', options)}:" \
f"{self.get_option('accountNumber', options)}:" \
f"{self.get_option('topicName', options)}"
current_app.logger.info(f"Publishing {notification_type} notification to topic {topic_arn}")
sns.publish(topic_arn, message, notification_type, region_name=self.get_option("region", options))

58
lemur/plugins/lemur_aws/sns.py Normal file
View File

@ -0,0 +1,58 @@
"""
.. module: lemur.plugins.lemur_aws.sts
:platform: Unix
:copyright: (c) 2020 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Jasmine Schladen <jschladen@netflix.com>
"""
import json
import arrow
import boto3
from flask import current_app
def publish(topic_arn, certificates, notification_type, **kwargs):
sns_client = boto3.client("sns", **kwargs)
message_ids = {}
subject = "Lemur: {0} Notification".format(notification_type.capitalize())
for certificate in certificates:
message_ids[certificate["name"]] = publish_single(sns_client, topic_arn, certificate, notification_type, subject)
return message_ids
def publish_single(sns_client, topic_arn, certificate, notification_type, subject):
response = sns_client.publish(
TopicArn=topic_arn,
Message=format_message(certificate, notification_type),
Subject=subject,
)
response_code = response["ResponseMetadata"]["HTTPStatusCode"]
if response_code != 200:
raise Exception(f"Failed to publish {notification_type} notification to SNS topic {topic_arn}. "
f"SNS response: {response_code} {response}")
current_app.logger.info(f"AWS SNS message published to topic [{topic_arn}] with message ID {response['MessageId']}")
current_app.logger.debug(f"AWS SNS message published to topic [{topic_arn}]: [{response}]")
return response["MessageId"]
def create_certificate_url(name):
return "https://{hostname}/#/certificates/{name}".format(
hostname=current_app.config.get("LEMUR_HOSTNAME"), name=name
)
def format_message(certificate, notification_type):
json_message = {
"notification_type": notification_type,
"certificate_name": certificate["name"],
"expires": arrow.get(certificate["validityEnd"]).format("YYYY-MM-DDTHH:mm:ss"), # 2047-12-31T22:00:00
"endpoints_detected": len(certificate["endpoints"]),
"owner": certificate["owner"],
"details": create_certificate_url(certificate["name"])
}
return json.dumps(json_message)

123
lemur/plugins/lemur_aws/tests/test_sns.py Normal file
View File

@ -0,0 +1,123 @@
import json
from datetime import timedelta
import arrow
import boto3
from moto import mock_sns, mock_sqs, mock_ses
from lemur.certificates.schemas import certificate_notification_output_schema
from lemur.plugins.lemur_aws.sns import format_message
from lemur.plugins.lemur_aws.sns import publish
from lemur.tests.factories import NotificationFactory, CertificateFactory
from lemur.tests.test_messaging import verify_sender_email
@mock_sns()
def test_format(certificate, endpoint):
data = [certificate_notification_output_schema.dump(certificate).data]
for certificate in data:
expected_message = {
"notification_type": "expiration",
"certificate_name": certificate["name"],
"expires": arrow.get(certificate["validityEnd"]).format("YYYY-MM-DDTHH:mm:ss"),
"endpoints_detected": 0,
"owner": certificate["owner"],
"details": "https://lemur.example.com/#/certificates/{name}".format(name=certificate["name"])
}
assert expected_message == json.loads(format_message(certificate, "expiration"))
@mock_sns()
@mock_sqs()
def create_and_subscribe_to_topic():
sns_client = boto3.client("sns", region_name="us-east-1")
topic_arn = sns_client.create_topic(Name='lemursnstest')["TopicArn"]
sqs_client = boto3.client("sqs", region_name="us-east-1")
queue = sqs_client.create_queue(QueueName="lemursnstestqueue")
queue_url = queue["QueueUrl"]
queue_arn = sqs_client.get_queue_attributes(QueueUrl=queue_url)["Attributes"]["QueueArn"]
sns_client.subscribe(TopicArn=topic_arn, Protocol="sqs", Endpoint=queue_arn)
return [topic_arn, sqs_client, queue_url]
@mock_sns()
@mock_sqs()
def test_publish(certificate, endpoint):
data = [certificate_notification_output_schema.dump(certificate).data]
topic_arn, sqs_client, queue_url = create_and_subscribe_to_topic()
message_ids = publish(topic_arn, data, "expiration", region_name="us-east-1")
assert len(message_ids) == len(data)
received_messages = sqs_client.receive_message(QueueUrl=queue_url)["Messages"]
for certificate in data:
expected_message_id = message_ids[certificate["name"]]
actual_message = next(
(m for m in received_messages if json.loads(m["Body"])["MessageId"] == expected_message_id), None)
actual_json = json.loads(actual_message["Body"])
assert actual_json["Message"] == format_message(certificate, "expiration")
assert actual_json["Subject"] == "Lemur: Expiration Notification"
def get_options():
return [
{"name": "interval", "value": 10},
{"name": "unit", "value": "days"},
{"name": "region", "value": "us-east-1"},
{"name": "accountNumber", "value": "123456789012"},
{"name": "topicName", "value": "lemursnstest"},
]
@mock_sns()
@mock_sqs()
@mock_ses() # because email notifications are also sent
def test_send_expiration_notification():
from lemur.notifications.messaging import send_expiration_notifications
verify_sender_email() # emails are sent to owner and security; SNS only used for configured notification
topic_arn, sqs_client, queue_url = create_and_subscribe_to_topic()
notification = NotificationFactory(plugin_name="aws-sns")
notification.options = get_options()
now = arrow.utcnow()
in_ten_days = now + timedelta(days=10, hours=1) # a bit more than 10 days since we'll check in the future
certificate = CertificateFactory()
certificate.not_after = in_ten_days
certificate.notifications.append(notification)
assert send_expiration_notifications([]) == (3, 0) # owner, SNS, and security
received_messages = sqs_client.receive_message(QueueUrl=queue_url)["Messages"]
assert len(received_messages) == 1
expected_message = format_message(certificate_notification_output_schema.dump(certificate).data, "expiration")
actual_message = json.loads(received_messages[0]["Body"])["Message"]
assert actual_message == expected_message
# Currently disabled as the SNS plugin doesn't support this type of notification
# def test_send_rotation_notification(endpoint, source_plugin):
# from lemur.notifications.messaging import send_rotation_notification
# from lemur.deployment.service import rotate_certificate
#
# notification = NotificationFactory(plugin_name="aws-sns")
# notification.options = get_options()
#
# new_certificate = CertificateFactory()
# rotate_certificate(endpoint, new_certificate)
# assert endpoint.certificate == new_certificate
#
# assert send_rotation_notification(new_certificate)
# Currently disabled as the SNS plugin doesn't support this type of notification
# def test_send_pending_failure_notification(user, pending_certificate, async_issuer_plugin):
# from lemur.notifications.messaging import send_pending_failure_notification
#
# assert send_pending_failure_notification(pending_certificate)

4
lemur/plugins/lemur_digicert/plugin.py
View File

@ -234,7 +234,7 @@ def handle_cis_response(response):
return response.json() return response.json()
@retry(stop_max_attempt_number=10, wait_fixed=10000) @retry(stop_max_attempt_number=10, wait_fixed=1000)
def get_certificate_id(session, base_url, order_id): def get_certificate_id(session, base_url, order_id):
"""Retrieve certificate order id from Digicert API.""" """Retrieve certificate order id from Digicert API."""
order_url = "{0}/services/v2/order/certificate/{1}".format(base_url, order_id) order_url = "{0}/services/v2/order/certificate/{1}".format(base_url, order_id)
@ -245,7 +245,7 @@ def get_certificate_id(session, base_url, order_id):
return response_data["certificate"]["id"] return response_data["certificate"]["id"]
@retry(stop_max_attempt_number=10, wait_fixed=10000) @retry(stop_max_attempt_number=10, wait_fixed=1000)
def get_cis_certificate(session, base_url, order_id): def get_cis_certificate(session, base_url, order_id):
"""Retrieve certificate order id from Digicert API, including the chain""" """Retrieve certificate order id from Digicert API, including the chain"""
certificate_url = "{0}/platform/cis/certificate/{1}/download".format(base_url, order_id) certificate_url = "{0}/platform/cis/certificate/{1}/download".format(base_url, order_id)

32
lemur/plugins/lemur_email/plugin.py
View File

@ -17,6 +17,7 @@ from lemur.plugins.bases import ExpirationNotificationPlugin
from lemur.plugins import lemur_email as email from lemur.plugins import lemur_email as email
from lemur.plugins.lemur_email.templates.config import env from lemur.plugins.lemur_email.templates.config import env
from lemur.plugins.utils import get_plugin_option
def render_html(template_name, options, certificates): def render_html(template_name, options, certificates):
@ -37,7 +38,7 @@ def render_html(template_name, options, certificates):
def send_via_smtp(subject, body, targets): def send_via_smtp(subject, body, targets):
""" """
Attempts to deliver email notification via SES service. Attempts to deliver email notification via SMTP.
:param subject: :param subject:
:param body: :param body:
@ -54,21 +55,26 @@ def send_via_smtp(subject, body, targets):
def send_via_ses(subject, body, targets): def send_via_ses(subject, body, targets):
""" """
Attempts to deliver email notification via SMTP. Attempts to deliver email notification via SES service.
:param subject: :param subject:
:param body: :param body:
:param targets: :param targets:
:return: :return:
""" """
client = boto3.client("ses", region_name="us-east-1") ses_region = current_app.config.get("LEMUR_SES_REGION", "us-east-1")
client.send_email( client = boto3.client("ses", region_name=ses_region)
Source=current_app.config.get("LEMUR_EMAIL"), source_arn = current_app.config.get("LEMUR_SES_SOURCE_ARN")
Destination={"ToAddresses": targets}, args = {
Message={ "Source": current_app.config.get("LEMUR_EMAIL"),
"Destination": {"ToAddresses": targets},
"Message": {
"Subject": {"Data": subject, "Charset": "UTF-8"}, "Subject": {"Data": subject, "Charset": "UTF-8"},
"Body": {"Html": {"Data": body, "Charset": "UTF-8"}}, "Body": {"Html": {"Data": body, "Charset": "UTF-8"}},
}, },
) }
if source_arn:
args["SourceArn"] = source_arn
client.send_email(**args)
class EmailNotificationPlugin(ExpirationNotificationPlugin): class EmailNotificationPlugin(ExpirationNotificationPlugin):
@ -111,3 +117,13 @@ class EmailNotificationPlugin(ExpirationNotificationPlugin):
elif s_type == "smtp": elif s_type == "smtp":
send_via_smtp(subject, body, targets) send_via_smtp(subject, body, targets)
@staticmethod
def filter_recipients(options, excluded_recipients, **kwargs):
notification_recipients = get_plugin_option("recipients", options)
if notification_recipients:
notification_recipients = notification_recipients.split(",")
# removing owner and security_email from notification_recipient
notification_recipients = [i for i in notification_recipients if i not in excluded_recipients]
return notification_recipients

23
lemur/plugins/lemur_email/tests/test_email.py
View File

@ -2,26 +2,21 @@ import os
from datetime import timedelta from datetime import timedelta
import arrow import arrow
import boto3
from moto import mock_ses from moto import mock_ses
from lemur.certificates.schemas import certificate_notification_output_schema from lemur.certificates.schemas import certificate_notification_output_schema
from lemur.plugins.lemur_email.plugin import render_html from lemur.plugins.lemur_email.plugin import render_html
from lemur.tests.factories import CertificateFactory from lemur.tests.factories import CertificateFactory
from lemur.tests.test_messaging import verify_sender_email
dir_path = os.path.dirname(os.path.realpath(__file__)) dir_path = os.path.dirname(os.path.realpath(__file__))
@mock_ses
def verify_sender_email():
ses_client = boto3.client("ses", region_name="us-east-1")
ses_client.verify_email_identity(EmailAddress="lemur@example.com")
def get_options(): def get_options():
return [ return [
{"name": "interval", "value": 10}, {"name": "interval", "value": 10},
{"name": "unit", "value": "days"}, {"name": "unit", "value": "days"},
{"name": "recipients", "value": "person1@example.com,person2@example.com"},
] ]
@ -59,7 +54,7 @@ def test_send_expiration_notification():
certificate.notifications[0].options = get_options() certificate.notifications[0].options = get_options()
verify_sender_email() verify_sender_email()
assert send_expiration_notifications([]) == (2, 0) assert send_expiration_notifications([]) == (3, 0) # owner, recipients (only counted as 1), and security
@mock_ses @mock_ses
@ -81,3 +76,15 @@ def test_send_pending_failure_notification(user, pending_certificate, async_issu
verify_sender_email() verify_sender_email()
assert send_pending_failure_notification(pending_certificate) assert send_pending_failure_notification(pending_certificate)
def test_filter_recipients(certificate, endpoint):
from lemur.plugins.lemur_email.plugin import EmailNotificationPlugin
options = [{"name": "recipients", "value": "security@example.com,bob@example.com,joe@example.com"}]
assert EmailNotificationPlugin.filter_recipients(options, []) == ["security@example.com", "bob@example.com",
"joe@example.com"]
assert EmailNotificationPlugin.filter_recipients(options, ["security@example.com"]) == ["bob@example.com",
"joe@example.com"]
assert EmailNotificationPlugin.filter_recipients(options, ["security@example.com", "bob@example.com",
"joe@example.com"]) == []

57
lemur/plugins/lemur_entrust/plugin.py
View File

@ -1,9 +1,9 @@
import arrow import arrow
import requests import requests
import json import json
import sys import sys
from flask import current_app from flask import current_app
from retrying import retry
from lemur.plugins import lemur_entrust as entrust from lemur.plugins import lemur_entrust as entrust
from lemur.plugins.bases import IssuerPlugin, SourcePlugin from lemur.plugins.bases import IssuerPlugin, SourcePlugin
@ -78,7 +78,6 @@ def process_options(options):
"eku": "SERVER_AND_CLIENT_AUTH", "eku": "SERVER_AND_CLIENT_AUTH",
"certType": product_type, "certType": product_type,
"certExpiryDate": validity_end, "certExpiryDate": validity_end,
# "keyType": "RSA", Entrust complaining about this parameter
"tracking": tracking_data "tracking": tracking_data
} }
return data return data
@ -87,7 +86,7 @@ def process_options(options):
def handle_response(my_response): def handle_response(my_response):
""" """
Helper function for parsing responses from the Entrust API. Helper function for parsing responses from the Entrust API.
:param content: :param my_response:
:return: :raise Exception: :return: :raise Exception:
""" """
msg = { msg = {
@ -100,27 +99,47 @@ def handle_response(my_response):
} }
try: try:
d = json.loads(my_response.content) data = json.loads(my_response.content)
except ValueError: except ValueError:
# catch an empty jason object here # catch an empty jason object here
d = {'response': 'No detailed message'} data = {'response': 'No detailed message'}
s = my_response.status_code status_code = my_response.status_code
if s > 399: if status_code > 399:
raise Exception(f"ENTRUST error: {msg.get(s, s)}\n{d['errors']}") raise Exception(f"ENTRUST error: {msg.get(status_code, status_code)}\n{data['errors']}")
log_data = { log_data = {
"function": f"{__name__}.{sys._getframe().f_code.co_name}", "function": f"{__name__}.{sys._getframe().f_code.co_name}",
"message": "Response", "message": "Response",
"status": s, "status": status_code,
"response": d "response": data
} }
current_app.logger.info(log_data) current_app.logger.info(log_data)
if d == {'response': 'No detailed message'}: if data == {'response': 'No detailed message'}:
# status if no data # status if no data
return s return status_code
else: else:
# return data from the response # return data from the response
return d return data
@retry(stop_max_attempt_number=3, wait_fixed=5000)
def order_and_download_certificate(session, url, data):
"""
Helper function to place a certificacte order and download it
:param session:
:param url: Entrust endpoint url
:param data: CSR, and the required order details, such as validity length
:return: the cert chain
:raise Exception:
"""
try:
response = session.post(url, json=data, timeout=(15, 40))
except requests.exceptions.Timeout:
raise Exception("Timeout for POST")
except requests.exceptions.RequestException as e:
raise Exception(f"Error for POST {e}")
return handle_response(response)
class EntrustIssuerPlugin(IssuerPlugin): class EntrustIssuerPlugin(IssuerPlugin):
@ -178,14 +197,8 @@ class EntrustIssuerPlugin(IssuerPlugin):
data = process_options(issuer_options) data = process_options(issuer_options)
data["csr"] = csr data["csr"] = csr
try: response_dict = order_and_download_certificate(self.session, url, data)
response = self.session.post(url, json=data, timeout=(15, 40))
except requests.exceptions.Timeout:
raise Exception("Timeout for POST")
except requests.exceptions.RequestException as e:
raise Exception(f"Error for POST {e}")
response_dict = handle_response(response)
external_id = response_dict['trackingId'] external_id = response_dict['trackingId']
cert = response_dict['endEntityCert'] cert = response_dict['endEntityCert']
if len(response_dict['chainCerts']) < 2: if len(response_dict['chainCerts']) < 2:
@ -200,6 +213,7 @@ class EntrustIssuerPlugin(IssuerPlugin):
return cert, chain, external_id return cert, chain, external_id
@retry(stop_max_attempt_number=3, wait_fixed=1000)
def revoke_certificate(self, certificate, comments): def revoke_certificate(self, certificate, comments):
"""Revoke an Entrust certificate.""" """Revoke an Entrust certificate."""
base_url = current_app.config.get("ENTRUST_URL") base_url = current_app.config.get("ENTRUST_URL")
@ -216,6 +230,7 @@ class EntrustIssuerPlugin(IssuerPlugin):
metrics.send("entrust_revoke_certificate", "counter", 1) metrics.send("entrust_revoke_certificate", "counter", 1)
return handle_response(response) return handle_response(response)
@retry(stop_max_attempt_number=3, wait_fixed=1000)
def deactivate_certificate(self, certificate): def deactivate_certificate(self, certificate):
"""Deactivates an Entrust certificate.""" """Deactivates an Entrust certificate."""
base_url = current_app.config.get("ENTRUST_URL") base_url = current_app.config.get("ENTRUST_URL")
@ -244,7 +259,7 @@ class EntrustIssuerPlugin(IssuerPlugin):
def get_ordered_certificate(self, order_id): def get_ordered_certificate(self, order_id):
raise NotImplementedError("Not implemented\n", self, order_id) raise NotImplementedError("Not implemented\n", self, order_id)
def canceled_ordered_certificate(self, pending_cert, **kwargs): def cancel_ordered_certificate(self, pending_cert, **kwargs):
raise NotImplementedError("Not implemented\n", self, pending_cert, **kwargs) raise NotImplementedError("Not implemented\n", self, pending_cert, **kwargs)

11
lemur/plugins/lemur_slack/plugin.py
View File

@ -112,6 +112,9 @@ class SlackNotificationPlugin(ExpirationNotificationPlugin):
""" """
A typical check can be performed using the notify command: A typical check can be performed using the notify command:
`lemur notify` `lemur notify`
While we receive a `targets` parameter here, it is unused, as Slack webhooks do not allow
dynamic re-targeting of messages. The webhook itself specifies a channel.
""" """
attachments = None attachments = None
if notification_type == "expiration": if notification_type == "expiration":
@ -124,7 +127,7 @@ class SlackNotificationPlugin(ExpirationNotificationPlugin):
raise Exception("Unable to create message attachments") raise Exception("Unable to create message attachments")
body = { body = {
"text": "Lemur {0} Notification".format(notification_type.capitalize()), "text": f"Lemur {notification_type.capitalize()} Notification",
"attachments": attachments, "attachments": attachments,
"channel": self.get_option("recipients", options), "channel": self.get_option("recipients", options),
"username": self.get_option("username", options), "username": self.get_option("username", options),
@ -133,8 +136,8 @@ class SlackNotificationPlugin(ExpirationNotificationPlugin):
r = requests.post(self.get_option("webhook", options), json.dumps(body)) r = requests.post(self.get_option("webhook", options), json.dumps(body))
if r.status_code not in [200]: if r.status_code not in [200]:
raise Exception("Failed to send message") raise Exception(f"Failed to send message. Slack response: {r.status_code} {body}")
current_app.logger.error( current_app.logger.info(
"Slack response: {0} Message Body: {1}".format(r.status_code, body) f"Slack response: {r.status_code} Message Body: {body}"
) )

7
lemur/plugins/lemur_slack/tests/test_slack.py
View File

@ -1,8 +1,10 @@
from datetime import timedelta from datetime import timedelta
import arrow import arrow
from moto import mock_ses
from lemur.tests.factories import NotificationFactory, CertificateFactory from lemur.tests.factories import NotificationFactory, CertificateFactory
from lemur.tests.test_messaging import verify_sender_email
def test_formatting(certificate): def test_formatting(certificate):
@ -38,9 +40,12 @@ def get_options():
] ]
@mock_ses() # because email notifications are also sent
def test_send_expiration_notification(): def test_send_expiration_notification():
from lemur.notifications.messaging import send_expiration_notifications from lemur.notifications.messaging import send_expiration_notifications
verify_sender_email() # emails are sent to owner and security; Slack only used for configured notification
notification = NotificationFactory(plugin_name="slack-notification") notification = NotificationFactory(plugin_name="slack-notification")
notification.options = get_options() notification.options = get_options()
@ -51,7 +56,7 @@ def test_send_expiration_notification():
certificate.not_after = in_ten_days certificate.not_after = in_ten_days
certificate.notifications.append(notification) certificate.notifications.append(notification)
assert send_expiration_notifications([]) == (2, 0) assert send_expiration_notifications([]) == (3, 0) # owner, Slack, and security
# Currently disabled as the Slack plugin doesn't support this type of notification # Currently disabled as the Slack plugin doesn't support this type of notification

4
lemur/sources/service.py
View File

@ -264,13 +264,14 @@ def create(label, plugin_name, options, description=None):
return database.create(source) return database.create(source)
def update(source_id, label, options, description): def update(source_id, label, plugin_name, options, description):
""" """
Updates an existing source. Updates an existing source.
:param source_id: Lemur assigned ID :param source_id: Lemur assigned ID
:param label: Source common name :param label: Source common name
:param options: :param options:
:param plugin_name:
:param description: :param description:
:rtype : Source :rtype : Source
:return: :return:
@ -278,6 +279,7 @@ def update(source_id, label, options, description):
source = get(source_id) source = get(source_id)
source.label = label source.label = label
source.plugin_name = plugin_name
source.options = options source.options = options
source.description = description source.description = description

1
lemur/sources/views.py
View File

@ -284,6 +284,7 @@ class Sources(AuthenticatedResource):
return service.update( return service.update(
source_id, source_id,
data["label"], data["label"],
data["plugin"]["slug"],
data["plugin"]["plugin_options"], data["plugin"]["plugin_options"],
data["description"], data["description"],
) )

8
lemur/static/app/angular/authorities/authority/options.tpl.html
View File

@ -21,13 +21,7 @@
</label> </label>
<div class="col-sm-10"> <div class="col-sm-10">
<select class="form-control" ng-model="authority.keyType" <select class="form-control" ng-model="authority.keyType"
ng-options="option.value as option.name for option in [ ng-options="option for option in ['RSA2048', 'RSA4096', 'ECCPRIME256V1', 'ECCSECP384R1', 'ECCSECP521R1']"
{'name': 'RSA-2048', 'value': 'RSA2048'},
{'name': 'RSA-4096', 'value': 'RSA4096'},
{'name': 'ECC-PRIME256V1', 'value': 'ECCPRIME256V1'},
{'name': 'ECC-SECP384R1', 'value': 'ECCSECP384R1'},
{'name': 'ECC-SECP521R1', 'value': 'ECCSECP521R1'}]"
ng-init="authority.keyType = 'RSA2048'"> ng-init="authority.keyType = 'RSA2048'">
</select> </select>
</div> </div>

7
lemur/static/app/angular/certificates/certificate/options.tpl.html
View File

@ -32,12 +32,7 @@
</label> </label>
<div class="col-sm-10"> <div class="col-sm-10">
<select class="form-control" ng-model="certificate.keyType" <select class="form-control" ng-model="certificate.keyType"
ng-options="option.value as option.name for option in [ ng-options="option for option in ['RSA2048', 'RSA4096', 'ECCPRIME256V1', 'ECCSECP384R1']"
{'name': 'RSA-2048', 'value': 'RSA2048'},
{'name': 'RSA-4096', 'value': 'RSA4096'},
{'name': 'ECC-PRIME256V1', 'value': 'ECCPRIME256V1'},
{'name': 'ECC-SECP384R1', 'value': 'ECCSECP384R1'}]"
ng-init="certificate.keyType = 'RSA2048'"></select> ng-init="certificate.keyType = 'RSA2048'"></select>
</div> </div>
</div> </div>

12
lemur/static/app/angular/destinations/destination/destination.js vendored
View File

@ -52,19 +52,19 @@ angular.module('lemur')
if (plugin.slug === $scope.destination.plugin.slug) { if (plugin.slug === $scope.destination.plugin.slug) {
plugin.pluginOptions = $scope.destination.plugin.pluginOptions; plugin.pluginOptions = $scope.destination.plugin.pluginOptions;
$scope.destination.plugin = plugin; $scope.destination.plugin = plugin;
_.each($scope.destination.plugin.pluginOptions, function (option) { PluginService.getByType('export').then(function (plugins) {
if (option.type === 'export-plugin') { $scope.exportPlugins = plugins;
PluginService.getByType('export').then(function (plugins) {
$scope.exportPlugins = plugins;
_.each($scope.destination.plugin.pluginOptions, function (option) {
if (option.type === 'export-plugin') {
_.each($scope.exportPlugins, function (plugin) { _.each($scope.exportPlugins, function (plugin) {
if (plugin.slug === option.value.slug) { if (plugin.slug === option.value.slug) {
plugin.pluginOptions = option.value.pluginOptions; plugin.pluginOptions = option.value.pluginOptions;
option.value = plugin; option.value = plugin;
} }
}); });
}); }
} });
}); });
} }
}); });

14
lemur/static/app/angular/notifications/notification/notification.js vendored
View File

@ -42,8 +42,8 @@ angular.module('lemur')
PluginService.getByType('notification').then(function (plugins) { PluginService.getByType('notification').then(function (plugins) {
$scope.plugins = plugins; $scope.plugins = plugins;
_.each($scope.plugins, function (plugin) { _.each($scope.plugins, function (plugin) {
if (plugin.slug === $scope.notification.pluginName) { if (plugin.slug === $scope.notification.plugin.slug) {
plugin.pluginOptions = $scope.notification.notificationOptions; plugin.pluginOptions = $scope.notification.plugin.pluginOptions;
$scope.notification.plugin = plugin; $scope.notification.plugin = plugin;
} }
}); });
@ -51,16 +51,6 @@ angular.module('lemur')
NotificationService.getCertificates(notification); NotificationService.getCertificates(notification);
}); });
PluginService.getByType('notification').then(function (plugins) {
$scope.plugins = plugins;
_.each($scope.plugins, function (plugin) {
if (plugin.slug === $scope.notification.pluginName) {
plugin.pluginOptions = $scope.notification.notificationOptions;
$scope.notification.plugin = plugin;
}
});
});
$scope.save = function (notification) { $scope.save = function (notification) {
NotificationService.update(notification).then( NotificationService.update(notification).then(
function () { function () {

4
lemur/static/app/angular/notifications/services.js vendored
View File

@ -27,7 +27,7 @@ angular.module('lemur')
}; };
NotificationService.getCertificates = function (notification) { NotificationService.getCertificates = function (notification) {
notification.getList('certificates').then(function (certificates) { notification.getList('certificates', {showExpired: 0}).then(function (certificates) {
notification.certificates = certificates; notification.certificates = certificates;
}); });
}; };
@ -40,7 +40,7 @@ angular.module('lemur')
NotificationService.loadMoreCertificates = function (notification, page) { NotificationService.loadMoreCertificates = function (notification, page) {
notification.getList('certificates', {page: page}).then(function (certificates) { notification.getList('certificates', {page: page, showExpired: 0}).then(function (certificates) {
_.each(certificates, function (certificate) { _.each(certificates, function (certificate) {
notification.roles.push(certificate); notification.roles.push(certificate);
}); });

12
lemur/static/app/angular/sources/source/source.js vendored
View File

@ -41,22 +41,14 @@ angular.module('lemur')
PluginService.getByType('source').then(function (plugins) { PluginService.getByType('source').then(function (plugins) {
$scope.plugins = plugins; $scope.plugins = plugins;
_.each($scope.plugins, function (plugin) { _.each($scope.plugins, function (plugin) {
if (plugin.slug === $scope.source.pluginName) { if (plugin.slug === $scope.source.plugin.slug) {
plugin.pluginOptions = $scope.source.plugin.pluginOptions;
$scope.source.plugin = plugin; $scope.source.plugin = plugin;
} }
}); });
}); });
}); });
PluginService.getByType('source').then(function (plugins) {
$scope.plugins = plugins;
_.each($scope.plugins, function (plugin) {
if (plugin.slug === $scope.source.pluginName) {
$scope.source.plugin = plugin;
}
});
});
$scope.save = function (source) { $scope.save = function (source) {
SourceService.update(source).then( SourceService.update(source).then(
function () { function () {

1
lemur/tests/test_certificates.py
View File

@ -802,6 +802,7 @@ def test_reissue_certificate(
assert new_cert.organization != certificate.organization assert new_cert.organization != certificate.organization
# Check for default value since authority does not have cab_compliant option set # Check for default value since authority does not have cab_compliant option set
assert new_cert.organization == LEMUR_DEFAULT_ORGANIZATION assert new_cert.organization == LEMUR_DEFAULT_ORGANIZATION
assert new_cert.description.startswith(f"Reissued by Lemur for cert ID {certificate.id}")
# update cab_compliant option to false for crypto_authority to maintain subject details # update cab_compliant option to false for crypto_authority to maintain subject details
update_options(crypto_authority.id, '[{"name": "cab_compliant","value":false}]') update_options(crypto_authority.id, '[{"name": "cab_compliant","value":false}]')

18
lemur/tests/test_messaging.py
View File

@ -1,11 +1,18 @@
from datetime import timedelta from datetime import timedelta
import arrow import arrow
import boto3
import pytest import pytest
from freezegun import freeze_time from freezegun import freeze_time
from moto import mock_ses from moto import mock_ses
@mock_ses
def verify_sender_email():
ses_client = boto3.client("ses", region_name="us-east-1")
ses_client.verify_email_identity(EmailAddress="lemur@example.com")
def test_needs_notification(app, certificate, notification): def test_needs_notification(app, certificate, notification):
from lemur.notifications.messaging import needs_notification from lemur.notifications.messaging import needs_notification
@ -78,6 +85,7 @@ def test_get_eligible_certificates(app, certificate, notification):
@mock_ses @mock_ses
def test_send_expiration_notification(certificate, notification, notification_plugin): def test_send_expiration_notification(certificate, notification, notification_plugin):
from lemur.notifications.messaging import send_expiration_notifications from lemur.notifications.messaging import send_expiration_notifications
verify_sender_email()
certificate.notifications.append(notification) certificate.notifications.append(notification)
certificate.notifications[0].options = [ certificate.notifications[0].options = [
@ -87,7 +95,9 @@ def test_send_expiration_notification(certificate, notification, notification_pl
delta = certificate.not_after - timedelta(days=10) delta = certificate.not_after - timedelta(days=10)
with freeze_time(delta.datetime): with freeze_time(delta.datetime):
assert send_expiration_notifications([]) == (2, 0) # this will only send owner and security emails (no additional recipients),
# but it executes 3 successful send attempts
assert send_expiration_notifications([]) == (3, 0)
@mock_ses @mock_ses
@ -104,12 +114,14 @@ def test_send_expiration_notification_with_no_notifications(
@mock_ses @mock_ses
def test_send_rotation_notification(notification_plugin, certificate): def test_send_rotation_notification(notification_plugin, certificate):
from lemur.notifications.messaging import send_rotation_notification from lemur.notifications.messaging import send_rotation_notification
verify_sender_email()
assert send_rotation_notification(certificate, notification_plugin=notification_plugin) assert send_rotation_notification(certificate)
@mock_ses @mock_ses
def test_send_pending_failure_notification(notification_plugin, async_issuer_plugin, pending_certificate): def test_send_pending_failure_notification(notification_plugin, async_issuer_plugin, pending_certificate):
from lemur.notifications.messaging import send_pending_failure_notification from lemur.notifications.messaging import send_pending_failure_notification
verify_sender_email()
assert send_pending_failure_notification(pending_certificate, notification_plugin=notification_plugin) assert send_pending_failure_notification(pending_certificate)

4
requirements-dev.txt
View File

@ -11,7 +11,7 @@ cffi==1.14.0 # via cryptography
cfgv==3.1.0 # via pre-commit cfgv==3.1.0 # via pre-commit
chardet==3.0.4 # via requests chardet==3.0.4 # via requests
colorama==0.4.3 # via twine colorama==0.4.3 # via twine
cryptography==3.2 # via secretstorage cryptography==3.2.1 # via secretstorage
distlib==0.3.0 # via virtualenv distlib==0.3.0 # via virtualenv
docutils==0.16 # via readme-renderer docutils==0.16 # via readme-renderer
filelock==3.0.12 # via virtualenv filelock==3.0.12 # via virtualenv
@ -24,7 +24,7 @@ keyring==21.2.0 # via twine
mccabe==0.6.1 # via flake8 mccabe==0.6.1 # via flake8
nodeenv==1.5.0 # via -r requirements-dev.in, pre-commit nodeenv==1.5.0 # via -r requirements-dev.in, pre-commit
pkginfo==1.5.0.1 # via twine pkginfo==1.5.0.1 # via twine
pre-commit==2.7.1 # via -r requirements-dev.in pre-commit==2.8.2 # via -r requirements-dev.in
pycodestyle==2.3.1 # via flake8 pycodestyle==2.3.1 # via flake8
pycparser==2.20 # via cffi pycparser==2.20 # via cffi
pyflakes==1.6.0 # via flake8 pyflakes==1.6.0 # via flake8

8
requirements-docs.txt
View File

@ -17,8 +17,8 @@ bcrypt==3.1.7 # via -r requirements.txt, flask-bcrypt, paramiko
beautifulsoup4==4.9.1 # via -r requirements.txt, cloudflare beautifulsoup4==4.9.1 # via -r requirements.txt, cloudflare
billiard==3.6.3.0 # via -r requirements.txt, celery billiard==3.6.3.0 # via -r requirements.txt, celery
blinker==1.4 # via -r requirements.txt, flask-mail, flask-principal, raven blinker==1.4 # via -r requirements.txt, flask-mail, flask-principal, raven
boto3==1.16.5 # via -r requirements.txt boto3==1.16.9 # via -r requirements.txt
botocore==1.19.5 # via -r requirements.txt, boto3, s3transfer botocore==1.19.9 # via -r requirements.txt, boto3, s3transfer
celery[redis]==4.4.2 # via -r requirements.txt celery[redis]==4.4.2 # via -r requirements.txt
certifi==2020.6.20 # via -r requirements.txt, requests certifi==2020.6.20 # via -r requirements.txt, requests
certsrv==2.1.1 # via -r requirements.txt certsrv==2.1.1 # via -r requirements.txt
@ -26,7 +26,7 @@ cffi==1.14.0 # via -r requirements.txt, bcrypt, cryptography, pynac
chardet==3.0.4 # via -r requirements.txt, requests chardet==3.0.4 # via -r requirements.txt, requests
click==7.1.1 # via -r requirements.txt, flask click==7.1.1 # via -r requirements.txt, flask
cloudflare==2.8.13 # via -r requirements.txt cloudflare==2.8.13 # via -r requirements.txt
cryptography==3.2 # via -r requirements.txt, acme, josepy, paramiko, pyopenssl, requests cryptography==3.2.1 # via -r requirements.txt, acme, josepy, paramiko, pyopenssl, requests
dnspython3==1.15.0 # via -r requirements.txt dnspython3==1.15.0 # via -r requirements.txt
dnspython==1.15.0 # via -r requirements.txt, dnspython3 dnspython==1.15.0 # via -r requirements.txt, dnspython3
docutils==0.15.2 # via sphinx docutils==0.15.2 # via sphinx
@ -92,7 +92,7 @@ six==1.15.0 # via -r requirements.txt, acme, bcrypt, cryptography,
snowballstemmer==2.0.0 # via sphinx snowballstemmer==2.0.0 # via sphinx
soupsieve==2.0.1 # via -r requirements.txt, beautifulsoup4 soupsieve==2.0.1 # via -r requirements.txt, beautifulsoup4
sphinx-rtd-theme==0.5.0 # via -r requirements-docs.in sphinx-rtd-theme==0.5.0 # via -r requirements-docs.in
sphinx==3.2.1 # via -r requirements-docs.in, sphinx-rtd-theme, sphinxcontrib-httpdomain sphinx==3.3.0 # via -r requirements-docs.in, sphinx-rtd-theme, sphinxcontrib-httpdomain
sphinxcontrib-applehelp==1.0.2 # via sphinx sphinxcontrib-applehelp==1.0.2 # via sphinx
sphinxcontrib-devhelp==1.0.2 # via sphinx sphinxcontrib-devhelp==1.0.2 # via sphinx
sphinxcontrib-htmlhelp==1.0.3 # via sphinx sphinxcontrib-htmlhelp==1.0.3 # via sphinx

8
requirements-tests.txt
View File

@ -10,16 +10,16 @@ aws-sam-translator==1.22.0 # via cfn-lint
aws-xray-sdk==2.5.0 # via moto aws-xray-sdk==2.5.0 # via moto
bandit==1.6.2 # via -r requirements-tests.in bandit==1.6.2 # via -r requirements-tests.in
black==20.8b1 # via -r requirements-tests.in black==20.8b1 # via -r requirements-tests.in
boto3==1.16.5 # via aws-sam-translator, moto boto3==1.16.9 # via aws-sam-translator, moto
boto==2.49.0 # via moto boto==2.49.0 # via moto
botocore==1.19.5 # via aws-xray-sdk, boto3, moto, s3transfer botocore==1.19.9 # via aws-xray-sdk, boto3, moto, s3transfer
certifi==2020.6.20 # via requests certifi==2020.6.20 # via requests
cffi==1.14.0 # via cryptography cffi==1.14.0 # via cryptography
cfn-lint==0.29.5 # via moto cfn-lint==0.29.5 # via moto
chardet==3.0.4 # via requests chardet==3.0.4 # via requests
click==7.1.2 # via black, flask click==7.1.2 # via black, flask
coverage==5.3 # via -r requirements-tests.in coverage==5.3 # via -r requirements-tests.in
cryptography==3.2 # via moto, python-jose, sshpubkeys cryptography==3.2.1 # via moto, python-jose, sshpubkeys
decorator==4.4.2 # via networkx decorator==4.4.2 # via networkx
docker==4.2.0 # via moto docker==4.2.0 # via moto
ecdsa==0.14.1 # via moto, python-jose, sshpubkeys ecdsa==0.14.1 # via moto, python-jose, sshpubkeys
@ -61,7 +61,7 @@ pyparsing==2.4.7 # via packaging
pyrsistent==0.16.0 # via jsonschema pyrsistent==0.16.0 # via jsonschema
pytest-flask==1.0.0 # via -r requirements-tests.in pytest-flask==1.0.0 # via -r requirements-tests.in
pytest-mock==3.3.1 # via -r requirements-tests.in pytest-mock==3.3.1 # via -r requirements-tests.in
pytest==6.1.1 # via -r requirements-tests.in, pytest-flask, pytest-mock pytest==6.1.2 # via -r requirements-tests.in, pytest-flask, pytest-mock
python-dateutil==2.8.1 # via botocore, faker, freezegun, moto python-dateutil==2.8.1 # via botocore, faker, freezegun, moto
python-jose[cryptography]==3.1.0 # via moto python-jose[cryptography]==3.1.0 # via moto
pytz==2019.3 # via moto pytz==2019.3 # via moto

6
requirements.txt
View File

@ -15,8 +15,8 @@ bcrypt==3.1.7 # via flask-bcrypt, paramiko
beautifulsoup4==4.9.1 # via cloudflare beautifulsoup4==4.9.1 # via cloudflare
billiard==3.6.3.0 # via celery billiard==3.6.3.0 # via celery
blinker==1.4 # via flask-mail, flask-principal, raven blinker==1.4 # via flask-mail, flask-principal, raven
boto3==1.16.5 # via -r requirements.in boto3==1.16.9 # via -r requirements.in
botocore==1.19.5 # via -r requirements.in, boto3, s3transfer botocore==1.19.9 # via -r requirements.in, boto3, s3transfer
celery[redis]==4.4.2 # via -r requirements.in celery[redis]==4.4.2 # via -r requirements.in
certifi==2020.6.20 # via -r requirements.in, requests certifi==2020.6.20 # via -r requirements.in, requests
certsrv==2.1.1 # via -r requirements.in certsrv==2.1.1 # via -r requirements.in
@ -24,7 +24,7 @@ cffi==1.14.0 # via bcrypt, cryptography, pynacl
chardet==3.0.4 # via requests chardet==3.0.4 # via requests
click==7.1.1 # via flask click==7.1.1 # via flask
cloudflare==2.8.13 # via -r requirements.in cloudflare==2.8.13 # via -r requirements.in
cryptography==3.2 # via -r requirements.in, acme, josepy, paramiko, pyopenssl, requests cryptography==3.2.1 # via -r requirements.in, acme, josepy, paramiko, pyopenssl, requests
dnspython3==1.15.0 # via -r requirements.in dnspython3==1.15.0 # via -r requirements.in
dnspython==1.15.0 # via dnspython3 dnspython==1.15.0 # via dnspython3
dyn==1.8.1 # via -r requirements.in dyn==1.8.1 # via -r requirements.in

1
setup.py
View File

@ -136,6 +136,7 @@ setup(
'aws_destination = lemur.plugins.lemur_aws.plugin:AWSDestinationPlugin', 'aws_destination = lemur.plugins.lemur_aws.plugin:AWSDestinationPlugin',
'aws_source = lemur.plugins.lemur_aws.plugin:AWSSourcePlugin', 'aws_source = lemur.plugins.lemur_aws.plugin:AWSSourcePlugin',
'aws_s3 = lemur.plugins.lemur_aws.plugin:S3DestinationPlugin', 'aws_s3 = lemur.plugins.lemur_aws.plugin:S3DestinationPlugin',
'aws_sns = lemur.plugins.lemur_aws.plugin:SNSNotificationPlugin',
'email_notification = lemur.plugins.lemur_email.plugin:EmailNotificationPlugin', 'email_notification = lemur.plugins.lemur_email.plugin:EmailNotificationPlugin',
'slack_notification = lemur.plugins.lemur_slack.plugin:SlackNotificationPlugin', 'slack_notification = lemur.plugins.lemur_slack.plugin:SlackNotificationPlugin',
'java_truststore_export = lemur.plugins.lemur_jks.plugin:JavaTruststoreExportPlugin', 'java_truststore_export = lemur.plugins.lemur_jks.plugin:JavaTruststoreExportPlugin',