diff --git a/docs/administration/index.rst b/docs/administration/index.rst index 9a1846e2..e57dd5c0 100644 --- a/docs/administration/index.rst +++ b/docs/administration/index.rst @@ -150,9 +150,6 @@ Lemur supports sending certification expiration notifications through SES and SM LEMUR_SECURITY_TEAM_EMAIL = ['security@example.com'] -.. data:: - - Authority Options ----------------- @@ -505,11 +502,19 @@ All commands default to `~/.lemur/lemur.conf.py` if a configuration is not speci .. data:: sync - Sync attempts to discover certificates in the environment that were not created by Lemur. There + Sync attempts to discover certificates in the environment that were not created by Lemur. If you wish to only sync + a few sources you can pass a comma delimited list of sources to sync :: - lemur sync --all + lemur sync source1,source2 + + + Additionally you can also list the available sources that Lemur can sync + + :: + + lemur sync -list Identity and Access Management diff --git a/docs/developer/internals/lemur.certificates.rst b/docs/developer/internals/lemur.certificates.rst index a19617c4..cf052148 100644 --- a/docs/developer/internals/lemur.certificates.rst +++ b/docs/developer/internals/lemur.certificates.rst @@ -28,15 +28,6 @@ certificates Package :undoc-members: :show-inheritance: -:mod:`sync` Module ------------------- - -.. automodule:: lemur.certificates.sync - :noindex: - :members: - :undoc-members: - :show-inheritance: - :mod:`verify` Module -------------------- diff --git a/docs/developer/plugins/index.rst b/docs/developer/plugins/index.rst index ae07f1c2..c813b4f7 100644 --- a/docs/developer/plugins/index.rst +++ b/docs/developer/plugins/index.rst @@ -215,9 +215,9 @@ certificate Lemur does not know about and adding the certificate to it's invento The `SourcePlugin` object has one default option of `pollRate`. This controls the number of seconds which to get new certificates. .. warning:: - Lemur currently has a very basic polling system of running a cron job every 15min to see which source plugins need to be run. - This means special consideration needs to be taken such that running all `SourcePlugins` does not take >15min to run. It also means - that the minimum resolution of a source plugin poll rate is effectively 15min. + Lemur currently has a very basic polling system of running a cron job every 15min to see which source plugins need to be run. A lock file is generated to guarentee that ] + only one sync is running at a time. It also means that the minimum resolution of a source plugin poll rate is effectively 15min. You can always specify a faster cron + job if you need a higher resolution sync job. The `SourcePlugin` object requires implementation of one function:: diff --git a/docs/quickstart/index.rst b/docs/quickstart/index.rst index f3a04a63..034141fb 100644 --- a/docs/quickstart/index.rst +++ b/docs/quickstart/index.rst @@ -42,13 +42,13 @@ Finally, activate your virtualenv:: Installing build dependencies ----------------------------- -If installing Lemur on true bare Ubuntu OS you will need to grab the following packages so that Lemur can correctly build it's -dependencies. +If installing Lemur on truely bare Ubuntu OS you will need to grab the following packages so that Lemur can correctly build it's +dependencies:: $ sudo apt-get update $ sudo apt-get install nodejs-legacy python-pip libpq-dev python-dev build-essential libssl-dev libffi-dev nginx git supervisor -And optionally if your database is going to be on the same host as the webserver. +And optionally if your database is going to be on the same host as the webserver:: $ sudo apt-get install postgres @@ -110,7 +110,7 @@ Update your configuration Once created you will need to update the configuration file with information about your environment, such as which database to talk to, where keys are stores etc.. -.. Note:: If you are unVfamiliar with with the SQLALCHEMY_DATABASE_URI string it can be broken up like so: +.. Note:: If you are unfamiliar with with the SQLALCHEMY_DATABASE_URI string it can be broken up like so: postgresql://userame:password@databasefqdn:databaseport/databasename Setup Postgres @@ -119,7 +119,7 @@ Setup Postgres For production a dedicated database is recommended, for this guide we will assume postgres has been installed and is on the same machine that Lemur is installed on. -First, set a password for the postgres user. For this guide, we will use **lemur** as an example but you should use the database password generated for by Lemur.:: +First, set a password for the postgres user. For this guide, we will use **lemur** as an example but you should use the database password generated for by Lemur:: $ sudo -u postgres psql postgres # \password postgres @@ -139,10 +139,17 @@ Initializing Lemur Lemur provides a helpful command that will initialize your database for you. It creates a default user (lemur) that is used by Lemur to help associate certificates that do not currently have an owner. This is most commonly the case when -Lemur has discovered certificates from a third party resource. This is also a default user that can be used to +Lemur has discovered certificates from a third party source. This is also a default user that can be used to administer Lemur. -**Make note of the password used as this will be use to first login to the Lemur UI** +In addition to create a new User, Lemur also creates a few default email notifications. These notifications are based +on a few configuration options such as `LEMUR_SECURITY_TEAM_EMAIL` they basically garentee that every cerificate within +Lemur will send one expiration notification to the security team. + +Additional notifications can be created through the UI or API. +See :ref:`Creating Notifications ` and :ref:`Command Line Interface ` for details. + +**Make note of the password used as this will be used during first login to the Lemur UI** .. code-block:: bash diff --git a/lemur/__init__.py b/lemur/__init__.py index 43c7e0df..5d91aacf 100644 --- a/lemur/__init__.py +++ b/lemur/__init__.py @@ -14,28 +14,27 @@ from lemur.users.views import mod as users_bp from lemur.roles.views import mod as roles_bp from lemur.auth.views import mod as auth_bp from lemur.domains.views import mod as domains_bp -from lemur.elbs.views import mod as elbs_bp from lemur.destinations.views import mod as destinations_bp from lemur.authorities.views import mod as authorities_bp -from lemur.listeners.views import mod as listeners_bp from lemur.certificates.views import mod as certificates_bp from lemur.status.views import mod as status_bp from lemur.plugins.views import mod as plugins_bp from lemur.notifications.views import mod as notifications_bp +from lemur.sources.views import mod as sources_bp + LEMUR_BLUEPRINTS = ( users_bp, roles_bp, auth_bp, domains_bp, - elbs_bp, destinations_bp, authorities_bp, - listeners_bp, certificates_bp, status_bp, plugins_bp, notifications_bp, + sources_bp ) diff --git a/lemur/authorities/service.py b/lemur/authorities/service.py index 0c831be8..23961ede 100644 --- a/lemur/authorities/service.py +++ b/lemur/authorities/service.py @@ -9,10 +9,12 @@ """ from flask import g +from flask import current_app from lemur import database from lemur.authorities.models import Authority from lemur.roles import service as role_service +from lemur.notifications import service as notification_service from lemur.roles.models import Role from lemur.certificates.models import Certificate @@ -56,9 +58,15 @@ def create(kwargs): cert.description = "This is the ROOT certificate for the {0} certificate authority".format(kwargs.get('caName')) cert.user = g.current_user + cert.notifications = notification_service.create_default_expiration_notifications( + 'DEFAULT_SECURITY', + current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL') + ) + # we create and attach any roles that the issuer gives us role_objs = [] for r in issuer_roles: + role = role_service.create( r['name'], password=r['password'], diff --git a/lemur/certificates/models.py b/lemur/certificates/models.py index d876a3da..83f3f690 100644 --- a/lemur/certificates/models.py +++ b/lemur/certificates/models.py @@ -23,7 +23,9 @@ from lemur.plugins.base import plugins from lemur.domains.models import Domain from lemur.constants import SAN_NAMING_TEMPLATE, DEFAULT_NAMING_TEMPLATE -from lemur.models import certificate_associations, certificate_destination_associations, certificate_notification_associations + +from lemur.models import certificate_associations, certificate_source_associations, \ + certificate_destination_associations, certificate_notification_associations def create_name(issuer, not_before, not_after, subject, san): @@ -222,8 +224,8 @@ class Certificate(db.Model): authority_id = Column(Integer, ForeignKey('authorities.id')) notifications = relationship("Notification", secondary=certificate_notification_associations, backref='certificate') destinations = relationship("Destination", secondary=certificate_destination_associations, backref='certificate') + sources = relationship("Source", secondary=certificate_source_associations, backref='certificate') domains = relationship("Domain", secondary=certificate_associations, backref="certificate") - elb_listeners = relationship("Listener", lazy='dynamic', backref='certificate') def __init__(self, body, private_key=None, chain=None): self.body = body @@ -277,5 +279,4 @@ class Certificate(db.Model): @event.listens_for(Certificate.destinations, 'append') def update_destinations(target, value, initiator): destination_plugin = plugins.get(value.plugin_name) - - destination_plugin.upload(target.body, target.private_key, target.chain, value.options) + destination_plugin.upload(target.name, target.body, target.private_key, target.chain, value.options) diff --git a/lemur/certificates/service.py b/lemur/certificates/service.py index 77f2f5b3..886ef165 100644 --- a/lemur/certificates/service.py +++ b/lemur/certificates/service.py @@ -134,8 +134,11 @@ def import_certificate(**kwargs): :param kwargs: """ from lemur.users import service as user_service - cert = Certificate(kwargs['public_certificate']) - cert.owner = kwargs.get('owner', current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL')) + from lemur.notifications import service as notification_service + cert = Certificate(kwargs['public_certificate'], chain=kwargs['intermediate_certificate']) + + # TODO future source plugins might have a better understanding of who the 'owner' is we should support this + cert.owner = kwargs.get('owner', current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL')[0]) cert.creator = kwargs.get('creator', user_service.get_by_email('lemur@nobody')) # NOTE existing certs may not follow our naming standard we will @@ -146,7 +149,9 @@ def import_certificate(**kwargs): if kwargs.get('user'): cert.user = kwargs.get('user') - database.update_list(cert, 'notifications', Notification, kwargs.get('notifications')) + notification_name = 'DEFAULT_SECURITY' + notifications = notification_service.create_default_expiration_notifications(notification_name, current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL')) + cert.notifications = notifications cert = database.create(cert) return cert @@ -156,18 +161,35 @@ def upload(**kwargs): """ Allows for pre-made certificates to be imported into Lemur. """ + from lemur.notifications import service as notification_service cert = Certificate( kwargs.get('public_cert'), kwargs.get('private_key'), kwargs.get('intermediate_cert'), ) - database.update_list(cert, 'destinations', Destination, kwargs.get('destinations')) - database.update_list(cert, 'notifications', Notification, kwargs.get('notifications')) + cert.description = kwargs.get('description') cert.owner = kwargs['owner'] cert = database.create(cert) + g.user.certificates.append(cert) + + database.update_list(cert, 'destinations', Destination, kwargs.get('destinations')) + + database.update_list(cert, 'notifications', Notification, kwargs.get('notifications')) + + # create default notifications for this certificate if none are provided + notifications = [] + if not kwargs.get('notifications'): + notification_name = "DEFAULT_{0}".format(cert.owner.split('@')[0].upper()) + notifications += notification_service.create_default_expiration_notifications(notification_name, [cert.owner]) + + notification_name = 'DEFAULT_SECURITY' + notifications += notification_service.create_default_expiration_notifications(notification_name, current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL')) + cert.notifications = notifications + + database.update(cert) return cert @@ -175,12 +197,11 @@ def create(**kwargs): """ Creates a new certificate. """ + from lemur.notifications import service as notification_service cert, private_key, cert_chain = mint(kwargs) cert.owner = kwargs['owner'] - database.update_list(cert, 'destinations', Destination, kwargs.get('destinations')) - database.create(cert) cert.description = kwargs['description'] g.user.certificates.append(cert) @@ -188,7 +209,20 @@ def create(**kwargs): # do this after the certificate has already been created because if it fails to upload to the third party # we do not want to lose the certificate information. + database.update_list(cert, 'destinations', Destination, kwargs.get('destinations')) + database.update_list(cert, 'notifications', Notification, kwargs.get('notifications')) + + # create default notifications for this certificate if none are provided + notifications = [] + if not kwargs.get('notifications'): + notification_name = "DEFAULT_{0}".format(cert.owner.split('@')[0].upper()) + notifications += notification_service.create_default_expiration_notifications(notification_name, [cert.owner]) + + notification_name = 'DEFAULT_SECURITY' + notifications += notification_service.create_default_expiration_notifications(notification_name, current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL')) + cert.notifications = notifications + database.update(cert) return cert @@ -297,7 +331,7 @@ def create_csr(csr_config): x509.SubjectAlternativeName(general_names), critical=True ) - # TODO support more CSR options, none of the authorities support these atm + # TODO support more CSR options, none of the authority plugins currently support these options # builder.add_extension( # x509.KeyUsage( # digital_signature=digital_signature, @@ -365,14 +399,6 @@ def stats(**kwargs): :param kwargs: :return: """ - query = database.session_query(Certificate) - - if kwargs.get('active') == 'true': - query = query.filter(Certificate.elb_listeners.any()) - - if kwargs.get('destination_id'): - query = query.filter(Certificate.destinations.any(Destination.id == kwargs.get('destination_id'))) - if kwargs.get('metric') == 'not_after': start = arrow.utcnow() end = start.replace(weeks=+32) @@ -385,10 +411,6 @@ def stats(**kwargs): attr = getattr(Certificate, kwargs.get('metric')) query = database.db.session.query(attr, func.count(attr)) - # TODO this could be cleaned up - if kwargs.get('active') == 'true': - query = query.filter(Certificate.elb_listeners.any()) - items = query.group_by(attr).all() keys = [] diff --git a/lemur/certificates/sync.py b/lemur/certificates/sync.py deleted file mode 100644 index b91af6b1..00000000 --- a/lemur/certificates/sync.py +++ /dev/null @@ -1,46 +0,0 @@ -""" -.. module: sync - :platform: Unix - :synopsis: This module contains various certificate syncing operations. - Because of the nature of the SSL environment there are multiple ways - a certificate could be created without Lemur's knowledge. Lemur attempts - to 'sync' with as many different datasources as possible to try and track - any certificate that may be in use. - - These operations are typically run on a periodic basis from either the command - line or a cron job. - - :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more - :license: Apache, see LICENSE for more details. -.. moduleauthor:: Kevin Glisson -""" -from flask import current_app - -from lemur.certificates import service as cert_service - -from lemur.plugins.base import plugins -from lemur.plugins.bases.source import SourcePlugin - - -def sync(): - for plugin in plugins: - new = 0 - updated = 0 - if isinstance(plugin, SourcePlugin): - if plugin.is_enabled(): - current_app.logger.error("Retrieving certificates from {0}".format(plugin.title)) - certificates = plugin.get_certificates() - - for certificate in certificates: - exists = cert_service.find_duplicates(certificate) - - if not exists: - cert_service.import_certificate(**certificate) - new += 1 - - if len(exists) == 1: - updated += 1 - - # TODO associated cert with source - # TODO update cert if found from different source - # TODO disassociate source if missing diff --git a/lemur/certificates/views.py b/lemur/certificates/views.py index 47511eaa..db8772aa 100644 --- a/lemur/certificates/views.py +++ b/lemur/certificates/views.py @@ -369,6 +369,7 @@ class CertificatesUpload(AuthenticatedResource): :statuscode 403: unauthenticated :statuscode 200: no error """ + self.reqparse.add_argument('description', type=str, location='json') self.reqparse.add_argument('owner', type=str, required=True, location='json') self.reqparse.add_argument('publicCert', type=pem_str, required=True, dest='public_cert', location='json') self.reqparse.add_argument('destinations', type=list, default=[], dest='destinations', location='json') diff --git a/lemur/database.py b/lemur/database.py index c2bb8f13..22a464ff 100644 --- a/lemur/database.py +++ b/lemur/database.py @@ -9,6 +9,8 @@ .. moduleauthor:: Kevin Glisson """ +from flask import current_app + from sqlalchemy import exc from sqlalchemy.sql import and_, or_ @@ -124,7 +126,8 @@ def get(model, value, field="id"): query = session_query(model) try: return query.filter(getattr(model, field) == value).one() - except Exception: + except Exception as e: + current_app.logger.exception(e) return diff --git a/lemur/destinations/service.py b/lemur/destinations/service.py index 38dc600f..f27a138c 100644 --- a/lemur/destinations/service.py +++ b/lemur/destinations/service.py @@ -5,6 +5,8 @@ :license: Apache, see LICENSE for more details. .. moduleauthor:: Kevin Glisson """ +from sqlalchemy import func + from lemur import database from lemur.destinations.models import Destination from lemur.certificates.models import Certificate @@ -28,9 +30,8 @@ def update(destination_id, label, options, description): Updates an existing destination. :param destination_id: Lemur assigned ID - :param destination_number: AWS assigned ID :param label: Destination common name - :param comments: + :param description: :rtype : Destination :return: """ @@ -107,3 +108,24 @@ def render(args): query = database.sort(query, Destination, sort_by, sort_dir) return database.paginate(query, page, count) + + +def stats(**kwargs): + """ + Helper that defines some useful statistics about destinations. + + :param kwargs: + :return: + """ + attr = getattr(Destination, kwargs.get('metric')) + query = database.db.session.query(attr, func.count(attr)) + + items = query.group_by(attr).all() + + keys = [] + values = [] + for key, count in items: + keys.append(key) + values.append(count) + + return {'labels': keys, 'values': values} diff --git a/lemur/destinations/views.py b/lemur/destinations/views.py index 55ff7071..21e7886e 100644 --- a/lemur/destinations/views.py +++ b/lemur/destinations/views.py @@ -353,7 +353,21 @@ class CertificateDestinations(AuthenticatedResource): return service.render(args) +class DestinationsStats(AuthenticatedResource): + """ Defines the 'certificates' stats endpoint """ + def __init__(self): + self.reqparse = reqparse.RequestParser() + super(DestinationsStats, self).__init__() + + def get(self): + self.reqparse.add_argument('metric', type=str, location='args') + args = self.reqparse.parse_args() + items = service.stats(**args) + return dict(items=items, total=len(items)) + + api.add_resource(DestinationsList, '/destinations', endpoint='destinations') -api.add_resource(Destinations, '/destinations/', endpoint='account') +api.add_resource(Destinations, '/destinations/', endpoint='destination') api.add_resource(CertificateDestinations, '/certificates//destinations', endpoint='certificateDestinations') +api.add_resource(DestinationsStats, '/destinations/stats', endpoint='destinationStats') diff --git a/lemur/elbs/models.py b/lemur/elbs/models.py deleted file mode 100644 index d57a9f1f..00000000 --- a/lemur/elbs/models.py +++ /dev/null @@ -1,44 +0,0 @@ -""" -.. module: lemur.elbs.models - :platform: Unix - :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more - :license: Apache, see LICENSE for more details. - -.. moduleauthor:: Kevin Glisson -""" -from sqlalchemy import Column, BigInteger, String, DateTime, PassiveDefault, func -from sqlalchemy.orm import relationship - -from lemur.database import db -from lemur.listeners.models import Listener - - -class ELB(db.Model): - __tablename__ = 'elbs' - id = Column(BigInteger, primary_key=True) - # account_id = Column(BigInteger, ForeignKey("accounts.id"), index=True) - region = Column(String(32)) - name = Column(String(128)) - vpc_id = Column(String(128)) - scheme = Column(String(128)) - dns_name = Column(String(128)) - listeners = relationship("Listener", backref='elb', cascade="all, delete, delete-orphan") - date_created = Column(DateTime, PassiveDefault(func.now()), nullable=False) - - def __init__(self, elb_obj=None): - if elb_obj: - self.region = elb_obj.connection.region.name - self.name = elb_obj.name - self.vpc_id = elb_obj.vpc_id - self.scheme = elb_obj.scheme - self.dns_name = elb_obj.dns_name - for listener in elb_obj.listeners: - self.listeners.append(Listener(listener)) - - def as_dict(self): - return {c.name: getattr(self, c.name) for c in self.__table__.columns} - - def serialize(self): - blob = self.as_dict() - del blob['date_created'] - return blob diff --git a/lemur/elbs/service.py b/lemur/elbs/service.py deleted file mode 100644 index d00110bf..00000000 --- a/lemur/elbs/service.py +++ /dev/null @@ -1,124 +0,0 @@ -""" -.. module: lemur.elbs.service - :platform: Unix - :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more - :license: Apache, see LICENSE for more details. - -.. moduleauthor:: Kevin Glisson - -""" -from sqlalchemy import func -from sqlalchemy.sql import and_ - -from lemur import database -from lemur.elbs.models import ELB -from lemur.listeners.models import Listener - - -def get_all(account_id, elb_name): - """ - Retrieves all ELBs in a given account - - :param account_id: - :param elb_name: - :rtype : Elb - :return: - """ - query = database.session_query(ELB) - return query.filter(and_(ELB.name == elb_name, ELB.account_id == account_id)).all() - - -def get_by_region_and_account(region, account_id): - query = database.session_query(ELB) - return query.filter(and_(ELB.region == region, ELB.account_id == account_id)).all() - - -def get_all_elbs(): - """ - Get all ELBs that Lemur knows about - - :rtype : list - :return: - """ - return ELB.query.all() - - -def get(elb_id): - """ - Retrieve an ELB with a give ID - - :rtype : Elb - :param elb_id: - :return: - """ - return database.get(ELB, elb_id) - - -def create(account, elb): - """ - Create a new ELB - - :param account: - :param elb: - """ - elb = ELB(elb) - account.elbs.append(elb) - database.create(elb) - - -def delete(elb_id): - """ - Delete an ELB - - :param elb_id: - """ - database.delete(get(elb_id)) - - -def render(args): - query = database.session_query(ELB) - - sort_by = args.pop('sort_by') - sort_dir = args.pop('sort_dir') - page = args.pop('page') - count = args.pop('count') - filt = args.pop('filter') - active = args.pop('active') - certificate_id = args.pop('certificate_id') - - if certificate_id: - query.filter(ELB.listeners.any(Listener.certificate_id == certificate_id)) - - if active == 'true': - query = query.filter(ELB.listeners.any()) - - if filt: - terms = filt.split(';') - query = database.filter(query, ELB, terms) - - query = database.find_all(query, ELB, args) - - if sort_by and sort_dir: - query = database.sort(query, ELB, sort_by, sort_dir) - - return database.paginate(query, page, count) - - -def stats(**kwargs): - attr = getattr(ELB, kwargs.get('metric')) - query = database.db.session.query(attr, func.count(attr)) - - if kwargs.get('account_id'): - query = query.filter(ELB.account_id == kwargs.get('account_id')) - - if kwargs.get('active') == 'true': - query = query.join(ELB.listeners) - query = query.filter(Listener.certificate_id != None) # noqa - - items = query.group_by(attr).all() - - results = [] - for key, count in items: - if key: - results.append({"key": key, "y": count}) - return results diff --git a/lemur/elbs/views.py b/lemur/elbs/views.py deleted file mode 100644 index 214d28e2..00000000 --- a/lemur/elbs/views.py +++ /dev/null @@ -1,78 +0,0 @@ -""" -.. module: lemur.elbs.service - :platform: Unix - :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more - :license: Apache, see LICENSE for more details. - -.. moduleauthor:: Kevin Glisson - -""" -from flask import Blueprint -from flask.ext.restful import reqparse, Api, fields -from lemur.elbs import service -from lemur.auth.service import AuthenticatedResource - -from lemur.common.utils import marshal_items, paginated_parser - - -mod = Blueprint('elbs', __name__) -api = Api(mod) - - -FIELDS = { - 'name': fields.String, - 'id': fields.Integer, - 'region': fields.String, - 'scheme': fields.String, - 'accountId': fields.Integer(attribute='account_id'), - 'vpcId': fields.String(attribute='vpc_id') -} - - -class ELBsList(AuthenticatedResource): - """ Defines the 'elbs' endpoint """ - def __init__(self): - super(ELBsList, self).__init__() - - @marshal_items(FIELDS) - def get(self): - parser = paginated_parser.copy() - parser.add_argument('owner', type=str, location='args') - parser.add_argument('id', type=str, location='args') - parser.add_argument('accountId', type=str, dest='account_id', location='args') - parser.add_argument('certificateId', type=str, dest='certificate_id', location='args') - parser.add_argument('active', type=str, default='true', location='args') - - args = parser.parse_args() - return service.render(args) - - -class ELBsStats(AuthenticatedResource): - def __init__(self): - self.reqparse = reqparse.RequestParser() - super(ELBsStats, self).__init__() - - def get(self): - self.reqparse.add_argument('metric', type=str, location='args') - self.reqparse.add_argument('accountId', dest='account_id', location='args') - self.reqparse.add_argument('active', type=str, default='true', location='args') - - args = self.reqparse.parse_args() - - items = service.stats(**args) - return {"items": items, "total": len(items)} - - -class ELBs(AuthenticatedResource): - def __init__(self): - self.reqparse = reqparse.RequestParser() - super(ELBs, self).__init__() - - @marshal_items(FIELDS) - def get(self, elb_id): - return service.get(elb_id) - - -api.add_resource(ELBsList, '/elbs', endpoint='elbs') -api.add_resource(ELBs, '/elbs/', endpoint='elb') -api.add_resource(ELBsStats, '/elbs/stats', endpoint='elbsStats') diff --git a/lemur/listeners/__init__.py b/lemur/listeners/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/lemur/listeners/models.py b/lemur/listeners/models.py deleted file mode 100644 index b72c9b48..00000000 --- a/lemur/listeners/models.py +++ /dev/null @@ -1,42 +0,0 @@ -""" -.. module: lemur.elbs.models - :platform: Unix - :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more - :license: Apache, see LICENSE for more details. - -.. moduleauthor:: Kevin Glisson - -""" -from sqlalchemy import Column, Integer, BigInteger, String, ForeignKey, DateTime, PassiveDefault, func - -from lemur.database import db -from lemur.certificates import service as cert_service -from lemur.certificates.models import Certificate, get_name_from_arn - - -class Listener(db.Model): - __tablename__ = 'listeners' - id = Column(BigInteger, primary_key=True) - certificate_id = Column(Integer, ForeignKey(Certificate.id), index=True) - elb_id = Column(BigInteger, ForeignKey("elbs.id"), index=True) - instance_port = Column(Integer) - instance_protocol = Column(String(16)) - load_balancer_port = Column(Integer) - load_balancer_protocol = Column(String(16)) - date_created = Column(DateTime, PassiveDefault(func.now()), nullable=False) - - def __init__(self, listener): - self.load_balancer_port = listener.load_balancer_port - self.load_balancer_protocol = listener.protocol - self.instance_port = listener.instance_port - self.instance_protocol = listener.instance_protocol - if listener.ssl_certificate_id not in ["Invalid-Certificate", None]: - self.certificate_id = cert_service.get_by_name(get_name_from_arn(listener.ssl_certificate_id)).id - - def as_dict(self): - return {c.name: getattr(self, c.name) for c in self.__table__.columns} - - def serialize(self): - blob = self.as_dict() - del blob['date_created'] - return blob diff --git a/lemur/listeners/service.py b/lemur/listeners/service.py deleted file mode 100644 index 6f2ae596..00000000 --- a/lemur/listeners/service.py +++ /dev/null @@ -1,159 +0,0 @@ -""" -.. module: lemur.listeners.service - :platform: Unix - :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more - :license: Apache, see LICENSE for more details. - -.. moduleauthor:: Kevin Glisson - -""" -from sqlalchemy import func - -from lemur import database - -from lemur.exceptions import CertificateUnavailable - -from lemur.elbs.models import ELB -from lemur.listeners.models import Listener -from lemur.elbs import service as elb_service -from lemur.certificates import service as certificate_service - -# from lemur.common.services.aws.elb import update_listeners, create_new_listeners, delete_listeners - - -def verify_attachment(certificate_id, elb_account_number): - """ - Ensures that the certificate we want ot attach to our listener is - in the same account as our listener. - - :rtype : Certificate - :param certificate_id: - :param elb_account_number: - :return: :raise CertificateUnavailable: - """ - cert = certificate_service.get(certificate_id) - - # we need to ensure that the specified cert is in our account - for account in cert.accounts: - if account.account_number == elb_account_number: - break - else: - raise CertificateUnavailable - return cert - - -def get(listener_id): - return database.get(Listener, listener_id) - - -def create(elb_id, instance_protocol, instance_port, load_balancer_port, load_balancer_protocol, certificate_id=None): - listener = Listener(elb_id, - instance_port, - instance_protocol, - load_balancer_port, - load_balancer_protocol - ) - - elb = elb_service.get(elb_id) - elb.listeners.append(listener) - account_number = elb.account.account_number - - cert = verify_attachment(certificate_id, account_number) - listener_tuple = (load_balancer_port, instance_port, load_balancer_protocol, cert.get_art(account_number),) - # create_new_listeners(account_number, elb.region, elb.name, [listener_tuple]) - - return {'message': 'Listener has been created'} - - -def update(listener_id, **kwargs): - listener = get(listener_id) - - # if the lb_port has changed we need to make sure we are deleting - # the listener on the old port to avoid listener duplication - ports = [] - if listener.load_balancer_port != kwargs.get('load_balancer_port'): - ports.append(listener.load_balancer_port) - else: - ports.append(kwargs.get('load_balancer_port')) - - certificate_id = kwargs.get('certificate_id') - - listener.instance_port = kwargs.get('instance_port') - listener.instance_protocol = kwargs.get('instance_protocol') - listener.load_balancer_port = kwargs.get('load_balancer_port') - listener.load_balancer_protocol = kwargs.get('load_balancer_protocol') - - elb = listener.elb - account_number = listener.elb.account.account_number - - arn = None - if certificate_id: - cert = verify_attachment(certificate_id, account_number) - cert.elb_listeners.append(listener) - arn = cert.get_arn(account_number) - - # remove certificate that is no longer wanted - if listener.certificate and not certificate_id: - listener.certificate.remove() - - database.update(listener) - listener_tuple = (listener.load_balancer_port, listener.instance_port, listener.load_balancer_protocol, arn,) - # update_listeners(account_number, elb.region, elb.name, [listener_tuple], ports) - - return {'message': 'Listener has been updated'} - - -def delete(listener_id): - # first try to delete the listener in aws - listener = get(listener_id) - # delete_listeners(listener.elb.account.account_number, listener.elb.region, listener.elb.name, [listener.load_balancer_port]) - # cleanup operation in lemur - database.delete(listener) - - -def render(args): - query = database.session_query(Listener) - - sort_by = args.pop('sort_by') - sort_dir = args.pop('sort_dir') - page = args.pop('page') - count = args.pop('count') - filt = args.pop('filter') - certificate_id = args.pop('certificate_id', None) - elb_id = args.pop('elb_id', None) - - if certificate_id: - query = database.get_all(Listener, certificate_id, field='certificate_id') - - if elb_id: - query = query.filter(Listener.elb_id == elb_id) - - if filt: - terms = filt.split(';') - query = database.filter(query, Listener, terms) - - query = database.find_all(query, Listener, args) - - if sort_by and sort_dir: - query = database.sort(query, Listener, sort_by, sort_dir) - - return database.paginate(query, page, count) - - -def stats(**kwargs): - attr = getattr(Listener, kwargs.get('metric')) - query = database.db.session.query(attr, func.count(attr)) - query = query.join(Listener.elb) - - if kwargs.get('account_id'): - query = query.filter(ELB.account_id == kwargs.get('account_id')) - - if kwargs.get('active') == 'true': - query = query.filter(Listener.certificate_id != None) # noqa - - items = query.group_by(attr).all() - results = [] - for key, count in items: - if key: - results.append({"key": key, "y": count}) - return results diff --git a/lemur/listeners/views.py b/lemur/listeners/views.py deleted file mode 100644 index b603d827..00000000 --- a/lemur/listeners/views.py +++ /dev/null @@ -1,128 +0,0 @@ -""" -.. module: lemur.listeners.service - :platform: Unix - :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more - :license: Apache, see LICENSE for more details. - -.. moduleauthor:: Kevin Glisson - -""" -from flask import Blueprint -from flask.ext.restful import reqparse, Api, fields - -from lemur.listeners import service -from lemur.auth.service import AuthenticatedResource -from lemur.auth.permissions import admin_permission -from lemur.common.utils import marshal_items, paginated_parser - - -mod = Blueprint('listeners', __name__) -api = Api(mod) - - -FIELDS = { - 'id': fields.Integer, - 'elbId': fields.Integer(attribute="elb_id"), - 'certificateId': fields.Integer(attribute="certificate_id"), - 'instancePort': fields.Integer(attribute="instance_port"), - 'instanceProtocol': fields.String(attribute="instance_protocol"), - 'loadBalancerPort': fields.Integer(attribute="load_balancer_port"), - 'loadBalancerProtocol': fields.String(attribute="load_balancer_protocol") -} - - -class ListenersList(AuthenticatedResource): - def __init__(self): - super(ListenersList, self).__init__() - - @marshal_items(FIELDS) - def get(self): - parser = paginated_parser.copy() - parser.add_argument('certificateId', type=int, dest='certificate_id', location='args') - args = parser.parse_args() - return service.render(args) - - -class ListenersCertificateList(AuthenticatedResource): - def __init__(self): - super(ListenersCertificateList, self).__init__() - - @marshal_items(FIELDS) - def get(self, certificate_id): - parser = paginated_parser.copy() - args = parser.parse_args() - args['certificate_id'] = certificate_id - return service.render(args) - - -class ListenersELBList(AuthenticatedResource): - def __init__(self): - super(ListenersELBList, self).__init__() - - @marshal_items(FIELDS) - def get(self, elb_id): - parser = paginated_parser.copy() - args = parser.parse_args() - args['elb_id'] = elb_id - return service.render(args) - - -class ListenersStats(AuthenticatedResource): - def __init__(self): - self.reqparse = reqparse.RequestParser() - super(ListenersStats, self).__init__() - - def get(self): - self.reqparse.add_argument('metric', type=str, location='args') - self.reqparse.add_argument('accountId', dest='account_id', location='args') - self.reqparse.add_argument('active', type=str, default='true', location='args') - - args = self.reqparse.parse_args() - - items = service.stats(**args) - return {"items": items, "total": len(items)} - - -class Listeners(AuthenticatedResource): - def __init__(self): - super(Listeners, self).__init__() - - @marshal_items(FIELDS) - def get(self, listener_id): - return service.get(listener_id) - - @admin_permission.require(http_exception=403) - @marshal_items(FIELDS) - def post(self): - self.reqparse.add_argument('elbId', type=str, dest='elb_id', required=True, location='json') - self.reqparse.add_argument('instanceProtocol', type=str, dest='instance_protocol', required=True, location='json') - self.reqparse.add_argument('instancePort', type=int, dest='instance_port', required=True, location='json') - self.reqparse.add_argument('loadBalancerProtocol', type=str, dest='load_balancer_protocol', required=True, location='json') - self.reqparse.add_argument('loadBalancerPort', type=int, dest='load_balancer_port', required=True, location='json') - self.reqparse.add_argument('certificateId', type=int, dest='certificate_id', location='json') - - args = self.reqparse.parse_args() - return service.create(**args) - - @admin_permission.require(http_exception=403) - @marshal_items(FIELDS) - def put(self, listener_id): - self.reqparse.add_argument('instanceProtocol', type=str, dest='instance_protocol', required=True, location='json') - self.reqparse.add_argument('instancePort', type=int, dest='instance_port', required=True, location='json') - self.reqparse.add_argument('loadBalancerProtocol', type=str, dest='load_balancer_protocol', required=True, location='json') - self.reqparse.add_argument('loadBalancerPort', type=int, dest='load_balancer_port', required=True, location='json') - self.reqparse.add_argument('certificateId', type=int, dest='certificate_id', location='json') - - args = self.reqparse.parse_args() - return service.update(listener_id, **args) - - @admin_permission.require(http_exception=403) - def delete(self, listener_id): - return service.delete(listener_id) - - -api.add_resource(ListenersList, '/listeners', endpoint='listeners') -api.add_resource(Listeners, '/listeners/', endpoint='listener') -api.add_resource(ListenersStats, '/listeners/stats', endpoint='listenersStats') -api.add_resource(ListenersCertificateList, '/certificates//listeners', endpoint='listenersCertificates') -api.add_resource(ListenersELBList, '/elbs//listeners', endpoint='elbListeners') diff --git a/lemur/manage.py b/lemur/manage.py index 21929980..8cee39e0 100755 --- a/lemur/manage.py +++ b/lemur/manage.py @@ -1,25 +1,27 @@ import os import sys import base64 +import time from gunicorn.config import make_settings from cryptography.fernet import Fernet +from lockfile import LockFile, LockTimeout + from flask import current_app -from flask.ext.script import Manager, Command, Option, Group, prompt_pass +from flask.ext.script import Manager, Command, Option, prompt_pass from flask.ext.migrate import Migrate, MigrateCommand, stamp from flask_script.commands import ShowUrls, Clean, Server from lemur import database from lemur.users import service as user_service from lemur.roles import service as role_service -from lemur.destinations import service as destination_service from lemur.certificates import service as cert_service - -from lemur.plugins.base import plugins +from lemur.sources import service as source_service +from lemur.notifications import service as notification_service from lemur.certificates.verify import verify_string -from lemur.certificates import sync +from lemur.sources.service import sync from lemur import create_app @@ -30,9 +32,8 @@ from lemur.authorities.models import Authority # noqa from lemur.certificates.models import Certificate # noqa from lemur.destinations.models import Destination # noqa from lemur.domains.models import Domain # noqa -from lemur.elbs.models import ELB # noqa -from lemur.listeners.models import Listener # noqa from lemur.notifications.models import Notification # noqa +from lemur.sources.models import Source # noqa manager = Manager(create_app) @@ -76,6 +77,7 @@ LEMUR_RESTRICTED_DOMAINS = [] LEMUR_EMAIL = '' LEMUR_SECURITY_TEAM_EMAIL = [] +LEMUR_DEFAULT_EXPIRATION_NOTIFICATION_INTERVALS = [30, 15, 2] # Logging @@ -91,14 +93,6 @@ SQLALCHEMY_DATABASE_URI = 'postgresql://lemur:lemur@localhost:5432/lemur' # AWS -# Lemur will need STS assume role access to every destination you want to monitor -# AWS_ACCOUNT_MAPPINGS = {{ -# '1111111111': 'myawsacount' -# }} - -## This is useful if you know you only want to monitor one destination -#AWS_REGIONS = ['us-east-1'] - #LEMUR_INSTANCE_PROFILE = 'Lemur' # Issuers @@ -177,52 +171,73 @@ def generate_settings(): return output -class Sync(Command): +@manager.option('-s', '--sources', dest='labels', default='', required=False) +@manager.option('-l', '--list', dest='view', default=False, required=False) +def sync_sources(labels, view): """ Attempts to run several methods Certificate discovery. This is run on a periodic basis and updates the Lemur datastore with the information it discovers. """ - option_list = [ - Group( - Option('-a', '--all', action="store_true"), - Option('-b', '--aws', action="store_true"), - Option('-d', '--cloudca', action="store_true"), - Option('-s', '--source', action="store_true"), - exclusive=True, required=True + if view: + sys.stdout.write("Active\tLabel\tDescription\n") + for source in source_service.get_all(): + sys.stdout.write( + "[{active}]\t{label}\t{description}!\n".format( + label=source.label, + description=source.description, + active=source.active + ) + ) + else: + start_time = time.time() + lock_file = "/tmp/.lemur_lock" + sync_lock = LockFile(lock_file) + + while not sync_lock.i_am_locking(): + try: + sync_lock.acquire(timeout=10) # wait up to 10 seconds + + if labels: + sys.stdout.write("[+] Staring to sync sources: {labels}!\n".format(labels=labels)) + labels = labels.split(",") + else: + sys.stdout.write("[+] Starting to sync ALL sources!\n") + + sync(labels=labels) + sys.stdout.write( + "[+] Finished syncing sources. Run Time: {time}\n".format( + time=(time.time() - start_time) + ) + ) + except LockTimeout: + sys.stderr.write( + "[!] Unable to acquire file lock on {file}, is there another sync running?\n".format( + file=lock_file + ) + ) + sync_lock.break_lock() + sync_lock.acquire() + sync_lock.release() + + sync_lock.release() + + +@manager.command +def notify(): + """ + Runs Lemur's notification engine, that looks for expired certificates and sends + notifications out to those that bave subscribed to them. + + :return: + """ + sys.stdout.write("Starting to notify subscribers about expiring certificates!\n") + count = notification_service.send_expiration_notifications() + sys.stdout.write( + "Finished notifying subscribers about expiring certificates! Sent {count} notifications!\n".format( + count=count ) - ] - - def run(self, all, aws, cloudca, source): - sys.stdout.write("[!] Starting to sync with external sources!\n") - - if all or aws: - sys.stdout.write("[!] Starting to sync with AWS!\n") - try: - sync.aws() - # sync_all_elbs() - sys.stdout.write("[+] Finished syncing with AWS!\n") - except Exception as e: - sys.stdout.write("[-] Syncing with AWS failed!\n") - - if all or cloudca: - sys.stdout.write("[!] Starting to sync with CloudCA!\n") - try: - sync.cloudca() - sys.stdout.write("[+] Finished syncing with CloudCA!\n") - except Exception as e: - sys.stdout.write("[-] Syncing with CloudCA failed!\n") - - sys.stdout.write("[!] Starting to sync with Source Code!\n") - - if all or source: - try: - sync.source() - sys.stdout.write("[+] Finished syncing with Source Code!\n") - except Exception as e: - sys.stdout.write("[-] Syncing with Source Code failed!\n") - - sys.stdout.write("[+] Finished syncing with external sources!\n") + ) class InitializeApp(Command): @@ -261,21 +276,20 @@ class InitializeApp(Command): else: sys.stdout.write("[-] Default user has already been created, skipping...!\n") - if current_app.config.get('AWS_ACCOUNT_MAPPINGS'): - if plugins.get('aws-destination'): - for account_name, account_number in current_app.config.get('AWS_ACCOUNT_MAPPINGS').items(): + sys.stdout.write("[+] Creating expiration email notifications!\n") + sys.stdout.write("[!] Using {recipients} as specified by LEMUR_SECURITY_TEAM_EMAIL for notifications\n") - destination = destination_service.get_by_label(account_name) + intervals = current_app.config.get("LEMUR_DEFAULT_EXPIRATION_NOTIFICATION_INTERVALS") + sys.stdout.write( + "[!] Creating {num} notifications for {intervals} days as specified by LEMUR_DEFAULT_EXPIRATION_NOTIFICATION_INTERVALS\n".format( + num=len(intervals), + intervals=",".join([str(x) for x in intervals]) + ) + ) + + recipients = current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL') + notification_service.create_default_expiration_notifications("DEFAULT_SECURITY", recipients=recipients) - options = dict(account_number=account_number) - if not destination: - destination_service.create(account_name, 'aws-destination', options, - description="This is an auto-generated AWS destination.") - sys.stdout.write("[+] Added new destination {0}:{1}!\n".format(account_number, account_name)) - else: - sys.stdout.write("[-] Account already exists, skipping...!\n") - else: - sys.stdout.write("[!] Skipping adding AWS destinations AWS plugin no available\n") sys.stdout.write("[/] Done!\n") @@ -475,7 +489,6 @@ def main(): manager.add_command("init", InitializeApp()) manager.add_command("create_user", CreateUser()) manager.add_command("create_role", CreateRole()) - manager.add_command("sync", Sync()) manager.run() diff --git a/lemur/migrations/versions/1ff763f5b80b_.py b/lemur/migrations/versions/1ff763f5b80b_.py new file mode 100644 index 00000000..e4a9af22 --- /dev/null +++ b/lemur/migrations/versions/1ff763f5b80b_.py @@ -0,0 +1,42 @@ +"""Adding in models for certificate sources + +Revision ID: 1ff763f5b80b +Revises: 4dc5ddd111b8 +Create Date: 2015-08-01 15:24:20.412725 + +""" + +# revision identifiers, used by Alembic. +revision = '1ff763f5b80b' +down_revision = '4dc5ddd111b8' + +from alembic import op +import sqlalchemy as sa + +import sqlalchemy_utils + + +def upgrade(): + ### commands auto generated by Alembic - please adjust! ### + op.create_table('sources', + sa.Column('id', sa.Integer(), nullable=False), + sa.Column('label', sa.String(length=32), nullable=True), + sa.Column('options', sqlalchemy_utils.types.json.JSONType(), nullable=True), + sa.Column('description', sa.Text(), nullable=True), + sa.Column('plugin_name', sa.String(length=32), nullable=True), + sa.PrimaryKeyConstraint('id') + ) + op.create_table('certificate_source_associations', + sa.Column('source_id', sa.Integer(), nullable=True), + sa.Column('certificate_id', sa.Integer(), nullable=True), + sa.ForeignKeyConstraint(['certificate_id'], ['certificates.id'], ondelete='cascade'), + sa.ForeignKeyConstraint(['source_id'], ['destinations.id'], ondelete='cascade') + ) + ### end Alembic commands ### + + +def downgrade(): + ### commands auto generated by Alembic - please adjust! ### + op.drop_table('certificate_source_associations') + op.drop_table('sources') + ### end Alembic commands ### diff --git a/lemur/migrations/versions/4c8915e461b3_.py b/lemur/migrations/versions/4c8915e461b3_.py new file mode 100644 index 00000000..f67d837f --- /dev/null +++ b/lemur/migrations/versions/4c8915e461b3_.py @@ -0,0 +1,41 @@ +"""Adding notifications + +Revision ID: 4c8915e461b3 +Revises: 3b718f59b8ce +Create Date: 2015-07-24 14:34:57.316273 + +""" + +# revision identifiers, used by Alembic. +revision = '4c8915e461b3' +down_revision = '3b718f59b8ce' + +from alembic import op +import sqlalchemy as sa +from sqlalchemy.dialects import postgresql + +import sqlalchemy_utils + + +def upgrade(): + ### commands auto generated by Alembic - please adjust! ### + op.create_table('notifications', + sa.Column('id', sa.Integer(), nullable=False), + sa.Column('label', sa.String(length=128), nullable=True), + sa.Column('description', sa.Text(), nullable=True), + sa.Column('options', sqlalchemy_utils.types.json.JSONType(), nullable=True), + sa.Column('active', sa.Boolean(), nullable=True), + sa.Column('plugin_name', sa.String(length=32), nullable=True), + sa.PrimaryKeyConstraint('id') + ) + op.drop_column(u'certificates', 'challenge') + op.drop_column(u'certificates', 'csr_config') + ### end Alembic commands ### + + +def downgrade(): + ### commands auto generated by Alembic - please adjust! ### + op.add_column(u'certificates', sa.Column('csr_config', sa.TEXT(), autoincrement=False, nullable=True)) + op.add_column(u'certificates', sa.Column('challenge', postgresql.BYTEA(), autoincrement=False, nullable=True)) + op.drop_table('notifications') + ### end Alembic commands ### diff --git a/lemur/migrations/versions/4dc5ddd111b8_.py b/lemur/migrations/versions/4dc5ddd111b8_.py new file mode 100644 index 00000000..8330744d --- /dev/null +++ b/lemur/migrations/versions/4dc5ddd111b8_.py @@ -0,0 +1,31 @@ +"""Creating a one-to-many relationship for notifications + +Revision ID: 4dc5ddd111b8 +Revises: 4c8915e461b3 +Create Date: 2015-07-24 15:02:04.398262 + +""" + +# revision identifiers, used by Alembic. +revision = '4dc5ddd111b8' +down_revision = '4c8915e461b3' + +from alembic import op +import sqlalchemy as sa + + +def upgrade(): + ### commands auto generated by Alembic - please adjust! ### + op.create_table('certificate_notification_associations', + sa.Column('notification_id', sa.Integer(), nullable=True), + sa.Column('certificate_id', sa.Integer(), nullable=True), + sa.ForeignKeyConstraint(['certificate_id'], ['certificates.id'], ondelete='cascade'), + sa.ForeignKeyConstraint(['notification_id'], ['notifications.id'], ondelete='cascade') + ) + ### end Alembic commands ### + + +def downgrade(): + ### commands auto generated by Alembic - please adjust! ### + op.drop_table('certificate_notification_associations') + ### end Alembic commands ### diff --git a/lemur/models.py b/lemur/models.py index 10ee07be..2ca4f6dd 100644 --- a/lemur/models.py +++ b/lemur/models.py @@ -8,9 +8,7 @@ :license: Apache, see LICENSE for more details. .. moduleauthor:: Kevin Glisson """ - from sqlalchemy import Column, Integer, ForeignKey - from lemur.database import db certificate_associations = db.Table('certificate_associations', @@ -25,12 +23,19 @@ certificate_destination_associations = db.Table('certificate_destination_associa ForeignKey('certificates.id', ondelete='cascade')) ) +certificate_source_associations = db.Table('certificate_source_associations', + Column('source_id', Integer, + ForeignKey('sources.id', ondelete='cascade')), + Column('certificate_id', Integer, + ForeignKey('certificates.id', ondelete='cascade')) + ) + certificate_notification_associations = db.Table('certificate_notification_associations', - Column('notification_id', Integer, - ForeignKey('notifications.id', ondelete='cascade')), - Column('certificate_id', Integer, - ForeignKey('certificates.id', ondelete='cascade')) - ) + Column('notification_id', Integer, + ForeignKey('notifications.id', ondelete='cascade')), + Column('certificate_id', Integer, + ForeignKey('certificates.id', ondelete='cascade')) + ) roles_users = db.Table('roles_users', Column('user_id', Integer, ForeignKey('users.id')), Column('role_id', Integer, ForeignKey('roles.id')) diff --git a/lemur/notifications/models.py b/lemur/notifications/models.py index 7318e5f3..97475686 100644 --- a/lemur/notifications/models.py +++ b/lemur/notifications/models.py @@ -17,12 +17,18 @@ from lemur.models import certificate_notification_associations class Notification(db.Model): __tablename__ = 'notifications' id = Column(Integer, primary_key=True) - label = Column(String(128)) + label = Column(String(128), unique=True) description = Column(Text()) options = Column(JSONType) active = Column(Boolean, default=True) plugin_name = Column(String(32)) - certificates = relationship("Certificate", secondary=certificate_notification_associations, passive_deletes=True, backref="notification", cascade='all,delete') + certificates = relationship( + "Certificate", + secondary=certificate_notification_associations, + passive_deletes=True, + backref="notification", + cascade='all,delete' + ) @property def plugin(self): diff --git a/lemur/notifications/service.py b/lemur/notifications/service.py index 517bc8ab..2c29d94d 100644 --- a/lemur/notifications/service.py +++ b/lemur/notifications/service.py @@ -24,6 +24,12 @@ from lemur.certificates import service as cert_service from lemur.plugins.base import plugins +def get_options(name, options): + for o in options: + if o.get('name') == name: + return o + + def _get_message_data(cert): """ Parse our the certification information needed for our notification @@ -34,7 +40,7 @@ def _get_message_data(cert): cert_dict = cert.as_dict() cert_dict['creator'] = cert.user.email cert_dict['domains'] = [x .name for x in cert.domains] - cert_dict['superseded'] = list(set([x.name for x in find_superseded(cert.domains) if cert.name != x])) + cert_dict['superseded'] = list(set([x.name for x in _find_superseded(cert) if cert.name != x])) return cert_dict @@ -44,8 +50,11 @@ def _deduplicate(messages): a roll up to the same set if the recipients are the same """ roll_ups = [] - for targets, data in messages: - for m, r in roll_ups: + for data, options in messages: + o = get_options('recipients', options) + targets = o['value'].split(',') + + for m, r, o in roll_ups: if r == targets: m.append(data) current_app.logger.info( @@ -53,7 +62,7 @@ def _deduplicate(messages): data['name'], ",".join(targets))) break else: - roll_ups.append(([data], targets, data.plugin_options)) + roll_ups.append(([data], targets, options)) return roll_ups @@ -62,21 +71,30 @@ def send_expiration_notifications(): This function will check for upcoming certificate expiration, and send out notification emails at given intervals. """ - notifications = 0 + sent = 0 - for plugin_name, notifications in database.get_all(Notification, 'active', field='status').group_by(Notification.plugin_name): - notifications += 1 + for plugin in plugins.all(plugin_type='notification'): + notifications = database.db.session.query(Notification)\ + .filter(Notification.plugin_name == plugin.slug)\ + .filter(Notification.active == True).all() # noqa - messages = _deduplicate(notifications) - plugin = plugins.get(plugin_name) + messages = [] + for n in notifications: + for c in n.certificates: + if _is_eligible_for_notifications(c): + messages.append((_get_message_data(c), n.options)) + + messages = _deduplicate(messages) for data, targets, options in messages: + sent += 1 plugin.send('expiration', data, targets, options) - current_app.logger.info("Lemur has sent {0} certification notifications".format(notifications)) + current_app.logger.info("Lemur has sent {0} certification notifications".format(sent)) + return sent -def get_domain_certificate(name): +def _get_domain_certificate(name): """ Fetch the SSL certificate currently hosted at a given domain (if any) and compare it against our all of our know certificates to determine if a new @@ -92,7 +110,7 @@ def get_domain_certificate(name): current_app.logger.info(str(e)) -def find_superseded(domains): +def _find_superseded(cert): """ Here we try to fetch any domain in the certificate to see if we can resolve it and to try and see if it is currently serving the certificate we are @@ -103,17 +121,22 @@ def find_superseded(domains): """ query = database.session_query(Certificate) ss_list = [] - for domain in domains: - dc = get_domain_certificate(domain.name) - if dc: - ss_list.append(dc) + + # determine what is current host at our domains + for domain in cert.domains: + dups = _get_domain_certificate(domain.name) + for c in dups: + if c.body != cert.body: + ss_list.append(dups) + current_app.logger.info("Trying to resolve {0}".format(domain.name)) - query = query.filter(Certificate.domains.any(Domain.name.in_([x.name for x in domains]))) + # look for other certificates that may not be hosted but cover the same domains + query = query.filter(Certificate.domains.any(Domain.name.in_([x.name for x in cert.domains]))) query = query.filter(Certificate.active == True) # noqa query = query.filter(Certificate.not_after >= arrow.utcnow().format('YYYY-MM-DD')) + query = query.filter(Certificate.body != cert.body) ss_list.extend(query.all()) - return ss_list @@ -129,8 +152,8 @@ def _is_eligible_for_notifications(cert): days = (cert.not_after - now.naive).days for notification in cert.notifications: - interval = notification.options['interval'] - unit = notification.options['unit'] + interval = get_options('interval', notification.options)['value'] + unit = get_options('unit', notification.options)['value'] if unit == 'weeks': interval *= 7 @@ -147,6 +170,63 @@ def _is_eligible_for_notifications(cert): return cert +def create_default_expiration_notifications(name, recipients): + """ + Will create standard 30, 10 and 2 day notifications for a given owner. If standard notifications + already exist these will be returned instead of new notifications. + + :param name: + :return: + """ + options = [ + { + 'name': 'unit', + 'type': 'select', + 'required': True, + 'validation': '', + 'available': ['days', 'weeks', 'months'], + 'helpMessage': 'Interval unit', + 'value': 'days', + }, + { + 'name': 'recipients', + 'type': 'str', + 'required': True, + 'validation': '^([\w+-.%]+@[\w-.]+\.[A-Za-z]{2,4},?)+$', + 'helpMessage': 'Comma delimited list of email addresses', + 'value': ','.join(recipients) + }, + ] + + intervals = current_app.config.get("LEMUR_DEFAULT_EXPIRATION_NOTIFICATION_INTERVALS") + + notifications = [] + for i in intervals: + n = get_by_label("{name}_{interval}_DAY".format(name=name, interval=i)) + if not n: + inter = [ + { + 'name': 'interval', + 'type': 'int', + 'required': True, + 'validation': '^\d+$', + 'helpMessage': 'Number of days to be alert before expiration.', + 'value': i, + } + ] + inter.extend(options) + n = create( + label="{name}_{interval}_DAY".format(name=name, interval=i), + plugin_name="email-notification", + options=list(inter), + description="Default {interval} day expiration notification".format(interval=i), + certificates=[] + ) + notifications.append(n) + + return notifications + + def create(label, plugin_name, options, description, certificates): """ Creates a new destination, that can then be used as a destination for certificates. @@ -163,7 +243,7 @@ def create(label, plugin_name, options, description, certificates): return database.create(notification) -def update(notification_id, label, options, description, certificates): +def update(notification_id, label, options, description, active, certificates): """ Updates an existing destination. @@ -178,6 +258,7 @@ def update(notification_id, label, options, description, certificates): notification.label = label notification.options = options notification.description = description + notification.active = active notification = database.update_list(notification, 'certificates', Certificate, certificates) return database.update(notification) diff --git a/lemur/notifications/views.py b/lemur/notifications/views.py index 884cea90..82bb3b86 100644 --- a/lemur/notifications/views.py +++ b/lemur/notifications/views.py @@ -110,6 +110,7 @@ class NotificationsList(AuthenticatedResource): :statuscode 200: no error """ parser = paginated_parser.copy() + parser.add_argument('active', type=bool, location='args') args = parser.parse_args() return service.render(args) @@ -346,6 +347,7 @@ class Notifications(AuthenticatedResource): """ self.reqparse.add_argument('label', type=str, location='json', required=True) self.reqparse.add_argument('plugin', type=dict, location='json', required=True) + self.reqparse.add_argument('active', type=bool, location='json') self.reqparse.add_argument('certificates', type=list, default=[], location='json') self.reqparse.add_argument('description', type=str, location='json') @@ -355,6 +357,7 @@ class Notifications(AuthenticatedResource): args['label'], args['plugin']['pluginOptions'], args['description'], + args['active'], args['certificates'] ) @@ -444,6 +447,7 @@ class CertificateNotifications(AuthenticatedResource): :statuscode 200: no error """ parser = paginated_parser.copy() + parser.add_argument('active', type=bool, location='args') args = parser.parse_args() args['certificate_id'] = certificate_id return service.render(args) diff --git a/lemur/plugins/base/v1.py b/lemur/plugins/base/v1.py index ce378b98..7026c99f 100644 --- a/lemur/plugins/base/v1.py +++ b/lemur/plugins/base/v1.py @@ -101,13 +101,18 @@ class IPlugin(local): Returns a list of tuples pointing to various resources for this plugin. >>> def get_resource_links(self): >>> return [ - >>> ('Documentation', 'http://sentry.readthedocs.org'), - >>> ('Bug Tracker', 'https://github.com/getsentry/sentry/issues'), - >>> ('Source', 'https://github.com/getsentry/sentry'), + >>> ('Documentation', 'http://lemury.readthedocs.org'), + >>> ('Bug Tracker', 'https://github.com/Netflix/lemur/issues'), + >>> ('Source', 'https://github.com/Netflix/lemur'), >>> ] """ return self.resource_links + def get_option(self, name, options): + for o in options: + if o.get(name): + return o['value'] + class Plugin(IPlugin): """ diff --git a/lemur/plugins/lemur_aws/elb.py b/lemur/plugins/lemur_aws/elb.py index d71b5013..b263d473 100644 --- a/lemur/plugins/lemur_aws/elb.py +++ b/lemur/plugins/lemur_aws/elb.py @@ -1,9 +1,9 @@ """ -.. module:: elb +.. module: elb :synopsis: Module contains some often used and helpful classes that are used to deal with ELBs -.. moduleauthor:: Kevin Glisson (kglisson@netflix.com) +.. moduleauthor:: Kevin Glisson """ import boto.ec2 diff --git a/lemur/plugins/lemur_aws/iam.py b/lemur/plugins/lemur_aws/iam.py index 9279c577..5e3bca0a 100644 --- a/lemur/plugins/lemur_aws/iam.py +++ b/lemur/plugins/lemur_aws/iam.py @@ -19,17 +19,17 @@ def get_name_from_arn(arn): return arn.split("/", 1)[1] -def upload_cert(account_number, cert, private_key, cert_chain=None): +def upload_cert(account_number, name, body, private_key, cert_chain=None): """ Upload a certificate to AWS :param account_number: - :param cert: + :param name: :param private_key: :param cert_chain: :return: """ - return assume_service(account_number, 'iam').upload_server_cert(cert.name, str(cert.body), str(private_key), + return assume_service(account_number, 'iam').upload_server_cert(name, str(body), str(private_key), cert_chain=str(cert_chain)) @@ -57,7 +57,7 @@ def get_all_server_certs(account_number): result = response['list_server_certificates_response']['list_server_certificates_result'] for cert in result['server_certificate_metadata_list']: - certs.append(cert) + certs.append(cert['arn']) if result['is_truncated'] == 'true': marker = result['marker'] @@ -72,7 +72,7 @@ def get_cert_from_arn(arn): :param arn: :return: """ - name = arn.split("/", 1)[1] + name = get_name_from_arn(arn) account_number = arn.split(":")[4] name = name.split("/")[-1] diff --git a/lemur/plugins/lemur_aws/plugin.py b/lemur/plugins/lemur_aws/plugin.py index 07b1f9a7..0c6fc09a 100644 --- a/lemur/plugins/lemur_aws/plugin.py +++ b/lemur/plugins/lemur_aws/plugin.py @@ -13,7 +13,7 @@ from lemur.plugins import lemur_aws as aws def find_value(name, options): for o in options: - if o.get(name): + if o['name'] == name: return o['value'] @@ -29,7 +29,7 @@ class AWSDestinationPlugin(DestinationPlugin): options = [ { 'name': 'accountNumber', - 'type': 'int', + 'type': 'str', 'required': True, 'validation': '/^[0-9]{12,12}$/', 'helpMessage': 'Must be a valid AWS account number!', @@ -41,8 +41,8 @@ class AWSDestinationPlugin(DestinationPlugin): # 'port': {'type': 'int'} # } - def upload(self, cert, private_key, cert_chain, options, **kwargs): - iam.upload_cert(find_value('accountNumber', options), cert, private_key, cert_chain=cert_chain) + def upload(self, name, body, private_key, cert_chain, options, **kwargs): + iam.upload_cert(find_value('accountNumber', options), name, body, private_key, cert_chain=cert_chain) e = find_value('elb', options) if e: @@ -68,14 +68,15 @@ class AWSSourcePlugin(SourcePlugin): }, ] - def get_certificates(self, **kwargs): + def get_certificates(self, options, **kwargs): certs = [] - arns = elb.get_all_server_certs(kwargs['account_number']) + arns = iam.get_all_server_certs(find_value('accountNumber', options)) for arn in arns: - cert_body = iam.get_cert_from_arn(arn) + cert_body, cert_chain = iam.get_cert_from_arn(arn) cert_name = iam.get_name_from_arn(arn) cert = dict( public_certificate=cert_body, + intermediate_certificate=cert_chain, name=cert_name ) certs.append(cert) diff --git a/lemur/plugins/lemur_email/plugin.py b/lemur/plugins/lemur_email/plugin.py index cecc6e37..90c53a67 100644 --- a/lemur/plugins/lemur_email/plugin.py +++ b/lemur/plugins/lemur_email/plugin.py @@ -19,12 +19,6 @@ from lemur.plugins import lemur_email as email from lemur.plugins.lemur_email.templates.config import env -def find_value(name, options): - for o in options: - if o.get(name): - return o['value'] - - class EmailNotificationPlugin(ExpirationNotificationPlugin): title = 'Email' slug = 'email-notification' diff --git a/lemur/plugins/service.py b/lemur/plugins/service.py index e69de29b..33965963 100644 --- a/lemur/plugins/service.py +++ b/lemur/plugins/service.py @@ -0,0 +1,7 @@ +""" +.. module: service + :platform: Unix + :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more + :license: Apache, see LICENSE for more details. +.. moduleauthor:: Kevin Glisson +""" diff --git a/lemur/elbs/__init__.py b/lemur/sources/__init__.py similarity index 100% rename from lemur/elbs/__init__.py rename to lemur/sources/__init__.py diff --git a/lemur/sources/models.py b/lemur/sources/models.py new file mode 100644 index 00000000..2d9870af --- /dev/null +++ b/lemur/sources/models.py @@ -0,0 +1,31 @@ +""" +.. module: lemur.sources.models + :platform: unix + :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more + :license: Apache, see LICENSE for more details. +.. moduleauthor:: Kevin Glisson +""" +import copy +from sqlalchemy import Column, Integer, String, Text, DateTime, Boolean +from sqlalchemy_utils import JSONType +from lemur.database import db + +from lemur.plugins.base import plugins + + +class Source(db.Model): + __tablename__ = 'sources' + id = Column(Integer, primary_key=True) + label = Column(String(32)) + options = Column(JSONType) + description = Column(Text()) + plugin_name = Column(String(32)) + active = Column(Boolean, default=True) + last_run = Column(DateTime) + + @property + def plugin(self): + p = plugins.get(self.plugin_name) + c = copy.deepcopy(p) + c.options = self.options + return c diff --git a/lemur/sources/service.py b/lemur/sources/service.py new file mode 100644 index 00000000..b097696e --- /dev/null +++ b/lemur/sources/service.py @@ -0,0 +1,198 @@ +""" +.. module: lemur.sources.service + :platform: Unix + :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more + :license: Apache, see LICENSE for more details. +.. moduleauthor:: Kevin Glisson +""" +from flask import current_app + +from lemur import database +from lemur.sources.models import Source +from lemur.certificates.models import Certificate +from lemur.certificates import service as cert_service +from lemur.destinations import service as destination_service + +from lemur.plugins.base import plugins + + +def _disassociate_certs_from_source(current_certificates, found_certificates, source_label): + missing = [] + for cc in current_certificates: + for fc in found_certificates: + if fc['public_certificate'] == cc.body: + break + else: + missing.append(cc) + + for c in missing: + for s in c.sources: + if s.label == source_label: + current_app.logger.info( + "Certificate {name} is no longer associated with {source}".format( + name=c.name, + source=source_label + ) + ) + c.sources.delete(s) + + +def sync_create(certificate, source): + cert = cert_service.import_certificate(**certificate) + cert.sources.append(source) + sync_update_destination(cert, source) + database.update(cert) + + +def sync_update(certificate, source): + for s in certificate.sources: + if s.label == source.label: + break + else: + certificate.sources.append(source) + + sync_update_destination(certificate, source) + database.update(certificate) + + +def sync_update_destination(certificate, source): + dest = destination_service.get_by_label(source.label) + if dest: + for d in certificate.destinations: + if d.label == source.label: + break + else: + certificate.destinations.append(dest) + + +def sync(labels=None): + new, updated = 0, 0 + c_certificates = cert_service.get_all_certs() + + for source in database.get_all(Source, True, field='active'): + # we should be able to specify, individual sources to sync + if labels: + if source.label not in labels: + continue + + current_app.logger.error("Retrieving certificates from {0}".format(source.label)) + s = plugins.get(source.plugin_name) + certificates = s.get_certificates(source.options) + + for certificate in certificates: + exists = cert_service.find_duplicates(certificate['public_certificate']) + + if not exists: + sync_create(certificate, source) + new += 1 + + # check to make sure that existing certificates have the current source associated with it + elif len(exists) == 1: + sync_update(exists[0], source) + updated += 1 + else: + current_app.logger.warning( + "Multiple certificates found, attempt to deduplicate the following certificates: {0}".format( + ",".join([x.name for x in exists]) + ) + ) + + # we need to try and find the absent of certificates so we can properly disassociate them when they are deleted + _disassociate_certs_from_source(c_certificates, certificates, source) + + +def create(label, plugin_name, options, description=None): + """ + Creates a new source, that can then be used as a source for certificates. + + :param label: Source common name + :param description: + :rtype : Source + :return: New source + """ + source = Source(label=label, options=options, plugin_name=plugin_name, description=description) + return database.create(source) + + +def update(source_id, label, options, description): + """ + Updates an existing source. + + :param source_id: Lemur assigned ID + :param label: Source common name + :rtype : Source + :return: + """ + source = get(source_id) + + source.label = label + source.options = options + source.description = description + + return database.update(source) + + +def delete(source_id): + """ + Deletes an source. + + :param source_id: Lemur assigned ID + """ + database.delete(get(source_id)) + + +def get(source_id): + """ + Retrieves an source by it's lemur assigned ID. + + :param source_id: Lemur assigned ID + :rtype : Source + :return: + """ + return database.get(Source, source_id) + + +def get_by_label(label): + """ + Retrieves a source by it's label + + :param label: + :return: + """ + return database.get(Source, label, field='label') + + +def get_all(): + """ + Retrieves all source currently known by Lemur. + + :return: + """ + query = database.session_query(Source) + return database.find_all(query, Source, {}).all() + + +def render(args): + sort_by = args.pop('sort_by') + sort_dir = args.pop('sort_dir') + page = args.pop('page') + count = args.pop('count') + filt = args.pop('filter') + certificate_id = args.pop('certificate_id', None) + + if certificate_id: + query = database.session_query(Source).join(Certificate, Source.certificate) + query = query.filter(Certificate.id == certificate_id) + else: + query = database.session_query(Source) + + if filt: + terms = filt.split(';') + query = database.filter(query, Source, terms) + + query = database.find_all(query, Source, args) + + if sort_by and sort_dir: + query = database.sort(query, Source, sort_by, sort_dir) + + return database.paginate(query, page, count) diff --git a/lemur/sources/views.py b/lemur/sources/views.py new file mode 100644 index 00000000..7e828f83 --- /dev/null +++ b/lemur/sources/views.py @@ -0,0 +1,367 @@ +""" +.. module: lemur.sources.views + :platform: Unix + :synopsis: This module contains all of the accounts view code. + :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more + :license: Apache, see LICENSE for more details. +.. moduleauthor:: Kevin Glisson +""" +from flask import Blueprint +from flask.ext.restful import Api, reqparse, fields +from lemur.sources import service + +from lemur.auth.service import AuthenticatedResource +from lemur.auth.permissions import admin_permission +from lemur.common.utils import paginated_parser, marshal_items + + +mod = Blueprint('sources', __name__) +api = Api(mod) + + +FIELDS = { + 'description': fields.String, + 'sourceOptions': fields.Raw(attribute='options'), + 'pluginName': fields.String(attribute='plugin_name'), + 'lastRun': fields.DateTime(attribute='last_run', dt_format='iso8061'), + 'label': fields.String, + 'id': fields.Integer, +} + + +class SourcesList(AuthenticatedResource): + """ Defines the 'sources' endpoint """ + def __init__(self): + self.reqparse = reqparse.RequestParser() + super(SourcesList, self).__init__() + + @marshal_items(FIELDS) + def get(self): + """ + .. http:get:: /sources + + The current account list + + **Example request**: + + .. sourcecode:: http + + GET /sources HTTP/1.1 + Host: example.com + Accept: application/json, text/javascript + + **Example response**: + + .. sourcecode:: http + + HTTP/1.1 200 OK + Vary: Accept + Content-Type: text/javascript + + { + "items": [ + { + "sourceOptions": [ + { + "name": "accountNumber", + "required": true, + "value": 111111111112, + "helpMessage": "Must be a valid AWS account number!", + "validation": "/^[0-9]{12,12}$/", + "type": "int" + } + ], + "pluginName": "aws-source", + "lastRun": "2015-08-01T15:40:58", + "id": 3, + "description": "test", + "label": "test" + } + ], + "total": 1 + } + + :query sortBy: field to sort on + :query sortDir: acs or desc + :query page: int. default is 1 + :query filter: key value pair. format is k=v; + :query limit: limit number. default is 10 + :reqheader Authorization: OAuth token to authenticate + :statuscode 200: no error + """ + parser = paginated_parser.copy() + args = parser.parse_args() + return service.render(args) + + @admin_permission.require(http_exception=403) + @marshal_items(FIELDS) + def post(self): + """ + .. http:post:: /sources + + Creates a new account + + **Example request**: + + .. sourcecode:: http + + POST /sources HTTP/1.1 + Host: example.com + Accept: application/json, text/javascript + + { + "sourceOptions": [ + { + "name": "accountNumber", + "required": true, + "value": 111111111112, + "helpMessage": "Must be a valid AWS account number!", + "validation": "/^[0-9]{12,12}$/", + "type": "int" + } + ], + "pluginName": "aws-source", + "id": 3, + "lastRun": "2015-08-01T15:40:58", + "description": "test", + "label": "test" + } + + **Example response**: + + .. sourcecode:: http + + HTTP/1.1 200 OK + Vary: Accept + Content-Type: text/javascript + + { + "sourceOptions": [ + { + "name": "accountNumber", + "required": true, + "value": 111111111112, + "helpMessage": "Must be a valid AWS account number!", + "validation": "/^[0-9]{12,12}$/", + "type": "int" + } + ], + "pluginName": "aws-source", + "id": 3, + "lastRun": "2015-08-01T15:40:58", + "description": "test", + "label": "test" + } + + :arg label: human readable account label + :arg description: some description about the account + :reqheader Authorization: OAuth token to authenticate + :statuscode 200: no error + """ + self.reqparse.add_argument('label', type=str, location='json', required=True) + self.reqparse.add_argument('plugin', type=dict, location='json', required=True) + self.reqparse.add_argument('description', type=str, location='json') + + args = self.reqparse.parse_args() + return service.create(args['label'], args['plugin']['slug'], args['plugin']['pluginOptions'], args['description']) + + +class Sources(AuthenticatedResource): + def __init__(self): + self.reqparse = reqparse.RequestParser() + super(Sources, self).__init__() + + @marshal_items(FIELDS) + def get(self, source_id): + """ + .. http:get:: /sources/1 + + Get a specific account + + **Example request**: + + .. sourcecode:: http + + GET /sources/1 HTTP/1.1 + Host: example.com + Accept: application/json, text/javascript + + **Example response**: + + .. sourcecode:: http + + HTTP/1.1 200 OK + Vary: Accept + Content-Type: text/javascript + + { + "sourceOptions": [ + { + "name": "accountNumber", + "required": true, + "value": 111111111112, + "helpMessage": "Must be a valid AWS account number!", + "validation": "/^[0-9]{12,12}$/", + "type": "int" + } + ], + "pluginName": "aws-source", + "id": 3, + "lastRun": "2015-08-01T15:40:58", + "description": "test", + "label": "test" + } + + :reqheader Authorization: OAuth token to authenticate + :statuscode 200: no error + """ + return service.get(source_id) + + @admin_permission.require(http_exception=403) + @marshal_items(FIELDS) + def put(self, source_id): + """ + .. http:put:: /sources/1 + + Updates an account + + **Example request**: + + .. sourcecode:: http + + POST /sources/1 HTTP/1.1 + Host: example.com + Accept: application/json, text/javascript + + { + "sourceOptions": [ + { + "name": "accountNumber", + "required": true, + "value": 111111111112, + "helpMessage": "Must be a valid AWS account number!", + "validation": "/^[0-9]{12,12}$/", + "type": "int" + } + ], + "pluginName": "aws-source", + "id": 3, + "lastRun": "2015-08-01T15:40:58", + "description": "test", + "label": "test" + } + + **Example response**: + + .. sourcecode:: http + + HTTP/1.1 200 OK + Vary: Accept + Content-Type: text/javascript + + { + "sourceOptions": [ + { + "name": "accountNumber", + "required": true, + "value": 111111111112, + "helpMessage": "Must be a valid AWS account number!", + "validation": "/^[0-9]{12,12}$/", + "type": "int" + } + ], + "pluginName": "aws-source", + "id": 3, + "lastRun": "2015-08-01T15:40:58", + "description": "test", + "label": "test" + } + + :arg accountNumber: aws account number + :arg label: human readable account label + :arg description: some description about the account + :reqheader Authorization: OAuth token to authenticate + :statuscode 200: no error + """ + self.reqparse.add_argument('label', type=str, location='json', required=True) + self.reqparse.add_argument('plugin', type=dict, location='json', required=True) + self.reqparse.add_argument('description', type=str, location='json') + + args = self.reqparse.parse_args() + return service.update(source_id, args['label'], args['plugin']['pluginOptions'], args['description']) + + @admin_permission.require(http_exception=403) + def delete(self, source_id): + service.delete(source_id) + return {'result': True} + + +class CertificateSources(AuthenticatedResource): + """ Defines the 'certificate/', endpoint='account') +api.add_resource(CertificateSources, '/certificates//sources', + endpoint='certificateSources') diff --git a/lemur/static/app/angular/certificates/certificate/certificate.js b/lemur/static/app/angular/certificates/certificate/certificate.js index 834c8206..b5253ea5 100644 --- a/lemur/static/app/angular/certificates/certificate/certificate.js +++ b/lemur/static/app/angular/certificates/certificate/certificate.js @@ -22,7 +22,7 @@ angular.module('lemur') $scope.notificationService = NotificationService; }) - .controller('CertificateCreateController', function ($scope, $modalInstance, CertificateApi, CertificateService, DestinationService, ELBService, AuthorityService, PluginService, MomentService, WizardHandler, LemurRestangular, NotificationService) { + .controller('CertificateCreateController', function ($scope, $modalInstance, CertificateApi, CertificateService, DestinationService, AuthorityService, PluginService, MomentService, WizardHandler, LemurRestangular, NotificationService) { $scope.certificate = LemurRestangular.restangularizeElement(null, {}, 'certificates'); $scope.create = function (certificate) { @@ -92,7 +92,6 @@ angular.module('lemur') $scope.plugins = plugins; }); - $scope.elbService = ELBService; $scope.authorityService = AuthorityService; $scope.destinationService = DestinationService; $scope.notificationService = NotificationService; diff --git a/lemur/static/app/angular/certificates/certificate/edit.tpl.html b/lemur/static/app/angular/certificates/certificate/edit.tpl.html new file mode 100644 index 00000000..a3cf3888 --- /dev/null +++ b/lemur/static/app/angular/certificates/certificate/edit.tpl.html @@ -0,0 +1,38 @@ +
+
+

Edit {{ certificate.name }}

+
+
+
+
+ +
+ + +

Enter a valid + email.

+
+
+
+ +
+ +

You must give a short description about this authority will be used for, this description should only include alphanumeric characters

+
+
+
+
+
+
+
+ + +
+
diff --git a/lemur/static/app/angular/certificates/certificate/notifications.tpl.html b/lemur/static/app/angular/certificates/certificate/notifications.tpl.html new file mode 100644 index 00000000..5f8136ac --- /dev/null +++ b/lemur/static/app/angular/certificates/certificate/notifications.tpl.html @@ -0,0 +1,28 @@ +
+ +
+
+ + + + +
+ + + + + + +
{{ notification.label }}{{ notification.description }} + +
+
+
diff --git a/lemur/static/app/angular/certificates/certificate/upload.js b/lemur/static/app/angular/certificates/certificate/upload.js index 685da6f0..6d51a1b4 100644 --- a/lemur/static/app/angular/certificates/certificate/upload.js +++ b/lemur/static/app/angular/certificates/certificate/upload.js @@ -2,22 +2,20 @@ angular.module('lemur') - .controller('CertificateUploadController', function ($scope, $modalInstance, CertificateService, LemurRestangular, DestinationService, NotificationService, ELBService, PluginService) { + .controller('CertificateUploadController', function ($scope, $modalInstance, CertificateService, LemurRestangular, DestinationService, NotificationService, PluginService) { $scope.certificate = LemurRestangular.restangularizeElement(null, {}, 'certificates'); $scope.upload = CertificateService.upload; $scope.destinationService = DestinationService; $scope.notificationService = NotificationService; - $scope.elbService = ELBService; PluginService.getByType('destination').then(function (plugins) { $scope.plugins = plugins; }); - $scope.attachELB = function (elb) { - $scope.certificate.attachELB(elb); - ELBService.getListeners(elb).then(function (listeners) { - $scope.certificate.elb.listeners = listeners; + $scope.save = function (certificate) { + CertificateService.upload(certificate).then(function () { + $modalInstance.close(); }); }; diff --git a/lemur/static/app/angular/certificates/certificate/upload.tpl.html b/lemur/static/app/angular/certificates/certificate/upload.tpl.html index 42723bb6..6ba63232 100644 --- a/lemur/static/app/angular/certificates/certificate/upload.tpl.html +++ b/lemur/static/app/angular/certificates/certificate/upload.tpl.html @@ -18,6 +18,16 @@ email.

+
+ +
+ +

You must give a short description about this authority will be used for, this description should only include alphanumeric characters

+
+
- +
diff --git a/lemur/static/app/angular/certificates/services.js b/lemur/static/app/angular/certificates/services.js index db9c1223..9344a3e9 100644 --- a/lemur/static/app/angular/certificates/services.js +++ b/lemur/static/app/angular/certificates/services.js @@ -77,18 +77,8 @@ angular.module('lemur') removeNotification: function (index) { this.notifications.splice(index, 1); }, - attachELB: function (elb) { - this.selectedELB = null; - if (this.elbs === undefined) { - this.elbs = []; - } - this.elbs.push(elb); - }, - removeELB: function (index) { - this.elbs.splice(index, 1); - }, findDuplicates: function () { - DomainService.findDomainByName(this.extensions.subAltNames[0]).then(function (domains) { //We should do a better job of searchin multiple domains + DomainService.findDomainByName(this.extensions.subAltNames[0]).then(function (domains) { //We should do a better job of searching for multiple domains this.duplicates = domains.total; }); }, @@ -205,18 +195,6 @@ angular.module('lemur') }); }; - CertificateService.getListeners = function (certificate) { - return certificate.getList('listeners').then(function (listeners) { - certificate.listeners = listeners; - }); - }; - - CertificateService.getELBs = function (certificate) { - return certificate.getList('listeners').then(function (elbs) { - certificate.elbs = elbs; - }); - }; - CertificateService.getDomains = function (certificate) { return certificate.getList('domains').then(function (domains) { certificate.domains = domains; diff --git a/lemur/static/app/angular/certificates/view/view.tpl.html b/lemur/static/app/angular/certificates/view/view.tpl.html index 563b4263..b19f14cd 100644 --- a/lemur/static/app/angular/certificates/view/view.tpl.html +++ b/lemur/static/app/angular/certificates/view/view.tpl.html @@ -101,14 +101,20 @@ -
- {{ notification.label }} -
+
    +
  • + {{ notification.label }} + {{ notification.description}} +
    • +
-
- {{ destination.label }} -
+
    +
  • + {{ destination.label }} + {{ destination.description }} +
    • +
diff --git a/lemur/static/app/angular/dashboard/dashboard.js b/lemur/static/app/angular/dashboard/dashboard.js index f0d94eab..69cfd554 100644 --- a/lemur/static/app/angular/dashboard/dashboard.js +++ b/lemur/static/app/angular/dashboard/dashboard.js @@ -9,13 +9,6 @@ angular.module('lemur') }) .controller('DashboardController', function ($scope, $rootScope, $filter, $location, LemurRestangular) { - var baseAccounts = LemurRestangular.all('accounts'); - - baseAccounts.getList() - .then(function (data) { - $scope.accounts = data; - }); - $scope.colours = [ { fillColor: 'rgba(41, 171, 224, 0.2)', @@ -89,4 +82,9 @@ angular.module('lemur') .then(function (data) { $scope.expiring = {labels: data.items.labels, values: [data.items.values]}; }); + + LemurRestangular.all('destinations').customGET('stats', {metric: 'certificates'}) + .then(function (data) { + $scope.destinations = {labels: data.items.labels, values: [data.items.values]}; + }); }); diff --git a/lemur/static/app/angular/dashboard/dashboard.tpl.html b/lemur/static/app/angular/dashboard/dashboard.tpl.html index a00880a4..c8d2c6a6 100644 --- a/lemur/static/app/angular/dashboard/dashboard.tpl.html +++ b/lemur/static/app/angular/dashboard/dashboard.tpl.html @@ -36,6 +36,17 @@
+
+
+
+
+

Destinations

+
+
+ +
+
+
diff --git a/lemur/static/app/angular/destinations/destination/destination.js b/lemur/static/app/angular/destinations/destination/destination.js index 7bb6b66b..321eecfb 100644 --- a/lemur/static/app/angular/destinations/destination/destination.js +++ b/lemur/static/app/angular/destinations/destination/destination.js @@ -23,6 +23,15 @@ angular.module('lemur') .controller('DestinationsEditController', function ($scope, $modalInstance, DestinationService, DestinationApi, PluginService, editId) { DestinationApi.get(editId).then(function (destination) { $scope.destination = destination; + PluginService.getByType('destination').then(function (plugins) { + $scope.plugins = plugins; + _.each($scope.plugins, function (plugin) { + if (plugin.slug === $scope.destination.pluginName) { + plugin.pluginOptions = $scope.destination.destinationOptions; + $scope.destination.plugin = plugin; + } + }); + }); }); PluginService.getByType('destination').then(function (plugins) { diff --git a/lemur/static/app/angular/elbs/elb/elb.js b/lemur/static/app/angular/elbs/elb/elb.js deleted file mode 100644 index d35be162..00000000 --- a/lemur/static/app/angular/elbs/elb/elb.js +++ /dev/null @@ -1,3 +0,0 @@ -/** - * Created by kglisson on 1/19/15. - */ diff --git a/lemur/static/app/angular/elbs/elb/elb.tpl.html b/lemur/static/app/angular/elbs/elb/elb.tpl.html deleted file mode 100644 index d0658f25..00000000 --- a/lemur/static/app/angular/elbs/elb/elb.tpl.html +++ /dev/null @@ -1,10 +0,0 @@ - - - - - - - - - - diff --git a/lemur/static/app/angular/elbs/services.js b/lemur/static/app/angular/elbs/services.js deleted file mode 100644 index 83d98dc5..00000000 --- a/lemur/static/app/angular/elbs/services.js +++ /dev/null @@ -1,59 +0,0 @@ -'use strict'; - -angular.module('lemur') - .service('ELBApi', function (LemurRestangular) { - LemurRestangular.extendModel('elbs', function (obj) { - return angular.extend(obj, { - attachListener: function (listener) { - if (this.listeners === undefined) { - this.listeners = []; - } - this.listeners.push(listener); - }, - removeListener: function (index) { - this.listeners.splice(index, 1); - } - }); - }); - return LemurRestangular.all('elbs'); - }) - .service('ELBService', function ($location, ELBApi, toaster) { - var ELBService = this; - ELBService.findELBByName = function (filterValue) { - return ELBApi.getList({'filter[name]': filterValue}) - .then(function (elbs) { - return elbs; - }); - }; - - ELBService.getListeners = function (elb) { - elb.getList('listeners').then(function (listeners) { - elb.listeners = listeners; - }); - return elb; - }; - - ELBService.create = function (elb) { - ELBApi.post(elb).then(function () { - toaster.pop({ - type: 'success', - title: 'ELB ' + elb.name, - body: 'Has been successfully created!' - }); - $location.path('elbs'); - }); - }; - - ELBService.update = function (elb) { - elb.put().then(function () { - toaster.pop({ - type: 'success', - title: 'ELB ' + elb.name, - body: 'Has been successfully updated!' - }); - $location.path('elbs'); - }); - }; - - return ELBService; - }); diff --git a/lemur/static/app/angular/elbs/view/view.js b/lemur/static/app/angular/elbs/view/view.js deleted file mode 100644 index 6a7bd825..00000000 --- a/lemur/static/app/angular/elbs/view/view.js +++ /dev/null @@ -1,34 +0,0 @@ -'use strict'; - -angular.module('lemur') - .config(function config($routeProvider) { - $routeProvider.when('/elbs', { - templateUrl: '/angular/elbs/view/view.tpl.html', - controller: 'ELBViewController' - }); - }) - - .controller('ELBViewController', function ($scope, ELBApi, ELBService, ngTableParams) { - $scope.filter = {}; - $scope.elbsTable = new ngTableParams({ - page: 1, // show first page - count: 10, // count per page - sorting: { - id: 'desc' // initial sorting - }, - filter: $scope.filter - }, { - total: 0, // length of data - getData: function ($defer, params) { - ELBApi.getList(params.url()) - .then(function (data) { - params.total(data.total); - $defer.resolve(data); - }); - } - }); - - $scope.toggleFilter = function (params) { - params.settings().$scope.show_filter = !params.settings().$scope.show_filter; - }; - }); diff --git a/lemur/static/app/angular/elbs/view/view.tpl.html b/lemur/static/app/angular/elbs/view/view.tpl.html deleted file mode 100644 index b3e4dad5..00000000 --- a/lemur/static/app/angular/elbs/view/view.tpl.html +++ /dev/null @@ -1,128 +0,0 @@ -
-
-

ELBs - Bring Balance to the Force

-
-
-
-
-
- -
-
-
-
- - - - - - - - - - - - - - -
-
{{ elb.name }}
- 		-
{{ elb.account.label }} -
- 		-
{{ elb.region }}
- 		-
- - -
- 		-
- - -
- 		-
- - -
-
- - - - - - - - - - - - - - - - - - - - - -
- Certificate NameInstance PortInstance ProtocolLoad Balancer PortLoad Balancer Protocol
-
- -
- 		-
- - -
-
-
- 		-
- -
- 		-
- -
- 		-
- -
- 		-
- -
- 		- -
-
-
-
-
diff --git a/lemur/static/app/angular/listeners/services.js b/lemur/static/app/angular/listeners/services.js deleted file mode 100644 index ef141c17..00000000 --- a/lemur/static/app/angular/listeners/services.js +++ /dev/null @@ -1,37 +0,0 @@ -'use strict'; - -angular.module('lemur') - .service('ListenerApi', function (LemurRestangular) { - return LemurRestangular.all('listeners'); - }) - .service('ListenerService', function ($location, ListenerApi) { - var ListenerService = this; - ListenerService.findListenerByName = function (filterValue) { - return ListenerApi.getList({'filter[name]': filterValue}) - .then(function (roles) { - return roles; - }); - }; - - ListenerService.create = function (role) { - ListenerApi.post(role).then(function () { - toaster.pop({ - type: 'success', - title: 'Listener ' + role.name, - body: 'Has been successfully created!' - }); - $location.path('roles/view'); - }); - }; - - ListenerService.update = function (role) { - role.put().then(function () { - toaster.pop({ - type: 'success', - title: 'Listener ' + role.name, - body: 'Has been successfully updated!' - }); - $location.path('roles/view'); - }); - }; - }); diff --git a/lemur/static/app/angular/notifications/notification/notification.js b/lemur/static/app/angular/notifications/notification/notification.js new file mode 100644 index 00000000..7cb1da50 --- /dev/null +++ b/lemur/static/app/angular/notifications/notification/notification.js @@ -0,0 +1,65 @@ +'use strict'; + +angular.module('lemur') + + .controller('NotificationsCreateController', function ($scope, $modalInstance, PluginService, NotificationService, CertificateService, LemurRestangular){ + $scope.notification = LemurRestangular.restangularizeElement(null, {}, 'notifications'); + + PluginService.getByType('notification').then(function (plugins) { + $scope.plugins = plugins; + }); + $scope.save = function (notification) { + NotificationService.create(notification).then( + function () { + $modalInstance.close(); + }, + function () { + + } + ); + }; + + $scope.cancel = function () { + $modalInstance.dismiss('cancel'); + }; + + $scope.certificateService = CertificateService; + }) + + .controller('NotificationsEditController', function ($scope, $modalInstance, NotificationService, NotificationApi, PluginService, CertificateService, editId) { + NotificationApi.get(editId).then(function (notification) { + $scope.notification = notification; + PluginService.getByType('notification').then(function (plugins) { + $scope.plugins = plugins; + _.each($scope.plugins, function (plugin) { + if (plugin.slug === $scope.notification.pluginName) { + plugin.pluginOptions = $scope.notification.notificationOptions; + $scope.notification.plugin = plugin; + } + }); + }); + NotificationService.getCertificates(notification); + }); + + PluginService.getByType('notification').then(function (plugins) { + $scope.plugins = plugins; + _.each($scope.plugins, function (plugin) { + if (plugin.slug === $scope.notification.pluginName) { + plugin.pluginOptions = $scope.notification.notificationOptions; + $scope.notification.plugin = plugin; + } + }); + }); + + $scope.save = function (notification) { + NotificationService.update(notification).then(function () { + $modalInstance.close(); + }); + }; + + $scope.cancel = function () { + $modalInstance.dismiss('cancel'); + }; + + $scope.certificateService = CertificateService; + }); diff --git a/lemur/static/app/angular/notifications/notification/notification.tpl.html b/lemur/static/app/angular/notifications/notification/notification.tpl.html new file mode 100644 index 00000000..8289ae5b --- /dev/null +++ b/lemur/static/app/angular/notifications/notification/notification.tpl.html @@ -0,0 +1,87 @@ +
+
+

CreateEdit Notification you gotta speak louder son!

+
+
+
+
+ +
+ +

You must enter an notification label

+
+
+
+ +
+ +
+
+
+ +
+ +
+
+
+ +
+ +
+ + + + +

{{ item.helpMessage }}

+
+
+ +
+
+ +
+
+ + + + +
+ + + + + + + + + + + +
{{ certificate.name }}{{ certificate.description }} + +
More
+
+
+
+
+
+ + +
+
+ diff --git a/lemur/static/app/angular/notifications/services.js b/lemur/static/app/angular/notifications/services.js new file mode 100644 index 00000000..9bb6affa --- /dev/null +++ b/lemur/static/app/angular/notifications/services.js @@ -0,0 +1,106 @@ +'use strict'; +angular.module('lemur') + .service('NotificationApi', function (LemurRestangular) { + LemurRestangular.extendModel('notifications', function (obj) { + return angular.extend(obj, { + attachCertificate: function (certificate) { + this.selectedCertificate = null; + if (this.certificates === undefined) { + this.certificates = []; + } + this.certificates.push(certificate); + }, + removeCertificate: function (index) { + this.certificate.splice(index, 1); + } + }); + }); + return LemurRestangular.all('notifications'); + }) + .service('NotificationService', function ($location, NotificationApi, PluginService, toaster) { + var NotificationService = this; + NotificationService.findNotificationsByName = function (filterValue) { + return NotificationApi.getList({'filter[label]': filterValue}) + .then(function (notifications) { + return notifications; + }); + }; + + NotificationService.getCertificates = function (notification) { + notification.getList('certificates').then(function (certificates) { + notification.certificates = certificates; + }); + }; + + NotificationService.getPlugin = function (notification) { + return PluginService.getByName(notification.pluginName).then(function (plugin) { + notification.plugin = plugin; + }); + }; + + + NotificationService.loadMoreCertificates = function (notification, page) { + notification.getList('certificates', {page: page}).then(function (certificates) { + _.each(certificates, function (certificate) { + notification.roles.push(certificate); + }); + }); + }; + + NotificationService.create = function (notification) { + return NotificationApi.post(notification).then( + function () { + toaster.pop({ + type: 'success', + title: notification.label, + body: 'Successfully created!' + }); + $location.path('notifications'); + }, + function (response) { + toaster.pop({ + type: 'error', + title: notification.label, + body: 'Was not created! ' + response.data.message + }); + }); + }; + + NotificationService.update = function (notification) { + return notification.put().then( + function () { + toaster.pop({ + type: 'success', + title: notification.label, + body: 'Successfully updated!' + }); + $location.path('notifications'); + }, + function (response) { + toaster.pop({ + type: 'error', + title: notification.label, + body: 'Was not updated! ' + response.data.message + }); + }); + }; + + NotificationService.updateActive = function (notification) { + notification.put().then( + function () { + toaster.pop({ + type: 'success', + title: notification.name, + body: 'Successfully updated!' + }); + }, + function (response) { + toaster.pop({ + type: 'error', + title: notification.name, + body: 'Was not updated! ' + response.data.message + }); + }); + }; + return NotificationService; + }); diff --git a/lemur/static/app/angular/notifications/view/view.js b/lemur/static/app/angular/notifications/view/view.js new file mode 100644 index 00000000..ae269e67 --- /dev/null +++ b/lemur/static/app/angular/notifications/view/view.js @@ -0,0 +1,96 @@ +'use strict'; + +angular.module('lemur') + + .config(function config($routeProvider) { + $routeProvider.when('/notifications', { + templateUrl: '/angular/notifications/view/view.tpl.html', + controller: 'NotificationsViewController' + }); + }) + + .controller('NotificationsViewController', function ($q, $scope, $modal, NotificationApi, NotificationService, ngTableParams, toaster) { + $scope.filter = {}; + $scope.notificationsTable = new ngTableParams({ + page: 1, // show first page + count: 10, // count per page + sorting: { + id: 'desc' // initial sorting + }, + filter: $scope.filter + }, { + total: 0, // length of data + getData: function ($defer, params) { + NotificationApi.getList(params.url()).then( + function (data) { + _.each(data, function (notification) { + NotificationService.getPlugin(notification); + }); + params.total(data.total); + $defer.resolve(data); + } + ); + } + }); + + $scope.getNotificationStatus = function () { + var def = $q.defer(); + def.resolve([{'title': 'Active', 'id': true}, {'title': 'Inactive', 'id': false}]); + return def; + }; + + $scope.remove = function (notification) { + notification.remove().then( + function () { + $scope.notificationsTable.reload(); + }, + function (response) { + toaster.pop({ + type: 'error', + title: 'Opps', + body: 'I see what you did there' + response.data.message + }); + } + ); + }; + + $scope.edit = function (notificationId) { + var modalInstance = $modal.open({ + animation: true, + templateUrl: '/angular/notifications/notification/notification.tpl.html', + controller: 'NotificationsEditController', + size: 'lg', + resolve: { + editId: function () { + return notificationId; + } + } + }); + + modalInstance.result.then(function () { + $scope.notificationsTable.reload(); + }); + + }; + + $scope.create = function () { + var modalInstance = $modal.open({ + animation: true, + controller: 'NotificationsCreateController', + templateUrl: '/angular/notifications/notification/notification.tpl.html', + size: 'lg' + }); + + modalInstance.result.then(function () { + $scope.notificationsTable.reload(); + }); + + }; + + $scope.toggleFilter = function (params) { + params.settings().$scope.show_filter = !params.settings().$scope.show_filter; + }; + + $scope.notificationService = NotificationService; + + }); diff --git a/lemur/static/app/angular/notifications/view/view.tpl.html b/lemur/static/app/angular/notifications/view/view.tpl.html new file mode 100644 index 00000000..1335e5e3 --- /dev/null +++ b/lemur/static/app/angular/notifications/view/view.tpl.html @@ -0,0 +1,52 @@ +
+
+

Notifications + you have to speak up son!

+
+
+
+ +
+
+ +
+
+
+
+ + + + + + + + + +
+
    +
  • {{ notification.label }}
    • +
  • {{ notification.description }}
    • +
+ 		+
    +
  • {{ notification.plugin.title }}
    • +
  • {{ notification.plugin.description }}
    • +
+ 		+
+ +
+ 		+
+ + +
+
+
+
+
+
diff --git a/lemur/static/app/angular/sources/services.js b/lemur/static/app/angular/sources/services.js new file mode 100644 index 00000000..d29897d1 --- /dev/null +++ b/lemur/static/app/angular/sources/services.js @@ -0,0 +1,59 @@ +'use strict'; +angular.module('lemur') + .service('SourceApi', function (LemurRestangular) { + return LemurRestangular.all('sources'); + }) + .service('SourceService', function ($location, SourceApi, PluginService, toaster) { + var SourceService = this; + SourceService.findSourcesByName = function (filterValue) { + return SourceApi.getList({'filter[label]': filterValue}) + .then(function (sources) { + return sources; + }); + }; + + SourceService.create = function (source) { + return SourceApi.post(source).then( + function () { + toaster.pop({ + type: 'success', + title: source.label, + body: 'Successfully created!' + }); + $location.path('sources'); + }, + function (response) { + toaster.pop({ + type: 'error', + title: source.label, + body: 'Was not created! ' + response.data.message + }); + }); + }; + + SourceService.update = function (source) { + return source.put().then( + function () { + toaster.pop({ + type: 'success', + title: source.label, + body: 'Successfully updated!' + }); + $location.path('sources'); + }, + function (response) { + toaster.pop({ + type: 'error', + title: source.label, + body: 'Was not updated! ' + response.data.message + }); + }); + }; + + SourceService.getPlugin = function (source) { + return PluginService.getByName(source.pluginName).then(function (plugin) { + source.plugin = plugin; + }); + }; + return SourceService; + }); diff --git a/lemur/static/app/angular/sources/source/source.js b/lemur/static/app/angular/sources/source/source.js new file mode 100644 index 00000000..b7378c77 --- /dev/null +++ b/lemur/static/app/angular/sources/source/source.js @@ -0,0 +1,56 @@ +'use strict'; + +angular.module('lemur') + + .controller('SourcesCreateController', function ($scope, $modalInstance, PluginService, SourceService, LemurRestangular){ + $scope.source = LemurRestangular.restangularizeElement(null, {}, 'sources'); + + PluginService.getByType('source').then(function (plugins) { + $scope.plugins = plugins; + }); + + $scope.save = function (source) { + SourceService.create(source).then(function () { + $modalInstance.close(); + }); + }; + + $scope.cancel = function () { + $modalInstance.dismiss('cancel'); + }; + }) + + .controller('SourcesEditController', function ($scope, $modalInstance, SourceService, SourceApi, PluginService, editId) { + SourceApi.get(editId).then(function (source) { + $scope.source = source; + PluginService.getByType('source').then(function (plugins) { + $scope.plugins = plugins; + _.each($scope.plugins, function (plugin) { + if (plugin.slug === $scope.source.pluginName) { + plugin.pluginOptions = $scope.source.sourceOptions; + $scope.source.plugin = plugin; + } + }); + }); + }); + + PluginService.getByType('source').then(function (plugins) { + $scope.plugins = plugins; + _.each($scope.plugins, function (plugin) { + if (plugin.slug === $scope.source.pluginName) { + plugin.pluginOptions = $scope.source.sourceOptions; + $scope.source.plugin = plugin; + } + }); + }); + + $scope.save = function (source) { + SourceService.update(source).then(function () { + $modalInstance.close(); + }); + }; + + $scope.cancel = function () { + $modalInstance.dismiss('cancel'); + }; + }); diff --git a/lemur/static/app/angular/sources/source/source.tpl.html b/lemur/static/app/angular/sources/source/source.tpl.html new file mode 100644 index 00000000..39e8c277 --- /dev/null +++ b/lemur/static/app/angular/sources/source/source.tpl.html @@ -0,0 +1,56 @@ +
+
+

CreateEdit Source oh the places you will go!

+
+
+
+
+ +
+ +

You must enter an source label

+
+
+
+ +
+ +
+
+
+ +
+ +
+
+
+ +
+ +
+ + + + +

{{ item.helpMessage }}

+
+
+ +
+
+
+
+ + +
+
+ diff --git a/lemur/static/app/angular/sources/view/view.js b/lemur/static/app/angular/sources/view/view.js new file mode 100644 index 00000000..031ea8d7 --- /dev/null +++ b/lemur/static/app/angular/sources/view/view.js @@ -0,0 +1,88 @@ +'use strict'; + +angular.module('lemur') + + .config(function config($routeProvider) { + $routeProvider.when('/sources', { + templateUrl: '/angular/sources/view/view.tpl.html', + controller: 'SourcesViewController' + }); + }) + + .controller('SourcesViewController', function ($scope, $modal, SourceApi, SourceService, ngTableParams, toaster) { + $scope.filter = {}; + $scope.sourcesTable = new ngTableParams({ + page: 1, // show first page + count: 10, // count per page + sorting: { + id: 'desc' // initial sorting + }, + filter: $scope.filter + }, { + total: 0, // length of data + getData: function ($defer, params) { + SourceApi.getList(params.url()).then( + function (data) { + _.each(data, function (source) { + SourceService.getPlugin(source); + }); + params.total(data.total); + $defer.resolve(data); + } + ); + } + }); + + $scope.remove = function (source) { + source.remove().then( + function () { + $scope.sourcesTable.reload(); + }, + function (response) { + toaster.pop({ + type: 'error', + title: 'Opps', + body: 'I see what you did there' + response.data.message + }); + } + ); + }; + + $scope.edit = function (sourceId) { + var modalInstance = $modal.open({ + animation: true, + templateUrl: '/angular/sources/source/source.tpl.html', + controller: 'SourcesEditController', + size: 'lg', + resolve: { + editId: function () { + return sourceId; + } + } + }); + + modalInstance.result.then(function () { + $scope.sourcesTable.reload(); + }); + + }; + + $scope.create = function () { + var modalInstance = $modal.open({ + animation: true, + controller: 'SourcesCreateController', + templateUrl: '/angular/sources/source/source.tpl.html', + size: 'lg' + }); + + modalInstance.result.then(function () { + $scope.sourcesTable.reload(); + }); + + }; + + $scope.toggleFilter = function (params) { + params.settings().$scope.show_filter = !params.settings().$scope.show_filter; + }; + + }); diff --git a/lemur/static/app/angular/sources/view/view.tpl.html b/lemur/static/app/angular/sources/view/view.tpl.html new file mode 100644 index 00000000..9655a7e2 --- /dev/null +++ b/lemur/static/app/angular/sources/view/view.tpl.html @@ -0,0 +1,47 @@ +
+
+

Sources + where are you from?

+
+
+
+ +
+
+ +
+
+
+
+ + + + + + + + +
+
    +
  • {{ source.label }}
    • +
  • {{ source.description }}
    • +
+ 		+
    +
  • {{ source.plugin.title }}
    • +
  • {{ source.plugin.description }}
    • +
+ 		+
+ + +
+
+
+
+
+
diff --git a/lemur/static/app/index.html b/lemur/static/app/index.html index 3ef93dd7..46dcd3a9 100644 --- a/lemur/static/app/index.html +++ b/lemur/static/app/index.html @@ -54,6 +54,7 @@
  • Authorities
  • Notifications
  • Destinations
    • +
  • Sources
  • Settings diff --git a/lemur/tests/test_notifications.py b/lemur/tests/test_notifications.py index e66c1984..78fe3053 100644 --- a/lemur/tests/test_notifications.py +++ b/lemur/tests/test_notifications.py @@ -6,7 +6,7 @@ def test_crud(session): notification = create('testnotify', 'email-notification', {}, 'notify1', []) assert notification.id > 0 - notification = update(notification.id, 'testnotify2', {}, 'notify2', []) + notification = update(notification.id, 'testnotify2', {}, 'notify2', True, []) assert notification.label == 'testnotify2' assert len(get_all()) == 1 diff --git a/lemur/tests/test_sources.py b/lemur/tests/test_sources.py new file mode 100644 index 00000000..8704ca23 --- /dev/null +++ b/lemur/tests/test_sources.py @@ -0,0 +1,134 @@ +from lemur.sources.service import * # noqa +from lemur.sources.views import * # noqa + +from json import dumps + + +def test_crud(session): + source = create('testdest', 'aws-source', {}, description='source1') + assert source.id > 0 + + source = update(source.id, 'testdest2', {}, 'source2') + assert source.label == 'testdest2' + + assert len(get_all()) == 1 + + delete(1) + assert len(get_all()) == 0 + + +def test_source_get(client): + assert client.get(api.url_for(Sources, source_id=1)).status_code == 401 + + +def test_source_post(client): + assert client.post(api.url_for(Sources, source_id=1), data={}).status_code == 405 + + +def test_source_put(client): + assert client.put(api.url_for(Sources, source_id=1), data={}).status_code == 401 + + +def test_source_delete(client): + assert client.delete(api.url_for(Sources, source_id=1)).status_code == 401 + + +def test_source_patch(client): + assert client.patch(api.url_for(Sources, source_id=1), data={}).status_code == 405 + + +VALID_USER_HEADER_TOKEN = { + 'Authorization': 'Basic ' + 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE0MzUyMzMzNjksInN1YiI6MSwiZXhwIjoxNTIxNTQ2OTY5fQ.1qCi0Ip7mzKbjNh0tVd3_eJOrae3rNa_9MCVdA4WtQI'} + + +def test_auth_source_get(client): + assert client.get(api.url_for(Sources, source_id=1), headers=VALID_USER_HEADER_TOKEN).status_code == 200 + + +def test_auth_source_post_(client): + assert client.post(api.url_for(Sources, source_id=1), data={}, headers=VALID_USER_HEADER_TOKEN).status_code == 405 + + +def test_auth_source_put(client): + assert client.put(api.url_for(Sources, source_id=1), data={}, headers=VALID_USER_HEADER_TOKEN).status_code == 403 + + +def test_auth_source_delete(client): + assert client.delete(api.url_for(Sources, source_id=1), headers=VALID_USER_HEADER_TOKEN).status_code == 403 + + +def test_auth_source_patch(client): + assert client.patch(api.url_for(Sources, source_id=1), data={}, headers=VALID_USER_HEADER_TOKEN).status_code == 405 + + +VALID_ADMIN_HEADER_TOKEN = { + 'Authorization': 'Basic ' + 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE0MzUyNTAyMTgsInN1YiI6MiwiZXhwIjoxNTIxNTYzODE4fQ.6mbq4-Ro6K5MmuNiTJBB153RDhlM5LGJBjI7GBKkfqA'} + + +def test_admin_source_get(client): + assert client.get(api.url_for(Sources, source_id=1), headers=VALID_ADMIN_HEADER_TOKEN).status_code == 200 + + +def test_admin_source_post(client): + assert client.post(api.url_for(Sources, source_id=1), data={}, headers=VALID_ADMIN_HEADER_TOKEN).status_code == 405 + + +def test_admin_source_put(client): + assert client.put(api.url_for(Sources, source_id=1), data={}, headers=VALID_ADMIN_HEADER_TOKEN).status_code == 400 + + +def test_admin_source_delete(client): + assert client.delete(api.url_for(Sources, source_id=1), headers=VALID_ADMIN_HEADER_TOKEN).status_code == 200 + + +def test_admin_source_patch(client): + assert client.patch(api.url_for(Sources, source_id=1), data={}, headers=VALID_ADMIN_HEADER_TOKEN).status_code == 405 + + +def test_sources_get(client): + assert client.get(api.url_for(SourcesList)).status_code == 401 + + +def test_sources_post(client): + assert client.post(api.url_for(SourcesList), data={}).status_code == 401 + + +def test_sources_put(client): + assert client.put(api.url_for(SourcesList), data={}).status_code == 405 + + +def test_sources_delete(client): + assert client.delete(api.url_for(SourcesList)).status_code == 405 + + +def test_sources_patch(client): + assert client.patch(api.url_for(SourcesList), data={}).status_code == 405 + + +def test_auth_sources_get(client): + assert client.get(api.url_for(SourcesList), headers=VALID_USER_HEADER_TOKEN).status_code == 200 + + +def test_auth_sources_post(client): + assert client.post(api.url_for(SourcesList), data={}, headers=VALID_USER_HEADER_TOKEN).status_code == 403 + + +def test_admin_sources_get(client): + resp = client.get(api.url_for(SourcesList), headers=VALID_ADMIN_HEADER_TOKEN) + assert resp.status_code == 200 + assert resp.json == {'items': [], 'total': 0} + + +def test_admin_sources_crud(client): + assert client.post(api.url_for(SourcesList), headers=VALID_ADMIN_HEADER_TOKEN).status_code == 400 + data = {'plugin': {'slug': 'aws-source', 'pluginOptions': {}}, 'label': 'test', 'description': 'test'} + resp = client.post(api.url_for(SourcesList), data=dumps(data), content_type='application/json', headers=VALID_ADMIN_HEADER_TOKEN) + assert resp.status_code == 200 + assert client.get(api.url_for(Sources, source_id=resp.json['id']), headers=VALID_ADMIN_HEADER_TOKEN).status_code == 200 + resp = client.get(api.url_for(SourcesList), headers=VALID_ADMIN_HEADER_TOKEN) + assert resp.status_code == 200 + assert resp.json['items'][0]['description'] == 'test' + assert client.delete(api.url_for(Sources, source_id=2), headers=VALID_ADMIN_HEADER_TOKEN).status_code == 200 + resp = client.get(api.url_for(SourcesList), headers=VALID_ADMIN_HEADER_TOKEN) + assert resp.status_code == 200 + assert resp.json == {'items': [], 'total': 0} diff --git a/setup.py b/setup.py index 7371881c..b508687f 100644 --- a/setup.py +++ b/setup.py @@ -42,7 +42,8 @@ install_requires = [ 'cryptography>=1.0dev', 'pyopenssl==0.15.1', 'pyjwt==1.0.1', - 'xmltodict==0.9.2' + 'xmltodict==0.9.2', + 'lockfile==0.10.2' ] tests_require = [ @@ -135,10 +136,10 @@ setup( 'lemur.plugins': [ 'verisign_issuer = lemur.plugins.lemur_verisign.plugin:VerisignIssuerPlugin', 'cloudca_issuer = lemur.plugins.lemur_cloudca.plugin:CloudCAIssuerPlugin', - 'cloudca_source = lemur.plugins.lemur_cloudca.plugin:CloudCASourcePlugin' + 'cloudca_source = lemur.plugins.lemur_cloudca.plugin:CloudCASourcePlugin', 'aws_destination = lemur.plugins.lemur_aws.plugin:AWSDestinationPlugin', - 'aws_source = lemur.plugins.lemur_aws.plugin:AWSSourcePlugin' - 'email_notification = lemur.plugins.lemur_email.plugin:EmailNotificationPlugin' + 'aws_source = lemur.plugins.lemur_aws.plugin:AWSSourcePlugin', + 'email_notification = lemur.plugins.lemur_email.plugin:EmailNotificationPlugin', ], }, classifiers=[