Merge pull request #29 from kevgliss/sources

Adding ability to define sources for lemur to sync with

docs/administration/index.rst

@@ -150,9 +150,6 @@ Lemur supports sending certification expiration notifications through SES and SM
         LEMUR_SECURITY_TEAM_EMAIL = ['security@example.com']
 
 
-.. data::
-
-
 Authority Options
 -----------------
 
@@ -505,11 +502,19 @@ All commands default to `~/.lemur/lemur.conf.py` if a configuration is not speci
 
 .. data:: sync
 
-    Sync attempts to discover certificates in the environment that were not created by Lemur. There
+    Sync attempts to discover certificates in the environment that were not created by Lemur. If you wish to only sync
+    a few sources you can pass a comma delimited list of sources to sync
 
     ::
 
-        lemur sync --all
+        lemur sync source1,source2
+
+
+    Additionally you can also list the available sources that Lemur can sync
+
+    ::
+
+        lemur sync -list
 
 
 Identity and Access Management

docs/developer/internals/lemur.certificates.rst

@@ -28,15 +28,6 @@ certificates Package
     :undoc-members:
     :show-inheritance:
 
-:mod:`sync` Module
-------------------
-
-.. automodule:: lemur.certificates.sync
-    :noindex:
-    :members:
-    :undoc-members:
-    :show-inheritance:
-
 :mod:`verify` Module
 --------------------
 

docs/developer/plugins/index.rst

@@ -215,9 +215,9 @@ certificate Lemur does not know about and adding the certificate to it's invento
 The `SourcePlugin` object has one default option of `pollRate`. This controls the number of seconds which to get new certificates.
 
  .. warning::
-    Lemur currently has a very basic polling system of running a cron job every 15min to see which source plugins need to be run.
+    Lemur currently has a very basic polling system of running a cron job every 15min to see which source plugins need to be run. A lock file is generated to guarentee that ]
-    This means special consideration needs to be taken such that running all `SourcePlugins` does not take >15min to run. It also means
+    only one sync is running at a time. It also means that the minimum resolution of a source plugin poll rate is effectively 15min. You can always specify a faster cron
-    that the minimum resolution of a source plugin poll rate is effectively 15min.
+    job if you need a higher resolution sync job.
 
 
 The `SourcePlugin` object requires implementation of one function::

docs/quickstart/index.rst

@@ -42,13 +42,13 @@ Finally, activate your virtualenv::
 Installing build dependencies
 -----------------------------
 
-If installing Lemur on true bare Ubuntu OS you will need to grab the following packages so that Lemur can correctly build it's
+If installing Lemur on truely bare Ubuntu OS you will need to grab the following packages so that Lemur can correctly build it's
-dependencies.
+dependencies::
 
     $ sudo apt-get update
     $ sudo apt-get install nodejs-legacy python-pip libpq-dev python-dev build-essential libssl-dev libffi-dev nginx git supervisor
 
-And optionally if your database is going to be on the same host as the webserver.
+And optionally if your database is going to be on the same host as the webserver::
 
     $ sudo apt-get install postgres
 
@@ -110,7 +110,7 @@ Update your configuration
 Once created you will need to update the configuration file with information about your environment,
 such as which database to talk to, where keys are stores etc..
 
-.. Note:: If you are unVfamiliar with with the SQLALCHEMY_DATABASE_URI string it can be broken up like so:
+.. Note:: If you are unfamiliar with with the SQLALCHEMY_DATABASE_URI string it can be broken up like so:
       postgresql://userame:password@databasefqdn:databaseport/databasename
 
 Setup Postgres
@@ -119,7 +119,7 @@ Setup Postgres
 For production a dedicated database is recommended, for this guide we will assume postgres has been installed and is on
 the same machine that Lemur is installed on.
 
-First, set a password for the postgres user.  For this guide, we will use **lemur** as an example but you should use the database password generated for by Lemur.::
+First, set a password for the postgres user.  For this guide, we will use **lemur** as an example but you should use the database password generated for by Lemur::
 
      $ sudo -u postgres psql postgres
      # \password postgres
@@ -139,10 +139,17 @@ Initializing Lemur
 
 Lemur provides a helpful command that will initialize your database for you. It creates a default user (lemur) that is
 used by Lemur to help associate certificates that do not currently have an owner. This is most commonly the case when
-Lemur has discovered certificates from a third party resource. This is also a default user that can be used to
+Lemur has discovered certificates from a third party source. This is also a default user that can be used to
 administer Lemur.
 
-**Make note of the password used as this will be use to first login to the Lemur UI**
+In addition to create a new User, Lemur also creates a few default email notifications. These notifications are based
+on a few configuration options such as `LEMUR_SECURITY_TEAM_EMAIL` they basically garentee that every cerificate within
+Lemur will send one expiration notification to the security team.
+
+Additional notifications can be created through the UI or API.
+See :ref:`Creating Notifications <CreatingNotifications>` and :ref:`Command Line Interface <CommandLineInterface>` for details.
+
+**Make note of the password used as this will be used during first login to the Lemur UI**
 
 .. code-block:: bash
 

lemur/__init__.py

@@ -14,28 +14,27 @@ from lemur.users.views import mod as users_bp
 from lemur.roles.views import mod as roles_bp
 from lemur.auth.views import mod as auth_bp
 from lemur.domains.views import mod as domains_bp
-from lemur.elbs.views import mod as elbs_bp
 from lemur.destinations.views import mod as destinations_bp
 from lemur.authorities.views import mod as authorities_bp
-from lemur.listeners.views import mod as listeners_bp
 from lemur.certificates.views import mod as certificates_bp
 from lemur.status.views import mod as status_bp
 from lemur.plugins.views import mod as plugins_bp
 from lemur.notifications.views import mod as notifications_bp
+from lemur.sources.views import mod as sources_bp
+
 
 LEMUR_BLUEPRINTS = (
     users_bp,
     roles_bp,
     auth_bp,
     domains_bp,
-    elbs_bp,
     destinations_bp,
     authorities_bp,
-    listeners_bp,
     certificates_bp,
     status_bp,
     plugins_bp,
     notifications_bp,
+    sources_bp
 )
 
 

lemur/authorities/service.py

@@ -9,10 +9,12 @@
 
 """
 from flask import g
+from flask import current_app
 
 from lemur import database
 from lemur.authorities.models import Authority
 from lemur.roles import service as role_service
+from lemur.notifications import service as notification_service
 
 from lemur.roles.models import Role
 from lemur.certificates.models import Certificate
@@ -56,9 +58,15 @@ def create(kwargs):
     cert.description = "This is the ROOT certificate for the {0} certificate authority".format(kwargs.get('caName'))
     cert.user = g.current_user
 
+    cert.notifications = notification_service.create_default_expiration_notifications(
+        'DEFAULT_SECURITY',
+        current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL')
+    )
+
     # we create and attach any roles that the issuer gives us
     role_objs = []
     for r in issuer_roles:
+
         role = role_service.create(
             r['name'],
             password=r['password'],

lemur/certificates/models.py

@@ -23,7 +23,9 @@ from lemur.plugins.base import plugins
 from lemur.domains.models import Domain
 
 from lemur.constants import SAN_NAMING_TEMPLATE, DEFAULT_NAMING_TEMPLATE
-from lemur.models import certificate_associations, certificate_destination_associations, certificate_notification_associations
+
+from lemur.models import certificate_associations, certificate_source_associations, \
+    certificate_destination_associations, certificate_notification_associations
 
 
 def create_name(issuer, not_before, not_after, subject, san):
@@ -222,8 +224,8 @@ class Certificate(db.Model):
     authority_id = Column(Integer, ForeignKey('authorities.id'))
     notifications = relationship("Notification", secondary=certificate_notification_associations, backref='certificate')
     destinations = relationship("Destination", secondary=certificate_destination_associations, backref='certificate')
+    sources = relationship("Source", secondary=certificate_source_associations, backref='certificate')
     domains = relationship("Domain", secondary=certificate_associations, backref="certificate")
-    elb_listeners = relationship("Listener", lazy='dynamic', backref='certificate')
 
     def __init__(self, body, private_key=None, chain=None):
         self.body = body
@@ -277,5 +279,4 @@ class Certificate(db.Model):
 @event.listens_for(Certificate.destinations, 'append')
 def update_destinations(target, value, initiator):
     destination_plugin = plugins.get(value.plugin_name)
-
+    destination_plugin.upload(target.name, target.body, target.private_key, target.chain, value.options)
-    destination_plugin.upload(target.body, target.private_key, target.chain, value.options)

lemur/certificates/service.py

@@ -134,8 +134,11 @@ def import_certificate(**kwargs):
     :param kwargs:
     """
     from lemur.users import service as user_service
-    cert = Certificate(kwargs['public_certificate'])
+    from lemur.notifications import service as notification_service
-    cert.owner = kwargs.get('owner', current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL'))
+    cert = Certificate(kwargs['public_certificate'], chain=kwargs['intermediate_certificate'])
+
+    # TODO future source plugins might have a better understanding of who the 'owner' is we should support this
+    cert.owner = kwargs.get('owner', current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL')[0])
     cert.creator = kwargs.get('creator', user_service.get_by_email('lemur@nobody'))
 
     # NOTE existing certs may not follow our naming standard we will
@@ -146,7 +149,9 @@ def import_certificate(**kwargs):
     if kwargs.get('user'):
         cert.user = kwargs.get('user')
 
-    database.update_list(cert, 'notifications', Notification, kwargs.get('notifications'))
+    notification_name = 'DEFAULT_SECURITY'
+    notifications = notification_service.create_default_expiration_notifications(notification_name, current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL'))
+    cert.notifications = notifications
 
     cert = database.create(cert)
     return cert
@@ -156,18 +161,35 @@ def upload(**kwargs):
     """
     Allows for pre-made certificates to be imported into Lemur.
     """
+    from lemur.notifications import service as notification_service
     cert = Certificate(
         kwargs.get('public_cert'),
         kwargs.get('private_key'),
         kwargs.get('intermediate_cert'),
     )
 
-    database.update_list(cert, 'destinations', Destination, kwargs.get('destinations'))
+    cert.description = kwargs.get('description')
-    database.update_list(cert, 'notifications', Notification, kwargs.get('notifications'))
 
     cert.owner = kwargs['owner']
     cert = database.create(cert)
+
     g.user.certificates.append(cert)
+
+    database.update_list(cert, 'destinations', Destination, kwargs.get('destinations'))
+
+    database.update_list(cert, 'notifications', Notification, kwargs.get('notifications'))
+
+    # create default notifications for this certificate if none are provided
+    notifications = []
+    if not kwargs.get('notifications'):
+        notification_name = "DEFAULT_{0}".format(cert.owner.split('@')[0].upper())
+        notifications += notification_service.create_default_expiration_notifications(notification_name, [cert.owner])
+
+    notification_name = 'DEFAULT_SECURITY'
+    notifications += notification_service.create_default_expiration_notifications(notification_name, current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL'))
+    cert.notifications = notifications
+
+    database.update(cert)
     return cert
 
 
@@ -175,12 +197,11 @@ def create(**kwargs):
     """
     Creates a new certificate.
     """
+    from lemur.notifications import service as notification_service
     cert, private_key, cert_chain = mint(kwargs)
 
     cert.owner = kwargs['owner']
 
-    database.update_list(cert, 'destinations', Destination, kwargs.get('destinations'))
-
     database.create(cert)
     cert.description = kwargs['description']
     g.user.certificates.append(cert)
@@ -188,7 +209,20 @@ def create(**kwargs):
 
     # do this after the certificate has already been created because if it fails to upload to the third party
     # we do not want to lose the certificate information.
+    database.update_list(cert, 'destinations', Destination, kwargs.get('destinations'))
+
     database.update_list(cert, 'notifications', Notification, kwargs.get('notifications'))
+
+    # create default notifications for this certificate if none are provided
+    notifications = []
+    if not kwargs.get('notifications'):
+        notification_name = "DEFAULT_{0}".format(cert.owner.split('@')[0].upper())
+        notifications += notification_service.create_default_expiration_notifications(notification_name, [cert.owner])
+
+    notification_name = 'DEFAULT_SECURITY'
+    notifications += notification_service.create_default_expiration_notifications(notification_name, current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL'))
+    cert.notifications = notifications
+
     database.update(cert)
     return cert
 
@@ -297,7 +331,7 @@ def create_csr(csr_config):
                 x509.SubjectAlternativeName(general_names), critical=True
             )
 
-    # TODO support more CSR options, none of the authorities support these atm
+    # TODO support more CSR options, none of the authority plugins currently support these options
     #    builder.add_extension(
     #        x509.KeyUsage(
     #            digital_signature=digital_signature,
@@ -365,14 +399,6 @@ def stats(**kwargs):
     :param kwargs:
     :return:
     """
-    query = database.session_query(Certificate)
-
-    if kwargs.get('active') == 'true':
-        query = query.filter(Certificate.elb_listeners.any())
-
-    if kwargs.get('destination_id'):
-        query = query.filter(Certificate.destinations.any(Destination.id == kwargs.get('destination_id')))
-
     if kwargs.get('metric') == 'not_after':
         start = arrow.utcnow()
         end = start.replace(weeks=+32)
@@ -385,10 +411,6 @@ def stats(**kwargs):
         attr = getattr(Certificate, kwargs.get('metric'))
         query = database.db.session.query(attr, func.count(attr))
 
-        # TODO this could be cleaned up
-        if kwargs.get('active') == 'true':
-            query = query.filter(Certificate.elb_listeners.any())
-
         items = query.group_by(attr).all()
 
     keys = []

lemur/certificates/sync.py

@@ -1,46 +0,0 @@
-"""
-.. module: sync
-    :platform: Unix
-    :synopsis: This module contains various certificate syncing operations.
-    Because of the nature of the SSL environment there are multiple ways
-    a certificate could be created without Lemur's knowledge. Lemur attempts
-    to 'sync' with as many different datasources as possible to try and track
-    any certificate that may be in use.
-
-    These operations are typically run on a periodic basis from either the command
-    line or a cron job.
-
-    :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
-    :license: Apache, see LICENSE for more details.
-.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
-"""
-from flask import current_app
-
-from lemur.certificates import service as cert_service
-
-from lemur.plugins.base import plugins
-from lemur.plugins.bases.source import SourcePlugin
-
-
-def sync():
-    for plugin in plugins:
-        new = 0
-        updated = 0
-        if isinstance(plugin, SourcePlugin):
-            if plugin.is_enabled():
-                current_app.logger.error("Retrieving certificates from {0}".format(plugin.title))
-                certificates = plugin.get_certificates()
-
-                for certificate in certificates:
-                    exists = cert_service.find_duplicates(certificate)
-
-                    if not exists:
-                        cert_service.import_certificate(**certificate)
-                        new += 1
-
-                    if len(exists) == 1:
-                        updated += 1
-
-                # TODO associated cert with source
-                # TODO update cert if found from different source
-                # TODO disassociate source if missing

lemur/certificates/views.py

@@ -369,6 +369,7 @@ class CertificatesUpload(AuthenticatedResource):
            :statuscode 403: unauthenticated
            :statuscode 200: no error
         """
+        self.reqparse.add_argument('description', type=str, location='json')
         self.reqparse.add_argument('owner', type=str, required=True, location='json')
         self.reqparse.add_argument('publicCert', type=pem_str, required=True, dest='public_cert', location='json')
         self.reqparse.add_argument('destinations', type=list, default=[], dest='destinations', location='json')

lemur/database.py

@@ -9,6 +9,8 @@
 
 .. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
 """
+from flask import current_app
+
 from sqlalchemy import exc
 from sqlalchemy.sql import and_, or_
 
@@ -124,7 +126,8 @@ def get(model, value, field="id"):
     query = session_query(model)
     try:
         return query.filter(getattr(model, field) == value).one()
-    except Exception:
+    except Exception as e:
+        current_app.logger.exception(e)
         return
 
 

lemur/destinations/service.py

@@ -5,6 +5,8 @@
     :license: Apache, see LICENSE for more details.
 .. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
 """
+from sqlalchemy import func
+
 from lemur import database
 from lemur.destinations.models import Destination
 from lemur.certificates.models import Certificate
@@ -28,9 +30,8 @@ def update(destination_id, label, options, description):
     Updates an existing destination.
 
     :param destination_id:  Lemur assigned ID
-    :param destination_number: AWS assigned ID
     :param label: Destination common name
-    :param comments:
+    :param description:
     :rtype : Destination
     :return:
     """
@@ -107,3 +108,24 @@ def render(args):
         query = database.sort(query, Destination, sort_by, sort_dir)
 
     return database.paginate(query, page, count)
+
+
+def stats(**kwargs):
+    """
+    Helper that defines some useful statistics about destinations.
+
+    :param kwargs:
+    :return:
+    """
+    attr = getattr(Destination, kwargs.get('metric'))
+    query = database.db.session.query(attr, func.count(attr))
+
+    items = query.group_by(attr).all()
+
+    keys = []
+    values = []
+    for key, count in items:
+        keys.append(key)
+        values.append(count)
+
+    return {'labels': keys, 'values': values}

lemur/destinations/views.py

@@ -353,7 +353,21 @@ class CertificateDestinations(AuthenticatedResource):
         return service.render(args)
 
 
+class DestinationsStats(AuthenticatedResource):
+    """ Defines the 'certificates' stats endpoint """
+    def __init__(self):
+        self.reqparse = reqparse.RequestParser()
+        super(DestinationsStats, self).__init__()
+
+    def get(self):
+        self.reqparse.add_argument('metric', type=str, location='args')
+        args = self.reqparse.parse_args()
+        items = service.stats(**args)
+        return dict(items=items, total=len(items))
+
+
 api.add_resource(DestinationsList, '/destinations', endpoint='destinations')
-api.add_resource(Destinations, '/destinations/<int:destination_id>', endpoint='account')
+api.add_resource(Destinations, '/destinations/<int:destination_id>', endpoint='destination')
 api.add_resource(CertificateDestinations, '/certificates/<int:certificate_id>/destinations',
                  endpoint='certificateDestinations')
+api.add_resource(DestinationsStats, '/destinations/stats', endpoint='destinationStats')

lemur/elbs/models.py

@@ -1,44 +0,0 @@
-"""
-.. module: lemur.elbs.models
-    :platform: Unix
-    :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
-    :license: Apache, see LICENSE for more details.
-
-.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
-"""
-from sqlalchemy import Column, BigInteger, String, DateTime, PassiveDefault, func
-from sqlalchemy.orm import relationship
-
-from lemur.database import db
-from lemur.listeners.models import Listener
-
-
-class ELB(db.Model):
-    __tablename__ = 'elbs'
-    id = Column(BigInteger, primary_key=True)
-    # account_id = Column(BigInteger, ForeignKey("accounts.id"), index=True)
-    region = Column(String(32))
-    name = Column(String(128))
-    vpc_id = Column(String(128))
-    scheme = Column(String(128))
-    dns_name = Column(String(128))
-    listeners = relationship("Listener", backref='elb', cascade="all, delete, delete-orphan")
-    date_created = Column(DateTime, PassiveDefault(func.now()), nullable=False)
-
-    def __init__(self, elb_obj=None):
-        if elb_obj:
-            self.region = elb_obj.connection.region.name
-            self.name = elb_obj.name
-            self.vpc_id = elb_obj.vpc_id
-            self.scheme = elb_obj.scheme
-            self.dns_name = elb_obj.dns_name
-            for listener in elb_obj.listeners:
-                self.listeners.append(Listener(listener))
-
-    def as_dict(self):
-        return {c.name: getattr(self, c.name) for c in self.__table__.columns}
-
-    def serialize(self):
-        blob = self.as_dict()
-        del blob['date_created']
-        return blob

lemur/elbs/service.py

@@ -1,124 +0,0 @@
-"""
-.. module: lemur.elbs.service
-    :platform: Unix
-    :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
-    :license: Apache, see LICENSE for more details.
-
-.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
-
-"""
-from sqlalchemy import func
-from sqlalchemy.sql import and_
-
-from lemur import database
-from lemur.elbs.models import ELB
-from lemur.listeners.models import Listener
-
-
-def get_all(account_id, elb_name):
-    """
-    Retrieves all ELBs in a given account
-
-    :param account_id:
-    :param elb_name:
-    :rtype : Elb
-    :return:
-    """
-    query = database.session_query(ELB)
-    return query.filter(and_(ELB.name == elb_name, ELB.account_id == account_id)).all()
-
-
-def get_by_region_and_account(region, account_id):
-    query = database.session_query(ELB)
-    return query.filter(and_(ELB.region == region, ELB.account_id == account_id)).all()
-
-
-def get_all_elbs():
-    """
-    Get all ELBs that Lemur knows about
-
-    :rtype : list
-    :return:
-    """
-    return ELB.query.all()
-
-
-def get(elb_id):
-    """
-    Retrieve an ELB with a give ID
-
-    :rtype : Elb
-    :param elb_id:
-    :return:
-    """
-    return database.get(ELB, elb_id)
-
-
-def create(account, elb):
-    """
-    Create a new ELB
-
-    :param account:
-    :param elb:
-    """
-    elb = ELB(elb)
-    account.elbs.append(elb)
-    database.create(elb)
-
-
-def delete(elb_id):
-    """
-    Delete an ELB
-
-    :param elb_id:
-    """
-    database.delete(get(elb_id))
-
-
-def render(args):
-    query = database.session_query(ELB)
-
-    sort_by = args.pop('sort_by')
-    sort_dir = args.pop('sort_dir')
-    page = args.pop('page')
-    count = args.pop('count')
-    filt = args.pop('filter')
-    active = args.pop('active')
-    certificate_id = args.pop('certificate_id')
-
-    if certificate_id:
-        query.filter(ELB.listeners.any(Listener.certificate_id == certificate_id))
-
-    if active == 'true':
-        query = query.filter(ELB.listeners.any())
-
-    if filt:
-        terms = filt.split(';')
-        query = database.filter(query, ELB, terms)
-
-    query = database.find_all(query, ELB, args)
-
-    if sort_by and sort_dir:
-        query = database.sort(query, ELB, sort_by, sort_dir)
-
-    return database.paginate(query, page, count)
-
-
-def stats(**kwargs):
-    attr = getattr(ELB, kwargs.get('metric'))
-    query = database.db.session.query(attr, func.count(attr))
-
-    if kwargs.get('account_id'):
-        query = query.filter(ELB.account_id == kwargs.get('account_id'))
-
-    if kwargs.get('active') == 'true':
-        query = query.join(ELB.listeners)
-        query = query.filter(Listener.certificate_id != None)  # noqa
-
-    items = query.group_by(attr).all()
-
-    results = []
-    for key, count in items:
-        if key:
-            results.append({"key": key, "y": count})
-    return results

lemur/elbs/views.py

@@ -1,78 +0,0 @@
-"""
-.. module: lemur.elbs.service
-    :platform: Unix
-    :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
-    :license: Apache, see LICENSE for more details.
-
-.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
-
-"""
-from flask import Blueprint
-from flask.ext.restful import reqparse, Api, fields
-from lemur.elbs import service
-from lemur.auth.service import AuthenticatedResource
-
-from lemur.common.utils import marshal_items, paginated_parser
-
-
-mod = Blueprint('elbs', __name__)
-api = Api(mod)
-
-
-FIELDS = {
-    'name': fields.String,
-    'id': fields.Integer,
-    'region': fields.String,
-    'scheme': fields.String,
-    'accountId': fields.Integer(attribute='account_id'),
-    'vpcId': fields.String(attribute='vpc_id')
-}
-
-
-class ELBsList(AuthenticatedResource):
-    """ Defines the 'elbs' endpoint """
-    def __init__(self):
-        super(ELBsList, self).__init__()
-
-    @marshal_items(FIELDS)
-    def get(self):
-        parser = paginated_parser.copy()
-        parser.add_argument('owner', type=str, location='args')
-        parser.add_argument('id', type=str, location='args')
-        parser.add_argument('accountId', type=str, dest='account_id', location='args')
-        parser.add_argument('certificateId', type=str, dest='certificate_id', location='args')
-        parser.add_argument('active', type=str, default='true', location='args')
-
-        args = parser.parse_args()
-        return service.render(args)
-
-
-class ELBsStats(AuthenticatedResource):
-    def __init__(self):
-        self.reqparse = reqparse.RequestParser()
-        super(ELBsStats, self).__init__()
-
-    def get(self):
-        self.reqparse.add_argument('metric', type=str, location='args')
-        self.reqparse.add_argument('accountId', dest='account_id', location='args')
-        self.reqparse.add_argument('active', type=str, default='true', location='args')
-
-        args = self.reqparse.parse_args()
-
-        items = service.stats(**args)
-        return {"items": items, "total": len(items)}
-
-
-class ELBs(AuthenticatedResource):
-    def __init__(self):
-        self.reqparse = reqparse.RequestParser()
-        super(ELBs, self).__init__()
-
-    @marshal_items(FIELDS)
-    def get(self, elb_id):
-        return service.get(elb_id)
-
-
-api.add_resource(ELBsList, '/elbs', endpoint='elbs')
-api.add_resource(ELBs, '/elbs/<int:elb_id>', endpoint='elb')
-api.add_resource(ELBsStats, '/elbs/stats', endpoint='elbsStats')

lemur/listeners/models.py

@@ -1,42 +0,0 @@
-"""
-.. module: lemur.elbs.models
-    :platform: Unix
-    :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
-    :license: Apache, see LICENSE for more details.
-
-.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
-
-"""
-from sqlalchemy import Column, Integer, BigInteger, String, ForeignKey, DateTime, PassiveDefault, func
-
-from lemur.database import db
-from lemur.certificates import service as cert_service
-from lemur.certificates.models import Certificate, get_name_from_arn
-
-
-class Listener(db.Model):
-    __tablename__ = 'listeners'
-    id = Column(BigInteger, primary_key=True)
-    certificate_id = Column(Integer, ForeignKey(Certificate.id), index=True)
-    elb_id = Column(BigInteger, ForeignKey("elbs.id"), index=True)
-    instance_port = Column(Integer)
-    instance_protocol = Column(String(16))
-    load_balancer_port = Column(Integer)
-    load_balancer_protocol = Column(String(16))
-    date_created = Column(DateTime, PassiveDefault(func.now()), nullable=False)
-
-    def __init__(self, listener):
-        self.load_balancer_port = listener.load_balancer_port
-        self.load_balancer_protocol = listener.protocol
-        self.instance_port = listener.instance_port
-        self.instance_protocol = listener.instance_protocol
-        if listener.ssl_certificate_id not in ["Invalid-Certificate", None]:
-            self.certificate_id = cert_service.get_by_name(get_name_from_arn(listener.ssl_certificate_id)).id
-
-    def as_dict(self):
-        return {c.name: getattr(self, c.name) for c in self.__table__.columns}
-
-    def serialize(self):
-        blob = self.as_dict()
-        del blob['date_created']
-        return blob

lemur/listeners/service.py

@@ -1,159 +0,0 @@
-"""
-.. module: lemur.listeners.service
-    :platform: Unix
-    :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
-    :license: Apache, see LICENSE for more details.
-
-.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
-
-"""
-from sqlalchemy import func
-
-from lemur import database
-
-from lemur.exceptions import CertificateUnavailable
-
-from lemur.elbs.models import ELB
-from lemur.listeners.models import Listener
-from lemur.elbs import service as elb_service
-from lemur.certificates import service as certificate_service
-
-# from lemur.common.services.aws.elb import update_listeners, create_new_listeners, delete_listeners
-
-
-def verify_attachment(certificate_id, elb_account_number):
-    """
-    Ensures that the certificate we want ot attach to our listener is
-    in the same account as our listener.
-
-    :rtype : Certificate
-    :param certificate_id:
-    :param elb_account_number:
-    :return: :raise CertificateUnavailable:
-    """
-    cert = certificate_service.get(certificate_id)
-
-    # we need to ensure that the specified cert is in our account
-    for account in cert.accounts:
-        if account.account_number == elb_account_number:
-            break
-    else:
-        raise CertificateUnavailable
-    return cert
-
-
-def get(listener_id):
-    return database.get(Listener, listener_id)
-
-
-def create(elb_id, instance_protocol, instance_port, load_balancer_port, load_balancer_protocol, certificate_id=None):
-    listener = Listener(elb_id,
-                        instance_port,
-                        instance_protocol,
-                        load_balancer_port,
-                        load_balancer_protocol
-               )
-
-    elb = elb_service.get(elb_id)
-    elb.listeners.append(listener)
-    account_number = elb.account.account_number
-
-    cert = verify_attachment(certificate_id, account_number)
-    listener_tuple = (load_balancer_port, instance_port, load_balancer_protocol, cert.get_art(account_number),)
-    # create_new_listeners(account_number, elb.region, elb.name, [listener_tuple])
-
-    return {'message': 'Listener has been created'}
-
-
-def update(listener_id, **kwargs):
-    listener = get(listener_id)
-
-    # if the lb_port has changed we need to make sure we are deleting
-    # the listener on the old port to avoid listener duplication
-    ports = []
-    if listener.load_balancer_port != kwargs.get('load_balancer_port'):
-        ports.append(listener.load_balancer_port)
-    else:
-        ports.append(kwargs.get('load_balancer_port'))
-
-    certificate_id = kwargs.get('certificate_id')
-
-    listener.instance_port = kwargs.get('instance_port')
-    listener.instance_protocol = kwargs.get('instance_protocol')
-    listener.load_balancer_port = kwargs.get('load_balancer_port')
-    listener.load_balancer_protocol = kwargs.get('load_balancer_protocol')
-
-    elb = listener.elb
-    account_number = listener.elb.account.account_number
-
-    arn = None
-    if certificate_id:
-        cert = verify_attachment(certificate_id, account_number)
-        cert.elb_listeners.append(listener)
-        arn = cert.get_arn(account_number)
-
-    # remove certificate that is no longer wanted
-    if listener.certificate and not certificate_id:
-        listener.certificate.remove()
-
-    database.update(listener)
-    listener_tuple = (listener.load_balancer_port, listener.instance_port, listener.load_balancer_protocol, arn,)
-    # update_listeners(account_number, elb.region, elb.name, [listener_tuple], ports)
-
-    return {'message': 'Listener has been updated'}
-
-
-def delete(listener_id):
-    # first try to delete the listener in aws
-    listener = get(listener_id)
-    # delete_listeners(listener.elb.account.account_number, listener.elb.region, listener.elb.name, [listener.load_balancer_port])
-    # cleanup operation in lemur
-    database.delete(listener)
-
-
-def render(args):
-    query = database.session_query(Listener)
-
-    sort_by = args.pop('sort_by')
-    sort_dir = args.pop('sort_dir')
-    page = args.pop('page')
-    count = args.pop('count')
-    filt = args.pop('filter')
-    certificate_id = args.pop('certificate_id', None)
-    elb_id = args.pop('elb_id', None)
-
-    if certificate_id:
-        query = database.get_all(Listener, certificate_id, field='certificate_id')
-
-    if elb_id:
-        query = query.filter(Listener.elb_id == elb_id)
-
-    if filt:
-        terms = filt.split(';')
-        query = database.filter(query, Listener, terms)
-
-    query = database.find_all(query, Listener, args)
-
-    if sort_by and sort_dir:
-        query = database.sort(query, Listener, sort_by, sort_dir)
-
-    return database.paginate(query, page, count)
-
-
-def stats(**kwargs):
-    attr = getattr(Listener, kwargs.get('metric'))
-    query = database.db.session.query(attr, func.count(attr))
-    query = query.join(Listener.elb)
-
-    if kwargs.get('account_id'):
-        query = query.filter(ELB.account_id == kwargs.get('account_id'))
-
-    if kwargs.get('active') == 'true':
-        query = query.filter(Listener.certificate_id != None)  # noqa
-
-    items = query.group_by(attr).all()
-    results = []
-    for key, count in items:
-        if key:
-            results.append({"key": key, "y": count})
-    return results

lemur/listeners/views.py

@@ -1,128 +0,0 @@
-"""
-.. module: lemur.listeners.service
-    :platform: Unix
-    :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
-    :license: Apache, see LICENSE for more details.
-
-.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
-
-"""
-from flask import Blueprint
-from flask.ext.restful import reqparse, Api, fields
-
-from lemur.listeners import service
-from lemur.auth.service import AuthenticatedResource
-from lemur.auth.permissions import admin_permission
-from lemur.common.utils import marshal_items, paginated_parser
-
-
-mod = Blueprint('listeners', __name__)
-api = Api(mod)
-
-
-FIELDS = {
-    'id': fields.Integer,
-    'elbId': fields.Integer(attribute="elb_id"),
-    'certificateId': fields.Integer(attribute="certificate_id"),
-    'instancePort': fields.Integer(attribute="instance_port"),
-    'instanceProtocol': fields.String(attribute="instance_protocol"),
-    'loadBalancerPort': fields.Integer(attribute="load_balancer_port"),
-    'loadBalancerProtocol': fields.String(attribute="load_balancer_protocol")
-}
-
-
-class ListenersList(AuthenticatedResource):
-    def __init__(self):
-        super(ListenersList, self).__init__()
-
-    @marshal_items(FIELDS)
-    def get(self):
-        parser = paginated_parser.copy()
-        parser.add_argument('certificateId', type=int, dest='certificate_id', location='args')
-        args = parser.parse_args()
-        return service.render(args)
-
-
-class ListenersCertificateList(AuthenticatedResource):
-    def __init__(self):
-        super(ListenersCertificateList, self).__init__()
-
-    @marshal_items(FIELDS)
-    def get(self, certificate_id):
-        parser = paginated_parser.copy()
-        args = parser.parse_args()
-        args['certificate_id'] = certificate_id
-        return service.render(args)
-
-
-class ListenersELBList(AuthenticatedResource):
-    def __init__(self):
-        super(ListenersELBList, self).__init__()
-
-    @marshal_items(FIELDS)
-    def get(self, elb_id):
-        parser = paginated_parser.copy()
-        args = parser.parse_args()
-        args['elb_id'] = elb_id
-        return service.render(args)
-
-
-class ListenersStats(AuthenticatedResource):
-    def __init__(self):
-        self.reqparse = reqparse.RequestParser()
-        super(ListenersStats, self).__init__()
-
-    def get(self):
-        self.reqparse.add_argument('metric', type=str, location='args')
-        self.reqparse.add_argument('accountId', dest='account_id', location='args')
-        self.reqparse.add_argument('active', type=str, default='true', location='args')
-
-        args = self.reqparse.parse_args()
-
-        items = service.stats(**args)
-        return {"items": items, "total": len(items)}
-
-
-class Listeners(AuthenticatedResource):
-    def __init__(self):
-        super(Listeners, self).__init__()
-
-    @marshal_items(FIELDS)
-    def get(self, listener_id):
-        return service.get(listener_id)
-
-    @admin_permission.require(http_exception=403)
-    @marshal_items(FIELDS)
-    def post(self):
-        self.reqparse.add_argument('elbId', type=str, dest='elb_id', required=True, location='json')
-        self.reqparse.add_argument('instanceProtocol', type=str, dest='instance_protocol', required=True, location='json')
-        self.reqparse.add_argument('instancePort', type=int, dest='instance_port', required=True, location='json')
-        self.reqparse.add_argument('loadBalancerProtocol', type=str, dest='load_balancer_protocol', required=True, location='json')
-        self.reqparse.add_argument('loadBalancerPort', type=int, dest='load_balancer_port', required=True, location='json')
-        self.reqparse.add_argument('certificateId', type=int, dest='certificate_id', location='json')
-
-        args = self.reqparse.parse_args()
-        return service.create(**args)
-
-    @admin_permission.require(http_exception=403)
-    @marshal_items(FIELDS)
-    def put(self, listener_id):
-        self.reqparse.add_argument('instanceProtocol', type=str, dest='instance_protocol', required=True, location='json')
-        self.reqparse.add_argument('instancePort', type=int, dest='instance_port', required=True, location='json')
-        self.reqparse.add_argument('loadBalancerProtocol', type=str, dest='load_balancer_protocol', required=True, location='json')
-        self.reqparse.add_argument('loadBalancerPort', type=int, dest='load_balancer_port', required=True, location='json')
-        self.reqparse.add_argument('certificateId', type=int, dest='certificate_id', location='json')
-
-        args = self.reqparse.parse_args()
-        return service.update(listener_id, **args)
-
-    @admin_permission.require(http_exception=403)
-    def delete(self, listener_id):
-        return service.delete(listener_id)
-
-
-api.add_resource(ListenersList, '/listeners', endpoint='listeners')
-api.add_resource(Listeners, '/listeners/<int:listener_id>', endpoint='listener')
-api.add_resource(ListenersStats, '/listeners/stats', endpoint='listenersStats')
-api.add_resource(ListenersCertificateList, '/certificates/<int:certificate_id>/listeners', endpoint='listenersCertificates')
-api.add_resource(ListenersELBList, '/elbs/<int:elb_id>/listeners', endpoint='elbListeners')

lemur/manage.py

@@ -1,25 +1,27 @@
 import os
 import sys
 import base64
+import time
 from gunicorn.config import make_settings
 
 from cryptography.fernet import Fernet
 
+from lockfile import LockFile, LockTimeout
+
 from flask import current_app
-from flask.ext.script import Manager, Command, Option, Group, prompt_pass
+from flask.ext.script import Manager, Command, Option, prompt_pass
 from flask.ext.migrate import Migrate, MigrateCommand, stamp
 from flask_script.commands import ShowUrls, Clean, Server
 
 from lemur import database
 from lemur.users import service as user_service
 from lemur.roles import service as role_service
-from lemur.destinations import service as destination_service
 from lemur.certificates import service as cert_service
-
+from lemur.sources import service as source_service
-from lemur.plugins.base import plugins
+from lemur.notifications import service as notification_service
 
 from lemur.certificates.verify import verify_string
-from lemur.certificates import sync
+from lemur.sources.service import sync
 
 from lemur import create_app
 
@@ -30,9 +32,8 @@ from lemur.authorities.models import Authority  # noqa
 from lemur.certificates.models import Certificate  # noqa
 from lemur.destinations.models import Destination  # noqa
 from lemur.domains.models import Domain  # noqa
-from lemur.elbs.models import ELB  # noqa
-from lemur.listeners.models import Listener  # noqa
 from lemur.notifications.models import Notification  # noqa
+from lemur.sources.models import Source  # noqa
 
 
 manager = Manager(create_app)
@@ -76,6 +77,7 @@ LEMUR_RESTRICTED_DOMAINS = []
 
 LEMUR_EMAIL = ''
 LEMUR_SECURITY_TEAM_EMAIL = []
+LEMUR_DEFAULT_EXPIRATION_NOTIFICATION_INTERVALS = [30, 15, 2]
 
 # Logging
 
@@ -91,14 +93,6 @@ SQLALCHEMY_DATABASE_URI = 'postgresql://lemur:lemur@localhost:5432/lemur'
 
 # AWS
 
-# Lemur will need STS assume role access to every destination you want to monitor
-# AWS_ACCOUNT_MAPPINGS = {{
-#    '1111111111': 'myawsacount'
-# }}
-
-## This is useful if you know you only want to monitor one destination
-#AWS_REGIONS = ['us-east-1']
-
 #LEMUR_INSTANCE_PROFILE = 'Lemur'
 
 # Issuers
@@ -177,52 +171,73 @@ def generate_settings():
     return output
 
 
-class Sync(Command):
+@manager.option('-s', '--sources', dest='labels', default='', required=False)
+@manager.option('-l', '--list', dest='view', default=False, required=False)
+def sync_sources(labels, view):
     """
     Attempts to run several methods Certificate discovery. This is
     run on a periodic basis and updates the Lemur datastore with the
     information it discovers.
     """
-    option_list = [
+    if view:
-        Group(
+        sys.stdout.write("Active\tLabel\tDescription\n")
-            Option('-a', '--all', action="store_true"),
+        for source in source_service.get_all():
-            Option('-b', '--aws', action="store_true"),
+            sys.stdout.write(
-            Option('-d', '--cloudca', action="store_true"),
+                "[{active}]\t{label}\t{description}!\n".format(
-            Option('-s', '--source', action="store_true"),
+                    label=source.label,
-            exclusive=True, required=True
+                    description=source.description,
+                    active=source.active
+                )
+            )
+    else:
+        start_time = time.time()
+        lock_file = "/tmp/.lemur_lock"
+        sync_lock = LockFile(lock_file)
+
+        while not sync_lock.i_am_locking():
+            try:
+                sync_lock.acquire(timeout=10)    # wait up to 10 seconds
+
+                if labels:
+                    sys.stdout.write("[+] Staring to sync sources: {labels}!\n".format(labels=labels))
+                    labels = labels.split(",")
+                else:
+                    sys.stdout.write("[+] Starting to sync ALL sources!\n")
+
+                sync(labels=labels)
+                sys.stdout.write(
+                    "[+] Finished syncing sources. Run Time: {time}\n".format(
+                        time=(time.time() - start_time)
+                    )
+                )
+            except LockTimeout:
+                sys.stderr.write(
+                    "[!] Unable to acquire file lock on {file}, is there another sync running?\n".format(
+                        file=lock_file
+                    )
+                )
+                sync_lock.break_lock()
+                sync_lock.acquire()
+                sync_lock.release()
+
+        sync_lock.release()
+
+
+@manager.command
+def notify():
+    """
+    Runs Lemur's notification engine, that looks for expired certificates and sends
+    notifications out to those that bave subscribed to them.
+
+    :return:
+    """
+    sys.stdout.write("Starting to notify subscribers about expiring certificates!\n")
+    count = notification_service.send_expiration_notifications()
+    sys.stdout.write(
+        "Finished notifying subscribers about expiring certificates! Sent {count} notifications!\n".format(
+            count=count
         )
-    ]
+    )
-
-    def run(self, all, aws, cloudca, source):
-        sys.stdout.write("[!] Starting to sync with external sources!\n")
-
-        if all or aws:
-            sys.stdout.write("[!] Starting to sync with AWS!\n")
-            try:
-                sync.aws()
-                # sync_all_elbs()
-                sys.stdout.write("[+] Finished syncing with AWS!\n")
-            except Exception as e:
-                sys.stdout.write("[-] Syncing with AWS failed!\n")
-
-        if all or cloudca:
-            sys.stdout.write("[!] Starting to sync with CloudCA!\n")
-            try:
-                sync.cloudca()
-                sys.stdout.write("[+] Finished syncing with CloudCA!\n")
-            except Exception as e:
-                sys.stdout.write("[-] Syncing with CloudCA failed!\n")
-
-            sys.stdout.write("[!] Starting to sync with Source Code!\n")
-
-        if all or source:
-            try:
-                sync.source()
-                sys.stdout.write("[+] Finished syncing with Source Code!\n")
-            except Exception as e:
-                sys.stdout.write("[-] Syncing with Source Code failed!\n")
-
-            sys.stdout.write("[+] Finished syncing with external sources!\n")
 
 
 class InitializeApp(Command):
@@ -261,21 +276,20 @@ class InitializeApp(Command):
         else:
             sys.stdout.write("[-] Default user has already been created, skipping...!\n")
 
-        if current_app.config.get('AWS_ACCOUNT_MAPPINGS'):
+        sys.stdout.write("[+] Creating expiration email notifications!\n")
-            if plugins.get('aws-destination'):
+        sys.stdout.write("[!] Using {recipients} as specified by LEMUR_SECURITY_TEAM_EMAIL for notifications\n")
-                for account_name, account_number in current_app.config.get('AWS_ACCOUNT_MAPPINGS').items():
 
-                    destination = destination_service.get_by_label(account_name)
+        intervals = current_app.config.get("LEMUR_DEFAULT_EXPIRATION_NOTIFICATION_INTERVALS")
+        sys.stdout.write(
+            "[!] Creating {num} notifications for {intervals} days as specified by LEMUR_DEFAULT_EXPIRATION_NOTIFICATION_INTERVALS\n".format(
+                num=len(intervals),
+                intervals=",".join([str(x) for x in intervals])
+            )
+        )
+
+        recipients = current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL')
+        notification_service.create_default_expiration_notifications("DEFAULT_SECURITY", recipients=recipients)
 
-                    options = dict(account_number=account_number)
-                    if not destination:
-                        destination_service.create(account_name, 'aws-destination', options,
-                                                   description="This is an auto-generated AWS destination.")
-                        sys.stdout.write("[+] Added new destination {0}:{1}!\n".format(account_number, account_name))
-                    else:
-                        sys.stdout.write("[-] Account already exists, skipping...!\n")
-            else:
-                sys.stdout.write("[!] Skipping adding AWS destinations AWS plugin no available\n")
         sys.stdout.write("[/] Done!\n")
 
 
@@ -475,7 +489,6 @@ def main():
     manager.add_command("init", InitializeApp())
     manager.add_command("create_user", CreateUser())
     manager.add_command("create_role", CreateRole())
-    manager.add_command("sync", Sync())
     manager.run()
 
 

lemur/migrations/versions/1ff763f5b80b_.py

@@ -0,0 +1,42 @@
+"""Adding in models for certificate sources
+
+Revision ID: 1ff763f5b80b
+Revises: 4dc5ddd111b8
+Create Date: 2015-08-01 15:24:20.412725
+
+"""
+
+# revision identifiers, used by Alembic.
+revision = '1ff763f5b80b'
+down_revision = '4dc5ddd111b8'
+
+from alembic import op
+import sqlalchemy as sa
+
+import sqlalchemy_utils
+
+
+def upgrade():
+    ### commands auto generated by Alembic - please adjust! ###
+    op.create_table('sources',
+    sa.Column('id', sa.Integer(), nullable=False),
+    sa.Column('label', sa.String(length=32), nullable=True),
+    sa.Column('options', sqlalchemy_utils.types.json.JSONType(), nullable=True),
+    sa.Column('description', sa.Text(), nullable=True),
+    sa.Column('plugin_name', sa.String(length=32), nullable=True),
+    sa.PrimaryKeyConstraint('id')
+    )
+    op.create_table('certificate_source_associations',
+    sa.Column('source_id', sa.Integer(), nullable=True),
+    sa.Column('certificate_id', sa.Integer(), nullable=True),
+    sa.ForeignKeyConstraint(['certificate_id'], ['certificates.id'], ondelete='cascade'),
+    sa.ForeignKeyConstraint(['source_id'], ['destinations.id'], ondelete='cascade')
+    )
+    ### end Alembic commands ###
+
+
+def downgrade():
+    ### commands auto generated by Alembic - please adjust! ###
+    op.drop_table('certificate_source_associations')
+    op.drop_table('sources')
+    ### end Alembic commands ###

lemur/migrations/versions/4c8915e461b3_.py

@@ -0,0 +1,41 @@
+"""Adding notifications
+
+Revision ID: 4c8915e461b3
+Revises: 3b718f59b8ce
+Create Date: 2015-07-24 14:34:57.316273
+
+"""
+
+# revision identifiers, used by Alembic.
+revision = '4c8915e461b3'
+down_revision = '3b718f59b8ce'
+
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import postgresql
+
+import sqlalchemy_utils
+
+
+def upgrade():
+    ### commands auto generated by Alembic - please adjust! ###
+    op.create_table('notifications',
+    sa.Column('id', sa.Integer(), nullable=False),
+    sa.Column('label', sa.String(length=128), nullable=True),
+    sa.Column('description', sa.Text(), nullable=True),
+    sa.Column('options', sqlalchemy_utils.types.json.JSONType(), nullable=True),
+    sa.Column('active', sa.Boolean(), nullable=True),
+    sa.Column('plugin_name', sa.String(length=32), nullable=True),
+    sa.PrimaryKeyConstraint('id')
+    )
+    op.drop_column(u'certificates', 'challenge')
+    op.drop_column(u'certificates', 'csr_config')
+    ### end Alembic commands ###
+
+
+def downgrade():
+    ### commands auto generated by Alembic - please adjust! ###
+    op.add_column(u'certificates', sa.Column('csr_config', sa.TEXT(), autoincrement=False, nullable=True))
+    op.add_column(u'certificates', sa.Column('challenge', postgresql.BYTEA(), autoincrement=False, nullable=True))
+    op.drop_table('notifications')
+    ### end Alembic commands ###

lemur/migrations/versions/4dc5ddd111b8_.py

@@ -0,0 +1,31 @@
+"""Creating a one-to-many relationship for notifications
+
+Revision ID: 4dc5ddd111b8
+Revises: 4c8915e461b3
+Create Date: 2015-07-24 15:02:04.398262
+
+"""
+
+# revision identifiers, used by Alembic.
+revision = '4dc5ddd111b8'
+down_revision = '4c8915e461b3'
+
+from alembic import op
+import sqlalchemy as sa
+
+
+def upgrade():
+    ### commands auto generated by Alembic - please adjust! ###
+    op.create_table('certificate_notification_associations',
+    sa.Column('notification_id', sa.Integer(), nullable=True),
+    sa.Column('certificate_id', sa.Integer(), nullable=True),
+    sa.ForeignKeyConstraint(['certificate_id'], ['certificates.id'], ondelete='cascade'),
+    sa.ForeignKeyConstraint(['notification_id'], ['notifications.id'], ondelete='cascade')
+    )
+    ### end Alembic commands ###
+
+
+def downgrade():
+    ### commands auto generated by Alembic - please adjust! ###
+    op.drop_table('certificate_notification_associations')
+    ### end Alembic commands ###

lemur/models.py

@@ -8,9 +8,7 @@
     :license: Apache, see LICENSE for more details.
 .. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
 """
-
 from sqlalchemy import Column, Integer, ForeignKey
-
 from lemur.database import db
 
 certificate_associations = db.Table('certificate_associations',
@@ -25,12 +23,19 @@ certificate_destination_associations = db.Table('certificate_destination_associa
                                                        ForeignKey('certificates.id', ondelete='cascade'))
                                                 )
 
+certificate_source_associations = db.Table('certificate_source_associations',
+                                           Column('source_id', Integer,
+                                                  ForeignKey('sources.id', ondelete='cascade')),
+                                           Column('certificate_id', Integer,
+                                                  ForeignKey('certificates.id', ondelete='cascade'))
+                                           )
+
 certificate_notification_associations = db.Table('certificate_notification_associations',
-                                                Column('notification_id', Integer,
+                                                 Column('notification_id', Integer,
-                                                       ForeignKey('notifications.id', ondelete='cascade')),
+                                                        ForeignKey('notifications.id', ondelete='cascade')),
-                                                Column('certificate_id', Integer,
+                                                 Column('certificate_id', Integer,
-                                                       ForeignKey('certificates.id', ondelete='cascade'))
+                                                        ForeignKey('certificates.id', ondelete='cascade'))
-                                                )
+                                                 )
 roles_users = db.Table('roles_users',
                        Column('user_id', Integer, ForeignKey('users.id')),
                        Column('role_id', Integer, ForeignKey('roles.id'))

lemur/notifications/models.py

@@ -17,12 +17,18 @@ from lemur.models import certificate_notification_associations
 class Notification(db.Model):
     __tablename__ = 'notifications'
     id = Column(Integer, primary_key=True)
-    label = Column(String(128))
+    label = Column(String(128), unique=True)
     description = Column(Text())
     options = Column(JSONType)
     active = Column(Boolean, default=True)
     plugin_name = Column(String(32))
-    certificates = relationship("Certificate", secondary=certificate_notification_associations, passive_deletes=True, backref="notification", cascade='all,delete')
+    certificates = relationship(
+        "Certificate",
+        secondary=certificate_notification_associations,
+        passive_deletes=True,
+        backref="notification",
+        cascade='all,delete'
+    )
 
     @property
     def plugin(self):

lemur/notifications/service.py

@@ -24,6 +24,12 @@ from lemur.certificates import service as cert_service
 from lemur.plugins.base import plugins
 
 
+def get_options(name, options):
+    for o in options:
+        if o.get('name') == name:
+            return o
+
+
 def _get_message_data(cert):
     """
     Parse our the certification information needed for our notification
@@ -34,7 +40,7 @@ def _get_message_data(cert):
     cert_dict = cert.as_dict()
     cert_dict['creator'] = cert.user.email
     cert_dict['domains'] = [x .name for x in cert.domains]
-    cert_dict['superseded'] = list(set([x.name for x in find_superseded(cert.domains) if cert.name != x]))
+    cert_dict['superseded'] = list(set([x.name for x in _find_superseded(cert) if cert.name != x]))
     return cert_dict
 
 
@@ -44,8 +50,11 @@ def _deduplicate(messages):
     a roll up to the same set if the recipients are the same
     """
     roll_ups = []
-    for targets, data in messages:
+    for data, options in messages:
-        for m, r in roll_ups:
+        o = get_options('recipients', options)
+        targets = o['value'].split(',')
+
+        for m, r, o in roll_ups:
             if r == targets:
                 m.append(data)
                 current_app.logger.info(
@@ -53,7 +62,7 @@ def _deduplicate(messages):
                         data['name'], ",".join(targets)))
                 break
         else:
-            roll_ups.append(([data], targets, data.plugin_options))
+            roll_ups.append(([data], targets, options))
     return roll_ups
 
 
@@ -62,21 +71,30 @@ def send_expiration_notifications():
     This function will check for upcoming certificate expiration,
     and send out notification emails at given intervals.
     """
-    notifications = 0
+    sent = 0
 
-    for plugin_name, notifications in database.get_all(Notification, 'active', field='status').group_by(Notification.plugin_name):
+    for plugin in plugins.all(plugin_type='notification'):
-        notifications += 1
+        notifications = database.db.session.query(Notification)\
+            .filter(Notification.plugin_name == plugin.slug)\
+            .filter(Notification.active == True).all()  # noqa
 
-        messages = _deduplicate(notifications)
+        messages = []
-        plugin = plugins.get(plugin_name)
+        for n in notifications:
+            for c in n.certificates:
+                if _is_eligible_for_notifications(c):
+                    messages.append((_get_message_data(c), n.options))
+
+        messages = _deduplicate(messages)
 
         for data, targets, options in messages:
+            sent += 1
             plugin.send('expiration', data, targets, options)
 
-    current_app.logger.info("Lemur has sent {0} certification notifications".format(notifications))
+        current_app.logger.info("Lemur has sent {0} certification notifications".format(sent))
+    return sent
 
 
-def get_domain_certificate(name):
+def _get_domain_certificate(name):
     """
     Fetch the SSL certificate currently hosted at a given domain (if any) and
     compare it against our all of our know certificates to determine if a new
@@ -92,7 +110,7 @@ def get_domain_certificate(name):
         current_app.logger.info(str(e))
 
 
-def find_superseded(domains):
+def _find_superseded(cert):
     """
     Here we try to fetch any domain in the certificate to see if we can resolve it
     and to try and see if it is currently serving the certificate we are
@@ -103,17 +121,22 @@ def find_superseded(domains):
     """
     query = database.session_query(Certificate)
     ss_list = []
-    for domain in domains:
+
-        dc = get_domain_certificate(domain.name)
+    # determine what is current host at our domains
-        if dc:
+    for domain in cert.domains:
-            ss_list.append(dc)
+        dups = _get_domain_certificate(domain.name)
+        for c in dups:
+            if c.body != cert.body:
+                ss_list.append(dups)
+
         current_app.logger.info("Trying to resolve {0}".format(domain.name))
 
-    query = query.filter(Certificate.domains.any(Domain.name.in_([x.name for x in domains])))
+    # look for other certificates that may not be hosted but cover the same domains
+    query = query.filter(Certificate.domains.any(Domain.name.in_([x.name for x in cert.domains])))
     query = query.filter(Certificate.active == True)  # noqa
     query = query.filter(Certificate.not_after >= arrow.utcnow().format('YYYY-MM-DD'))
+    query = query.filter(Certificate.body != cert.body)
     ss_list.extend(query.all())
-
     return ss_list
 
 
@@ -129,8 +152,8 @@ def _is_eligible_for_notifications(cert):
     days = (cert.not_after - now.naive).days
 
     for notification in cert.notifications:
-        interval = notification.options['interval']
+        interval = get_options('interval', notification.options)['value']
-        unit = notification.options['unit']
+        unit = get_options('unit', notification.options)['value']
         if unit == 'weeks':
             interval *= 7
 
@@ -147,6 +170,63 @@ def _is_eligible_for_notifications(cert):
             return cert
 
 
+def create_default_expiration_notifications(name, recipients):
+    """
+    Will create standard 30, 10 and 2 day notifications for a given owner. If standard notifications
+    already exist these will be returned instead of new notifications.
+
+    :param name:
+    :return:
+    """
+    options = [
+        {
+            'name': 'unit',
+            'type': 'select',
+            'required': True,
+            'validation': '',
+            'available': ['days', 'weeks', 'months'],
+            'helpMessage': 'Interval unit',
+            'value': 'days',
+        },
+        {
+            'name': 'recipients',
+            'type': 'str',
+            'required': True,
+            'validation': '^([\w+-.%]+@[\w-.]+\.[A-Za-z]{2,4},?)+$',
+            'helpMessage': 'Comma delimited list of email addresses',
+            'value': ','.join(recipients)
+        },
+    ]
+
+    intervals = current_app.config.get("LEMUR_DEFAULT_EXPIRATION_NOTIFICATION_INTERVALS")
+
+    notifications = []
+    for i in intervals:
+        n = get_by_label("{name}_{interval}_DAY".format(name=name, interval=i))
+        if not n:
+            inter = [
+                {
+                    'name': 'interval',
+                    'type': 'int',
+                    'required': True,
+                    'validation': '^\d+$',
+                    'helpMessage': 'Number of days to be alert before expiration.',
+                    'value': i,
+                }
+            ]
+            inter.extend(options)
+            n = create(
+                label="{name}_{interval}_DAY".format(name=name, interval=i),
+                plugin_name="email-notification",
+                options=list(inter),
+                description="Default {interval} day expiration notification".format(interval=i),
+                certificates=[]
+            )
+        notifications.append(n)
+
+    return notifications
+
+
 def create(label, plugin_name, options, description, certificates):
     """
     Creates a new destination, that can then be used as a destination for certificates.
@@ -163,7 +243,7 @@ def create(label, plugin_name, options, description, certificates):
     return database.create(notification)
 
 
-def update(notification_id, label, options, description, certificates):
+def update(notification_id, label, options, description, active, certificates):
     """
     Updates an existing destination.
 
@@ -178,6 +258,7 @@ def update(notification_id, label, options, description, certificates):
     notification.label = label
     notification.options = options
     notification.description = description
+    notification.active = active
     notification = database.update_list(notification, 'certificates', Certificate, certificates)
 
     return database.update(notification)

lemur/notifications/views.py

@@ -110,6 +110,7 @@ class NotificationsList(AuthenticatedResource):
            :statuscode 200: no error
         """
         parser = paginated_parser.copy()
+        parser.add_argument('active', type=bool, location='args')
         args = parser.parse_args()
         return service.render(args)
 
@@ -346,6 +347,7 @@ class Notifications(AuthenticatedResource):
         """
         self.reqparse.add_argument('label', type=str, location='json', required=True)
         self.reqparse.add_argument('plugin', type=dict, location='json', required=True)
+        self.reqparse.add_argument('active', type=bool, location='json')
         self.reqparse.add_argument('certificates', type=list, default=[], location='json')
         self.reqparse.add_argument('description', type=str, location='json')
 
@@ -355,6 +357,7 @@ class Notifications(AuthenticatedResource):
             args['label'],
             args['plugin']['pluginOptions'],
             args['description'],
+            args['active'],
             args['certificates']
         )
 
@@ -444,6 +447,7 @@ class CertificateNotifications(AuthenticatedResource):
            :statuscode 200: no error
         """
         parser = paginated_parser.copy()
+        parser.add_argument('active', type=bool, location='args')
         args = parser.parse_args()
         args['certificate_id'] = certificate_id
         return service.render(args)

lemur/plugins/base/v1.py

@@ -101,13 +101,18 @@ class IPlugin(local):
         Returns a list of tuples pointing to various resources for this plugin.
         >>> def get_resource_links(self):
         >>>     return [
-        >>>         ('Documentation', 'http://sentry.readthedocs.org'),
+        >>>         ('Documentation', 'http://lemury.readthedocs.org'),
-        >>>         ('Bug Tracker', 'https://github.com/getsentry/sentry/issues'),
+        >>>         ('Bug Tracker', 'https://github.com/Netflix/lemur/issues'),
-        >>>         ('Source', 'https://github.com/getsentry/sentry'),
+        >>>         ('Source', 'https://github.com/Netflix/lemur'),
         >>>     ]
         """
         return self.resource_links
 
+    def get_option(self, name, options):
+        for o in options:
+            if o.get(name):
+                return o['value']
+
 
 class Plugin(IPlugin):
     """

lemur/plugins/lemur_aws/elb.py

@@ -1,9 +1,9 @@
 """
-.. module:: elb
+.. module: elb
     :synopsis: Module contains some often used and helpful classes that
     are used to deal with ELBs
 
-.. moduleauthor:: Kevin Glisson (kglisson@netflix.com)
+.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
 """
 import boto.ec2
 

lemur/plugins/lemur_aws/iam.py

@@ -19,17 +19,17 @@ def get_name_from_arn(arn):
     return arn.split("/", 1)[1]
 
 
-def upload_cert(account_number, cert, private_key, cert_chain=None):
+def upload_cert(account_number, name, body, private_key, cert_chain=None):
     """
     Upload a certificate to AWS
 
     :param account_number:
-    :param cert:
+    :param name:
     :param private_key:
     :param cert_chain:
     :return:
     """
-    return assume_service(account_number, 'iam').upload_server_cert(cert.name, str(cert.body), str(private_key),
+    return assume_service(account_number, 'iam').upload_server_cert(name, str(body), str(private_key),
                                                                     cert_chain=str(cert_chain))
 
 
@@ -57,7 +57,7 @@ def get_all_server_certs(account_number):
         result = response['list_server_certificates_response']['list_server_certificates_result']
 
         for cert in result['server_certificate_metadata_list']:
-            certs.append(cert)
+            certs.append(cert['arn'])
 
         if result['is_truncated'] == 'true':
             marker = result['marker']
@@ -72,7 +72,7 @@ def get_cert_from_arn(arn):
     :param arn:
     :return:
     """
-    name = arn.split("/", 1)[1]
+    name = get_name_from_arn(arn)
     account_number = arn.split(":")[4]
     name = name.split("/")[-1]
 

lemur/plugins/lemur_aws/plugin.py

@@ -13,7 +13,7 @@ from lemur.plugins import lemur_aws as aws
 
 def find_value(name, options):
     for o in options:
-        if o.get(name):
+        if o['name'] == name:
             return o['value']
 
 
@@ -29,7 +29,7 @@ class AWSDestinationPlugin(DestinationPlugin):
     options = [
         {
             'name': 'accountNumber',
-            'type': 'int',
+            'type': 'str',
             'required': True,
             'validation': '/^[0-9]{12,12}$/',
             'helpMessage': 'Must be a valid AWS account number!',
@@ -41,8 +41,8 @@ class AWSDestinationPlugin(DestinationPlugin):
     #    'port': {'type': 'int'}
     # }
 
-    def upload(self, cert, private_key, cert_chain, options, **kwargs):
+    def upload(self, name, body, private_key, cert_chain, options, **kwargs):
-        iam.upload_cert(find_value('accountNumber', options), cert, private_key, cert_chain=cert_chain)
+        iam.upload_cert(find_value('accountNumber', options), name, body, private_key, cert_chain=cert_chain)
 
         e = find_value('elb', options)
         if e:
@@ -68,14 +68,15 @@ class AWSSourcePlugin(SourcePlugin):
         },
     ]
 
-    def get_certificates(self, **kwargs):
+    def get_certificates(self, options, **kwargs):
         certs = []
-        arns = elb.get_all_server_certs(kwargs['account_number'])
+        arns = iam.get_all_server_certs(find_value('accountNumber', options))
         for arn in arns:
-            cert_body = iam.get_cert_from_arn(arn)
+            cert_body, cert_chain = iam.get_cert_from_arn(arn)
             cert_name = iam.get_name_from_arn(arn)
             cert = dict(
                 public_certificate=cert_body,
+                intermediate_certificate=cert_chain,
                 name=cert_name
             )
             certs.append(cert)

lemur/plugins/lemur_email/plugin.py

@@ -19,12 +19,6 @@ from lemur.plugins import lemur_email as email
 from lemur.plugins.lemur_email.templates.config import env
 
 
-def find_value(name, options):
-    for o in options:
-        if o.get(name):
-            return o['value']
-
-
 class EmailNotificationPlugin(ExpirationNotificationPlugin):
     title = 'Email'
     slug = 'email-notification'

lemur/plugins/service.py

@@ -0,0 +1,7 @@
+"""
+.. module: service
+    :platform: Unix
+    :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
+    :license: Apache, see LICENSE for more details.
+.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
+"""

lemur/sources/models.py

@@ -0,0 +1,31 @@
+"""
+.. module: lemur.sources.models
+    :platform: unix
+    :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
+    :license: Apache, see LICENSE for more details.
+.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
+"""
+import copy
+from sqlalchemy import Column, Integer, String, Text, DateTime, Boolean
+from sqlalchemy_utils import JSONType
+from lemur.database import db
+
+from lemur.plugins.base import plugins
+
+
+class Source(db.Model):
+    __tablename__ = 'sources'
+    id = Column(Integer, primary_key=True)
+    label = Column(String(32))
+    options = Column(JSONType)
+    description = Column(Text())
+    plugin_name = Column(String(32))
+    active = Column(Boolean, default=True)
+    last_run = Column(DateTime)
+
+    @property
+    def plugin(self):
+        p = plugins.get(self.plugin_name)
+        c = copy.deepcopy(p)
+        c.options = self.options
+        return c

lemur/sources/service.py

@@ -0,0 +1,198 @@
+"""
+.. module: lemur.sources.service
+    :platform: Unix
+    :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
+    :license: Apache, see LICENSE for more details.
+.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
+"""
+from flask import current_app
+
+from lemur import database
+from lemur.sources.models import Source
+from lemur.certificates.models import Certificate
+from lemur.certificates import service as cert_service
+from lemur.destinations import service as destination_service
+
+from lemur.plugins.base import plugins
+
+
+def _disassociate_certs_from_source(current_certificates, found_certificates, source_label):
+    missing = []
+    for cc in current_certificates:
+        for fc in found_certificates:
+            if fc['public_certificate'] == cc.body:
+                break
+        else:
+            missing.append(cc)
+
+    for c in missing:
+        for s in c.sources:
+            if s.label == source_label:
+                current_app.logger.info(
+                    "Certificate {name} is no longer associated with {source}".format(
+                        name=c.name,
+                        source=source_label
+                    )
+                )
+                c.sources.delete(s)
+
+
+def sync_create(certificate, source):
+    cert = cert_service.import_certificate(**certificate)
+    cert.sources.append(source)
+    sync_update_destination(cert, source)
+    database.update(cert)
+
+
+def sync_update(certificate, source):
+    for s in certificate.sources:
+        if s.label == source.label:
+            break
+    else:
+        certificate.sources.append(source)
+
+    sync_update_destination(certificate, source)
+    database.update(certificate)
+
+
+def sync_update_destination(certificate, source):
+    dest = destination_service.get_by_label(source.label)
+    if dest:
+        for d in certificate.destinations:
+            if d.label == source.label:
+                break
+        else:
+            certificate.destinations.append(dest)
+
+
+def sync(labels=None):
+    new, updated = 0, 0
+    c_certificates = cert_service.get_all_certs()
+
+    for source in database.get_all(Source, True, field='active'):
+        # we should be able to specify, individual sources to sync
+        if labels:
+            if source.label not in labels:
+                continue
+
+        current_app.logger.error("Retrieving certificates from {0}".format(source.label))
+        s = plugins.get(source.plugin_name)
+        certificates = s.get_certificates(source.options)
+
+        for certificate in certificates:
+            exists = cert_service.find_duplicates(certificate['public_certificate'])
+
+            if not exists:
+                sync_create(certificate, source)
+                new += 1
+
+            # check to make sure that existing certificates have the current source associated with it
+            elif len(exists) == 1:
+                sync_update(exists[0], source)
+                updated += 1
+            else:
+                current_app.logger.warning(
+                    "Multiple certificates found, attempt to deduplicate the following certificates: {0}".format(
+                        ",".join([x.name for x in exists])
+                    )
+                )
+
+        # we need to try and find the absent of certificates so we can properly disassociate them when they are deleted
+        _disassociate_certs_from_source(c_certificates, certificates, source)
+
+
+def create(label, plugin_name, options, description=None):
+    """
+    Creates a new source, that can then be used as a source for certificates.
+
+    :param label: Source common name
+    :param description:
+    :rtype : Source
+    :return: New source
+    """
+    source = Source(label=label, options=options, plugin_name=plugin_name, description=description)
+    return database.create(source)
+
+
+def update(source_id, label, options, description):
+    """
+    Updates an existing source.
+
+    :param source_id:  Lemur assigned ID
+    :param label: Source common name
+    :rtype : Source
+    :return:
+    """
+    source = get(source_id)
+
+    source.label = label
+    source.options = options
+    source.description = description
+
+    return database.update(source)
+
+
+def delete(source_id):
+    """
+    Deletes an source.
+
+    :param source_id: Lemur assigned ID
+    """
+    database.delete(get(source_id))
+
+
+def get(source_id):
+    """
+    Retrieves an source by it's lemur assigned ID.
+
+    :param source_id: Lemur assigned ID
+    :rtype : Source
+    :return:
+    """
+    return database.get(Source, source_id)
+
+
+def get_by_label(label):
+    """
+    Retrieves a source by it's label
+
+    :param label:
+    :return:
+    """
+    return database.get(Source, label, field='label')
+
+
+def get_all():
+    """
+    Retrieves all source currently known by Lemur.
+
+    :return:
+    """
+    query = database.session_query(Source)
+    return database.find_all(query, Source, {}).all()
+
+
+def render(args):
+    sort_by = args.pop('sort_by')
+    sort_dir = args.pop('sort_dir')
+    page = args.pop('page')
+    count = args.pop('count')
+    filt = args.pop('filter')
+    certificate_id = args.pop('certificate_id', None)
+
+    if certificate_id:
+        query = database.session_query(Source).join(Certificate, Source.certificate)
+        query = query.filter(Certificate.id == certificate_id)
+    else:
+        query = database.session_query(Source)
+
+    if filt:
+        terms = filt.split(';')
+        query = database.filter(query, Source, terms)
+
+    query = database.find_all(query, Source, args)
+
+    if sort_by and sort_dir:
+        query = database.sort(query, Source, sort_by, sort_dir)
+
+    return database.paginate(query, page, count)

lemur/sources/views.py

@@ -0,0 +1,367 @@
+"""
+.. module: lemur.sources.views
+    :platform: Unix
+    :synopsis: This module contains all of the accounts view code.
+    :copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
+    :license: Apache, see LICENSE for more details.
+.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
+"""
+from flask import Blueprint
+from flask.ext.restful import Api, reqparse, fields
+from lemur.sources import service
+
+from lemur.auth.service import AuthenticatedResource
+from lemur.auth.permissions import admin_permission
+from lemur.common.utils import paginated_parser, marshal_items
+
+
+mod = Blueprint('sources', __name__)
+api = Api(mod)
+
+
+FIELDS = {
+    'description': fields.String,
+    'sourceOptions': fields.Raw(attribute='options'),
+    'pluginName': fields.String(attribute='plugin_name'),
+    'lastRun': fields.DateTime(attribute='last_run', dt_format='iso8061'),
+    'label': fields.String,
+    'id': fields.Integer,
+}
+
+
+class SourcesList(AuthenticatedResource):
+    """ Defines the 'sources' endpoint """
+    def __init__(self):
+        self.reqparse = reqparse.RequestParser()
+        super(SourcesList, self).__init__()
+
+    @marshal_items(FIELDS)
+    def get(self):
+        """
+        .. http:get:: /sources
+
+           The current account list
+
+           **Example request**:
+
+           .. sourcecode:: http
+
+              GET /sources HTTP/1.1
+              Host: example.com
+              Accept: application/json, text/javascript
+
+           **Example response**:
+
+           .. sourcecode:: http
+
+              HTTP/1.1 200 OK
+              Vary: Accept
+              Content-Type: text/javascript
+
+              {
+                "items": [
+                    {
+                        "sourceOptions": [
+                            {
+                                "name": "accountNumber",
+                                "required": true,
+                                "value": 111111111112,
+                                "helpMessage": "Must be a valid AWS account number!",
+                                "validation": "/^[0-9]{12,12}$/",
+                                "type": "int"
+                            }
+                        ],
+                        "pluginName": "aws-source",
+                        "lastRun": "2015-08-01T15:40:58",
+                        "id": 3,
+                        "description": "test",
+                        "label": "test"
+                    }
+                ],
+                "total": 1
+              }
+
+           :query sortBy: field to sort on
+           :query sortDir: acs or desc
+           :query page: int. default is 1
+           :query filter: key value pair. format is k=v;
+           :query limit: limit number. default is 10
+           :reqheader Authorization: OAuth token to authenticate
+           :statuscode 200: no error
+        """
+        parser = paginated_parser.copy()
+        args = parser.parse_args()
+        return service.render(args)
+
+    @admin_permission.require(http_exception=403)
+    @marshal_items(FIELDS)
+    def post(self):
+        """
+        .. http:post:: /sources
+
+           Creates a new account
+
+           **Example request**:
+
+           .. sourcecode:: http
+
+              POST /sources HTTP/1.1
+              Host: example.com
+              Accept: application/json, text/javascript
+
+              {
+                "sourceOptions": [
+                    {
+                        "name": "accountNumber",
+                        "required": true,
+                        "value": 111111111112,
+                        "helpMessage": "Must be a valid AWS account number!",
+                        "validation": "/^[0-9]{12,12}$/",
+                        "type": "int"
+                    }
+                ],
+                "pluginName": "aws-source",
+                "id": 3,
+                "lastRun": "2015-08-01T15:40:58",
+                "description": "test",
+                "label": "test"
+              }
+
+           **Example response**:
+
+           .. sourcecode:: http
+
+              HTTP/1.1 200 OK
+              Vary: Accept
+              Content-Type: text/javascript
+
+              {
+                "sourceOptions": [
+                    {
+                        "name": "accountNumber",
+                        "required": true,
+                        "value": 111111111112,
+                        "helpMessage": "Must be a valid AWS account number!",
+                        "validation": "/^[0-9]{12,12}$/",
+                        "type": "int"
+                    }
+                ],
+                "pluginName": "aws-source",
+                "id": 3,
+                "lastRun": "2015-08-01T15:40:58",
+                "description": "test",
+                "label": "test"
+              }
+
+           :arg label: human readable account label
+           :arg description: some description about the account
+           :reqheader Authorization: OAuth token to authenticate
+           :statuscode 200: no error
+        """
+        self.reqparse.add_argument('label', type=str, location='json', required=True)
+        self.reqparse.add_argument('plugin', type=dict, location='json', required=True)
+        self.reqparse.add_argument('description', type=str, location='json')
+
+        args = self.reqparse.parse_args()
+        return service.create(args['label'], args['plugin']['slug'], args['plugin']['pluginOptions'], args['description'])
+
+
+class Sources(AuthenticatedResource):
+    def __init__(self):
+        self.reqparse = reqparse.RequestParser()
+        super(Sources, self).__init__()
+
+    @marshal_items(FIELDS)
+    def get(self, source_id):
+        """
+        .. http:get:: /sources/1
+
+           Get a specific account
+
+           **Example request**:
+
+           .. sourcecode:: http
+
+              GET /sources/1 HTTP/1.1
+              Host: example.com
+              Accept: application/json, text/javascript
+
+           **Example response**:
+
+           .. sourcecode:: http
+
+              HTTP/1.1 200 OK
+              Vary: Accept
+              Content-Type: text/javascript
+
+              {
+                "sourceOptions": [
+                    {
+                        "name": "accountNumber",
+                        "required": true,
+                        "value": 111111111112,
+                        "helpMessage": "Must be a valid AWS account number!",
+                        "validation": "/^[0-9]{12,12}$/",
+                        "type": "int"
+                    }
+                ],
+                "pluginName": "aws-source",
+                "id": 3,
+                "lastRun": "2015-08-01T15:40:58",
+                "description": "test",
+                "label": "test"
+              }
+
+           :reqheader Authorization: OAuth token to authenticate
+           :statuscode 200: no error
+        """
+        return service.get(source_id)
+
+    @admin_permission.require(http_exception=403)
+    @marshal_items(FIELDS)
+    def put(self, source_id):
+        """
+        .. http:put:: /sources/1
+
+           Updates an account
+
+           **Example request**:
+
+           .. sourcecode:: http
+
+              POST /sources/1 HTTP/1.1
+              Host: example.com
+              Accept: application/json, text/javascript
+
+              {
+                "sourceOptions": [
+                    {
+                        "name": "accountNumber",
+                        "required": true,
+                        "value": 111111111112,
+                        "helpMessage": "Must be a valid AWS account number!",
+                        "validation": "/^[0-9]{12,12}$/",
+                        "type": "int"
+                    }
+                ],
+                "pluginName": "aws-source",
+                "id": 3,
+                "lastRun": "2015-08-01T15:40:58",
+                "description": "test",
+                "label": "test"
+              }
+
+           **Example response**:
+
+           .. sourcecode:: http
+
+              HTTP/1.1 200 OK
+              Vary: Accept
+              Content-Type: text/javascript
+
+              {
+                "sourceOptions": [
+                    {
+                        "name": "accountNumber",
+                        "required": true,
+                        "value": 111111111112,
+                        "helpMessage": "Must be a valid AWS account number!",
+                        "validation": "/^[0-9]{12,12}$/",
+                        "type": "int"
+                    }
+                ],
+                "pluginName": "aws-source",
+                "id": 3,
+                "lastRun": "2015-08-01T15:40:58",
+                "description": "test",
+                "label": "test"
+              }
+
+           :arg accountNumber: aws account number
+           :arg label: human readable account label
+           :arg description: some description about the account
+           :reqheader Authorization: OAuth token to authenticate
+           :statuscode 200: no error
+        """
+        self.reqparse.add_argument('label', type=str, location='json', required=True)
+        self.reqparse.add_argument('plugin', type=dict, location='json', required=True)
+        self.reqparse.add_argument('description', type=str, location='json')
+
+        args = self.reqparse.parse_args()
+        return service.update(source_id, args['label'], args['plugin']['pluginOptions'], args['description'])
+
+    @admin_permission.require(http_exception=403)
+    def delete(self, source_id):
+        service.delete(source_id)
+        return {'result': True}
+
+
+class CertificateSources(AuthenticatedResource):
+    """ Defines the 'certificate/<int:certificate_id/sources'' endpoint """
+    def __init__(self):
+        super(CertificateSources, self).__init__()
+
+    @marshal_items(FIELDS)
+    def get(self, certificate_id):
+        """
+        .. http:get:: /certificates/1/sources
+
+           The current account list for a given certificates
+
+           **Example request**:
+
+           .. sourcecode:: http
+
+              GET /certificates/1/sources HTTP/1.1
+              Host: example.com
+              Accept: application/json, text/javascript
+
+           **Example response**:
+
+           .. sourcecode:: http
+
+              HTTP/1.1 200 OK
+              Vary: Accept
+              Content-Type: text/javascript
+
+              {
+                "items": [
+                    {
+                        "sourceOptions": [
+                            {
+                                "name": "accountNumber",
+                                "required": true,
+                                "value": 111111111112,
+                                "helpMessage": "Must be a valid AWS account number!",
+                                "validation": "/^[0-9]{12,12}$/",
+                                "type": "int"
+                            }
+                        ],
+                        "pluginName": "aws-source",
+                        "id": 3,
+                        "lastRun": "2015-08-01T15:40:58",
+                        "description": "test",
+                        "label": "test"
+                    }
+                ],
+                "total": 1
+              }
+
+           :query sortBy: field to sort on
+           :query sortDir: acs or desc
+           :query page: int. default is 1
+           :query filter: key value pair. format is k=v;
+           :query limit: limit number. default is 10
+           :reqheader Authorization: OAuth token to authenticate
+           :statuscode 200: no error
+        """
+        parser = paginated_parser.copy()
+        args = parser.parse_args()
+        args['certificate_id'] = certificate_id
+        return service.render(args)
+
+
+api.add_resource(SourcesList, '/sources', endpoint='sources')
+api.add_resource(Sources, '/sources/<int:source_id>', endpoint='account')
+api.add_resource(CertificateSources, '/certificates/<int:certificate_id>/sources',
+                 endpoint='certificateSources')

lemur/static/app/angular/certificates/certificate/certificate.js

@@ -22,7 +22,7 @@ angular.module('lemur')
     $scope.notificationService = NotificationService;
   })
 
-  .controller('CertificateCreateController', function ($scope, $modalInstance, CertificateApi, CertificateService, DestinationService, ELBService, AuthorityService, PluginService, MomentService, WizardHandler, LemurRestangular, NotificationService) {
+  .controller('CertificateCreateController', function ($scope, $modalInstance, CertificateApi, CertificateService, DestinationService, AuthorityService, PluginService, MomentService, WizardHandler, LemurRestangular, NotificationService) {
     $scope.certificate = LemurRestangular.restangularizeElement(null, {}, 'certificates');
 
     $scope.create = function (certificate) {
@@ -92,7 +92,6 @@ angular.module('lemur')
         $scope.plugins = plugins;
     });
 
-    $scope.elbService = ELBService;
     $scope.authorityService = AuthorityService;
     $scope.destinationService = DestinationService;
     $scope.notificationService = NotificationService;

lemur/static/app/angular/certificates/certificate/edit.tpl.html

@@ -0,0 +1,38 @@
+<div class="modal-header">
+    <div class="modal-title">
+        <h3 class="modal-header">Edit <span class="text-muted"><small>{{ certificate.name }}</small></span></h3>
+    </div>
+    <div class="modal-body">
+        <form name="editForm" class="form-horizontal" role="form" novalidate>
+            <div class="form-group"
+                 ng-class="{'has-error': editForm.owner.$invalid, 'has-success': !editForm.owner.$invalid&&editForm.owner.$dirty}">
+                <label class="control-label col-sm-2">
+                    Owner
+                </label>
+                <div class="col-sm-10">
+                    <input type="email" name="owner" ng-model="certificate.owner" placeholder="owner@netflix.com"
+                           class="form-control" required/>
+
+                    <p ng-show="editForm.owner.$invalid && !editForm.owner.$pristine" class="help-block">Enter a valid
+                        email.</p>
+                </div>
+            </div>
+            <div class="form-group"
+                 ng-class="{'has-error': editForm.description.$invalid, 'has-success': !editForm.$invalid&&editForm.description.$dirty}">
+                <label class="control-label col-sm-2">
+                    Description
+                </label>
+                <div class="col-sm-10">
+                    <textarea name="description" ng-model="certificate.description" placeholder="Something elegant" class="form-control" ng-pattern="/^[\w\-\s]+$/" required></textarea>
+                    <p ng-show="editForm.description.$invalid && !editForm.description.$pristine" class="help-block">You must give a short description about this authority will be used for, this description should only include alphanumeric characters</p>
+                </div>
+            </div>
+            <div ng-include="'angular/certificates/certificate/notifications.tpl.html'"></div>
+            <div ng-include="'angular/certificates/certificate/destinations.tpl.html'"></div>
+        </form>
+    </div>
+    <div class="modal-footer">
+        <button type="submit" ng-click="save(certificate)" ng-disabled="editForm.$invalid" class="btn btn-success">Save</button>
+        <button ng-click="cancel()" class="btn btn-danger">Cancel</button>
+    </div>
+</div>

lemur/static/app/angular/certificates/certificate/notifications.tpl.html

@@ -0,0 +1,28 @@
+<div class="form-group">
+    <label class="control-label col-sm-2">
+        Notifications
+    </label>
+    <div class="col-sm-10">
+        <div class="input-group">
+            <input type="text" ng-model="certificate.selectedNotification" placeholder="Email"
+                   typeahead="notification.label for notification in notificationService.findNotificationsByName($viewValue)" typeahead-loading="loadingDestinations"
+                   class="form-control input-md" typeahead-on-select="certificate.attachNotification($item)" typeahead-min-wait="50"
+                   tooltip="By default Lemur will always notify you about this certificate through Email notifications."
+                   tooltip-trigger="focus" tooltip-placement="top">
+          <span class="input-group-btn">
+            <button ng-model="notifications.show" class="btn btn-md btn-default" btn-checkbox btn-checkbox-true="1" btn-checkbox-false="0">
+                <span class="badge">{{ certificate.notifications.length || 0 }}</span>
+            </button>
+          </span>
+        </div>
+        <table class="table">
+            <tr ng-repeat="notification in certificate.notifications track by $index">
+                <td><a class="btn btn-sm btn-info" href="#/notifications/{{ notification.id }}/certificates">{{ notification.label }}</a></td>
+                <td><span class="text-muted">{{ notification.description }}</span></td>
+                <td>
+                    <button type="button" ng-click="certificate.removeNotification($index)" class="btn btn-danger btn-sm pull-right">Remove</button>
+                </td>
+            </tr>
+        </table>
+    </div>
+</div>

lemur/static/app/angular/certificates/certificate/upload.js

@@ -2,22 +2,20 @@
 
 angular.module('lemur')
 
-  .controller('CertificateUploadController', function ($scope, $modalInstance, CertificateService, LemurRestangular, DestinationService, NotificationService, ELBService, PluginService) {
+  .controller('CertificateUploadController', function ($scope, $modalInstance, CertificateService, LemurRestangular, DestinationService, NotificationService, PluginService) {
     $scope.certificate = LemurRestangular.restangularizeElement(null, {}, 'certificates');
     $scope.upload = CertificateService.upload;
 
     $scope.destinationService = DestinationService;
     $scope.notificationService = NotificationService;
-    $scope.elbService = ELBService;
 
     PluginService.getByType('destination').then(function (plugins) {
         $scope.plugins = plugins;
     });
 
-    $scope.attachELB = function (elb) {
+    $scope.save = function (certificate) {
-      $scope.certificate.attachELB(elb);
+      CertificateService.upload(certificate).then(function () {
-      ELBService.getListeners(elb).then(function (listeners) {
+        $modalInstance.close();
-        $scope.certificate.elb.listeners = listeners;
       });
     };
 

lemur/static/app/angular/certificates/certificate/upload.tpl.html

@@ -18,6 +18,16 @@
                         email.</p>
                 </div>
             </div>
+            <div class="form-group"
+                 ng-class="{'has-error': uploadForm.description.$invalid, 'has-success': !uploadForm.$invalid&&uploadForm.description.$dirty}">
+                <label class="control-label col-sm-2">
+                    Description
+                </label>
+                <div class="col-sm-10">
+                    <textarea name="description" ng-model="certificate.description" placeholder="Something elegant" class="form-control" ng-pattern="/^[\w\-\s]+$/" required></textarea>
+                    <p ng-show="uploadForm.description.$invalid && !uploadForm.description.$pristine" class="help-block">You must give a short description about this authority will be used for, this description should only include alphanumeric characters</p>
+                </div>
+            </div>
             <div class="form-group"
                  ng-class="{'has-error': uploadForm.publicCert.$invalid, 'has-success': !uploadForm.publicCert.$invalid&&uploadForm.publicCert.$dirty}">
                 <label class="control-label col-sm-2">
@@ -66,7 +76,7 @@
         </form>
     </div>
     <div class="modal-footer">
-        <button type="submit" ng-click="upload(certificate)" ng-disabled="uploadForm.$invalid" class="btn btn-success">Import</button>
+        <button type="submit" ng-click="save(certificate)" ng-disabled="uploadForm.$invalid" class="btn btn-success">Import</button>
         <button ng-click="cancel()" class="btn btn-danger">Cancel</button>
     </div>
 </div>

lemur/static/app/angular/certificates/services.js

@@ -77,18 +77,8 @@ angular.module('lemur')
         removeNotification: function (index) {
           this.notifications.splice(index, 1);
         },
-        attachELB: function (elb) {
-          this.selectedELB = null;
-          if (this.elbs === undefined) {
-            this.elbs = [];
-          }
-          this.elbs.push(elb);
-        },
-        removeELB: function (index) {
-          this.elbs.splice(index, 1);
-        },
         findDuplicates: function () {
-          DomainService.findDomainByName(this.extensions.subAltNames[0]).then(function (domains) { //We should do a better job of searchin multiple domains
+          DomainService.findDomainByName(this.extensions.subAltNames[0]).then(function (domains) { //We should do a better job of searching for multiple domains
             this.duplicates = domains.total;
           });
         },
@@ -205,18 +195,6 @@ angular.module('lemur')
       });
     };
 
-    CertificateService.getListeners = function (certificate) {
-      return certificate.getList('listeners').then(function (listeners) {
-        certificate.listeners = listeners;
-      });
-    };
-
-    CertificateService.getELBs = function (certificate) {
-      return certificate.getList('listeners').then(function (elbs) {
-        certificate.elbs = elbs;
-      });
-    };
-
     CertificateService.getDomains = function (certificate) {
       return certificate.getList('domains').then(function (domains) {
         certificate.domains = domains;

lemur/static/app/angular/certificates/view/view.tpl.html

@@ -101,14 +101,20 @@
                         </ul>
                     </tab>
                     <tab heading="Notifications">
-                        <div class="list-group">
+                        <ul class="list-group">
-                            <a href="#/domains/{{ domain.id }}" class="list-group-item" ng-repeat="notification in certificate.notifications">{{ notification.label }}</a>
+                            <li class="list-group-item" ng-repeat="notification in certificate.notifications">
-                        </div>
+                                <strong>{{ notification.label }}</strong>
+                                <span class="pull-right">{{ notification.description}}</span>
+                            </li>
+                        </ul>
                     </tab>
                     <tab heading="Destinations">
-                        <div class="list-group">
+                        <ul class="list-group">
-                            <a href="#/domains/{{ domain.id }}" class="list-group-item" ng-repeat="destination in certificate.destinations">{{ destination.label }}</a>
+                            <li class="list-group-item" ng-repeat="destination in certificate.destinations">
-                        </div>
+                                <strong>{{ destination.label }}</strong>
+                                <span class="pull-right">{{ destination.description }}</span>
+                            </li>
+                        </ul>
                     </tab>
                     <tab heading="Domains">
                         <div class="list-group">

lemur/static/app/angular/dashboard/dashboard.js

@@ -9,13 +9,6 @@ angular.module('lemur')
   })
   .controller('DashboardController', function ($scope, $rootScope, $filter, $location, LemurRestangular) {
 
-    var baseAccounts = LemurRestangular.all('accounts');
-
-    baseAccounts.getList()
-      .then(function (data) {
-        $scope.accounts = data;
-      });
-
     $scope.colours = [
       {
         fillColor: 'rgba(41, 171, 224, 0.2)',
@@ -89,4 +82,9 @@ angular.module('lemur')
       .then(function (data) {
         $scope.expiring = {labels: data.items.labels, values: [data.items.values]};
       });
+
+    LemurRestangular.all('destinations').customGET('stats', {metric: 'certificates'})
+      .then(function (data) {
+        $scope.destinations = {labels: data.items.labels, values: [data.items.values]};
+      });
   });

lemur/static/app/angular/dashboard/dashboard.tpl.html

@@ -36,6 +36,17 @@
         </div>
       </div>
     </div>
+    <div class="row"></div>
+    <div class="col-md-6">
+        <div class="panel panel-default">
+            <div class="panel-heading">
+                <h3 class="panel-title">Destinations</h3>
+            </div>
+            <div class="panel-body">
+                <canvas id="destinationPie" class="chart chart-pie" data="destinations.values" labels="destinations.labels" colours="colours" legend="true"></canvas>
+            </div>
+        </div>
+    </div>
   </div>
   <!-- /.row -->
 </div>

lemur/static/app/angular/destinations/destination/destination.js

@@ -23,6 +23,15 @@ angular.module('lemur')
   .controller('DestinationsEditController', function ($scope, $modalInstance, DestinationService, DestinationApi, PluginService, editId) {
     DestinationApi.get(editId).then(function (destination) {
       $scope.destination = destination;
+      PluginService.getByType('destination').then(function (plugins) {
+        $scope.plugins = plugins;
+        _.each($scope.plugins, function (plugin) {
+          if (plugin.slug === $scope.destination.pluginName) {
+            plugin.pluginOptions = $scope.destination.destinationOptions;
+            $scope.destination.plugin = plugin;
+          }
+        });
+      });
     });
     
     PluginService.getByType('destination').then(function (plugins) {

lemur/static/app/angular/elbs/elb/elb.js

@@ -1,3 +0,0 @@
-/**
- * Created by kglisson on 1/19/15.
- */

lemur/static/app/angular/elbs/elb/elb.tpl.html

@@ -1,10 +0,0 @@
-<!DOCTYPE html>
-<html>
-<head lang="en">
-    <meta charset="UTF-8">
-    <title></title>
-</head>
-<body>
-
-</body>
-</html>

lemur/static/app/angular/elbs/services.js

@@ -1,59 +0,0 @@
-'use strict';
-
-angular.module('lemur')
-  .service('ELBApi', function (LemurRestangular) {
-    LemurRestangular.extendModel('elbs', function (obj) {
-      return angular.extend(obj, {
-        attachListener: function (listener) {
-          if (this.listeners === undefined) {
-            this.listeners = [];
-          }
-          this.listeners.push(listener);
-        },
-        removeListener: function (index) {
-          this.listeners.splice(index, 1);
-        }
-      });
-    });
-    return LemurRestangular.all('elbs');
-  })
-  .service('ELBService', function ($location, ELBApi, toaster) {
-    var ELBService = this;
-    ELBService.findELBByName = function (filterValue) {
-      return ELBApi.getList({'filter[name]': filterValue})
-        .then(function (elbs) {
-          return elbs;
-        });
-    };
-
-    ELBService.getListeners = function (elb) {
-      elb.getList('listeners').then(function (listeners) {
-        elb.listeners = listeners;
-      });
-      return elb;
-    };
-
-    ELBService.create = function (elb) {
-      ELBApi.post(elb).then(function () {
-        toaster.pop({
-          type: 'success',
-          title: 'ELB ' + elb.name,
-          body: 'Has been successfully created!'
-        });
-        $location.path('elbs');
-      });
-    };
-
-    ELBService.update = function (elb) {
-      elb.put().then(function () {
-        toaster.pop({
-          type: 'success',
-          title: 'ELB ' + elb.name,
-          body: 'Has been successfully updated!'
-        });
-        $location.path('elbs');
-      });
-    };
-
-    return ELBService;
-  });

lemur/static/app/angular/elbs/view/view.js

@@ -1,34 +0,0 @@
-'use strict';
-
-angular.module('lemur')
-  .config(function config($routeProvider) {
-    $routeProvider.when('/elbs', {
-      templateUrl: '/angular/elbs/view/view.tpl.html',
-      controller: 'ELBViewController'
-    });
-  })
-
-  .controller('ELBViewController', function ($scope, ELBApi, ELBService, ngTableParams) {
-    $scope.filter = {};
-    $scope.elbsTable = new ngTableParams({
-      page: 1,            // show first page
-      count: 10,          // count per page
-      sorting: {
-        id: 'desc'     // initial sorting
-      },
-      filter: $scope.filter
-    }, {
-      total: 0,           // length of data
-      getData: function ($defer, params) {
-        ELBApi.getList(params.url())
-          .then(function (data) {
-            params.total(data.total);
-            $defer.resolve(data);
-          });
-      }
-    });
-
-    $scope.toggleFilter = function (params) {
-      params.settings().$scope.show_filter = !params.settings().$scope.show_filter;
-    };
-  });

lemur/static/app/angular/elbs/view/view.tpl.html

@@ -1,128 +0,0 @@
-<div class="row">
-  <div class="col-md-12">
-    <h2 class="featurette-heading">ELBs
-      <span class="text-muted"><small>Bring Balance to the Force</small></span></h2>
-    <div class="panel panel-default">
-      <div class="panel-heading">
-        <div class="btn-group pull-right">
-        </div>
-        <div class="btn-group">
-          <button ng-click="toggleFilter(elbsTable)" class="btn btn-default">Filter</button>
-        </div>
-        <div class="clearfix"></div>
-      </div>
-      <div class="table-responsive">
-        <table ng-table="elbsTable" show-filter="false" class="table" template-pagination="angular/pager.html" >
-          <tbody>
-          <tr ng-class="{'even-row': $even }" ng-repeat-start="elb in $data track by $index">
-            <td data-title="'Name'" sortable="'name'" filter="{ 'name': 'text' }">
-              <div class="text-center">{{ elb.name }}</div>
-            </td>
-            <td data-title="'Account'" sortable="'account_id'">
-              <div class="text-center"><span class="label label-primary">{{ elb.account.label }}</span>
-              </div>
-            </td>
-            <td data-title="'Region'" sortable="'region'" filter="{ 'region': 'text' }">
-              <div class="text-center">{{ elb.region }}</div>
-            </td>
-            <td data-title="'VPC'">
-              <div class="text-center">
-                <i class="fa fa-check" ng-show="elb.vpcId" data-placement="top"
-                   data-title="{{ elb.vpcId }}" bs-tooltip></i>
-                <i class="fa fa-times" ng-show="!elb.vpcId"></i>
-              </div>
-            </td>
-            <td data-title="'Internet Accessible'" sortable="'scheme'">
-              <div class="text-center">
-                <i class="fa fa-check" ng-show="elb.scheme == 'internet-facing'"></i>
-                <i class="fa fa-times" ng-show="elb.scheme == 'internal'"></i>
-              </div>
-            </td>
-            <td>
-              <div class="text-center">
-                <i ng-show="!button.toggle" class="fa fa-chevron-down" ng-model="button.toggle"
-                   bs-checkbox></i>
-                <i ng-show="button.toggle" class="fa fa-chevron-up" ng-model="button.toggle"
-                   bs-checkbox></i>
-              </div>
-            </td>
-          </tr>
-          <tr class="warning" ng-show="button.toggle" ng-repeat-end>
-            <td colspan="6">
-              <table class="table">
-                <tbody>
-                <tr>
-                  <th>
-                    <!--<button data-placement="right" data-title="Add Listener" class="btn btn-sm btn-default pull-left" ng-click="addListener()" bs-tooltip><i class="fa fa-plus"></i></button>--></th>
-                  <th>Certificate Name</th>
-                  <th>Instance Port</th>
-                  <th>Instance Protocol</th>
-                  <th>Load Balancer Port</th>
-                  <th>Load Balancer Protocol</th>
-                  <th></th>
-                </tr>
-                <tr class="warning" ng-repeat="listener in elb.listeners">
-                  <td>
-                    <div>
-                      <button data-title="Remove listener" data-placement="right"
-                              class="btn btn-sm btn-danger"
-                              ng-click="removeListener(elb, listener, $index)" bs-tooltip><i
-                        class="fa fa-remove"></i></button>
-                    </div>
-                  </td>
-                  <td>
-                    <div class="text-center">
-                      <input type="text" class="form-control input-sm"
-                             ng-model="listener.certificate.name"
-                             ng-options="cert.name for cert in getCertificate($viewValue, elb)"
-                             placeholder="Certificate name..." bs-typeahead>
-
-                      <div ng-show="showCert">
-                      </div>
-                    </div>
-                  </td>
-                  <td>
-                    <div class="text-center">
-                      <input type="text" ng-model="listener.instancePort"
-                             placeholder="Port number..." class="form-control input-sm"/>
-                    </div>
-                  </td>
-                  <td>
-                    <div class="text-center">
-                      <button type="button" class="btn btn-default btn-sm"
-                              ng-model="listener.instanceProtocol"
-                              ng-options="value for value in protocols" bs-select>
-                        Action <span class="caret"></span>
-                      </button>
-                    </div>
-                  </td>
-                  <td>
-                    <div class="text-center">
-                      <input type="text" ng-model="listener.loadBalancerPort"
-                             placeholder="Port number..." class="form-control input-sm"/>
-                    </div>
-                  </td>
-                  <td>
-                    <div class="text-center">
-                      <button type="button" class="btn btn-default btn-sm"
-                              ng-model="listener.loadBalancerProtocol"
-                              ng-options="value for value in protocols" bs-select>
-                        Action <span class="caret"></span>
-                      </button>
-                    </div>
-                  </td>
-                  <td>
-                    <button data-placement="right" data-title="Save" class="btn btn-sm btn-primary"
-                            ng-click="saveListener(listener)" bs-tooltip><i
-                      class="fa fa-floppy-o"></i></button>
-                  </td>
-                </tr>
-                </tbody>
-              </table>
-            </td>
-          </tr>
-          </tbody>
-        </table>
-      </div>
-    </div>
-  </div>

lemur/static/app/angular/listeners/services.js

@@ -1,37 +0,0 @@
-'use strict';
-
-angular.module('lemur')
-  .service('ListenerApi', function (LemurRestangular) {
-    return LemurRestangular.all('listeners');
-  })
-  .service('ListenerService', function ($location, ListenerApi) {
-    var ListenerService = this;
-    ListenerService.findListenerByName = function (filterValue) {
-      return ListenerApi.getList({'filter[name]': filterValue})
-        .then(function (roles) {
-          return roles;
-        });
-    };
-
-    ListenerService.create = function (role) {
-      ListenerApi.post(role).then(function () {
-        toaster.pop({
-          type: 'success',
-          title: 'Listener ' + role.name,
-          body: 'Has been successfully created!'
-        });
-        $location.path('roles/view');
-      });
-    };
-
-    ListenerService.update = function (role) {
-      role.put().then(function () {
-        toaster.pop({
-          type: 'success',
-          title: 'Listener ' + role.name,
-          body: 'Has been successfully updated!'
-        });
-        $location.path('roles/view');
-      });
-    };
-  });

lemur/static/app/angular/notifications/notification/notification.js

@@ -0,0 +1,65 @@
+'use strict';
+
+angular.module('lemur')
+
+  .controller('NotificationsCreateController', function ($scope, $modalInstance, PluginService, NotificationService, CertificateService, LemurRestangular){
+    $scope.notification = LemurRestangular.restangularizeElement(null, {}, 'notifications');
+
+    PluginService.getByType('notification').then(function (plugins) {
+      $scope.plugins = plugins;
+    });
+    $scope.save = function (notification) {
+      NotificationService.create(notification).then(
+        function () {
+          $modalInstance.close();
+        },
+        function () {
+
+        }
+      );
+    };
+
+    $scope.cancel = function () {
+      $modalInstance.dismiss('cancel');
+    };
+
+    $scope.certificateService = CertificateService;
+  })
+
+  .controller('NotificationsEditController', function ($scope, $modalInstance, NotificationService, NotificationApi, PluginService, CertificateService, editId) {
+    NotificationApi.get(editId).then(function (notification) {
+      $scope.notification = notification;
+      PluginService.getByType('notification').then(function (plugins) {
+        $scope.plugins = plugins;
+        _.each($scope.plugins, function (plugin) {
+          if (plugin.slug === $scope.notification.pluginName) {
+            plugin.pluginOptions = $scope.notification.notificationOptions;
+            $scope.notification.plugin = plugin;
+          }
+        });
+      });
+      NotificationService.getCertificates(notification);
+    });
+
+    PluginService.getByType('notification').then(function (plugins) {
+      $scope.plugins = plugins;
+      _.each($scope.plugins, function (plugin) {
+        if (plugin.slug === $scope.notification.pluginName) {
+          plugin.pluginOptions = $scope.notification.notificationOptions;
+          $scope.notification.plugin = plugin;
+        }
+      });
+    });
+
+    $scope.save = function (notification) {
+      NotificationService.update(notification).then(function () {
+        $modalInstance.close();
+      });
+    };
+
+    $scope.cancel = function () {
+      $modalInstance.dismiss('cancel');
+    };
+
+    $scope.certificateService = CertificateService;
+  });

lemur/static/app/angular/notifications/notification/notification.tpl.html

@@ -0,0 +1,87 @@
+<div class="modal-header">
+    <div class="modal-title">
+        <h3 class="modal-header"><span ng-show="!notification.fromServer">Create</span><span ng-show="notification.fromServer">Edit</span> Notification <span class="text-muted"><small>you gotta speak louder son!</small></span></h3>
+    </div>
+    <div class="modal-body">
+        <form name="createForm" class="form-horizontal" role="form" novalidate>
+            <div class="form-group"
+                 ng-class="{'has-error': createForm.label.$invalid, 'has-success': !createForm.label.$invalid&&createForm.label.$dirty}">
+                <label class="control-label col-sm-2">
+                    Label
+                </label>
+                <div class="col-sm-10">
+                    <input name="label" ng-model="notification.label" placeholder="Label" class="form-control" required/>
+                    <p ng-show="createForm.label.$invalid && !createForm.label.$pristine" class="help-block">You must enter an notification label</p>
+                </div>
+            </div>
+            <div class="form-group">
+                <label class="control-label col-sm-2">
+                    Description
+                </label>
+                <div class="col-sm-10">
+                    <textarea name="comments" ng-model="notification.description" placeholder="Something elegant" class="form-control" ></textarea>
+                </div>
+            </div>
+            <div class="form-group">
+                <label class="control-label col-sm-2">
+                    Plugin
+                </label>
+                <div class="col-sm-10">
+                    <select class="form-control" ng-model="notification.plugin" ng-options="plugin.title for plugin in plugins" required></select>
+                </div>
+            </div>
+            <div class="form-group" ng-repeat="item in notification.plugin.pluginOptions">
+                <ng-form name="subForm" class="form-horizontal" role="form" novalidate>
+                    <div ng-class="{'has-error': subForm.sub.$invalid, 'has-success': !subForm.sub.$invalid&&subForm.sub.$dirty}">
+                        <label class="control-label col-sm-2">
+                            {{ item.name | titleCase }}
+                        </label>
+                        <div class="col-sm-10">
+                            <input name="sub" ng-if="item.type == 'int'" type="number" ng-pattern="item.validation" class="form-control" ng-model="item.value"/>
+                            <select name="sub" ng-if="item.type == 'select'" class="form-control" ng-options="i as (i | titleCase) for i in item.available" ng-model="item.value"></select>
+                            <input name="sub" ng-if="item.type == 'bool'" class="form-control" type="checkbox" ng-model="item.value">
+                            <input name="sub" ng-if="item.type == 'str'" ng-pattern="item.validation" type="text" class="form-control" ng-model="item.value"/>
+                            <p ng-show="subForm.sub.$invalid && !subForm.sub.$pristine" class="help-block">{{ item.helpMessage }}</p>
+                        </div>
+                    </div>
+                </ng-form>
+            </div>
+            <div class="form-group">
+                <label class="control-label col-sm-2">
+                    Certificates
+                </label>
+                <div class="col-sm-10">
+                    <div class="input-group">
+                        <input type="text" ng-model="notification.selectedCertificate" placeholder="Certificate Name"
+                               typeahead="certificate.name for certificate in certificateService.findCertificatesByName($viewValue)" typeahead-loading="loadingCertificates"
+                               class="form-control input-md" typeahead-on-select="notification.attachCertificate($item)" typeahead-min-wait="50">
+                        <span class="input-group-btn">
+                            <button ng-model="certificates.show" class="btn btn-md btn-default" btn-checkbox btn-checkbox-true="1" btn-checkbox-false="0">
+                                <span class="badge">{{ notification.certificates.total || 0 }}</span>
+                            </button>
+                        </span>
+                    </div>
+                    <table ng-show="notification.certificates" class="table">
+                        <tr ng-repeat="certificate in notification.certificates track by $index">
+                            <td><a class="btn btn-sm btn-info" href="#">{{ certificate.name }}</a></td>
+                            <td><span class="text-muted">{{ certificate.description }}</span></td>
+                            <td>
+                                <button type="button" ng-click="notification.removeCertificate($index)" class="btn btn-danger btn-sm pull-right">Remove</button>
+                            </td>
+                        </tr>
+                        <tr>
+                            <td></td>
+                            <td></td>
+                            <td><a class="pull-right" ng-click="loadMoreCertificates()"><strong>More</strong></a></td>
+                        </tr>
+                    </table>
+                </div>
+            </div>
+        </form>
+    </div>
+    <div class="modal-footer">
+        <button ng-click="save(notification)" type="submit" ng-disabled="createForm.$invalid" class="btn btn-primary">Save</button>
+        <button ng-click="cancel()" class="btn btn-danger">Cancel</button>
+    </div>
+</div>
+

lemur/static/app/angular/notifications/services.js

@@ -0,0 +1,106 @@
+'use strict';
+angular.module('lemur')
+  .service('NotificationApi', function (LemurRestangular) {
+    LemurRestangular.extendModel('notifications', function (obj) {
+      return angular.extend(obj, {
+        attachCertificate: function (certificate) {
+          this.selectedCertificate = null;
+          if (this.certificates === undefined) {
+            this.certificates = [];
+          }
+          this.certificates.push(certificate);
+        },
+        removeCertificate: function (index) {
+          this.certificate.splice(index, 1);
+        }
+      });
+    });
+    return LemurRestangular.all('notifications');
+  })
+  .service('NotificationService', function ($location,  NotificationApi, PluginService, toaster) {
+    var NotificationService = this;
+    NotificationService.findNotificationsByName = function (filterValue) {
+      return NotificationApi.getList({'filter[label]': filterValue})
+        .then(function (notifications) {
+          return notifications;
+        });
+    };
+
+    NotificationService.getCertificates = function (notification) {
+      notification.getList('certificates').then(function (certificates) {
+        notification.certificates = certificates;
+      });
+    };
+
+    NotificationService.getPlugin = function (notification) {
+      return PluginService.getByName(notification.pluginName).then(function (plugin) {
+        notification.plugin = plugin;
+      });
+    };
+
+
+    NotificationService.loadMoreCertificates = function (notification, page) {
+      notification.getList('certificates', {page: page}).then(function (certificates) {
+        _.each(certificates, function (certificate) {
+          notification.roles.push(certificate);
+        });
+      });
+    };
+
+    NotificationService.create = function (notification) {
+      return NotificationApi.post(notification).then(
+        function () {
+          toaster.pop({
+            type: 'success',
+            title: notification.label,
+            body: 'Successfully created!'
+          });
+          $location.path('notifications');
+        },
+        function (response) {
+          toaster.pop({
+            type: 'error',
+            title: notification.label,
+            body: 'Was not created! ' + response.data.message
+          });
+        });
+    };
+
+    NotificationService.update = function (notification) {
+      return notification.put().then(
+        function () {
+          toaster.pop({
+            type: 'success',
+            title: notification.label,
+            body: 'Successfully updated!'
+          });
+          $location.path('notifications');
+        },
+        function (response) {
+          toaster.pop({
+            type: 'error',
+            title: notification.label,
+            body: 'Was not updated! ' + response.data.message
+          });
+        });
+    };
+
+    NotificationService.updateActive = function (notification) {
+      notification.put().then(
+        function () {
+          toaster.pop({
+            type: 'success',
+            title: notification.name,
+            body: 'Successfully updated!'
+          });
+        },
+        function (response) {
+          toaster.pop({
+            type: 'error',
+            title: notification.name,
+            body: 'Was not updated! ' + response.data.message
+          });
+        });
+    };
+    return NotificationService;
+  });

lemur/static/app/angular/notifications/view/view.js

@@ -0,0 +1,96 @@
+'use strict';
+
+angular.module('lemur')
+
+  .config(function config($routeProvider) {
+    $routeProvider.when('/notifications', {
+      templateUrl: '/angular/notifications/view/view.tpl.html',
+      controller: 'NotificationsViewController'
+    });
+  })
+
+  .controller('NotificationsViewController', function ($q, $scope, $modal, NotificationApi, NotificationService, ngTableParams, toaster) {
+    $scope.filter = {};
+    $scope.notificationsTable = new ngTableParams({
+      page: 1,            // show first page
+      count: 10,          // count per page
+      sorting: {
+        id: 'desc'     // initial sorting
+      },
+      filter: $scope.filter
+    }, {
+      total: 0,           // length of data
+      getData: function ($defer, params) {
+        NotificationApi.getList(params.url()).then(
+          function (data) {
+            _.each(data, function (notification) {
+              NotificationService.getPlugin(notification);
+            });
+            params.total(data.total);
+            $defer.resolve(data);
+          }
+        );
+      }
+    });
+
+    $scope.getNotificationStatus = function () {
+      var def = $q.defer();
+      def.resolve([{'title': 'Active', 'id': true}, {'title': 'Inactive', 'id': false}]);
+      return def;
+    };
+
+    $scope.remove = function (notification) {
+      notification.remove().then(
+        function () {
+          $scope.notificationsTable.reload();
+        },
+        function (response) {
+          toaster.pop({
+            type: 'error',
+            title: 'Opps',
+            body: 'I see what you did there' + response.data.message
+          });
+        }
+      );
+    };
+
+    $scope.edit = function (notificationId) {
+      var modalInstance = $modal.open({
+        animation: true,
+        templateUrl: '/angular/notifications/notification/notification.tpl.html',
+        controller: 'NotificationsEditController',
+        size: 'lg',
+        resolve: {
+          editId: function () {
+            return notificationId;
+          }
+        }
+      });
+
+      modalInstance.result.then(function () {
+        $scope.notificationsTable.reload();
+      });
+
+    };
+
+    $scope.create = function () {
+      var modalInstance = $modal.open({
+        animation: true,
+        controller: 'NotificationsCreateController',
+        templateUrl: '/angular/notifications/notification/notification.tpl.html',
+        size: 'lg'
+      });
+
+      modalInstance.result.then(function () {
+        $scope.notificationsTable.reload();
+      });
+
+    };
+
+    $scope.toggleFilter = function (params) {
+      params.settings().$scope.show_filter = !params.settings().$scope.show_filter;
+    };
+
+    $scope.notificationService = NotificationService;
+
+  });

lemur/static/app/angular/notifications/view/view.tpl.html

@@ -0,0 +1,52 @@
+<div class="row">
+    <div class="col-md-12">
+        <h2 class="featurette-heading">Notifications
+            <span class="text-muted"><small>you have to speak up son!</small></span></h2>
+        <div class="panel panel-default">
+            <div class="panel-heading">
+                <div class="btn-group pull-right">
+                    <button ng-click="create()" class="btn btn-primary">Create</button>
+                </div>
+                <div class="btn-group">
+                    <button ng-click="toggleFilter(notificationsTable)" class="btn btn-default">Filter</button>
+                </div>
+                <div class="clearfix"></div>
+            </div>
+            <div class="table-responsive">
+                <table ng-table="notificationsTable" class="table table-striped" show-filter="false" template-pagination="angular/pager.html" >
+                    <tbody>
+                    <tr ng-repeat="notification in $data track by $index">
+                        <td data-title="'Label'" sortable="'label'" filter="{ 'label': 'text' }">
+                            <ul class="list-unstyled">
+                                <li>{{ notification.label }}</li>
+                                <li><span class="text-muted">{{ notification.description }}</span></li>
+                            </ul>
+                        </td>
+                        <td data-title="'Plugin'">
+                            <ul class="list-unstyled">
+                                <li>{{ notification.plugin.title }}</li>
+                                <li><span class="text-muted">{{ notification.plugin.description }}</span></li>
+                            </ul>
+                        </td>
+                        <td data-title="'Active'" filter="{ 'active': 'select' }" filter-data="getNotificationStatus()">
+                            <form>
+                                <switch ng-change="notificationService.updateActive(notification)" id="status" name="status" ng-model="notification.active" class="green small"></switch>
+                            </form>
+                        </td>
+                        <td data-title="''">
+                            <div class="btn-group-vertical pull-right">
+                                <button tooltip="Edit Notification" ng-click="edit(notification.id)" class="btn btn-sm btn-info">
+                                    Edit
+                                </button>
+                                <button tooltip="Delete Notification" ng-click="remove(notification)" type="button" class="btn btn-sm btn-danger pull-left">
+                                    Remove
+                                </button>
+                            </div>
+                        </td>
+                    </tr>
+                    </tbody>
+                </table>
+            </div>
+        </div>
+    </div>
+</div>

lemur/static/app/angular/sources/services.js

@@ -0,0 +1,59 @@
+'use strict';
+angular.module('lemur')
+  .service('SourceApi', function (LemurRestangular) {
+    return LemurRestangular.all('sources');
+  })
+  .service('SourceService', function ($location,  SourceApi, PluginService, toaster) {
+    var SourceService = this;
+    SourceService.findSourcesByName = function (filterValue) {
+      return SourceApi.getList({'filter[label]': filterValue})
+        .then(function (sources) {
+          return sources;
+        });
+    };
+
+    SourceService.create = function (source) {
+      return SourceApi.post(source).then(
+        function () {
+          toaster.pop({
+            type: 'success',
+            title: source.label,
+            body: 'Successfully created!'
+          });
+          $location.path('sources');
+        },
+        function (response) {
+          toaster.pop({
+            type: 'error',
+            title: source.label,
+            body: 'Was not created! ' + response.data.message
+          });
+        });
+    };
+
+    SourceService.update = function (source) {
+      return source.put().then(
+        function () {
+          toaster.pop({
+            type: 'success',
+            title: source.label,
+            body: 'Successfully updated!'
+          });
+          $location.path('sources');
+        },
+        function (response) {
+          toaster.pop({
+            type: 'error',
+            title: source.label,
+            body: 'Was not updated! ' + response.data.message
+          });
+        });
+    };
+
+    SourceService.getPlugin = function (source) {
+      return PluginService.getByName(source.pluginName).then(function (plugin) {
+        source.plugin = plugin;
+      });
+    };
+    return SourceService;
+  });

lemur/static/app/angular/sources/source/source.js

@@ -0,0 +1,56 @@
+'use strict';
+
+angular.module('lemur')
+
+  .controller('SourcesCreateController', function ($scope, $modalInstance, PluginService, SourceService, LemurRestangular){
+    $scope.source = LemurRestangular.restangularizeElement(null, {}, 'sources');
+
+    PluginService.getByType('source').then(function (plugins) {
+        $scope.plugins = plugins;
+    });
+
+    $scope.save = function (source) {
+      SourceService.create(source).then(function () {
+        $modalInstance.close();
+      });
+    };
+
+    $scope.cancel = function () {
+      $modalInstance.dismiss('cancel');
+    };
+  })
+
+  .controller('SourcesEditController', function ($scope, $modalInstance, SourceService, SourceApi, PluginService, editId) {
+    SourceApi.get(editId).then(function (source) {
+      $scope.source = source;
+      PluginService.getByType('source').then(function (plugins) {
+        $scope.plugins = plugins;
+        _.each($scope.plugins, function (plugin) {
+          if (plugin.slug === $scope.source.pluginName) {
+            plugin.pluginOptions = $scope.source.sourceOptions;
+            $scope.source.plugin = plugin;
+          }
+        });
+      });
+    });
+    
+    PluginService.getByType('source').then(function (plugins) {
+      $scope.plugins = plugins;
+      _.each($scope.plugins, function (plugin) {
+        if (plugin.slug === $scope.source.pluginName) {
+          plugin.pluginOptions = $scope.source.sourceOptions;
+          $scope.source.plugin = plugin;
+        }
+      });
+    });
+
+    $scope.save = function (source) {
+      SourceService.update(source).then(function () {
+        $modalInstance.close();
+      });
+    };
+
+    $scope.cancel = function () {
+      $modalInstance.dismiss('cancel');
+    };
+  });

lemur/static/app/angular/sources/source/source.tpl.html

@@ -0,0 +1,56 @@
+<div class="modal-header">
+    <div class="modal-title">
+        <h3 class="modal-header"><span ng-show="!source.fromServer">Create</span><span ng-show="source.fromServer">Edit</span> Source <span class="text-muted"><small>oh the places you will go!</small></span></h3>
+    </div>
+    <div class="modal-body">
+        <form name="createForm" class="form-horizontal" role="form" novalidate>
+            <div class="form-group"
+                 ng-class="{'has-error': createForm.label.$invalid, 'has-success': !createForm.label.$invalid&&createForm.label.$dirty}">
+                <label class="control-label col-sm-2">
+                    Label
+                </label>
+                <div class="col-sm-10">
+                    <input name="label" ng-model="source.label" placeholder="Label" class="form-control" required/>
+                    <p ng-show="createForm.label.$invalid && !createForm.label.$pristine" class="help-block">You must enter an source label</p>
+                </div>
+            </div>
+            <div class="form-group">
+                <label class="control-label col-sm-2">
+                    Description
+                </label>
+                <div class="col-sm-10">
+                    <textarea name="comments" ng-model="source.description" placeholder="Something elegant" class="form-control" ></textarea>
+                </div>
+            </div>
+            <div class="form-group">
+                <label class="control-label col-sm-2">
+                    Plugin
+                </label>
+                <div class="col-sm-10">
+                    <select class="form-control" ng-model="source.plugin" ng-options="plugin.title for plugin in plugins" required></select>
+                </div>
+            </div>
+            <div class="form-group" ng-repeat="item in source.plugin.pluginOptions">
+                <ng-form name="subForm" class="form-horizontal" role="form" novalidate>
+                    <div ng-class="{'has-error': subForm.sub.$invalid, 'has-success': !subForm.sub.$invalid&&subForm.sub.$dirty}">
+                        <label class="control-label col-sm-2">
+                            {{ item.name | titleCase }}
+                        </label>
+                        <div class="col-sm-10">
+                            <input name="sub" ng-if="item.type == 'int'" type="number" ng-pattern="/^[0-9]{12,12}$/" class="form-control" ng-model="item.value"/>
+                            <select name="sub" ng-if="item.type == 'select'" class="form-control" ng-options="i for i in item.available" ng-model="item.value"></select>
+                            <input name="sub" ng-if="item.type == 'bool'" class="form-control" type="checkbox" ng-model="item.value">
+                            <input name="sub" ng-if="item.type == 'str'" type="text" class="form-control" ng-model="item.value"/>
+                            <p ng-show="subForm.sub.$invalid && !subForm.sub.$pristine" class="help-block">{{ item.helpMessage }}</p>
+                        </div>
+                    </div>
+                </ng-form>
+            </div>
+        </form>
+    </div>
+    <div class="modal-footer">
+        <button ng-click="save(source)" type="submit" ng-disabled="createForm.$invalid" class="btn btn-primary">Save</button>
+        <button ng-click="cancel()" class="btn btn-danger">Cancel</button>
+    </div>
+</div>
+

lemur/static/app/angular/sources/view/view.js

@@ -0,0 +1,88 @@
+'use strict';
+
+angular.module('lemur')
+
+  .config(function config($routeProvider) {
+    $routeProvider.when('/sources', {
+      templateUrl: '/angular/sources/view/view.tpl.html',
+      controller: 'SourcesViewController'
+    });
+  })
+
+  .controller('SourcesViewController', function ($scope, $modal, SourceApi, SourceService, ngTableParams, toaster) {
+    $scope.filter = {};
+    $scope.sourcesTable = new ngTableParams({
+      page: 1,            // show first page
+      count: 10,          // count per page
+      sorting: {
+        id: 'desc'     // initial sorting
+      },
+      filter: $scope.filter
+    }, {
+      total: 0,           // length of data
+      getData: function ($defer, params) {
+        SourceApi.getList(params.url()).then(
+          function (data) {
+            _.each(data, function (source) {
+              SourceService.getPlugin(source);
+            });
+            params.total(data.total);
+            $defer.resolve(data);
+          }
+        );
+      }
+    });
+
+    $scope.remove = function (source) {
+      source.remove().then(
+        function () {
+            $scope.sourcesTable.reload();
+          },
+          function (response) {
+            toaster.pop({
+              type: 'error',
+              title: 'Opps',
+              body: 'I see what you did there' + response.data.message
+            });
+          }
+      );
+    };
+
+    $scope.edit = function (sourceId) {
+      var modalInstance = $modal.open({
+        animation: true,
+        templateUrl: '/angular/sources/source/source.tpl.html',
+        controller: 'SourcesEditController',
+        size: 'lg',
+        resolve: {
+            editId: function () {
+              return sourceId;
+            }
+        }
+      });
+
+      modalInstance.result.then(function () {
+        $scope.sourcesTable.reload();
+      });
+
+    };
+
+    $scope.create = function () {
+      var modalInstance = $modal.open({
+        animation: true,
+        controller: 'SourcesCreateController',
+        templateUrl: '/angular/sources/source/source.tpl.html',
+        size: 'lg'
+      });
+
+      modalInstance.result.then(function () {
+        $scope.sourcesTable.reload();
+      });
+
+    };
+
+    $scope.toggleFilter = function (params) {
+      params.settings().$scope.show_filter = !params.settings().$scope.show_filter;
+    };
+
+  });

lemur/static/app/angular/sources/view/view.tpl.html

@@ -0,0 +1,47 @@
+<div class="row">
+  <div class="col-md-12">
+    <h2 class="featurette-heading">Sources
+      <span class="text-muted"><small>where are you from?</small></span></h2>
+    <div class="panel panel-default">
+      <div class="panel-heading">
+        <div class="btn-group pull-right">
+          <button ng-click="create()" class="btn btn-primary">Create</button>
+        </div>
+        <div class="btn-group">
+          <button ng-click="toggleFilter(sourcesTable)" class="btn btn-default">Filter</button>
+        </div>
+        <div class="clearfix"></div>
+      </div>
+      <div class="table-responsive">
+        <table ng-table="sourcesTable" class="table table-striped" show-filter="false" template-pagination="angular/pager.html" >
+          <tbody>
+          <tr ng-repeat="source in $data track by $index">
+            <td data-title="'Label'" sortable="'label'" filter="{ 'label': 'text' }">
+              <ul class="list-unstyled">
+                <li>{{ source.label }}</li>
+                <li><span class="text-muted">{{ source.description }}</span></li>
+              </ul>
+            </td>
+              <td data-title="'Plugin'">
+                  <ul class="list-unstyled">
+                      <li>{{ source.plugin.title }}</li>
+                      <li><span class="text-muted">{{ source.plugin.description }}</span></li>
+                  </ul>
+              </td>
+            <td data-title="''">
+              <div class="btn-group-vertical pull-right">
+                <button tooltip="Edit Source" ng-click="edit(source.id)" class="btn btn-sm btn-info">
+                  Edit
+                </button>
+                <button tooltip="Delete Source" ng-click="remove(source)" type="button" class="btn btn-sm btn-danger pull-left">
+                  Remove
+                </button>
+              </div>
+            </td>
+          </tr>
+          </tbody>
+        </table>
+      </div>
+    </div>
+  </div>
+</div>

lemur/static/app/index.html

@@ -54,6 +54,7 @@
                     <li><a href="/#/authorities">Authorities</a></li>
                     <li><a href="/#/notifications">Notifications</a></li>
                     <li><a href="/#/destinations">Destinations</a></li>
+                    <li><a href="/#/sources">Sources</a></li>
                     <li></li>
                     <li class="dropdown" dropdown on-toggle="toggled(open)">
                         <a href class="dropdown-toggle" dropdown-toggle>Settings <span class="caret"></span></a>

lemur/tests/test_notifications.py

@@ -6,7 +6,7 @@ def test_crud(session):
     notification = create('testnotify', 'email-notification', {}, 'notify1', [])
     assert notification.id > 0
 
-    notification = update(notification.id, 'testnotify2', {}, 'notify2', [])
+    notification = update(notification.id, 'testnotify2', {}, 'notify2', True, [])
     assert notification.label == 'testnotify2'
 
     assert len(get_all()) == 1

lemur/tests/test_sources.py

@@ -0,0 +1,134 @@
+from lemur.sources.service import *  # noqa
+from lemur.sources.views import *  # noqa
+
+from json import dumps
+
+
+def test_crud(session):
+    source = create('testdest', 'aws-source', {}, description='source1')
+    assert source.id > 0
+
+    source = update(source.id, 'testdest2', {}, 'source2')
+    assert source.label == 'testdest2'
+
+    assert len(get_all()) == 1
+
+    delete(1)
+    assert len(get_all()) == 0
+
+
+def test_source_get(client):
+    assert client.get(api.url_for(Sources, source_id=1)).status_code == 401
+
+
+def test_source_post(client):
+    assert client.post(api.url_for(Sources, source_id=1), data={}).status_code == 405
+
+
+def test_source_put(client):
+    assert client.put(api.url_for(Sources, source_id=1), data={}).status_code == 401
+
+
+def test_source_delete(client):
+    assert client.delete(api.url_for(Sources, source_id=1)).status_code == 401
+
+
+def test_source_patch(client):
+    assert client.patch(api.url_for(Sources, source_id=1), data={}).status_code == 405
+
+
+VALID_USER_HEADER_TOKEN = {
+    'Authorization': 'Basic ' + 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE0MzUyMzMzNjksInN1YiI6MSwiZXhwIjoxNTIxNTQ2OTY5fQ.1qCi0Ip7mzKbjNh0tVd3_eJOrae3rNa_9MCVdA4WtQI'}
+
+
+def test_auth_source_get(client):
+    assert client.get(api.url_for(Sources, source_id=1), headers=VALID_USER_HEADER_TOKEN).status_code == 200
+
+
+def test_auth_source_post_(client):
+    assert client.post(api.url_for(Sources, source_id=1), data={}, headers=VALID_USER_HEADER_TOKEN).status_code == 405
+
+
+def test_auth_source_put(client):
+    assert client.put(api.url_for(Sources, source_id=1), data={}, headers=VALID_USER_HEADER_TOKEN).status_code == 403
+
+
+def test_auth_source_delete(client):
+    assert client.delete(api.url_for(Sources, source_id=1), headers=VALID_USER_HEADER_TOKEN).status_code == 403
+
+
+def test_auth_source_patch(client):
+    assert client.patch(api.url_for(Sources, source_id=1), data={}, headers=VALID_USER_HEADER_TOKEN).status_code == 405
+
+
+VALID_ADMIN_HEADER_TOKEN = {
+    'Authorization': 'Basic ' + 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE0MzUyNTAyMTgsInN1YiI6MiwiZXhwIjoxNTIxNTYzODE4fQ.6mbq4-Ro6K5MmuNiTJBB153RDhlM5LGJBjI7GBKkfqA'}
+
+
+def test_admin_source_get(client):
+    assert client.get(api.url_for(Sources, source_id=1), headers=VALID_ADMIN_HEADER_TOKEN).status_code == 200
+
+
+def test_admin_source_post(client):
+    assert client.post(api.url_for(Sources, source_id=1), data={}, headers=VALID_ADMIN_HEADER_TOKEN).status_code == 405
+
+
+def test_admin_source_put(client):
+    assert client.put(api.url_for(Sources, source_id=1), data={}, headers=VALID_ADMIN_HEADER_TOKEN).status_code == 400
+
+
+def test_admin_source_delete(client):
+    assert client.delete(api.url_for(Sources, source_id=1), headers=VALID_ADMIN_HEADER_TOKEN).status_code == 200
+
+
+def test_admin_source_patch(client):
+    assert client.patch(api.url_for(Sources, source_id=1), data={}, headers=VALID_ADMIN_HEADER_TOKEN).status_code == 405
+
+
+def test_sources_get(client):
+    assert client.get(api.url_for(SourcesList)).status_code == 401
+
+
+def test_sources_post(client):
+    assert client.post(api.url_for(SourcesList), data={}).status_code == 401
+
+
+def test_sources_put(client):
+    assert client.put(api.url_for(SourcesList), data={}).status_code == 405
+
+
+def test_sources_delete(client):
+    assert client.delete(api.url_for(SourcesList)).status_code == 405
+
+
+def test_sources_patch(client):
+    assert client.patch(api.url_for(SourcesList), data={}).status_code == 405
+
+
+def test_auth_sources_get(client):
+    assert client.get(api.url_for(SourcesList), headers=VALID_USER_HEADER_TOKEN).status_code == 200
+
+
+def test_auth_sources_post(client):
+    assert client.post(api.url_for(SourcesList), data={}, headers=VALID_USER_HEADER_TOKEN).status_code == 403
+
+
+def test_admin_sources_get(client):
+    resp = client.get(api.url_for(SourcesList), headers=VALID_ADMIN_HEADER_TOKEN)
+    assert resp.status_code == 200
+    assert resp.json == {'items': [], 'total': 0}
+
+
+def test_admin_sources_crud(client):
+    assert client.post(api.url_for(SourcesList), headers=VALID_ADMIN_HEADER_TOKEN).status_code == 400
+    data = {'plugin': {'slug': 'aws-source', 'pluginOptions': {}}, 'label': 'test', 'description': 'test'}
+    resp = client.post(api.url_for(SourcesList), data=dumps(data), content_type='application/json', headers=VALID_ADMIN_HEADER_TOKEN)
+    assert resp.status_code == 200
+    assert client.get(api.url_for(Sources, source_id=resp.json['id']), headers=VALID_ADMIN_HEADER_TOKEN).status_code == 200
+    resp = client.get(api.url_for(SourcesList), headers=VALID_ADMIN_HEADER_TOKEN)
+    assert resp.status_code == 200
+    assert resp.json['items'][0]['description'] == 'test'
+    assert client.delete(api.url_for(Sources, source_id=2), headers=VALID_ADMIN_HEADER_TOKEN).status_code == 200
+    resp = client.get(api.url_for(SourcesList), headers=VALID_ADMIN_HEADER_TOKEN)
+    assert resp.status_code == 200
+    assert resp.json == {'items': [], 'total': 0}

setup.py

@@ -42,7 +42,8 @@ install_requires = [
     'cryptography>=1.0dev',
     'pyopenssl==0.15.1',
     'pyjwt==1.0.1',
-    'xmltodict==0.9.2'
+    'xmltodict==0.9.2',
+    'lockfile==0.10.2'
 ]
 
 tests_require = [
@@ -135,10 +136,10 @@ setup(
         'lemur.plugins': [
             'verisign_issuer = lemur.plugins.lemur_verisign.plugin:VerisignIssuerPlugin',
             'cloudca_issuer = lemur.plugins.lemur_cloudca.plugin:CloudCAIssuerPlugin',
-            'cloudca_source = lemur.plugins.lemur_cloudca.plugin:CloudCASourcePlugin'
+            'cloudca_source = lemur.plugins.lemur_cloudca.plugin:CloudCASourcePlugin',
             'aws_destination = lemur.plugins.lemur_aws.plugin:AWSDestinationPlugin',
-            'aws_source = lemur.plugins.lemur_aws.plugin:AWSSourcePlugin'
+            'aws_source = lemur.plugins.lemur_aws.plugin:AWSSourcePlugin',
-            'email_notification = lemur.plugins.lemur_email.plugin:EmailNotificationPlugin'
+            'email_notification = lemur.plugins.lemur_email.plugin:EmailNotificationPlugin',
         ],
     },
     classifiers=[