Infra
/
lemur
5
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

initial commit

This commit is contained in:
Kevin Glisson 2015-06-22 13:47:27 -07:00
commit 4330ac9c05
228 changed files with 16656 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.bowerrc Normal file
View File

@ -0,0 +1,3 @@
{
"directory": "lemur/static/app/vendor/bower_components"
}

1
.gitattributes vendored Normal file
View File

@ -0,0 +1 @@
* text=auto

28
.gitignore vendored Normal file
View File

@ -0,0 +1,28 @@
.coverage
.tox
.DS_Store
/.tmp/
*.egg-info
*.pyc
*.log
*.egg
*.db
*.pid
*.enc
MANIFEST
test.conf
pip-log.txt
/htmlcov
/cover
/build
/dist
/node_modules/
/bower_components/
/docs/html
/docs/doctrees
/lemur/static/dist/
/lemur/static/app/vendor/
/wheelhouse
docs/_build
.editorconfig
.idea

4
.jshintignore Normal file
View File

@ -0,0 +1,4 @@
tests/
lemur/static/lemur/scripts/lib/
lemur/static/lemur/dist/
lemur/static/lemur/vendor/

24
.jshintrc Normal file
View File

@ -0,0 +1,24 @@
{
"node": true,
"browser": true,
"esnext": true,
"bitwise": true,
"camelcase": true,
"curly": true,
"eqeqeq": true,
"immed": true,
"indent": 2,
"latedef": true,
"newcap": true,
"noarg": true,
"quotmark": "single",
"regexp": true,
"undef": true,
"unused": true,
"strict": true,
"trailing": true,
"smarttabs": true,
"globals": {
"angular": false
}
}

7
.travis.yml Normal file
View File

@ -0,0 +1,7 @@
language: node_js
node_js:
- '0.8'
- '0.10'
before_script:
- 'npm install -g bower grunt-cli'
- 'bower install'

1
AUTHORS Normal file
View File

@ -0,0 +1 @@
- Kevin Glisson (kglisson@netflix.com)

0
CHANGES Normal file
View File

202
LICENSE Normal file
View File

@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2014 Netflix, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

19
README.rst Normal file
View File

@ -0,0 +1,19 @@
Lemur
*****
Lemur manages SSL certificate creation. It provides a central portal for developers to issuer their own SSL certificates with 'sane' defaults.
It works on CPython 2.7. It is known
to work on Ubuntu Linux and OS X.
Project resources
=================
- `Documentation <http://lemur.readthedocs.org/>`_
- `Source code <https://github.com/netflix/lemur>`_
- `Issue tracker <https://github.com/netflix/lemur/issues>`_
.. image:: https://badges.gitter.im/Join%20Chat.svg
:alt: Join the chat at https://gitter.im/Netflix/lemur
:target: https://gitter.im/Netflix/lemur?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge

49
bower.json Normal file
View File

@ -0,0 +1,49 @@
{
"name": "lemur",
"repository": {
"type": "git",
"url": "git://github.com/netflix/lemur.git"
},
"private": true,
"dependencies": {
"angular": "1.3",
"json3": "~3.3",
"es5-shim": "~4.0",
"jquery": "~2.1",
"angular-resource": "1.2.15",
"angular-cookies": "1.2.15",
"angular-sanitize": "1.2.15",
"angular-route": "1.2.15",
"angular-strap": "~2.0.2",
"restangular": "~1.4.0",
"ng-table": "~0.5.4",
"ngAnimate": "*",
"moment": "~2.6.0",
"angular-animate": "~1.4.0",
"angular-loading-bar": "~0.6.0",
"fontawesome": "~4.2.0",
"angular-wizard": "~0.4.0",
"bootswatch": "3.3.1+2",
"angular-spinkit": "~0.3.3",
"angular-bootstrap": "~0.12.0",
"angular-ui-switch": "~0.1.0",
"angular-chart.js": "~0.7.1",
"satellizer": "~0.9.4",
"angularjs-toaster": "~0.4.14"
},
"devDependencies": {
"angular-mocks": "~1.3",
"angular-scenario": "~1.3"
},
"resolutions": {
"bootstrap": "~3.3.1",
"angular": "1.3"
},
"ignore": [
"**/.*",
"node_modules",
"bower_components",
"test",
"tests"
]
}

2
config-default.py Normal file
View File

@ -0,0 +1,2 @@
import os
_basedir = os.path.abspath(os.path.dirname(__file__))

177
docs/Makefile Normal file
View File

@ -0,0 +1,177 @@
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " texinfo to make Texinfo files"
@echo " info to make Texinfo files and run them through makeinfo"
@echo " gettext to make PO message catalogs"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " xml to make Docutils-native XML files"
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
clean:
rm -rf $(BUILDDIR)/*
html:
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
singlehtml:
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
@echo
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/lemur.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/lemur.qhc"
devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/lemur"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/lemur"
@echo "# devhelp"
epub:
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
@echo
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."
latexpdf:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
latexpdfja:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through platex and dvipdfmx..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text files are in $(BUILDDIR)/text."
man:
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
@echo
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
texinfo:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
@echo "Run \`make' in that directory to run these through makeinfo" \
"(use \`make info' here to do that automatically)."
info:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo "Running Texinfo files through makeinfo..."
make -C $(BUILDDIR)/texinfo info
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
gettext:
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
@echo
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."
xml:
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
@echo
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
pseudoxml:
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
@echo
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."

540
docs/administration/index.rst Normal file
View File

@ -0,0 +1,540 @@
Configuration
=============
.. warning::
There are many secrets that Lemur uses that must be protected. All of these options are set via the Lemur configruation
file. It is highly advised that you do not store your secrets in this file! Lemur provides functions
that allow you to encrypt files at rest and decrypt them when it's time for deployment. See :ref:`Credential Management <CredentialManagement>`
for more information.
Basic Configuration
-------------------
.. data:: LOG_LEVEL
:noindex:
::
LOG_LEVEL = "DEBUG"
.. data:: LOG_FILE
:noindex:
::
LOG_FILE = "/logs/lemur/lemur-test.log"
.. data:: debug
:noindex:
Sets the flask debug flag to true (if supported by the webserver)
::
debug = False
.. warning::
This should never be used in a production environment as it exposes Lemur to
remote code execution through the debug console.
.. data:: CORS
:noindex:
Allows for cross domain requests, this is most commonly used for development but could
be use in production if you decided to host the webUI on a different domain than the server.
Use this cautiously, if you're not sure. Set it to `False`
::
CORS = False
.. data:: SQLACHEMY_DATABASE_URI
:noindex:
If you have ever used sqlalchemy before this is the standard connection string used. Lemur uses a postgres database and the connection string would look something like:
::
SQLALCHEMY_DATABASE_URI = 'postgresql://<user>:<password>@<hostname>:5432/lemur'
.. data:: LEMUR_MAIL
:noindex:
Lemur mail service
::
LEMUR_MAIL = 'lemur.example.com'
.. data:: LEMUR_SECURITY_TEAM_EMAIL
:noindex:
This is an email or list of emails that should be notified when a certificate is expiring. It is also the contact email address for any discovered certificate.
::
LEMUR_SECURITY_TEAM_EMAIL = ['security@example.com']
.. data:: LEMUR_RESTRICTED_DOMAINS
:noindex:
This allows the administrator to mark a subset of domains or domains matching a particular regex as
*restricted*. This means that only an administrator is allows to issue the domains in question.
.. data:: LEMUR_TOKEN_SECRET
:noindex:
The TOKEN_SECRET is the secret used to create JWT tokens that are given out to users. This should be securely generated and be kept private.
See `SECRET_KEY` for methods on secure secret generation.
::
LEMUR_TOKEN_SECRET = 'supersecret'
An example of how you might generate a random string:
>>> import random
>>> secret_key = ''.join(random.choice(string.ascii_uppercase) for x in range(6))
>>> secret_key = secret_key + ''.join(random.choice("~!@#$%^&*()_+") for x in range(6))
>>> secret_key = secret_key + ''.join(random.choice(string.ascii_lowercase) for x in range(6))
>>> secret_key = secret_key + ''.join(random.choice(string.digits) for x in range(6))
.. data:: LEMUR_ENCRYPTION_KEY
:noindex:
The LEMUR_ENCRYPTION_KEY is used to encrypt data at rest within Lemur's database. Without this key Lemur will refuse
to start.
See `LEMUR_TOKEN_SECRET` for methods of secure secret generation.
::
LEMUR_ENCRYPTION_KEY = 'supersupersecret'
Authority Options
-----------------
Authorities will each have their own configuration options. There are currently two plugins bundled with Lemur,
Verisign/Symantec and CloudCA
.. data:: VERISIGN_URL
:noindex:
This is the url for the verisign API
.. data:: VERISIGN_PEM_PATH
:noindex:
This is the path to the mutual SSL certificate used for communicating with Verisign
.. data:: CLOUDCA_URL
:noindex:
This is the URL for CLoudCA API
.. data:: CLOUDCA_PEM_PATH
:noindex:
This is the path to the mutual SSL Certificate use for communicating with CLOUDCA
.. data:: CLOUDCA_BUNDLE
:noindex:
This is the path to the CLOUDCA certificate bundle
Authentication
--------------
Lemur currently supports Basic Authentication and Ping OAuth2, additional flows can be added relatively easily
If you are not using PING you do not need to configure any of these options
.. data:: PING_SECRET
:noindex:
::
PING_SECRET = 'somethingsecret'
.. data:: PING_ACCESS_TOKEN_URL
:noindex:
::
PING_ACCESS_TOKEN_URL = "https://<yourpingserver>/as/token.oauth2"
.. data:: PING_USER_API_URL
:noindex:
::
PING_USER_API_URL = "https://<yourpingserver>/idp/userinfo.openid"
.. data:: PING_JWKS_URL
:noindex:
::
PING_JWKS_URL = "https://<yourpingserver>/pf/JWKS"
Notifications
=============
Lemur currently has very basic support for notifications. Notifications are send to the certificate creator, owner and
security team as specified by the `SECURITY_TEAM_EMAIL` configuration parameter.
The template for all of these notifications lives under lemur/template/event.html and can be easily modified to fit your
needs.
Certificates marked as in-active will **not** be notified of upcoming expiration. This enables a user to essentially
silence the expiration. If a certificate is active and is expiring the above will be notified at 30, 15, 5, 2 days
respectively. Lemur will not attempt to notify about certificate that have already expired.
AWS Configuration
=================
In order for Lemur to manage it's own account and other accounts we must ensure it has the correct AWS permissions.
.. note:: AWS usage is completely optional. Lemur can upload, find and manage SSL certificates in AWS. But is not required to do so.
AWS Configuration Options
-------------------------
.. data:: AWS_ACCOUNT_MAPPINGS
:noindex:
Lemur maintains it's own internal table of AWS accounts with their alias and account numbers, this variable is used during setup to bootstrap
your particular enviroment.
Defaults to ``{}``.
::
AWS_ACCOUNT_MAPPINGS = {
'awsaccountalias': 111111111111
}
Setting up IAM roles
--------------------
Lemur uses boto heavily to talk to all the AWS resources it manages. By default it uses the on-instance credentials to make the necessary calls.
In order to limit the permissions we will create a new two IAM roles for Lemur. You can name them whatever you would like but for example sake we will be calling them LemurInstanceProfile and Lemur.
Lemur uses to STS to talk to different accounts. For managing one account this isn't necessary but we will still use it so that we can easily add new accounts.
LemurInstanceProfile is the IAM role you will launch your instance with. It actually has almost no rights. In fact it should really only be able to use STS to assume role to the Lemur role.
Here is are example polices for the LemurInstanceProfile:
SES-SendEmail
.. code-block:: python
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ses:SendEmail"
],
"Resource": "*"
}
]
}
STS-AssumeRole
.. code-block:: python
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action":
"sts:AssumeRole",
"Resource": "*"
}
]
}
Next we will create the the Lemur IAM role. Lemur
Here is an example policy for Lemur:
IAM-ServerCertificate
.. code-block:: python
{
"Statement": [
{
"Action": [
"iam:ListServerCertificates",
"iam:UpdateServerCertificate",
"iam:GetServerCertificate",
"iam:UploadServerCertificate"
],
"Resource": [
"*"
],
"Effect": "Allow",
"Sid": "Stmt1404836868000"
}
]
}
.. code-block:: python
{
"Statement": [
{
"Action": [
"elasticloadbalancing:DescribeInstanceHealth",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"elasticloadbalancing:DescribeLoadBalancerPolicyTypes",
"elasticloadbalancing:DescribeLoadBalancerPolicies",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DeleteLoadBalancerListeners",
"elasticloadbalancing:CreateLoadBalancerListeners"
],
"Resource": [
"*"
],
"Effect": "Allow",
"Sid": "Stmt1404841912000"
}
]
}
Setting up STS access
---------------------
Once we have setup our accounts we need to ensure that we create a trust relationship so that LemurInstanceProfile can assume the Lemur role.
In the AWS console select the Lemur IAM role and select the Trust Relationships tab and click Edit Trust Relationship
Below is an example policy:
.. code-block:: python
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::<awsaccountnumber>:role/LemurInstanceProfile",
]
},
"Action": "sts:AssumeRole"
}
]
}
Adding N+1 accounts
-------------------
To add another account we go to the new account and create a new Lemur IAM role with the same policy as above.
Then we would go to the account that Lemur is running is and edit the trust relationship policy.
An example policy:
.. code-block:: python
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::<awsaccountnumber>:role/LemurInstanceProfile",
"arn:aws:iam::<awsaccountnumber1>:role/LemurInstanceProfile",
]
},
"Action": "sts:AssumeRole"
}
]
}
Setting up SES
--------------
Lemur has built in support for sending it's certificate notifications via Amazon's simple email service (SES). To force
Lemur to use SES ensure you are the running as the IAM role defined above and that you have followed the steps outlined
in Amazon's documentation `Setting up Amazon SES <http://docs.aws.amazon.com/ses/latest/DeveloperGuide/setting-up-ses.html>`_
The configuration::
LEMUR_MAIL = 'lemur.example.com'
Will be sender of all notifications, so ensure that it is verified with AWS.
SES if the default notification gateway and will be used unless SMTP settings are configured in the application configuration
settings.
Upgrading Lemur
===============
Lemur provides an easy way to upgrade between versions. Simply download the newest
version of Lemur from pypi and then apply any schema cahnges with the following command.
.. code-block:: bash
$ lemur db upgrade
.. note:: Internally, this uses `Alembic <https://alembic.readthedocs.org/en/latest/>`_ to manage database migrations.
.. _CommandLineInterface:
Command Line Interface
======================
Lemur installs a command line script under the name ``lemur``. This will allow you to
perform most required operations that are unachievable within the web UI.
If you're using a non-standard configuration location, you'll need to prefix every command with
--config (excluding create_config, which is a special case). For example::
lemur --config=/etc/lemur.conf.py help
For a list of commands, you can also use ``lemur help``, or ``lemur [command] --help``
for help on a specific command.
.. note:: The script is powered by a library called `Flask-Script <https://github.com/smurfix/flask-script>`_
Builtin Commands
----------------
All commands default to `~/.lemur/lemur.conf.py` if a configuration is not specified.
.. data:: create_config
Creates a default configuration file for Lemur.
Path defaults to ``~/.lemur/lemur.config.py``
::
lemur create_config .
.. note::
This command is a special case and does not depend on the configuration file
being set.
.. data:: init
Initializes the configuration file for Lemur.
::
lemur -c /etc/lemur.conf.py init
.. data:: start
Starts a Lemur service. You can also pass any flag that Gunicorn uses to specify the webserver configuration.
::
lemur start -w 6 -b 127.0.0.1:8080
.. data:: db upgrade
Performs any needed database migrations.
::
lemur db upgrade
.. data:: create_user
Creates new users within Lemur.
::
lemur create_user -u jim -e jim@example.com
.. data:: create_role
Creates new roles within Lemur.
::
lemur create_role -n example -d "a new role"
.. data:: check_revoked
Traverses every certificate that Lemur is aware of and attempts to understand it's validity.
It utilizes both OCSP and CRL. If Lemur is unable to come to a conclusion about a certificates
validity it's status is marked 'unknown'
.. data:: sync
Sync attempts to discover certificates in the environment that were not created by Lemur. There
::
lemur sync --all
Identity and Access Management
==============================
Lemur uses a Role Based Access Control (RBAC) mechanism to control which users have access to which resources. When a
user is first created in Lemur the can be assigned one or more roles. These roles are typically dynamically created
depending on a external identity provider (Google, LDAP, etc.,) or are hardcoded within Lemur and associated with special
meaning.
Within Lemur there are three main permissions: AdminPermission, CreatorPermission, OwnerPermission. Sub-permissions such
as ViewPrivateKeyPermission are compositions of these three main Permissions.
Lets take a look at how these permissions used:
Each `Authority` has a set of roles associated with it. If a user is also associated with the same roles
that the `Authority` is associated with it Lemur allows that user to user/view/update that `Authority`.
This RBAC is also used when determining which users can access which certificate private key. Lemur's current permission
structure is setup such that if the user is a `Creator` or `Owner` of a given certificate they are allow to view that
private key.
These permissions are applied to the user upon login and refreshed on every request.
.. seealso::
`Flask-Principal <https://pythonhosted.org/Flask-Principal>`_

2
docs/changelog.rst Normal file
View File

@ -0,0 +1,2 @@
Change Log
==========

262
docs/conf.py Normal file
View File

@ -0,0 +1,262 @@
# -*- coding: utf-8 -*-
#
# security_monkey documentation build configuration file, created by
# sphinx-quickstart on Sat Jun 7 18:43:48 2014.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
import sys
import os
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
sys.path.insert(0, os.path.abspath('..'))
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.autodoc',
'sphinxcontrib.autohttp.flask',
'sphinx.ext.todo',
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'lemur'
copyright = u'2015, Kevin Glisson'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '0.1'
# The full version, including alpha/beta/rc tags.
release = '0.1.1'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['_build']
# The reST default role (used for this markup: `text`) to use for all
# documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# If true, keep warnings as "system message" paragraphs in the built documents.
#keep_warnings = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'alabaster'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
#html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
#html_extra_path = []
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_domain_indices = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'lemurdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
('index', 'lemur.tex', u'Lemur Documentation',
u'Kevin Glisson', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'Lemur', u'Lemur Documentation',
[u'Kevin Glisson'], 1)
]
# If true, show URL addresses after external links.
#man_show_urls = False
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
('index', 'Lemur', u'Lemur Documentation',
u'Kevin Glisson', 'Lemur', 'SSL Certificate Management',
'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'
# If true, do not generate a @detailmenu in the "Top" node's menu.
#texinfo_no_detailmenu = False

197
docs/developer/index.rst Normal file
View File

@ -0,0 +1,197 @@
Contributing
============
Want to contribute back to Lemur? This page describes the general development flow,
our philosophy, the test suite, and issue tracking.
Documentation
-------------
If you're looking to help document Lemur, you can get set up with Sphinx, our documentation tool,
but first you will want to make sure you have a few things on your local system:
* python-dev (if you're on OS X, you already have this)
* pip
* virtualenvwrapper
Once you've got all that, the rest is simple:
::
# If you have a fork, you'll want to clone it instead
git clone git://github.com/netflix/lemur.git
# Create a python virtualenv
mkvirtualenv lemur
# Make the magic happen
make dev-docs
Running ``make dev-docs`` will install the basic requirements to get Sphinx running.
Building Documentation
~~~~~~~~~~~~~~~~~~~~~~
Inside the ``docs`` directory, you can run ``make`` to build the documentation.
See ``make help`` for available options and the `Sphinx Documentation <http://sphinx-doc.org/contents.html>`_ for more information.
Developing Against HEAD
-----------------------
We try to make it easy to get up and running in a development environment using a git checkout
of Lemur. You'll want to make sure you have a few things on your local system first:
* python-dev (if you're on OS X, you already have this)
* pip
* virtualenv (ideally virtualenvwrapper)
* node.js (for npm and building css/javascript)
* (Optional) Potgresql
Once you've got all that, the rest is simple:
::
# If you have a fork, you'll want to clone it instead
git clone git://github.com/lemur/lemur.git
# Create a python virtualenv
mkvirtualenv lemur
# Make the magic happen
make
Running ``make`` will do several things, including:
* Setting up any submodules (including Bootstrap)
* Installing Python requirements
* Installing NPM requirements
.. note::
You will want to store your virtualenv out of the ``lemur`` directory you cloned above,
otherwise ``make`` will fail.
Create a default Lemur configuration just as if this were a production instance:
::
lemur init
You'll likely want to make some changes to the default configuration (we recommend developing against Postgres, for example). Once done, migrate your database using the following command:
::
lemur upgrade
.. note:: The ``upgrade`` shortcut is simply a shorcut to Alembic's upgrade command.
Coding Standards
----------------
Lemur follows the guidelines laid out in `pep8 <http://www.python.org/dev/peps/pep-0008/>`_ with a little bit
of flexibility on things like line length. We always give way for the `Zen of Python <http://www.python.org/dev/peps/pep-0020/>`_. We also use strict mode for JavaScript, enforced by jshint.
You can run all linters with ``make lint``, or respectively ``lint-python`` or ``lint-js``.
Spacing
~~~~~~~
Python:
4 Spaces
JavaScript:
2 Spaces
CSS:
2 Spaces
HTML:
2 Spaces
Running the Test Suite
----------------------
The test suite consists of multiple parts, testing both the Python and JavaScript components in Lemur. If you've setup your environment correctly, you can run the entire suite with the following command:
::
make test
If you only need to run the Python tests, you can do so with ``make test-python``, as well as ``test-js`` for the JavaScript tests.
You'll notice that the test suite is structured based on where the code lives, and strongly encourages using the mock library to drive more accurate individual tests.
.. note:: We use py.test for the Python test suite, and a combination of phantomjs and jasmine for the JavaScript tests.
Static Media
------------
Lemur uses a library that compiles it's static media assets (LESS and JS files) automatically. If you're developing using
runserver you'll see changes happen not only in the original files, but also the minified or processed versions of the file.
If you've made changes and need to compile them by hand for any reason, you can do so by running:
::
lemur compilestatic
The minified and processed files should be committed alongside the unprocessed changes.
Developing with Flask
----------------------
Because Lemur is just Flask, you can use all of the standard Flask functionality. The only difference is you'll be accessing commands that would normally go through manage.py using the ``lemur`` CLI helper instead.
For example, you probably don't want to use ``lemur start`` for development, as it doesn't support anything like
automatic reloading on code changes. For that you'd want to use the standard builtin ``runserver`` command:
::
lemur runserver
DDL (Schema Changes)
--------------------
Schema changes should always introduce the new schema in a commit, and then introduce code relying on that schema in a followup commit. This also means that new columns must be NULLable.
Removing columns and tables requires a slightly more painful flow, and should resemble the follow multi-commit flow:
- Remove all references to the column or table (but dont remove the Model itself)
- Remove the model code
- Remove the table or column
Contributing Back Code
----------------------
All patches should be sent as a pull request on GitHub, include tests, and documentation where needed. If you're fixing a bug or making a large change the patch **must** include test coverage.
Uncertain about how to write tests? Take a look at some existing tests that are similar to the code you're changing, and go from there.
You can see a list of open pull requests (pending changes) by visiting https://github.com/netflix/lemur/pulls
Plugins
=======
.. toctree::
:maxdepth: 1
plugins/index
Internals
=========
.. toctree::
:maxdepth: 1
internals/lemur

20
docs/developer/internals/lemur.accounts.rst Normal file
View File

@ -0,0 +1,20 @@
accounts Package
================
:mod:`models` Module
--------------------
.. automodule:: lemur.accounts.models
:members:
:undoc-members:
:show-inheritance:
:mod:`service` Module
---------------------
.. automodule:: lemur.accounts.service
:members:
:undoc-members:
:show-inheritance:

11
docs/developer/internals/lemur.analyze.rst Normal file
View File

@ -0,0 +1,11 @@
analyze Package
===============
:mod:`service` Module
---------------------
.. automodule:: lemur.analyze.service
:members:
:undoc-members:
:show-inheritance:

20
docs/developer/internals/lemur.auth.rst Normal file
View File

@ -0,0 +1,20 @@
auth Package
============
:mod:`permissions` Module
-------------------------
.. automodule:: lemur.auth.permissions
:members:
:undoc-members:
:show-inheritance:
:mod:`service` Module
---------------------
.. automodule:: lemur.auth.service
:members:
:undoc-members:
:show-inheritance:

20
docs/developer/internals/lemur.authorities.rst Normal file
View File

@ -0,0 +1,20 @@
authorities Package
===================
:mod:`models` Module
--------------------
.. automodule:: lemur.authorities.models
:members:
:undoc-members:
:show-inheritance:
:mod:`service` Module
---------------------
.. automodule:: lemur.authorities.service
:members:
:undoc-members:
:show-inheritance:

43
docs/developer/internals/lemur.certificates.rst Normal file
View File

@ -0,0 +1,43 @@
certificates Package
====================
:mod:`exceptions` Module
------------------------
.. automodule:: lemur.certificates.exceptions
:members:
:undoc-members:
:show-inheritance:
:mod:`models` Module
--------------------
.. automodule:: lemur.certificates.models
:members:
:undoc-members:
:show-inheritance:
:mod:`service` Module
---------------------
.. automodule:: lemur.certificates.service
:members:
:undoc-members:
:show-inheritance:
:mod:`sync` Module
------------------
.. automodule:: lemur.certificates.sync
:members:
:undoc-members:
:show-inheritance:
:mod:`verify` Module
--------------------
.. automodule:: lemur.certificates.verify
:members:
:undoc-members:
:show-inheritance:

34
docs/developer/internals/lemur.common.rst Normal file
View File

@ -0,0 +1,34 @@
common Package
==============
:mod:`crypto` Module
--------------------
.. automodule:: lemur.common.crypto
:members:
:undoc-members:
:show-inheritance:
:mod:`health` Module
--------------------
.. automodule:: lemur.common.health
:members:
:undoc-members:
:show-inheritance:
:mod:`utils` Module
-------------------
.. automodule:: lemur.common.utils
:members:
:undoc-members:
:show-inheritance:
Subpackages
-----------
.. toctree::
lemur.common.services

35
docs/developer/internals/lemur.common.services.aws.rst Normal file
View File

@ -0,0 +1,35 @@
aws Package
===========
:mod:`elb` Module
-----------------
.. automodule:: lemur.common.services.aws.elb
:members:
:undoc-members:
:show-inheritance:
:mod:`iam` Module
-----------------
.. automodule:: lemur.common.services.aws.iam
:members:
:undoc-members:
:show-inheritance:
:mod:`ses` Module
-----------------
.. automodule:: lemur.common.services.aws.ses
:members:
:undoc-members:
:show-inheritance:
:mod:`sts` Module
-----------------
.. automodule:: lemur.common.services.aws.sts
:members:
:undoc-members:
:show-inheritance:

19
docs/developer/internals/lemur.common.services.issuers.plugins.cloudca.rst Normal file
View File

@ -0,0 +1,19 @@
cloudca Package
===============
:mod:`cloudca` Module
---------------------
.. automodule:: lemur.common.services.issuers.plugins.cloudca.cloudca
:members:
:undoc-members:
:show-inheritance:
:mod:`constants` Module
-----------------------
.. automodule:: lemur.common.services.issuers.plugins.cloudca.constants
:members:
:undoc-members:
:show-inheritance:

11
docs/developer/internals/lemur.common.services.issuers.plugins.rst Normal file
View File

@ -0,0 +1,11 @@
plugins Package
===============
Subpackages
-----------
.. toctree::
lemur.common.services.issuers.plugins.cloudca
lemur.common.services.issuers.plugins.verisign

19
docs/developer/internals/lemur.common.services.issuers.plugins.verisign.rst Normal file
View File

@ -0,0 +1,19 @@
verisign Package
================
:mod:`constants` Module
-----------------------
.. automodule:: lemur.common.services.issuers.plugins.verisign.constants
:members:
:undoc-members:
:show-inheritance:
:mod:`verisign` Module
----------------------
.. automodule:: lemur.common.services.issuers.plugins.verisign.verisign
:members:
:undoc-members:
:show-inheritance:

26
docs/developer/internals/lemur.common.services.issuers.rst Normal file
View File

@ -0,0 +1,26 @@
issuers Package
===============
:mod:`issuer` Module
--------------------
.. automodule:: lemur.common.services.issuers.issuer
:members:
:undoc-members:
:show-inheritance:
:mod:`manager` Module
---------------------
.. automodule:: lemur.common.services.issuers.manager
:members:
:undoc-members:
:show-inheritance:
Subpackages
-----------
.. toctree::
lemur.common.services.issuers.plugins

11
docs/developer/internals/lemur.common.services.rst Normal file
View File

@ -0,0 +1,11 @@
services Package
================
Subpackages
-----------
.. toctree::
lemur.common.services.aws
lemur.common.services.issuers

19
docs/developer/internals/lemur.domains.rst Normal file
View File

@ -0,0 +1,19 @@
domains Package
===============
:mod:`models` Module
--------------------
.. automodule:: lemur.domains.models
:members:
:undoc-members:
:show-inheritance:
:mod:`service` Module
---------------------
.. automodule:: lemur.domains.service
:members:
:undoc-members:
:show-inheritance:

35
docs/developer/internals/lemur.elbs.rst Normal file
View File

@ -0,0 +1,35 @@
elbs Package
============
:mod:`models` Module
--------------------
.. automodule:: lemur.elbs.models
:members:
:undoc-members:
:show-inheritance:
:mod:`service` Module
---------------------
.. automodule:: lemur.elbs.service
:members:
:undoc-members:
:show-inheritance:
:mod:`sync` Module
------------------
.. automodule:: lemur.elbs.sync
:members:
:undoc-members:
:show-inheritance:
:mod:`views` Module
-------------------
.. automodule:: lemur.elbs.views
:members:
:undoc-members:
:show-inheritance:

27
docs/developer/internals/lemur.listeners.rst Normal file
View File

@ -0,0 +1,27 @@
listeners Package
=================
:mod:`models` Module
--------------------
.. automodule:: lemur.listeners.models
:members:
:undoc-members:
:show-inheritance:
:mod:`service` Module
---------------------
.. automodule:: lemur.listeners.service
:members:
:undoc-members:
:show-inheritance:
:mod:`views` Module
-------------------
.. automodule:: lemur.listeners.views
:members:
:undoc-members:
:show-inheritance:

20
docs/developer/internals/lemur.roles.rst Normal file
View File

@ -0,0 +1,20 @@
roles Package
=============
:mod:`models` Module
--------------------
.. automodule:: lemur.roles.models
:members:
:undoc-members:
:show-inheritance:
:mod:`service` Module
---------------------
.. automodule:: lemur.roles.service
:members:
:undoc-members:
:show-inheritance:

87
docs/developer/internals/lemur.rst Normal file
View File

@ -0,0 +1,87 @@
:mod:`constants` Module
-----------------------
.. automodule:: lemur.constants
:members:
:undoc-members:
:show-inheritance:
:mod:`database` Module
----------------------
.. automodule:: lemur.database
:members:
:undoc-members:
:show-inheritance:
:mod:`decorators` Module
------------------------
.. automodule:: lemur.decorators
:members:
:undoc-members:
:show-inheritance:
:mod:`exceptions` Module
------------------------
.. automodule:: lemur.exceptions
:members:
:undoc-members:
:show-inheritance:
:mod:`extensions` Module
------------------------
.. automodule:: lemur.extensions
:members:
:undoc-members:
:show-inheritance:
:mod:`factory` Module
---------------------
.. automodule:: lemur.factory
:members:
:undoc-members:
:show-inheritance:
:mod:`manage` Module
--------------------
.. automodule:: lemur.manage
:members:
:undoc-members:
:show-inheritance:
:mod:`models` Module
--------------------
.. automodule:: lemur.models
:members:
:undoc-members:
:show-inheritance:
:mod:`notifications` Module
---------------------------
.. automodule:: lemur.notifications
:members:
:undoc-members:
:show-inheritance:
Subpackages
-----------
.. toctree::
lemur.accounts
lemur.auth
lemur.authorities
lemur.certificates
lemur.common
lemur.domains
lemur.roles
lemur.status
lemur.users

11
docs/developer/internals/lemur.status.rst Normal file
View File

@ -0,0 +1,11 @@
status Package
==============
:mod:`views` Module
-------------------
.. automodule:: lemur.status.views
:members:
:undoc-members:
:show-inheritance:

19
docs/developer/internals/lemur.users.rst Normal file
View File

@ -0,0 +1,19 @@
users Package
=============
:mod:`models` Module
--------------------
.. automodule:: lemur.users.models
:members:
:undoc-members:
:show-inheritance:
:mod:`service` Module
---------------------
.. automodule:: lemur.users.service
:members:
:undoc-members:
:show-inheritance:

151
docs/developer/plugins/index.rst Normal file
View File

@ -0,0 +1,151 @@
Writing a Plugin
================
**The plugin interface is a work in progress.**
Several interfaces exist for extending Lemur:
* Issuers (lemur.issuers)
Structure
---------
A plugins layout generally looks like the following::
setup.py
lemur_pluginname/
lemur_pluginname/__init__.py
lemur_pluginname/plugin.py
The ``__init__.py`` file should contain no plugin logic, and at most, a VERSION = 'x.x.x' line. For example,
if you want to pull the version using pkg_resources (which is what we recommend), your file might contain::
try:
VERSION = __import__('pkg_resources') \
.get_distribution(__name__).version
except Exception, e:
VERSION = 'unknown'
Inside of ``plugin.py``, you'll declare your Plugin class::
import lemur_pluginname
from lemur.common.services.issuers.plugins import Issuer
class PluginName(Plugin):
title = 'Plugin Name'
slug = 'pluginname'
description = 'My awesome plugin!'
version = lemur_pluginname.VERSION
author = 'Your Name'
author_url = 'https://github.com/yourname/lemur_pluginname'
def widget(self, request, group, **kwargs):
return "<p>Absolutely useless widget</p>"
And you'll register it via ``entry_points`` in your ``setup.py``::
setup(
# ...
entry_points={
'lemur.plugins': [
'pluginname = lemur_pluginname.issuers:PluginName'
],
},
)
That's it! Users will be able to install your plugin via ``pip install <package name>`` and configure it
via the web interface based on the hooks you enabled.
Permissions
===========
As described in the plugin interface, Lemur provides a suite of permissions.
In most cases, a admin (that is, if User.is_admin is ``True``), will be granted implicit permissions
on everything.
This page attempts to describe those permissions, and the contextual objects along with them.
.. data:: add_project
Controls whether a user can create a new project.
::
>>> has_perm('add_project', user)
Testing
=======
Lemur provides a basic py.test-based testing framework for extensions.
In a simple project, you'll need to do a few things to get it working:
setup.py
--------
Augment your setup.py to ensure at least the following:
.. code-block:: python
setup(
# ...
install_requires=[
'lemur',
]
)
conftest.py
-----------
The ``conftest.py`` file is our main entry-point for py.test. We need to configure it to load the Lemur pytest configuration:
.. code-block:: python
from __future__ import absolute_import
pytest_plugins = [
'lemur.utils.pytest'
]
Test Cases
----------
You can now inherit from Lemur's core test classes. These are Django-based and ensure the database and other basic utilities are in a clean state:
.. code-block:: python
# test_myextension.py
from __future__ import absolute_import
from lemur.testutils import TestCase
class MyExtensionTest(TestCase):
def test_simple(self):
assert 1 != 2
Running Tests
-------------
Running tests follows the py.test standard. As long as your test files and methods are named appropriately (``test_filename.py`` and ``test_function()``) you can simply call out to py.test:
::
$ py.test -v
============================== test session starts ==============================
platform darwin -- Python 2.7.9 -- py-1.4.26 -- pytest-2.6.4/python2.7
plugins: django
collected 1 items
tests/test_myextension.py::MyExtensionTest::test_simple PASSED
=========================== 1 passed in 0.35 seconds ============================

60
docs/developer/rest.rst Normal file
View File

@ -0,0 +1,60 @@
Lemur's front end is entirely API driven. Any action that you can accomplish via the UI can also be accomplished by the
UI. The following is documents and provides examples on how to make requests to the Lemur API.
Authentication
--------------
.. automodule:: lemur.auth.views
:members:
:undoc-members:
:show-inheritance:
Accounts
--------
.. automodule:: lemur.accounts.views
:members:
:undoc-members:
:show-inheritance:
Users
-----
.. automodule:: lemur.users.views
:members:
:undoc-members:
:show-inheritance:
Roles
-----
.. automodule:: lemur.roles.views
:members:
:undoc-members:
:show-inheritance:
Certificates
------------
.. automodule:: lemur.certificates.views
:members:
:undoc-members:
:show-inheritance:
Authorities
-----------
.. automodule:: lemur.authorities.views
:members:
:undoc-members:
:show-inheritance:
Domains
-------
.. automodule:: lemur.domains.views
:members:
:undoc-members:
:show-inheritance:

30
docs/faq.rst Normal file
View File

@ -0,0 +1,30 @@
Frequently Asked Questions
==========================
Common Problems
---------------
In my startup logs I see *'Aborting... Lemur cannot locate db encryption key, is ENCRYPTION_KEY set?'*
You likely have not correctly configured **ENCRYPTION_KEY**. See
:doc:`administration/configuration` for more information.
How do I
--------
... script the Lemur installation to bootstrap things like roles and users?
Lemur is a simple Flask (Python) application that runs using a utility
runner. A script that creates a project and default user might look something
like this:
.. code-block:: python
# Bootstrap the Flask environment
from flask import current_app
from lemur.users.service import create as create_user
from lemur.roles.service import create as create_role
from lemur.accounts.service import create as create_account
role = create_role('aRole', 'this is a new role')
create_user('admin', 'password', 'lemur@nobody', True, [role]

10
docs/guide/index.rst Normal file
View File

@ -0,0 +1,10 @@
Creating Certificates
=====================
Creating Users
==============
Creating Roles
==============

67
docs/index.rst Normal file
View File

@ -0,0 +1,67 @@
Lemur
=====
Lemur is a SSL management service. It attempts to help track and create certificates. By removing common issues with
CSR creation it gives normal developers 'sane' SSL defaults and helps security teams push SSL usage throughout an organization.
Installation
------------
.. toctree::
:maxdepth: 2
quickstart/index
production/index
User Guide
----------
.. toctree::
:maxdepth: 2
guide/index
Administration
--------------
.. toctree::
:maxdepth: 2
administration/index
plugins/index
Developers
----------
.. toctree::
:maxdepth: 2
developer/index
REST API
--------
.. toctree::
:maxdepth: 2
developer/rest
FAQ
----
.. toctree::
:maxdepth: 1
faq
Reference
---------
.. toctree::
:maxdepth: 1
changelog
license/index

20
docs/license/index.rst Normal file
View File

@ -0,0 +1,20 @@
License
=======
Lemur is licensed under a three clause APACHE License.
The full license text can be found below (:ref:`lemur-license`).
Authors
-------
Lemur was originally written and is maintained by Kevin Glisson.
A list of additional contributors can be seen on `GitHub <https://github.com/netflix/lemur/contributors>`_.
.. _lemur-license:
Lemur License
-------------
.. include:: ../../LICENSE

20
docs/plugins/index.rst Normal file
View File

@ -0,0 +1,20 @@
Plugins
=======
There are several interfaces currently available to extend Lemur. These are a work in
progress and the API is not frozen.
Bundled Plugins
---------------
Lemur includes several plugins by default. Including extensive support for AWS, VeriSign/Symantec and CloudCA services.
3rd Party Extensions
--------------------
The following extensions are available and maintained by members of the Lemur community:
Have an extension that should be listed here? Submit a `pull request <https://github.com/netflix/lemur>`_ and we'll
get it added.
Want to create your own extension? See :doc:`../developer/plugins/index` to get started.

277
docs/production/index.rst Normal file
View File

@ -0,0 +1,277 @@
Production
**********
There are several steps needed to make Lemur production ready. Here we focus on making Lemur more reliable and secure.
Basics
======
Because of the sensitivity of the information stored and maintain by Lemur it is important that you follow standard host hardening practices:
- Run Lemur with a limited user
- Disabled any unneeded service
- Enable remote logging
.. _CredentialManagement:
Credential Management
---------------------
Lemur often contains credentials such as mutual SSL keys that are used to communicate with third party resources and for encrypting stored secrets. Lemur comes with the ability
to automatically encrypt these keys such that your keys not be in clear text.
The keys are located within lemur/keys and broken down by environment
To utilize this ability use the following commands:
``lemur lock``
and
``lemur unlock``
If you choose to use this feature ensure that the KEY are decrypted before Lemur starts as it will have trouble communicating with the database otherwise.
SSL
====
Nginx
-----
Nginx is a very popular choice to serve a Python project:
- It's fast.
- It's lightweight.
- Configuration files are simple.
Nginx doesn't run any Python process, it only serves requests from outside to
the Python server.
Therefor there are two steps:
- Run the Python process.
- Run Nginx.
You will benefit from having:
- the possibility to have several projects listening to the port 80;
- your web site processes won't run with admin rights, even if --user doesn't
work on your OS;
- the ability to manage a Python process without touching Nginx or the other
processes. It's very handy for updates.
You must create a Nginx configuration file for Lemur. On GNU/Linux, they usually
go into /etc/nginx/conf.d/. Name it lemur.conf.
The minimal configuration file to run the site is::
server {
listen 80;
server_name www.yourwebsite.com;
location / {
proxy_pass http://127.0.0.1:5000;
}
}
`proxy_pass` just passes the external request to the Python process.
The port much match the one used by the 0bin process of course.
You can make some adjustments to get a better user experience::
server_tokens off;
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
server {
listen 80;
return 301 https://$host$request_uri;
}
server {
listen 443;
access_log /var/log/nginx/log/lemur.access.log;
error_log /var/log/nginx/log/lemur.error.log;
location /api {
proxy_pass http://127.0.0.1:5000;
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location / {
root /apps/lemur/lemur/static/dist;
index index.html;
}
}
This makes Nginx serve the favicon and static files which is is much better at than python.
It is highly recommended that you deploy SSL when deploying Lemur. This may be obvious given Lemur's purpose but the
sensitive nature of Lemur and what it controls makes this essential. This is a sample config for Lemur that also terminates SSL::
server_tokens off;
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
server {
listen 80;
return 301 https://$host$request_uri;
}
server {
listen 443;
access_log /var/log/nginx/log/lemur.access.log;
error_log /var/log/nginx/log/lemur.error.log;
# certs sent to the client in SERVER HELLO are concatenated in ssl_certificate
ssl_certificate /path/to/signed_cert_plus_intermediates;
ssl_certificate_key /path/to/private_key;
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;
# Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
ssl_dhparam /path/to/dhparam.pem;
# modern configuration. tweak to your needs.
ssl_protocols TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK';
ssl_prefer_server_ciphers on;
# HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
add_header Strict-Transport-Security max-age=15768000;
# OCSP Stapling ---
# fetch OCSP records from URL in ssl_certificate and cache them
ssl_stapling on;
ssl_stapling_verify on;
## verify chain of trust of OCSP response using Root CA and Intermediate certs
ssl_trusted_certificate /path/to/root_CA_cert_plus_intermediates;
resolver <IP DNS resolver>;
location /api {
proxy_pass http://127.0.0.1:5000;
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location / {
root /apps/lemur/lemur/static/dist;
index index.html;
}
}
Apache
------
An example apache config::
<VirtualHost *:443>
...
SSLEngine on
SSLCertificateFile /path/to/signed_certificate
SSLCertificateChainFile /path/to/intermediate_certificate
SSLCertificateKeyFile /path/to/private/key
SSLCACertificateFile /path/to/all_ca_certs
# intermediate configuration, tweak to your needs
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
SSLHonorCipherOrder on
# HSTS (mod_headers is required) (15768000 seconds = 6 months)
Header always set Strict-Transport-Security "max-age=15768000"
...
</VirtualHost>
Also included in the configurations above are several best practices when it comes to deploying SSL. Things like enabling
HSTS, disabling vulnerable ciphers are all good ideas when it comes to deploying Lemur into a production environment.
.. seealso::
`Mozilla SSL Configuration Generator <https://mozilla.github.io/server-side-tls/ssl-config-generator/>`_
.. _UsingSupervisor:
Supervisor
==========
Supervisor is a very nice way to manage you Python processes. We won't cover
the setup (which is just apt-get install supervisor or pip install supervisor
most of the time), but here is a quick overview on how to use it.
Create a configuration file named supervisor.ini::
[unix_http_server]
file=/tmp/supervisor.sock;
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock;
[rpcinterface:supervisor]
supervisor.rpcinterface_factory=supervisor.rpcinterface:make_main_rpcinterface
[supervisord]
logfile=/tmp/lemur.log
logfile_maxbytes=50MB
logfile_backups=2
loglevel=trace
pidfile=/tmp/supervisord.pid
nodaemon=false
minfds=1024
minprocs=200
user=lemur
[program:lemur]
command=python /path/to/lemur/manage.py manage.py start
directory=/path/to/lemur/
environment=PYTHONPATH='/path/to/lemur/'
user=lemur
autostart=true
autorestart=true
The 4 first entries are just boiler plate to get you started, you can copy
them verbatim.
The last one define one (you can have many) process supervisor should manage.
It means it will run the command::
python manage.py start
In the directory, with the environment and the user you defined.
This command will be ran as a daemon, in the background.
`autostart` and `autorestart` just make it fire and forget: the site will always be
running, even it crashes temporarily or if you restart the machine.
The first time you run supervisor, pass it the configuration file::
supervisord -c /path/to/supervisor.ini
Then you can manage the process by running::
supervisorctl -c /path/to/supervisor.ini
It will start a shell from were you can start/stop/restart the service
You can read all errors that might occurs from /tmp/lemur.log.

237
docs/quickstart/index.rst Normal file
View File

@ -0,0 +1,237 @@
Quickstart
**********
This guide will step you through setting up a Python-based virtualenv, installing the required packages, and configuring the basic web service.
Dependencies
------------
Some basic prerequisites which you'll need in order to run Lemur:
* A UNIX-based operating system. We test on Ubuntu, develop on OS X
* Python 2.7
* PostgreSQL
* Ngnix
.. note:: Lemur was built with in AWS in mind. This means that things such as databases (RDS), mail (SES), and SSL (ELB),
are largely handled for us. Lemur does **not** require AWS to function. Our guides and documentation try to be
be as generic as possible and are not intended to document every step of launching Lemur into a given environment.
Setting up an Environment
-------------------------
The first thing you'll need is the Python ``virtualenv`` package. You probably already
have this, but if not, you can install it with::
pip install -U virtualenv
Once that's done, choose a location for the environment, and create it with the ``virtualenv``
command. For our guide, we're going to choose ``/www/lemur/``::
virtualenv /www/lemur/
Finally, activate your virtualenv::
source /www/lemur/bin/activate
.. note:: Activating the environment adjusts your PATH, so that things like pip now
install into the virtualenv by default.
Installing Lemur
----------------
Once you've got the environment setup, you can install Lemur and all its dependencies with
the same command you used to grab virtualenv::
pip install -U lemur
Once everything is installed, you should be able to execute the Lemur CLI, via ``lemur``, and get something
like the following:
.. code-block:: bash
$ lemur
usage: lemur [--config=/path/to/settings.py] [command] [options]
Installing from Source
~~~~~~~~~~~~~~~~~~~~~~
If you're installing the Lemur source (e.g. from git), you'll also need to install **npm**.
Once your system is prepared, symlink your source into the virtualenv:
.. code-block:: bash
$ python setup.py develop
.. Note:: This command will install npm dependencies as well as compile static assets.
Creating a configuration
------------------------
Before we run Lemur we must create a valid configuration file for it.
The Lemur cli comes with a simple command to get you up and running quickly.
Simply run:
.. code-block:: bash
$ lemur create_config
.. Note:: This command will create a default configuration under `~/.lemur/lemur.conf.py` you
can specify this location by passing the `config_path` parameter to the `create_config` command.
You can specify `-c` or `--config` to any Lemur command to specify the current environment
you are working in. Lemur will also look under the environmental variable `LEMUR_CONF` should
that be easier to setup in your environment.
Once created you will need to update the configuration file with information about your environment,
such as which database to talk to, where keys are stores etc..
Initializing Lemur
------------------
Lemur provides a helpful command that will initialize your database for you. It creates a default user (lemur) that is
used by Lemur to help associate certificates that do not currently have an owner. This is most commonly the case when
Lemur has discovered certificates from a third party resource. This is also a default user that can be used to
administer Lemur.
.. code-block:: bash
$ lemur init
.. note:: It is recommended that once the 'lemur' user is created that you create individual users for every day access.
There is currently no way for a user to self enroll for Lemur access, they must have an administrator create an account
for them or be enrolled automatically through SSO. This can be done through the CLI or UI.
See :ref:`Creating Users <CreatingUsers>` and :ref:`Command Line Interface <CommandLineInterface>` for details
.. note::
This assumes you have already created a postgres database and have specified the right postgres URI in the
lemur configuration. See the `Postgres Documentation <http://www.postgresql.org/docs/9.0/static/tutorial-createdb.html>`_
for details.
Starting the Web Service
------------------------
Lemur provides a built-in webserver (powered by gunicorn and eventlet) to get you off the ground quickly.
To start the webserver, you simply use ``lemur start``. If you opted to use an alternative configuration path
you can pass that via the --config option.
::
# Lemur's server runs on port 5000 by default. Make sure your client reflects
# the correct host and port!
lemur --config=/etc/lemur.conf.py start
You should now be able to test the web service by visiting `http://localhost:5000/`.
Setup a Reverse Proxy
---------------------
By default, Lemur runs on port 5000. Even if you change this, under normal conditions you won't be able to bind to
port 80. To get around this (and to avoid running Lemur as a privileged user, which you shouldn't), we recommend
you setup a simple web proxy.
Proxying with Nginx
~~~~~~~~~~~~~~~~~~~
You'll use the builtin HttpProxyModule within Nginx to handle proxying::
location / {
proxy_pass http://localhost:5000;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
See :doc:`../production/index` for more details on using Nginx.
Proxying with Apache
~~~~~~~~~~~~~~~~~~~~
Apache requires the use of mod_proxy for forwarding requests::
ProxyPass / http://localhost:5000/
ProxyPassReverse / http://localhost:5000/
ProxyPreserveHost On
RequestHeader set X-Forwarded-Proto "https" env=HTTPS
You will need to enable ``headers``, ``proxy``, and ``proxy_http`` apache modules to use these settings.
See :doc:`../production/index` for more details on using Apache.
Running Lemur as a Service
---------------------------
We recommend using whatever software you are most familiar with for managing Lemur processes. One option is
`Supervisor <http://supervisord.org/>`_.
Configure ``supervisord``
~~~~~~~~~~~~~~~~~~~~~~~~~
Configuring Supervisor couldn't be more simple. Just point it to the ``lemur`` executable in your virtualenv's bin/
folder and you're good to go.
::
[program:lemur-web]
directory=/www/lemur/
command=/www/lemur/bin/lemur start
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile syslog
stderr_logfile syslog
See :ref:`Using Supervisor <UsingSupervisor>` for more details on using Supervisor.
Syncing
-------
Lemur uses periodic sync tasks to make sure it is up-to-date with it's environment. As always things can change outside
of Lemur, but we do our best to reconcile those changes.
.. code-block:: bash
$ crontab -e
* 3 * * * lemur sync
* 3 * * * lemur check_revoked
Additional Utilities
--------------------
If you're familiar with Python you'll quickly find yourself at home, and even more so if you've used Flask. The
``lemur`` command is just a simple wrapper around Flask's ``manage.py``, which means you get all of the
power and flexibility that goes with it.
Some of those which you'll likely find useful are:
lock
~~~~
Encrypts sensitive key material - This is most useful for storing encrypted secrets in source code.
unlock
~~~~~~
Decrypts sensitive key material - Used to decrypt the secrets stored in source during deployment.
What's Next?
------------
The above gets you going, but for production there are several different security considerations to take into account,
remember Lemur is handling sensitive data and security is imperative.
See :doc:`../production/index` for more details on how to configure Lemur for production.

231
gulp/build.js Normal file
View File

@ -0,0 +1,231 @@
var gulp = require('gulp'),
minifycss = require('gulp-minify-css'),
concat = require('gulp-concat'),
less = require('gulp-less'),
gulpif = require('gulp-if'),
order = require('gulp-order'),
gutil = require('gulp-util'),
rename = require('gulp-rename'),
foreach = require('gulp-foreach'),
debug = require('gulp-debug'),
path =require('path'),
merge = require('merge-stream'),
del = require('del'),
size = require('gulp-size'),
plumber = require('gulp-plumber'),
autoprefixer = require('gulp-autoprefixer'),
jshint = require('gulp-jshint'),
inject = require('gulp-inject'),
cache = require('gulp-cache'),
ngAnnotate = require('gulp-ng-annotate'),
csso = require('gulp-csso'),
useref = require('gulp-useref'),
filter = require('gulp-filter'),
rev = require('gulp-rev'),
revReplace = require('gulp-rev-replace'),
imagemin = require('gulp-imagemin'),
minifyHtml = require('gulp-minify-html'),
bowerFiles = require('main-bower-files'),
replace = require('gulp-replace-task');
gulp.task('default', ['clean'], function () {
gulp.start('fonts', 'styles');
});
gulp.task('clean', function (cb) {
del(['.tmp', 'lemur/static/dist'], cb);
});
gulp.task('dev:fonts', function () {
var fileList = [
'lemur/static/app/vendor/bower_components/bootstrap/dist/fonts/*',
'lemur/static/app/vendor/bower_components/fontawesome/fonts/*'
];
return gulp.src(fileList)
.pipe(gulp.dest('.tmp/fonts'));
});
gulp.task('dev:styles', function () {
var baseContent = '@import "lemur/static/app/vendor/bower_components/bootstrap/less/bootstrap.less";@import "lemur/static/app/vendor/bower_components/bootswatch/$theme$/variables.less";@import "lemur/static/app/vendor/bower_components/bootswatch/$theme$/bootswatch.less";@import "lemur/static/app/vendor/bower_components/bootstrap/less/utilities.less";';
var isBootswatchFile = function (file) {
var suffix = 'bootswatch.less';
return file.path.indexOf(suffix, file.path.length - suffix.length) !== -1;
};
var isBootstrapFile = function (file) {
var suffix = 'bootstrap-',
fileName = path.basename(file.path);
return fileName.indexOf(suffix) === 0;
};
var fileList = [
'lemur/static/app/styles/lemur.css',
'lemur/static/app/vendor/bower_components/bootswatch/sandstone/bootswatch.less',
'lemur/static/app/vendor/bower_components/fontawesome/css/font-awesome.css',
'lemur/static/app/vendor/bower_components/angular-spinkit/src/angular-spinkit.css',
'lemur/static/app/vendor/bower_components/angular-chart.js/dist/angular-chart.css',
'lemur/static/app/vendor/bower_components/angular-loading-bar/src/loading-bar.css',
'lemur/static/app/vendor/bower_components/angular-ui-switch/angular-ui-switch.css',
'lemur/static/app/vendor/bower_components/angular-wizard/dist/angular-wizard.css',
'lemur/static/app/vendor/bower_components/ng-table/ng-table.css',
'lemur/static/app/vendor/bower_components/angularjs-toaster/toaster.css'
];
return gulp.src(fileList)
.pipe(gulpif(isBootswatchFile, foreach(function (stream, file) {
var themeName = path.basename(path.dirname(file.path)),
content = replaceAll(baseContent, '$theme$', themeName),
file = string_src('bootstrap-' + themeName + '.less', content);
return file;
})))
.pipe(less())
.pipe(gulpif(isBootstrapFile, foreach(function (stream, file) {
var fileName = path.basename(file.path),
themeName = fileName.substring(fileName.indexOf('-') + 1, fileName.indexOf('.'));
// http://stackoverflow.com/questions/21719833/gulp-how-to-add-src-files-in-the-middle-of-a-pipe
// https://github.com/gulpjs/gulp/blob/master/docs/recipes/using-multiple-sources-in-one-task.md
return merge(stream, gulp.src(['.tmp/styles/font-awesome.css', '.tmp/styles/lemur.css']))
.pipe(concat('style-' + themeName + ".css"));
})))
.pipe(plumber())
.pipe(concat('styles.css'))
.pipe(minifycss())
.pipe(autoprefixer('last 1 version'))
.pipe(gulp.dest('.tmp/styles'))
.pipe(size());
});
// http://stackoverflow.com/questions/1144783/replacing-all-occurrences-of-a-string-in-javascript
function escapeRegExp(string) {
return string.replace(/([.*+?^=!:${}()|\[\]\/\\])/g, "\\$1");
}
function replaceAll(string, find, replace) {
return string.replace(new RegExp(escapeRegExp(find), 'g'), replace);
}
function string_src(filename, string) {
var src = require('stream').Readable({ objectMode: true });
src._read = function () {
this.push(new gutil.File({ cwd: "", base: "", path: filename, contents: new Buffer(string) }));
this.push(null);
};
return src;
}
gulp.task('dev:scripts', function () {
return gulp.src(['lemur/static/app/angular/**/*.js'])
.pipe(jshint())
.pipe(jshint.reporter('jshint-stylish'))
.pipe(size());
});
gulp.task('build:extras', function () {
return gulp.src(['lemur/static/app/*.*', '!lemur/static/app/*.html'])
.pipe(gulp.dest('lemur/static/dist'));
});
function injectHtml(isDev) {
return gulp.src('lemur/static/app/index.html')
.pipe(
inject(gulp.src(bowerFiles({ base: 'app' }), {
read: false
}), {
starttag: '<!-- inject:bower:{{ext}} -->',
addRootSlash: false,
ignorePath: isDev ? ['lemur/static/app/', '.tmp/'] : null
})
)
.pipe(inject(gulp.src(['lemur/static/app/angular/**/*.js'], {
read: false
}), {
read: false,
starttag: '<!-- inject:{{ext}} -->',
addRootSlash: false,
ignorePath: isDev ? ['lemur/static/app/', '.tmp/'] : null
}))
.pipe(inject(gulp.src(['.tmp/styles/**/*.css'], {
read: false
}), {
read: false,
starttag: '<!-- inject:{{ext}} -->',
addRootSlash: false,
ignorePath: isDev ? ['lemur/static/app/', '.tmp/'] : null
}))
.pipe(
gulpif(!isDev,
inject(gulp.src('lemur/static/dist/ngviews/ngviews.min.js'), {
read: false,
starttag: '<!-- inject:ngviews -->',
addRootSlash: false
})
)
)
.pipe(gulp.dest('.tmp/'));
}
gulp.task('dev:inject', ['dev:styles', 'dev:scripts'], function () {
return injectHtml(true);
});
gulp.task('build:inject', ['dev:styles', 'dev:scripts', 'build:ngviews'], function () {
return injectHtml(false);
});
gulp.task('build:ngviews', function () {
return gulp.src(['lemur/static/app/angular/**/*.html'])
.pipe(minifyHtml({
empty: true,
spare: true,
quotes: true
}))
.pipe(gulp.dest('lemur/static/dist/angular'))
.pipe(size());
});
gulp.task('build:html', ['dev:styles', 'dev:scripts', 'build:ngviews', 'build:inject'], function () {
var jsFilter = filter('**/*.js');
var cssFilter = filter('**/*.css');
var assets = useref.assets();
return gulp.src('.tmp/index.html')
.pipe(assets)
.pipe(rev())
.pipe(jsFilter)
.pipe(ngAnnotate())
.pipe(jsFilter.restore())
.pipe(cssFilter)
.pipe(csso())
.pipe(cssFilter.restore())
.pipe(assets.restore())
.pipe(useref())
.pipe(revReplace())
.pipe(gulp.dest('lemur/static/dist'))
.pipe(size());
});
gulp.task('build:fonts', ['dev:fonts'], function () {
return gulp.src('.tmp/fonts/**/*')
.pipe(gulp.dest('lemur/static/dist/fonts'));
});
gulp.task('build:images', function () {
return gulp.src('lemur/static/app/images/**/*')
.pipe(cache(imagemin({
optimizationLevel: 3,
progressive: true,
interlaced: true
})))
.pipe(gulp.dest('lemur/static/dist/images'))
.pipe(size());
});
gulp.task('build', ['build:ngviews', 'build:inject', 'build:images', 'build:fonts', 'build:html', 'build:extras']);

56
gulp/server.js Normal file
View File

@ -0,0 +1,56 @@
'use strict';
var gulp = require('gulp');
var browserSync = require('browser-sync');
var httpProxy = require('http-proxy');
/* This configuration allow you to configure browser sync to proxy your backend */
/*
var proxyTarget = 'http://localhost/context/'; // The location of your backend
var proxyApiPrefix = 'api'; // The element in the URL which differentiate between API request and static file request
var proxy = httpProxy.createProxyServer({
target: proxyTarget
});
function proxyMiddleware(req, res, next) {
if (req.url.indexOf(proxyApiPrefix) !== -1) {
proxy.web(req, res);
} else {
next();
}
}
*/
function browserSyncInit(baseDir, files, browser) {
browser = browser === undefined ? 'default' : browser;
browserSync.instance = browserSync.init(files, {
startPath: '/index.html',
server: {
baseDir: baseDir
},
browser: browser,
ghostMode: false
});
}
gulp.task('serve', ['watch'], function () {
browserSyncInit([
'.tmp',
'app'
], [
'.tmp/*.html',
'.tmp/styles/**/*.css',
'lemur/static/app/angular/**/*.js',
'lemur/static/app/partials/**/*.html',
'lemur/static/app/images/**/*',
'lemur/static/app/angular/**/*',
'lemur/static/app/index.html'
]);
});
gulp.task('serve:dist', ['build'], function () {
browserSyncInit('lemur/static/dist');
});

12
gulp/watch.js Normal file
View File

@ -0,0 +1,12 @@
'use strict';
var gulp = require('gulp');
gulp.task('watch', ['dev:styles', 'dev:scripts', 'dev:inject'] ,function () {
gulp.watch('app/styles/**/*.less', ['dev:styles']);
gulp.watch('app/styles/**/*.css', ['dev:styles']);
gulp.watch('app/**/*.js', ['dev:scripts']);
gulp.watch('app/images/**/*', ['build:images']);
gulp.watch('bower.json', ['dev:inject']);
});

31
gulpfile.js Normal file
View File

@ -0,0 +1,31 @@
/**
* Created by kglisson on 1/19/15.
*/
'use strict';
var gulp = require('gulp');
require('require-dir')('./gulp');
gulp.task('default', function () {
var c = {
reset: '\x1b[0m',
bold: '\x1b[1m',
green: '\x1b[32m',
magenta: '\x1b[35m'
};
console.log('');
console.log(c.green + c.bold + 'Main Commands' + c.reset);
console.log(c.green + '-------------------------------------------' + c.reset);
console.log(c.green + 'clean' + c.reset + ' - delete the .tmp/ and dist/ folders.');
console.log(c.green + 'build' + c.reset + ' - execute the release build and output into the dist/ folder.');
console.log(c.green + 'serve:dist' + c.reset + ' - execute the release build and output into the dist/ folder then run a local server for the files.');
console.log(c.green + 'serve' + c.reset + ' - run JShint and LESS compiler to produce .tmp/ folder. Then serve up the app on a local server.');
console.log('');
console.log(c.green + c.bold + 'All Commands' + c.reset);
console.log(c.green + '-------------------------------------------' + c.reset);
console.log(Object.keys(gulp.tasks).sort().join('\n'));
console.log('');
return;
});

54
hooks/pre-commit Normal file
View File

@ -0,0 +1,54 @@
#!/usr/bin/env python
import glob
import os
import sys
os.environ['PYFLAKES_NODOCTEST'] = '1'
# pep8.py uses sys.argv to find setup.cfg
sys.argv = [os.path.join(os.path.dirname(__file__), os.pardir, os.pardir)]
# git usurbs your bin path for hooks and will always run system python
if 'VIRTUAL_ENV' in os.environ:
site_packages = glob.glob(
'%s/lib/*/site-packages' % os.environ['VIRTUAL_ENV'])[0]
sys.path.insert(0, site_packages)
def py_lint(files_modified):
from flake8.main import DEFAULT_CONFIG
from flake8.engine import get_style_guide
# remove non-py files and files which no longer exist
files_modified = filter(lambda x: x.endswith('.py'), files_modified)
flake8_style = get_style_guide(parse_argv=True, config_file=DEFAULT_CONFIG)
report = flake8_style.check_files(files_modified)
return report.total_errors != 0
def js_lint(files_modified):
has_errors = False
if os.system('node_modules/.bin/jshint src/sentry'):
has_errors = True
return has_errors
def main():
from flake8.hooks import run
gitcmd = "git diff-index --cached --name-only HEAD"
_, files_modified, _ = run(gitcmd)
files_modified = filter(lambda x: os.path.exists(x), files_modified)
if any((py_lint(files_modified), js_lint(files_modified))):
return 1
return 0
if __name__ == '__main__':
sys.exit(main())

69
lemur/__init__.py Normal file
View File

@ -0,0 +1,69 @@
"""
.. module: lemur
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from flask import jsonify
from lemur import factory
from lemur.users.views import mod as users
from lemur.roles.views import mod as roles
from lemur.auth.views import mod as auth
from lemur.domains.views import mod as domains
from lemur.elbs.views import mod as elbs
from lemur.accounts.views import mod as accounts
from lemur.authorities.views import mod as authorities
from lemur.listeners.views import mod as listeners
from lemur.certificates.views import mod as certificates
from lemur.status.views import mod as status
LEMUR_BLUEPRINTS = (
users,
roles,
auth,
domains,
elbs,
accounts,
authorities,
listeners,
certificates,
status
)
def create_app(config=None):
app = factory.create_app(app_name=__name__, blueprints=LEMUR_BLUEPRINTS, config=config)
configure_hook(app)
return app
def configure_hook(app):
"""
:param app:
:return:
"""
from flask.ext.principal import PermissionDenied
from lemur.decorators import crossdomain
if app.config.get('CORS'):
@app.after_request
@crossdomain(origin="http://localhost:3000", methods=['PUT', 'HEAD', 'GET', 'POST', 'OPTIONS', 'DELETE'])
def after(response):
return response
@app.errorhandler(PermissionDenied)
def handle_invalid_usage(error):
response = {'message': 'You are not allow to access this resource'}
response.status_code = 403
return response

0
lemur/accounts/__init__.py Normal file
View File

29
lemur/accounts/models.py Normal file
View File

@ -0,0 +1,29 @@
"""
.. module: lemur.accounts.models
:platform: unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from sqlalchemy import Column, Integer, String, Text
from sqlalchemy.orm import relationship
from lemur.database import db
class Account(db.Model):
__tablename__ = 'accounts'
id = Column(Integer, primary_key=True)
account_number = Column(String(32), unique=True)
label = Column(String(32))
notes = Column(Text())
elbs = relationship("ELB", backref='account', cascade="all, delete, delete-orphan")
def as_dict(self):
return {c.name: getattr(self, c.name) for c in self.__table__.columns}
def serialize(self):
blob = self.as_dict()
blob['elbs'] = [x.id for x in self.elbs]
return blob

112
lemur/accounts/service.py Normal file
View File

@ -0,0 +1,112 @@
"""
.. module: lemur.accounts.views
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from lemur import database
from lemur.accounts.models import Account
from lemur.certificates.models import Certificate
def create(account_number, label=None, comments=None):
"""
Creates a new account, that can then be used as a destination for certificates.
:param account_number: AWS assigned ID
:param label: Account common name
:param comments:
:rtype : Account
:return: New account
"""
acct = Account(account_number=account_number, label=label, notes=comments)
return database.create(acct)
def update(account_id, account_number, label, comments=None):
"""
Updates an existing account.
:param account_id: Lemur assigned ID
:param account_number: AWS assigned ID
:param label: Account common name
:param comments:
:rtype : Account
:return:
"""
account = get(account_id)
account.account_number = account_number
account.label = label
account.notes = comments
return database.update(account)
def delete(account_id):
"""
Deletes an account.
:param account_id: Lemur assigned ID
"""
database.delete(get(account_id))
def get(account_id):
"""
Retrieves an account by it's lemur assigned ID.
:param account_id: Lemur assigned ID
:rtype : Account
:return:
"""
return database.get(Account, account_id)
def get_by_account_number(account_number):
"""
Retrieves an account by it's amazon assigned ID.
:rtype : Account
:param account_number: AWS assigned ID
:return:
"""
return database.get(Account, account_number, field='account_number')
def get_all():
"""
Retrieves all account currently known by Lemur.
:return:
"""
query = database.session_query(Account)
return database.find_all(query, Account, {}).all()
def render(args):
sort_by = args.pop('sort_by')
sort_dir = args.pop('sort_dir')
page = args.pop('page')
count = args.pop('count')
filt = args.pop('filter')
certificate_id = args.pop('certificate_id', None)
if certificate_id:
query = database.session_query(Account).join(Certificate, Account.certificate)
query = query.filter(Certificate.id == certificate_id)
else:
query = database.session_query(Account)
if filt:
terms = filt.split(';')
query = database.filter(query, Account, terms)
query = database.find_all(query, Account, args)
if sort_by and sort_dir:
query = database.sort(query, Account, sort_by, sort_dir)
return database.paginate(query, page, count)

300
lemur/accounts/views.py Normal file
View File

@ -0,0 +1,300 @@
"""
.. module: lemur.accounts.views
:platform: Unix
:synopsis: This module contains all of the accounts view code.
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from flask import Blueprint
from flask.ext.restful import Api, reqparse, fields
from lemur.accounts import service
from lemur.auth.service import AuthenticatedResource
from lemur.auth.permissions import admin_permission
from lemur.common.utils import paginated_parser, marshal_items
mod = Blueprint('accounts', __name__)
api = Api(mod)
FIELDS = {
'accountNumber': fields.Integer(attribute='account_number'),
'label': fields.String,
'comments': fields.String(attribute='notes'),
'id': fields.Integer,
}
class AccountsList(AuthenticatedResource):
""" Defines the 'accounts' endpoint """
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(AccountsList, self).__init__()
@marshal_items(FIELDS)
def get(self):
"""
.. http:get:: /accounts
The current account list
**Example request**:
.. sourcecode:: http
GET /accounts HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"items": [
{
"id": 2,
"accountNumber": 222222222,
"label": "account2",
"comments": "this is a thing"
},
{
"id": 1,
"accountNumber": 11111111111,
"label": "account1",
"comments": "this is a thing"
},
]
"total": 2
}
:query sortBy: field to sort on
:query sortDir: acs or desc
:query page: int. default is 1
:query filter: key value pair. format is k=v;
:query limit: limit number. default is 10
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
"""
parser = paginated_parser.copy()
args = parser.parse_args()
return service.render(args)
@admin_permission.require(http_exception=403)
@marshal_items(FIELDS)
def post(self):
"""
.. http:post:: /accounts
Creates a new account
**Example request**:
.. sourcecode:: http
POST /accounts HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
{
"accountNumber": 11111111111,
"label": "account1,
"comments": "this is a thing"
}
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"id": 1,
"accountNumber": 11111111111,
"label": "account1",
"comments": "this is a thing"
}
:arg accountNumber: aws account number
:arg label: human readable account label
:arg comments: some description about the account
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
"""
self.reqparse.add_argument('accountNumber', type=int, dest="account_number", location='json', required=True)
self.reqparse.add_argument('label', type=str, location='json', required=True)
self.reqparse.add_argument('comments', type=str, location='json')
args = self.reqparse.parse_args()
return service.create(args['account_number'], args['label'], args['comments'])
class Accounts(AuthenticatedResource):
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(Accounts, self).__init__()
@marshal_items(FIELDS)
def get(self, account_id):
"""
.. http:get:: /accounts/1
Get a specific account
**Example request**:
.. sourcecode:: http
GET /accounts/1 HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"id": 1,
"accountNumber": 11111111111,
"label": "account1",
"comments": "this is a thing"
}
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
"""
return service.get(account_id)
@admin_permission.require(http_exception=403)
@marshal_items(FIELDS)
def put(self, account_id):
"""
.. http:post:: /accounts/1
Updates an account
**Example request**:
.. sourcecode:: http
POST /accounts/1 HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
{
"accountNumber": 11111111111,
"label": "labelChanged,
"comments": "this is a thing"
}
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"id": 1,
"accountNumber": 11111111111,
"label": "labelChanged",
"comments": "this is a thing"
}
:arg accountNumber: aws account number
:arg label: human readable account label
:arg comments: some description about the account
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
"""
self.reqparse.add_argument('accountNumber', type=int, dest="account_number", location='json', required=True)
self.reqparse.add_argument('label', type=str, location='json', required=True)
self.reqparse.add_argument('comments', type=str, location='json')
args = self.reqparse.parse_args()
return service.update(account_id, args['account_number'], args['label'], args['comments'])
@admin_permission.require(http_exception=403)
def delete(self, account_id):
service.delete(account_id)
return {'result': True}
class CertificateAccounts(AuthenticatedResource):
""" Defines the 'certificate/<int:certificate_id/accounts'' endpoint """
def __init__(self):
super(CertificateAccounts, self).__init__()
@marshal_items(FIELDS)
def get(self, certificate_id):
"""
.. http:get:: /certificates/1/accounts
The current account list for a given certificates
**Example request**:
.. sourcecode:: http
GET /certificates/1/accounts HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"items": [
{
"id": 2,
"accountNumber": 222222222,
"label": "account2",
"comments": "this is a thing"
},
{
"id": 1,
"accountNumber": 11111111111,
"label": "account1",
"comments": "this is a thing"
},
]
"total": 2
}
:query sortBy: field to sort on
:query sortDir: acs or desc
:query page: int. default is 1
:query filter: key value pair. format is k=v;
:query limit: limit number. default is 10
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
"""
parser = paginated_parser.copy()
args = parser.parse_args()
args['certificate_id'] = certificate_id
return service.render(args)
api.add_resource(AccountsList, '/accounts', endpoint='accounts')
api.add_resource(Accounts, '/accounts/<int:account_id>', endpoint='account')
api.add_resource(CertificateAccounts, '/certificates/<int:certificate_id>/accounts', endpoint='certificateAccounts')

0
lemur/analyze/__init__.py Normal file
View File

62
lemur/analyze/service.py Normal file
View File

@ -0,0 +1,62 @@
"""
.. module: lemur.analyze.service
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
#def analyze(endpoints, truststores):
# results = {"headings": ["Endpoint"],
# "results": [],
# "time": datetime.now().strftime("#Y%m%d %H:%M:%S")}
#
# for store in truststores:
# results['headings'].append(os.path.basename(store))
#
# for endpoint in endpoints:
# result_row = [endpoint]
# for store in truststores:
# result = {'details': []}
#
# tests = []
# for region, ip in REGIONS.items():
# try:
# domain = dns.name.from_text(endpoint)
# if not domain.is_absolute():
# domain = domain.concatenate(dns.name.root)
#
# my_resolver = dns.resolver.Resolver()
# my_resolver.nameservers = [ip]
# answer = my_resolver.query(domain)
#
# #force the testing of regional enpoints by changing the dns server
# response = requests.get('https://' + str(answer[0]), verify=store)
# tests.append('pass')
# result['details'].append("{}: SSL testing completed without errors".format(region))
#
# except SSLError as e:
# log.debug(e)
# if 'hostname' in str(e):
# tests.append('pass')
# result['details'].append("{}: This test passed ssl negotiation but failed hostname verification becuase the hostname is not included in the certificate".format(region))
# elif 'certificate verify failed' in str(e):
# tests.append('fail')
# result['details'].append("{}: This test failed to verify the SSL certificate".format(region))
# else:
# tests.append('fail')
# result['details'].append("{}: {}".format(region, str(e)))
#
# except Exception as e:
# log.debug(e)
# tests.append('fail')
# result['details'].append("{}: {}".format(region, str(e)))
#
# #any failing tests fails the whole endpoint
# if 'fail' in tests:
# result['test'] = 'fail'
# else:
# result['test'] = 'pass'
#
# result_row.append(result)
# results['results'].append(result_row)
# return results
#

0
lemur/auth/__init__.py Normal file
View File

62
lemur/auth/permissions.py Normal file
View File

@ -0,0 +1,62 @@
"""
.. module: permissions
:platform: Unix
:synopsis: This module defines all the permission used within Lemur
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from functools import partial
from collections import namedtuple
from flask.ext.principal import Permission, RoleNeed
# Permissions
operator_permission = Permission(RoleNeed('operator'))
admin_permission = Permission(RoleNeed('secops@netflix.com'))
CertificateCreator = namedtuple('certificate', ['method', 'value'])
CertificateCreatorNeed = partial(CertificateCreator, 'certificateView')
CertificateOwner = namedtuple('certificate', ['method', 'value'])
CertificateOwnerNeed = partial(CertificateOwner, 'certificateView')
class ViewKeyPermission(Permission):
def __init__(self, role_id, certificate_id):
c_need = CertificateCreatorNeed(unicode(certificate_id))
o_need = CertificateOwnerNeed(unicode(role_id))
super(ViewKeyPermission, self).__init__(o_need, c_need, RoleNeed('admin'))
class UpdateCertificatePermission(Permission):
def __init__(self, role_id, certificate_id):
c_need = CertificateCreatorNeed(unicode(certificate_id))
o_need = CertificateOwnerNeed(unicode(role_id))
super(UpdateCertificatePermission, self).__init__(o_need, c_need, RoleNeed('admin'))
RoleUser = namedtuple('role', ['method', 'value'])
ViewRoleCredentialsNeed = partial(RoleUser, 'roleView')
class ViewRoleCredentialsPermission(Permission):
def __init__(self, role_id):
need = ViewRoleCredentialsNeed(unicode(role_id))
super(ViewRoleCredentialsPermission, self).__init__(need, RoleNeed('admin'))
AuthorityCreator = namedtuple('authority', ['method', 'value'])
AuthorityCreatorNeed = partial(AuthorityCreator, 'authorityUse')
AuthorityOwner = namedtuple('authority', ['method', 'value'])
AuthorityOwnerNeed = partial(AuthorityOwner, 'role')
class AuthorityPermission(Permission):
def __init__(self, authority_id, roles):
needs = [RoleNeed('admin'), AuthorityCreatorNeed(unicode(authority_id))]
for r in roles:
needs.append(AuthorityOwnerNeed(unicode(r)))
super(AuthorityPermission, self).__init__(*needs)

188
lemur/auth/service.py Normal file
View File

@ -0,0 +1,188 @@
"""
.. module: lemur.auth.service
:platform: Unix
:synopsis: This module contains all of the authentication duties for
lemur
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
import jwt
import json
import base64
import binascii
from functools import wraps
from datetime import datetime, timedelta
from flask import g, current_app, jsonify, request
from flask.ext.restful import Resource
from flask.ext.principal import identity_loaded, RoleNeed, UserNeed
from flask.ext.principal import Identity, identity_changed
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.rsa import RSAPublicNumbers
from lemur.users import service as user_service
from lemur.auth.permissions import CertificateOwnerNeed, CertificateCreatorNeed, \
AuthorityCreatorNeed, AuthorityOwnerNeed, ViewRoleCredentialsNeed
def base64url_decode(data):
if isinstance(data, unicode):
data = str(data)
rem = len(data) % 4
if rem > 0:
data += b'=' * (4 - rem)
return base64.urlsafe_b64decode(data)
def base64url_encode(data):
return base64.urlsafe_b64encode(data).replace(b'=', b'')
def get_rsa_public_key(n, e):
"""
Retrieve an RSA public key based on a module and exponent as provided by the JWKS format.
:param n:
:param e:
:return: a RSA Public Key in PEM format
"""
n = int(binascii.hexlify(base64url_decode(n)), 16)
e = int(binascii.hexlify(base64url_decode(e)), 16)
pub = RSAPublicNumbers(e, n).public_key(default_backend())
return pub.public_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PublicFormat.SubjectPublicKeyInfo
)
def create_token(user):
"""
Create a valid JWT for a given user, this token is then used to authenticate
sessions until the token expires.
:param user:
:return:
"""
expiration_delta = timedelta(days=int(current_app.config.get('TOKEN_EXPIRATION', 1)))
payload = {
'sub': user.id,
'iat': datetime.now(),
'exp': datetime.now() + expiration_delta
}
token = jwt.encode(payload, current_app.config['TOKEN_SECRET'])
return token.decode('unicode_escape')
def login_required(f):
"""
Validates the JWT and ensures that is has not expired.
:param f:
:return:
"""
@wraps(f)
def decorated_function(*args, **kwargs):
if not request.headers.get('Authorization'):
response = jsonify(message='Missing authorization header')
response.status_code = 401
return response
token = request.headers.get('Authorization').split()[1]
try:
payload = jwt.decode(token, current_app.config['TOKEN_SECRET'])
except jwt.DecodeError:
return dict(message='Token is invalid'), 403
except jwt.ExpiredSignatureError:
return dict(message='Token has expired'), 403
except jwt.InvalidTokenError:
return dict(message='Token is invalid'), 403
g.current_user = user_service.get(payload['sub'])
if not g.current_user.id:
return dict(message='You are not logged in'), 403
# Tell Flask-Principal the identity changed
identity_changed.send(current_app._get_current_object(), identity=Identity(g.current_user.id))
return f(*args, **kwargs)
return decorated_function
def fetch_token_header(token):
"""
Fetch the header out of the JWT token.
:param token:
:return: :raise jwt.DecodeError:
"""
token = token.encode('utf-8')
try:
signing_input, crypto_segment = token.rsplit(b'.', 1)
header_segment, payload_segment = signing_input.split(b'.', 1)
except ValueError:
raise jwt.DecodeError('Not enough segments')
try:
return json.loads(base64url_decode(header_segment))
except TypeError, binascii.Error:
raise jwt.DecodeError('Invalid header padding')
@identity_loaded.connect
def on_identity_loaded(sender, identity):
"""
Sets the identity of a given option, assigns additional permissions based on
the role that the user is a part of.
:param sender:
:param identity:
"""
# load the user
user = user_service.get(identity.id)
# add the UserNeed to the identity
identity.provides.add(UserNeed(identity.id))
# identity with the roles that the user provides
if hasattr(user, 'roles'):
for role in user.roles:
identity.provides.add(CertificateOwnerNeed(unicode(role.id)))
identity.provides.add(ViewRoleCredentialsNeed(unicode(role.id)))
identity.provides.add(RoleNeed(role.name))
# apply ownership for authorities
if hasattr(user, 'authorities'):
for authority in user.authorities:
identity.provides.add(AuthorityCreatorNeed(unicode(authority.id)))
# apply ownership of certificates
if hasattr(user, 'certificates'):
for certificate in user.certificates:
identity.provides.add(CertificateCreatorNeed(unicode(certificate.id)))
g.user = user
class AuthenticatedResource(Resource):
"""
Inherited by all resources that need to be protected by authentication.
"""
method_decorators = [login_required]
def __init__(self):
super(AuthenticatedResource, self).__init__()

257
lemur/auth/views.py Normal file
View File

@ -0,0 +1,257 @@
"""
.. module: lemur.auth.views
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
import jwt
import base64
import requests
from flask import g, Blueprint, current_app, abort
from flask.ext.restful import reqparse, Resource, Api
from flask.ext.principal import Identity, identity_changed
from lemur.common.crypto import unlock
from lemur.auth.permissions import admin_permission
from lemur.users import service as user_service
from lemur.roles import service as role_service
from lemur.certificates import service as cert_service
from lemur.auth.service import AuthenticatedResource, create_token, fetch_token_header, get_rsa_public_key
mod = Blueprint('auth', __name__)
api = Api(mod)
class Login(Resource):
"""
Provides an endpoint for Lemur's basic authentication. It takes a username and password
combination and returns a JWT token.
This token token is required for each API request and must be provided in the Authorization Header for the request.
::
Authorization:Bearer <token>
Tokens have a set expiration date. You can inspect the token expiration be base64 decoding the token and inspecting
it's contents.
.. note:: It is recommended that the token expiration is fairly short lived (hours not days). This will largely depend \
on your uses cases but. It is important to not that there is currently no build in method to revoke a users token \
and force re-authentication.
"""
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(Login, self).__init__()
def post(self):
"""
.. http:post:: /auth/login
Login with username:password
**Example request**:
.. sourcecode:: http
POST /auth/login HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
{
"username": "test",
"password": "test"
}
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"token": "12343243243"
}
:arg username: username
:arg password: password
:statuscode 401: invalid credentials
:statuscode 200: no error
"""
self.reqparse.add_argument('username', type=str, required=True, location='json')
self.reqparse.add_argument('password', type=str, required=True, location='json')
args = self.reqparse.parse_args()
if '@' in args['username']:
user = user_service.get_by_email(args['username'])
else:
user = user_service.get_by_username(args['username'])
if user and user.check_password(args['password']):
# Tell Flask-Principal the identity changed
identity_changed.send(current_app._get_current_object(),
identity=Identity(user.id))
return dict(token=create_token(user))
return dict(message='The supplied credentials are invalid'), 401
def get(self):
return {'username': g.current_user.username, 'roles': [r.name for r in g.current_user.roles]}
class Ping(Resource):
"""
This class serves as an example of how one might implement an SSO provider for use with Lemur. In
this example we use a OpenIDConnect authentication flow, that is essentially OAuth2 underneath. If you have an
OAuth2 provider you want to use Lemur there would be two steps:
1. Define your own class that inherits from :class:`flask.ext.restful.Resource` and create the HTTP methods the \
provider uses for it's callbacks.
2. Add or change the Lemur AngularJS Configuration to point to your new provider
"""
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(Ping, self).__init__()
def post(self):
self.reqparse.add_argument('clientId', type=str, required=True, location='json')
self.reqparse.add_argument('redirectUri', type=str, required=True, location='json')
self.reqparse.add_argument('code', type=str, required=True, location='json')
args = self.reqparse.parse_args()
# take the information we have received from Meechum to create a new request
params = {
'client_id': args['clientId'],
'grant_type': 'authorization_code',
'scope': 'openid email profile address',
'redirect_uri': args['redirectUri'],
'code': args['code']
}
# you can either discover these dynamically or simply configure them
access_token_url = current_app.config.get('PING_ACCESS_TOKEN_URL')
user_api_url = current_app.config.get('PING_USER_API_URL')
# the secret and cliendId will be given to you when you signup for meechum
basic = base64.b64encode('{0}:{1}'.format(args['clientId'], current_app.config.get("PING_SECRET")))
headers = {'Authorization': 'Basic {0}'.format(basic)}
# exchange authorization code for access token.
r = requests.post(access_token_url, headers=headers, params=params)
id_token = r.json()['id_token']
access_token = r.json()['access_token']
# fetch token public key
header_data = fetch_token_header(id_token)
jwks_url = current_app.config.get('PING_JWKS_URL')
# retrieve the key material as specified by the token header
r = requests.get(jwks_url)
for key in r.json()['keys']:
if key['kid'] == header_data['kid']:
secret = get_rsa_public_key(key['n'], key['e'])
algo = header_data['alg']
break
else:
return dict(message='Key not found'), 403
# validate your token based on the key it was signed with
try:
jwt.decode(id_token, secret, algorithms=[algo], audience=args['clientId'])
except jwt.DecodeError:
return dict(message='Token is invalid'), 403
except jwt.ExpiredSignatureError:
return dict(message='Token has expired'), 403
except jwt.InvalidTokenError:
return dict(message='Token is invalid'), 403
user_params = dict(access_token=access_token, schema='profile')
# retrieve information about the current user.
r = requests.get(user_api_url, params=user_params)
profile = r.json()
user = user_service.get_by_email(profile['email'])
# update their google 'roles'
roles = []
# Legacy edge case - 'admin' has some special privileges associated with it
if 'secops@netflix.com' in profile['googleGroups']:
roles.append(role_service.get_by_name('admin'))
for group in profile['googleGroups']:
role = role_service.get_by_name(group)
if not role:
role = role_service.create(group, description='This is a google group based role created by Lemur')
roles.append(role)
# if we get an sso user create them an account
# we still pick a random password in case sso is down
if not user:
# every user is an operator (tied to the verisignCA)
v = role_service.get_by_name('verisign')
if v:
roles.append(v)
user = user_service.create(
profile['email'],
cert_service.create_challenge(),
profile['email'],
True,
profile.get('thumbnailPhotoUrl'),
roles
)
else:
# we add 'lemur' specific roles, so they do not get marked as removed
for ur in user.roles:
if ur.authority_id:
roles.append(ur)
# update any changes to the user
user_service.update(
user.id,
profile['email'],
profile['email'],
True,
profile.get('thumbnailPhotoUrl'), # incase profile isn't google+ enabled
roles
)
# Tell Flask-Principal the identity changed
identity_changed.send(current_app._get_current_object(), identity=Identity(user.id))
return dict(token=create_token(user))
class Unlock(AuthenticatedResource):
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(Unlock, self).__init__()
@admin_permission.require(http_exception=403)
def post(self):
self.reqparse.add_argument('password', type=str, required=True, location='json')
args = self.reqparse.parse_args()
unlock(args['password'])
return {
"message": "You have successfully unlocked this Lemur instance",
"type": "success"
}
api.add_resource(Login, '/auth/login', endpoint='login')
api.add_resource(Ping, '/auth/ping', endpoint='ping')
api.add_resource(Unlock, '/auth/unlock', endpoint='unlock')

0
lemur/authorities/__init__.py Normal file
View File

58
lemur/authorities/models.py Normal file
View File

@ -0,0 +1,58 @@
"""
.. module: lemur.authorities.models
:platform: unix
:synopsis: This module contains all of the models need to create a authority within Lemur.
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from sqlalchemy.orm import relationship
from sqlalchemy import Column, Integer, String, Text, func, ForeignKey, DateTime, PassiveDefault, Boolean
from sqlalchemy.dialects.postgresql import JSON
from lemur.database import db
from lemur.certificates.models import cert_get_cn, cert_get_not_after, cert_get_not_before
class Authority(db.Model):
__tablename__ = 'authorities'
id = Column(Integer, primary_key=True)
owner = Column(String(128))
name = Column(String(128), unique=True)
body = Column(Text())
chain = Column(Text())
bits = Column(Integer())
cn = Column(String(128))
not_before = Column(DateTime)
not_after = Column(DateTime)
active = Column(Boolean, default=True)
date_created = Column(DateTime, PassiveDefault(func.now()), nullable=False)
plugin_name = Column(String(64))
description = Column(Text)
options = Column(JSON)
roles = relationship('Role', backref=db.backref('authority'), lazy='dynamic')
user_id = Column(Integer, ForeignKey('users.id'))
certificates = relationship("Certificate", backref='authority')
def __init__(self, name, owner, plugin_name, body, roles=None, chain=None, description=None):
self.name = name
self.body = body
self.chain = chain
self.owner = owner
self.plugin_name = plugin_name
cert = x509.load_pem_x509_certificate(str(body), default_backend())
self.cn = cert_get_cn(cert)
self.not_before = cert_get_not_before(cert)
self.not_after = cert_get_not_after(cert)
self.roles = roles
self.description = description
def as_dict(self):
return {c.name: getattr(self, c.name) for c in self.__table__.columns}
def serialize(self):
blob = self.as_dict()
return blob

173
lemur/authorities/service.py Normal file
View File

@ -0,0 +1,173 @@
"""
.. module: lemur.authorities.service
:platform: Unix
:synopsis: This module contains all of the services level functions used to
administer authorities in Lemur
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from flask import g
from lemur import database
from lemur.authorities.models import Authority
from lemur.roles import service as role_service
from lemur.roles.models import Role
import lemur.certificates.service as cert_service
from lemur.common.services.issuers.manager import get_plugin_by_name
def update(authority_id, active=None, roles=None):
"""
Update a an authority with new values.
:param authority_id:
:param roles: roles that are allowed to use this authority
:rtype : Authority
:return:
"""
authority = get(authority_id)
if roles:
authority = database.update_list(authority, 'roles', Role, roles)
if active:
authority.active = active
return database.update(authority)
def create(kwargs):
"""
Create a new authority.
:param name: name of the authority
:param roles: roles that are allowed to use this authority
:param options: available options for authority
:param description:
:rtype : Authority
:return:
"""
issuer = get_plugin_by_name(kwargs.get('pluginName'))
kwargs['creator'] = g.current_user.email
cert_body, intermediate, issuer_roles = issuer.create_authority(kwargs)
cert = cert_service.save_cert(cert_body, None, intermediate, None, None, None)
cert.user = g.current_user
# we create and attach any roles that cloudCA gives us
role_objs = []
for r in issuer_roles:
role = role_service.create(r['name'], password=r['password'], description="CloudCA auto generated role",
username=r['username'])
# the user creating the authority should be able to administer it
if role.username == 'admin':
g.current_user.roles.append(role)
role_objs.append(role)
authority = Authority(
kwargs.get('caName'),
kwargs['ownerEmail'],
kwargs['pluginName'],
cert_body,
description=kwargs['caDescription'],
chain=intermediate,
roles=role_objs
)
# do this last encase we need to roll back/abort
database.update(cert)
authority = database.create(authority)
g.current_user.authorities.append(authority)
return authority
def get_all():
"""
Get all authorities that are currently in Lemur.
:rtype : List
:return:
"""
query = database.session_query(Authority)
return database.find_all(query, Authority, {}).all()
def get(authority_id):
"""
Retrieves an authority given it's ID
:rtype : Authority
:param authority_id:
:return:
"""
return database.get(Authority, authority_id)
def get_by_name(authority_name):
"""
Retrieves an authority given it's name.
:param authority_name:
:rtype : Authority
:return:
"""
return database.get(Authority, authority_name, field='name')
def get_authority_role(ca_name):
"""
Attempts to get the authority role for a given ca uses current_user
as a basis for accomplishing that.
:param ca_name:
"""
if g.current_user.is_admin:
authority = get_by_name(ca_name)
#TODO we should pick admin ca roles for admin
return authority.roles[0]
else:
for role in g.current_user.roles:
if role.authority:
if role.authority.name == ca_name:
return role
def render(args):
"""
Helper that helps us render the REST Api responses.
:param args:
:return:
"""
query = database.session_query(Authority)
sort_by = args.pop('sort_by')
sort_dir = args.pop('sort_dir')
page = args.pop('page')
count = args.pop('count')
filt = args.pop('filter')
if filt:
terms = filt.split(';')
if 'active' in filt: # this is really weird but strcmp seems to not work here??
query = query.filter(Authority.active == terms[1])
else:
query = database.filter(query, Authority, terms)
# we make sure that a user can only use an authority they either own are are a member of - admins can see all
if not g.current_user.is_admin:
authority_ids = []
for role in g.current_user.roles:
if role.authority:
authority_ids.append(role.authority.id)
query = query.filter(Authority.id.in_(authority_ids))
query = database.find_all(query, Authority, args)
if sort_by and sort_dir:
query = database.sort(query, Authority, sort_by, sort_dir)
return database.paginate(query, page, count)

372
lemur/authorities/views.py Normal file
View File

@ -0,0 +1,372 @@
"""
.. module: lemur.authorities.views
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from flask import Blueprint, g
from flask.ext.restful import reqparse, fields, Api
from lemur.authorities import service
from lemur.roles import service as role_service
from lemur.certificates import service as certificate_service
from lemur.auth.service import AuthenticatedResource
from lemur.auth.permissions import AuthorityPermission
from lemur.common.utils import paginated_parser, marshal_items
FIELDS = {
'name': fields.String,
'description': fields.String,
'options': fields.Raw,
'pluginName': fields.String,
'body': fields.String,
'chain': fields.String,
'active': fields.Boolean,
'notBefore': fields.DateTime(dt_format='iso8601', attribute='not_before'),
'notAfter': fields.DateTime(dt_format='iso8601', attribute='not_after'),
'id': fields.Integer,
}
mod = Blueprint('authorities', __name__)
api = Api(mod)
class AuthoritiesList(AuthenticatedResource):
""" Defines the 'authorities' endpoint """
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(AuthoritiesList, self).__init__()
@marshal_items(FIELDS)
def get(self):
"""
.. http:get:: /authorities
The current list of authorities
**Example request**:
.. sourcecode:: http
GET /authorities HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"items": [
{
"id": 1,
"name": "authority1",
"description": "this is authority1",
"pluginName": null,
"chain": "-----Begin ...",
"body": "-----Begin ...",
"active": true,
"notBefore": "2015-06-05T17:09:39",
"notAfter": "2015-06-10T17:09:39"
"options": null
}
]
"total": 1
}
:query sortBy: field to sort on
:query sortDir: acs or desc
:query page: int. default is 1
:query filter: key value pair. format is k=v;
:query limit: limit number. default is 10
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
:note: this will only show certificates that the current user is authorized to use
"""
parser = paginated_parser.copy()
args = parser.parse_args()
return service.render(args)
@marshal_items(FIELDS)
def post(self):
"""
.. http:post:: /authorities
Create an authority
**Example request**:
.. sourcecode:: http
POST /authorities HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
{
"caDN": {
"country": "US",
"state": "CA",
"location": "A Location",
"organization": "ExampleInc",
"organizationalUnit": "Operations",
"commonName": "a common name"
},
"caType": "root",
"caSigningAlgo": "sha256WithRSA",
"caSensitivity": "medium",
"keyType": "RSA2048",
"pluginName": "cloudca",
"validityStart": "2015-06-11T07:00:00.000Z",
"validityEnd": "2015-06-13T07:00:00.000Z",
"caName": "DoctestCA",
"ownerEmail": "jimbob@example.com",
"caDescription": "Example CA",
"extensions": {
"subAltNames": {
"names": []
}
},
}
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"id": 1,
"name": "authority1",
"description": "this is authority1",
"pluginName": null,
"chain": "-----Begin ...",
"body": "-----Begin ...",
"active": true,
"notBefore": "2015-06-05T17:09:39",
"notAfter": "2015-06-10T17:09:39"
"options": null
}
:arg caName: authority's name
:arg caDescription: a sensible description about what the CA with be used for
:arg ownerEmail: the team or person who 'owns' this authority
:arg validityStart: when this authority should start issuing certificates
:arg validityEnd: when this authority should stop issuing certificates
:arg extensions: certificate extensions
:arg pluginName: name of the plugin to create the authority
:arg caType: the type of authority (root/subca)
:arg caParent: the parent authority if this is to be a subca
:arg caSigningAlgo: algorithm used to sign the authority
:arg keyType: key type
:arg caSensitivity: the sensitivity of the root key, for CloudCA this determines if the root keys are stored
in an HSM
:arg caKeyName: name of the key to store in the HSM (CloudCA)
:arg caSerialNumber: serial number of the authority
:arg caFirstSerial: specifies the starting serial number for certificates issued off of this authority
:reqheader Authorization: OAuth token to authenticate
:statuscode 403: unauthenticated
:statuscode 200: no error
"""
self.reqparse.add_argument('caName', type=str, location='json', required=True)
self.reqparse.add_argument('caDescription', type=str, location='json', required=False)
self.reqparse.add_argument('ownerEmail', type=str, location='json', required=True)
self.reqparse.add_argument('caDN', type=dict, location='json', required=False)
self.reqparse.add_argument('validityStart', type=str, location='json', required=False) # TODO validate
self.reqparse.add_argument('validityEnd', type=str, location='json', required=False) # TODO validate
self.reqparse.add_argument('extensions', type=dict, location='json', required=False)
self.reqparse.add_argument('pluginName', type=str, location='json', required=True)
self.reqparse.add_argument('caType', type=str, location='json', required=False)
self.reqparse.add_argument('caParent', type=str, location='json', required=False)
self.reqparse.add_argument('caSigningAlgo', type=str, location='json', required=False)
self.reqparse.add_argument('keyType', type=str, location='json', required=False)
self.reqparse.add_argument('caSensitivity', type=str, location='json', required=False)
self.reqparse.add_argument('caKeyName', type=str, location='json', required=False)
self.reqparse.add_argument('caSerialNumber', type=int, location='json', required=False)
self.reqparse.add_argument('caFirstSerial', type=int, location='json', required=False)
args = self.reqparse.parse_args()
return service.create(args)
class Authorities(AuthenticatedResource):
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(Authorities, self).__init__()
@marshal_items(FIELDS)
def get(self, authority_id):
"""
.. http:get:: /authorities/1
One authority
**Example request**:
.. sourcecode:: http
GET /authorities/1 HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"id": 1,
"name": "authority1",
"description": "this is authority1",
"pluginName": null,
"chain": "-----Begin ...",
"body": "-----Begin ...",
"active": true,
"notBefore": "2015-06-05T17:09:39",
"notAfter": "2015-06-10T17:09:39"
"options": null
}
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
"""
return service.get(authority_id)
@marshal_items(FIELDS)
def put(self, authority_id):
"""
.. http:put:: /authorities/1
Update a authority
**Example request**:
.. sourcecode:: http
PUT /authorities/1 HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
{
"roles": [],
"active": false
}
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"id": 1,
"name": "authority1",
"description": "this is authority1",
"pluginname": null,
"chain": "-----begin ...",
"body": "-----begin ...",
"active": false,
"notbefore": "2015-06-05t17:09:39",
"notafter": "2015-06-10t17:09:39"
"options": null
}
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
"""
self.reqparse.add_argument('roles', type=list, location='json')
self.reqparse.add_argument('active', type=str, location='json')
args = self.reqparse.parse_args()
authority = service.get(authority_id)
role = role_service.get_by_name(authority.owner)
# all the authority role members should be allowed
roles = [x.name for x in authority.roles]
# allow "owner" roles by team DL
roles.append(role)
permission = AuthorityPermission(authority_id, roles)
# we want to make sure that we cannot add roles that we are not members of
if not g.current_user.is_admin:
role_ids = set([r['id'] for r in args['roles']])
user_role_ids = set([r.id for r in g.current_user.roles])
if not role_ids.issubset(user_role_ids):
return dict(message="You are not allowed to associate a role which you are not a member of"), 400
if permission.can():
return service.update(authority_id, active=args['active'], roles=args['roles'])
return dict(message="You are not authorized to update this authority"), 403
class CertificateAuthority(AuthenticatedResource):
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(CertificateAuthority, self).__init__()
@marshal_items(FIELDS)
def get(self, certificate_id):
"""
.. http:get:: /certificates/1/authority
One authority for given certificate
**Example request**:
.. sourcecode:: http
GET /certificates/1/authority HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"id": 1,
"name": "authority1",
"description": "this is authority1",
"pluginName": null,
"chain": "-----Begin ...",
"body": "-----Begin ...",
"active": true,
"notBefore": "2015-06-05T17:09:39",
"notAfter": "2015-06-10T17:09:39"
"options": null
}
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
"""
return certificate_service.get(certificate_id).authority
api.add_resource(AuthoritiesList, '/authorities', endpoint='authorities')
api.add_resource(Authorities, '/authorities/<int:authority_id>', endpoint='authority')
api.add_resource(CertificateAuthority, '/certificates/<int:certificate_id>/authority', endpoint='certificateAuthority')

0
lemur/certificates/__init__.py Normal file
View File

87
lemur/certificates/exceptions.py Normal file
View File

@ -0,0 +1,87 @@
"""
.. module: lemur.certificates.exceptions
:synopsis: Defines all monterey specific exceptions
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from flask import current_app
from lemur.exceptions import LemurException
class UnknownAuthority(LemurException):
def __init__(self, authority):
self.code = 404
self.authority = authority
self.data = {"message": "The authority specified '{}' is not a valid authority".format(self.authority)}
current_app.logger.warning(self)
def __str__(self):
return repr(self.data['message'])
class InsufficientDomains(LemurException):
def __init__(self):
self.code = 400
self.data = {"message": "Need at least one domain specified in order create a certificate"}
current_app.logger.warning(self)
def __str__(self):
return repr(self.data['message'])
class InvalidCertificate(LemurException):
def __init__(self):
self.code = 400
self.data = {"message": "Need at least one domain specified in order create a certificate"}
current_app.logger.warning(self)
def __str__(self):
return repr(self.data['message'])
class UnableToCreateCSR(LemurException):
def __init__(self):
self.code = 500
self.data = {"message": "Unable to generate CSR"}
current_app.logger.error(self)
def __str__(self):
return repr(self.data['message'])
class UnableToCreatePrivateKey(LemurException):
def __init__(self):
self.code = 500
self.data = {"message": "Unable to generate Private Key"}
current_app.logger.error(self)
def __str__(self):
return repr(self.data['message'])
class MissingFiles(LemurException):
def __init__(self, path):
self.code = 500
self.path = path
self.data = {"path": self.path, "message": "Expecting missing files"}
current_app.logger.error(self)
def __str__(self):
return repr(self.data['message'])
class NoPersistanceFound(LemurException):
def __init__(self):
self.code = 500
self.data = {"code": 500, "message": "No peristence method found, Lemur cannot persist sensitive information"}
current_app.logger.error(self)
def __str__(self):
return repr(self.data['message'])

307
lemur/certificates/models.py Normal file
View File

@ -0,0 +1,307 @@
"""
.. module: lemur.certificates.models
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
import os
import datetime
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from flask import current_app
from sqlalchemy.orm import relationship
from sqlalchemy import Integer, ForeignKey, String, DateTime, PassiveDefault, func, Column, Text, Boolean
from sqlalchemy_utils import EncryptedType
from lemur.database import db
from lemur.domains.models import Domain
from lemur.users import service as user_service
from lemur.constants import SAN_NAMING_TEMPLATE, DEFAULT_NAMING_TEMPLATE, NONSTANDARD_NAMING_TEMPLATE
from lemur.models import certificate_associations, certificate_account_associations
def cert_get_cn(cert):
"""
Attempts to get a sane common name from a given certificate.
:param cert:
:return: Common name or None
"""
try:
return cert.subject.get_attributes_for_oid(
x509.OID_COMMON_NAME
)[0].value.strip()
except Exception as e:
current_app.logger.error("Unable to get CN! {0}".format(e))
def cert_get_domains(cert):
"""
Attempts to get an domains listed in a certificate.
If 'subjectAltName' extension is not available we simply
return the common name.
:param cert:
:return: List of domains
"""
domains = []
try:
ext = cert.extensions.get_extension_for_oid(x509.OID_SUBJECT_ALTERNATIVE_NAME)
entries = ext.get_values_for(x509.DNSName)
for entry in entries:
domains.append(entry.split(":")[1].strip(", "))
except Exception as e:
current_app.logger.warning("Failed to get SubjectAltName: {0}".format(e))
domains.append(cert_get_cn(cert))
return domains
def cert_get_serial(cert):
"""
Fetch the serial number from the certificate.
:param cert:
:return: serial number
"""
return cert.serial
def cert_is_san(cert):
"""
Determines if a given certificate is a SAN certificate.
SAN certificates are simply certificates that cover multiple domains.
:param cert:
:return: Bool
"""
domains = cert_get_domains(cert)
if len(domains) > 1:
return True
return False
def cert_is_wildcard(cert):
"""
Determines if certificate is a wildcard certificate.
:param cert:
:return: Bool
"""
domains = cert_get_domains(cert)
if len(domains) == 1 and domains[0][0:1] == "*":
return True
return False
def cert_get_bitstrength(cert):
"""
Calculates a certificates public key bit length.
:param cert:
:return: Integer
"""
return cert.public_key().key_size * 8
def cert_get_issuer(cert):
"""
Gets a sane issuer from a given certificate.
:param cert:
:return: Issuer
"""
try:
return cert.subject.get_attributes_for_oid(x509.OID_ORGANIZATION_NAME)[0].value
except Exception as e:
current_app.logger.error("Unable to get issuer! {0}".format(e))
def cert_is_internal(cert):
"""
Uses an internal resource in order to determine if
a given certificate was issued by an 'internal' certificate
authority.
:param cert:
:return: Bool
"""
if cert_get_issuer(cert) in current_app.config.get('INTERNAL_CA', []):
return True
return False
def cert_get_not_before(cert):
"""
Gets the naive datetime of the certificates 'not_before' field.
This field denotes the first date in time which the given certificate
is valid.
:param cert:
:return: Datetime
"""
return cert.not_valid_before
def cert_get_not_after(cert):
"""
Gets the naive datetime of the certificates 'not_after' field.
This field denotes the last date in time which the given certificate
is valid.
:param cert:
:return: Datetime
"""
return cert.not_valid_after
def get_name_from_arn(arn):
"""
Extract the certificate name from an arn.
:param arn: IAM SSL arn
:return: name of the certificate as uploaded to AWS
"""
return arn.split("/", 1)[1]
def get_account_number(arn):
"""
Extract the account number from an arn.
:param arn: IAM SSL arn
:return: account number associated with ARN
"""
return arn.split(":")[4]
class Certificate(db.Model):
__tablename__ = 'certificates'
id = Column(Integer, primary_key=True)
owner = Column(String(128))
body = Column(Text())
private_key = Column(EncryptedType(String, os.environ.get('LEMUR_ENCRYPTION_KEY')))
challenge = Column(EncryptedType(String, os.environ.get('LEMUR_ENCRYPTION_KEY')))
csr_config = Column(Text())
status = Column(String(128))
deleted = Column(Boolean, index=True)
name = Column(String(128))
chain = Column(Text())
bits = Column(Integer())
issuer = Column(String(128))
serial = Column(String(128))
cn = Column(String(128))
description = Column(String(1024))
active = Column(Boolean, default=True)
san = Column(String(1024))
not_before = Column(DateTime)
not_after = Column(DateTime)
date_created = Column(DateTime, PassiveDefault(func.now()), nullable=False)
user_id = Column(Integer, ForeignKey('users.id'))
authority_id = Column(Integer, ForeignKey('authorities.id'))
accounts = relationship("Account", secondary=certificate_account_associations, backref='certificate')
domains = relationship("Domain", secondary=certificate_associations, backref="certificate")
elb_listeners = relationship("Listener", lazy='dynamic', backref='certificate')
def __init__(self, body, private_key=None, challenge=None, chain=None, csr_config=None):
self.body = body
# We encrypt the private_key on creation
self.private_key = private_key
self.chain = chain
self.csr_config = csr_config
self.challenge = challenge
cert = x509.load_pem_x509_certificate(str(self.body), default_backend())
self.bits = cert_get_bitstrength(cert)
self.issuer = cert_get_issuer(cert)
self.serial = cert_get_serial(cert)
self.cn = cert_get_cn(cert)
self.san = cert_is_san(cert)
self.not_before = cert_get_not_before(cert)
self.not_after = cert_get_not_after(cert)
self.name = self.create_name
for domain in cert_get_domains(cert):
self.domains.append(Domain(name=domain))
@property
def create_name(self):
"""
Create a name for our certificate. A naming standard
is based on a series of templates. The name includes
useful information such as Common Name, Validation dates,
and Issuer.
:rtype : str
:return:
"""
# aws doesn't allow special chars
if self.cn:
subject = self.cn.replace('*', "WILDCARD")
if self.san:
t = SAN_NAMING_TEMPLATE
else:
t = DEFAULT_NAMING_TEMPLATE
temp = t.format(
subject=subject,
issuer=self.issuer,
not_before=self.not_before.strftime('%Y%m%d'),
not_after=self.not_after.strftime('%Y%m%d')
)
else:
t = NONSTANDARD_NAMING_TEMPLATE
temp = t.format(
issuer=self.issuer,
not_before=self.not_before.strftime('%Y%m%d'),
not_after=self.not_after.strftime('%Y%m%d')
)
return temp
@property
def is_expired(self):
if self.not_after < datetime.datetime.now():
return True
@property
def is_unused(self):
if self.elb_listeners.count() == 0:
return True
@property
def is_revoked(self):
# we might not yet know the condition of the cert
if self.status:
if 'revoked' in self.status:
return True
def get_arn(self, account_number):
"""
Generate a valid AWS IAM arn
:rtype : str
:param account_number:
:return:
"""
return "arn:aws:iam::{}:server-certificate/{}".format(account_number, self.name)
def as_dict(self):
return {c.name: getattr(self, c.name) for c in self.__table__.columns}
def serialize(self):
blob = self.as_dict()
# TODO this should be done with relationships
user = user_service.get(self.user_id)
if user:
blob['creator'] = user.username
return blob

446
lemur/certificates/service.py Normal file
View File

@ -0,0 +1,446 @@
"""
.. module: service
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
import os
import arrow
import string
import random
import hashlib
import datetime
import subprocess
from sqlalchemy import func, or_
from flask import g, current_app
from lemur import database
from lemur.common.services.aws import iam
from lemur.common.services.issuers.manager import get_plugin_by_name
from lemur.certificates.models import Certificate
from lemur.certificates.exceptions import UnableToCreateCSR, \
UnableToCreatePrivateKey, MissingFiles
from lemur.accounts.models import Account
from lemur.accounts import service as account_service
from lemur.authorities.models import Authority
from lemur.roles.models import Role
def get(cert_id):
"""
Retrieves certificate by it's ID.
:param cert_id:
:return:
"""
return database.get(Certificate, cert_id)
def get_by_name(name):
"""
Retrieves certificate by it's Name.
:param name:
:return:
"""
return database.get(Certificate, name, field='name')
def delete(cert_id):
"""
Delete's a certificate.
:param cert_id:
"""
database.delete(get(cert_id))
def disassociate_aws_account(certs, account):
"""
Removes the account association from a certificate. We treat AWS as a completely
external service. Certificates are added and removed from this service but a record
of that certificate is always kept and tracked by Lemur. This allows us to migrate
certificates to different accounts with ease.
:param certs:
:param account:
"""
account_certs = Certificate.query.filter(Certificate.accounts.any(Account.id == 1)).\
filter(~Certificate.body.in_(certs)).all()
for a_cert in account_certs:
try:
a_cert.accounts.remove(account)
except Exception as e:
current_app.logger.debug("Skipping {0} account {1} is already disassociated".format(a_cert.name, account.label))
continue
database.update(a_cert)
def get_all_certs():
"""
Retrieves all certificates within Lemur.
:return:
"""
return Certificate.query.all()
def find_duplicates(cert_body):
"""
Finds certificates that already exist within Lemur. We do this by looking for
certificate bodies that are the same. This is the most reliable way to determine
if a certificate is already being tracked by Lemur.
:param cert_body:
:return:
"""
return Certificate.query.filter_by(body=cert_body).all()
def update(cert_id, owner, active):
"""
Updates a certificate.
:param cert_id:
:param owner:
:param active:
:return:
"""
cert = get(cert_id)
cert.owner = owner
cert.active = active
return database.update(cert)
def mint(issuer_options):
"""
Minting is slightly different for each authority.
Support for multiple authorities is handled by individual plugins.
:param issuer_options:
"""
authority = issuer_options['authority']
issuer = get_plugin_by_name(authority.plugin_name)
# NOTE if we wanted to support more issuers it might make sense to
# push CSR creation down to the plugin
path = create_csr(issuer.get_csr_config(issuer_options))
challenge, csr, csr_config, private_key = load_ssl_pack(path)
issuer_options['challenge'] = challenge
issuer_options['creator'] = g.user.email
cert_body, cert_chain = issuer.create_certificate(csr, issuer_options)
cert = save_cert(cert_body, private_key, cert_chain, challenge, csr_config, issuer_options.get('accounts'))
cert.user = g.user
cert.authority = authority
database.update(cert)
# securely delete pack after saving it to RDS and IAM (if applicable)
delete_ssl_pack(path)
return cert, private_key, cert_chain,
def import_certificate(**kwargs):
"""
Uploads already minted certificates and pulls the required information into Lemur.
This is to be used for certificates that are reated outside of Lemur but
should still be tracked.
Internally this is used to bootstrap Lemur with external
certificates, and used when certificates are 'discovered' through various discovery
techniques. was still in aws.
:param kwargs:
"""
cert = Certificate(kwargs['public_certificate'])
cert.owner = kwargs.get('owner', )
cert.creator = kwargs.get('creator', 'Lemur')
# NOTE existing certs may not follow our naming standard we will
# overwrite the generated name with the actual cert name
if kwargs.get('name'):
cert.name = kwargs.get('name')
if kwargs.get('user'):
cert.user = kwargs.get('user')
if kwargs.get('account'):
cert.accounts.append(kwargs.get('account'))
cert = database.create(cert)
return cert
def save_cert(cert_body, private_key, cert_chain, challenge, csr_config, accounts):
"""
Determines if the certificate needs to be uploaded to AWS or other services.
:param cert_body:
:param private_key:
:param cert_chain:
:param challenge:
:param csr_config:
:param account_ids:
"""
cert = Certificate(cert_body, private_key, challenge, cert_chain, csr_config)
# if we have an AWS accounts lets upload them
if accounts:
for account in accounts:
account = account_service.get(account['id'])
iam.upload_cert(account.account_number, cert, private_key, cert_chain)
cert.accounts.append(account)
return cert
def upload(**kwargs):
"""
Allows for pre-made certificates to be imported into Lemur.
"""
# save this cert the same way we save all of our certs, including uploading
# to aws if necessary
cert = save_cert(
kwargs.get('public_cert'),
kwargs.get('private_key'),
kwargs.get('intermediate_cert'),
None,
None,
kwargs.get('accounts')
)
cert.owner = kwargs['owner']
cert = database.create(cert)
g.user.certificates.append(cert)
return cert
def create(**kwargs):
"""
Creates a new certificate.
"""
cert, private_key, cert_chain = mint(kwargs)
cert.owner = kwargs['owner']
database.create(cert)
g.user.certificates.append(cert)
database.update(g.user)
return cert
def render(args):
"""
Helper function that allows use to render our REST Api.
:param args:
:return:
"""
query = database.session_query(Certificate)
time_range = args.pop('time_range')
account_id = args.pop('account_id')
show = args.pop('show')
owner = args.pop('owner')
creator = args.pop('creator') # TODO we should enabling filtering by owner
filt = args.pop('filter')
if filt:
terms = filt.split(';')
if 'issuer' in terms:
# we can't rely on issuer being correct in the cert directly so we combine queries
sub_query = database.session_query(Authority.id)\
.filter(Authority.name.ilike('%{0}%'.format(terms[1])))\
.subquery()
query = query.filter(
or_(
Certificate.issuer.ilike('%{0}%'.format(terms[1])),
Certificate.authority_id.in_(sub_query)
)
)
return database.sort_and_page(query, Certificate, args)
if 'account' in terms:
query = query.filter(Certificate.accounts.any(Account.id == terms[1]))
elif 'active' in filt: # this is really weird but strcmp seems to not work here??
query = query.filter(Certificate.active == terms[1])
else:
query = database.filter(query, Certificate, terms)
if show:
sub_query = database.session_query(Role.name).filter(Role.user_id == g.user.id).subquery()
query = query.filter(
or_(
Certificate.user_id == g.user.id,
Certificate.owner.in_(sub_query)
)
)
if account_id:
query = query.filter(Certificate.accounts.any(Account.id == account_id))
if time_range:
to = arrow.now().replace(weeks=+time_range).format('YYYY-MM-DD')
now = arrow.now().format('YYYY-MM-DD')
query = query.filter(Certificate.not_after <= to).filter(Certificate.not_after >= now)
return database.sort_and_page(query, Certificate, args)
def create_csr(csr_config):
"""
Given a list of domains create the appropriate csr
for those domains
:param csr_config:
"""
# we create a no colliding file name
path = create_path(hashlib.md5(csr_config).hexdigest())
challenge = create_challenge()
challenge_path = os.path.join(path, 'challenge.txt')
with open(challenge_path, 'w') as c:
c.write(challenge)
csr_path = os.path.join(path, 'csr_config.txt')
with open(csr_path, 'w') as f:
f.write(csr_config)
#TODO use cloudCA to seed a -rand file for each call
#TODO replace openssl shell calls with cryptograph
with open('/dev/null', 'w') as devnull:
code = subprocess.call(['openssl', 'genrsa',
'-out', os.path.join(path, 'private.key'), '2048'],
stdout=devnull, stderr=devnull)
if code != 0:
raise UnableToCreatePrivateKey(code)
with open('/dev/null', 'w') as devnull:
code = subprocess.call(['openssl', 'req', '-new', '-sha256', '-nodes',
'-config', csr_path, "-key", os.path.join(path, 'private.key'),
"-out", os.path.join(path, 'request.csr')], stdout=devnull, stderr=devnull)
if code != 0:
raise UnableToCreateCSR(code)
return path
def create_path(domain_hash):
"""
:param domain_hash:
:return:
"""
path = os.path.join('/tmp', domain_hash)
try:
os.mkdir(path)
except OSError as e:
now = datetime.datetime.now()
path = os.path.join('/tmp', "{}.{}".format(domain_hash, now.strftime('%s')))
os.mkdir(path)
current_app.logger.warning(e)
current_app.logger.debug("Writing ssl files to: {}".format(path))
return path
def load_ssl_pack(path):
"""
Loads the information created by openssl to be used by other functions.
:param path:
"""
if len(os.listdir(path)) != 4:
raise MissingFiles(path)
with open(os.path.join(path, 'challenge.txt')) as c:
challenge = c.read()
with open(os.path.join(path, 'request.csr')) as r:
csr = r.read()
with open(os.path.join(path, 'csr_config.txt')) as config:
csr_config = config.read()
with open(os.path.join(path, 'private.key')) as key:
private_key = key.read()
return (challenge, csr, csr_config, private_key,)
def delete_ssl_pack(path):
"""
Removes the temporary files associated with CSR creation.
:param path:
"""
subprocess.check_call(['srm', '-r', path])
def create_challenge():
"""
Create a random and strongish csr challenge.
"""
challenge = ''.join(random.choice(string.ascii_uppercase) for x in range(6))
challenge += ''.join(random.choice("~!@#$%^&*()_+") for x in range(6))
challenge += ''.join(random.choice(string.ascii_lowercase) for x in range(6))
challenge += ''.join(random.choice(string.digits) for x in range(6))
return challenge
def stats(**kwargs):
"""
Helper that defines some useful statistics about certifications.
:param kwargs:
:return:
"""
query = database.session_query(Certificate)
if kwargs.get('active') == 'true':
query = query.filter(Certificate.elb_listeners.any())
if kwargs.get('account_id'):
query = query.filter(Certificate.accounts.any(Account.id == kwargs.get('account_id')))
if kwargs.get('metric') == 'not_after':
start = arrow.utcnow()
end = start.replace(weeks=+32)
items = database.db.session.query(Certificate.issuer, func.count(Certificate.id))\
.group_by(Certificate.issuer)\
.filter(Certificate.not_after <= end.format('YYYY-MM-DD')) \
.filter(Certificate.not_after >= start.format('YYYY-MM-DD')).all()
else:
attr = getattr(Certificate, kwargs.get('metric'))
query = database.db.session.query(attr, func.count(attr))
# TODO this could be cleaned up
if kwargs.get('active') == 'true':
query = query.filter(Certificate.elb_listeners.any())
items = query.group_by(attr).all()
keys = []
values = []
for key, count in items:
keys.append(key)
values.append(count)
return {'labels': keys, 'values': values}

169
lemur/certificates/sync.py Normal file
View File

@ -0,0 +1,169 @@
"""
.. module: sync
:platform: Unix
:synopsis: This module contains various certificate syncing operations.
Because of the nature of the SSL environment there are multiple ways
a certificate could be created without Lemur's knowledge. Lemur attempts
to 'sync' with as many different datasources as possible to try and track
any certificate that may be in use.
This include querying AWS for certificates attached to ELBs, querying our own
internal CA for certificates issued. As well as some rudimentary source code
scraping that attempts to find certificates checked into source code.
These operations are typically run on a periodic basis from either the command
line or a cron job.
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
import requests
from bs4 import BeautifulSoup
from flask import current_app
from lemur.users import service as user_service
from lemur.accounts import service as account_service
from lemur.certificates import service as cert_service
from lemur.certificates.models import Certificate, get_name_from_arn
from lemur.common.services.aws.iam import get_all_server_certs
from lemur.common.services.aws.iam import get_cert_from_arn
from lemur.common.services.issuers.manager import get_plugin_by_name
def aws():
"""
Attempts to retrieve all certificates located in known AWS accounts
:raise e:
"""
new = 0
updated = 0
# all certificates 'discovered' by lemur are tracked by the lemur
# user
user = user_service.get_by_email('lemur@nobody')
# we don't need to check regions as IAM is a global service
for account in account_service.get_all():
certificate_bodies = []
try:
cert_arns = get_all_server_certs(account.account_number)
except Exception as e:
current_app.logger.error("Failed to to get Certificates from '{}/{}' reason {}".format(
account.label, account.account_number, e.message)
)
raise e
current_app.logger.info("found {} certs from '{}/{}' ... ".format(
len(cert_arns), account.account_number, account.label)
)
for cert in cert_arns:
cert_body = get_cert_from_arn(cert.arn)[0]
certificate_bodies.append(cert_body)
existing = cert_service.find_duplicates(cert_body)
if not existing:
cert_service.import_certificate(
**{'owner': 'secops@netflix.com',
'creator': 'Lemur',
'name': get_name_from_arn(cert.arn),
'account': account,
'user': user,
'public_certificate': cert_body
}
)
new += 1
elif len(existing) == 1: # we check to make sure we know about the current account for this certificate
for e_account in existing[0].accounts:
if e_account.account_number == account.account_number:
break
else: # we have a new account
existing[0].accounts.append(account)
updated += 1
else:
current_app.logger.error(
"Multiple certificates with the same body found, unable to correctly determine which entry to update"
)
# make sure we remove any certs that have been removed from AWS
cert_service.disassociate_aws_account(certificate_bodies, account)
current_app.logger.info("found {} new certificates in aws {}".format(new, account.label))
def cloudca():
"""
Attempts to retrieve all certificates that are stored in CloudCA
"""
user = user_service.get_by_email('lemur@nobody')
# sync all new certificates/authorities not created through lemur
issuer = get_plugin_by_name('cloudca')
authorities = issuer.get_authorities()
total = 0
new = 1
for authority in authorities:
certs = issuer.get_cert(ca_name=authority)
for cert in certs:
total += 1
cert['user'] = user
existing = cert_service.find_duplicates(cert['public_certificate'])
if not existing:
new += 1
try:
cert_service.import_certificate(**cert)
except NameError as e:
current_app.logger.error("Cannot import certificate {0}".format(cert))
current_app.logger.debug("Found {0} total certificates in cloudca".format(total))
current_app.logger.debug("Found {0} new certificates in cloudca".format(new))
def source():
"""
Attempts to track certificates that are stored in Source Code
"""
new = 0
keywords = ['"--- Begin Certificate ---"']
endpoint = current_app.config.get('LEMUR_SOURCE_SEARCH')
maxresults = 25000
current_app.logger.info("Searching {0} for new certificates".format(endpoint))
for keyword in keywords:
current_app.logger.info("Looking for keyword: {0}".format(keyword))
url = "{}/source/s?n={}&start=1&sort=relevancy&q={}&project=github%2Cperforce%2Cstash".format(endpoint, maxresults, keyword)
current_app.logger.debug("Request url: {0}".format(url))
r = requests.get(url, timeout=20)
if r.status_code != 200:
current_app.logger.error("Unable to retrieve: {0} Status Code: {1}".format(url, r.status_code))
continue
soup = BeautifulSoup(r.text, "lxml")
results = soup.find_all(title='Download')
for result in results:
parts = result['href'].split('/')
path = "/".join(parts[:-1])
filename = parts[-1:][0]
r = requests.get("{0}{1}/{2}".format(endpoint, path, filename))
if r.status_code != 200:
current_app.logger.error("Unable to retrieve: {0} Status Code: {1}".format(url, r.status_code))
continue
try:
# validate we have a real certificate
cert = Certificate(r.content)
# do a lookup to see if we know about this certificate
existing = cert_service.find_duplicates(r.content)
if not existing:
current_app.logger.debug(cert.name)
cert_service.import_certificate()
new += 1
except Exception as e:
current_app.logger.debug("Could not parse the following 'certificate': {0} Reason: {1}".format(r.content, e))

135
lemur/certificates/verify.py Normal file
View File

@ -0,0 +1,135 @@
"""
.. module: lemur.certificates.verify
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
import os
import re
import hashlib
import requests
import subprocess
from OpenSSL import crypto
from flask import current_app
def ocsp_verify(cert_path, issuer_chain_path):
"""
Attempts to verify a certificate via OCSP. OCSP is a more modern version
of CRL in that it will query the OCSP URI in order to determine if the
certificate as been revoked
:param cert_path:
:param issuer_chain_path:
:return bool: True if certificate is valid, False otherwise
"""
command = ['openssl', 'x509', '-noout', '-ocsp_uri', '-in', cert_path]
p1 = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
url, err = p1.communicate()
p2 = subprocess.Popen(['openssl', 'ocsp', '-issuer', issuer_chain_path,
'-cert', cert_path, "-url", url.strip()], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
message, err = p2.communicate()
if 'error' in message or 'Error' in message:
raise Exception("Got error when parsing OCSP url")
elif 'revoked' in message:
return
elif 'good' not in message:
raise Exception("Did not receive a valid response")
return True
def crl_verify(cert_path):
"""
Attempts to verify a certificate using CRL.
:param cert_path:
:return: True if certificate is valid, False otherwise
:raise Exception: If certificate does not have CRL
"""
s = "(http(s)?\://[a-zA-Z0-9\-\.]+\.[a-zA-Z]{2,3}/\S*?$)"
regex = re.compile(s, re.MULTILINE)
x509 = crypto.load_certificate(crypto.FILETYPE_PEM, open(cert_path, 'rt').read())
for x in range(x509.get_extension_count()):
ext = x509.get_extension(x)
if ext.get_short_name() == 'crlDistributionPoints':
r = regex.search(ext.get_data())
points = r.groups()
break
else:
raise Exception("Certificate does not have a CRL distribution point")
for point in points:
if point:
response = requests.get(point)
crl = crypto.load_crl(crypto.FILETYPE_ASN1, response.content)
revoked = crl.get_revoked()
for r in revoked:
if x509.get_serial_number() == r.get_serial():
return
return True
def verify(cert_path, issuer_chain_path):
"""
Verify a certificate using OCSP and CRL
:param cert_path:
:param issuer_chain_path:
:return: True if valid, False otherwise
"""
# OCSP is our main source of truth, in a lot of cases CRLs
# have been deprecated and are no longer updated
try:
return ocsp_verify(cert_path, issuer_chain_path)
except Exception as e:
current_app.logger.debug("Could not use OCSP: {0}".format(e))
try:
return crl_verify(cert_path)
except Exception as e:
current_app.logger.debug("Could not use CRL: {0}".format(e))
raise Exception("Failed to verify")
raise Exception("Failed to verify")
def make_tmp_file(string):
"""
Creates a temporary file for a given string
:param string:
:return: Full file path to created file
"""
m = hashlib.md5()
m.update(string)
hexdigest = m.hexdigest()
path = os.path.join(os.path.dirname(os.path.abspath(__file__)), hexdigest)
with open(path, 'w') as f:
f.write(string)
return path
def verify_string(cert_string, issuer_string):
"""
Verify a certificate given only it's string value
:param cert_string:
:param issuer_string:
:return: True if valid, False otherwise
"""
cert_path = make_tmp_file(cert_string)
issuer_path = make_tmp_file(issuer_string)
status = verify(cert_path, issuer_path)
remove_tmp_file(cert_path)
remove_tmp_file(issuer_path)
return status
def remove_tmp_file(file_path):
os.remove(file_path)

575
lemur/certificates/views.py Normal file
View File

@ -0,0 +1,575 @@
"""
.. module: lemur.certificates.views
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from flask import Blueprint, make_response, jsonify
from flask.ext.restful import reqparse, Api, fields
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from lemur.certificates import service
from lemur.authorities.models import Authority
from lemur.auth.service import AuthenticatedResource
from lemur.auth.permissions import ViewKeyPermission, AuthorityPermission, UpdateCertificatePermission
from lemur.roles import service as role_service
from lemur.common.utils import marshal_items, paginated_parser
mod = Blueprint('certificates', __name__)
api = Api(mod)
FIELDS = {
'name': fields.String,
'id': fields.Integer,
'bits': fields.Integer,
'deleted': fields.String,
'issuer': fields.String,
'serial': fields.String,
'owner': fields.String,
'chain': fields.String,
'san': fields.String,
'active': fields.Boolean,
'description': fields.String,
'notBefore': fields.DateTime(dt_format='iso8601', attribute='not_before'),
'notAfter': fields.DateTime(dt_format='iso8601', attribute='not_after'),
'cn': fields.String,
'status': fields.String,
'body': fields.String
}
def valid_authority(authority_options):
"""
Defends against invalid authorities
:param authority_name:
:return: :raise ValueError:
"""
name = authority_options['name']
authority = Authority.query.filter(Authority.name == name).one()
if not authority:
raise ValueError("Unable to find authority specified")
if not authority.active:
raise ValueError("Selected authority [{0}] is not currently active".format(name))
return authority
def pem_str(value, name):
"""
Used to validate that the given string is a PEM formatted string
:param value:
:param name:
:return: :raise ValueError:
"""
try:
x509.load_pem_x509_certificate(str(value), default_backend())
except Exception as e:
raise ValueError("The parameter '{0}' needs to be a valid PEM string".format(name))
return value
def private_key_str(value, name):
"""
User to validate that a given string is a RSA private key
:param value:
:param name:
:return: :raise ValueError:
"""
try:
serialization.load_pem_private_key(str(value), backend=default_backend())
except Exception as e:
raise ValueError("The parameter '{0}' needs to be a valid RSA private key".format(name))
return value
class CertificatesList(AuthenticatedResource):
""" Defines the 'certificates' endpoint """
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(CertificatesList, self).__init__()
@marshal_items(FIELDS)
def get(self):
"""
.. http:get:: /certificates
The current list of certificates
**Example request**:
.. sourcecode:: http
GET /certificates HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"items": [
{
"id": 1,
"name": "cert1",
"description": "this is cert1",
"bits": 2048,
"deleted": false,
"issuer": "ExampeInc.",
"serial": "123450",
"chain": "-----Begin ...",
"body": "-----Begin ...",
"san": true,
"owner": 'bob@example.com",
"active": true,
"notBefore": "2015-06-05T17:09:39",
"notAfter": "2015-06-10T17:09:39",
"cn": "example.com",
"status": "unknown"
}
]
"total": 1
}
:query sortBy: field to sort on
:query sortDir: acs or desc
:query page: int. default is 1
:query filter: key value pair. format is k=v;
:query limit: limit number. default is 10
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
"""
parser = paginated_parser.copy()
parser.add_argument('timeRange', type=int, dest='time_range', location='args')
parser.add_argument('owner', type=bool, location='args')
parser.add_argument('id', type=str, location='args')
parser.add_argument('active', type=bool, location='args')
parser.add_argument('accountId', type=int, dest="account_id", location='args')
parser.add_argument('creator', type=str, location='args')
parser.add_argument('show', type=str, location='args')
args = parser.parse_args()
return service.render(args)
@marshal_items(FIELDS)
def post(self):
"""
.. http:post:: /certificates
Creates a new certificate
**Example request**:
.. sourcecode:: http
POST /certificates HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
{
"country": "US",
"state": "CA",
"location": "A Place",
"organization": "ExampleInc.",
"organizationalUnit": "Operations",
"owner": "bob@example.com",
"description": "test",
"selectedAuthority": "timetest2",
"authority": {
"body": "-----BEGIN...",
"name": "timetest2",
"chain": "",
"notBefore": "2015-06-05T15:20:59",
"active": true,
"id": 50,
"notAfter": "2015-06-17T15:21:08",
"description": "dsfdsf"
},
"extensions": {
"basicConstraints": {},
"keyUsage": {
"isCritical": true,
"useKeyEncipherment": true,
"useDigitalSignature": true
},
"extendedKeyUsage": {
"isCritical": true,
"useServerAuthentication": true
},
"subjectKeyIdentifier": {
"includeSKI": true
},
"subAltNames": {
"names": []
}
},
"commonName": "test",
"validityStart": "2015-06-05T07:00:00.000Z",
"validityEnd": "2015-06-16T07:00:00.000Z"
}
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"id": 1,
"name": "cert1",
"description": "this is cert1",
"bits": 2048,
"deleted": false,
"issuer": "ExampeInc.",
"serial": "123450",
"chain": "-----Begin ...",
"body": "-----Begin ...",
"san": true,
"owner": "jimbob@example.com",
"active": false,
"notBefore": "2015-06-05T17:09:39",
"notAfter": "2015-06-10T17:09:39",
"cn": "example.com",
"status": "unknown"
}
:arg extensions: extensions to be used in the certificate
:arg description: description for new certificate
:arg owner: owner email
:arg validityStart: when the certificate should start being valid
:arg validityEnd: when the certificate should expire
:arg authority: authority that should issue the certificate
:arg country: country for the CSR
:arg state: state for the CSR
:arg location: location for the CSR
:arg organization: organization for CSR
:arg commonName: certiifcate common name
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
"""
self.reqparse.add_argument('extensions', type=dict, location='json')
self.reqparse.add_argument('accounts', type=list, location='json')
self.reqparse.add_argument('elbs', type=list, location='json')
self.reqparse.add_argument('owner', type=str, location='json')
self.reqparse.add_argument('validityStart', type=str, location='json') # parse date
self.reqparse.add_argument('validityEnd', type=str, location='json') # parse date
self.reqparse.add_argument('authority', type=valid_authority, location='json')
self.reqparse.add_argument('description', type=str, location='json')
self.reqparse.add_argument('country', type=str, location='json')
self.reqparse.add_argument('state', type=str, location='json')
self.reqparse.add_argument('location', type=str, location='json')
self.reqparse.add_argument('organization', type=str, location='json')
self.reqparse.add_argument('organizationalUnit', type=str, location='json')
self.reqparse.add_argument('owner', type=str, location='json')
self.reqparse.add_argument('commonName', type=str, location='json')
args = self.reqparse.parse_args()
authority = args['authority']
role = role_service.get_by_name(authority.owner)
# all the authority role members should be allowed
roles = [x.name for x in authority.roles]
# allow "owner" roles by team DL
roles.append(role)
permission = AuthorityPermission(authority.id, roles)
if permission.can():
return service.create(**args)
return dict(message="You are not authorized to use {0}".format(args['authority'].name)), 403
class CertificatesUpload(AuthenticatedResource):
""" Defines the 'certificates' upload endpoint """
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(CertificatesUpload, self).__init__()
@marshal_items(FIELDS)
def post(self):
"""
.. http:post:: /certificates/upload
Upload a certificate
**Example request**:
.. sourcecode:: http
POST /certificates/upload HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
{
"owner": "joe@exmaple.com",
"publicCert": "---Begin Public...",
"intermediateCert": "---Begin Public...",
"privateKey": "---Begin Private..."
"accounts": []
}
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"id": 1,
"name": "cert1",
"description": "this is cert1",
"bits": 2048,
"deleted": false,
"issuer": "ExampeInc.",
"serial": "123450",
"chain": "-----Begin ...",
"body": "-----Begin ...",
"san": true,
"owner": "joe@example.com",
"active": true,
"notBefore": "2015-06-05T17:09:39",
"notAfter": "2015-06-10T17:09:39",
"cn": "example.com",
"status": "unknown"
}
:arg owner: owner email for certificate
:arg publicCert: valid PEM public key for certificate
:arg intermediateCert valid PEM intermediate key for certificate
:arg privateKey: valid PEM private key for certificate
:arg accounts: list of aws accounts to upload the certificate to
:reqheader Authorization: OAuth token to authenticate
:statuscode 403: unauthenticated
:statuscode 200: no error
"""
self.reqparse.add_argument('owner', type=str, required=True, location='json')
self.reqparse.add_argument('publicCert', type=pem_str, required=True, dest='public_cert', location='json')
self.reqparse.add_argument('accounts', type=list, dest='accounts', location='json')
self.reqparse.add_argument('intermediateCert', type=pem_str, dest='intermediate_cert', location='json')
self.reqparse.add_argument('privateKey', type=private_key_str, dest='private_key', location='json')
args = self.reqparse.parse_args()
if args.get('accounts'):
if args.get('private_key'):
return service.upload(**args)
else:
raise Exception("Private key must be provided in order to upload certificate to AWS")
return service.upload(**args)
class CertificatesStats(AuthenticatedResource):
""" Defines the 'certificates' stats endpoint """
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(CertificatesStats, self).__init__()
def get(self):
self.reqparse.add_argument('metric', type=str, location='args')
self.reqparse.add_argument('range', default=32, type=int, location='args')
self.reqparse.add_argument('accountId', dest='account_id', location='args')
self.reqparse.add_argument('active', type=str, default='true', location='args')
args = self.reqparse.parse_args()
items = service.stats(**args)
return dict(items=items, total=len(items))
class CertificatePrivateKey(AuthenticatedResource):
def __init__(self):
super(CertificatePrivateKey, self).__init__()
def get(self, certificate_id):
"""
.. http:get:: /certificates/1/key
Retrieves the private key for a given certificate
**Example request**:
.. sourcecode:: http
GET /certificates/1/key HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"key": "----Begin ...",
}
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
"""
cert = service.get(certificate_id)
role = role_service.get_by_name(cert.owner)
permission = ViewKeyPermission(certificate_id, hasattr(role, 'id'))
if permission.can():
response = make_response(jsonify(key=cert.private_key), 200)
response.headers['cache-control'] = 'private, max-age=0, no-cache, no-store'
response.headers['pragma'] = 'no-cache'
return response
return dict(message='You are not authorized to view this key'), 403
class Certificates(AuthenticatedResource):
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(Certificates, self).__init__()
@marshal_items(FIELDS)
def get(self, certificate_id):
"""
.. http:get:: /certificates/1
One certificate
**Example request**:
.. sourcecode:: http
GET /certificates/1 HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"id": 1,
"name": "cert1",
"description": "this is cert1",
"bits": 2048,
"deleted": false,
"issuer": "ExampeInc.",
"serial": "123450",
"chain": "-----Begin ...",
"body": "-----Begin ...",
"san": true,
"owner": "bob@example.com",
"active": true,
"notBefore": "2015-06-05T17:09:39",
"notAfter": "2015-06-10T17:09:39",
"cn": "example.com",
"status": "unknown"
}
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
"""
return service.get(certificate_id)
@marshal_items(FIELDS)
def put(self, certificate_id):
"""
.. http:put:: /certificates/1
Update a certificate
**Example request**:
.. sourcecode:: http
PUT /certificates/1 HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
{
"owner": "jimbob@example.com",
"active": false
}
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"id": 1,
"name": "cert1",
"description": "this is cert1",
"bits": 2048,
"deleted": false,
"issuer": "ExampeInc.",
"serial": "123450",
"chain": "-----Begin ...",
"body": "-----Begin ...",
"san": true,
"owner": "jimbob@example.com",
"active": false,
"notBefore": "2015-06-05T17:09:39",
"notAfter": "2015-06-10T17:09:39",
"cn": "example.com",
"status": "unknown"
}
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
"""
self.reqparse.add_argument('active', type=bool, location='json')
self.reqparse.add_argument('owner', type=str, location='json')
args = self.reqparse.parse_args()
cert = service.get(certificate_id)
role = role_service.get_by_name(cert.owner)
permission = UpdateCertificatePermission(certificate_id, hasattr(role, 'id'))
if permission.can():
return service.update(certificate_id, args['owner'], args['active'])
return dict(message='You are not authorized to update this certificate'), 403
api.add_resource(CertificatesList, '/certificates', endpoint='certificates')
api.add_resource(Certificates, '/certificates/<int:certificate_id>', endpoint='certificate')
api.add_resource(CertificatesStats, '/certificates/stats', endpoint='certificateStats')
api.add_resource(CertificatesUpload, '/certificates/upload', endpoint='certificateUpload')
api.add_resource(CertificatePrivateKey, '/certificates/<int:certificate_id>/key', endpoint='privateKeyCertificates')

0
lemur/common/__init__.py Normal file
View File

185
lemur/common/crypto.py Normal file
View File

@ -0,0 +1,185 @@
"""
.. module: lemur.common.crypto
:platform: Unix
:synopsis: This module contains all cryptographic function's in Lemur
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
import os
import ssl
import StringIO
import functools
from Crypto import Random
from Crypto.Cipher import AES
from hashlib import sha512
from flask import current_app
from lemur.factory import create_app
old_init = ssl.SSLSocket.__init__
@functools.wraps(old_init)
def ssl_bug(self, *args, **kwargs):
kwargs['ssl_version'] = ssl.PROTOCOL_TLSv1
old_init(self, *args, **kwargs)
ssl.SSLSocket.__init__ = ssl_bug
def derive_key_and_iv(password, salt, key_length, iv_length):
"""
Derives the key and iv from the password and salt.
:param password:
:param salt:
:param key_length:
:param iv_length:
:return: key, iv
"""
d = d_i = ''
while len(d) < key_length + iv_length:
d_i = sha512(d_i + password + salt).digest()
d += d_i
return d[:key_length], d[key_length:key_length+iv_length]
def encrypt(in_file, out_file, password, key_length=32):
"""
Encrypts a file.
:param in_file:
:param out_file:
:param password:
:param key_length:
"""
bs = AES.block_size
salt = Random.new().read(bs - len('Salted__'))
key, iv = derive_key_and_iv(password, salt, key_length, bs)
cipher = AES.new(key, AES.MODE_CBC, iv)
out_file.write('Salted__' + salt)
finished = False
while not finished:
chunk = in_file.read(1024 * bs)
if len(chunk) == 0 or len(chunk) % bs != 0:
padding_length = bs - (len(chunk) % bs)
chunk += padding_length * chr(padding_length)
finished = True
out_file.write(cipher.encrypt(chunk))
def decrypt(in_file, out_file, password, key_length=32):
"""
Decrypts a file.
:param in_file:
:param out_file:
:param password:
:param key_length:
:raise ValueError:
"""
bs = AES.block_size
salt = in_file.read(bs)[len('Salted__'):]
key, iv = derive_key_and_iv(password, salt, key_length, bs)
cipher = AES.new(key, AES.MODE_CBC, iv)
next_chunk = ''
finished = False
while not finished:
chunk, next_chunk = next_chunk, cipher.decrypt(in_file.read(1024 * bs))
if len(next_chunk) == 0:
padding_length = ord(chunk[-1])
if padding_length < 1 or padding_length > bs:
raise ValueError("bad decrypt pad (%d)" % padding_length)
# all the pad-bytes must be the same
if chunk[-padding_length:] != (padding_length * chr(padding_length)):
# this is similar to the bad decrypt:evp_enc.c from openssl program
raise ValueError("bad decrypt")
chunk = chunk[:-padding_length]
finished = True
out_file.write(chunk)
def encrypt_string(string, password):
"""
Encrypts a string.
:param string:
:param password:
:return:
"""
in_file = StringIO.StringIO(string)
enc_file = StringIO.StringIO()
encrypt(in_file, enc_file, password)
enc_file.seek(0)
return enc_file.read()
def decrypt_string(string, password):
"""
Decrypts a string.
:param string:
:param password:
:return:
"""
in_file = StringIO.StringIO(string)
out_file = StringIO.StringIO()
decrypt(in_file, out_file, password)
out_file.seek(0)
return out_file.read()
def lock(password):
"""
Encrypts Lemur's KEY_PATH. This directory can be used to store secrets needed for normal
Lemur operation. This is especially useful for storing secrets needed for communication
with third parties (e.g. external certificate authorities).
Lemur does not assume anything about the contents of the directory and will attempt to
encrypt all files contained within. Currently this has only been tested against plain
text files.
:param password:
"""
dest_dir = os.path.join(current_app.config.get("KEY_PATH"), "encrypted")
if not os.path.exists(dest_dir):
current_app.logger.debug("Creating encryption directory: {0}".format(dest_dir))
os.makedirs(dest_dir)
for root, dirs, files in os.walk(os.path.join(current_app.config.get("KEY_PATH"), 'decrypted')):
for f in files:
source = os.path.join(root, f)
dest = os.path.join(dest_dir, f + ".enc")
with open(source, 'rb') as in_file, open(dest, 'wb') as out_file:
encrypt(in_file, out_file, password)
def unlock(password):
"""
Decrypts Lemur's KEY_PATH, allowing lemur to use the secrets within.
This reverses the :func:`lock` function.
:param password:
"""
dest_dir = os.path.join(current_app.config.get("KEY_PATH"), "decrypted")
source_dir = os.path.join(current_app.config.get("KEY_PATH"), "encrypted")
if not os.path.exists(dest_dir):
current_app.logger.debug("Creating decryption directory: {0}".format(dest_dir))
os.makedirs(dest_dir)
for root, dirs, files in os.walk(source_dir):
for f in files:
source = os.path.join(source_dir, f)
dest = os.path.join(dest_dir, ".".join(f.split(".")[:-1]))
with open(source, 'rb') as in_file, open(dest, 'wb') as out_file:
current_app.logger.debug("Writing file: {0} Source: {1}".format(dest, source))
decrypt(in_file, out_file, password)

15
lemur/common/health.py Normal file
View File

@ -0,0 +1,15 @@
"""
.. module: lemur.common.health
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from flask import Blueprint
mod = Blueprint('healthCheck', __name__)
@mod.route('/healthcheck')
def health():
return 'ok'

0
lemur/common/services/__init__.py Normal file
View File

0
lemur/common/services/aws/__init__.py Normal file
View File

140
lemur/common/services/aws/elb.py Normal file
View File

@ -0,0 +1,140 @@
"""
.. module:: elb
:synopsis: Module contains some often used and helpful classes that
are used to deal with ELBs
.. moduleauthor:: Kevin Glisson (kglisson@netflix.com)
"""
import boto.ec2
from flask import current_app
from lemur.exceptions import InvalidListener
from lemur.common.services.aws.sts import assume_service
def is_valid(listener_tuple):
"""
There are a few rules that aws has when creating listeners,
this function ensures those rules are met before we try and create
or update a listener.
While these could be caught with boto exception handling, I would
rather be nice and catch these early before we sent them out to aws.
It also gives us an opportunity to create nice user warnings.
This validity check should also be checked in the frontend
but must also be enforced by server.
:param listener_tuple:
"""
current_app.logger.debug(listener_tuple)
lb_port, i_port, lb_protocol, arn = listener_tuple
current_app.logger.debug(lb_protocol)
if lb_protocol.lower() in ['ssl', 'https']:
if not arn:
raise InvalidListener
return listener_tuple
def get_all_regions():
"""
Retrieves all current EC2 regions.
:return:
"""
regions = []
for r in boto.ec2.regions():
regions.append(r.name)
return regions
def get_all_elbs(account_number, region):
"""
Fetches all elb objects for a given account and region.
:param account_number:
:param region:
"""
marker = None
elbs = []
return assume_service(account_number, 'elb', region).get_all_load_balancers()
# TODO create pull request for boto to include elb marker support
# while True:
# app.logger.debug(response.__dict__)
# raise Exception
# result = response['list_server_certificates_response']['list_server_certificates_result']
#
# for elb in result['server_certificate_metadata_list']:
# elbs.append(elb)
#
# if result['is_truncated'] == 'true':
# marker = result['marker']
# else:
# return elbs
def attach_certificate(account_number, region, name, port, certificate_id):
"""
Attaches a certificate to a listener, throws exception
if certificate specified does not exist in a particular account.
:param account_number:
:param region:
:param name:
:param port:
:param certificate_id:
"""
return assume_service(account_number, 'elb', region).set_lb_listener_SSL_certificate(name, port, certificate_id)
def create_new_listeners(account_number, region, name, listeners=None):
"""
Creates a new listener and attaches it to the ELB.
:param account_number:
:param region:
:param name:
:param listeners:
:return:
"""
listeners = [is_valid(x) for x in listeners]
return assume_service(account_number, 'elb', region).create_load_balancer_listeners(name, listeners=listeners)
def update_listeners(account_number, region, name, listeners, ports):
"""
We assume that a listener with a specified port already exists. We can then
delete the old listener on the port and create a new one in it's place.
If however we are replacing a listener e.g. changing a port from 80 to 443 we need
to make sure we kept track of which ports we needed to delete so that we don't create
two listeners (one 80 and one 443)
:param account_number:
:param region:
:param name:
:param listeners:
:param ports:
"""
# you cannot update a listeners port/protocol instead we remove the only one and
# create a new one in it's place
listeners = [is_valid(x) for x in listeners]
assume_service(account_number, 'elb', region).delete_load_balancer_listeners(name, ports)
return create_new_listeners(account_number, region, name, listeners=listeners)
def delete_listeners(account_number, region, name, ports):
"""
Deletes a listener from an ELB.
:param account_number:
:param region:
:param name:
:param ports:
:return:
"""
return assume_service(account_number, 'elb', region).delete_load_balancer_listeners(name, ports)

104
lemur/common/services/aws/iam.py Normal file
View File

@ -0,0 +1,104 @@
"""
.. module: lemur.common.services.aws.iam
:platform: Unix
:synopsis: Contains helper functions for interactive with AWS IAM Apis.
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from flask import current_app
from lemur.common.services.aws.sts import assume_service
def ssl_split(param_string):
"""
:param param_string:
:return:
"""
output = {}
parts = str(param_string).split("/")
for part in parts:
if "=" in part:
key, value = part.split("=", 1)
output[key] = value
return output
def upload_cert(account_number, cert, private_key, cert_chain=None):
"""
Upload a certificate to AWS
:param account_number:
:param cert:
:param private_key:
:param cert_chain:
:return:
"""
return assume_service(account_number, 'iam').upload_server_cert(cert.name, str(cert.body), str(private_key), cert_chain=str(cert_chain))
def delete_cert(account_number, cert):
"""
Delete a certificate from AWS
:param account_number:
:param cert:
:return:
"""
return assume_service(account_number, 'iam').delete_server_cert(cert.name)
def get_all_server_certs(account_number):
"""
Use STS to fetch all of the SSL certificates from a given account
:param account_number:
"""
marker = None
certs = []
while True:
response = assume_service(account_number, 'iam').get_all_server_certs(marker=marker)
result = response['list_server_certificates_response']['list_server_certificates_result']
for cert in result['server_certificate_metadata_list']:
certs.append(cert)
if result['is_truncated'] == 'true':
marker = result['marker']
else:
return certs
def get_cert_from_arn(arn):
"""
Retrieves an SSL certificate from a given ARN.
:param arn:
:return:
"""
name = arn.split("/", 1)[1]
account_number = arn.split(":")[4]
name = name.split("/")[-1]
response = assume_service(account_number, 'iam').get_server_certificate(name.strip())
return digest_aws_cert_response(response)
def digest_aws_cert_response(response):
"""
Processes an AWS certifcate response and retrieves the certificate body and chain.
:param response:
:return:
"""
chain = None
cert = response['get_server_certificate_response']['get_server_certificate_result']['server_certificate']
body = cert['certificate_body']
if 'certificate_chain' in cert:
chain = cert['certificate_chain']
return str(body), str(chain),

29
lemur/common/services/aws/ses.py Normal file
View File

@ -0,0 +1,29 @@
"""
.. module: lemur.common.services.aws
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from flask import current_app
import boto.ses
from lemur.templates.config import env
def send(subject, data, email_type, recipients):
"""
Configures all Lemur email messaging
:param subject:
:param data:
:param email_type:
:param recipients:
"""
conn = boto.connect_ses()
#jinja template depending on type
template = env.get_template('{}.html'.format(email_type))
body = template.render(**data)
conn.send_email(current_app.config.get("LEMUR_EMAIL"), subject, body, recipients, format='html')

41
lemur/common/services/aws/sts.py Normal file
View File

@ -0,0 +1,41 @@
"""
.. module: lemur.common.services.aws.sts
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
import boto
import boto.ec2.elb
from flask import current_app
def assume_service(account_number, service, region=None):
conn = boto.connect_sts()
role = conn.assume_role('arn:aws:iam::{0}:role/{1}'.format(
account_number, current_app.config.get('LEMUR_INSTANCE_PROFILE', 'Lemur')), 'blah')
if service in 'iam':
return boto.connect_iam(
aws_access_key_id=role.credentials.access_key,
aws_secret_access_key=role.credentials.secret_key,
security_token=role.credentials.session_token)
elif service in 'elb':
return boto.ec2.elb.connect_to_region(
region,
aws_access_key_id=role.credentials.access_key,
aws_secret_access_key=role.credentials.secret_key,
security_token=role.credentials.session_token)
elif service in 'vpc':
return boto.connect_vpc(
aws_access_key_id=role.credentials.access_key,
aws_secret_access_key=role.credentials.secret_key,
security_token=role.credentials.session_token)

0
lemur/common/services/issuers/__init__.py Normal file
View File

32
lemur/common/services/issuers/issuer.py Normal file
View File

@ -0,0 +1,32 @@
"""
.. module: authority
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from flask import current_app
class Issuer(object):
"""
This is the base class from which all of the supported
issuers will inherit from.
"""
def __init__(self):
self.dry_run = current_app.config.get('DRY_RUN')
def create_certificate(self):
raise NotImplementedError
def create_authority(self):
raise NotImplementedError
def get_authorities(self):
raise NotImplementedError
def get_csr_config(self):
raise NotImplementedError

37
lemur/common/services/issuers/manager.py Normal file
View File

@ -0,0 +1,37 @@
"""
.. module: lemur.common.services.issuers.manager
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson (kglisson@netflix.com)
"""
import pkgutil
from importlib import import_module
from flask import current_app
from lemur.common.services.issuers import plugins
# TODO make the plugin dir configurable
def get_plugin_by_name(plugin_name):
"""
Fetches a given plugin by it's name. We use a known location for issuer plugins and attempt
to load it such that it can be used for issuing certificates.
:param plugin_name:
:return: a plugin `class` :raise Exception: Generic error whenever the plugin specified can not be found.
"""
for importer, modname, ispkg in pkgutil.iter_modules(plugins.__path__):
try:
issuer = import_module('lemur.common.services.issuers.plugins.{0}.{0}'.format(modname))
if issuer.__name__ == plugin_name:
# we shouldn't return bad issuers
issuer_obj = issuer.init()
return issuer_obj
except Exception as e:
current_app.logger.warn("Issuer {0} was unable to be imported: {1}".format(modname, e))
else:
raise Exception("Could not find the specified plugin: {0}".format(plugin_name))

0
lemur/common/services/issuers/plugins/__init__.py Normal file
View File

5
lemur/common/services/issuers/plugins/cloudca/__init__.py Normal file
View File

@ -0,0 +1,5 @@
try:
VERSION = __import__('pkg_resources') \
.get_distribution(__name__).version
except Exception, e:
VERSION = 'unknown'

346
lemur/common/services/issuers/plugins/cloudca/cloudca.py Normal file
View File

@ -0,0 +1,346 @@
"""
.. module: lemur.common.services.issuers.plugins.cloudca
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
import ssl
import base64
from json import dumps
import arrow
import requests
from requests.adapters import HTTPAdapter
from flask import current_app
from lemur.exceptions import LemurException
from lemur.common.services.issuers.issuer import Issuer
from lemur.common.services.issuers.plugins import cloudca
from lemur.authorities import service as authority_service
API_ENDPOINT = '/v1/ca/netflix'
class CloudCAException(LemurException):
def __init__(self, message):
self.message = message
current_app.logger.error(self)
def __str__(self):
return repr("CloudCA request failed: {0}".format(self.message))
class CloudCAHostNameCheckingAdapter(HTTPAdapter):
def cert_verify(self, conn, url, verify, cert):
super(CloudCAHostNameCheckingAdapter, self).cert_verify(conn, url, verify, cert)
conn.assert_hostname = False
def remove_none(options):
"""
Simple function that traverse the options and removed any None items
CloudCA really dislikes null values.
:param options:
:return:
"""
new_dict = {}
for k, v in options.items():
if v:
new_dict[k] = v
# this is super hacky and gross, cloudca doesn't like null values
if new_dict.get('extensions'):
if len(new_dict['extensions']['subAltNames']['names']) == 0:
del new_dict['extensions']['subAltNames']
return new_dict
def get_default_issuance(options):
"""
Gets the default time range for certificates
:param options:
:return:
"""
if not options.get('validityStart') and not options.get('validityEnd'):
start = arrow.utcnow()
options['validityStart'] = start.floor('second').isoformat()
options['validityEnd'] = start.replace(years=current_app.config.get('CLOUDCA_DEFAULT_VALIDITY')).ceil('second').isoformat()
return options
def convert_to_pem(der):
"""
Converts DER to PEM Lemur uses PEM internally
:param der:
:return:
"""
decoded = base64.b64decode(der)
return ssl.DER_cert_to_PEM_cert(decoded)
def convert_date_to_utc_time(date):
"""
Converts a python `datetime` object to the current date + current time in UTC.
:param date:
:return:
"""
d = arrow.get(date)
return arrow.utcnow().replace(day=d.naive.day).replace(month=d.naive.month).replace(year=d.naive.year).replace(microsecond=0)
def process_response(response):
"""
Helper function that processes responses from CloudCA.
:param response:
:return: :raise CloudCAException:
"""
if response.status_code == 200:
res = response.json()
if res['returnValue'] != 'success':
current_app.logger.debug(res)
if res.get('data'):
raise CloudCAException(" ".join([res['returnMessage'], res['data']['dryRunResultMessage']]))
else:
raise CloudCAException(res['returnMessage'])
else:
raise CloudCAException("There was an error with your request: {0}".format(response.status_code))
return response.json()
def get_auth_data(ca_name):
"""
Creates the authentication record needed to authenticate a user request to CloudCA.
:param ca_name:
:return: :raise CloudCAException:
"""
role = authority_service.get_authority_role(ca_name)
if role:
return {
"authInfo": {
"credType": "password",
"credentials": {
"username": role.username,
"password": role.password # we only decrypt when we need to
}
}
}
raise CloudCAException("You do not have the required role to issue certificates from {0}".format(ca_name))
class CloudCA(Issuer):
title = 'CloudCA'
slug = 'cloudca'
description = 'Enables the creation of certificates from the cloudca API.'
version = cloudca.VERSION
author = 'Kevin Glisson'
author_url = 'https://github.com/netflix/lemur'
def __init__(self, *args, **kwargs):
self.session = requests.Session()
self.session.mount('https://', CloudCAHostNameCheckingAdapter())
self.url = current_app.config.get('CLOUDCA_URL')
if current_app.config.get('CLOUDCA_PEM_PATH') and current_app.config.get('CLOUDCA_BUNDLE'):
self.session.cert = current_app.config.get('CLOUDCA_PEM_PATH')
self.ca_bundle = current_app.config.get('CLOUDCA_BUNDLE')
else:
current_app.logger.warning("No CLOUDCA credentials found, lemur will be unable to request certificates from CLOUDCA")
super(CloudCA, self).__init__(*args, **kwargs)
def create_authority(self, options):
"""
Creates a new certificate authority
:param options:
:return:
"""
# this is weird and I don't like it
endpoint = '{0}/createCA'.format(API_ENDPOINT)
options['caDN']['email'] = options['ownerEmail']
if options['caType'] == 'subca':
options = dict(options.items() + self.auth_data(options['caParent']).items())
options['validityStart'] = convert_date_to_utc_time(options['validityStart']).isoformat()
options['validityEnd'] = convert_date_to_utc_time(options['validityEnd']).isoformat()
response = self.session.post(self.url + endpoint, data=dumps(remove_none(options)), timeout=10, verify=self.ca_bundle)
json = process_response(response)
roles = []
for cred in json['data']['authInfo']:
role = {
'username': cred['credentials']['username'],
'password': cred['credentials']['password'],
'name': "_".join([options['caName'], cred['credentials']['username']])
}
roles.append(role)
if options['caType'] == 'subca':
cert = convert_to_pem(json['data']['certificate'])
else:
cert = convert_to_pem(json['data']['rootCertificate'])
intermediates = []
for i in json['data']['intermediateCertificates']:
intermediates.append(convert_to_pem(i))
return cert, "".join(intermediates), roles,
def get_authorities(self):
"""
Retrieves authorities that were made outside of Lemur.
:return:
"""
endpoint = '{0}/listCAs'.format(API_ENDPOINT)
authorities = []
for ca in self.get(endpoint)['data']['caList']:
try:
authorities.append(ca['caName'])
except AttributeError as e:
current_app.logger.error("No authority has been defined for {}".format(ca['caName']))
return authorities
def create_certificate(self, csr, options):
"""
Creates a new certificate from cloudca
If no start and end date are specified the default issue range
will be used.
:param csr:
:param options:
"""
endpoint = '{0}/enroll'.format(API_ENDPOINT)
# lets default to two years if it's not specified
# we do some last minute data massaging
options = get_default_issuance(options)
cloudca_options = {
'extensions': options['extensions'],
'validityStart': convert_date_to_utc_time(options['validityStart']).isoformat(),
'validityEnd': convert_date_to_utc_time(options['validityEnd']).isoformat(),
'creator': options['creator'],
'ownerEmail': options['owner'],
'caName': options['authority'].name,
'csr': csr,
'comment': options['description']
}
response = self.post(endpoint, remove_none(cloudca_options))
# we return a concatenated list of intermediate because that is what aws
# expects
cert = convert_to_pem(response['data']['certificate'])
intermediates = [convert_to_pem(response['data']['rootCertificate'])]
for i in response['data']['intermediateCertificates']:
intermediates.append(convert_to_pem(i))
return cert, "".join(intermediates),
def get_csr_config(self, issuer_options):
"""
Get a valid CSR for use with CloudCA
:param issuer_options:
:return:
"""
return cloudca.constants.CSR_CONFIG.format(**issuer_options)
def random(self, length=10):
"""
Uses CloudCA as a decent source of randomness.
:param length:
:return:
"""
endpoint = '/v1/random/{0}'.format(length)
response = self.session.get(self.url + endpoint, verify=self.ca_bundle)
return response
def get_cert(self, ca_name=None, cert_handle=None):
"""
Returns a given cert from CloudCA.
:param ca_name:
:param cert_handle:
:return:
"""
endpoint = '{0}/getCert'.format(API_ENDPOINT)
response = self.session.post(self.url + endpoint, data=dumps({'caName': ca_name}), timeout=10, verify=self.ca_bundle)
raw = process_response(response)
certs = []
for c in raw['data']['certList']:
cert = convert_to_pem(c['certValue'])
intermediates = []
for i in c['intermediateCertificates']:
intermediates.append(convert_to_pem(i))
certs.append({
'public_certificate': cert,
'intermediate_cert': "\n".join(intermediates),
'owner': c['ownerEmail']
})
return certs
def post(self, endpoint, data):
"""
HTTP POST to CloudCA
:param endpoint:
:param data:
:return:
"""
if self.dry_run:
endpoint += '?dry_run=1'
data = dumps(dict(data.items() + get_auth_data(data['caName']).items()))
# we set a low timeout, if cloudca is down it shouldn't bring down
# lemur
response = self.session.post(self.url + endpoint, data=data, timeout=10, verify=self.ca_bundle)
return process_response(response)
def get(self, endpoint):
"""
HTTP GET to CloudCA
:param endpoint:
:return:
"""
if self.dry_run:
endpoint += '?dry_run=1'
response = self.session.get(self.url + endpoint, timeout=10, verify=self.ca_bundle)
return process_response(response)
def init():
return CloudCA()

27
lemur/common/services/issuers/plugins/cloudca/constants.py Normal file
View File

@ -0,0 +1,27 @@
CSR_CONFIG = """
# Configuration for standard CSR generation for Netflix
# Used for procuring CloudCA certificates
# Author: kglisson
# Contact: secops@netflix.com
[ req ]
# Use a 2048 bit private key
default_bits = 2048
default_keyfile = key.pem
prompt = no
encrypt_key = no
# base request
distinguished_name = req_distinguished_name
# distinguished_name
[ req_distinguished_name ]
countryName = "{country}" # C=
stateOrProvinceName = "{state}" # ST=
localityName = "{location}" # L=
organizationName = "{organization}" # O=
organizationalUnitName = "{organizationalUnit}" # OU=
# This is the hostname/subject name on the certificate
commonName = "{commonName}" # CN=
"""

5
lemur/common/services/issuers/plugins/verisign/__init__.py Normal file
View File

@ -0,0 +1,5 @@
try:
VERSION = __import__('pkg_resources') \
.get_distribution(__name__).version
except Exception, e:
VERSION = 'unknown'

159
lemur/common/services/issuers/plugins/verisign/constants.py Normal file
View File

@ -0,0 +1,159 @@
CSR_CONFIG = """
# Configuration for standard CSR generation for Netflix
# Used for procuring VeriSign certificates
# Author: jachan
# Contact: cloudsecurity@netflix.com
[ req ]
# Use a 2048 bit private key
default_bits = 2048
default_keyfile = key.pem
prompt = no
encrypt_key = no
# base request
distinguished_name = req_distinguished_name
# extensions
# Uncomment the following line if you are requesting a SAN cert
{is_san_comment}req_extensions = req_ext
# distinguished_name
[ req_distinguished_name ]
countryName = "US" # C=
stateOrProvinceName = "CALIFORNIA" # ST=
localityName = "Los Gatos" # L=
organizationName = "Netflix, Inc." # O=
organizationalUnitName = "{OU}" # OU=
# This is the hostname/subject name on the certificate
commonName = "{DNS[0]}" # CN=
[ req_ext ]
# Uncomment the following line if you are requesting a SAN cert
{is_san_comment}subjectAltName = @alt_names
[alt_names]
# Put your SANs here
{DNS_LINES}
"""
VERISIGN_INTERMEDIATE = """
-----BEGIN CERTIFICATE-----
MIIFFTCCA/2gAwIBAgIQKC4nkXkzkuQo8iGnTsk3rjANBgkqhkiG9w0BAQsFADCB
yjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMTk5OSBWZXJp
U2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxW
ZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0
aG9yaXR5IC0gRzMwHhcNMTMxMDMxMDAwMDAwWhcNMjMxMDMwMjM1OTU5WjB+MQsw
CQYDVQQGEwJVUzEdMBsGA1UEChMUU3ltYW50ZWMgQ29ycG9yYXRpb24xHzAdBgNV
BAsTFlN5bWFudGVjIFRydXN0IE5ldHdvcmsxLzAtBgNVBAMTJlN5bWFudGVjIENs
YXNzIDMgU2VjdXJlIFNlcnZlciBDQSAtIEc0MIIBIjANBgkqhkiG9w0BAQEFAAOC
AQ8AMIIBCgKCAQEAstgFyhx0LbUXVjnFSlIJluhL2AzxaJ+aQihiw6UwU35VEYJb
A3oNL+F5BMm0lncZgQGUWfm893qZJ4Itt4PdWid/sgN6nFMl6UgfRk/InSn4vnlW
9vf92Tpo2otLgjNBEsPIPMzWlnqEIRoiBAMnF4scaGGTDw5RgDMdtLXO637QYqzu
s3sBdO9pNevK1T2p7peYyo2qRA4lmUoVlqTObQJUHypqJuIGOmNIrLRM0XWTUP8T
L9ba4cYY9Z/JJV3zADreJk20KQnNDz0jbxZKgRb78oMQw7jW2FUyPfG9D72MUpVK
Fpd6UiFjdS8W+cRmvvW1Cdj/JwDNRHxvSz+w9wIDAQABo4IBQDCCATwwHQYDVR0O
BBYEFF9gz2GQVd+EQxSKYCqy9Xr0QxjvMBIGA1UdEwEB/wQIMAYBAf8CAQAwawYD
VR0gBGQwYjBgBgpghkgBhvhFAQc2MFIwJgYIKwYBBQUHAgEWGmh0dHA6Ly93d3cu
c3ltYXV0aC5jb20vY3BzMCgGCCsGAQUFBwICMBwaGmh0dHA6Ly93d3cuc3ltYXV0
aC5jb20vcnBhMC8GA1UdHwQoMCYwJKAioCCGHmh0dHA6Ly9zLnN5bWNiLmNvbS9w
Y2EzLWczLmNybDAOBgNVHQ8BAf8EBAMCAQYwKQYDVR0RBCIwIKQeMBwxGjAYBgNV
BAMTEVN5bWFudGVjUEtJLTEtNTM0MC4GCCsGAQUFBwEBBCIwIDAeBggrBgEFBQcw
AYYSaHR0cDovL3Muc3ltY2QuY29tMA0GCSqGSIb3DQEBCwUAA4IBAQBbF1K+1lZ7
9Pc0CUuWysf2IdBpgO/nmhnoJOJ/2S9h3RPrWmXk4WqQy04q6YoW51KN9kMbRwUN
gKOomv4p07wdKNWlStRxPA91xQtzPwBIZXkNq2oeJQzAAt5mrL1LBmuaV4oqgX5n
m7pSYHPEFfe7wVDJCKW6V0o6GxBzHOF7tpQDS65RsIJAOloknO4NWF2uuil6yjOe
soHCL47BJ89A8AShP/U3wsr8rFNtqVNpT+F2ZAwlgak3A/I5czTSwXx4GByoaxbn
5+CdKa/Y5Gk5eZVpuXtcXQGc1PfzSEUTZJXXCm5y2kMiJG8+WnDcwJLgLeVX+OQr
J+71/xuzAYN6
-----END CERTIFICATE-----
"""
VERISIGN_ROOT = """
-----BEGIN CERTIFICATE-----
MIIEGjCCAwICEQCbfgZJoz5iudXukEhxKe9XMA0GCSqGSIb3DQEBBQUAMIHKMQsw
CQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZl
cmlTaWduIFRydXN0IE5ldHdvcmsxOjA4BgNVBAsTMShjKSAxOTk5IFZlcmlTaWdu
LCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxRTBDBgNVBAMTPFZlcmlT
aWduIENsYXNzIDMgUHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3Jp
dHkgLSBHMzAeFw05OTEwMDEwMDAwMDBaFw0zNjA3MTYyMzU5NTlaMIHKMQswCQYD
VQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlT
aWduIFRydXN0IE5ldHdvcmsxOjA4BgNVBAsTMShjKSAxOTk5IFZlcmlTaWduLCBJ
bmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxRTBDBgNVBAMTPFZlcmlTaWdu
IENsYXNzIDMgUHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkg
LSBHMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMu6nFL8eB8aHm8b
N3O9+MlrlBIwT/A2R/XQkQr1F8ilYcEWQE37imGQ5XYgwREGfassbqb1EUGO+i2t
KmFZpGcmTNDovFJbcCAEWNF6yaRpvIMXZK0Fi7zQWM6NjPXr8EJJC52XJ2cybuGu
kxUccLwgTS8Y3pKI6GyFVxEa6X7jJhFUokWWVYPKMIno3Nij7SqAP395ZVc+FSBm
CC+Vk7+qRy+oRpfwEuL+wgorUeZ25rdGt+INpsyow0xZVYnm6FNcHOqd8GIWC6fJ
Xwzw3sJ2zq/3avL6QaaiMxTJ5Xpj055iN9WFZZ4O5lMkdBteHRJTW8cs54NJOxWu
imi5V5cCAwEAATANBgkqhkiG9w0BAQUFAAOCAQEAERSWwauSCPc/L8my/uRan2Te
2yFPhpk0djZX3dAVL8WtfxUfN2JzPtTnX84XA9s1+ivbrmAJXx5fj267Cz3qWhMe
DGBvtcC1IyIuBwvLqXTLR7sdwdela8wv0kL9Sd2nic9TutoAWii/gt/4uhMdUIaC
/Y4wjylGsB49Ndo4YhYYSq3mtlFs3q9i6wHQHiT+eo8SGhJouPtmmRQURVyu565p
F4ErWjfJXir0xuKhXFSbplQAz/DxwceYMBo7Nhbbo27q/a2ywtrvAkcTisDxszGt
TxzhT5yvDwyd93gN2PQ1VoDat20Xj50egWTh/sVFuq1ruQp6Tk9LhO5L8X3dEQ==
-----END CERTIFICATE-----
"""
OLD_VERISIGN_INTERMEDIATE = """
-----BEGIN CERTIFICATE-----
MIIFlTCCBH2gAwIBAgIQLP62CQ7ireLp/CI3JPG2vzANBgkqhkiG9w0BAQUFADCB
yjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMTk5OSBWZXJp
U2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxW
ZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0
aG9yaXR5IC0gRzMwHhcNMTAwMjA4MDAwMDAwWhcNMjAwMjA3MjM1OTU5WjCBtTEL
MAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZW
ZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQg
aHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykxMDEvMC0GA1UEAxMmVmVy
aVNpZ24gQ2xhc3MgMyBTZWN1cmUgU2VydmVyIENBIC0gRzMwggEiMA0GCSqGSIb3
DQEBAQUAA4IBDwAwggEKAoIBAQCxh4QfwgxF9byrJZenraI+nLr2wTm4i8rCrFbG
5btljkRPTc5v7QlK1K9OEJxoiy6Ve4mbE8riNDTB81vzSXtig0iBdNGIeGwCU/m8
f0MmV1gzgzszChew0E6RJK2GfWQS3HRKNKEdCuqWHQsV/KNLO85jiND4LQyUhhDK
tpo9yus3nABINYYpUHjoRWPNGUFP9ZXse5jUxHGzUL4os4+guVOc9cosI6n9FAbo
GLSa6Dxugf3kzTU2s1HTaewSulZub5tXxYsU5w7HnO1KVGrJTcW/EbGuHGeBy0RV
M5l/JJs/U0V/hhrzPPptf4H1uErT9YU3HLWm0AnkGHs4TvoPAgMBAAGjggGIMIIB
hDASBgNVHRMBAf8ECDAGAQH/AgEAMHAGA1UdIARpMGcwZQYLYIZIAYb4RQEHFwMw
VjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL2NwczAqBggr
BgEFBQcCAjAeGhxodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhMA4GA1UdDwEB
/wQEAwIBBjBtBggrBgEFBQcBDARhMF+hXaBbMFkwVzBVFglpbWFnZS9naWYwITAf
MAcGBSsOAwIaBBSP5dMahqyNjmvDz4Bq1EgYLHsZLjAlFiNodHRwOi8vbG9nby52
ZXJpc2lnbi5jb20vdnNsb2dvLmdpZjAoBgNVHREEITAfpB0wGzEZMBcGA1UEAxMQ
VmVyaVNpZ25NUEtJLTItNjAdBgNVHQ4EFgQUDURcFlNEwYJ+HSCrJfQBY9i+eaUw
NAYDVR0fBC0wKzApoCegJYYjaHR0cDovL2NybC52ZXJpc2lnbi5jb20vcGNhMy1n
My5jcmwwDQYJKoZIhvcNAQEFBQADggEBAHREFQzFWA4YY+3z8CjDeuuSSG/ghSBJ
olwwlpIX4IjoeYuzT864Hzk2tTeEeODf4YFIVsSxah8nUsGdpgVTUGPPoUJOMXvn
8wJeBSlUDXBwv3td5XbPIPXHy6vmIS6phYRetZUgq1CDTI/pvtWZKXTGM/eYXlLF
6QDvXevUHQjfb3cqQvfLljws85xLxbNFmz7cy9YmiLOd5n+gFC6X5hzSDO7+DDMi
o//+4Q/nk/UId1UCsobqYWVmqs017AmyiAPO/v3sGncYYQY2BMYgla74dZfeDNu4
MXA68Mb6ZdlkhGEmZYVBcOmkaKs+P+SggTofsK27BlpugAtNWjEy5JY=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIEOzCCA6SgAwIBAgIQSsnqCI7m94zHpfn6OaSTljANBgkqhkiG9w0BAQUFADBf
MQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xNzA1BgNVBAsT
LkNsYXNzIDMgUHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkw
HhcNMTEwNjA5MDAwMDAwWhcNMjExMTA3MjM1OTU5WjCByjELMAkGA1UEBhMCVVMx
FzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVz
dCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMTk5OSBWZXJpU2lnbiwgSW5jLiAtIEZv
ciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxWZXJpU2lnbiBDbGFzcyAz
IFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IC0gRzMwggEi
MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDLupxS/HgfGh5vGzdzvfjJa5QS
ME/wNkf10JEK9RfIpWHBFkBN+4phkOV2IMERBn2rLG6m9RFBjvotrSphWaRnJkzQ
6LxSW3AgBFjResmkabyDF2StBYu80FjOjYz16/BCSQudlydnMm7hrpMVHHC8IE0v
GN6SiOhshVcRGul+4yYRVKJFllWDyjCJ6NzYo+0qgD9/eWVXPhUgZggvlZO/qkcv
qEaX8BLi/sIKK1Hmdua3RrfiDabMqMNMWVWJ5uhTXBzqnfBiFgunyV8M8N7Cds6v
92ry+kGmojMUyeV6Y9OeYjfVhWWeDuZTJHQbXh0SU1vHLOeDSTsVropouVeXAgMB
AAGjggEGMIIBAjAPBgNVHRMBAf8EBTADAQH/MD0GA1UdIAQ2MDQwMgYEVR0gADAq
MCgGCCsGAQUFBwIBFhxodHRwczovL3d3dy52ZXJpc2lnbi5jb20vY3BzMDEGA1Ud
HwQqMCgwJqAkoCKGIGh0dHA6Ly9jcmwudmVyaXNpZ24uY29tL3BjYTMuY3JsMA4G
A1UdDwEB/wQEAwIBBjBtBggrBgEFBQcBDARhMF+hXaBbMFkwVzBVFglpbWFnZS9n
aWYwITAfMAcGBSsOAwIaBBSP5dMahqyNjmvDz4Bq1EgYLHsZLjAlFiNodHRwOi8v
bG9nby52ZXJpc2lnbi5jb20vdnNsb2dvLmdpZjANBgkqhkiG9w0BAQUFAAOBgQBl
2Sr58sJgybnqQQfKNrcYL2iu/gMk5mdU7nTDLNn1M8Fetw6Tz3iejrImFBFT0cjC
EiG0PXsq2BzUS2TsiU+/lYeH3pVk9HPGF9+9GZCX6GmBEmlmStMkQA5ZdRWwRHQX
op4GYNOwg7jdL+afe2dcFqFH284ueQXZ8fT4PuJKoQ==
-----END CERTIFICATE-----
"""

194
lemur/common/services/issuers/plugins/verisign/verisign.py Normal file
View File

@ -0,0 +1,194 @@
"""
.. module: lemur.common.services.issuers.plugins.verisign.verisign
:platform: Unix
:synopsis: This module is responsible for communicating with the VeriSign VICE 2.0 API.
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
import arrow
import requests
import xmltodict
from flask import current_app
from lemur.common.services.issuers.issuer import Issuer
from lemur.common.services.issuers.plugins import verisign
from lemur.certificates.exceptions import InsufficientDomains
# https://support.venafi.com/entries/66445046-Info-VeriSign-Error-Codes
VERISIGN_ERRORS = {
"0x30c5": "Domain Mismatch when enrolling for an SSL certificate, a domain in your request has not been added to verisign",
"0x482d": "Cannot issue SHA1 certificates expiring after 31/12/2016",
"0x3a10": "Invalid X509 certificate format.: an unsupported certificate format was submitted",
"0x4002": "Internal QM Error. : Internal Database connection error.",
"0x3301": "Bad transaction id or parent cert not renewable.: User try to renew a certificate that is not yet ready for renew or the transaction id is wrong",
"0x3069": "Challenge phrase mismatch: The challenge phrase submitted does not match the original one",
"0x3111": "Unsupported Product: User submitted a wrong product or requested cipher is not supported",
"0x30e8": "CN or org does not match the original one.: the submitted CSR contains a common name or org that does not match the original one",
"0x1005": "Duplicate certificate: a certificate with the same common name exists already",
"0x0194": "Incorrect Signature Algorithm: The requested signature algorithm is not supported for the key type. i.e. an ECDSA is submitted for an RSA key",
"0x6000": "parameter missing or incorrect: This is a general error code for missing or incorrect parameters. The reason will be in the response message. i.e. 'CSR is missing, 'Unsupported serverType' when no supported serverType could be found., 'invalid transaction id'",
"0x3063": "Certificate not allowed: trying to issue a certificate that is not configured for the account",
"0x23df": "No MDS Data Returned: internal connection lost or server not responding. this should be rare",
"0x3004": "Invalid Account: The users mpki account associated with the certificate is not valid or not yet active",
"0x4101": "Internal Error: internal server error, user should try again later. (Also check that State is spelled out",
"0x3101": "Missing admin role: Your account does not have the admin role required to access the webservice API",
"0x3085": "Account does not have webservice feature.: Your account does not the the webservice role required to access the webservice API",
"0x9511": "Corrupted CSR : the submitted CSR was mal-formed",
"0xa001": "Public key format does not match.: The public key format does not match the original cert at certificate renewal or replacement. E.g. if you try to renew or replace an RSA cert with a DSA or ECC key based CSR",
"0x0143": "Certificate End Date Error: You are trying to replace a certificate with validity end date exceeding the original cert. or the certificate end date is not valid",
"0x482d": "SHA1 validity check error: What error code do we get when we submit the SHA1 SSL requests with the validity more than 12/31/2016?",
"0x482e": "What error code do we get when we cannot complete the re-authentication for domains with a newly-approved gTLD 30 days after the gTLD approval",
"0x4824": "Per CA/B Forum baseline requirements, non-FQDN certs cannot exceed 11/1/2015. Examples: hostname, foo.cba (.cba is a pending gTLD)",
"eE0x48": "Currently the maximum cert validity is 4-years",
"0x4826": "OU misleading. See comments",
"0x4827": "Org re-auth past due. EV org has to go through re-authentication every 13 months; OV org has to go through re-authentication every 39 months",
"0x482a": "Domain re-auth past due. EV domain has to go through re-authentication every 13 months; OV domain has to go through re-authentication every 39 months.",
"0x482b": "No org address was set to default, should not happen",
"0x482c": "signature algorithm does not match intended key type in the CSR (e.g. CSR has an ECC key, but the signature algorithm is sha1WithRSAEncryption)",
"0x600E": "only supports ECC keys with the named curve NIST P-256, aka secp256r1 or prime256v1, other ECC key sizes will get this error ",
"0x6013": "only supports DSA keys with (2048, 256) as the bit lengths of the prime parameter pair (p, q), other DSA key sizes will get this error",
"0x600d": "RSA key size < 2A048",
"0x4828": "Verisign certificates can be at most two years in length",
"0x3043": "Certificates must have a validity of at least 1 day"
}
class Verisign(Issuer):
title = 'VeriSign'
slug = 'verisign'
description = 'Enables the creation of certificates by the VICE2.0 verisign API.'
version = verisign.VERSION
author = 'Kevin Glisson'
author_url = 'https://github.com/netflix/lemur'
def __init__(self, *args, **kwargs):
self.session = requests.Session()
self.session.cert = current_app.config.get('VERISIGN_PEM_PATH')
super(Verisign, self).__init__(*args, **kwargs)
@staticmethod
def handle_response(content):
"""
Helper function that helps with parsing responses from the Verisign API.
:param content:
:return: :raise Exception:
"""
d = xmltodict.parse(content)
global VERISIGN_ERRORS
if d.get('Error'):
status_code = d['Error']['StatusCode']
elif d.get('Response'):
status_code = d['Response']['StatusCode']
if status_code in VERISIGN_ERRORS.keys():
raise Exception(VERISIGN_ERRORS[status_code])
return d
def create_certificate(self, csr, issuer_options):
"""
Creates a Verisign certificate.
:param csr:
:param issuer_options:
:return: :raise Exception:
"""
url = current_app.config.get("VERISIGN_URL") + '/enroll'
data = {
'csr': csr,
'challenge': issuer_options['challenge'],
'serverType': 'Apache',
'certProductType': 'Server',
'firstName': current_app.config.get("VERISIGN_FIRST_NAME"),
'lastName': current_app.config.get("VERISIGN_LAST_NAME"),
'signatureAlgorithm': 'sha256WithRSAEncryption',
'email': current_app.config.get("VERISIGN_EMAIL")
}
if issuer_options.get('validityEnd'):
data['specificEndDate'] = arrow.get(issuer_options['validityEnd']).replace(days=-1).format("MM/DD/YYYY")
now = arrow.utcnow()
then = arrow.get(issuer_options['validityEnd'])
if then < now.replace(years=+1):
data['validityPeriod'] = '1Y'
elif then < now.replace(years=+2):
data['validityPeriod'] = '2Y'
else:
raise Exception("Verisign issued certificates cannot exceed two years in validity")
current_app.logger.info("Requesting a new verisign certificate: {0}".format(data))
response = self.session.post(url, data=data)
cert = self.handle_response(response.content)['Response']['Certificate']
return cert, verisign.constants.VERISIGN_INTERMEDIATE,
def get_csr_config(self, issuer_options):
"""
Used to generate a valid CSR for the given Certificate Authority.
:param issuer_options:
:return: :raise InsufficientDomains:
"""
domains = []
if issuer_options.get('commonName'):
domains.append(issuer_options.get('commonName'))
if issuer_options.get('extensions'):
for n in issuer_options['extensions']['subAltNames']['names']:
if n['value']:
domains.append(n['value'])
is_san_comment = "#"
dns_lines = []
if len(domains) < 1:
raise InsufficientDomains
elif len(domains) > 1:
is_san_comment = ""
for domain_line in list(set(domains)):
dns_lines.append("DNS.{} = {}".format(len(dns_lines) + 1, domain_line))
return verisign.constants.CSR_CONFIG.format(
is_san_comment=is_san_comment,
OU=issuer_options.get('organizationalUnit', 'Operations'),
DNS=domains,
DNS_LINES="\n".join(dns_lines))
@staticmethod
def create_authority(options):
"""
Creates an authority, this authority is then used by Lemur to allow a user
to specify which Certificate Authority they want to sign their certificate.
:param options:
:return:
"""
role = {'username': '', 'password': '', 'name': 'verisign'}
return verisign.constants.VERISIGN_ROOT, "", [role]
def get_available_units(self):
"""
Uses the Verisign to fetch the number of available unit's left. This can be used to get tabs
on the number of certificates that can be issued.
:return:
"""
url = current_app.config.get("VERISIGN_URL") + '/getTokens'
response = self.session.post(url, headers={'content-type': 'application/x-www-form-urlencoded'})
return self.handle_response(response.content)['Response']['Order']
def get_authorities(self):
pass
def init():
return Verisign()

63
lemur/common/utils.py Normal file
View File

@ -0,0 +1,63 @@
"""
.. module: lemur.common.utils
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from functools import wraps
from flask import current_app
from flask.ext.restful import marshal
from flask.ext.restful.reqparse import RequestParser
from flask.ext.sqlalchemy import Pagination
class marshal_items(object):
def __init__(self, fields, envelope=None):
self.fields = fields
self.envelop = envelope
def __call__(self, f):
def _filter_items(items):
filtered_items = []
for item in items:
filtered_items.append(marshal(item, self.fields))
return filtered_items
@wraps(f)
def wrapper(*args, **kwargs):
try:
resp = f(*args, **kwargs)
# this is a bit weird way to handle non standard error codes returned from the marshaled function
if isinstance(resp, tuple):
return resp[0], resp[1]
if isinstance(resp, Pagination):
return {'items': _filter_items(resp.items), 'total': resp.total}
if isinstance(resp, list):
return _filter_items(resp)
return marshal(resp, self.fields)
except Exception as e:
# this is a little weird hack to respect flask restful parsing errors on marshaled functions
if hasattr(e, 'code'):
return {'message': e.data['message']}, 400
else:
current_app.logger.exception(e)
return {'message': e.message}, 400
return wrapper
paginated_parser = RequestParser()
paginated_parser.add_argument('count', type=int, default=10, location='args')
paginated_parser.add_argument('page', type=int, default=1, location='args')
paginated_parser.add_argument('sortDir', type=str, dest='sort_dir', location='args')
paginated_parser.add_argument('sortBy', type=str, dest='sort_by', location='args')
paginated_parser.add_argument('filter', type=str, location='args')

10
lemur/constants.py Normal file
View File

@ -0,0 +1,10 @@
"""
.. module: lemur.constants
:copyright: (c) 2015 by Netflix Inc.
:license: Apache, see LICENSE for more details.
"""
SAN_NAMING_TEMPLATE = "SAN-{subject}-{issuer}-{not_before}-{not_after}"
DEFAULT_NAMING_TEMPLATE = "{subject}-{issuer}-{not_before}-{not_after}"
NONSTANDARD_NAMING_TEMPLATE = "{issuer}-{not_before}-{not_after}"

278
lemur/database.py Normal file
View File

@ -0,0 +1,278 @@
"""
.. module: lemur.database
:platform: Unix
:synopsis: This module contains all of the database related methods
needed for lemur to interact with a datastore
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from flask import current_app
from sqlalchemy import exc
from sqlalchemy.sql import and_, or_
from lemur.extensions import db
from lemur.exceptions import AttrNotFound, IntegrityError
def filter_none(kwargs):
"""
Remove all `None` values froma given dict. SQLAlchemy does not
like to have values that are None passed to it.
:param kwargs: Dict to filter
:return: Dict without any 'None' values
"""
n_kwargs = {}
for k, v in kwargs.items():
if v:
n_kwargs[k] = v
return n_kwargs
def session_query(model):
"""
Returns a SQLAlchemy query object for the specified `model`.
If `model` has a ``query`` attribute already, that object will be returned.
Otherwise a query will be created and returned based on `session`.
:param model: sqlalchemy model
:return: query object for model
"""
return model.query if hasattr(model, 'query') else db.session.query(model)
def create_query(model, kwargs):
"""
Returns a SQLAlchemy query object for specified `model`. Model
filtered by the kwargs passed.
:param model:
:param kwargs:
:return:
"""
s = session_query(model)
return s.filter_by(**kwargs)
def commit():
"""
Helper to commit the current session.
"""
db.session.commit()
def add(model):
"""
Helper to add a `model` to the current session.
:param model:
:return:
"""
db.session.add(model)
def find_all(query, model, kwargs):
"""
Returns a query object that ensures that all kwargs
are present.
:param query:
:param model:
:param kwargs:
:return:
"""
conditions = []
kwargs = filter_none(kwargs)
for attr, value in kwargs.items():
if not isinstance(value, list):
value = value.split(',')
conditions.append(getattr(model, attr).in_(value))
return query.filter(and_(*conditions))
def find_any(query, model, kwargs):
"""
Returns a query object that allows any kwarg
to be present.
:param query:
:param model:
:param kwargs:
:return:
"""
or_args = []
for attr, value in kwargs.items():
or_args.append(or_(getattr(model, attr) == value))
exprs = or_(*or_args)
return query.filter(exprs)
def get(model, value, field="id"):
"""
Returns one object filtered by the field and value.
:param model:
:param value:
:param field:
:return:
"""
query = session_query(model)
try:
return query.filter(getattr(model, field) == value).one()
except:
return
def get_all(model, value, field="id"):
"""
Returns query object with the fields and value filtered.
:param model:
:param value:
:param field:
:return:
"""
query = session_query(model)
return query.filter(getattr(model, field) == value)
def create(model):
"""
Helper that attempts to create a new instance of an object.
:param model:
:return: :raise IntegrityError:
"""
try:
db.session.add(model)
commit()
db.session.refresh(model)
except exc.IntegrityError as e:
raise IntegrityError(e.orig.diag.message_detail)
return model
def update(model):
"""
Helper that attempts to update a model.
:param model:
:return:
"""
commit()
db.session.refresh(model)
return model
def delete(model):
"""
Helper that attempts to delete a model.
:param model:
"""
db.session.delete(model)
db.session.commit()
def filter(query, model, terms):
"""
Helper that searched for 'like' strings in column values.
:param query:
:param model:
:param terms:
:return:
"""
return query.filter(getattr(model, terms[0]).ilike('%{}%'.format(terms[1])))
def sort(query, model, field, direction):
"""
Returns objects of the specified `model` in the field and direction
given
:param query:
:param model:
:param field:
:param direction:
"""
try:
field = getattr(model, field)
direction = getattr(field, direction)
query = query.order_by(direction())
return query
except AttributeError as e:
raise AttrNotFound(field)
def paginate(query, page, count):
"""
Returns the items given the count and page specified
:param query:
:param page:
:param count:
"""
return query.paginate(page, count)
def update_list(model, model_attr, item_model, items):
"""
Helper that correctly updates a models items
depending on what has changed
:param model_attr:
:param item_model:
:param items:
:param model:
:return:
"""
ids = []
for i in items:
ids.append(i['id'])
for i in getattr(model, model_attr):
if i.id not in ids:
getattr(model, model_attr).remove(i)
for i in items:
for item in getattr(model, model_attr):
if item.id == i['id']:
break
else:
getattr(model, model_attr).append(get(item_model, i['id']))
return model
def sort_and_page(query, model, args):
"""
Helper that allows us to combine sorting and paging
:param query:
:param model:
:param args:
:return:
"""
sort_by = args.pop('sort_by')
sort_dir = args.pop('sort_dir')
page = args.pop('page')
count = args.pop('count')
query = find_all(query, model, args)
if sort_by and sort_dir:
query = sort(query, model, sort_by, sort_dir)
return paginate(query, page, count)

55
lemur/decorators.py Normal file
View File

@ -0,0 +1,55 @@
"""
.. module: lemur.decorators
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
"""
from datetime import timedelta
from flask import make_response, request, current_app
from functools import update_wrapper
def crossdomain(origin=None, methods=None, headers=None,
max_age=21600, attach_to_all=True,
automatic_options=True):
if methods is not None:
methods = ', '.join(sorted(x.upper() for x in methods))
if headers is not None and not isinstance(headers, basestring):
headers = ', '.join(x.upper() for x in headers)
if not isinstance(origin, basestring):
origin = ', '.join(origin)
if isinstance(max_age, timedelta):
max_age = max_age.total_seconds()
def get_methods():
if methods is not None:
return methods
options_resp = current_app.make_default_options_response()
return options_resp.headers['allow']
def decorator(f):
def wrapped_function(*args, **kwargs):
if automatic_options and request.method == 'OPTIONS':
resp = current_app.make_default_options_response()
else:
resp = make_response(f(*args, **kwargs))
if not attach_to_all and request.method != 'OPTIONS':
return resp
h = resp.headers
h['Access-Control-Allow-Origin'] = origin
h['Access-Control-Allow-Methods'] = get_methods()
h['Access-Control-Max-Age'] = str(max_age)
#if headers is not None:
h['Access-Control-Allow-Headers'] = "Origin, X-Requested-With, Content-Type, Accept, Authorization " # headers
h['Access-Control-Allow-Credentials'] = 'true'
return resp
f.provide_automatic_options = False
return update_wrapper(wrapped_function, f)
return decorator

0
lemur/domains/__init__.py Normal file
View File

27
lemur/domains/models.py Normal file
View File

@ -0,0 +1,27 @@
"""
.. module: lemur.domains.models
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from sqlalchemy import Column, Integer, String
from lemur.database import db
class Domain(db.Model):
__tablename__ = 'domains'
id = Column(Integer, primary_key=True)
name = Column(String(256))
def as_dict(self):
return {c.name: getattr(self, c.name) for c in self.__table__.columns}
def serialize(self):
blob = self.as_dict()
blob['certificates'] = [x.id for x in self.certificate]
return blob

64
lemur/domains/service.py Normal file
View File

@ -0,0 +1,64 @@
"""
.. module: lemur.domains.service
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from lemur.domains.models import Domain
from lemur.certificates.models import Certificate
from lemur import database
def get(domain_id):
"""
Fetches one domain
:param domain_id:
:return:
"""
return database.get(Domain, domain_id)
def get_all():
"""
Fetches all domains
:return:
"""
query = database.session_query(Domain)
return database.find_all(query, Domain, {}).all()
def render(args):
"""
Helper to parse REST Api requests
:param args:
:return:
"""
query = database.session_query(Domain).join(Certificate, Domain.certificate)
sort_by = args.pop('sort_by')
sort_dir = args.pop('sort_dir')
page = args.pop('page')
count = args.pop('count')
filt = args.pop('filter')
certificate_id = args.pop('certificate_id', None)
if filt:
terms = filt.split(';')
query = database.filter(query, Domain, terms)
if certificate_id:
query = query.filter(Certificate.id == certificate_id)
query = database.find_all(query, Domain, args)
if sort_by and sort_dir:
query = database.sort(query, Domain, sort_by, sort_dir)
return database.paginate(query, page, count)

182
lemur/domains/views.py Normal file
View File

@ -0,0 +1,182 @@
"""
.. module: lemur.domains.views
:platform: Unix
:copyright: (c) 2015 by Netflix Inc., see AUTHORS for more
:license: Apache, see LICENSE for more details.
.. moduleauthor:: Kevin Glisson <kglisson@netflix.com>
"""
from flask import Blueprint
from flask.ext.restful import reqparse, Api, fields
from lemur.domains import service
from lemur.auth.service import AuthenticatedResource
from lemur.common.utils import paginated_parser, marshal_items
FIELDS = {
'id': fields.Integer,
'name': fields.String
}
mod = Blueprint('domains', __name__)
api = Api(mod)
class DomainsList(AuthenticatedResource):
""" Defines the 'domains' endpoint """
def __init__(self):
super(DomainsList, self).__init__()
@marshal_items(FIELDS)
def get(self):
"""
.. http:get:: /domains
The current domain list
**Example request**:
.. sourcecode:: http
GET /domains HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"items": [
{
"id": 1,
"name": "www.example.com",
},
{
"id": 2,
"name": "www.example2.com",
}
]
"total": 2
}
:query sortBy: field to sort on
:query sortDir: acs or desc
:query page: int. default is 1
:query filter: key value pair. format is k=v;
:query limit: limit number. default is 10
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
"""
parser = paginated_parser.copy()
args = parser.parse_args()
return service.render(args)
class Domains(AuthenticatedResource):
def __init__(self):
self.reqparse = reqparse.RequestParser()
super(Domains, self).__init__()
@marshal_items(FIELDS)
def get(self, domain_id):
"""
.. http:get:: /domains/1
Fetch one domain
**Example request**:
.. sourcecode:: http
GET /domains HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"id": 1,
"name": "www.example.com",
}
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
"""
return service.get(domain_id)
class CertificateDomains(AuthenticatedResource):
""" Defines the 'domains' endpoint """
def __init__(self):
super(CertificateDomains, self).__init__()
@marshal_items(FIELDS)
def get(self, certificate_id):
"""
.. http:get:: /certificates/1/domains
The current domain list
**Example request**:
.. sourcecode:: http
GET /domains HTTP/1.1
Host: example.com
Accept: application/json, text/javascript
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: text/javascript
{
"items": [
{
"id": 1,
"name": "www.example.com",
},
{
"id": 2,
"name": "www.example2.com",
}
]
"total": 2
}
:query sortBy: field to sort on
:query sortDir: acs or desc
:query page: int. default is 1
:query filter: key value pair. format is k=v;
:query limit: limit number. default is 10
:reqheader Authorization: OAuth token to authenticate
:statuscode 200: no error
:statuscode 403: unauthenticated
"""
parser = paginated_parser.copy()
args = parser.parse_args()
args['certificate_id'] = certificate_id
return service.render(args)
api.add_resource(DomainsList, '/domains', endpoint='domains')
api.add_resource(Domains, '/domains/<int:domain_id>', endpoint='domain')
api.add_resource(CertificateDomains, '/certificates/<int:certificate_id>/domains', endpoint='certificateDomains')

0
lemur/elbs/__init__.py Normal file
View File

Some files were not shown because too many files have changed in this diff Show More