Commit Graph

150 Commits

Author SHA1 Message Date
lmitul 6a762d463f Documenting connection pool config settings (#1197) 2018-04-10 16:50:58 -07:00
Dmitry Zykov 4a0103a88d SFTP destination plugin (#1170)
* add sftp destination plugin
2018-04-03 10:30:19 -07:00
iTitou a316cbba73 [add] Docs and default config for metric plugins (#1148) 2018-03-27 15:51:32 -07:00
Harm Weites 5d18838868 Use Cloudflare as DNS provider for LE certs (#945)
* Use Cloudflare as DNS provider for LE certs

* Better handle dns_provider plugins
2018-02-22 08:17:28 -08:00
James Chuong 2578970f7d Async Certificate Issuing using Pending Certificates (#1037)
* Add PendingCertificate model

This change creates a DB table called pending_certificates and
associated mapping relationship tables from pending certificate to
roles, rotation policy, destination, sources, etc.

The table is generated on initialization of Lemur. A pending
certificate holds most of the information of a Certificate, while it has
not be issued so that it can later backfill the information when the CA
has issued the certificate.

Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0
Tickets: PBL-36499

* Create a PendingCertificate if cert is empty

IssuePlugins should return empty cert bodies if the request failed to
complete immediately (such as Digicert).  This way, we can immediately
return the certificate, or if not just place into PendingCertificates
for later processing.

+ Fix relation from Certificate to Pending Certificate, as view only.
There is no real need for anything more than that since Pending cert
only needs to know the cert to replace when it is issued later.

+ Made PendingCertificate private key be empty: UI does not allow
private key on 'Create' but only on 'Import'.  For Instart, we require
the private key but upstream does not necessarily need it.  Thus, if
someone at Instart wants to create a CSR / key combo, they should
manually issue the cert themselves and import later.  Otherwise you
should let Lemur generate that.  This keeps the workflow transparent for
upstream Lemur users.

Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26
Tickets: PBL-36343

* Fix empty private_key when create Pending Cert

On creation of a certificate with a CSR, there is no option for private
key.  In this case, we actually have a dictionary with private_key as
key, but the value is None.  This fixes the strip() called on NoneType.

Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199
Tickets: PBL-36499

* Source sync finds and uses pending certificate

When a source syncs certificates, it will check for a pending
certificate.  If that is found via external_id (given by digicert as
order_id) then it will use the found Pending Certificate's fields to
create a new certificate.  Then the pending certificate is deleted.

Tickets: PBL-36343
Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9

* Add Lemur static files and views for pending certs

This adds the basic static files to view pending certificates in a
table.

Tickets: PBL-36343
Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9

* Add CLI and plugin based pending fetch

This change uses the adds a new function to issuer plugins to fetch
certificates like source, but for one order.  This way, we can control
which pending certificates to try and populate instead of getting all
certificates from source.

Tickets: PBL-36343
Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6

* Revert source using Pending Certificate

Tickets: PBL-36343
Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51

* Fix PendingCertificate init getting authority id

Should get authority id from authority.id instead of the authority_id
key in kwargs.

Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623
Tickets: n/a

* Add fixtures and basic test for PendingCertificate

Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34
Tickets: PBL-36343

* Add User to create_certificate parameters

create_certificate now takes a User, which will be used to populate the
'creator' field in certificates.service.upload().  This allows the UI
populate with the current user if the owner does not exist in Lemur.

+ Fix chain being replaced with version from pending certificate, which
may be empty (depends on plugin implementation).

Change-Id: I516027b36bc643c4978b9c4890060569e03f3049
Tickets: n/a

* Fix permalink and filters to pending certs

Fixes the permalink button to get a single pending certificate
Add argument filter parsing for the pending certificate API
Fix comment on API usage
Added get_by_name for pending_certificate (currently unused, but useful
for CLI, instead of using IDs)

Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db
Tickets: PBL-36910

* Update displayed fields for Pending Certificates

There are a number of unused / unpopulated fields from Certificate UI
that does apply to Pending Certificates.  Those ones were removed, and
added other useful fields:
Owner, number of attempts to fetch and date created

Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b
Tickets: PBL-36910

* Add common name (cn) to Pending Certificate model

Fixes the UI missing the CN for Pending Certificate, as it was
originally being parsed from the generated certificate.  In the case of
pending certificate, the CN from the user generates the request, which
means a pending cert can trust the original user putting in the CN
instead of having to parse the not-yet-generated certificate.  There is
no real possibility to return a certificate from a pending certificate
where the CN has changed since it was initially ordered.

Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab
Tickets: PBL-36910

* Fix missing imports for service filter

+ Removed duplicate get_by_name function from old merge

Change-Id: I04ae6852533aa42988433338de74390e2868d69b
Tickets: PBL-36910

* Add private key viewing to Pending Certificates

Add private key API for Pending Certificates, with the same
authorization as Certificates (only owner, creator or owner-roles can
view private key).

Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b
Tickets: PBL-36910

* Add edit capability to pending certificates

Like editing certificates, we should be able to modify some parts of a
pending certificate so the resulting certificate has the right
references, owner, etc.

+ Added API to update pending certificate
+ Fix UI to use pending certificate scope instead of reusing Certificate
+ Change pending_certificate.replaces to non-passive association, so
that updates do affect it (similar to roles/notifications/etc)

Tickets: PBL-36910
Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6

* Add common_name parsing instead using kwargs

To fix tests where common name may not be passed in, use the CSR
generated to find the official common name.

Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4
Tickets: PBL-36343

* Add Cancel to pending certificates and plugins

This allows pending certificates to be cancelled, which will be handled
by the issuer plugin.

Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9
Tickets: PBL-36910

* Add API for Cancelling Pending Certificate

Added the DELETE handler for pending_certificates, which will cancel and
delete the pending certificate from the pending certs table on
successful cancellation via Issuer Plugin.

+ Add UT for testing cancel API

Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4
Tickets: PBL-36910

* Remove Export from Pending Certificates

Pending Certificates doesn't need an export since it should just be
fetched by Lemur via plugins, and the CSR is viewable via the UI.

Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae
Tickets: PBL-36910

* Add cancel button functionality to UI

This adds the Cancel option to the dropdown of pending certificates.

+ Adds modal window for Note (may not be required for all issuers, just
Digicert)
+ Add schema for cancel input
+ Fix Digitcert plugin for non-existant orders

When an order is actually issued, then attempting to cancel will return
a 403 from Digicert.  This is a case where it should only be done once
we know the pending cert has been sitting for too long.

Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0
Tickets: PBL-36910

* Fix test_pending_cancel UT

This change creates and injects a pending cert, which will then be used
for the ID so it can be canceled by the unit test.

Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346
Tickets: PBL-36343

* Fix test_digicert on non-existent order

cancelling a non-existent order is fine since we're cancelling it

Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c
Tickets: PBL-36343

* Add migrations for PendingCertificates

Added revision for Pending Certificates table and foreign key mapping
tables.

Change-Id: Ife8202cef1e6b99db377851264639ba540b749db
Tickets: n/a

* Fix relationship copy from Pending to Certificate

When a Pending Certificate is changed to a full Certificate, the
relationship fields are not copied via vars() function, as it's not a
column but mapped via association table.  This adds an explicit copy for
these relations.  Which will properly copy them to the new Certificate,
and thus also update destinations.

Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685
Tickets: PBL-36343

* Fix renaming of certificates and unit tests

The rename flag was not used to rename certificates on creation as
expected.

Fixed unit test, instead of expunging the session, just copy the
pending_certificate so we don't have a weird reference to the object
that can't be copied via vars() function.

Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0
Tickets: PBL-36343

* Updated developer docs for async certs

Added blurb for implementing new issuer functions.

Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c
Tickets: n/a
2018-02-22 08:13:16 -08:00
James Chuong 763c5e8356 Add DIGICERT_ORDER_TYPE to Digicert plugin (#1025)
* Add DIGICERT_ORDER_TYPE to Digicert plugin

This allows lemur.conf.py to control which kind of certificate to
order.  User defined options are not currently supported in the the UI,
so we cannot create multiple Digicert authorities at runtime for
separate certificate types.

Change-Id: I06c216ec3c476e0001b240530626a86464be999e

* Fix Mock URL for Digicert test

Change-Id: Ida7c0ed1bd120c9024bea091c03b7d1ecfa66498

* Add documentation for DIGICERT_ORDER_TYPE

Change-Id: I0bc347883b628416eb7f13a7c60c937dcb6ae0c2
2018-01-13 18:06:17 -08:00
Josh Becigneul dfb9e3a0c8 Add nodejs-legacy to provide the 'node' command (#1004)
Affecting Ubuntu 16.04.3 LTS:

Following the directions of http://lemur.readthedocs.io/en/latest/quickstart/index.html, the make release command fails as the command 'node' cannot be found.

Adding nodejs-legacy solves the issue and allows the build to complete.

(lemur) lemur@lemur1:/www/lemur$ make release
--> Installing dependencies
npm install
npm WARN deprecated gulp-minify-css@1.2.4: Please use gulp-clean-css
npm WARN deprecated bower@1.8.2: ...psst! Your project can stop working at any moment because its dependencies can change. Prevent this by migrating to Yarn                                                                                 : https://bower.io/blog/2017/how-to-migrate-away-from-bower/
npm WARN deprecated gulp-foreach@0.1.0: Either use gulp-tap or gulp-flatmap, depending on your needs
npm WARN deprecated express@2.5.11: express 2.x series is deprecated
npm WARN deprecated connect@1.9.2: connect 1.x series is deprecated
npm WARN deprecated minimatch@2.0.10: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated minimatch@0.2.14: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated graceful-fs@1.2.3: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as poss                                                                                 ible. Use 'npm ls graceful-fs' to find it in the tree.
npm WARN deprecated node-uuid@1.4.8: Use uuid module instead
npm WARN deprecated minimatch@0.3.0: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN prefer global marked@0.3.6 should be installed with -g

> optipng-bin@3.1.4 postinstall /www/lemur/node_modules/optipng-bin
> node lib/install.js

sh: 1: node: not found
npm WARN install:optipng-bin@3.1.4 optipng-bin@3.1.4 postinstall: `node lib/install.js`
npm WARN install:optipng-bin@3.1.4 spawn ENOENT

> jpegtran-bin@3.2.0 postinstall /www/lemur/node_modules/jpegtran-bin
> node lib/install.js

sh: 1: node: not found
npm WARN install:jpegtran-bin@3.2.0 jpegtran-bin@3.2.0 postinstall: `node lib/install.js`
npm WARN install:jpegtran-bin@3.2.0 spawn ENOENT

> gifsicle@3.0.4 postinstall /www/lemur/node_modules/gifsicle
> node lib/install.js

sh: 1: node: not found
npm WARN install:gifsicle@3.0.4 gifsicle@3.0.4 postinstall: `node lib/install.js`
npm WARN install:gifsicle@3.0.4 spawn ENOENT

> Lemur@ postinstall /www/lemur
> bower install --allow-root --config.interactive=false

/usr/bin/env: ‘node’: No such file or directory


Makefile:24: recipe for target 'release' failed
make: *** [release] Error 1
(lemur) lemur@lemur1:/www/lemur$ which node
(lemur) lemur@lemur1:/www/lemur$

Installing the package to solve the issue.

vsnine@lemur1:~$ sudo apt-get install nodejs-legacy
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  nodejs-legacy
0 upgraded, 1 newly installed, 0 to remove and 79 not upgraded.
Need to get 27.7 kB of archives.
After this operation, 81.9 kB of additional disk space will be used.
Get:1 http://ca.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 nodejs-legacy all 4.2.6~dfsg-1ubuntu4.1 [27.7 kB]
Fetched 27.7 kB in 0s (52.4 kB/s)
Selecting previously unselected package nodejs-legacy.
(Reading database ... 73230 files and directories currently installed.)
Preparing to unpack .../nodejs-legacy_4.2.6~dfsg-1ubuntu4.1_all.deb ...
Unpacking nodejs-legacy (4.2.6~dfsg-1ubuntu4.1) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up nodejs-legacy (4.2.6~dfsg-1ubuntu4.1) ...
vsnine@lemur1:~$ which node
/usr/bin/node
vsnine@lemur1:~$
2017-11-27 09:37:14 -08:00
Johannes Langer 9319dda0ec Added ability to ignore cert for oauth2 provider (#971)
* Added ability to ignore cert for oauth2 provider

This is useful for development environments where the OAuth provider
doesn't have a valid cert!

* Setting default for OAUTH2_VERIFY_CERT to true
2017-10-20 16:36:14 -07:00
ClontarfX ea6f5c920b Update index.rst (#942)
Fixed typo for libsasl3-dev (was libsas13-dev).
2017-09-27 09:44:03 -07:00
Marti Raudsepp 97d83890e0 Various minor cleanups and fixes (#938)
* Documentation fixes

* Various docstring and help string fixes

* Minor code cleanups

* Removed redundant .gitignore entry, ignored package-lock.json.
* 'return' statement in certificates.service.render was redundant
* Split up too long line
* Non-matching tags in templates
2017-09-25 15:33:42 -07:00
Marti Raudsepp ec5dec4a16 Add option to disable owner email address in CSR subject (#939) 2017-09-25 15:32:08 -07:00
Aaron Mell 444be5bb7f Updated Quikstart (#934)
Got some failures doing a clean install on Ubuntu 17.04 Zesty Zapus (Final) from virtualboxes.org
2017-09-22 12:35:25 -07:00
Ian Stahnke a6dab5e1ee a bit more ldap documentaion (#930) 2017-09-21 06:00:26 -07:00
Francisco Santos 2ecfaa41cf Add pyldap mock for readthedocs (#912) 2017-09-11 09:18:03 -07:00
Francisco Santos 7106c4fdcf Sync docs requirements.txt (#910) 2017-09-10 10:41:46 -07:00
Ian Stahnke ff4d1edd63 remove duplicated ldap_bind_uri description (#898) 2017-09-04 10:12:40 -07:00
Ian Stahnke 79d12578c7 basic ldap support (#842) 2017-09-03 20:41:43 -07:00
Marti Raudsepp 82b43b5a9d Create signal hooks and handler for dumping CSR and certificate details (#882) 2017-08-28 17:35:56 -07:00
kevgliss aca6d6346f Removing legacy requirement for nodejs. Closes #866 (#887) 2017-08-25 10:12:56 -07:00
Marti Raudsepp 7762d6ed52 Reworked sensitive domain name and restriction logic (#878)
* This is a fix for a potential security issue; the old code had edge
  cases with unexpected behavior.
* LEMUR_RESTRICTED_DOMAINS is no more, instead LEMUR_WHITELISTED_DOMAINS
  is a list of *allowed* domain name patterns. Per discussion in PR #600
* Domain restrictions are now checked everywhere: in domain name-like
  CN (common name) values and SAN DNSNames, including raw CSR requests.
* Common name values that contain a space are exempt, since they cannot
  be valid domain names.
2017-08-16 19:24:49 -07:00
kevgliss 7507f6be50 Updating documentation (#849) 2017-07-05 20:17:19 -07:00
Asbjørn Kjær 35cc7ef8d7 Adding support for private DigiCert certificates (#835) 2017-06-14 09:20:24 -07:00
Henry Megarry 31f4cf0253 adding url context path to html templates (#814) 2017-05-25 10:20:32 -07:00
Arthur Lutz 97dceb5623 fixed typo in supervisord example config (#790) 2017-05-12 09:18:32 -07:00
marc-sensenich 23b6df536f Fix Minor Typo in index.rst (#793)
Changed LEMUR_DEFAUTL_ORGANIZATION to LEMUR_DEFAULT_ORGANIZATION
2017-05-12 09:17:52 -07:00
csv7 381cd2e1ff Updated apache config (#776)
You guys asked for one that worked... It took me a little while to tweak, esp. since I'm not a guru with python.  The comment about needing mod_wsgi isn't true, unless you want to run lemur as a cgi program... I suspect that's from an older version that ran as cgi and not as a standalone webserver.
2017-05-04 08:45:55 -07:00
kevgliss 2a2d5a5583 Adding an example digicert url. Closes #700. (#775) 2017-05-01 10:59:49 -07:00
kevgliss 6367a98134 Creating a user named 'lemur' in postgres (#773)
Creating a user named 'lemur' in postgres
2017-04-28 15:31:08 -07:00
Rick Breidenstein 0bbe2b0331 config LEMUR_MAIL to LEMUR_EMAIL (#772)
I referenced https://github.com/Netflix/lemur/blob/master/lemur/plugins/lemur_email/plugin.py and it appears this configuration option should be "LEMUR_EMAIL"
2017-04-28 15:01:21 -07:00
Paul Van de Vreede 989e3733a2 Add docker setup for running tests on a docker enabled dev environment. (#771) 2017-04-28 09:28:06 -07:00
Paul Van de Vreede 604cd60dbe Return correct intermediate certificate on digicert creation. (#762)
This commit also removes the unused DIGICERT_INTERMEDIATE env
var as it is not used.
2017-04-27 09:14:20 -07:00
Michael Treacher f90076abe9 Update index.rst (#754)
Seems the api for these actions have changed. Thought I would update the documentation around this. Let me know if I've misunderstood something.
2017-04-19 16:06:32 -07:00
Rick Breidenstein 221851abc1 supervisor ; cause services not to start (#744)
the ; in the supervisor/conf.d/app.conf file cause the service not to start.
2017-04-06 09:21:13 -07:00
Brint O'Hearn e18a188723 Spell fixes in docs (#740) 2017-03-30 21:09:30 -07:00
Henry Megarry dd39b9ebe8 adding url context path to build, adding documentation on url contextpath (#737) 2017-03-28 15:21:13 -07:00
Jason Spriggs 15896a3b11 Fix spelling error in LEMUR_DEFAULT_COUNTRY (#734) 2017-03-22 15:49:16 -07:00
Rick Breidenstein a4707c5fc9 added a few steps (#731)
Added a few steps that are needed during the install on a fresh Ubuntu image
2017-03-18 21:36:26 -07:00
Rick Breidenstein 7b4d31d4f6 added steps for loading custom plugin (#725)
* added steps for loading custom plugin

added steps for loading a custom plugin into Lemur once the files have been put into place (/www/lemur/lemur/plugins/) and the setup.py file (/www/lemur/setup.py) has been modified.

* updated __init__.py section


except Exception as e:
2017-03-14 09:30:22 -07:00
Rick Breidenstein 522e182694 added python3-dev to dependencies (#724)
make release fails without it
2017-03-13 15:45:10 -07:00
Rick Breidenstein 6c8a6620d2 specify python3 when creating virtualenv (#723)
Lemur is developed against Python3.5. If you do not specify the Python version it is possible the virtualenv will be built on a different version.
2017-03-13 13:58:44 -07:00
kevgliss a4068001a3 Updating docs to align with normal deployment. (#718) 2017-03-12 15:01:21 -07:00
Nevins 0326e1031f adding generic OAuth2 provider (#685)
* adding support for Okta Oauth2

* renaming to OAuth2

* adding documentation of options

* fixing flake8 problems
2017-02-03 10:36:49 -08:00
kevgliss 9f6ad08c50 Updating hooks. (#660) 2017-01-18 14:16:31 -08:00
Marti Raudsepp b327963925 Plugin base classes: update method signatures & fix raise (#598)
This way IDEs can verify method overrides in subclasses, otherwise these
are flagged as erroneous.

Changed base classes to properly raise NotImplementedError; previously
they would cause "TypeError: exceptions must derive from BaseException"

Also fixed exception handling in sources.service.clean().
2016-12-14 13:42:29 -08:00
Marti Raudsepp 71ddbb409c Minor documentation fixes/tweaks (#597)
Mostly typos, grammar errors and inconsistent indentation in code
examples.

Some errors detected using Topy (https://github.com/intgr/topy), all
changes verified by hand.
2016-12-14 09:29:04 -08:00
kevgliss fbcedc2fa0 Specifying a recommended postgres version (#592) 2016-12-13 11:22:10 -08:00
kevgliss a40bc65fd4 Default authority. (#549)
* Enabling the specification of a default authority, if no default is found then the first available authority is selected

* PEP8

* Skipping tests relying on keytool
2016-12-01 15:42:03 -08:00
kevgliss f141ae78f3 Typo. (#485) 2016-11-10 14:40:59 -08:00
kevgliss 89470a0ce0 Adding default validity and retry logic. (#483) 2016-11-10 11:23:37 -08:00
kevgliss 25a6c722b6 Adding digicert documentation. (#480) 2016-11-08 14:56:05 -08:00