Configure werkzeug to output JSON error messages for the benefit of
downstream clients. This also allows for metrics collection in all cases
where werkzeug is outputting an exception.
* Implement CFSSL issuer plugin
Implement a Lemur plugin for generating certificates from the open
source certificate authority CFSSL
(https://github.com/cloudflare/cfssl). The plugin interacts with CFSSL
through the CFSSL REST API. The CFSSL configuration is defined in the
lemur.conf.py property file using property names prefixed with "CFSSL_".
* Update documentation to include CFSSL plugin
* Renaming 'active' to 'notify' as this is clearer and more aligned to what this value is actually controlling. 'active' is now a property that depends on whether any endpoints were found to be using the certificate. Also added logic for issue #405 disallowing for a certificates' notifications to be silenced when it is actively deployed on an endpoint.
* Adding migration script to alter 'active' column.
Following the quickstart instructions, I ran into issues at `make develop` because
several dependencies couldn't be resolved. Several errors like this in the output
of `npm install`:
```
npm ERR! TypeError: Cannot read property 'latest' of undefined
npm ERR! at next (/usr/share/npm/lib/cache.js:687:35)
npm ERR! at /usr/share/npm/lib/cache.js:675:5
npm ERR! at saved (/usr/share/npm/node_modules/npm-registry-client/lib/get.js:142:7)
npm ERR! at /usr/lib/nodejs/graceful-fs/polyfills.js:133:7
npm ERR! at Object.oncomplete (fs.js:107:15)
npm ERR! If you need help, you may report this log at:
npm ERR! <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR! <npm-@googlegroups.com>
npm ERR! System Linux 3.13.0-92-generic
npm ERR! command "/usr/bin/nodejs" "/usr/bin/npm" "install"
npm ERR! cwd /home/lemur/lemur
npm ERR! node -v v0.10.25
npm ERR! npm -v 1.3.10
npm ERR! type non_object_property_load
```
`npm list` yielded this output at the bottom:
```
npm ERR! missing: @types/jasmine@^2.2.31, required by protractor@4.0.9
npm ERR! missing: @types/node@^6.0.35, required by protractor@4.0.9
npm ERR! missing: @types/q@^0.0.30, required by protractor@4.0.9
npm ERR! missing: @types/selenium-webdriver@~2.53.30, required by protractor@4.0.9
npm ERR! missing: adm-zip@0.4.7, required by protractor@4.0.9
npm ERR! missing: chalk@^1.1.3, required by protractor@4.0.9
npm ERR! missing: glob@^7.0.3, required by protractor@4.0.9
npm ERR! missing: jasmine@2.5.2, required by protractor@4.0.9
npm ERR! missing: jasminewd2@0.0.10, required by protractor@4.0.9
npm ERR! missing: optimist@~0.6.0, required by protractor@4.0.9
npm ERR! missing: q@1.4.1, required by protractor@4.0.9
npm ERR! missing: saucelabs@~1.3.0, required by protractor@4.0.9
npm ERR! missing: selenium-webdriver@2.53.3, required by protractor@4.0.9
npm ERR! missing: source-map-support@~0.4.0, required by protractor@4.0.9
npm ERR! missing: webdriver-manager@^10.2.2, required by protractor@4.0.9
npm ERR! not ok code 0
```
lemur depends explicitly on gulp-protractor 0.0.11 explicitly
gulp-protractor 0.0.11 depends on protractor at _any_ version (*)
The latest versions of protractor (@4) require much newer versions of nodejs
according to the Compatibility section of
https://www.npmjs.com/package/protractor.
gulp-protractor 0.0.12 fixes and constrains some of these dependencies better
and adds a debug option, and fixes a few typos in comments and metadata.
https://github.com/mllrsohn/gulp-protractor/compare/0.0.11...0.0.12
When exporting a certificate, the password is an optional parameter.
When a password is not supplied by the caller, a default password is
generated by the method. The generation library creates the random
password as a bytes object. The bytes object raises an error in the
'keytool' command used to export the certificate. The keytool is
expecting the password to be a str object.
The fix is to decode the generated password from a bytes object to a str
object.
The associated Java plugin tests have been updated to verify the export
method returns the password as a str object. In addition, the tests have
been updated to correctly test the export methods response object. The
original tests treated the response as a single object. The current
export methods return a tuple of data (type, password, data).
In order to make the tests compatible with both Python2 and Python3, the
'six' library was used to test the password is in fact a string.
* Update the private key regex validation
Private keys provided by the Let's Encrypt certificate authority as part
of their certificate bundle fail the import/upload certificate private
key validation. The validation is looking for a specific character
sequence at the begin of the certificate. In order to support valid
Let's Encrypt private keys, the regex has been updated to check for both
the existing sequence and the Let's Encrypt character sequence.
Example Let's Encrypt private key:
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCvsiwV8A5+r0tQ
QzUAJO0DfoEb9tMWvoFi0DLs9tx88IwMqItPPl9+RNzQnv9qqZR1h4W97sxP8aWY
...
AeS667IJO/2DMKaGiEldaVZtgqdUhCL8Rm4XUFVb1GjLa03E4VRU6W7eQ4hgT2a7
cHDAR8MiovNyfT0fm8Xz3ac=
-----END PRIVATE KEY-----
* Add private key regex for footer
Update the import/upload private key validation regex to verify both the
header and footer are matching.
* Activate the AWS S3 destination plugin
Add the AWS S3 destination plugin to the list of available Lemur
plugins.
Update the S3 destination plugin's "accountNumber" option to be of type
'str' to handle account numbers starting with zeros.
Update Lemur's utils for parsing certificates to correctly encode the
X509 certificates before loading for python3.
* Add S3 destination plugin test
Added simple test to verify S3 destination plugin is available.
When importing a certificate, the private key is passed to the
import/upload process from the UI as a str object. In Python3 this
raises two issues when processing the private key - the private key
validation fails and database insert of the certificate fails.
The fix in both cases is to correctly encode the private key as a bytes
object.
* Fix test certificates module hanging issue
When executing the lemur/tests/test_certificates.py module's tests, all
tests are executed, but the test process appears to hang and never
completes with the display of the results for the tests.
The hanging issue is traced to the two test methods:
test_import(logged_in_user) and test_upload(logged_in_user). The issue
has to do with the test methods' using the logged_in_user(app) fixture from
the conftest.py module as the method parameter.
The test methods at issue require the session, db, and app fixtures to
be initialized for the tests to complete successfully. The
logged_in_user() fixture only initializes the app fixture. Updating the
test_import() and test_upload() methods parameters to be the "session"
fixture fixes the hanging issue and the tests complete successfully.
This is the command being used to execute the tests...
$ py.test -s -v lemur/tests/test_certificates.py
* Update fix for test certificates hanging issue
Based on feedback from the original pull request for this fix, added the
session fixture to the logged_in_user fixture and reverted the
test_import() and test_upload() methods to use the logged_in_user
(instead of the session fixture).
The pre-commit module contained a reference to the variable
DEFAULT_CONFIG which does not exist in the version of flake8 packaged
with the project. The DEFAULT_CONFIG defines the path to the current
user's local flake8 config file.
The flake8 2.6.2 version packaged with project has been refactored to
set the local flake8 config file in the flake8/main.py module.
Lemur's documentation already mentions LEMUR_RESTRICTED_DOMAINS, a list
of regular expressions matching domains only administrators can issue
certificates for. An option to mark domains as sensitive existed in the
API, however the configuration option was not implemented.
Now both ways of sensitivity are checked in the same place.
