* Fixes for Bandit
This commit fixes a couple of issues so that Bandit can run
cleanly using medium+ severity and confidence filtering.
* Adding Lemur Bandit job to TravisCI
* Ensures that both AKI serial/issue _and_ keyid won't be included.
Validation issues crop up if both types of AKI fields are present.
* Ensure that SAN extension includes the certificate's common name
* Fix scenario where subAltNames are getting dropped when applying a template
* Ensure that SAN includes the CN
* Ensuring that getting here without a SAN extension won't break things.
* New cleaner approach
* Some bits of handling the extensions are a bit hacky, requiring access to attributes inside the objects in x509.
I think this is pretty clean though.
* lintian check
* Fixing tests
* Allow owner to be specified when syncing certs.
* Ensuring non-endpoint plugins don't fail to complete syncing.
* Adding in some additional error handling.
* Adds additional constraints to the max notification time. With an increasing number of certificates we need to limit the max notification time to reduce the number of certificates that need to be analyzed for notification eligibility.
* UI adjustments to make Key Agreement, Encipher Only, and Decipher Only relationship more user-friendly
* whitespace typo
* Issue #663 switching Encipher/Decipher Only options to be mutually exclusive and un-checkable radio buttons.
* Found a bug in the fields schema that was dropping Key Agreement bit if encipher/decipher only weren't checked
* subAltNames were getting wiped out every time a template was selected
* isCritical variables aren't presented in the UI, nor is this information used in determining to use them.
* Renaming the function so it sounds less root-specific
* Refactoring lemur_cryptography
* Adding to the certificate interface an easy way to request the subject and public_key of a certificate
* Turning the create authority functionality into a wrapper of creating a CSR in the certificate codebase and issueing that certificate in this plugin. (Dependent on https://github.com/Netflix/lemur/pull/666 changes first)
* Ensuring that intermediate certificates and signed certificates retain their chain cert data
* Handling extensions that are the responsibility of the CA
Implementing authority_key_identifier for lemur_cryptography signatures and including skeletons of handling the certificate_info_access and crl_distribution_points
* Fixing errors found with linter
* Updating plugin unit tests
* Changing this for Python3. Underlying cryptography library expects these to be bytes now.
* Updating tests to match new function names/interfaces
* Another naming update in the plugin tests
* Appears that create_csr won't like this input without an owner.
* Undoing last commit and putting it into the right place this time.
* create_csr should be good now with these options, and chain certs will be blank in tests
* This won't be blank in issue_certificate, like it will in creating an authority.
* Much cleaner
* unnecessary import
* Allowing that create_csr can be called with an additional flag in the csr_config to adjust the BasicConstraints for a CA.
* If there are no SANs, skip adding a blank list of SANs.
* Adding handling for all the extended key usage, key usage, and subject key identifier extensions.
* Fixing lint checks. I was overly verbose.
* This implements marshalling of the certificate extensions into x509 ExtensionType objects in the schema validation code.
* Will create x509 ExtensionType objects in the schema validation stage
* Allows errors parsing incoming options to bubble up to the requestor as ValidationErrors.
* Cleans up create_csr a lot in the certificates/service.py
* Makes BasicConstraints _just another extension_, rather than a hard-coded one
* Adds BasicConstraints option for path_length to the UI for creating an authority
* Removes SAN types which cannot be handled from the UI for authorities and certificates.
* Fixes Certificate() object model so that it doesn't just hard-code only SAN records in the extensions property and actually returns the extensions how you expect to see them. Since Lemur is focused on using these data in the "CSR" phase of things, extensions that don't get populated until signing will be in dict() form.* Trying out schema validation of extensions
* Aligning certificate creation between authority and certificate workflows
* Correctly missing and mis-named fields in schemas
* Re-ordering KeyUsage and ExtendedKeyUsage for consistency and clarity
* Adding client authentication to the authority options.
* Missing blank lines for pyflakes linting
* Updating tests for new fields/names/typos
* is_critical wasn't in the schema, so was getting dropped.
* isCritical in the Javascript wasn't getting assigned if it was unchecked. Now, it will be assumed false if missing.
* The display of critical or not in the list of added custom OIDs was unclear when it was just true/false with no heading. Now it will be displayed as critical or nothing instead.
* The namespace for the checkbox for isCritical was wrong, and didn't get processed with the oid/type/value variables.
* Combining Authority Key Identifier extension options in the schema.
This makes processing them in the cert/csr generation stage make more sense because they are two options in the same x.509 extension. They were already in the same part of the schema for authorities, but this makes the certificates follow the same pattern, and it allows them to share the same schema/validation layout.
* Updating schema tests to match changes
* Fixing an idiot typo
* I promise to stop using Travis as a typo-corrector soon.
* Fixing an IAM syncing issue. Were duplicates were not properly sync'd with Lemur. This resulted in a visibility gap. Even 'duplicates' need to sync'd to Lemur such that we can track rotation correctly. Failing on duplicates lead to missing those certificates and the endpoints onto which they were deployed. This commit removes the duplicate handling altogether.
* Fixing tests.
This way IDEs can verify method overrides in subclasses, otherwise these
are flagged as erroneous.
Changed base classes to properly raise NotImplementedError; previously
they would cause "TypeError: exceptions must derive from BaseException"
Also fixed exception handling in sources.service.clean().
Mostly typos, grammar errors and inconsistent indentation in code
examples.
Some errors detected using Topy (https://github.com/intgr/topy), all
changes verified by hand.
* Enabling the specification of a default authority, if no default is found then the first available authority is selected
* PEP8
* Skipping tests relying on keytool
* Initial work on certificate rotation.
* Adding ability to get additional certificate info.
* - Adding endpoint rotation.
- Removes the g requirement from all services to enable easier testing.
Configure werkzeug to output JSON error messages for the benefit of
downstream clients. This also allows for metrics collection in all cases
where werkzeug is outputting an exception.
* Implement CFSSL issuer plugin
Implement a Lemur plugin for generating certificates from the open
source certificate authority CFSSL
(https://github.com/cloudflare/cfssl). The plugin interacts with CFSSL
through the CFSSL REST API. The CFSSL configuration is defined in the
lemur.conf.py property file using property names prefixed with "CFSSL_".
* Update documentation to include CFSSL plugin
* Renaming 'active' to 'notify' as this is clearer and more aligned to what this value is actually controlling. 'active' is now a property that depends on whether any endpoints were found to be using the certificate. Also added logic for issue #405 disallowing for a certificates' notifications to be silenced when it is actively deployed on an endpoint.
* Adding migration script to alter 'active' column.
When exporting a certificate, the password is an optional parameter.
When a password is not supplied by the caller, a default password is
generated by the method. The generation library creates the random
password as a bytes object. The bytes object raises an error in the
'keytool' command used to export the certificate. The keytool is
expecting the password to be a str object.
The fix is to decode the generated password from a bytes object to a str
object.
The associated Java plugin tests have been updated to verify the export
method returns the password as a str object. In addition, the tests have
been updated to correctly test the export methods response object. The
original tests treated the response as a single object. The current
export methods return a tuple of data (type, password, data).
In order to make the tests compatible with both Python2 and Python3, the
'six' library was used to test the password is in fact a string.
* Update the private key regex validation
Private keys provided by the Let's Encrypt certificate authority as part
of their certificate bundle fail the import/upload certificate private
key validation. The validation is looking for a specific character
sequence at the begin of the certificate. In order to support valid
Let's Encrypt private keys, the regex has been updated to check for both
the existing sequence and the Let's Encrypt character sequence.
Example Let's Encrypt private key:
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCvsiwV8A5+r0tQ
QzUAJO0DfoEb9tMWvoFi0DLs9tx88IwMqItPPl9+RNzQnv9qqZR1h4W97sxP8aWY
...
AeS667IJO/2DMKaGiEldaVZtgqdUhCL8Rm4XUFVb1GjLa03E4VRU6W7eQ4hgT2a7
cHDAR8MiovNyfT0fm8Xz3ac=
-----END PRIVATE KEY-----
* Add private key regex for footer
Update the import/upload private key validation regex to verify both the
header and footer are matching.
* Activate the AWS S3 destination plugin
Add the AWS S3 destination plugin to the list of available Lemur
plugins.
Update the S3 destination plugin's "accountNumber" option to be of type
'str' to handle account numbers starting with zeros.
Update Lemur's utils for parsing certificates to correctly encode the
X509 certificates before loading for python3.
* Add S3 destination plugin test
Added simple test to verify S3 destination plugin is available.
When importing a certificate, the private key is passed to the
import/upload process from the UI as a str object. In Python3 this
raises two issues when processing the private key - the private key
validation fails and database insert of the certificate fails.
The fix in both cases is to correctly encode the private key as a bytes
object.
* Fix test certificates module hanging issue
When executing the lemur/tests/test_certificates.py module's tests, all
tests are executed, but the test process appears to hang and never
completes with the display of the results for the tests.
The hanging issue is traced to the two test methods:
test_import(logged_in_user) and test_upload(logged_in_user). The issue
has to do with the test methods' using the logged_in_user(app) fixture from
the conftest.py module as the method parameter.
The test methods at issue require the session, db, and app fixtures to
be initialized for the tests to complete successfully. The
logged_in_user() fixture only initializes the app fixture. Updating the
test_import() and test_upload() methods parameters to be the "session"
fixture fixes the hanging issue and the tests complete successfully.
This is the command being used to execute the tests...
$ py.test -s -v lemur/tests/test_certificates.py
* Update fix for test certificates hanging issue
Based on feedback from the original pull request for this fix, added the
session fixture to the logged_in_user fixture and reverted the
test_import() and test_upload() methods to use the logged_in_user
(instead of the session fixture).
Lemur's documentation already mentions LEMUR_RESTRICTED_DOMAINS, a list
of regular expressions matching domains only administrators can issue
certificates for. An option to mark domains as sensitive existed in the
API, however the configuration option was not implemented.
Now both ways of sensitivity are checked in the same place.
* Fixed an issue were default notifications were added even when updating a certificate, resulting in duplicate notifications.
* Ensuring imported certificates get the same treatment.
