- update vault destination to support multi cert under one object
- added san list as key value
- read and update object with new keys, keeping other keys, allowing
us to keep an iterable list of keys in an object for deploying multiple
certs to a single node
The plugin now supports loading details from local files rather than requiring them to be entered through the UI. This is especially relaent when Lemur is deployed on Kubernetes as the certificate, token, and current namespace will be injected into the pod. The location these details are injected are the defaults if no configuration details are supplied.
The plugin now supports deploying the secret in three different formats:
* Full - matches the formate used by the plugin prior to these changes.
* TLS - creates a secret of type kubernetes.io/tls and includes the certificate chain and private key, this format is used by many kubernetes features.
* Certificate - creates a secret containing only the certificate chain, suitable for use as trust authority where private keys should _NOT_ be deployed.
The deployed secret can now have a name set through the configuration options; the setting allows the insertion of the placeholder '{common_name}' which will be replaced by the certificate's common name value.
Debug level logging has been added.
The Kubernetes plugin was broken. There were two major issues:
* The server certificate was entered in a string input making it impossible (as far as I know) to enter a valid PEM certificate.
* The base64 encoding calls were passing strings where bytes were expected.
The fix to the first issue depends on #2218 and a change in the options structure. I've also included some improved input validation and logging.
The destination configuration page did not previously support a textarea input as was supported on most other pages. The validation of string inputs was not being performed. This commit addresses both of those issues and corrects the validation expressions for the AWS and S3 destination plugins so that they continue to function. The SFTP destination plugin does not have any string validation. The Kubernetes plugin does not work at all as far as I can tell; there will be another PR in the coming days to address that.
In preparation for certificate integrity-checking: invalid certificate
chains and mismatching private keys will no longer be allowed anywhere
in Lemur code.
The test vector certs were generated using the Lemur "cryptography"
authority plugin.
* Certificates are now more similar to real-world usage: long serial
numbers, etc.
* Private key is included for all certs, so it's easy to re-generate
anything if needed.
* ecc: add the support for ECC
update generate_private_key to support ECC. Move key types to constant. Update UI for the new key types
* ecc: Remove extra line to fix linting
* ecc: Fix flake8 lint problems
* Update options.tpl.html
* lemur_digicert: Do not truncate valid_to anymore
The valid_to field for Digicert supports YYYY-MM-DDTHH:MM:SSZ so we should stop truncating
* lemur_digicert: Update unit tests for valid_to
* Add PendingCertificate model
This change creates a DB table called pending_certificates and
associated mapping relationship tables from pending certificate to
roles, rotation policy, destination, sources, etc.
The table is generated on initialization of Lemur. A pending
certificate holds most of the information of a Certificate, while it has
not be issued so that it can later backfill the information when the CA
has issued the certificate.
Change-Id: I277c16b776a71fe5edaf0fa0e76bbedc88924db0
Tickets: PBL-36499
* Create a PendingCertificate if cert is empty
IssuePlugins should return empty cert bodies if the request failed to
complete immediately (such as Digicert). This way, we can immediately
return the certificate, or if not just place into PendingCertificates
for later processing.
+ Fix relation from Certificate to Pending Certificate, as view only.
There is no real need for anything more than that since Pending cert
only needs to know the cert to replace when it is issued later.
+ Made PendingCertificate private key be empty: UI does not allow
private key on 'Create' but only on 'Import'. For Instart, we require
the private key but upstream does not necessarily need it. Thus, if
someone at Instart wants to create a CSR / key combo, they should
manually issue the cert themselves and import later. Otherwise you
should let Lemur generate that. This keeps the workflow transparent for
upstream Lemur users.
Change-Id: Ib74722a5ed5792d4b10ca702659422739c95ae26
Tickets: PBL-36343
* Fix empty private_key when create Pending Cert
On creation of a certificate with a CSR, there is no option for private
key. In this case, we actually have a dictionary with private_key as
key, but the value is None. This fixes the strip() called on NoneType.
Change-Id: I7b265564d8095bfc83d9d4cd14ae13fea3c03199
Tickets: PBL-36499
* Source sync finds and uses pending certificate
When a source syncs certificates, it will check for a pending
certificate. If that is found via external_id (given by digicert as
order_id) then it will use the found Pending Certificate's fields to
create a new certificate. Then the pending certificate is deleted.
Tickets: PBL-36343
Change-Id: I4f7959da29275ebc47a3996741f7e98d3e2d29d9
* Add Lemur static files and views for pending certs
This adds the basic static files to view pending certificates in a
table.
Tickets: PBL-36343
Change-Id: Ia4362e6664ec730d05d280c5ef5c815a6feda0d9
* Add CLI and plugin based pending fetch
This change uses the adds a new function to issuer plugins to fetch
certificates like source, but for one order. This way, we can control
which pending certificates to try and populate instead of getting all
certificates from source.
Tickets: PBL-36343
Change-Id: Ifc1747ccdc2cba09a81f298b31ddddebfee1b1d6
* Revert source using Pending Certificate
Tickets: PBL-36343
Change-Id: I05121bc951e0530d804070afdb9c9e09baa0bc51
* Fix PendingCertificate init getting authority id
Should get authority id from authority.id instead of the authority_id
key in kwargs.
Change-Id: Ie56df1a5fb0ab2729e91050f3ad1a831853e0623
Tickets: n/a
* Add fixtures and basic test for PendingCertificate
Change-Id: I4cca34105544d40dac1cc50a87bba93d8af9ab34
Tickets: PBL-36343
* Add User to create_certificate parameters
create_certificate now takes a User, which will be used to populate the
'creator' field in certificates.service.upload(). This allows the UI
populate with the current user if the owner does not exist in Lemur.
+ Fix chain being replaced with version from pending certificate, which
may be empty (depends on plugin implementation).
Change-Id: I516027b36bc643c4978b9c4890060569e03f3049
Tickets: n/a
* Fix permalink and filters to pending certs
Fixes the permalink button to get a single pending certificate
Add argument filter parsing for the pending certificate API
Fix comment on API usage
Added get_by_name for pending_certificate (currently unused, but useful
for CLI, instead of using IDs)
Change-Id: Iaa48909c45606bec65dfb193c13d6bd0e816f6db
Tickets: PBL-36910
* Update displayed fields for Pending Certificates
There are a number of unused / unpopulated fields from Certificate UI
that does apply to Pending Certificates. Those ones were removed, and
added other useful fields:
Owner, number of attempts to fetch and date created
Change-Id: I3010a715f0357ba149cf539a19fdb5974c5ce08b
Tickets: PBL-36910
* Add common name (cn) to Pending Certificate model
Fixes the UI missing the CN for Pending Certificate, as it was
originally being parsed from the generated certificate. In the case of
pending certificate, the CN from the user generates the request, which
means a pending cert can trust the original user putting in the CN
instead of having to parse the not-yet-generated certificate. There is
no real possibility to return a certificate from a pending certificate
where the CN has changed since it was initially ordered.
Change-Id: I88a4fa28116d5d8d293e58970d9777ce73fbb2ab
Tickets: PBL-36910
* Fix missing imports for service filter
+ Removed duplicate get_by_name function from old merge
Change-Id: I04ae6852533aa42988433338de74390e2868d69b
Tickets: PBL-36910
* Add private key viewing to Pending Certificates
Add private key API for Pending Certificates, with the same
authorization as Certificates (only owner, creator or owner-roles can
view private key).
Change-Id: Ie5175154a10fe0007cc0e9f35b80c0a01ed48d5b
Tickets: PBL-36910
* Add edit capability to pending certificates
Like editing certificates, we should be able to modify some parts of a
pending certificate so the resulting certificate has the right
references, owner, etc.
+ Added API to update pending certificate
+ Fix UI to use pending certificate scope instead of reusing Certificate
+ Change pending_certificate.replaces to non-passive association, so
that updates do affect it (similar to roles/notifications/etc)
Tickets: PBL-36910
Change-Id: Ibbcb166a33f0337e1b14f426472261222f790ce6
* Add common_name parsing instead using kwargs
To fix tests where common name may not be passed in, use the CSR
generated to find the official common name.
Change-Id: I09f9258fa92c2762d095798676ce210c5d7a3da4
Tickets: PBL-36343
* Add Cancel to pending certificates and plugins
This allows pending certificates to be cancelled, which will be handled
by the issuer plugin.
Change-Id: Ibd6b5627c3977e33aca7860690cfb7f677236ca9
Tickets: PBL-36910
* Add API for Cancelling Pending Certificate
Added the DELETE handler for pending_certificates, which will cancel and
delete the pending certificate from the pending certs table on
successful cancellation via Issuer Plugin.
+ Add UT for testing cancel API
Change-Id: I11b1d87872e4284f6e4f9c366a15da4ddba38bc4
Tickets: PBL-36910
* Remove Export from Pending Certificates
Pending Certificates doesn't need an export since it should just be
fetched by Lemur via plugins, and the CSR is viewable via the UI.
Change-Id: I9a3e65ea11ac5a85316f6428e7f526c3c09178ae
Tickets: PBL-36910
* Add cancel button functionality to UI
This adds the Cancel option to the dropdown of pending certificates.
+ Adds modal window for Note (may not be required for all issuers, just
Digicert)
+ Add schema for cancel input
+ Fix Digitcert plugin for non-existant orders
When an order is actually issued, then attempting to cancel will return
a 403 from Digicert. This is a case where it should only be done once
we know the pending cert has been sitting for too long.
Change-Id: I256c81ecd142dd51dcf8e38802d2c202829887b0
Tickets: PBL-36910
* Fix test_pending_cancel UT
This change creates and injects a pending cert, which will then be used
for the ID so it can be canceled by the unit test.
Change-Id: I686e7e0fafd68cdaeb26438fb8504d79de77c346
Tickets: PBL-36343
* Fix test_digicert on non-existent order
cancelling a non-existent order is fine since we're cancelling it
Change-Id: I70c0e82ba2f4b8723a7f65b113c19e6eeff7e68c
Tickets: PBL-36343
* Add migrations for PendingCertificates
Added revision for Pending Certificates table and foreign key mapping
tables.
Change-Id: Ife8202cef1e6b99db377851264639ba540b749db
Tickets: n/a
* Fix relationship copy from Pending to Certificate
When a Pending Certificate is changed to a full Certificate, the
relationship fields are not copied via vars() function, as it's not a
column but mapped via association table. This adds an explicit copy for
these relations. Which will properly copy them to the new Certificate,
and thus also update destinations.
Change-Id: I322032ce4a9e3e67773f7cf39ee4971054c92685
Tickets: PBL-36343
* Fix renaming of certificates and unit tests
The rename flag was not used to rename certificates on creation as
expected.
Fixed unit test, instead of expunging the session, just copy the
pending_certificate so we don't have a weird reference to the object
that can't be copied via vars() function.
Change-Id: I962943272ed92386ab6eab2af4ed6d074d4cffa0
Tickets: PBL-36343
* Updated developer docs for async certs
Added blurb for implementing new issuer functions.
Change-Id: I1caed6e914bcd73214eae2d241e4784e1b8a0c4c
Tickets: n/a
* Add DIGICERT_ORDER_TYPE to Digicert plugin
This allows lemur.conf.py to control which kind of certificate to
order. User defined options are not currently supported in the the UI,
so we cannot create multiple Digicert authorities at runtime for
separate certificate types.
Change-Id: I06c216ec3c476e0001b240530626a86464be999e
* Fix Mock URL for Digicert test
Change-Id: Ida7c0ed1bd120c9024bea091c03b7d1ecfa66498
* Add documentation for DIGICERT_ORDER_TYPE
Change-Id: I0bc347883b628416eb7f13a7c60c937dcb6ae0c2
This plugin allows a certificate to be exported as a CSR via OpenSSL
x509. The workflow will be:
* Create self-signed cert via Cryptography authority
* Export CSR via this plugin
* Sign your own cert outside of Lemur
* Import new cert with private key
Change-Id: Id3f7db2506bd959236cd3a6df622841058abda5a
* Changes required for functional Route53 operations
* Changes required for functional ACME operations with Route53
* Changes required for functional ACME operations with Route53, need external ID
* Documentation fixes
* Various docstring and help string fixes
* Minor code cleanups
* Removed redundant .gitignore entry, ignored package-lock.json.
* 'return' statement in certificates.service.render was redundant
* Split up too long line
* Non-matching tags in templates
* Adds the ability for destination plugins to be sub-classed from ExportDestination. These plugins have the extra option of specifying an export plugin before the destination receives the data. Closes#807.
* fixing tests
* Fixes for Bandit
This commit fixes a couple of issues so that Bandit can run
cleanly using medium+ severity and confidence filtering.
* Adding Lemur Bandit job to TravisCI
* Ensures that both AKI serial/issue _and_ keyid won't be included.
Validation issues crop up if both types of AKI fields are present.
* Ensure that SAN extension includes the certificate's common name
* Fix scenario where subAltNames are getting dropped when applying a template
* Ensure that SAN includes the CN
* Ensuring that getting here without a SAN extension won't break things.
* New cleaner approach
* Some bits of handling the extensions are a bit hacky, requiring access to attributes inside the objects in x509.
I think this is pretty clean though.
* lintian check
* Fixing tests
* Renaming the function so it sounds less root-specific
* Refactoring lemur_cryptography
* Adding to the certificate interface an easy way to request the subject and public_key of a certificate
* Turning the create authority functionality into a wrapper of creating a CSR in the certificate codebase and issueing that certificate in this plugin. (Dependent on https://github.com/Netflix/lemur/pull/666 changes first)
* Ensuring that intermediate certificates and signed certificates retain their chain cert data
* Handling extensions that are the responsibility of the CA
Implementing authority_key_identifier for lemur_cryptography signatures and including skeletons of handling the certificate_info_access and crl_distribution_points
* Fixing errors found with linter
* Updating plugin unit tests
* Changing this for Python3. Underlying cryptography library expects these to be bytes now.
* Updating tests to match new function names/interfaces
* Another naming update in the plugin tests
* Appears that create_csr won't like this input without an owner.
* Undoing last commit and putting it into the right place this time.
* create_csr should be good now with these options, and chain certs will be blank in tests
* This won't be blank in issue_certificate, like it will in creating an authority.
* Much cleaner
* unnecessary import
* Fixing an IAM syncing issue. Were duplicates were not properly sync'd with Lemur. This resulted in a visibility gap. Even 'duplicates' need to sync'd to Lemur such that we can track rotation correctly. Failing on duplicates lead to missing those certificates and the endpoints onto which they were deployed. This commit removes the duplicate handling altogether.
* Fixing tests.
This way IDEs can verify method overrides in subclasses, otherwise these
are flagged as erroneous.
Changed base classes to properly raise NotImplementedError; previously
they would cause "TypeError: exceptions must derive from BaseException"
Also fixed exception handling in sources.service.clean().
Mostly typos, grammar errors and inconsistent indentation in code
examples.
Some errors detected using Topy (https://github.com/intgr/topy), all
changes verified by hand.
* Enabling the specification of a default authority, if no default is found then the first available authority is selected
* PEP8
* Skipping tests relying on keytool
* Initial work on certificate rotation.
* Adding ability to get additional certificate info.
* - Adding endpoint rotation.
- Removes the g requirement from all services to enable easier testing.
* Implement CFSSL issuer plugin
Implement a Lemur plugin for generating certificates from the open
source certificate authority CFSSL
(https://github.com/cloudflare/cfssl). The plugin interacts with CFSSL
through the CFSSL REST API. The CFSSL configuration is defined in the
lemur.conf.py property file using property names prefixed with "CFSSL_".
* Update documentation to include CFSSL plugin
When exporting a certificate, the password is an optional parameter.
When a password is not supplied by the caller, a default password is
generated by the method. The generation library creates the random
password as a bytes object. The bytes object raises an error in the
'keytool' command used to export the certificate. The keytool is
expecting the password to be a str object.
The fix is to decode the generated password from a bytes object to a str
object.
The associated Java plugin tests have been updated to verify the export
method returns the password as a str object. In addition, the tests have
been updated to correctly test the export methods response object. The
original tests treated the response as a single object. The current
export methods return a tuple of data (type, password, data).
In order to make the tests compatible with both Python2 and Python3, the
'six' library was used to test the password is in fact a string.
* Activate the AWS S3 destination plugin
Add the AWS S3 destination plugin to the list of available Lemur
plugins.
Update the S3 destination plugin's "accountNumber" option to be of type
'str' to handle account numbers starting with zeros.
Update Lemur's utils for parsing certificates to correctly encode the
X509 certificates before loading for python3.
* Add S3 destination plugin test
Added simple test to verify S3 destination plugin is available.
