Keycloak implementation of SPI

.github/workflows/ci-release-dev.yaml

@@ -15,8 +15,14 @@ jobs:
       - uses: actions/checkout@v4
       - id: set_version
         run: |
+          # In order to publish Helm charts we need valid semantic version, so we get the latest release tag to prefix the version with.
+          git fetch --tags
+
+          # Try to get the latest tag, fallback to 0.0.0 if no tag is found
+          LATEST_RELEASE=$(git describe --tags --abbrev=0 2>/dev/null || echo "v0.0.0")
+
           BRANCH=$(echo $(git rev-parse --abbrev-ref HEAD) | tr -d '0123456789/.')
-          VERSION=dev-$BRANCH-$(git rev-parse --short HEAD)
+          VERSION=$LATEST_RELEASE-dev-$BRANCH-$(git rev-parse --short HEAD)
           echo "version=$VERSION" >> $GITHUB_OUTPUT
           echo "Set version to $VERSION"
   docker-release:

.gitignore

@@ -17,3 +17,6 @@ _helm_sync_dir/
 
 helm/Chart.yaml
 helm/values.yaml
+
+# Built binaries
+cmd/idp-connect

DEVELOPMENT.md

@@ -1,17 +1,112 @@
-## Development
+# Development
 
 This is a development guide for users wanting to help contribute to the project.
 
 ## Environment
 
 Before you begin, make sure you have all of the required tools installed:
 
-```
+```sh
 ./env/validate-env.sh
 ```
 
-## TODO:
+Add any new connector implementations to `cmd/idp-connect.go` so that they can become valid server options to start.
+
+## Keycloak
+
+You can test the manipulation of self-service clients using a dedicated realm in a Keycloak instance. Create a new realm using curl and the admin credentials using the examples below.
+
+First, create a token to access the Keycloak REST API. This is a short-lived token, so you may need to repeat this step later on:
+
+```sh
+KEYCLOAK_URL=http://$(kubectl --context mgmt -n keycloak get service keycloak -o jsonpath='{.status.loadBalancer.ingress[0].*}'):8080
+
+KEYCLOAK_TOKEN=$(curl -Ssm 10 --fail-with-body \
+  -d "client_id=admin-cli" \
+  -d "username=admin" \
+  -d "password=admin" \
+  -d "grant_type=password" \
+  "$KEYCLOAK_URL/realms/master/protocol/openid-connect/token" |
+  jq -r .access_token)
+```
+
+Create the new realm:
+
+```sh
+REALM=my-realm
+
+curl -Ssm 10 --fail-with-body -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -H "Content-Type: application/json" \
+  -d '{ "realm": "'${REALM}'", "enabled": true }' \
+  $KEYCLOAK_URL/admin/realms
+```
+
+You'll need to provision a client in this realm that permits service accounts and has permissions to manipulate self-service clients. For convenience, we'll also treat this client as a _resource server_ in which we will store API products as _resources_. You can create such a client like this:
+
+```sh
+KEYCLOAK_CLIENT=gloo-portal
+
+# Create initial token to register the client
+INITIAL_TOKEN=$(curl -Ssm 10 --fail-with-body -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -H "Content-Type: application/json" \
+  -d '{ "expiration": 0, "count": 1 }' \
+  $KEYCLOAK_URL/admin/realms/${REALM}/clients-initial-access |
+  jq -r .token)
+
+# Register the client
+read -r KEYCLOAK_CLIENT_INTERNAL_ID KEYCLOAK_SECRET <<<$(curl -Ssm 10 --fail-with-body -H "Authorization: bearer ${INITIAL_TOKEN}" -H "Content-Type: application/json" \
+  -d '{ "clientId": "'${KEYCLOAK_CLIENT}'", "name": "Solo.io Gloo Portal Resource Server" }' \
+  ${KEYCLOAK_URL}/realms/${REALM}/clients-registrations/default |
+  jq -r '[.id, .secret] | @tsv')
+
+echo "Management client ID: ${KEYCLOAK_CLIENT}"
+echo "Management client secret: ${KEYCLOAK_SECRET}"
+
+# Set up the client as we need
+curl -Ssm 10 --fail-with-body -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -H "Content-Type: application/json" \
+  -X PUT -d '{ "serviceAccountsEnabled": true, "authorizationServicesEnabled": true }' \
+  ${KEYCLOAK_URL}/admin/realms/${REALM}/clients/${KEYCLOAK_CLIENT_INTERNAL_ID}
+
+# Get the internal ID of the client's service account user
+SA_USER_ID=$(curl -Ssm 10 --fail-with-body -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -H "Content-Type: application/json" \
+  ${KEYCLOAK_URL}/admin/realms/${REALM}/clients/${KEYCLOAK_CLIENT_INTERNAL_ID}/service-account-user |
+  jq -r .id)
+
+# Get the ID of the 'realm-management' client
+REALM_MGMT_CLIENT_ID=$(curl -Ssm 10 --fail-with-body -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" \
+  "${KEYCLOAK_URL}/admin/realms/${REALM}/clients?clientId=realm-management" |
+  jq -r '.[].id')
+
+# Get the ID of the 'manage-clients' role
+ROLE_ID=$(curl -Ssm 10 --fail-with-body -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -H "Content-Type: application/json" \
+  ${KEYCLOAK_URL}/admin/realms/${REALM}/users/${SA_USER_ID}/role-mappings/clients/${REALM_MGMT_CLIENT_ID}/available | jq -r '.[] | select(.name=="manage-clients") | .id')
+
+# Add the 'manage-clients' role to the service account user
+curl -Ssm 10 --fail-with-body -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -H "Content-Type: application/json" \
+  -d '[ { "id": "'${ROLE_ID}'", "name": "manage-clients", "composite": false, "clientRole": true, "containerId": "'${REALM_MGMT_CLIENT_ID}'" } ]' \
+  ${KEYCLOAK_URL}/admin/realms/${REALM}/users/${SA_USER_ID}/role-mappings/clients/${REALM_MGMT_CLIENT_ID}
+```
+
+The values of `KEYCLOAK_CLIENT` and `KEYCLOAK_SECRET` should be supplied to the Keycloak flavour of `idp-connect` at runtime (via `--client-id` and `--client-secret`) so that the service can obtain tokens and manipulate self-service clients on behalf of this management client. In the example used so far, you can start the service like this:
+
+```sh
+./idp-connect keycloak --issuer ${KEYCLOAK_URL}/realms/${REALM} --client-id ${KEYCLOAK_CLIENT} --client-secret ${KEYCLOAK_SECRET}
+ ```
+
+IDP Connect will use the token endpoint to obtain a token for the management client. You can replicate this for testing purposes like this:
+
+```sh
+MGMT_TOKEN=$(curl -Ssm 10 --fail-with-body \
+  -u ${KEYCLOAK_CLIENT}:${KEYCLOAK_SECRET} \
+  -d "grant_type=urn:ietf:params:oauth:grant-type:uma-ticket" \
+  -d "audience=${KEYCLOAK_CLIENT}" \
+  ${KEYCLOAK_URL}/realms/${REALM}/protocol/openid-connect/token |
+  jq -r .access_token)
+
+# Test the token by listing the clients in the realm
+curl -Ssm 10 --fail-with-body -H "Authorization: Bearer ${MGMT_TOKEN}" ${KEYCLOAK_URL}/admin/realms/${REALM}/clients | jq .
+```
+
+## TODO
 
 * Create middleware to handle login requests and responses and exposing those metrics via Prometheus metrics
 * Cognito
-  * Develop auth mechanism when Cognito is running in EKS, taking advantage of AWS IAM Role for Service Accounts
+  * Develop auth mechanism when Cognito is running in EKS, taking advantage of AWS IAM Role for Service Accounts

README.md

@@ -8,12 +8,30 @@ it is the responsibility of the SPI to create the credential associated with tha
 Here is a list of Identity Providers that we currently support:
 
 * Amazon Cognito
+* Keycloak
+
+## Configuration Instructions
+
+### Keycloak
+
+A Keycloak client must be created for the Keycloak IDP Connect service to use. Provide the ID and secret of this client in the `--client-id` and `--client-secret` IDP Connect arguments respectively. This client must meet some requirements:
+
+* The client must have the `manage-client` permission needed for IDP Connect to be able to manipulate self-service clients.
+* **Authorization** must be enabled on this client, as this client will also act as an OAuth2 [resource server](https://www.keycloak.org/docs/latest/authorization_services/index.html#_resource_server_overview).
+* **Service accounts roles** (or OAuth2 _client credentials_) must be enabled, to allow IDP Connect to use this client directly to manage other clients and resources.
+
+#### Related documentation
+
+* Keycloak's support for client registration: <https://www.keycloak.org/docs/latest/securing_apps/#_client_registration>
+* Resource authorization in Keycloak: <https://www.keycloak.org/docs/latest/authorization_services/>
+* IDP Connect will manipulate resources using Keycloak's Authorization Services, which is based on [User-Managed Access (UMA)](https://docs.kantarainitiative.org/uma/rec-uma-core.html)
 
 ## Production
 
 IDP Connect provides a straightforward and easy-to-setup way of configuring credentials for the applications in your system; however,
  we expect that the needs of your system are and will evolve beyond the scope of this simple implementation. The SPI we provide provides a hook on top of which you can build a customizable system to service any number of more advanced use cases.
 
 TODO: Add information for devs
+
 * Install tools
-* (Potential) Allow for AWS IAM Roles for service accounts as cognito auth method.
+* (Potential) Allow for AWS IAM Roles for service accounts as cognito auth method.

api/v1/openapi.yaml

@@ -11,34 +11,24 @@ paths:
       description: Creates an application of type oauth2. This is intended to be integrated with an Open Id Connect Provider that the IDP Connect implementation integrates with. Note that the `clientSecret` is never stored in the database and is shown to the user only once. Keep this secret to make future requests to the API products in the Portal.
       operationId: CreateOAuthApplication
       requestBody:
-        description: (Required) name for creating name of the client.
+        description: (Required) Unique identifier for creating client.
         required: true
         content:
           application/json:
             schema:
               type: object
               required:
-                - name
+                - id
               properties:
-                name:
+                id:
                   type: string
-                  example: "example-user-pool-developer-1"
+                  example: "a0897e6d0ea94f589c38278bca4e9342"
       responses:
         '201':
           content:
             application/json:
               schema:
-                type: object
-                properties:
-                  clientId:
-                    type: string
-                    example: a0897e6d0ea94f589c38278bca4e9342
-                  clientSecret:
-                    type: string
-                    example: c94dbd582d594e8aa04934f9c7ef0f52
-                  clientName:
-                    type: string
-                    example: "example-user-pool-developer-1"
+                $ref: '#/components/schemas/OAuthApplication'
           description: Successfully created client.
         '400':
           description: Invalid input.
@@ -74,7 +64,7 @@ paths:
           content:
             application/json:
               schema:
-                  $ref: '#/components/schemas/Error'
+                $ref: '#/components/schemas/Error'
         '500':
           description: Unexpected error deleting application.
           content:
@@ -108,8 +98,8 @@ paths:
                 apiProducts:
                   type: array
                   items:
-                      type: string
-                      example: "example-api-product"
+                    type: string
+                    example: "example-api-product"
       responses:
         '204':
           description: Successfully added API Product to application.
@@ -174,6 +164,27 @@ paths:
       summary: Creates API Product in the OpenID Connect Provider. Then, you can add this API Product to the application for your Portal applications with the `PUT /applications/{id}/api-products` API request.
       tags:
         - API Products
+    get:
+      description: Get all API Products in the Open Id Connect Provider. The Portal uses the results to keep the API Products in the IdP in sync with Portal by creating and deleting as needed.
+      operationId: GetAPIProducts
+      responses:
+        '200':
+          description: Successfully retrieved API Products.
+          content:
+            application/json:
+              schema:
+                type: array
+                items:
+                  $ref: '#/components/schemas/ApiProduct'
+        '500':
+          description: Unexpected error retrieving API Products.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Error'
+      summary: Get all API Products in the OpenID Connect Provider.
+      tags:
+        - API Products
   /api-products/{name}:
     delete:
       description: Deletes API Product in the Open Id Connect Provider for a given unique identifier.
@@ -215,6 +226,20 @@ components:
         description:
           type: string
           example: "example API Product description"
+    OAuthApplication:
+      required:
+        - clientId
+        - clientSecret
+      properties:
+        clientId:
+          type: string
+          example: a0897e6d0ea94f589c38278bca4e9342
+        clientSecret:
+          type: string
+          example: c94dbd582d594e8aa04934f9c7ef0f52
+        clientName:
+          type: string
+          example: "example-user-pool-developer-1"
     Error:
       required:
         - code

cmd/idp-connect.go

@@ -7,6 +7,7 @@ import (
 	"github.com/spf13/cobra"
 
 	"github.com/solo-io/gloo-portal-idp-connect/internal/cognito"
+	"github.com/solo-io/gloo-portal-idp-connect/internal/keycloak"
 	"github.com/solo-io/gloo-portal-idp-connect/internal/version"
 )
 
@@ -28,6 +29,7 @@ func rootCommand(ctx context.Context) *cobra.Command {
 
 	cmd.AddCommand(
 		cognito.Command(),
+		keycloak.Command(),
 	)
 
 	return cmd