Reconciliation with Keycloak

What this is about

Every group lives in two places: in the CIAS database and as a copy in Keycloak. A change writes both one after the other, and something can go wrong in between. Keycloak is briefly away, or the process stops exactly between the two steps. Then CIAS is correct, but the copy lags behind.

The sync state makes this visible, and the reconciliation repairs it.

The two states

stateDiagram-v2
    [*] --> PENDING: group created
    PENDING --> SYNCHRONIZED: Keycloak accepted the change
    SYNCHRONIZED --> PENDING: group changed (before the Keycloak step)
    PENDING --> PENDING: Keycloak unreachable
    SYNCHRONIZED --> PENDING: adding a member failed
    PENDING --> SYNCHRONIZED: reconciliation

State	Meaning	What you do
`SYNCHRONIZED`	CIAS and Keycloak matched at the last write	nothing
`PENDING`	Keycloak does not yet hold everything CIAS manages. So people may be missing roles	start a reconciliation

CIAS sets PENDING before it touches Keycloak, not only when Keycloak fails. If the process died between the two steps, a state that is only set on failure would stay clean even though the copy is missing. That is exactly the case the state exists for.

When a group stays PENDING:

Create or change, and Keycloak does not accept the copy.
Add members, and Keycloak does not add all of them.
A default group gets a new member after a registration, and Keycloak cannot be reached, see The default group.

Taking never leaves PENDING: if Keycloak fails while removing or deleting, the whole call fails and CIAS stays as it was. See Manage groups and members.

What a reconciliation does

A reconciliation, group by group

1

CIAS
reads all groups from the CIAS database
2

CIAS→Keycloak
per group: create the copy if it is missing. Set description, default group, realm roles and client roles exactly to the CIAS state, including emptying clients the group no longer carries
3

CIAS→Keycloak
per group: align the members. First remove whoever is extra in Keycloak, then add whoever is missing
4

CIAS
per group: set SYNCHRONIZED. If a group fails, it goes into the report with a reason, the others carry on
5

CIAS→Keycloak
reads all groups with the cias-managed mark and deletes those CIAS no longer knows

Result: Report: created, reconciled, removed, failed

The order is deliberate. If CIAS deleted first, it could delete a group that the same run is about to create again. In the end everything would be right, but on the way all members would briefly have lost their permissions.

A reconciliation looks at every group, not only the PENDING ones. Even a group that looks clean can differ after an interruption or a manual change in Keycloak. If a group is already right, it only costs reads.

When reconciliation runs

Two occasions

at startupon request

When: The application has started, and codamai.cias.authorization.startup.enabled is true.

After startup, a reconciliation runs without a caller. The result is in the log: one line with the numbers, or a warning with the failed groups. If the run fails, the application starts anyway. A group without a copy is bad, but better than a platform that does not come up because Keycloak is restarting. The setting defaults to false; the standalone CIAS does not set it.

Result: report in the log

When: A platform administrator calls POST /cias/admin/groups/reconcile.

The same run. Anyone who is not a platform administrator gets 403 cias.authorization.denied.

Result: 200 with the report, even if groups failed

Why no timer? If a background run sometimes repaired differences and sometimes did not, nobody could tell from outside whether a difference exists right now or has already been fixed. That is why there are exactly two occasions, and a person decides both: starting the application and the call.

The report

Request

POST /cias/admin/groups/reconcile
Authorization: Bearer <token of a platform administrator>

Response

HTTP 200
{
  "created":  ["einkauf"],
  "repaired": ["grundrechte", "support"],
  "removed":  ["alt-vertrieb"],
  "failed":   [
    { "group": "lager", "reason": "…" }
  ]
}

Field	Meaning
`created`	groups whose copy was missing in Keycloak and is there now
`repaired`	groups whose copy already existed. They were checked and, if needed, brought to the CIAS state
`removed`	groups with the `cias-managed` mark that CIAS no longer knows, now deleted in Keycloak
`failed`	what did not work, per group with a reason. Empty means: everything done

The call answers 200 even if failed has entries. A single status code for forty groups would hide which group has the problem. If CIAS could not read the group list from Keycloak at all, failed contains an entry (provider listing), and CIAS deletes nothing in this run.

The decision

What does the reconciliation do with a group?

In CIAS?	In Keycloak with the `cias-managed` mark?	Result
yes	no	create in Keycloak, set roles and members → `created`
yes	yes	bring to the CIAS state → `repaired`
no	yes	delete in Keycloak → `removed`
no	no, group without the mark	leave alone, CIAS does not even see it