CodamAIDocs
Topicdone

When CIAS is not reachable

What CDMS does in standalone mode when CIAS goes down: keep serving known tenants, reject unknown ones, tenant-bound attributes cannot be read.

Variants
tenant in cachetenant unknowntenant-bound attributesrequest without a tenantrestart during the outage

What this is about

Only in standalone mode can CIAS fail on its own. Embedded, CIAS stands and falls with the application: if CIAS is gone, CDMS is gone too, and there is nobody left to ask.

Standalone the situation is different. The CDMS service keeps running but can no longer reach CIAS — and still has to decide on every request.

This page covers the request path. The full picture of outages, including Keycloak and the administration, is under When CIAS or Keycloak fails.

Two questions, one rule

On the request path the CDMS service asks CIAS two things. They are built independently of each other and follow the same rule:

QuestionWho asksRemembersWhen nothing is remembered
Is this tenant served?the tenant gateevery answer, including every nothe request is refused
What does this person hold in this tenant?the attribute lookupevery answer, including an empty onethe request is refused

The second question is only asked when a module registered an attribute per tenant at all. If none does, there is nothing here that could fail: the values in the token are then the whole truth. See One value per person or per tenant.

What counts as an outage

More than you would think at first. Every failure to ask is an outage, because none of them is safer than the others:

"CIAS did not answer"
  1. 1
    CDMS
    no connection after 2 seconds, or no answer after 2 seconds
    The timeouts are short and not a tuning knob. Better to fail than to wait, because "failed" has a defined answer.
  2. 2
    CDMS
    a refused connection, any other error code from CIAS, an answer that cannot be read
  3. 3
    CDMS
    an answer without the field served
    A missing field must never read as "not served". Otherwise a change to the response format would refuse every customer at once, while every log said they had been suspended.
  4. 4
    CDMS
    for the attribute lookup, a 404 as well
    Result: A person without a record is answered with 200 and an empty list. So a 404 means: this endpoint does not exist.

What is not an outage is a 404 from the tenant lookup. That is a real answer and means “no tenant carries this key”. The gate refuses and remembers it.

A 401 or a 403 on the lookup is not an outage either. It is a statement about this service’s own credentials — and it refuses the request immediately, without consulting the memory. The difference to every other failure above is that those pass once CIAS answers again. A rejected service token does not; it only gets better with a new one.

Embedded the rule is the same, only the causes differ: the method call can fail, for example because the system database is unreachable. The same branch applies then.

What the gate answers then

The tenant gate while CIAS does not answer
Remembered answer for this tenantOutcome for the request
present, at most 15 minutes old, and it was "served"the request runs normally
present, at most 15 minutes old, and it was "not served"403 cias.authentication.tenant-not-served – a remembered refusal stays a refusal
present, but older than 15 minutes403 cias.authentication.tenant-not-served
none403 cias.authentication.tenant-not-served

As long as CIAS does not answer, the remembered answer stands — but for at most 15 minutes (codamai.cias.tenant-gate.stale-ceiling). After that the request is refused as if nothing had ever been remembered. The remembering time of 30 seconds only decides when the question is asked again — and it can only be asked again once CIAS answers.

So the two durations do different jobs: the remembering time applies while CIAS answers, the staleness limit while it does not. Without the limit a service that never reaches CIAS again would answer from memory until it is restarted — and would never learn that a tenant has been suspended in the meantime.

The same applies word for word to the attribute values: remembered values still apply, and without remembered values the request is refused. An empty answer is a full remembered answer here, because “this person holds nothing here” is a statement and not a missing one.

The variants

Who keeps working and who does not

When: In the seconds before the outage, somebody from this tenant was already working.

The gate knows the answer and hands it out. For the users of this tenant nothing changes – provided their attribute values are remembered too, if there are any. Everything CDMS can do without CIAS keeps working: reading, writing, files, history.

Result: The request reaches the application.

When: Nothing is in the memory for this tenant – nobody has worked for it since this node last started.

The gate has nothing to go on and refuses. This is where "reject when in doubt" takes effect: letting the request through would mean inventing an answer to "may this customer work?".

Result: 403 cias.authentication.tenant-not-served – the same answer as for a suspended or unknown customer.

When: The tenant is admitted, but the person's values in this tenant cannot be read and are not remembered.

The filter chain refuses instead of carrying on with the values from the token or with none at all. Both would be dangerous: the value in the token belongs to whichever tenant wrote it last, and an empty list is a single * away from switching an attribute filter off entirely.

Result: 403 cias.authentication.tenant-not-served, with the same key as a tenant refusal.

When: The application runs in SINGLE, or the request belongs to no tenant.

Then the gate has nothing to ask and the attribute lookup has nothing to look up. An outage of CIAS is invisible to such requests.

Result: The request runs normally.

When: The CDMS node restarts while CIAS still does not answer.

The memory only lives in RAM and is empty after a start. The node then knows not a single tenant.

Result: Every request with a tenant is refused until CIAS answers again.

What keeps running and what does not

While CIAS does not answer
Logging in and refreshing tokensworks, because that is Keycloak’s job, not CIAS’s
Requests in a remembered tenantkeep running, as long as the remembered answer is less than 15 minutes old
Requests in a tenant without a remembered answerare refused
Requests once the outage has lasted 15 minutesare refused
Requests without a tenantkeep running
A customer who was just createdcannot work until CIAS answers
The administration UI of CIASis unreachable, because it lives in the CIAS service
Writing, files, history in CDMSkeep running, they do not need CIAS

Watch out

Next

Sources in the code and the knowledge base
  • CIAS/cias-authentication – TenantGate.admit (catch branch, remembered answer, LOOKUP_UNAVAILABLE), TenantGateProperties, AttributeLookup.valuesFor (catch branch, rethrow), AttributeLookupProperties, TokenParser.admit (refusal when the attributes cannot be read), TenantAdmission, NoTenantBoundAttributes, TenantBoundAttributeWiringCheck
  • CIAS/cias-tenancy-client – RemoteTenantLookupAdapter (404 is an answer, 401/403 and everything else is an outage, missing `served`), RemoteTenantBoundAttributeAdapter (404 is an outage, missing `attributes`, values must be lists of text), CiasTenancyClientProperties (2 s/2 s), ClientCredentialsTenantLookupCredentials (IAM unreachable is an outage, IAM rejecting the client is a refusal), StaticTenantLookupCredentials (does not refresh), TenantLookupUnavailableException, AttributeLookupUnavailableException
  • CIAS/cias-tenancy – LocalTenantLookupAdapter, TenantLookupController
  • CIAS/cias-kernel/docs/adr – ADR-022 §4 and §5; CIAS/cias-authentication/docs/adr – ADR-021, ADR-042 §4
Search