CodamAIDocs
Topicdone

What happens in the database when reading

The field selection drives the SQL query: only the requested columns, references via LEFT JOIN, no loading in the background. This page explains why this is fast and when it gets expensive.

Variants
Tuple projectionLEFT JOINcount, determine IDs, fetch fieldsexpanded single reference as a read of its ownnested lists as separate queriesdeep nestingsubtypes of abstract models

What this is about

Many frameworks first load a whole object and then cut away what the client does not want. CDMS does it the other way round: the response is evaluated before the database query, and the query reads only the columns you requested.

Three terms that appear on this page:

TermMeaning
ProjectionThe query reads only selected columns instead of the whole row. In CDMS the technique is called “tuple projection”: the result is a list of values instead of a complete object.
LEFT JOINConnects a table with a second one without losing rows that have no connection. An employee without a company stays in the result, their company field is empty.
Lazy loadingA framework secretly loads a reference as soon as the code accesses it. This does not happen when reading in CDMS.

From the response to the SQL query

On the left is the response, on the right what CDMS makes of it (simplified):

flowchart LR
    subgraph R["response"]
        direction TB
        r1["firstname"]
        r2["lastname"]
        r3["{ field: company, response: [companyname] }"]
    end
    subgraph S["SQL for employee"]
        direction TB
        s1["SELECT e.id, e._createdOn, e._updatedOn, type,<br/>e.firstname, e.lastname,<br/>c.id, type of c"]
        s2["FROM employee e<br/>LEFT JOIN company c ON e.company_id = c.id"]
        s3["WHERE e.id = '7f3…' AND row filters"]
        s1 --> s2 --> s3
    end
    subgraph T["SQL for company"]
        direction TB
        t1["SELECT c.id, c._createdOn, c._updatedOn, type,<br/>c.companyname"]
        t2["FROM company c<br/>WHERE c.id = 'a1…' AND row filters"]
        t1 --> t2
    end
    R ==> S
    S ==>|"company.id"| T

You can see three things here:

  1. Only requested columns. From employee, CDMS reads firstname and lastname, plus always id, _createdOn, _updatedOn and the type. The query does not touch other columns.
  2. The reference comes via LEFT JOIN, but only with id and type. This way CDMS knows which object company points to without reading its columns.
  3. The expanded reference is a read of its own. With the id, CDMS reads the company through its own model. Its permissions, its row filters and its read hooks apply, exactly as with a direct read.

The three steps of a query

Every query, including reading a single object, runs in three steps:

How CDMS reads a table
  1. 1
    CDMS→Database
    count: How many rows match the filter?
    This gives totalCount in meta. If there are zero rows, CDMS is already done here.
  2. 2
    CDMS→Database
    determine IDs: Which rows go on this page? With filter, sorting, limit and page, but only with the IDs
  3. 3
    CDMS→Database
    fetch fields: For exactly these IDs, read the requested columns, references via LEFT JOIN
  4. 4
    CDMS
    assembles the values into objects, in the order from step 2

The values come back as a flat list with names like company.id. CDMS turns this back into a nested structure: { "company": { "id": … } }.

When it gets expensive

A single query is fast thanks to the projection. What makes it expensive is the number of queries. Every expanded reference and every expanded list is a query of its own, and that per object that contains it.

How many queries are created?

When: POST /hr/employee/query with limit: 25 and response: ["firstname", "lastname"]

One query on employee, without a JOIN.

Result: 1 query (in three steps).

When: … plus { "field": "company", "response": ["companyname"] }

The employee query gets a LEFT JOIN for the id of the company. After that, CDMS reads the company for each of the 25 employees.

Result: up to 1 + 25 queries. Employees without a company need no second read.

When: response: ["*"] on employee with the references company and department

* requests every reference with its id. This too is a read of its own for every reference, so that the permissions and row filters of the target model apply.

Result: up to 1 + 25 × 2 queries.

When: POST /hr/company/query with limit: 25 and { "field": "employees", "response": ["firstname"] }

A search of its own on employee for each of the 25 companies. Without limit in the list, every search returns all employees of its company.

Result: 1 + 25 queries, with 500 employees per company 12,500 objects.

When: … plus { "field": "department", "response": ["name"] } inside employees

For every employee read, a query for their department is added. The number multiplies from level to level.

Result: 1 + 25 + 25 × (employees per company) queries.

Subtypes of abstract models

If a field is not in the parent model but in a subtype, CDMS connects the subtype’s table via the shared id. For an abstract list, CDMS first searches the parent model for IDs and types only, and then reads all matches of each subtype with one query (id IN (…)). See Reading through abstract types.

Decision table

What does an entry in the response cost?
EntryEffect on the database
simple field, e.g. "firstname"one more column in the same query
"+"all simple columns in the same query
single reference via * or { field }one LEFT JOIN, plus one read per object
list via * or { field }one search per object, without limit with all entries
object inside an objectmultiplies with the level above

Rules of thumb

Overview and detail
Overview
POST /hr/employee/query
  • { "response": ["firstname", "lastname"], "parameter": { "limit": 25 } }
  • one query, no matter how many references the model has
Detail
POST /hr/employee/read/{id}
  • { "response": ["+", { "field": "company", "response": ["companyname"] }] }
  • additional queries only for the one opened object

Where to go next

Sources in the code and the knowledge base
  • CDMS/cdms-persistence-database – AbstractDatabasePersistence.queryObjects, SelectionBuilder, TupleMapper
  • CDMS/cdms-system-layer – AbstractLayer.recursiveRead, recursiveQuery, fetchAndSetModel, fetchAndSetList
  • documentation/20-api/03-response-requests.md
Search