Skip to main content

Settings and environment variables

The prod profile and its environment variables​

An application runs in production with the prod profile (SPRING_PROFILES_ACTIVE=prod, which the image sets). Its src/main/resources/application-prod.properties (the sample's and the demo's; the scaffold writes one) reads every setting from an environment variable and no value is written in the file: the database password is required, and every other secret's placeholder defaults to blank, which is unset (${JCM_S3_SECRET_KEY:}). Startup stops before anything connects when a required variable is not set, or is blank, naming each one and never a value:

JCM_DB_URL is not set; the prod profile needs it
VariableSettingNotes
JCM_DB_URL, JCM_DB_USERNAME, JCM_DB_PASSWORDmain.datasource.url, .username, .passwordRequired: the database (below)
JCM_PUBLIC_URLjcm.public-urlThe address users reach (https://appeals.example.org): where an identity provider sends the browser back after sign-in and logout. Unset, the request's own address stands in, which is right behind a proxy that forwards it (below)
JCM_SERVER_NAMEjcm.server-nameThe name in each response's X-Jcm-Server header (default: the host name, below)
JCM_SERVERSjcm.serversHow many servers run the application (default 1); above 1 the studio refuses editing. Set it to the real count: every server also reports on the console (as.studio.applied) when it has not applied the newest studio publish
JCM_STORAGEjcm.storage.kindlocal (default), s3 or azure (File storage, below)
JCM_FILES_DIRjcm.storage.local.dirThe local store's folder (default /data/files, the image's volume)
JCM_S3_BUCKET, JCM_S3_REGION, JCM_S3_ENDPOINT, JCM_S3_PATH_STYLEjcm.storage.s3.bucket, .region, .endpoint, .path-styleWith JCM_STORAGE=s3; path style defaults to false
JCM_S3_ACCESS_KEY, JCM_S3_SECRET_KEYjcm.storage.s3.access-key, .secret-keyTogether, or neither (the AWS default credentials chain)
JCM_AZURE_CONNECTION_STRING, JCM_AZURE_CONTAINERjcm.storage.azure.connection-string, .containerWith JCM_STORAGE=azure
JCM_OIDC_ISSUER, JCM_OIDC_CLIENT_ID, JCM_OIDC_CLIENT_SECRETjcm.identity.oidc.issuer, .client-id, .client-secretWith jcm-identity-oidc-starter; sign-in stays on the login form while the issuer is unset. With the issuer set, startup refuses a blank secret (JCM_OIDC_CLIENT_SECRET is not set; the prod profile needs it while jcm.identity.oidc.issuer is set ...)
JCM_OIDC_PUBLIC_CLIENTjcm.identity.oidc.public-clienttrue for a public client, which signs in with PKCE alone and has no secret (default false)
JCM_LDAP_URL, JCM_LDAP_BASE_DN, JCM_LDAP_MANAGER_DN, JCM_LDAP_MANAGER_PASSWORDjcm.identity.ldap.url, .base-dn, .manager-dn, .manager-passwordWith jcm-identity-ldap-starter; off while the URL is unset
JCM_IDENTITY_LOCAL_ACCOUNTSjcm.identity.local-accountstrue opens /login/local, the break-glass sign-in for database accounts (default false; Identity-provider outage, below)
JCM_ADMIN_PASSWORDjcm.identity.local-admin.passwordThe break-glass admin account's password; required at first start (startup refuses without it, or with admin), kept once set
JCM_SEED_PASSWORDjcm.seed.passwordThe demo's seeded users' password, while jcm.seed.enabled=true makes users: the demo needs it at its first start under prod, and stops without it (jcm.seed.password must be set to seed users); later starts keep the users' passwords
JCM_MAIL_HOST, JCM_MAIL_PORT, JCM_MAIL_FROMspring.mail.host, .port, jmix.email.from-addressWithout a host nothing is sent and startup warns; port defaults to 25, the sender to DoNotReply@localhost
JCM_MAIL_USERNAME, JCM_MAIL_PASSWORDspring.mail.username, .passwordThe mail server's account; blank, no authentication. StartTLS: SPRING_MAIL_PROPERTIES_MAIL_SMTP_STARTTLS_ENABLE=true (and ..._AUTH=true where the server needs it said)

With jcm-mailbox-starter and a mailbox file, each connection's settings come from the environment too, one set per connection name (here intake; see the contract's Mailbox intake). No password may be written in a packaged properties file under prod, only a ${...} placeholder, and startup names any of host, user or password that is not set:

VariableSettingNotes
JCM_MAILBOX_INTAKE_HOST, JCM_MAILBOX_INTAKE_PORTjcm.mailbox.connections.intake.host, .portPort defaults to 993
JCM_MAILBOX_INTAKE_SECURITYjcm.mailbox.connections.intake.securitytls (default) or starttls; none is refused under prod
JCM_MAILBOX_INTAKE_USERNAME, JCM_MAILBOX_INTAKE_PASSWORD.username, .passwordThe mailbox's own account; never in the image or the repository
JCM_MAILBOX_INTAKE_SOURCE, JCM_MAILBOX_INTAKE_DONE, JCM_MAILBOX_INTAKE_HELD.folders.source, .done, .heldDefault INBOX, JCM-Ingested, JCM-Not-ingested; all three must exist
JCM_MAILBOX_ALLOW_UNSCANNEDjcm.mailbox.allow-unscannedtrue files mail from outside with no document scanner installed (a WARN at every start); without it, and without a scanner, startup refuses
jcm.mailbox.connections.intake.host=${JCM_MAILBOX_INTAKE_HOST}
jcm.mailbox.connections.intake.port=${JCM_MAILBOX_INTAKE_PORT:993}
jcm.mailbox.connections.intake.security=${JCM_MAILBOX_INTAKE_SECURITY:tls}
jcm.mailbox.connections.intake.username=${JCM_MAILBOX_INTAKE_USERNAME}
jcm.mailbox.connections.intake.password=${JCM_MAILBOX_INTAKE_PASSWORD}
jcm.mailbox.connections.intake.folders.source=${JCM_MAILBOX_INTAKE_SOURCE:INBOX}
jcm.mailbox.connections.intake.folders.done=${JCM_MAILBOX_INTAKE_DONE:JCM-Ingested}
jcm.mailbox.connections.intake.folders.held=${JCM_MAILBOX_INTAKE_HELD:JCM-Not-ingested}

The poll's run holds one idle database connection for its length: size the pool for it, and do not end idle transactions on PostgreSQL (idle_in_transaction_session_timeout) shorter than a poll.

A blank optional variable is the same as an unset one. The OIDC and LDAP lines are in the profile only of an application with that starter (the demo has no LDAP lines). Any other setting is set by its environment form, which Spring Boot reads without a line in the file: dots and hyphens become underscores, in capitals (JCM_IDENTITY_OIDC_USERNAME_CLAIM, JCM_IDENTITY_LOCKOUT_ATTEMPTS, SPRING_MAIL_PROPERTIES_MAIL_SMTP_STARTTLS_ENABLE). Such a form also wins over the profile's own line (MAIN_DATASOURCE_URL stands in for JCM_DB_URL); given blank, it counts as not set.

A value is used as written: a generated secret that holds $, { or } reads intact, and nothing in the log or the startup report ever shows a value. Should a JCM_* value still not be readable as a setting, startup stops with JCM_DB_PASSWORD could not be read, naming the variable only.

The profile also sets, as values, what production needs on: jcm.identity.require-admin-password=true and jcm.storage.probe-blocks-readiness=true (the two checks that keep readiness down, Health below), and server.forward-headers-strategy=framework (behind a proxy, below). It sets nothing under jcm.studio, so studio editing stays off, no ui.login.default*, so nothing fills in the login form, and nothing under management.

The dev profile (application-dev.properties, for a developer's own machine) keeps what is for a local run only: the login form filled in with admin and the warning to change that password, studio editing on, and jcm.servers=1.

Hosted on the JCM control plane​

An application with jcm-tenant-starter and a tenant id reports usage and health and is held to its signed plan. With no tenant id it contacts nobody, and a self-hosted deployment sets none of this. The variables are read without the prod profile naming them:

VariableNotes
JCM_TENANT_IDThe tenant; unset, the application is not hosted
JCM_TENANT_CONTROL_PLANE_URLhttps, unless localhost
JCM_TENANT_CREDENTIALThe tenant's credential; a secret, only here or in the file below
JCM_TENANT_CREDENTIAL_FILEA mounted secret: an absolute path outside the project
JCM_TENANT_ENTITLEMENT_FILEOptional: a mounted plan document, used instead of asking the control plane. Only this variable may name it; it must not be older than the plan already held, nor dated more than five minutes ahead of the server's clock
JCM_TENANT_ENVIRONMENTOptional: the name reported, such as prod
JCM_TENANT_APPLICATION_VERSIONOptional: the version reported

The application reaches out to the control plane on https only, once per report and at startup; no redirect is followed. The control plane being down never stops an application that holds a valid plan: the last one is kept in the database (JCM_TENANT_STATE). A first start with no plan anywhere is refused. Expiry never stops a running application, and a restart is allowed for 30 days after it; the console warns through that time. Anyone with jcm-tenant-viewer can see exactly what was last reported, under Plan and usage. jcm.jobs.scheduling-enabled=false stops reports as it stops every job.

This is proven against the contract (cli/tenant-plane-api.yaml) and the fake control plane only; the hosted side and the plan signing key are not built. Until then a hosted start is refused with This build of JCM holds no plan signing key.

Enforcement runs inside the application and trusts the server's clock, so it does not stop a tenant who controls the build: the hosted build must add jcm-tenant-starter, set these variables and run the prod profile. In the report, active users are enabled accounts rather than recent sign-ins, and bytes leaves out letter files, which have no recorded size. A refused start leaves the database untouched when a mounted or fetched document decided it; a refusal that rests on the cached plan, or a first start with no document, comes after the changelogs. Hosted plan and usage has every setting and refusal.

Replacing a plan dated ahead​

A plan dated more than five minutes ahead of the server's clock is refused, so one is only held if the clock was wrong when it came or has been wound back. as.tenant.plan, the console's plan page and the log then say so, because no plan issued before that date can replace it. A running application is not stopped, but once that plan has lapsed a restart is refused, and the refusal says the date is the cause. The database needs no change:

  1. Check the server's clock first. If it was wrong, correct it; if the plan's date is then past, the next plan is taken.
  2. Otherwise read the held plan's issue date, in the warning or on the console's plan page, and ask the control plane's operator for a recovery document: an ordinary plan issued now, with replaces_issued_at set to exactly that date.
  3. The control plane serves it and the next report cycle or restart takes it. Or mount it, name it in JCM_TENANT_ENTITLEMENT_FILE and restart; then remove the variable, or the file stays in charge.

The document must not have expired. Under the prod profile only a signed document does this, so the clock or an ordinary setting does not bring an old plan back; the test profile's key can sign anything and is no defence, and neither is write access to JCM_TENANT_STATE. The replaced plan is refused from then on, even once its date has passed.

A start that says the cached plan document cannot be used means it does not verify: another tenant's, or signed with a retired key. Check JCM_TENANT_ID; otherwise a plan issued at or after the date the refusal gives replaces it.

Behind a proxy or load balancer​

server.forward-headers-strategy=framework makes the application read X-Forwarded-Proto, -Host, -Port and -Prefix from the proxy, so the request's own address is the one users reached and redirects go back to it. Set JCM_PUBLIC_URL as well wherever the proxy does not send those headers. The proxy must overwrite, not pass on, any X-Forwarded-* header a client sends.

The sign-in lockout (below) does not read the client's address at all, so a proxy that appends to X-Forwarded-For (nginx's $proxy_add_x_forwarded_for, an AWS load balancer's default) does not weaken it: a client that sends a new X-Forwarded-For on every try is locked out as any other.

Sign-in lockout​

Every sign-in form has a lockout, always: the application's own form (database accounts, and LDAP, which signs in there) and the break-glass route /login/local. After jcm.identity.lockout.attempts wrong passwords (default 5) for one username within jcm.identity.lockout.window (default 15m), that username's sign-in is refused, the right password too, for jcm.identity.lockout.duration (default 15m) after the last of them; then it signs in again, and a successful sign-in clears the count. The form shows its usual wrong-username-or-password error meanwhile, and the log says Sign-in for '<username>' is locked for PT15M after 5 wrong passwords. The count is kept per username (as typed, in lower case) and nothing else: never per address or header, so neither a proxy that passes the client's X-Forwarded-For on nor a guesser that changes it helps. The failures are rows in the database (JCM_SIGN_IN_FAILURE), so every server of a cluster counts the same ones. The cost: someone who knows a username can keep that person locked out by typing wrong passwords for it; the person's identity provider (under OIDC) is unaffected. To unlock a username before the lock ends, delete its rows: DELETE FROM JCM_SIGN_IN_FAILURE WHERE USERNAME_KEY = 'worker-a'. A setting that is not positive is refused at startup (jcm.identity.lockout.attempts must be 1 or more; the sign-in lockout cannot be turned off). Sign-in through OIDC is the provider's, and so is its lockout.

Identity-provider outage: break-glass​

While the identity provider (OIDC) or the directory (LDAP) is down, nobody signs in through it, and an OIDC application cannot even start, since every start asks the issuer for its configuration. The way in is the break-glass admin account (or another database account with a password), and it takes a restart:

  1. Unset JCM_OIDC_ISSUER (or JCM_LDAP_URL) and set JCM_IDENTITY_LOCAL_ACCOUNTS=true in the environment the application starts in. Keep JCM_ADMIN_PASSWORD as it is (or set it, if admin's password is not known).
  2. Restart every server. The prod profile starts with no provider and local accounts on (the sample's ProdProfileTest proves that start); readiness comes up as usual.
  3. Sign in as admin at /login/local (or on the login form, which is the database's while no provider is set). The sign-in lockout guards both. The console warns while local accounts are on (as.identity.local-accounts).
  4. When the provider is back, set JCM_OIDC_ISSUER (or JCM_LDAP_URL) again, unset JCM_IDENTITY_LOCAL_ACCOUNTS, and restart. People signed in through the provider before keep their accounts and roles: their rows are linked to the provider and nothing in the outage changed that.

While no provider is set, every database account with a password can sign in on the application's form, not just admin; accounts a provider created have no password and cannot. Keep the outage window short.

Every response carries X-Jcm-Server: <name>, the name of the server that answered and nothing else (no product or version), so an operator, and the cluster test, can see which server a request reached. The name is JCM_SERVER_NAME when set, else the host name: in a container that is the container's, but on a virtual machine the host name is often an internal name that carries the private address (ip-10-0-1-23.eu-west-2.compute.internal), so set JCM_SERVER_NAME (app-1, app-2) where responses reach the internet and that name should not.