Skip to content

Changelog

Threadmill 1.1.1 is a patch release for the Oracle store. See Upgrade from 1.1.0.

  • Fixed the Oracle baseline migration failing with ORA-12899 at statement 28 of V1__baseline.sql on databases with MAX_STRING_SIZE=EXTENDED. The threadmill_concurrency_groups.idle_sort virtual column declared RAW(2000), but Oracle sizes its NLSSORT expression at RAW(2058) under EXTENDED; the column now takes its derived type. A schema stopped at that statement resumes with the corrected statement, and a completed 1.1.0 schema stays valid.

Threadmill 1.1.0 adds an Oracle Database backend. It is a compatible minor release: core, the PostgreSQL, Redis and in-memory stores and their storage formats are unchanged, so upgrading from 1.0.x is a version bump. See Upgrade from 1.0.0.

  • Added threadmill-store-oracle, an OracleJobStore for Oracle Database 19c and later (issue #137). It passes the same shared contract suite as the other backends and requires the AL32UTF8 database character set and the default NLS_COMP=BINARY, both checked at startup. It uses standard JDBC only; applications supply the Oracle driver. The schema user needs CREATE TABLE, CREATE TRIGGER and a tablespace quota. See Oracle schema.
  • Added OracleMigrationRunner. Oracle commits each DDL statement, so the runner records progress per statement and resumes an interrupted migration where it stopped. Concurrent migrators are serialized.
  • Added Spring Boot auto-configuration for the Oracle store, including join_transaction enqueue. With both JDBC stores on the classpath, Spring picks one from threadmill.store.jdbc-type, then the spring.datasource.url scheme, and otherwise keeps PostgreSQL. Oracle schema handling is set with threadmill.store.oracle.schema-mode.
  • Oracle has no cross-node wake channel: a job enqueued on one node is picked up by another within pollInterval (default 500 ms).
  • The shared store contract now requires that a null mutex release holder never frees another holder’s mutex.
  • Upgraded Jackson 2 to 2.21.7 and constrained Spring Boot’s test-scope Jackson 3 to 3.1.7, resolving GHSA-7hhh-6rmp-j9qf, GHSA-p6pp-m3f8-5c89, GHSA-cxp5-3px4-pw24, GHSA-wv8q-qhhj-9h54, GHSA-gx83-3vf8-gh7j, GHSA-q4xh-88c3-wmh7 and GHSA-wjgm-6hv5-3cvf. Applications that manage Jackson through their own BOM should move to the same patched releases.
  • Moved the dashboard UI build to Tailwind CSS 4.3.3 through the @tailwindcss/vite plugin, replacing Tailwind 3, PostCSS config and Autoprefixer. Tailwind 3 resolved braces (GHSA-vfj7-8cjw-p6xm, no patched release) through fast-glob, micromatch and chokidar, plus a vulnerable postcss-selector-parser (GHSA-rj75-hqrm-r3gf); Tailwind 4 has neither dependency. source-map-js moves to 1.2.2 (GHSA-68fv-2mgg-jv7q). npm audit reports zero known vulnerabilities. The theme moves into src/styles.css, with a small base layer that keeps Tailwind 3’s border, placeholder and button cursor defaults. The packaged console now targets the browsers Tailwind 4 supports (Safari 16.4+, Chrome 111+, Firefox 128+).

Threadmill 1.0.0 adds production hardening, operational tooling and migration support across the PostgreSQL, Redis and in-memory backends. See the compatibility and upgrade guide before upgrading from 0.3.0. Delivery remains at least once; handlers must be idempotent.

  • Hardened execution persistence through datastore outages. Finished handlers retain a finalizer until their outcome commits or shutdown transfers recovery to another node. Heartbeats renew only confirmed active attempts, so a lost claim acknowledgement cannot leave an abandoned job permanently shielded.
  • Persisted retry decisions and attempt-local execution revisions. Recovery retains the original retry policy, while delayed progress/log/check-in writes cannot overwrite acknowledged newer diagnostics or liveness.
  • Fixed concurrency-hold release for unclaimed jobs and execution cleanup across timeout, shutdown and orphan recovery. Metrics and tracing distinguish overlapping contexts for the same job and unwind their resources on every exit.
  • Bounded atomic bulk inserts and deferred Spring enqueues to 1,000 jobs and 8 MiB of encoded bodies. New JSON jobs reserve lifecycle space; bounded diagnostic compaction preserves identity, payload and execution policy.
  • Made deferred enqueue failures observable through AfterCommitEnqueueFailure. The default after_commit mode does not make business and job writes atomic; use PostgreSQL join_transaction with the same DataSource or a durable outbox.
  • Bounded maintenance and retention scans with resumable cursors. Retention protects pending/unknown retries, waiting workflows and live deduplication; idle queue and concurrency metadata can be reclaimed safely under churn.
  • Added PostgreSQL migrations V7–V11 for execution revisions, maintenance and retention indexes, and sharded queue monitoring counters. Historical V1–V6 migrations remain unchanged. Monitoring and queue discovery use bounded or indexed operations instead of repeatedly scanning all retained jobs.
  • Fixed Redis claim locks surviving timed-out/interrupted acquisitions, partial bulk acquisition and failed workflow preparation. Token-checked cleanup preserves replacement ownership and caller interruption, and tries remaining locks after an ordinary cleanup failure. Expiry remains the crash fallback.
  • Raised the Redis data-node minimum to 7.4 and added an offline format-2 index migration for existing namespaces. Script-cache recovery uses key-routed EVALSHA/EVAL, including after Cluster promotion and live slot migration. Standalone, Sentinel and Cluster contracts use real Redis; failover does not imply zero acknowledged-write loss.
  • Hardened application-classloader handling, dashboard authentication/error responses, Redis search capability enforcement, and optional asynchronous metrics refresh. Updated locked dependencies, including the patched Vitest development toolchain.
  • Expanded regression, migration, topology and production validation. The soak harness records live invariants, operation latency, retention and fault evidence, and recovers uncertain producer acknowledgements across datastore pauses/restarts without changing job IDs.
  • Updated installation and migration documentation for 1.0.0. Commercial support starts with 1.0 through hemju.com, with inquiries to sales@hemju.com. LingoHub is a reference customer.
  • Removed runtime backward-compatibility layers. Persisted handler and payload type tags now resolve exact current class names; TypeNameAliases, PayloadMigration, PayloadMigrations, JobSerializers, and JobDefinitionMigrator are gone. The JSON serializer no longer accepts unknown payload fields. Drain or rewrite affected durable work before renaming executable types or changing payload shapes.
  • Removed deprecated and transitional APIs: the test-support ForwardingJobStore alias, ThreadmillMetrics.recordClaimLatency, the timestamp-taking Redis queue-score overload, legacy Sentinel credential and TLS verification properties/constructors, shortened JobStoreCapabilities construction, and fingerprint-free recurring-state factories. The duplicate engine-side scoped-value constant is also gone; use JobExecutionContexts.CURRENT. DashboardApiService requires an explicit definition validator. Spring dashboard validation now uses exact handler and payload names.
  • Removed Redis v0.2.1 queue-score and per-queue-age-index upgrade machinery, layout markers, node layout heartbeats, repair scripts, and mixed-version detection. Redis deployments must already use the current priority score and age index before starting 0.3.0; mixed-version rolling operation is not supported.
  • Made the current contracts explicit instead of silently supplying fallback behavior. Every JobStore capability, every PostgreSQL transaction-boundary capability, and engine-backed JobExecutionContext deadline, cancellation, check-in, and result operation must be implemented. A missing operation is now a compile failure.
  • JobStore decorators now share the core ForwardingJobStore base (issue #131). The OpenTelemetry and Micrometer decorators forward every SPI operation, including external-transaction and remote-wake capabilities. The reflective JobStoreDecoratorContract fails when any current or future operation does not reach the delegate.
  • Process-fatal JVM errors are no longer swallowed as ordinary job or loop failures (issue #101). Every intentional catch (Throwable) now classifies VirtualMachineError / ThreadDeath — including through bounded wrapper and cause chains — and rethrows it before logging, interceptor invocation, failure serialization, or retry work. Long-lived dispatcher, maintenance, registry, and recurring loops catch RuntimeException rather than Throwable, worker tasks run through ExecutorService.execute so a fatal reaches the thread’s uncaught-exception path instead of an ignored Future, and the per-job watchdog is cancelled before the fatal escapes and before terminal finalization begins. A handler AssertionError remains an ordinary per-job failure. Threadmill still does not call System.exit: configure -XX:+ExitOnOutOfMemoryError (or an equivalent uncaught-exception handler) plus a process supervisor, because until the process stops, a fatal worker’s job stays PROCESSING and holds its concurrency slot.
  • Micrometer store gauges now refresh on their own bounded pull path instead of only on job completion (issue #100), so queue depths, per-state counts, and oldest-age gauges stay truthful when processing stalls and when a queue is paused or newly created. A failed refresh retains the last successful snapshot, keeps ages advancing, and exposes threadmill.metrics.snapshot.stale, a snapshot age, and a refresh-error counter rather than reporting zero. Concurrent gauge readers use the cached snapshot instead of queueing behind an in-flight refresh. meteredStore() now records claim latency and failures, persisted orphan reclaims, and rejected store-write attempts at the real JobStore boundary — contractual stale-version, oversize, and invalid-argument rejections are excluded from store-health signals — and per-queue meters are capped by a configurable hard tag limit (default 100) that reports overflow through threadmill.metrics.queue.tags.omitted.
  • Fixed PostgreSQL keyed claim polling permanently re-reading the same first page of concurrency-blocked keys, which starved claimable work behind a hot key (issue #92). Queue-scoped keys are now keyset-paged from a bounded, process-local cursor with least-recently-used eviction, cursor progress survives concurrent polls through identity-checked updates, the common path still costs one loose-scan query, and a drained tail restarts within the same poll instead of reporting an empty one.
  • Release publication is now fail-closed (issue #99): productionCheck cleans and checks every subproject from one ordered task graph, supported aggregate Central Portal publication requires that gate, direct per-module publication is rejected, and a tag is rejected before the expensive build unless it exactly matches the effective non-SNAPSHOT versions of all published modules.
  • Hardened dashboard executable-definition edits (issue #95). Pending-job and recurring-task updates that supply a handler or payload now require ADMIN, pass through a host-aware DashboardJobDefinitionValidator, resolve the same handler aliases as the runtime, reject direct or inherited raw and incompatible JobHandler declarations, and deserialize the payload before any store write. This is an intentional authorization change: REPLACE_JOB and UPDATE_RECURRING still permit operational queue, priority, schedule, trigger, policy, zone, and enabled-state edits, but no longer select executable classes. Spring securely returns 501 for definition edits when it cannot select one JobSerializer or an unambiguous alias registry; ordinary edits remain available. Job replacement also preserves existing producer-deduplication metadata when only the executable definition changes.
  • Dependency vulnerability checks now run as a distinct pull-request check, a nightly main scan, and a fail-closed pre-publish release step. Each path uses pinned Node/npm plus a checksum-verified OSV Scanner and evaluates every Git-tracked Gradle/npm lockfile. The policy, advisory-specific reachability record, and time-bounded exception rules are documented.
  • Patched the remaining locked dependency advisories: Spring Boot 4.0.8 with Spring Framework 7.0.9, Spring Security 7.0.7, Logback 1.5.38, and a Tomcat 11.0.25 test floor; Lettuce 6.8.2.RELEASE with Netty 4.1.137.Final; Browserslist 4.28.9; and PostCSS Selector Parser 6.1.4. Lettuce 6.8 adds netty-resolver-dns/netty-codec-dns to the published Redis module’s runtime graph; these are upstream compile dependencies, not optional extras.
  • Redis Cluster and Sentinel now support production ACL/TLS configuration (issue #93). Cluster credentials and verified TLS apply to every seed; Sentinel keeps Redis data-node credentials separate from Sentinel control-plane credentials while using Lettuce’s shared TLS policy for both. Spring exposes the matching properties, full peer verification defaults on, CA-only verification is also supported, and caller-managed RedisClusterClient instances can be injected for custom TLS resources, mutual TLS, and rotating credentials. Topology descriptions and configuration-owned startup failures redact usernames and passwords while retaining a sanitized exception-type chain. Real authenticated/mTLS Cluster and Sentinel tests cover the injected-client and store paths.
  • Fixed a release-blocking Redis Cluster CROSSSLOT failure in ordinary job insertion and transitions. Optional Lua KEYS entries used empty strings, which hash outside the engine slot; absent keys now use the reserved {threadmill}:no_key same-slot sentinel loaded from one Java constant. The complete shared job-store contract now runs through an actual Cluster client and asserts that the protocol sentinel is never materialized as a Redis key.
  • Handlers can see the engine’s deadline coming (issue #119). JobExecutionContext gains deadline() and remaining(), computed by the same rule the timeout watchdog uses — claimedAt plus the effective timeout before the first check-in, the most recent check-in plus noProgressTimeout after it — and capped at the end of the node’s shutdownGracePeriod once close() begins, so a handler that runs a loop of costly steps can stop before the interrupt instead of being cut off mid-step by a timeout or a rolling deploy. cancellation() / isCancelled() report why the engine abandoned an attempt (TIMEOUT or SHUTDOWN); the record is written immediately before the worker thread is interrupted and never cleared. JobExecutionContext.current() resolves the running context from code below the handler that has no ctx parameter. All additions are default methods; existing implementations keep compiling.
  • The failure path now classifies a shutdown interrupt from the engine’s own cancellation record rather than from the exception type. A handler interrupted inside socket I/O on a virtual thread surfaces SocketException: Closed by interrupt, not InterruptedException, and was previously billed as a handler fault — an attempt burned and the retry delayed by backoff. It is now SHUTDOWN: rescheduled immediately, no attempt consumed.
  • Documented the interrupt contract on @Job(timeout), threadmill.jobTimeout, threadmill.noProgressTimeout, and in the handler and long-running-job guides: the worker thread is interrupted; on virtual threads that can abort in-flight socket I/O (guaranteed for java.net.Socket and interruptible channels) and leaves the interrupt flag set; a timeout interrupt is re-asserted every watchdog tick until the handler returns while a shutdown interrupt is delivered once; a handler must treat it as cancellation.
  • Hardened the timeout path around the new API: the watchdog keeps interrupting a cancelled attempt even if cleanup code calls checkIn(), a worker that registers during the shutdown sweep still classifies its interrupt as SHUTDOWN, and a per-job timeout override beyond ProcessingNodeConfig.MAX_TIMEOUT (one hundred years) degrades to the global timeout instead of overflowing before the handler runs. The config rejects jobTimeout, noProgressTimeout, and shutdownGracePeriod above that bound.
  • Queue claim order is now exactly (priority DESC, job id) on every store (issue #91). Redis queue and unkeyed-lane ZSET scores are exact negated priorities, so a lower-priority job can no longer overtake after about 116 days and extreme priorities lose no timestamp precision; JobId now defines allocation-free unsigned UUID natural order, matching Redis and PostgreSQL even across Java’s signed UUID.compareTo boundary. Redis upgrades legacy scores in bounded, atomic pages that cannot resurrect a concurrent claim or overwrite a replacement. The layout stays rescored while a legacy-scoring node heartbeat is live, then a cluster-wide lease-backed mutex elects one node for the final exact pass before it becomes priority_only_v1; rolling upgrades may temporarily perturb ordering but do not lose or duplicate jobs. The deprecated RedisKeys.queueScore(int, long) compatibility overload is marked for removal; enqueue time is ignored.
  • Redis oldestEnqueuedAt — the threadmill.queue.oldest.enqueued.age gauge and the dashboard queue view — is now one head read of a per-queue age index ({threadmill}:queue_enqueued_at:{queue}, scored by current_state_at) instead of a Lua scan that did one HGET per ENQUEUED member inside one atomic call on every metrics scrape. The index is maintained inside the same atomic scripts as queue membership. A store started against data written by v0.2.1 or earlier backfills it once from the queue ZSETs (bounded SSCAN / ZSCAN pages from Java) and records the {threadmill}:layout:queue_enqueued_at state: backfilled while any old-release node heartbeat is live (every start reconciles exactly), complete once none remains (finalized from a read, so the last old node’s exit needs no restart; later starts check two ZCARDs per queue). New-release nodes write {threadmill}:node:layout:{nodeId} next to their heartbeat for that detection. The read verifies its head against the queue ZSET and drops a stale head through an atomic compare-and-remove. Upgrade procedure and mixed-version guarantees are in the Redis README.

Tagged but never published. Both release builds failed in the pre-publish gate — first on a missing browser in the release workflow, then on the correctness simulation’s drain deadline expiring on a loaded CI runner — so no artifact was signed or uploaded for either version and neither exists on Maven Central. Their contents shipped as 0.2.4.

  • Fixed Postgres self-owned writes being silently rolled back when the host DataSource hands out connections with autoCommit=false (issue #111). Queue pauses, execution and node heartbeats, maintenance leases, retention, mutexes, recurring definitions/state, and the expired-dedup fallback now use explicit Threadmill-owned transactions, restoring the connection’s prior auto-commit mode after commit or rollback. saveAtomic, softDelete, claimReady, and replaceJob now use that same boundary instead of forcing auto-commit on return. Migration history bootstrap and destructive schema reset also commit explicitly. The expired-dedup fallback now keeps its SELECT … FOR UPDATE lock through the delete, closing a lost-update window that also existed on auto-commit pools.
  • The recurring materializer now repairs a timing-fingerprint mismatch found during its under-mutex definition reload (issue #112). The current task definition wins: Threadmill preserves run bookkeeping and pending nudge demand, recomputes timing forward from the tick, and does not fire or CATCH_UP the obsolete trigger’s overdue backlog. A listed fingerprint mismatch now triggers that repair even when the stale nextRunAt is still in the future, so a faster replacement schedule cannot remain dormant until the obsolete fire time. Legacy null fingerprints are adopted without dropping or moving an existing fire, and detected crashed edits emit an operator-visible warning.
  • Added fixed process-separated nudge simulations for Postgres and Redis (issue #114). They hard-kill a maintenance leader after an accepted nudge and prove the standby serves it, then hard-kill a producer after its durable work write but before its nudge and prove the regular recurring backstop drains the row. Cross-process JSON-lines traces record process ids, leadership, trigger origins, and the verified event ordering. Ready markers are atomically published and trace writes drain their buffers fully; the fixed real-backend simulation is part of productionCheck. The failover phase uses a one-minute backstop and five-minute leader poll, then edits the task to an eight-second backstop for the producer-crash phase, eliminating process-start timing races. Simulation work uses its own datastore namespace and long-lived per-process connections.
  • Recorded the issue #113 decision not to add per-task lifecycle generations to the JobStore SPI. Issue #114 proves hard-kill handoff and backstop recovery, while the remaining revision-reuse ABA additionally requires an old materializer to resume after outliving its mutex lease and a same-name delete/re-registration. A persistent generation would not replace the existing lifecycle orderings and does not justify its cross-backend storage, migration, and SPI cost without evidence that sequence occurs.
  • Recurring tasks can declare claim-time exclusivity (issue #110). A CronTask gains an exclusive flag, surfaced as @Recurring(exclusive = true) and as an exclusive parameter on Scheduler.defineRecurring. Every materialized instance — scheduled, caught-up, or manually triggered from the dashboard — is claimed under the derived key recurring:<name> in ConcurrencyMode.EXCLUSIVE, so the store refuses to admit a second instance while one is processing. This replaces the per-handler advisory lock a singleton sweep usually grows. It does not close the lease-expiry reclaim window: reclaim releases the concurrency slot as part of the terminal failure save, so handlers stay idempotent by contract. Postgres adds the additive migration V6__cron_task_exclusive.sql; Redis stores the flag as a cron-task hash field with the existing overwrite-on-re-upsert semantics.
  • The recurring pile-up guard no longer opens a window between a failed instance and its retry (issue #110). FAILED is not terminal — a retry may follow — but the guard treated every FAILED as finished, so a materializer tick landing between the failure save and RetryInterceptor’s reschedule save could create a fresh instance beside a retrying one. A FAILED instance now blocks while its retry budget is not provably spent and the failure is younger than a five-second handoff grace. A retry-exhausted instance still never blocks, and the age bound keeps an instance that is terminal under a per-exception-type policy — which the guard cannot read — from blocking its task until the stranded-failure recovery scan reaches it.
  • DashboardPayloads.CronTaskView gains an exclusive component.
  • Documented the three overlap windows and the reclaim window’s unclosability, with fencing guidance for effects that must not happen twice (issue #110).
  • On-demand materialization for recurring tasks (“nudge”, issue #108). Scheduler.nudgeRecurring(taskName) (Spring: JobScheduler.nudgeRecurring) requests that a registered recurring task materialize an instance as soon as possible, turning frequent pollers into wake-driven pumps with a slow backstop schedule. The nudge goes through the normal machinery — same pile-up guard, same missed-run policy — with four guarantees: at least one run starts after every accepted nudge (a nudge during an in-flight run produces exactly one follow-up after it completes, because the in-flight run may have read its inputs before the nudge’s triggering write committed); a burst of nudges coalesces to at most the current run plus one follow-up; the nudge is a durable store write consumed by the maintenance tick, so there is no transient signal to lose (worst-case latency one maintenancePollInterval, default 1 s); and a nudged run never moves the schedule — a cron task’s next fire stays the regular wall-clock match and an interval trigger’s phase is preserved. Nudging an unknown task throws; nudging a disabled task throws — an explicit pause wins — and an enabled-flip clears any pending nudge. The coalescing bound is failure-free: consistent with at-least-once, a crash between the follow-up’s insert and the request’s clear can produce an extra run, never lose one. Storage: additive Postgres migration V5__cron_state_nudge.sql adds threadmill_cron_task_state.nudge_requested_at plus a store-generated, never-reset nudge_revision — the revision, not the collision-prone wall-clock timestamp, is the compare-and-clear identity; Redis stores both as schedule-state hash fields. CronTaskScheduleState gained read-only nudgeRequestedAt / nudgeRevision components.
  • Spring nudges are after-commit in every enqueue mode, including join_transaction. Validation fails fast at call time, the write lands on commit, and a rollback discards it — identical semantics in all three modes. This is the one write that deliberately does not join the caller’s transaction: coalescing is one store cell per task, so a joined nudge would hold that row’s write lock for the whole business transaction and serialize every concurrent producer of that task (silently — correct at low rate, collapsing under load), and all it would buy is closing a crash window the design explicitly does not need closed. An in-JVM per-task coalescer additionally bounds the store write rate under bursts, and never retains a caller beyond its own covering write (follow-up generations run on a dedicated virtual thread).
  • Spring: nudge by handler class, jobScheduler.nudgeRecurring(OutboxPump.class). A @Recurring task’s durable identity defaults to the handler’s fully-qualified class name, so the string overload forced callers to hard-code it and broke on a rename or package move; the class overload resolves the registered name through the handler registry and matches the rest of this API, where the handler class is always the first argument. Nudging a registered handler that is not @Recurring fails loudly. The string overload remains for tasks registered imperatively through the core Scheduler, where the caller chooses the name.
  • Breaking (SPI): JobStore gained two abstract operations, requestCronNudge(name, requestedAt) → ACCEPTED | UNKNOWN_TASK | DISABLED and clearCronNudge(name, observedRevision). Third-party store implementations must add both before recompiling (old binaries throw AbstractMethodError when nudged). Implementation contract: acceptance must atomically check task existence + enabled and advance a monotonic, never-reset revision; the clear must compare-and-clear on that revision and must not touch it; upsertCronTaskState must preserve both nudge fields; a nudge must never resurrect state for a deleted task. The contract test suite (AbstractJobStoreContractTest) pins all of this.
  • Recurring instances now carry threadmill.cron.origin metadata (schedule / nudge / manual), surfaced in three places: handler code (JobExecutionContext.cronOrigin()), the dashboard API (JobSummary.cronOrigin, deliberately visible on redacted read-level views — the value set is closed, so no metadata can leak through it), and Micrometer (threadmill.jobs.recurring.runs{origin=schedule|nudge|manual|other}, cardinality-clamped). The operations console renders it as a badge beside the handler, so a nudged run is distinguishable from a scheduled one at a glance.
  • Breaking (Spring): renamed @Job(maxRetries) to @Job(maxAttempts) (issue #104). The value always counted total attempts — maxRetries = 1 meant one attempt and zero retries — so every existing number keeps its runtime meaning under the honest name; only the attribute name changes. Non-positive values other than the -1 sentinel now fail startup with a descriptive error instead of being silently replaced by the engine default: the natural “disable retries” spelling 0 used to run up to five attempts. The rename also reaches ThreadmillJobRegistry.Registration: its component changes from int maxRetries to a nullable Integer maxAttempts (constructor, accessor, and binary signature change; null means “no explicit budget”). Code that constructs registrations directly or reads them via ThreadmillRecurringRegistrar.recurring() must follow the new shape.
  • Per-job retry metadata is now stamped only for an explicit @Job(maxAttempts) (issue #104). Previously the resolved default was stamped on every Spring-enqueued job, so per-job metadata permanently shadowed per-exception-type RetryInterceptor policies. Jobs without an explicit budget now fall through to the policy chain, and @Recurring handlers without an explicit budget register their CronTask with a null retry budget, so later threadmill.default-max-attempts changes are honored by already-registered tasks.
  • Fixed startup re-registration wiping an overdue recurring next_run_at (issue #105). Scheduler.upsertCron now preserves the schedule state — including an overdue next-run and an interval trigger’s phase — while the re-registered schedule is unchanged, so the task’s MissedRunPolicy decides what happens to restart-missed firings: CATCH_UP materialises every missed fire (capped per tick) and DROP collapses the backlog into one make-up run shortly after startup. A real edit — changed trigger, changed zone on a cron trigger, or re-enabling a disabled task — still recomputes the next-run from now so an edited cron never fires stale times. CronExpression gained source-based value equality to support the unchanged-schedule comparison.
  • CronTaskScheduleState now carries a timingFingerprint recording which trigger timing its next_run_at was computed from, written atomically with the state row (additive Postgres migration V4__cron_state_timing_fingerprint.sql; a Redis hash field; legacy rows read as null and adopt the current fingerprint without moving their recorded timing). The unchanged-schedule decision reads this fingerprint rather than comparing stored task definitions, so a crash between the separate task-definition and state writes can never pair a new trigger with old timing undetectably — the retry detects the mismatch and recomputes. The fingerprint deliberately excludes the zone for interval triggers (CronTask documents the zone as ignored there), so nodes with different system-default zones do not treat the same interval as a schedule edit.
  • DROP missed-run recovery is now phase-exact and nominally stamped: the single make-up instance represents the most recent nominal fire (its cronFireTime() carries that nominal time, not the recovery wall-clock), and the next run advances from the nominal fire — an every-6h task that was due at 06:00 and recovered at 07:00 next fires at 12:00, not 13:00. Behavioral note: with overdue state now surviving restarts, an existing DROP task whose boundary passed during downtime fires once shortly after startup instead of silently skipping to the next boundary — the same behavior it always had across a live materializer stall.
  • Patched OSV-flagged dependencies for the release: PostgreSQL JDBC 42.7.12, Netty 4.1.136.Final, Micrometer 1.15.12, a test-classpath constraint on tools.jackson.core:jackson-databind 3.1.5 (Spring Boot 4.0.7’s BOM still pins 3.1.4), and refreshed dashboard build dependencies (nanoid, postcss). npm audit and the OSV scan report zero known vulnerabilities for the release lockfiles.
  • Fixed Spring lifecycle ordering so ProcessingNode starts before remote-wake subscription and stops after the subscription closes. Spring applications now fail fast when no durable store is configured; the in-memory store must be explicitly enabled with threadmill.store.memory.enabled=true.
  • Required the per-task correctness mutex for recurring definition registration, reconciliation, dashboard mutation, and deletion. Contended mutations now fail explicitly instead of proceeding without their lock.
  • Retained terminal-save responsibility through store outages. A worker keeps retrying PROCESSING -> SUCCEEDED / FAILED with capped backoff until the write commits or node shutdown begins, preventing completed attempts from being stranded behind refreshed owner heartbeats.
  • Hardened persisted invariants with defensive JobSnapshot copies, bounded handler type names, safe ProcessingNode lifecycle behavior, startup migration checksum/description validation, and additive PostgreSQL V3 integrity constraints.
  • Bounded Redis claim candidate discovery with rotating HSCAN cursors and pipelined per-key probes, avoiding backlog-wide registry reads under large keyed workloads.
  • Added Gradle wrapper verification, dependency locking and SHA-256 metadata, immutable GitHub Action pins, and mandatory META-INF/LICENSE / NOTICE inspection for every published JAR. Refreshed dashboard build tooling to patched Babel, esbuild, and Vite releases; npm audit reports zero known vulnerabilities for the release lockfile.
  • Upgraded Jackson, Spring Boot, Spring Framework, Spring Security, Netty, PostgreSQL JDBC, Logback, and AssertJ to patched maintenance releases. The release OSV gate now scans every committed Gradle and npm lockfile instead of passing an unsupported version-catalog file to the scanner.
  • Synchronized installation versions, release/security guidance, API names, dashboard security documentation, Redis execution-model documentation, and soak-result wording.
  • Fixed @Job(timeout) and @Job(maxRetries) being silently ignored for @Recurring handlers (issue #84). A CronTask now carries an optional per-instance timeout and maxAttempts; RecurringMaterializer and the dashboard’s manual trigger stamp them onto every materialized instance as JobRunner.META_TIMEOUT_SECONDS / RetryInterceptor.META_MAX_ATTEMPTS, so the annotations behave identically on the recurring and enqueue paths. Previously every recurring instance ran under the global jobTimeout and the default retry budget, so a long-running recurring job could be interrupted mid-run and retried (duplicating external side effects).
  • Added a per-instance timeout / maxAttempts parameter to Scheduler.defineCronTask, defineIntervalTask, and defineRecurring for core (non-Spring) callers. Existing signatures are unchanged (null keeps the engine defaults).
  • Added the additive Postgres migration V2__cron_task_overrides.sql (nullable timeout_seconds and max_attempts on threadmill_cron_tasks); existing rows default to the engine behaviour. Redis stores the same as optional hash fields cleared by an override-less re-registration.
  • Fixed Dispatcher release of claimed-but-unrun jobs (node-tag mismatch, dispatch failure, shutdown mid-batch): releases now route through the single failure path as FailureCause.SHUTDOWN, so the job is rescheduled immediately without consuming a retry attempt and its claim-time concurrency slot is freed. Previously the release attempted an illegal PROCESSING→SCHEDULED transition, always threw, and left the job to orphan reclaim.
  • Fixed the default job id and createdAt to derive from a single clock read in Job.Builder.build(), keeping UUIDv7 id ordering consistent with the engine’s in-key claim-admission order.
  • Validated with a 12-hour PostgreSQL endurance soak (mixed workload, ~5.5 million jobs, 3 nodes, node churn every 10 minutes): passed with all invariants green.

First public release under the Apache-2.0 license.

  • Added claim-time per-key concurrency with ConcurrencyMode, workflow-root inheritance, store-backed enforcement in memory/Postgres/Redis, and documentation for import/export and tenant event-processing shapes.
  • Added queue-family lanes with anchored * / ? patterns, stride-scheduled QueueWeights, discovery retention, Spring configuration, and soak coverage across all stores.
  • Fixed workflow-root concurrency release for failed intermediate workflow steps by abandoning descendants that can no longer be promoted.
  • Added the Spring ergonomic API: @Job, typed handler discovery, and JobScheduler.
  • Added Redis standalone, Sentinel, and Cluster configuration. Cluster uses a single {threadmill} hash slot for v1 Lua correctness.
  • Added producer-side deduplication with Created / Coalesced results.
  • Added long-running job check-ins, progress updates, bounded logs, and no-progress timeout handling.
  • Removed the experimental alternate framework module and public positioning for now. The core remains framework-agnostic so additional integrations can be added later.