Bill Gernert
Automation & Tooling

Job-search platform

A self-hosted application I built and operate on the same platform that runs everything else - a real app with a real data pipeline, held to the same review and migration discipline as the infrastructure.

What it is and whyHow I use it inArchitectureLessons learned and gotchasImpact

What it is & why I use it

The job-search platform is a self-hosted application I built to run on top of AutomationLab. It matters here because the lab operates a stateful application with a database and a scheduled data pipeline. The application uses the same GitOps and pipeline discipline as every other workload.

Building it end to end - schema, data collection, deployment - is also where a lot of the platform's rougher edges showed up first, because an application with persistent state exercises parts of the system that stateless services never touch.

How I use it in the lab

  • Deployed like everything else. The app ships through the same build-and-deploy path as the rest of the cluster - reviewed changes, versioned images, reconciled state - not a hand-deployed pet.
  • A scheduled data pipeline. Staged jobs collect and enrich data on a schedule, so the app's content stays current without manual intervention.
  • Database-backed with real migrations. Schema changes are version-controlled migrations that apply as an explicit pipeline stage - which turned out to be the single most important operational detail (see the gotcha).
  • Secrets from the manager, config from git. Connection details come from the secrets manager at run time; everything non-secret is declared in the repo.

Architecture

Job-search platform architecture: Application delivery with gated migrations The scheduled collector and enrichment job prepare data, the migration gate updates the schema first, and the application and database move together under pipeline control. ARCHITECTURE FLOW APPLICATION DELIVERY WITH GATED MIGRATIONS Schedule: Start the collection cycle SCHEDULE Start the collection cycle Collect: Fetch the permitted source data COLLECT Fetch the permitted source data Enrich: Normalize and add derived fields ENRICH Normalize and add derived fields Migration gate: Apply schema changes before use MIGRATION GATE Apply schema changes before use Application: Run against the expected schema APPLICATION Run against the expected schema Database: Store the migrated application data DATABASE Store the migrated application data Cyan. Automated step or path. Amber. Operator or policy gate. Green. Recorded or healthy outcome. Dashed box. Stored state or record. Rounded box. Actor or process. Arrow. Direction of work or data.
The application, database, and scheduled collect and enrich work share a delivery path where migrations gate schema changes. Hostnames, connection strings, addresses, and data are omitted.

Lessons learned & gotchas

Gotcha - a merged migration is not an applied migration The one that cost me real confusion: merging a PR that adds a database migration does not change the schema. The migration only takes effect when the pipeline's migrate stage actually runs - here, as part of the scheduled data pipeline. So right after a merge, the code expects a column the live database does not have yet, and you get errors that look like a bug in the app when the real cause is "the schema change has not been applied." The rule I now follow: never assume a merged migration is live - run the migrate stage (or verify against the database) before trusting it.
  • Deploy order matters for stateful apps. Code that depends on a new column has to land after the migration that adds it, not merely after the migration is merged. "Merged" and "applied" are different points in time, and the gap is where outages live.
  • Verify schema against the database, not the repo. The repo tells you what the schema should be; only the database tells you what it is. When something looks broken after a change, check the actual schema first.
  • Stateful applications expose platform gaps. Stateless services hide a lot of sins. A database, migrations, and a schedule showed that the platform could carry production-shaped workloads.

Impact

1 app
built and operated end to end
Scheduled
collect + enrich pipeline
Gated
schema migrations

The job-search platform is the workload that turns AutomationLab from "infrastructure I automated" into "a platform that runs a real application." The migration lesson is the one I carry everywhere now: shipping the code is not the same as changing the world it runs in, and the discipline is verifying the change actually landed - not assuming a merge made it so.