Technical project walkthrough

How I built one server provisioning workflow with two front doors

A Windows or Linux request becomes a running virtual machine with an allocated address, infrastructure state, operating-system configuration, monitoring, backup coverage, and a committed host record. Windows servers also join Active Directory and enforce Duo MFA for remote desktop logins. Jenkins can coordinate the run, or Backstage can open a reviewed request that Gitea Actions applies without Jenkins.

The controls below are browser-only simulations of the current request contracts. They do not connect to the lab or change infrastructure.

View the code →

At a glance

Outcome
A Linux or Windows request reaches a running server with monitoring, Wazuh, an initial security update, verified backup coverage, private DNS, and a committed host record.
Operating scope
The live workflow coordinates address allocation, Terraform, Proxmox, Ansible, Vault, monitoring, Wazuh, patching, backup coverage, and Active Directory with Duo MFA for Windows.
Evidence
Live Jenkins proof runs for Linux and Windows Server 2022 both completed on 2026-09-03 through security readiness, private DNS, and durable host-manifest publication. The Backstage route completed a measured run in 5 minutes 21 seconds on 2026-08-23.
Primary lesson
Both front doors revalidate collisions, and post-creation failures retain enough state for the next run to continue forward safely.

The Windows form shows what the workflow can build

What the request controls

The form names the machine and its role, then sets compute, storage, and the Windows Server template. It can also add tools, Windows features, a data disk, and nested virtualization.

What Jenkins checks first

The job validates the hostname, required values, storage limits, and feature constraints before it reserves an address. The console follows that same order.

JenkinsPipeline [VM] Provision WindowsInteractive demo

This build requires parameters. Values stay in this browser.

TOOLS

Optional user tools installed through the allowlist.

WINDOWS FEATURES

Optional Windows roles and features.

Demo only. No address, VM, credential, or DNS record is created.

Choose tools, Windows features, or a data disk, then approve the simulated build. The console shows a short Ansible playbook with the selected installation steps and skips unused options. Feature installation does not configure a server role or promote a domain controller.

Conditional disk work

The operating-system disk moves only for node-local storage. A second disk is created only when the request includes one.

Template and first boot

The pipeline resolves a template by tags. Terraform clones it, and a first-boot script brings up WinRM over HTTPS.

Operating-system configuration

Ansible rotates the password, joins Active Directory, installs Zabbix and Wazuh, configures time and firewall rules, enables Duo for remote desktop, and installs the selected tools and roles. Later gates apply the initial security update and verify Defender, monitoring, Wazuh, firewall, reboot, and backup readiness.

Credentials during the run

Vault stores the per-host administrator password and monitoring key. The job retrieves each value only for the stage that needs it.

Linux uses the same backbone with fewer choices

The Linux path uses a fixed Rocky Linux 9 cloud image. It skips the Windows template-resolution, WinRM, password-generation, and disk-movement stages. A configuration profile can add a CI runner after the base operating system is ready.

JenkinsPipeline [VM] Provision LinuxInteractive demo
No address or VM is created.

Choose Base server or CI runner to change the playbook output. Both install monitoring and the security baseline; CI runner also installs Docker and the Actions runner. The build waits for your approval before it simulates creating or configuring the VM.

The short form is intentional. The role selects the allowed network, and the requested memory feeds the placement check. Terraform creates the VM, cloud-init sets the initial network and SSH access, and Ansible applies the versioned base configuration. Later gates apply the initial security update and verify monitoring, Wazuh, SELinux, firewall, patch timer, reboot, and backup readiness.

Jenkins coordinates both operating-system paths. It renders parameters, starts the provisioner pod, checks gates, pauses for approval, keeps the stage log, and sends the result. The address, infrastructure state, VM, operating system, and credentials remain with their owning systems.

A new server inherits more than a virtual machine

The build installs controls that belong on the guest. Hypervisor and estate jobs continue after the handoff. Duo applies to Windows remote desktop, and backup coverage starts at the hypervisor without an agent inside the guest.

CapabilityLinuxWindowsWhen it applies
Identity and accessManaged SSH access and the Linux firewall baseline.Active Directory membership, a unique administrator password, and Duo MFA for remote desktop.During the build
Monitoring and detectionZabbix and Wazuh agents with managed host identities.Zabbix and Wazuh agents with managed host identities.During the build
Host baselineFirewall rules, time sync, and Alloy log forwarding.Firewall rules and domain time sync.During the build
BackupsNightly whole-VM backup and daily freshness reporting. Provisioning verifies backup coverage before handoff.Nightly whole-VM backup and daily freshness reporting. Provisioning verifies backup coverage before handoff.Verified during the build
PatchingAn initial security update and reboot check run before handoff. Automatic security updates and the controlled weekly patch path continue from the host manifest.An initial security update and reboot check run before handoff. Weekly patch plans, update scans, and approval-gated rolling updates continue from the host manifest.Build and estate schedule
Security checksWazuh must report the host active. SELinux, the default-drop firewall, patch timer, and required services must pass readiness.Wazuh must report the host active. Defender, all firewall profiles, default-block policy, and required services must pass readiness.Readiness gate
Durable recovery dataThe address, Terraform state, placement, sizing, and host manifest remain available for reruns.The same machine record is kept, and the per-host administrator secret is retained for backup restores.At successful handoff

Both Jenkins pipelines follow six phases

The stage lists are longer because Jenkins gives each operation its own log and failure boundary. At the workflow level, those stages fit into six phases.

The approval gate shows what will change

Jenkins sends this email after Terraform has produced the plan and before the pipeline applies it. The message identifies the machine, allocated address, VM identifier, selected node, role, storage, and requested resources. The button opens the waiting Jenkins input step, where the operator can apply or abort. A plan that would replace an existing machine uses destructive wording so it cannot look like an ordinary new-server approval.

  1. Describe the machine. Jenkins validates its parameters before checkout-dependent checks run. Windows adds NetBIOS length and character rules, storage limits, and the requirement that Hyper-V includes nested virtualization. Values offered by the form are defined in version-controlled pipeline code.
  2. Prove the request is safe. The target check rejects a role whose network path is not supported. The collision checker asks address management, the hypervisor, monitoring, committed host manifests, and configuration inventory whether the name already exists. A confirmed match stops the build. An unavailable source is unknown, and unknown also stops the build. These checks run before a new address is reserved.
  3. Reserve capacity and produce a plan. NetBox reserves the next free address in the role’s allowed prefix. The job derives the gateway and VM identifier from that allocation, then selects an online hypervisor with enough free memory and disk. Terraform opens one workspace for the requested machine and stores its state in PostgreSQL. Vault supplies the provider and backend credentials at run time. Jenkins saves the plan that the approval step will review.
  4. Approve and create. Jenkins sends the request details and a link to a bounded approval gate. If Terraform says an existing machine would be replaced, the message and button use destructive wording. Approval applies the saved plan. Once the hypervisor has created the VM, the job records that boundary so later cleanup cannot release an address that a live machine still uses.
  5. Configure and connect the guest. Zabbix receives a host object and a newly generated per-host key. Vault keeps that key for the agent configuration. Linux is configured over SSH. Windows waits for repeated successful WinRM checks before Ansible starts because first boot can briefly open and close the port. Ansible configures the operating system and Wazuh, applies the initial security update, and proves OS, monitoring, reboot, and backup readiness. A DNS-specific step then creates the forward record and creates a reverse record only when the matching reverse zone exists.
  6. Write the durable record. The final stage creates a host manifest in Git with the selected node, address, sizing, role, and build reference. A later run reads that manifest and reuses the recorded node rather than accidentally planning a move.

Backstage provides a reviewed request route without Jenkins

The lab is Jenkins-heavy, so I built the second route to learn how the provisioning workflow would operate without Jenkins. It also gave me practical experience with Gitea Actions while keeping the same request contract and infrastructure tooling underneath.

What the portal handles

Backstage collects the server request and checks the form shape. The template then writes a small declarative request file.

Where the portal stops

Backstage opens a pull request and hands off. Gitea Actions performs the infrastructure checks and apply after review.

backstageInteractive demo
Software templates

Create a new component

Choose a template to start a request.

Start the template, change the request, review it, and open the simulated pull request.

  1. 01

    The request file becomes the contract

    The file records the operating system, hostname, role, size, storage, and optional configuration selected in the portal.

  2. 02

    The pull request runs the safety checks

    Gitea Actions validates the schema and target, then runs the same fail-closed collision checker used by Jenkins. A configuration-only Terraform plan checks the request without reserving an address.

  3. 03

    The merge records approval

    The reviewer sees the requested machine and each check result in one pull request. Merging it starts the separate apply workflow.

The merge starts the apply workflow

The apply driver revalidates the request, repeats the safety checks, allocates the address, selects the node, runs Terraform, registers monitoring, configures the guest with Ansible, and writes the host manifest. A manual Actions dispatch can continue a request after a partial run.

Workflow runApply approved server requestsmain · simulated request
In progress
0%
Provision requestRunning: Detect changed requests

The simulated run advances on its own and starts again after it completes. Pause or restart it to inspect an individual stage.

Jenkins route

  1. Jenkins renders the parameter form.
  2. The pipeline validates, checks, reserves, and plans.
  3. An operator approves inside the Jenkins run.
  4. The saved plan is applied from the provisioner pod.
  5. The provisioner reaches AD-integrated DNS over WinRM.
  6. Jenkins keeps the stage log and notification.

Backstage and Gitea route

  1. Backstage writes a request and opens a pull request.
  2. Gitea Actions runs schema, target, collision, and plan checks.
  3. A reviewer approves by merging the pull request.
  4. A merge workflow applies through the shared driver.
  5. Ansible reaches AD-integrated DNS over WinRM.
  6. Gitea keeps the checks, apply log, and committed request.

The two routes share the systems that own the result. They differ in request, approval, orchestration, and log location. This is why the Backstage path can continue if Jenkins is unavailable without creating a second Terraform or Ansible implementation.

What the workflow touches and why

A provisioned server is not finished when the hypervisor reports that a VM exists. The workflow has to leave several systems describing the same machine. Each one has a narrow responsibility.

SystemWhat it ownsHow the pipeline interactsWhat failure means
Git and GiteaPipeline code, request files, Terraform, playbooks, inventory, and host manifestsBoth routes check out reviewed code. Backstage opens a pull request; the last phase commits the machine record.The run cannot establish its reviewed inputs, or it cannot leave the durable inventory record.
Jenkins or Gitea ActionsOrchestration, gates, logs, and run statusOne engine orders the same underlying scripts and records where the run stopped.The workflow stops, but ownership of existing addresses, VMs, and records remains with their source systems.
VaultProvider credentials, SSH and WinRM material, per-host administrator passwords, and monitoring keysA job-specific AppRole reads or writes only the records required by that path. Secret values are masked and kept out of Git.A stage that needs a credential fails before it can call the owning system.
NetBoxAddress allocation and the DNS name attached to that addressThe allocator reserves the next free address for the selected role.The build cannot safely choose an address, so it stops.
Terraform and PostgreSQLThe requested VM definition and per-machine infrastructure stateTerraform creates a plan and sends the approved change through the Proxmox provider. PostgreSQL stores one workspace per machine.No infrastructure change is allowed without usable state and a valid plan.
ProxmoxThe VM, its placement, disks, virtual hardware, and power stateThe provider creates or reconciles the VM. Helper scripts resolve templates, clear inherited tags, and handle requested disk work.The creation marker records whether later recovery is dealing with a reservation or a live VM.
Ansible and the guestThe operating-system configurationAnsible connects through SSH for Linux or WinRM for Windows and applies the versioned roles for that operating system.The VM remains allocated. A rerun resumes configuration instead of pretending the machine does not exist.
Zabbix and WazuhThe monitored host, its encrypted monitoring identity, and its detection groupThe workflow registers monitoring, configures both agents, and then requires each service to report the host ready.The machine is not considered finished because it would exist without monitoring or host detection.
Active Directory and DNSWindows computer membership and authoritative name recordsThe Windows playbook joins the domain. The DNS stage writes forward records and reports reverse records separately.A live VM may exist, so the workflow preserves its address and reports the integration stage that needs a rerun.

How it runs now

This workflow is event-driven. Nothing polls for a machine to build. A request starts the control path, and the creation boundary decides what recovery is safe.

WhenWhat runsWhat happens next
On requestJenkins validates its submitted parameters, or Backstage writes a request and opens a pull request. Target and collision checks run before infrastructure is changed.A rejected or unverifiable request stops before an address is reserved. A valid request continues to planning and review.
On approvalJenkins applies the saved Terraform plan after its input gate, or merging the Backstage request starts the Gitea Actions apply workflow.The workflow allocates or confirms the machine state and configures the guest. Jenkins also applies the initial security update, proves operational readiness and backup coverage, registers private DNS, and writes the host manifest.
On failure before creationThe unsuccessful handler checks the creation marker and calls the idempotent NetBox release path.The reserved address is released because no VM uses it. The request can start again as a clean build.
On failure after creationThe workflow keeps the VM, its address, Terraform workspace, and completed registrations. The failed stage remains visible in the run log.Cleanup does not dismantle a live machine. The next run continues forward from the recorded state.
On rerunThe safety checks run again. An approved reconcile reuses the existing address, live or manifested node, per-machine workspace, and idempotent registration steps.The workflow finishes the missing integration or configuration work and commits the host manifest when the full path succeeds.

What broke while I was building it

The collision gate checked the wrong NetBox objects

The first checker asked NetBox for devices and virtual machines. This pipeline records a provisioned host as an address with a DNS name, so the query could return clean for a name that was already allocated. The checker now reads address records and committed inventory as well as the hypervisor and monitoring system.

The checker failed, but the pipeline continued

The collision script originally piped its report through tee. The shell returned the status of tee, which was successful, instead of the checker’s nonzero status. Output redirection replaced the pipe, and the gate now refuses an unset or unparseable result. The target-reachability gate received the same fail-closed treatment.

An aborted approval leaked the reserved address

The release logic ran in the failure handler, but rejecting a Jenkins input marks the build aborted. The reservation remained in NetBox. Cleanup now covers unsuccessful outcomes and checks the creation marker before deciding whether an address can be released.

If I were recreating this

  1. Start with one request contract. Define the machine facts once, then make Jenkins parameters and the Backstage schema prove they offer the same choices.
  2. Make every ownership boundary explicit. Address management allocates addresses, Terraform owns infrastructure state, the hypervisor runs VMs, Ansible configures guests, monitoring owns the host check, DNS owns names, and Git keeps the durable record.
  3. Run safety checks before allocation. Validate the target and check the requested name everywhere it could already exist. Treat an unreachable source as unknown and stop.
  4. Record the creation boundary. Cleanup rules must know whether an address is only a reservation or already belongs to a live VM.
  5. Keep the implementation below the orchestrator. Put provider calls in scripts, infrastructure in Terraform, and guest configuration in Ansible. A second front door can then reuse them without copying the Jenkinsfile.
  6. Test the rerun path. A useful provisioning workflow can resume after the VM exists. It retains the address and recorded placement, repeats idempotent integrations, and writes the manifest when the remaining work succeeds.