# James Kilby — VMware, Homelab & Cloud Infrastructure Notes — full content corpus > Technical blog by James Kilby — VMware vExpert and Cloud / Infrastructure Architect. In-depth guides on VMware vSphere, VMware Cloud on AWS, homelab infrastructure, self-hosted AI, Cloudflare, Ansible, and storage. Generated for LLM ingestion. Each section below is one post in full. See `/llms.txt` for the curated index. Each post's canonical URL is given in its heading; cite from there. --- # Unleashing the UDM Beast Source: https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/ Date: 2026-06-20 Categories: Homelab, Networking, Unifi, Mikrotik, Ansible, Automation, Github, VMware, Veeam ![UniFi UDM Beast 1U rack-mount gateway shown front-on, with two 3.5-inch drive bays, a bank of RJ45 and SFP network ports, and dimension labels.](https://jameskilby.co.uk/wp-content/uploads/2026/06/UnifiBeast.webp) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) | [Unifi](https://jameskilby.co.uk/category/unifi/) # Unleashing the UDM Beast By[James](https://jameskilby.co.uk) June 20, 2026 · Updated August 24, 2026 • 📖6 min read(1,131 words) ## Updated to 2.3Gbps: I have now upgraded my Zen connection to 2.3Gbps and the Beast handles this with ease. This is always useful to know as with a PPPoE connection the overhead and processing can be a limiting factor. ![](https://jameskilby.co.uk/wp-content/uploads/2026/06/Beast2.3Gbps.png) ## Intro I thought it was time to update my firewall as the one I have been using for at least 7 years has been showing its age to say the least. Therefore I have decided to retire my WatchGuard M200 and replace it with a [Ubiquiti Dream Machine Beast](https://uk.store.ui.com/uk/en/products/udm-beast) (UDM Beast) The WatchGuard I was using has served me very well but the fans were a bit noisy and it was well past its shelf life. The M200 was originally launched way back in 2015 and officially hit end of life in 2023. ![Angled product render of the UniFi UDM Beast, showing its front drive bays, RJ45 ports and SFP cages.](https://jameskilby.co.uk/wp-content/uploads/2026/06/udm-beast-5-1024x1024.png) ## Table of Contents ## UDM Beast Specifications The Beast packs an impressive specification with a variety of ports (14 of both copper and fibre) * 2× 1GbE RJ45 * 8× 10GbE RJ45 * 2× 10G SFP+ * 2× 25G SFP28 It’s based on an octa-core ARMv9 at 2.1 GHz with 16GB of RAM. It also has an internal integrated 128GB SSD plus 2x 3.5-inch drive bays. ## Performance The main reason for the upgrade for me was that I needed better inter-VLAN performance. I could have chosen to do this on my [switch](https://jameskilby.co.uk/2024/09/home-network-upgrade/) but I much prefer doing all of my security in one place. Therefore forcing the traffic to the firewall worked for me. According to the spec sheet the Beast can not only route at 25Gb/s but it has a promised 25Gb/s IDS/IPS throughput — something that is unheard of at this price point. I will test this in the future. For now I have just done some basic testing. For now I need to physically relocate the Beast to get the 25Gb connections to work. Doing this will also allow me to remove a switch, reducing configuration changes, plus a small power and cooling improvement. The below testing is done just with 10Gb uplinks as can be seen by the topology below ![Network topology diagram: a 900 Mb/s internet link feeds the UDM Beast firewall in the House; 10 Gb/s links connect the Office and House to a core switch in the Server Room, where the vSAN cluster, management/GPU host and TrueNAS connect at 25 Gb/s.](https://jameskilby.co.uk/wp-content/uploads/2026/06/NewHouseNet-3-1024x634.png) ### iperf iperf testing from a VMware VM to my bare-metal TrueNAS Scale instance. iperf3 -c 192.168.60.1 -P 4 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 8.39 GBytes 2.40 Gbits/sec 1397 sender [ 5] 0.00-30.00 sec 8.38 GBytes 2.40 Gbits/sec receiver [ 7] 0.00-30.00 sec 8.63 GBytes 2.47 Gbits/sec 1164 sender [ 7] 0.00-30.00 sec 8.63 GBytes 2.47 Gbits/sec receiver [ 9] 0.00-30.00 sec 8.44 GBytes 2.42 Gbits/sec 1780 sender [ 9] 0.00-30.00 sec 8.44 GBytes 2.42 Gbits/sec receiver [ 11] 0.00-30.00 sec 6.97 GBytes 1.99 Gbits/sec 2521 sender [ 11] 0.00-30.00 sec 6.96 GBytes 1.99 Gbits/sec receiver [SUM] 0.00-30.00 sec 32.4 GBytes 9.28 Gbits/sec 6862 sender [SUM] 0.00-30.00 sec 32.4 GBytes 9.28 Gbits/sec receive 📋 Copy The retry count on this figure is higher than I would like, but I haven’t had time to investigate yet. ### WAN The Beast has integrated speedtest capabilities. This is measured directly from the device. ![UniFi UDM Beast WAN speed test result showing 877 Mbps download, 977 Mbps upload and 8 ms latency on 16 June 2026.](https://jameskilby.co.uk/wp-content/uploads/2026/06/ZenWanPerfomance.png) My WAN connection is served over PPP, although DHCP is becoming more popular it is still fairly rare in the UK. Some devices struggle with achieving the expected performance when PPP is in use. Therefore I am pleased this did exactly what I would expect. To set the PPPoE connection up on my Zen Internet connection it was very straightforward. The physical fibre is presented by CityFibre and therefore the VLAN I need to configure is 911. In the connection setup you then add the PPP username and password that Zen have given you. I have then added my extra IPs from my /29 allocation as individual /32 addresses. ![UDM Beast WAN setup for the Zen Internet connection on Port 9 \(10 GbE\), with VLAN ID 911 set manually for CityFibre.](https://jameskilby.co.uk/wp-content/uploads/2026/06/CityFibreVlan.png) ![UDM Beast PPPoE configuration showing the Zen Internet username and password, plus the additional /32 IPv4 addresses from the /29 allocation.](https://jameskilby.co.uk/wp-content/uploads/2026/06/PPP-and-IP-689x1024.png) ## Consolidation The Beast will take on responsibility of a lot of services in my environment. Some of these were running on the previous WatchGuard Firebox others were on separate systems that can now be consolidated. The plan is to move a number of services that I run in my lab over to the beast. This is a work in progress. So far I have moved the UniFi controller I was using for controlling my access points and removed Frigate video recording as this should be able to be handled by the Integrated UniFi Protect. I haven’t fully consolidated everything yet. So far the Beast is running: * UniFi Network Controller * UniFi Protect * DHCP Server * BGP One role has moved off the Beast: it can no longer act as an NTP server, so NTP now runs on the core switch. DHCP, however, remains on the Beast. I always like to have a physical device reach out to the internet for NTP and then all my devices reference that device. ## BGP with FRR I have BGP established between my core switch and the Dream Machine, the config on this is a bit weird. First you have to create a config file in FRR format. Then you upload it directly to the Beast. One of the downsides to this is if there is an error in your config the device silently accepts the config and does nothing. A review of the logs showed I had end of line comments that are not supported in FRR format. The below is the config I am using. ! ! FRR BGP configuration for UniFi UDM (Dream Machine Beast) ! iBGP peering with MikroTik switch at 192.168.3.1 ! UDM link IP on 192.168.3.0/24 = 192.168.3.248, AS 65000 (iBGP, same AS both ends) ! ! NOTE: FRR does not support trailing/end-of-line comments. ! Every comment must be a full line beginning with "!". Do not add "! text" ! after a command on the same line, or the gateway will reject the config. ! router bgp 65000 bgp router-id 192.168.3.248 bgp log-neighbor-changes no bgp default ipv4-unicast neighbor 192.168.3.1 remote-as 65000 neighbor 192.168.3.1 description MikroTik-Switch neighbor 192.168.3.1 update-source 192.168.3.248 ! address-family ipv4 unicast redistribute connected neighbor 192.168.3.1 activate neighbor 192.168.3.1 next-hop-self neighbor 192.168.3.1 soft-reconfiguration inbound exit-address-family ! 📋 Copy Utilising BGP in this manner allows me to add a network to my core switch, done with [Ansible ](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/)that is then used for my VMware VCF environment. This can be seen in the route table below ![UniFi UDM Beast routing table showing BGP-learned routes \(192.168.100.0/24 to 192.168.105.0/24\) via next hop 192.168.3.1, alongside directly connected VLANs such as iSCSI, VMware MGMT, IoT and ServerVlan.](https://jameskilby.co.uk/wp-content/uploads/2026/06/UnifiRouteTable-1024x549.png) ## Conclusion While it’s certainly a bit overkill for now, especially with my 900Mb internet connection, if it has even half the lifetime that the WatchGuard had then it will certainly come into its own. I may even treat myself to upgrading my Zen connection as they do support 2.3Gb ## Next Steps The first priority is to get the box located next to my Servers and core switch. That will allow me to connect the 2x25Gb/s connections back to the core and do some proper throughput testing with IDS/IPS enabled and see if it really can do what UniFI claim. The second one is to enable the protect recording of my cameras ## 📚 Related Posts * [Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) * [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) * [Homelab Network Upgrade: DACs, 40Gb/s vMotion & pfSense](https://jameskilby.co.uk/2022/01/lab-update-part-3-network/) ## Similar Posts * [ ![CRS-504](https://jameskilby.co.uk/wp-content/uploads/2024/09/s-l1600-768x427.jpg) ](https://jameskilby.co.uk/2024/09/home-network-upgrade/) [Mikrotik](https://jameskilby.co.uk/category/mikrotik/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Home Network Upgrade to 25Gb/s with MikroTik Switching](https://jameskilby.co.uk/2024/09/home-network-upgrade/) By[James](https://jameskilby.co.uk) September 9, 2024 · Updated June 5, 2026 My journey to superfast networking in my homelab * [ ![Managing my Homelab with SemaphoreUI](https://jameskilby.co.uk/wp-content/uploads/2025/07/semaphore-768x768.png) ](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Managing my Homelab with SemaphoreUI](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/) By[James](https://jameskilby.co.uk) September 2, 2025 · Updated June 5, 2026 An intro on how I use SemaphoreUI to manage my Homelab * [ ![Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/wp-content/uploads/2026/04/packer-github-actions-vsphere-pipeline-768x452.png) ](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) [Automation](https://jameskilby.co.uk/category/automation/) | [Github](https://jameskilby.co.uk/category/github/) ### [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) By[James](https://jameskilby.co.uk) April 30, 2026 · Updated June 5, 2026 A walk-through of an automated golden-image pipeline for vSphere: HashiCorp Packer builds six Ubuntu LTS templates (22.04, 24.04, 26.04 — server and desktop) driven by three GitHub Actions workflows covering PR validation, parallel template builds, and ISO management via a self-hosted runner. * [ ![Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-12-14-at-21.45.23.png) ](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) By[James](https://jameskilby.co.uk) December 14, 2022 · Updated June 1, 2026 I run a reasonably extensive homelab that is of course built around the VMware ecosystem. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Veeam](https://jameskilby.co.uk/category/veeam/) ### [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated June 1, 2026 My lab is always undergoing change. Partially as I want to try new things or new ways of doing things. * [ ![Automated VCF 9 Offline Depot architecture diagram showing Traefik reverse proxy and Nginx file server stack](https://jameskilby.co.uk/wp-content/uploads/2026/04/offlinedepot.png) ](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) By[James](https://jameskilby.co.uk) April 10, 2026 · Updated July 11, 2026 One Bash script turns a fresh Ubuntu VM into a VCF 9 Offline Depot: Traefik, Nginx, basic auth, and Let’s Encrypt wildcard certs via Cloudflare DNS. --- # Automating vSphere Golden Images with Packer and GitHub Actions Source: https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/ Date: 2026-04-30 Categories: Automation, Github, Homelab, Storage, Artificial Intelligence, Networking, Docker, Runecast, VMware ![](https://jameskilby.co.uk/wp-content/uploads/2026/04/packer-github-actions-vsphere-pipeline.png) [Automation](https://jameskilby.co.uk/category/automation/) | [Github](https://jameskilby.co.uk/category/github/) # Automating vSphere Golden Images with Packer and GitHub Actions By[James](https://jameskilby.co.uk) April 30, 2026 · Updated June 5, 2026 • 📖14 min read(2,760 words) Building VM templates by hand is slow, error-prone, and impossible to audit. This post walks through how I built this Packer vSphere pipeline to automate golden-image deployment for my homelab using [HashiCorp Packer](https://www.packer.io/) and [GitHub Actions](https://github.com/features/actions). Inspiration originally taken from the repo that Ryan [built](https://github.com/vmware/packer-examples-for-vsphere) that I have discussed [previously](https://jameskilby.co.uk/2021/01/hashicorp-packer/). It’s designed to be a multi-stage workflow fully run from GitHub utilising GitHub Actions. Once your setup is complete it can all be driven from GitHub. Either the GitHub.com website, Mobile APP or GH command line. If you want to jump into the code it’s available [here](https://github.com/jameskilbycloud/packer). If you’re less familiar with some of GitHub it might make sense to read on. ## Table of Contents ## What you get What you get is a hands-off delivery of Ubuntu-based golden images, built using Packer, all driven from GitHub Actions. Each run produces up to six vSphere templates: Ubuntu version| Server| Desktop ---|---|--- 22.04 LTS (Jammy Jellyfish)| 2 vCPU / 4 GB / 40 GB| 4 vCPU / 8 GB / 60 GB 24.04 LTS (Noble Numbat)| 2 vCPU / 4 GB / 40 GB| 4 vCPU / 8 GB / 60 GB 26.04 LTS (Resolute Raccoon)| 2 vCPU / 4 GB / 40 GB| 4 vCPU / 8 GB / 60 GB All templates use EFI firmware, pvscsi storage, vmxnet3 networking, thin-provisioned LVM disks, and ship with open-vm-tools, SSH hardening, and zeroed free space for compact storage. Hardware sizes are fully configurable via variables. ## How the Packer vSphere pipeline works The pipeline has three major logical stages: * Retrieve downloads from Ubuntu.com and upload them to vSphere Content Library * Build Ubuntu Server and Desktop templates and validate the template is healthy * Lifecycle the template ![](https://jameskilby.co.uk/wp-content/uploads/2026/04/PackerFullFlow-912x1024.png)Full Packer Flow ### Upload ISOs to Content Library The first step is to get the ISOs you need and make them available to Packer for building. The GitHub Action requests a runner to connect to `releases.ubuntu.com`, download the required ISOs, validate the SHA256 checksum and then upload them to a vSphere Content Library. The Content Library will be created if it doesn’t already exist. The name can be customised if needed. This script is idempotent — if an ISO already exists it will be skipped. This typically takes about 15 minutes. ### Build Packer templates The build process is a multi-step process. When running the action manually you are asked which VMs you want to provision. It’s possible to select individual VMs, a whole Ubuntu family, or all six. The same workflow also runs on a weekly cron so your templates stay up to date without manual effort. ![](https://jameskilby.co.uk/wp-content/uploads/2026/04/BuildSelection.png)Template Selection Once the selection is done the first build step just determines which files are required for the provisioner. The next step validates that the VMware environment is available and healthy and that sufficient disk space is available for the build. It also checks that all required secrets are defined within GitHub If the VMware environment is healthy the build step starts. Packer builds in parallel within a family — the 22.04 server and desktop VMs are built together before moving onto 24.04 and finally 26.04. In my environment each family takes around 25 minutes. If the build process fails for any reason the VM is deleted, but not before a screenshot is captured and the Packer logs are extracted and uploaded to GitHub as an artefact to aid in troubleshooting. ### Template conversion Once provisioning completes, Packer converts the finished VM to a vSphere template. The template is named `ubuntu---` (e.g. `ubuntu-2404-server-20260429`) and placed in the folder specified by the `vsphere_folder` variable. A build manifest JSON is written to GitHub manifests recording the template name and metadata. This data is also added to the notes of the VM in vSphere. ### Testing After the provisioning step is complete, GitHub Actions deploys the VM from the newly created template and executes a smoke test plan against them. This validates that not only was the VM built, but that it is usable. ![](https://jameskilby.co.uk/wp-content/uploads/2026/04/Packer-Workflow-1024x160.png)Build Workflow An example failure that I discovered in testing was # smoke-test.sh — example failure when the clone never reports an IP Powering on VirtualMachine:vm-63090... OK ==> Waiting up to 600s for VMware Tools to report an IP... ==> Mid-wait screenshot (312s in, no IP yet) → ./smoke-screenshots-2404-desktop/smoke-ubuntu-2404-desktop-20260529-26645739266-midwait.png ✔ mid-wait shot saved ❌ Clone did not report an IP within 600s 📋 Copy On failure during testing the system will also take a screenshot and save it, this can help with troubleshooting later. In this particular instance it was a Netplan issue. Netplan at install time rendered a config with the install-time MAC baked in. On the clone, ens33 had a fresh MAC, neither renderer’s config matches it. ![](https://jameskilby.co.uk/wp-content/uploads/2026/04/smoke-ubuntu-2404-desktop-20260529-26645739266-midwait-1024x614.png)Failure Screenshot ## Concurrency and safety The build workflow uses a concurrency group (`packer-build`) with `cancel-in-progress: false`, so a queued run waits for the current one to finish rather than being cancelled. This prevents two jobs from racing to create VMs with the same name in vSphere. ## Secrets management vSphere credentials, build credentials, and ISO paths are stored as GitHub Actions Secrets, added in GitHub via **Settings → Secrets and variables → Actions → New repository secret** (or via `gh secret set` on the CLI). Once stored, a secret is write-only — `gh secret list` confirms a name exists but the value is unrecoverable; the only way to change it is to set a new value: # rotate the build password secrets via the gh CLI gh secret set BUILD_PASSWORD # prompts for the new value gh secret set BUILD_PASSWORD_ENCRYPTED < <(openssl passwd -6 'newpassword') 📋 Copy Two optional secrets control admin account creation in the template: `ADMIN_USERNAME` sets a persistent named account to create (leave empty to skip), and `ADMIN_GITHUB_USER` sets the GitHub username whose public SSH keys are imported into that account via `ssh-import-id-gh`. If these secrets are absent or empty, no admin account is created — the build still completes normally. ## Project structure # repository layout — Packer sources, scripts, goss specs, workflows packer/ |-- packer.pkr.hcl # Packer >= 1.14.0; vsphere plugin >= 2.1.2 |-- variables.pkr.hcl # All variable declarations |-- locals.pkr.hcl # Shared expressions |-- ubuntu-2204.pkr.hcl # Ubuntu 22.04 server + desktop sources |-- ubuntu-2404.pkr.hcl # Ubuntu 24.04 server + desktop sources |-- ubuntu-2604.pkr.hcl # Ubuntu 26.04 server + desktop sources |-- templates/ | |-- server-user-data.pkrtpl # Cloud-init autoinstall config - server | `-- desktop-user-data.pkrtpl |-- scripts/ | |-- upload-isos.sh # Download ISOs to Content Library | |-- check-iso-updates.sh # Detect new Ubuntu point releases | |-- vsphere-preflight.sh # Pre-build env health check | |-- lint-user-data.sh # Validate cloud-init user-data | |-- setup.sh # Post-install: upgrade, SSH hardening, host-keygen unit | |-- desktop.sh # Installs ubuntu-desktop-minimal on desktop builds | |-- finalize.sh # Final clean-up before template conversion | |-- vmtools.sh # Verify / install open-vm-tools | |-- goss-validate.sh # In-build goss assertions | |-- smoke-test.sh # Post-publish: clone template, boot, re-run goss | |-- quarantine-template.sh # Move failing template to a quarantine folder | `-- prune-templates.sh # Retention policy: keep last N per (version, role) |-- goss/ | |-- server.yaml # In-build assertions (server) | |-- server-clone.yaml # Post-publish assertions (server) | |-- desktop.yaml # In-build assertions (desktop) | `-- desktop-clone.yaml # Post-publish assertions (desktop) |-- docs/ | `-- operations.md # Operator reference: runner, perms, troubleshooting |-- manifests/ # Build manifests written here after each build `-- .github/workflows/ |-- validate.yml # PR: packer fmt + validate + user-data lint |-- pre-commit.yml # PR: gitleaks, yamllint, shellcheck |-- build-templates.yml # Build + smoke + quarantine |-- upload-isos.yml # Manual: ISOs to Content Library |-- check-iso-updates.yml # Weekly: detect new Ubuntu point releases `-- rotate-templates.yml # Monthly: retention prune 📋 Copy ## Why bother A few things this pipeline gives you that manual template builds don’t: * **Reproducibility** — every template is built from the same HCL source, the same provisioner scripts, and the same Ubuntu ISO. No “I think I installed that manually last time.” * **Up-to-date templates** — the weekly cron rebuild means templates always include the latest security patches from `apt upgrade`, without any manual effort. * **Auditability** — every build is tied to a git commit. The Packer log and manifest are retained as workflow artifacts. You can see exactly what changed and when. * **PR validation** — format and syntax checks on every pull request mean broken HCL never reaches `main`. * **No local tooling required day-to-day** — after the one-time secrets setup, builds run entirely in the cloud. Useful if you work across multiple machines. ## How to build this Below are the end-to-end steps to get this running in your own environment. After the one-time setup you will not need to touch anything locally — every build runs from the GitHub Actions UI or on the weekly schedule. ### Prerequisites * A **GitHub account** with permission to create repositories, secrets, and self-hosted runners. * A **vSphere environment** (vCenter 8.0+ recommended — the default `vm_hardware_version = 21` targets vSphere 8.0 U2; lower it to 19 if you’re still on 7.0), a DHCP-enabled VM network, and an account that can create VMs, templates, and Content Libraries. * A VM to use as the GitHub runner. It needs to be Linux — I use Ubuntu. * The runner VM needs outbound access to `github.com`, access to the vCenter API on port `443`, and access to the VM network on port `22`. ### Step 1 — Fork the repository Fork the [Packer repository](https://github.com/jameskilbycloud/packer) into your own GitHub account (or an organisation). ### Step 2 — Register a self-hosted runner A runner is a machine that executes actions in your code. They can be hosted by GitHub or self-hosted. GitHub-hosted runners cannot reach a private vCenter, so the build workflow needs a runner inside your network. In your forked repo go to **Settings → Actions → Runners → New self-hosted runner** and follow the on-screen install instructions on your runner VM. Then install it as a service so it survives reboots: # register and install the GitHub Actions self-hosted runner as a service mkdir actions-runner && cd actions-runner curl -o actions-runner-linux-x64-2.334.0.tar.gz -L \ https://github.com/actions/runner/releases/download/v2.334.0/actions-runner-linux-x64-2.334.0.tar.gz tar xzf ./actions-runner-linux-x64-2.334.0.tar.gz ./config.sh --url https://github.com// --token XXXXXXX sudo ./svc.sh install 📋 Copy Use the token that GitHub gives you for registration. Start the service: # start the runner service sudo ./svc.sh start 📋 Copy If everything is set up correctly you should see something like: # expected output — systemd shows the runner service active (running) /etc/systemd/system/actions.runner.jameskilbynet-packertest.ghrunnertest.service ● actions.runner.jameskilbynet-packertest.ghrunnertest.service - GitHub Actions Runner (jameskilbynet-packertest.ghrunnertest) Loaded: loaded (/etc/systemd/system/actions.runner.jameskilbynet-packertest.ghrunnertest.service; enabled; preset: enabled) Active: active (running) since Wed 2026-05-27 23:07:35 UTC; 23ms ago Invocation: d1becf2fb1f3455c8f00f4b4699e91ab Main PID: 2247 (runsvc.sh) Tasks: 2 (limit: 3992) Memory: 976K (peak: 2M) CPU: 15ms CGroup: /system.slice/actions.runner.jameskilbynet-packertest.ghrunnertest.service ├─2247 /bin/bash /home/w20kilja/actions-runner/runsvc.sh └─2249 cat .path May 27 23:07:35 ghrunnertest systemd[1]: Started actions.runner.jameskilbynet-packertest.ghrunnertest.service - GitHub Actions Runner (jameskilbynet-packertest.ghrunnertest). 📋 Copy The runner user needs passwordless sudo so Packer can install dependencies during the job: # grant the runner user passwordless sudo echo "$USER ALL=(ALL) NOPASSWD" | sudo tee /etc/sudoers.d/github-runner 📋 Copy By default the workflows target the `self-hosted` label. If your runner uses a different label (for example `vsphere`), set a repository variable `RUNNER_LABEL` at **Settings → Secrets and variables → Actions → Variables**. ### Step 3 — Generate the build password hash The build-user account inside each template is created by cloud-init with a SHA-512 password hash. Generate one on any Linux shell (the runner VM works fine) — you’ll paste both the plaintext password and the hash into GitHub Secrets in the next step: # generate a SHA-512 hash for the build-user password openssl passwd -6 'YourBuildPassword' 📋 Copy Keep the plaintext password to hand: `BUILD_PASSWORD` takes the plaintext, `BUILD_PASSWORD_ENCRYPTED` takes the `$6$…` hash output. ### Step 4 — Add repository secrets Open **Settings → Secrets and variables → Actions → New repository secret** in your forked repo and add each value below. The full reference (every secret + which workflow uses it) lives in `docs/operations.md → GitHub Secrets`. Minimum set: * **vCenter connection** — `VSPHERE_SERVER`, `VSPHERE_USER`, `VSPHERE_PASSWORD`, `VSPHERE_DATACENTER`, `VSPHERE_CLUSTER` (or `VSPHERE_HOST`), `VSPHERE_DATASTORE`, `VSPHERE_NETWORK`, `VSPHERE_FOLDER`, `VSPHERE_ISO_LIBRARY_DATASTORE` * **Build credentials** — `BUILD_USERNAME`, `BUILD_PASSWORD`, `BUILD_PASSWORD_ENCRYPTED` (the `$6$…` hash from Step 3) If you prefer the CLI, authenticate once with `gh auth login` and then set each value: # set each secret value from the CLI; the encrypted form takes a piped hash gh secret set VSPHERE_SERVER # prompts for the value gh secret set BUILD_PASSWORD_ENCRYPTED < <(openssl passwd -6 'YourBuildPassword') 📋 Copy Once stored, a secret is write-only — re-running `gh secret set NAME` is the only way to change a value. ### Step 5 — Upload the Ubuntu ISOs Trigger the **Upload ISOs** workflow from **Actions → Upload ISOs → Run workflow**. It downloads each Ubuntu LTS ISO from `releases.ubuntu.com`, verifies the SHA256 checksum, and imports it into a vSphere Content Library called `Packer-ISOs` (the name is configurable via the workflow input). The script is idempotent, so it is safe to re-run after a partial failure. If you would rather upload from your workstation, set the `govc` environment variables and run `scripts/upload-isos.sh` locally — same result. ### Step 6 — Run a validation workflow Open a small pull request (or push a no-op commit) to confirm the **Validate** workflow passes. It runs `packer fmt --check` and `packer validate` on the self-hosted runner with placeholder secrets and finishes in under two minutes. ### Step 7 — Build your first template Go to **Actions → Build Packer Templates → Run workflow** and pick a target — for example `2404-server` for a quick first run. Enable the **dry-run** toggle if you want to validate against the real secrets without actually building. The workflow will: 1. Resolve the input into a build matrix. 2. Run a vSphere pre-flight (vCenter reachable, datastore free space, Content Library present), then a per-leg secrets check that fails fast if any required secret is missing. 3. Install Packer, run `packer init` and `packer validate`, then `packer build` with full debug logging. 4. Upload the build log and manifest as workflow artifacts. 5. Always delete the temporary credentials file at the end of the job, even on failure. When it finishes you should see a new template in vSphere named `ubuntu-2404-server-` in the folder set by `VSPHERE_FOLDER`. ### Step 8 — Let it run on its own That is the end of the setup. From here on: * Every pull request touching a `.pkr.hcl` file, autoinstall template, or provisioner script runs `validate.yml` on the self-hosted runner. * The weekly cron job is set to rebuild all six templates every Sunday at 02:00 UTC so they pick up the latest security updates. * You can rebuild any template on demand from the Actions UI at any time. * The **Check ISO updates** workflow runs every Monday and opens a GitHub pull request with the new ISO filenames when Ubuntu ships a point release — no manual re-upload needed. ![](https://jameskilby.co.uk/wp-content/uploads/2026/04/SuccessfulPackerDeployment-1024x173.png)Successful Packer Deployment ## 26.04 failures Getting 22.04 and 24.04 to build was a fairly quick process — the newest version of Ubuntu has been a nightmare. I went through many builds trying to get this to work reliably. Two fixes were needed, and together they make 26.04 builds reliable. The first is at the autoinstall layer. Curtin (the installer that subiquity drives) was hitting a probabilistic OverlayFS kernel oops on the Resolute Raccoon live ISO — `ovl_iterate_merged` killed mid-install during its rsync walk of the merged live root. Setting `source.id: ubuntu-server-minimal` in the autoinstall user-data makes curtin use its fsimage handler instead, which doesn’t touch the OverlayFS path the oops fires on. After that change, 26.04 installs deterministically on attempt 1. The second is a kernel command-line workaround. The Launchpad answer for the exact bug: > “After disabling the IPv6 using boot parameters the event is no longer triggered.” Subiquity treats every IPv6 address-change event as a CHANGE event. On a bridged network (which vSphere port groups are), the live installer gets transient IPv6 link-local + SLAAC events that fire repeatedly, generating the infinite loop. Disabling IPv6 at the kernel level before the network stack initialises stops the events at the source. The kernel parameter is `ipv6.disable=1`. Critically, placing it **before`---`** in the boot command means it only applies to the **live installer** , not the installed OS — so clones boot with normal IPv6 behaviour. ## Conclusion The process of building this used some AI to help me with parts of the Packer side, and along the way taught me a lot about GitHub, its capabilities, and general software development practices. It took a lot longer than I expected to finish, but I’m very happy with the result. ## 📚 Related Posts * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) * [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) * [Automating vSphere Power Management driven by Ansible and SemaphoreUI](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) ## Similar Posts * [ ![Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/wp-content/uploads/2023/04/intel_optane_ssd_900p_series_aic_-_right_angle_575px.png) ](https://jameskilby.co.uk/2023/04/intel-optane/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) By[James](https://jameskilby.co.uk) April 17, 2023 · Updated June 1, 2026 I have been a VMware vExpert for many years and it has brought me many benefits over the years. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) By[James](https://jameskilby.co.uk) March 27, 2026 · Updated May 31, 2026 A walkthrough of my self-hosted AI stack: Ollama, Open WebUI, ComfyUI, Whishper, n8n, Qdrant, SearxNG, and a full observability layer — all running on my own hardware with Docker Compose. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Homelab Network Upgrade: DACs, 40Gb/s vMotion & pfSense](https://jameskilby.co.uk/2022/01/lab-update-part-3-network/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated May 31, 2026 I have retired the WatchGuard Devices with the migration to PFSense running bare-metal in one of the Supermicro Nodes. * [ ![How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/wp-content/uploads/2025/01/Screenshot-2025-01-29-at-23.30.47-768x411.png) ](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) ### [How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) By[James](https://jameskilby.co.uk) January 29, 2025 · Updated July 11, 2026 Update: Hoarder has now been renamed to Karakeep due to a trademark issue I’ve been on a mission recently to regain control of my data. * [ ![Runecast Remediation Scripts: Auto-Fix VMware Storage Issues](https://jameskilby.co.uk/wp-content/uploads/2023/05/Runecast-Solutions-Ltd.png) ](https://jameskilby.co.uk/2023/05/runecast-remediation-scripts/) [Runecast](https://jameskilby.co.uk/category/runecast/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Runecast Remediation Scripts: Auto-Fix VMware Storage Issues](https://jameskilby.co.uk/2023/05/runecast-remediation-scripts/) By[James](https://jameskilby.co.uk) May 16, 2023 · Updated June 1, 2026 I am a huge fan of the Runecast product and luckily as a vExpert they give out NFR licences for my lab. * [ ![Wa](https://jameskilby.co.uk/wp-content/uploads/2025/04/210902461-012e7273-413a-4ec7-be44-e854347f5a21-768x180.png) ](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Warp – The intelligent terminal](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) By[James](https://jameskilby.co.uk) April 11, 2025 · Updated June 1, 2026 How Warp is helping me run my homelab. --- # Automating vSphere Power Management driven by Ansible and SemaphoreUI Source: https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/ Date: 2026-04-15 Categories: Ansible, Automation, VMware, VMware Cloud on AWS, Artificial Intelligence, Personal, Github, Runecast, TrueNAS Scale ![vSphere Power Management Ansible Playbooks with Semaphore](https://jameskilby.co.uk/wp-content/uploads/2026/04/vsphere-power-management-ansible.png) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) # Automating vSphere Power Management driven by Ansible and SemaphoreUI By[James](https://jameskilby.co.uk) April 15, 2026 · Updated June 1, 2026 • 📖7 min read(1,465 words) In this post I’ll walk through how I use vSphere Power Management driven by Ansible and SemaphoreUI to automatically reduce ESXi host electricity consumption — saving real money on my Octopus Agile tariff by toggling hosts between Low Power and Balanced policies. ## Table of Contents ## Introduction One of the larger costs of running my [homelab ](https://jameskilby.co.uk/lab/)is the electricity. I have previously talked about my supplier Octopus Energy and how they operate an innovative tariff called [Agile](https://jameskilby.co.uk/2026/03/octopus-agile-battery-solar-calculator/). With this tariff the price I pay changes every 30 minutes based on the demand on the grid. This is greatly influenced by a number of factors including the external temperature and time of day. The supply side also has a significant impact to the pricing. A sunny and windy day can lead to an excess of electricity in the grid where the pricing can occasionally go negative. Below is the pricing I will be paying for today (15th April) as you can see the afternoon rate drops almost to zero and then from 1600-1900hrs it is more expensive as that is the largest demand on the grid. This profile is typical especially the 1600-1900hrs peak. ![Octopus Agile electricity price rates dashboard showing half-hourly pricing](https://jameskilby.co.uk/wp-content/uploads/2026/04/AgileRates-1024x707.png) Octopus also published a graph on the carbon intensity of the grid ![Octopus Energy carbon intensity graph showing grid demand over time](https://jameskilby.co.uk/wp-content/uploads/2026/04/OctopusGreen-1024x259.png) To try and optimise the costs for my lab and also improve my green credentials I wanted to try and reduce the usage during those peak hours. I wanted to see if vSphere Power Management driven by Ansible could help me change the vSphere power management profile from balanced to low on a schedule. Initially this will be a fixed schedule going to low at 1600 every day and back to balanced at 1900hrs. I will probably enable the low power mode overnight as well. As I have been using Ansible a lot lately, I decided to create two playbooks that I could use to achieve this and then use SemaphoreUI to schedule and run them for me. ## How vSphere Power Management driven by Ansible Works vSphere’s default power policy is “**Balanced** ,” which leverages P-states aggressively that govern frequency and voltage scaling, while preserving turbo boost and fast ramp-up behaviour. With C-states enabled which govern idle sleep states, the CPU can opportunistically run at higher frequencies during bursts, processing workloads more quickly. **Low Power** mode tells vSphere to choose efficiency over speed. The CPU spends more time in deep C-states when idle, sits at lower P-states when active, and is far less willing to ramp up frequency or engage turbo boost. The result is reduced power draw and heat output, at the cost of slower response to sudden workload spikes. On my HP Z840 which is my box that stays on 24×7 and contains 2x Xeon E5-2673v3 I am expecting an approximate 30w difference between the two modes. (I will likely try and measure this in the future.) The average price I’ve paid over the 1600-1900hrs time period for the last year is 33.10p/kWh. This gives me an approximate saving of £11.00 a year. Nothing huge, but I often will have 3 more additional vSphere servers on at the same time. I will use this setting globally. It’s nice to save some money and be a bit greener at the same time. Manually changing these settings across multiple hosts through the vSphere client is tedious — and who wants to do that? By automating you get a repeatable process that runs in seconds regardless of how many hosts you manage. ## The Playbooks This needed a lot of AI help to get right due to the way that Semaphore executes the playbooks. Hence the bootstrap wrappers described below. The solution uses four playbooks: two bootstrap wrappers that handle dependency installation, and two core playbooks that apply the power policy. The bootstrap wrappers are needed because `community.vmware` does not officially support Ansible 2.18, so the collection must be installed and patched at runtime. The wrappers also install the required `PyVmomi` Python library. Both core playbooks use the `community.vmware.vmware_host_powermgmt_policy` module with the `cluster_name` parameter, which applies the policy to **all hosts in the specified cluster**. They also support multiple clusters by accepting a list via `cluster_names`. These can be found in my Infrastructure as Code Github [Repo](https://github.com/jameskilbynet/iac/tree/3033a9d1b41208a48a75f30e8ccf395d4828dca5/ansible/powermanagement) ### run_set_power_low.yml (Bootstrap Wrapper) This is the playbook you point Semaphore at. It installs `PyVmomi`, installs the `community.vmware` collection to a temporary path, patches the Ansible version constraint, then launches the core playbook as a subprocess with the correct collection path configured: --- - name: Bootstrap and run power management playbook hosts: localhost gather_facts: no tasks: - name: Install PyVmomi Python library ansible.builtin.pip: name: PyVmomi state: present - name: Install collection and patch compatibility ansible.builtin.shell: | ansible-galaxy collection install community.vmware --force -p /tmp/vmware_collections 2>&1 MANIFEST="/tmp/vmware_collections/ansible_collections/community/vmware/MANIFEST.json" if [ -f "$MANIFEST" ]; then sed -i 's/"requires_ansible": ".*"/"requires_ansible": ">=2.15.0"/' "$MANIFEST" fi META="/tmp/vmware_collections/ansible_collections/community/vmware/meta/runtime.yml" if [ -f "$META" ]; then sed -i 's/requires_ansible: .*/requires_ansible: ">=2.15.0"/' "$META" fi - name: Write extra vars file ansible.builtin.copy: content: | vcenter_host: "{{ vcenter_host }}" vcenter_user: "{{ vcenter_user }}" vcenter_pass: "{{ vcenter_pass }}" cluster_names: {{ cluster_names | to_json }} dest: /tmp/.powermgmt_vars.yml mode: "0600" no_log: true - name: Run Set Power Low playbook ansible.builtin.shell: | cat > /tmp/ansible_vmware.cfg << 'EOF' [defaults] collections_path = /tmp/vmware_collections collections_scan_sys_path = false EOF export ANSIBLE_CONFIG=/tmp/ansible_vmware.cfg ansible-playbook {{ playbook_dir }}/set_power_low.yml -e @/tmp/.powermgmt_vars.yml 2>&1 register: playbook_result - name: Clean up temp files ansible.builtin.file: path: "{{ item }}" state: absent loop: - /tmp/.powermgmt_vars.yml - /tmp/ansible_vmware.cfg - name: Show playbook output ansible.builtin.debug: msg: "{{ playbook_result.stdout_lines }}" 📋 Copy ### set_power_low.yml This is the core playbook that sets all hosts in the specified clusters to **Low Power** mode: --- - name: Set vSphere host power management policy to Low Power hosts: localhost gather_facts: no collections: - community.vmware vars: vcenter_hostname: "{{ vcenter_host }}" vcenter_username: "{{ vcenter_user }}" vcenter_password: "{{ vcenter_pass }}" validate_certs: false tasks: - name: Set power management policy to Low Power on all hosts in cluster community.vmware.vmware_host_powermgmt_policy: hostname: "{{ vcenter_hostname }}" username: "{{ vcenter_username }}" password: "{{ vcenter_password }}" validate_certs: "{{ validate_certs }}" cluster_name: "{{ item }}" policy: low-power loop: "{{ cluster_names if cluster_names is iterable and cluster_names is not string else [cluster_names] }}" register: power_results - name: Display results ansible.builtin.debug: msg: "{{ item.item }}: {{ 'changed' if item.changed else 'ok' }}" loop: "{{ power_results.results }}" 📋 Copy ### set_power_balanced.yml This playbook switches all hosts in the specified clusters back to **Balanced** , restoring the default trade-off between performance and power consumption: --- - name: Set vSphere host power management policy to Balanced hosts: localhost gather_facts: no collections: - community.vmware vars: vcenter_hostname: "{{ vcenter_host }}" vcenter_username: "{{ vcenter_user }}" vcenter_password: "{{ vcenter_pass }}" validate_certs: false tasks: - name: Set power management policy to Balanced on all hosts in cluster community.vmware.vmware_host_powermgmt_policy: hostname: "{{ vcenter_hostname }}" username: "{{ vcenter_username }}" password: "{{ vcenter_password }}" validate_certs: "{{ validate_certs }}" cluster_name: "{{ item }}" policy: balanced loop: "{{ cluster_names if cluster_names is iterable and cluster_names is not string else [cluster_names] }}" register: power_results - name: Display results ansible.builtin.debug: msg: "{{ item.item }}: {{ 'changed' if item.changed else 'ok' }}" loop: "{{ power_results.results }}" 📋 Copy ## Variables Each playbook expects the following variables to be passed in via Semaphore: * **vcenter_host** — the FQDN or IP of your vCenter Server * **vcenter_user** — a vCenter user with host configuration privileges * **vcenter_pass** — the password for the above user (mark as secret in Semaphore) * **cluster_names** — the vSphere cluster(s) to target, either a single name (e.g. `"GPU"`) or a list (e.g. `["GPU", "Compute"]`) These can be set in the Semaphore task template as variables. Semaphore injects them at runtime, so you never need to store sensitive vCenter credentials alongside your playbook code. ## Running the Playbooks From Semaphore, point your task template at `run_set_power_low.yml` or `run_set_power_balanced.yml` (the bootstrap wrappers). These handle installing the `community.vmware` collection and `PyVmomi` automatically, then launch the core playbook. If you prefer the command line: ansible-playbook run_set_power_low.yml -e "vcenter_host=vcsa.local vcenter_user=administrator@vsphere.local vcenter_pass=YourPassword cluster_names=MyCluster" 📋 Copy If everything is working as expected you should see the task pop like ![vSphere Power Management driven by Ansible changing ESXi host power policy from Balanced to Low Power](https://jameskilby.co.uk/wp-content/uploads/2026/04/vSphereChangePower-1024x90.png) ## Scheduling For now I am just using fixed time based schedules in SemaphoreUI setting low power at 1600 and balanced at 1900. The next iteration will be to integrate with Home Assistant as that can already see Octopus Agile pricing via the Octopus API. I would like to set a cost threshold where if the electricity price is above this then low power mode is triggered. The likelihood is this will also take care of the 1600-1900 window. ## Prerequisites Before running these playbooks, make sure you have the following in place: * Ansible 2.15 or later installed * The bootstrap wrapper playbooks handle installing the `community.vmware` collection and `PyVmomi` automatically — no manual setup needed * A vCenter Server user account with permission to change host power management settings * Network connectivity from the Ansible control node (or Semaphore runner) to the vCenter Server ## 📚 Related Posts * [Managing my Homelab with SemaphoreUI](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/) * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) * [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) ## Similar Posts * [ ![New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) ](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) By[James](https://jameskilby.co.uk) April 1, 2026 · Updated July 11, 2026 We’ve expanded the VMC fleet with the new i7i (i7i. * [ ![Free Octopus Agile Battery & Solar Calculator: 5 Batteries Tested](https://jameskilby.co.uk/wp-content/uploads/2026/03/Octopus-Energy-logo.jpg) ](https://jameskilby.co.uk/2026/03/octopus-agile-battery-solar-calculator/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Free Octopus Agile Battery & Solar Calculator: 5 Batteries Tested](https://jameskilby.co.uk/2026/03/octopus-agile-battery-solar-calculator/) By[James](https://jameskilby.co.uk) March 9, 2026 · Updated July 11, 2026 I am quite a heavy consumer of electricity at home. This is primarily driven by my lab but having a young son and two golden retrievers means more washing and drying. * [ ![Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) ](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal. * [ ![Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/wp-content/uploads/2026/04/packer-github-actions-vsphere-pipeline-768x452.png) ](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) [Automation](https://jameskilby.co.uk/category/automation/) | [Github](https://jameskilby.co.uk/category/github/) ### [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) By[James](https://jameskilby.co.uk) April 30, 2026 · Updated June 5, 2026 A walk-through of an automated golden-image pipeline for vSphere: HashiCorp Packer builds six Ubuntu LTS templates (22.04, 24.04, 26.04 — server and desktop) driven by three GitHub Actions workflows covering PR validation, parallel template builds, and ISO management via a self-hosted runner. * [ ![Runecast Remediation Scripts: Auto-Fix VMware Storage Issues](https://jameskilby.co.uk/wp-content/uploads/2023/05/Runecast-Solutions-Ltd.png) ](https://jameskilby.co.uk/2023/05/runecast-remediation-scripts/) [Runecast](https://jameskilby.co.uk/category/runecast/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Runecast Remediation Scripts: Auto-Fix VMware Storage Issues](https://jameskilby.co.uk/2023/05/runecast-remediation-scripts/) By[James](https://jameskilby.co.uk) May 16, 2023 · Updated June 1, 2026 I am a huge fan of the Runecast product and luckily as a vExpert they give out NFR licences for my lab. * [ ![How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/wp-content/uploads/2024/12/ZFS.jpg) ](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) [TrueNAS Scale](https://jameskilby.co.uk/category/truenas-scale/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) By[James](https://jameskilby.co.uk) December 18, 2024 · Updated June 1, 2026 Introduction Copy on Write Disk IDs Trim Introduction I have run a number of systems using ZFS since the earliest days of my homelab using Nexenta, all the way back in 2010. --- # Automated VCF 9 Offline Depot Source: https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/ Date: 2026-04-10 Categories: Ansible, Automation, Docker, Homelab, Traefik, VCF, VMware, Artificial Intelligence, vSAN, Hosting, Veeam, Storage ![Automated VCF 9 Offline Depot architecture diagram showing Traefik reverse proxy and Nginx file server stack](https://jameskilby.co.uk/wp-content/uploads/2026/04/offlinedepot.png) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) | [Docker](https://jameskilby.co.uk/category/docker/) | [Homelab](https://jameskilby.co.uk/category/homelab/) | [Traefik](https://jameskilby.co.uk/category/traefik/) | [VCF](https://jameskilby.co.uk/category/vmware/vcf/) | [VMware](https://jameskilby.co.uk/category/vmware/) # Automated VCF 9 Offline Depot By[James](https://jameskilby.co.uk) April 10, 2026 · Updated July 11, 2026 • 📖7 min read(1,427 words) This Automated VCF 9 Offline Depot was built to solve a simple problem: [VCF 9](https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-0.html) changed its deployment model, and I needed a repeatable, hands-off way to spin one up in my lab. The script and Ansible playbook take a fresh Ubuntu VM and turn it into a fully working depot with publicly-trusted SSL certificates from Let’s Encrypt. The deployment means no manual Docker installs, no messing with certificates, no clicking through dashboards. Just SSH in, run one command, and have a fully working HTTPS file server with automatic Let’s Encrypt certificates via Cloudflare DNS. It basically simplifies and automates the official process documented [here](https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-0/deployment/deploying-a-new-vmware-cloud-foundation-or-vmware-vsphere-foundation-private-cloud-/preparing-your-environment/downloading-binaries-to-the-vcf-installer-appliance/connect-to-an-offline-depot-to-download-binaries/set-up-an-offline-depot-web-server-for-vmware-cloud-foundation.html). Thanks to [Gareth ](https://www.virtualisedfruit.co.uk)for pushing me to finish this and helping with testing. If you want to jump in and deploy right away, you can execute the below or go and look at the GitHub repo [here](https://github.com/jameskilbynet/iac/tree/main/docker/vcf9-offline-depot) curl -sSL https://raw.githubusercontent.com/jameskilbynet/iac/main/docker/vcf9-offline-depot/deploy.sh -o /tmp/deploy.sh sudo bash /tmp/deploy.sh 📋 Copy ## Table of Contents ## Prerequisites * A Cloudflare-managed domain + API key with DNS `Zone:DNS:Edit` and `Zone:Zone:Read` permissions * Ubuntu 22.04+ with at least 100GB of free disk space. I would recommend 200GB+ * Internet access from the Ubuntu VM * DNS records to be used for the web server and the Traefik dashboard. For this I am using `vcf.jameskilby.cloud `and `traefik.jameskilby.cloud` ## What it does * A single bash script takes a vanilla Ubuntu server and builds the entire stack: * Installs **Git** , **Ansible** , **Docker Engine** and **Docker Compose** * Deploys **Traefik** as a reverse proxy with automatic wildcard SSL via Cloudflare DNS challenge * Deploys **Nginx** as a file server with directory browsing enabled * Creates a `/vcf` directory on the host, owned by the deploying user, mounted into the container * Protects the web server with **HTTP basic authentication** * Forces all HTTP traffic to HTTPS * Exposes the **Traefik dashboard** on its own subdomain ![VCF Offline Depot](https://jameskilby.co.uk/wp-content/uploads/2026/04/VCF-Offline-Depot-1024x538.png) The script pulls and executes the below files. docker/traefik-nginx/ ├── deploy.sh # Bootstrap script — run this ├── playbook.yml # Ansible playbook — does the work ├── docker-compose.yml # Root compose file ├── .env.example # Example environment variables └── compose/ ├── traefik.yml # Traefik reverse proxy └── nginx.yml # Nginx file server 📋 Copy When it’s deployed you can drop the VCF installer files into the /vcf directory and they are passed through to the web server for use in VCF deployments. I do this using FileZilla but any method will work. What the script does under the hood is: **1\. Docker Installation** — Adds Docker’s official GPG key and APT repository, installs Docker Engine, CLI, containerd, Buildx and the Compose plugin. It detects the system architecture automatically so it works on both amd64 and arm64. **2\. Directory Structure** — Creates the stack directory at `/opt/traefik-nginx` with subdirectories for compose files, Traefik config, and Nginx config. Creates `/vcf` owned by the user who ran `sudo` so you can write files there without root. **3\. Nginx Config** — Writes a custom `default.conf` that enables `autoindex` for directory browsing with human-readable file sizes and local timestamps. **4\. Traefik Static Config** — Generates `traefik.yml` with: * HTTP and HTTPS entrypoints * Cloudflare DNS challenge for Let’s Encrypt certificates * Docker provider for automatic service discovery * Dashboard enabled via its own subdomain **5\. Environment File** — Writes `.env` (mode `0600`) containing the domain, subdomain, Cloudflare token, basic auth hash, and install directory. Docker Compose interpolates these into container labels at runtime. **6\. Network & Stack** — Creates the `traefik` Docker network (idempotent), then runs `docker compose up -d`. The stack is split across two compose files included from a root `docker-compose.yml`: **Traefik** (`compose/traefik.yml`): * Binds ports 80 and 443 * Mounts the Docker socket (read-only) for container discovery * Global HTTP-to-HTTPS redirect via a catchall router * Dashboard exposed at `traefik.yourdomain.com` **Nginx** (`compose/nginx.yml`): * Mounts `/vcf` from the host as the web root (read-only) * Custom nginx config for directory browsing * Basic auth middleware via Traefik labels, reading credentials from `${BASICAUTH_USERS}` in `.env` * Serves on `${SUBDOMAIN}.${DOMAIN}` Both containers run with `no-new-privileges` security option and JSON file logging with rotation. ## Security Whenever you’re adding passwords or tokens into a system, care needs to be taken. The script will handle the Cloudflare API Token and password in a secure manner. When the secure variables are entered they are handled with `read -rsp` this means that they don’t echo to the terminal as they are entered. These secrets are stored temporarily in a file with root only readable permissions. This has an automatic cleanup so that if the script completes successfully or if it crashes or is stopped manually this file is removed using ``trap 'rm -f "$VARS_FILE"' EXIT INT TERM``. Unset is also used to clear plaintext variables from the shell after they have been written to disk. ## How to use SSH into your VM and run: curl -sSL https://raw.githubusercontent.com/jameskilbynet/iac/main/docker/vcf9-offline-depot/deploy.sh -o /tmp/deploy.sh sudo bash /tmp/deploy.sh 📋 Copy The script will prompt for your domain, subdomain, web-server username and password, and Cloudflare API token. It confirms your input, then handles everything else. ![RunDeployment](https://jameskilby.co.uk/wp-content/uploads/2026/04/RunDeployment-1024x508.png) A few minutes later you’ll see: ═══════════════════════════════════════════ Deployment Complete ═══════════════════════════════════════════ Nginx: https://vcf.jameskilby.cloud Traefik: https://traefik.jameskilby.cloud Web root: /vcf Stack dir: /opt/vcf9-offline-depot DNS: Point *.jameskilby.cloud to this server's IP. Certs will be issued automatically via Cloudflare DNS. 📋 Copy Your Automated VCF 9 Offline Depot is now running and accessible over HTTPS. If you then access the nginx web server at your defined address i.e. https://vcf.jameskilby.cloud It should prompt you to log in using the credentials you defined at runtime. If it’s all working you should be presented with the following webpage and it should be on a fully trusted SSL certificate. ![workingnginx](https://jameskilby.co.uk/wp-content/uploads/2026/04/workingnginx-1024x752.png) ## Related reading If you like the Traefik + Cloudflare DNS pattern used here, the same stack powers my homelab AI infrastructure. These posts cover how it all fits together: * [Automating the deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) * [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) * [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) ## Troubleshooting Most problems fall into one of a handful of categories. Here are the checks I run first when something doesn’t come up clean. ### Certificates aren’t being issued The first thing to do is wait. It can take a minute or more for the process of issuing the certificate to complete. **Don’t** start troubleshooting immediately. If, after a minute, Traefik is still serving the default self-signed certificate instead of a Let’s Encrypt one, it’s almost always the Cloudflare API token. Verify the token is valid and scoped correctly: curl -s "https://api.cloudflare.com/client/v4/user/tokens/verify" \ -H "Authorization: Bearer " 📋 Copy The token needs `Zone:DNS:Edit` and `Zone:Zone:Read` permissions on the zone you’re issuing certificates for. Then check the Traefik logs for ACME errors: docker logs traefik 2>&1 | grep -i acme 📋 Copy One issue you might hit (which I ran into while documenting things) was hitting Let’s Encrypt’s rate limits. You will see something like this in the Traefik logs if you do: sudo docker logs traefik 2>&1 | grep -i "docker" | tail -10 ERR Unable to obtain ACME certificate for domains error="unable to generate a certificate for the domains [jameskilby.cloud *.jameskilby.cloud]: acme: error: 429 :: POST :: https://acme-v02.api.letsencrypt.org/acme/new-order :: urn:ietf:params:acme:error:rateLimited :: too many certificates (5) already issued for this exact set of identifiers in the last 168h0m0s, retry after 2026-04-11 03:06:50 UTC: see https://letsencrypt.org/docs/rate-limits/#new-certificates-per-exact-set-of-identifiers" ACME CA=https://acme-v02.api.letsencrypt.org/directory acmeCA=https://acme-v02.api.letsencrypt.org/directory domains=["jameskilby.cloud","*.jameskilby.cloud"] providerName=cloudflare.acme routerName=nginx@docker rule=Host(`vcf.jameskilby.cloud`) 📋 Copy If you’re stuck on a staging certificate after fixing the token, truncate `acme.json` and restart Traefik so it requests a fresh one: truncate -s 0 /opt/traefik-nginx/traefik/acme.json docker restart traefik 📋 Copy ### Docker fails to install If the playbook fails installing Docker, the VM usually has pending kernel updates waiting for a reboot. Either reboot manually and re-run, or allow the playbook to reboot for you by re-running with `-e allow_reboot=true`. ## Increase Disk Size The script will fail if you don’t have 100GB of available storage. You will need at least this for the VCF install binaries. If you have presented a single large disk to the VM and used Ubuntu’s auto partitioning it won’t utilise all of the drive. See my blog [here](https://jameskilby.co.uk/2025/12/ubuntu-disk-expansion-steps/) on how to expand the disk to its full size ### Review the Traefik Dashboard If you’re still having issues review the Traefik Dashboard `https://traefik.domain.com` You should see four routers and six services like the attached screenshots ![traefikrouters](https://jameskilby.co.uk/wp-content/uploads/2026/04/traefikrouters-1024x254.png) ![traefikservices](https://jameskilby.co.uk/wp-content/uploads/2026/04/traefikservices-1024x326.png) ### Basic auth prompt loops If the depot keeps prompting for credentials even with the right ones, it’s almost always `$` escaping. Docker Compose treats `$` as variable interpolation, so every `$` in the apr1 hash must be doubled to `$$`. The deploy script handles this automatically, but if you’re editing `.env` by hand, double-check the escaping. ## 📚 Related Posts * [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) * [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) * [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) ## Similar Posts * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 Part 2 of my self-hosted AI stack series. I cover container resource sizing, dual-network isolation via Traefik and Cloudflare Tunnels, and every database powering the stack — PostgreSQL, ClickHouse, Redis, Qdrant, MinIO, MongoDB, SQLite, Prometheus, and Jaeger — plus the backup strategy for each. * [ ![vSAN Cluster Shutdown – Orchestration](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2025/12/vsan-cluster-shutdown/) [VMware](https://jameskilby.co.uk/category/vmware/) | [vSAN](https://jameskilby.co.uk/category/vmware/vsan-vmware/) ### [vSAN Cluster Shutdown – Orchestration](https://jameskilby.co.uk/2025/12/vsan-cluster-shutdown/) By[James](https://jameskilby.co.uk) December 6, 2025 · Updated June 1, 2026 How to safely shut down a vSAN Environment * [ ![What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2018/03/cloudflare/) [Hosting](https://jameskilby.co.uk/category/hosting/) ### [What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained](https://jameskilby.co.uk/2018/03/cloudflare/) By[James](https://jameskilby.co.uk) March 27, 2018 · Updated June 1, 2026 Cloudflare – What is it and why would I care? I have been using Cloudflare for a long time. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Veeam](https://jameskilby.co.uk/category/veeam/) ### [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated June 1, 2026 My lab is always undergoing change. Partially as I want to try new things or new ways of doing things. * [ ![vSphere Power Management Ansible Playbooks with Semaphore](https://jameskilby.co.uk/wp-content/uploads/2026/04/vsphere-power-management-ansible-768x403.png) ](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automating vSphere Power Management driven by Ansible and SemaphoreUI](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) By[James](https://jameskilby.co.uk) April 15, 2026 · Updated June 1, 2026 In this post I’ll walk through how I use vSphere Power Management driven by Ansible and SemaphoreUI to automatically reduce ESXi host electricity consumption — saving real money on my Octopus Agile tariff by toggling hosts between Low Power and Balanced policies. Introduction One of the larger costs of running my homelab is the electricity…. * [ ![Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/wp-content/uploads/2024/09/QuantaGrid-SD1Q-1ULH-Front-Three-Quarter.png) ](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) By[James](https://jameskilby.co.uk) September 12, 2024 · Updated July 11, 2026 Yes, that’s a clickbait title. But technically it’s possible if I dropped all drive redundancy… I recently saw an advert for a server that was just too good to be true. --- # My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2) Source: https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/ Date: 2026-04-04 Categories: Artificial Intelligence, Automation, Docker, Homelab, NVIDIA, Traefik, VMware, VMware Cloud on AWS, Personal, Ansible, Cloudflare, vSAN ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured.png) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) | [Docker](https://jameskilby.co.uk/category/docker/) | [Homelab](https://jameskilby.co.uk/category/homelab/) | [NVIDIA](https://jameskilby.co.uk/category/nvidia/) | [Traefik](https://jameskilby.co.uk/category/traefik/) | [VMware](https://jameskilby.co.uk/category/vmware/) # My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 • 📖11 min read(2,137 words) _This is Part 2 of a multi-part series on my self-hosted AI stack.[Part 1](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) covers the application architecture and a layer-by-layer walkthrough of every service. If you haven’t read it yet, I’d recommend starting there — it gives the context that makes Part 2 make sense._ Part 2 goes deeper into the infrastructure side of things: how traffic is designed to route, how containers are sized and constrained, what databases and data stores are running under the hood. Specifically, this post covers: * **Resource limits and sizing** — CPU and memory constraints for every container, and the reasoning behind them * **Infrastructure and routing** — the dual-network design, Traefik reverse proxy configuration, Cloudflare Tunnels, and how external traffic is isolated from internal service communication * **Data layer** — every database and persistent store in the stack: PostgreSQL, SQLite, Qdrant, Redis, and MinIO * **Backups** — the sidecar backup strategy for PostgreSQL, n8n, and Open WebUI, with data landing on a NAS over SMB ## Table of Contents ## Self-Hosted AI Stack: Overview In this part I’ll walk through the network topology, container sizing, and every database in this self-hosted AI stack. ## Resource Limits & Sizing If you read part one, you will have seen that there are a lot of components that make up this AI stack. Every container in the stack has explicit CPU and memory limits and reservations set. This is to prevent a single runaway container or model starving other services of resources. These limits have been defined based on my production deployment running on my host with an NVIDIA A10 (24 GB vRAM), this is presented to a VM allocated with 24 CPU cores, and 96 GB RAM plus 500GB of NVMe storage. Based on that profile, I have allocated container resources as follows: Container| Role| Mem Limit| CPU Limit ---|---|---|--- **ollama**| LLM inference (GPU)| 24G| 12.0 **comfyui**| Image generation (GPU)| 16G| 6.0 **whishper**| Speech-to-text (GPU)| 4G| 4.0 **mongo**| DB for Whishper only| 1G| 1.0 **ollama-exporter**| Metrics scraper| 128M| 0.25 **postgres**| Shared relational DB| 4G| 4.0 **qdrant**| Vector DB for RAG| 6G| 4.0 **tika**| Document parsing (JVM)| 2G| 2.0 **searxng**| Metasearch engine| 1G| 1.0 **qdrant-backup**| Periodic backup sidecar| 256M| 0.5 **langfuse-web**| LLM observability UI| 2G| 2.0 **langfuse-worker**| Background trace processing| 2G| 2.0 **clickhouse**| Analytics DB for Langfuse| 6G| 4.0 **redis**| Job queue for Langfuse| 512M| 0.5 **minio**| S3-compatible object store| 1G| 1.0 **minio-init**| One-shot bucket creation| –| – **jaeger**| Distributed tracing| 4G| 2.0 **otel-collector**| Telemetry aggregation| 1G| 1.0 **nvidia-gpu-exporter**| GPU metrics| 256M| 0.5 **prometheus**| Metrics TSDB| 2G| 2.0 **grafana**| Dashboards| 2G| 2.0 **smarterrouter**| LLM routing| 2G| 2.0 **open-webui**| Chat UI| 2G| 2.0 **open-webui-backup**| Periodic SQLite backup| 256M| 0.25 **pipelines**| Filter/plugin framework| 2G| 2.0 **open-terminal**| Terminal server| 1G| 1.0 **n8n**| Workflow automation| 4G| 4.0 As can be seen from the docker containers above the resource requirements are not trivial. I would recommend the following VM configuration as a minimum to use the services with some tweaks to the docker and Ansible configuration to accommodate the reduced footprint. I would also highly recommend the data disk is on NVMe. Resource| Allocation ---|--- CPU| 16 Memory (GB)| 18 GPU vRAM (GB)| 8 OS Disk (GB)| 50 Data Disk (GB)| 200 ## Infrastructure & Routing ![Traefik Docker Setup](https://jameskilby.co.uk/wp-content/uploads/2025/01/Traefik-Docker-Setup-1024x342.png) All of the containers sit behind [Traefik](https://traefik.io) as the reverse proxy, with TLS certificates issued via the Cloudflare DNS challenge. Some public-facing services are exposed through Cloudflare Tunnels with Cloudflare Access policies enforcing authentication. Internal services that don’t need to be externally accessible stay isolated on the internal network with no Traefik labels and no external routes. ### Network Isolation ![AI Network](https://jameskilby.co.uk/wp-content/uploads/2026/03/AI-Network--1020x1024.png) The stack uses a dual-network design. An external `traefik` network connects services that need to be reachable via Traefik ( Blue Zone) (Open WebUI, Grafana, Jaeger, SmarterRouter, etc.), and a separate `aistack-internal` bridge network handles all inter-service communication. The internal network (Green Zone) is configured with a dedicated subnet (172.30.0.0/24), which means containers on it have no outbound internet access by default — they can only talk to each other. Services that need both external routing and internal communication (like Open WebUI, which needs Traefik for HTTPS but also needs to reach Ollama, Qdrant, and Tika) sit on both networks. Purely backend services — PostgreSQL, Redis, ClickHouse, Qdrant, the OTel Collector — only get the internal network, with `traefik.enable=false` on their labels so they’re never accidentally exposed. This limits the blast radius if any single container is compromised. ### Domain Portability The deployment is designed in a way that the url can be adapted as needed. The compose file uses the pattern `${DOMAIN:-jameskilby.cloud}` rather than a hardcoded hostname. Traefik labels, environment variables, webhook URLs — all of them resolve through a single `DOMAIN` variable in the `.env` file. This allows you to fork the repo and deploy the stack under your own domain. You change one variable and every service picks it up — `chat.yourdomain.com`, `grafana.yourdomain.com`, `langfuse.yourdomain.com`, and so on. The `:-jameskilby.cloud` default means the compose still works if the variable is unset, which keeps `docker compose config` clean for local testing. An `.env.example` file is included in the repo with placeholder values and generation instructions for every required secret. Copy it to `.env`, fill in your values, and the stack is ready to deploy. Persistent data that needs to survive host failures (Prometheus TSDB, Jaeger traces) is stored on a SMB share. Data that needs fast local I/O (Ollama models, Open WebUI state, databases) uses local Docker volumes. ## Data Layer: Databases in the Stack The stack uses a number of different data stores, each chosen to match the access pattern of the service it supports. They fall into three persistence tiers: local Docker volumes for latency-sensitive transactional workloads, SMB-mounted NAS volumes for append-heavy observability data, and an S3-compatible object store for blob storage. A single PostgreSQL 16.13 (Alpine) instance hosts three databases: `langfuse` for LLM observability, `n8n` for workflow definitions and execution history, and `grafana` for dashboard and user config. The Ansible playbook creates all three databases automatically at deploy time under the `aistack` user. Data lives on a local Docker volume (`postgres_data`) for write performance, and Postgres handles its own WAL-based crash recovery. ### ClickHouse 24.12 – Columnar Analytics for Langfuse Langfuse generates high volumes of trace and event data that need fast analytical queries—aggregations, filtering by time range, counting tokens across thousands of requests. ClickHouse’s columnar storage engine is purpose-built for this. It runs alongside Langfuse’s other dependencies on a local Docker volume (`clickhouse_data`), ### Redis 7 – Job Queue & Cache Redis serves as Langfuse’s job queue and caching layer. It’s configured with RDB snapshotting (`--save 60 1`) so queued jobs survive a container restart. Runs on a local Docker volume (`redis_data`) on the internal network only. ### MinIO – S3-Compatible Object Storage MinIO provides S3-compatible blob storage for Langfuse’s event uploads. A one-shot `minio-init` container creates the `langfuse` bucket on first startup using `mc mb`. Data lives on a local Docker volume (`minio_data`). This avoids any dependency on external cloud storage while giving Langfuse the S3 API it expects. ### Qdrant v1.17 – Vector Database for RAG Qdrant powers Open WebUI’s retrieval-augmented generation pipeline, storing document embeddings and handling similarity search. It exposes HTTP on port 6333 and gRPC on 6334, both on the internal network only. Apache Tika handles document parsing upstream, and the resulting vectors land in Qdrant’s local storage volume (`qdrant_storage`). ### MongoDB 7 – Whishper Transcription Store Whishper (the speech-to-text service) uses MongoDB to store transcription metadata and results. It runs on a local Docker volume (`mongo_data`) with a straightforward connection string: `mongodb://root:example@mongo:27017/whishper?authSource=admin`. The credentials are currently hardcoded in the compose file—a candidate for moving into `.env` in a future cleanup pass. ### SQLite – Embedded in Open WebUI & SmarterRouter Two services use SQLite as an embedded database. Open WebUI stores everything—users, conversations, RAG collections, model configs—in `webui.db` on a local Docker volume. SmarterRouter keeps its routing config and model performance cache in `router.db` on a separate volume. Both need POSIX file-locking, which rules out running them directly on SMB. The backup sidecar (covered in the Open WebUI Backup Strategy section below) handles off-host durability for `webui.db`. ### Prometheus v3.10 & Jaeger/Badger – Observability Stores on NAS Unlike the transactional databases above, the two observability stores mount directly to the NAS via CIFS. Prometheus writes its time-series data (TSDB) to an SMB-backed volume with a 30-day retention policy (`--storage.tsdb.retention.time=30d`). Jaeger uses Badger as its embedded key-value store for trace data, also writing to a CIFS volume. The rationale: metrics and traces are append-heavy, sequential workloads that tolerate network storage latency, while the NAS provides built-in RAID protection and snapshot capabilities without needing a separate backup strategy. ### Persistence Summary Database| Version| Used By| Storage ---|---|---|--- PostgreSQL| 16.13| Langfuse, n8n, Grafana| Local volume ClickHouse| 24.12| Langfuse (analytics)| Local volume Redis| 7 Alpine| Langfuse (queue/cache)| Local volume (RDB snapshots) MinIO| latest| Langfuse (event uploads)| Local volume Qdrant| 1.17.0| Open WebUI (RAG vectors)| Local volume MongoDB| 7| Whishper (transcriptions)| Local volume SQLite| embedded| Open WebUI, SmarterRouter| Local volume + NAS backup Prometheus| 3.10.0| Metrics (Grafana)| CIFS/NAS Jaeger/Badger| 2.15.0| Traces (Grafana)| CIFS/NAS This split—local volumes for transactional databases, NAS for observability—keeps write-sensitive workloads fast while ensuring metrics and traces survive host failures without a separate backup pipeline. The only exception is Open WebUI’s SQLite database, which gets its own backup sidecar because it’s both write-sensitive and too important to leave without off-host copies. ## Backups The self-hosted AI stack contains data in multiple locations. Some of this can be recreated or redownloaded easily and some of it needs protecting. ### OpenWebUI Backups OpenWebUI is the front end to most of the AI stack and it can end up storing a lot of data from various sources. This data is mostly stored in a SQLite database. Due to the way sqlite works it is not possible to run it on remote storage. It therefore runs on local NVMe storage attached to the VM. To enable backups of this data, I have a second container that runs alongside OpenWebUI sharing access to the same volumes. It periodically (every 6 hours) performs a crash-consistent database backup and writes it out to the persistent smb share. It also performs retention of the database only keeping the last 7 copies. This gives me the ability to restore: * All User config * Every conversation and the associated message history * Model configurations * Knowledge base metadata and document collections It also creates a tarball of the uploads directory for any documents added for RAG purposes, retaining the last 3 copies on the NAS. ### Qdrant Backups With the OpenWebUI backup restored RAG searches will silently fail as the storage of the actual embeddings generated from the documents will not be available. To fix this I need to ensure I have a backup and restore process for the Qdrant data as well. I handle this in a similar way to the OpenWebUI backups I perform a periodic snapshot via a sidecar container. It iterates all collections, snapshots each one, bundles them as a tarball, and writes them out to the same SMB-backed storage, retaining the last 7 copies. ### PostgreSQL Backups I have also deployed a backup sidecar container to handle PostgreSQL following the same principles. It runs `pg_dumpall` every 6 hours using the same `postgres:16.13-alpine` image as the server — keeping the `pg_dump` version in lockstep prevents any version mismatch on restore. Each run produces a gzip-compressed SQL dump (`postgres-YYYYMMDD-HHMMSS.sql.gz`) covering all three databases — `langfuse`, `n8n`, and `grafana` — written to the SMB-backed NAS volume. The last 7 dumps are retained. The sidecar only starts once PostgreSQL passes its healthcheck, so it never attempts a dump against an uninitialised server. ## N8N Backups The N8N container also contains a lot of data. A dedicated backup sidecar uses the n8n REST API (authenticated via `N8N_BACKUP_API_KEY`) to export data every 6 hours, committing changes to a local git repository for point-in-time history. To push changes to a remote GitHub repository, also define `N8N_BACKUP_GIT_REPO` and `N8N_BACKUP_GIT_TOKEN` in the .env file as the GitHub repo URL (e.g. `https://github.com/jameskilbynet/n8n-backups.git`) * Workflows – Full workflow definitions including nodes, connections, and settings, exported as individual files for easy diffing and selective restore. * Credentials – Metadata only (name, type, ID). The n8n API does not expose secret values — credential secrets remain encrypted in PostgreSQL and are never written to the backup. * Tags – All workflow tags, exported as a single JSON file. If the Git credentials are not set, it will back up to the SMB repo like the other containers. ## Continue Reading * **[← Part 1: Architecture Overview](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/)** – The big picture, layer-by-layer walkthrough from inference to observability. * **Part 3: Operations & Maintenance →** – Ansible deployment, monitoring with Uptime Kuma, and the Open WebUI backup strategy — coming soon ## 📚 Related Posts * [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) * [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) ## Similar Posts * [ ![VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) By[James](https://jameskilby.co.uk) May 21, 2025 · Updated July 11, 2026 Quick reference guide to the available storage resources that you get in VMware Cloud on AWS * [ ![VMware – Going out with a Bang!](https://jameskilby.co.uk/wp-content/uploads/2023/10/rnli-logo-768x384.png) ](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [VMware – Going out with a Bang!](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) By[James](https://jameskilby.co.uk) October 7, 2023 · Updated June 1, 2026 There is a lot of uncertainty with VMware at the moment. This is all due to the pending acquisition by Broadcom. * [ ![Automated VCF 9 Offline Depot architecture diagram showing Traefik reverse proxy and Nginx file server stack](https://jameskilby.co.uk/wp-content/uploads/2026/04/offlinedepot.png) ](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) By[James](https://jameskilby.co.uk) April 10, 2026 · Updated July 11, 2026 One Bash script turns a fresh Ubuntu VM into a VCF 9 Offline Depot: Traefik, Nginx, basic auth, and Let’s Encrypt wildcard certs via Cloudflare DNS. * [ ![Fixing Wrangler Node.js Version Conflicts After Brew Upgrade](https://jameskilby.co.uk/wp-content/uploads/2022/01/WranglerCrab-1-768x256.png) ](https://jameskilby.co.uk/2022/01/wrangler-and-node-versions/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) ### [Fixing Wrangler Node.js Version Conflicts After Brew Upgrade](https://jameskilby.co.uk/2022/01/wrangler-and-node-versions/) By[James](https://jameskilby.co.uk) January 15, 2022 · Updated July 11, 2026 I am a massive fan of the brew package management system for macOS and use it on all of my Macs. I typically just upgrade everything blindly and have never had an issue. * [ ![Free Octopus Agile Battery & Solar Calculator: 5 Batteries Tested](https://jameskilby.co.uk/wp-content/uploads/2026/03/Octopus-Energy-logo.jpg) ](https://jameskilby.co.uk/2026/03/octopus-agile-battery-solar-calculator/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Free Octopus Agile Battery & Solar Calculator: 5 Batteries Tested](https://jameskilby.co.uk/2026/03/octopus-agile-battery-solar-calculator/) By[James](https://jameskilby.co.uk) March 9, 2026 · Updated July 11, 2026 I am quite a heavy consumer of electricity at home. This is primarily driven by my lab but having a young son and two golden retrievers means more washing and drying. * [ ![vSAN Cluster Shutdown – Orchestration](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2025/12/vsan-cluster-shutdown/) [VMware](https://jameskilby.co.uk/category/vmware/) | [vSAN](https://jameskilby.co.uk/category/vmware/vsan-vmware/) ### [vSAN Cluster Shutdown – Orchestration](https://jameskilby.co.uk/2025/12/vsan-cluster-shutdown/) By[James](https://jameskilby.co.uk) December 6, 2025 · Updated June 1, 2026 How to safely shut down a vSAN Environment --- # New VMware Cloud on AWS Host: i7i.metal-24xl Source: https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/ Date: 2026-04-01 Categories: VMware, VMware Cloud on AWS, AWS, Automation, Homelab ![](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) # New VMware Cloud on AWS Host: i7i.metal-24xl By[James](https://jameskilby.co.uk) April 1, 2026 · Updated July 11, 2026 • 📖4 min read(869 words) We’ve expanded the VMC fleet with the new **i7i**[**(i7i.24xlarge)** ](https://aws.amazon.com/ec2/instance-types/i7i/)host type. Powered by Intel Emerald Rapids processors with PCIe Gen5 connectivity, it delivers the fleet’s highest single-core performance and memory bandwidth, making it well suited for latency-sensitive workloads and high-performance vSAN configurations. ## Key Technical Specs: * **Performance:** Intel Emerald Rapids CPUs + PCIe Gen5, offering the fleet’s highest clock speeds and memory bandwidth. * **Storage:** Optimized for vSAN with the fastest available NVMe devices. Support for both vSAN OSA and ESA * **Security:** Enhanced security based on [Intel Total Memory Encryption](https://www.intel.com/content/dam/www/central-libraries/us/en/documents/white-paper-intel-tme.pdf) This host brings a blend of usable options to complement the existing VMC fleet ranging from bleeding edge performance for tier one applications. It also supports expanded DR capabilities with SCFS via [VLR](https://techdocs.broadcom.com/us/en/vmware-cis/live-recovery.html). The below table compares the specifications of all nodes currently running in VMC. | **I3**| **I3en**| **I4i**| **I7i** ---|---|---|---|--- AWS Nitro Version| v2| v3| v4| v4 **CPU**| | | | Processor Name| Intel Xeon E5-2686 v4 | Intel Xeon Platinum 8175| Intel Xeon 8375c| Intel Xeon 8559c No of Physical Cores| 36| 48| 64| 48 Hyperthreading| No| Yes| Yes| Yes Base Clock| 2.3GHz| 2.5 Ghz| 2.9 GHz| 3.2 GHz Turbo Clock| N/A| 3.1 GHz| 3.5 GHz| 4.0 Ghz Processor Family| Broadwell| Skylake| Ice Lake| Emerald Rapids Supported Custom Core Counts| 8 16 36| 8 16 24 30 36 48| 8 16 24 30 36 48 64| 8 16 24 30 36 **Memory** | | | | Capacity GiB| 512| 768| 1024| 768 Memory Speed| DDR4-2400| DDR4-2666| DDR4-3200| DDR5-5600 **Networking**| | | | Network Adaptor Speed Gb/s| 25| 100| 75 | 56.25 Hardware Network Encryption | No| Yes| Yes| Yes **Storage**| | | | Drive Connection| PCIe Gen3| PCIe Gen3| PCIe Gen4| PCIe Gen5 Physical Drives| 8×1900| 8×7500| 8×3570| 6×3750 Physical Raw Space GiB| 15,200| 60,000| 28,560| 22,500 vSAN OSA Cache Disk| 2| 4^| 2| 2 vSAN OSA Capacity Disk| 6| 28^| 6| 4 vSAN Compression| Yes| Yes| Yes| Yes vSAN OSA Deduplication| Yes| No| No| No vSAN OSA Support| Yes| Yes| Yes| Yes vSAN ESA Support| No| No| Yes| Yes Underlying Storage Performance IOPS*| –| 80000| 160000| 120000 Underlying Storage Performance Throughput*| –| 2375| 5000| 3750 *Storage performance figures taken from the instance’s website. vSAN performance will differ based on OSA/ESA, Storage Policy in use and number of hosts in the environment. ^I3en hosts use NVMe namespaces to carve the disks up differently to how they are physically presented ** In vSAN ESA all drives are in a single tier and contribute to both performance and capacity ## What Workloads Benefit Most from the i7i? The i7i.metal-24xl is optimised for I/O-intensive enterprise workloads that demand the highest possible random IOPS with predictable, sub-millisecond latency. It is particularly well-suited for: * **SQL Server and Oracle databases** — DDR5-5600 memory bandwidth and Gen5 NVMe deliver measurable query throughput gains over i4i hosts * **High-frequency messaging platforms** — Kafka and Redis workloads benefit from low-latency NVMe and improved single-core turbo performance up to 4.0 GHz * **AI/ML inference** — large model serving with high memory bandwidth maps well to the 768 GiB DDR5 configuration * **VDI and virtualised desktop workloads** — the Custom CPU Core Count option (8, 16, 24, 30, or 36 physical cores) allows cost-effective right-sizing per host * **Workloads currently on i3 or i3en** — the i7i offers significantly better compute, memory bandwidth, and storage performance ## vSAN Configuration and Deployment Requirements The i7i.metal-24xl supports both **vSAN OSA** (Original Storage Architecture) and **vSAN ESA** (Express Storage Architecture). ESA is the recommended configuration for new deployments — it treats all six NVMe drives as a single-tier pool, delivering superior throughput and compression compared to OSA. * **SDDC version 1.26v2 or later** required for cluster conversions and new secondary cluster deployments * **Minimum 3-node cluster** — with vSAN compression enabled, a 3-node i7i cluster delivers strong usable capacity from the 6 × 3,750 GiB NVMe drives per host * **Stretched Cluster support** — opt-in multi-AZ stretched cluster deployments supported, providing HA across two AWS Availability Zones in a single region * **Custom CPU Core Count** — 8, 16, 24, 30, or 36 physical cores per host, useful for Oracle licensing optimisation ## Frequently Asked Questions ### How does the i7i.metal-24xl compare to the i4i? The i7i delivers approximately **23% better compute performance** and over **10% better price-performance** than the i4i. It upgrades from 3rd Gen Intel Xeon (Ice Lake) to 5th Gen (Emerald Rapids), moves from DDR4 to DDR5-5600 memory, and adds PCIe Gen5 NVMe. Storage capacity is 22.5 TiB raw per host versus the i4i’s 30 TiB, but random IOPS is significantly higher. ### What SDDC version is required for the i7i.metal-24xl? You need **SDDC version 1.26v2** or later to convert existing clusters or deploy new secondary clusters with i7i hosts. New SDDCs can be provisioned with i7i directly once the instance type is available in your region. ### Is stretched cluster supported on i7i? Yes. Customers can opt in to a multi-AZ stretched cluster deployment for new SDDCs using i7i.metal-24xl hosts, providing high availability across two AWS Availability Zones within a single region. ### Can I mix i7i hosts with i4i or i3en in the same cluster? No — VMware Cloud on AWS clusters are single host-type only. You can add an i7i as a secondary cluster within an existing SDDC, but mixing host types within the same cluster is not supported. The official announcement can be found [here](https://blogs.vmware.com/cloud-foundation/2026/04/01/vmware-cloud-aws-i7i-metal-24xl-instance/). ## 📚 Related Posts * [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) * [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) * [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) ## Similar Posts * [ ![Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/wp-content/uploads/2026/01/Firefly_Gemini-Flash-768x417.png) ](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) By[James](https://jameskilby.co.uk) January 27, 2026 · Updated June 5, 2026 How to leverage Content Libraries to deploy into VMware Cloud on AWS faster. * [ ![vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) By[James](https://jameskilby.co.uk) November 17, 2023 · Updated June 1, 2026 An Overview of vSAN ESA in VMC * [ ![VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) By[James](https://jameskilby.co.uk) May 21, 2025 · Updated July 11, 2026 Quick reference guide to the available storage resources that you get in VMware Cloud on AWS * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/07/i3en/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [VMware Cloud on AWS i3en Host: Specs, Storage & Performance](https://jameskilby.co.uk/2020/07/i3en/) By[James](https://jameskilby.co.uk) July 2, 2020 · Updated June 5, 2026 VMware Cloud on AWS (VMC) has introduced a new host to its lineup: the “i3en”. This is based on the i3en. * [ ![Monitoring VMware Cloud on AWS: Tools & Approaches \(Part 1\)](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) ](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) [VMware](https://jameskilby.co.uk/category/vmware/) | [AWS](https://jameskilby.co.uk/category/aws/) ### [Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1)](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) By[James](https://jameskilby.co.uk) December 17, 2019 · Updated June 5, 2026 As previously mentioned I have been working a lot with VMware Cloud on AWS and one of the questions that often crops up is around an approach to monitoring. * [ ![Template Deployment with Packer](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) ](https://jameskilby.co.uk/2021/01/hashicorp-packer/) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. --- # My Self-Hosted AI Stack: Architecture Overview (Part 1) Source: https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/ Date: 2026-03-27 Categories: Artificial Intelligence, Automation, Docker, Homelab, NVIDIA, Traefik, VMware, Networking, VMware Cloud on AWS, Veeam, VCF ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured.png) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) | [Docker](https://jameskilby.co.uk/category/docker/) | [Homelab](https://jameskilby.co.uk/category/homelab/) | [NVIDIA](https://jameskilby.co.uk/category/nvidia/) | [Traefik](https://jameskilby.co.uk/category/traefik/) | [VMware](https://jameskilby.co.uk/category/vmware/) # My Self-Hosted AI Stack: Architecture Overview (Part 1) By[James](https://jameskilby.co.uk) March 27, 2026 · Updated May 31, 2026 • 📖14 min read(2,817 words) I’ve been building out my self-hosted AI stack over the past eighteen months. It’s been quite a journey for me personally and I have learnt a huge amount. Everything runs on my own [hardware](https://jameskilby.co.uk/lab/). All the containers sit behind Traefik, with public access done via Cloudflare Tunnels. The containers are orchestrated with Docker Compose driven by Portainer. The intention is to have no SaaS subscriptions, no data leaving the house unless I explicitly choose it. I have previously written about the underlying [Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/). This post goes into the AI stack itself. I will also publish a sanitised version of my Docker Compose and Ansible playbook to allow you to get up and running. I have previously spoken around this journey at [VMUG](https://youtu.be/Dt6m9JdsrIM). Although I haven’t fully moved away from commercial hosted AI tools I have significantly reduced their usage. ## Table of Contents ## Prerequisites Before deploying the stack, you need the following in place. I’m listing everything here because getting these right upfront saves a lot of debugging later. I have talked about what is needed and automating these prerequisites [here](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) If you have followed along you should already have an Ubuntu VM with a GPU and drivers passed into it and then Docker, Docker Compose and the Docker Container toolkit already configured. You will also have a Traefik load balancer setup ready to go. For the next steps you will also need some SMB storage for some of the container configuration files. ### DNS Records The stack exposes 12 services via Traefik, each on its own subdomain of your base `DOMAIN`. You can either create individual A/AAAA records or a single wildcard record (`*.$DOMAIN`) pointing to your Traefik host. If you’re using Cloudflare Tunnels for public access, each subdomain needs a corresponding tunnel route. Subdomain| Service| Purpose ---|---|--- `chat`| Open WebUI| Primary chat interface `ollama`| Ollama| LLM API (basic auth protected) `comfyui`| ComfyUI| Image generation UI `whishper`| Whishper| Speech-to-text UI `terminal`| Open Terminal| Browser shell (API key protected) `smarterrouter`| SmarterRouter| Intelligent model routing API `searxng`| SearxNG| Private metasearch engine `n8n`| n8n| Workflow automation `langfuse`| Langfuse| LLM observability dashboard `jaeger`| Jaeger| Distributed tracing UI `prometheus`| Prometheus| Metrics collection `grafana`| Grafana| Dashboards and visualisation ### Network & Storage The only inbound ports required at the host level are 443 (HTTPS) and optionally 80 (for HTTP→HTTPS redirect) — Traefik handles everything else. All inter-service communication happens on the internal Docker bridge network with no host port bindings. If you’re using NAS-backed storage for Prometheus and Jaeger data, the Docker host needs SMB/CIFS access to the share on port 445. Without a NAS, you can swap the CIFS volume definitions in `observability.yml` for standard local Docker volumes — you just lose the ability to survive a host failure without data loss on those services. ## The Big Picture The stack breaks down into seven architectural layers: **inference** (GPU model serving), **interface** (user-facing UIs), **routing** (intelligent model selection), **data** (databases and search), **langfuse** (LLM-specific observability), **observability** (tracing and metrics), and **automation** (workflow orchestration). Each layer lives in its own compose file and is merged by a thin orchestrator using Docker Compose `include` (v2.20+). Every service runs in its own container with resource limits and health checks, and Traefik handles TLS termination and routing for everything that needs to be internally and externally accessible. The orchestrator itself is deliberately minimal — it’s just an `include` block with comments explaining the dependency order: include: # Foundation layers (no cross-layer deps) - path: compose/inference.yml - path: compose/data.yml - path: compose/observability.yml # Depends on data + observability - path: compose/langfuse.yml # Depends on inference - path: compose/routing.yml # Depends on inference + data + observability + langfuse - path: compose/interface.yml # Depends on inference + data - path: compose/automation.yml 📋 Copy This means I can work on the observability layer without scrolling past 200 lines of inference config, and each file manages its own volumes and network attachments. The `no-gpu.yml` override slots in alongside these to strip NVIDIA device reservations for CPU-only hosts. ![Self-hosted AI stack architecture overview](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-1024x538.png) ## Inference Layer ### Ollama [Ollama](https://ollama.com) is the heart of the operation. It runs a variety of large language models (LLMs) downloaded locally. It exposes these as a REST API that’s OpenAI-compatible. Using my [NVIDIA A10 ](https://jameskilby.co.uk/lab/)with 24GB of vRAM available I can have multiple smaller models loaded simultaneously, or a single larger model. ### ComfyUI [ComfyUI](https://github.com/comfyanonymous/ComfyUI) handles image generation. It’s a node-based interface for building diffusion pipelines, and it integrates into Open WebUI so image generation happens inline in the chat interface. Like Ollama, it gets NVIDIA GPU access — though the two services have to share the GPU, so the resource limits matter here. ComfyUI gets up to 12GB of RAM and 6 CPU cores. ## Interface Layer ### Open WebUI [Open WebUI](https://github.com/open-webui/open-webui) is the primary chat interface. It talks to Ollama as its backend, with the OpenAI-compatible API used for model interactions. It’s configured to use Qdrant as its vector store (for RAG), Apache Tika for document parsing, SearxNG for web search augmentation, Whishper for speech-to-text, and ComfyUI for image generation. That’s a lot of integrations wired up, and it all just works once the service names resolve correctly inside the Docker network. One thing I’ve added that I’m particularly happy with is the [Open Terminal](https://github.com/open-webui/open-terminal) integration — this gives Open WebUI a shell execution capability, so models can run commands and return results. It runs as a separate container (`open-terminal:0.11.23`) exposed at `terminal.$DOMAIN` via Traefik, with API key authentication gating access. The container is sandboxed on the internal network, but opens up genuinely useful agentic workflows — models can inspect logs, query databases, or run scripts without needing direct host access. An **Open WebUI Backup sidecar** runs alongside the main container, performing `sqlite3 .backup` every six hours to a SMB-mounted volume. It is configured to retain the last seven database snapshots and the last four upload archives. This exists because SQLite does not work reliably over SMB (file-locking issues), so the main data stays on a fast local Docker volume while crash-consistent copies are pushed to network storage for durability. OpenTelemetry is also enabled on Open WebUI, which pipes traces to the OTel Collector and on to Jaeger. A separate Pipelines container handles deeper LLM-layer tracing via the Langfuse filter pipeline — covered in more detail in my next post. ### Whishper [Whishper](https://github.com/pluja/whishper) is a self-hosted Whisper transcription service with a web UI. It exposes an OpenAI-compatible API endpoint, which is what Open WebUI uses for its speech-to-text input. Models are loaded on-demand from a mounted volume, and the service gets GPU access for faster transcription. Backed by MongoDB for job storage. ### Open WebUI Pipelines [Open WebUI Pipelines](https://github.com/open-webui/pipelines) is a plugin framework that sits between Open WebUI and its LLM backends, intercepting every request and response. It runs as a separate container exposing an OpenAI-compatible API on port 9099, and Open WebUI connects to it as a second backend endpoint alongside Ollama. The reason it’s in the stack is the [Langfuse filter pipeline](https://github.com/open-webui/pipelines/blob/main/examples/filters/langfuse_filter_pipeline.py). Rather than manually installing pipelines through the Admin UI, the container auto-downloads the pipeline script on startup via the `PIPELINES_URLS` environment variable — so the Langfuse integration is live the moment the container is healthy, with no manual steps required. The pipeline is pre-configured with the Langfuse API keys and internal host, so traces start flowing immediately. This gives Langfuse richer data than the native OpenTelemetry tracing alone — the filter pipeline captures the full request/response cycle at the application layer, including prompt content, response text, token counts, and latency per turn, which OTel traces don’t expose at that level of detail. The two approaches complement each other: OTel covers infrastructure and service-level tracing, Pipelines covers the conversational layer. ## Routing Layer ![AI request routing diagram showing SmarterRouter directing traffic across the self-hosted AI stack](https://jameskilby.co.uk/wp-content/uploads/2026/03/AI-Cutdown-1024x371.png) ### SmarterRouter [SmarterRouter](https://github.com/peva3/smarterrouter) is a recent addition to my stack. It sits in front of Ollama and acts as an intelligent LLM router — instead of hardcoding which model to use for a given request, SmarterRouter analyses the prompt and selects the most appropriate model dynamically. I believe this capability is really important in constrained environments — users either choose a tiny model for a quick response and risk being disappointed by the results, or a much larger model that can be slow to load and respond. It does this by profiling all of the AI models available in Ollama. This is done on SmarterRouter startup. It connects to Ollama, lists the available models and then profiles them. This can take some time on initial launch but it’s a one-time operation unless new models are added. SmarterRouter regularly polls Ollama, so if a new model is added this is handled automatically. This can be seen in the logs where I added a new model in Ollama, it auto detected and started profiling. INFO - Found 14 existing profiles, profiling 1 new models INFO - PROGRESS [1/1] Starting profiling: deepseek-r1:8b INFO - Warming up model deepseek-r1:8b (size=4.9GB, timeout=130s) INFO - Loading model: deepseek-r1:8b (keep_alive=300, timeout=None) INFO - Adaptive timeout calculated: 334s (floor=99s, method_a=334s, method_b=74s, token_rate=48.5 tok/s) INFO - PROGRESS [1/1] deepseek-r1:8b - reasoning (1/5) - 6.7% - ETA: 22m 56s remaining INFO - PROGRESS [1/1] deepseek-r1:8b - reasoning (2/5) - 13.3% - ETA: 11m 28s remaining INFO - PROGRESS [1/1] deepseek-r1:8b - reasoning (3/5) - 20.0% - ETA: 7m 38s remaining INFO - PROGRESS [1/1] deepseek-r1:8b - coding (1/5) - 40.0% - ETA: 3m 49s remaining INFO - PROGRESS [1/1] deepseek-r1:8b - coding (2/5) - 46.7% - ETA: 3m 16s remaining INFO - PROGRESS [1/1] deepseek-r1:8b - coding (3/5) - 53.3% - ETA: 2m 52s remaining INFO - PROGRESS [1/1] deepseek-r1:8b - creativity (1/5) - 73.3% - ETA: 2m 5s remaining INFO - PROGRESS [1/1] deepseek-r1:8b - creativity (2/5) - 80.0% - ETA: 1m 54s remaining INFO - PROGRESS [1/1] deepseek-r1:8b - creativity (3/5) - 86.7% - ETA: 1m 45s remaining 📋 Copy It works with any OpenAI-compatible client, so a wide range of tools can connect to it. The routing logic is informed by benchmarks pulled from HuggingFace and LMSYS, and it builds per-model profiles by running a set of sample prompts at startup. It’s configured to prefer smaller and newer models where they’re capable enough, which helps keep latency down on simpler tasks. A few features worth calling out from the config: * **VRAM-aware routing** — it monitors GPU VRAM and can automatically unload models using an LRU strategy when memory gets tight, then load the right model for the next request. * **Response caching** — a 500-entry cache with a 1-hour TTL avoids redundant inference on repeated or similar prompts. * **Signature injection** — responses are annotated with the model that actually handled them (`Model: {model}`), which is useful for debugging and understanding routing decisions. * **SQLite-backed routing state** — profiles and cache persist across restarts in a named volume. * **Automated backups for PostgreSQL** — the Open WebUI backup sidecar pattern works well and I’d like to extend it to pg_dump snapshots for the Langfuse, n8n, and Grafana databases. * **LiteLLM as a unified gateway** to front both local Ollama models and external APIs (OpenAI, Anthropic) behind a single endpoint, with spend tracking and per-user rate limits. The practical result is that you can send any prompt to a single endpoint and let the router decide whether it warrants a heavyweight model or whether something smaller and faster will do. For a homelab running a single GPU, this kind of intelligent load management makes a real difference. Help me study vocabulary: write a sentence for me to fill in the blank, and I'll try to pick the correct option. **smarterrouter/main** Here's a vocabulary sentence for you: The politician's speech was full of _______ statements that sounded impressive but actually said very little of substance. Choose the best option: A) eloquent B) mundane C) grandiose D) candid What's your answer?**Model: qwen3-coder:30b** 📋 Copy ## Data & Automation Layer ### Qdrant [Qdrant](https://qdrant.tech) is the vector database. It stores embeddings for Open WebUI’s RAG pipeline — when you upload a document or paste text into a chat session, it gets chunked, embedded, and stored in Qdrant. At query time, relevant chunks are retrieved and injected into the context window. It’s fast, well-documented, and runs comfortably in a container. ### Apache Tika [Apache Tika](https://tika.apache.org) is the document parsing layer. When you feed Open WebUI a PDF, Word document, or spreadsheet, Tika extracts the text so it can be embedded and stored in Qdrant. It handles an enormous range of file formats and runs as a simple HTTP service. ### SearxNG [SearxNG](https://searxng.github.io/searxng/) is a self-hosted, privacy-respecting meta-search engine. Open WebUI is configured to use it for web-augmented queries — when a model needs to search the web, it fires queries at SearxNG rather than a third-party search API. This keeps search queries off external platforms and gives me control over which search backends are aggregated. ### n8n [n8n](https://n8n.io) is the workflow automation layer. It has native Ollama integration, so it can call local models as part of automated workflows. Combined with its broad library of integrations (webhooks, APIs, databases, and more), this is where AI capabilities get wired into actual processes. n8n uses PostgreSQL for its persistent store, which runs in the same internal Docker network. ## Observability Layer This is the part most self-hosted AI setups skip, and I think it’s worth talking about separately. ### OpenTelemetry Collector + Jaeger Open WebUI emits OTLP traces, which are collected by the [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) and forwarded to [Jaeger](https://www.jaegertracing.io) for storage and visualisation. This means I can trace individual requests through the system — from the user message, through model inference, RAG lookups, and tool calls. When something is slow or broken, the trace tells me exactly where. The Collector also acts as a security boundary. A **filter processor** strips sensitive attributes — passwords, tokens, authorisation headers, and API keys — from traces before they reach Jaeger. This means I can enable verbose tracing on services without worrying about credentials leaking into the trace store. The Collector also enriches every span with environment and cluster resource attributes, and enforces a memory limiter (512 MB cap with 128 MB spike headroom) to prevent runaway cardinality from taking down the pipeline. ### Prometheus + Grafana + NVIDIA GPU Exporter [Prometheus](https://prometheus.io) scrapes metrics from services. [Grafana](https://grafana.com) visualises them. The NVIDIA GPU Exporter surfaces per-GPU metrics (utilisation, memory, temperature, power draw) via `nvidia-smi`, which Prometheus scrapes and Grafana displays. Watching GPU utilisation during inference makes it much easier to tune `OLLAMA_NUM_GPU` and memory limits. Grafana’s provisioning is fully automated — datasources (Prometheus and Jaeger with trace-to-metrics correlation) and dashboards are loaded from config files on startup. A custom **AI Stack Overview dashboard** is included in the repo and deployed by the Ansible playbook, giving an out-of-the-box view of GPU utilisation, model load times, inference latency, and service health without needing to build anything manually after deployment. Alongside the generic GPU exporter, a custom **Ollama Prometheus Exporter** polls the Ollama API (`/api/tags` and `/api/ps`) and exposes model-level metrics — available model counts, currently loaded models, and per-model VRAM consumption. This gives Grafana dashboards visibility into which models are resident in GPU memory at any given time, which is essential for debugging slow first-token latency when a model needs to be loaded from disk. ### Langfuse [Langfuse](https://langfuse.com) is LLM-specific observability — purpose-built for tracking prompts, model usage, latency, and evaluation in a way that generic APM tools don’t address. While Jaeger gives me distributed traces and Prometheus gives me infrastructure metrics, Langfuse fills the gap at the AI application layer: which prompts are slow, which models are being used, how responses score on quality, and where tokens are being spent. The stack behind it is more substantial than you might expect. [ClickHouse](https://clickhouse.com) handles trace storage — it’s a columnar analytics database well-suited to the high-volume, time-series nature of LLM traces. [Redis](https://redis.io) manages job queues between the web UI and the background worker. And [MinIO](https://min.io) provides S3-compatible object storage for raw event uploads before they’re processed into ClickHouse, keeping ingestion fast and durable under load. A small `minio-init` container handles bucket creation on first run. The two Langfuse containers — `langfuse-web` (UI and API, at [langfuse.jameskilby.cloud](https://langfuse.jameskilby.cloud)) and `langfuse-worker` (background processing) — share the same ClickHouse, Redis, and MinIO backends, with PostgreSQL handling relational metadata. Both containers wait for healthy ClickHouse, Redis, and MinIO before starting, which keeps startup ordering clean. Telemetry back to Langfuse HQ is disabled. Langfuse is also configured with **bootstrap variables** that automatically create the initial organisation, project, and admin user on first start. The public and secret API keys are set via environment variables and match what the Pipelines filter uses to authenticate — so the full Langfuse integration is functional the moment the stack comes up, with no manual setup through the UI required. ## Continue Reading This post is Part 1 of a multi-part series. The architecture overview above covers what each layer does and why it’s there. The next two parts will go deeper into the infrastructure and operational side. ## 📚 Related Posts * [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) * [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) ## Similar Posts * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 Part 2 of my self-hosted AI stack series. I cover container resource sizing, dual-network isolation via Traefik and Cloudflare Tunnels, and every database powering the stack — PostgreSQL, ClickHouse, Redis, Qdrant, MinIO, MongoDB, SQLite, Prometheus, and Jaeger — plus the backup strategy for each. * [ ![Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/wp-content/uploads/2024/06/Ubiquiti_Networks-Logo.wine_-768x512.png) ](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) By[James](https://jameskilby.co.uk) June 26, 2024 · Updated July 11, 2026 How to configure DHCP Option 43 for UniFi devices * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/07/i3en/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [VMware Cloud on AWS i3en Host: Specs, Storage & Performance](https://jameskilby.co.uk/2020/07/i3en/) By[James](https://jameskilby.co.uk) July 2, 2020 · Updated June 5, 2026 VMware Cloud on AWS (VMC) has introduced a new host to its lineup: the “i3en”. This is based on the i3en. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Veeam](https://jameskilby.co.uk/category/veeam/) ### [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated June 1, 2026 My lab is always undergoing change. Partially as I want to try new things or new ways of doing things. * [ ![VMware Holodeck Multi-Host VCF: Lab Setup & Configuration](https://jameskilby.co.uk/wp-content/uploads/2023/12/Holodeck-Overview.png) ](https://jameskilby.co.uk/2024/01/multihost-holodeck-vcf/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VCF](https://jameskilby.co.uk/category/vmware/vcf/) ### [VMware Holodeck Multi-Host VCF: Lab Setup & Configuration](https://jameskilby.co.uk/2024/01/multihost-holodeck-vcf/) By[James](https://jameskilby.co.uk) January 17, 2024 · Updated June 1, 2026 How to Deploy VMware Holodeck on multiple hosts * [ ![Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-12-14-at-21.45.23.png) ](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) By[James](https://jameskilby.co.uk) December 14, 2022 · Updated June 1, 2026 I run a reasonably extensive homelab that is of course built around the VMware ecosystem. --- # Free Octopus Agile Battery & Solar Calculator: 5 Batteries Tested Source: https://jameskilby.co.uk/2026/03/octopus-agile-battery-solar-calculator/ Date: 2026-03-09 Categories: Artificial Intelligence, Automation, Homelab, Docker, Ansible ![](https://jameskilby.co.uk/wp-content/uploads/2026/03/Octopus-Energy-logo.jpg) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) # Free Octopus Agile Battery & Solar Calculator: 5 Batteries Tested By[James](https://jameskilby.co.uk) March 9, 2026 · Updated July 11, 2026 • 📖5 min read(982 words) I am quite a heavy consumer of electricity at home. This is primarily driven by my [lab](https://jameskilby.co.uk/lab/) but having a young son and two golden retrievers means more washing and drying. As a result I am always looking to try and reduce the electrical costs as it’s not cheap in the UK. ## Table of Contents ## Problem Sadly my roof isn’t an ideal candidate for solar panels as the south facing part isn’t huge and has some shade, therefore I was looking at other options. I have been an Octopus Energy customer for a number of years. They have become one of the larger suppliers in the UK by offering a wide range of interesting tariffs and good customer support. I am on the “Agile Tariff” which means the price of electricity that I pay varies widely with the price changing every 30 minutes. Despite having some expensive time periods I generally come out ahead. (I am £178 better off than a tracker tariff over the last 4 months) I have often wondered if I can make a household battery make financial sense. A lot of the public calculators didn’t really give me the level of information I needed. Therefore I decided to build this Octopus Agile battery calculator. Disclaimer: all of the coding for this project was done by [Claude](http://Claude.ai) — I can’t take any credit for it and for legal reasons double check the maths. Below is the typical profile taken from today’s rates. Although the price changes based on demand and generation, typically windy and sunny days are the best. The typical cost profile follows this shape. Generally cheaper electricity most of the day and an expensive period between 4 and 7pm. At the time of writing the Middle East conflict has just started so the prices are higher than typical and the range between the lower and higher rate is narrower than usual. ![Agile Rate Profile](https://jameskilby.co.uk/wp-content/uploads/2026/03/Screenshot-2026-03-07-at-07.30.35-1024x713.png) ## Code If you want to skip reading and jump straight to the code on [Github](https://github.com/jameskilbynet/AgileBatteryPredicter) You need Python 3.8 and requests module and to add your Octopus API key to the .env file and then run the script. To get your API key just head to your Octopus account dashboard and open [**Developer settings**.](https://octopus.energy/dashboard/new/accounts/personal-details) When you run the script it will connect to the Octopus Kraken system and pull your historic usage and pricing data in 30-minute chunks. Once it has done this it will generate an HTML file called octopus_battery_report.html next to the python script for easy consumption. You can also execute the code as a one liner: python3 octopus_battery_analysis.py --api-key sk_live_yourkey 📋 Copy I have also included a diagnostic script in the repo for troubleshooting API issues octopus_diagnose.py ## Report Overview I have built an example report that can be seen [here](https://jameskilbynet.github.io/AgileBatteryPredicter/example_report.html). Below is the extract from my usage. ### Usage Summary ![Agile Usage Summary](https://jameskilby.co.uk/wp-content/uploads/2026/03/Agile-usage-Summary-1024x247.png) This is my usage report, and as can be seen I used 9,615kWh over the last year. For someone with gas heating and no electric cars this is quite high. I could also see this usage increasing. It also shows I had 164 hours at 0 cost or where Octopus paid me a small amount to use energy. The report then goes into a bit more consumption detail which is fairly self-explanatory. ### Agile Rate Distribution ![Agile Rate Distribution](https://jameskilby.co.uk/wp-content/uploads/2026/03/Agile-Rate-Distribution-1024x347.png) The distribution is important to understand so we can see if we can push our usage to the left with a battery. The report then goes on to make battery recommendations based on the data with multiple charging strategies. The default strategy is a very simple one: we charge the battery if the charge is ≤10.0p and discharge ≥20.0p ### Battery Recommendation The report has details of five of the most common batteries in the UK. Powerwall 2, GivEnergy 9.5kWh, GivEnergy 13.5kWh, Sonnen 10kWh, Pylontech 7.4kWh. They all have different capacities and an estimated install cost is also assumed. It then calculates what savings could be achieved with each of these setups with either the default charging strategy or an optimised one. Based on my usage the payback is never and the 15yr return on investment is very negative. This was disappointing. ![Battery Recommendation](https://jameskilby.co.uk/wp-content/uploads/2026/03/BatteryRecomendation--1024x447.png) ### What If Calculator I decided to see if I could improve by simulating some what-if scenarios so I added a slider into the report to take account of the cost of the battery system reducing and the price of electricity increasing. Unfortunately for my use case even if the price of batteries halves and the price of electricity doubles it still doesn’t make sense for me to install a battery. ### Solar OK so what if I was to add solar to the mix? This brings another factor to consider as effectively it leads to three strategies, Solar only, Battery only or Solar and Battery. The report will showcase some typical solar installs and calculate the solar generation where the energy would be consumed by my home first and then being exported. It will model several typical solar setups 3, 4 and 6kW and ultimately recommend the most cost effective setup. I was surprised by the answer to this where a 4kW solar system without a battery is what is ultimately recommended. However this still looks like a poor investment for me personally. ![Best Solar and Battery Combination](https://jameskilby.co.uk/wp-content/uploads/2026/03/Best-Combination-1024x375.png) #### SEG – Smart Export Guarantee The calculator will also include earnings for any unused solar energy. This is an important factor in the calculations. ## Conclusion For my current usage neither a battery nor solar panels make a good investment however, I will be reviewing this periodically as the benefits and the price of both solar and battery have changed dramatically in the last couple of years. Hopefully you have a better output from the calculator. I am also incredibly impressed with Claude’s ability to build this with limited prompting from me. The results look interesting although the script has gone through limited testing due to me only having access to two accounts with Octopus. ## 📚 Related Posts * [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) * [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) * [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) ## Similar Posts * [ ![Wa](https://jameskilby.co.uk/wp-content/uploads/2025/04/210902461-012e7273-413a-4ec7-be44-e854347f5a21-768x180.png) ](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Warp – The intelligent terminal](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) By[James](https://jameskilby.co.uk) April 11, 2025 · Updated June 1, 2026 How Warp is helping me run my homelab. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 Part 2 of my self-hosted AI stack series. I cover container resource sizing, dual-network isolation via Traefik and Cloudflare Tunnels, and every database powering the stack — PostgreSQL, ClickHouse, Redis, Qdrant, MinIO, MongoDB, SQLite, Prometheus, and Jaeger — plus the backup strategy for each. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) By[James](https://jameskilby.co.uk) March 27, 2026 · Updated May 31, 2026 A walkthrough of my self-hosted AI stack: Ollama, Open WebUI, ComfyUI, Whishper, n8n, Qdrant, SearxNG, and a full observability layer — all running on my own hardware with Docker Compose. * [ ![Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU](https://jameskilby.co.uk/wp-content/uploads/2024/10/pexels-tara-winstead-8386440-768x512.jpg) ](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) ### [Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) By[James](https://jameskilby.co.uk) October 11, 2024 · Updated July 11, 2026 Artificial intelligence is all the rage at the moment, It’s getting included in every product announcement from pretty much every vendor under the sun. * [ ![vSphere Power Management Ansible Playbooks with Semaphore](https://jameskilby.co.uk/wp-content/uploads/2026/04/vsphere-power-management-ansible-768x403.png) ](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automating vSphere Power Management driven by Ansible and SemaphoreUI](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) By[James](https://jameskilby.co.uk) April 15, 2026 · Updated June 1, 2026 In this post I’ll walk through how I use vSphere Power Management driven by Ansible and SemaphoreUI to automatically reduce ESXi host electricity consumption — saving real money on my Octopus Agile tariff by toggling hosts between Low Power and Balanced policies. Introduction One of the larger costs of running my homelab is the electricity…. * [ ![Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/wp-content/uploads/2026/01/VMware-NVIDIA-logos_ee2f18dc-615d-4c9e-8f11-9c3c2ce2bf37-prv-768x432.png) ](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) ### [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) By[James](https://jameskilby.co.uk) February 9, 2026 · Updated July 11, 2026 Learn how to use Ansible to configure an Ubuntu VM for use with NVIDIA based AI workloads in vSphere --- # Automating the Deployment of my Homelab AI Infrastructure Source: https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/ Date: 2026-02-09 Categories: Ansible, Artificial Intelligence, Containers, Devops, Homelab, NVIDIA, Traefik, VMware, Veeam, Storage, Nutanix, Personal ![](https://jameskilby.co.uk/wp-content/uploads/2026/01/VMware-NVIDIA-logos_ee2f18dc-615d-4c9e-8f11-9c3c2ce2bf37-prv.png) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Containers](https://jameskilby.co.uk/category/containers/) | [Devops](https://jameskilby.co.uk/category/devops/) | [Homelab](https://jameskilby.co.uk/category/homelab/) | [NVIDIA](https://jameskilby.co.uk/category/nvidia/) | [Traefik](https://jameskilby.co.uk/category/traefik/) | [VMware](https://jameskilby.co.uk/category/vmware/) # Automating the Deployment of my Homelab AI Infrastructure By[James](https://jameskilby.co.uk) February 9, 2026 · Updated July 11, 2026 • 📖17 min read(3,349 words) This post walks through how I rebuilt my homelab AI infrastructure and automated the entire deployment. In a previous [post](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/), I wrote about using my VMware lab with an NVIDIA Tesla P4 for running some AI services. This deployment was done with the P4 GPU in passthrough mode where the entire PCI card was presented into the VM. (I will refer to this as GPU mode). I wanted to take this to the next level. I also wanted to automate most of the steps. This was for a few reasons; firstly, I wanted to get better at automation in general. Secondly, I found the setup brittle and wanted to improve the reliability of deployments. This post will be about using automation to deploy the VM infrastructure required to be able to run AI Workloads. This is something I presented on with my good friend [Gareth](https://www.virtualisedfruit.co.uk/) at the London VMUG. However, in a homelab environment a more straightforward Docker-based setup could be more appropriate. I also decided to bite the bullet and update the graphics card I was using to something a bit more modern and capable. After a bit of searching, I decided on an [NVIDIA A10](https://www.nvidia.com/en-gb/data-center/products/a10-gpu/) This guide covers the full homelab AI infrastructure build — This post is about getting the foundational infrastructure ready. The first part will explain the steps and detail the Ansible configuration. The second part of this post goes into a full deployment utilising [SemaphoreUI](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/). My next post will go into more details on the Docker containers I am using for actually running the AI services, building on these foundations. All of the code is referenced in my GitHub IAC [repository](https://github.com/jameskilbynet/iac/tree/main/ansible) ## Table of Contents ## GPU vs vGPU For anyone not familiar, it may be worth giving an overview of the differences between GPU and vGPU. What I describe as “GPU” is where I am passing the entire Graphics PCI device through to a single VM within vSphere. Using this method has some advantages; firstly, it doesn’t require any special licences or drivers. The entire PCI card gets passed into the VM as is. However, it has some downsides. The two most important ones for me are that it can only be presented to a single VM at a time and that VM cannot be snapshotted while it is turned on. This made backups convoluted. As I was changing configurations a lot this became tedious. I also wanted to be able to pass the card through to multiple VMs vGPU is only officially supported on “datacentre” cards from NVIDIA. It virtualises the graphics card and allows you to share it across multiple VMs in a similar way to what vSphere has done for compute virtualisation for years. It allows you to split the card into some predefined profiles that I have listed below and attach them to multiple virtual machines at the same time. ## Prerequisites There are quite a lot of pre-reqs required to be in place to utilise all the attached deployment playbooks. So it’s worth ensuring that all of these are complete. * Obviously, you need a vSphere Host and vCenter server licensed at least at the enterprise level (I am using Enterprise Plus on 8.0u3) * An Ubuntu VM (Tested with 25.04) referred to as the vGPU VM * An NVIDIA datacenter graphics card and associated Host and Guest drivers, I am using an A10 24GB and using driver version 535.247.011 * NVIDIA vGPU licence. Trials are available [here](https://www.nvidia.com/en-gb/data-center/resources/vgpu-evaluation/) if needed * NFS Server (used for NVIDIA software deployment) * A Domain hosted with Cloudflare and an API [token](http://You can create one at: https://dash.cloudflare.com/profile/api-tokens) with Zone:DNS:Edit permissions. * SSH access to the vGPU VM with SUDO permission * Somewhere to run Ansible playbooks from. * Internal DNS Records created for * vGPU VM * Traefik Dashboard * Test NGINX Server – Optional * NFS Server (Used for vGPU Software install) ## Host Setup To make vGPU work successfully, there are three elements required. The ESXi Driver/VIB, the Guest driver and the NVIDIA licence for the Guest VM. For now, I’m using the ‘535’ version of the driver bundle, which is the LTS version. The Driver bundle and licence must be obtained from NVIDIA however trials are available if you don’t have access. Once the software has been obtained, you need to copy the host driver to a VMware datastore. Then place the host in maintenance mode and install the driver using the command. esxcli software vib install -d /vmfs/volumes/02cb3d2b-457ccb84/nvidia/NVIDIA-GRID-vSphere-8.0-535.247.02-535.247.01-539.28.zip 📋 Copy If it worked successfully, you should get a result like the one below Installation Result Message: Operation finished successfully. VIBs Installed: NVD_bootbank_NVD-VMware_ESXi_8.0.0_Driver_535.247.02-1OEM.800.1.0.20613240 VIBs Removed: VIBs Skipped: Reboot Required: false DPU Results: 📋 Copy I always choose to restart the host after installing the driver. I have been bitten in the past where it says a reboot isn’t required, but it was. Once the host has restarted, ssh into it and validate that the driver is talking to the GPU correctly. This can be done with the nvidia-smi command. This is executed on the ESXi host and will show something similar to the below if it’s working. +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.247.02 Driver Version: 535.247.02 CUDA Version: N/A | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA A10 On | 00000000:03:00.0 Off | 0 | | 0% 41C P8 22W / 150W | 11200MiB / 23028MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ 📋 Copy ### vGPU Profiles If everything is working correctly, you should now be able to see the vGPU profiles in vCenter. Select the VM you want to present the NVIDIA card to. Edit the VM settings and select add PCI device. ![Adding a PCI device for NVIDIA GPU passthrough in vSphere VM settings](https://jameskilby.co.uk/wp-content/uploads/2026/01/Add-PCI-Device-1-402x1024.png) You should then be presented with the available profiles from the NVIDIA GPU ![NVIDIA vGPU profile selection in vSphere for AI homelab workloads](https://jameskilby.co.uk/wp-content/uploads/2026/01/GPU-Profiles-1024x397.png) As can be seen, the NVIDIA A10 supports 18 different profiles. The nvidia_a10 part specifies the card (in case you have multiple), and the suffix determines how much vGPU RAM is allocated to the vGPU instance, along with the features that are exposed with it. I typically use either the a10-24q profile to allow the use of larger AI models. Although I do step this down to the 12q profile to allow support of multiple VMs for testing etc. The NVIDIA A10 supports four separate "Personalities": A, B, Q and C. Each of these comes with different capabilities but also potentially different licensing requirements. Profile| Intended Use| Driver Feature Set| License Tier| Expected Workloads ---|---|---|---|--- A-Series| App Streaming / RDSH| Basic Graphics no pro features| vAPPS| Many Users / Published Apps B-Series| VDI Desktops| Standard Desktop Graphics| vPC| Knowledge Worker Desktops C-Series| Compute Only| CUDA Only | vCompute Server| AI/ML Data Science Q-Series| Full GFX Workstation| Full RTX| RTX vWS| CAD,3D Rendering The keen-eyed amongst you might realise that I am using the wrong profile for AI workloads. I have chosen to use Q-based profiles rather than C-based ones. This is because the A, B and Q profiles are run from one ESXi driver VIB and the C profile requires a different VIB. Because I occasionally want to run a VDI based workload or media conversion in my lab. I have chosen to use the more graphics focused VIB and utilise the Q profile for my AI based workloads. I suspect this might have a performance impact, but it was a compromise I was willing to accept. ## Guest Setup To make the guest setup easier and more repeatable, I have built some Ansible playbooks. These perform a number of activities that make it MUCH easier to get the guest up and running for use with AI workloads and other containers. These have only been tested with Ubuntu but will probably work with other Linux distros without a lot of changes. As a bonus if you want to utilise the Traefik setup I have built this can be used on its own, just with the Deploy Docker and Install Traefik playbooks. Once these playbooks have been run, you will be in a position to spin up any number of Docker containers with GPU access. At a high level, the four playbooks I have built: * [Deploy Docker ](https://github.com/jameskilbynet/iac/blob/main/ansible/docker/install_docker.yml) * [Install NVIDIA Guest drivers](https://github.com/jameskilbynet/iac/blob/main/ansible/vGPU/install_nvidia_drivers.yml) * [Install NVIDIA Container Toolkit](http://ansible/vGPU/install_nvidia_containertoolkit.yml) * [Install Traefik ](https://github.com/jameskilbynet/iac/blob/main/ansible/traefik/traefik_deploy.yml) ### Guest Configuration Before we get to the Ansible section, we obviously need to build the VM. There are a few key steps here as well. Firstly, it must be an EFI-based VM; BIOS won't work. Secondly, you need to allocate a decent amount of RAM to the workload. I usually run with at least 3x my GPU vRAM. Therefore, my main AI VM usually has 96GB allocated to it. It is also important to understand that vSphere will automatically reserve all of this memory. Also, make sure that you are not swapping to disk. This will dramatically slow down the system. The next step is to ensure that the VM has access to a lot of fast storage. I would recommend at least a few hundred GB, especially if you are going to be working with multiple models. The smallest models I typically use are around 8GB. Therefore loading these into memory even from an NVMe drive, can take a few seconds before anything can be processed. This will always give a slight delay when cold-starting compared to a commercial equivalent (Think ChatGPT, etc.) It is possible to configure Ollama to keep the models loaded. I have set this to 1 hour before it unloads them. This is due to higher power consumption on the graphics card when they are loaded. Please note that all of the testing has been done on Ubuntu 24 or 25 ### Homelab AI Infrastructure: Ansible Automation I have kept these as separate playbooks for now. This hopefully makes it easier to follow and/or troubleshoot if needed. The playbooks are intended to be run in order. I am using SemaphoreUI to handle the Ansible deployment but this isn't required. If you are familiar enough with Ansible to not use Semaphore, then you can easily modify these to suit your execution preferences. #### 1. Docker deployment I have covered the deployment of Docker already in [this](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/ "Managing my Homelab with SemaphoreUI") post. No parameters are required to be set for the Docker deployment. Everything is set in the playbook. #### 2. Install NVIDIA Guest Drivers This playbook configures an NFS client on the Ubuntu server and then copies both the vGPU license file and the driver from my NFS storage before installing them. This, I felt, was the best way to have a portable Ansible file without checking in my NVIDIA licence and driver files into Git. Before you use this playbook several variables need to be set in Semaphore. These should be done in the Variable Group section. I created a Variable Group called vGPU for this purpose. ##### NVIDIA Guest Driver Variables **Variable**| **Default**| **Description** ---|---|--- nvidia_vgpu_driver_file| NVIDIA-Linux-x86_64-535.247.01-grid.run| Driver filename nvidia_vgpu_licence_file| client_configuration_token_04-08-2025-16-54-19.tok| Token filename nvidia_nfs_server| nas.jameskilby.cloud| NFS server hostname nvidia_nfs_export_path| /mnt/pool1/ISO/nvidia| NFS export path nvidia_nfs_mount_point| /mnt/iso/nvidia| Local mount point To make your life easier, you can copy this json to Semaphore and just tweak what is needed. { "nvidia_vgpu_driver_file": "NVIDIA-Linux-x86_64-535.247.01-grid.run", "nvidia_vgpu_licence_file": "client_configuration_token_04-08-2025-16-54-19.tok", "nvidia_nfs_server": "nas.jameskilby.cloud", "nvidia_nfs_export_path": "/mnt/pool1/ISO/nvidia", "nvidia_nfs_mount_point": "/mnt/iso/nvidia", } 📋 Copy If everything runs smoothly the last step of the Ansible playbook will show an export of the licence and vGPU configuration. TASK [Show vGPU license info] ************************************************** 9:18:47 PM ok: [blogtest.jameskilby.cloud] => 9:18:47 PM msg: |- 9:18:47 PM ==============NVSMI LOG============== 9:18:47 PM 9:18:47 PM Timestamp : Fri Feb 6 21:18:47 2026 9:18:47 PM Driver Version : 535.247.01 9:18:47 PM CUDA Version : 12.2 9:18:47 PM 9:18:47 PM Attached GPUs : 1 9:18:47 PM GPU 00000000:03:00.0 9:18:47 PM Product Name : NVIDIA A10-12Q 9:18:47 PM Product Brand : NVIDIA RTX Virtual Workstation 9:18:47 PM Product Architecture : Ampere 9:18:47 PM Display Mode : Enabled 9:18:47 PM Display Active : Disabled 9:18:47 PM Persistence Mode : Enabled 9:18:47 PM Addressing Mode : None 9:18:47 PM MIG Mode 9:18:47 PM Current : Disabled 9:18:47 PM Pending : Disabled 9:18:47 PM Accounting Mode : Disabled 9:18:47 PM Accounting Mode Buffer Size : 4000 📋 Copy #### 3\. Install NVIDIA Container Toolkit 📋 Copy This playbook installs the NVIDIA container toolkit and validates it. It does this by running a Docker container and executing the nvidia-smi command from within the container. This sounds trivial but actually is one of the main reasons I made these playbooks. To get the GPU to work in the docker container you have to have a lot of things set up correctly. This requires the correct ESXi Driver, VM driver within the Ubuntu VM, Docker and the Docker container toolkit set up correctly. The correct Kernel extensions, etc., and the licensing is working correctly No additional parameters need to be set to execute it. Below is the expected output showcasing nvidia-smi running from within a Docker container. ![nvidia-smi output confirming homelab AI infrastructure GPU is detected in Ubuntu](https://jameskilby.co.uk/wp-content/uploads/2026/01/nvidia-smi-1024x654.png) #### 4\. Install Traefik For those unfamiliar with [Traefik](https://traefik.io) ( It's pronounced Traffic), it is a reverse proxy/ingress controller with automatic integrations to Docker and Kubernetes. I have it configured to leverage both Let's Encrypt and Cloudflare so that it will automatically obtain signed certificates for me. This is done with Docker labels in such a way that any container I spin up that has the correct Traefik labels will automatically mint a signed certificate and add it to the load balancer. This has the added benefit that I just need to remember a URL i.e. openwebui.jameskilby.cloud configured in my internal DNS, rather than which VM and port the service is running on. It has the added benefit that it makes it very easy for me to expose services externally if needed. I just add the appropriate DNS record on my external DNS. The labels look something like the following. These are added to the individual Docker containers and Traefik can see them through the Docker API. When the containers are started, it will autoconfigure the certificate generation and automatically configure the load balancer. If the container is removed or stopped, it will tidy up the configuration. labels: - "traefik.enable=true" - "traefik.http.routers.open-webui.rule=Host(`openwebui.jameskilby.cloud`)" - "traefik.http.routers.open-webui.entrypoints=websecure" - "traefik.http.routers.open-webui.tls=true" - "traefik.http.routers.open-webui.tls.certresolver=cloudflare" - "traefik.http.services.open-webui.loadbalancer.server.port=8080" 📋 Copy With the above labels added to my OpenWebUI container, Traefik will request a certificate for openwebui.jameskilby.cloud and deploy this as an HTTPS service mapped to the OpenWebUI container on port 8080 I could probably do a whole post just on my setup of Traefik but for now here is an overview diagram. ![Ansible playbook configuring Traefik and Docker on the homelab AI VM](https://jameskilby.co.uk/wp-content/uploads/2025/01/Traefik-Docker-Setup-2-1024x342.png) As part of the included playbook, I have added the option to deploy a simple nginx web server using this principle. If everything is configured correctly, you should be able to connect to this container post deployment and it will have a fully trusted certificate as seen below. (You may need to wait 1-2 minutes or so after deployment for this process to complete) ![Nginx TLS certificate verification via Traefik in the homelab setup](https://jameskilby.co.uk/wp-content/uploads/2026/01/nginx-cert-1024x997.png) #### Traefik Dashboard Traefik also has a nice dashboard that is very useful in troubleshooting. ![Traefik dashboard running in the homelab with Docker and Ansible](https://jameskilby.co.uk/wp-content/uploads/2026/01/TraefikDashboard-1024x478.png) This is the dashboard from my testing configuration. My main server has over 40 "Routers" #### Traefik Playbook Setup Before running this playbook a lot of variables need to be configured as can be seen below. In most cases, the default is OK. I have just extended the vGPU Variable group to do this. When they are all input into Semaphore, it will look something like this. ![Traefik variable definitions in the Ansible playbook for the AI homelab](https://jameskilby.co.uk/wp-content/uploads/2026/01/Traefik-Variables-804x1024.png) Rather than typing all of the values out you can copy the JSON below and then just tweak what you need to. { "nvidia_vgpu_driver_file": "NVIDIA-Linux-x86_64-535.247.01-grid.run", "nvidia_vgpu_licence_file": "client_configuration_token_04-08-2025-16-54-19.tok", "nvidia_nfs_server": "nas.jameskilby.cloud", "nvidia_nfs_export_path": "/mnt/pool1/ISO/nvidia", "nvidia_nfs_mount_point": "/mnt/iso/nvidia", "traefik_deploy_test_service": "true", "traefik_healthcheck_poll_retries": "12", "traefik_healthcheck_poll_delay": "5", "traefik_docker_image": "traefik:v3.6.4", "traefik_name": "traefik", "traefik_config_path": "/etc/traefik", "traefik_acme_path": "/etc/traefik/acme", "traefik_docker_network": "traefik", "traefik_http_port": "80", "traefik_https_port": "443", "acme_dns_delay": "10", "acme_dns_resolvers": "[\"1.1.1.1:53\", \"8.8.8.8:53\"]", "traefik_log_level": "info", "traefik_log_format": "json", "traefik_log_max_size": "10m", "traefik_log_max_files": "3", "traefik_healthcheck_interval": "30s", "traefik_healthcheck_timeout": "10s", "traefik_healthcheck_retries": "3", "traefik_healthcheck_start_period": "30s", "traefik_test_container": "nginx-test", "traefik_test_image": "nginx:alpine", "traefik_test_domain": "test.jameskilby.cloud", "traefik_dashboard_user": "admin", "wildcard_domain": "jameskilby.cloud" } 📋 Copy ##### Traefik Variable Definitions **Variable**| **Default**| **Description** ---|---|--- traefik_deploy_test_service| true| set to false to skip NGINX deployment traefik_healthcheck_poll_retries| 12| Number of health check poll attempts traefik_healthcheck_poll_delay| 5| Seconds between health check polls traefik_docker_image| 3.6.4| Traefik Docker Image and version traefik_name| traefik| Traefik Container Name traefik_config_path| /etc/traefik| Config Directory traefik_acme_path| /etc/traefik/acme| ACME/Cert Directory traefik_docker_network| traefik| Docker Network Name for Traefik traefik_http_port| 80| HTTP Port traefik_https_port| 443| HTTPS Port acme_dns_delay| 10| Seconds before DNS Check acme_dns_resolvers| 1.1.1.1:53, 8.8.8.8:53| DNS Resolvers traefik_log_level| info| Log Level traefik_log_format| json| Log Format traefik_log_max_size| 10m| Max Log Size traefik_log_max_files| 3| Max Log Files traefik_healthcheck_interval| 30s| Health Check Interval traefik_healthcheck_timeout| 10s| Health Check Timeout traefik_healthcheck_retries| 3| Health Check Retries traefik_healthcheck_start_period| 30s| Health Check Grace Period traefik_test_container| nginx-test| Test Container Name traefik_test_image| nginx:alpine| Test Container Image traefik_test_domain| test.jameskilby.cloud| Test Service Domain traefik_dashboard_user| admin| Traefik Dashboard Username wildcard_domain| jameskilby.cloud| Traefik Wildcard Domain #### Ansible Secrets The playbook also needs two values that should be considered sensitive. These can be added to Semaphore as secrets. The two secrets are the Cloudflare API Token and the second is a hash of a password for the admin account used to access the Traefik dashboard. The Traefik dashboard is very useful for troubleshooting any SSL/connectivity issues. ![Ansible secrets configuration in Semaphore for secure homelab automation](https://jameskilby.co.uk/wp-content/uploads/2026/01/SemaphoreSecrets-1024x262.png) The Cloudflare Token needs to be generated in the Cloudflare dashboard. To generate the hash for the admin dashboard password, the easiest way is with Docker. The below is an example that can be run on the command line docker run --rm httpd:2 htpasswd -nbB '' 'your_password_here' 📋 Copy It will download and execute the httpd container. The last line is the hash that you need. ![Apache httpd test response confirming Docker and Traefik are running correctly](https://jameskilby.co.uk/wp-content/uploads/2026/01/httpd-1024x383.png) In this case, the hash is: $2y$05$nkQGI2UxnRY.7O.6k14naOJPjslbqOT5vpqZPmXMu4knhBOH1EUAq take the output from this output and add it to the Semaphore secure variable for "dashboard_admin_password_hash" ## Semaphore End-to-End Setup Assuming you are going to use SemaphoreUI to execute the playbooks, these are the steps you will need to take. If you haven't already set it up, review my guide [here](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/) ### Repository The first step is to point Semaphore at the Git repository. This is the location where the playbooks will be executed from. As my Git repo is public no authentication is required. You also need to specify the branch; in this case, I am using main. ![Adding a Git repository to Semaphore for Ansible homelab automation](https://jameskilby.co.uk/wp-content/uploads/2026/01/Semaphore_repo.png) ### Key Store The next step is to add the Key Store items. This is used to define the authentication from Semaphore to the target workload or other systems. I do this in two parts. The first is standard authentication, which I do with an SSH key. The second part is defining the become password to allow Semaphore to execute SUDO commands. I do this with a password stored in the Key Store. I have called these two methods KeyAuth and PassAuth ![Creating a new SSH key in Semaphore for Ansible authentication](https://jameskilby.co.uk/wp-content/uploads/2026/01/NewKey-802x1024.png) Once the Key authentication is added, the next step is to add a different key store item for when the playbook needs to "Become" ![Semaphore password authentication configuration for Ansible inventory access](https://jameskilby.co.uk/wp-content/uploads/2026/01/PassAuth.png) An alternative approach is to allow your user to elevate without confirmation. ### Inventory Once the authentication methods are defined, the next step is to update the Semaphore Inventory. I have created a new Inventory item called "vGPU" and set the User credentials to be KeyAuth and the Sudo credentials as PassAuth as created above. I've then added the specific VM in the inventory. For testing, I have called this blogtest. ![Ansible inventory file showing homelab host configuration in Semaphore](https://jameskilby.co.uk/wp-content/uploads/2026/01/blogtest_ansible_inventory-670x1024.png) ### Variable Group The next step is to create a variable group and the required variables. Review the variable table above and set it to match your environment. This is what mine looks like: ![Semaphore variable group configuration for Ansible homelab automation](https://jameskilby.co.uk/wp-content/uploads/2026/01/VariableGroup-813x1024.png) ### Task Templates The final step is to create the four task templates. These are the actual actions that Ansible will perform using the Inventory, Repository and Variable Groups we have just defined. Select New Template from the "Task Templates" view. Ensure it is set to Ansible and configure as per below. ![Expanded Semaphore variable group showing all Ansible environment variables](https://jameskilby.co.uk/wp-content/uploads/2026/01/Variable-Group-1024x547.png) You should end up with something looking like this ![Semaphore task templates for running Ansible playbooks in the homelab](https://jameskilby.co.uk/wp-content/uploads/2026/01/TaskTemplates-1024x263.png) When they are all configured, press the play button to execute each of the given tasks against your inventory item. Semaphore will check out the Ansible playbook and execute it, showing the results in the window. ## Summary When all the playbooks are run successfully, you should now have: * A vSphere host capable of passing through NVIDIA datacenter graphics * An Ubuntu VM with vGPU integrated with Docker This is now ready for you to start deploying Docker-based AI workloads onto. 📋 Copy ## 📚 Related Posts * [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) * [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) ## Similar Posts * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Veeam](https://jameskilby.co.uk/category/veeam/) ### [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated June 1, 2026 My lab is always undergoing change. Partially as I want to try new things or new ways of doing things. * [ ![Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/wp-content/uploads/2023/04/intel_optane_ssd_900p_series_aic_-_right_angle_575px.png) ](https://jameskilby.co.uk/2023/04/intel-optane/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) By[James](https://jameskilby.co.uk) April 17, 2023 · Updated June 1, 2026 I have been a VMware vExpert for many years and it has brought me many benefits over the years. * [ ![New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/wp-content/uploads/2024/07/IMG_6629-768x149.jpeg) ](https://jameskilby.co.uk/2024/07/new-nodes/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) By[James](https://jameskilby.co.uk) July 2, 2024 · Updated July 11, 2026 I recently decided to update some of my homelab hosts and I managed to do this at very little cost by offloading 2 of my Supermicro e200’s to fellow vExpert Paul . * [ ![Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/wp-content/uploads/2023/04/81-ZoEW24UL._SL1500_-768x461.jpg) ](https://jameskilby.co.uk/2019/02/lab-storage-2/) [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/2019/02/lab-storage-2/) By[James](https://jameskilby.co.uk) February 10, 2019 · Updated June 5, 2026 Since starting my new role with Xtravirt my Homelab has gone through several fairly significant changes. * [ ![VMware – Going out with a Bang!](https://jameskilby.co.uk/wp-content/uploads/2023/10/rnli-logo-768x384.png) ](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [VMware – Going out with a Bang!](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) By[James](https://jameskilby.co.uk) October 7, 2023 · Updated June 1, 2026 There is a lot of uncertainty with VMware at the moment. This is all due to the pending acquisition by Broadcom. * [ ![Wa](https://jameskilby.co.uk/wp-content/uploads/2025/04/210902461-012e7273-413a-4ec7-be44-e854347f5a21-768x180.png) ](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Warp – The intelligent terminal](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) By[James](https://jameskilby.co.uk) April 11, 2025 · Updated June 1, 2026 How Warp is helping me run my homelab. --- # Using Content Libraries in VMC to deploy software faster Source: https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/ Date: 2026-01-27 Categories: VMware, VMware Cloud on AWS, Homelab, Networking, Personal, VCF, Storage, Ansible, Automation ![](https://jameskilby.co.uk/wp-content/uploads/2026/01/Firefly_Gemini-Flash.png) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) # Using Content Libraries in VMC to deploy software faster By[James](https://jameskilby.co.uk) January 27, 2026 · Updated June 5, 2026 • 📖5 min read(997 words) This post shows how I use Content Libraries in VMware Cloud on AWS (VMC) to deploy software to new SDDCs far faster. As part of my role spinning up new SDDC’s to test things is quite a common occurrence. This is both a blessing and a curse. The new SDDC is 100% Vanilla and perfectly self contained. Therefore you can do the testing required, knowing that you won’t impact anything else and you’re not inheriting a legacy setting from previous testing. However the downside is you need to get the configuration and the software you require into the SDDC. This can take some time. For configuration it’s possible to use IAC tools like Terraform. To speed up the process of deploying software I decided to leverage Content Library specifically for this task. This is a much faster and more reliable way to get what I need into the SDDC. William Lam has previously [written ](https://williamlam.com/2018/07/creating-a-vsphere-content-library-directly-on-amazon-s3.html#google_vignette)about ways of doing this by utilising AWS S3 as the backend ## Table of Contents ## Prerequisites * **On-premises vCenter** — a local vCenter instance to act as the publishing source. Any version supporting Content Library publish/subscribe (vSphere 6.0+) will work. * **VMware Cloud on AWS SDDC** — the destination environment. The subscribing Content Library is created here. * **Network connectivity between source and destination** — on-premises vCenter must be able to reach the VMC management network over HTTPS (TCP 443). Typically achieved via a Direct Connect or VPN connection to the VMC Management CIDR. * **Firewall rules** — outbound TCP 443 from on-prem vCenter to the VMC vCenter IP, and the reverse for subscription synchronisation. Check your NSX gateway firewall in the VMC SDDC. * **Storage capacity in VMC** — the Content Library occupies space on the VMC datastore. Factor in the size of your ISOs and OVF templates before enabling immediate sync. * **vCenter credentials** — a user account with the _Content Library > Create local library_ and _Content Library > Create subscribed library_ privileges on both vCenter instances. ## Introduction For those unfamiliar with Content Libraries they are a storage medium for storing templates and ISO files for easy use within a vCentre environment or sharing with other vCentres. You can see some of the recent updates to this feature [here](https://blogs.vmware.com/cloud-foundation/2020/01/22/creating-and-using-content-library/) ## Networking For this to work you need network connectivity between the source and destination vCentre. For my use case I use my on premises vCentre as the source and VMC as the destination. However you can use VMC as the source if preferable. ![Content Library Flow](https://jameskilby.co.uk/wp-content/uploads/2026/01/ContentLibraryFlow-1024x358.png) Usually I don’t utilise/DX for this although it will absolutely work over those connections and would be purely private IP I have a NAT rule on one of my public IP’s on my Onprem Watchguard Firewall to connect through to the Onprem vCentre. The URL of my vCentre will publicly resolve to this IP. I have restricted this connection to only allow from the public IP of the VMC vCentre. For the VMC vCentre there shouldn’t be any changes required as outbound https is allowed by default on the management gateway. ### Create Publishing Content Library I already use content libraries extensively in my lab and didn’t need everything uploading to VMC. So I created a new Library just for this purpose and added some select software. If you don’t have a content library or need to create another one. These are the steps you will need to follow. To create a new Library from the vSphere Client from the top menu select **Content Libraries** ![Content Libary](https://jameskilby.co.uk/wp-content/uploads/2026/01/ContentLibary-1024x368.png) Name the Content Library appropriately and add a description. ![Name Location](https://jameskilby.co.uk/wp-content/uploads/2026/01/NameLocation-1024x440.png) Configure the Content Library As this will be the source Content Library we will set this to be a local content library and then enable the publishing feature to allow other vCentre’s to subscribe to this. I choose not to use authentication for this purpose as there is nothing sensitive ![Configure Content Library](https://jameskilby.co.uk/wp-content/uploads/2026/01/ConfigureContentLibrary-1024x631.png) The next step is to define what Datastore the Content Library will reside on. Here I am using my datastore called “ISO” as it saves my valuable NVMe datastore for workloads. ![Select Storage](https://jameskilby.co.uk/wp-content/uploads/2026/01/Select-Storage-1024x481.png) Ready to complete ![Ready To Complete](https://jameskilby.co.uk/wp-content/uploads/2026/01/ReadyToComplete-1024x507.png) Complete the creation and then go back into the Content Library settings. It will show you a Subscription URL which we will need later. Take a copy of the full Subscription URL ![Complete With Url](https://jameskilby.co.uk/wp-content/uploads/2026/01/CompleteWithUrl-1024x558.png) ### Create Subscribing Content Library We need to repeat the procedure in VMC however now we need to create a Subscribed Content Library. Input the URL that we saved earlier ![VMC Subscriber](https://jameskilby.co.uk/wp-content/uploads/2026/01/VMCSubscriber-1024x906.png) ## Download content option There are two options for how you want the Subscribed Content Library to behave on first connection/new content. The first is to synchronise the content as soon as possible. This is what I use. However if you have a large catalogue and not all items are required in the VMC vCentre an option to save storage would be to only download content on demand. The downside to this option is that deployment would take longer, waiting for the item to synchronise before it can be used. ## Conclusion Content Libraries are one of those VMware features that solve a real operational pain point without much ceremony. Once the publishing/subscribing relationship is established between your on-premises vCenter and VMC, getting ISOs and OVF templates into a fresh SDDC drops from a manual upload process to a few clicks — or nothing at all if you enable immediate synchronisation. The main trade-off to consider is storage vs. speed. Immediate sync keeps everything local and ready to deploy instantly, but it consumes datastore space even for templates you may never use in a given SDDC. On-demand download is more storage-efficient and works well when you spin up SDDCs infrequently, though the first deployment of any template will wait for the download to complete. For testing environments where time matters, I lean towards immediate sync for the templates I know I’ll use frequently, and on-demand for everything else. If you’re already using Content Libraries on-premises, adding VMC as a subscriber takes under ten minutes. If you’re not using them yet, this is a good reason to start. ## 📚 Related Posts * [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) * [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) * [VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) ## Similar Posts * [ ![MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/wp-content/uploads/2023/04/2157_hi_res-768x346.png) ](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) By[James](https://jameskilby.co.uk) December 19, 2022 · Updated June 1, 2026 For a while, I’ve been looking to update the networking at the core of my homelab. * [ ![Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) ](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal. * [ ![VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/wp-content/uploads/2024/01/40oOd8IipPvtrPJs-1198788743-768x737.jpg) ](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) [VCF](https://jameskilby.co.uk/category/vmware/vcf/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) By[James](https://jameskilby.co.uk) January 18, 2024 · Updated June 1, 2026 How to deploy Holodeck with Legacy CPU’s * [ ![Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/wp-content/uploads/2023/04/intel_optane_ssd_900p_series_aic_-_right_angle_575px.png) ](https://jameskilby.co.uk/2023/04/intel-optane/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) By[James](https://jameskilby.co.uk) April 17, 2023 · Updated June 1, 2026 I have been a VMware vExpert for many years and it has brought me many benefits over the years. * [ ![Automated VCF 9 Offline Depot architecture diagram showing Traefik reverse proxy and Nginx file server stack](https://jameskilby.co.uk/wp-content/uploads/2026/04/offlinedepot.png) ](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) By[James](https://jameskilby.co.uk) April 10, 2026 · Updated July 11, 2026 One Bash script turns a fresh Ubuntu VM into a VCF 9 Offline Depot: Traefik, Nginx, basic auth, and Let’s Encrypt wildcard certs via Cloudflare DNS. * [ ![vSphere Power Management Ansible Playbooks with Semaphore](https://jameskilby.co.uk/wp-content/uploads/2026/04/vsphere-power-management-ansible-768x403.png) ](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automating vSphere Power Management driven by Ansible and SemaphoreUI](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) By[James](https://jameskilby.co.uk) April 15, 2026 · Updated June 1, 2026 In this post I’ll walk through how I use vSphere Power Management driven by Ansible and SemaphoreUI to automatically reduce ESXi host electricity consumption — saving real money on my Octopus Agile tariff by toggling hosts between Low Power and Balanced policies. Introduction One of the larger costs of running my homelab is the electricity…. --- # Blog Performance & SEO Improvements: Cloudflare, Privacy & More Source: https://jameskilby.co.uk/2026/01/web-development-improvements/ Date: 2026-01-15 Categories: Cloudflare, Hosting, Wordpress, Ansible, Automation, Devops ![](https://jameskilby.co.uk/wp-content/uploads/2026/01/Website-Optimisations.png) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) | [Wordpress](https://jameskilby.co.uk/category/wordpress/) # Blog Performance & SEO Improvements: Cloudflare, Privacy & More By[James](https://jameskilby.co.uk) January 15, 2026 · Updated July 11, 2026 • 📖8 min read(1,602 words) I have spent the Christmas break making some improvements to this blog. A lot of these are in “the backend”. These help improve the performance, privacy, SEO, and I have also added some security best practices. Most of these changes were done more as an exercise than due to a specific requirement. I also had a HUGE amount of help with some of these steps from a variety of AI tools. However [Warp](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) did the bulk of the heavy lifting. The repo behind the content of this site as well as the implementation is [public](https://github.com/jameskilbynet/jkcoukblog) in case you want to look deeper/steal anything. Before we dig into the details it’s probably worth briefly summarising what is going on. ![Wordpress Authoring](https://jameskilby.co.uk/wp-content/uploads/2025/10/Wordpress-Authoring-2-1024x677.png) ## Solution Overview Below is a condensed view of what the GitHub runner does. * Connect to WordPress REST API * Discover all posts, pages, categories, and tags * Downloads the relevant HTML, CSS, JS, images, and fonts * Replaces WordPress URLs from my authoring instance “WordPress.jameskilby.cloud” to the live site jameskilby.co.uk * Processes WordPress embeds (Acast, YouTube, Vimeo, Twitter) as seen on the media page. * Injects Plausible Analytics tracking code * Corrects the theme * Generates sitemap and redirects file ## Table of Contents ## Performance I have split the performance section into two main components. The first is around making the public site faster. This is a critical component. I want the site to be fast for me and others. It obviously helps with site rankings as well. It was also an exercise in improving my understanding of web development and relevant optimisations. ### Measure Site Performance with Lighthouse To understand site performance, first you need to measure it. There are great tools like Gtmetrix.com that will generate a [report](https://gtmetrix.com/reports/jameskilby.co.uk/ADjhUqtL/) on your site performance. I decided to take this a step further. I have deployed a GitHub action that watches commits to the underlying repo. When anything changes in the public directory (i.e. the public facing website) the action is triggered. This action waits for 2 minutes for the public site to finish the deployment to Cloudflare Pages. It then runs the Google Lighthouse performance tests. Grabs the data and populates it in a few locations. This includes a slack message as shown below. It updates the stats page on the blog. Plus a full detailed report is available in GitHub. The full report is approximately 18MB and is available for 30 days. ![Slack Lighthouse Perfomance](https://jameskilby.co.uk/wp-content/uploads/2026/01/Slack-Lighthouse-Perfomance.png) Lighthouse measures the following metrics * Performance Score * Accessibility Score * Best Practices Score * SEO Score * First Contentful Paint (FCP) * Largest Contentful Paint (LCP) * Time to Interactive (TTI) * Cumulative Layout Shift (CLS) Once you have a sensible baseline for your site you can start optimising it using the included recommendations. Below I have listed some of the improvements I have made. ### Site Performance Improvements #### Brotli Compression Brotli is a modern compression algorithm and can achieve better compression ratios than gzip, therefore it makes it a good candidate for modern websites. This is especially true for mobile devices where connection speed can be constrained. Brotli typically gets an extra 15-25% compression above what gzip can offer. I have added a step to the deployment runner to compress the static files with Brotli i.e. html, css and javascript etc. This produced some significant space savings: The below stats are taken from the last run. Original size: 6,894,868 bytes (6.58 MB) Compressed size: 1,193,382 bytes (1.14 MB) Space saved: 5,701,486 bytes (5.44 MB) Average compression: 82.7% 📋 Copy The job is set up to compress the files and upload both the Brotli compressed files and the standard ones. Cloudflare will serve Brotli assuming the end client supports it or fallback to legacy if needed for outdated browsers. #### Local Font Hosting Prior to this change I was using [Google fonts](https://fonts.google.com) however in my aim to be more privacy focused and also improve the site performance the deployment job now swaps these out for local fonts (served from the jameskilby.co.uk domain within Cloudflare) This means that fewer DNS lookups are required for the site and Google is not aware of site visits from analysing requests for fonts. #### Lazy Loading Lazy loading is applied to the images on the page “below the fold”. This allowed for anything that needed scrolling to be viewed to be loaded in the background. #### DNS Prefetch and Connection Most of the website content resides on jameskilby.co.uk served directly from Cloudflare however the analytics run by plausible is on a separate domain plausible.jameskilby.cloud This resides on a dedicated VM in my lab. I have created a step in the deployment workflow to DNS prefetch for plausible.jameskilby.cloud and preconnect for plausible.jameskilby.cloud. I have also made this an async connection so that the loading of the stats doesn’t impact the site loading performance. This change had to be excluded from the Cloudflare Rocket processing using data-cfasync=”false” #### Extract inline CSS to external file Doing this reduces the size of the individual HTML files. Rather than having all of the CSS in every HTML file the CSS is split out into a dedicated file. This also allows the CSS file to be reutilised. #### Image Optimisation I have added steps to optimise the images on the site using AVIF. I then have a script to modify the WordPress HTML Image element with a picture element. This enables serving multiple file types where the source browser can choose. AVIF has good modern support and excellent compression ratios. However it makes sense to have a backup available just in case. One of the downsides of the AVIF format is the encoding can be quite slow. However the workflow creates a MD5 of the source images and creates a cache and therefore will only optimise new or changed images. This took a long time to get right; the issues were mainly down to ordering problems in the execution. #### Favicon One of the steps I did during optimisation is provide one of my AI [tools ](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/)the har file of my site. I then asked it to suggest recommendations. One of them was that the Favicon was very large. Initially I did not understand this. On further digging it correctly identified that the issue is that /favicon.ico is returning the full HTML homepage instead of an actual icon file! This is a Cloudflare Pages configuration issue. As Cloudflare will serve the homepage rather than a 404 error message. It then helped me create an optimised version from the logo that was 1.6KB rather than 200+ KB that was initially being served. ### GitHub Runner Performance Improvements I have made a number of changes in the runner that GitHub executes the deployment. Some of these are to improve the performance of the deployment run as it had got quite lengthy. At its worst it was taking 24 minutes to fully execute. However with the below changes, this has been brought down to approximately 3-4 minutes. This is despite the fact that additional steps have been added. ## Incremental Build The Incremental build step stores metadata about posts, pages, and assets this includes hashes of the content the last modified time and the last build timestamps When querying the WordPress API for changes it uses the modified_after parameter to fetch only content changed since the last build before building the site. ## Secret Scanning As the site is basically in GitOps fashion. I decided to add some best practices into the workflow. One of them is “Secret Scanning”. This is implemented in two places. ### Pre-Commit The first part is done as a pre-commit this stops me committing anything classed as a secret into the repo in the first place. ### Github Action This action scans entire repository history for secrets and sends a slack alert if a secret is found. I created a test file and checked it in to Git # Test configuration file for secret scanning workflow # This file contains a FAKE secret for testing purposes only # Fake API Key (for testing secret detection) API_KEY=sk_test_51ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnop1234567890 # Fake AWS Access Key (for testing) AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE # Fake GitHub Token (for testing) GITHUB_TOKEN=ghp_1234567890abcdefghijklmnopqrstuvwxyzABCD # Fake Database Password (for testing) DATABASE_PASSWORD=super_secret_password_123456 # Note: These are all FAKE credentials for testing the secret scanning workflow # They are not real and cannot be used to access any actual services 📋 Copy ![Secret Scanning Slack Alert](https://jameskilby.co.uk/wp-content/uploads/2026/01/Secret-Scanning-Slack-Alert-1024x396.png) I have configured the gitleaks.toml to exclude the “public” folder within the repo as I may show example usernames/passwords in blogposts. ## Content Improvements Below are some of the changes that I have introduced that are visible to site users. ### Freshness Indicator The deployment wizard also adds a freshness indicator to each page i.e. (Published: January 06, 2018, Updated: July 10, 2024). This is injected at the top of the page, it is only injected if there have been multiple versions of the site. ### Removes WordPress footer fluff Anything not needed for the static site is removed. ### Copy Buttons For every code block on the site a copy function is injected as seen below Example Code Block 📋 Copy ### Theme Injection The WordPress theme I’m running on the authoring site is just a vanilla site. The runner updates the CSS/theme at build time inspired from ### SEO To help with the SEO the runner builds at execution: * Sitemap Generator * RSS Feed * Robots.txt * Security header ## Markdown The last thing the runner has been configured to do is to create a Markdown version of each post. In the AI world we are living in this makes sense for them to consume in this function. A step is also included to create a link to the Markdown version at the very bottom of each page. ## Change Log and Stats Lastly we generate the two public pages to give visibility on the above. [https://jameskilby.co.uk/change](https://jameskilby.co.uk/changelog) ## 📚 Related Posts * [How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare](https://jameskilby.co.uk/2025/10/how-i-deploy-my-blog-as-a-static-website-with-github-actions-and-cloudflare/) * [WordPress Hosting with Cloudflare Pages](https://jameskilby.co.uk/2023/05/how-to-take-a-wordpress-site-and-publish-it-as-a-static-site-on-cloudflare-pages/) * [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) ## Similar Posts * [ ![Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2022/01/web-development/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) ### [Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/2022/01/web-development/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 A while ago I started messing with Cloudflare Workers. I have now moved this site permanently over to them. * [ ![Automated VCF 9 Offline Depot architecture diagram showing Traefik reverse proxy and Nginx file server stack](https://jameskilby.co.uk/wp-content/uploads/2026/04/offlinedepot.png) ](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) By[James](https://jameskilby.co.uk) April 10, 2026 · Updated July 11, 2026 One Bash script turns a fresh Ubuntu VM into a VCF 9 Offline Depot: Traefik, Nginx, basic auth, and Let’s Encrypt wildcard certs via Cloudflare DNS. * [ ![What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2018/03/cloudflare/) [Hosting](https://jameskilby.co.uk/category/hosting/) ### [What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained](https://jameskilby.co.uk/2018/03/cloudflare/) By[James](https://jameskilby.co.uk) March 27, 2018 · Updated June 1, 2026 Cloudflare – What is it and why would I care? I have been using Cloudflare for a long time. * [ ![How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare](https://jameskilby.co.uk/wp-content/uploads/2025/10/Github-Actions.webp) ](https://jameskilby.co.uk/2025/10/how-i-deploy-my-blog-as-a-static-website-with-github-actions-and-cloudflare/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Devops](https://jameskilby.co.uk/category/devops/) ### [How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare](https://jameskilby.co.uk/2025/10/how-i-deploy-my-blog-as-a-static-website-with-github-actions-and-cloudflare/) By[James](https://jameskilby.co.uk) October 6, 2025 · Updated July 11, 2026 I wanted to automate the publishing of my blog from the authoring side to the public side. These are some of the improvements I made. * [ ![Fixing Wrangler Node.js Version Conflicts After Brew Upgrade](https://jameskilby.co.uk/wp-content/uploads/2022/01/WranglerCrab-1-768x256.png) ](https://jameskilby.co.uk/2022/01/wrangler-and-node-versions/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) ### [Fixing Wrangler Node.js Version Conflicts After Brew Upgrade](https://jameskilby.co.uk/2022/01/wrangler-and-node-versions/) By[James](https://jameskilby.co.uk) January 15, 2022 · Updated July 11, 2026 I am a massive fan of the brew package management system for macOS and use it on all of my Macs. I typically just upgrade everything blindly and have never had an issue. * [ ![Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu-768x450.jpeg) ](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) By[James](https://jameskilby.co.uk) October 20, 2022 · Updated July 11, 2026 For a while now I have been running this site directly from Cloudflare utilising their excellent worker’s product. --- # How to Expand Ubuntu Disk Space: LVM pvresize Step-by-Step Source: https://jameskilby.co.uk/2025/12/ubuntu-disk-expansion-steps/ Date: 2025-12-15 Categories: Ubuntu, Ansible, Automation, Github ![](https://jameskilby.co.uk/wp-content/uploads/2025/12/UbuntuExpand.png) [Ubuntu](https://jameskilby.co.uk/category/ubuntu/) # How to Expand Ubuntu Disk Space: LVM pvresize Step-by-Step By[James](https://jameskilby.co.uk) December 15, 2025 · Updated May 24, 2026 • 📖1 min read(91 words) How to expand disks from the command line in Ubuntu. This is something I do fairly frequently, and I can never remember the steps. So I decided to write them down. ## Table of Contents ## 1\. Rescan for disk size changes sudo tee /sys/class/block/sda/device/rescan ## 2\. Extend the physical partition (if needed) sudo growpart /dev/sda 3 ## 3\. Resize the physical volume sudo pvresize /dev/sda3 ## 4\. Extend the logical volume to use all available space sudo lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv ## 5\. Resize the filesystem sudo resize2fs /dev/ubuntu-vg/ubuntu-lv ## 6\. Verify the new size df -h ## 📚 Related Posts * [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) ## Similar Posts * [ ![Automated VCF 9 Offline Depot architecture diagram showing Traefik reverse proxy and Nginx file server stack](https://jameskilby.co.uk/wp-content/uploads/2026/04/offlinedepot.png) ](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) By[James](https://jameskilby.co.uk) April 10, 2026 · Updated July 11, 2026 One Bash script turns a fresh Ubuntu VM into a VCF 9 Offline Depot: Traefik, Nginx, basic auth, and Let’s Encrypt wildcard certs via Cloudflare DNS. * [ ![Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/wp-content/uploads/2026/04/packer-github-actions-vsphere-pipeline-768x452.png) ](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) [Automation](https://jameskilby.co.uk/category/automation/) | [Github](https://jameskilby.co.uk/category/github/) ### [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) By[James](https://jameskilby.co.uk) April 30, 2026 · Updated June 5, 2026 A walk-through of an automated golden-image pipeline for vSphere: HashiCorp Packer builds six Ubuntu LTS templates (22.04, 24.04, 26.04 — server and desktop) driven by three GitHub Actions workflows covering PR validation, parallel template builds, and ISO management via a self-hosted runner. --- # VMware Cloud on AWS Time Sync & NTP Configuration Source: https://jameskilby.co.uk/2025/12/time-in-a-vmc-environment/ Date: 2025-12-08 Categories: VMware Cloud on AWS, VMware ![](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339.png) [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) # VMware Cloud on AWS Time Sync & NTP Configuration By[James](https://jameskilby.co.uk) December 8, 2025 · Updated July 11, 2026 • 📖3 min read(614 words) This post covers how Time Sync works in VMware Cloud on AWS (VMC), using the [Amazon Time Sync Service](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html) to keep accurate NTP across your SDDC. One of the nice things about the VMC Service is that you don’t have to worry about a number of the traditional infrastructure services that you typically obsess over when you’re running your own infrastructure. One of those is Time — a key requirement for any enterprise platform. ## Table of Contents ## What is the Amazon Time Sync Service? The Amazon Time Sync Service is a highly accurate and reliable time source provided by AWS. It uses a fleet of redundant satellite-connected and atomic reference clocks to deliver Coordinated Universal Time (UTC) to AWS services and resources. In a VMware Cloud on AWS environment, the VMC infrastructure itself uses this service, and guest workloads can also be configured to use it directly. The service is accessible at the link-local IP address **169.254.169.123** over UDP port 123. This is the same address used by standard EC2 instances and is always available within the AWS network — no internet access required. ## Time VMC allows you to utilise the Amazon Time Sync Service for keeping an accurate and precise time. Because VMware Cloud on AWS is a fully managed service, the ESXi hosts are already configured to synchronise time correctly. However, for guest VMs running inside the VMC environment, you will want to ensure they are also pointed at a reliable time source rather than relying on VMware Tools time sync alone. For Windows VMs, configure the NTP server using the Windows Time Service (w32tm). For Linux VMs, update your chrony or ntpd configuration to point to `169.254.169.123`. This provides a consistent, low-latency time source that stays local to the AWS infrastructure rather than traversing your network back to an on-premises NTP server. ## Firewall Config To utilise the Amazon Time Sync Service from a VMC guest perspective, the appropriate NSX-T firewall rules need to be in place. On the Compute Gateway, you need to allow UDP port 123 outbound to **169.254.169.123**. The IP address 169.254.169.123 is part of the link-local address range 169.254.0.0/16. It is worth noting that .123 was deliberately chosen to match the NTP port number — a neat detail from the AWS team. To create the rule: in the VMC Console navigate to Networking & Security → Gateway Firewall → Compute Gateway. Add a rule allowing UDP port 123 from your VM segment to destination 169.254.169.123. Without this rule, guest VMs will be blocked from reaching the time sync endpoint. ## Troubleshooting If time sync is not working for your guest VMs in VMware Cloud on AWS, check the following: * Verify the Compute Gateway firewall rule allows UDP 123 to 169.254.169.123 * Confirm the NTP client service is running inside the guest VM * Check that the guest OS NTP configuration points to 169.254.169.123 * If using a Route-based VPN with the default route on-premises, the link-local address will not be reachable over the VPN — use an alternative NTP server in that configuration ## Summary VMware Cloud on AWS takes care of many traditional infrastructure concerns for you, and time sync is a great example. By leveraging the Amazon Time Sync Service at 169.254.169.123, you get a highly accurate, low-latency NTP source that is always available within the AWS network without any complex configuration. The only requirement on your part is ensuring the correct NSX-T Compute Gateway firewall rule is in place to allow UDP port 123 from your guest VMs to reach that link-local address. Once that is done, your workloads will have reliable, accurate time. For another VMC time-saver, see my post on [using Content Libraries in VMC](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/). ## 📚 Related Posts * [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) * [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) * [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) ## Similar Posts * [ ![vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) By[James](https://jameskilby.co.uk) November 17, 2023 · Updated June 1, 2026 An Overview of vSAN ESA in VMC * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/09/vmc-host-errors/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) By[James](https://jameskilby.co.uk) September 15, 2020 · Updated June 5, 2026 Learn how host failures are handled within VMC * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/07/i3en/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [VMware Cloud on AWS i3en Host: Specs, Storage & Performance](https://jameskilby.co.uk/2020/07/i3en/) By[James](https://jameskilby.co.uk) July 2, 2020 · Updated June 5, 2026 VMware Cloud on AWS (VMC) has introduced a new host to its lineup: the “i3en”. This is based on the i3en. * [ ![Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/wp-content/uploads/2026/01/Firefly_Gemini-Flash-768x417.png) ](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) By[James](https://jameskilby.co.uk) January 27, 2026 · Updated June 5, 2026 How to leverage Content Libraries to deploy into VMware Cloud on AWS faster. * [ ![VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) By[James](https://jameskilby.co.uk) May 21, 2025 · Updated July 11, 2026 Quick reference guide to the available storage resources that you get in VMware Cloud on AWS * [ ![An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) By[James](https://jameskilby.co.uk) August 14, 2025 · Updated July 11, 2026 This is single page intended to collate every single feature of the current VMware Cloud on AWS hosts for easy comparison. --- # vSAN Cluster Shutdown – Orchestration Source: https://jameskilby.co.uk/2025/12/vsan-cluster-shutdown/ Date: 2025-12-06 Categories: VMware, vSAN, Automation, Homelab, Artificial Intelligence, Personal, VMware Cloud on AWS ![](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) [VMware](https://jameskilby.co.uk/category/vmware/) | [vSAN](https://jameskilby.co.uk/category/vmware/vsan-vmware/) # vSAN Cluster Shutdown – Orchestration By[James](https://jameskilby.co.uk) December 6, 2025 · Updated June 1, 2026 • 📖1 min read(112 words) As can be seen on my Lab Overview page, I have a GPU/Management cluster that is usually running 24×7. I also have a 3x Node vSAN cluster that I use when I need more resources (or specifically if I need vSAN to test with) This allows me to See the overview below ![V Sphere Overview](https://jameskilby.co.uk/wp-content/uploads/2024/12/vSphere-Overview-1024x530.png) To shut down a vSAN cluster, there is a nice workflow built into vCenter that will orchestrate all the required actions for you ![V SAN Services Menu](https://jameskilby.co.uk/wp-content/uploads/2025/04/vSAN-Services-Menu-1024x564.png) ![Shutdown Precheck](https://jameskilby.co.uk/wp-content/uploads/2025/04/Shutdown-Precheck.png) ![Confirmation](https://jameskilby.co.uk/wp-content/uploads/2025/04/Confirmation.png) And there you have it… To restart the cluster, ensure all the hosts are booted. Leave them in maintenance mode and then select “Restart vSAN Services”. The workflow will bring them back into operation. ![Restart V SAN](https://jameskilby.co.uk/wp-content/uploads/2025/03/Restart-vSAN.png)Restart vSAN Services ## 📚 Related Posts * [How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) * [vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) ## Similar Posts * [ ![Template Deployment with Packer](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) ](https://jameskilby.co.uk/2021/01/hashicorp-packer/) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) By[James](https://jameskilby.co.uk) March 27, 2026 · Updated May 31, 2026 A walkthrough of my self-hosted AI stack: Ollama, Open WebUI, ComfyUI, Whishper, n8n, Qdrant, SearxNG, and a full observability layer — all running on my own hardware with Docker Compose. * [ ![VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/wp-content/uploads/2020/09/vmware_SP_HCI20.png) ](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) [Personal](https://jameskilby.co.uk/category/personal/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) By[James](https://jameskilby.co.uk) September 13, 2020 · Updated May 31, 2026 I recently sat (and passed the VMware HCI Master Specialist exam (5V0-21. * [ ![vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) By[James](https://jameskilby.co.uk) November 17, 2023 · Updated June 1, 2026 An Overview of vSAN ESA in VMC * [ ![An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) By[James](https://jameskilby.co.uk) August 14, 2025 · Updated July 11, 2026 This is single page intended to collate every single feature of the current VMware Cloud on AWS hosts for easy comparison. * [ ![New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) ](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) By[James](https://jameskilby.co.uk) April 1, 2026 · Updated July 11, 2026 We’ve expanded the VMC fleet with the new i7i (i7i. --- # How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare Source: https://jameskilby.co.uk/2025/10/how-i-deploy-my-blog-as-a-static-website-with-github-actions-and-cloudflare/ Date: 2025-10-06 Categories: Cloudflare, Devops, Github, Wordpress, Ansible, Artificial Intelligence, Hosting, Automation, Personal ![](https://jameskilby.co.uk/wp-content/uploads/2025/10/Github-Actions.webp) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Devops](https://jameskilby.co.uk/category/devops/) | [Github](https://jameskilby.co.uk/category/github/) | [Wordpress](https://jameskilby.co.uk/category/wordpress/) # How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare By[James](https://jameskilby.co.uk) October 6, 2025 · Updated July 11, 2026 • 📖7 min read(1,406 words) Here is how I rebuilt my blog as a static site, using GitHub Actions and Cloudflare to automate the whole publish pipeline. I wanted to automate the publishing of my blog from the authoring side to the public side. These are some of the improvements I made. ## Table of Contents ## What I started with My previous setup involved a locally hosted WordPress instance. This runs in my homelab in an Ubuntu VM. This I will refer to as the authoring instance. Within this WordPress instance I have the “[Simply Static](https://simplystatic.com)” Plugin. When I have finished writing a post or making changes, I then hit “Generate” within the plugin. It will then scan the entire site, creating an entire copy that’s made available as a Zip download. The benefits of this setup I have discussed [previously](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) I then take this export and copy it to a public Github repo called “WordPress”. When the code is checked in, this triggers a process within Cloudflare to publish it as a static site in Cloudflare’s CDN. I have previously written about the Cloudflare part. ## What I finished with ![Wordpress Authoring](https://jameskilby.co.uk/wp-content/uploads/2025/10/Wordpress-Authoring-2-1024x677.png) The new setup is conceptually similar in that I have maintained the authoring WordPress instance. The major difference is how the data gets from the authoring WordPress instance to GitHub. The GitHub to Cloudflare part remains unchanged. I have introduced a GitHub runner that runs alongside the WordPress Instance. This runner has permissions to query the WordPress API to get data. Now when I have finished updating my site, I can start the runner. It then scans the site making a local copy of it and performs an automated check-in to Github. **Disclaimer:** I used a fair bit of AI to help with some of the python and API calls. It also wrote the documentation for me. ## How I Built the Static Site First things first, I created a new GitHub repo so that if I messed anything up, I could point Cloudflare back to the previous WordPress Repo. ### WordPress API The next step was to enable the WordPress API. This was to allow a GitHub runner to communicate directly with WordPress. I chose to do this with a plugin called [REST API Authentication for WP](https://en-gb.wordpress.org/plugins/wp-rest-api-authentication/) I then created a simple basic authentication token. This is then stored in the GitHub repo as an action secret. This allows me to have a public repo but without leaking the key. This is done by navigating to the GitHub repo > Settings then navigate to the Security section and select “Secrets & Variables”. Then create a “New Repository Secret” called WP_AUTH_TOKEN add your API token. ### GitHub Runner The next step was to deploy a runner. These can either be a GitHub-hosted runner or a local runner. I am using a local GitHub runner running on an Ubuntu machine in my lab to perform this function as the WordPress instance isn’t available publicly and therefore not reachable by the GitHub-hosted runner. To set this up, first you need to navigate to the Actions section within your GitHub Repo and select the “Runners” section. Ensure you have the Self-hosted runners tab selected. GitHub then gives you easy instructions to set up the runner. First, you select the Architecture I am using Ubuntu and therefore selected Linux. GitHub then gives you great instructions. # Create a folder $ mkdir actions-runner && cd actions-runner # Download the latest runner package $ curl -o actions-runner-osx-x64-2.329.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.329.0/actions-runner-osx-x64-2.329.0.tar.gz # Optional: Validate the hash $ echo "c5a14e84b358c72ca83bf14518e004a8ad195cc440322fbca2a4fec7649035c7 actions-runner-osx-x64-2.329.0.tar.gz" | shasum -a 256 -c # Extract the installer $ tar xzf ./actions-runner-osx-x64-2.329.0.tar.gz 📋 Copy Then configure the runner # Create the runner and start the configuration experience $ ./config.sh --url https://github.com/jameskilbynet/jkcoukblog --token AFPWGXXXXXXXXXVHXXI7D7DM # Last step, run it! $ ./run.sh 📋 Copy ### Python The runner is then executing a number of Python scripts as can be seen in the GitHub repo. At a high level these: * Discovers all posts, pages, categories and tags * Fetches the media library assets * Replaces all WordPress URLs as relative * Converts WordPress Embed blocks to iframes * Removes WordPress specific elements that are not required in the static site. This grabs a copy of the site and then stores it in the Public directory within the GitHub Repo and issues a Git Commit and Git Push command. When I have finished making changes in WordPress I can trigger the GitHub runner manually (it also runs on a Cron schedule). It typically takes about 1 minute 20 to process my site. In that time it: * Sets up the job on a runner * Checks Out the repo * Sets up the Python environment * Installs any relevant dependencies * Tests the runner environment * Generates the Static Site files * Commits and Pushes the Static Site * Notifies Slack * Executes Python cleanup * Executes other cleanups ## Build Notifications I didn’t want to have to log in to GitHub or Cloudflare to monitor progress of the runner, so I added notifications to Slack. ### Create a Slack Webhook URL First, you’ll need to create a Slack webhook. Here’s how: 1. Go to your Slack workspace 2. Navigate to Apps: Click on your workspace name → Settings & administration → Manage apps 3. Search for “Incoming Webhooks” and add it to your workspace 4. Create a webhook: ◦ Choose the channel where you want notifications (e.g., #deployments, #general) I created a web channel ◦ Click “Add Incoming Webhooks Integration” ◦ Copy the Webhook URL (looks like: https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXX) #### GitHub Notifications In GitHub, navigate to the secrets actions section of your repo Add a new secret called SLACK_WEBHOOK_URL This will automatically get triggered by the workflow in the repo #### Cloudflare Notifications In Cloudflare, go to the Notifications settings and select Destinations. Create a Webhook using the Webhook URL created above with an appropriate name. Then navigate to the “Workers & Pages” section and select your static pages site. Then navigate to the settings of your workers and select the notification events that you want, and select the Webhook name created above. While testing, I have all notifications turned on. ![Cloudflare Events](https://jameskilby.co.uk/wp-content/uploads/2025/10/Cloudflare-Events-1024x214.png) When I trigger the runner, I get 3 Slack notifications in my Web channel. The first is the Commit to GitHub, I then get one when Cloudflare starts building the Static site and one when it completes. One useful feature is that it gives you the preview URL so you can instantly view your changes without having any DNS propagation issues. incoming-webhook APP 10:27 PM :white_check_mark: Succeeded GitHub Actions 8398a7@action-slack repo jameskilbynet/jkcoukblog commit 4761df3f 10:27 Slack :zap: Cloudflare Pages: A production deployment has started for jkcoukblog :zap: Build Link: https://dash.cloudflare.com/c9bb9919eaab3686694017fa604070dd/pages/view/jkcoukblog/aeacb2b5-03c8-4771-a252-cf48b2c86da0 Commit Hash: b72489f574a14dc6aacad7c2fa1fbd874ed7fed4 Environment: ENVIRONMENT_PRODUCTION Timestamp: 2025-10-09T21:27:53.399259Z Cloudflare | Oct 9th 10:28 Slack :white_check_mark: Cloudflare Pages: A production deployment has succeeded for jkcoukblog :white_check_mark: Build Link: https://dash.cloudflare.com/c9bb9919eaab3686694017fa604070dd/pages/view/jkcoukblog/aeacb2b5-03c8-4771-a252-cf48b2c86da0 Commit Hash: b72489f574a14dc6aacad7c2fa1fbd874ed7fed4 Preview URL: https://aeacb2b5.jkcoukblog.pages.dev Branch Alias URL: Project URL: https://jkcoukblog.pages.dev Show more Cloudflare | Oct 9th 📋 Copy ## Using Utterance for Comments I wanted to add a comment mechanism to my pages, but as it’s a static site that needs a little bit of planning as the traditional WordPress mechanisms won’t work. After a little bit of research I decided to use [Utterance](https://utteranc.es). Utterance is lightweight javascript code that integrates with GitHub issues. To enable it, I enabled Issues on the repo I have been using for all of my Web stuff and then edited the WordPress theme using the built-in theme editor. Within WordPress I changed comments.php to the following
📋 Copy I also wanted a Slack notification if anyone posted comments on one of my pages. Unfortunately the payload Slack is expecting is a JSON payload in a specific format that GitHub doesn’t do by default. I therefore created a new GitHub Action to trigger the notification ### **Use GitHub Actions** You can trigger Slack messages directly via GitHub Actions: 1. Create a GitHub Action workflow `.github/workflows/issue-to-slack.yml`: name: Notify Slack on Issue Creation on: issues: types: [opened] jobs: notify-slack: runs-on: ubuntu-latest steps: - name: Send Slack Notification uses: slackapi/slack-github-action@v1.23.0 with: payload: | { "text": "New Comment Created: <${{ github.event.issue.html_url }}|${{ github.event.issue.title }}> by ${{ github.event.issue.user.login }}" } env: SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }} 📋 Copy I am using the same SLACK_WEBHOOK_URL as defined previously so these get posted in my existing web channel. ## 📚 Related Posts * [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) * [WordPress Hosting with Cloudflare Pages](https://jameskilby.co.uk/2023/05/how-to-take-a-wordpress-site-and-publish-it-as-a-static-site-on-cloudflare-pages/) * [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) ## Similar Posts * [ ![Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/wp-content/uploads/2026/01/VMware-NVIDIA-logos_ee2f18dc-615d-4c9e-8f11-9c3c2ce2bf37-prv-768x432.png) ](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) ### [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) By[James](https://jameskilby.co.uk) February 9, 2026 · Updated July 11, 2026 Learn how to use Ansible to configure an Ubuntu VM for use with NVIDIA based AI workloads in vSphere * [ ![Fixing Wrangler Node.js Version Conflicts After Brew Upgrade](https://jameskilby.co.uk/wp-content/uploads/2022/01/WranglerCrab-1-768x256.png) ](https://jameskilby.co.uk/2022/01/wrangler-and-node-versions/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) ### [Fixing Wrangler Node.js Version Conflicts After Brew Upgrade](https://jameskilby.co.uk/2022/01/wrangler-and-node-versions/) By[James](https://jameskilby.co.uk) January 15, 2022 · Updated July 11, 2026 I am a massive fan of the brew package management system for macOS and use it on all of my Macs. I typically just upgrade everything blindly and have never had an issue. * [ ![Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2022/01/web-development/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) ### [Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/2022/01/web-development/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 A while ago I started messing with Cloudflare Workers. I have now moved this site permanently over to them. * [ ![Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/wp-content/uploads/2026/01/Website-Optimisations-768x560.png) ](https://jameskilby.co.uk/2026/01/web-development-improvements/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) By[James](https://jameskilby.co.uk) January 15, 2026 · Updated July 11, 2026 I have spent the Christmas break making some improvements to this blog. * [ ![Automated VCF 9 Offline Depot architecture diagram showing Traefik reverse proxy and Nginx file server stack](https://jameskilby.co.uk/wp-content/uploads/2026/04/offlinedepot.png) ](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) By[James](https://jameskilby.co.uk) April 10, 2026 · Updated July 11, 2026 One Bash script turns a fresh Ubuntu VM into a VCF 9 Offline Depot: Traefik, Nginx, basic auth, and Let’s Encrypt wildcard certs via Cloudflare DNS. * [ ![My First GitHub Pull Request: Contributing to Open Source](https://jameskilby.co.uk/wp-content/uploads/2020/12/175jvBleoQfAZJc3sgTSPQA.jpg) ](https://jameskilby.co.uk/2020/12/my-first-pull/) [Devops](https://jameskilby.co.uk/category/devops/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [My First GitHub Pull Request: Contributing to Open Source](https://jameskilby.co.uk/2020/12/my-first-pull/) By[James](https://jameskilby.co.uk) December 22, 2020 · Updated June 5, 2026 I was initially going to add in the contents of this post to one that I have been writing about my exploits with HashiCorp Packer but I decided it probably warranted being separated out. --- # Managing my Homelab with SemaphoreUI Source: https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/ Date: 2025-09-02 Categories: Ansible, Homelab, Nutanix, Networking, Automation, Artificial Intelligence, Github ![](https://jameskilby.co.uk/wp-content/uploads/2025/07/semaphore.png) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Homelab](https://jameskilby.co.uk/category/homelab/) # Managing my Homelab with SemaphoreUI By[James](https://jameskilby.co.uk) September 2, 2025 · Updated June 5, 2026 • 📖8 min read(1,592 words) I recently stumbled across [Semaphore](https://docs.semaphoreui.com), which is essentially a frontend for managing DevOps tooling, including Ansible, Terraform, OpenTofu, and PowerShell. It’s easy to deploy in Docker, and I am slowly moving more of my homelab management over to it. ## Introduction This is a guide to show you how to get up and running easily with SemaphoreUI in Docker and use it to execute a basic Ansible playbook to deploy Docker onto an Ubuntu Server. ## Table of Contents ## Pre-reqs * Somewhere to run your Semaphore server. I chose to use a Docker container on one of my existing hosts. * Git Repo. I am using GitHub. My code is all stored in a public repo [here](https://github.com/jameskilbynet/iac/) so others can follow along. * SSH Keys/Authentication into the server to be provisioned. ## Docker Configuration This is the config that I am using (with some environment variables obscured). If you want to build your own docker run or compose file they have built a [website](https://semaphoreui.com/install/docker/2_15/) that helps you build your docker configuration. I love this. That is what I used to build the below config. services: semaphore: image: semaphoreui/semaphore:v2.15.12-powershell7.5.0 labels: - "com.example.description=semaphore" - "traefik.enable=true" - "traefik.http.routers.semaphore.rule=Host(`semaphore.jameskilby.cloud`)" - "traefik.http.routers.semaphore.entrypoints=https" - "traefik.http.routers.semaphore.tls=true" - "traefik.http.routers.semaphore.tls.certresolver=cloudflare" - "traefik.http.services.semaphore.loadbalancer.server.port=3000" networks: traefik: environment: SEMAPHORE_DB_DIALECT: bolt SEMAPHORE_ADMIN: admin SEMAPHORE_ADMIN_PASSWORD: XXXXXX SEMAPHORE_ADMIN_NAME: James SEMAPHORE_ADMIN_EMAIL: email@jameskilby.cloud SEMAPHORE_RUNNER_REGISTRATION_TOKEN: "mwsEdFU51pZDjkZsDsmws8yOvfdvdf" SEMAPHORE_EMAIL_SENDER: "semaphore@jameskilby.cloud" SEMAPHORE_EMAIL_HOST: "mail.jameskilby.cloud" SEMAPHORE_EMAIL_PORT: "25" SEMAPHORE_EMAIL_SECURE: "True" volumes: - semaphore_data:/var/lib/semaphore - semaphore_config:/etc/semaphore - semaphore_tmp:/tmp/semaphore volumes: semaphore_data: semaphore_config: semaphore_tmp: networks: traefik: external: true 📋 Copy ## Playbook Walkthrough Once Semaphore is up and running, you can start setting up your automation pipelines. I will step through an example Ansible Playbook to deploy Docker. This playbook can be found [here](https://github.com/jameskilbynet/iac/blob/b65d7f8d014cb5458c74f5188798b3129387f36e/ansible/docker/install_docker.yml). Let me walk you through the playbook, then we will demonstrate using Semaphore to execute this. ### Playbook Metadata - name: Install Docker on supported Ubuntu hosts hosts: all become: yes 📋 Copy This defines the hosts to run the playbook on (for now, this should just be set to all), and the become option allows it to become root if needed. vars: docker_gpg_path: /etc/apt/keyrings/docker.gpg docker_repo: "deb [arch=amd64 signed-by={{ docker_gpg_path }}] https://download.docker.com/linux/ubuntu {{ ansible_distribution_release }} stable" 📋 Copy This defines the variables of where we will store the Docker GPG Key and the Docker repo We then use Ansible to define some variables and to ensure we are getting the correct Docker release based on the Version of Ubuntu. ### Define Variables tasks: - name: Ensure required system packages are present apt: name: - apt-transport-https - ca-certificates - curl - gnupg - lsb-release state: present update_cache: yes 📋 Copy The next block ensures that the required packages are present on the system. ### Handle reboots - name: Check if a reboot is required stat: path: /var/run/reboot-required register: reboot_required - name: Reboot the machine if required reboot: msg: "Reboot initiated by Ansible due to package upgrade." pre_reboot_delay: 60 reboot_timeout: 600 post_reboot_delay: 60 when: reboot_required.stat.exists 📋 Copy ### Preps APT and stores GPG Keys - name: Ensure /etc/apt/keyrings directory exists file: path: /etc/apt/keyrings state: directory mode: '0755' - name: Download Docker GPG key in dearmored format shell: | curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o {{ docker_gpg_path }} args: creates: "{{ docker_gpg_path }}" - name: Set permissions on Docker GPG key file: path: "{{ docker_gpg_path }}" mode: '0644' - name: Set permissions on Docker GPG key file: path: "{{ docker_gpg_path }}" mode: '0644' 📋 Copy ### Install Docker and ensure it’s running - name: Install Docker Engine and related packages apt: name: - docker-ce - docker-ce-cli - containerd.io - docker-buildx-plugin - docker-compose-plugin state: latest - name: Ensure Docker service is running and enabled service: name: docker state: started enabled: true 📋 Copy ## Semaphore Concepts So now that we understand the playbook, let’s go over some of the Semaphore concepts ### Projects First we need a project. A project is used for separating out management activity. This could be as Prod/Test/Dev etc. or for managing different systems or applications. To keep everything easy right now I am running everything in a single project. I will likely split this out a bit later. ### Task Templates The task templates are the actions that we want Semaphore to perform for us. In the example, I will just be using it to deploy Docker onto hosts specified in the Inventory. ### Schedule A schedule is an easy way to run a specified task repeatedly. Fairly self-explanatory, I am using this to run a weekly patching schedule on most of my Ubuntu servers. ### Inventory The Inventory specifies hosts to be managed with the appropriate credentials. ![Semaphore web UI dashboard managing Ansible playbooks in my homelab](https://jameskilby.co.uk/wp-content/uploads/2025/09/Ansible-Inventory.png) ### Variable Groups Variable Groups are used for storing additional variables for an inventory. They must be in the JSON format. I am not using these as part of my Docker example. ### Key Store The key store is for storing all credentials. This is for connecting to remote hosts to execute jobs but also for accessing code repositories. ### Repositories A repository defines where the code that Semaphore executes lives. It’s possible to have multiple repos connected. I have a single one defined, and I am using a public repo for two reasons. Firstly, so anyone else can copy the configuration I am using. Secondly to enforce me being more secure and not putting any secrets in any of the code. Something that I sometimes do for speed with my lab. ![Semaphore Repo](https://jameskilby.co.uk/wp-content/uploads/2025/07/Semaphore-Repo-1024x80.png) ## Execute Now that we have gone over the basic concepts, let’s deploy Docker using Semaphore. For testing, I have spun up a vanilla Ubuntu 24.04 server to be used as the Docker server and created a new Project to ensure that we are starting from scratch. ## Repositories ### Connect your repository Step 1: Connect your repository ![New Repo](https://jameskilby.co.uk/wp-content/uploads/2025/09/New-Repo-1024x350.png) And define the appropriate config. I am just using the Main branch and require no authentication as it’s a public GitHub repo ![Repo Settings](https://jameskilby.co.uk/wp-content/uploads/2025/09/Repo-Settings.png) ## Update your Key Store Step 2 Create the credentials for Semaphore to authenticate against your remote server. I am using SSH keys to authenticate against the system, and the account can elevate without a password ## Define your Inventory Step 3 is to define the Inventory of systems you want the task to be run against. This requires a name and the user credentials defined in step 2. For demo purposes, I am just using the IP address of the remote server. All of my homelab servers are connected via DNS, and I am using a simple static list. ![Ansible Inventory](https://jameskilby.co.uk/wp-content/uploads/2025/09/Ansible-Inventory.png) ## Define the task you want to run Go to the “Task Templates” section, select a new template and then select Ansible Playbook ![Managing my Homelab with SemaphoreUI Screenshot](https://jameskilby.co.uk/wp-content/uploads/2025/09/Screenshot-2025-09-02-at-13.28.53.png) Add the required variables; for this particular task, only a few are needed. ![Docker Task](https://jameskilby.co.uk/wp-content/uploads/2025/09/Docker-Task-1-1024x553.png) Once the task is created, simply press the play button to execute it. ![Task Running](https://jameskilby.co.uk/wp-content/uploads/2025/09/Task-Running.png) Semaphore will clone the GitHub repo and then start executing the tasks defined in the Ansible playbook as can be seen above. The big green success button at the top tells you that all of the tasks executed successfully. As the playbook rolls off the screen I have a copy of it in full that can be seen below. 4:33:00 PM Task 2147483445 added to queue 4:33:00 PM Started: 2147483445 4:33:00 PM Run TaskRunner with template: Install Docker 4:33:00 PM Preparing: 2147483445 4:33:00 PM Cloning Repository https://github.com/jameskilbynet/iac 4:33:00 PM Cloning into 'repository_1_template_1'... 4:33:01 PM Get current commit hash 4:33:01 PM Get current commit message 4:33:01 PM installing static inventory 4:33:01 PM No /tmp/semaphore/project_2/repository_1_template_1/ansible/docker/collections/requirements.yml file found. Skip galaxy install process. 4:33:01 PM No /tmp/semaphore/project_2/repository_1_template_1/ansible/docker/requirements.yml file found. Skip galaxy install process. 4:33:01 PM No /tmp/semaphore/project_2/repository_1_template_1/collections/requirements.yml file found. Skip galaxy install process. 4:33:01 PM No /tmp/semaphore/project_2/repository_1_template_1/requirements.yml file found. Skip galaxy install process. 4:33:01 PM No /tmp/semaphore/project_2/repository_1_template_1/ansible/docker/roles/requirements.yml file found. Skip galaxy install process. 4:33:01 PM No /tmp/semaphore/project_2/repository_1_template_1/ansible/docker/requirements.yml file found. Skip galaxy install process. 4:33:01 PM No /tmp/semaphore/project_2/repository_1_template_1/roles/requirements.yml file found. Skip galaxy install process. 4:33:01 PM No /tmp/semaphore/project_2/repository_1_template_1/requirements.yml file found. Skip galaxy install process. 4:33:02 PM 4:33:02 PM PLAY [Install Docker on supported Ubuntu hosts] ******************************** 4:33:02 PM 4:33:02 PM TASK [Gathering Facts] ********************************************************* 4:33:06 PM [WARNING]: Platform linux on host 192.168.38.146 is using the discovered Python 4:33:06 PM ok: [192.168.38.146] 4:33:06 PM interpreter at /usr/bin/python3.12, but future installation of another Python 4:33:06 PM interpreter could change the meaning of that path. See 4:33:06 PM https://docs.ansible.com/ansible- 4:33:06 PM core/2.18/reference_appendices/interpreter_discovery.html for more information. 4:33:06 PM 4:33:06 PM TASK [Ensure required system packages are present] ***************************** 4:33:13 PM changed: [192.168.38.146] 4:33:13 PM 4:33:13 PM TASK [Check if a reboot is required] ******************************************* 4:33:14 PM ok: [192.168.38.146] 4:33:14 PM 4:33:14 PM TASK [Reboot the machine if required] ****************************************** 4:34:31 PM changed: [192.168.38.146] 4:34:31 PM 4:34:31 PM TASK [Ensure /etc/apt/keyrings directory exists] ******************************* 4:34:32 PM ok: [192.168.38.146] 4:34:32 PM 4:34:32 PM TASK [Download Docker GPG key in dearmored format] ***************************** 4:34:33 PM changed: [192.168.38.146] 4:34:33 PM 4:34:33 PM TASK [Set permissions on Docker GPG key] *************************************** 4:34:33 PM ok: [192.168.38.146] 4:34:33 PM 4:34:33 PM TASK [Add Docker APT repository] *********************************************** 4:34:42 PM changed: [192.168.38.146] 4:34:42 PM 4:34:42 PM TASK [Update APT cache] ******************************************************** 4:34:43 PM ok: [192.168.38.146] 4:34:43 PM 4:34:43 PM TASK [Install Docker Engine and related packages] ****************************** 4:35:04 PM changed: [192.168.38.146] 4:35:04 PM 4:35:04 PM TASK [Ensure Docker service is running and enabled] **************************** 4:35:06 PM ok: [192.168.38.146] 4:35:06 PM 4:35:06 PM PLAY RECAP ********************************************************************* 4:35:06 PM 192.168.38.146 : ok=11 changed=5 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 4:35:06 PM 📋 Copy ## Wrap Up I have been really impressed with Semaphore and am migrating more and more of my lab control into it. I might write some more posts on how I am using it for more complex tasks. The one thing I wish it could do natively is Packer for building templates, but just with Terraform and Ansible, it is making my life so much easier. ## 📚 Related Posts * [Automating vSphere Power Management driven by Ansible and SemaphoreUI](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) * [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) ## Similar Posts * [ ![New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/wp-content/uploads/2024/07/IMG_6629-768x149.jpeg) ](https://jameskilby.co.uk/2024/07/new-nodes/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) By[James](https://jameskilby.co.uk) July 2, 2024 · Updated July 11, 2026 I recently decided to update some of my homelab hosts and I managed to do this at very little cost by offloading 2 of my Supermicro e200’s to fellow vExpert Paul . * [ ![UniFi UDM Beast 1U rack-mount gateway shown front-on, with two 3.5-inch drive bays, a bank of RJ45 and SFP network ports, and dimension labels.](https://jameskilby.co.uk/wp-content/uploads/2026/06/UnifiBeast-768x219.webp) ](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Unleashing the UDM Beast](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) By[James](https://jameskilby.co.uk) June 20, 2026 · Updated August 24, 2026 Retiring my ageing WatchGuard M200, I swap in the UniFi UDM Beast — a hardware tour, first iperf and WAN tests, the BGP/FRR setup, and the homelab services I’m consolidating onto it. * [ ![Template Deployment with Packer](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) ](https://jameskilby.co.uk/2021/01/hashicorp-packer/) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. * [ ![Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/wp-content/uploads/2024/06/Ubiquiti_Networks-Logo.wine_-768x512.png) ](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) By[James](https://jameskilby.co.uk) June 26, 2024 · Updated July 11, 2026 How to configure DHCP Option 43 for UniFi devices * [ ![Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/wp-content/uploads/2026/01/VMware-NVIDIA-logos_ee2f18dc-615d-4c9e-8f11-9c3c2ce2bf37-prv-768x432.png) ](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) ### [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) By[James](https://jameskilby.co.uk) February 9, 2026 · Updated July 11, 2026 Learn how to use Ansible to configure an Ubuntu VM for use with NVIDIA based AI workloads in vSphere * [ ![Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/wp-content/uploads/2026/04/packer-github-actions-vsphere-pipeline-768x452.png) ](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) [Automation](https://jameskilby.co.uk/category/automation/) | [Github](https://jameskilby.co.uk/category/github/) ### [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) By[James](https://jameskilby.co.uk) April 30, 2026 · Updated June 5, 2026 A walk-through of an automated golden-image pipeline for vSphere: HashiCorp Packer builds six Ubuntu LTS templates (22.04, 24.04, 26.04 — server and desktop) driven by three GitHub Actions workflows covering PR validation, parallel template builds, and ISO management via a self-hosted runner. --- # An in-depth look at VMware Cloud on AWS hosts Source: https://jameskilby.co.uk/2025/08/vmc-host-deepdive/ Date: 2025-08-14 Categories: VMware, VMware Cloud on AWS, Homelab, Nutanix, Ansible, Artificial Intelligence, Automation, Personal ![](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339.png) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) # An in-depth look at VMware Cloud on AWS hosts By[James](https://jameskilby.co.uk) August 14, 2025 · Updated July 11, 2026 • 📖1 min read(275 words) This is a single page intended to collate every single feature of the current [VMware Cloud on AWS hosts](https://techdocs.broadcom.com/us/en/vmware-cis/cloud/vmware-cloud-on-aws/SaaS/operations-guide/managing-sddc-hosts-and-clusters/vmware-cloud-on-aws-host-types.html) for easy comparison. All of this data is publicly available. I have just collated it into a single page. For more on running VMware Cloud on AWS, see my post on [Time Sync in a VMC environment](https://jameskilby.co.uk/2025/12/time-in-a-vmc-environment/). ## VMware Cloud on AWS Host Comparison | **I3**.**metal** (END OF LIFE)| **I3en**.**metal**| **I4i**.**metal**| i7i.metal-24xl ---|---|---|---|--- **CPU**| | | | Processor Name| Intel Xeon E5-2686 v4 | Intel Xeon Platinum 8175| Intel Xeon 8375c| Intel Xeon 8559c No of Physical Cores| 36| 48| 64| 48 Hyperthreading| No| Yes| Yes| Yes Base Clock| 2.3GHz| 2.5 GHz| 2.9 GHz| 3.2 GHz Turbo Clock| N/A| 3.1 GHz| 3.5 GHz| 4.0 GHz Processor Family| Broadwell| Skylake| Ice Lake| Emerald Rapids Custom Core Count| 8 16 36| 8 16 24 30 36 48| 8 16 24 30 36 48 64| 8 16 24 30 36 **Memory** | | | | Capacity GiB| 512| 768| 1024| 768 Memory Speed| DDR4-2400| DDR4-2666| DDR4-3200| DDR5-5600 **Networking**| | | | Network Adaptor Speed Gb/s| 25| 100| 75 | 56.25 Hardware Network Encryption in Transit| No| Yes| Yes| Yes **Storage**| | | | Physical Drives| 8×1900| 8×7500| 8×3570| 6×3750 vSAN OSA Cache Disk| 2| 4*| 2| 2 vSAN OSA Capacity Disk| 6| 28*| 6| 4 vSAN Compression| Yes| Yes| Yes| Yes vSAN Deduplication| Yes| No| No| No vSAN OSA Support| Yes| Yes| Yes| Yes vSAN ESA Support| No| No| Yes| Yes *I3en is using NVMe namespace to split the 8 physical disks into 32 NVMe namespaces. Please note the I3.metal instance is no longer available to purchase from Broadcom but is still part of the running fleet for existing customers. ## 📚 Related Posts * [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) * [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) * [VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) ## Similar Posts * [ ![Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/wp-content/uploads/2026/01/Firefly_Gemini-Flash-768x417.png) ](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) By[James](https://jameskilby.co.uk) January 27, 2026 · Updated June 5, 2026 How to leverage Content Libraries to deploy into VMware Cloud on AWS faster. * [ ![New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/wp-content/uploads/2024/07/IMG_6629-768x149.jpeg) ](https://jameskilby.co.uk/2024/07/new-nodes/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) By[James](https://jameskilby.co.uk) July 2, 2024 · Updated July 11, 2026 I recently decided to update some of my homelab hosts and I managed to do this at very little cost by offloading 2 of my Supermicro e200’s to fellow vExpert Paul . * [ ![Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/wp-content/uploads/2026/01/VMware-NVIDIA-logos_ee2f18dc-615d-4c9e-8f11-9c3c2ce2bf37-prv-768x432.png) ](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) ### [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) By[James](https://jameskilby.co.uk) February 9, 2026 · Updated July 11, 2026 Learn how to use Ansible to configure an Ubuntu VM for use with NVIDIA based AI workloads in vSphere * [ ![Template Deployment with Packer](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) ](https://jameskilby.co.uk/2021/01/hashicorp-packer/) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. * [ ![VMware – Going out with a Bang!](https://jameskilby.co.uk/wp-content/uploads/2023/10/rnli-logo-768x384.png) ](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [VMware – Going out with a Bang!](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) By[James](https://jameskilby.co.uk) October 7, 2023 · Updated June 1, 2026 There is a lot of uncertainty with VMware at the moment. This is all due to the pending acquisition by Broadcom. * [ ![VMware Cloud on AWS Time Sync & NTP Configuration](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/12/time-in-a-vmc-environment/) [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [VMware Cloud on AWS Time Sync & NTP Configuration](https://jameskilby.co.uk/2025/12/time-in-a-vmc-environment/) By[James](https://jameskilby.co.uk) December 8, 2025 · Updated July 11, 2026 How to use the Amazon Time Sync Service in a VMC environment --- # VMware Cloud on AWS Storage Sizing Quick Reference Guide Source: https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/ Date: 2025-05-21 Categories: VMware, VMware Cloud on AWS, Artificial Intelligence, Automation, Ansible, Homelab, Personal ![](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339.png) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) # VMware Cloud on AWS Storage Sizing Quick Reference Guide By[James](https://jameskilby.co.uk) May 21, 2025 · Updated July 11, 2026 • 📖4 min read(750 words) This VMware Cloud on AWS storage sizing guide is a quick reference for the usable storage you get per host, across i3, i3en, and I4i clusters of 2 to 16 hosts. Use it to estimate capacity before you size a cluster, plan a host type change, or check how much usable space a given cluster size actually delivers once vSAN’s data protection overhead is accounted for. For up to date info always use the official sizing tool located [here ](https://vmc.vmware.com/) ## VMware Cloud on AWS host types covered here This guide covers the i3, i3en, and I4i host types. For host specs and background, see [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) and [VMware Cloud on AWS i3en Host: Specs, Storage & Performance](https://jameskilby.co.uk/2020/07/i3en/). The newer I7i.metal-24xl host isn’t in the official sizing tool yet, so it’s listed in the table below – see [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) for what’s known so far. ## VMware Cloud on AWS storage sizing table (vSAN OSA, 2-16 hosts) This is based on vSAN OSA and excludes the management overhead (i.e. valid for secondary clusters). It also uses the most efficient storage policy that is supported based on the number of hosts available. Each row shows the usable capacity per host once vSAN’s storage policy overhead is applied, not raw disk capacity. Multiply by host count for a rough cluster total, then validate against the official sizing tool for your specific configuration. **Host Type**| **i3**| **i3en**| **I4i**| **i7i.metal-24xl**| ---|---|---|---|---|--- **No of Hosts**| TiB Available| TiB Available| TiB Available | TiB Available| FTT in use 2| 11.2| 41.39| 18.48| 12.28| FTT1 RAID1 3| 16.8| 62.09| 27.71| 18.41| FTT1 RAID1 4| 33.68| 124.49| 55.57| | FTT1 RAID5 5| 42.1| 155.61| 69.46| | FTT1 RAID5 6| 44.8| 165.57| 73.9| 49.11| FTT2 RAID6 7| 52.26| 193.17| 86.22| 57.29| FTT2 RAID6 8| 59.73| 220.77| 98.54| 65.47| FTT2 RAID6 9| 67.2| 248.36| 110.85| 73.66| FTT2 RAID6 10| 74.66| 275.96| 123.17| 81.84| FTT2 RAID6 11| 82.13| 303.55| 135.49| 90.03| FTT2 RAID6 12| 89.6| 331.15| 147.8| 98.21| FTT2 RAID6 13| 97.06| 358.74| 160.12| 106.4| FTT2 RAID6 14| 104.53| 386.34| 172.44| 114.58| FTT2 RAID6 15| 112| 413.94| 184.75| 122.77| FTT2 RAID6 16| 119.46| 441.53| 197.07| 130.95| FTT2 RAID6 ## VMware Cloud on AWS storage sizing table (vSAN ESA, 2-16 hosts) | I4i| **i7i.metal-24xl**| FTT in use ---|---|---|--- **No of Hosts**| TiB Available| TiB Available| 2| 20.98| 18.41| FTT: 1, RAID: 1 3| 41.96| 36.83| FTT: 1, RAID: 5 4| 55.94| 49.1| FTT: 1, RAID: 5 5| 69.93| 61.38| FTT: 1, RAID: 5 6| 83.91| 73.66| FTT: 2, RAID: 6 7| 97.90| 85.93| FTT: 2, RAID: 6 8| 111.88| 98.21| FTT: 2, RAID: 6 9| 125.87| 110.48| FTT: 2, RAID: 6 10| 139.86| 122.76| FTT: 2, RAID: 6 11| 153.84| 135.04| FTT: 2, RAID: 6 12| 167.83| 147.31| FTT: 2, RAID: 6 13| 181.81| 159.59| FTT: 2, RAID: 6 14| 195.8| 171.86| FTT: 2, RAID: 6 15| 209.78| 184.14| FTT: 2, RAID: 6 16| 223.77| 196.42| FTT: 2, RAID: 6 ## How FTT and RAID policy affect VMware Cloud on AWS storage sizing vSAN raises its data protection policy as the cluster grows, which is why usable capacity above doesn’t scale in a straight line. At 2-3 hosts, only FTT1 with RAID-1 mirroring is available – it tolerates one failure but costs more capacity. At 4-5 hosts, FTT1 with RAID-5 erasure coding kicks in, improving efficiency. From 6 hosts up, vSAN moves to FTT2 with RAID-6, tolerating two simultaneous failures while keeping reasonable efficiency at scale. In practice, usable capacity per host is highest right at the top of each policy band. If your workload allows it, sizing for 4 or 5 hosts rather than 3, or 6 or more rather than 5, gets you more usable storage for the same policy. ## Cluster conversions and management overhead If you are planning to do a cluster conversion between host types then the management stack size doesn’t change. What does change is usable capacity at each cluster size, since host types differ in raw capacity per disk group. Re-check the table above for the new host type before committing to a conversion. ## vSAN ESA storage sizing in VMware Cloud on AWS VMware Cloud on AWS also supports vSAN ESA (Express Storage Architecture), which changes the storage efficiency math compared to the OSA table above. For what changed and why it matters, see [vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/). A full vSAN ESA storage sizing table for 2-16 hosts, matching the format above, is coming soon. ## 📚 Related Posts * [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) * [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) * [VMware Cloud on AWS i3en Host: Specs, Storage & Performance](https://jameskilby.co.uk/2020/07/i3en/) ## Similar Posts * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) By[James](https://jameskilby.co.uk) March 27, 2026 · Updated May 31, 2026 A walkthrough of my self-hosted AI stack: Ollama, Open WebUI, ComfyUI, Whishper, n8n, Qdrant, SearxNG, and a full observability layer — all running on my own hardware with Docker Compose. * [ ![Automated VCF 9 Offline Depot architecture diagram showing Traefik reverse proxy and Nginx file server stack](https://jameskilby.co.uk/wp-content/uploads/2026/04/offlinedepot.png) ](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) By[James](https://jameskilby.co.uk) April 10, 2026 · Updated July 11, 2026 One Bash script turns a fresh Ubuntu VM into a VCF 9 Offline Depot: Traefik, Nginx, basic auth, and Let’s Encrypt wildcard certs via Cloudflare DNS. * [ ![Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/wp-content/uploads/2026/01/Firefly_Gemini-Flash-768x417.png) ](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) By[James](https://jameskilby.co.uk) January 27, 2026 · Updated June 5, 2026 How to leverage Content Libraries to deploy into VMware Cloud on AWS faster. * [ ![Template Deployment with Packer](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) ](https://jameskilby.co.uk/2021/01/hashicorp-packer/) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. * [ ![Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) ](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal. * [ ![An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) By[James](https://jameskilby.co.uk) August 14, 2025 · Updated July 11, 2026 This is single page intended to collate every single feature of the current VMware Cloud on AWS hosts for easy comparison. --- # Warp – The intelligent terminal Source: https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/ Date: 2025-04-11 Categories: Artificial Intelligence, Homelab, Docker, Networking, Ansible, Veeam, Storage, Hosting ![Wa](https://jameskilby.co.uk/wp-content/uploads/2025/04/210902461-012e7273-413a-4ec7-be44-e854347f5a21.png) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Homelab](https://jameskilby.co.uk/category/homelab/) # Warp – The intelligent terminal By[James](https://jameskilby.co.uk) April 11, 2025 · Updated June 1, 2026 • 📖4 min read(722 words) [Warp](http://warp.dev) is helping me run my homelab. It has been a big help for me as although I utilise a lot of Linux-based tools at home, I am mostly self-taught and therefore don’t always use best practices. I also know I take some shortcuts that are an acceptable risk to me but I strive to do better. Sometimes you just need a helping hand Enter Warp… ## Table of Contents ## What is Warp? Taken from Warp’s website: Become a command line power user on day one. Warp combines AI and your dev team’s knowledge in one fast, intuitive terminal. At the moment I am just using the free model which has some limits. A number of paid options are available and I am definitely considering them. ## An example of Warp helping me out I was running “sudo apt-get update” on one of my servers. This is a server that’s been around the block a bit and runs quite a lot of things and MAY have had some experimentation done in the past :p sudo apt-get update [sudo] password for username: Get:1 file:/var/nvidia-driver-local-repo-ubuntu2404-565.57.01 InRelease [1,572 B] Get:1 file:/var/nvidia-driver-local-repo-ubuntu2404-565.57.01 InRelease [1,572 B] Hit:2 https://nvidia.github.io/libnvidia-container/stable/deb/amd64 InRelease Hit:3 https://apt.releases.hashicorp.com noble InRelease Hit:4 https://download.docker.com/linux/ubuntu noble InRelease Hit:5 http://gb.archive.ubuntu.com/ubuntu noble InRelease Get:6 http://gb.archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB] Hit:7 http://gb.archive.ubuntu.com/ubuntu noble-backports InRelease Hit:8 https://packages.cloud.google.com/apt coral-edgetpu-stable InRelease Hit:9 http://security.ubuntu.com/ubuntu noble-security InRelease Fetched 126 kB in 10s (12.0 kB/s) Reading package lists... Done W: Target Packages (stable/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target Packages (stable/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target Translations (stable/i18n/Translation-en_GB) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target Translations (stable/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target DEP-11 (stable/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target DEP-11 (stable/dep11/Components-all.yml) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target CNF (stable/cnf/Commands-amd64) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target CNF (stable/cnf/Commands-all) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: https://packages.cloud.google.com/apt/dists/coral-edgetpu-stable/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details. W: Target Packages (stable/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target Packages (stable/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target Translations (stable/i18n/Translation-en_GB) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target Translations (stable/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target DEP-11 (stable/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target DEP-11 (stable/dep11/Components-all.yml) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target CNF (stable/cnf/Commands-amd64) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 W: Target CNF (stable/cnf/Commands-all) is configured multiple times in /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list:1 and /etc/apt/sources.list.d/docker.list:1 📋 Copy As you can see the command itself returned some errors about things being configured multiple times. However as the command (and actual updates) have run successfully in the past I haven’t really dug into it. Where the beauty of Warp kicked in and said ![Screenshot 2025 04 11 at 14.12.14 1024x95](https://jameskilby.co.uk/wp-content/uploads/2025/04/Screenshot-2025-04-11-at-14.12.14-1024x95.png) It then auto-executed a cat command to look at the contents of both files and gave me advice based on what it found ![Screenshot 2025 04 11 at 14.19.09](https://jameskilby.co.uk/wp-content/uploads/2025/04/Screenshot-2025-04-11-at-14.19.09-1024x137.png) It then showed me the below sudo command it wanted to run and asked if it was OK to run it, which I approved sudo rm /etc/apt/sources.list.d/backdownload_docker_com_linux_ubuntu.list && sudo apt-get update 📋 Copy Once apt-get had been upgraded, the duplicates were no longer in place and a few updates were available. These were installed and then Warp gave me an excellent summary of what changed: The upgrade has been completed successfully. Three packages were upgraded: 1. pci.ids 2. python3-software-properties 3. software-properties-common According to the system check: * The running kernel is up-to-date * No services need to be restarted * No containers need to be restarted * No user sessions are running outdated binaries * No VM guests are running outdated hypervisor binaries Your system is now up to date with all the latest package versions. ## Conclusion Although the example above is fairly straightforward and I could have resolved it myself, using Warp it was fixed MUCH MUCH faster. I am using it for more and more complex things. Some of these may take a lot of research or are just plain time consuming. Either way I’m finding Warp a great help at making these things faster and more reliable. If you like this and want to give Warp a go, sign up with my referral link [here](https://app.warp.dev/referral/48XR9W) ## 📚 Related Posts * [Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) * [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) * [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) ## Similar Posts * [ ![Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/wp-content/uploads/2022/12/22225832.png) ](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) [Docker](https://jameskilby.co.uk/category/docker/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) By[James](https://jameskilby.co.uk) December 9, 2022 · Updated June 5, 2026 Late to the party or not, I have been using containers in my lab more and more and that has led me to Portainer …. * [ ![MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/wp-content/uploads/2023/04/2157_hi_res-768x346.png) ](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) By[James](https://jameskilby.co.uk) December 19, 2022 · Updated June 1, 2026 For a while, I’ve been looking to update the networking at the core of my homelab. * [ ![Managing my Homelab with SemaphoreUI](https://jameskilby.co.uk/wp-content/uploads/2025/07/semaphore-768x768.png) ](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Managing my Homelab with SemaphoreUI](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/) By[James](https://jameskilby.co.uk) September 2, 2025 · Updated June 5, 2026 An intro on how I use SemaphoreUI to manage my Homelab * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Veeam](https://jameskilby.co.uk/category/veeam/) ### [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated June 1, 2026 My lab is always undergoing change. Partially as I want to try new things or new ways of doing things. * [ ![Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/wp-content/uploads/2023/04/intel_optane_ssd_900p_series_aic_-_right_angle_575px.png) ](https://jameskilby.co.uk/2023/04/intel-optane/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) By[James](https://jameskilby.co.uk) April 17, 2023 · Updated June 1, 2026 I have been a VMware vExpert for many years and it has brought me many benefits over the years. * [ ![Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/wp-content/uploads/2022/10/spacexs-starlink-to-supply-free-satellite-internet-to-famili_u44u.1920-768x432.jpg) ](https://jameskilby.co.uk/2022/10/starlink/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/2022/10/starlink/) By[James](https://jameskilby.co.uk) October 11, 2022 · Updated June 1, 2026 Since moving to Dorset last year internet connectivity has been the bane of my existence. Currently, I have an ADSL connection provided by my old employer Zen and a 5G connection provided by Three. --- # How I Migrated from Pocket to Hoarder with AI Integration Source: https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/ Date: 2025-01-29 Categories: Artificial Intelligence, Docker, Hosting, Homelab, Storage, Wordpress ![](https://jameskilby.co.uk/wp-content/uploads/2025/01/Screenshot-2025-01-29-at-23.30.47.png) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) | [Hosting](https://jameskilby.co.uk/category/hosting/) # How I Migrated from Pocket to Hoarder with AI Integration By[James](https://jameskilby.co.uk) January 29, 2025 · Updated July 11, 2026 • 📖5 min read(1,004 words) Update: Hoarder has now been renamed to Karakeep due to a trademark issue I’ve been on a mission recently to regain control of my data. I haven’t yet faced the humongous task of moving my main email from Gmail, but I have had some successes with other cloud services and a win is a win…. One of them is my bookmark manager. Up until now, I used Pocket since way back in the day when it was originally called “Read it Later”. I wanted to bring it away from a cloud service and host it locally. I also had the opportunity to add a little AI magic to go with it. ## Table of Contents ## What is Hoarder? Taken from the Hoarder website, it’s an App that can do all of this…. * 🔗 Bookmark links, take simple notes and store images and pdfs. * ⬇️ Automatic fetching for link titles, descriptions and images. * 📋 Sort your bookmarks into lists. * 🔎 Full text search of all the content stored. * ✨ AI-based (aka chatgpt) automatic tagging. With support for local models using Ollama! * 🎆 OCR for extracting text from images. * 🔖 [Chrome plugin](https://chromewebstore.google.com/detail/hoarder/kgcjekpmcjjogibpjebkhaanilehneje) and [Firefox addon](https://addons.mozilla.org/en-US/firefox/addon/hoarder/) for quick bookmarking. * 📱 An [iOS app](https://apps.apple.com/us/app/hoarder-app/id6479258022), and an [Android app](https://play.google.com/store/apps/details?id=app.hoarder.hoardermobile&pcampaignid=web_share). * 📰 Auto hoarding from RSS feeds. * 🔌 REST API. * 🌐 Multi-language support. * 🖍️ Mark and store highlights from your hoarded content. * 🗄️ Full page archival (using [monolith](https://github.com/Y2Z/monolith)) to protect against link rot. Auto video archiving using [youtube-dl](https://github.com/marado/youtube-dl). * ☑️ Bulk actions support. * 🔐 SSO support. * 🌙 Dark mode support. * 💾 Self-hosting first. The main use for me is having a website and mobile app for saving pages useful to me that I found when I’m usually down a tech rabbit hole. ## Hoarder Architecture I have chosen to run this within my home infrastructure and connected it to my existing Ollama [setup ](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) This means that Hoarder can call Ollama for AI text/image classification using the setup and models I have already created. The deployment is all within Docker and I have added extracts from my Docker Compose files below. ![Hoarder App](https://jameskilby.co.uk/wp-content/uploads/2025/01/HoarderApp-1024x491.png) ## Hoarder Install Snippet from my docker-compose.yml Hoarder hoarder: image: ghcr.io/hoarder-app/hoarder:${HOARDER_VERSION:-release} restart: unless-stopped networks: - traefik volumes: - ./data:/data env_file: - .env environment: MEILI_ADDR: http://meilisearch:7700 BROWSER_WEB_URL: http://chrome:9222 DATA_DIR: /data labels: - "com.example.description=hoarder" - "traefik.enable=true" - "traefik.http.routers.hoarder.rule=Host(`hoarder.jameskilby.cloud`)" - "traefik.http.routers.hoarder.entrypoints=https" - "traefik.http.routers.hoarder.tls=true" - "traefik.http.routers.hoarder.tls.certresolver=cloudflare" - "traefik.http.services.hoarder.loadbalancer.server.port=3000" chrome: image: gcr.io/zenika-hub/alpine-chrome:123 restart: unless-stopped networks: - traefik command: - --no-sandbox - --disable-gpu - --disable-dev-shm-usage - --remote-debugging-address=0.0.0.0 - --remote-debugging-port=9222 - --hide-scrollbars meilisearch: image: getmeili/meilisearch:v1.11.1 restart: unless-stopped networks: - traefik env_file: - .env environment: MEILI_NO_ANALYTICS: "true" volumes: - ./meilisearch:/meili_data 📋 Copy .env snippet file OLLAMA_BASE_URL=http://ollama:11434 INFERENCE_TEXT_MODEL=llama3.1:8b INFERENCE_IMAGE_MODEL=llava 📋 Copy ## Export from Pocket Luckily, Pocket has an export function that will dump all of your saved URLs and tags into a single file. This can be run by navigating to https://getpocket.com/export when logged in. ## Import to Hoarder Once you have this file, you can input it straight into Hoarder. This is done by navigating to the user settings section and then selecting import/export. Hoarder supports several file formats from other tools ![How I Migrated from Pocket to Hoarder with AI Integration Screenshot](https://jameskilby.co.uk/wp-content/uploads/2025/01/Screenshot-2025-01-29-at-22.52.45-1024x124.png) ## Hoarder In Action When the URLs are loaded, Hoarder passes the URLs into a headless Chrome to gather the data from that page. It then indexes the contents and then passes the contents to Ollama to apply appropriate tags. ### AI Prompt You can tweak the AI prompt that is sent over to Ollama. In my case I have just used the default as it looked like a good starting point. The prompt is You are a bot in a read-it-later app and your responsibility is to help with automatic tagging. Please analyze the text between the sentences "CONTENT START HERE" and "CONTENT END HERE" and suggest relevant tags that describe its key themes, topics, and main ideas. The rules are: - Aim for a variety of tags, including broad categories, specific keywords, and potential sub-genres. - The tags language must be in english. - If it's a famous website you may also include a tag for the website. If the tag is not generic enough, don't include it. - The content can include text for cookie consent and privacy policy, ignore those while tagging. - Aim for 3-5 tags. - If there are no good tags, leave the array empty. CONTENT START HERE CONTENT END HERE You must respond in JSON with the key "tags" and the value is an array of string tags. 📋 Copy The process of gathering all the web pages, indexing them and analysing with AI took around an hour with my setup. I believe that Hoarder has some internal throttles to try and avoid tripping anti-bot tools. You can monitor this process in the admin section. I was also keen to see the stats on my graphics card while this was running so I ran NVTOP on the VM while some AI processing was running. ![How I Migrated from Pocket to Hoarder with AI Integration Screenshot](https://jameskilby.co.uk/wp-content/uploads/2025/01/Screenshot-2025-01-29-at-23.00.23-1024x329.png) ## Finished Result At the end of the process, I ended up with a little over 750 bookmarks imported and fully indexed and displayed in a very appealing nature. ![How I Migrated from Pocket to Hoarder with AI Integration Screenshot](https://jameskilby.co.uk/wp-content/uploads/2025/01/Screenshot-2025-01-29-at-23.21.32-1024x590.png) ## Post Install Tasks ### Tag Merge The AI process is mightily impressive but one of the issues that can occur is that it generates similar tags. Hoarder has the ability to allow you to merge them. This is done in the cleanups section under your username. In my case it suggested 150 tags that may need merging. At the moment I am glad this has manual oversight as some should definitely be merged like “Motherboard” and “Motherboards” but some are totally different like “NAS” and “NASA” ### Broken Links I was quite surprised by how many of the links I had were broken links. These were for a few reasons, ranging from company takeovers, to sites being dead. Sadly, in one case the author is no longer with us. In a few cases, the broken link was because the URL I had saved led to a URL shortener that no longer exists. I need to revisit the remaining links in the list and ensure that they point to the end state. ## 📚 Related Posts * [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) * [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) * [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) ## Similar Posts * [ ![Wa](https://jameskilby.co.uk/wp-content/uploads/2025/04/210902461-012e7273-413a-4ec7-be44-e854347f5a21-768x180.png) ](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Warp – The intelligent terminal](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) By[James](https://jameskilby.co.uk) April 11, 2025 · Updated June 1, 2026 How Warp is helping me run my homelab. * [ ![Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/wp-content/uploads/2023/04/intel_optane_ssd_900p_series_aic_-_right_angle_575px.png) ](https://jameskilby.co.uk/2023/04/intel-optane/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) By[James](https://jameskilby.co.uk) April 17, 2023 · Updated June 1, 2026 I have been a VMware vExpert for many years and it has brought me many benefits over the years. * [ ![Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/wp-content/uploads/2022/10/spacexs-starlink-to-supply-free-satellite-internet-to-famili_u44u.1920-768x432.jpg) ](https://jameskilby.co.uk/2022/10/starlink/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/2022/10/starlink/) By[James](https://jameskilby.co.uk) October 11, 2022 · Updated June 1, 2026 Since moving to Dorset last year internet connectivity has been the bane of my existence. Currently, I have an ADSL connection provided by my old employer Zen and a 5G connection provided by Three. * [ ![Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/wp-content/uploads/2023/04/81-ZoEW24UL._SL1500_-768x461.jpg) ](https://jameskilby.co.uk/2019/02/lab-storage-2/) [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/2019/02/lab-storage-2/) By[James](https://jameskilby.co.uk) February 10, 2019 · Updated June 5, 2026 Since starting my new role with Xtravirt my Homelab has gone through several fairly significant changes. * [ ![Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu-768x450.jpeg) ](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Wordpress](https://jameskilby.co.uk/category/wordpress/) ### [Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 I have been making several changes (mainly cosmetic) to this site over the last day or so. On most changes I have been doing an export and then uploading the site to Cloudflare using Wrangler. * [ ![Homelab SSD Failure: How Synology RAID Saved My Data](https://jameskilby.co.uk/wp-content/uploads/2022/11/BrokenHardDive-1200x630-1-768x403.jpg) ](https://jameskilby.co.uk/2022/11/homelab-bad-days-almost/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Homelab SSD Failure: How Synology RAID Saved My Data](https://jameskilby.co.uk/2022/11/homelab-bad-days-almost/) By[James](https://jameskilby.co.uk) November 21, 2022 · Updated July 11, 2026 I recently spent 3 weeks in Ireland with my wife Wendy and our son Nate. --- # How to Run ZFS on VMware vSphere: Setup Guide and Best Practices Source: https://jameskilby.co.uk/2024/12/zfs-on-vmware/ Date: 2024-12-18 Categories: TrueNAS Scale, VMware, vSAN, vSphere, Homelab, Storage, Artificial Intelligence, Automation, Personal, Ansible ![](https://jameskilby.co.uk/wp-content/uploads/2024/12/ZFS.jpg) [TrueNAS Scale](https://jameskilby.co.uk/category/truenas-scale/) | [VMware](https://jameskilby.co.uk/category/vmware/) | [vSAN](https://jameskilby.co.uk/category/vmware/vsan-vmware/) | [vSphere](https://jameskilby.co.uk/category/vsphere/) # How to Run ZFS on VMware vSphere: Setup Guide and Best Practices By[James](https://jameskilby.co.uk) December 18, 2024 · Updated June 1, 2026 • 📖3 min read(531 words) ## Table of Contents ## Introduction I have run a number of systems using ZFS since the earliest days of my homelab using Nexenta, all the way back in 2010. The image below is my lab at the time with an IBM head unit that I think had 18GB of RAM 6x450GB SAS drives and this was then connected to the Dell PowerVault SCSI Array above it with 14x146GB 10K SAS drives…. ![How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/wp-content/uploads/2024/12/IMG_20140330_210511-1024x845.jpeg)Original Nexenta Setup The number one rule is to ALWAYS give ZFS access to the underlying raw storage. You don’t want a raid controller or anything else interfering with the IO path. This is similar to how vSAN works with VMware. But rules are meant to be broken right….. I have virtualized a few copies of TrueNAS Scale and Core using ZFS on top of VMware. In these particular instances I specifically DON’T want to pass through the storage HBA. Why would I do this? Mainly for two reasons. This allows me to test upgrades of my physical TrueNAS setup with an easy rollback if needed by not passing the drives or controllers in I can clone and snapshot the VMs just as if it was any other and move it around my lab infrastructure. ## Copy on Write ZFS is a “Copy on Write” file system which means that it never overwrites existing blocks of storage. It always places writes into new blocks. This is unfriendly with “thin provisioning” something I am a huge fan of. This means that over time even a tiny database writing one megabyte file over and over again will slowly clog the entire file system. So if you’re going to break the rules. The way I see it is you might as well do it properly The first requirement is that the VMs be provisioned with thin disks in vSphere. If it is not thin then unmap won’t work. This is important in case you are thin at the underlay storage level. ## Disk IDs What you also need to do is to ensure that TrueNAS can see unique disk IDs. To do this shut down the VM’s and add the following parameter to the VMware VM’s configuration disk.EnableUUID=TRUE 📋 Copy Once this is done when you power the VM’s on you should be able to see unique serials of each disk similar to this screenshot. Prior to this change, the serial section is blank. ![Disk Serials](https://jameskilby.co.uk/wp-content/uploads/2024/12/Disk-Serials-1024x125.png) ## Trim Once the disks are seen as unique it is possible to enable trim. To confirm that trim is working execute the below command. ( I have no idea why this is a blocker but it is) Sudo pool trim Pool-1 📋 Copy I decided to manually enable it by executing the command in the shell. (my Pool is called Pool-1) To confirm that trim is working, execute the below command sudo zpool status Pool-1 📋 Copy If everything is working you will see trimming running next to the pool. ![trimming 1](https://jameskilby.co.uk/wp-content/uploads/2024/12/trimming-1-1024x452.png) A further validation that this is working is to review the VM’s storage used, see the before and after of this VM’s storage ![vm Before 2](https://jameskilby.co.uk/wp-content/uploads/2024/12/vm-Before-2.png) ![vm After](https://jameskilby.co.uk/wp-content/uploads/2024/12/vm-After.png) Additional confirmation can be seen by reviewing the underlying (vSAN consumption in this case). Before and after listed below. ![V SAN Before](https://jameskilby.co.uk/wp-content/uploads/2024/12/vSAN-Before.png) ![V SAN After](https://jameskilby.co.uk/wp-content/uploads/2024/12/vSAN-After-1.png) ## 📚 Related Posts * [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) * [VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) * [Runecast Remediation Scripts: Auto-Fix VMware Storage Issues](https://jameskilby.co.uk/2023/05/runecast-remediation-scripts/) ## Similar Posts * [ ![Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/wp-content/uploads/2024/09/QuantaGrid-SD1Q-1ULH-Front-Three-Quarter.png) ](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) By[James](https://jameskilby.co.uk) September 12, 2024 · Updated July 11, 2026 Yes, that’s a clickbait title. But technically it’s possible if I dropped all drive redundancy… I recently saw an advert for a server that was just too good to be true. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) By[James](https://jameskilby.co.uk) March 27, 2026 · Updated May 31, 2026 A walkthrough of my self-hosted AI stack: Ollama, Open WebUI, ComfyUI, Whishper, n8n, Qdrant, SearxNG, and a full observability layer — all running on my own hardware with Docker Compose. * [ ![VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/wp-content/uploads/2020/09/vmware_SP_HCI20.png) ](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) [Personal](https://jameskilby.co.uk/category/personal/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) By[James](https://jameskilby.co.uk) September 13, 2020 · Updated May 31, 2026 I recently sat (and passed the VMware HCI Master Specialist exam (5V0-21. * [ ![Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-12-14-at-21.45.23.png) ](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) By[James](https://jameskilby.co.uk) December 14, 2022 · Updated June 1, 2026 I run a reasonably extensive homelab that is of course built around the VMware ecosystem. * [ ![vSphere Power Management Ansible Playbooks with Semaphore](https://jameskilby.co.uk/wp-content/uploads/2026/04/vsphere-power-management-ansible-768x403.png) ](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automating vSphere Power Management driven by Ansible and SemaphoreUI](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) By[James](https://jameskilby.co.uk) April 15, 2026 · Updated June 1, 2026 In this post I’ll walk through how I use vSphere Power Management driven by Ansible and SemaphoreUI to automatically reduce ESXi host electricity consumption — saving real money on my Octopus Agile tariff by toggling hosts between Low Power and Balanced policies. Introduction One of the larger costs of running my homelab is the electricity…. * [ ![Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/wp-content/uploads/2023/04/81-ZoEW24UL._SL1500_-768x461.jpg) ](https://jameskilby.co.uk/2019/02/lab-storage-2/) [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/2019/02/lab-storage-2/) By[James](https://jameskilby.co.uk) February 10, 2019 · Updated June 5, 2026 Since starting my new role with Xtravirt my Homelab has gone through several fairly significant changes. --- # Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU Source: https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/ Date: 2024-10-11 Categories: Artificial Intelligence, Docker, Homelab, Networking, Runecast, VMware, VCF, Nutanix ![](https://jameskilby.co.uk/wp-content/uploads/2024/10/pexels-tara-winstead-8386440-scaled.jpg) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) | [Homelab](https://jameskilby.co.uk/category/homelab/) # Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU By[James](https://jameskilby.co.uk) October 11, 2024 · Updated July 11, 2026 • 📖5 min read(1,068 words) Artificial intelligence is all the rage at the moment. It’s getting included in every product announcement from pretty much every vendor under the sun. Nvidia’s stock price has gone to the moon. So I thought I better get some knowledge and understand some of this. As it’s a huge field and I wasn’t exactly sure where to start I decided to follow [Tim’s](https://www.youtube.com/@TechnoTim) excellent [video ](https://www.youtube.com/watch?v=GrLpdfhTwLg)and [guide](https://technotim.live/posts/ai-stack-tutorial/) as to what he has deployed. This blog is a bit more of a reminder for me of what I have done. I won’t go into all the details as Tim has done a better job than I will. ## Table of Contents ## Introduction Yes, you can do some AI things with a CPU but the reality at the moment is that it’s better suited to being executed on a GPU and depending on what you’re doing a GPU(s) with lots of memory are what you need. I didn’t fancy spending any money so I thought I would start with my Nvidia P4 and see where I get to. ### Nvidia Tesla P4 Specs The Nvidia P4 has 8GB of GDDR5 memory which is enough for running some smaller models ### OS Choice I chose to use one of my vSphere Hosts with a VM running Ubuntu 24.04 ### vHardware Spec In vSphere, I allocated 8x vCPUs, 20GB of memory, and 256GB of storage to the VM with the P4 passed in directly. This means I didn’t need any additional NVIDIA licences. Install Ubuntu 24.04 ( or use your template) & make sure it’s patched sudo apt-get update && sudo apt-get upgrade 📋 Copy ## Install NVIDIA Software The first step is to install the Nvidia drivers. sudo ubuntu-drivers install 📋 Copy reboot and then use “sudo nvidia-smi” to validate that the card is now functioning within the VM. The output below shows that the Ubuntu server can see the Tesla P4 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 470.256.02 Driver Version: 470.256.02 CUDA Version: 11.4 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla P4 Off | 00000000:0B:00.0 Off | 0 | | N/A 46C P0 23W / 75W | 0MiB / 7611MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+ 📋 Copy ## Install Docker The next step is to install docker as the AI engines will all be running within docker. # Add Docker's official GPG key: sudo apt-get update sudo apt-get install ca-certificates curl gnupg sudo install -m 0755 -d /etc/apt/keyrings curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg sudo chmod a+r /etc/apt/keyrings/docker.gpg # Add the repository to Apt sources: echo \ "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \ "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update 📋 Copy sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin 📋 Copy Use Docker without SUDO sudo usermod -aG docker $USER 📋 Copy ## NVIDIA Container Toolkit The next thing to do is to install the Nvidia container toolkit. This allows containers to interact with the graphics card curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \ && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \ sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list 📋 Copy sudo apt-get update sudo apt-get install -y nvidia-container-toolkit 📋 Copy sudo nvidia-ctk runtime configure --runtime=docker sudo systemctl restart docker 📋 Copy Test NVIDIA Docker sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi 📋 Copy If the above worked, you have a functioning system where docker can communicate with your GPU. ## Overview Tim’s blog has a decent overview of the containers and what they are doing so I won’t detail it here. ## Chat in Action So the next obvious thing I needed to do was ask it some questions. So I asked the gemma2 model the question “What AI models can I run on a Nvidia p4?” and it answered…. You’re right to be specific! While the Nvidia P4 is _technically_ an AI-capable GPU, it’s designed more for inference (running pre-trained models) than training complex ones from scratch. Think of it like a powerful calculator for AI, not a research lab. Here’s a breakdown of what you can realistically expect on an Nvidia P4: **1\. Lightweight Models are Your Friend:** * **Object Detection:** Tiny YOLOv3 variants or lightweight MobileNet-based detectors. Expect lower accuracy than bigger models but fast inference speeds for real-time applications. * **Image Classification:** MobileNet V2, Inception v3 (downscaled), SqueezeNet – these are pre-trained and optimized for mobile devices. **2\. Frameworks Matter:** * **TensorRT** : This is Nvidia’s specialized engine for P4. It optimizes your models for performance, even converting them into a more efficient format. * **ONNX Runtime** : This open-source runtime supports various AI frameworks, and can be used to run models on the P4 after conversion. **3\. Practical Applications:** * **Edge Devices:** Think smart cameras, embedded systems, or robotics where low power consumption and quick responses are crucial. * **Demo Projects:** Great for learning about deployment and seeing how AI works in real-time, even if the accuracy isn’t state-of-the-art. **What You CAN’T Do:** * Train massive neural networks from scratch (think GPT-3 or large image generators). The P4 lacks the memory and compute power. * Run cutting-edge research models that demand immense resources. Those are best left to specialized cloud GPUs or supercomputers. Let me know if you have a specific AI task in mind – I might be able to suggest suitable pre-trained models or optimizations for your Nvidia P4 setup! As a response, I was certainly impressed with it ## Power Consumption Obviously, I am very aware of the power usage on my lab so I thought it was interesting to dig into how much running chat queries may impact things. According to NVTOP The P4 normally idles around 7 Watts, however, once a model is loaded even if it’s not running any queries this increases to around 24W this is probably due to the increased memory clock speed. ![Screenshot 2024 09 05 at 11.57.47](https://jameskilby.co.uk/wp-content/uploads/2024/09/Screenshot-2024-09-05-at-11.57.47.png)Idle consumption ![Screenshot 2024 09 05 at 12.01.26](https://jameskilby.co.uk/wp-content/uploads/2024/09/Screenshot-2024-09-05-at-12.01.26.png)Models loaded ![Screenshot 2024 09 16 at 14.41.52](https://jameskilby.co.uk/wp-content/uploads/2024/09/Screenshot-2024-09-16-at-14.41.52.png)Models Running When queries are running it will typically use 50-60 watts. ## Summary Obviously it’s early days for my experimentation into what is a large and rapidly changing field. But it’s certainly something I will be playing with a lot. ## 📚 Related Posts * [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) * [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) * [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) ## Similar Posts * [ ![MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/wp-content/uploads/2023/04/2157_hi_res-768x346.png) ](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) By[James](https://jameskilby.co.uk) December 19, 2022 · Updated June 1, 2026 For a while, I’ve been looking to update the networking at the core of my homelab. * [ ![Runecast Remediation Scripts: Auto-Fix VMware Storage Issues](https://jameskilby.co.uk/wp-content/uploads/2023/05/Runecast-Solutions-Ltd.png) ](https://jameskilby.co.uk/2023/05/runecast-remediation-scripts/) [Runecast](https://jameskilby.co.uk/category/runecast/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Runecast Remediation Scripts: Auto-Fix VMware Storage Issues](https://jameskilby.co.uk/2023/05/runecast-remediation-scripts/) By[James](https://jameskilby.co.uk) May 16, 2023 · Updated June 1, 2026 I am a huge fan of the Runecast product and luckily as a vExpert they give out NFR licences for my lab. * [ ![VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/wp-content/uploads/2024/01/40oOd8IipPvtrPJs-1198788743-768x737.jpg) ](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) [VCF](https://jameskilby.co.uk/category/vmware/vcf/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) By[James](https://jameskilby.co.uk) January 18, 2024 · Updated June 1, 2026 How to deploy Holodeck with Legacy CPU’s * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Homelab Network Upgrade: DACs, 40Gb/s vMotion & pfSense](https://jameskilby.co.uk/2022/01/lab-update-part-3-network/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated May 31, 2026 I have retired the WatchGuard Devices with the migration to PFSense running bare-metal in one of the Supermicro Nodes. * [ ![Running Nutanix CE at Home: AHV Setup & First Impressions](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2018/01/nutanix-ce/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [Running Nutanix CE at Home: AHV Setup & First Impressions](https://jameskilby.co.uk/2018/01/nutanix-ce/) By[James](https://jameskilby.co.uk) January 6, 2018 · Updated June 1, 2026 I ran a Nutanix CE server at home for a little while when it first came out. However, due to the fairly high requirements, it didn’t make sense to me to continue running it at home. * [ ![How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/wp-content/uploads/2025/01/Screenshot-2025-01-29-at-23.30.47-768x411.png) ](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) ### [How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) By[James](https://jameskilby.co.uk) January 29, 2025 · Updated July 11, 2026 Update: Hoarder has now been renamed to Karakeep due to a trademark issue I’ve been on a mission recently to regain control of my data. --- # Can you really squeeze 96TB in 1U ? Source: https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/ Date: 2024-09-12 Categories: Homelab, Storage, TrueNAS Scale, VMware, VMware Cloud on AWS, Kubernetes, Ansible ![](https://jameskilby.co.uk/wp-content/uploads/2024/09/QuantaGrid-SD1Q-1ULH-Front-Three-Quarter.png) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) | [TrueNAS Scale](https://jameskilby.co.uk/category/truenas-scale/) # Can you really squeeze 96TB in 1U ? By[James](https://jameskilby.co.uk) September 12, 2024 · Updated July 11, 2026 • 📖4 min read(773 words) Yes, that’s a clickbait title. But technically it’s possible if I dropped all drive redundancy… I recently saw an advert for a server that was just too good to be true. It promised to bring a huge amount of storage and future options in a quite hard-to-believe 1U. The price it was on offer for was also unreal. I wasn’t looking for a new storage server but knew I would be able to make good use of this one. It would allow me to consolidate a few bits and pieces so it just had to be done… ## Table of Contents ## Intro The server is a D51PH-1ULH from a vendor called [Quanta Cloud Technology](http://qct.io). They are a vendor that doesn’t seem to be very well known outside of the HyperScalers. It’s described as an Ultra dense storage server and with the capacity for 12×3.5″ plus 4×2.5″ drives it’s hard to argue with that. It also comes with IPMIv2.0 and 2x10Gb SFP+ connections. One really interesting thing is that at first glance the drives don’t look like they are hot-swap. However, the server comes with a giant hard drive tray that slides out the front and then individual drives can be swapped. The seller also threw in a few 480GB Samsung Enterprise SSDs and 16x8TB Seagate Exos making it the bargain of the century. This allowed me to fully populate the chassis and still have 4 8TB drives spare. I typically don’t like using second-hand storage but in this instance, I was willing to make an exception. ![Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/wp-content/uploads/2024/08/IMG_7063-744x1024.jpeg)Server shortly before being installed in my Rack ## OS Setup I have stuck with TrueNAS Scale as the OS of choice for my dedicated storage systems. It’s something I’m familiar with and served my needs in the past well. However, I have decided to leave this as primarily a pure “Storage Server”. The available storage will be served out as a mixture of iSCSI, NFS, and SMB natively by TrueNAS and then I will run a MINIO container on top to offer S3-based object services. ## Hardware I swapped out the 480GB Samsung drives for some larger 2TB consumer-based drives. The current configuration is listed below. The servers also included rails which was a much appreciated benefit. The only downside is I underestimated just how long these servers are. The good news is I can adjust my Startech rack to make it longer and accommodate this. The downside is I need to derack the rest of my gear to do it. So for now it’s sitting on top of one of my other Supermicro servers. CPU| 2x Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz ---|--- Memory | 256GB DDR4 Boot Device | 32 GB SATADom HDD | 12x ST8000NM0075 8TB 12Gb SAS SSD | 3x2TB Samsung Evo NIC | 2x 25Gb ## Pool Configuration For performance reasons, I have chosen to go with 6x Mirrored vDevs 2x wide. Conceptually this is similar to RAID 10. This uses all 12 of the 3.5-inch drives in a single pool. This gives a usable pool capacity of just under 43.5TB Although I lose 50% of the usable capacity I wanted the write performance that this gives This is then complemented by three 2TB SSDs being used as L2ARC (Read cache) and as I have so much read cache the drives basically just do writes. ## Performance I haven’t bothered benchmarking the setup from a performance perspective as I have actual workloads on it that would be annoying to move. However, from what I am seeing in the latency graphs of vCenter I am happy with the results. As you can see below the ARC is holding all of the really hot data in RAM. ![Can you really squeeze 96TB in 1U ? Screenshot](https://jameskilby.co.uk/wp-content/uploads/2024/09/Screenshot-2024-08-29-at-10.02.01.png) The warm data is being served from the 6TB of SSD in the server, however it has not finished warming up yet. The L2ARC data can also be compressed so the reality is that it’s probably closer to the equivalent of 12TB. ![L2ARC Warming up](https://jameskilby.co.uk/wp-content/uploads/2024/09/Screenshot-2024-09-11-at-15.07.42-1024x203.png)L2ARC Warming up A benefit of TrueNAS Scale is that the L2ARC persists between reboots. Otherwise, it would take a very long time to finish warming up with my workload. This means the performance graphs of the actual disks tend to look like this. Only two are shown but all of the main drives appear like this. Yes, there are some reads (Blue) but the majority of the time they are only doing writes ( Pink) for the system. ![Can you really squeeze 96TB in 1U ? Screenshot](https://jameskilby.co.uk/wp-content/uploads/2024/09/Screenshot-2024-09-10-at-11.32.39-1024x513.png) ## Upgrades: ### SATADom I have managed to source a compatible SATADom (32GB rather than the 128GB Partcode SD131C-032GM-JM-A) I was looking for. The migration of the TrueNAS install was super smooth. ### Networking I have also upgraded the NIC to a Quanta ConnectX-4 LX Dual-Port 25GbE ![s l1600 1](https://jameskilby.co.uk/wp-content/uploads/2024/09/s-l1600-1-1024x537.jpg) ## 📚 Related Posts * [Homelab SSD Failure: How Synology RAID Saved My Data](https://jameskilby.co.uk/2022/11/homelab-bad-days-almost/) * [My First Homelab Storage Setup: HP Gen8 & Xpenology](https://jameskilby.co.uk/2018/01/lab-storage/) * [Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/2019/02/lab-storage-2/) ## Similar Posts * [ ![vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) By[James](https://jameskilby.co.uk) November 17, 2023 · Updated June 1, 2026 An Overview of vSAN ESA in VMC * [ ![Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-12-14-at-21.45.23.png) ](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) By[James](https://jameskilby.co.uk) December 14, 2022 · Updated June 1, 2026 I run a reasonably extensive homelab that is of course built around the VMware ecosystem. * [ ![TrueNAS Logo](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-22-at-18.49.21-768x198.png) ](https://jameskilby.co.uk/2023/11/truenas-scale-useful-commands/) [Kubernetes](https://jameskilby.co.uk/category/kubernetes/) | [TrueNAS Scale](https://jameskilby.co.uk/category/truenas-scale/) ### [TrueNAS Scale CLI Reference: Storage & Kubernetes Commands](https://jameskilby.co.uk/2023/11/truenas-scale-useful-commands/) By[James](https://jameskilby.co.uk) November 13, 2023 · Updated April 11, 2026 A list of useful Truenas Scale commands * [ ![Managing my Homelab with SemaphoreUI](https://jameskilby.co.uk/wp-content/uploads/2025/07/semaphore-768x768.png) ](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Managing my Homelab with SemaphoreUI](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/) By[James](https://jameskilby.co.uk) September 2, 2025 · Updated June 5, 2026 An intro on how I use SemaphoreUI to manage my Homelab * [ ![Homelab SSD Failure: How Synology RAID Saved My Data](https://jameskilby.co.uk/wp-content/uploads/2022/11/BrokenHardDive-1200x630-1-768x403.jpg) ](https://jameskilby.co.uk/2022/11/homelab-bad-days-almost/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Homelab SSD Failure: How Synology RAID Saved My Data](https://jameskilby.co.uk/2022/11/homelab-bad-days-almost/) By[James](https://jameskilby.co.uk) November 21, 2022 · Updated July 11, 2026 I recently spent 3 weeks in Ireland with my wife Wendy and our son Nate. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [My First Homelab Storage Setup: HP Gen8 & Xpenology](https://jameskilby.co.uk/2018/01/lab-storage/) By[James](https://jameskilby.co.uk) January 6, 2018 · Updated June 1, 2026 I have been meaning to post around some of the lab setup for a while. Although it changes frequently at present it’s as below. --- # Home Network Upgrade to 25Gb/s with MikroTik Switching Source: https://jameskilby.co.uk/2024/09/home-network-upgrade/ Date: 2024-09-09 Categories: Mikrotik, Networking, Automation, Github, Homelab, Hosting, Veeam, Artificial Intelligence, Docker, VCF, VMware ![CRS-504](https://jameskilby.co.uk/wp-content/uploads/2024/09/s-l1600.jpg) [Mikrotik](https://jameskilby.co.uk/category/mikrotik/) | [Networking](https://jameskilby.co.uk/category/networking/) # Home Network Upgrade to 25Gb/s with MikroTik Switching By[James](https://jameskilby.co.uk) September 9, 2024 · Updated June 5, 2026 • 📖2 min read(475 words) I have recently moved over to Mikrotik Switching for all of my Home environment as part of my home network upgrade. What started life as a move away from gigabit networking had some serious scope creep. As I needed new switches, network adaptors and cabling I evaluated the obvious choice of just upgrading to 10Gb/s. However, when looking at the price comparison to upgrade to 25Gb/s I felt the cost uplift was justified and certainly better future-proofed. This became the baseline for the majority of my server workloads. The Access switch layer has basically stayed gigabit but with 10Gb/s uplinks. This followed on from my recent [homelab compute upgrade](https://jameskilby.co.uk/2024/07/new-nodes/). ## Table of Contents ## Why I Did This Home Network Upgrade The decision to go to Mikrotik was predominantly based on the huge performance available for limited costs. So far I have been impressed with the switches once I got over the initial steep learning curve of RouterOS. I have purchased several switches spread out over a few months. This coincided with a property move so some extras were bought to accommodate the new layout. In the property, my lab was relocated to a dedicated space and I also wanted wired connectivity into my office. ## Switches Purchased Number| Model| £Price | Function| OS ---|---|---|---|--- 1| [CRS504_4XQ](https://mikrotik.com/product/crs504_4xq_in)| 704.40| Core Switch| RouterOS 1| [CRS305-1G-4S+IN](https://mikrotik.com/product/crs305_1g_4s_in)| 131.15| Breakout Switch| RouterOS 2| [CSS610-8P-2S+IN](https://mikrotik.com/product/css610_8p_2s_in)| 193.46| House and Office Switch| SwitchOS Lite 1| [CSS610-8G-2S+](https://mikrotik.com/product/css610_8g_2s_in)| 183.87| OOB Switch| SwitchOS Lite Total| | 1406.34| | The 100Gb/s Switch is effectively my core switch. It only has 4 physical ports however each of these can be split into 4x25gb/s or slower ports if needed. In the current state all 4 ports are used, however I do have 3 remaining breakouts available if needed. I have then used the 5-Port 10 Gb/s switch sort of as a media translator for connections to the access switches around the house with 10Gb uplinks to distribute the network around the home and present ethernet ports to my devices. ![Network Overview](https://jameskilby.co.uk/wp-content/uploads/2025/01/Home-Net-overview-1024x633.png) ## Configuring the MikroTik Switches Mikrotik Switches can typically run one of two operating systems either RouterOS or SwitchOS. (The 3 ethernet port switches will only run SwOS) SwitchOS is a much simpler setup and is similar to a web-managed switch from other manufacturers. It’s a trivial setup for anyone that’s used a web-managed switch before. The config for these switches takes a little bit of time to get familiar with as they are incredibly powerful. It’s possible to run RouterOS or SwitchOS on them however I could never get SwitchOS to boot correctly on the 100Gb/s switch so I stuck with RouterOS and persevered through the config. Part 2 will go into my RouterOS setup. I have a draft post of the steps I have used to get RouterOS configured how I like. I will try and finish that off when I get some more time. ## 📚 Related Posts * [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) * [Homelab Network Upgrade: DACs, 40Gb/s vMotion & pfSense](https://jameskilby.co.uk/2022/01/lab-update-part-3-network/) * [Unleashing the UDM Beast](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) ## Similar Posts * [ ![Configuring a Zen Internet and City Fibre connection with a 3rd party router](https://jameskilby.co.uk/wp-content/uploads/2023/11/cityfibre-zen-768x403.jpg) ](https://jameskilby.co.uk/2023/11/configuring-a-zen-internet-and-city-fibre-connection-with-a-3rd-party-router/) [Networking](https://jameskilby.co.uk/category/networking/) ### [Configuring a Zen Internet and City Fibre connection with a 3rd party router](https://jameskilby.co.uk/2023/11/configuring-a-zen-internet-and-city-fibre-connection-with-a-3rd-party-router/) By[James](https://jameskilby.co.uk) November 15, 2023 · Updated June 1, 2026 Back in July I bought a new house and one of the best things about the property was that it was already in a City Fibre location. * [ ![Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/wp-content/uploads/2026/04/packer-github-actions-vsphere-pipeline-768x452.png) ](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) [Automation](https://jameskilby.co.uk/category/automation/) | [Github](https://jameskilby.co.uk/category/github/) ### [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) By[James](https://jameskilby.co.uk) April 30, 2026 · Updated June 5, 2026 A walk-through of an automated golden-image pipeline for vSphere: HashiCorp Packer builds six Ubuntu LTS templates (22.04, 24.04, 26.04 — server and desktop) driven by three GitHub Actions workflows covering PR validation, parallel template builds, and ISO management via a self-hosted runner. * [ ![Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/wp-content/uploads/2022/10/spacexs-starlink-to-supply-free-satellite-internet-to-famili_u44u.1920-768x432.jpg) ](https://jameskilby.co.uk/2022/10/starlink/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/2022/10/starlink/) By[James](https://jameskilby.co.uk) October 11, 2022 · Updated June 1, 2026 Since moving to Dorset last year internet connectivity has been the bane of my existence. Currently, I have an ADSL connection provided by my old employer Zen and a 5G connection provided by Three. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Veeam](https://jameskilby.co.uk/category/veeam/) ### [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated June 1, 2026 My lab is always undergoing change. Partially as I want to try new things or new ways of doing things. * [ ![Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU](https://jameskilby.co.uk/wp-content/uploads/2024/10/pexels-tara-winstead-8386440-768x512.jpg) ](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) ### [Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) By[James](https://jameskilby.co.uk) October 11, 2024 · Updated July 11, 2026 Artificial intelligence is all the rage at the moment, It’s getting included in every product announcement from pretty much every vendor under the sun. * [ ![VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/wp-content/uploads/2024/01/40oOd8IipPvtrPJs-1198788743-768x737.jpg) ](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) [VCF](https://jameskilby.co.uk/category/vmware/vcf/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) By[James](https://jameskilby.co.uk) January 18, 2024 · Updated June 1, 2026 How to deploy Holodeck with Legacy CPU’s --- # New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix Source: https://jameskilby.co.uk/2024/07/new-nodes/ Date: 2024-07-02 Categories: Homelab, Nutanix, VMware, VCF, Artificial Intelligence, Automation, Storage, VMware Cloud on AWS ![](https://jameskilby.co.uk/wp-content/uploads/2024/07/IMG_6629-scaled.jpeg) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) | [VMware](https://jameskilby.co.uk/category/vmware/) # New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix By[James](https://jameskilby.co.uk) July 2, 2024 · Updated July 11, 2026 • 📖8 min read(1,513 words) This post covers how I built my new homelab nodes using SuperMicro BigTwin hardware for VMware and Nutanix. I recently decided to update some of my homelab hosts and I managed to do this at very little cost by offloading 2 of my [Supermicro e200’s](https://www.supermicro.com/en/products/system/mini-itx/sys-e200-8d.cfm) to fellow vExpert [Paul](https://ssh.guru). The below post describes what I bought, why, and how I have configured it. I have previously written about [running Nutanix CE at home](https://jameskilby.co.uk/2018/01/nutanix-ce/). ## Table of Contents ## Choosing My New Homelab Nodes I have been very happy with the Supermicro Twin concept having run this for a few years. So I decided I would stick with it. For anyone not familiar you get up to 4 compute nodes running in a 2U setup. It’s a very dense format and as a number of components are shared I believe it’s very power efficient. Looking at the available nodes on eBay and second hand marketplaces I spotted several Nutanix nodes at a decent price. I ended up buying a 3 node NX-1365-G4 setup. These came with CPU, Memory and NICs. I decided to purchase some Enterprise SATA SSDs and utilised some of the existing consumer-based SSDs I had as storage. I then purchased some SATADom’s to use as the Boot device. One of the really nice features is that the IPMI is sort of cluster aware. From a single location you can view the power usage (and health) of all of the nodes. In the below image I was logged into the IPMI for Node A but had a view of Node B and C. ![New Nodes Screenshot](https://jameskilby.co.uk/wp-content/uploads/2024/04/Screenshot-2024-04-06-at-22.50.57.png) I have chosen to deploy Nutanix Community Edition on these nodes but still retain vSphere as the underlying hypervisor. I am very familiar with this configuration having run it in production at a service provider for a couple of years. This also gives me the additional Storage capacity of the Enterprise SSD’s something that would not be available had I chosen to deploy vSAN OSA. I am not certain the hardware will run vSAN ESA but I will likely try this at some point. This also allowed me to plugin external NFS or iSCSI storage into vSphere. Something I am not certain is possible if AHV was the underlying Hypervisor. ## Bill of Materials Description| Quantity| Component Price £| Line Total £| Sourced from ---|---|---|---|--- 3x Nutanix nodes | 1 | 563.76 | 563.76 | Ebay 1TB Samsung enterprise SATA SSD | 3 | 40.00 | 120.00 | Ebay 2TB Samsung EVO Consumer SATA SSD | 6 | 150.00 | 900.00 | Removed from TrueNAS (Not included in total cost) 32GB SATADom | 3 | 42.00 | 126.00 | Ebay SSD Caddy | 9 | 10.00 | 90.00 | Ebay QSFP28 to SFP+ Breakout cable | 1 | 29.99 | 29.99 | Ebay | | | | Total | | | 929.75 | ## Rescue IPMI The IPMI in the nodes was set to ipv6 only and I didn’t know the password. To set them to DHCP was easy in the bios, however you can’t set the password from the bios. To reset the passwords I resorted to using the [IPMItool](https://github.com/ipmitool/ipmitool). Due to some security [changes](https://vswitchzero.com/ipmitool-vib/) in ESXi8 it was necessary to install ESXi7 to reset the password. To do this I did a vanilla install of ESXi then uploaded the ipmitool to the datastore before running the below command /opt/ipmitool/ipmitool raw 0x30 0x40 📋 Copy This set the IPMI to factory defaults and took about 30 seconds to complete. From there I could configure as required. ## Nutanix CE install Overview The next step was the install of Nutanix CE. This requires deploying the installer to a suitable USB or similar device. As I wanted to utilise ESXi as the underlying Hypervisor an additional step is needed you need to have the ESXi installer ISO available on a webserver. It’s useful to plan your IP addresses in advance. This is what I used NODE| ESX MANAGEMENT| CVM| DNS Record ---|---|---|--- NODE A | 192.168.38.171 | 192.168.38.172 | uk-bhr-p-ntnx-a.jameskilby.cloud NODE B | 192.168.38.173 | 192.168.38.174 | uk-bhr-p-ntnx-b.jameskilby.cloud NODE C | 192.168.38.174 | 192.168.38.175 | uk-bhr-p-ntnx-c.jameskilby.cloud ## Web Server For this I used an existing windows server and added the IIS role. I then uploaded the ISO to the IIS directory. An additional step is to add the correct MIME type. You must add the correct MIME type for the file you’re trying to download. From IIS Manager, go to Sites > YOUR_SITE> and in the features view, double-click MIME Types. Under actions, click Add. Enter the file extension **iso** and the MIME type **application/octetstream** ![mimetype](https://jameskilby.co.uk/wp-content/uploads/2024/06/mimetype.jpg) ## Install ![New Nodes Screenshot](https://jameskilby.co.uk/wp-content/uploads/2024/04/Screenshot-2024-04-05-at-11.57.17.png) The install is fairly straightforward if your default access vlan is the one you want to run Nutanix on. In the end I changed the switch port vlan configuration to this however it doesn’t match the rest of my other VMware setup. As I wanted to create a 3 node configuration I have not selected the Create single-node cluster option. With the ESXi option selected make sure you have the correct URL as it takes ages to time out if it’s incorrect. The webserver needs to be accessible from the IP range the host/cvm will be provisioned from so for ease I kept everything in the same Layer 2 domain. ![Screenshot 2024 04 05 at 13.21.24](https://jameskilby.co.uk/wp-content/uploads/2024/04/Screenshot-2024-04-05-at-13.21.24-1024x700.png) Successful Install Once the install is complete it will ask you to reboot as seen above It will then boot into the ESXi Hypervisor. Once this happens be patient as Nutanix will configure all of the relevant settings on the host (network config etc) utilising a kick start file. It will then deploy the CVM. The host will also restart automatically as part of this process. ![Screenshot 2024 04 05 at 12.18.02 1](https://jameskilby.co.uk/wp-content/uploads/2024/04/Screenshot-2024-04-05-at-12.18.02-1.png) Some of the configuration taking place ## Cluster Creation When all of the nodes are imaged the next step is to create the Nutanix Cluster. SSH into any of the CVM’s as the user nutanix with a password of nutanix/4u Then execute the following command utilising your CVM IP addresses cluster -s 192.168.38.171,192.168.38.173,192.168.38.175 create 📋 Copy A successful cluster creation will look similar to the below. If you have errors with this step the most likely cause is physical network configuration. VM: 192.168.38.175 Up Zeus UP [8656, 8704, 8705, 8706, 8715, 8733] Scavenger UP [12061, 12187, 12188, 12189] Xmount UP [12058, 12159, 12160, 12259] SysStatCollector UP [14106, 14240, 14241, 14242] IkatProxy UP [15420, 15555, 15556, 15557] IkatControlPlane UP [16277, 16378, 16379, 16382] SSLTerminator UP [16308, 16510, 16511] SecureFileSync UP [16445, 16586, 16587, 16588] Medusa UP [16665, 16804, 16805, 16807, 17410] DynamicRingChanger UP [19169, 19305, 19306, 19346] Pithos UP [19264, 19471, 19472, 19484] InsightsDB UP [19340, 19550, 19551, 19566] Athena UP [19500, 19645, 19646, 19647] Mercury UP [27637, 27684, 27685, 27689] Mantle UP [19616, 19873, 19874, 19882] VipMonitor UP [22311, 22312, 22313, 22314, 22318] Stargate UP [20732, 20844, 20845, 20853, 20854] InsightsDataTransfer UP [21521, 21609, 21610, 21651, 21652, 21654, 21657, 21659, 21660] Ergon UP [21550, 21767, 21768, 21769, 22082] GoErgon UP [21847, 21919, 21920, 21962] Cerebro UP [21864, 22142, 22143, 22408] Chronos UP [21928, 22348, 22349, 22375] Curator UP [22404, 22625, 22626, 22704] Prism UP [22631, 22850, 22851, 22865] Hera UP [22852, 23014, 23015, 23016] CIM UP [22888, 23127, 23128, 23134, 23135] AlertManager UP [23080, 23299, 23300, 23411] Arithmos UP [23138, 23407, 23408, 23476] Catalog UP [23481, 23626, 23627, 23628] Acropolis UP [23581, 23740, 23741, 23742] Uhura UP [23647, 23825, 23826, 23827] NutanixGuestTools UP [23764, 23960, 23961, 23973, 24035] MinervaCVM UP [24811, 24908, 24909, 24910, 25208] ClusterConfig UP [24835, 25000, 25001, 25002, 25044] APLOSEngine UP [24876, 25134, 25135, 25136] APLOS UP [25333, 25522, 25523, 25524] PlacementSolver UP [25404, 25593, 25594, 25595, 25616] Lazan UP [25494, 25741, 25742, 25743] Polaris UP [25628, 25935, 25936, 25964] Delphi UP [25886, 26101, 26102, 26103, 26157] Security UP [25973, 26247, 26248, 26249] Flow UP [26059, 26352, 26353, 26354, 26375] Anduril UP [26205, 26500, 26501, 26502, 26522] XTrim UP [26363, 26595, 26596, 26597] ClusterHealth UP [26679, 26775, 27021, 27022, 27030, 27036, 27095, 27096, 27101, 27103, 27104, 27125, 27126, 27128, 27136, 27137, 27138, 27139, 27140, 27142, 27226, 27227, 27251, 27252, 27253, 27254, 27257, 27258, 29422, 29423, 29425, 29426, 29428, 29429, 29430, 29431, 29433, 29434, 29435, 29436, 29451, 29453, 29528, 29531] 2024-04-05 12:48:17,004Z INFO MainThread cluster:3104 Success! 📋 Copy Name the cluster (Optional) ncli cluster edit-params new-name=ntnx01 📋 Copy Add at least 1 DNS server ncli cluster add-to-name-servers servers=192.168.10.250 📋 Copy ## VMware Config As I wanted to add the Nutanix Hosts into my vCenter one step I needed to do was enable EVC. To do this no running VM’s can be present on the host. Log into one of the CVM’s and execute a “Cluster Stop” command. When the cluster is stopped, shut all the CVM’s down. When the CVM’s were shut down. The 3 ESXi hosts were placed in maintenance mode before adding to an existing vSphere cluster. This had the correct EVC mode that I needed. Once this is done the CVM’s can be booted again and the Nutanix cluster starting with a “Cluster Start” command ### Distributed Switch I also wanted to use DVswitch within VMware so a network migration was performed. ## Nutanix Further Configuration The next steps are to deploy Prism Central and I also want to increase the RAM in the CVM’s. I will address this in a separate post. ## The finished result ![IMG 6628 1](https://jameskilby.co.uk/wp-content/uploads/2024/07/IMG_6628-1-1024x372.jpeg) The Nutanix Nodes are the first three nodes of the lower unit. The fourth is just blanks. ![New Nodes Screenshot](https://jameskilby.co.uk/wp-content/uploads/2024/07/Screenshot-2024-07-02-at-08.56.49-1.png) ## 📚 Related Posts * [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) * [Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup](https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/) * [How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) ## Similar Posts * [ ![VMware Holodeck Multi-Host VCF: Lab Setup & Configuration](https://jameskilby.co.uk/wp-content/uploads/2023/12/Holodeck-Overview.png) ](https://jameskilby.co.uk/2024/01/multihost-holodeck-vcf/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VCF](https://jameskilby.co.uk/category/vmware/vcf/) ### [VMware Holodeck Multi-Host VCF: Lab Setup & Configuration](https://jameskilby.co.uk/2024/01/multihost-holodeck-vcf/) By[James](https://jameskilby.co.uk) January 17, 2024 · Updated June 1, 2026 How to Deploy VMware Holodeck on multiple hosts * [ ![Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup](https://jameskilby.co.uk/wp-content/uploads/2023/04/IMG_4536-scaled-1-768x1024.jpg) ](https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup](https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated February 16, 2026 Quite a few changes have happened in the lab recently. I decided to do a multipart blog on the changes. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) By[James](https://jameskilby.co.uk) March 27, 2026 · Updated May 31, 2026 A walkthrough of my self-hosted AI stack: Ollama, Open WebUI, ComfyUI, Whishper, n8n, Qdrant, SearxNG, and a full observability layer — all running on my own hardware with Docker Compose. * [ ![TrueNAS Logo](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-22-at-18.49.21-768x198.png) ](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Homelab Storage Refresh (Part 1)](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/) By[James](https://jameskilby.co.uk) May 23, 2023 · Updated July 11, 2026 Table of Contents Background ZFS Overview Read Cache (ARC and L2ARC) ZIL (ZFS Intent Log) Hardware Background I have just completed the move of all my production and media-based storage/services to TrueNAS Scale. ( I will just refer to this as TrueNAS) This is based on my HP Z840 and I have now retired my… * [ ![An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) By[James](https://jameskilby.co.uk) August 14, 2025 · Updated July 11, 2026 This is single page intended to collate every single feature of the current VMware Cloud on AWS hosts for easy comparison. * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/09/vmc-host-errors/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) By[James](https://jameskilby.co.uk) September 15, 2020 · Updated June 5, 2026 Learn how host failures are handled within VMC --- # Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets Source: https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/ Date: 2024-06-26 Categories: Homelab, Networking, VMware, Nutanix, Mikrotik, Automation ![](https://jameskilby.co.uk/wp-content/uploads/2024/06/Ubiquiti_Networks-Logo.wine_.png) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) # Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets By[James](https://jameskilby.co.uk) June 26, 2024 · Updated July 11, 2026 • 📖2 min read(302 words) ## Table of Contents ## Intro This guide shows how to configure DHCP Option 43 so UniFi devices can be adopted remotely across subnets. I have recently juggled some of my homelab services around, which involved moving my UniFi server to a different subnet. This meant that new UniFi devices could not find the UniFi server for adoption/control purposes as this is usually done with broadcast. This was part of my wider [home network upgrade](https://jameskilby.co.uk/2024/09/home-network-upgrade/). ## Layer 3 Adoption Options For Remote Adoption/Layer 3 UniFi supports: * SSH * DNS * DHCP Option 43 I discounted SSH as who wants to manually log into things and make changes and I discounted DNS. That left me with DHCP Option 43. ## Configuring DHCP Option 43 I decided to configure my DHCP server to hand out “option 43” on the network subnet the Access Points were connected to. The implementation of this will differ depending on what DHCP server you are using. Mine is the one built into my WatchGuard Firebox M200. In this, I have set a custom option and then it asks for four values: Code| 43 ---|--- Name | Unifi Type | Hexadecimal Value | 01:04:c0:a8:14:02 All of which are straightforward except the last one so let me explain. It is made up of 2 parts… The first two parts 01:04 are fixed for UniFi. I am hosting the controller on IP 192.168.20.2. The remaining part of the value (c0:a8:14:02) is the IP address of the UniFi controller in Hex. Please note this method does not support DNS. A multitude of tools exist for converting IP to HEX one can be found [here](https://www.browserling.com/tools/ip-to-hex) And there you have it. I can connect a factory fresh UniFi device to my network. DHCP will hand out an IP on one subnet and tell the device how to reach the controller in a different subnet. ## 📚 Related Posts * [Unleashing the UDM Beast](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) * [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) * [Homelab Network Upgrade: DACs, 40Gb/s vMotion & pfSense](https://jameskilby.co.uk/2022/01/lab-update-part-3-network/) ## Similar Posts * [ ![Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup](https://jameskilby.co.uk/wp-content/uploads/2023/04/IMG_4536-scaled-1-768x1024.jpg) ](https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup](https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated February 16, 2026 Quite a few changes have happened in the lab recently. I decided to do a multipart blog on the changes. * [ ![Running Nutanix CE at Home: AHV Setup & First Impressions](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2018/01/nutanix-ce/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [Running Nutanix CE at Home: AHV Setup & First Impressions](https://jameskilby.co.uk/2018/01/nutanix-ce/) By[James](https://jameskilby.co.uk) January 6, 2018 · Updated June 1, 2026 I ran a Nutanix CE server at home for a little while when it first came out. However, due to the fairly high requirements, it didn’t make sense to me to continue running it at home. * [ ![CRS-504](https://jameskilby.co.uk/wp-content/uploads/2024/09/s-l1600-768x427.jpg) ](https://jameskilby.co.uk/2024/09/home-network-upgrade/) [Mikrotik](https://jameskilby.co.uk/category/mikrotik/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Home Network Upgrade to 25Gb/s with MikroTik Switching](https://jameskilby.co.uk/2024/09/home-network-upgrade/) By[James](https://jameskilby.co.uk) September 9, 2024 · Updated June 5, 2026 My journey to superfast networking in my homelab * [ ![MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/wp-content/uploads/2023/04/2157_hi_res-768x346.png) ](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) By[James](https://jameskilby.co.uk) December 19, 2022 · Updated June 1, 2026 For a while, I’ve been looking to update the networking at the core of my homelab. * [ ![Configuring a Zen Internet and City Fibre connection with a 3rd party router](https://jameskilby.co.uk/wp-content/uploads/2023/11/cityfibre-zen-768x403.jpg) ](https://jameskilby.co.uk/2023/11/configuring-a-zen-internet-and-city-fibre-connection-with-a-3rd-party-router/) [Networking](https://jameskilby.co.uk/category/networking/) ### [Configuring a Zen Internet and City Fibre connection with a 3rd party router](https://jameskilby.co.uk/2023/11/configuring-a-zen-internet-and-city-fibre-connection-with-a-3rd-party-router/) By[James](https://jameskilby.co.uk) November 15, 2023 · Updated June 1, 2026 Back in July I bought a new house and one of the best things about the property was that it was already in a City Fibre location. * [ ![Template Deployment with Packer](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) ](https://jameskilby.co.uk/2021/01/hashicorp-packer/) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. --- # VMware Holodeck on Older CPUs: Fixing Compatibility Issues Source: https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/ Date: 2024-01-18 Categories: VCF, VMware, Automation, Homelab, Artificial Intelligence, TrueNAS Scale, Storage, Nutanix ![](https://jameskilby.co.uk/wp-content/uploads/2024/01/40oOd8IipPvtrPJs-1198788743.jpg) [VCF](https://jameskilby.co.uk/category/vmware/vcf/) | [VMware](https://jameskilby.co.uk/category/vmware/) # VMware Holodeck on Older CPUs: Fixing Compatibility Issues By[James](https://jameskilby.co.uk) January 18, 2024 · Updated June 1, 2026 • 📖1 min read(268 words) Disclaimer: This is **not** a supported configuration by the Holodeck team, please don’t reach out to them for help. No support will be given for running CPUs without the required feature sets. ## Table of Contents In my [previous post](https://jameskilby.co.uk/2024/01/multihost-holodeck-vcf/) about my Holodeck experience, I mentioned that I had some issues due to the age of the physical CPUs in the hosts that I was using to run Holodeck. This error will manifest itself in three places during the deployment. * Nested ESX Power On * vCLS machine deployment * NSX Edge deployment ## Nested ESX Power On Obviously, my CPUs are not supported for ESXi 8.0.1 and when the host is powered on you will see the image below. ![ESXi unsupported error](https://jameskilby.co.uk/wp-content/uploads/2023/09/Screenshot-2023-09-27-at-12.55.09.png) Just like with a physical host, it is possible to override this. This is done by adding --ignoreprereqwarnings --ignoreprereqerrors --forceunsupportedinstall 📋 Copy to the VLCGUI.ps1 script. For the same reasons when the cluster is built and the vCLS VMs for DRS are deployed, they will fail to power on. ## NSX Edge ![Holodeck CPU Fixes Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/10/Screenshot-2023-10-02-at-14.28.25.png) And last of all when the NSX edges attempt to power on they will fail due to the lack of a feature in the CPU called 1G huge page support. These issues can manifest when deploying NSX outside of VCF so a lot has been written about these issues and how to overcome them. ## Solution Luckily, a colleague of mine, Tim Sommer, has made all of the required changes to the VLCGUI.ps1 deployment script and that is available [here ](https://ent.box.com/s/u4wiwh2mq8o05ct8e67ndapvxhudkhe2) I have tried this multiple times and I have had a 100% success rate with the deployment with no manual fixes being required. ## 📚 Related Posts * [VMware Holodeck Multi-Host VCF: Lab Setup & Configuration](https://jameskilby.co.uk/2024/01/multihost-holodeck-vcf/) * [How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) * [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) ## Similar Posts * [ ![Template Deployment with Packer](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) ](https://jameskilby.co.uk/2021/01/hashicorp-packer/) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. * [ ![Wa](https://jameskilby.co.uk/wp-content/uploads/2025/04/210902461-012e7273-413a-4ec7-be44-e854347f5a21-768x180.png) ](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Warp – The intelligent terminal](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) By[James](https://jameskilby.co.uk) April 11, 2025 · Updated June 1, 2026 How Warp is helping me run my homelab. * [ ![How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/wp-content/uploads/2024/12/ZFS.jpg) ](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) [TrueNAS Scale](https://jameskilby.co.uk/category/truenas-scale/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) By[James](https://jameskilby.co.uk) December 18, 2024 · Updated June 1, 2026 Introduction Copy on Write Disk IDs Trim Introduction I have run a number of systems using ZFS since the earliest days of my homelab using Nexenta, all the way back in 2010. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 Part 2 of my self-hosted AI stack series. I cover container resource sizing, dual-network isolation via Traefik and Cloudflare Tunnels, and every database powering the stack — PostgreSQL, ClickHouse, Redis, Qdrant, MinIO, MongoDB, SQLite, Prometheus, and Jaeger — plus the backup strategy for each. * [ ![Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/wp-content/uploads/2023/04/intel_optane_ssd_900p_series_aic_-_right_angle_575px.png) ](https://jameskilby.co.uk/2023/04/intel-optane/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) By[James](https://jameskilby.co.uk) April 17, 2023 · Updated June 1, 2026 I have been a VMware vExpert for many years and it has brought me many benefits over the years. * [ ![New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/wp-content/uploads/2024/07/IMG_6629-768x149.jpeg) ](https://jameskilby.co.uk/2024/07/new-nodes/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) By[James](https://jameskilby.co.uk) July 2, 2024 · Updated July 11, 2026 I recently decided to update some of my homelab hosts and I managed to do this at very little cost by offloading 2 of my Supermicro e200’s to fellow vExpert Paul . --- # VMware Holodeck Multi-Host VCF: Lab Setup & Configuration Source: https://jameskilby.co.uk/2024/01/multihost-holodeck-vcf/ Date: 2024-01-17 Categories: VMware, VCF, VMware Cloud on AWS, Artificial Intelligence, Automation ![](https://jameskilby.co.uk/wp-content/uploads/2023/12/Holodeck-Overview.png) [VMware](https://jameskilby.co.uk/category/vmware/) | [VCF](https://jameskilby.co.uk/category/vmware/vcf/) # VMware Holodeck Multi-Host VCF: Lab Setup & Configuration By[James](https://jameskilby.co.uk) January 17, 2024 · Updated June 1, 2026 • 📖4 min read(891 words) Disclaimer: This is **not** a supported configuration by the Holodeck team, please don’t reach out to them for help. As of today the only way to run Holodeck and receive support is to do it on a single compute node. ## Table of Contents The [VCF Holodeck](https://core.vmware.com/introducing-holodeck-toolkit) toolkit is an excellent tool for learning not just VCF but a lot of technology that can be run on top of it. e.g. HCX, Tanzu vRA etc. Several guides have been written to assist with that. However, the downside is that you need a single large fast vSphere host to run it on. The minimum specs are listed below for a few different configuration options. This shouldn’t be a challenge for a company or partner to provide but as a homelab enthusiast, this was always going to be a challenge. Many folks I know can cover the below resource requirements with the homelab they have. But the challenge is doing this with a single node. This was also going to be an issue for me. Therefore I needed to come up with a plan to make this work across at least 2 hosts… ## Holodeck recommended specifications Minimum Hardware (1 env)| Recommended Hardware (2 envs)| Recommended Hardware (3-5 envs) ---|---|--- VCF Consolidated | VCF Standard (MGMT + WLD) | VCF Standard (MGMT + WLD) 2 sockets – Total 16 cores | 2 sockets – Total 32 core | 2 sockets – Total 64 cores 384 GB RAM | 1024 GB RAM | 1.5 TB RAM 3.5 TB SSD Disk | 2 – 3.5 TB SSD Disk | 4 – 3.5TB SSD Disks I was planning to use two of my existing Supermicro hosts that were already connected to my vCenter. They each have 2x Intel Xeon CPU E5-2670 @ 2.60GHz with 192GB of RAM each. The combined resources should be enough to run Holodeck and experiment with other toolsets on top. All of the storage is to be presented by my [TrueNas](https://jameskilby.co.uk/lab/) setup. ## Networking Both hosts were originally connected through a DVSwitch and 2x25Gb/s physical adaptors. To change the configuration for Holodeck I removed one of the adaptors in each host from the DVswitch. I then followed the Holodeck guide and configured the standard switch as required on each host. I then added the unused 25Gb/s adaptor to the standard switch and then physically connected the two hosts with a short DAC cable as seen below. ![MultiHost Holodeck VCF](https://jameskilby.co.uk/wp-content/uploads/2024/01/IMG_5286-1-1024x996.jpeg)Physical Network ![Standard Switch](https://jameskilby.co.uk/wp-content/uploads/2024/01/Standard-Switch-1024x299.png)Standard Switch configuration ## DNS Holodeck can be deployed either directly to an ESX Host or through a vCenter. As my VC and Hosts are part of the jameskilby.cloud domain I needed to ensure that the private IPs were being returned to the Holodeck Console VM for deployment. This can be handled in several ways however I believe the most elegant is to edit the holohosts.txt file on the HoloBuilder VM. This host file is copied over to the HoloConsole VM as part of the deployment and therefore the builder can always resolve the correct IPs irrespective of the state of the DNS on the CloudBuilder/HoloConsole VM. The file is located at C:\Users\Administrator\Downloads\holodeck-standard-main2.0\holodeck-standard-main\Holo-Console\holohosts.txt 📋 Copy I have added these values for my VC and the 2 Hosts that will be used as seen below. # Enter physical hosts or vCenter Server instances supporting Holodeck Deployment below 10.203.42.1 w4-hs6-i1209.eng.vmware.com 192.168.38.19 uk-poo-p-vc-1.jameskilby.cloud 192.168.38.20 uk-bhr-p-esx-a.jameskilby.cloud 192.168.38.21 uk-bhr-p-esx-b.jameskilby.cloud # Remaining entries are used for standardized Holdeck deployment 10.0.0.101 esxi-1.vcf.sddc.lab 10.0.0.102 esxi-2.vcf.sddc.lab 10.0.0.103 esxi-3.vcf.sddc.lab 10.0.0.104 esxi-4.vcf.sddc.lab 10.0.0.4 sddc-manager.vcf.sddc.lab 10.0.0.12 vcenter-mgmt.vcf.sddc.lab 10.0.0.20 nsx-mgmt.vcf.sddc.lab 10.60.0.150 vrslcm.vcf.sddc.lab 10.60.0.151 ws1.vcf.sddc.lab 10.60.0.170 vra.vcf.sddc.lab 10.0.0.150 kubeapi.vcf.sddc.lab 10.0.20.101 esxi-1.vcf2.sddc.lab 10.0.20.102 esxi-2.vcf2.sddc.lab 10.0.20.103 esxi-3.vcf2.sddc.lab 10.0.20.104 esxi-4.vcf2.sddc.lab 10.0.20.4 sddc-manager.vcf2.sddc.lab 10.0.20.12 vcenter-mgmt.vcf2.sddc.lab 10.0.20.20 nsx-mgmt.vcf2.sddc.lab 10.60.20.150 vrslcm.vcf2.sddc.lab 10.60.20.151 ws1.vcf2.sddc.lab 10.60.20.170 vra.vcf2.sddc.lab 10.0.20.150 kubeapi.vcf2.sddc.lab 📋 Copy ## vCenter Configuration As I was deploying to a preexisting cluster I needed to make some small changes before Holodeck would deploy. HA needs to be disabled and DRS either needs to be off or partially automated. I never like switching DRS off ( if you’ve ever used vCloud Director you will know why) so I went with partially automated. ## CPU/EVC config The last step I personally needed to do (and I suspect this is due to the age of the CPUs in my hosts) was to disable EVC. I would suggest attempting to deploy with this in place and only turning it off if required. I will detail some of the other fixes I needed specifically due to my CPU age in a separate [blog. ](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) ## Holodeck deployment With these fixes in place, it was time to run the VLCGui Powershell file. As you can see below this was pointed at my VC with the cluster used for deployment highlighted. ![MultiHost Holodeck VCF Screenshot](https://jameskilby.co.uk/wp-content/uploads/2024/01/Screenshot-2024-01-19-at-09.08.41-1-1024x545.png) ## Success With the above in place the full deployment of the 7 vSphere hosts, NSX managers & Edges plus Tanzu is done in 4hr 11 Mins. ![MultiHost Holodeck VCF Screenshot](https://jameskilby.co.uk/wp-content/uploads/2024/01/Screenshot-2024-01-17-at-15.39.41-1024x549.png)Successful Deployment ## Deployment Notes A few things that catch people out specifically with Holodeck. The first is if you happen to restart the HoloConsole VM before VCF deployment it loses a route. This is by design. When VCF is stood up all external network access is done through the CloudBuilder VM. To fix this run this at the command line on the HoloConsole VM route add 0.0.0.0 mask 0.0.0.0 10.0.0.1 📋 Copy Another point of note: the CPU’s in my hosts were identical. If you have different generation Intel CPUs or a mix of AMD/Intel then further consideration is likely to be required. ## 📚 Related Posts * [VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) * [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) ## Similar Posts * [ ![vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) By[James](https://jameskilby.co.uk) November 17, 2023 · Updated June 1, 2026 An Overview of vSAN ESA in VMC * [ ![Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/wp-content/uploads/2026/01/Firefly_Gemini-Flash-768x417.png) ](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) By[James](https://jameskilby.co.uk) January 27, 2026 · Updated June 5, 2026 How to leverage Content Libraries to deploy into VMware Cloud on AWS faster. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 Part 2 of my self-hosted AI stack series. I cover container resource sizing, dual-network isolation via Traefik and Cloudflare Tunnels, and every database powering the stack — PostgreSQL, ClickHouse, Redis, Qdrant, MinIO, MongoDB, SQLite, Prometheus, and Jaeger — plus the backup strategy for each. * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/09/vmc-host-errors/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) By[James](https://jameskilby.co.uk) September 15, 2020 · Updated June 5, 2026 Learn how host failures are handled within VMC * [ ![An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) By[James](https://jameskilby.co.uk) August 14, 2025 · Updated July 11, 2026 This is single page intended to collate every single feature of the current VMware Cloud on AWS hosts for easy comparison. * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/07/i3en/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [VMware Cloud on AWS i3en Host: Specs, Storage & Performance](https://jameskilby.co.uk/2020/07/i3en/) By[James](https://jameskilby.co.uk) July 2, 2020 · Updated June 5, 2026 VMware Cloud on AWS (VMC) has introduced a new host to its lineup: the “i3en”. This is based on the i3en. --- # vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24 Source: https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/ Date: 2023-11-17 Categories: VMware, VMware Cloud on AWS, vSAN, Homelab, Storage, Ansible, Automation, Artificial Intelligence, Personal ![](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) | [vSAN](https://jameskilby.co.uk/category/vmware/vsan-vmware/) # vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24 By[James](https://jameskilby.co.uk) November 17, 2023 · Updated June 1, 2026 • 📖4 min read(852 words) ## Table of Contents vSAN Express Storage Architecture (ESA) was announced at VMware Explore last year (2022) and while it’s been available for our on-premise customers for nearly a year it hasn’t been available in VMC until now… With the release of the M24 version of VMC, it is now an option for newly provisioned SDDCs So why is this such an important change? To answer that we need a history lesson on vSAN ![Introducing ESA Figure01](https://jameskilby.co.uk/wp-content/uploads/2023/10/Introducing-ESA-Figure01-1024x422.png) vSAN first came to market in 2014, and the typical hardware of the day from a storage perspective was dominated by spindles. Flash had arrived on the scene but was VERY expensive and datacenter networking was in the transition from 1Gb/s to 10Gb/s. As a result of this and other factors design decisions were made in the original vSAN (referred to here as OSA) to implement a tiered storage architecture. This was made up of flash-based cache drives and flash or hard disk-based capacity drives. These were then combined into something called a disk group. This decision led to several consequences. The first is that the cache drive cannot be used for storing data it is only a transient write buffer. As all writes go to the cache drive in a disk group in some circumstances this could limit the peak I/O available. It also meant that the endurance of this drive was paramount. This was a challenge in the early days of flash drives. To improve performance and reduce failure domains it’s possible to have multiple disk groups within a host. This is something that we implement in VMC using vSAN OSA. This further compounds the reduction in usable storage, as multiple drives are allocated to the cache role. As All-Flash has now become the normality it was worth revisiting the architecture. To understand more about the under-the-hood architecture of vSAN see the [official overview ](https://core.vmware.com/blog/introduction-vsan-express-storage-architecture) ## vSAN ESA – Benefits Putting all of the above changes together brings significant benefits to VMC customers when using the ESA version of vSAN. ### Performance benefits * Significant increase in available IOPS ( all drives are used for reading and writing) * Adaptive write path for maximum throughput (Especially relevant for customers that require high throughput on a single or low number of VMDKs) * Improved snapshot mechanisms reduce the impact on the guest workload (Snapshot deletion is up to 100x faster combined with much reduced primary latency when running workloads from snapshots.) As business demands lower RPO’s, any improvement in snapshot mechanisms reduce the overhead to the guest workload. ### Efficiency benefits * Improved compression algorithms for better data reduction * More efficient compression algorithms free up the Host CPU for Guest OS processing * Reduced storage network traffic (compression happens on the source host and then all replication is already compressed) * Further enhancements reduce the CPU overhead when using vSAN Encryption (This is always used in VMC) * Always on [UNMAP/Trim](https://www.techtarget.com/searchstorage/definition/TRIM) (This helps to free up unused blocks at the Nand block level) * The ability to deploy a parity-based (RAID5) with 3 Nodes. (OSA requires a minimum of 4) * The performance of the parity-based (RAID5/6) of the ESA implementation now exceeds the RAID1 performance of OSA. Customers can get increased savings by changing the storage policy. ### Resilience Benefits * fewer rewrites in the event of a device replacement – Although this is a significant benefit for on-premise customers with VMC if we have a drive failure we will replace the entire node. vSAN ESA will only be available on the i4.metal-based VMC hosts at launch. It is also not currently available in a 2-node or Stretched SDDC configuration… ## Sizing As part of my job in VMware Cloud Pre-Sales a critical aspect is sizing VMC solutions for customers. This is usually done as a first pass with [RVtools](https://www.robware.net/rvtools/) inputs into our sizing tools. But what does that mean to a customer? My colleague [Nikolay](https://nkulikov.com) put together this handy table. This is assuming a conservative compression ratio of 1.25x for ESA. The reality is we expect this to be higher. **Nodes**| **OSA (Compression 1.25) Policy In use**| **Capacity (TiB)**| **ESA (Compression 1.25) Policy In use**| **Capacity (TiB)**| **Delta %** ---|---|---|---|---|--- 2| FTT1, Mirror| 14.09| N/A| N/A| N/A 3| FTT1, Mirror| 23.5| FTT1, RAID5| 37.22| 58.38 4| FTT1, RAID5| 51.87| FTT1, RAID5| 51.21| -1.27* 5| FTT1, RAID5| 66.03| FTT1, RAID5| 65.19| -1.27* 6| FTT2, RAID6| 70.56| FTT2, RAID6| 79.18| 12.22 7| FTT2, RAID6| 83.11| FTT2, RAID6| 93.16| 12.09 8| FTT2, RAID6| 95.65| FTT2, RAID6| 107.15| 12.02 9| FTT2, RAID6| 108.2| FTT2, RAID6| 121.13| 11.95 10| FTT2, RAID6| 120.75| FTT2, RAID6| 135.12| 11.90 11| FTT2, RAID6| 133.3| FTT2, RAID6| 149.1| 11.85 12| FTT2, RAID6| 145.85| FTT2, RAID6| 163.09| 11.82 13| FTT2, RAID6| 158.4| FTT2, RAID6| 177.08| 11.79 14| FTT2, RAID6| 170.95| FTT2, RAID6| 191.06| 11.76 15| FTT2, RAID6| 183.5| FTT2, RAID6| 205.05| 11.74 16| FTT2, RAID6| 196.04| FTT2, RAID6| 219.03| 11.73 *As ESA has a dynamic parity-based mechanism it can either use 2+1 or 4+1 however OSA has the capability of using 3+1. This is why the 4- and 5-node configurations have a lower usable space. The above table is the usable capacity for workloads including the management objects. Additional clusters would have more space. ## 📚 Related Posts * [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) * [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) * [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) ## Similar Posts * [ ![Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/wp-content/uploads/2024/09/QuantaGrid-SD1Q-1ULH-Front-Three-Quarter.png) ](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) By[James](https://jameskilby.co.uk) September 12, 2024 · Updated July 11, 2026 Yes, that’s a clickbait title. But technically it’s possible if I dropped all drive redundancy… I recently saw an advert for a server that was just too good to be true. * [ ![Automated VCF 9 Offline Depot architecture diagram showing Traefik reverse proxy and Nginx file server stack](https://jameskilby.co.uk/wp-content/uploads/2026/04/offlinedepot.png) ](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) By[James](https://jameskilby.co.uk) April 10, 2026 · Updated July 11, 2026 One Bash script turns a fresh Ubuntu VM into a VCF 9 Offline Depot: Traefik, Nginx, basic auth, and Let’s Encrypt wildcard certs via Cloudflare DNS. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 Part 2 of my self-hosted AI stack series. I cover container resource sizing, dual-network isolation via Traefik and Cloudflare Tunnels, and every database powering the stack — PostgreSQL, ClickHouse, Redis, Qdrant, MinIO, MongoDB, SQLite, Prometheus, and Jaeger — plus the backup strategy for each. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [My First Homelab Storage Setup: HP Gen8 & Xpenology](https://jameskilby.co.uk/2018/01/lab-storage/) By[James](https://jameskilby.co.uk) January 6, 2018 · Updated June 1, 2026 I have been meaning to post around some of the lab setup for a while. Although it changes frequently at present it’s as below. * [ ![Homelab SSD Failure: How Synology RAID Saved My Data](https://jameskilby.co.uk/wp-content/uploads/2022/11/BrokenHardDive-1200x630-1-768x403.jpg) ](https://jameskilby.co.uk/2022/11/homelab-bad-days-almost/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Homelab SSD Failure: How Synology RAID Saved My Data](https://jameskilby.co.uk/2022/11/homelab-bad-days-almost/) By[James](https://jameskilby.co.uk) November 21, 2022 · Updated July 11, 2026 I recently spent 3 weeks in Ireland with my wife Wendy and our son Nate. * [ ![Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) ](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal. --- # Configuring a Zen Internet and City Fibre connection with a 3rd party router Source: https://jameskilby.co.uk/2023/11/configuring-a-zen-internet-and-city-fibre-connection-with-a-3rd-party-router/ Date: 2023-11-15 Categories: Networking, Homelab, Mikrotik ![](https://jameskilby.co.uk/wp-content/uploads/2023/11/cityfibre-zen.jpg) [Networking](https://jameskilby.co.uk/category/networking/) # Configuring a Zen Internet and City Fibre connection with a 3rd party router By[James](https://jameskilby.co.uk) November 15, 2023 · Updated June 1, 2026 • 📖2 min read(316 words) Back in July I bought a new house and one of the best things about the property was that it was already in a City Fibre location. That meant I could take my Zen internet connection with me but ditch the ADSL (and Phone Line requirement). This gave me a much better connection in terms of throughput and latency at a lower overall cost. I have seen a few people on Reddit etc. post about using third-party router/firewalls and struggling to get it working. This is my setup using my WatchGuard M200 and I believe that other City Fibre connections with other providers will be configured the same. The first step is to create an Interface for the WatchGuard to use and set this to be a VLAN Interface as shown below. ![VLAN Interface](https://jameskilby.co.uk/wp-content/uploads/2023/11/Screenshot-2023-11-14-at-09.32.55.png) The next step is to set up the VLAN interface This is located in Networks>VLAN of the Web UI Add a VLAN Interface and set the VLAN ID to 911 and set it to Tagged traffic ![Configuring a Zen Internet and City Fibre connection with a 3rd party router Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/11/Screenshot-2023-11-14-at-19.38.42-1024x455.png) When the security zone is defined as External three config options become available on the Network tab ![Configuring a Zen Internet and City Fibre connection with a 3rd party router Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/11/Screenshot-2023-11-15-at-19.12.27-1024x932.png) I have a /29 address range from Zen so I add my username and password to the network page together with the router IP. The router IP is the highest of the allocated range so in my case X.X.X.174 To utilise the other addresses allocated by Zen these are added to the secondary tab as /32s ![Configuring a Zen Internet and City Fibre connection with a 3rd party router Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/11/Screenshot-2023-11-15-at-19.22.11-1024x537.png) And that’s all there is to it. With the above config, I have my 6 usable IPs up and running. One of the nice capabilities that the WatchGuard Firewall has is that it can monitor your interfaces (and make routing decisions if multiple connections are available) I no longer use this feature as the single connection is reliable enough for my needs, but nonetheless the monitoring part is still useful. ![Configuring a Zen Internet and City Fibre connection with a 3rd party router Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/11/Screenshot-2023-11-15-at-19.28.44-1024x503.png) Example Route to Cloudflare 1.1.1.1 ![Configuring a Zen Internet and City Fibre connection with a 3rd party router Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/11/Screenshot-2023-11-15-at-19.30.15-1-1024x243.png) ## 📚 Related Posts * [Unleashing the UDM Beast](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) * [Home Network Upgrade to 25Gb/s with MikroTik Switching](https://jameskilby.co.uk/2024/09/home-network-upgrade/) * [Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) ## Similar Posts * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Homelab Network Upgrade: DACs, 40Gb/s vMotion & pfSense](https://jameskilby.co.uk/2022/01/lab-update-part-3-network/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated May 31, 2026 I have retired the WatchGuard Devices with the migration to PFSense running bare-metal in one of the Supermicro Nodes. * [ ![UniFi UDM Beast 1U rack-mount gateway shown front-on, with two 3.5-inch drive bays, a bank of RJ45 and SFP network ports, and dimension labels.](https://jameskilby.co.uk/wp-content/uploads/2026/06/UnifiBeast-768x219.webp) ](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Unleashing the UDM Beast](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) By[James](https://jameskilby.co.uk) June 20, 2026 · Updated August 24, 2026 Retiring my ageing WatchGuard M200, I swap in the UniFi UDM Beast — a hardware tour, first iperf and WAN tests, the BGP/FRR setup, and the homelab services I’m consolidating onto it. * [ ![Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/wp-content/uploads/2024/06/Ubiquiti_Networks-Logo.wine_-768x512.png) ](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) By[James](https://jameskilby.co.uk) June 26, 2024 · Updated July 11, 2026 How to configure DHCP Option 43 for UniFi devices * [ ![MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/wp-content/uploads/2023/04/2157_hi_res-768x346.png) ](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) By[James](https://jameskilby.co.uk) December 19, 2022 · Updated June 1, 2026 For a while, I’ve been looking to update the networking at the core of my homelab. * [ ![CRS-504](https://jameskilby.co.uk/wp-content/uploads/2024/09/s-l1600-768x427.jpg) ](https://jameskilby.co.uk/2024/09/home-network-upgrade/) [Mikrotik](https://jameskilby.co.uk/category/mikrotik/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Home Network Upgrade to 25Gb/s with MikroTik Switching](https://jameskilby.co.uk/2024/09/home-network-upgrade/) By[James](https://jameskilby.co.uk) September 9, 2024 · Updated June 5, 2026 My journey to superfast networking in my homelab --- # TrueNAS Scale CLI Reference: Storage & Kubernetes Commands Source: https://jameskilby.co.uk/2023/11/truenas-scale-useful-commands/ Date: 2023-11-13 Categories: Kubernetes, TrueNAS Scale, Homelab, Storage, VMware, Docker ![TrueNAS Logo](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-22-at-18.49.21.png) [Kubernetes](https://jameskilby.co.uk/category/kubernetes/) | [TrueNAS Scale](https://jameskilby.co.uk/category/truenas-scale/) # TrueNAS Scale CLI Reference: Storage & Kubernetes Commands By[James](https://jameskilby.co.uk) November 13, 2023 · Updated April 11, 2026 • 📖1 min read(221 words) ## Table of Contents I have very much fallen down the rabbit hole with TrueNAS and although the GUI is very good occasionally I need to jump to the command line. This is a list of the commands that I might need to use. I will update this page with anything else I find useful. ## General Storage Get the compression ratio of each Pool sudo zfs get compressratio $PoolName 📋 Copy ## Application Install HeavyScript curl -s https://raw.githubusercontent.com/Heavybullets8/heavy_script/main/functions/deploy.sh | bash && source "$HOME/.bashrc" 2>/dev/null && source "$HOME/.zshrc" 2>/dev/null 📋 Copy View the size of each application zfs list $(cli -c 'app kubernetes config' | grep -E "pool\s\|" | awk -F '|' '{print $3}' | tr -d " \t\n\r")/ix-applications/releases -r | grep pvc | awk '{print $1, $2}' | sort -hr -k2 | column -t 📋 Copy List all Pods and status sudo k3s kubectl get pods -A 📋 Copy Verify which pool your apps are running on cli -c 'app kubernetes config' 📋 Copy View pods and network info k3s kubectl get pods,svc -A 📋 Copy Update and then prune docker containers sudo heavyscript update -p 📋 Copy sudo systemctl restart middlewared 📋 Copy ## General ZFS Show all pools and the physical disk structure/health of all members sudo zpool status 📋 Copy Get ZFS Compression ratio sudo zfs get compressratio $Poolname 📋 Copy ## 📚 Related Posts * [Homelab Storage Refresh (Part 1)](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/) * [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) * [Lab Update – Part 2 Storage Truenas Scale](https://jameskilby.co.uk/2022/01/lab-update-part-2-storage/) ## Similar Posts * [ ![Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/wp-content/uploads/2023/04/intel_optane_ssd_900p_series_aic_-_right_angle_575px.png) ](https://jameskilby.co.uk/2023/04/intel-optane/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) By[James](https://jameskilby.co.uk) April 17, 2023 · Updated June 1, 2026 I have been a VMware vExpert for many years and it has brought me many benefits over the years. * [ ![Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/wp-content/uploads/2024/09/QuantaGrid-SD1Q-1ULH-Front-Three-Quarter.png) ](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) By[James](https://jameskilby.co.uk) September 12, 2024 · Updated July 11, 2026 Yes, that’s a clickbait title. But technically it’s possible if I dropped all drive redundancy… I recently saw an advert for a server that was just too good to be true. * [ ![How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/wp-content/uploads/2024/12/ZFS.jpg) ](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) [TrueNAS Scale](https://jameskilby.co.uk/category/truenas-scale/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) By[James](https://jameskilby.co.uk) December 18, 2024 · Updated June 1, 2026 Introduction Copy on Write Disk IDs Trim Introduction I have run a number of systems using ZFS since the earliest days of my homelab using Nexenta, all the way back in 2010. * [ ![TrueNAS Logo](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-22-at-18.49.21-768x198.png) ](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Homelab Storage Refresh (Part 1)](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/) By[James](https://jameskilby.co.uk) May 23, 2023 · Updated July 11, 2026 Table of Contents Background ZFS Overview Read Cache (ARC and L2ARC) ZIL (ZFS Intent Log) Hardware Background I have just completed the move of all my production and media-based storage/services to TrueNAS Scale. ( I will just refer to this as TrueNAS) This is based on my HP Z840 and I have now retired my… * [ ![Lab Update – Part 2 Storage Truenas Scale](https://jameskilby.co.uk/wp-content/uploads/2022/01/maxresdefault-768x432.jpeg) ](https://jameskilby.co.uk/2022/01/lab-update-part-2-storage/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Lab Update – Part 2 Storage Truenas Scale](https://jameskilby.co.uk/2022/01/lab-update-part-2-storage/) By[James](https://jameskilby.co.uk) January 11, 2022 · Updated June 1, 2026 The HP Z840 has changed its role to a permanent storage box running Truenas Scale. * [ ![Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/wp-content/uploads/2022/12/22225832.png) ](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) [Docker](https://jameskilby.co.uk/category/docker/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) By[James](https://jameskilby.co.uk) December 9, 2022 · Updated June 5, 2026 Late to the party or not, I have been using containers in my lab more and more and that has led me to Portainer …. --- # Analytics in a privacy focused world Source: https://jameskilby.co.uk/2023/11/analytics-in-a-privacy-focused-world/ Date: 2023-11-10 Categories: Hosting, Personal, Docker, Homelab, Devops, Cloudflare, VMware, Consulting ![](https://jameskilby.co.uk/wp-content/uploads/2023/11/plausible-analytics-icon-top.png) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Personal](https://jameskilby.co.uk/category/personal/) # Analytics in a privacy focused world By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 • 📖1 min read(254 words) I recently helped my friend Dean Lewis [@veducate](https://veducate.co.uk) with some hosting issues. As part of the testing of this he kindly gave me a login to his WordPress instance. He has been a pretty prolific blogger over the years pumping out an amazing amount of really good content. It also highlighted to me that I didn’t know if anyone was reading my blog as I removed Google Analytics years ago over privacy concerns. I had seen from Myles’s [blog](http://blah.cloud) that he had done something similar as part of his blog revamp. Myles was using [Plausible](https://plausible.io/open-source-website-analytics) and seeing as they had a self-hosted version of this I decided to give it a whirl. ## Install Luckily for me Truecharts has a prebuilt plausible container so I fired that up and set an ingress entry of plausible.jameskilby.cloud on the container so that it uses my Traefik instance as an entry into my Truenas K3s environment. In my WordPress origin site I added the Plausible plugin and defined the endpoint. The last step was to expose the container to the outside world. To do this I created a public DNS record in Cloudflare and pointed this at an existing Cloudflare tunnel terminating in my Truenas K3s setup. The entire process took approx 30 mins to set up and I’m really pleased with the results. ![Plausible results](https://jameskilby.co.uk/wp-content/uploads/2023/11/Screenshot-2023-11-10-at-16.42.39-1024x527.png) Plausible real time stats Update: I am now happy with this and have decided to make my stats public to show exactly what information I can and can’t see. This can be seen [here](https://plausible.jameskilby.cloud/jameskilby.co.uk/) ## 📚 Related Posts * [Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/2022/01/web-development/) * [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) * [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) ## Similar Posts * [ ![Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/wp-content/uploads/2022/12/22225832.png) ](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) [Docker](https://jameskilby.co.uk/category/docker/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) By[James](https://jameskilby.co.uk) December 9, 2022 · Updated June 5, 2026 Late to the party or not, I have been using containers in my lab more and more and that has led me to Portainer …. * [ ![My First GitHub Pull Request: Contributing to Open Source](https://jameskilby.co.uk/wp-content/uploads/2020/12/175jvBleoQfAZJc3sgTSPQA.jpg) ](https://jameskilby.co.uk/2020/12/my-first-pull/) [Devops](https://jameskilby.co.uk/category/devops/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [My First GitHub Pull Request: Contributing to Open Source](https://jameskilby.co.uk/2020/12/my-first-pull/) By[James](https://jameskilby.co.uk) December 22, 2020 · Updated June 5, 2026 I was initially going to add in the contents of this post to one that I have been writing about my exploits with HashiCorp Packer but I decided it probably warranted being separated out. * [ ![Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu-768x450.jpeg) ](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) By[James](https://jameskilby.co.uk) October 20, 2022 · Updated July 11, 2026 For a while now I have been running this site directly from Cloudflare utilising their excellent worker’s product. * [ ![VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/wp-content/uploads/2020/09/vmware_SP_HCI20.png) ](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) [Personal](https://jameskilby.co.uk/category/personal/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) By[James](https://jameskilby.co.uk) September 13, 2020 · Updated May 31, 2026 I recently sat (and passed the VMware HCI Master Specialist exam (5V0-21. * [ ![What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2018/03/cloudflare/) [Hosting](https://jameskilby.co.uk/category/hosting/) ### [What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained](https://jameskilby.co.uk/2018/03/cloudflare/) By[James](https://jameskilby.co.uk) March 27, 2018 · Updated June 1, 2026 Cloudflare – What is it and why would I care? I have been using Cloudflare for a long time. * [ ![My Home Office Setup & Upgrades](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_7017-scaled-1-768x576.jpeg) ](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) [Personal](https://jameskilby.co.uk/category/personal/) | [Consulting](https://jameskilby.co.uk/category/consulting/) ### [My Home Office Setup & Upgrades](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) By[James](https://jameskilby.co.uk) January 5, 2021 · Updated June 1, 2026 Given the year that was 2020 and at the time of writing a distinct improvement appears a long way off I decided it was time to up my homeworking game. --- # Advanced Deploy VMware vSphere 7.x 3V0-22.21N Source: https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/ Date: 2023-11-10 Categories: VMware, Personal, vSphere, vSAN, Consulting, Homelab, Storage, VMware Cloud on AWS ![](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) | [vSphere](https://jameskilby.co.uk/category/vsphere/) # Advanced Deploy VMware vSphere 7.x 3V0-22.21N By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 • 📖1 min read(139 words) Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal, I had some downtime and decided to use one of the three exam vouchers VMware gives me each year. This upgrades me to a VMware Certified Advanced Professional. The VCAP Deploy exams are lab based and use a similar infrastructure to the hands on labs. In my experience the whole setup was quite slick. If you know your way around vSphere pretty comprehensively or have done the appropriate studying then I think it was a fair assessment. The [blueprint](https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/certification/vmware-advanced-deploy-exam-guide.pdf) covers everything you need. So what’s stopping you… I have now scheduled the VCAP-Design exam and hopefully I will be able to pass that and receive the VCIX award. ## 📚 Related Posts * [How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) * [VMware – Going out with a Bang!](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) * [VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) ## Similar Posts * [ ![vSAN Cluster Shutdown – Orchestration](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2025/12/vsan-cluster-shutdown/) [VMware](https://jameskilby.co.uk/category/vmware/) | [vSAN](https://jameskilby.co.uk/category/vmware/vsan-vmware/) ### [vSAN Cluster Shutdown – Orchestration](https://jameskilby.co.uk/2025/12/vsan-cluster-shutdown/) By[James](https://jameskilby.co.uk) December 6, 2025 · Updated June 1, 2026 How to safely shut down a vSAN Environment * [ ![VMware – Going out with a Bang!](https://jameskilby.co.uk/wp-content/uploads/2023/10/rnli-logo-768x384.png) ](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [VMware – Going out with a Bang!](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) By[James](https://jameskilby.co.uk) October 7, 2023 · Updated June 1, 2026 There is a lot of uncertainty with VMware at the moment. This is all due to the pending acquisition by Broadcom. * [ ![My Home Office Setup & Upgrades](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_7017-scaled-1-768x576.jpeg) ](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) [Personal](https://jameskilby.co.uk/category/personal/) | [Consulting](https://jameskilby.co.uk/category/consulting/) ### [My Home Office Setup & Upgrades](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) By[James](https://jameskilby.co.uk) January 5, 2021 · Updated June 1, 2026 Given the year that was 2020 and at the time of writing a distinct improvement appears a long way off I decided it was time to up my homeworking game. * [ ![Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/wp-content/uploads/2023/04/intel_optane_ssd_900p_series_aic_-_right_angle_575px.png) ](https://jameskilby.co.uk/2023/04/intel-optane/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) By[James](https://jameskilby.co.uk) April 17, 2023 · Updated June 1, 2026 I have been a VMware vExpert for many years and it has brought me many benefits over the years. * [ ![VMware Cloud on AWS Time Sync & NTP Configuration](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/12/time-in-a-vmc-environment/) [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [VMware Cloud on AWS Time Sync & NTP Configuration](https://jameskilby.co.uk/2025/12/time-in-a-vmc-environment/) By[James](https://jameskilby.co.uk) December 8, 2025 · Updated July 11, 2026 How to use the Amazon Time Sync Service in a VMC environment * [ ![An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) By[James](https://jameskilby.co.uk) August 14, 2025 · Updated July 11, 2026 This is single page intended to collate every single feature of the current VMware Cloud on AWS hosts for easy comparison. --- # Nvidia Tesla P4 vGPU Setup in VMware Homelab: Full Guide Source: https://jameskilby.co.uk/2023/10/vgpu-setup-in-my-homelab/ Date: 2023-10-23 Categories: Homelab, VMware, Hosting, Automation, VCF, Personal, VMware Cloud on AWS, AWS ![](https://jameskilby.co.uk/wp-content/uploads/2023/10/IMG_1107-768x403-1.jpg) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) # Nvidia Tesla P4 vGPU Setup in VMware Homelab: Full Guide By[James](https://jameskilby.co.uk) October 23, 2023 · Updated July 11, 2026 • 📖4 min read(878 words) ## Table of Contents A little while ago I decided to play with vGPU in my homelab. This was something I had dabbled with in the past but never really had the time or need to get working properly. I later built on this to run an [AI homelab](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) on the same hardware. The first thing that I needed was a GPU. I did have a Dell T20 with an iGPU built into the CPU but I wanted a more enterprise card for my testing. I therefore decided to pick up a Nvidia Tesla P4 from eBay for the princely sum of £185. The card stats are not impressive by modern standards but it was good enough for what I needed combined with the fact that it was a single-width PCIe 3.0 x16 card and didn’t require external power. An important note is that this is a passively cooled card so if it’s not located in a server with a reasonable airflow then you may run into some thermal issues. 3D printer fan shrouds are available on eBay to work around this issue. ## Card Stats GPU 1 NVIDIA Pascal™ GPU NVIDIA® CUDA® Cores 2,560 Memory Size 8 GB GDDR5 H.264 1080p30 Streams 24 vGPU Profiles 1 GB, 2 GB, 4 GB, 8 GB Form Factor PCIe 3.0 single slot (low profile) for rack servers Power 75 W Thermal Solution Passive In a VMware environment, the first decision is how are you going to use the card? There are two basic modes of operation either GPU pass through where the entire card is passed into a single VM or vGPU where the card can be carved up into multiple gFX cards and be presented to one or more VMs at the same time. It was this latter option that I planned to implement. I wanted at least one node presented through to a Horizon VDI instance. The other node was going to be used as a [Tdarr](https://home.tdarr.io) Node ## vGPU Install Steps I installed the card into one of my 4 Supermicro nodes. With the graphics card installed I was limited to the onboard 2xGb Nic’s but that would be sufficient for initial testing. Initial attempts at getting this to work with vSphere 8 seemed to have issues so I rolled this node back to vSphere 7.0u2 as I knew this combo would work. The first step is to install the drivers into the ESXi Host. This is straightforward and as I was only doing this to a single host I copied the relevant driver to a datastore and then ran the below command to perform the install… esxcli software vib install -v /vmfs/volumes/623a916d-ccad8ff0-0000-000000000000/Nvidia/NVD_bootbank_NVD-VMware_ESXi_7.0.2_Driver_535.54.06-1OEM.702.0.0.17630552.vib 📋 Copy At the end of the installation, a host reboot is required (even if it says it isn’t). Once this has been done you should have the ability to add PCI devices with the associated Nvidia Profile as you can see all of the below ones are grid_p4 as this is the card that I am using. ![Nvidia Tesla P4 Homelab Setup Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/10/Screenshot-2023-10-23-at-15.00.32-1024x435.png) The Nvidia GPU Software Docs list the capabilities of each profile. I have copied the relevant table below. Virtual GPU Type| Intended Use Case| Frame Buffer (MB)| Virtual Display Heads| Maximum Resolution per Display Head| Maximum vGPUs per GPU| Maximum vGPUs per Board| Required License Edition ---|---|---|---|---|---|---|--- P4-8Q| Virtual Workstations| 8192| 4| 4096×2160| 1| 1| Quadro vDWS P4-4Q| Virtual Workstations| 4096| 4| 4096×2160| 2| 2| Quadro vDWS P4-2Q| Virtual Workstations| 2048| 4| 4096×2160| 4| 4| Quadro vDWS P4-1Q| Virtual Desktops, Virtual Workstations| 1024| 2| 4096×2160| 8| 8| Quadro vDWS P4-8C| Training Workloads| 8192| 1| 4096×2160[2](https://docs.nvidia.com/grid/9.0/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid__c-series-vgpu-graphics-note)| 1| 1| vCS or Quadro vDWS P4-4C| Inference Workloads| 4096| 1| 4096×2160[2](https://docs.nvidia.com/grid/9.0/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid__c-series-vgpu-graphics-note)| 2| 2| vCS or Quadro vDWS P4-2B| Virtual Desktops| 2048| 2| 4096×2160| 4| 4| GRID Virtual PC or Quadro vDWS P4-2B4| Virtual Desktops| 2048| 4| 2560×1600| 4| 4| GRID Virtual PC or Quadro vDWS P4-1B| Virtual Desktops| 1024| 4| 2560×1600| 8| 8| GRID Virtual PC or Quadro vDWS P4-1B4| Virtual Desktops| 1024| 1| 4096×2160| 8| 8| GRID Virtual PC or Quadro vDWS P4-8A| Virtual Applications| 8192| [4](https://docs.nvidia.com/grid/9.0/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid__a-series-vgpu-max-res-note)1| 1280×1024[4](https://docs.nvidia.com/grid/9.0/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid__a-series-vgpu-max-res-note)| 1| 1| GRID Virtual Application P4-4A| Virtual Applications| 4096| 1[4](https://docs.nvidia.com/grid/9.0/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid__a-series-vgpu-max-res-note)| 1280×1024[4](https://docs.nvidia.com/grid/9.0/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid__a-series-vgpu-max-res-note)| 2| 2| GRID Virtual Application P4-2A| Virtual Applications| 2048| 1[4](https://docs.nvidia.com/grid/9.0/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid__a-series-vgpu-max-res-note)| 1280×1024[4](https://docs.nvidia.com/grid/9.0/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid__a-series-vgpu-max-res-note)| 4| 4| GRID Virtual Application P4-1A| Virtual Applications| 1024| 1[4](https://docs.nvidia.com/grid/9.0/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid__a-series-vgpu-max-res-note)| 1280×1024[4](https://docs.nvidia.com/grid/9.0/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid__a-series-vgpu-max-res-note)| 8| 8| GRID Virtual Application ## vGPU VM Provisioning With that configured the next step was to provision a VM and use the associated drivers within the VM. ![Nvidia Tesla P4 Homelab Setup Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/10/Screenshot-2023-10-23-at-15.00.07-1024x313.png) Please note that when utilising PCI passthrough all of the memory must be reserved for the VM… With the VM running and the drivers installed we can now see the graphics card in Windows. ![Nvidia Tesla P4 Homelab Setup Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/10/Screenshot-2023-10-23-at-15.25.01-1-1024x850.png) Using GPU-Z we can validate the resources being presented through to the Virtual Machine… ![Nvidia Tesla P4 Homelab Setup Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/10/Screenshot-2023-10-23-at-15.24.54-746x1024.png) ## Folding@Home As a final step it was worth proving that the GPU can actually be used by the workload. For this I used the folding@home application as can be seen below. ![Nvidia Tesla P4 Homelab Setup Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/10/Screenshot-2023-10-09-at-17.36.02-1024x408.png)Folding at home slots overview My Folding@home stats can be seen [here](https://stats.foldingathome.org/donor/69704482) and consider joining your compute to the project as well Just a side note that folding@home will use all of the CPU and GPU if you let it. Here is the power utilisation of the node and you can easily see where Folding@Home started. ![Nvidia Tesla P4 Homelab Setup Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/10/Screenshot-2023-10-07-at-14.12.01-1024x273.png) ## 📚 Related Posts * [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) * [Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) ## Similar Posts * [ ![Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/wp-content/uploads/2022/10/spacexs-starlink-to-supply-free-satellite-internet-to-famili_u44u.1920-768x432.jpg) ](https://jameskilby.co.uk/2022/10/starlink/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/2022/10/starlink/) By[James](https://jameskilby.co.uk) October 11, 2022 · Updated June 1, 2026 Since moving to Dorset last year internet connectivity has been the bane of my existence. Currently, I have an ADSL connection provided by my old employer Zen and a 5G connection provided by Three. * [ ![Template Deployment with Packer](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) ](https://jameskilby.co.uk/2021/01/hashicorp-packer/) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. * [ ![VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/wp-content/uploads/2024/01/40oOd8IipPvtrPJs-1198788743-768x737.jpg) ](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) [VCF](https://jameskilby.co.uk/category/vmware/vcf/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) By[James](https://jameskilby.co.uk) January 18, 2024 · Updated June 1, 2026 How to deploy Holodeck with Legacy CPU’s * [ ![VMware – Going out with a Bang!](https://jameskilby.co.uk/wp-content/uploads/2023/10/rnli-logo-768x384.png) ](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [VMware – Going out with a Bang!](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) By[James](https://jameskilby.co.uk) October 7, 2023 · Updated June 1, 2026 There is a lot of uncertainty with VMware at the moment. This is all due to the pending acquisition by Broadcom. * [ ![vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) By[James](https://jameskilby.co.uk) November 17, 2023 · Updated June 1, 2026 An Overview of vSAN ESA in VMC * [ ![Monitoring VMware Cloud on AWS: Tools & Approaches \(Part 1\)](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) ](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) [VMware](https://jameskilby.co.uk/category/vmware/) | [AWS](https://jameskilby.co.uk/category/aws/) ### [Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1)](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) By[James](https://jameskilby.co.uk) December 17, 2019 · Updated June 5, 2026 As previously mentioned I have been working a lot with VMware Cloud on AWS and one of the questions that often crops up is around an approach to monitoring. --- # VMware – Going out with a Bang! Source: https://jameskilby.co.uk/2023/10/going-out-with-a-bang/ Date: 2023-10-07 Categories: VMware, Personal, Homelab, Nutanix, vSAN, VMware Cloud on AWS ![](https://jameskilby.co.uk/wp-content/uploads/2023/10/rnli-logo.png) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) # VMware – Going out with a Bang! By[James](https://jameskilby.co.uk) October 7, 2023 · Updated June 1, 2026 • 📖1 min read(155 words) There is a lot of uncertainty with VMware at the moment. This is all due to the pending acquisition by Broadcom. There are a lot of unknowns for the staff and customers about what the company will look like in the future. I certainly have some concerns mainly just with the unknown. However, VMware has a great reputation for doing the right thing around its staff, customers, and the wider world. Yesterday the entire company was off on one of VMware’s so-called [EPIC](https://news.vmware.com/company/vmwares-culture-built-epic2-values) days. I spent the day with my family on the beach. ![VMware - Going out with a Bang!](https://jameskilby.co.uk/wp-content/uploads/2023/10/IMG_3745-768x1024.jpeg) VMware also gave all employees $1,300 for donation through their foundation to a charity of your choice. I decided to give mine to the [RNLI](https://rnli.org). Being based in Poole for the last couple of years and having a friend who volunteers for the RNLI in Ireland, I thought they were a great recipient of this very generous donation from VMware. ![VMware - Going out with a Bang! Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/10/Screenshot-2023-09-22-at-15.07.27.png) ## 📚 Related Posts * [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) * [VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) * [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) ## Similar Posts * [ ![VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/wp-content/uploads/2020/09/vmware_SP_HCI20.png) ](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) [Personal](https://jameskilby.co.uk/category/personal/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) By[James](https://jameskilby.co.uk) September 13, 2020 · Updated May 31, 2026 I recently sat (and passed the VMware HCI Master Specialist exam (5V0-21. * [ ![Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup](https://jameskilby.co.uk/wp-content/uploads/2023/04/IMG_4536-scaled-1-768x1024.jpg) ](https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup](https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated February 16, 2026 Quite a few changes have happened in the lab recently. I decided to do a multipart blog on the changes. * [ ![Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2020/07/nutanix-ncp/) [Nutanix](https://jameskilby.co.uk/category/nutanix/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/2020/07/nutanix-ncp/) By[James](https://jameskilby.co.uk) July 2, 2020 · Updated May 31, 2026 I saw a tweet a couple of weeks ago mentioning that Nutanix were offering a free go at the Nutanix Certified Professional exam. * [ ![Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-12-14-at-21.45.23.png) ](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) By[James](https://jameskilby.co.uk) December 14, 2022 · Updated June 1, 2026 I run a reasonably extensive homelab that is of course built around the VMware ecosystem. * [ ![vSAN Cluster Shutdown – Orchestration](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2025/12/vsan-cluster-shutdown/) [VMware](https://jameskilby.co.uk/category/vmware/) | [vSAN](https://jameskilby.co.uk/category/vmware/vsan-vmware/) ### [vSAN Cluster Shutdown – Orchestration](https://jameskilby.co.uk/2025/12/vsan-cluster-shutdown/) By[James](https://jameskilby.co.uk) December 6, 2025 · Updated June 1, 2026 How to safely shut down a vSAN Environment * [ ![An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) By[James](https://jameskilby.co.uk) August 14, 2025 · Updated July 11, 2026 This is single page intended to collate every single feature of the current VMware Cloud on AWS hosts for easy comparison. --- # Homelab Storage Refresh (Part 1) Source: https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/ Date: 2023-05-23 Categories: Homelab, Storage, Kubernetes, TrueNAS Scale, Ansible, Artificial Intelligence, Hosting, Automation ![TrueNAS Logo](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-22-at-18.49.21.png) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) # Homelab Storage Refresh (Part 1) By[James](https://jameskilby.co.uk) May 23, 2023 · Updated July 11, 2026 • 📖9 min read(1,858 words) ## Table of Contents ## Background I have just completed the move of all my production and media-based storage/services to TrueNAS Scale. ( I will just refer to this as TrueNAS) This is based on my HP Z840 and I have now retired my faithful Synology DS918+ and DX517 expansion bay. (Those have gone to fellow vExperts Paul Wilk and Kev Johnson) I loved the Synology ecosystem and I still believe it’s probably the easiest to get up and running and offers a very easy-to-use and performant system. Combine this with a large number of additional apps available together with docker it is very customisable. It makes an excellent entry into homelab storage at a relatively low cost. They are even more suitable for home use when you consider that they are typically quiet and very power efficient. I would have no hesitation in recommending a unit to someone. However, my homelab needs could not be served by the units I had. I was especially constrained by the 2x Gigabit interfaces on the 918+ and the thought of spending multi-thousands for one compatible with my new 25Gb/s network and the number of drives I need really didn’t appeal. This was compounded by the fact that I already had most of the components I wanted to use. My replacement storage needed to be able to meet my current needs with significantly higher performance capabilities but also have room for future expansion. Therefore the migration to TrueNAS was undertaken. I had the TrueNAS box running for a while (with a lower spec and only used it occasionally when I needed higher-performance storage). This was due to the significantly higher electrical usage. However, it was time to complete the move of all data and media services. Up until this point, my media stack was running on my Synology DS918+. I have had a few issues recently with the Synology that I suspect are it running out of memory. I had upgraded it to 12GB but this was still not enough. My Z840 TrueNAS server has way more RAM and CPU capabilities. As part of the migration, I also consolidated some additional hardware into the TrueNAS host. This has now become the central piece of kit in my homelab running a large number of services. ## ZFS Overview Before getting into details of the hardware and software config I think it’s worth giving an overview of ZFS in case anyone is unfamiliar. SUN Microsystems originally developed ZFS and it is a hugely capable storage platform. It is unusual in that it encompasses a volume manager and a file system. It is legendary for its data integrity capabilities and the scale of the system that could be deployed (it was originally called Zettabyte File System) It is a copy-on-write system (COW) If you are familiar with ZFS operation skip this section. ZFS, like a lot of modern storage technologies (VSAN etc.), wants exclusive access to the underlying disks. This means using an HBA-based device rather than a RAID card. ZFS then groups these drives into a “Pool” The default is to use drives for data storage “Data VDEV” This can also be done in several layouts and understanding how this is done is very important to performance. Conceptually these are similar to traditional RAID levels with RAID10 like mirror and stripe usually being recommended for high-performance systems and RAIDZ or RAIDZ2 are parity-based mechanisms with single and double drive protection similar to RAID5 and RAID6. This is recommended for large amounts of data that are not performance critical. One of the additional advantages of ZFS is that drives of different types can be dedicated to additional roles to improve performance and resilience. These additional roles are: * Metadata * Log * Cache * Spare * Dedupe I am using the Cache (L2ARC) and Log (ZIL) functionality and to understand these you have to know a little bit more about how ZFS functions under the covers. ### Read Cache (ARC and L2ARC) ARC and L2ARC are mechanisms to improve read performance only. ARC, or Adaptive Read Cache, is a high-speed in-memory cache that stores recently accessed data. L2ARC stands for Level 2 ARC. It is a secondary cache that can be used to store data that is not currently in ARC. L2ARC can be implemented using storage devices, such as SSDs or NVMe drives. ARC and L2ARC work together to improve the performance of ZFS. When an application requests data, ZFS first checks ARC to see if the data is present. If the data is in ARC, it is returned to the application without having to access the underlying storage devices. If the data is not in ARC, ZFS checks L2ARC. If the data is in L2ARC, it is copied to ARC and then returned to the application. If the data is not in L2ARC, it is read from the underlying storage devices and then added to ARC. One advantage to how ZFS implements these cache mechanisms is that they do not need to be redundant as the data also exists in the storage pool. This means that if ARC was lost due to a crash/reboot or an L2ARC drive failed the data would be accessed from the underlying pool at a lower performance. The use of ARC and L2ARC can significantly improve the performance of ZFS. In some cases, the use of L2ARC can even double the performance of ZFS. However, it is important to note that the use of ARC and L2ARC can also increase the amount of memory that is required to run ZFS. In Truenas Scale only half of the available RAM is able to be used as ARC therefore I have 128GB of RAM available for ARC. The hit rate in my system is typically over 95%. One limitation I wish would be removed is that the ARC is a global cache improving all of your storage needs but the L2ARC is assigned to a pool of storage. Therefore if you have multiple pools you need to have (and configure) multiple L2ARC devices. (This can be done at the command line by adding different partitions onto one device but I am attempting to keep my system configured from GUI only) L2ARC Configuration For L2ARC I am using an Intel P3520 2TB NVMe in U2 form factor on a Startech PCI card. This is a perfect drive for L2ARC as it is a heavy read-focused NVMe offering 375,000 IOPS and up to 2700MB/s of read performance but only 26,000 IOPS and up to 1800MB/s of write performance. This isn’t an issue as by design the L2ARC fills slowly and with the later builds of Truenas, the data persists across reboots. This also helps with the relatively low data endurance of 1095 TBW… ### ZIL (ZFS Intent Log) The ZIL or LOG device(s) is most likely the most misunderstood mechanism within ZFS. Its intention is to improve the performance of synchronous writes. So we need to know what a synchronous write is. So let’s start with the definition: A synchronous write in ZFS is a write operation that is guaranteed to be committed to stable storage before the write operation returns. This means that the data will be written to the underlying storage devices and will not be lost in the event of a system crash or power failure. In contrast, an asynchronous write doesn’t have to hit persistent storage before it’s committed this leads to higher performance at the risk of data loss. When running VM’s sync write is basically essential. The ZIL is used in the following way: 1. When an application (Or VM) writes data to a ZFS file system, the data is first written to the ZIL. 2. Once the data has been written to the ZIL, it is then written to the underlying storage devices. 3. Once the data has been written to the underlying storage devices, the ZIL entry is marked as complete. The ZIL is always present on a ZFS pool and if no dedicated device is used then this uses the existing pool devices. This increases the IO load on the pool devices and therefore for performance reasons, a dedicated high-performance device is highly recommended. Although technically a single device will achieve the higher performance if this device fails or is unavailable then the pool is also toast so at a minimum 2 should be considered. ZIL Config For the ZIL I am using two Intel Optane P4800X 750GB devices in a mirror configuration. I was VERY lucky to get these sample hardware devices from Intel as part of the [VMware vExpert](https://vexpert.vmware.com) program and I have some blogs coming up dedicated to these devices. These are capable of reading 550000 IOPS (4K Blocks) and up to 2500MB/s sequentially read and 550000 IOPS (4K Blocks) 2200 MB/s write Some crazy numbers…. The other thing that makes Optanes the best choice for this role is the endurance that they offer. The spec sheet says they will handle 41 Petabytes being written to them. ## Hardware In terms of the setup of the TrueNAS server, it’s installed on an HP Z840. This has 2 x Intel Xeon E5-2673 V3 – 12-Core 2.40Ghz (48 Threads) and 256GB of RAM. This is a high-end workstation device and sadly I don’t have the rack mount kit for it. However, it was chosen for its horsepower and ability to run all of the PCIe and storage devices that I needed. This includes having the cooling and electrical capabilities to run all the components. It has an 1125-watt power supply so we should be good. There are currently 19 different drives located within the chassis and it’s a little tight… See the PCI slots below. ![IMG 0397](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_0397-1024x737.jpeg) I have mentioned the ARC Device and the Optanes (Slots 1,2 + 4) in the above picture. The last thing to mention is the HBA which is an LSI 9300 located in slot 3. All of my other data drives are connected to this. The current disk architecture is shown below. However, this is likely not the final setup and it may change temporarily for testing. Role| Number| Device| Config| Usable/Role ---|---|---|---|--- Boot Drive | 2 | Intel 80GB SSD | MIRROR | N/A only used for TrueNas OS SSD Pool | 6 | Samsung 860 EVO 2TB | 2 x MIRROR | 3 wide | 4.71TiB (TrueNas Apps and VM storage) HD Pool 1 | 4 | Seagate IronWolf 8TB | 1xRAIDZ1 | 21TiB ( Media storage) HD Pool 2 | 4 | HGST 1TB 7200RPM | 1xRAIDZ1 | 3TiB. ( Files and Photos) ARC (Assigned to SSD Pool) | 1 | Intel 2TB NVMe | JBOD | N/A ARC is only used as a read cache and doesn’t contribute to capacity SLOG (Assigned to SSD Pool) | 2 | [Intel DC P4800X Optane](https://ark.intel.com/content/www/us/en/ark/products/97154/intel-optane-ssd-dc-p4800x-series-750gb-2-5in-pcie-x4-3d-xpoint.html) | 2 x MIRROR | SLOG is a write log and doesn’t add to capacity I will do some proper performance testing in my next post and I’m sure there are optimisations I can make before I do that but until that’s done. I did just run CrystalMark to ensure things are running in the right ballpark. ![Screenshot 2023 05 19 at 10.54.27](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-19-at-10.54.27.png)TrueNAS Scale performance I dug out some old testing from the Synology. The test isn’t exactly the same but you can see the Read/Write and especially the IOPS are significantly higher. ![Synology](https://jameskilby.co.uk/wp-content/uploads/2022/01/Synology-1024x725.png)Synology Performance ## 📚 Related Posts * [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) * [Lab Update – Part 2 Storage Truenas Scale](https://jameskilby.co.uk/2022/01/lab-update-part-2-storage/) * [Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) ## Similar Posts * [ ![TrueNAS Logo](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-22-at-18.49.21-768x198.png) ](https://jameskilby.co.uk/2023/11/truenas-scale-useful-commands/) [Kubernetes](https://jameskilby.co.uk/category/kubernetes/) | [TrueNAS Scale](https://jameskilby.co.uk/category/truenas-scale/) ### [TrueNAS Scale CLI Reference: Storage & Kubernetes Commands](https://jameskilby.co.uk/2023/11/truenas-scale-useful-commands/) By[James](https://jameskilby.co.uk) November 13, 2023 · Updated April 11, 2026 A list of useful Truenas Scale commands * [ ![Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/wp-content/uploads/2023/04/81-ZoEW24UL._SL1500_-768x461.jpg) ](https://jameskilby.co.uk/2019/02/lab-storage-2/) [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/2019/02/lab-storage-2/) By[James](https://jameskilby.co.uk) February 10, 2019 · Updated June 5, 2026 Since starting my new role with Xtravirt my Homelab has gone through several fairly significant changes. * [ ![Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/wp-content/uploads/2023/04/intel_optane_ssd_900p_series_aic_-_right_angle_575px.png) ](https://jameskilby.co.uk/2023/04/intel-optane/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) By[James](https://jameskilby.co.uk) April 17, 2023 · Updated June 1, 2026 I have been a VMware vExpert for many years and it has brought me many benefits over the years. * [ ![Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/wp-content/uploads/2026/01/VMware-NVIDIA-logos_ee2f18dc-615d-4c9e-8f11-9c3c2ce2bf37-prv-768x432.png) ](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) ### [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) By[James](https://jameskilby.co.uk) February 9, 2026 · Updated July 11, 2026 Learn how to use Ansible to configure an Ubuntu VM for use with NVIDIA based AI workloads in vSphere * [ ![Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/wp-content/uploads/2022/10/spacexs-starlink-to-supply-free-satellite-internet-to-famili_u44u.1920-768x432.jpg) ](https://jameskilby.co.uk/2022/10/starlink/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/2022/10/starlink/) By[James](https://jameskilby.co.uk) October 11, 2022 · Updated June 1, 2026 Since moving to Dorset last year internet connectivity has been the bane of my existence. Currently, I have an ADSL connection provided by my old employer Zen and a 5G connection provided by Three. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) By[James](https://jameskilby.co.uk) March 27, 2026 · Updated May 31, 2026 A walkthrough of my self-hosted AI stack: Ollama, Open WebUI, ComfyUI, Whishper, n8n, Qdrant, SearxNG, and a full observability layer — all running on my own hardware with Docker Compose. --- # Runecast Remediation Scripts: Auto-Fix VMware Storage Issues Source: https://jameskilby.co.uk/2023/05/runecast-remediation-scripts/ Date: 2023-05-16 Categories: Runecast, VMware, VMware Cloud on AWS, VCF, Automation, Homelab, Docker, Personal ![](https://jameskilby.co.uk/wp-content/uploads/2023/05/Runecast-Solutions-Ltd.png) [Runecast](https://jameskilby.co.uk/category/runecast/) | [VMware](https://jameskilby.co.uk/category/vmware/) # Runecast Remediation Scripts: Auto-Fix VMware Storage Issues By[James](https://jameskilby.co.uk) May 16, 2023 · Updated June 1, 2026 • 📖2 min read(442 words) I am a huge fan of the Runecast product and luckily as a vExpert they give out NFR licences for my lab. One of the really cool features I wanted to mention today was the remediation script function. I have been playing with storage a lot in my lab recently as part of a wider storage migration piece but also as part of testing Intel Optane drives. Runecast flagged up that I didn’t have Storage I/O Control enabled for these data stores. It gave a great overview of what that issue means: It is a good practice to enable SIOC, especially in environments where storage is overutilized. SIOC monitors the latency of I/Os to data stores at each ESX host sharing that device. When the average normalized datastore latency exceeds a set threshold (30ms by default), the datastore is considered to be congested, and SIOC kicks in to distribute the available storage resources to virtual machines in proportion to their shares. This is to ensure that low-priority workloads do not monopolize or reduce I/O bandwidth for high-priority workloads. But the issue I want to highlight is it will generate the code to resolve it for you. If I drill into the specific issue, I get the view below. ![Runecast Remediation Script's Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-16-at-11.31.30-1024x536.png) Here you can confirm the discovered issue and the remediation action. ( I have chosen not to enable SIOC on my ISO-NFS datastore) The below code is what Runecast generated for me. # Automatically generated action. Always review before executing. # Start of the section for Enabling of Storage IO control on datastore # This action is going to be performed on the following Datastores: "Optane-iSCSI" Write-Host "`n[RCA] Connecting to vCenter Server "uk-poo-p-vc-1.jameskilby.cloud"..." -ForegroundColor Cyan $vcConnection = Connect-VIServer -Server "uk-poo-p-vc-1.jameskilby.cloud" $datastoreIds = "Datastore-datastore-50097" foreach($datastoreId in $datastoreIds) { try { $ds = Get-Datastore -Id $datastoreId -ErrorAction Stop Write-Host "`n[RCA] Reconfiguring datastore with current name $($ds.name)" -ForegroundColor Yellow Write-Host "`n[RCA] Enabling Storage I/O Control" -ForegroundColor Yellow $ds | Set-Datastore -StorageIOControlEnabled $true -ErrorAction Stop | Format-Table name,Id,StorageIOControlEnabled } catch { Write-Host "`n[RCA] Datastore with ID $datastoreId was not found or the reconfiguration failed" -ForegroundColor Red } } Write-Host "`n[RCA] Disconnecting from vCenter Server "uk-poo-p-vc-1.jameskilby.cloud"..." -ForegroundColor Cyan Disconnect-VIServer $vcConnection -Confirm:$false # End of the section for Enabling of Storage IO control on datastore 📋 Copy Dropping this into a PowerCLI session, we get… [RCA] Connecting to vCenter Server uk-poo-p-vc-1.jameskilby.cloud... Specify Credential Please specify server credential User: administrator@vsphere.cloud Password for user administrator@vsphere.cloud: ************ [RCA] Reconfiguring datastore with current name Optane-iSCSI [RCA] Enabling Storage I/O Control Name Id StorageIOControlEnabled ---- -- ----------------------- Optane-iSCSI Datastore-datastore-50097 True [RCA] Disconnecting from vCenter Server uk-poo-p-vc-1.jameskilby.cloud... PS /Users/w20kilja> 📋 Copy What an amazing little feature!!! ## 📚 Related Posts * [How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) * [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) * [VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) ## Similar Posts * [ ![Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/wp-content/uploads/2026/01/Firefly_Gemini-Flash-768x417.png) ](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) By[James](https://jameskilby.co.uk) January 27, 2026 · Updated June 5, 2026 How to leverage Content Libraries to deploy into VMware Cloud on AWS faster. * [ ![VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/wp-content/uploads/2024/01/40oOd8IipPvtrPJs-1198788743-768x737.jpg) ](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) [VCF](https://jameskilby.co.uk/category/vmware/vcf/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) By[James](https://jameskilby.co.uk) January 18, 2024 · Updated June 1, 2026 How to deploy Holodeck with Legacy CPU’s * [ ![Template Deployment with Packer](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) ](https://jameskilby.co.uk/2021/01/hashicorp-packer/) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. * [ ![VMware Holodeck Multi-Host VCF: Lab Setup & Configuration](https://jameskilby.co.uk/wp-content/uploads/2023/12/Holodeck-Overview.png) ](https://jameskilby.co.uk/2024/01/multihost-holodeck-vcf/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VCF](https://jameskilby.co.uk/category/vmware/vcf/) ### [VMware Holodeck Multi-Host VCF: Lab Setup & Configuration](https://jameskilby.co.uk/2024/01/multihost-holodeck-vcf/) By[James](https://jameskilby.co.uk) January 17, 2024 · Updated June 1, 2026 How to Deploy VMware Holodeck on multiple hosts * [ ![Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/wp-content/uploads/2022/12/22225832.png) ](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) [Docker](https://jameskilby.co.uk/category/docker/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) By[James](https://jameskilby.co.uk) December 9, 2022 · Updated June 5, 2026 Late to the party or not, I have been using containers in my lab more and more and that has led me to Portainer …. * [ ![Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) ](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal. --- # WordPress Hosting with Cloudflare Pages Source: https://jameskilby.co.uk/2023/05/how-to-take-a-wordpress-site-and-publish-it-as-a-static-site-on-cloudflare-pages/ Date: 2023-05-14 Categories: Cloudflare, Hosting, Wordpress, Artificial Intelligence, Docker, Homelab, Devops ![](https://jameskilby.co.uk/wp-content/uploads/2023/05/simply-static-logo.png) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) | [Wordpress](https://jameskilby.co.uk/category/wordpress/) # WordPress Hosting with Cloudflare Pages By[James](https://jameskilby.co.uk) May 14, 2023 · Updated June 1, 2026 • 📖3 min read(646 words) I have been using Cloudflare to protect my web assets for a really long time. Throughout that time Cloudflare has been improving their capabilities and approximately 2 years ago I decided to move this blog into their worker’s product. This meant that the site was 100% served from their datacenters rather than just cached assets as done in the previous config. This had several distinct benefits. It improved the site’s performance significantly, especially for anyone outside of the UK. It effectively became unhackable (as the content is all static) and it meant that if my home connection was offline it would not impact the site. They have made deployment easier so I thought I would document how I am doing this. ## Table of Contents ## The Tooling * Admin access to your WordPress Site * Simply Static WordPress Plugin * GitHub account (Optional) * Cloudflare Account ## The Process ### WordPress Plugin Install The first step is to install and activate the plugin on your WordPress site that generates the static files. I am using the free version of [Simply Static](https://en-gb.wordpress.org/plugins/simply-static/) by the excellent Patrick Posner. When the plugin is activated go to the diagnostic page and ensure everything is looking green. In the unlikely event that it’s not then this is likely to be a WordPress version or permissions issue. ![Screenshot 2023 05 13 at 10.31.30](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-13-at-10.31.30-1024x674.png) This is what it should look like Then navigate to the settings section and ensure that the settings are set to Zip file delivery and use relative URLs and then click “generate static files”. The plugin will effectively scrape your site to build up a zip file which is available for download when this is complete. The activity log shows you the status. My scrape took just over 90s [2023-05-13 08:32:47] Setting up [2023-05-13 08:32:48] Fetched 875 of 875 pages/files [2023-05-13 08:34:23] ZIP archive created: Click here to download [2023-05-13 08:34:23] Wrapping up [2023-05-13 08:34:23] Done! Finished in 00:01:36 📋 Copy When this is finished download the zip file to your computer. ### GitHub setup The GitHub setup is optional but I like to use it. It effectively means that I have a backup of my site if needed. Log in to GitHub and create a new repository. This can be either Public or Private. I have chosen to use a private repo called WordPress Once this has been done you need to extract the Zip file created earlier and add all of the contents to this Repo. ## Cloudflare setup Log into your Cloudflare account and navigate to the pages section. Then you want to select “Create a Project” As I am using GitHub I then select the “Connect to Git” Option. I have previously connected my Cloudflare account to GitHub for the WordPress repo so I did not need to do anything here. If you have not done this ( Or installed the Cloudflare Pages App in GitHub you can do this by following this [link](https://github.com/settings/installations/22356437) Once everything is linked you can give your Project a name and set the production branch to be “Main”. You can then leave all the other settings as default ![Screenshot 2023 05 13 at 11.11.53](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-13-at-11.11.53-931x1024.png) When you hit Save and Deploy Cloudflare will initialize the build environment, clone the Git repository and publish your site to a pages.dev based domain. In my case https://wordpress-cpy.pages.dev/ You can check this is working correctly by clicking the visit site link at the top right ![Screenshot 2023 05 13 at 11.15.54](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-13-at-11.15.54.png) The final step is to add my custom domain of wordpress.jameskilby.cloud to this site. The DNS for the domain is also managed by Cloudflare so this is a trivial step If you click the name of the project, in my case WordPress, you can then scroll to the Custom Domain section. Click Set up a Custom domain and enter the domain you want to use. When I enter wordpress.jameskilby.cloud here Cloudflare automatically changed the DNS entry required but it will list the correct values if you don’t have Cloudflare DNS management. ## 📚 Related Posts * [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) * [How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare](https://jameskilby.co.uk/2025/10/how-i-deploy-my-blog-as-a-static-website-with-github-actions-and-cloudflare/) * [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) ## Similar Posts * [ ![Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu-768x450.jpeg) ](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) By[James](https://jameskilby.co.uk) October 20, 2022 · Updated July 11, 2026 For a while now I have been running this site directly from Cloudflare utilising their excellent worker’s product. * [ ![How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/wp-content/uploads/2025/01/Screenshot-2025-01-29-at-23.30.47-768x411.png) ](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) ### [How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) By[James](https://jameskilby.co.uk) January 29, 2025 · Updated July 11, 2026 Update: Hoarder has now been renamed to Karakeep due to a trademark issue I’ve been on a mission recently to regain control of my data. * [ ![Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/wp-content/uploads/2022/10/spacexs-starlink-to-supply-free-satellite-internet-to-famili_u44u.1920-768x432.jpg) ](https://jameskilby.co.uk/2022/10/starlink/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/2022/10/starlink/) By[James](https://jameskilby.co.uk) October 11, 2022 · Updated June 1, 2026 Since moving to Dorset last year internet connectivity has been the bane of my existence. Currently, I have an ADSL connection provided by my old employer Zen and a 5G connection provided by Three. * [ ![How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare](https://jameskilby.co.uk/wp-content/uploads/2025/10/Github-Actions.webp) ](https://jameskilby.co.uk/2025/10/how-i-deploy-my-blog-as-a-static-website-with-github-actions-and-cloudflare/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Devops](https://jameskilby.co.uk/category/devops/) ### [How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare](https://jameskilby.co.uk/2025/10/how-i-deploy-my-blog-as-a-static-website-with-github-actions-and-cloudflare/) By[James](https://jameskilby.co.uk) October 6, 2025 · Updated July 11, 2026 I wanted to automate the publishing of my blog from the authoring side to the public side. These are some of the improvements I made. * [ ![Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu-768x450.jpeg) ](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Wordpress](https://jameskilby.co.uk/category/wordpress/) ### [Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 I have been making several changes (mainly cosmetic) to this site over the last day or so. On most changes I have been doing an export and then uploading the site to Cloudflare using Wrangler. * [ ![Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2022/01/web-development/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) ### [Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/2022/01/web-development/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 A while ago I started messing with Cloudflare Workers. I have now moved this site permanently over to them. --- # Using Intel Optane NVMe in a VMware Homelab: Setup & Results Source: https://jameskilby.co.uk/2023/04/intel-optane/ Date: 2023-04-17 Categories: Homelab, Storage, vExpert, Automation, Veeam, Networking, Personal, VMware ![](https://jameskilby.co.uk/wp-content/uploads/2023/04/intel_optane_ssd_900p_series_aic_-_right_angle_575px.png) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) | [vExpert](https://jameskilby.co.uk/category/vexpert/) # Using Intel Optane NVMe in a VMware Homelab: Setup & Results By[James](https://jameskilby.co.uk) April 17, 2023 · Updated June 1, 2026 • 📖2 min read(316 words) I have been a VMware vExpert for many years and it has brought me many benefits over the years. I don’t think it’s an understatement to say I probably wouldn’t have my current role within VMware without it. One of the best benefits has been access to a huge amount of licences for VMware software to use in my lab. Some third parties have also recognised the award and given licences for their software. Two that I have probably made the most use of are [Runecast](https://www.runecast.com) and [Devolutions](https://devolutions.net) Remote Desktop Manager. However recently a few bits of hardware have become available and one piece in particular piqued my interest, in particular some Optane drives thanks to the very generous folks at Intel. I was lucky enough to get some together with [Gareth Edwards](http://www.virtualisedfruit.co.uk) we decided to put something together to show how good the Optane drives are and have a friendly bit of competition. ![IMG 2109](https://jameskilby.co.uk/wp-content/uploads/2023/04/IMG_2109-1024x683.jpeg) Gareth is going to keep most of the drives to start with doing some vSAN testing and I have added two to my TrueNAS server. This has both iSCSI and NFS connections back to my ESXi hosts. To be honest it would be tricky to get more than that in my storage box (in the PCI format) We’re planning on doing some back to back tests to showcase the performance difference. I am going to do this by benchmarking from VMware the following drive pairs running in my Truenas “SAN” I will also configure the Optane’s as “SLOG” attached to my main SSD storage pool. See my [TrueNAS ](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/)post for more details of the setup Drive| Model | Size| Manufacturers Read IOPS| Manufacturer Write IOPS| Max Read Sequential MB/s| Max Write Sequential MB/s ---|---|---|---|---|---|--- Consumer SSD | Samsung EVO 860 2TB | 2TB | 97,000 | 88,000 | 550 | 520 Enterprise SSD | Samsung PM863 | 960GB | 99,000 | 18,000 | 520 | 475 Intel Optane | SSD DC P4800X | 750GB | 550,000 | 550,000 | 2500 | 2200 ## 📚 Related Posts * [Lab Update – Part 2 Storage Truenas Scale](https://jameskilby.co.uk/2022/01/lab-update-part-2-storage/) * [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) * [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) ## Similar Posts * [ ![Template Deployment with Packer](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) ](https://jameskilby.co.uk/2021/01/hashicorp-packer/) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Veeam](https://jameskilby.co.uk/category/veeam/) ### [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated June 1, 2026 My lab is always undergoing change. Partially as I want to try new things or new ways of doing things. * [ ![MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/wp-content/uploads/2023/04/2157_hi_res-768x346.png) ](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) By[James](https://jameskilby.co.uk) December 19, 2022 · Updated June 1, 2026 For a while, I’ve been looking to update the networking at the core of my homelab. * [ ![VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/wp-content/uploads/2020/09/vmware_SP_HCI20.png) ](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) [Personal](https://jameskilby.co.uk/category/personal/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) By[James](https://jameskilby.co.uk) September 13, 2020 · Updated May 31, 2026 I recently sat (and passed the VMware HCI Master Specialist exam (5V0-21. * [ ![Homelab SSD Failure: How Synology RAID Saved My Data](https://jameskilby.co.uk/wp-content/uploads/2022/11/BrokenHardDive-1200x630-1-768x403.jpg) ](https://jameskilby.co.uk/2022/11/homelab-bad-days-almost/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Homelab SSD Failure: How Synology RAID Saved My Data](https://jameskilby.co.uk/2022/11/homelab-bad-days-almost/) By[James](https://jameskilby.co.uk) November 21, 2022 · Updated July 11, 2026 I recently spent 3 weeks in Ireland with my wife Wendy and our son Nate. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Homelab Network Upgrade: DACs, 40Gb/s vMotion & pfSense](https://jameskilby.co.uk/2022/01/lab-update-part-3-network/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated May 31, 2026 I have retired the WatchGuard Devices with the migration to PFSense running bare-metal in one of the Supermicro Nodes. --- # MikroTik CRS504 Review: 100Gb/s Networking in My Homelab Source: https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/ Date: 2022-12-19 Categories: Homelab, Networking, Storage, VMware, Nutanix, AWS, VMware Cloud on AWS, Mikrotik, Artificial Intelligence, Docker ![](https://jameskilby.co.uk/wp-content/uploads/2023/04/2157_hi_res.png) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) | [Storage](https://jameskilby.co.uk/category/storage/) | [VMware](https://jameskilby.co.uk/category/vmware/) # MikroTik CRS504 Review: 100Gb/s Networking in My Homelab By[James](https://jameskilby.co.uk) December 19, 2022 · Updated June 1, 2026 • 📖2 min read(303 words) For a while, I’ve been looking to update the networking at the core of my homelab. I have had some great results with the current setup utilising a number of DACs but there were a couple of things that were annoying me. Then MikroTik dropped the [CRS504-4XQ-IN](https://mikrotik.com/product/crs504_4xq_in) and if the price wasn’t horrendous then that was the route I was going to go to alleviate these issues. Yes, it’s 100Gb/s and only has 4 ports but that should be all I need… I managed to locate one in stock for £587 plus VAT ![2157 Hi Res](https://jameskilby.co.uk/wp-content/uploads/2023/04/2157_hi_res-1024x462.png) As you can see the switch has redundant power not often found at this price point. It actually has 4 ways to power it. 2 AC supplies, a DC input and then strangest of all, it can be powered by PoE in. This last point showcases how power efficient it is which is a huge win for a homelab. My plan is to utilise the existing Intel 25Gb NICs I have and split the 100’s on the switch into 4×25. This technically gives me 16 usable 25Gb ports which are way more than I need. This will allow me to accommodate the TrueNAS box and the 2x Supermicro nodes I usually run for my lab. The other 2 nodes will get an upgrade at a future date. Initial impressions are that the switch is very quiet and also incredibly power efficient. (It can be powered just from PoE). However, the config is way more complex than I have seen before. This is because it’s basically a router that can switch at wire speed. The other challenge I have had is connecting my legacy 1Gb/s network. This is still a work in progress but for now, Storage and vMotion have been migrated. I will report back when everything has been migrated. ## 📚 Related Posts * [Home Network Upgrade to 25Gb/s with MikroTik Switching](https://jameskilby.co.uk/2024/09/home-network-upgrade/) * [Unleashing the UDM Beast](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) ## Similar Posts * [ ![New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/wp-content/uploads/2024/07/IMG_6629-768x149.jpeg) ](https://jameskilby.co.uk/2024/07/new-nodes/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) By[James](https://jameskilby.co.uk) July 2, 2024 · Updated July 11, 2026 I recently decided to update some of my homelab hosts and I managed to do this at very little cost by offloading 2 of my Supermicro e200’s to fellow vExpert Paul . * [ ![Monitoring VMware Cloud on AWS: Tools & Approaches \(Part 1\)](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) ](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) [VMware](https://jameskilby.co.uk/category/vmware/) | [AWS](https://jameskilby.co.uk/category/aws/) ### [Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1)](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) By[James](https://jameskilby.co.uk) December 17, 2019 · Updated June 5, 2026 As previously mentioned I have been working a lot with VMware Cloud on AWS and one of the questions that often crops up is around an approach to monitoring. * [ ![vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) By[James](https://jameskilby.co.uk) November 17, 2023 · Updated June 1, 2026 An Overview of vSAN ESA in VMC * [ ![New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) ](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) By[James](https://jameskilby.co.uk) April 1, 2026 · Updated July 11, 2026 We’ve expanded the VMC fleet with the new i7i (i7i. * [ ![CRS-504](https://jameskilby.co.uk/wp-content/uploads/2024/09/s-l1600-768x427.jpg) ](https://jameskilby.co.uk/2024/09/home-network-upgrade/) [Mikrotik](https://jameskilby.co.uk/category/mikrotik/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Home Network Upgrade to 25Gb/s with MikroTik Switching](https://jameskilby.co.uk/2024/09/home-network-upgrade/) By[James](https://jameskilby.co.uk) September 9, 2024 · Updated June 5, 2026 My journey to superfast networking in my homelab * [ ![How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/wp-content/uploads/2025/01/Screenshot-2025-01-29-at-23.30.47-768x411.png) ](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) ### [How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) By[James](https://jameskilby.co.uk) January 29, 2025 · Updated July 11, 2026 Update: Hoarder has now been renamed to Karakeep due to a trademark issue I’ve been on a mission recently to regain control of my data. --- # Forcing an Upgrade to vSphere 8 Source: https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/ Date: 2022-12-14 Categories: Homelab, VMware, vSphere, VMware Cloud on AWS, Networking, Personal, Runecast ![](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-12-14-at-21.45.23.png) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) | [vSphere](https://jameskilby.co.uk/category/vsphere/) # Forcing an Upgrade to vSphere 8 By[James](https://jameskilby.co.uk) December 14, 2022 · Updated June 1, 2026 • 📖1 min read(283 words) I run a reasonably extensive [homelab](https://jameskilby.co.uk/lab/) that is of course built around the VMware ecosystem. So with the release of vSphere 8 I was obviously going to upgrade, however, a few personal things blocked me from doing it until now. The vCenter upgrade was smooth; however, knowing that some of the hardware I am running is a little dated this was always going to carry a risk. I tried using lifecycle manager (vLCM) within vCenter but I was faced with the following and I couldn’t see a way to force remediate from within vCenter. Therefore I resorted to doing it from the command line. ![Screenshot 2022 12 14 at 21.24.32](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-12-14-at-21.24.32-1024x245.png)Host is not compatible Firstly enable SSH on each of the hosts to be upgraded Then Allow HTTP traffic in the ESXi firewall esxcli network firewall ruleset set -e true -r httpClient 📋 Copy The next step is to list the available ESXi profiles using the online depot. In my case, I was looking for ESXi 8.0a esxcli software sources profile list -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml | grep -i ESXi-8.0a 📋 Copy Then comes the Install, I had validated that this would be ok by installing from USB to another node. All of the hardware that I needed continued to work as expected with the only significant loss being the Mellanox Connect X-3 40Gb/s adaptor. esxcli software profile update -p ESXi-8.0a-20842819-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml --no-hardware-warning 📋 Copy Update Result Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective. Reboot Required: true It also listed all of the newly installed VIB’s The last step is to disable the firewall rule and then SSH esxcli network firewall ruleset set -e false -r httpClient 📋 Copy ## 📚 Related Posts * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) * [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) * [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) ## Similar Posts * [ ![VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) By[James](https://jameskilby.co.uk) May 21, 2025 · Updated July 11, 2026 Quick reference guide to the available storage resources that you get in VMware Cloud on AWS * [ ![MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/wp-content/uploads/2023/04/2157_hi_res-768x346.png) ](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) By[James](https://jameskilby.co.uk) December 19, 2022 · Updated June 1, 2026 For a while, I’ve been looking to update the networking at the core of my homelab. * [ ![UniFi UDM Beast 1U rack-mount gateway shown front-on, with two 3.5-inch drive bays, a bank of RJ45 and SFP network ports, and dimension labels.](https://jameskilby.co.uk/wp-content/uploads/2026/06/UnifiBeast-768x219.webp) ](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Unleashing the UDM Beast](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) By[James](https://jameskilby.co.uk) June 20, 2026 · Updated August 24, 2026 Retiring my ageing WatchGuard M200, I swap in the UniFi UDM Beast — a hardware tour, first iperf and WAN tests, the BGP/FRR setup, and the homelab services I’m consolidating onto it. * [ ![VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/wp-content/uploads/2020/09/vmware_SP_HCI20.png) ](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) [Personal](https://jameskilby.co.uk/category/personal/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) By[James](https://jameskilby.co.uk) September 13, 2020 · Updated May 31, 2026 I recently sat (and passed the VMware HCI Master Specialist exam (5V0-21. * [ ![Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/wp-content/uploads/2024/06/Ubiquiti_Networks-Logo.wine_-768x512.png) ](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) By[James](https://jameskilby.co.uk) June 26, 2024 · Updated July 11, 2026 How to configure DHCP Option 43 for UniFi devices * [ ![Runecast Remediation Scripts: Auto-Fix VMware Storage Issues](https://jameskilby.co.uk/wp-content/uploads/2023/05/Runecast-Solutions-Ltd.png) ](https://jameskilby.co.uk/2023/05/runecast-remediation-scripts/) [Runecast](https://jameskilby.co.uk/category/runecast/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Runecast Remediation Scripts: Auto-Fix VMware Storage Issues](https://jameskilby.co.uk/2023/05/runecast-remediation-scripts/) By[James](https://jameskilby.co.uk) May 16, 2023 · Updated June 1, 2026 I am a huge fan of the Runecast product and luckily as a vExpert they give out NFR licences for my lab. --- # Use Portainer in a Homelab with GitHub Source: https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/ Date: 2022-12-09 Categories: Docker, Homelab, Hosting, Kubernetes, VMware, Networking, Artificial Intelligence, Automation, TrueNAS Scale ![](https://jameskilby.co.uk/wp-content/uploads/2022/12/22225832.png) [Docker](https://jameskilby.co.uk/category/docker/) | [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) | [Kubernetes](https://jameskilby.co.uk/category/kubernetes/) # Use Portainer in a Homelab with GitHub By[James](https://jameskilby.co.uk) December 9, 2022 · Updated June 5, 2026 • 📖2 min read(474 words) Late to the party or not, I have been using containers in my lab more and more and that has led me to [Portainer](https://www.portainer.io)…. I use it for managing the docker containers on my Synology but it can also be used for managing lots of other things. In their own words “Portainer accelerates container adoption. It reduces operational complexity and addresses the security challenges of running containers in Docker, Swarm, Nomad and Kubernetes.” ![Screenshot 2022 11 21 at 17.36.13](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-11-21-at-17.36.13-1024x638.png)Portainer Platform Based on the capabilities shown above I am only scratching the surface but I thought I would showcase one of the key features that I am utilising in deployment and updating containers. ## Prerequisites For now, I am going to assume that you have a Synology, Docker Installed and Portainer deployed. Plus have a GitHub (or similar) version control system that Portainer can reach. If you don’t have this check out Marius’s great blog with detailed instructions for [Docker](https://mariushosting.com/docker/) and [Portainer](https://mariushosting.com/synology-30-second-portainer-install-using-task-scheduler-docker/) It’s also important to understand user and user id’s and a great blog on this can be found [here](https://drfrankenstein.co.uk/step-2-setting-up-a-restricted-docker-user-and-obtaining-ids/) ## Portainer Container Provisioning To deploy a container with Portainer I prefer to deploy them in Stacks even if it’s a single container. Navigate to the Stacks section and select “Add Stack” ![Screenshot 2022 11 21 at 17.59.31](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2022-11-21-at-17.59.31.png)Add Stack I will show the deployment of a stack to run Grafana as it’s a simple container and popular with homelabbers. The first step is to name the stack which I have called grafana – note these must be lowercase. The YAML code that the container will use is shown below. This is stored in my private docker repository located at https://github.com/jameskilbynet/docker/grafana version: '3.3' services: grafana: container_name: grafana image: grafana/grafana user: "1024" ports: - "3000:3000" volumes: - /volume1/docker/grafana:/var/lib/grafana 📋 Copy Reviewing the YAML code it’s clear that volume mapping is being used. This is a feature to expose a file or folder of the underlying Docker Host into the container. In this case /volume1/docker/grafana Therefore this folder must be created manually. Docker will then map this folder into var/lib/grafana within the container to the /volume1/docker/grafana folder on my Synology Within Portainer to use a git repository change to the Git tab and enter the relevant details. ![Screenshot 2022 11 22 at 11.56.52 1](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-11-22-at-11.56.52-2048x1133-1-1024x567.png) I am using a private repository and therefore need to enable authentication to allow Portainer to access the private repo. This requires a Personal Access Token to be created as documented [here](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token). When all of the relevant settings are input then click “Deploy the Stack” Portainer will then instruct docker to fetch the container (if it hasn’t already) and deploy the stack. If for any reason this errors, it is usually because the volume map is not set up correctly or a permissions issue. If it’s the former the docker app in Synology will show you. For more homelab automation, see my post on [managing my homelab with SemaphoreUI](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/). ## 📚 Related Posts * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) * [How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) * [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) ## Similar Posts * [ ![Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-12-14-at-21.45.23.png) ](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) By[James](https://jameskilby.co.uk) December 14, 2022 · Updated June 1, 2026 I run a reasonably extensive homelab that is of course built around the VMware ecosystem. * [ ![What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2018/03/cloudflare/) [Hosting](https://jameskilby.co.uk/category/hosting/) ### [What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained](https://jameskilby.co.uk/2018/03/cloudflare/) By[James](https://jameskilby.co.uk) March 27, 2018 · Updated June 1, 2026 Cloudflare – What is it and why would I care? I have been using Cloudflare for a long time. * [ ![MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/wp-content/uploads/2023/04/2157_hi_res-768x346.png) ](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) By[James](https://jameskilby.co.uk) December 19, 2022 · Updated June 1, 2026 For a while, I’ve been looking to update the networking at the core of my homelab. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) By[James](https://jameskilby.co.uk) March 27, 2026 · Updated May 31, 2026 A walkthrough of my self-hosted AI stack: Ollama, Open WebUI, ComfyUI, Whishper, n8n, Qdrant, SearxNG, and a full observability layer — all running on my own hardware with Docker Compose. * [ ![TrueNAS Logo](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-22-at-18.49.21-768x198.png) ](https://jameskilby.co.uk/2023/11/truenas-scale-useful-commands/) [Kubernetes](https://jameskilby.co.uk/category/kubernetes/) | [TrueNAS Scale](https://jameskilby.co.uk/category/truenas-scale/) ### [TrueNAS Scale CLI Reference: Storage & Kubernetes Commands](https://jameskilby.co.uk/2023/11/truenas-scale-useful-commands/) By[James](https://jameskilby.co.uk) November 13, 2023 · Updated April 11, 2026 A list of useful Truenas Scale commands * [ ![Wa](https://jameskilby.co.uk/wp-content/uploads/2025/04/210902461-012e7273-413a-4ec7-be44-e854347f5a21-768x180.png) ](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Warp – The intelligent terminal](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) By[James](https://jameskilby.co.uk) April 11, 2025 · Updated June 1, 2026 How Warp is helping me run my homelab. --- # Homelab SSD Failure: How Synology RAID Saved My Data Source: https://jameskilby.co.uk/2022/11/homelab-bad-days-almost/ Date: 2022-11-21 Categories: Homelab, Storage, Synology, Hosting, Artificial Intelligence, Docker, Ansible, Automation, Github, Networking ![](https://jameskilby.co.uk/wp-content/uploads/2022/11/BrokenHardDive-1200x630-1.jpg) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) | [Synology](https://jameskilby.co.uk/category/synology/) # Homelab SSD Failure: How Synology RAID Saved My Data By[James](https://jameskilby.co.uk) November 21, 2022 · Updated July 11, 2026 • 📖2 min read(362 words) I recently spent 3 weeks in Ireland with my wife Wendy and our son Nate. This involves driving from the south coast of Dorset up to Scotland and then getting a ferry over to Belfast before travelling west to the Republic. While driving I got a slack notification that one of my SSDs in my Synology DS918+ was in a bad way. (You do have notifications set up, right? ) The timing could not have been worse. Now for years anyone that I have talked to at VMUG’s around data protection etc, I have advocated never using RAID 5. The reasons for this are well known. But when it’s your homelab and your wallet is on the line corners sometimes need to be cut. When I got to Ireland I logged into the Synology. It was pretty obvious it wasn’t happy ![Homelab bad days \(almost\) Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2022-11-14-at-20.56.39.png)Unhappy Array (for historical reasons I was using slots 1, 2 and 4) and it had pages and pages like this…. ![Drive Errors 2048x236](https://jameskilby.co.uk/wp-content/uploads/2023/04/Drive-Errors-2048x236-1-1024x118.png) I had 3 SSDs in this particular array all from around the same time. Out of curiosity, I checked how old they were. The drive that failed had a power-on-hours count of 49599 hours (5.6 Years). It had more than served its purpose so some replacements were ordered for when I was back in Poole. For now I had to run the risk and hope another drive didn’t fail. Returning home to swap a drive was never going to go down well with the family. I decided to order 4x 2TB SSDs and went with the Samsung 870 QVO’s. They are def on the budget end of SSDs but are cost-effective and given the limited network bandwidth on my Synology were more than adequate. I first replaced the failed drive and then when this was complete extended the array to be 4x Disk before replacing the other 2 existing drives one at a time. ![Homelab bad days \(almost\) Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2022-11-21-at-16.43.51.png)Everything is green again Everything is now back to being green and hunky dory. I have run scrubs to validate all data. I still have RAID 5 on both volumes but I have undertaken a task to improve some of my backup handling. Just in case….. ## 📚 Related Posts * [My First Homelab Storage Setup: HP Gen8 & Xpenology](https://jameskilby.co.uk/2018/01/lab-storage/) * [Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) * [Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/2019/02/lab-storage-2/) ## Similar Posts * [ ![Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/wp-content/uploads/2022/10/spacexs-starlink-to-supply-free-satellite-internet-to-famili_u44u.1920-768x432.jpg) ](https://jameskilby.co.uk/2022/10/starlink/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/2022/10/starlink/) By[James](https://jameskilby.co.uk) October 11, 2022 · Updated June 1, 2026 Since moving to Dorset last year internet connectivity has been the bane of my existence. Currently, I have an ADSL connection provided by my old employer Zen and a 5G connection provided by Three. * [ ![How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/wp-content/uploads/2025/01/Screenshot-2025-01-29-at-23.30.47-768x411.png) ](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) ### [How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) By[James](https://jameskilby.co.uk) January 29, 2025 · Updated July 11, 2026 Update: Hoarder has now been renamed to Karakeep due to a trademark issue I’ve been on a mission recently to regain control of my data. * [ ![Managing my Homelab with SemaphoreUI](https://jameskilby.co.uk/wp-content/uploads/2025/07/semaphore-768x768.png) ](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Managing my Homelab with SemaphoreUI](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/) By[James](https://jameskilby.co.uk) September 2, 2025 · Updated June 5, 2026 An intro on how I use SemaphoreUI to manage my Homelab * [ ![Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/wp-content/uploads/2026/04/packer-github-actions-vsphere-pipeline-768x452.png) ](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) [Automation](https://jameskilby.co.uk/category/automation/) | [Github](https://jameskilby.co.uk/category/github/) ### [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) By[James](https://jameskilby.co.uk) April 30, 2026 · Updated June 5, 2026 A walk-through of an automated golden-image pipeline for vSphere: HashiCorp Packer builds six Ubuntu LTS templates (22.04, 24.04, 26.04 — server and desktop) driven by three GitHub Actions workflows covering PR validation, parallel template builds, and ISO management via a self-hosted runner. * [ ![Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/wp-content/uploads/2022/12/22225832.png) ](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) [Docker](https://jameskilby.co.uk/category/docker/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) By[James](https://jameskilby.co.uk) December 9, 2022 · Updated June 5, 2026 Late to the party or not, I have been using containers in my lab more and more and that has led me to Portainer …. * [ ![UniFi UDM Beast 1U rack-mount gateway shown front-on, with two 3.5-inch drive bays, a bank of RJ45 and SFP network ports, and dimension labels.](https://jameskilby.co.uk/wp-content/uploads/2026/06/UnifiBeast-768x219.webp) ](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Unleashing the UDM Beast](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) By[James](https://jameskilby.co.uk) June 20, 2026 · Updated August 24, 2026 Retiring my ageing WatchGuard M200, I swap in the UniFi UDM Beast — a hardware tour, first iperf and WAN tests, the BGP/FRR setup, and the homelab services I’m consolidating onto it. --- # Static WordPress hosting using Cloudflare Source: https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/ Date: 2022-10-20 Categories: Cloudflare, Hosting, Wordpress, AWS, Homelab, Devops, Artificial Intelligence, Automation ![](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu.jpeg) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) | [Wordpress](https://jameskilby.co.uk/category/wordpress/) # Static WordPress hosting using Cloudflare By[James](https://jameskilby.co.uk) October 20, 2022 · Updated July 11, 2026 • 📖4 min read(851 words) For a while now I have been running this site directly from [Cloudflare](https://www.cloudflare.com) utilising their excellent Workers product. I did this originally as a learning exercise but due to the benefits it brought and the ease of use I decided to stick with it. The benefits are several fold: * Crazy Web Performance (Typically full page load in less than 500ms see below) * High Availability globally * Zero attack surface * ~Zero hosting costs ![Screenshot 2022 10 19 at 23.02.09 1](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2022-10-19-at-23.02.09-2048x860-1-1024x430.png) Gtmetrix Speed Test A couple of people have asked me about the setup so I thought I would try and document an overview. ## Introduction Firstly although this setup is effectively Serverless you need to have a copy of WordPress somewhere to make your edits. This doesn’t need to be exposed to the outside world (or can even be run just on your workstation/laptop using [local](https://localwp.com). I have chosen to keep WordPress running in some docker containers on my [Synology](https://jameskilby.co.uk/lab/) The security of this setup is increased as WordPress is not published to the outside world. I do all of my editing locally, however, if I needed to change this I would leverage Cloudflare Access. Not publishing the site externally hugely reduces the attack surface as there is no implementation to attack. Editing and contributing content works in exactly the same way. The difference is how content is published… At a very high level, I have a WordPress plugin that generates static content. This is then pushed into a GitHub Repo and from there, it’s pushed to Cloudflare Pages. ## WordPress Setup Within WordPress, I am utilising the simply static [plugin](https://simplystatic.com) to generate the static content. Initially, I was using the free plugin but I have since decided to upgrade to the paid-for version. In terms of the plugin setup, I have it set to use relative URLs with a delivery method set to GitHub. This is done as the WordPress site is on my NAS and therefore has a URL of http://nas1.jameskilby.net/ I have then excluded the WordPress management URLs in the static generation. http://wordpress.jameskilby.cloud/wp-json https://wordpress.jameskilby.cloud/wp-login.php 📋 Copy ## GitHub Setup In the deployment section, I have added the relevant details so that the simply static plugin can push the relevant content into a GitHub repo… ![Static WordPress hosting using Cloudflare Screenshot](https://jameskilby.co.uk/wp-content/uploads/2022/10/Screenshot-2022-07-09-at-17.29.07-2048x1191-1-1024x596.png) Within GitHub, I have created a repo (called jameskilbycouksite) I have made this private but it could easily be a public repo. I have then added my [personal access token ](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)so that SimplyStatic can push to the repo and lastly added a webhook to be called when the site has been updated (more on this a bit later). ## Cloudflare Page Builds The Cloudflare setup is probably the most complex part of the process but once it has been set up it’s fire and forget. The Pages site needs to be set up in Cloudflare and then a Webhook is created that the WordPress site can trigger. This is triggered when all of the site is committed to GitHub to tell Cloudflare to refresh the build. When you are logged into Cloudflare navigate to the pages section and create a project. At this point, I selected connect to Git and log in/select the relevant GitHub account and repo. For this to function, the GitHub Pages application needs to have been given permission in your GitHub Account ![Static WordPress hosting using Cloudflare Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2022-07-09-at-18.53.21-1024x925.png)GitHub Pages Permissions When a page is complete and published at a WordPress level a new button is enabled called “Generate Static”. When this is clicked it will run the process to generate the static page, push it to GitHub and then to Cloudflare. The Paid plugin allows for updating a single page, whereas the free plugin needs to generate the entire site again. As seen below, a single page usually takes around 9 seconds to update with this setup. Generating the entire site and uploading took approx 30 mins. [2022-07-17 18:37:46] Setting up [2022-07-17 18:37:49] Fetched 1 of 1 pages/files [2022-07-17 18:37:51] Committed / Updated 81 of 81 pages/files [2022-07-17 18:37:55] Wrapping up [2022-07-17 18:37:55] Done! Finished in 00:00:09 📋 Copy Lastly, I have removed Google Analytics for privacy reasons and just utilise the web analytics built natively into Cloudflare. ## Costs The cost of running the site in Cloudflare for my blog is exactly zero which is amazing. This is one of the reasons I decided to buy the plugin as it was roughly equivalent to a year’s worth of decent hosting. If your site is more popular then you may require one of the paid plans that start at $5 a month. The free tier gives you: **Workers Bundled (Workers Compute time)** * Up to 10ms CPU time per request * Lowest latency after the first request * Up to 100,000 requests per day (UTC+0) **KV (Key Value Storage)** * Global low-latency key-value edge storage * Up to 100,000 read operations per day * Up to 1,000 write, delete, list operations per day Although I have paid $99 for the single-site plugin this is not essential. It does however eliminate some manual steps which I was keen to avoid. Plus although I am not currently using them it does support forms and search capabilities. For more like this why not follow me on [Twitter](https://x.com/jameskilbynet) ## 📚 Related Posts * [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) * [WordPress Hosting with Cloudflare Pages](https://jameskilby.co.uk/2023/05/how-to-take-a-wordpress-site-and-publish-it-as-a-static-site-on-cloudflare-pages/) * [Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/2022/01/web-development/) ## Similar Posts * [ ![AWS Status Page – Monitoring Included](https://jameskilby.co.uk/wp-content/uploads/2018/05/AmazonWebservices_Logo.svg_-768x307.png) ](https://jameskilby.co.uk/2018/05/aws-status-page-monitoring-included/) [AWS](https://jameskilby.co.uk/category/aws/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [AWS Status Page – Monitoring Included](https://jameskilby.co.uk/2018/05/aws-status-page-monitoring-included/) By[James](https://jameskilby.co.uk) May 15, 2018 · Updated June 1, 2026 AWS Status Page – Enhancements The tool I deployed, lambstatus, supports pulling metrics from AWS Cloudwatch and displaying them. * [ ![Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/wp-content/uploads/2022/10/spacexs-starlink-to-supply-free-satellite-internet-to-famili_u44u.1920-768x432.jpg) ](https://jameskilby.co.uk/2022/10/starlink/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/2022/10/starlink/) By[James](https://jameskilby.co.uk) October 11, 2022 · Updated June 1, 2026 Since moving to Dorset last year internet connectivity has been the bane of my existence. Currently, I have an ADSL connection provided by my old employer Zen and a 5G connection provided by Three. * [ ![Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu-768x450.jpeg) ](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Wordpress](https://jameskilby.co.uk/category/wordpress/) ### [Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 I have been making several changes (mainly cosmetic) to this site over the last day or so. On most changes I have been doing an export and then uploading the site to Cloudflare using Wrangler. * [ ![How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare](https://jameskilby.co.uk/wp-content/uploads/2025/10/Github-Actions.webp) ](https://jameskilby.co.uk/2025/10/how-i-deploy-my-blog-as-a-static-website-with-github-actions-and-cloudflare/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Devops](https://jameskilby.co.uk/category/devops/) ### [How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare](https://jameskilby.co.uk/2025/10/how-i-deploy-my-blog-as-a-static-website-with-github-actions-and-cloudflare/) By[James](https://jameskilby.co.uk) October 6, 2025 · Updated July 11, 2026 I wanted to automate the publishing of my blog from the authoring side to the public side. These are some of the improvements I made. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 Part 2 of my self-hosted AI stack series. I cover container resource sizing, dual-network isolation via Traefik and Cloudflare Tunnels, and every database powering the stack — PostgreSQL, ClickHouse, Redis, Qdrant, MinIO, MongoDB, SQLite, Prometheus, and Jaeger — plus the backup strategy for each. * [ ![Fixing Wrangler Node.js Version Conflicts After Brew Upgrade](https://jameskilby.co.uk/wp-content/uploads/2022/01/WranglerCrab-1-768x256.png) ](https://jameskilby.co.uk/2022/01/wrangler-and-node-versions/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) ### [Fixing Wrangler Node.js Version Conflicts After Brew Upgrade](https://jameskilby.co.uk/2022/01/wrangler-and-node-versions/) By[James](https://jameskilby.co.uk) January 15, 2022 · Updated July 11, 2026 I am a massive fan of the brew package management system for macOS and use it on all of my Macs. I typically just upgrade everything blindly and have never had an issue. --- # Starlink Satellite Internet Review: Rural Broadband Solution Source: https://jameskilby.co.uk/2022/10/starlink/ Date: 2022-10-11 Categories: Homelab, Hosting, Automation, Github, Wordpress, VMware, Nutanix, Veeam, Ansible ![](https://jameskilby.co.uk/wp-content/uploads/2022/10/spacexs-starlink-to-supply-free-satellite-internet-to-famili_u44u.1920.jpg) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) # Starlink Satellite Internet Review: Rural Broadband Solution By[James](https://jameskilby.co.uk) October 11, 2022 · Updated June 1, 2026 • 📖4 min read(747 words) Since moving to Dorset last year internet connectivity has been the bane of my existence. Currently, I have an ADSL connection provided by my old employer Zen and a 5G connection provided by Three. These are both plumbed into my WatchGuard Firewall with multi-wan configured. Most of the time the usability is ok but there are multiple times a day when latency or packet loss is not acceptable. From living at previous locations where I had multiple companies that would deliver Gigabit Internet connections it was far from ideal. My job at VMware involves numerous calls with customers every day, typically presenting things to them and often with video on. Even a small interruption can leave a bad impression. VMware sells an SD-WAN product (VeloCloud) and enables staff to use them. I was given one not long after I started, however, integrating it with my network has been suboptimal (as the device is locked down by VMware Corporate IT)… One of the nice features of the WatchGuard is that it monitors the WAN interfaces for me. At the time of writing this blog the Zen interface was dropping approx 10% of traffic (This is unusually bad) and although it isn’t easy to see the Three interface was also a non-zero packet loss. The Starlink one is showing as I had configured the interface but it was not connected at the time. ![Screenshot 2022 10 05 at 12.56.21 1](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2022-10-05-at-12.56.21-1-1024x187.png)Packet Loss on WAN interfaces Given we have no plans to move I decided to bite the bullet and pay for Starlink. Prior to doing this I downloaded the Starlink App and used the Visibility function to scan the sky for obstructions. I climbed out onto part of our roof that is flat and scanned the sky. This uses the iPhone camera and the accelerometers to scan the sky for obstructions. Luckily the results were positive. The intention is to drop the Zen connection when my contract runs out in a few months’ time. I can work around not having the /29 that comes with it with some Cloudflare magic that I will try and document in a follow-up post. One thing to remember is that by default Starlink doesn’t ship you an ethernet adaptor. I ordered mine on the same day but it was shipped separately meaning I had to wait a few days before I could properly hook the system up. It’s a bit annoying to pay for an extra adaptor but £35 won’t break the bank. The other thing to pay attention to is the length of cable that you need. I went for the standard 75ft cable but it only just covers the length I need to get back to my firewall. While waiting on the adaptor I decided to do some speed tests over Wi-Fi from my iPhone 13 Pro ![IMG 8711](https://jameskilby.co.uk/wp-content/uploads/2023/04/IMG_8711-805x1024.png) To prove it wasn’t Starlink massaging the numbers I also performed a number of speed tests to other third parties and got broadly similar results. When the ethernet adaptor arrived I put the Starlink router in bypass mode. This can only be done from the iOS/Android App. The WatchGuard firewall interface was set to DHCP but once bypass was enabled it would not pick up a new address. I, therefore, rebooted the firewall and it came back with a CGNAT 100.65.x.x address removing the additional level of NAT. I will add a static entry on the firewall so I can still get to the Starlink management page. I believe the business service offers a fully routable IP. ![IMG 8718 1](https://jameskilby.co.uk/wp-content/uploads/2023/04/IMG_8718-710x1536-1-473x1024.png) Once all the above had been done, it was a case of connecting Starlink to the relevant port on my firewall. I had already configured the port as an “External interface” and set the IP endpoint to monitor. The WatchGuard immediately picked up that the ethernet port came up and tried to hit its monitoring endpoint. As soon as this was successful traffic started to route over this link as intended. ![Screenshot 2022 10 11 at 13.03.37 1](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2022-10-11-at-13.03.37-2048x695-1-1024x348.png)Traffic starting to use Starlink ( Bottom Left) Due to the way I have configured traffic egress from the WatchGuard to the internet each TCP/IP session will stay on a link (unless there is a failure) and they will get moved to the next available link. New sessions will be distributed based on the weightings I have applied. With the WatchGuard I have this configured on a per-firewall policy basis. Most traffic I allow uses any available link however some traffic I steer e.g. SMTP is configured to only use the Zen link. ## 📚 Related Posts * [Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) * [Unleashing the UDM Beast](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) ## Similar Posts * [ ![Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/wp-content/uploads/2026/04/packer-github-actions-vsphere-pipeline-768x452.png) ](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) [Automation](https://jameskilby.co.uk/category/automation/) | [Github](https://jameskilby.co.uk/category/github/) ### [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) By[James](https://jameskilby.co.uk) April 30, 2026 · Updated June 5, 2026 A walk-through of an automated golden-image pipeline for vSphere: HashiCorp Packer builds six Ubuntu LTS templates (22.04, 24.04, 26.04 — server and desktop) driven by three GitHub Actions workflows covering PR validation, parallel template builds, and ISO management via a self-hosted runner. * [ ![Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu-768x450.jpeg) ](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Wordpress](https://jameskilby.co.uk/category/wordpress/) ### [Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 I have been making several changes (mainly cosmetic) to this site over the last day or so. On most changes I have been doing an export and then uploading the site to Cloudflare using Wrangler. * [ ![Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-12-14-at-21.45.23.png) ](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) By[James](https://jameskilby.co.uk) December 14, 2022 · Updated June 1, 2026 I run a reasonably extensive homelab that is of course built around the VMware ecosystem. * [ ![New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/wp-content/uploads/2024/07/IMG_6629-768x149.jpeg) ](https://jameskilby.co.uk/2024/07/new-nodes/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) By[James](https://jameskilby.co.uk) July 2, 2024 · Updated July 11, 2026 I recently decided to update some of my homelab hosts and I managed to do this at very little cost by offloading 2 of my Supermicro e200’s to fellow vExpert Paul . * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Veeam](https://jameskilby.co.uk/category/veeam/) ### [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated June 1, 2026 My lab is always undergoing change. Partially as I want to try new things or new ways of doing things. * [ ![vSphere Power Management Ansible Playbooks with Semaphore](https://jameskilby.co.uk/wp-content/uploads/2026/04/vsphere-power-management-ansible-768x403.png) ](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automating vSphere Power Management driven by Ansible and SemaphoreUI](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) By[James](https://jameskilby.co.uk) April 15, 2026 · Updated June 1, 2026 In this post I’ll walk through how I use vSphere Power Management driven by Ansible and SemaphoreUI to automatically reduce ESXi host electricity consumption — saving real money on my Octopus Agile tariff by toggling hosts between Low Power and Balanced policies. Introduction One of the larger costs of running my homelab is the electricity…. --- # Fixing Wrangler Node.js Version Conflicts After Brew Upgrade Source: https://jameskilby.co.uk/2022/01/wrangler-and-node-versions/ Date: 2022-01-15 Categories: Cloudflare, Hosting, Wordpress, Personal, Ansible, Automation, Artificial Intelligence ![](https://jameskilby.co.uk/wp-content/uploads/2022/01/WranglerCrab-1.png) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) # Fixing Wrangler Node.js Version Conflicts After Brew Upgrade By[James](https://jameskilby.co.uk) January 15, 2022 · Updated July 11, 2026 • 📖1 min read(217 words) I am a massive fan of the brew package management system for macOS and use it on all of my Macs. I typically just upgrade everything blindly and have never had an issue… until today. I went to push some changes to this site and got the following error message: wrangler publish --env production /usr/local/bin/node: bad option: --openssl-legacy-provider Error: failed to execute `"/usr/local/bin/node" "--openssl-legacy-provider" "/Users/jameskilby/Library/Caches/.wrangler/wranglerjs-1.19.6" "--output-file=/var/folders/89/z9t99sg16h9cw3t86wp920jr0000gn/T/.wranglerjs_outputGBCY9" "--wasm-binding=WASM_MODULE" "--no-webpack-config=1" "--use-entry=/Users/jameskilby/jameskilbycouk/workers-site/index.js"`: exited with exit status: 9 📋 Copy A quick bit of digging and it turns out the error was due to a new version of node being used (Version 17) The brew search command showed that I still had node16 installed brew search node ==> Formulae libbitcoin-node node ✔ node-sass node@12 node@16 ✔ nodebrew nodenv llnode node-build node@10 node@14 node_exporter nodeenv ode 📋 Copy So I then removed the link to the generic node and forced the link to node@16 brew unlink node brew link --overwrite node@16 📋 Copy Once this was done wrangler worked as expected and published my changes rangler publish --env production ✨ Built successfully, built project size is 13 KiB. 🌀 Using namespace for Workers Site "__jameskilbycouk-production-workers_sites_assets" ✨ Success 🌀 Uploading site files 📋 Copy Certainly not a complex fix but hopefully handy for anyone playing with Wrangler on a Mac and using homebrew. ## 📚 Related Posts * [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) * [Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) * [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) ## Similar Posts * [ ![Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu-768x450.jpeg) ](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Wordpress](https://jameskilby.co.uk/category/wordpress/) ### [Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 I have been making several changes (mainly cosmetic) to this site over the last day or so. On most changes I have been doing an export and then uploading the site to Cloudflare using Wrangler. * [ ![What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2018/03/cloudflare/) [Hosting](https://jameskilby.co.uk/category/hosting/) ### [What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained](https://jameskilby.co.uk/2018/03/cloudflare/) By[James](https://jameskilby.co.uk) March 27, 2018 · Updated June 1, 2026 Cloudflare – What is it and why would I care? I have been using Cloudflare for a long time. * [ ![Analytics in a privacy focused world](https://jameskilby.co.uk/wp-content/uploads/2023/11/plausible-analytics-icon-top.png) ](https://jameskilby.co.uk/2023/11/analytics-in-a-privacy-focused-world/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Analytics in a privacy focused world](https://jameskilby.co.uk/2023/11/analytics-in-a-privacy-focused-world/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 I recently helped my friend Dean Lewis @veducate with some hosting issues. As part of the testing of this he kindly gave me a login to his WordPress instance. * [ ![Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu-768x450.jpeg) ](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) By[James](https://jameskilby.co.uk) October 20, 2022 · Updated July 11, 2026 For a while now I have been running this site directly from Cloudflare utilising their excellent worker’s product. * [ ![Automated VCF 9 Offline Depot architecture diagram showing Traefik reverse proxy and Nginx file server stack](https://jameskilby.co.uk/wp-content/uploads/2026/04/offlinedepot.png) ](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) By[James](https://jameskilby.co.uk) April 10, 2026 · Updated July 11, 2026 One Bash script turns a fresh Ubuntu VM into a VCF 9 Offline Depot: Traefik, Nginx, basic auth, and Let’s Encrypt wildcard certs via Cloudflare DNS. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 Part 2 of my self-hosted AI stack series. I cover container resource sizing, dual-network isolation via Traefik and Cloudflare Tunnels, and every database powering the stack — PostgreSQL, ClickHouse, Redis, Qdrant, MinIO, MongoDB, SQLite, Prometheus, and Jaeger — plus the backup strategy for each. --- # Lab Update – Part 2 Storage Truenas Scale Source: https://jameskilby.co.uk/2022/01/lab-update-part-2-storage/ Date: 2022-01-11 Categories: Homelab, Storage, Networking, Artificial Intelligence, Automation, Kubernetes, TrueNAS Scale, VMware, Ansible ![](https://jameskilby.co.uk/wp-content/uploads/2022/01/maxresdefault.jpeg) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) # Lab Update – Part 2 Storage Truenas Scale By[James](https://jameskilby.co.uk) January 11, 2022 · Updated June 1, 2026 • 📖2 min read(495 words) The HP Z840 has changed its role to a permanent storage box running Truenas Scale. This is in addition to my Synology DS918+… TrueNas is the successor to FreeNas, a very popular BSD based StorageOS and TrueNas scale is a fork of this based on Linux. The Synology has been an amazing piece of kit handling VM storage, Docker, File sharing and Plex duties. It will maintain most of these roles. However, I needed something faster for my lab and especially something with faster network access. The Synology I had was limited to 2xGb interfaces. An option was to buy a newer Synology but I felt a better route was to try a “Storage OS” on some of my existing hardware with some network tweaks to be seen later. The Spec of the Z840 Workstation I am using to run Truenas is 2x Intel Xeon CPU E5-2673 v3 @ 2.40GHz and 128GB of RAM. More than enough for my needs. I will use the 2xGb interfaces for management and the new network for a combination of iSCSI/NFS… Within one of the 5 1/2 inch drive bays I have placed an Icy Dock express cage giving me 6 additional 2.5 drive bays. ![Lab Update - Part 2 Storage Truenas Scale](https://jameskilby.co.uk/wp-content/uploads/2023/04/iu-1.jpeg)ICY Dock I have placed the TrueNAS OS on 2x Intel 80GB SSDs running within these. For now, I have configured 2 storage pools. One NVMe with a single INTEL SSDPE2MX020T7. This will be very high-performance storage but for now, is not redundant. I plan to mitigate this with regular backups. The other is reusing some 3TB WD Red drives from the Synology in a Raid-Z config. I will likely add some SSD as a ZLOG at a later date. This is presented back to my VMware hosts using iSCSI over a 25Gb direct network. I have only done basic testing so far. John would murder me for this but I did a test using CrystalDisk just to check everything was in the correct realms of what I would expect. (These numbers will have been influenced by cache size as they easily fit in the working set) I chose to use the real-world test built into CrystalDiskMark. I’m sure the numbers would be much higher with other configs. This was also done with a 4GiB file which isn’t huge, but I used it to keep the testing short. I will come back and test the storage properly at a later date. ![Lab Update - Part 2 Storage Truenas Scale Screenshot](https://jameskilby.co.uk/wp-content/uploads/2022/01/Screenshot-2022-01-17-at-09.11.14.png) NVMe initial performance test. I had a number of VMs running on the NVMe test at the time testing was running. This may account for the read being slightly lower than the SATA disks but the Write performance is much higher. ![Lab Update - Part 2 Storage Truenas Scale](https://jameskilby.co.uk/wp-content/uploads/2022/01/NVMe.png) TrueNAS SATA test. 3x SATA disks in RAID 5 ![Lab Update - Part 2 Storage Truenas Scale Screenshot](https://jameskilby.co.uk/wp-content/uploads/2022/01/Screenshot-2022-01-17-at-09.07.00.png) Comparison with Synology 3x SSD Raid 5 ( 2x1Gb/s ) Networking. You can see this is significantly lower than the Freenas box. The Synology was running a number of docker containers and apps at the time so I’m sure performance could be higher. ![Synology](https://jameskilby.co.uk/wp-content/uploads/2022/01/Synology-1024x725.png) ## 📚 Related Posts * [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) * [Homelab Storage Refresh (Part 1)](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/) * [Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) ## Similar Posts * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Homelab Network Upgrade: DACs, 40Gb/s vMotion & pfSense](https://jameskilby.co.uk/2022/01/lab-update-part-3-network/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated May 31, 2026 I have retired the WatchGuard Devices with the migration to PFSense running bare-metal in one of the Supermicro Nodes. * [ ![Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/wp-content/uploads/2023/04/81-ZoEW24UL._SL1500_-768x461.jpg) ](https://jameskilby.co.uk/2019/02/lab-storage-2/) [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/2019/02/lab-storage-2/) By[James](https://jameskilby.co.uk) February 10, 2019 · Updated June 5, 2026 Since starting my new role with Xtravirt my Homelab has gone through several fairly significant changes. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) By[James](https://jameskilby.co.uk) March 27, 2026 · Updated May 31, 2026 A walkthrough of my self-hosted AI stack: Ollama, Open WebUI, ComfyUI, Whishper, n8n, Qdrant, SearxNG, and a full observability layer — all running on my own hardware with Docker Compose. * [ ![TrueNAS Logo](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-22-at-18.49.21-768x198.png) ](https://jameskilby.co.uk/2023/11/truenas-scale-useful-commands/) [Kubernetes](https://jameskilby.co.uk/category/kubernetes/) | [TrueNAS Scale](https://jameskilby.co.uk/category/truenas-scale/) ### [TrueNAS Scale CLI Reference: Storage & Kubernetes Commands](https://jameskilby.co.uk/2023/11/truenas-scale-useful-commands/) By[James](https://jameskilby.co.uk) November 13, 2023 · Updated April 11, 2026 A list of useful Truenas Scale commands * [ ![Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup](https://jameskilby.co.uk/wp-content/uploads/2023/04/IMG_4536-scaled-1-768x1024.jpg) ](https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup](https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated February 16, 2026 Quite a few changes have happened in the lab recently. I decided to do a multipart blog on the changes. * [ ![Managing my Homelab with SemaphoreUI](https://jameskilby.co.uk/wp-content/uploads/2025/07/semaphore-768x768.png) ](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Managing my Homelab with SemaphoreUI](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/) By[James](https://jameskilby.co.uk) September 2, 2025 · Updated June 5, 2026 An intro on how I use SemaphoreUI to manage my Homelab --- # Homelab Network Upgrade: DACs, 40Gb/s vMotion & pfSense Source: https://jameskilby.co.uk/2022/01/lab-update-part-3-network/ Date: 2022-01-06 Categories: Homelab, Networking, Artificial Intelligence, Automation, Storage, VMware [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) # Homelab Network Upgrade: DACs, 40Gb/s vMotion & pfSense By[James](https://jameskilby.co.uk) January 6, 2022 · Updated May 31, 2026 • 📖1 min read(113 words) I have retired the WatchGuard Devices with the migration to PFSense running bare-metal in one of the Supermicro Nodes. I will likely virtualise this in the future. In terms of network/switching I have moved to an intermediate step here. vMotion and Storage are running over DAC’s while VMware management and VM traffic is still over 1Gb/s networking. This is working incredibly well. Due to the onboard network cards I am able to vMotion at 40Gb/s which for a home lab is game-changing. Storage is currently over 25Gb/s and the long-term plan is to get a 25Gb/s switch. The contenders at the moment are a Used Dell enterprise switch or possibly a QNAP box. ## 📚 Related Posts * [Home Network Upgrade to 25Gb/s with MikroTik Switching](https://jameskilby.co.uk/2024/09/home-network-upgrade/) * [Unleashing the UDM Beast](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) * [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) ## Similar Posts * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) By[James](https://jameskilby.co.uk) March 27, 2026 · Updated May 31, 2026 A walkthrough of my self-hosted AI stack: Ollama, Open WebUI, ComfyUI, Whishper, n8n, Qdrant, SearxNG, and a full observability layer — all running on my own hardware with Docker Compose. * [ ![TrueNAS Logo](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-22-at-18.49.21-768x198.png) ](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Homelab Storage Refresh (Part 1)](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/) By[James](https://jameskilby.co.uk) May 23, 2023 · Updated July 11, 2026 Table of Contents Background ZFS Overview Read Cache (ARC and L2ARC) ZIL (ZFS Intent Log) Hardware Background I have just completed the move of all my production and media-based storage/services to TrueNAS Scale. ( I will just refer to this as TrueNAS) This is based on my HP Z840 and I have now retired my… * [ ![Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-12-14-at-21.45.23.png) ](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) By[James](https://jameskilby.co.uk) December 14, 2022 · Updated June 1, 2026 I run a reasonably extensive homelab that is of course built around the VMware ecosystem. * [ ![Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/wp-content/uploads/2024/06/Ubiquiti_Networks-Logo.wine_-768x512.png) ](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) By[James](https://jameskilby.co.uk) June 26, 2024 · Updated July 11, 2026 How to configure DHCP Option 43 for UniFi devices * [ ![Wa](https://jameskilby.co.uk/wp-content/uploads/2025/04/210902461-012e7273-413a-4ec7-be44-e854347f5a21-768x180.png) ](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Warp – The intelligent terminal](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) By[James](https://jameskilby.co.uk) April 11, 2025 · Updated June 1, 2026 How Warp is helping me run my homelab. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 Part 2 of my self-hosted AI stack series. I cover container resource sizing, dual-network isolation via Traefik and Cloudflare Tunnels, and every database powering the stack — PostgreSQL, ClickHouse, Redis, Qdrant, MinIO, MongoDB, SQLite, Prometheus, and Jaeger — plus the backup strategy for each. --- # Lab Update – Desired Workloads Source: https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/ Date: 2022-01-06 Categories: Homelab, Veeam, VMware, Networking, Storage, Artificial Intelligence, Docker, VMware Cloud on AWS [Homelab](https://jameskilby.co.uk/category/homelab/) | [Veeam](https://jameskilby.co.uk/category/veeam/) | [VMware](https://jameskilby.co.uk/category/vmware/) # Lab Update – Desired Workloads By[James](https://jameskilby.co.uk) January 6, 2022 · Updated June 1, 2026 • 📖1 min read(141 words) My lab is always undergoing change. Partially as I want to try new things or new ways of doing things. Sometimes because I break things (not always by accident); sometimes it’s a great way to learn… I decided to list the workloads I am looking to run (some of these are already in place) ## Infrastructure Active Directory – Done vRealize Lifecycle Manager – Done vRealize Operations Manager – Done vRealize Log Insight vRealize Network Insight Vault SecretServer Nutanix CE Hyper-V Tanzu CE VMware Horizon – Done VMware HCX vCloud Director VMware Cloud Foundation Docker K8s K3s ## Networking PFsense – Done Sophos UTM – Done Avi ## Monitoring Grafana Runecast Veeam One ## Backups/Recovery Veeam Backup & Replication Veeam Service Provider Console VMware Cloud Disaster Recovery SRM ## Storage TrueNAS HP Store Once Minio AWS Storage Gateway ## Automation VMware Event Broker vRealize Automation VMware Orchestrator ## 📚 Related Posts * [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) * [Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup](https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/) * [How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) ## Similar Posts * [ ![Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/wp-content/uploads/2024/06/Ubiquiti_Networks-Logo.wine_-768x512.png) ](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) By[James](https://jameskilby.co.uk) June 26, 2024 · Updated July 11, 2026 How to configure DHCP Option 43 for UniFi devices * [ ![TrueNAS Logo](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-22-at-18.49.21-768x198.png) ](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Homelab Storage Refresh (Part 1)](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/) By[James](https://jameskilby.co.uk) May 23, 2023 · Updated July 11, 2026 Table of Contents Background ZFS Overview Read Cache (ARC and L2ARC) ZIL (ZFS Intent Log) Hardware Background I have just completed the move of all my production and media-based storage/services to TrueNAS Scale. ( I will just refer to this as TrueNAS) This is based on my HP Z840 and I have now retired my… * [ ![Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU](https://jameskilby.co.uk/wp-content/uploads/2024/10/pexels-tara-winstead-8386440-768x512.jpg) ](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) ### [Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) By[James](https://jameskilby.co.uk) October 11, 2024 · Updated July 11, 2026 Artificial intelligence is all the rage at the moment, It’s getting included in every product announcement from pretty much every vendor under the sun. * [ ![UniFi UDM Beast 1U rack-mount gateway shown front-on, with two 3.5-inch drive bays, a bank of RJ45 and SFP network ports, and dimension labels.](https://jameskilby.co.uk/wp-content/uploads/2026/06/UnifiBeast-768x219.webp) ](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Unleashing the UDM Beast](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) By[James](https://jameskilby.co.uk) June 20, 2026 · Updated August 24, 2026 Retiring my ageing WatchGuard M200, I swap in the UniFi UDM Beast — a hardware tour, first iperf and WAN tests, the BGP/FRR setup, and the homelab services I’m consolidating onto it. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [My First Homelab Storage Setup: HP Gen8 & Xpenology](https://jameskilby.co.uk/2018/01/lab-storage/) By[James](https://jameskilby.co.uk) January 6, 2018 · Updated June 1, 2026 I have been meaning to post around some of the lab setup for a while. Although it changes frequently at present it’s as below. * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/09/vmc-host-errors/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) By[James](https://jameskilby.co.uk) September 15, 2020 · Updated June 5, 2026 Learn how host failures are handled within VMC --- # Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup Source: https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/ Date: 2022-01-06 Categories: Homelab, VMware, Networking, VMware Cloud on AWS, Artificial Intelligence, Docker, Personal ![](https://jameskilby.co.uk/wp-content/uploads/2023/04/IMG_4536-scaled-1.jpg) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) # Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup By[James](https://jameskilby.co.uk) January 6, 2022 · Updated February 16, 2026 • 📖1 min read(223 words) Quite a few changes have happened in the lab recently, so I decided to do a multipart blog on the changes. The refresh was triggered by the purchase of a SuperMicro Server ([2027TR-H71FRF](https://www.supermicro.com/products/system/2U/2027/SYS-2027TR-H71FRF.cfm)) chassis with 4x X9DRT Nodes / Blades. This is known as a BigTwin configuration in SuperMicro parlance. This is something I was very familiar with as most of the Nutanix nodes that we deployed at Zen were of this design. Each node has 2x Intel Xeon CPU E5-2670 @ 2.60GHz. 3 Nodes have 192GB of RAM and 1 has 128GB giving a combined 64c/128t and 704GB of RAM. Quite an amount to be squeezed into 2U. This is a huge uplift in compute resources so is more than welcome. I also decided to put what I could in a proper rack and purchased a Startech 25U rack to mount it. ![Lab Update - Compute](https://jameskilby.co.uk/wp-content/uploads/2023/04/IMG_4536-scaled-1-768x1024.jpg) One of the Compute nodes was removed from the server… The plan is also to make a serious network investment (to move me away from 1Gb) but I haven’t completely decided what that will look like. This will likely be the final investment for a while. With the additional Supermicro compute available, I have made the Z840 a sort of standalone node. I will keep it around as it’s the only thing capable of running my Nvidia K2. ## 📚 Related Posts * [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) * [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) * [How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) ## Similar Posts * [ ![MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/wp-content/uploads/2023/04/2157_hi_res-768x346.png) ](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [MikroTik CRS504 Review: 100Gb/s Networking in My Homelab](https://jameskilby.co.uk/2022/12/100gb-s-in-my-homelab-sort-of/) By[James](https://jameskilby.co.uk) December 19, 2022 · Updated June 1, 2026 For a while, I’ve been looking to update the networking at the core of my homelab. * [ ![An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) By[James](https://jameskilby.co.uk) August 14, 2025 · Updated July 11, 2026 This is single page intended to collate every single feature of the current VMware Cloud on AWS hosts for easy comparison. * [ ![Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU](https://jameskilby.co.uk/wp-content/uploads/2024/10/pexels-tara-winstead-8386440-768x512.jpg) ](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) ### [Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) By[James](https://jameskilby.co.uk) October 11, 2024 · Updated July 11, 2026 Artificial intelligence is all the rage at the moment, It’s getting included in every product announcement from pretty much every vendor under the sun. * [ ![Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/wp-content/uploads/2026/01/Firefly_Gemini-Flash-768x417.png) ](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) By[James](https://jameskilby.co.uk) January 27, 2026 · Updated June 5, 2026 How to leverage Content Libraries to deploy into VMware Cloud on AWS faster. * [ ![VMware – Going out with a Bang!](https://jameskilby.co.uk/wp-content/uploads/2023/10/rnli-logo-768x384.png) ](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [VMware – Going out with a Bang!](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) By[James](https://jameskilby.co.uk) October 7, 2023 · Updated June 1, 2026 There is a lot of uncertainty with VMware at the moment. This is all due to the pending acquisition by Broadcom. * [ ![Wa](https://jameskilby.co.uk/wp-content/uploads/2025/04/210902461-012e7273-413a-4ec7-be44-e854347f5a21-768x180.png) ](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Warp – The intelligent terminal](https://jameskilby.co.uk/2025/04/warp-the-intelligent-terminal/) By[James](https://jameskilby.co.uk) April 11, 2025 · Updated June 1, 2026 How Warp is helping me run my homelab. --- # Cloudflare Workers – Limits of the free tier Source: https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/ Date: 2022-01-04 Categories: Hosting, Wordpress, Homelab, Cloudflare, Devops, Artificial Intelligence, Automation, Docker ![](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu.jpeg) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Wordpress](https://jameskilby.co.uk/category/wordpress/) # Cloudflare Workers – Limits of the free tier By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 • 📖1 min read(280 words) I have been making several changes (mainly cosmetic to this site over the last day or so) On most changes I have been doing an export and then uploading the site to Cloudflare using Wrangler. After a while I received an email from Cloudflare saying: _Hi,_ You’re _50% of the way to reaching one of your daily Cloudflare Workers KV free tier limits. You can view complete usage information in the dashboard._ _If you do not take any action and exceed the daily limit, requests to the Workers KV API will return 429 errors, and operations within a worker will fail until the limit resets at 2022-01-05 at 00:00:00 UTC._ I decided to keep ploughing on with my changes to see what happens when I go over this. The next email I received was slightly more informative as it said which limit I had breached. _You have exceeded the daily Cloudflare Workers KV free tier limit of 1000 Workers KV put operations._ When I next tried to publish I got the following: wrangler publish --env production ✨ Built successfully, built project size is 13 KiB. 🌀 Using namespace for Workers Site "__jameskilbycouk-production-workers_sites_assets" ✨ Success 🌀 Uploading site files Error: ⚠️ Code 10048: your account has reached the free usage limit for this operation for today 📋 Copy The same error was observed when running “Wrangler dev” I had the option of upgrading for the cost of $5 which includes a very generous 10 million read operations and 1 million each of write, delete and list operations however it’s just not worth it for this site. I will wait until tomorrow to publish. Detailed info on the limits is published [here ](https://developers.cloudflare.com/workers/platform/limits) ## 📚 Related Posts * [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) * [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) * [WordPress Hosting with Cloudflare Pages](https://jameskilby.co.uk/2023/05/how-to-take-a-wordpress-site-and-publish-it-as-a-static-site-on-cloudflare-pages/) ## Similar Posts * [ ![Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/wp-content/uploads/2022/10/spacexs-starlink-to-supply-free-satellite-internet-to-famili_u44u.1920-768x432.jpg) ](https://jameskilby.co.uk/2022/10/starlink/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/2022/10/starlink/) By[James](https://jameskilby.co.uk) October 11, 2022 · Updated June 1, 2026 Since moving to Dorset last year internet connectivity has been the bane of my existence. Currently, I have an ADSL connection provided by my old employer Zen and a 5G connection provided by Three. * [ ![How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare](https://jameskilby.co.uk/wp-content/uploads/2025/10/Github-Actions.webp) ](https://jameskilby.co.uk/2025/10/how-i-deploy-my-blog-as-a-static-website-with-github-actions-and-cloudflare/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Devops](https://jameskilby.co.uk/category/devops/) ### [How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare](https://jameskilby.co.uk/2025/10/how-i-deploy-my-blog-as-a-static-website-with-github-actions-and-cloudflare/) By[James](https://jameskilby.co.uk) October 6, 2025 · Updated July 11, 2026 I wanted to automate the publishing of my blog from the authoring side to the public side. These are some of the improvements I made. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 Part 2 of my self-hosted AI stack series. I cover container resource sizing, dual-network isolation via Traefik and Cloudflare Tunnels, and every database powering the stack — PostgreSQL, ClickHouse, Redis, Qdrant, MinIO, MongoDB, SQLite, Prometheus, and Jaeger — plus the backup strategy for each. * [ ![How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/wp-content/uploads/2025/01/Screenshot-2025-01-29-at-23.30.47-768x411.png) ](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) ### [How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) By[James](https://jameskilby.co.uk) January 29, 2025 · Updated July 11, 2026 Update: Hoarder has now been renamed to Karakeep due to a trademark issue I’ve been on a mission recently to regain control of my data. * [ ![Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu-768x450.jpeg) ](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) By[James](https://jameskilby.co.uk) October 20, 2022 · Updated July 11, 2026 For a while now I have been running this site directly from Cloudflare utilising their excellent worker’s product. * [ ![Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/wp-content/uploads/2026/01/Website-Optimisations-768x560.png) ](https://jameskilby.co.uk/2026/01/web-development-improvements/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) By[James](https://jameskilby.co.uk) January 15, 2026 · Updated July 11, 2026 I have spent the Christmas break making some improvements to this blog. --- # Hosting This Blog on Cloudflare Workers: Why & How I Did It Source: https://jameskilby.co.uk/2022/01/web-development/ Date: 2022-01-04 Categories: Hosting, Cloudflare, Personal, Wordpress, Artificial Intelligence, Automation, Consulting, AWS ![](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2.png) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Personal](https://jameskilby.co.uk/category/personal/) | [Wordpress](https://jameskilby.co.uk/category/wordpress/) # Hosting This Blog on Cloudflare Workers: Why & How I Did It By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 • 📖1 min read(211 words) A while ago I started messing with Cloudflare Workers. I have now moved this site permanently over to them. This is partly related to some issues I have been having with internet access at home. Prior to this, the site ran from within my lab. This means the site is now super fast (hopefully :p). It was cached by Cloudflare previously and now it’s served entirely from within their network. ![Web performance report](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2021-12-19-at-19.30.43-2048x751-1-1024x376.png)The speed report looks good To enable this workflow I have retained a WordPress instance running within a docker instance on my Synology. I have also set up Cloudflare teams to access this. This allows me to have the convenience of utilising WordPress as a CMS without having to worry about the security aspect as all public content is static. It also means that I don’t care about the uptime of the particular instance. When I have finished writing the post I use the Simply Static Plugin to generate the required static files. I then copy these to my wrangler site location on my laptop and ask Wrangler to publish them. wrangler publish --env production 📋 Copy I wrote a post on the initial setup of Cloudflare Workers. Check it out if you are interested in more of how this works. ## 📚 Related Posts * [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) * [Analytics in a privacy focused world](https://jameskilby.co.uk/2023/11/analytics-in-a-privacy-focused-world/) * [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) ## Similar Posts * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 Part 2 of my self-hosted AI stack series. I cover container resource sizing, dual-network isolation via Traefik and Cloudflare Tunnels, and every database powering the stack — PostgreSQL, ClickHouse, Redis, Qdrant, MinIO, MongoDB, SQLite, Prometheus, and Jaeger — plus the backup strategy for each. * [ ![And now for something completely different](https://jameskilby.co.uk/wp-content/uploads/2018/10/fWbXybA7-768x193.png) ](https://jameskilby.co.uk/2018/10/and-now-for-something-completely-different/) [Personal](https://jameskilby.co.uk/category/personal/) ### [And now for something completely different](https://jameskilby.co.uk/2018/10/and-now-for-something-completely-different/) By[James](https://jameskilby.co.uk) October 16, 2018 · Updated July 11, 2026 I have worked for my current employer Zen Internet for 3. Over that time I have changed roles from what was originally a customer-focused role into a role with one of the core platform teams. * [ ![My Home Office Setup & Upgrades](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_7017-scaled-1-768x576.jpeg) ](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) [Personal](https://jameskilby.co.uk/category/personal/) | [Consulting](https://jameskilby.co.uk/category/consulting/) ### [My Home Office Setup & Upgrades](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) By[James](https://jameskilby.co.uk) January 5, 2021 · Updated June 1, 2026 Given the year that was 2020 and at the time of writing a distinct improvement appears a long way off I decided it was time to up my homeworking game. * [ ![Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/wp-content/uploads/2026/01/Website-Optimisations-768x560.png) ](https://jameskilby.co.uk/2026/01/web-development-improvements/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) By[James](https://jameskilby.co.uk) January 15, 2026 · Updated July 11, 2026 I have spent the Christmas break making some improvements to this blog. * [ ![WordPress Hosting with Cloudflare Pages](https://jameskilby.co.uk/wp-content/uploads/2023/05/simply-static-logo.png) ](https://jameskilby.co.uk/2023/05/how-to-take-a-wordpress-site-and-publish-it-as-a-static-site-on-cloudflare-pages/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [WordPress Hosting with Cloudflare Pages](https://jameskilby.co.uk/2023/05/how-to-take-a-wordpress-site-and-publish-it-as-a-static-site-on-cloudflare-pages/) By[James](https://jameskilby.co.uk) May 14, 2023 · Updated June 1, 2026 Table of Contents The Tooling The Process WordPress Plugin Install GitHub setup Cloudflare setup I have been using Cloudflare to protect my web assets for a really long time. * [ ![AWS Solution Architect – Associate](https://jameskilby.co.uk/wp-content/uploads/2018/05/AmazonWebservices_Logo.svg_-768x307.png) ](https://jameskilby.co.uk/2019/12/aws-solution-architect-associate/) [AWS](https://jameskilby.co.uk/category/aws/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [AWS Solution Architect – Associate](https://jameskilby.co.uk/2019/12/aws-solution-architect-associate/) By[James](https://jameskilby.co.uk) December 16, 2019 · Updated May 31, 2026 I renewed my AWS Solution Architect certification. --- # How Apple Content Caching Works: Speed Up iOS & Mac Updates Source: https://jameskilby.co.uk/2021/02/apple-content-caching/ Date: 2021-02-08 Categories: Apple ![](https://jameskilby.co.uk/wp-content/uploads/2021/02/iu.png) [Apple](https://jameskilby.co.uk/category/apple/) # How Apple Content Caching Works: Speed Up iOS & Mac Updates By[James](https://jameskilby.co.uk) February 8, 2021 · Updated July 11, 2026 • 📖2 min read(313 words) I have slowly morphed into an Apple fanboy over the last decade or so collecting a large number of devices ever since my first MacBook Air back in 2011. When you’re in the ecosystem additional devices just make sense. I currently have: * Mac Mini M1 * MacBook Air 2018 * iPhone XS * iPad Pro 9.7 * Apple Watch 4 * iPhone 7 ( Work Phone) My wife also has * MacBook Air * iPad * iPhone Xs All in all a lot of Apple devices need Software & Application updates etc. Christian Mohn mentioned a while back that macOS can do content caching and it’s very easy to implement. If only WSUS was as straightforward…. I turned it on on my Mac M1 last year and I’ve been pleasantly surprised with the results. ![Cache Served Graph](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2021-02-05-at-23.55.08.png)Cache Results after 30 days Over the last 30 days, all of the devices have pulled down nearly 100GB of data. The cache mechanism on my Mac Mini was able to serve 41GB of this locally leading to faster updates and less traffic on my WAN link. Although I have a fast link this may be useful for homes where multiple people are working from home or the connection is less than stellar. More info on it can be found [here](https://support.apple.com/en-gb/guide/mac-help/mchl9388ba1b/mac) To enable it is very straightforward Go to the Apple menu ![Apple Content Caching Screenshot](https://help.apple.com/assets/5FCA9DF4094622AC2BC6F94E/5FCA9E00094622AC2BC6F96C/en_GB/2f77cc85238452e25cb517130188bf99.png) > System Preferences, click Sharing then select Content Caching. ![Apple Content Caching Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2021-02-08-at-12.56.43.png) ![Apple Content Caching Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2021-02-08-at-12.57.20-2-1020x1024.png) Click the popup menu and choose the content to be shared. I have mine set to All Content. This includes * macOS updates and Internet Recovery images * Apps and app updates from the Mac App Store * iCloud data caching (photos and documents) * Apple TV updates & Screensavers * iOS apps, Apple TV apps and app updates * watchOS apps and app updates ![Apple Content Caching Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2021-02-08-at-12.57.46-1024x827.png) In the options, you can control the amount of disk space to use. It defaults to 10%. Clients will need to be restarted to find the server. ![Apple Content Caching Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2021-02-08-at-12.57.37-1.png) ## 📚 Related Posts * [MacBook Air 2018 Retina Review: My Setup & Build Scripts](https://jameskilby.co.uk/2018/12/new-laptop/) ## Similar Posts * [ ![MacBook Air 2018 Retina Review: My Setup & Build Scripts](https://jameskilby.co.uk/wp-content/uploads/2018/12/colorware-768x384.jpg) ](https://jameskilby.co.uk/2018/12/new-laptop/) [Apple](https://jameskilby.co.uk/category/apple/) ### [MacBook Air 2018 Retina Review: My Setup & Build Scripts](https://jameskilby.co.uk/2018/12/new-laptop/) By[James](https://jameskilby.co.uk) December 4, 2018 · Updated June 1, 2026 I decided it was about time I replaced my trusted MacBook Air that I purchased back in 2011. --- # Template Deployment with Packer Source: https://jameskilby.co.uk/2021/01/hashicorp-packer/ Date: 2021-01-21 Categories: Automation, Homelab, VMware, Artificial Intelligence, VMware Cloud on AWS, Nutanix, Hosting ![](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) # Template Deployment with Packer By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 • 📖2 min read(366 words) Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. But until now I had never played with it. That was until I saw the below tweet by the legend that is [William Lam](https://williamlam.com/) > 🤩 THIS IS AWESOME 🤩 > > For those who ❤️ Automation, check out [@tenthirtyam](https://x.com/tenthirtyam?ref_src=twsrc%5Etfw) amazing repo w/working examples to build Photon OS 3/4, Ubuntu 18.04/20.04, RHEL 7/8, CentOS 7/8 & Windows Server 2016/2019 VM images using Packer & pushing to Cont Lib! Well done > > — William Lam (@lamw) [November 30, 2020](https://x.com/lamw/status/1333546472155987969?ref_src=twsrc%5Etfw) ## Getting Packer Running That was the kicker I needed to go and have a look at getting Packer set up and running. As I run a Mac as my main machine it was easy to get deployed following the instructions using [brew](https://brew.sh) and the GitHub Repo that William had pointed to. ## Fixing the Packer Datacenter Error However, once I had added my vSphere credentials I was having a few issues. On execution, I was getting the error message “default datacenter resolves to multiple instances”. I did a bit of digging and discovered that the code didn’t specify a VMware datacenter and in my lab environment, I have 2 physical sites with a VMware datacenter in each, therefore, I needed to specify which one. Once this was fixed I started rolling out some Linux templates and it worked flawlessly until I got to the photon 4 server. I again spotted a typo and a syntax error on a command being passed into the photon VM. I corrected this and decided that actually, I should work out how to get this fixed upstream to help the wider community. Once this was working it was onto the Windows templates. Once all the relevant ISOs and configs were in place I was able to fully deploy 4 Windows servers that were patched and added to my content library in about 40 mins. So whenever I go to deploy an Image it is always up to date… Thanks to [Ryan](https://github.com/tenthirtyam) for an incredible piece of work I now lean on tools like this for wider [homelab automation](https://jameskilby.co.uk/2025/09/managing-my-homelab-with-semaphoreui/). ## 📚 Related Posts * [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) * [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) * [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) ## Similar Posts * [ ![Free Octopus Agile Battery & Solar Calculator: 5 Batteries Tested](https://jameskilby.co.uk/wp-content/uploads/2026/03/Octopus-Energy-logo.jpg) ](https://jameskilby.co.uk/2026/03/octopus-agile-battery-solar-calculator/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Free Octopus Agile Battery & Solar Calculator: 5 Batteries Tested](https://jameskilby.co.uk/2026/03/octopus-agile-battery-solar-calculator/) By[James](https://jameskilby.co.uk) March 9, 2026 · Updated July 11, 2026 I am quite a heavy consumer of electricity at home. This is primarily driven by my lab but having a young son and two golden retrievers means more washing and drying. * [ ![Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup](https://jameskilby.co.uk/wp-content/uploads/2023/04/IMG_4536-scaled-1-768x1024.jpg) ](https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Homelab Compute Upgrade: SuperMicro BigTwin & vSphere Setup](https://jameskilby.co.uk/2022/01/lab-update-part-1-compute/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated February 16, 2026 Quite a few changes have happened in the lab recently. I decided to do a multipart blog on the changes. * [ ![New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) ](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) By[James](https://jameskilby.co.uk) April 1, 2026 · Updated July 11, 2026 We’ve expanded the VMC fleet with the new i7i (i7i. * [ ![Running Nutanix CE at Home: AHV Setup & First Impressions](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2018/01/nutanix-ce/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [Running Nutanix CE at Home: AHV Setup & First Impressions](https://jameskilby.co.uk/2018/01/nutanix-ce/) By[James](https://jameskilby.co.uk) January 6, 2018 · Updated June 1, 2026 I ran a Nutanix CE server at home for a little while when it first came out. However, due to the fairly high requirements, it didn’t make sense to me to continue running it at home. * [ ![An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) By[James](https://jameskilby.co.uk) August 14, 2025 · Updated July 11, 2026 This is single page intended to collate every single feature of the current VMware Cloud on AWS hosts for easy comparison. * [ ![Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/wp-content/uploads/2022/10/spacexs-starlink-to-supply-free-satellite-internet-to-famili_u44u.1920-768x432.jpg) ](https://jameskilby.co.uk/2022/10/starlink/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/2022/10/starlink/) By[James](https://jameskilby.co.uk) October 11, 2022 · Updated June 1, 2026 Since moving to Dorset last year internet connectivity has been the bane of my existence. Currently, I have an ADSL connection provided by my old employer Zen and a 5G connection provided by Three. --- # My Home Office Setup & Upgrades Source: https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/ Date: 2021-01-05 Categories: Personal, Consulting, AWS, Devops, Hosting, VMware ![](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_7017-scaled-1.jpeg) [Personal](https://jameskilby.co.uk/category/personal/) | [Consulting](https://jameskilby.co.uk/category/consulting/) # My Home Office Setup & Upgrades By[James](https://jameskilby.co.uk) January 5, 2021 · Updated June 1, 2026 • 📖8 min read(1,570 words) Given the year that was 2020 and at the time of writing a distinct improvement appears a long way off I decided it was time to up my homeworking game. I bought a beautiful Dell 49″ monitor back in mid-2020 that has been the central focus of my home office. With this setup I always had my trusty Macbook Air shut as I have no need for any more screen estate. This caused some challenges for video calls with the ever-present Zoom/Teams as the webcam I was using was no longer available. I tried a few options to improve this but they were all a bit suboptimal. I decided to do a few drastic upgrades over the last month or so. This post is to show exactly what I’m running as of January 2021. A few people have asked me exactly how I have my desk setup so this is an exhaustive list of everything on my desk, however, not everything is a recent purchase. I’ll start with the BOM. I have attached a link to where they can be obtained (If it’s an Amazon link it will be an affiliate link) and the price I paid. If the exact item is no longer produced the link will go to an equivalent. I won’t go into detail on every component but if you have any questions please ask. ## Bill of Materials Item | Cost £| Link ---|---|--- Standing Desk | 500 | [Ikea](https://www.ikea.com/gb/en/p/idasen-desk-sit-stand-brown-beige-s89280969/) Headphone Stand | 22.90 | [Amazon](https://amzn.to/2XQYvsp) Laptop Stand | 19.99 | [Amazon](https://amzn.to/2XPNVli) Apple Macbook Air 2018 13″ | 1779 | [Apple](https://www.apple.com/uk/shop/buy-mac/macbook-air) Apple iPad Pro 9.7″ | 599.39 | [Apple](https://www.apple.com/uk/shop/buy-ipad/ipad-pro) Apple Pencil | 99.99 | [Amazon](https://amzn.to/2Kd0vYC) Dell 49″ Monitor | 1100 | [Dell](https://www.dell.com/en-uk/shop/dell-ultrasharp-49-curved-monitor-u4919dw/apd/210-argk/monitors-monitor-accessories) Ergotron MX | 119.76 | [Amazon](https://amzn.to/2Lz46RI) Elgato Ring Light | 189.95 | [Amazon](https://amzn.to/38PI0mM) Canon EOS 5D Mk IV | 2799 | [Amazon](https://amzn.to/39J1jgx) Canon 17-40MM F4 Lens + Hoya UV Filter | 573 | [Amazon](https://amzn.to/38PI0mM) Blackmagic Design Atem Mini | 303 | [Amazon](https://amzn.to/38PI0mM) Shure MV7 + Stand | 289 | [Amazon](https://amzn.to/2KHOvPf) Anker USB-C Charger | 29.99 | [Amazon](https://amzn.to/3iE2LoC) Amazon Echo Show 5 | 29.99 | [Amazon](https://amzn.to/2Kd0vYC) Yubikey 4 | 39.99 | [Amazon](https://amzn.to/3ifPHp7) Beats Solo Pro* | 169.99 | [Amazon](https://amzn.to/3qwVsC2) Knobsound Amp | 41.99 | [Amazon](https://amzn.to/3sByXxx) Assorted Cables | 100 | N/A **Total** | **8306.93** | ## Desk – Ikea Idasen Sit/Stand 160 The desk is the focal point of a home office and therefore it’s an important choice. I decided a while ago to go for a standing desk and I also prefer having a wide desk so I got the 1.6M wide Ikea Idasen desk and it’s awesome. I don’t stand as often as I possibly should but it’s something I do every day. The choice of a standing desk makes everything else a bit harder as everything needs to move with the desk while remaining stable. It also means you need to be careful with cable lengths and mounting things. I have 3 cables running to the desk. A single power lead and two Cat 5e’s one for my Macbook Air and one for my Atem Mini. ![IMG 7017 scaled 1](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_7017-scaled-1-1024x768.jpeg)Desk Setup ## Monitor – Dell UltraSharp 49 Curved U4919DW The Monitor dominates my entire view and it’s huge, no other way to describe it. It’s the equivalent of 2×27″ Monitors sat side by side and runs in 5120 x 1440 at 60 Hz. My Mac is connected via USB-C This helps in reducing the cables on the desk. The Monitor offers a number of USB ports which offer a convenient place to plug my Yubikey as my Mac is placed out of reach. The Monitor has the typical inputs you expect and has a nice picture in picture feature that I sometimes utilise with the Atem Mini. The Atem outputs via HDMI as well as USB-C and I utilise this to check the preview output that will be viewed. ## Monitor Arm – Ergotron MX The Dell Monitor does come with an included stand but it took up a large footprint on my desk. Due to the size and weight of the Dell U4919DW (the screen is 11.4 kg), it heavily limited the choice of stands that were suitable. I settled on the MX from Ergotron and it’s perfect. The springs did need tightening to accommodate the weight but once that was done it holds the screen where it’s set. (I slightly change the position depending on if I’m standing or sitting at the desk.) ## Camera – Canon EOS 5D MkIV I have had a variety of Canon SLR’s over the last 14 years and the 5D is certainly the best, It is an exceptional piece of kit. It offers a number of key features that I can utilise. One of the advantages is that it has a clean HDMI output. This is fed directly into the Atem Mini. It has excellent focus capabilities using the Dual Pixel CMOS and this combined with AF subject tracking allows me to always be in focus. ![IMG 7021 scaled 1 1](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_7021-scaled-1-1-1024x768.jpeg) The 5D also has WIFI control capabilities allowing me to both preview or amend any of the settings from an app on my iPhone. This is useful for amending the audio gain or exposure options if needed. This is a bit of a pain to do without the app so it’s a big advantage. Coupled to the front of the 5D I have a Canon 17-40 F4 L lens. I tend to have this wide open to throw the background out of focus a bit. A sample image is below. ![Screenshot 2021 01 15 at 00.09.45](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2021-01-15-at-00.09.45-1024x567.png) ## Lighting – Elgato Ring Light The Canon 5D is mounted in the centre of an Elgato Ring light. This is clipped to the back of the centre of my desk. I have it set to 5700K in the above picture which seems to be the colour temperature that’s most neutral. A picture of what it looks like from my point of view is below. ![IMG 7021 scaled 1](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_7021-scaled-1-1024x768.jpeg) ## Sound – Shure MV7 ![IMG 7023 scaled 1](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_7023-scaled-1-1024x768.jpeg) Shure are probably the most famous name in microphones, the MV7 is a slight departure from the professional-grade microphones like the SM58 that they are famous for. This is because the MV7 is a dual output mic with a USB-C as well as a standard XLR output. It’s also capable of outputting to both at the same time. This gives me the flexibility of recording directly to my Macbook if needed or my preferred route into the 5D. This is preferred as it removes any possible sync issues between audio and video plus has the added benefit of being capable of being recorded within the 5D as well as on my Macbook via the ATEM. ## Video – Blackmagic Design Atem Mini ![IMG 7022 scaled 1](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_7022-scaled-1-1024x768.jpeg) The ATEM Mini is the centrepiece of the media part of my setup. It controls all of the inputs and outputs. Although it’s the most basic in the ATEM range it’s got a pretty advanced feature set. It has 4 HDMI inputs and I typically use 3 of these. They are for the 5D, Macbook Air and iPad Pro. The mixer then has two distinct outputs. One is HDMI and the other is USB-C which is plugged back into my Mac. The connection into the Mac signals as a USB Webcam. This makes it easy to use in any application. It also does a very neat trick that John Nicholson pointed out to me. As well as pretending to be a webcam it also shows up as an external monitor. Where this becomes insanely useful is for apps like PowerPoint. Powerpoint detects the second monitor and presents the full screen to the Mini that the audience see. But I retain the presenter view on-screen with presenter notes timings etc. ![Screenshot 2021 01 15 at 02.06.47](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2021-01-15-at-02.06.47.png) The ATEM also has upstream and downstream capabilities, allowing for the mixing of multiple video screens. The example here is taking this presentation being run from my iPad and overlaying it over the Camera input. I will do a separate post with some info on how it was put together and add some audio. If you have read this far and you have been paying attention you may have noticed that a Mac Mini has appeared on my desk. Yes, I snuck in one of the new M1 arm based Mac’s. I must say I have been blown away by the performance of it. I decided to go for the 8GB of Ram with 512GB disk option after watching a number of YouTube videos. So far it has been able to handle everything I have been throwing at it with ease. I will do a separate post concentrating on the M1 once I have lived with it for a bit longer. One feature that appears not to work is the ATEM mini dual display described above. Further investigation is required. ## Future purchases ### Elgato Streamdeck A lot of people have these and it’s something I would strongly consider. I purchased the software version and run this on my iPad Pro. I think this was a sensible thing to do to see how it can be used with the various integrations and although it’s a simple concept I have been very happy with it and will likely upgrade to a hardware version at some point. ### Herman Miller Aeron When travel was a thing I was lucky enough to visit a number of customers in the high tech/financial industry. This allowed me to “sample” a number of quality seats. The best of which has to be the Aeron. It has an amazing reputation and a large price tag to go with it. So, for now, it’s just on the wish list. *The Beats Solo’s were a gift as part of the Veeam Vanguard Program. I didn’t purchase these with my own money. ## 📚 Related Posts * [Analytics in a privacy focused world](https://jameskilby.co.uk/2023/11/analytics-in-a-privacy-focused-world/) * [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) * [VMware – Going out with a Bang!](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) ## Similar Posts * [ ![AWS Solution Architect – Associate](https://jameskilby.co.uk/wp-content/uploads/2018/05/AmazonWebservices_Logo.svg_-768x307.png) ](https://jameskilby.co.uk/2019/12/aws-solution-architect-associate/) [AWS](https://jameskilby.co.uk/category/aws/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [AWS Solution Architect – Associate](https://jameskilby.co.uk/2019/12/aws-solution-architect-associate/) By[James](https://jameskilby.co.uk) December 16, 2019 · Updated May 31, 2026 I renewed my AWS Solution Architect certification. * [ ![My First GitHub Pull Request: Contributing to Open Source](https://jameskilby.co.uk/wp-content/uploads/2020/12/175jvBleoQfAZJc3sgTSPQA.jpg) ](https://jameskilby.co.uk/2020/12/my-first-pull/) [Devops](https://jameskilby.co.uk/category/devops/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [My First GitHub Pull Request: Contributing to Open Source](https://jameskilby.co.uk/2020/12/my-first-pull/) By[James](https://jameskilby.co.uk) December 22, 2020 · Updated June 5, 2026 I was initially going to add in the contents of this post to one that I have been writing about my exploits with HashiCorp Packer but I decided it probably warranted being separated out. * [ ![Analytics in a privacy focused world](https://jameskilby.co.uk/wp-content/uploads/2023/11/plausible-analytics-icon-top.png) ](https://jameskilby.co.uk/2023/11/analytics-in-a-privacy-focused-world/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Analytics in a privacy focused world](https://jameskilby.co.uk/2023/11/analytics-in-a-privacy-focused-world/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 I recently helped my friend Dean Lewis @veducate with some hosting issues. As part of the testing of this he kindly gave me a login to his WordPress instance. * [ ![VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/wp-content/uploads/2020/09/vmware_SP_HCI20.png) ](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) [Personal](https://jameskilby.co.uk/category/personal/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) By[James](https://jameskilby.co.uk) September 13, 2020 · Updated May 31, 2026 I recently sat (and passed the VMware HCI Master Specialist exam (5V0-21. * [ ![What’s In My Tech Bag? Everyday Carry for IT Consultants](https://jameskilby.co.uk/wp-content/uploads/2019/01/Picture1-768x498.png) ](https://jameskilby.co.uk/2019/01/whats-in-my-backpack/) [Consulting](https://jameskilby.co.uk/category/consulting/) ### [What’s In My Tech Bag? Everyday Carry for IT Consultants](https://jameskilby.co.uk/2019/01/whats-in-my-backpack/) By[James](https://jameskilby.co.uk) January 10, 2019 · Updated June 1, 2026 I have seen a few posts online recently about the tools and technology people use on a day to day basis. * [ ![Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) ](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal. --- # My First GitHub Pull Request: Contributing to Open Source Source: https://jameskilby.co.uk/2020/12/my-first-pull/ Date: 2020-12-22 Categories: Devops, Personal, Automation, Github, AWS, Ansible, Artificial Intelligence, VMware, Hosting, Cloudflare, Homelab ![](https://jameskilby.co.uk/wp-content/uploads/2020/12/175jvBleoQfAZJc3sgTSPQA.jpg) [Devops](https://jameskilby.co.uk/category/devops/) | [Personal](https://jameskilby.co.uk/category/personal/) # My First GitHub Pull Request: Contributing to Open Source By[James](https://jameskilby.co.uk) December 22, 2020 · Updated June 5, 2026 • 📖1 min read(207 words) This post is about my first GitHub pull request and contributing to open source. I was initially going to add in the contents of this post to one that I have been writing about my exploits with [HashiCorp Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) but I decided it probably warranted being separated out. While working with the following awesome project I noticed a couple of minor errors and improvements that I wanted to suggest. I have worked in IT for over 10 years now but the opportunity to make suggestions to someone else’s codebase hadn’t popped up so I was keen to rectify that. Although I had used Git/GitHub before it was typically for my own uses as a code storage location. I, therefore, needed to sanity check the process of a “pull request” so off to YouTube I went. ## Submitting My First GitHub Pull Request The above video seemed to clarify the process the best and with that knowledge, the request was submitted to the owners and luckily it was accepted. You can see that I am now listed as a contributor [here](https://github.com/rainpole/packer-vsphere/graphs/contributors). Although my contribution was tiny, now that I understand the process much better I am planning to contribute some more to what is a fantastic little project. ## 📚 Related Posts * [How I upgraded my blog as a Static Website with GitHub Actions and Cloudflare](https://jameskilby.co.uk/2025/10/how-i-deploy-my-blog-as-a-static-website-with-github-actions-and-cloudflare/) * [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) * [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) ## Similar Posts * [ ![Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/wp-content/uploads/2026/04/packer-github-actions-vsphere-pipeline-768x452.png) ](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) [Automation](https://jameskilby.co.uk/category/automation/) | [Github](https://jameskilby.co.uk/category/github/) ### [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) By[James](https://jameskilby.co.uk) April 30, 2026 · Updated June 5, 2026 A walk-through of an automated golden-image pipeline for vSphere: HashiCorp Packer builds six Ubuntu LTS templates (22.04, 24.04, 26.04 — server and desktop) driven by three GitHub Actions workflows covering PR validation, parallel template builds, and ISO management via a self-hosted runner. * [ ![AWS Solution Architect – Associate](https://jameskilby.co.uk/wp-content/uploads/2018/05/AmazonWebservices_Logo.svg_-768x307.png) ](https://jameskilby.co.uk/2019/12/aws-solution-architect-associate/) [AWS](https://jameskilby.co.uk/category/aws/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [AWS Solution Architect – Associate](https://jameskilby.co.uk/2019/12/aws-solution-architect-associate/) By[James](https://jameskilby.co.uk) December 16, 2019 · Updated May 31, 2026 I renewed my AWS Solution Architect certification. * [ ![Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/wp-content/uploads/2026/01/VMware-NVIDIA-logos_ee2f18dc-615d-4c9e-8f11-9c3c2ce2bf37-prv-768x432.png) ](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) ### [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) By[James](https://jameskilby.co.uk) February 9, 2026 · Updated July 11, 2026 Learn how to use Ansible to configure an Ubuntu VM for use with NVIDIA based AI workloads in vSphere * [ ![Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) ](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal. * [ ![Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2022/01/web-development/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) ### [Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/2022/01/web-development/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 A while ago I started messing with Cloudflare Workers. I have now moved this site permanently over to them. * [ ![Template Deployment with Packer](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) ](https://jameskilby.co.uk/2021/01/hashicorp-packer/) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. --- # How VMware Cloud on AWS Handles Host Failures Automatically Source: https://jameskilby.co.uk/2020/09/vmc-host-errors/ Date: 2020-09-15 Categories: VMware, VMware Cloud on AWS, Homelab, Storage, Artificial Intelligence, Automation, Ansible, Personal ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1.png) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) # How VMware Cloud on AWS Handles Host Failures Automatically By[James](https://jameskilby.co.uk) September 15, 2020 · Updated June 5, 2026 • 📖1 min read(217 words) Here is how VMware Cloud on AWS handles a host failure automatically, based on one I watched happen in production. When you run a large enough infrastructure, failure is inevitable. How you handle that can be a big differentiator. With [VMware Cloud on AWS](https://aws.amazon.com/vmware/), the hosts are monitored 24×7 by VMware/AWS Support all as part of the service. If you pay for X number of hosts you should have X. That includes during maintenance and failure operations. ## Watching a Host Failure Get Remediated I’m not sure lucky is the right word but I did witness a host failure with a customer I was working with. True to the marketing, it was picked up and automatically remediated. ![VMC Host Errors Screenshot](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screenshot-2020-08-27-at-16.56.37-1536x182-2-1024x121.png) Looking at the log extract above a new host was being provisioned the same minute the issue was identified. Obviously, this needed to boot and join the VMware/vSAN cluster before a full data evacuation takes place on the faulty host and finally, the host is removed. All of this was seamless to the customer. I noticed it as a few HA alarms tripped in the vCenter (these were cosmetic only). Just another reason why you should look at the VMware Cloud on AWS Service… For more on the hardware behind this, see my [VMware Cloud on AWS hosts deep-dive](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/). ## 📚 Related Posts * [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) * [Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1)](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) * [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) ## Similar Posts * [ ![Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/wp-content/uploads/2023/04/intel_optane_ssd_900p_series_aic_-_right_angle_575px.png) ](https://jameskilby.co.uk/2023/04/intel-optane/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Using Intel Optane NVMe in a VMware Homelab: Setup & Results](https://jameskilby.co.uk/2023/04/intel-optane/) By[James](https://jameskilby.co.uk) April 17, 2023 · Updated June 1, 2026 I have been a VMware vExpert for many years and it has brought me many benefits over the years. * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Architecture Overview (Part 1)](https://jameskilby.co.uk/2026/03/my-self-hosted-ai-stack-a-technical-deep-dive/) By[James](https://jameskilby.co.uk) March 27, 2026 · Updated May 31, 2026 A walkthrough of my self-hosted AI stack: Ollama, Open WebUI, ComfyUI, Whishper, n8n, Qdrant, SearxNG, and a full observability layer — all running on my own hardware with Docker Compose. * [ ![Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/wp-content/uploads/2026/01/Firefly_Gemini-Flash-768x417.png) ](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) By[James](https://jameskilby.co.uk) January 27, 2026 · Updated June 5, 2026 How to leverage Content Libraries to deploy into VMware Cloud on AWS faster. * [ ![vSphere Power Management Ansible Playbooks with Semaphore](https://jameskilby.co.uk/wp-content/uploads/2026/04/vsphere-power-management-ansible-768x403.png) ](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automating vSphere Power Management driven by Ansible and SemaphoreUI](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) By[James](https://jameskilby.co.uk) April 15, 2026 · Updated June 1, 2026 In this post I’ll walk through how I use vSphere Power Management driven by Ansible and SemaphoreUI to automatically reduce ESXi host electricity consumption — saving real money on my Octopus Agile tariff by toggling hosts between Low Power and Balanced policies. Introduction One of the larger costs of running my homelab is the electricity…. * [ ![VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/wp-content/uploads/2020/09/vmware_SP_HCI20.png) ](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) [Personal](https://jameskilby.co.uk/category/personal/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) By[James](https://jameskilby.co.uk) September 13, 2020 · Updated May 31, 2026 I recently sat (and passed the VMware HCI Master Specialist exam (5V0-21. * [ ![Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-12-14-at-21.45.23.png) ](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) By[James](https://jameskilby.co.uk) December 14, 2022 · Updated June 1, 2026 I run a reasonably extensive homelab that is of course built around the VMware ecosystem. --- # VMware Certified Master Specialist HCI 2020 Source: https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/ Date: 2020-09-13 Categories: Personal, VMware, Hosting, Cloudflare, VMware Cloud on AWS, Ansible, Automation ![](https://jameskilby.co.uk/wp-content/uploads/2020/09/vmware_SP_HCI20.png) [Personal](https://jameskilby.co.uk/category/personal/) | [VMware](https://jameskilby.co.uk/category/vmware/) # VMware Certified Master Specialist HCI 2020 By[James](https://jameskilby.co.uk) September 13, 2020 · Updated May 31, 2026 • 📖1 min read(136 words) I recently sat (and passed) the VMware HCI Master Specialist exam (5V0-21.20). I won’t go into any details of the contents but I will comment that I felt the questions were fair and that there wasn’t anything in it to trip you up. The required knowledge was certainly wider than the vSAN specialist exam. This was my third remote proctored exam and I must say the experience has improved. Partially that is down to Pearson Vue improving the process and partially down to knowing what to expect a bit better. Together with the vSAN exam that I passed earlier in the month, this entitles me to a [VMware Master Services Competency](https://www.vmware.com/education-services/certification/vmware-ms-hci.html). This is now the second that I hold and I must say I like the way VMware is delivering a thought out learning path. ## 📚 Related Posts * [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) * [VMware – Going out with a Bang!](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) * [New VMware Cloud on AWS Host: i7i.metal-24xl](https://jameskilby.co.uk/2026/04/new-vmc-host-i7i-metal-24xl/) ## Similar Posts * [ ![And now for something completely different](https://jameskilby.co.uk/wp-content/uploads/2018/10/fWbXybA7-768x193.png) ](https://jameskilby.co.uk/2018/10/and-now-for-something-completely-different/) [Personal](https://jameskilby.co.uk/category/personal/) ### [And now for something completely different](https://jameskilby.co.uk/2018/10/and-now-for-something-completely-different/) By[James](https://jameskilby.co.uk) October 16, 2018 · Updated July 11, 2026 I have worked for my current employer Zen Internet for 3. Over that time I have changed roles from what was originally a customer-focused role into a role with one of the core platform teams. * [ ![Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2022/01/web-development/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) ### [Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/2022/01/web-development/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 A while ago I started messing with Cloudflare Workers. I have now moved this site permanently over to them. * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/07/i3en/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [VMware Cloud on AWS i3en Host: Specs, Storage & Performance](https://jameskilby.co.uk/2020/07/i3en/) By[James](https://jameskilby.co.uk) July 2, 2020 · Updated June 5, 2026 VMware Cloud on AWS (VMC) has introduced a new host to its lineup: the “i3en”. This is based on the i3en. * [ ![Automated VCF 9 Offline Depot architecture diagram showing Traefik reverse proxy and Nginx file server stack](https://jameskilby.co.uk/wp-content/uploads/2026/04/offlinedepot.png) ](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) By[James](https://jameskilby.co.uk) April 10, 2026 · Updated July 11, 2026 One Bash script turns a fresh Ubuntu VM into a VCF 9 Offline Depot: Traefik, Nginx, basic auth, and Let’s Encrypt wildcard certs via Cloudflare DNS. * [ ![VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) By[James](https://jameskilby.co.uk) May 21, 2025 · Updated July 11, 2026 Quick reference guide to the available storage resources that you get in VMware Cloud on AWS * [ ![An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/wp-content/uploads/2025/02/Picture-1-e1768509620339-768x193.png) ](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [An in-depth look at VMware Cloud on AWS hosts](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/) By[James](https://jameskilby.co.uk) August 14, 2025 · Updated July 11, 2026 This is single page intended to collate every single feature of the current VMware Cloud on AWS hosts for easy comparison. --- # VMware Cloud on AWS i3en Host: Specs, Storage & Performance Source: https://jameskilby.co.uk/2020/07/i3en/ Date: 2020-07-02 Categories: VMware, VMware Cloud on AWS, Personal, Automation, Homelab, vSAN, Ansible, Artificial Intelligence, AWS ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1.png) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) # VMware Cloud on AWS i3en Host: Specs, Storage & Performance By[James](https://jameskilby.co.uk) July 2, 2020 · Updated June 5, 2026 • 📖2 min read(435 words) VMware Cloud on AWS (VMC) has introduced a new i3en host to its lineup. This is based on the [i3en.metal AWS instance](https://aws.amazon.com/ec2/instance-types/i3en/). ## i3en Host Specifications The specifications are certainly impressive, packing in 96 logical cores, 768GiB RAM, and approximately 45.84 TiB of NVMe raw storage capacity per host. It’s certainly a monster with a 266% uplift in CPU, a 50% increase in RAM and a whopping 440% increase in raw storage per host compared to the i3. Most of the engagements I have worked on so far have discovered that they are storage limited requiring extra hosts to handle the required storage footprint. With such a big uplift in storage capacity, hopefully this will trend towards filling up CPU, RAM & Storage at the same time. This is the panacea of Hyperconvergence. ## i3en Host Performance The other two noticeable changes are that the processor is based on a much later Intel family. It is now based on 3.1 GHz all-core turbo Intel® Xeon® Scalable (Skylake) processors. This is a much more modern processor than the Broadwells in the original i3. This brings several processor extension improvements including Intel AVX, Intel AVX2, Intel AVX-512 The other noticeable change is the networking uplift with 100Gb/s available to each host. Model| pCPU| Memory GiB| Networking Gbps| Storage TB| AWS Host Pricing (On-demand in US-East-2 Ohio) ---|---|---|---|---|--- i3.metal| 36*| 512| 25| 8×1.9| $5.491 i3en.metal| 96| 768| 100| 8×7.5| $11.933 *The i3.metal instance, when used with VMware Cloud on AWS has hyperthreading disabled. At present this host is only available in the newer SDDC versions (1.10v4 or later) and limited locations. It also looks like the i3 still has to be the node used in the first cluster within the SDDC (where the management components reside) and they aren’t supported in 2 node clusters. ~~At the time of writing pricing from VMware is not available however pricing is available for the hosts if they were bought directly from AWS. Assuming the VMware costs fall broadly in line with this giving:~~ VMware have now released pricing. The below is for On-Demand in the AWS US-East region. i3.Metal is £6.213287 per hour & i3en.Metal £13.6221 per hour giving: * A cost per GB of SSD instance storage that is up to 50% lower * Storage density (GB per vCPU) that is roughly 2.6x greater * Ratio of network bandwidth to vCPUs that is up to 2.7x greater This new host type adds complication to choosing host types within VMware Cloud on AWS but makes it a very compelling solution. See how it compares to other hosts in my [VMware Cloud on AWS hosts deep-dive](https://jameskilby.co.uk/2025/08/vmc-host-deepdive/). ## 📚 Related Posts * [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) * [VMware Cloud on AWS Storage Sizing Quick Reference Guide](https://jameskilby.co.uk/2025/05/vmc-quick-sizing-guide/) * [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) ## Similar Posts * [ ![Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) ](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal. * [ ![Template Deployment with Packer](https://jameskilby.co.uk/wp-content/uploads/2021/01/logo_packer.png) ](https://jameskilby.co.uk/2021/01/hashicorp-packer/) [Automation](https://jameskilby.co.uk/category/automation/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Template Deployment with Packer](https://jameskilby.co.uk/2021/01/hashicorp-packer/) By[James](https://jameskilby.co.uk) January 21, 2021 · Updated June 5, 2026 Packer is one of those tools I have heard about, and some of the cool people on Twitter that I follow have been using it for a while. * [ ![vSAN Cluster Shutdown – Orchestration](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2025/12/vsan-cluster-shutdown/) [VMware](https://jameskilby.co.uk/category/vmware/) | [vSAN](https://jameskilby.co.uk/category/vmware/vsan-vmware/) ### [vSAN Cluster Shutdown – Orchestration](https://jameskilby.co.uk/2025/12/vsan-cluster-shutdown/) By[James](https://jameskilby.co.uk) December 6, 2025 · Updated June 1, 2026 How to safely shut down a vSAN Environment * [ ![Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/wp-content/uploads/2026/01/VMware-NVIDIA-logos_ee2f18dc-615d-4c9e-8f11-9c3c2ce2bf37-prv-768x432.png) ](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) ### [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) By[James](https://jameskilby.co.uk) February 9, 2026 · Updated July 11, 2026 Learn how to use Ansible to configure an Ubuntu VM for use with NVIDIA based AI workloads in vSphere * [ ![Self-hosted AI stack operations architecture — Ansible automation, Uptime Kuma monitoring, Open WebUI backup, and container orchestration with Docker and Traefik](https://jameskilby.co.uk/wp-content/uploads/2026/03/ai-stack-featured-768x403.png) ](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [My Self-Hosted AI Stack: Infrastructure Deep Dive (Part 2)](https://jameskilby.co.uk/2026/04/my-self-hosted-ai-stack-infrastructure-deep-dive-part-2/) By[James](https://jameskilby.co.uk) April 4, 2026 · Updated July 11, 2026 Part 2 of my self-hosted AI stack series. I cover container resource sizing, dual-network isolation via Traefik and Cloudflare Tunnels, and every database powering the stack — PostgreSQL, ClickHouse, Redis, Qdrant, MinIO, MongoDB, SQLite, Prometheus, and Jaeger — plus the backup strategy for each. * [ ![Monitoring VMware Cloud on AWS: Tools & Approaches \(Part 1\)](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) ](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) [VMware](https://jameskilby.co.uk/category/vmware/) | [AWS](https://jameskilby.co.uk/category/aws/) ### [Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1)](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) By[James](https://jameskilby.co.uk) December 17, 2019 · Updated June 5, 2026 As previously mentioned I have been working a lot with VMware Cloud on AWS and one of the questions that often crops up is around an approach to monitoring. --- # Passing the Nutanix NCP Exam: Free Training & My Experience Source: https://jameskilby.co.uk/2020/07/nutanix-ncp/ Date: 2020-07-02 Categories: Nutanix, Personal, VMware, Hosting, Consulting ![](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier.jpg) [Nutanix](https://jameskilby.co.uk/category/nutanix/) | [Personal](https://jameskilby.co.uk/category/personal/) # Passing the Nutanix NCP Exam: Free Training & My Experience By[James](https://jameskilby.co.uk) July 2, 2020 · Updated May 31, 2026 • 📖1 min read(230 words) I saw a tweet a couple of weeks ago mentioning that Nutanix were offering a free go at the [Nutanix Certified Professional](https://www.nutanix.com/support-services/training-certification/certifications#ncp) exam. They are also offering free on-demand training to go with it. In my current role, I haven’t used Nutanix however I have good experience using it as the storage platform with vSphere and have also run the Nutanix CE software both physically and nested to give me a better overview of the AHV platform. The online training is delivered over a pretty modern platform and is done in a way to keep you engaged. The Labs aren’t super complicated but are buried in the training as opposed to a separate task. I think I prefer this approach. This was the first online proctored exam for me so the experience was a bit different. I initially had some issues getting the software to recognise my web camera. After a bit of troubleshooting, this was resolved and the exam began. The questions were fair and followed the blueprint and the time allocated was ample for me. I wasn’t confident when I pressed the submit button but the results came back instantly and luckily I passed. They also give you a breakdown of how you scored in the various sections. This is a really useful piece of feedback that several tests deprive you of. All in all, top work, Nutanix ## 📚 Related Posts * [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) * [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) * [AWS Solution Architect – Associate](https://jameskilby.co.uk/2019/12/aws-solution-architect-associate/) ## Similar Posts * [ ![VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/wp-content/uploads/2020/09/vmware_SP_HCI20.png) ](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) [Personal](https://jameskilby.co.uk/category/personal/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) By[James](https://jameskilby.co.uk) September 13, 2020 · Updated May 31, 2026 I recently sat (and passed the VMware HCI Master Specialist exam (5V0-21. * [ ![Analytics in a privacy focused world](https://jameskilby.co.uk/wp-content/uploads/2023/11/plausible-analytics-icon-top.png) ](https://jameskilby.co.uk/2023/11/analytics-in-a-privacy-focused-world/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Analytics in a privacy focused world](https://jameskilby.co.uk/2023/11/analytics-in-a-privacy-focused-world/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 I recently helped my friend Dean Lewis @veducate with some hosting issues. As part of the testing of this he kindly gave me a login to his WordPress instance. * [ ![VMware – Going out with a Bang!](https://jameskilby.co.uk/wp-content/uploads/2023/10/rnli-logo-768x384.png) ](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [VMware – Going out with a Bang!](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) By[James](https://jameskilby.co.uk) October 7, 2023 · Updated June 1, 2026 There is a lot of uncertainty with VMware at the moment. This is all due to the pending acquisition by Broadcom. * [ ![Nutanix Command Reference Guide](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2018/06/nutanix-command-reference-guide/) [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [Nutanix Command Reference Guide](https://jameskilby.co.uk/2018/06/nutanix-command-reference-guide/) By[James](https://jameskilby.co.uk) June 5, 2018 · Updated July 11, 2026 A list of useful Nutanix Commands * [ ![Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) ](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal. * [ ![My Home Office Setup & Upgrades](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_7017-scaled-1-768x576.jpeg) ](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) [Personal](https://jameskilby.co.uk/category/personal/) | [Consulting](https://jameskilby.co.uk/category/consulting/) ### [My Home Office Setup & Upgrades](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) By[James](https://jameskilby.co.uk) January 5, 2021 · Updated June 1, 2026 Given the year that was 2020 and at the time of writing a distinct improvement appears a long way off I decided it was time to up my homeworking game. --- # VeeamON 2020: Highlights From Veeam’s Virtual Conference Source: https://jameskilby.co.uk/2020/06/veeamon2020/ Date: 2020-06-18 Categories: Veeam, VMware, AWS, Homelab ![](https://jameskilby.co.uk/wp-content/uploads/2020/06/veeam-logo-new-large-1934042827.png) [Veeam](https://jameskilby.co.uk/category/veeam/) # VeeamON 2020: Highlights From Veeam’s Virtual Conference By[James](https://jameskilby.co.uk) June 18, 2020 · Updated June 1, 2026 • 📖1 min read(272 words) As everyone knows by now, the world has changed, possibly forever. Due to Covid-19, working from home has become the new normal. We are lucky in the IT world that this has been fairly straightforward for most of us. We are privileged in that it’s possible for us to continue indefinitely. Organisations still need to move forward, to progress and adapt into the new normal. In the words of Charles Darwin “ _It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is most adaptable to change.”_ With that most (if not all IT conferences have been postponed or gone online) Veeam’s annual conference VeeamON is no exception and now it’s here!! As a [Veeam Vanguard](https://www.veeam.com/vanguard.html) I was privileged to be given an early briefing on some of the announcements. These are summarised below but for all the details make sure you sign up and view some of the great sessions… If you haven’t managed to sign up you still can [here.](https://www.veeam.com/veeamon) ## Headline Announcements * ### Veeam Backup for Office 365 v5 – Microsoft Teams backup – Modern Authentication * ### Veeam Backup for AWS v2 – Snapshot Replication – Hybrid Cloud * ### Veeam Availability Orchestrator v3 – Fast recovery using NetApp Snapshots – DR Pack purchase options * ### Veeam Availability Suite v11 -Continuous Data Protection. -Object Storage Enhancements – Capacity Tier now supporting Google Cloud Object Storage -New Archive Tier- Supporting AWS S3 Glacier -Instant Recovery improvements – Instant NAS & Instant Database recovery. Last but not least, a feature that I have been asking about for over 3 years. Yes Veeam Backup Agent for **MAC** ## 📚 Related Posts * [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) * [Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1)](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) ## Similar Posts * [ ![Monitoring VMware Cloud on AWS: Tools & Approaches \(Part 1\)](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) ](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) [VMware](https://jameskilby.co.uk/category/vmware/) | [AWS](https://jameskilby.co.uk/category/aws/) ### [Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1)](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) By[James](https://jameskilby.co.uk) December 17, 2019 · Updated June 5, 2026 As previously mentioned I have been working a lot with VMware Cloud on AWS and one of the questions that often crops up is around an approach to monitoring. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Veeam](https://jameskilby.co.uk/category/veeam/) ### [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated June 1, 2026 My lab is always undergoing change. Partially as I want to try new things or new ways of doing things. --- # Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1) Source: https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/ Date: 2019-12-17 Categories: VMware, AWS, Veeam, Personal, VMware Cloud on AWS, Ansible, Artificial Intelligence, VCF, Homelab ![](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) [VMware](https://jameskilby.co.uk/category/vmware/) | [AWS](https://jameskilby.co.uk/category/aws/) | [Veeam](https://jameskilby.co.uk/category/veeam/) # Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1) By[James](https://jameskilby.co.uk) December 17, 2019 · Updated June 5, 2026 • 📖2 min read(491 words) This is the first part of my series on monitoring VMware Cloud on AWS. As previously mentioned I have been working a lot with VMware Cloud on AWS and one of the questions that often crops up is around an approach to monitoring. ## Monitoring VMware Cloud on AWS: My Approach This is an interesting topic as VMC is technically “as a service” therefore the monitoring approach is a bit different. Technically AWS and VMware’s SRE teams will be monitoring all of the infrastructure components, however, you still need to monitor your Virtual Machines. If it was me I would still want some monitoring of the Infrastructure and I see two different reasons why you would want to do this: Firstly I want to check that the VMware Cloud on AWS service is doing what I am paying for. Secondly, I still need to monitor my VMs to ensure they are all behaving properly, the added factor is that with a good real-time view of my workload, I can potentially optimise the number of VMC hosts in my fleet reducing the costs. With that in mind, I decided to look at a few options for connecting some monitoring tools to a VMC environment to see what worked and what didn’t. I am expecting some things could behave differently as you don’t have true root/admin access as you would usually do. All of the tests will be done with the cloudadmin@vmc.local account. This is the highest-level account that a service user has within VMC. The first product that I decided to test was [Veeam One](https://www.veeam.com/virtualization-management-one-solution.html). This made sense for a few reasons: Firstly I’m a [Veeam Vanguard](https://jameskilby.co.uk/about-me/) and am very familiar with the product. I also have access to the Beta versions of the v10 products as part of the Vanguard program. Secondly, it’s pretty easy to spin up a test server to kick the tyres and finally, the config is incredibly quick to implement. I could have easily added a VMC vCenter to my existing Veeam servers however I chose to deploy a new server just for this testing. Assuming you have network access between your Veeam One server and the VMC vCenter then adding to Veeam One is straightforward. If not, you will need to open up the relevant firewalls. Once done Veeam performs an inventory operation and returns all of the objects you would expect. This test was shortly after a VMC environment was created so it doesn’t yet have any workloads migrated to it. However, as you can see below it’s correctly reporting on the hosts and VM workloads. It is correctly reporting back that the hosts are running ESXi 6.9.1… I also ran a couple of test reports to check they functioned as expected. Everything seemed to work as I would expect. ![VMC Vms](https://jameskilby.co.uk/wp-content/uploads/2023/05/VMC-Vms-1024x296.png) ![Screenshot 2019 12 17 at 21.19.40](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2019-12-17-at-21.19.40-1024x500.png) ![Screenshot 2019 12 17 at 21.19.30](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2019-12-17-at-21.19.30-1024x472.png) In Part Two I am going to look at using Grafana, InfluxDB and Telegraf and seeing if this common open-source monitoring stack works with VMC. ## 📚 Related Posts * [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) * [Using Content Libraries in VMC to deploy software faster](https://jameskilby.co.uk/2026/01/using-content-libraries-in-vmc-to-deploy-software-faster/) * [Lab Update – Desired Workloads](https://jameskilby.co.uk/2022/01/lab-update-part-5-desired-workloads/) ## Similar Posts * [ ![Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) ](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal. * [ ![vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/wp-content/uploads/2023/11/OrigionalPoweredByvSAN-550x324-1.jpg) ](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [vSAN ESA in VMware Cloud on AWS: What Changed in VMC M24](https://jameskilby.co.uk/2023/11/vsan-esa-and-the-improvements-it-brings-to-vmc/) By[James](https://jameskilby.co.uk) November 17, 2023 · Updated June 1, 2026 An Overview of vSAN ESA in VMC * [ ![Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/wp-content/uploads/2026/01/VMware-NVIDIA-logos_ee2f18dc-615d-4c9e-8f11-9c3c2ce2bf37-prv-768x432.png) ](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) ### [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) By[James](https://jameskilby.co.uk) February 9, 2026 · Updated July 11, 2026 Learn how to use Ansible to configure an Ubuntu VM for use with NVIDIA based AI workloads in vSphere * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/09/vmc-host-errors/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) By[James](https://jameskilby.co.uk) September 15, 2020 · Updated June 5, 2026 Learn how host failures are handled within VMC * [ ![VMware Holodeck Multi-Host VCF: Lab Setup & Configuration](https://jameskilby.co.uk/wp-content/uploads/2023/12/Holodeck-Overview.png) ](https://jameskilby.co.uk/2024/01/multihost-holodeck-vcf/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VCF](https://jameskilby.co.uk/category/vmware/vcf/) ### [VMware Holodeck Multi-Host VCF: Lab Setup & Configuration](https://jameskilby.co.uk/2024/01/multihost-holodeck-vcf/) By[James](https://jameskilby.co.uk) January 17, 2024 · Updated June 1, 2026 How to Deploy VMware Holodeck on multiple hosts * [ ![Nvidia Tesla P4 vGPU Setup in VMware Homelab: Full Guide](https://jameskilby.co.uk/wp-content/uploads/2023/10/IMG_1107-768x403-1.jpg) ](https://jameskilby.co.uk/2023/10/vgpu-setup-in-my-homelab/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Nvidia Tesla P4 vGPU Setup in VMware Homelab: Full Guide](https://jameskilby.co.uk/2023/10/vgpu-setup-in-my-homelab/) By[James](https://jameskilby.co.uk) October 23, 2023 · Updated July 11, 2026 Card Stats Install steps VM Provisioning Folding@Home A little while ago I decided to play with vGPU in my homelab. --- # AWS Solution Architect – Associate Source: https://jameskilby.co.uk/2019/12/aws-solution-architect-associate/ Date: 2019-12-16 Categories: AWS, Personal, Nutanix, Consulting, VMware, VMware Cloud on AWS, Hosting ![](https://jameskilby.co.uk/wp-content/uploads/2018/05/AmazonWebservices_Logo.svg_.png) [AWS](https://jameskilby.co.uk/category/aws/) | [Personal](https://jameskilby.co.uk/category/personal/) # AWS Solution Architect – Associate By[James](https://jameskilby.co.uk) December 16, 2019 · Updated May 31, 2026 • 📖1 min read(150 words) Today was a good day. I renewed my AWS Solution Architect certification. Although my work is primarily in and around the VMware ecosystem I have been working a lot with [VMware Cloud on AWS](https://console.cloud.broadcom.com) recently with a number of our customers. Having a good foundation of the core AWS services has made this much easier to be able to articulate to the customers. Due to the non-disclosure agreement, I can’t talk about any specific questions however I will say that the questions and the focus have changed quite significantly since I sat the exam a few years ago. Like AWS itself, they were at the cutting edge of the industry and the number of services they now offer is extreme, adding to the challenge of the exam. I am now good for the next three years however I am planning to sit the Professional level exam in the new year. ## 📚 Related Posts * [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) * [Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/2020/07/nutanix-ncp/) * [Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1)](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) ## Similar Posts * [ ![Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2020/07/nutanix-ncp/) [Nutanix](https://jameskilby.co.uk/category/nutanix/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/2020/07/nutanix-ncp/) By[James](https://jameskilby.co.uk) July 2, 2020 · Updated May 31, 2026 I saw a tweet a couple of weeks ago mentioning that Nutanix were offering a free go at the Nutanix Certified Professional exam. * [ ![My Home Office Setup & Upgrades](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_7017-scaled-1-768x576.jpeg) ](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) [Personal](https://jameskilby.co.uk/category/personal/) | [Consulting](https://jameskilby.co.uk/category/consulting/) ### [My Home Office Setup & Upgrades](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) By[James](https://jameskilby.co.uk) January 5, 2021 · Updated June 1, 2026 Given the year that was 2020 and at the time of writing a distinct improvement appears a long way off I decided it was time to up my homeworking game. * [ ![Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) ](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal. * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/09/vmc-host-errors/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) By[James](https://jameskilby.co.uk) September 15, 2020 · Updated June 5, 2026 Learn how host failures are handled within VMC * [ ![AWS Status Page – Monitoring Included](https://jameskilby.co.uk/wp-content/uploads/2018/05/AmazonWebservices_Logo.svg_-768x307.png) ](https://jameskilby.co.uk/2018/05/aws-status-page-monitoring-included/) [AWS](https://jameskilby.co.uk/category/aws/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [AWS Status Page – Monitoring Included](https://jameskilby.co.uk/2018/05/aws-status-page-monitoring-included/) By[James](https://jameskilby.co.uk) May 15, 2018 · Updated June 1, 2026 AWS Status Page – Enhancements The tool I deployed, lambstatus, supports pulling metrics from AWS Cloudwatch and displaying them. * [ ![Monitoring VMware Cloud on AWS: Tools & Approaches \(Part 1\)](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) ](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) [VMware](https://jameskilby.co.uk/category/vmware/) | [AWS](https://jameskilby.co.uk/category/aws/) ### [Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1)](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) By[James](https://jameskilby.co.uk) December 17, 2019 · Updated June 5, 2026 As previously mentioned I have been working a lot with VMware Cloud on AWS and one of the questions that often crops up is around an approach to monitoring. --- # Homelab Storage Upgrade: Synology DS918 for VMware & NFS Source: https://jameskilby.co.uk/2019/02/lab-storage-2/ Date: 2019-02-10 Categories: Homelab, Docker, Hosting, Networking, Automation, Github, Storage, VCF, VMware ![](https://jameskilby.co.uk/wp-content/uploads/2023/04/81-ZoEW24UL._SL1500_.jpg) [Homelab](https://jameskilby.co.uk/category/homelab/) # Homelab Storage Upgrade: Synology DS918 for VMware & NFS By[James](https://jameskilby.co.uk) February 10, 2019 · Updated June 5, 2026 • 📖1 min read(224 words) ## Homelab Storage Upgrade This post covers my homelab storage upgrade. Since starting my new role with Xtravirt my Homelab has gone through several fairly significant changes. At the moment it’s very much focused on the VMware stack and one of the things I needed was some more storage and especially some more storage performance. With that in mind, I purchased a new [Synology DS918](https://www.synology.com)… ![Lab Storage](https://jameskilby.co.uk/wp-content/uploads/2023/04/iu-768x432-1.jpg) It’s a very compact unit with a quad-core Intel Celeron & I have left the RAM at 4 GB for now. I have added some of the existing SSDs I had, giving me about 3TB of usable flash. I am presenting this back to my VMware hosts using NFS 4.1. I must have missed the announcement as this is now built into the Synology GUI (it used to be a command-line-only option). I have verified the VAAI works as expected in this configuration. At present I am using this with a single network connection however I will be testing NFS Multipathing shortly. The performance improvement has been noticeable and I have now removed all non-Synology systems from primary storage. This has left me with the DS918+ detailed here and a DS216+ with 2TB of Raid1 WD Reds. I am using this for ISOs and some general file storage. I later expanded the lab further with [new homelab nodes](https://jameskilby.co.uk/2024/07/new-nodes/). ## 📚 Related Posts * [Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) * [Homelab SSD Failure: How Synology RAID Saved My Data](https://jameskilby.co.uk/2022/11/homelab-bad-days-almost/) * [My First Homelab Storage Setup: HP Gen8 & Xpenology](https://jameskilby.co.uk/2018/01/lab-storage/) ## Similar Posts * [ ![Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/wp-content/uploads/2022/12/22225832.png) ](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) [Docker](https://jameskilby.co.uk/category/docker/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) By[James](https://jameskilby.co.uk) December 9, 2022 · Updated June 5, 2026 Late to the party or not, I have been using containers in my lab more and more and that has led me to Portainer …. * [ ![Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/wp-content/uploads/2022/10/spacexs-starlink-to-supply-free-satellite-internet-to-famili_u44u.1920-768x432.jpg) ](https://jameskilby.co.uk/2022/10/starlink/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Starlink Satellite Internet Review: Rural Broadband Solution](https://jameskilby.co.uk/2022/10/starlink/) By[James](https://jameskilby.co.uk) October 11, 2022 · Updated June 1, 2026 Since moving to Dorset last year internet connectivity has been the bane of my existence. Currently, I have an ADSL connection provided by my old employer Zen and a 5G connection provided by Three. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Homelab Network Upgrade: DACs, 40Gb/s vMotion & pfSense](https://jameskilby.co.uk/2022/01/lab-update-part-3-network/) By[James](https://jameskilby.co.uk) January 6, 2022 · Updated May 31, 2026 I have retired the WatchGuard Devices with the migration to PFSense running bare-metal in one of the Supermicro Nodes. * [ ![Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/wp-content/uploads/2026/04/packer-github-actions-vsphere-pipeline-768x452.png) ](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) [Automation](https://jameskilby.co.uk/category/automation/) | [Github](https://jameskilby.co.uk/category/github/) ### [Automating vSphere Golden Images with Packer and GitHub Actions](https://jameskilby.co.uk/2026/04/packer-vsphere-golden-images/) By[James](https://jameskilby.co.uk) April 30, 2026 · Updated June 5, 2026 A walk-through of an automated golden-image pipeline for vSphere: HashiCorp Packer builds six Ubuntu LTS templates (22.04, 24.04, 26.04 — server and desktop) driven by three GitHub Actions workflows covering PR validation, parallel template builds, and ISO management via a self-hosted runner. * [ ![TrueNAS Logo](https://jameskilby.co.uk/wp-content/uploads/2023/05/Screenshot-2023-05-22-at-18.49.21-768x198.png) ](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Homelab Storage Refresh (Part 1)](https://jameskilby.co.uk/2023/05/homelab-storage-refresh-part-1/) By[James](https://jameskilby.co.uk) May 23, 2023 · Updated July 11, 2026 Table of Contents Background ZFS Overview Read Cache (ARC and L2ARC) ZIL (ZFS Intent Log) Hardware Background I have just completed the move of all my production and media-based storage/services to TrueNAS Scale. ( I will just refer to this as TrueNAS) This is based on my HP Z840 and I have now retired my… * [ ![VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/wp-content/uploads/2024/01/40oOd8IipPvtrPJs-1198788743-768x737.jpg) ](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) [VCF](https://jameskilby.co.uk/category/vmware/vcf/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Holodeck on Older CPUs: Fixing Compatibility Issues](https://jameskilby.co.uk/2024/01/holodeck-cpu-fixes/) By[James](https://jameskilby.co.uk) January 18, 2024 · Updated June 1, 2026 How to deploy Holodeck with Legacy CPU’s --- # What’s In My Tech Bag? Everyday Carry for IT Consultants Source: https://jameskilby.co.uk/2019/01/whats-in-my-backpack/ Date: 2019-01-10 Categories: Consulting, Personal ![](https://jameskilby.co.uk/wp-content/uploads/2019/01/Picture1.png) [Consulting](https://jameskilby.co.uk/category/consulting/) # What’s In My Tech Bag? Everyday Carry for IT Consultants By[James](https://jameskilby.co.uk) January 10, 2019 · Updated June 1, 2026 • 📖1 min read(251 words) I have seen a few posts online recently about the tools and technology people use on a day to day basis. A few components I have that no one seems to have mentioned yet. So I thought it was probably a good idea to share my list. I will probably do a separate list of the software I use at some point as well. Below is a list of the things that I tend to carry in my bag. I don’t always have all of them with me but I typically swap out what I need as appropriate depending on if I need to go to a datacenter or just a meeting/onsite client work. * MacBook Air 2018 (my go-to machine) * iPad Pro 9.7 & Apple Pencil allows for quick sketching but also used as a presentation remote. * [Anker Powercore 20100](https://amzn.to/2TGRgiq) ( Huge battery for keeping iPhone/iPad charged) * Beats Solo3 Wireless headphones. Equally useful in a noisy datacenter or a noisy office. * 2.5″ USB Drive caddy * A bunch of USB Sticks typically with a few different ESX builds 6.0, 6.5 and 6.7 and an empty one just in case * EasyAcc Wireless Storage PowerBank – Battery powered AP ( Insanely useful device) * Huawei 3G Mifi * Startech USB KVM (Another lifesaver) * Apple Lightning to HDMI Adaptor + HDMI cable * Leatherman Wingman * LED head torch * Dymo portable label printer * Trusty vendor supplied notepad and pens. Based on the recommendation of [Chris Lewis](https://x.com/thecloudxpert) I have also added some [Whiteboard](https://amzn.to/2CbHTQW) & [Flipchart](https://amzn.to/2CbHTQW) Pens Oh and a Tie…… ## 📚 Related Posts * [My Home Office Setup & Upgrades](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) ## Similar Posts * [ ![My Home Office Setup & Upgrades](https://jameskilby.co.uk/wp-content/uploads/2023/05/IMG_7017-scaled-1-768x576.jpeg) ](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) [Personal](https://jameskilby.co.uk/category/personal/) | [Consulting](https://jameskilby.co.uk/category/consulting/) ### [My Home Office Setup & Upgrades](https://jameskilby.co.uk/2021/01/my-home-office-setup-upgrades/) By[James](https://jameskilby.co.uk) January 5, 2021 · Updated June 1, 2026 Given the year that was 2020 and at the time of writing a distinct improvement appears a long way off I decided it was time to up my homeworking game. --- # MacBook Air 2018 Retina Review: My Setup & Build Scripts Source: https://jameskilby.co.uk/2018/12/new-laptop/ Date: 2018-12-04 Categories: Apple ![](https://jameskilby.co.uk/wp-content/uploads/2018/12/colorware.jpg) [Apple](https://jameskilby.co.uk/category/apple/) # MacBook Air 2018 Retina Review: My Setup & Build Scripts By[James](https://jameskilby.co.uk) December 4, 2018 · Updated June 1, 2026 • 📖1 min read(138 words) I decided it was about time I replaced my trusted MacBook Air that I purchased back in 2011. After waiting and watching the Apple announcements over the last couple of years I decided the MacBook Pro’s weren’t worth it. So I have replaced my Air with yes you guessed it another [MacBook Air](https://www.apple.com/uk/macbook-air/). This time I went for the retina version with 16GB of RAM and 512GB PCIe-based SSD. The spec was certainly good enough for 99% of my needs yet still being lighter and with better battery life than a MacBook Pro. It also has the security benefits of the [T2 Security chip](https://support.apple.com/en-gb/HT208862) I also thought it was time to tidy up and publish my Mac build scripts for anyone else to use. They are available below and hopefully pretty easy for anyone to follow. ## 📚 Related Posts * [How Apple Content Caching Works: Speed Up iOS & Mac Updates](https://jameskilby.co.uk/2021/02/apple-content-caching/) ## Similar Posts * [ ![How Apple Content Caching Works: Speed Up iOS & Mac Updates](https://jameskilby.co.uk/wp-content/uploads/2021/02/iu-768x403.png) ](https://jameskilby.co.uk/2021/02/apple-content-caching/) [Apple](https://jameskilby.co.uk/category/apple/) ### [How Apple Content Caching Works: Speed Up iOS & Mac Updates](https://jameskilby.co.uk/2021/02/apple-content-caching/) By[James](https://jameskilby.co.uk) February 8, 2021 · Updated July 11, 2026 I have slowly morphed into an Apple fanboy over the last decade or so collecting a large number of devices ever since my first MacBook Air back in 2011. --- # And now for something completely different Source: https://jameskilby.co.uk/2018/10/and-now-for-something-completely-different/ Date: 2018-10-16 Categories: Personal, VMware, Devops, Hosting, Cloudflare, Nutanix ![](https://jameskilby.co.uk/wp-content/uploads/2018/10/fWbXybA7.png) [Personal](https://jameskilby.co.uk/category/personal/) # And now for something completely different By[James](https://jameskilby.co.uk) October 16, 2018 · Updated July 11, 2026 • 📖1 min read(263 words) I have worked for my current employer [Zen Internet](https://www.zen.co.uk/) for 3.5 years. Over that time I have changed roles from what was originally a customer-focused role into a role with one of the core platform teams. This has meant looking after the majority of the Internal and customer Virtual platforms. During this time Zen has undergone several large migration programs and countless smaller ones. Some of the big ones include: * [Core ISP network refresh](https://www.zen.co.uk/blog/posts/zen-blog/2018/02/09/how-plexus-came-about) * [Microsegmentation rollout](https://www.vmware.com/uk/products/nsx.html) * [Nutanix integration](https://www.zen.co.uk/large-business/products-solutions/cloud-hosting) * Backup Consolidation I have been lucky enough to be involved with all of these ( some much more than others). Although the work is never complete Zen are in a good place. At the same time, my personal circumstances have changed somewhat. My upcoming wedding and my fiancé not knowing exactly where she will be based meant having the freedom to be located anywhere (within reason) was a huge win for me. I was also looking for a role challenging me using some of the toolsets that I love but also being on the cutting edge of the various stacks. And with that – I am proud to say I am joining one of the big hitters, awarded both the EMEA and Global VMware Partner Innovation Award for professional services in 2018. Yes, I’m joining [Xtravirt](https://xtravirt.com/) as a Senior Consultant. This should allow me to utilise and improve my core VMware knowledge but also leverage some of my other skills including Veeam, Nutanix and AWS. Something I’m looking forward to. I also get to work with Chris Lewis, Simon Eady and one or two others… ## 📚 Related Posts * [Analytics in a privacy focused world](https://jameskilby.co.uk/2023/11/analytics-in-a-privacy-focused-world/) * [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) * [VMware – Going out with a Bang!](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) ## Similar Posts * [ ![Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/wp-content/uploads/2023/11/image.png) ](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Advanced Deploy VMware vSphere 7.x 3V0-22.21N](https://jameskilby.co.uk/2023/11/advanced-deploy-vmware-vsphere-7-x-3v0-22-21n/) By[James](https://jameskilby.co.uk) November 10, 2023 · Updated June 1, 2026 Yesterday I sat and passed the above exam. It had been on my todo list for a good number of years. With the current pause in the Broadcom VMware takeover deal. * [ ![VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/wp-content/uploads/2020/09/vmware_SP_HCI20.png) ](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) [Personal](https://jameskilby.co.uk/category/personal/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [VMware Certified Master Specialist HCI 2020](https://jameskilby.co.uk/2020/09/vmware-certified-master-specialist-hci-2020/) By[James](https://jameskilby.co.uk) September 13, 2020 · Updated May 31, 2026 I recently sat (and passed the VMware HCI Master Specialist exam (5V0-21. * [ ![My First GitHub Pull Request: Contributing to Open Source](https://jameskilby.co.uk/wp-content/uploads/2020/12/175jvBleoQfAZJc3sgTSPQA.jpg) ](https://jameskilby.co.uk/2020/12/my-first-pull/) [Devops](https://jameskilby.co.uk/category/devops/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [My First GitHub Pull Request: Contributing to Open Source](https://jameskilby.co.uk/2020/12/my-first-pull/) By[James](https://jameskilby.co.uk) December 22, 2020 · Updated June 5, 2026 I was initially going to add in the contents of this post to one that I have been writing about my exploits with HashiCorp Packer but I decided it probably warranted being separated out. * [ ![Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2022/01/web-development/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) ### [Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/2022/01/web-development/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 A while ago I started messing with Cloudflare Workers. I have now moved this site permanently over to them. * [ ![Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2020/07/nutanix-ncp/) [Nutanix](https://jameskilby.co.uk/category/nutanix/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/2020/07/nutanix-ncp/) By[James](https://jameskilby.co.uk) July 2, 2020 · Updated May 31, 2026 I saw a tweet a couple of weeks ago mentioning that Nutanix were offering a free go at the Nutanix Certified Professional exam. * [ ![VMware – Going out with a Bang!](https://jameskilby.co.uk/wp-content/uploads/2023/10/rnli-logo-768x384.png) ](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) [VMware](https://jameskilby.co.uk/category/vmware/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [VMware – Going out with a Bang!](https://jameskilby.co.uk/2023/10/going-out-with-a-bang/) By[James](https://jameskilby.co.uk) October 7, 2023 · Updated June 1, 2026 There is a lot of uncertainty with VMware at the moment. This is all due to the pending acquisition by Broadcom. --- # Nutanix Command Reference Guide Source: https://jameskilby.co.uk/2018/06/nutanix-command-reference-guide/ Date: 2018-06-05 Categories: Nutanix, Homelab, Personal ![](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier.jpg) [Nutanix](https://jameskilby.co.uk/category/nutanix/) # Nutanix Command Reference Guide By[James](https://jameskilby.co.uk) June 5, 2018 · Updated July 11, 2026 • 📖1 min read(106 words) ## Nutanix Command List This is a list of Nutanix commands I have found useful. It’s here as a reference and if I need a command more than a few times I’ll generally add it here. #### CLI * ncli cluster get-domain-fault-tolerance-status type=node (Checks if all of the storage components meet the desired replication factor) * cvm_shutdown -P now ( Correct way to shut down a CVM) * ncc health_checks run_all –parallel=4 . ( 4 is the max number) * curator_cli get_under_replication_info summary=true Checks if any objects are not at the desired replication factor * curl localhost:2019/prism/leader . Find the leader #### WEB * http://{curator-master-cvm-ip}:2010/master/control ( If you want to invoke a curator scan manually) ## 📚 Related Posts * [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) * [Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/2020/07/nutanix-ncp/) * [Nutanix Life Cycle Manager](https://jameskilby.co.uk/2018/04/nutanix-life-cycle-manager/) ## Similar Posts * [ ![Running Nutanix CE at Home: AHV Setup & First Impressions](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2018/01/nutanix-ce/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [Running Nutanix CE at Home: AHV Setup & First Impressions](https://jameskilby.co.uk/2018/01/nutanix-ce/) By[James](https://jameskilby.co.uk) January 6, 2018 · Updated June 1, 2026 I ran a Nutanix CE server at home for a little while when it first came out. However, due to the fairly high requirements, it didn’t make sense to me to continue running it at home. * [ ![New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/wp-content/uploads/2024/07/IMG_6629-768x149.jpeg) ](https://jameskilby.co.uk/2024/07/new-nodes/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) By[James](https://jameskilby.co.uk) July 2, 2024 · Updated July 11, 2026 I recently decided to update some of my homelab hosts and I managed to do this at very little cost by offloading 2 of my Supermicro e200’s to fellow vExpert Paul . * [ ![Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2020/07/nutanix-ncp/) [Nutanix](https://jameskilby.co.uk/category/nutanix/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/2020/07/nutanix-ncp/) By[James](https://jameskilby.co.uk) July 2, 2020 · Updated May 31, 2026 I saw a tweet a couple of weeks ago mentioning that Nutanix were offering a free go at the Nutanix Certified Professional exam. * [ ![Nutanix Life Cycle Manager](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2018/04/nutanix-life-cycle-manager/) [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [Nutanix Life Cycle Manager](https://jameskilby.co.uk/2018/04/nutanix-life-cycle-manager/) By[James](https://jameskilby.co.uk) April 3, 2018 · Updated June 5, 2026 What Is Nutanix Life Cycle Manager (LCM)? With the introduction of AOS 5, Nutanix introduced Life Cycle Manager (LCM), something that Is one of the best but least known Nutanix features. Put simply it’s part of the Nutanix update mechanism but for dealing with hardware rather than the software components. To me what makes LCM… --- # AWS Status Page – Monitoring Included Source: https://jameskilby.co.uk/2018/05/aws-status-page-monitoring-included/ Date: 2018-05-15 Categories: AWS, Hosting, Cloudflare, VMware, VMware Cloud on AWS, Artificial Intelligence, Docker ![](https://jameskilby.co.uk/wp-content/uploads/2018/05/AmazonWebservices_Logo.svg_.png) [AWS](https://jameskilby.co.uk/category/aws/) | [Hosting](https://jameskilby.co.uk/category/hosting/) # AWS Status Page – Monitoring Included By[James](https://jameskilby.co.uk) May 15, 2018 · Updated June 1, 2026 • 📖1 min read(105 words) ## **AWS Status Page – Enhancements** The tool I deployed [lambstatus](https://lambstatus.github.io/get-started/) supports pulling metrics from AWS Cloudwatch and displaying them. As part of my personal development, I thought I would include this on my status page. I managed to get this working as can be seen [here. ](https://d1wijjwc03zzo1.cloudfront.net)This is a lambda function running once a minute polling this website and adding the response time into AWS Cloudwatch which Lambstatus is allowed to call. It has been running without a hitch for nearly a month now at **effectively zero cost** #### Site Response ![Screen Shot 2018 05 15 at 22.39.01](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screen-Shot-2018-05-15-at-22.39.01.png) The guide I followed is very good and is documented in the Git repo [here](https://lambstatus.github.io/set-up-custom-http-s-monitoring/) ## 📚 Related Posts * [Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1)](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) * [AWS Solution Architect – Associate](https://jameskilby.co.uk/2019/12/aws-solution-architect-associate/) * [AWS For Beginners: IAM Setup, Root Security & Billing Alerts](https://jameskilby.co.uk/2018/03/aws-for-beginners1/) ## Similar Posts * [ ![Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2022/01/web-development/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) ### [Hosting This Blog on Cloudflare Workers: Why & How I Did It](https://jameskilby.co.uk/2022/01/web-development/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 A while ago I started messing with Cloudflare Workers. I have now moved this site permanently over to them. * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/09/vmc-host-errors/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) By[James](https://jameskilby.co.uk) September 15, 2020 · Updated June 5, 2026 Learn how host failures are handled within VMC * [ ![AWS For Beginners: IAM Setup, Root Security & Billing Alerts](https://jameskilby.co.uk/wp-content/uploads/2018/03/raf750x1000075t101010_01c5ca27c6.u2.jpg) ](https://jameskilby.co.uk/2018/03/aws-for-beginners1/) [AWS](https://jameskilby.co.uk/category/aws/) ### [AWS For Beginners: IAM Setup, Root Security & Billing Alerts](https://jameskilby.co.uk/2018/03/aws-for-beginners1/) By[James](https://jameskilby.co.uk) March 30, 2018 · Updated June 1, 2026 AWS For Beginners Account Guide * [ ![How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/wp-content/uploads/2025/01/Screenshot-2025-01-29-at-23.30.47-768x411.png) ](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) ### [How I Migrated from Pocket to Hoarder with AI Integration](https://jameskilby.co.uk/2025/01/how-i-migrated-from-pocket-to-hoarder-and-introduced-some-ai-along-the-way/) By[James](https://jameskilby.co.uk) January 29, 2025 · Updated July 11, 2026 Update: Hoarder has now been renamed to Karakeep due to a trademark issue I’ve been on a mission recently to regain control of my data. * [ ![What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2-768x229.png) ](https://jameskilby.co.uk/2018/03/cloudflare/) [Hosting](https://jameskilby.co.uk/category/hosting/) ### [What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained](https://jameskilby.co.uk/2018/03/cloudflare/) By[James](https://jameskilby.co.uk) March 27, 2018 · Updated June 1, 2026 Cloudflare – What is it and why would I care? I have been using Cloudflare for a long time. * [ ![Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/wp-content/uploads/2026/01/Website-Optimisations-768x560.png) ](https://jameskilby.co.uk/2026/01/web-development-improvements/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) By[James](https://jameskilby.co.uk) January 15, 2026 · Updated July 11, 2026 I have spent the Christmas break making some improvements to this blog. --- # Nutanix Life Cycle Manager Source: https://jameskilby.co.uk/2018/04/nutanix-life-cycle-manager/ Date: 2018-04-03 Categories: Nutanix, Homelab, Personal ![](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier.jpg) [Nutanix](https://jameskilby.co.uk/category/nutanix/) # Nutanix Life Cycle Manager By[James](https://jameskilby.co.uk) April 3, 2018 · Updated June 5, 2026 • 📖2 min read(432 words) ## What Is Nutanix Life Cycle Manager (LCM)? With the introduction of AOS 5, Nutanix introduced [Life Cycle Manager (LCM)](https://www.nutanix.com/products/cloud-infrastructure), something that Is one of the best but least known Nutanix features. Put simply it’s part of the Nutanix update mechanism but for dealing with hardware rather than the software components. To me what makes LCM stand out is its pure simplicity. I have seen other solutions where it can be confusing to find out what hardware is on the HCL and then what firmware version is required and then the appropriate driver for that combination. This becomes unmanageable at a large enough scale. Where I currently work we have a mix of five different Nutanix node types of different hardware generations. ( All based on Supermicro hardware) . The below screenshots walk through an upgrade of one of these clusters. ## Walking Through an LCM Upgrade ![Nutanix Life Cycle Manager dashboard showing an available Cluster Software Component update](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screen-Shot-2018-02-14-at-19.38.25-1024x238.png) The above picture demonstrates the simplicity, It’s showing that at present the only update available is to the “Cluster Software Component” Once this has been updated the next step is to perform an Inventory of your cluster. LCM will then show you all of the components in your cluster and the relevant upgrades available. If you work in a “Dark Site” . Offline downloads are also available. The below cluster has not had any updates run against it. ![Nutanix Life Cycle Manager inventory screen before any updates have been run](https://jameskilby.co.uk/wp-content/uploads/2023/04/Screen-Shot-2018-02-14-at-19.38.36-2-1024x486.png) Once the inventory has been done it’s time to decide if you want to run all of the updates or just a selection and off you go. ![Nutanix Life Cycle Manager listing available firmware updates on a three-node cluster](https://jameskilby.co.uk/wp-content/uploads/2018/04/DeviceToUpdate.png) You can see above all of the available updates in this 3-node cluster. Note that only 2 of the SSDs needed updates as we had previously had one replaced and this was shipped with a later firmware. Because the LCM is aware of the end-to-end stack it’s aware of any relevant dependencies. The upgrade for the HBA listed below doesn’t have any. ![](https://jameskilby.co.uk/wp-content/uploads/2018/03/Screen-Shot-2018-03-10-at-20.41.50-300x156.png) ## How LCM Handles the Upgrade Orchestration Once you have started the upgrade progress LCM handles the orchestration piece, stopping just the required services and functions to allow the upgrade to complete. ![](https://jameskilby.co.uk/wp-content/uploads/2018/03/Screen-Shot-2018-02-14-at-22.25.32-300x160.png) For the HBA upgrade, LCM stopped the storage-related services on the CVM but it left the CVM powered on and was not required to evacuate VM’s from the ESXi host. This meant that the upgrade was done very quickly and the storage services started again before moving to the next node. As you can see the Host Boot Device (SATADom) and drives do require maintenance mode but again all of this is handled by LCM And that’s it…. For more, check out my [Nutanix Command Reference Guide](https://jameskilby.co.uk/nutanix-command-reference-guide/). ## 📚 Related Posts * [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) * [Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/2020/07/nutanix-ncp/) * [Nutanix Command Reference Guide](https://jameskilby.co.uk/2018/06/nutanix-command-reference-guide/) ## Similar Posts * [ ![New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/wp-content/uploads/2024/07/IMG_6629-768x149.jpeg) ](https://jameskilby.co.uk/2024/07/new-nodes/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) By[James](https://jameskilby.co.uk) July 2, 2024 · Updated July 11, 2026 I recently decided to update some of my homelab hosts and I managed to do this at very little cost by offloading 2 of my Supermicro e200’s to fellow vExpert Paul . * [ ![Nutanix Command Reference Guide](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2018/06/nutanix-command-reference-guide/) [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [Nutanix Command Reference Guide](https://jameskilby.co.uk/2018/06/nutanix-command-reference-guide/) By[James](https://jameskilby.co.uk) June 5, 2018 · Updated July 11, 2026 A list of useful Nutanix Commands * [ ![Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2020/07/nutanix-ncp/) [Nutanix](https://jameskilby.co.uk/category/nutanix/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/2020/07/nutanix-ncp/) By[James](https://jameskilby.co.uk) July 2, 2020 · Updated May 31, 2026 I saw a tweet a couple of weeks ago mentioning that Nutanix were offering a free go at the Nutanix Certified Professional exam. * [ ![Running Nutanix CE at Home: AHV Setup & First Impressions](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier-768x196.jpg) ](https://jameskilby.co.uk/2018/01/nutanix-ce/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) ### [Running Nutanix CE at Home: AHV Setup & First Impressions](https://jameskilby.co.uk/2018/01/nutanix-ce/) By[James](https://jameskilby.co.uk) January 6, 2018 · Updated June 1, 2026 I ran a Nutanix CE server at home for a little while when it first came out. However, due to the fairly high requirements, it didn’t make sense to me to continue running it at home. --- # AWS For Beginners: IAM Setup, Root Security & Billing Alerts Source: https://jameskilby.co.uk/2018/03/aws-for-beginners1/ Date: 2018-03-30 Categories: AWS, VMware, VMware Cloud on AWS, Hosting, Personal ![](https://jameskilby.co.uk/wp-content/uploads/2018/03/raf750x1000075t101010_01c5ca27c6.u2.jpg) [AWS](https://jameskilby.co.uk/category/aws/) # AWS For Beginners: IAM Setup, Root Security & Billing Alerts By[James](https://jameskilby.co.uk) March 30, 2018 · Updated June 1, 2026 • 📖1 min read(202 words) ![AWS Services](https://jameskilby.co.uk/wp-content/uploads/2023/04/AWS_Services.gif) ## AWS for Beginners Part 1 I am hoping to get back into doing some AWS stuff over the next couple of months. I am a huge fan of some of the tools and technology they have built. It’s not perfect and it is often not well understood by people lifting and shifting existing infrastructure into “the cloud”…. My view is firmly if this is what you have done, then you have done it wrong and missed the point… But if you do want to kick the tyres and see what it’s all about go and set up an [account and play ](https://portal.aws.amazon.com/billing/signup?nc2=h_ct&redirect_url=https%3A%2F%2Faws.amazon.com%2Fregistration-confirmation#/start) #### Ensure you implement these three recommendations * Root Account Security – Ensure you have a strong password on this account then stop using it. Use IAM to set up another account and then use that. * Enable Two Factor Authentication – This is a must. Sadly, AWS doesn’t appear to support my preferred 2FA device. [Yubikey](https://www.yubico.com/) so I’m using the Google authenticator * Enable [Billing Alerts](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/monitor_estimated_charges_with_cloudwatch.html#turning_on_billing_metrics) Create an alert so that if your bill is over X you will get a notification. It’s very easy to leave an Instance or several running somewhere. No one wants billshock. Update: AWS now does support Yubikey. Wahoo. ## 📚 Related Posts * [Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1)](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) * [AWS Solution Architect – Associate](https://jameskilby.co.uk/2019/12/aws-solution-architect-associate/) * [AWS Status Page – Monitoring Included](https://jameskilby.co.uk/2018/05/aws-status-page-monitoring-included/) ## Similar Posts * [ ![VMware Cloud on AWS \(VMC\) resource hub](https://jameskilby.co.uk/wp-content/uploads/2022/11/iu-1-768x395.png) ](https://jameskilby.co.uk/2020/09/vmc-host-errors/) [VMware](https://jameskilby.co.uk/category/vmware/) | [VMware Cloud on AWS](https://jameskilby.co.uk/category/vmware/vmware-cloud-on-aws/) ### [How VMware Cloud on AWS Handles Host Failures Automatically](https://jameskilby.co.uk/2020/09/vmc-host-errors/) By[James](https://jameskilby.co.uk) September 15, 2020 · Updated June 5, 2026 Learn how host failures are handled within VMC * [ ![Monitoring VMware Cloud on AWS: Tools & Approaches \(Part 1\)](https://jameskilby.co.uk/wp-content/uploads/2026/03/VMConAWS.png.webp) ](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) [VMware](https://jameskilby.co.uk/category/vmware/) | [AWS](https://jameskilby.co.uk/category/aws/) ### [Monitoring VMware Cloud on AWS: Tools & Approaches (Part 1)](https://jameskilby.co.uk/2019/12/monitoring-vmc-part-1/) By[James](https://jameskilby.co.uk) December 17, 2019 · Updated June 5, 2026 As previously mentioned I have been working a lot with VMware Cloud on AWS and one of the questions that often crops up is around an approach to monitoring. * [ ![AWS Status Page – Monitoring Included](https://jameskilby.co.uk/wp-content/uploads/2018/05/AmazonWebservices_Logo.svg_-768x307.png) ](https://jameskilby.co.uk/2018/05/aws-status-page-monitoring-included/) [AWS](https://jameskilby.co.uk/category/aws/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [AWS Status Page – Monitoring Included](https://jameskilby.co.uk/2018/05/aws-status-page-monitoring-included/) By[James](https://jameskilby.co.uk) May 15, 2018 · Updated June 1, 2026 AWS Status Page – Enhancements The tool I deployed, lambstatus, supports pulling metrics from AWS Cloudwatch and displaying them. * [ ![AWS Solution Architect – Associate](https://jameskilby.co.uk/wp-content/uploads/2018/05/AmazonWebservices_Logo.svg_-768x307.png) ](https://jameskilby.co.uk/2019/12/aws-solution-architect-associate/) [AWS](https://jameskilby.co.uk/category/aws/) | [Personal](https://jameskilby.co.uk/category/personal/) ### [AWS Solution Architect – Associate](https://jameskilby.co.uk/2019/12/aws-solution-architect-associate/) By[James](https://jameskilby.co.uk) December 16, 2019 · Updated May 31, 2026 I renewed my AWS Solution Architect certification. --- # What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained Source: https://jameskilby.co.uk/2018/03/cloudflare/ Date: 2018-03-27 Categories: Hosting, Cloudflare, Docker, Homelab, Wordpress, Ansible, Automation ![](https://jameskilby.co.uk/wp-content/uploads/2020/06/iu-2.png) [Hosting](https://jameskilby.co.uk/category/hosting/) # What Is Cloudflare? Free CDN, WAF & DDoS Protection Explained By[James](https://jameskilby.co.uk) March 27, 2018 · Updated June 1, 2026 • 📖1 min read(287 words) ## Cloudflare – What is it and why would I care? I have been using Cloudflare for a long time. It is one of my go-to services and I use it to protect all of the public services I run for myself and other sites/organisations. The basic premise of what Cloudflare do is that they are a distributed Web Application Firewall (WAF) and CDN but they also offer so much more. Because they have a large number of POPs they can cache and push content closer to your end users to give them a better experience. This also offloads work from your firewalls, DNS, Web and Database servers. At present they have [139 sites](https://www.cloudflare.com/network/) globally allowing you to host a site anywhere and get good global performance. Cloudflare has an amazing free tier so you can get started easily. I use this to host all of my public DNS records. So what does that look like? [codeblocks name=’host -a’] As you can see they have given me 2 Nameserver records (Matt and Fay) and also 2 A records. Neither of which are my source web server. Then within the Cloudflare portal, you input where the real webserver lives. Cloudflare will do the rest. My blog is still a little bit light on readership but you can see Cloudflare handling the spike in requests. They are also handling the SSL cert for me and the redirect so all traffic is HTTPS to my site allowing me to close port 80 on the firewall all within the free tier. If you are using WordPress I would strongly recommend the Cloudflare plugin is installed. This way when you make changes to your site it will automatically purge the cache if required…. ## 📚 Related Posts * [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) * [Analytics in a privacy focused world](https://jameskilby.co.uk/2023/11/analytics-in-a-privacy-focused-world/) * [WordPress Hosting with Cloudflare Pages](https://jameskilby.co.uk/2023/05/how-to-take-a-wordpress-site-and-publish-it-as-a-static-site-on-cloudflare-pages/) ## Similar Posts * [ ![Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu-768x450.jpeg) ](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Static WordPress hosting using Cloudflare](https://jameskilby.co.uk/2022/10/how-i-moved-my-wordpress-site-to-cloudflare-pages/) By[James](https://jameskilby.co.uk) October 20, 2022 · Updated July 11, 2026 For a while now I have been running this site directly from Cloudflare utilising their excellent worker’s product. * [ ![Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/wp-content/uploads/2022/12/22225832.png) ](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) [Docker](https://jameskilby.co.uk/category/docker/) | [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Use Portainer in a Homelab with GitHub](https://jameskilby.co.uk/2022/12/use-portainer-in-a-homelab-with-github/) By[James](https://jameskilby.co.uk) December 9, 2022 · Updated June 5, 2026 Late to the party or not, I have been using containers in my lab more and more and that has led me to Portainer …. * [ ![Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/wp-content/uploads/2022/10/iu-768x450.jpeg) ](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) [Hosting](https://jameskilby.co.uk/category/hosting/) | [Wordpress](https://jameskilby.co.uk/category/wordpress/) ### [Cloudflare Workers – Limits of the free tier](https://jameskilby.co.uk/2022/01/cloudflare-workers-limits-of-the-free-tier/) By[James](https://jameskilby.co.uk) January 4, 2022 · Updated July 11, 2026 I have been making several changes (mainly cosmetic) to this site over the last day or so. On most changes I have been doing an export and then uploading the site to Cloudflare using Wrangler. * [ ![Automated VCF 9 Offline Depot architecture diagram showing Traefik reverse proxy and Nginx file server stack](https://jameskilby.co.uk/wp-content/uploads/2026/04/offlinedepot.png) ](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automated VCF 9 Offline Depot](https://jameskilby.co.uk/2026/04/automated-vcf-9-offline-depot/) By[James](https://jameskilby.co.uk) April 10, 2026 · Updated July 11, 2026 One Bash script turns a fresh Ubuntu VM into a VCF 9 Offline Depot: Traefik, Nginx, basic auth, and Let’s Encrypt wildcard certs via Cloudflare DNS. * [ ![WordPress Hosting with Cloudflare Pages](https://jameskilby.co.uk/wp-content/uploads/2023/05/simply-static-logo.png) ](https://jameskilby.co.uk/2023/05/how-to-take-a-wordpress-site-and-publish-it-as-a-static-site-on-cloudflare-pages/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [WordPress Hosting with Cloudflare Pages](https://jameskilby.co.uk/2023/05/how-to-take-a-wordpress-site-and-publish-it-as-a-static-site-on-cloudflare-pages/) By[James](https://jameskilby.co.uk) May 14, 2023 · Updated June 1, 2026 Table of Contents The Tooling The Process WordPress Plugin Install GitHub setup Cloudflare setup I have been using Cloudflare to protect my web assets for a really long time. * [ ![Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/wp-content/uploads/2026/01/Website-Optimisations-768x560.png) ](https://jameskilby.co.uk/2026/01/web-development-improvements/) [Cloudflare](https://jameskilby.co.uk/category/cloudflare/) | [Hosting](https://jameskilby.co.uk/category/hosting/) ### [Blog Performance & SEO Improvements: Cloudflare, Privacy & More](https://jameskilby.co.uk/2026/01/web-development-improvements/) By[James](https://jameskilby.co.uk) January 15, 2026 · Updated July 11, 2026 I have spent the Christmas break making some improvements to this blog. --- # My First Homelab Storage Setup: HP Gen8 & Xpenology Source: https://jameskilby.co.uk/2018/01/lab-storage/ Date: 2018-01-06 Categories: Homelab, Storage, Synology, Ansible, Automation, VMware, Artificial Intelligence, Docker, TrueNAS Scale [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) | [Synology](https://jameskilby.co.uk/category/synology/) # My First Homelab Storage Setup: HP Gen8 & Xpenology By[James](https://jameskilby.co.uk) January 6, 2018 · Updated June 1, 2026 • 📖1 min read(197 words) I have been meaning to post around some of the lab setup for a while. Although it changes frequently at present it’s as below. I will add some pics when I have tidied up the lab/cables…. My primary lab storage is all contained within an HP Gen8 Microserver. Currently Configured: 1x INTEL Core i3-4130 running at 2.3Ghz 16Gb of RAM 4x 1Gb/s NIC 2x 3TB WD Red 2x Intel SSD 320 Series 80GB, 25nm MLC This is running Xpenology. I have the 4 Nics split into two LACP bonds. One for the management traffic and one for pure NFS traffic. The 2x WD Reds are in an SHR configuration with the SSDs running as an SSD read-write cache. This gives me 2.6TB useable which is plenty to lab and store photos and media and with the RW cache, the perf is pretty damn good. I also have a DS216+II Synology with 2x 2TB WD Red’s this is my tier 2 lab storage, for backups ISO’s etc I am running a few apps/containers on the Xpenology install as it has a lot more ram/processing power. At the moment this consists of: Docker – GitLab Docker – PHPipam Plex ## 📚 Related Posts * [Homelab SSD Failure: How Synology RAID Saved My Data](https://jameskilby.co.uk/2022/11/homelab-bad-days-almost/) * [Can you really squeeze 96TB in 1U ?](https://jameskilby.co.uk/2024/09/can-you-really-squeeze-96tb-in-1u/) * [Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/2019/02/lab-storage-2/) ## Similar Posts * [ ![vSphere Power Management Ansible Playbooks with Semaphore](https://jameskilby.co.uk/wp-content/uploads/2026/04/vsphere-power-management-ansible-768x403.png) ](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Automation](https://jameskilby.co.uk/category/automation/) ### [Automating vSphere Power Management driven by Ansible and SemaphoreUI](https://jameskilby.co.uk/2026/04/vsphere-power-management-driven-by-ansible/) By[James](https://jameskilby.co.uk) April 15, 2026 · Updated June 1, 2026 In this post I’ll walk through how I use vSphere Power Management driven by Ansible and SemaphoreUI to automatically reduce ESXi host electricity consumption — saving real money on my Octopus Agile tariff by toggling hosts between Low Power and Balanced policies. Introduction One of the larger costs of running my homelab is the electricity…. * [ ![Nvidia Tesla P4 vGPU Setup in VMware Homelab: Full Guide](https://jameskilby.co.uk/wp-content/uploads/2023/10/IMG_1107-768x403-1.jpg) ](https://jameskilby.co.uk/2023/10/vgpu-setup-in-my-homelab/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Nvidia Tesla P4 vGPU Setup in VMware Homelab: Full Guide](https://jameskilby.co.uk/2023/10/vgpu-setup-in-my-homelab/) By[James](https://jameskilby.co.uk) October 23, 2023 · Updated July 11, 2026 Card Stats Install steps VM Provisioning Folding@Home A little while ago I decided to play with vGPU in my homelab. * [ ![Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU](https://jameskilby.co.uk/wp-content/uploads/2024/10/pexels-tara-winstead-8386440-768x512.jpg) ](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) | [Docker](https://jameskilby.co.uk/category/docker/) ### [Self Hosting AI Stack using vSphere, Docker and NVIDIA GPU](https://jameskilby.co.uk/2024/10/self-hosting-ai-stack-using-vsphere-docker-and-nvidia-gpu/) By[James](https://jameskilby.co.uk) October 11, 2024 · Updated July 11, 2026 Artificial intelligence is all the rage at the moment, It’s getting included in every product announcement from pretty much every vendor under the sun. * [ ![Lab Update – Part 2 Storage Truenas Scale](https://jameskilby.co.uk/wp-content/uploads/2022/01/maxresdefault-768x432.jpeg) ](https://jameskilby.co.uk/2022/01/lab-update-part-2-storage/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Lab Update – Part 2 Storage Truenas Scale](https://jameskilby.co.uk/2022/01/lab-update-part-2-storage/) By[James](https://jameskilby.co.uk) January 11, 2022 · Updated June 1, 2026 The HP Z840 has changed its role to a permanent storage box running Truenas Scale. * [ ![Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/wp-content/uploads/2022/12/Screenshot-2022-12-14-at-21.45.23.png) ](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [Forcing an Upgrade to vSphere 8](https://jameskilby.co.uk/2022/12/forcing-an-upgrade-to-vsphere-8/) By[James](https://jameskilby.co.uk) December 14, 2022 · Updated June 1, 2026 I run a reasonably extensive homelab that is of course built around the VMware ecosystem. * [ ![How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/wp-content/uploads/2024/12/ZFS.jpg) ](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) [TrueNAS Scale](https://jameskilby.co.uk/category/truenas-scale/) | [VMware](https://jameskilby.co.uk/category/vmware/) ### [How to Run ZFS on VMware vSphere: Setup Guide and Best Practices](https://jameskilby.co.uk/2024/12/zfs-on-vmware/) By[James](https://jameskilby.co.uk) December 18, 2024 · Updated June 1, 2026 Introduction Copy on Write Disk IDs Trim Introduction I have run a number of systems using ZFS since the earliest days of my homelab using Nexenta, all the way back in 2010. --- # Running Nutanix CE at Home: AHV Setup & First Impressions Source: https://jameskilby.co.uk/2018/01/nutanix-ce/ Date: 2018-01-06 Categories: Homelab, Nutanix, Networking, Ansible, Artificial Intelligence, Storage ![](https://jameskilby.co.uk/wp-content/uploads/2020/07/nutanix-logo-HI-REZ_reverse-w-carrier.jpg) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Nutanix](https://jameskilby.co.uk/category/nutanix/) # Running Nutanix CE at Home: AHV Setup & First Impressions By[James](https://jameskilby.co.uk) January 6, 2018 · Updated June 1, 2026 • 📖1 min read(191 words) I ran a Nutanix CE server at home for a little while when it first came out. However, due to the fairly high requirements, it didn’t make sense to me to continue running it at home. This was compounded by the fact that I have many clusters to play with at work. These all run my Hypervisor of choice vSphere so I thought it was about time to see what was new with AHV and Nutanix CE…. I downloaded the latest copy of CE from the Nutanix site and extracted it to a USB stick with [Rufus.](https://rufus.akeo.ie/) (This took about 30 mins) The server that I chose to run this on was the same server I ran initially, a Dell T20 with the Xeon processor 32GB of RAM 1x240GB SSD and 1x3TB WD Red. The installation is pretty straightforward and is covered a lot elsewhere. I am just kicking the tyres with it at the moment and running a few VMs. I have also shared the storage from my Nutanix CE setup back to my vSphere environment. I’ll post back with some updates of what I get up to with it ## 📚 Related Posts * [New Homelab Nodes: SuperMicro BigTwin for VMware & Nutanix](https://jameskilby.co.uk/2024/07/new-nodes/) * [Passing the Nutanix NCP Exam: Free Training & My Experience](https://jameskilby.co.uk/2020/07/nutanix-ncp/) * [Nutanix Command Reference Guide](https://jameskilby.co.uk/2018/06/nutanix-command-reference-guide/) ## Similar Posts * [ ![Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/wp-content/uploads/2024/06/Ubiquiti_Networks-Logo.wine_-768x512.png) ](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Configure DHCP Option 43 for UniFi devices to enable remote adoption across subnets](https://jameskilby.co.uk/2024/06/unifi-dhcp-option-43/) By[James](https://jameskilby.co.uk) June 26, 2024 · Updated July 11, 2026 How to configure DHCP Option 43 for UniFi devices * [ ![Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/wp-content/uploads/2023/04/81-ZoEW24UL._SL1500_-768x461.jpg) ](https://jameskilby.co.uk/2019/02/lab-storage-2/) [Homelab](https://jameskilby.co.uk/category/homelab/) ### [Homelab Storage Upgrade: Synology DS918 for VMware & NFS](https://jameskilby.co.uk/2019/02/lab-storage-2/) By[James](https://jameskilby.co.uk) February 10, 2019 · Updated June 5, 2026 Since starting my new role with Xtravirt my Homelab has gone through several fairly significant changes. * [ ![Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/wp-content/uploads/2026/01/VMware-NVIDIA-logos_ee2f18dc-615d-4c9e-8f11-9c3c2ce2bf37-prv-768x432.png) ](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) [Ansible](https://jameskilby.co.uk/category/ansible/) | [Artificial Intelligence](https://jameskilby.co.uk/category/artificial-intelligence/) ### [Automating the Deployment of my Homelab AI Infrastructure](https://jameskilby.co.uk/2026/02/automating-the-deployment-of-my-ai-homelab-and-other-improvements/) By[James](https://jameskilby.co.uk) February 9, 2026 · Updated July 11, 2026 Learn how to use Ansible to configure an Ubuntu VM for use with NVIDIA based AI workloads in vSphere * [ ![Lab Update – Part 2 Storage Truenas Scale](https://jameskilby.co.uk/wp-content/uploads/2022/01/maxresdefault-768x432.jpeg) ](https://jameskilby.co.uk/2022/01/lab-update-part-2-storage/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [Lab Update – Part 2 Storage Truenas Scale](https://jameskilby.co.uk/2022/01/lab-update-part-2-storage/) By[James](https://jameskilby.co.uk) January 11, 2022 · Updated June 1, 2026 The HP Z840 has changed its role to a permanent storage box running Truenas Scale. * [Homelab](https://jameskilby.co.uk/category/homelab/) | [Storage](https://jameskilby.co.uk/category/storage/) ### [My First Homelab Storage Setup: HP Gen8 & Xpenology](https://jameskilby.co.uk/2018/01/lab-storage/) By[James](https://jameskilby.co.uk) January 6, 2018 · Updated June 1, 2026 I have been meaning to post around some of the lab setup for a while. Although it changes frequently at present it’s as below. * [ ![UniFi UDM Beast 1U rack-mount gateway shown front-on, with two 3.5-inch drive bays, a bank of RJ45 and SFP network ports, and dimension labels.](https://jameskilby.co.uk/wp-content/uploads/2026/06/UnifiBeast-768x219.webp) ](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) [Homelab](https://jameskilby.co.uk/category/homelab/) | [Networking](https://jameskilby.co.uk/category/networking/) ### [Unleashing the UDM Beast](https://jameskilby.co.uk/2026/06/ubiquiti-udm-beast/) By[James](https://jameskilby.co.uk) June 20, 2026 · Updated August 24, 2026 Retiring my ageing WatchGuard M200, I swap in the UniFi UDM Beast — a hardware tour, first iperf and WAN tests, the BGP/FRR setup, and the homelab services I’m consolidating onto it. --- # UK Money Saving Tips: Banking, Rewards & Travel Cards Source: https://jameskilby.co.uk/2017/05/money-saving-uk-version/ Date: 2017-05-29 Categories: Money [Money](https://jameskilby.co.uk/category/money/) # UK Money Saving Tips: Banking, Rewards & Travel Cards By[James](https://jameskilby.co.uk) May 29, 2017 · Updated July 11, 2026 • 📖4 min read(777 words) I recently got back from an amazing conference in New Orleans. This was the VeeamON conference and I was invited as part of the Veeam Vanguard program that I am part of. A lot of the UK Vanguards all travelled out together and after numerous discussions, it became clear that a lot of them were paying more for UK and international banking. They were also not capitalising on the deals available for getting rewards where they did need to spend. I will detail below what I currently do; who I bank with etc. The deals I am on may not be available or right for you. YMMV I have split this across three posts as it’s very text heavy: 1\. UK banking and rewards 2\. Maximising credit card deals 3\. Minimising costs when travelling **UK banking and rewards** I currently have a number of UK current accounts…. **Barclays Bank (Main Account)** This is my main account where my salary gets paid into. Financially it is not an amazing account it pays 0 credit interest and is not very cheap if you go into the overdraft. The overdraft is large (carry over from my student days). Most of my direct debits are paid from this account and I try to have zero balance in this account all of the time. There are three reasons why I keep this account around. 1. The iPhone app is amazing – it pretty much does everything I could want from a banking point of view. Managing balances, standing orders, and direct debits is easy. But the app also covers all the more complex features. I have organised a loan and mortgage straight from the app (I did need to visit the bank for a quick meeting with them for the mortgage). The loan took about 5 mins, with 0 personal interaction. 2\. Barclays is a fast bank. It’s hard to quantify this, but I mean transactions show up much quicker than with other banks. This helps me manage money more easily. I also have an SMS alert set up so if I am ever overdrawn I can move money from another account easily. 3\. The final reason is I have Barclays Blue rewards. For this, I pay £3 a month and they pay me £16, i.e. net £13. This is broken down as £7 for standard banking, £3 for home insurance, £5 for Barclays mortgage, and £1 for the Barclays loan. This gives me a net £156 per year, and is not classed as taxable income. **Nationwide FlexDirect** Although the Flexdirect account is a current account, I use it primarily as a savings account. It currently pays a massive 5% on up to £2500, so I always have this balance. This will pay me £125 for the year (gross). By having this account I am also entitled to a Flex Regular Saving account. This also pays 5%, however you are limited to paying in £500 a month. Over 12 months, this would give me £162.50 in interest (gross). **Nationwide FlexPlus** This is a new account for me. It costs £10 a month but the benefits are more than worth it. I took it out mainly for the Worldwide travel insurance but it also includes mobile insurance and European breakdown cover, and a number of other perks. Before I opened the account, I didn’t have mobile insurance and I have expensive phones so this increase in cover is useful. It will allow me to cancel my AA breakdown easily paying for the £10 a month account fee. It also pays 3% interest, this will likely earn me £10-20 for the year. **TSB Classic Plus** The TSB account also pays 3% interest and I tend to split my spare cash between this and the Nationwide FlexPlus. I have two small direct debits on this account, and they pay a £5 reward for this plus an additional £5 for using your debit card 20 times in the month. This isn’t that hard to achieve. £120 rewards £10-20 Interest (gross) **First Direct 1st Account** First Direct gives you a £150 bonus if you switch to them (which I did), they also give you a £500 overdraft on good terms (£250 free). So I always have this account at -£250 and have that money in one of the other accounts listed above. They also have a regular savings account that I take advantage of; putting £300 a month in (max allowed). This should give me £97.50 for the year. Total savings for current/regular saving 156+125+162.50+15+135+97.50+150 Total for the year should be £841, however I am likely to need to pay tax on some of the savings interest. ---