Field notes · 5 min read
What an AI Hall Changes for Cabling Engineers
Twenty-two engineers, one month, one AI cluster in Johor. What changed on the floor when the racks were GPU racks, and what stayed the same.
Most of what gets written about AI halls is about power and cooling. Less is written about what they change for the engineers pulling and testing the fibre. We spent June in Johor with twenty-two engineers on optical validation for an AI cluster. Some things were different. More stayed the same than people expect.
What stayed the same
The install. Pre-terminated trunks are pulled, dressed, plugged and tested as they are anywhere else.
A tidy rack in an AI hall looks like a tidy rack in a colocation hall, and an untidy one causes the same problems six months later. An installer who does good work in a colo hall does good work here.
What changed: testing became most of the job
On a standard build, testing comes at the end. On this one it took more hours than the install. Every link was validated from the server NIC, through leaf and spine, to the demarcation point.
Every link, with no sampling. When a single rack costs what a GPU rack costs, the client wants a passing result for each link.
That changes how you staff a hall. We ran two streams side by side: one pulling and dressing, one testing behind it. The testing stream was never the smaller of the two, and it never became an end phase that ate the remaining float.
Every link was validated from the server NIC to the demarcation point. None were sampled.
What changed: the faults were harder to see
The fault that cost us the most time was raw bit error rate. A link with a contaminated end-face or a marginal bend often still comes up.
The port shows connected and the lights are green, so it passes a visual check. It still fails the acceptance test, and the only way to find it is to read the error counters.
Chasing that is different from chasing a dead link. The inspection scope and the test set are the only way the problem is found at all, so they stay in use all shift. An engineer who leaves testing to the end of the day hands back links that fail later.
What changed: the handbook mattered as much as the install
We wrote a validation handbook for the job: acceptance criteria, the test evidence format, every fault tag in the client’s portal and a step-by-step fix for each. With it, any engineer could work a fault without waiting for someone senior to look at it.
We revised it mid-job, because the first version’s acceptance criteria were too loose once real racks were in front of us. A handbook written before anyone has touched the site is a draft until the first-of-kind proves it.
By the end of the month the handbook was worth more to the client than any single day of install, and it stayed in use after we left.
Three questions to ask before an AI hall
- Who owns the test pack? If the answer is “the client’s QA team”, you are buying installers, not a delivered package.
- What happens to a marginal link? “Retest it until it passes” is the wrong answer. A link at the edge on test day fails the first time somebody moves a cable in that rack.
- Is there a written standard? Ask for the document that says what a finished rack and a passing link look like on their jobs, separate from the method statement.
Labour sets the date
Power, grid connections and planning decide whether an AI hall gets built. Labour decides whether it gets built on time. A consented site with power secured and nobody to cable it is still an empty building.
If you are building high-density halls and need fibre engineers and testers who have worked on one before, send us the roles and the dates. Request engineers
Need engineers, or the package delivered?
Send the roles, the dates and the region, or the scope, the specification and the programme. On active projects we reply the same working day.
Check the paperwork and the earliest start first →