Introduction

This book and its companion came from the same surprise.

I’m a software engineer by training. I spent years building systems, sitting through incident reviews, and developing the reflexes the profession beats into you: pin your dependencies, write the test first, never trust a result you can’t reproduce. When I moved into teaching, I brought those reflexes to software engineering apprenticeships, and they landed exactly as I expected.

Then I was asked to teach a software engineering module on a data science degree apprenticeship. The brief looked easy from where I stood. The learners were already fluent in Python. They could do things with data I found genuinely hard. All they needed from me was the engineering: version control, testing, structuring code, getting it to run somewhere other than a laptop. Familiar territory. I assumed it would be the simplest teaching I’d ever done.

It wasn’t, and the reason it wasn’t is the reason this book exists.

The practices I treated as obvious — the ones I’d stopped noticing I held — turned out to be genuinely alien to people who were not, in any sense, beginners. I’d explain why you write a test, and a learner who could derive a loss function from scratch would look at me as though I’d suggested decorating the office. Not because they couldn’t understand it. Because in their world, the question a test answers — does this code do exactly what I specified, every time? — wasn’t the question they spent their days asking. Their work was exploratory. The output was an insight, not a service. A notebook that produced the right chart had done its job. Wrapping that notebook in the machinery of reliable software felt, reasonably, like effort spent on the wrong thing.

That was the surprise: the gap ran in both directions, and it wasn’t about ability. It was about what each discipline optimises for. Software engineering is built to eliminate uncertainty — to make behaviour repeatable and guarantees enforceable. Data science is built to work inside uncertainty — to quantify it, model it, and draw careful conclusions from incomplete information. Each side has a rigour the other lacks, and each finds the other’s rigour slightly baffling at first. My engineering habits looked like ceremony to a data scientist. Their comfort with “good enough, on average” looked like recklessness to me. We were both right, and both missing half the picture.

The companion volume, Thinking in Uncertainty, is the bridge built from my side: it helps software engineers develop the statistical and exploratory thinking that data scientists bring naturally. This book is the bridge built from the other side. It assumes you can already do the hard part — the modelling, the analysis, the wringing of signal out of noise — and adds the engineering scaffolding that makes your work something other people can run, trust, and build on.

I’ve tried hard not to write a conversion manual. The exploratory, notebook-driven way of working isn’t a phase you grow out of; it’s the right tool for discovery, and you should keep using it. Engineering practices aren’t a replacement for it. They’re what you reach for at a specific moment: when you’ve found something worth keeping, and “it works on my machine” stops being good enough. The skill this book teaches isn’t becoming a software engineer. It’s knowing which engineering practice to apply, when, and how much — so the investment pays off rather than slowing you down.

The title, Building with Certainty, is a deliberate counterpoint to its companion. It doesn’t mean certainty about the world; the world is uncertain, and pretending otherwise is the mistake the other book exists to prevent. It means the quieter, more achievable kind of certainty engineering can give you: confidence that your code does what you think it does, on someone else’s machine, next month, without you in the room. That your colleague can reproduce your numbers. That when something breaks, you’ll know. For work that increasingly has to leave the laptop and survive contact with production, that certainty is worth building.

If you’re a data scientist who has been asked to “productionise” a model, hand a project over, or simply make your own work trustworthy six months from now, this is where I’d start you. And if you’re just curious what your software engineering colleagues are on about, it’s a good place to start too.

Matthew Gibbons
North Cornwall, UK
June 2026

--- # Content: CC BY-NC-SA 4.0 | Code: MIT - see /LICENSE.md number-sections: false --- # Introduction {.unnumbered} This book and its companion came from the same surprise. I'm a software engineer by training. I spent years building systems, sitting through incident reviews, and developing the reflexes the profession beats into you: pin your dependencies, write the test first, never trust a result you can't reproduce. When I moved into teaching, I brought those reflexes to software engineering apprenticeships, and they landed exactly as I expected. Then I was asked to teach a software engineering module on a data science degree apprenticeship. The brief looked easy from where I stood. The learners were already fluent in Python. They could do things with data I found genuinely hard. All they needed from me was the engineering: version control, testing, structuring code, getting it to run somewhere other than a laptop. Familiar territory. I assumed it would be the simplest teaching I'd ever done. It wasn't, and the reason it wasn't is the reason this book exists. The practices I treated as obvious — the ones I'd stopped noticing I held — turned out to be genuinely alien to people who were not, in any sense, beginners. I'd explain why you write a test, and a learner who could derive a loss function from scratch would look at me as though I'd suggested decorating the office. Not because they couldn't understand it. Because in their world, the question a test answers — *does this code do exactly what I specified, every time?* — wasn't the question they spent their days asking. Their work was exploratory. The output was an insight, not a service. A notebook that produced the right chart had done its job. Wrapping that notebook in the machinery of reliable software felt, reasonably, like effort spent on the wrong thing. That was the surprise: the gap ran in both directions, and it wasn't about ability. It was about what each discipline optimises for. Software engineering is built to eliminate uncertainty — to make behaviour repeatable and guarantees enforceable. Data science is built to work *inside* uncertainty — to quantify it, model it, and draw careful conclusions from incomplete information. Each side has a rigour the other lacks, and each finds the other's rigour slightly baffling at first. My engineering habits looked like ceremony to a data scientist. Their comfort with "good enough, on average" looked like recklessness to me. We were both right, and both missing half the picture. The companion volume, *Thinking in Uncertainty*, is the bridge built from my side: it helps software engineers develop the statistical and exploratory thinking that data scientists bring naturally. This book is the bridge built from the other side. It assumes you can already do the hard part — the modelling, the analysis, the wringing of signal out of noise — and adds the engineering scaffolding that makes your work something other people can run, trust, and build on. I've tried hard not to write a conversion manual. The exploratory, notebook-driven way of working isn't a phase you grow out of; it's the right tool for discovery, and you should keep using it. Engineering practices aren't a replacement for it. They're what you reach for at a specific moment: when you've found something worth keeping, and "it works on my machine" stops being good enough. The skill this book teaches isn't *becoming a software engineer*. It's knowing which engineering practice to apply, when, and how much — so the investment pays off rather than slowing you down. The title, *Building with Certainty*, is a deliberate counterpoint to its companion. It doesn't mean certainty about the world; the world is uncertain, and pretending otherwise is the mistake the other book exists to prevent. It means the quieter, more achievable kind of certainty engineering can give you: confidence that your code does what you think it does, on someone else's machine, next month, without you in the room. That your colleague can reproduce your numbers. That when something breaks, you'll know. For work that increasingly has to leave the laptop and survive contact with production, that certainty is worth building. If you're a data scientist who has been asked to "productionise" a model, hand a project over, or simply make your own work trustworthy six months from now, this is where I'd start you. And if you're just curious what your software engineering colleagues are on about, it's a good place to start too. ::: {.text-end} Matthew Gibbons\ North Cornwall, UK\ June 2026 :::