John Doak is the co-author of Go for DevOps. We got the chance to sit down with him and find out more about his experience of writing with Packt.
Q. What is/are your specialist tech area(s)?
John: Go, SRE, DevOps, Networking, Distributed Computing, SDKs
Q: How did you become an author for Packt? Tell us about your journey. What was your motivation for writing this book?
John: I was approached by Packt after they became aware of different websites I run on Go language development and SRE. From there I found a co-author who has worked in a similar problem space, but with an entire different direction. At Google, we needed to invent everything to deal with the enormous scale. He came from a startup background, where you need to make what exists fit your needs. And the rest, as they say, is history.
Q: What kind of research did you do, and how long did you spend researching before beginning the book?
John: There wasn’t a lot of research needed. We both were familiar with the problem space and are active in it. Of course as your writing example code that deals with things like Slack or Packer, you need to make sure you are up on the latest versions and APIs, so there was still “research”. But we were writing about what we know from practical experience and not writing about something we haven’t used in real life.
Q: Did you face any challenges during the writing process? How did you overcome them?
John: Of course. First, writing a book is a huge time commitment. Both David and I have full time jobs outside writing a book that pays our bills. We have family commitments. Packt has aggressive timelines that are fairly hard to hit and it was always a challenge to hit any of them.
The book grew from an original 300 page count to over 600. That is missing the complete code examples in the GitHub repository that probably exceed 50K lines of code.
Editing a book of this size is also a huge challenge. Originally we wrote everything in markdown, which goes to a translation into a Word document. Things get lost in that translation and you try to catch them as best as you can. There are several editing passes and you are doing your best to catch problems. Then there is a translation to PDF, where it needs to be read again. The code is almost impossible to read in Word format, so you have to catch those errors in PDF form. You also get to see how your formatting is going to be once the code is in PDF, which requires you to do revisions to try to avoid having code span multiple lines if you can to make it easier to read for the reader. And nothing slows down your writing like having to edit previous chapters while writing new ones.
Our book also had an additional co-writer who had to bow out after about 9 months, which threw us for a loop. These things happen in any collaborative environment, but that pushed more work down to us and additional time was needed.
Overcoming the challenges for this book was really three things:
Perseverance
An awesome co-writer
A supportive family + friends
Not giving up at the first sign of problems. Putting in the time, which of course you think about before you start the process but you never fully grasp what that is going to be like. Reality sets in and you just keep pushing.
Having an awesome co-writer who helps keep your spirts up I found to be essential. I’ve heard that being a writer is a lonely occupation. I can honestly say I would not have wanted to write this book without David being there to keep me sane and help with the cognitive load.
And while I like to think of myself as self reliant, like I’m a grizzly bear or something, the truth is I’m just a wolf in a wolf pack. Family and friends supporting me, lending me their ears and helping me is really how I get through my days to do something like this. My wife was supportive throughout the entire process and really was the one to encourage me to do this. She is already an author of a book that she wrote with a friend (of the non-technical variety). She is awesome in so many ways and this book would not have existed without her.
Q. What’s your take on the technologies discussed in the book? Where do you see these technologies heading in the future?
John: Well, tech has what I call short and long tail technologies and you can’t always be sure which way they will go.
Go itself is going to be on the long end of things. 20 years from now it will still be around kicking, much in the way C, C++ and others are. I don’t see it going the way of Perl or Ruby, which had decent lifespans. Rust, which is seen as Go’s competitor, isn’t going to eat Go’s lunch. Rust is for developers who can spend 10x or more time developing something because they absolutely require speed that no garbage collected language can provide. But it is much harder to write code in and large program compile times are similar to C++. In the DevOps space, Rust isn’t even in the running and I doubt ever will be.
There are entire sections on REST, gRPC and building command line tools. The RPC technologies look to be safe bets for the next 10 years at least. The packages like Cobra we use for command line tools might change, but writing command line tools will be something that exists until we all plugin to the matrix.
Kubernetes is an interesting one. It is the free software of choice for running infrastructure. It can run everything and the kitchen sink. But it is a monster. Even Google has problems running their managed version. Compared to Google’s internal Borg system (which Kubernetes is based on), it is vastly more complex because it tries to solve everyone’s problem. Where in as Borg was simple and easy to use, but didn’t allow you to run a program written in Fortran from 1980. I imagine it will be the choice for a while, but I do think there is room for a competitor that is easier to use. HashiCorp’s HashiStack seems to be more like Borg was and easier to deal with. I certainly could see some other new “hotness” come in and 10 years from now seeing a lot more competition with something that is easier to manage.
Terraform and Packer are certainly leaders in Infrastructure as a Service and image management. Competitors to these could come at any time and sweep the market. But these certainly seem to be safe bets for the next 3 to 5 years.
OpenTelemetry I think is here to stay for the long term and has buy in from major cloud providers. It is simply the only real open source, multi-language tracing/metrics solution out there. In the book we use Prometheus and Jaeger to act as collectors for this data. Those may or may not be around in the 5 to 10 year period. I can see room for a lot of improvement there and in many cloud providers, you may end up using their native collectors.
GitHub actions probably don’t have a competitor in the near future. While Microsoft’s purchase of GitHub seemed to scare some people to places like GitLab, GitHub has remained the defacto source control location of the internet. GitHub is unlikely to replace actions with something else anytime soon, so I see this as a safe bet.
The idea of ChatOps, which we use in this book, is a trend that may or may not continue in the future. It allows you to unite your tools into a single place to allow better collaboration. It can answer questions for services across the company, instead of having to ask the oncall engineer. This is a technology I know Facebook and Microsoft have leveraged within some of their DevOps groups. We chose Slack for our examples here. Slack may or may not be the cool thing for chatting 10 years from now. Some things like IRC seem to never die (Google still uses it internally I think), but other chat programs like AIM, ICQ, Google Hangouts, Google Talk, Google Allo, Google Duo, Microsoft Messenger, MSN Messenger, … seem to last a few years to be replaced with something new. Hence, in the book, we design an architecture to make transitioning your ChatOps to various messenger technologies (future proofing).
More interesting will be the direction of DevOps for the future. I like to think of DevOps as a set of methodologies around managing software and infrastructure. SRE, which is the discipline at Google I am most familiar with, had two core things that made its model different from a standard Ops model if you get to its roots:
* SRE could say no to developers based on SLA models where developers had exceeded acceptable risk
* SRE would hand the pager back to developers if it exceeded a threshold
Unfortunately, I think varying interests will always chip away at that model until those two things no longer hold true and you are left with more of an Ops model. Facebook turned their SREs into Production Engineers, which I believe tried to address those by embedding people with operational expertise into teams and building core infrastructure (however my knowledge is not firsthand, I haven’t worked for Facebook).
And in many places you can’t afford dedicated DevOps teams and you are forced into a NoOps environment. The reality here is that you are pushing the DevOps tasks onto the developers or you are pushing it into your Platform as a service (PaaS) provider. But at some point, things go wrong and in the end, your company is responsible for its product. So you will have DevOps engineers one way or the other, but it will be your developers doing double duty. This is not a bad model if your developers can remain disciplined and you can lean on an infrastructure provider for expertise with issues.
But no matter which way the DevOps trend moves, the lessons that are taught in this book on methodologies for change will be useful even with shifts in DevOps trends or technology shifts.
Q: Why should readers choose this book over others already on the market? How would you differentiate your book from its competition?
John: Using Go specifically for DevOps hasn’t been addressed by a book that I’m aware of, so we are kind of a first mover for writing about how Go is used in this way. As this book points out, the companies providing products for the DevOps world are writing these tools in Go, so it is a huge advantage to be able to interface with them and the packages they provide.
Large companies like Google also have chosen Go as the language of choice for SRE type work. This isn’t because Go was written there, it was quite a journey from a world of Python scripts and services to services written in Go. And those choices tend to trickle out of these companies into other places.
There are certainly other books on DevOps or specifically DevOps technologies. This book isn’t about the DevOps philosophies, there are other books to look at if you are interested in running a DevOps organization. And there are books that teach some basic Python and a few things you can do with it to help deploy software of make changes on machines with scripts.
We teach you not only the basics of writing in the Go language, but building robust command line tools, REST + gRPC services, leveraging existing infrastructure by extending them with Go (Kubernetes, GitHub, Terraform, Packer), building out observability using OpenTelemetry, building workflow engines, leveraging rate limiting and policy servers to limit blast radius of bad releases and so much more. Each of these backed up with detailed example code that goes beyond what we could fit in a book.
We won’t deep dive on technology like Kubernetes or Terraform, there are large books for that if you are using that technology. But we will show how you can extend them with Go to fit your environment.
Q. What are the key takeaways you want readers to come away from the book with?
John: An understanding of how to leverage Go to easy their development cycle and the methodologies that you can use to deliver software services safely.
Q. Do you have a blog that readers can follow?
John: Readers can follow me at:
www.gophersre.com
www.golangbasics.com
www.obscuredworld.com
Q. Can you share any blogs, websites and forums to help readers gain a holistic view of the tech they are learning?
John: For Go:
https://go.dev/blog/
https://dave.cheney.net
https://golangnews.com
https://blog.gopheracademy.com
DevOps
Q. How would you describe your author journey with Packt? Would you recommend Packt to aspiring authors?
John: It is a difficult path to write a book. Packt is going to provide overall book guidelines from experience publishing books, editors looking for problems in your writing and common gotchas and may provide technical reviewers if you don’t provide them yourself. They will handle the details of turning what you write into a formatted book that others can digest.
At the beginning, they will force you to think about the book before you write it by having your provide chapter and section breakdowns with synopsis information to each. They require a certain amount of content to make a book.
Everything else is something that you bring to the experience including a large chunk of your time. Packt is going to be there to remind you that there are deadlines, pushing for about a 9 month cycle on delivering all content and several months of editing to the final product.
What I’d say is that you are going to get what you put into it and Packt is there to make sure it gets published in a format that someone can read, both electronically and in book form (which is what a publisher’s job is). The hard part, that is going to be up to you.
Q. What are your favorite tech journals? How do you keep yourself up to date on tech?
John: Day to day life in the tech field, looking at new cloud products, listening to developer conferences that AWS, Google and Microsoft provide. I constantly read articles on sources like golangnews, reddit and other places. Read articles published on outages at large sites to learn from their mistakes with the technology they have deployed.
Q. How did you organize, plan, and prioritize your work and write the book?
John: Between myself and my co-author, we would divide the book into sections we were responsible for. Then I’d get up early in the mornings or spend my evenings working on the book content,. You loose a lot of weekends working on a book this size.
You have to remember each day that, the deadlines are getting closer, so you have to push to get things done. And at some point, your edits are going to start coming, which has you doing two things at once. So it doesn’t matter if you are tired after you finish your day job, you still have work to do.
Q. What is that one writing tip that you found most crucial and would like to share with aspiring authors?
John: Work on the book every day. Don’t let yourself off the hook. If you write 1 page a day, every day, you will find yourself with a novel at the end of a year. It is very easy to give yourself excuses and find yourself in real trouble. Also, remember that none of your book is going to work without examples in your repo. That is a hidden cost that many authors will forget about.
You can find John’s book on Amazon by following this link: Please click here.
