Good afternoon, NDC. Before we get started, there are some things that you are not going to see in this talk, and I will tell you what they are right at the beginning so if that’s what you’re here for, you can leave with no hard feelings. You will not see any code. You will not see any slides. You will not see any PowerPoint animations. And unfortunately you will not see Pieter.
This talk was originally pitched as a… Pieter Hintjens did a keynote at Coding Serbia last year where he talked about the laws of physics as they are applied to software systems. And we were talking about this in a bar in Vilnius. And basically I loved this idea and we started bouncing things backwards and forwards. It was really really interesting. I said to him why don’t we do it at NDC Oslo as a double act. He said that sounds like a good idea. For reasons those of you know him well understand, he’s not here. So, if you want to see PowerPoint, I believe they have some in the next room.
I am going to talk to you about the the laws of physics. There are all of these wonderful laws that people have discovered and refined and proposed and proved over the years. And some of these laws can apply to the software projects and the teams and the communities that we work in every day. I’m assuming most of the people here in this room, we’re all developers of some sort or another. We worked on teams, we get stuff done, we build systems. And there are interesting observations about the way that we interact. Because software is about people, and people are subject to the same physical laws that govern the universe we live in, just like everything else. And some of this is going to be fairly entertaining and high-level, and some of it is actually going to be fairly specific.
And we’re going to do it in chronological order, starting with Isaac Newton. Newton was a genius. Newton basically founded the principles of modern physics and mechanics. Newton was so smart that in the 16th century he came up with a set of laws which tell you why distributed software teams will fail. He just didn’t know that’s what he’d done until four centuries later when we looked it up and went, “That’s interesting…”
So Newton was a fascinating guy. He basically had three different careers. First he was a physicist, or a natural scientist, at the University of Cambridge. And he published the Principia Mathematica, where he basically laid out all these rules about how things work. How planets move. How gravity works. How all these kinds of things happen. How they behave. How to predict their influence. Then he became an alchemist for a while. And we don’t talk about that very much because he wasn’t nearly as successful at turning lead into gold as he was at understanding planetary motion. And then he came to London and he ran the Royal Mint, and did that for the sort of last chapter of his life.
But we’re going to talk about Newton’s laws of motion. So, Newton’s First Law, which was published in the Principia Mathematica in 1600 and something basically says, something that is at rest, something that’s not moving, will not move. It will remain at rest until a force acts upon it. And something that is already moving will continue moving at a constant speed in a straight line until a force acts upon it.
Things don’t happen unless there is a force, a motive, an incentive, for something to stop happening. This is why when you wake up in the morning you think, “I’m not gonna move. I am at rest right now and Newton says I should remain at rest.” And then you remember that you have bills to pay, and if you don’t get up and get online or go into your job or whatever, then you probably won’t get paid. And so there is an economic motive. There is a force acting upon you which will get you out of bed and get you moving.
And the same thing is true of teams and companies. So you know, organizations, startups, will see an opportunity. They’ll think hey, we were talking in the show last night about the fact that the CEO of Uber doesn’t have a driver’s license. His driver’s license expired. And the joke is that it’s so Silicon Valley that when your driver’s license expires, instead of getting a new one you do a startup that will get people to pick you up from your door because it’s more fun and it’s easier.
So there are two kinds of force here. There’s sort of attractive force. There’s economic opportunity. There’s seeing something and going, “There’s something here.” We could go and do something. We could get rich. We could change the world. We could make this a better place. And then there is, if you like, the push force, the fear, where you’re being driven to do something—sometimes it’s by threats from competition, sometimes its legislative, sometimes ts you need to do some work because a piece of your stack is like a ticking time bomb and you know that you’re going to run out of address space or run out of capacity.
So you have this idea of forces. Now, you ever worked in a company where they’ve been doing some happily for ten years and you turn up and you try and change something and they’re just like, “No no. We don’t do that. This is not how we work. We do this.” This is the same thing. Once they’re moving in a straight line, they’ll just keep trucking on. And Newton’s laws, anyone who remembers studying them in high school, you always had these thing in your examinations where it’s like “imagine a weight sliding down a smooth frictionless plane in a vacuum.” Which…doesn’t happen, you know. Unless you’re going ski jumping on the moon.
But friction is a force. And friction applies to real objects in the real world. It’s why things slow down. And friction applies to organizations as well. Even if you’ve got a team who are working and the goal is focused, you’re building a project, you have something to ship. Day by day, the kind of novelty and the initial push— You know, you’ve all had the kick-off meeting where it’s really exciting and it’s inspiring and it’s visionary. And a couple of weeks down the line you’re like this isn’t really quite as much fun as it used to be. And so it slows down. Because that organizational friction is a force. And that force will slow down the people and the teams that you’re depending on to make these things happen.
Newton’s Second Law. This is the one you probably all remember from school, “F=ma”. Force is mass times acceleration. Acceleration is how fast you can change something. You can change something by moving it very very very slowly. But to actually change direction— So, acceleration is any change in direction. If you’re moving, if you stop that’s an acceleration; negative acceleration. If you’re standing still and you start moving, that’s an acceleration. If you’re going this way and you need to go that way; if you’re running Mac you need to switch to windows; if you need to get everything from Perl to Python it’s a change of direction. And that’s an acceleration.
And the force required to effect that acceleration scales with the size of the thing that you are trying to change. This is why startups can go, “Ooh, hang on. This chat room that we’ve got, people are quite liking the fact they can share photographs on it. Let’s turn our company into a photo-sharing thing which has comments.” And that’s how Flickr turned from a chat service into an online photo album. Because they’re small. They don’t have a lot of mass, they don’t have a lot of inertia.
Startups can pivot really easily. You compare this to big organizations, Microsoft, Oracle, Sun Microsystems. These organizations are big. Start thinking about these in terms of supertankers. Massive, massive, heavy structures, that do not change direction easily. Now the thing about F=ma, you need to think about the time that it is going to take. You can steer a supertanker by very gently pushing for about six weeks, and it will very slowly come about. If you want to do a handbrake turn in a supertanker, stuff is going to get broken. Because you are applying a massive force to effect a big acceleration on something which is massive.
And if you’ve ever— I’m sure there are people here who have been in organizations where somebody has used the fact that they are powerful to try and implement this kind of change… You get an email comes down from the CTO, “From tomorrow there will be no more Github, We are using SourceSafe for everything.” And they can do it because they are powerful, so they can wield massive force. But trying to wield that force to a large organization to effect that change in a short space of time will break things. If they do this with enough authority behind them sure, by the end of next week there will be no Github users left in the organization. But it’s not necessarily because they’ve all gone and done the thing that you wanted them to do. It’ll be because they’ve left.
So this is the equivalent—you know, visualize someone handbrake-turning a supertanker into an iceberg. It’s a very destructive mental picture, but when you’re trying to effect large changes to large organizations in a very short space of time, that’s effectively what you’re trying to do.
Newton’s Third Law, for every action there is an equal and opposite reaction. This is the thing, you push something, it pushes back. I’m standing here right now. I’m pushing down, floors pushing me back up. By happy coincidence, they’re exactly the same. It’s why I can’t fly but it’s also why I’m not sinking into the ground.
You get this in organizations as well. You try and change something, there will be resistance. You propose something, people will push back at you. “Hey you know, I think we should stop sending out Word documents and we should do our updates on the wiki.”
They’ll be like, “No, we don’t like the wiki.”
It’s like, “The wiki’s great. Everyone can read it. It’s online. Anyone can edit it.”
“Oh, I don’t like anyone editing it.”
You get this reaction that pushes back.
The other interesting thing about the equal and opposite reaction is if you have one of these massive organizations, lots of inertia, lots of people, lots of money, you can use that as a springboard. Very very small organizations, if they try and fire off something to go and explore a new opportunity or a new venture, it can actually split the team. Because suddenly they don’t have the bandwidth to cope anymore.
But if you are say, Microsoft, who are steaming along down the software industry with however many billion dollars in Windows and Office revenue behind you, and you suddenly realize that Sony and Nintendo and Sega are about to start winning the console war and you think it would be a great idea if Microsoft had a console… You can’t steer the supertanker. Microsoft the organization could not have pivoted to focus on games consoles as their primary area of business.
But what they could do is leverage the mass that they had to spin off the Xbox division without worrying that that was going to disrupt what they were doing. Xbox may have failed, but I don’t think anyone would ever consider that spinning off a small team to go and develop a games console was an actual threat to Microsoft. There’s the marketing risk and there’s the odd piece of brand fallout, but in terms of the core operation of that organization and the way that it works, the fact that they have that mass and that inertia allows them to spin things off against it at relatively little risk to what they’re doing.
So that’s Newton’s predictions about software teams in organizations, written down in Latin in the 16th century. And here we are today exploring them and thinking about how they apply to what we’re doing.
There is then a thing called the equivalency principle. The equivalency principle is an observation in physics that effectively says it is impossible to tell the difference between gravity and acceleration. You don’t know if you’re in orbit or your falling, until you hit the ground. Because you cannot distinguish— As an observer, you cannot distinguish between acceleration that is because of gravity, and acceleration that is because of a force that’s being applied to it.
Science fiction tropes love this. You ever read any of those books where they have the spacecraft that accelerates all away so that there’s gravity from the fact that the engines are pushing? And then when they get to the halfway point, the whole ship turns round and it does the rest of the journey in reverse and it’s breaking the whole way? And so that acceleration and then breaking provides an artificial gravity for all the people who are on board. It’s an idea Arthur C. Clarke used a couple of times.
Now, the interesting thing about the equivalency principle is that it applies to observers within the frame of reference that you’re talking about. There is a thing in aviation that’s called a graveyard spiral. Now, a graveyard spiral is when you’re flying in a cloud, you can’t see anything, your instruments are out, and you think you are flying straight and level. And what you’re actually doing is you are banking in a turn like this. But because the centripetal force from the turn feels just like gravity, you think you are in level flight. And because the gravity is coming from the turn, you’re not conscious of the fact that you’re falling.
Now, to somebody outside it is completely obvious that you are not in level flight, you are doing this. [indicates a downward spiral] Someone could see you on radar or something, they’d be like, “Hey. Pull up pull up pull up. You’re in a deep problem situation here.” The point about this is that from within our teams and within our organizations it can be very very difficult to tell whether we are doing the right things or not, until you hit the ground and realize that you’ve been falling all of this time.
Projects where you’re delivering code, you’re working hard, you’re shipping things… And then suddenly you realize that actually you’re not going to get out in time; your version one product is targeting the wrong device; there are issues with your deployment strategy; something happens and…splat. You hit the ground, you faceplant, and the things over.
So the point about this, the equivalency principle is going to bite you because from inside it’s impossible to tell the difference between being in a stable orbit and being in freefall. So you need external observations. You need somebody from outside who is looking in or observing what you are doing to allow you to tell the difference between these two scenarios.
Which means you need to measure things. You need to set up some metrics. And the problem with measuring things. Is something called the Uncertainty Principle. So, Werner Heisenberg has come over to say, London, from Germany. He’s driving down the motorway. He’s in his BMW. And because in Germany the Autobahn has no speed limit, he’s got his foot down, he’s going a fair old clip. And the police pull him over. And they say to him, “Hello, Mr. Heisenberg. Do you know how fast you were going?”
And he says, “No, officer. But I know exactly where I was.”
And the policeman says, “We just clocked you doing 120mph, mate.”
And Heisenberg says, “Great, now I’m lost!”
Okay, so the people who are laughing know what the Uncertainty Principle. Heisenberg basically observe that in subatomic physics, you cannot measure the position of a particle without affecting its velocity. So the more accurately you know the position, the less accurately you know how fast it’s moving. And the more accurately you can measure how fast it’s moving, the less accurately you know where it is. To measure more accurate velocity you measure over a longer distance, which means you have more uncertainty around where the particle is during the measurement. To measure its position, the only way to do that is to bounce something off it. Imagine you’ve got a soccer ball and a dark hall, and you’re trying to find out where it is by throwing golf balls. And sooner or later one of them is going to go “ping!” and come back at you. But as you do that, the soccer ball is going to start rolling because someone just threw a golf ball at it.
In software, we try and measure things all the time. Because it’s very very difficult in a lot of organizations and in a lot of programming methodologies and things to tell how good your progress is, to tell how well you are doing. Are you going to hit your deadline or not? And by trying to measure things about the way our systems work, you can actually interfere with them to the extent they stop working.
A completely degenerate example of this, someone comes in and goes, “Look, this whole fixed salary thing is nonsense. We’re gonna pay you all per line of code. Software is code, and we need code because we need to finish it. So from now on at the end of every day tell us how many lines of code you have written, and we will pay you one dollar for every line of code.” How many people in here would be millionaires by the end of the morning? Because when they start measuring lines of code, you stop going, “No, I’m not doing unit tests anymore. I’m just gonna put newlines in.” And then, “Okay, I’m going to put empty statements.”
And there’s all sorts of measuring things. It’s a quite common in agile. One of the reasons why I agile teams and scrum says you should estimate everything in points is because people used to estimate in hours. And then people would use the hours as a yardstick to measure them against. “You said this ticket would take four hours. Why isn’t it finished? You started at 1:00 and I’ve checked in with you 2:00 and 3:00 and 4:00 and 5:00, and it’s still not done.” And you’re like, “Well, no four hours was a sort of… Out of all the tickets we do in a year and divide them by the number of tickets you’ll probably got some averages out, but these are not absolute accurate metrics.”
And there’s all kinds of ways of measuring things. Anyone ever build websites and then go and run them through the W3C XHTML validator so you get that green tick that says well done, good job? Your customers don’t care if your HTML validates. The only person who cares about that is you. And there’s nothing wrong with the sort of of craftsmanship in professional pride. But never once have I had someone who’s like, “Yeah, new product is ready. Customers want to buy it. But we’re not shipping it because the HTML doesn’t validate yet.” It’s like, it doesn’t matter. That’s not the thing you should be measuring.
Look at why you’re there. What was the the force that got you moving in the first place? How do you measure that? If you’re try to be Uber, how many drivers, how many rides, how much revenue? If you’re trying to be Spotify, how many bands, how many albums, how many streams? What is it you are trying to do? Because if you try and measure something else, you are going to end up affecting the team and the system that is trying to deliver something. Because they’re going to start delivering the thing that’s being measured. Because that’s what they’re being assessed on. That’s where the sense of value is coming from.
So there you go. That’s kind of the deep theoretical physics stuff. It’s going to get a little bit silly now. Murphy’s Law. Murphy was, apocryphally, an engineer working in the United States Air Force in the 1960s. They were doing experiments with the deceleration on the rocket sleds. So they’d get a crash test dummy and they would put it in a chair on a sledge on a railway. And they would put a rocket on the back of it and they’d smack it into a wall as hard as they could. And then they were like, “Damn, we didn’t actually get any readings out of that. All we did is broke the dummy.”
So they came up with this idea of using strain gauges on all the seat belts, so that when the thing crashed into the wall they’d be able to tell afterwards how much strain each of the contact points on the harness had been subjected to. And John Stapp I believe was his name, was the Air Force colonel conducting the experiment. And Murphy was the guy who installed the strain meters. And the strain meters were symmetrical. They were physically symmetrical, but they only worked one way round. And he installed every single one of them backwards. And the phrase Murphy’s Law was immortalized in a paper that Stapp wrote afterwards where, depending which version of this you read and where the provenance comes from, is either “If something can fail, it will fail” or “If there’s a way of getting it wrong that bugger will do it.”
Murphy’s Law is interesting because Murphy’s Law is something that you want to keep in your head when you’re thinking about user experience interaction and the way that you design and build your systems. So, one of the sort of great examples of this is the old joke, when the inventor of USB dies, they’re going to lower his coffin into the hole, stop, pick it up, turn over, and lower it in again.
USB is… So for starters, you get systems where you can actually destroy something. One of the interesting distinctions between the United Kingdom, where I’m from, and most of Europe is that you can’t put our mains plugs in upside down. With alternating current, it doesn’t really matter because live/neutral, neutral/life. There aren’t very many consumer devices that are sensitive to that. But I always find it a bit interesting you know, I have an extension cord where I put a European plug on a UK four-way brick so I can plug laptops and things in. I had to go and buy a Norwegian plug for it. And I open the thing up, I’m like which way round do you wire this? And it’s that actually it doesn’t matter.
But if it did matter, then every time you plugged in a kettle or a laptop or anything in Europe you have a 50⁄50 chance of blowing it up. Because if the plug fits the wrong way round and it shouldn’t… Now, USB looks like it’ll fit the wrong way round, but then you realize as you get in and there’s actually a thing inside it that stops you inserting it backwards.
Nine-volt batteries, the PP3 batteries, the rectangular ones. You know they’ve got a little crown and they’ve got a little knobble on the top so that you can’t connect them the wrong way round? It’s exactly that. If it matters which way round something is going to be done, make it impossible to get it wrong. Don’t build a strain gauge that looks exactly the same until you crash it into a wall with a crash test dummy on it.
The Apple Lightning connector, the one they use on the new iPhones I think is a brilliant example of this, because it works both ways. It’s like a European plug. It’s a USB connector but it doesn’t matter if it’s this way up or that way up, so you can’t get it wrong. So you never again have that thing of of scrabbling and going, “Oh, it won’t fit,” and having to turn it over and doing it the other way like you do with Micro-USB.
The other thing where Murphy’s Law is interesting is thinking about the interactions between your users and your systems. Because if somebody is going to— You’re building an input, say, on a form and you’re testing it, somebody out there is going to do the wrong thing. Somebody is going to do it.
The joke I love about this is a bad software tester walks into a bar. They order a beer, two beers, glass of wine, gin and tonic, packet of peanuts… Check. It’s good to go. The bars is ready. A good software tester walks into a bar. They one beer, two beers, a glass of wine, -1 beers, a million beers, infinity beers, a lizard. Because they’re testing the edge cases. Because it is all too common for us as developers, because we have developed these mental models of the interaction models of the systems were building, you have a picture in your head of what your user is like. And sometimes when you meet a real one it can be a real shock. Because they are nothing, at all, like the person you imagines was going to be using your system.
So when you are testing, when you are developing systems and putting things together, don’t always think “how can we make this right?” Concentrate on the golden path by all means. If your users are smart, they’re paying attention, they’re doing the right thing, make that as seamless and painless and pleasant as possible. Because user experience is so fundamental to delighting users. But also think about what can go wrong. Because Murphy’s Law says if something can go wrong, it will, eventually. There is a corollary that says it’ll also go wrong at the worst possible time.
Okay, so ten laws of software development. We’ve had Newton one, two, three. We’ve had the equivalency principle. We’ve had the uncertainty principle. We’ve had Murphy’s Law.
Zipf’s Law. Zipf was a linguist, and Zipf discovered something which is true, and is freaky as hell, and no one knows why it is the case. Zips was analyzing the frequency of words in natural language—spoken language, and written language. The most common word in English is “the.” The second most common word in English is “of.” Which in any reasonable corpus of written English occurs about half as much. The third most common word is “and.” It occurs approximately one-third as much. The 5,570th most common word in English is “source,” and it took us almost exactly 1/5,570th as often as the word “the.” There is this remarkable power law distribution of natural language, and no one understands why this is the case.
And when you start looking for it, you find that the same power law distribution starts cropping up in all kinds of places. If you look at the populations of large cities, the United States is a good example of this. The second most populous city has about half as many people as the first. The third most populous has about a third as many people. And this distribution follows all the way down.
And there is one theory that this it’s a product of the fact that we are human beings and we’ve subconsciously…as a society, as a group, we tailor and we optimize the things we have to worry about to keep them at a manageable scale. We want something which is infrequent. Naturally, we are wary of using it more often because it is infrequent. So it seems alien. The word “the” is very familiar. People use it readily because they’re comfortable with it because they see it all the time. It’s always there. It’s omnipresent. A word like “source” you need a more specialist application for.
And this 80⁄20 rule crops up all over the place. It occurs in codebases. If you look at the number of function calls and function routines and cyclomatic complexity of your code, then you’ll find a similar power law distribution. You have something at the top, you’ll have something that’s used about half as much, about a third as much.
And this is very closely aligned to something called the Pareto Principle, which is also known as the 80⁄20 rule. Which also applies in all sorts of situations. Basically, 80% of your users will only ever use 20% of your features. 80% of the—have you ever had the experience where you come in to work on a Monday or you sit down to work at home on a Monday, and by the end of the day you’ve pretty much finished something? And then it takes you the whole rest of the week just to get the couple of bugs out and get it rendering in this one browser and work out why that unit test is failing? 20%, literally one day out of five—Monday Tuesday Wednesday Thursday Friday—Monday is the 20% of the time where you get 80% of the feature set done and delivered. And then it takes you the rest of the week to do all the little snagging edge cases that really aren’t that important, but if you’re going to ship the thing they’ve got to be right. Eighty percent of the bugs and defect reports in your software will come from 20% of the lines of code. I’m not going to go as far as saying that on teams 80% of the work is done by 20% of the people, but I’m sure some of you are thinking, “Hang on…”
And yeah, Zipf’s Law, it’s true. It holds true for all sorts of data sets and things. It’s really interesting to think about when you’re prioritizing and you’re deciding the feature set and scale of the systems that you’re going to build. But we don’t know why it’s the case. There’s just something about things created by people that likes them to follow a power law distribution. And I would be fascinated if one day they actually find out why.
So, seven laws. We’re going to move on to the last three. The last three are where we start really talking about laws all of software as opposed to taking laws of physics and and trying to make them fit and stuff. So the last three laws we’re going to talk about are Amdahl’s Law, Conway’s Law, and Moore’s Law.
Amdahl’s Law is about the rate of improvement you can achieve in parallel computation by adding more resources to it. So a good example of this, say we’re organizing a party. We’re going to have a nice big NDC afterparty tonight, and we need to prepare for it. And we’ve got…150 people in this room? Fifty people in this room, say. So we want to make a little little bag with sweets and stickers and stuff for everybody, because it’s not a party unless you get a little bag. And we need to go and pick up the cake. And the cake is in Sweden, because we messed up the ordering.
So we have a workload. We’ve got fifty— Say it takes ten minutes to make one of these little party bags. So we’ve got fifty pieces of work that take ten minutes each. So that’s 500 minutes, ten hours give or take. And somebody needs to get a train to Sweden, which will take four hours, and pick up the cake and bring it back; take another four hours. So if you, sir— I nominated you and said, “You’re gonna have to do all the work because this is your party. You’re gonna have to do it,” how long’s it going to take you? So you’ve got ten hours to make up all the party bags. And then eight hours to get to Sweden, bring the cake back. So you’re looking at eighteen hours, say. Take short days, long lunches. Call it three days’ worth of work.
Get someone to help you. So you, sir. You’re like, “Come on. Let’s help out.” You’re going to make half of the party bags here. So, you’ve got to make twenty-five, you’ve got to make twenty-five. So, suddenly, you’ve reduce the workload involved. You can do a day and you can do a day. You’ve got all the bags made made and then someone has to go and pick up the cake. So someone says, “Well this is good. Why don’t we get someone else to go and pick up the cake while we make party bags.” So we’ll get you to help out. You go to Sweden, pick the cake up. Bang, we’ve gone from three days, to two days, to one day.
You, you, you, you, you, you, you, everyone here all decide they’re all going to pitch in and make party bags. How long’s it going to take? One day. There is no way adding more people can achieve that in less than a day, because we have a piece of timebound work that cannot be parallelized. There is no way that someone can bring the cake back while someone is on the train to Sweden to pick it up. What it does is it imposes an upper limits on the number of effective cores, threads, processes, that will give you any improvement in how a piece of work is being done.
So, Moore’s Law, the other one we’re talking about here. Gordon Moore worked for Intel, I’m sure you’re all familiar with Moore’s Law. Basically it said that computers got twice as fast for the same money every eighteen months. Originally, his observation was—this was back in the early 1980s, I think? And he observed that the number of transistors you could fit on the same piece of silicon was doubling every eighteen months. And he thought this would probably be true for about another two or three years.
It has proved to be true for two or three decades. It was absolutely spot-on. And because so much systems design and interaction and system capability scales directly with the amount of processing power, which is the number of transistors you can fit on a silicon wafer, basically computers got twice as fast every eighteen months. For the same price. So on the one hand we have systems becoming more more powerful in terms of the amount of parallelism they can do. On the other hand, we have Amdahl’s Law, which imposes limits on how much of an improvement you can get by adding more process to it.
Now, what’s interesting with Amdahl’s Law is think about in terms of people. So, go back to our example. There we’ve got a project that we cannot parallelize. There is no way we can organize a party— You know, it’s half past three now. The conference finishes at half past six. There’s no way we can have our party ready in three hours no matter how many people we hire to help us organize it.
If you are working on a software team and you have activities which cannot be parallelized and a timebound, they impose an upper limit on how much faster you can go by adding more people to it. The classic example of this is meetings. If you have eight hours worth of meetings that are necessary because that’s how you’re project works, there is no way you can hire enough people to do that project in less than eight hours. And so it imposes an upper limit on— Take the total amount of time. You’ve got twenty days’ work to do? Except two days of them are meetings. You can get ten people on that team, but the eleventh person will have nothing to do. Because the meetings are happening and everyone else is already busy, and that’s it. All of the work is being taken care of. All your tickets are in flight. Everything is already happening.
So these patterns of locking components. If you think about distributed system design. So, imagine for one second that we had an online commerce web site. Customer’s going to connect to our web site. They are going to place an order. We’re going to talk to the inventory system. Determine that we’ve got that particular type of…I don’t know, NDC party cake. Ordering a cake online, because we’re not going to go to Sweden.
So we go on, we order the cake. Check with the inventory, do we have the cake, yes. Respond. Check with the account system, is this customer validated? Yes. Okay. Send a message to the warehouse system, please ship this cake to this address. Yes. And then send a response back.
And those systems may take three, four, five seconds to recover. But what we’re going to do, whenever anyone hits our website we are going to set up a massive distributed transaction and take an atomic lock on every single component in the entire system, until all of them, they are not allowed to deal with any other requests or process any other work. So whilst we’re ordering the cake, the website is effectively down for everyone else in the world. The warehouse system is down for everyone else in the world. The stock control system is down. The delivery system is down. When you look at like that it seems absolutely crazy, doesn’t it. No one would ever build software like that.
But you’ll call half a dozen people into a room and sit them down to have a meeting and say, “None of you can do anything else for the next two hours.” We are getting very very good at modeling distributed systems in terms of autonomous actors that have their own workloads, they have their own inputs, they have their own outputs, and they can deal with the work that comes to them as effectively as possible, and these systems are easy to scale up.
And this brings us on to the tenth of the laws we’re talking about today, which is Conway’s Law. Mel Conway observed in 1964 that organizations that create systems will tend to create systems that reflect the communication structures of the organizations that built them. This is something that I’ve done a lot of thinking, a lot of research, and a lot of speaking about. Peter Hintjens summarized this as “if you have shit people you’ll have shit code.” Which is a fairly piffy kind of synopsis of how it works.
What it means is if you’ve got two teams who for whatever reason communicate amongst themselves better than they communicate with the other team, you’re going to end up with two very tightly-coupled components with some kind of interface layer between them that doesn’t work quite so well. If you’ve three, four, five teams, you’re going to end up with three, four, five different components. If you’ve got teams that talk to each other very readily, then the interface between those two teams is going to be good. It’s going to be fit for purpose because those people will communicate. If you’ve got two teams who are very bad at maintaining the boundary, you are going to get tight coupling, because two components that were supposed to be separate end up being bound together.
So we have these three laws. We have Moore’s Law, which says that systems are getting cheaper. Computation is getting cheaper, everything’s getting more expensive. And you know, when I first started doing software development, I probably spent about an hour a day waiting for my PC to do things. Now I probably spend about an hour a day waiting for my PC to do things. Because we’re not using all of this amazing computing power to do the same things quicker. We’re using it to do more things. And so the fact that the amount of time you spend interacting with the system remained constant must mean that the complexity of the systems we are building is doubling. There is a lovely quote, “the greatest achievement of the software engineering industry is to completely neutralize the astonishing advances made by the hardware engineering industry.” So the system’s are getting twice as powerful, but they’re not getting twice as fast at actually doing real stuff, which means that the work they’re doing is getting twice as complicated. Which means we are building more and more and more complicated systems, and the complexity of those systems is doubling every few years.
Conway’s Law says that the systems will reflect the communication structures of the organizations that created them. And Amdahl’s Law says that you have this upper limit on how effective your team sizes can be if you have things like meetings and work that cannot be scaled and broken down.
So effectively, the message we can take, the way that we can synthesize all of these laws into recommendations and things we can think about and how we do approach the task of building software differently… We’ve got to get better at communicating asynchronously. We’ve got to get better at interacting with teams and organizations. The open source model kinda got this right, almost from the word go. How many people ever went to a meeting for an open source project they work on? Let alone a meeting at nine o’clock in the morning where coffee and donuts are served.
They got this distributed, asynchronous, people do work when they’re ready, we maintain information and statistics and people can work on it in their own time. That model is going to become more more widespread. Because in order to continue keeping pace with the development of the systems we’re building, we need bigger teams. And there is a limit on how big teams get if you are doing things like having meetings. So, get rid of all the meetings. Make a massive distributed team that’s basically everybody in the world. And love the fact that everything you do is going to be running twice as fast in two years’ time but it won’t matter because you’re going be doing twice as much with it.
And I think we’re gonna stop it there. Thank you.
Are there any questions? I love it when there’s no questions. It means you have all achieved complete enlightenment. So yes, thank you all very much for coming. Enjoy the last bit of the conference.