Managing Costs of a Software Project

Developing software is an extremely time consuming process, requiring long hours of uninterrupted focus on the technical task at hand. That coupled with constantly learning new technologies, analysing requirements, designing, documenting,  coding, debugging, testing, installing etc. makes software development a stressful and difficult job. But no matter how difficult the job, for certain types of people the process of imagining something and bringing it to life from nothing proves to be work that is incredibly rewarding and self satisfying. Most successful software developers that I’ve met over the years love coding and most cannot imagine doing anything else with their lives.

Before I started working as a software developer professionally, I  remember spending hours sitting at my computer for hours on end writing code just for fun, without any interruptions or any other worries. When I was done my code had no bugs whatsoever and by that measure the code was perfect. If I wanted to implement another feature, I would just start coding it right away. The only limits were my own imagination. However, after I started working as a software consultant for a few years I noticed a few things that would kill the joy of coding. It’s for these reasons that many developers end up changing careers or at the very best keep switching between jobs in the hope of finding a light at the end of tunnel i.e. bringing joy back into coding.

The truth is that if you want to enjoy your career in the real world, you cannot continue sticking your head in the sand and pretend that you’re just a coding geek, and ignore the business aspects. You may not enjoy getting involved in the business aspects of software, but as the saying goes, “if you can’t beat them, join them” and believe me when I tell you that you definitely cannot beat the business people for the simply reason that they are pragmatic realists while most coding geeks are idealists with their heads in the cloud … or sand or whatever else geeky developers put their heads in. As George Carlin once said “Inside every cynical person is a disappointed idealist.” Continuing on a path of avoiding business meetings will turn you into a disappointed and grumpy pessimist, ending up complaining to your grandkids over Christmas dinner about how unfair the world is. So you may as well join the business people and turn yourself into a pragmatic realist that somehow always seems to find a solution where others see only problems.

Here’s some advise for getting involved in the business side of things. It all starts with managing ongoing costs of projects. Firstly, every software project has three components that need to be managed and there isn’t a single business meeting in which these components are not brought up:

  • Money: in the real world, people need to put food on the table, bills need to be paid and companies need to make a profit otherwise they go out of business. For this reason every little thing needs to have a monetary value assigned to it. So if you have an idea for a feature you’d like to develop, you cannot just start coding it because someone needs to pay for your time and effort. Your customer/boss will only pay for you to do it if it provides an ROI (Return On Investment) to the company i.e. a financial benefit the company gets from having this feature/application developed. Therefore before you start coding, you first have to create a proposal, estimate costs, prepare an ROI analysis, present it to the customer and wait for approval. Depending on the costs, complexity of the project and other factors this process could take a week, a month, a year, or it might never get approved (probably because your proposal sucked).
  • Time: regardless of whether you’re billing by the hour or you’re getting paid a flat salary, your time is being paid for. Therefore any feature or app you are developing must have a time constraint assigned to it because time equals money. To make matters worse, there are deadlines imposed by the customer because they need it by a certain date. If that’s not bad enough, you’re competing with other software companies, and so in order to win a contract, you’ll then have your salesman promising the customer (on your behalf) that you will in fact delivery the finished software in half the time for a fraction of the money that other companies are offering it for. While you’re still standing there dealing with the shock and horror of the situation, the salesman will then turn around and spew out motivational quotes to you which he heard at some sales seminar that he went to last week … or maybe he’ll just quote Barack Obama by saying “Yes, we can!”.
  • People: the more people are involved in a project the more politics you can expect. That’s just a fact of life. The reason being that different types of people have different agendas with different incentives in their job. So once again, before you get to write any code, people need to agree on what this code needs to do and how it needs to do it, how long it’s going to take, what budget is going to be allocated to the project and who is going to handle which work. Going forward you can expect millions of emails, phone calls, conference calls and endless recurring meetings that will make want to pull your hair out … assuming you have any hair left to pull out. Throughout this process both functional and technical specifications need to be written up which essentially serve as contracts for what will be delivered as the finished software to the customer.

You would think that people would be aware of all the above, but most people, especially inexperienced or non-technical stakeholders, somehow still believe that software projects are purely about just writing code. They either fail to foresee all the above mentioned factors or they simply choose to stick their heads in the sand and not learn from past mistakes. So they typically end up quoting purely for the time required to write the code, thereby setting up unrealistic expectations with the customers. On the customers’ end, it goes without saying that most customers are not software people and therefore never expect to pay for anything other than the time required to write code.

When these naive people inevitably run into these problems they throw their toys out the cot, arguments ensue which lead to meltdowns and they just generally become stressed out and unhappy with life. Many of the software companies then end up going out of business because of failed projects or loss of money due to time that was never billed for.

In order to to survive in this industry and avoid projects going pear shaped, you have to start managing expectations. Expectations translate into managing money, time and people and it all starts with the first quote to the customer. A million questions will arise when doing the first quote on a project. Here are some examples:

  • “Should we quote for emails, phone calls, meetings and other project management related work?”
  • “What if the customer refuses to pay for anything other than purely coding time?”
  • “Should we then just act like a charity and give that work away for free?”
  • “What percentage of the project do we spend doing project management versus time spent developing software?”
  • “What if unexpected things go wrong and the costs go up?”
  • “What if the customer keeps changing their requirements resulting in scope creep?”
  • “Should we then quote for requirements analysis and documentation work?”
  • “Will our competitors be cheaper than us because they’re not quoting for all the above?”
  • “What if our competitors undercut us by deliberately underestimating the time and effort required and oversimplifying the customers’ requirements?” This happens on a regular basis by the way. Many sales people do this because they know that once they get started on a project the customer will not be able to simply turn back if the project ends up costing more than what was originally quoted for. This approach is like the drug dealer sales approach i.e. the first one’s free, after that we’ll nail you.
  • “If that happens, will the customer be experienced enough to realise that they’re being played by our competitors and that in the long term it will eventually cost them more than they bargained for?”
  • “Should we then preempt the competitors’ tactics and deliberately under quote just so we can get the deal i.e. should we stoop to our competitors’ level?”

These are all difficult and valid questions to ask yourself before quoting on a project. What I can tell you though, beyond a shadow of a doubt, is that if you’re only charging for purely coding time, your company will inevitable go out of business. The reason being that in my experience at least 50% of the work required on most projects excludes coding time.

So what is the best way to quote and bill then?

  • Wrong sales approach: Under quote and over bill i.e. the drug dealer sales approach. As mentioned above, most inexperienced sales people will under quote and over bill in this manner and most inexperienced customers will accept these quotes, which unfortunately perpetuates these bad practices i.e. people learning from bad examples. This approach is equivalent to over promising and under delivering in terms of costing. There are all sorts of other problems which this approach can create:
    • By quoting for purely coding work and/or cutting the budget, you are cutting the time on the project, thereby putting unnecessary pressure on the developers who will end up taking shortcuts in their code, resulting in massive amounts of technical dept being accrued. In the long run, the project will end up costing way more anyway. Hence I say that you would be putting yourself and/or other developers under unnecessary pressure.
    • It goes without saying that taking shortcuts leads to lower quality code being produced and the software being filled with bugs.
    • Due to the low quality code and over-budget costs, your reputation will end up being tarnished in process. So even if you end up finishing the project successfully, that very same customer will most likely never call you again for the next project.
    • In the end it really comes down to the customer getting what they pay for. That being the reason for many internal and enterprise products only being as good as they need to be i.e. sales people undercutting competitors, underestimating requirements and customers being cheap and shortsighted.
  • Right sales approach: Over quote and under bill: it’s always difficult to estimate the costs and timelines of a project. For this reason and all the reasons mentioned above, I have found the best strategy to be as follows:
    • Sales meetings: During your first sales meeting/s with the customer, get a sense of the their requirements. You don’t need to get into too many details about every custom single business rule and how exactly each feature will be developed. The reason being that the customer is not paying for these sales meetings. Hence you don’t want to spend too much time doing the analysis and design work before any payments have been made or approvals have been granted i.e. you do not want to work for free. Furthermore, it can often happen that you’ll go through a long and rigorous analysis phase, designing and fully documenting a proposed solution only later to find out that the customer took your design to a competitor and ended up paying them for only their coding time. You may or may not be surprised to know that this in fact happens quite often. If you do this enough times your company will end up going out of business. The purpose of the sales meetings should only be to:
      • Sales Pitch: Pitch yours and the company’s services to the customer. The main aim of your first sales meeting should simply be to prove your competence and skills to the customer.
      • Requirements Overview: get an overview of their requirements for the software they’re looking to develop. Don’t get bogged down into specifics. The specifics should be left up to the analysis phase when the customer is actually paying for your time.
      • Proposal: put together a proposal for the solution you plan on implementing. This proposal should not be a functional and especially not a technical specification. There should be no mention of specific features or technical details of how the software will be developed.
        • Technical Information: from a technical perspective it should only provide an overview of the technologies (hardware, frameworks, SDKs and programming languages) you plan on utilising to provide the solution. You can  provide them with generic architecture diagrams, but don’t spend too much time creating custom diagrams that are specific to their business.
        • Solution Benefits: you should mention the benefits of implementing such a solution. This could be an increase in productivity, accuracy, automation etc.
        • Costing: this is the tough part that most people get wrong. Only through experience will it become easier to prepare costing estimates.
          • Development Costs: firstly, you need to think of the time required to develop the software by breaking down the tasks. If you’re using XP and Agile, these could be your stories or whatever you like to call them. If you’ve done similar projects in the past it will be easy to make an estimate. However for features that you’ve never implemented before, I would advise you to always be conservative i.e. think about the time required for research, testing debugging etc. If you’re not the one developing the software, this process will be become even more difficult in which case you will need to speak to the developers and get estimates from them.
            • Developer Confidence: Keep in mind that you cannot just take the developers’ estimates at face value. The reason being that some developers provide optimistic estimates, while others provide pessimistic estimates and this is based on their confidence, which may or may not necessarily have anything to do with their actual skill level. Confident developers typically tend to under estimate the time and effort required for coding, while under confident developers will overestimate time and effort.
            • Code Sloppiness: Some developers can be incredible quick to write code but they code can be sloppy. For such developers, you need to add buffer time for them to go back to refactor and debug etc. Others are perfectionists and take really long to code, but when their work is done it’s actually done. Therefore, you need to take all of this into account on the overall costing.
            • Buffer Time: Once you’ve come up with a fair estimate, it is best to always add a further buffer by multiplying the estimated hours with a factor. This factor is something of gut feeling you get through experience. For example, suppose a developer gives you an estimate of 10 days of development, you could then multiply that by 1.5, thereby your estimate to the customer being 15 days. The reason for this buffer is to take into account unexpected issues that can and do always pop up. Some may say that allocating extra time encourages procrastination, but I think that’s non-sense because it is the project manager’s job to check on up progress and ensure that people are meeting their timelines.
            • Do not quote on details: when quoting for development work, do not make the mistake of separating and displaying costs for every little coding task e.g. do not quote for each and every single button that they want on a form. Instead group the costs together and have only one or two development line items on the quote. The reasoning behind this is because you do not want the customer interrogating you about every single line item e.g. “Why would it take 8 hours to develop a single button.” The purpose of not quoting on details is not to be dishonest about the costs, but rather to avoid having technical debates with a customer that isn’t technical i.e. the customer has no idea how much work goes behind implementing the logic of that specific button and therefore to the customer it just looks like a simple button, while there could be 2000 lines of code behind that button for it to do what it’s supposed to do.
          • Project Management Costs: think about all the time that will be spent on the phone, writing emails, in meetings and driving around to those meetings. You must also take into account the kind of people the customers are when making this estimate. If your customers are professionals that know exactly what they want and they themselves plan ahead, then the project management costs can be reduced. However, you will have customers that know very little about software projects, are unprofessional, will constantly change their minds about what they want and the goal posts and timelines will constantly change. For unprofessional customers, I’d advise you to increase the project management costs substantially because you’re going to be doing a lot of baby sitting.
          • Analysis Costs: this is the time required for you to sit in workshops and meetings to discuss and finalise requirements and write up documentation. This time needs to be paid for because at least a third of your project will involve analysis work. It is for this reason that you should never make the mistake of starting the analysis phase before presenting the proposal and getting the customer’s go-ahead on the project.
        • ROI Analysis: finally once you’ve come up with a final number for the costing of a project, you then need to sell it against a value you’re providing to the customer’s business i.e. you need to give your customer a financial reason to spend the money on doing the project. Simply listing reasons is not going to cut it. Your motivation for doing the project needs to be have financials attached to it and that is typically the ROI (Return On Investment) calculation. In most cases, customers will already know why they want the software implemented, that being the reason why they called you in to take on the project. However, that may not always be the case, especially in cases when you are the one approaching them. In such cases you need to show the customer that doing the project will be an investment in their business and like any investment it must have a return/payback in a certain amount of time. So for example, if the project costs will be $100 000, you need to show the customer that the software will reduce costs in other areas of their business thereby enabling them to save X amount of money every month.  Those savings will amount to the total investment in a certain amount of time and that will be your final ROI payback period. Going forward the customer will continue to save for as long as they continue using your solution. In other words, the purpose of an ROI calculation is to establish a win-win deal with the customer whereby you earn money from the project implementation and the customer wins even more in savings over time. Selling based on ROI is the most difficult kind of sale you can make because you will need to have a thorough understanding of the customer’s business in order to put together an ROI calculation. However this kind of sale is also the most rewarding because instead of selling based on billable hours, you’re selling based on the value your software solution provides. It always helps if you are already familiar with the specific industry that the customer is in. If not, then you will be required to do a lot of research into the industry. Either way you will need to ask the customer a multitude of questions regarding their business before you can understand how to solve their problems and cut their costs.
      • Project inception: once you’ve presented the proposal to the customer you can expect to have a few backwards and forwards rounds of meetings and discussions about the proposal. Do not make the mistake of starting a project without the go-ahead from the customer. If it is a new customer that you’ve never dealt with in the past, I would advise you get a formal approval in written form from the customer. This can take the form of a PO (Purchase Order) or otherwise some sort of a signed contract.
      • Analysis and Specification Documents: it is only at this stage once you’ve gotten the go-ahead from the customer that you can start working. However, it is still not time to start coding just yet.
        • Project Plan: First and foremost you need to put a project plan together. The project plan will consist of the timelines of when each project deliverable (feature/work) will be completed and which resources (people) will be working on those deliverables. Both you and the customer need to agree on the project plan and the timelines need to be in line/comparable to the project costing in your proposal.
        • Functional and/or Technical Specifications: included in the project plan should be time allocated for analysis work and writing up the functional and/or technical specification. This phase of the project will involve having workshops and meetings to discuss the requirements of the software. Throughout this process, you will document the requirements in a formal specification document.  Many developers will say that documentation is a waste of time, but in the business world these documents will essentially server two purposes:
          • Contract for work required: The specifications will serve as a contract for what the customer can expect of the software. No more and no less than what is specified in the document will be delivered to the customer for the agreed upon price. It is for this reason that many software companies only quote the customer after the analysis phase, because they will say that they cannot quote for the project without fully understanding all the project requirements in detail. However, remember that I mentioned that in the costing process you should always add a buffer i.e. allocate extra budget and time for unexpected costs. This buffer that you added in your costing should cover any unexpected work that is being requested during the analysis phase and it will also protect you from doing the analysis phase at the risk of not getting paid for this work.
          • Mitigating scope creep: without a specification in place, the customer can continue asking for changes and you will end up spending way more time on the project than you initially anticipated all the while the customer is getting all this additional work for free.
      • Billing:  Many software companies bill only at the end of the project with an agreement in place that the customer only pays once the project has been completed and the solution has been implemented and tested. This is a huge mistake. The reason being that the customer will be under the impression that the project costs you presented in the proposal are a flat fee and in such a scenario the customer will not take into account the accruing costs of changing requirements, calling you in for additional meetings etc. All of that can land you in a mountain of scope creep costing your company ridiculous amounts of money. Hence it is important to:
        • Bill actual hours not estimated hours:  It is crucial to make the customer understand that the costing you did in the proposal are only estimated costs and that they will be billed the actual hours worked instead of these estimated hours. This clearly tells the customer that it will not be a flat fee for the project, but instead can change based on changes in the requirements. If you added a buffer to the budget (over estimated costs) in your proposal, the customer will end up being pleasantly surprised when the costs of the project are less than the estimated costs. However, if the customer continuously asks for changes throughout the project even after the specification document has been agreed on, these changes will be handled as CCRs (Change Control Requests) which fall outside the scope of the initial specification. The CCRs must be paid for and given enough of these CCRs the total cost of the project will end up going over the initial costing in the proposal. Having these mechanisms in place will encourage the customer to think twice before asking for changes to the initial agreement and will protect you against scope creep.
        • Bill in cycles: Bill the customer for the hours worked in continuous cycles e.g. monthly or even every two weeks. The developers and other resources on the project will then have to keep track of their hours worked and ensure that the costs are not going over budget. If the costs do end up running over budget, either because your initial estimation in the proposal was wrong or because of CCRs, the customer must be alerted of it and approval must be gotten from the customer before continuing with the project.

Keeping all of the above in mind, here’s a summary of golden rules to follow in order to manage the costs of a project.

  1. Always over quote and under bill instead of under quote and over bill.
  2. Do not get into too many details in the sales cycle i.e. when drawing up the proposal and estimating the costs of the project.
  3. Bill actual hours worked instead of estimated hours: Explain to customers that the quote is purely an estimation and that they will be billed actual hours worked rather than estimated hours. Explain to them that if there are no change requests, they can most likely expect the total cost to be less than the estimated cost in the initial proposal.
  4. Never work for free: aside from the sales meetings, proposal and costing, no other work should be done for free without formal approval from the customer. Even after the project has started, any change requests must be paid for.

Why do large software companies open-source their projects that they’ve invested so much into?

This is a good question, because it seems like a bit of a contradiction: these companies are there to make money and they’re spending money to develop these products but they’re still “giving” their code away for “free”.

Obviously I don’t know exactly what the thought process is by the executives of these companies, but I can take a guess. My guess is that open-source is great if you want to:

  • Lower costs: get others in the open-source community to work for free in testing and spotting bugs in your software.
  • Lower risk: this goes along with lowering costs i.e. lowering the costs lowers you investment risk. Furthermore, getting feedback from the open-source community is a lot quicker than having to wait for the software to be released and only thereafter get feedback from your users, by which time they might have created a negative impression with the end-users (if the software is full of bugs).
  • Increase adoption rate: most of the products being open-sourced are SDKs and other development tools. This is very beneficial to developers using those development tools i.e. because it gives the developers more freedom and control. In the long term they end up with more developers/users hooked on their platforms at a quicker rate. As an example, I’m guessing it’s one of the reasons Microsoft opted to finally open-source .NET because it would get a lot more developers hooked on their platform, programming languages and development tools, which ultimately results in more apps being developed using Microsoft technology. Once they’ve increased the number of apps on the market developed using .NET, I’m pretty sure Microsoft have plans to capitalise on that in one way or another.

When to open-source and when not to open-source

If you’re a developer and have no intentions of every getting paid for writing code, then deciding between open-sourcing your software or not can be a simple decision. However, if you have any intentions to get compensated for your work, then the decision to open-source or not can be a complicated and long winded debate which has gone on for decades. On the one side you have the capitalists/conservatives that sell software for a price and their pricing models are relatively simple to understand especially in commercial and non-enterprize environments i.e. you get this product for this price. On the other hand, you have the open-source socialists/liberals/anarchists, advocating for making software “open”, transparent and most of the time offering it for free. However, for anybody that has ever delved into the open-source debate, they will quickly realise that open-source is in fact not that clear and transparent after all.

It’s all good and well to spew out socialist rhetoric about how we should all be more altruistic by volunteering our time and skills to open and transparent software communities, but once we’ve all had our turn at singing kumba-ya, the truth is that in reality there’s no such thing as free lunch. Meaning that although many open-source enthusiasts pitch the idea of so called free software, the users will still end up paying for it one way or another, be it in some sort of fee, time, energy and/or frustration. The reason being that in the real world, time equals money and money makes the world go round. Therefore, any developer working on a “free” and open-source project still needs to pay the bills, put food on the table and at the very least still have money left over to buy a laptop on which to work on. So how do open-source developers get compensated? Open-source projects typically start off as side-line hobby projects, then perhaps moving on to offering paid for versions and/or charging for services.

Over the years I’ve thought about this debate countless number of times and like most analytical people I’ve searched far and wide for an absolute answer i.e. is open-source good or bad. When I was younger I leaned more towards the open-source point of view and their anarchist/hippie rhetoric about how corporations are evil, money-grubbing establishments designed to create monopolies and control people’s lives. Now that I’ve gotten older, my view has become a bit more conservative. Maybe it’s got something to do with there being a grain of truth in Winston Churchill’s wise words where he said that “any man who is under 30, and is not a liberal, has no heart; and any man who is over 30, and is not a conservative, has no brains.”

So here’s my attempt at an absolute answer as to when to open-source and when not to:

  • How most open-source projects start off: as the saying goes, “necessity is the mother of all inventions”. By that token, most open-source projects start out from a need/problem that someone has. A developer may want to achieve something that they cannot do with commercial software that is available or they just don’t have the money to purchase commercial software that can perform the given task. While developing this new app for their own needs, it suddenly dawns on the developer that other people may have the same need/problem that their new app can solve. At this point the developer begins to think about how they could possibly get others to use this new app and possibly capitalise on it. After careful consideration they realise that there’s more to selling software than simply putting a price tag on it i.e. a company needs to be setup, marketing needs to be done, payments need to be captured, orders and invoices need to be created, taxes need to be paid and amongst many other things, all sorts of people need to be hired e.g. marketing, sales, finance, support etc. Starting and managing a company requires a lot of work and responsibility. Due to the fact that a descent amount of capital is required, it also involves a huge amount of risk i.e. it’s not always easy to predict whether or not a software product will be a hit or not. So as opposed to going through that whole nightmare, our friendly developer decides to rather just upload his code to a popular open-source repository that all his friends can access. If nothing else, it will look good on their resume.
  • If the software is a hit: if there proves to be a substantial demand for the software, the developer may begin thinking about capitalising on it. At this point the developer will probably be stuck between a rock and a hard place. If he closes down the open-source repository and begins to charge for the software he’s fear is that the users will just continue using the latest open-source version available and never purchase any new versions. Even if there are some people willing to pay for the software, the problem is that the developer will never be able to sell it to enterprise customers because no large company will agree to adopting a product that is being supported by a single individual i.e. it would just be too risky for the company. So it’s a bit of a chicken and egg scenario i.e. you need to generate revenue to hire more people, but customers are hesitant to commit to a one man show. On the other hand if he keeps the open-source repository available, then there is no way to make money on it. To get out of this dilemma, most developers in this situation either begin selling services or offering paid for upgraded versions with additional features.
  • Charging for services approach: most open-source developers will write the code for free to develop the software, but if users need any help with specific feature requests, installations, advice or any consulting services, they will be charged an hourly rate or are sold a 1/3 year Service Level Agreement. As a bit of a realist, it is at this point that I start questioning the incentives put in place for this type of revenue stream. I ask myself, what incentives do open-source developers have to produce quality software that is easy install and use when their primary revenue stream comes from selling services? If charging for services is your only revenue stream on an open-source project, then there is a clear conflict of interest in this approach and I would advise against it. Reason being that what users want is software that is easy to install, configure and use, while at the same time the developers are well aware that the easier the software is to configure and use, the less services they will sell. So what you end up with, is open-source software that is counterintuitive and users are constantly left with more questions than answers. In this kind game with these kind of incentives in place, there is not much that is open and transparent to the users i.e. the software and code is available and free for anybody to use, but nobody knows how to set it up and us it. You may be of the opinion that it doesn’t matter whether there’s a conflict of interest or not, as long as you’re making some money off of your efforts. However, my argument would be that users are not stupid, they always know when there is a conflict of interest and sooner or later they will start posing questions. This approach also creates confusion that results in users asking why you’re doing some work for free while charging for other work. This is where open-source can lack transparency. With the above said, there are some more negative side effects with this approach:
    • Minimal documentation: the whole point of writing documentation is to make it easier for users to understand the process of installing, configuring and using a given piece of software. Given the fact that many open-source software is offered for free, the developers have almost no incentives to write comprehensive documentation i.e. the more difficult it is for a user to get up and running and troubleshoot issues themselves, the more services the developers can sell. What you naturally end up with is open-source software that often has little to no documentation aside from a simple two page quick start guide. The counter argument from open-source developers is that “the code is the documentation, just look at the code”. But if we cross over from the delusional hippie world into the real world, we realise that most users are not coders and therefore having the code adds zero value to their lives. Even as a coder myself, if I’m interested in using someone else’s software I will neither have the time nor the interest in doing support on other people’s products. Personally I just want to be able to click install, use the software and move on with my life because I have enough of my own code to deal with, without having to deal with other people’s broken code.
    • Lower quality of open-source software: in terms of quality, the argument made by open-source purists is that their software is of much higher quality than that of proprietary software. According to them this is due to the fact that open-source communities are larger and there are more people testing, spotting bugs and fixing them as opposed to small teams that develop proprietary software. This is based in Linus Torvalds’ law that “given enough eyeballs, all bugs are shallow”. That does indeed sound plausible, but on the other hand it seems to me like a bit of a contradiction to the many open-source companies that rely on services as their primary revenue stream. If the quality of open-source software is so incredibly high, then how do they make their money on services i.e. if there are no bugs to fix, the apps are easy to use and everything just works, then what kind of services are these developers selling to pay the bills? Developers creating proprietary software on the other hand have every incentive to write high quality code and create intuitive user interfaces, because if they don’t, then nobody will buy their software.
  • Paid for upgraded versions approach: an alternative to only offering services is to develop additional features into the open-source software and offer those features at a premium i.e. offering a “free” open-source community version as well as a paid for version with additional features that are enticing enough that make users want to spend money. Open-source can be fantastic when you want to lower your risk while at the same time increase the user adoption rate of your software. By open-sourcing your software you can start a project without having to take on the risk of starting a company, investing capital etc. and if there proves to be a high enough demand for your software, you can then take it to the next level by offering additional features and charging for them. This is the only time I would see open-source as good idea i.e. assuming that you actually want be compensated for your work.

In summary, before open-sourcing your software you should consider the number of features that you have in mind and whether or not there will ultimately be a demand for those features on the market. There are many apps that can be incredibly useful, but beyond their basic features there are a limited number of additional features that can be developed and incorporated into future paid for versions. A great example would an SDK (Software Development Kit) to interface with third party application or hardware: there are only so many features you can add to such an SDK. This kind of software with limited useful features should either be open-sourced without any intention of ever making money or it should never be open-sourced, but rather be released and sold as proprietary software right from the start.

Technical engineering vs social engineering in a software project

I started off my career in a product development environment; I sat behind a desk all day every day writing code. The only work related discussions I ever had to have was with the CTO (Chief Technical Officer). He told me what to do and how to do it. I was incredibly lucky to have fallen under his wing, because he was an incredibly talented developer and manager with years of experience. Not only did I learn a lot from him but I could always count on the fact that he always had answer to any problem I was facing.

However, in retrospect I realise I was working in a sheltered cocoon. Wanting to explore other opportunities I later moved on to a software consulting company. Soon after having started as a software consultant, I realised that I was in over my head. Interestingly enough, the technical challenges I was facing were relatively trivial compared to what I was previously doing. At the time I couldn’t for the life of me understand why I was struggling. Through experience, I later learned that the problems were not technical in nature, but they were rather social engineering problems i.e. I wasn’t in charge of managing the people on the project and even if I was, I wouldn’t have known how to manage them.

Every software project will always be made up of three major components:

  • Problem: at the core of every project is an underlying necessity for a problem to be solved.
  • People: there are expectations that are set out in every project by the stakeholders. These expectations are essentially agreements as to how a solution will be developed for the above problem to be solved. The stake holders involved in setting out the expectations are mostly made up of two crowds of people.
    • People requesting the software solution.
    • People providing the software solution.
  • Solution: once the expectations have been set out, they need to be met by the software developers i.e. developing the software that solves the defined problem according to the expectations that have been set out.

Most inexperienced software developers and business/sales people often believe that developing software for a customer/investor is all about the technical solutions that need to be implemented to make the customer happy. These kinds of people often accept the word of the customer as the word of God i.e. they walk around saying “the customer is always right” or the “customer is King”. Personally I believe they do this because they either lack the experience, a backbone and/or foresight. Hence they are “yes people” that often say yes to whatever the customer asks for, no matter how ridiculous/incorrect these requests may be and without considering the big picture. Hence they become a puppet being played like a ping pong ball by the customer’s stakeholders, and thereby ultimately ending up with a failed project on their hands.

Part of the reason this happens is because as kids, most of us are brought up to think that our superiors (parents) are always right and that they have our best interests at heart. When we enter the working world, we tend to see our managers and/or customers as our new superiors because they’re paying paying us. Therefore our natural instinct tells us to believe the same principle applies to them i.e. that the customer is always right. However, in the real world a consultant is hired to recommend and provide solutions, not to just simply take instructions. Simply put, the customer is not your parent.

Furthermore, as the name implies, each stakeholder has their own individual interests and incentives that motivate them. Whenever there are a multitude of varying interests, you can bet that there will always be conflicts of interests. Hence it would be foolish to allow the stakeholders to control the expectations and proposed solutions, thereby controlling the project itself. Your job as a consultant is to understand the needs of the businesses, the needs of each stakeholder, what motivates them and what their incentives are, so that you can gather enough information to make your own proposal for a solution. The consultant must be the one in the driver’s seat of every project.

Simply put: the consultant should never ask what needs to be done, but instead tell the customer what needs to be done and how it will be done based on the problem at hand. This means that at least 50% of the consultant’s work involves setting the right expectations, thereby requiring social engineering to be applied before a single line of code is written.

So what is social engineering in the context of software development?” Here a few points on this:

  • Understanding the customer’s business:
    • A customer will often request for technical solution to solve what is often only a simple operations problem. For example; in a warehouse they may request for software with a simpler and more intuitive process flow to be used by employees handling stock on the warehouse floor. Doing so may increase the productivity by 10-15%, but given the fact that according to statistics a warehouse worker spends 50% of their time walking to different locations, the more serious problem at hand are the long distances between the bin locations, rather than the process of data processing on a mobile device. In this example, software cannot solve the problem of the physical distances between bin locations, instead it can only optimise the existing processes which may be inherently flawed. As a consultant you need to understand these nuances and draft your proposal accordingly.
  • ROI (Return On Investment) & Budget: keeping in mind that that the primary aim of most businesses is to make money, the need for most software projects comes down to a necessity to optimise current processes by cutting costs and increasing productivity. Hence, any recommendations should be applicable to the customer’s business needs i.e. do not make recommendations to develop features just because it would be fun for you to develop them.
    • ROI: any feature to be developed, needs a cost associated with it and an ROI to be proven i.e. the customer is paying X amount to have this feature developed, therefore how much money will they be saving and how long will it take before the feature is paying for itself?
    • Budget: you also need to keep in mind that although you may be offering a great solution and a great ROI, the customer you’re proposing the solution to may not always have the budget to invest in it. This applies to any sort of business. For example, someone may present you with a bargain to purchase a block of flats that you can rent out and get a fantastic ROI, but if you don’t have the budget to make the investment, it really doesn’t matter how great the solution/business opportunity is. Hence, you should never be shy to discuss the budget with a customer upfront. This will prevent you and the customer from wasting each other’s time.
  • Understand your audience: IT staff for example, are more concerned with the network and IT infrastructure of your proposed solution than the usability or how it improves the business. Finance people are more concerned with the costs and ROI than with the actual features being offered by your solution. Operations people are more concerned with efficiency of the solution rather than the costs or technical architecture. Finally, end users are more concerned with the features and usability of the solution than with any of the above. You need to pay careful attention to who you are talking to at any given time as well as what you say, how you say it and what questions you ask.
  • Managing expectations: customers often do not understand the implications of developing software, such as the amount of effort and time that will be required. Based on that you should not allow the customer to dictate how long something should take and how much effort will go into it. As a software developer, it is your job to inform the customers of the above and be firm about it. They must accept your estimations or go somewhere else. Needless to say that the estimations and expectations you set should be reasonable and some flexibility (within get reason) on your end will prove to go a long way.
  • Getting paid: finally you need to worry about when and how much you will get paid. You will come across many customers that expect you to work for free with only a promise or a maybe of getting paid. Therefore it is imperative that you address this question with the customer right from the start in order to set the right expections. Also keep in mind that many customers that deal with software consultants are often under the impression that they are buying software, which couldn’t be further from the truth i.e. they are not buying software, they are buying your time and expertise which you need to make crystal clear to them right from the start.

To summarize: a software project is more than just someone telling you what feature to develop and then coding it. Therefore it is imperative that you understand the social engineering aspects so that you can manage the customer instead of the customer managing you. With the above said in mind, half the project is performed in boardroom discussions and over emails and phone calls, not just behind a computer simply writing pretty code.

Will software developers continue to be in high demand in the future?

These are some of the trends that I have seen over the years:

  • Data agnostics software: if you’ve ever spent time jumping from project to project developing custom software for various customers’ requirements, you will pretty quickly come to the conclusion that you’re basically developing the same software over and over again and the reason you’re doing it is because each customer has different business rules. So to make your own life easier, you will inevitably start thinking about removing the business logic out of the code and making the software more and more customisable thereby making it data agnostic so that it doesn’t know anything about the data its working with. By developing data agnostic software, you are basically handing over the power and responsibility to your customers enabling them to implement the business rules themselves instead of relying on you to change code every time they change their business rules. Doing this is all good and great for the customer and even for your own company, because you can then resell your software to many more customers without having to code business logic for each new customer. However, the problem is that other competing software companies that are still coding custom business logic will be blown out of the water by you i.e. their customers will now rather just buy your out-of-the-box and customisable software from. The end result being that there will be less and less demand for software developers building custom business applications. That is why for example most corporates prefer implementing large systems like ERP, CRM, WMS, CMS etc. that are trusted and have been proven to work as opposed to developing their own system from scratch. Although these large software systems are more and more business logic and data agnostic, for the time being there will still be a need to have technical people installing and customising these “out-of-the-box” products. However the “technical” people required to do so will be less and less technical i.e. less and less technical skills will be required to perform customisations for each customer.
    • Example: back in the 1990s, if you wanted a simple company website you would have hired a web developer to put together a few HTML pages i.e. Home, About, Contact Us page etc. At some point, some clever guys decided to develop a CMS (Content Management System), which is exactly that: a data agnostic piece of software that doesn’t know or care about what content you’ve got, but it gives you the power to configure it yourself. You would still need an IT guy to perform the configuration and handle the hosting, but you no longer needed a web developer.
  • SAAS (Software As A Service) & PAAS (Platform As A Service) in the cloud: to make matters worse (or better , depending on your perspective), these large systems (ERP, CRM, WMS, CMS etc.) are now being moved into the cloud and offered as a service i.e. monthly payments to access the software online. That means that business people no longer even need technical IT staff to manage the hosting or to install and configure anything because that is now handled by the SAAS providers. Thus, once again making more technical people redundant.
    • Example: at this point the customer no longer even needs a CMS hosted on their own server. Instead they can just create their own website on wordpress.com without having any technical knowledge. If integration with a payment portal is still too difficult, they can even use Facebook Store or shopify.com.
  • Platform agnostic software: I started my career in mobile development, back when the popular operating systems available to develop for were Symbian and Windows Mobile/CE. If someone wanted a mobile app developed, they would have needed to hire a developer to code the app from scratch. Coding an app for Symbian was incredibly difficult with a 6–12 month learning curve, Hence requiring highly skilled Symbian C++ developers. In the company that I was working for, we were developing a .NET Compact Framework to run on Symbian, thereby allowing less skilled .NET developers to write code targeting the .NET Compact Framework and thereafter run that same app on Symbian. The very same people that I was working for ended up starting another company (devicemagic.com), allowing people with limited technical skills to put together a mobile app that will run on any of the popular operating systems like Android and iOS. Once again, if your requirements for a mobile app are relatively simple (data capturing, taking pictures etc.) then you no longer to hire expensive iOS, Android or .NET developers to build the app for you.
  • AI (Artificial Intelligence): at this stage AI is still a baby, but the baby is growing. Once fully grown it will further exacerbate the situation where we might see autonomous software writing code by itself based on your specifications. With AI programming languages will not even be needed anymore because the only reason programming languages exist is to enable humans to define the execution of a program. If the machine is generating the code, it will simply generate binary code.

The moral of the story being that it’s survival of the fittest i.e. the big fish will continue to get bigger by eating the smaller fish. The name of the game is consolidation; of technology, money and power. But to be fair humans have been playing this game since the beginning of time to the point where the little people on the ground get fed up and come out with their pitchforks, after which war breaks out, people die, everything gets destroyed and the survivors start rebuilding everything from scratch thereby starting the cycle all over again. But even knowing this I still can’t stop myself from recoding that function in my code to remove the business logic so that I can reuse it with my next customer.

To answer your question: as long as software still exists in this world, there will always be a need for software developers, but personally I think that the demand will drop in the long term. In a few decades, only the most talented developers will have jobs and they will most likely be working for the big fish, like Microsoft, Apple, Google, Oracle, SAP etc.

Knowing this, what can you do about it if you’re currently a software developer? The answer is; not much … except enjoy the proverbial gravy train that you’re currently on.

In the interim: focus on the following caveats of the above mentioned trends/technologies:

  • Security & Trust: there are still plenty of companies out there that are hesitant to move their data and infrastructure to the cloud. This especially applies to financial institutions which hate the idea of putting their money (numbers) into the cloud hosted by a third party. Their concerns being centred around the security of their data and whether or not they can trust the cloud hosting providers with their data/money.
  • Control: many of these companies are still run by control freaks that want highly customised software that works exactly the way they wanted to work. They will never be able to get that with so called “out-of-the-box” solutions. Thus they will still require highly skilled developers. For how long … only time will tell.

For the future: plan ahead by deciding between two different paths:

  • Technical: if you decide to stay the course and focus on being technical, then you better make sure that you are part of the best of the best. Average won’t cut it if you’re planning on working for one of a handful of tech companies in the world. Keep in mind that age will catch up with you sooner or later, and competing with twenty something year old guys that have no family, commitments or a life for that matter, will prove to be incredibly difficult.
  • Business: alternatively, you can become more business minded, worrying less about the details of software and more about how to sell it or manage the people implementing it.

Not all software companies are in the software business

Many people have the misconception that a software business is purely about developing software i.e. that as long as code is being written that is valuable to someone else, then money can be made. This may be partly true, but it is a very narrow sort of view on what it takes to generate revenues.

The question of “what are we selling and when should we request payment?” determines the type of software business you are working for and/or managing. There are three main types of software businesses:

  1. Software as a means to an end: there are many businesses that dedicate the majority of their resources (money, people, time) to developing software but they do not directly generate money from the the software itself. Instead the software they develop is nothing more than a means to an end to support and/or automate their primary business, which may be anything from marketing/advertising, to selling physical products or perhaps managing logistics. The quality of the software produced by these companies has less impact on their profits than the primary service/products that they provide. Here’s a few examples of large corporations already doing this:
    1. Google may for example develop software as part of their business strategy, but they are primarily just an advertising agency and they just happened to have found a way to reach a wider audience by developing a search engine. More recently they have started making money on Google Apps but it pales in comparison to their revenues on advertising through their search engine. They have indeed developed Android, the most successful mobile OS (Operating System) in the world, but as far I know they make a minimal amount of profit on it, while Microsoft gain the majority of the profits which Google have to pay out to them for royalties. Customers that advertise on Google care less about the features Google provides than the ROI (Return On Investment) they get from advertising on their search engine. On the flip side, the people using the Google search engine do care about the quality of the search results, but those people are not paying to use the software.
    2. Facebook too is nothing more than an advertising agency that have just happened to find a different way of reaching a wider world wide audience. They do this by attracting people to a social networking website, keeping them engaged and gathering information about their interests (likes). Once again, the paying customers don’t care so much about the features available on the software, as long as their advertisements are reaching the target audience. Facebook is not exactly in the business of developing software as it is in the business of connecting people and feeding them targeted advertisements.
    3. Amazon for the most part is nothing more than a shop/store that happens to be online and they have thereby managed to automate the process of selling products to a wider world wide market. Only as of more recently have they started selling software products and services such as AWS (Amazon Web Services), but this is not their core business. Again, the features provided by the Amazon website have little to do with their customer satisfaction; their customers care about the products available on the website and how quickly those products can be delivered to them. Therefore Amazon is not so much  in the business of selling software as it is in the business of selling and delivering physical products.
    4. Uber is simply a company that acts as a middleman managing taxis. They may as well be a personal assistant that helps you coordinate the travelling via a taxi: contacting the taxi driver, telling you where to meet, keeping track of distances travelled and of course keeping track of the costs. They just happened to find a way of automating this whole administration head ache by developing software. The customers using Uber once again don’t care so much about the features on the software, as long as they get a taxi to take from the A to B at an affordable price.
    5. Apple: is in fact a hardware company that just happens to have developed their own operating systems and developer tools as a strategic way of improving and controlling the performance of their hardware and keeping their users within their ecosystem. The same applies to every other OEM (Original Equipment Manufacturer): they may have internal software engineers, but they are not on the frontline of what keeps the company going.
  2. Software Consulting: a consulting business charges for services rendered, meaning that you’re not selling software but rather your time, skills and effort. The consulting business is very much like prostitution: a salesperson (pimp) sells the services of software developers (prostitutes) to customers and theses services are billed by the hour. A software consulting business is not purely in the business of developing software, but rather in the business of providing services and adding value to customers.
  3. Product Development: traditionally software was regarded as any other engineering endeavour: we create a product, we sell as many units of it as we possibly can and make lots of money. Microsoft is a great example and the original company to have implemented this strategy.  These kinds of companies are the only companies that exist purely for the purpose of developing software. The challenge these companies have is that they will not be making any money until the first product is finished and ready to be shipped off. Therefore it requires massive funding and involves great amount of risk. With this approach, customers are purchasing the software products for no other reason than for the software itself and the features it provides. Therefore in order for such a company to survive it needs to provide finished software products that are of the highest quality and offer the largest amount of features.

The moral of the story is that the activities of a company do not always have a direct impact on the bottom line, and thus do not necessarily dictate the the kind of business that a company is in i.e. employees can be spending 90% of their time developing an advertising platform, but if the money is made from selling advertisements, then the company is not in the business of writing software, but in the business of selling advertisements.

The kind of work needed to make money in the tech industry

The fundamental difference between poor people and rich people is that poor people believe that the harder they work the more money they will make, while the rich know that they have to rely on leverage to to make the big money. Leverage means either relying on others to do the work for you or earning interest on investments or earning commission.

  1. Other people’s work: if you’re at the beginning of your career it will take you years to get to top managerial positions where you get to leverage off of other people’s work. Not to mention that not everybody is capable of getting to that level – you have to be the best of the best.
  2. Investments: many makes money, so if you don’t already have a lot of it to invest it will take you years before you earn that kind of money on interest/dividends.
  3. Commission: this leaves you with the last option; start selling IT solutions if that’s what turns you on. If you’re a passionate technologist, then your first obstacle will be suppressing and overcoming your passion for technology (or whatever else you love), meaning that you will have to prioritise money over everything else. It’s easier to do that if you’re not particularly good at what you you’re currently doing. That is typically why they say that to become a salesman you have to fail at everything else first. That’s not to say that sales isn’t interesting or challenging, it is, but you have to have a certain aptitude for it: you have to be a risk taker, good with people and an exceptional strategist to make the really big bucks i.e. that’s the difference between a guy selling sun glasses on the side of the road and a successful corporate salesman. Examples:
    1. Steve Jobs is a prime example of such a man that wasn’t that good as an engineer but he was a genius at sales and marketing. He managed to get others to build solutions for him and he would sell it on to the world.
    2. Ideally you should be both an exceptional engineer and salesman, but that’s very rare. Bill Gates is such an example and because of it he and Microsoft had an almost complete monopoly in the software industry, across the world, while he was in charge.
    3. Linus Torvalds on the other hand was the exact opposite of Steve Jobs; he’s probably the best coder that’s ever lived but he’s never cared about money and is not much of a people person, hence he has not made a fraction of the money that either Steve Jobs or Bill Gates made … but he’s made enough from what I’ve heard.
    4. Mark Zuckerberg is a bit of an anomaly which has made him a hero to pure technologists around the world: he’s an exceptional coder but not a salesman (if The Social Network movie is anything to go by). However he was at the right time at the right place, has a passion for connecting people and very smart to leverage off of exceptional salesman like Sean Parker, Peter Thiel etc. without letting them get the upper hand.

Moral of the story: get into sales if you want to make the big bucks, but if you love technology more than money you probably won’t make that much of it in your life … which isn’t necessarily a terrible thing as long as you enjoy what you’re doing.