Technical engineering vs social engineering in a software project

I started off my career in a product development environment; I sat behind a desk all day every day writing code. The only work related discussions I ever had to have was with the CTO (Chief Technical Officer). He told me what to do and how to do it. I was incredibly lucky to have fallen under his wing, because he was an incredibly talented developer and manager with years of experience. Not only did I learn a lot from him but I could always count on the fact that he always had answer to any problem I was facing.

However, in retrospect I realise I was working in a sheltered cocoon. Wanting to explore other opportunities I later moved on to a software consulting company. Soon after having started as a software consultant, I realised that I was in over my head. Interestingly enough, the technical challenges I was facing were relatively trivial compared to what I was previously doing. At the time I couldn’t for the life of me understand why I was struggling. Through experience, I later learned that the problems were not technical in nature, but they were rather social engineering problems i.e. I wasn’t in charge of managing the people on the project and even if I was, I wouldn’t have known how to manage them.

Every software project will always be made up of three major components:

  • Problem: at the core of every project is an underlying necessity for a problem to be solved.
  • People: there are expectations that are set out in every project by the stakeholders. These expectations are essentially agreements as to how a solution will be developed for the above problem to be solved. The stake holders involved in setting out the expectations are mostly made up of two crowds of people.
    • People requesting the software solution.
    • People providing the software solution.
  • Solution: once the expectations have been set out, they need to be met by the software developers i.e. developing the software that solves the defined problem according to the expectations that have been set out.

Most inexperienced software developers and business/sales people often believe that developing software for a customer/investor is all about the technical solutions that need to be implemented to make the customer happy. These kinds of people often accept the word of the customer as the word of God i.e. they walk around saying “the customer is always right” or the “customer is King”. Personally I believe they do this because they either lack the experience, a backbone and/or foresight. Hence they are “yes people” that often say yes to whatever the customer asks for, no matter how ridiculous/incorrect these requests may be and without considering the big picture. Hence they become a puppet being played like a ping pong ball by the customer’s stakeholders, and thereby ultimately ending up with a failed project on their hands.

Part of the reason this happens is because as kids, most of us are brought up to think that our superiors (parents) are always right and that they have our best interests at heart. When we enter the working world, we tend to see our managers and/or customers as our new superiors because they’re paying paying us. Therefore our natural instinct tells us to believe the same principle applies to them i.e. that the customer is always right. However, in the real world a consultant is hired to recommend and provide solutions, not to just simply take instructions. Simply put, the customer is not your parent.

Furthermore, as the name implies, each stakeholder has their own individual interests and incentives that motivate them. Whenever there are a multitude of varying interests, you can bet that there will always be conflicts of interests. Hence it would be foolish to allow the stakeholders to control the expectations and proposed solutions, thereby controlling the project itself. Your job as a consultant is to understand the needs of the businesses, the needs of each stakeholder, what motivates them and what their incentives are, so that you can gather enough information to make your own proposal for a solution. The consultant must be the one in the driver’s seat of every project.

Simply put: the consultant should never ask what needs to be done, but instead tell the customer what needs to be done and how it will be done based on the problem at hand. This means that at least 50% of the consultant’s work involves setting the right expectations, thereby requiring social engineering to be applied before a single line of code is written.

So what is social engineering in the context of software development?” Here a few points on this:

  • Understanding the customer’s business:
    • A customer will often request for technical solution to solve what is often only a simple operations problem. For example; in a warehouse they may request for software with a simpler and more intuitive process flow to be used by employees handling stock on the warehouse floor. Doing so may increase the productivity by 10-15%, but given the fact that according to statistics a warehouse worker spends 50% of their time walking to different locations, the more serious problem at hand are the long distances between the bin locations, rather than the process of data processing on a mobile device. In this example, software cannot solve the problem of the physical distances between bin locations, instead it can only optimise the existing processes which may be inherently flawed. As a consultant you need to understand these nuances and draft your proposal accordingly.
  • ROI (Return On Investment) & Budget: keeping in mind that that the primary aim of most businesses is to make money, the need for most software projects comes down to a necessity to optimise current processes by cutting costs and increasing productivity. Hence, any recommendations should be applicable to the customer’s business needs i.e. do not make recommendations to develop features just because it would be fun for you to develop them.
    • ROI: any feature to be developed, needs a cost associated with it and an ROI to be proven i.e. the customer is paying X amount to have this feature developed, therefore how much money will they be saving and how long will it take before the feature is paying for itself?
    • Budget: you also need to keep in mind that although you may be offering a great solution and a great ROI, the customer you’re proposing the solution to may not always have the budget to invest in it. This applies to any sort of business. For example, someone may present you with a bargain to purchase a block of flats that you can rent out and get a fantastic ROI, but if you don’t have the budget to make the investment, it really doesn’t matter how great the solution/business opportunity is. Hence, you should never be shy to discuss the budget with a customer upfront. This will prevent you and the customer from wasting each other’s time.
  • Understand your audience: IT staff for example, are more concerned with the network and IT infrastructure of your proposed solution than the usability or how it improves the business. Finance people are more concerned with the costs and ROI than with the actual features being offered by your solution. Operations people are more concerned with efficiency of the solution rather than the costs or technical architecture. Finally, end users are more concerned with the features and usability of the solution than with any of the above. You need to pay careful attention to who you are talking to at any given time as well as what you say, how you say it and what questions you ask.
  • Managing expectations: customers often do not understand the implications of developing software, such as the amount of effort and time that will be required. Based on that you should not allow the customer to dictate how long something should take and how much effort will go into it. As a software developer, it is your job to inform the customers of the above and be firm about it. They must accept your estimations or go somewhere else. Needless to say that the estimations and expectations you set should be reasonable and some flexibility (within get reason) on your end will prove to go a long way.
  • Getting paid: finally you need to worry about when and how much you will get paid. You will come across many customers that expect you to work for free with only a promise or a maybe of getting paid. Therefore it is imperative that you address this question with the customer right from the start in order to set the right expections. Also keep in mind that many customers that deal with software consultants are often under the impression that they are buying software, which couldn’t be further from the truth i.e. they are not buying software, they are buying your time and expertise which you need to make crystal clear to them right from the start.

To summarize: a software project is more than just someone telling you what feature to develop and then coding it. Therefore it is imperative that you understand the social engineering aspects so that you can manage the customer instead of the customer managing you. With the above said in mind, half the project is performed in boardroom discussions and over emails and phone calls, not just behind a computer simply writing pretty code.

Will software developers continue to be in high demand in the future?

These are some of the trends that I have seen over the years:

  • Data agnostics software: if you’ve ever spent time jumping from project to project developing custom software for various customers’ requirements, you will pretty quickly come to the conclusion that you’re basically developing the same software over and over again and the reason you’re doing it is because each customer has different business rules. So to make your own life easier, you will inevitably start thinking about removing the business logic out of the code and making the software more and more customisable thereby making it data agnostic so that it doesn’t know anything about the data its working with. By developing data agnostic software, you are basically handing over the power and responsibility to your customers enabling them to implement the business rules themselves instead of relying on you to change code every time they change their business rules. Doing this is all good and great for the customer and even for your own company, because you can then resell your software to many more customers without having to code business logic for each new customer. However, the problem is that other competing software companies that are still coding custom business logic will be blown out of the water by you i.e. their customers will now rather just buy your out-of-the-box and customisable software from. The end result being that there will be less and less demand for software developers building custom business applications. That is why for example most corporates prefer implementing large systems like ERP, CRM, WMS, CMS etc. that are trusted and have been proven to work as opposed to developing their own system from scratch. Although these large software systems are more and more business logic and data agnostic, for the time being there will still be a need to have technical people installing and customising these “out-of-the-box” products. However the “technical” people required to do so will be less and less technical i.e. less and less technical skills will be required to perform customisations for each customer.
    • Example: back in the 1990s, if you wanted a simple company website you would have hired a web developer to put together a few HTML pages i.e. Home, About, Contact Us page etc. At some point, some clever guys decided to develop a CMS (Content Management System), which is exactly that: a data agnostic piece of software that doesn’t know or care about what content you’ve got, but it gives you the power to configure it yourself. You would still need an IT guy to perform the configuration and handle the hosting, but you no longer needed a web developer.
  • SAAS (Software As A Service) & PAAS (Platform As A Service) in the cloud: to make matters worse (or better , depending on your perspective), these large systems (ERP, CRM, WMS, CMS etc.) are now being moved into the cloud and offered as a service i.e. monthly payments to access the software online. That means that business people no longer even need technical IT staff to manage the hosting or to install and configure anything because that is now handled by the SAAS providers. Thus, once again making more technical people redundant.
    • Example: at this point the customer no longer even needs a CMS hosted on their own server. Instead they can just create their own website on wordpress.com without having any technical knowledge. If integration with a payment portal is still too difficult, they can even use Facebook Store or shopify.com.
  • Platform agnostic software: I started my career in mobile development, back when the popular operating systems available to develop for were Symbian and Windows Mobile/CE. If someone wanted a mobile app developed, they would have needed to hire a developer to code the app from scratch. Coding an app for Symbian was incredibly difficult with a 6–12 month learning curve, Hence requiring highly skilled Symbian C++ developers. In the company that I was working for, we were developing a .NET Compact Framework to run on Symbian, thereby allowing less skilled .NET developers to write code targeting the .NET Compact Framework and thereafter run that same app on Symbian. The very same people that I was working for ended up starting another company (devicemagic.com), allowing people with limited technical skills to put together a mobile app that will run on any of the popular operating systems like Android and iOS. Once again, if your requirements for a mobile app are relatively simple (data capturing, taking pictures etc.) then you no longer to hire expensive iOS, Android or .NET developers to build the app for you.
  • AI (Artificial Intelligence): at this stage AI is still a baby, but the baby is growing. Once fully grown it will further exacerbate the situation where we might see autonomous software writing code by itself based on your specifications. With AI programming languages will not even be needed anymore because the only reason programming languages exist is to enable humans to define the execution of a program. If the machine is generating the code, it will simply generate binary code.

The moral of the story being that it’s survival of the fittest i.e. the big fish will continue to get bigger by eating the smaller fish. The name of the game is consolidation; of technology, money and power. But to be fair humans have been playing this game since the beginning of time to the point where the little people on the ground get fed up and come out with their pitchforks, after which war breaks out, people die, everything gets destroyed and the survivors start rebuilding everything from scratch thereby starting the cycle all over again. But even knowing this I still can’t stop myself from recoding that function in my code to remove the business logic so that I can reuse it with my next customer.

To answer your question: as long as software still exists in this world, there will always be a need for software developers, but personally I think that the demand will drop in the long term. In a few decades, only the most talented developers will have jobs and they will most likely be working for the big fish, like Microsoft, Apple, Google, Oracle, SAP etc.

Knowing this, what can you do about it if you’re currently a software developer? The answer is; not much … except enjoy the proverbial gravy train that you’re currently on.

In the interim: focus on the following caveats of the above mentioned trends/technologies:

  • Security & Trust: there are still plenty of companies out there that are hesitant to move their data and infrastructure to the cloud. This especially applies to financial institutions which hate the idea of putting their money (numbers) into the cloud hosted by a third party. Their concerns being centred around the security of their data and whether or not they can trust the cloud hosting providers with their data/money.
  • Control: many of these companies are still run by control freaks that want highly customised software that works exactly the way they wanted to work. They will never be able to get that with so called “out-of-the-box” solutions. Thus they will still require highly skilled developers. For how long … only time will tell.

For the future: plan ahead by deciding between two different paths:

  • Technical: if you decide to stay the course and focus on being technical, then you better make sure that you are part of the best of the best. Average won’t cut it if you’re planning on working for one of a handful of tech companies in the world. Keep in mind that age will catch up with you sooner or later, and competing with twenty something year old guys that have no family, commitments or a life for that matter, will prove to be incredibly difficult.
  • Business: alternatively, you can become more business minded, worrying less about the details of software and more about how to sell it or manage the people implementing it.

Not all software companies are in the software business

Many people have the misconception that a software business is purely about developing software i.e. that as long as code is being written that is valuable to someone else, then money can be made. This may be partly true, but it is a very narrow sort of view on what it takes to generate revenues.

The question of “what are we selling and when should we request payment?” determines the type of software business you are working for and/or managing. There are three main types of software businesses:

  1. Software as a means to an end: there are many businesses that dedicate the majority of their resources (money, people, time) to developing software but they do not directly generate money from the the software itself. Instead the software they develop is nothing more than a means to an end to support and/or automate their primary business, which may be anything from marketing/advertising, to selling physical products or perhaps managing logistics. The quality of the software produced by these companies has less impact on their profits than the primary service/products that they provide. Here’s a few examples of large corporations already doing this:
    1. Google may for example develop software as part of their business strategy, but they are primarily just an advertising agency and they just happened to have found a way to reach a wider audience by developing a search engine. More recently they have started making money on Google Apps but it pales in comparison to their revenues on advertising through their search engine. They have indeed developed Android, the most successful mobile OS (Operating System) in the world, but as far I know they make a minimal amount of profit on it, while Microsoft gain the majority of the profits which Google have to pay out to them for royalties. Customers that advertise on Google care less about the features Google provides than the ROI (Return On Investment) they get from advertising on their search engine. On the flip side, the people using the Google search engine do care about the quality of the search results, but those people are not paying to use the software.
    2. Facebook too is nothing more than an advertising agency that have just happened to find a different way of reaching a wider world wide audience. They do this by attracting people to a social networking website, keeping them engaged and gathering information about their interests (likes). Once again, the paying customers don’t care so much about the features available on the software, as long as their advertisements are reaching the target audience. Facebook is not exactly in the business of developing software as it is in the business of connecting people and feeding them targeted advertisements.
    3. Amazon for the most part is nothing more than a shop/store that happens to be online and they have thereby managed to automate the process of selling products to a wider world wide market. Only as of more recently have they started selling software products and services such as AWS (Amazon Web Services), but this is not their core business. Again, the features provided by the Amazon website have little to do with their customer satisfaction; their customers care about the products available on the website and how quickly those products can be delivered to them. Therefore Amazon is not so much  in the business of selling software as it is in the business of selling and delivering physical products.
    4. Uber is simply a company that acts as a middleman managing taxis. They may as well be a personal assistant that helps you coordinate the travelling via a taxi: contacting the taxi driver, telling you where to meet, keeping track of distances travelled and of course keeping track of the costs. They just happened to find a way of automating this whole administration head ache by developing software. The customers using Uber once again don’t care so much about the features on the software, as long as they get a taxi to take from the A to B at an affordable price.
    5. Apple: is in fact a hardware company that just happens to have developed their own operating systems and developer tools as a strategic way of improving and controlling the performance of their hardware and keeping their users within their ecosystem. The same applies to every other OEM (Original Equipment Manufacturer): they may have internal software engineers, but they are not on the frontline of what keeps the company going.
  2. Software Consulting: a consulting business charges for services rendered, meaning that you’re not selling software but rather your time, skills and effort. The consulting business is very much like prostitution: a salesperson (pimp) sells the services of software developers (prostitutes) to customers and theses services are billed by the hour. A software consulting business is not purely in the business of developing software, but rather in the business of providing services and adding value to customers.
  3. Product Development: traditionally software was regarded as any other engineering endeavour: we create a product, we sell as many units of it as we possibly can and make lots of money. Microsoft is a great example and the original company to have implemented this strategy.  These kinds of companies are the only companies that exist purely for the purpose of developing software. The challenge these companies have is that they will not be making any money until the first product is finished and ready to be shipped off. Therefore it requires massive funding and involves great amount of risk. With this approach, customers are purchasing the software products for no other reason than for the software itself and the features it provides. Therefore in order for such a company to survive it needs to provide finished software products that are of the highest quality and offer the largest amount of features.

The moral of the story is that the activities of a company do not always have a direct impact on the bottom line, and thus do not necessarily dictate the the kind of business that a company is in i.e. employees can be spending 90% of their time developing an advertising platform, but if the money is made from selling advertisements, then the company is not in the business of writing software, but in the business of selling advertisements.

The kind of work needed to make money in the tech industry

The fundamental difference between poor people and rich people is that poor people believe that the harder they work the more money they will make, while the rich know that they have to rely on leverage to to make the big money. Leverage means either relying on others to do the work for you or earning interest on investments or earning commission.

  1. Other people’s work: if you’re at the beginning of your career it will take you years to get to top managerial positions where you get to leverage off of other people’s work. Not to mention that not everybody is capable of getting to that level – you have to be the best of the best.
  2. Investments: many makes money, so if you don’t already have a lot of it to invest it will take you years before you earn that kind of money on interest/dividends.
  3. Commission: this leaves you with the last option; start selling IT solutions if that’s what turns you on. If you’re a passionate technologist, then your first obstacle will be suppressing and overcoming your passion for technology (or whatever else you love), meaning that you will have to prioritise money over everything else. It’s easier to do that if you’re not particularly good at what you you’re currently doing. That is typically why they say that to become a salesman you have to fail at everything else first. That’s not to say that sales isn’t interesting or challenging, it is, but you have to have a certain aptitude for it: you have to be a risk taker, good with people and an exceptional strategist to make the really big bucks i.e. that’s the difference between a guy selling sun glasses on the side of the road and a successful corporate salesman. Examples:
    1. Steve Jobs is a prime example of such a man that wasn’t that good as an engineer but he was a genius at sales and marketing. He managed to get others to build solutions for him and he would sell it on to the world.
    2. Ideally you should be both an exceptional engineer and salesman, but that’s very rare. Bill Gates is such an example and because of it he and Microsoft had an almost complete monopoly in the software industry, across the world, while he was in charge.
    3. Linus Torvalds on the other hand was the exact opposite of Steve Jobs; he’s probably the best coder that’s ever lived but he’s never cared about money and is not much of a people person, hence he has not made a fraction of the money that either Steve Jobs or Bill Gates made … but he’s made enough from what I’ve heard.
    4. Mark Zuckerberg is a bit of an anomaly which has made him a hero to pure technologists around the world: he’s an exceptional coder but not a salesman (if The Social Network movie is anything to go by). However he was at the right time at the right place, has a passion for connecting people and very smart to leverage off of exceptional salesman like Sean Parker, Peter Thiel etc. without letting them get the upper hand.

Moral of the story: get into sales if you want to make the big bucks, but if you love technology more than money you probably won’t make that much of it in your life … which isn’t necessarily a terrible thing as long as you enjoy what you’re doing.

Product vs Customer vs Profit

My personal philosophy is that in the software business there are three things that need to be focused on and their priority should be as follows:

Product and its source code: the features of the product and its quality determine how happy you make your customers. If your customer is happy he will purchase the product, which in turn means you have a profitable business. So one should always listen to the customer’s needs. Conversely, the product being yours means you as the founder know more about the product than anybody else, meaning you’re the boss of your own product. Therefore if a customer submits a feature request, it is just that, a “feature request” – not a direct order by the customer i.e. it is up to you as the founder of the product to decide whether the feature will be implemented or not and when it will be implemented because that feature or your prioritization of the feature could negatively impact another customer. That is why I believe the product should be prioritized ahead of the customer, which in the grand scheme of things will prove to be beneficial for both you and all the end customers even if you upset one or two customers along the way.

Customer: always prioritize the customer’s needs over making a quick buck by ripping them off: Many amateur sales people do a “hit & run” with a customer. However I feel that the best sales people have empathy for customers providing them with good deals… but not too much empathy that one gives things away for free or too cheaply.

Profit: personally I believe that money is just a by-product of one’s work and success. Focus on the work; do the work for the work itself because you enjoy it and you have a vision. This in turn will result in the money being made. If you’re going to run the comrades marathon, there is no way you are going to run all those kilometers if your only goal is to win the race so you can get a prize. With that mentality you will quit as soon as you realize you’re not in the lead. Instead if you run for the sake of running because you enjoy it, then you will finish the race and you will feel satisfied with yourself … that’s intrinsic motivation.