Anna Frazzetto's Blog

Digital Innovations and Technology Solutions

App Dev Productivity Measures: Can We Do More?

This week I had the pleasure and the challenge of hosting a Webinar on how businesses can set themselves up for success in an offshore engagement. If you know this blog, you know I love to talk about offshoring so my satisfaction in the event is easy to understand. The challenges came from the audience, a group of smart business leaders and professionals, who peppered me with thoughtful, tough questions.

I hate to pick favorites but one question from the event has stuck with me this week: What metrics/models can be used to accurately measure productivity of offshore/outsourced application development? I have worked in application outsourcing and offshoring so long that this struggle for clear development measures is nothing new in my world. Software development has long been famous for the inaccuracy in estimating time and costs, which has resulted in the adoption of numerous measurement tools, cost models and sizing standards. I immediately launched into an explanation of some of the many tools and processes Harvey Nash Offshore uses to measure and report on productivity across the software development life cycle:

Development Metrics--Measuring developer output in various ways, such as the number and quality of lines of code produced each day
QA Metrics--Measuring tester output by analyzing the number of errors found on a daily basis
Process Quality Assurance Team--A team in our offshore development centers in Vietnam 100% dedicated to tracking, measuring and improving the process adherence and excellence of our application development teams
Accreditation--Maintaining third-party accreditation of industry-recognized standards of excellence, such as CMMI Level 5, ISO 27001 and BS7799

However, that didn't end the discussion and I am glad. The questionnaire challenged if measures like daily code output were truly accurate in revealing the cost of output. After all, the idea behind object-oriented programming is to reuse units of programming logic. It was a fantastic, technology-smart pushback on efficiency.

We in the offshore industry pride ourselves on delivering efficiency, and here's a place where measurement is extremely hard. We have come up with many ways of estimating time and cost on a project (Story Points, Use Case Points, etc.) However, once we move beyond estimates and into the actual work, can we actually measure the effort and innovation of the application development process in code and bugs alone?

No we cannot--which is exactly the idea behind the Harvey Nash Process Quality Assurance Team. These senior technologists on our offshore team spend 100% of their time monitoring all software development projects, streamlining processes and reducing any and all inefficiencies they come across. They work with teams and individual developers and testers to constantly improve their work and results. It's a continuous effort by Harvey Nash to infuse best practices and push productivity higher by working with our teams to improve their daily, even hourly, work.

It's also the idea behind our apprenticeship program in which any new developer or tester on the Harvey Nash Vietnam team spend approximately 18 months learning. They don't work directly on client projects until they are experts in the methodologies and processes that have made our 4,500 strong offshore team an engine of highly efficient IT innovators for our clients worldwide.

But do those two efforts to maximize individual and team productivity give me a simple productivity metric to hand over? A chart I can give to clients to say this number right here shows you exactly how much innovation, knowledge, efficiency and revenue you gained this month as a result of the efficiency practices and knowledge management of our offshore team? It doesn't yet and here's my argument as to why. Productivity in the end is a measure that must always be tied to that fourth important measurement factor: revenue.

Revenue realized as a result of the offshore investment is a key aspect of the productivity equation, and I believe one that will receive greater attention from the offshore industry in the years ahead. It actually underscores the importance of ensuring your offshore provider is a strategic partner--not just a service provider. Just as the "IT department" has had to learn how to measure its productivity in relation to business revenue, we in offshore are working to find measurements that are truly reflective of "bottom-line productivity." There's not a quick, easy remedy here, but that's not going to keep me from working on it. Like I said, I love a big challenge.

On Tuesday, March 10, Harvey Nash's VP of Technology Solutions, Anna Frazzetto--an IT industry veteran and frequent presenter at SIMposium and HDI--hosted a Webinar for CIOs, CTOs and senior leaders of IT. During her presentation she shared the practical steps to help build a business case for offshoring, what to look for in an offshore location and the top five keys to offshoring success. The presentation concluded with a Q&A session, and following is a transcript of the questions and answers posed by participants.

What are some key things we need to look at from a contract perspective?
From a contractual perspective, you need to lock it in under U.S. currency. From the get-go you need to know that it's a U.S. contract enforceable in the U.S., and you need to know the pricing structure associated with it. Doing it this way ensures the currency exchange becomes the vendor's responsibility and not your responsibility.

Another item to look at related to contracts is the types of increases that should be factored in. For example, a country's inflation rate could be 30-40%, but you want to cap a cost of living increase at 10%. Again, this is something that becomes the vendor's issues. I think that's where most of the challenges have been--in not locking the cost of living increase in and the vendor going back to client and requiring them to pay an extra 25-30% to keep their team in place.

Have you seen a correlation between high turnover rates and the maturity of the outsourced vendor and countries?
Yes and no. I answer it this way because I do think there is another component and that's what is accepted culturally. For example, although India has been in business for many years and does have a level of maturity, it's culturally accepted to migrate from job to job for a better career opportunity. In China and India, people view themselves as their own career agent and that's socially accepted. In Vietnam and Singapore, it's not socially or culturally accepted. I can speak specifically to Vietnam, and know that the employer is their career agent. It's a lot like how the U.S. was 15 years ago. You trusted your employer to present you with opportunities for career advancement. That's obviously changed in the U.S. and we can talk about the Gen Y population and where they've taken ownership of their own careers. So, that's why I answer the question yes and no because I do think that culture has a big role in turnover.

What are the most critical issues overlooked or underestimated when considering offshoring?
In my opinion, it's cost and time. There's no such thing as being able to take this little nugget and throw it over the fence and the offshore team being able to deliver on it. It doesn't work like that.

You have to invest the time. Think about if you were setting up a new group in your environment in the U.S. and hired a group of people. If you just expected them to learn it on their own and came back to them in three months asking if the project was complete, the answer is most likely going to be "no." Now obviously I'm exaggerating a bit for effect, but if the offshore team is 10,000 miles away, you need to invest the time in training them and getting them up-to-speed on the project.

It's also essential that you have a communication methodology in place. It doesn't have to be a phone call in every instance--it can be Instant Messaging, Skype, video conferencing--but you really need to be investing in the communication effort.

Finally, I would say you also need to evaluate the cost savings you're really going to get from an offshore solution.

Does Harvey Nash offer offshore project management and QA as well as software development work?
Yes, we do. We offer what I like to refer to as the "application umbrella." Project management, ongoing maintenance and support, QA, testing, design, architecture, development. We also do BPO and on that side it is traditionally digitizing transaction-heavy processes that we can help automate.

How would you compare Vietnam to some of the other countries you mentioned such as Thailand and India?
I think the comparison is best said in the words of our clients.

We have one client that's in the Chicago area, a leading edge technology firm and we were working with their senior architect. The gentleman came back from Vietnam and was going through all checks of the work we were producing. He said to me, "I am truly humbled. I was expecting challenges with the deliverables, the code, the quality. But I am truly humbled at the work you've done and so impressed with the team."

It's what I keep saying about Vietnam. It's truly the hidden treasure of Southeast Asia. The technology capabilities, the quality of the students and the technical talent is truly high-end. Now it doesn't compare to the number of graduates in India and/or the number of people.

Something you also want to factor when you compare India to Vietnam, let's say, is the cost from a turnover and quality perspective. Some of the quality from India has suffered because of the challenge retaining talent.

I think Thailand will be a strong contender. Singapore will also be a strong contender. They have great technical capability, but it's a really small country so I'm not sure they will have scalability.

What are your recommendations for managing a multi-vendor scenario?
I would limit the number of vendors you use. I believe Gartner is recommending that you should have a top three list of offshoring providers. This allows you to better control and manage the providers and their efforts. I would recommend that you look at balancing geography, so not having them all in same geographical location as we discussed in today's presentation. You should also look at their area of expertise. That's an easy, natural way to divide the work. Finally, I would make sure your vendor has a single point of contact for you and if that person is not available, a single person for escalation.

Can you share your perspective on the situation with Satyam and India?
I think the situation highlights some of the key points we made today. I think the risk factors that we highlighted and Satyam being a family owned business--there is a risk factor associated with that. You want to make sure you understand the financial stability of the company.

If you have a multi-vendor arrangement you do mitigate your risks. If something happens, you already have the processes in place and can move to another vendor with minimal consequences.

It's an unfortunate situation, and I think it gives offshoring a bad name. Particularly because shortly after, the announcement came out about WIPRO and World Bank. I think it's left a lot of people asking what's next?

(You can read more of Anna's perspective in her blog, Big Integrity Hits for Indian Outsourcing.)

How do you recommend the transfer of knowledge be tackled?
I think that's a great question to conclude today's Webinar. Most people I talk to think the knowledge transfer happens at the beginning and end of the project. But it should actually be done through the evolution of the project.

Obviously the offshore team gains domain knowledge up front. This is done through online training, in person training with either you going to the offshore location to train the team, or a portion of the team coming to the U.S. and getting trained, and then returning to train the rest of the team.

It's important to have a process in place as you go through each milestone and deliverable so you never feel like you've lost control of what's going on. You must have the right process in place to exchange knowledge throughout the engagement.

For more information about Harvey Nash Offshore Development Services, please contact Anna Frazzetto at