Hiring an External Development Team: Interview Prospective Vendors
When you're talking to potential development firms that you're considering hiring for building your product, make sure you dive deep into the company's background and approach to software development. It's critical to be thorough in your interviewing process so that, of course, you find the right vendor for your company. But as importantly, all parties need to have the same expectations entering a project.
This article is part 3 in a 5-part series explaining some best practices for how to approach the process of hiring a team when outsourcing software development. Previously we've discussed, getting organized and how to find potential vendors. In this post, we'll cover questions you should ask.
Of course, you'll have different follow-up questions with each vendor, but the table below contains questions I'd suggest asking every vendor with whom you speak.
A note on using point systems and other quantitative evaluation methods: unless you're the government and you need to measure contenders in such a fashion, your gut is often a better indicator of which firm to choose when compared with seemingly arbitrary scales.
Some Questions to Ask
Make sure to get answers to the questions below when you're interviewing a vendor or reviewing their proposal. Of course, add whatever other questions you think will be necessary so you can make a decision in which you're completely confident in your development team.
| Question | What to look for |
|---|---|
| What type of projects do you work on? | Look for vendors who have experience building unique products, not building simple websites or configuring Content Management Systems (CMS's).
There's a different set of expertise and project management processes for CMS vs. custom app development. If the vendor doesn't have a team of people very experienced in custom app development, be wary. Red Flag: A company that has significant experience building CMS's. |
| Have you created libraries to streamline software development? | Experts don't reinvent the wheel. They stand on the shoulders of giants and leverage existing frameworks for common tasks such as authentication, testing, server deployment and the like.
However, this is sort of a trick question: small tools, scripts, and code libraries that augment the process can be very helpful. Experts will have created small tools that help with very specific and isolated tasks. For example, our team created the rspec_api_documentation tool to help document APIs. Red Flag: A completely custom approach to every problem, and on the opposite end of the spectrum: any "one size fits all" solutions. |
| What tools do you use to streamline development? | Feel comfortable with firms that use widely-respected tools, and bonus points for tools that are open source.
Be extremely wary of building your product off of proprietary tools (even if they're "open source") that are not widely-supported. This could leave you in the lurch if you need to stop working with the firm. For firms using Ruby on Rails, ruby-toolbox.com is a great resource for researching tools. Red Flag: Something proprietary that nobody else has ever heard of. |
| What framework will you use to build our application? | Look for vendors that use web application frameworks such as Ruby on Rails, Django or CakePHP. These are frameworks for creating custom applications, like your product.
Stay away from strangers with candy and vendors who say they'll use WordPress, Joomla, Drupal, Magento, [insert name of CMS or eCommerce framework here] as the backbone of your product. CMS's should only be used for content sites, and e-commerce frameworks should only be used for e-commerce sites. A Cessna with a single propeller is great for traveling a couple hundred miles at speeds under 150 mph. But I wouldn't retrofit it with jet engines and then try to fly across the Pacific at 600mph. Red Flag: DotNetNuke, Joomla, Drupal, ExpressionEngine, WordPress, and (let's be honest) .NET. |
| How will I know where we stand on scope and budget? | With the right project management system, you should be able to go to a website, such as Pivotal Tracker, and see the status of all features. This should be SUPER easy for you to do.
For knowing where you stand on budget, consider how often the firm invoices and how budget is reported back to you. SmartLogic, for example, invoices weekly. Getting invoices weekly protects you from getting shocked at the end of the month, or even worse, the end of a phase in the project. Whether your firm uses this method or simply tracks hours in a spreadsheet, they should have a clear answer to this question. Red Flag: Any budget or scope updates that feel infrequent enough that the information would be too late. |
| How does the estimation process work? | Estimating a budget after just a few sales discussions is hard to do, especially with products for startups who may need to change their strategy given customer feedback. That's why communication is so important. The whole point of agile development is that the product will change. Expect a ballpark and as your product changes, make sure the firm can tell you how changes in scope will affect cost.
Red Flag: Fixed-price bids. |
| Are your project managers technical? | Can it work for a non-tech person to manage a technical project? Probably, but it's a risk.
Some people might say developers should be focused on coding and not managing client relationships. But separating developers from the client means you've got more chances for miscommunication and mistakes. The ideal project manager is a developer with project management experience. The PM should be supported by a simple process. Red Flag: The answer "No. Besides, developers don't know how to talk." |
| How often can I access the latest version of the app? | A development shop focused on developing products will have a continuous integration (CI) system in place. CI is an automated process by which changes to code are deployed to various servers and tested against an automated test suite. Features and code should be deployed continuously, and automatically, so that all stakeholders can see the most up-to-date code in action.
Be wary of having to ask to see progress or features in development. The last thing you want is to pay for development for long periods of time without being able to see and review features; this increases the chance of being unpleasantly surprised. Red Flag: Anything less frequent than several times a week. Another red flag is if you get the feeling they don't want you to see and approve work frequently. |
| Describe your QA process. | You want a company that has an automated QA process and that practices Test-Driven Development (TDD). You should feel confident that there won't be regressions (the scenario where new features break old features).
Be wary of companies that have manual testing processes; they don't scale. Red Flag: Anything with a manual component. Atomic Red Flag: The answer "The client is responsible for testing all code changes." |
| Describe a situation where a project did not go well. What happened and what did you change? | If a vendor tries to tell you they've both (1) never faced any issues on a project; and (2) bore no responsibility for said issues then they are lying or you are that company's first customer. Nobody is perfect and no project goes off without a hitch.
You want a vendor that can act as a trusted partner and that can be honest with you. You also want a vendor that is constantly improving its process. At SmartLogic we are extremely quick to change how we do business. Generally the changes we institute result in improvements to our process and client relationships. When that's not the case we're just as quick to change and try something new. What you don't want is a vendor that repeats the same mistakes and is unable to learn from said mistakes. Red Flag: (1) We've never really had a client dissatisified with our work; or (2) it was all the client's fault; or (3) no clear explanation on steps taken to mitigate risk of the situation happening in the future. |
After you conduct interviews, read our guide to development proposals and signing a deal with a vendor.