11th June 2019

Keeping your SaaS vendor honest during product selection.

Reading Time: 2 minutes

There’s a blurry line between the Solutions Architecture and Business Analysis – one of the things we’ll often find within that area are the Non-Functional Requirements. From an Architectural perspective I believe NFRs to be the life-blood of the project, especially within organisations which are heavily regulated and/or have str

Typical Business Analyst focuses on the functional requirements, if you’re lucky you’ll get some cucumber style user stories and gherkin style acceptance criteria but it’s rare that they’ll pay too much attention to the technology. It’s often the case that a Business Analyst community will, by default use some relatively low-quality boilerplate non-functional requirements, often these can be horribly out of date.

In an ideal world your non-functional requirements will be reflective of the direction of travel, architecturally. Companies seeking to adopt a more cloud-centric ways of working should reflect those desires within their requirements and score vendors according to their alignment.

A common theme I see is that when it comes to the selection of SaaS products the perception can often be that non-functional requirements are largely irrelevant. The argument will often be made that when you’ve got nothing to really manage or maintain then why do you need to care about the physical security, the release management strategy or how they backup the data within their platform.

I sit across from views such as these often, the focus will typically be towards the SLA, Licence structure and very little else. It’s an approach I dislike and frequently resist – I feel strongly that a trust but verify approach is more sensible than hoping for the best.

I’ve sat across from more well rehearsed glossy sales-pitches than I care to imagine. During those sessions I’ve seen experienced sales engineers struggle to hide the realities of their operations when probed about areas such as the underlying technology and the maturity of their own internal governance.

My suspicion is often that many of them are building products far too quickly and still have a start-up mentality. Those competing in crowded or innovative spaces are more likely to sacrifice governance for features and diverting effort away from developing functionality to support a proper release management process, for example, isn’t likely.

Architects often earn their money in the less-sexy areas of the Information Services arena and insisting on assurances of the quality of the product is one area where I feel we can do just that. It’s why I’ll often insist on putting forth a set of NFRs designed to probe their product and their processes – the clients I work with all deserve a solid view of the quality and maturity of organisation they’re engaging with.

That’s not to say it need be a list of non functional requirements, a “SaaS adoption checklist” would do much the same thing. But it’s critical that it be tied to your RFP/RFI process – that’s when you have the most leverage.

Often I’ll try and look into areas such as;

  • Security: Identity Management Integration, Surfacing Security Events, If the platform fails gracefully, Cadence of Security Testing, Patching Strategies
  • Availability: The geography of their locations, How they calculated their SLA, Failure domains and impact
  • Analysis and Audit: Their ability to share events
  • Integration: How much of the platform (data, workflows and such) can be consumable by API.
  • Robustness: Describing the release management strategy

If my experience is anything to go by you’ll get some resistance – but it’s better to find out about an immature vendor now, than at 9AM after a failed patch wipes out the platform.