AI-as-a-Service: AI cloud services should make the technology suitable for everyday use

AI-as-a-Service

Artificial intelligence (AI) and machine learning (ML) inspire the imagination of many SaaS providers. Wouldn’t it be great if we could replace complicated input masks with an easy-to-use bot? Why should we have to type in travel expense receipts? A photo with clever AI running in the background can do it, right? In practice, teams attempting to implement such projects encounter a lot of problems. Above all, there is a lack of relevant development experience in many cases. Completed AI services like Cognitive Service from Microsoft promise a remedy for that. Instead of having to laboriously develop everything from scratch, you simply get consumable Web APIs with usage-based costs. Does this mean that typical SaaS projects are on the fast lane towards an AI future?

Off-the-rack AI has limited value

The first step in answering this question begins with the availability of data. Granted, there are AI services like Microsoft’s Text Analytics [1] and Computer Vision [2] or Google’s Cloud Vision API [3] that are completely ready for use. For example, recognizing the language of a text with Text Analytics does not require training data or an understanding of machine learning. If you can send a text to a Web API, you’re ready to go. For some applications, this might be enough as a start (e.g., assigning a support case to a team member who speaks the correct language). But in most cases, that is not enough. AI and machine learning only represent a real added value if they are adapted to a specific application.

Adaptable AI Services

If no ready-made AI service are available off-the-rack, that does not mean you have to build everything from the ground up with libraries like TensorFlow or Microsoft Cognitive Toolkit (aka CNTK). There is a middle ground: customizable AI and ML models that you can train with your own data. Here are two examples from Microsoft’s product portfolio:

With Custom Vision Service [4] images can be tagged according to an individual logic. Instead of writing the algorithm by hand or having to create a deep learning model from scratch, you provide training data in the form of correctly indexed images. They are used to train a base model provided by Microsoft. The result is an individualized model with a Web API that allows you to tag new images (prediction). With this service, it is even possible to export the trained model to run it locally
The Language Understanding Service (LUIS) [5] helps to process speech. When a user formulates a query in natural language, it is not easy to determine the user’s Intent (e.g., navigate, order a product, book a trip, etc.) and any parameters contained in the sentence (Entity, e.g., destination, product name, date of travel). However, this ability is indispensable when programming a bot, for example. LUIS solves precisely this problem. Training data is provided in the form of pattern sets (Utterances) with correct assignment to Intents and Parameters (see Figure 1). The trained model can be deployed with just a few clicks. You can use the Web API that you receive as a result directly or link it to the Azure Bot Service to develop a bot.

Figure 1: Microsoft LUIS

Data is the gold standard

These two examples demonstrate the fundamentally new approach to “programming” (semi)-finished AI services as compared to the classical development of program libraries. Our role as a developer is no longer writing an algorithm. We need to take care of the training data. This task is anything but trivial because the quality of the emerging Deep Learning model stands and falls with the quality of the training data. If there is not enough data available or if the existing training data sets are faulty (e.g., incorrectly tagged), of poor quality (e.g., poor image quality, very similar photos) or not representative (e.g., pattern sets that no genuine user would ever use), the result is useless. In addition, training data alone does not suffice. Additional datasets are required for testing the models.

Data is the new gold standard in the world of AI and ML. Ready-made AI services in the cloud do not change this fact. On the contrary. As a team that wants to enter this world, you first must ask yourself how you can acquire the necessary data. This hurdle is also what makes getting started for start-ups so difficult. Established companies either have existing databases or can rely on an existing community that can motivate them to test AI-based software components such as bots, provide feedback, and resulting from this, indirectly supply the necessary training data.

Iterative model development

An important aspect in this context is iterative model development. Customizable AI services, such as those listed above, contain ready-made components that can be used to verify real-world data (e.g., sentences that users have spoken to a bot or images that have been uploaded for tagging). If you discover classification errors, you can easily add the real data to the training set with correct metadata, thereby improving the AI model step by step (see also Figure 2).

Figure 2: Tagging an image in Custom Vision Service

For this iterative approach to work in practice, mechanisms must be in place that make versioning, testing, and implementing models simple and robust. AI services are usually available serverless. As a development team, you do not have to worry about operating or scaling the servers in any way. You can deploy the model with one click, differentiate between test and production environment, have version management built in, export the models for archiving in source code management, and much more. Such functions keep the time required for administration and DevOps processes to a minimum.

Figure 3: Deployment of a LUIS model

API for meta-programming

For SaaS providers, there is another important feature of AI services: not only are all functions interactively available via a Web UI, but the same functions can also be automated via Web APIs. When you develop your own multi-tenant SaaS solution, you are often not able to lump all customers together. Every customer has different requirements. The data models differ, workflows are customer-specific, master data is naturally different for each customer, just to name a few aspects. For example, if you want to offer your individual bot to every SaaS customer, the model must differ from customer to customer in order to produce a high-quality result. The training data are different, and in many cases, the models also differ structurally.

AI service APIs enable you to conduct meta-programming as a SaaS provider. This means that you write a program that is not used by the end user, but rather creates another program – in this case an AI model with the help of an AI service.

Challenges

That all sounds alluring, doesn’t it? AI and ML can easily be used in any project, even if you have no relevant prior knowledge and the budget is limited. That statement is basically correct, but in detail there are some challenges to master. The first one has already been briefly mentioned above. You need a lot of good quality training data. Given the current context of GDPR, this is not only a technical, but also a legal hurdle (see also my column, A Fine Line: SaaS as a balancing act between machine learning and data protection).

The second challenge is the risk that one will expect more from the selected AI service than it can offer. As mentioned above, modern AI services have options for customizing the ready-made models. But you cannot control all aspects. After all, reduced complexity is precisely the strength of these services. Compared to traditional cloud SaaS and PaaS services, evaluating AI services is much more difficult. So far you could compare feature lists. This is not so easy with AI services. Suppose you want to develop a SaaS solution that involves the detection of license plates. Are Microsoft’s Computer Vision services suitable for that? Can you build a good solution with it, if you prepare the picture material for training and real operations accordingly? Would Google’s counterpart service deliver better results? In my experience, you cannot answer these questions in theory for projects in practice. You need to build prototypes or need help from people who have domain-specific experience with the selected AI services.

Conclusion

AI and ML projects often turn into adventures into which vast amounts of money and resources are sunk. In many cases, ready-made AI services in the cloud that can be adapted to each respective domain provide a shortcut and reduce project risk. However, anyone who believes that such projects are trivial will be sorely disappointed. Handling the data, automating the accompanying DevOps processes, evaluating the available AI services of various vendors, just to name a few aspects, will compel you to serious engagement with the topic. Otherwise, although you may get quick results, it will not offer real added value from the user’s perspective.

In his column “Stropek as a Service” Rainer Stropek takes up exciting aspects such as financing, customer lifetime value, and important topics such as billing, customer loyalty through quality or APIs – all from the perspective of an entrepreneur who has been active in the IT industry for 20 years and gained intensive experience with SaaS for many years.