Artificial Intelligence Decision Tree

Posted on February 15, 2019 by Ivan Kosyakov

Let’s discuss decision points for selecting right components for Artificial Intelligence (AI) solutions. This is also an update to Machine Learning Decision Tree (v1). Keep in mind here that AI is a broader term compared to Machine Learning.

Artificial Intelligence functionality in the decision tree is divided between following groups:

  • AI in Business Applications: Dynamics 365 AI (AI for Customer Service, Market Insights, Sales), Microsoft 365 AI (Office 365 Workplace Analytics, ML in Power BI, O365 Search).
  • Knowledge Mining: O365 Search, Azure Search.
  • AI Apps and Agents: Azure Bot Service (Framework), Cognitive Services (Vision APIs, Speech APIs, Language APIs, Search APIs, Custom APIs).
  • Machine Learning Tools: Azure Notebooks, Jupiter Notebooks, Code, PyCharm, Visual Studio, Azure ML Studio.
  • Cloud-based Machine Learning: Azure ML Service, Azure ML Studio, Data Preparation (Azure Data Factory, Azure Databricks), Model Training/Testing (Azure Databricks, Azure HDInsight, Data Science VM), Container Registry, Model Deployment (Azure Container Instances, Azure Kubernetes Service, Azure Batch, Azure IoT Edge), Azure Infrastructure (CPUs, GPUs, FPGAs).
  • On-premises Machine Learning: Edge Devices, Cognitive Services Containers, SQL Server ML Services, On-prem Hadoop.
  • Machine Learning Frameworks: Deep Learning (ONNX, PyTorch, TensorFlow), General ML (Spark MLllib, SparkR, SparklyR, MML Spark).

The text description of the decision points will be available in a few days…

Reference materials

Other Decision Trees/Maps

Filed under: Article, Decision Tree | Tagged: , | Leave a comment »

Azure Search with Knowledge-based Cognitive Capabilities

Posted on October 12, 2018 by Ivan Kosyakov

Azure Search is a search-as-a-service cloud solution that gives developers APIs and tools for adding a rich search experience over private, heterogenous content in web, mobile, and enterprise applications.

In a context of organizations, applications with search capabilities can be used by External Customers or Internal Business Users. Example of Cognitive Search on top of publicly available JFK files (see JFK Files Public Site and JFK Files Project).

When you use Full Text Search, query execution is done over a user-defined index on top files and searchable datasets.

Cognitive search

Cognitive search can be added to create searchable information out of non-searchable content by attaching AI algorithms to an indexing pipeline. AI integration is provided through cognitive skills, enriching source documents before creating a search index.

Cognitive Skills are based on the same AI algorithms used in Cognitive Services APIs:

  1. Natural language processing skills include entity recognition, language detection, key phrase extraction, text manipulation, and sentiment detection. With these skills, unstructured text becomes structured, mapped to searchable and filterable fields in an index.
  2. Image processing skills include OCR and identification of visual features, such as facial detection, image interpretation, image recognition (famous people and landmarks) or attributes like colors or image orientation. You can create text-representations of image content, searchable using all the query capabilities of Azure Search.
  3. Custom skills are a way to insert transformations unique to application content. A custom skill executes independently, applying whatever enrichment step you require. For example, you could define field-specific custom entities, build custom classification models to differentiate business and financial contracts and documents, or add a speech recognition skill to reach deeper into audio files for relevant content.

Skills can be chained. For instance, you may want to use the language you detected to improve the accuracy of the key-phrase extractor.

Video from Ignite: AI for Knowledge Mining by Luis Cabrera:

Azure Search Global Distribution

To reduce latency for remote users (in a case of geo-distributed workloads) it makes sense to create search services in each corresponding region (that is in closer proximity to these users). For example, you may use multiple Azure Search indexers in different regions that will point to the same datastore. To route requests to multiple geo-located websites that are then backed by multiple Azure Search Services, you may use Azure Traffic Manager. This approach also provides high availability and load balancing.

List of Azure Search Features

Reference

Filed under: Article, Video | Tagged: , , , , , | Leave a comment »

Modern Data Platform Map and Video

Posted on September 20, 2018 by Ivan Kosyakov

Last update: Dec 4, 2018

Modern Data Platform Map represents reference organizational layout of most important data pillars and services and corresponding groups of specialists in enterprises.

In the following video I make a quick overview of Microsoft Data Platform. I will provide more details in subsequent posts and videos. Please post your questions, suggestions and feedback below.

You may also check for details following data pillars and products:

Filed under: Article, Map, Video | Tagged: , , , , , , , , , , , | Leave a comment »