Azure Search is a search-as-a-service cloud solution that gives developers APIs and tools for adding a rich search experience over private, heterogenous content in web, mobile, and enterprise applications.
In a context of organizations, applications with search capabilities can be used by External Customers or Internal Business Users. Example of Cognitive Search on top of publicly available JFK files (see JFK Files Public Site and JFK Files Project).
When you use Full Text Search, query execution is done over a user-defined index on top files and searchable datasets.
Cognitive search
Cognitive search can be added to create searchable information out of non-searchable content by attaching AI algorithms to an indexing pipeline. AI integration is provided through cognitive skills, enriching source documents before creating a search index.
Cognitive Skills are based on the same AI algorithms used in Cognitive Services APIs:
- Natural language processing skills include entity recognition, language detection, key phrase extraction, text manipulation, and sentiment detection. With these skills, unstructured text becomes structured, mapped to searchable and filterable fields in an index.
- Image processing skills include OCR and identification of visual features, such as facial detection, image interpretation, image recognition (famous people and landmarks) or attributes like colors or image orientation. You can create text-representations of image content, searchable using all the query capabilities of Azure Search.
- Custom skills are a way to insert transformations unique to application content. A custom skill executes independently, applying whatever enrichment step you require. For example, you could define field-specific custom entities, build custom classification models to differentiate business and financial contracts and documents, or add a speech recognition skill to reach deeper into audio files for relevant content.
Skills can be chained. For instance, you may want to use the language you detected to improve the accuracy of the key-phrase extractor.
Video from Ignite: AI for Knowledge Mining by Luis Cabrera:
Azure Search Global Distribution
To reduce latency for remote users (in a case of geo-distributed workloads) it makes sense to create search services in each corresponding region (that is in closer proximity to these users). For example, you may use multiple Azure Search indexers in different regions that will point to the same datastore. To route requests to multiple geo-located websites that are then backed by multiple Azure Search Services, you may use Azure Traffic Manager. This approach also provides high availability and load balancing.
List of Azure Search Features
- Full text search – queries can be formulated using Simple query syntax or Lucene query syntax.
- Cognitive Search (Preview) – AI-powered algorithms for image and text analysis can be applied to an indexing pipeline to extract text information from raw content. A few examples of built-in skills include optical character recognition (making scanned JPEGs searchable), entity recognition (identifying an organization, name, or location), and key phrase recognition. You can also code custom skills to attach to the pipeline.
- Data integration with JSON data structures. Indexers can be used to automatically crawl Azure SQL Database, Azure Cosmos DB, or Azure Blob storage for searchable content in primary data stores. Azure Blob indexers can perform document cracking to extract text from major file formats, including Microsoft Office, PDF, and HTML documents.
- Custom lexical analyzers and Language analyzers from Lucene or Microsoft are used for text processing during indexing and search operations.
- Geo-search can be used to process, filter, and display geographic locations.
- User experience features: Auto-complete (preview), Search suggestions, Synonyms, Faceted navigation, Filters, Hit highlighting, Sorting, Paging, Simple scoring for relevance ranking.
- Administrative and development tools: Search traffic analytics, Service administration, Import data wizard, Search explorer.
Reference
- Azure Search Documentation
- Cognitive Search (Preview)
- Azure Search performance and optimization considerations
- Azure Search – frequently asked questions (FAQ)
- How full text search works in Azure Search
- JFK Files Public Site
- JFK Files Project
Filed under: Article, Video | Tagged: Artificial Intelligence, Azure Search, Cognitive Search, Cognitive Services, Full-text Search, Machine Learning | Leave a comment »
Business Excellence 