by Changjian Gao and Jingjing Jia

Collaborations between open-source communities are a magical thing. Not only do passionate, intelligent, and creative volunteers keep open-source solutions innovative, they also work to bring different tools together in interesting and useful ways. Milvus, the world’s most popular vector database, and JuiceFS, a shared file system designed for cloud-native environments, were united in this spirit by their respective open-source communities. This article explains what JuiceFS is, how to build a Milvus cluster based on JuiceFS shared file storage, and the performance users can expect using this solution.

What is JuiceFS?

JuiceFS is a high-performance, open-source distributed POSIX…

Milvus is an ongoing open-source software (OSS) project focused on building the world’s fastest and most reliable vector database. New features inside Milvus v1.1.0 are the first of many updates to come, thanks to long-term support from the open-source community and sponsorship from Zilliz. This blog article covers the new features, improvements, and bug fixes included with Milvus v1.1.0.

New features

Like any OSS project, Milvus is a perpetual work in progress. We strive to listen to our users and the open-source community to prioritize the features that matter most. The latest update, Milvus v1.1.0, offers the following new features:

Specify partitions with `get_entity_by_id()` method calls

To further…

We are proud to announce Milvus v1.0, a stable, long-term support version of the world’s most popular open-source vector database. Discover the highlight features that Milvus v1.0 has to offer.

Milvus is an open-source vector database designed to manage massive million, billion, or even trillion vector datasets. Milvus has broad applications spanning new drug discovery, computer vision, autonomous driving, recommendation engines, chatbots, and much more.

In March, 2021 Zilliz, the company behind Milvus, released the platform’s first long-term support version-Milvus v1.0. After months of extensive testing, a stable, production ready version of the world’s most popular vector database is ready for prime time. This blog article covers some Milvus fundamentals as well as key features of v1.0.

Milvus distributions

Milvus is available in CPU-only and GPU-enabled distributions. The former relies exclusively on…

Artificial intelligence influences what we read, the things we buy, who we become friends with, and more everyday human behavior. A huge part of what makes many modern AI applications possible is vector similarity search. Understanding how this underlying technology works can help demystify AI, providing insight into what computer intelligence is really made of.

Imagine a room with a wall of screens displaying closed circuit video feeds from dozens of cameras, like a security office in a film. In the movies, there is often a guard responsible for keeping an eye on the screens that inevitably falls asleep, allowing something bad to happen. Although intuition and other distinctly “people skills” are useful in security, most would agree that the human attention span isn’t well suited for always-on, 24/7 video monitoring. …

With 71% of Americans getting their news recommendations from social platforms, personalized content has quickly become how new media is discovered. Whether people are searching for specific topics, or interacting with recommended content, everything users see is optimized by algorithms to improve click-through rates (CTR), engagement, and relevance. Sohu is a NASDAQ-listed Chinese online media, video, search and gaming group. It leveraged Milvus, an open-source vector database built by Zilliz, to build a semantic vector search engine inside its news app. …

From Unsplash

Compile time can be compounded by complex internal and external dependencies that evolve throughout the development process, as well as changes in compilation environments such as the operating system or hardware architectures. Following are common issues one may encounter when working on large-scale AI or MLOps projects:

Prohibitively long compilation — Code integration is done hundreds of times each day. With hundreds of thousands of lines of code in place, even a small change could result in a full compilation that typically takes one or more hours.

Complex compilation environment — The project code needs to be compiled under different…

Banks and other financial institutions have long been early adopters of open-source software for big data processing and analytics. In 2010, Morgan Stanley began using the open-source Apache Hadoop framework as part of a small experiment. The company was struggling to successfully scale traditional databases to the massive volumes of data its scientists wanted to leverage, so it decided to explore alternative solutions. Hadoop is now a staple at Morgan Stanley, helping with everything from managing CRM data to portfolio analysis. …

Online shopping surged in 2020, up 44% in large part due to the coronavirus pandemic. As people sought to socially distance and avoid contact with strangers, no-contact delivery became an incredibly desirable option for many consumers. This popularity has also led to people buying a greater variety of goods online, including niche items that can be hard to describe using a traditional keyword search.

To help users overcome the limitations of keyword-based queries, companies can build image search engines that allow users to use images instead of words for search. Not only does this allow users to find items that…

Learn how Milvus is used to mitigate threats to critical data and strengthen cybersecurity with real-time virus detection.

Cybersecurity remains a persistent threat to both individuals and businesses, with data privacy concerns increasing for 86% of companies in 2020 and just 23% of consumers believing their personal data is very secure. As malware becomes steadily more omnipresent and sophisticated, a proactive approach to threat detection has become essential. Trend Micro is a global leader in hybrid cloud security, network defense, small business security, and endpoint security. To protect Android devices from viruses, the company built Trend Micro Mobile Security — a mobile app that compares APKs (Android Application Package) from the Google Play Store to a database of…

In this talk, our speaker- Dr.Yi covers how various index types can be used in Milvus to accelerate vector similarity search on large datasets.

Vector similarity search & indexing methods

Dr. Xiaomeng Yi, Senior Researcher, Zilliz

Download the session slides here!

❓Q&A for the session:

Q: What are some common issues when indexing large datasets?

A: For building index, the most common issue is performance. Typically, for a 32-core server, it would take minutes to hours to build index for one billion vectors. For search, memory usage is another critical issue. …


Open source #distributed #similarity_search engine for large-scale #vectors. #ANNS #FAISS #SPTAG #DL

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store