Snowflake Introduces AI Data Ingestion
What happened
Snowflake has launched OpenFlow, a new service that uses AI-powered workflows for automated data ingestion. The tool reflects a broader industry trend toward AI-driven data pipelines. The system aims to reduce manual work and enable the creation of more intelligent, automated backend systems.
Why it matters
- OpenFlow is built on the open-source dataflow system Apache NiFi; Snowflake acquired Datavolo, a company founded by the co-creators of NiFi, to integrate the technology into its platform. - The service integrates with Snowflake Cortex, allowing pipelines to call on large language models (LLMs) to perform tasks like semantic chunking or summarizing unstructured data like text and images as it is ingested. - For streaming data, OpenFlow connects directly to sources like Apache Kafka and Amazon Kinesis, using a new Snowpipe Streaming integration that provides 10GB/s throughput with a five-second query latency. - It is designed to process and structure multimodal data from sources like Google Drive, SharePoint, and Box, making it usable for building AI applications such as "chat with your data" experiences. - A related feature, Document AI, uses Snowflake's proprietary Arctic-TILT large language model to extract data from documents, including from tables, logos, and even handwritten signatures. - Developers can deploy data pipelines on Snowflake's managed infrastructure or in their own cloud environment, with both options operating as managed services. - All data processing and AI enrichment occur within Snowflake's security perimeter, meaning data doesn't need to be moved to a third-party service, and it adheres to existing role-based access controls.
Key numbers
- For streaming data, OpenFlow connects directly to sources like Apache Kafka and Amazon Kinesis, using a new Snowpipe Streaming integration that provides 10GB/s throughput with a five-second query latency.
What happens next
- The system aims to reduce manual work and enable the creation of more intelligent, automated backend systems.
Quick answers
What happened in Snowflake Introduces AI Data Ingestion?
Snowflake has launched OpenFlow, a new service that uses AI-powered workflows for automated data ingestion. The tool reflects a broader industry trend toward AI-driven data pipelines. The system aims to reduce manual work and enable the creation of more intelligent, automated backend systems.
Why does Snowflake Introduces AI Data Ingestion matter?
OpenFlow is built on the open-source dataflow system Apache NiFi; Snowflake acquired Datavolo, a company founded by the co-creators of NiFi, to integrate the technology into its platform. The service integrates with Snowflake Cortex, allowing pipelines to call on large language models (LLMs) to perform tasks like semantic chunking or summarizing unstructured data like text and images as it is ingested. For streaming data, OpenFlow connects directly to sources like Apache Kafka and Amazon Kinesis, using a new Snowpipe Streaming integration that provides 10GB/s throughput with a five-second query latency. It is designed to process and structure multimodal data from sources like Google Drive, SharePoint, and Box, making it usable for building AI applications such as "chat with your data" experiences. A related feature, Document AI, uses Snowflake's proprietary Arctic-TILT large language model to extract data from documents, including from tables, logos, and even handwritten signatures. Developers can deploy data pipelines on Snowflake's managed infrastructure or in their own cloud environment, with both options operating as managed services. All data processing and AI enrichment occur within Snowflake's security perimeter, meaning data doesn't need to be moved to a third-party service, and it adheres to existing role-based access controls.
