Stream processing was considered niche, with some inherent limitations over traditional batch processing, including transactional correctness issues, lack of tools, and poor scalability. But these limitations were not unbeatable, and today, with over 150k organizations using Kafka in production, it’s safe to say that this is a technology that will be a fundamental part of the data architecture for all types of data in every organization, for all the data in the world.

Earlier this week, Current 2023: The Next Generation of Kafka Summit kicked off year two of its run as the industry conference for all things Apache Kafka and real-time data streaming.

Hosted in San Jose, CA, the summit welcomed over 2.1k in-person attendees (and 3.5k+ who joined virtually), showcased two major keynotes, and hosted over 100+ breakout sessions led by subject matter experts from nearly every major industry. And the Clurgo team was there!

Data streaming is all that enterprises can’t live without.

Surely, without data streaming technology, we couldn’t publish alerts in real-time and crowdsource observations:

• “It’s a capability that we didn’t have before” – Joe Foster, Cloud Computing Program Manager at NASA explained.
• “Without Kafka, our customers would never get a car,” Tobias Nothaft, Sub Product Owner, Streaming Platform at BMW.
• “Without Apache Kafka elasticity and log management capabilities Warner Bros. Discovery won’t be able to launch its new video streaming service this year, called ‘Max,’ built on Confluent service,” Girish Rao, SVP Engineering explained.

Business data is always streaming

I can’t agree more with Senior Product Manager at Confluent Martijn Visser that business data is always streaming. It’s either bounded (“batch”) or unbounded. ApacheFlink handles both in the same way, making it easier to query both historical and live data. Therefore it’s interesting to see Confluent embellishing its Kafka-based message brokering cloud service with real-time analytics capabilities, through a managed version of Flink. The service aims to eliminate the burden of setting up analytics on the enterprise backend, and its serverless scale-up model promises to save provisioning costs as well, the company promises.

Enterprise efficient data engineering culture

Natan Silnitsky, Backend Infra Tech Lead at Wix, gave another amazing session around effective microservices communication (2.5k microservices using Kafka at Wix sounds wow!). The fresh insights around temporary workflow orchestration features were the most inspiring! Last but not least I want to applaud Xinran Waibel, Netflix Data Engineering Culture Advocate for an impressive session at Current 2023 in which she explained the way enterprises can build a highly efficient data engineering culture.

Thank you, Current23 organizers. Thank you, Clients and partners! This week was a blast!