Event-Driven Architecture (EDA) vs Request/Response (RR)
In this video, Adam Bellemare compares and contrasts Event-Driven and Request-Driven Architectures to give you a better idea of the tradeoffs and benefits involved with each.
To learn more about Event-Driven Architectures, check out Adam’s video on the 4 Key Types of EDA: • 4 Key Types of Event-D...
Many developers start in the synchronous request-response (RR) world, using REST and RPC to build inter-service communications. But tight service-to-service coupling, scalability, fan-out sensitivity, and data access issues can still remain.
In contrast, Event-Driven Architectures (EDA) provide a powerful way to decouple services in both time and space, letting you subscribe and react to the events that matter most for your line of business.
RELATED RESOURCES
► Tips to Help your Event-Driven Architecture Succeed - • How to Unlock the Powe...
► Introduction to Apache Kafka - cnfl.io/3Q7Pdn8
► Introduction to Kafka Connect - cnfl.io/3W24w4C
► Designing Events and Event Streams - cnfl.io/3vYIWU5
► Event Sourcing and Event Storage with Apache Kafka - cnfl.io/3JlXoso
► Designing Event-Driven Microservices - cnfl.io/444Kpoz
CHAPTERS
00:00 - Intro
01:04 - Reactivity
02:11 - Coupling in Space and Time
03:20 - Consistency
04:32 - Historical State
06:36 - Architectural Flexibility
09:09 - Data Access and Data Reuse
10:56 - Summary
-
ABOUT CONFLUENT
Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion - designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io.
#eventdrivenarchitecture #apachekafka #kafka #confluent #rest #rpc
Adam here. I took a crack at explaining some of the key differences between two popular interservice architectures. Let me know what you think, and if there are any other videos you'd like to see.
Love it! Thank you for making this, Adam.
Good work!
Very clean, thank you!
Thanks, best illustration of the two different architectures I've seen so far!
you guys are making some high quality content here. Keep up the great work! 👏
thank you, Adam. An amazing, short but very valuable video. Please continue enlighting us more. Also, I feel that this way of presenting is much better than not seeing the tutor, just the whiteboard. It makes me feel engaged.
Hey, thanks for the feedback! I'm going to have a few more coming out in the next month or so, so stay tuned for more.
Hello, Adam! I absolutely love and admire your effort (and other Confluent speakers) in making these very complex topics so easy to understand and grasp. Absolute best out there. Laconic an informative. Big thanks!
Adam here. Thanks! I appreciate the kind words.
Amazing explanation! Thanks for this 👍😃
Adam drops another knowledge bomb! Respect
🤓
Amazing video! Question about your setup, have you been teaching yourself to write backwards? My mind doesn’t quite wrap around how this video is filmed, it looks like the transparent “whiteboard” is in front, with you behind it writing.
Hi, Adam here. I'm using what's called a "lightboard", which is effectively just a sheet of glass, lit by a strip of LEDs around the edge. The camera shoots through the board, recording everything backwards. In post-processing, we flip the video on its vertical axis, in effect "mirroring" the view. Notice how I am using my left hand to draw in the video - in reality, I was using my right hand! it's just when we flip it in post, it also flips me :) If you want a better description, try searching "lightboard" in youtube, I know there are several people who have done a "how does it work" type videos.
Hi Adam, I wanted to check what all are the ways to get the completion status in EDA from fulfilment store? I can think of polling only. Which I believe shouldn't be recommended. Could you suggest the best approach.
I prefer an orchestrating ordering process that triggers events for underlying services to act on. These services will obtain the necessary data by making API calls to other services. Highly flexible through the use of process driven approach. Decoupled through event driven services. Consistency through well defined APIs.
Adam here. If you're using events as triggers to make API calls to other services, you lose replayability of the events. I've seen services work this way, and it's fine if you don't care about history or about high load on your servers hosting the API calls.
@@ConfluentDevXTeam my orchestrator can replay itself based on history and each event driven service can scale up horizontally to consume more work.
Why would you orchestrate some business process based on events (I assume), if services still making sync calls to other API's? That feels like orchestrated choreograhy. Data and temporal coupling are still there. Could you please explain underlying reason to do so?
There are sometimes cases where the full data a service needs to process an event would be too large for e.g. a Kafka message. In that special case the service could obtain additional data via a synchronous call to another API. If the data provided by the API are immutable then replayability won't be lost.
@@kohlimhg Yep, we also call it "claim cheque/check" pattern. The complexity with this pattern is stitching together the permissions and access controls between the Kafka record and the system that you present the claim check to. One good trick is to put all the work in the serializer and deserializer, such that it's transparent to the consumer.
For EDA, do we need to use CDC technology for it?
Adam here. No, you don't need to, though it is a good way to get started by bootstrapping data into your Kafka broker. Some applications, such as one made with Kafka Streams, Flink, or FlinkSQL, can natively emit their own events to Kafka - events as input, and events as output. But if you're starting with absolutely no event-driven applications or event streams at all, then learning how to use CDC (such as Debezium) is a very good place to start.
I am a student Software engineering and I found this quite interesting. Is there any academic/research paper out there that discuss this topic in details you could point out peharps. Thank you.
Adam here. I don't know of any whitepaper for this subject, particularly as this is really a "versus" type of discussion. You may find some luck searching for one or the other terms (EDA, or RR) on its own. Otherwise, I'd just be googling this for you. Lots of this is stuff I've picked up in my own experience over the years, listening to others, etc.
My question is about his setup, the board he is using to write in front of the camera is an Acrylic board correct? can anyone correct me if am wrong here? and what is the best marker to use I do deliver online training and consultation and I want to use the same method
Adam here. This is a homemade board that I made using modular aluminum framing, and initially I tried using a 6' x 4' sheet of acrylic. However, I found that my acrylic sheet would "fog up" if I turned the lights on too bright, and furthermore, it was very easy to scratch. I tried very hard to not scratch it, but within a week of light use I ended up with a few deep enough scratches that I couldn't hide them in post-processing. At that point, I switched to locally sourced low-lead glass (aka Starphire type), 1/4" thick. The sheet weighs about 55 lbs at a size of 64" x 44" (I made it smaller to match the actual 4K resolution), and I had to modify the frame to install some glass clamps. But it's much more durable, easier to clean, and clearer at higher brightness. I think I paid about $220 for the acrylic sheet, and $550 for the low-lead glass (both delivered). I would recommend skipping acrylic, as it can be hard to source a clear one and it just scratches too easily for sustained high-definition usage.
Oh, and in terms of markers, I found that "EXPO NEON Marker Dry-Eraser" worked best in terms of contrast and visibility. I've also tried some liquid-chalk markers to mixed results, some show up well, some do not.
In the EDA, 1)How do we verify the transaction got verified? (In an event where subscribers lose an event) 2) In a micro services scenario where multiple nodes are running, how do we prevent duplicates? (how to stop Processing the same order twice)
Adam here. 1) If you're talking about distributed transactions, you're going to need to look at the saga pattern. The subscribers won't "lose" an event unless they have written buggy code, in which case it's their responsibility to fix it. 2) One option is to check out "effectively/exactly-once processing" in Apache Kafka. It'll ensure your system doesn't create duplicate events, regardless of if or when it fails in its processing. However, it's still possible to cause side effects (like calling an external REST API) each time you process the event, despite exactly-once. However, this is the same as if you were to call a REST API, get a timeout, and retry calling it - resulting in duplicate processing. My advice is to make your code idempotent so that duplicate processing has no side effects. If you can't do that, then you're going to have to build a system to guard your consumer against duplicate processing, such as consuming records 1 by 1 (slow) or using durable state to keep track of precisely which records have been processed and which have not via atomic updates.
I don't understand the need for Kafka here, the storefront could easily keep a history of order data to use for inventory, data lakes, ML etc. etc.? Also, depending on how the request-response's model's architecture is planned, it could work with "events" as well. Just don't design it to require an immediate response, but rather poll for a list of the fulfilled orders regularly and update its "order status" attribute. If there is a clear boundary between req-res and EDA, I still can't see it. All depends on how it's implemented, right? In the EDA example, the storefront would at some point need to display the fulfilled order, so it still needs to consume "responses" from the fulfillment, just it's asking Kafka instead the fulfillment service. You still need to define a data structure for the event and hope all your future application will be able to consume it, it's still a hard contract. Isn't it true you could create an asynchronous req-res Application? The immediate need for a response seems contrived and a beginner's mistake, frankly.
How would you pick one over the other? What are the use cases.
Adam here. There are many ways you _can_ solve problems, which means that everyone has a different opinion on the matter. Given that I won't be able to explore this all in a youtube comment, we can start with the certainties. You'll be very safe using Request-Response and RPC-style communication whenever you need client-server communication. You'll also be safe if you use event-driven architecture to drive services that don't require blocking low-latency calls, like managing real-product inventory, handling advertising campaigns, and processing orders and payments. The whole middle ground between these two is where things can get muddy, where you'll get different answers depending on who you ask, and ultimately you'll end up with "it depends". What I like about EDAs is that if you invest a bit of time and effort into building well-defined event-streams, then you unlock the possibility to choose whichever is best for the task at hand - RR OR EDA.
@@ConfluentDevXTeam thanks for the explanation!
Fulfilment is spelled without two Ls
Adam here! I'm a Canadian, and I always spell it as "Fulfilment" (British/AUS/CAD/NZ Spelling) in my personal life. But my editors insisted I use "Fulfillment" (US English). I did more than a few takes where I had to stop and rewind before I decided to just start with the word written on the board.
Event driven architecture is a headache for developers, it has a lot of pitfalls and i recommend never do it
Adam here. I'm sorry to hear you haven't had a good experience with EDAs, but I encourage you not to discard it as a tool from your kit. My experience has been quite the opposite, where developers love it and embrace it once they understand where to use it and what it's best suited for.